DESIGN AND ANALYSIS OF ALGORITHMS MC9223

%N&' & undamentals

&N'()D%*'&)N of algorithmic problem solving ) Important problem types )

undamentals of the analysis of algorithm efficiency ) analysis frame work ) Asymptotic notations ) *athematical analysis for recursive and non-recursive algorithms.

Part A 1. What is meant by algorithm? An algorithm is a step by step process for solving a problem or program. It is easy to understand by the programmer who is going to implement the algorithm. 2. Definition / Notation of algorithm. An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. . "roblem

Algorithm

Input

+ompiler

'utput

3. Write the rules / important points to de elop an algorithm. The non-ambiguity requirement for each step of an algorithm. The range of inputs for which an algorithm works has been specified. The same algorithms are representing in different ways. Several algorithms for solving the same problem. !. "ist out the implementation of algorithm. inding a method for solving a problem. !very step of an algorithm should be in a clear manner. "suedcode and flowchart are also used to describe the algorithm. All the steps in an algorithm should be done in manually. inally implement the algorithm in terms of programming language. #. What are $hara$teristi$s of an algorithm? I#"$T %ero or more quantities are e&ternally supplied. '$T"$T At least one quantity is produced. (! I#IT!#!SS !ach instruction is clear and unambiguous.

I#IT!#!SS If we trace out the instructions of an algorithm, then for all cases, the algorithm terminates after a finite number of steps. ! !+TI,!#!SS !very instruction must very basic so that it can be carried out, in principle, by a person using only pencil - paper.

+. "ist out the types of algorithms. !uclid.s algorithm +onsecutive integer checking algorithm *iddle-school algorithm. ,. What are the fundamentals steps in ol ed in Algorithmi$ Problem -ol ing? ./%N 20111 $nderstanding the problem (eciding on Exact vs. approximate problem solving Appropriate data structure (esign an algorithm "roving correctness Analy/ing an algorithm Time efficiency : how fast the algorithm runs Space efficiency: how much extra memory the algorithm needs. Simplicity and generality +oding an algorithm 2. Define se3uential algorithms. The central assumption is that instructions are e&ecuted one after another, one operation at a time. Accordingly algorithms are designed to be e&ecuted on such machines are called sequential algorithms. 4. Define parallel algorithms Some new computers can e&ecute operations concurrently0 i.e. in parallel. Algorithms that take advantage of this capability are called parallel algorithms 10. 5o6 to *hoosing bet6een 78a$t and Appropriate Problem -ol ing? The principal decision is to choose between solving the problem e&actly and solving it appropriately. The former case algorithm is called e&act algorithm latter case, an algorithm is called appropriate algorithm. 11. What is Data -tru$ture? Algorithms + Data structures !rograms

A data structure is a speciali/ed format for organi/ing and storing data. 1eneral data structure types include the array, the file, the record, the table, the tree, and so on. Any data structure is designed to organi/e data to suit a specific purpose so that it can be accessed and worked with in appropriate ways. 12. What is an algorithm design te$hni3ue? An algorithm design technique is a general approach to solving problems algorithmically that is applicable to a variety of problems for different areas of computing. 13. What is pseudo$ode?

A pseudocode is a mi&ture of a natural language and programming language like constructs. A pseudocode is usually more precise than an natural language. 1!. What are the measures used to analy9e an algorithm? There are two kinds of algorithm efficiency2 .D7* 20101

Time efficiency: indicates how fast the algorithm runs. Space efficiency: indicates how much extra memory the algorithm needs.

1#. Write the &mportant Problem 'ypes of algorithm? Sorting Searching String processing 1raph problems +ombination problems 1eometric problems #umerical problems 1+. Define -orting 6ith an e8amples. 3earrange the items of a given list in ascending order. Sorting can be done to sort list of numbers, characters from an alphabet, character strings and records maintained by schools about students and libraries about books and companies about their employees. !&amples of sorting algorithms Selection sort 4ubble sort Insertion sort *erge sort 5eap sort 1,. What is meant by -orting :ey? A specially chosen piece of information used to guide sorting. I.e., sort student records by names. 12. What are the t6o properties of sorting algorithm? Stability2 A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input. In place2 A sorting algorithm is in place if it does not require e&tra memory, e&cept, possibly for a few memory units. 14. Define -ear$hing 6ith e8amples The searching problem deals with finding a given value, called a search key, in a given set. !&amples of searching algorithms Sequential searching 4inary searching 20. What is meant by -tring? A string is a sequence of characters from an alphabet. Strings comprises of Te&t strings2 letters, numbers, and special characters. 4it strings comprises of /eros and ones. 21. What is -tring mat$hing?

String matching means searching for a given word 6 pattern in a te&t. 22. What is meant by ;raph Problems? A graph is a collection of points called vertices, some of which are connected by line segments called edges. 1raphs can be used for2 *odeling real-life problems *odeling 777 Transportation +ommunication networks "ro8ect scheduling

23. "ist out the e8amples of 6idely used graph algorithms. 1raph traversal algorithms Shortest-path algorithms Topological sorting 2!. Define tra eling salesman problem. Traveling salesman problem which finds the shortest tour through n cities that visits every city e&actly once. 2#. Define ;raph $oloring problem. The graph coloring problem assigns the smallest number of colors to vertices of a graph so that no two ad8acent vertices are the same color. 2+. What is *ombinational Problems? The traveling salesman problems and the graph coloring problems are e&amples of combination problems. These problems ask to find a combinatorial ob8ect ) such as permutation, a combination, or a subset ) that satisfies certain constraints and has some desired property. 2,. Define ;eometri$ Problems. 1eometric algorithms deal with geometric ob8ects such as points, lines and polygons. Ancient 1reeks have invented procedures for solving problems of constructing simple geometric shapes ) triangles, circles etc. 22. Define Numeri$al Problems< #umerical problems another large area of application, are problems that involve mathematical ob8ects of continuous nature2 solving equations and systems of equations, computing definite integrals, evaluating functions. 24. 5o6 to analysis =rame6or: of algorithm? *easuring an input.s si/e *easuring running time 'rders of growth 9of the algorithm.s efficiency function: 7orst-base, best-case and average efficiency 30. 5o6 6ill you measuring an &nput>s -i9e of a problem? *ost programs runs longer on larger inputs e&ample it takes longer to sort larger arrays, multiply larger matrices. !fficiency is defined as a function of input si/e. Input si/e depends on the problem.

!&ample ;, what is the input si/e of the problem of sorting n numbers< !&ample =, what is the input si/e of adding two n by n matrices< 31. Define ?asi$ operation. The most important operation of the algorithm, the operation contributing the most to the total running time. or e&ample, the basic operation is usually the most time-consuming operation in the algorithm.s innermost loop. 78amples Problem Input size measure Basic operation >ey comparison addition multiplication

Search for a key in a list of n items #umber of items in list, n Add two n by n matrices "olynomial !valuation (imensions of matrices, n "rder of the polynomial

31. 78plain the formula of ?asi$ operation. ?et cop # 4e the time of e&ecution of an algorithms basic operation on a particular computer ?et *@nA ) be the number of times this operation needs to be e&ecuted for this algorithm. Then we can estimate the running time T9n: of a program implementing this algorithm on that computer by the formula. T@nA B copC @nA. 32. Define Worst $ase 7ffi$ien$y The worst case efficiency of an algorithm is its !fficiency for the worst case input of si/e n. or which the algorithm runs the longest among all possible inputs of si/e n. Analy/e the algorithm to see what kind of inputs yield largest value of the basic operations count +9n: among the possible inputs of si/e n and compute the worst case value +worst9n:. 33. Define ?est $ase 7ffi$ien$y The best case efficiency of an algorithm is its !fficiency for the best case input of si/e n. or which the algorithm runs the fastest among all possible inputs of si/e n. Then we should ascertain the value of +9n: on these inputs. !&ample for sequential search, best case inputs will be lists of si/e n with their first elements equal to a search key according to +best9n: @;. 3!. Define A erage $ase 7ffi$ien$y The average case efficiency of an algorithm is its !fficiency for a typical6random input of si/e n. #'T the average of worst and best case. 3#. What are the standard assumptions of A erage $ase 7ffi$ien$y? The probability of a successful search is equal to p 9AB@pB@;: The probability of the first match occuririn in the ith position of the list is the same for every i.

3+. Define Asymptoti$ Notations 6ith e8amples.

.D7* 20101

Asymptotic notations used to compare orders of growth of an algorithm.s basic operation count for the algorithms efficiency. Types are ) @big )hA C @big omegaA D @big thetaA .D7* 20121

3,. Write a brief note on )Enotation.

A function t$n% is said to be in "$g$n%%& denoted t$n% "$g$n%%& if t$n% is bounded above by some constant multiple of g$n% for all large n, i.e., if there e&ist some positive constant c and some nonnegative integer n' such that t$n% cg$n% for all n n' 32. Define Enotation. A function t$n% is said to be in $g$n%%& denoted t$n% $g$n%%& if t$n% is bounded below by some constant multiple of g$n% for all large n, i.e., if there e&ist some positive constant c and some nonnegative integer n' such that t$n% cg$n% for all n n' 34. Define Enotation @'hetaA. ./%N 20121 A function t$n% is said to be in $g$n%%& denoted t$n% $g$n%%& if t$n% is bounded both above and below by some positive constant multiples of g$n% for all large n, i.e., if there e&ist some positive constant c; and c= and some nonnegative integer n' such that c( g$n% t$n% c) g$n% for all n n' !0. What is re$ursi e algorithm? An algorithm is said to be recursive if the same algorithm is invoked in the body. An algorithm that calls itself is direct recursive. Algorithm A is said to be indeed recursive if it calls another algorithm, which in turn calls A. !1. Write the ;eneral plan for analy9ing the effi$ien$y for non re$ursi e algorithms? ./%N 20111 (ecide on a parameter indicating an input.s si/e. Identify the algorithm.s basic operation. +heck whether the number of times the basic operation is e&ecuted depends only in the si/e of an input. If it depends on some additional properties the worst case, average case, best case efficiencies have to be investigated. Set up sum e&pressing the number of times the algorithm.s basic operation is e&ecuted. $sing standard formulas and rules of sum manipulation, either find a closed form formula for the count or, at the very least, establish its order of growth. !2. Write the ;eneral plan to follo6 in analysis re$ursi e algorithms? (ecide on parameter n indicating input si/e Identify algorithm.s basic operation (etermine worst, average, and best case for input of si/e n

Set up a recurrence relation and initial condition9s: for +9n:-the number of times the basic operation will be e&ecuted for an input of si/e n. Solve the recurrence or estimate the order of magnitude of the solutions

!3. Write demerits of re$ursi e algorithms2 *any programming languages do not support recursion0 hence, recursive mathematical function is implemented using iterative methods. !ven though mathematical functions can be easily implemented using recursion it is always at the cost of e&ecution time and memory space. A recursive procedure can be called from within or outside itself and to ensure its proper functioning it has to save in some order the return addresses so that, a return to the proper location will result when the return to a calling statement is made. The recursive programs needs considerably more storage and will take more time. !!. What is performan$e measurement? "erformance measurement is concerned with obtaining the space and the time requirements of a particular algorithm. !#. Define input si9e. The input si/e of any instance of a problem is defined to be the number of words 9or the number of elements: needed to describe that instance. !+. Write algorithm using NonEre$ursi e fun$tion to fine sum of n numbers. algorithm sum9a,n: C s 2 @ A.A for i@; to n do s 2 - s D aEiF0 return s0 G !,. Write an algorithm using (e$ursi e fun$tion to fine sum of n numbersF algorithm rsum 9a,n: C if9nH A: then return A.A0 else return rsum9a, n- ;: D a9n:0 !4. Define Doubly "in:ed list. .D7* 20111 A (oubly linked list is a linked data structure that consists of a set of sequentially linked records called nodes. !ach node contains two fields, called links that are references to the previous and to the ne&t node in the sequence of nodes. The beginning and ending nodesI previous and ne&t links, respectively, point to some kind of terminator, typically a sentinel node or null, to facilitate traversal of the list. If there is only one sentinel node, then the list is circularly linked via the sentinel node. It can be conceptuali/ed as two singly linked lists formed from the same data items, but in opposite sequential orders. #0. Distinguish bet6een Worst $ase and ?est $ase effi$ien$y. .D7* 20111 The worst case scenario, on the other hand, describes the absolute worst set of input for a given algorithm. ?etIs look at a quick sort, which can perform terribly if you always choose

the smallest or largest element of a sub list for the pivot value. This will cause quick sort to degenerate to '9n=:. The best case scenario for an algorithm is the arrangement of data for which this algorithm performs best. Take a binary search for e&ample. The best case scenario for this search is that the target value is at the very center of the data youIre searching. So the best case time comple&ity for this would be '9;:. (iscounting the best and worst cases, we usually want to look at the average performance of an algorithm. These are the cases for which the algorithm performs Jnormally.J #1. Differen$e bet6een se3uential and parallel algorithms. ./%N 20121 -e3uential algorithms< The central assumption is that instructions are e&ecuted one after another, one operation at a time. Accordingly algorithms are designed to be e&ecuted on such machines are called sequential algorithms. Parallel algorithms< Some new computers can e&ecute operations concurrently0 i.e. in parallel. Algorithms that take advantage of this capability are called parallel algorithms #2. Why is an algorithm important in problem sol ing? .D7* 20121

The term Kalgorithmic problem solvingL is deliberately ambiguous. 'n the one hand, it can be read as Kalgorithmic-problem solvingL0 this means solving problems that require the formulation of an algorithm for their solution. 'n the other hand, it can be read as Kalgorithmic problem-solvingL0 this means e&ploiting what has been learned from the e&perience of developing computer software over the last MA years that has helped us to hone our problem-solving skills. Part ? 1. Dis$uss about algorithm and its types 6ith e8amples. An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. 'ypes of algorithms< !uclid.s algorithm +onsecutive integer checking algorithm *iddle-school algorithm.

As an e&ample of illustrating the notion of algorithm, we consider three methods for solving the same problem2 *omputing the greatest $ommon di isor. 1reatest common divisor of two non negative, not- both-/ero integers m and n, denoted by g$d@mFnAF is defined as the largest integer that divides both m and n evenly. @iA 7u$lid>s algorithm<

7u$lid>s algorithm is a famous method which applies repeated equality. or e&ample2 gcd9NA,=O: @ gdc9=O,;=:@gcd9;=,A:@;=. gcd9m, n:2 the largest integer that divides both m and n. irst try -- !uclid.s algorithm2 gcd9m, n: @ gcd9n, m mod n:

Step;2 If n @ A, return the value of m as the answer and stop0 otherwise, proceed to Step=2 (ivide m by n and assign the value of the remainder to r. StepP2 Assign the value of n to m and the value of r to n. 1o to Step ;. Pseudo$ode of 7u$lid>s Algorithm< Algorithm Euclid$m& n% 66+omputes gcd9m, n: by !uclid.s algorithm 66Input2 Two nonnegative, not-both-/ero integers m and n 66'utput2 1reatest common divisor of m and n while n Q A do r m mod n mn nr return m @iiA *onse$uti e &nteger Algorithm Step;2 Assign the value of minCm, nG to t. Step=2 (ivide m by t. If the remainder of this division is A, go to StepP0otherwise, go to Step= StepP2 (ivide n by t. If the remainder of this division is A, return the value of t as the answer and stop0 otherwise, proceed to StepO. StepO2 (ecrease the value of t by ;. 1o to Step=.

@iiiA GiddleEs$hool pro$edure Step;2 ind the prime factors of m. Step=2 ind the prime factors of n. StepP2 Identify all the common factors in the two prime e&pansions found in Step; and Step=. 9If p is a common factor occurring "m and "n times in m and n, respectively, it should be repeated in minC"m, "nG times.: StepO2 +ompute the product of all the common factors and return it as the gcd of the numbers given.

Thus for the numbers NA,=O we get2 NA @ = . = . P . M =O @ = . = . = . P gcd9NA,=O: @ = . = . P @ ;= 2. "ist out the fundamentals steps of Problem sol ing algorithm. Dis$uss briefly about analy9e phase of algorithm. .D7* 2011F /%N7 H D7* 20121 @)(A

Dis$uss in detail< =eatures of the top do6n design and bottom up design approa$hes in problem sol ing. .D7* 20121 Structure of Fundamentals of Algorithmic Problem Solving

%nderstanding the problem o 3eading the problems description carefully, Asking questions do a few e&amples by hand0 think about special cases, etc. De$iding on o !&act vs. appro&imate problem solving o Appropriate data structure Design an algorithm Pro ing $orre$tness Analy9ing an algorithm o Time efficiency 2 how fast the algorithm runs o Space efficiency2 how much e&tra memory the algorithm needs. o Simplicity and generality *oding an algorithm Understanding the problem

An input to an algorithm specifies an instance of the problem the algorithms solves. It is very important to specify e&actly to range of instances the algorithms needs to handle. ailure to do this, the algorithm may work correctly for a ma8ority of input but crash on some KboundaryL value. A correct algorithm is not one that works most of the time but one that works correctly for all legitimate inputs. Ascertaining the Capabilities of a Computational !evice 'nce having understood a problem, one must ascertain the capabilities of the computational device the algorithm is intended for. In the case of random-access memory 93A*:, its central assumption is that instructions are e&ecuted one after another, one operation at a time. Accordingly algorithms are designed to be e&ecuted on such machines are called se3uential algorithms. Some new computers can e&ecute operations concurrently0 i.e. in parallel. Algorithms that take advantage of this capability are called parallel algorithms Choosing bet"een #$act and Appropriate Problem Solving The ne&t principal decision is to choose between solving the problem e&actly or solving it appropriately. The former case algorithm is called e8a$t algorithm latter case, an algorithm is called appropriate algorithm. !eciding on Appropriate !ata Structure In the new world of ob8ect oriented programming, data structures remain crucially important for both design and analysis of algorithms. Algorithms + Data structures !rograms Algorithm !esign Techni%ues An algorithm design technique is a general approach to solving problems algorithmically that is applicable to a variety of problems for different areas of computing. &ethods of Specifying an Algorithm 'nce having designed an algorithm, you need to specify it in some fashion. $sing natural language is obviously difficult. A pseudocode is a mi&ture of a natural language and programming language like constructs. A pseudocode is usually more precise than an natural language. Proving an Algorithm's Correctness 'nce an algorithm has been specified, you have to prove its correctness. That is you have to prove that the algorithm yields a required result for every legitimate input in a finite amount of time. or algorithms, a proof of correctness is to use mathematical induction because an algorithm.s iteration provides a natural sequence of steps needed for such proofs. The notion of correctness for appro&imation algorithms is less straightforward than it is for e&act algorithms. !rrors for appro&imation algorithms should not e&ceed a predefined limit. Analyzing an Algorithm

After correctness, the most important is effi$ien$y. There are two kinds of algorithm efficiency2 Time efficiency2 indicates how fast the algorithm runs. Space efficiency2 indicates how much e&tra memory the algorithm needs. Another characteristic of an algorithm is simplicity. Simplicity is an important algorithm characteristic. 4ecause simple algorithms are easier to understand and easier to program. 3esulting programs usually contain fewer bugs. Simper algorithms are also more efficient than more complicated alternatives. Another characteristic of an algorithm is generality. There are two issues here2 *enerality of the problem the algorithm solves ) it is easier to design an algorithm for a more general problem. 4ut there are situations where designing a more general algorithm is unnecessary or difficult or even impossible. The range of inputs it accepts ) your main concern should be designing an algorithm that can handle a range of inputs that is natural for the problem at hand.

Coding an Algorithm *ost algorithms are ultimately implemented as computer programs. Transition of algorithm to a program must be done carefully. ,alidity of the computer program is established by testing. Testing of computer programs is an art rather than science. 3. 7numerate and e8plain briefly important types of problem. Sorting Searching String processing 1raph problems +ombination problems 1eometric problems #umerical problems ./%N 20111

Sorting 3earrange the items of a given list in ascending order. Sorting can be done to sort list of numbers, characters from an alphabet, character strings and records maintained by schools about students and libraries about books and companies about their employees. +nput: A sequence of n numbers Ba;, a=, R, anS "utput: A reordering BaT;, aT=, R, aT nS of the input sequence such that aT )U a T( U , U aTn. -orting :ey A specially chosen piece of information used to guide sorting. I.e., sort student records by names. !&amples of sorting algorithms Selection sort

4ubble sort Insertion sort *erge sort 5eap sort

!valuate sorting algorithm comple&ity2 the number of key comparisons. '6o properties Stability2 A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input. In place2 A sorting algorithm is in place if it does not require e&tra memory, e&cept, possibly for a few memory units.

Searching The searching problem deals with finding a given value, called a search key, in a given set. The e&amples of searching algorithms are as follows, Sequential searching 4inary searching

String A string is a sequence of characters from an alphabet. Strings comprises of te&t strings2 letters, numbers, and special characters. 4it strings comprises of /eros and ones. String matching2 searching for a given word6pattern in a te&t. (raph Problems A graph is a collection of points called vertices, some of which are connected by line segments called edges. 1raphs can be used for2 *odeling real-life problems *odeling 777 transportation communication networks "ro8ect scheduling R

78amples of 6idely used graph algorithms< *raph traversal algorithms Shortest#path algorithms Topological sorting

'he 6idely :no6n graph problems are< Traveling salesman problem ) 7hich finds the shortest tour through n cities that visits every city e&actly once.

The graph coloring problem ) assigns the smallest number of colors to vertices of a graph so that no two ad8acent vertices are the same color. Combinational Problems

The traveling salesman problems and the graph coloring problems are e&amples of combination problems. These problems ask to find a combinatorial ob8ect ) such as permutation, a combination, or a subset ) that satisfies certain constraints and has some desired property. (eometric Problems 1eometric algorithms deal with geometric ob8ects such as points, lines and polygons. Ancient 1reeks have invented procedures for solving problems of constructing simple geometric shapes ) triangles, circles etc. )umerical Problems #umerical problems another large area of application, are problems that involve mathematical ob8ects of continuous nature2 solving equations and systems of equations, computing definite integrals, evaluating functions. !. What is an algorithm? What are the analysis frame6or:s in measuring the effi$ien$y? ;i e e8amples. .D7* 20101 Algorithm An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Analysis of algorithms means to investigate an algorithm.s efficiency with respect to resources2 running time and memory space. Time efficiency2 how fast an algorithm runs. Space efficiency2 the space an algorithm requires. Analysis Frame"or* *easuring an input.s si/e *easuring running time 'rders of growth 9of the algorithm.s efficiency function: 7orst-base, best-case and average efficiency

&easuring an Input's Size *ost programs runs longer on larger inputs e&ample it takes longer to sort larger arrays, multiply larger matrices. !fficiency is defined as a function of input si/e. Input si/e depends on the problem. !&ample ;, what is the input si/e of the problem of sorting n numbers< !&ample =, what is the input si/e of adding two n by n matrices<

Units for &easuring +unning Time

*easure the running time using standard unit of time measurements, such as seconds, minutes. o This (epends on the speed of the computer. +ount the number of times each of an algorithm.s operations is e&ecuted. o This approach is (ifficult and unnecessary +ount the number of times an algorithm.s basic operation is e&ecuted.

?asi$ operation2 the most important operation of the algorithm, the operation contributing the most to the total running time. or e&ample, the basic operation is usually the most time-consuming operation in the algorithm.s innermost loop. Input Size and Basic ,peration #$amples Problem Search for a key in a list of n items Add two n by n matrices "olynomial !valuation Input size measure #umber of items in list, n (imensions of matrices, n "rder of the polynomial Basic operation >ey comparison addition multiplication

'ime effi$ien$y is analy9ed by determining the number of repetitions of the basi$ operation as a fun$tion of input si9e. "et cop # 4e the time of e&ecution of an algorithms basic operation on a particular computer ?et *@nA ) be the number of times this operation needs to be e&ecuted for this algorithm. Then we can estimate the running time T9n: of a program implementing this algorithm on that computer by the formula. ,rder of gro"th T@nA B copC @nA

An order of growth is a set of functions whose asymptotic growth behavior is considered equivalent. or e&ample, = n, ;AAn and n D ; belong to the same order of growth, which is written "9n: in 4ig-'h notation and often called linear because every function in the set grows linearly with n. )rder of gro6th '9;: '9logb n: '9n: '9n logb n: '9n=: '9n : '9c :
n P

Name constant logarithmic 9for any b: linear Ken log enL quadratic cubic e&ponential 9for any c:

-orst.Case/ Best.Case/ and Average.Case #fficiency Algorithm efficiency depends on the input si/e n but for some algorithms efficiency depends on not only on the input si/e also on type of input. *onsider an e8ample< -e3uential -ear$h "roblem2 1iven a list of n elements and a search key >, find an element equal to >, if any. Algorithm2 Scan the list and compare its successive elements with > until either a matching element is found 9successful search: of the list is e&hausted 9unsuccessful search: -e3uential -ear$h Algorithm A?1'3IT5* SequentialSearch9AEA..n-;F, >: 66Searches for a given value in a given array by sequential search 66Input2 An array AEA..n-;F and a search key > 66'utput2 3eturns the inde& of the first element of A that matches > or ); if there are no matching elements i A while i B n and AEiF Q > do iiD; if i B n 66AEIF @ > return i else return -; -orst case #fficiency The worst case efficiency of an algorithm is its !fficiency for the worst case input of si/e n. or which the algorithm runs the longest among all possible inputs of si/e n. Analy/e the algorithm to see what kind of inputs yield largest value of the basic operations count +9n: among the possible inputs of si/e n and compute the worst case value +worst9n:. Best case #fficiency The best case efficiency of an algorithm is its !fficiency for the best case input of si/e n. or which the algorithm runs the fastest among all possible inputs of si/e n. Then we should ascertain the value of +9n: on these inputs. !&ample for sequential search, best case inputs will be lists of si/e n with their first elements equal to a search key according to +best9n: @;. Average case #fficiency The average case efficiency of an algorithm is its !fficiency for a typical6random input of si/e n. #'T the average of worst and best case. 5ow to find the average case efficiency< +onsider the sequential search. The standard assumptions are2 a: The probability of a successful search is equal to p 9AB@pB@;:

b: The probability of the first match occurring in the position of the list is the same for every i. The average number of key comparisons +avg9n: as follows2 The first match occurring in the ith position of the list is p6n for every i, and the number of comparisons made by the algorithm is i. In the case of an unsuccessful search, the number of comparisons is n with the probability of such a search being 9;-p:. +avg9n: @ E;.p6n D =.p6n D . . . D i.p6n D . . . D n.p6nF D n. 9;-p: @ p6n E ;D=D. . . D I D . . . D nF D n 9;-p: @ p6n . n9nD;:6= D n9;-p: @ p9nD;:6= D n9;-p: #. ?riefly e8plain asymptoti$ Notations and types 6ith its graphs. Asymptotic notations used to compare orders of growth of an algorithm.s basic operation count for the algorithms efficiency. Types are , @big )hA C @big omegaA D @big thetaA )@g@nAA< class of functions f9n: that grow no faster than g9n: C@g@nAA< class of functions f9n: that grow at least as fast as g9n: D @g@nAA< class of functions f9n: that grow at same rate as g9n:

7here, t9n: ) algorithms running time +9n: ) indicates the basic operation count g9n: ) simple functions to compare the count with. )Enotation ormal definition

A function t$n% is said to be in "$g$n%%& denoted t$n% "$g$n%%& if t$n% is bounded above by some constant multiple of g$n% for all large n, i.e., if there e&ist some positive constant c and some nonnegative integer n' such that t9n: cg9n: for all n nA !&ample2 ;AAn + - '9n=: ;AAn + - )'' n + n $for all n M: @ ;A;n )') n( Thus the value of the constants c and nA ;AAn D M;AAn D Mn 9for all nS;: @ ;AMn To complete the proof with c @ ;AM and nA@ ;

 Enotation ormal definition A function t$n% is said to be in $g$n%%& denoted t$n% $g$n%%& if t$n% is bounded below by some constant multiple of g$n% for all large n, i.e., if there e&ist some positive constant c and some nonnegative integer n' such that t9n: cg9n: for all n nA

!&ample of the formal proof that n. 9n=:2 n. n= for n A. i.e. we can select c@; and nA @ A Enotation ormal definition A function t$n% is said to be in $g$n%%& denoted t$n% $g$n%%& if t$n% is bounded both above and below by some positive constant multiples of g$n% for all large n, i.e., if there e&ist some positive constant c; and c= and some nonnegative integer n' such that

c= g9n: t9n: c; g9n: for all n nA

!&ample let us prove that ;6=n9n-;: 9n=: irst we prove the right inequality 9upper bound:2 ;6=n9n-;: @ ;6=n= - ;6=n ;6=n= for all n A. Second we prove the left inequality 9lower bound:2 ;6=n9n-;: @ ;6=n= - ;6=n ;6=n= - ;6=n ;6=n for all n = @ ;6On= 5ence we can select c= @ V, c;@ W and no @ = +. Des$ribe briefly about mathemati$al analysis of non re$ursi e algorithms. .D7* 2011 H /%N7 20121 78ample 12 +onsider the problem of finding the value of the largest element in a list of n numbers. Assume the list is implemented as array. The following is a pseudocode of a standard algorithm for solving the problem. Algorithm *a&!lement 9AEA..n-;F: 66(etermines the value of the largest element in a given array 66Input 2 An array AEA..n-;F of real numbers 66'utput2 The value of the largest element in A *a&val AEAF for i ; to n ) ; do if AEiF S ma&val *a&val AEiF return ma&val Inout Si/e here is the array n. The operation to be e&ecuted most often is for loop. There are two operations in the loops body2

 +omparison AEiFSma&val Assignment ma&val AEiF 7hich of these two operations must we consider basic operation< +omparison is e&ecuted on each repetition of the loop and not assignment ?et us denote +9n: the number of lines this comparison is e&ecuted. The algorithm makes one comparison on each e&ecution of the loop, which is repeated for each value of the loop.s variable i within the bounds between ; and n-;. Therefore we get the following sum for +9n:2 +9n: @ X ; @ n-; 9n:.
i@; n-;

The 1eneral plan to follow in analy/ing non recursive algorithms is2 ;. (ecide on a parameter indicating an input.s si/e. =. Identify the algorithm.s basic operation. P. +heck whether the number of times the basic operation is e&ecuted depends only in the si/e of an input. If it depends on some additional properties the worst case, average case, best case efficiencies have to be investigated. O. Set up sum e&pressing the number of times the algorithm.s basic operation is e&ecuted. M. $sing standard formulas and rules of sum manipulation, either find a closed form formula for the count or , at the very least, establish its order of growth. 78ample 2< The following algorithm finds the representation of a positive decimal integer. number of binary digits in the binary

A";)(&'5G 4inary9n: 66Input2 A positive decimal integer n 66'utput2 The number of binary digits in n.s binary representation count ; 6hile nS; do count count D; nEn6=F return count In this the most frequently e&ecuted operation is not inside the 6hile loop but rather the comparison nS; that determines whether the loop.s body will be e&ecuted. Since the number if times the comparison will be e&ecuted is larger than the number of repetitions of the loop.s body by e&actly ;, the choice is not that important. A more significant feature of this e&ample is the fact that the loop.s variable takes on only a few values between its lower and upper limits0 therefore we have to use an alternative way of computing the number of times the loop is e&ecuted. Since the value of n is about halved on each repetition of the loop, the answer would be log=n. ,. Define (e$ursi e algorithm. 78plain the mathemati$al analysis pro$edures 6ith 'o6ers of 5anoi pu99le e8ample. .D7* 2010 H /%N7 20121

 3ecursive evaluation of nY 3ecursive solution to the Towers of 5anoi pu//le 3ecursive solution to the number of binary digits problem !&ample 3ecursive evaluation of n Y 9;: Iterative (efinition 3ecursive definition

Algorithm /$n% 66 +omputes nY recursively 66 Input2 A nonnegative integer n 66'utput2 the value of nY if n@A return ; else return / 9n -;: Z n

66base case 66general case

!&ample 3ecursive evaluation of n Y 9=: Two 3ecurrences o The one for the factorial function value2 9n: 9n: @ 9n ) ;: Z n for every n S A 9A: @ ; o The one for number of multiplications to compute nY, *9n:

*9n: @ *9n ) ;: D ; for every n S A *9A: @ A *9n: [ 9n: -teps in Gathemati$al Analysis of (e$ursi e Algorithms (ecide on parameter n indicating input si/e Identify algorithm.s basic operation (etermine worst, average, and best case for input of si/e n Set up a recurrence relation and initial condition9s: for +9n:-the number of times the basic operation will be e&ecuted for an input of si/e n. Solve the recurrence or estimate the order of magnitude of the solutions

!&ample 2 The Towers of 5anoi "u//le In this pu//le we have n disks of different si/es and three pegs. Initially all the disks are in the first peg in order of si&w, the largest on the bottom and the smallest on top. The goal is to move all the disks to the third peg, using the second one as an au&iliary, if necessary. 7e can move only one disk at a time, an it is forbidden to place a larger disk on top of the smaller one. To move nS; disks from peg ; to peg P9with peg = as au&iliary:, we first moce recursively n ) ; disks from peg ; to peg = 9with peg P as au&iliary:, then move the largest disk directly from peg ; to peg P, and finally move recursively n ) ; disks from peg

= to pe P9using peg; as au&iliary:. If n@; then we can directly move the disk directly from the source peg to the destination peg. The number of moves *9n: depends on n only, and we get the following recurrence equation for it2 *9n: @ *9n ) ;: D ;D*9n - ;: for nS;. 7ith the obvious initial condition *9;: @ ;, we have the following recurrence relation for the number of moves *9n:2 *9n: @ =*9n ) ;: D ; for every n S ; *9;: @ ; 7e solve this recurrence by te same method of backward substitutions2 *9n: @ =*9n ) ;: D ; sub. *9n ) ;: @ =*9n ) =: D ; @=E=*9n ) =: D ;F D ; @ ==*9n ) =:D=D; sub. *9n ) =: @ =*9n ) P: D ; @== E=*9n ) P:D;FD=D; @ =P *9n ) P:D ==D=D; The ne&t one will be =O *9n ) O:D =PD ==D=D; And after the ith substitution, we get *9n: @ =i *9n ) i:D =i-;D =i-=DRD=D; @ =i *9n ) i:D =i-; Since the initial condition is specified for n@;, which is achieved for i@n-;, we get the following formula for the solution to recurrence *9n:@ = @= Problem
n-;

*9n ) 9n -;:: D = n-; -; *9;: D = n-; -; @ =

n-;

n-;

D=

n-;

-; @ =

-;

2. Write an algorithm for finding ma8imum element of an array perform bestF 6orst and a erage $ase $omple8ity 6ith appropriate order notations. ./%N 20111 +onsider the problem of finding the value of the largest element in a list of n numbers. Assume the list is implemented as array. The following is a pseudocode of a standard algorithm for solving the problem. Algorithm *a&!lement 9AEA..n-;F: 66(etermines the value of the largest element in a given array 66Input 2 An array AEA..n-;F of real numbers 66'utput2 The value of the largest element in A *a&val AEAF for i ; to n ) ; do if AEiF S ma&val *a&val AEiF return ma&val Input Si/e here is the array n. The operation to be e&ecuted most often is for loop. There are two operations in the loops body2 +omparison AEiFSma&val Assignment ma&val AEiF 7hich of these two operations must we consider basic operation< +omparison is e&ecuted on each repetition of the loop and not assignment ?et us denote +9n: the number of lines this comparison is e&ecuted. The algorithm makes one comparison on each e&ecution of the loop, which is repeated for each value of the loop.s variable i within the bounds between ; and n-;. Therefore we get the following sum for +9n:2 +9n: @ X ; @ n-; 9n:.
i@; n-;

Program< \includeBstdio.hS

int main 9: C int i0 int aE;AF @ C ;A, MM, ], O, =PO, =A, PA, OA, ==, PO G0 int ma&val @ aEAF0 for 9i @ A0 i B ;A0 iDD: C if 9aEiF S ma&val: C ma&val @ aEiF0 G 3eturn ma&val G printf 9J*a&imum element in an array 2 ^d_nJ, ma&val:0 return A0

?est $ase< 4est case - finding the ma& element as the first 9'9;::, Worst *ase< 7orst case - it is the last element checked 9O(n):. A erage *ase< o o o The tricky part is the average case. To find the average case - we need the e&pected number of iterationsY Since you can stop after you find the ma&imum, we can split the problem into two parts2 EA,n-;:2 Since on average 9assuming uniform independent distribution for each element: - the number n has probability ;6n to be in each place, then the e&pected number of iterations for this part is ;6n D =Z99n-;:6n:6n D P Z 99n-;:6n:`=6n D ... D 9n-;: Z 99n-;:6n:`9n-=:6n ;

o The above formula yields an ugly formula which is O(n) o The last element is going to be checked if the first n-; elements did not contain the
value n2 so you need to add to the above n* ((n-1)/n)^(n-1), which is O(n) as well 9lim to infinity is 1/e * n:.

o This totals in O(n) average time solution.

4. 78plain ;raph and ;eometri$ problems in detail. .D7* 2011 H /%N 20121

1raph and geometric problems have been studied by computer science researchers using the framework of analysis of algorithms. 1raph theory is the study of the properties

of graphs. 1raph algorithms are one of the oldest classes of algorithms and they have been studied for almost PAA years. (raph Problems A graph is a collection of points called vertices, some of which are connected by line segments called edges. 1raphs provide essential models for many applications areas of computer science, and at the same time, they are fascinating ob8ects of study in pure and applied mathematics. There have been a number of e&citing recent developments in graph theory that are important for designers of algorithms to know about. +orrespondingly, the algorithmic viewpoint of computer science has stimulated much research in graph theory. 1raph theory and graph algorithms are inseparably intertwined sub8ects. 1raphs can be used for2 *odeling real-life problems *odeling 777 transportation communication networks "ro8ect scheduling R 78amples of 6idely used graph algorithms< *raph traversal algorithms Shortest#path algorithms Topological sorting

'he 6idely :no6n graph problems are< Traveling salesman problem ) 7hich finds the shortest tour through n cities that visits every city e&actly once. The graph coloring problem ) assigns the smallest number of colors to vertices of a graph so that no two ad8acent vertices are the same color.

(eometric Problems 1eometric algorithms deal with geometric ob8ects such as points, lines and polygons. Ancient 1reeks have invented procedures for solving problems of constructing simple geometric shapes ) triangles, circles etc. The main impetus for the development of geometric algorithms came from the progress in computer graphics, computer-aided design and manufacturing. In addition, algorithms are also designed for geometric problems that are classical in nature. The success of the field can be e&plained from the beauty of the geometry problems studied, the solutions obtained, and by the many application domains- computer graphics, geographic information systems, robotics and others, in which geometric algorithms play a crucial role. 1raph and geometric algorithms are at the heart of many computer applications. So, it is e&pected that computer scientists and professional programmers know frequently used algorithms and generic techniques for efficient organi/ation and retrieval of data, modeling, understanding and solving graph and geometric problems.

10. Dis$uss about WorstE*ase effi$ien$y. .D7* 20111 The worst case efficiency of an algorithm is its !fficiency for the worst case input of si/e n. or which the algorithm runs the longest among all possible inputs of si/e n. Analy/e the algorithm to see what kind of inputs yield largest value of the basic operations count +9n: among the possible inputs of si/e n and compute the worst case value +worst9n:. 78amples< Se%uential Search Assume that the data in the array is in no particular order. 7orst-case2 #ot found or searching for the item that is in the last position. f9n: @ n ,rdered Se%uential Search 3equires the precondition that the data in the array is ordered by the key field that will be used for the search. This additional overhead will affect the efficiency of the search, but for now will not be considered 9efficiency of sorting algorithms is the topic of the other half of this chapter:. 6orstE$ase< Searching for the item that is greater than or equal to the item in the last position. f@nA I n ?inary -ear$h 2 3equires the precondition that the data in the array is ordered by the key field that will be used for the search. This additional overhead will affect the efficiency of the search, but for now will not be considered. WorstE$ase< #ot found or searching for the item that would be the last one checked before determining it is not in the array. f@nA I log2n 9see below for the derivation of this result: Determining the 6orstE$ase $al$ulation for the binary sear$h... The binary search begins by looking at the middle item in the array 9at position n6=:. This results in three possibilities2 a. This is the item you are looking for, in which case you are now done. 4ut this would not be the worst case, therefore we will ignore this possibility. b. The item you are looking for is less than 9i.e. comes before: the middle item. Therefore, you can throw away the middle item and the second half of the list and search the first half of the list by the same process. This results in searching from among n6= items. c. The item you are looking for is greater than 9i.e. comes after: the middle item. Therefore, you can throw away the middle item and the first half of the list and search the second half of the list by the same process. This results in searching from among n6= items. The first comparison results in n6= items left over.

o o o o

The second comparison results in 9n6=:6= @ n69==: items left over. The third comparison results in 99n6=:6=:6= @ n69=P: items left over. The fourth comparison results in n69=O: items left over. !ventually, the kth comparison results in n69= k: items left over. If this results in one item left then only one more check needs to be made.

Therefore, appro&imately k comparisons are needed to search the array. After k comparisons, there would be only ; item left over. Therefore... n/@2:A I 1 2: I n : I log2n Since log=n is less than n, we may conclude that a binary search is more efficient that a linear search. 11. Write short notes on sorting ./%N 20121

Sorting is arranging or ordering the data in some logical order either increasing or decreasing. *any Sorting algorithms are available to sort the given set of elements. i0 Internal Sorting If the data to be sorted remains in main memory and also the sorting is carried out in main memory it is called internal sorting. The types of internal sorting are as follows, Insertion sort Selection sort *erge Sort 3adi& Sort auick Sort 5eap Sort 4ubble Sort ii0 #$ternal Sorting If the data resides in au&iliary memory and is brought into main memory in blocks for sorting and then result is returned back to au&iliary memory is called e&ternal sorting. The !&ternal sorting methods are applied only when the number of data elements to be sorted is too large. These methods involve as much e&ternal processing as processing in the +"$. The following are the e&amples of e&ternal sorting. Sorting with (isk Sorting with tapes. -orting Algorithms o Insertion Sort o Selection Sort o auicksort

*ergesort Sorting time may depend on2 initial order of data data structure used to represent data memory available si/e of record sort algorithm ?ocation of list2 internal storage, e&ternal device, etc. and more ,1n230

Selection Sort

In this sorting we find the smallest element in this list and put it in the first position. Then find the second smallest element in the list and put it in the second position. And so on. ;. ind smallest item in list. =. !&change it with JfirstJ item in list. P. 3epeat, beginning with the rest of the list. 4uic*sort ,1nlog31n00

auicksort was invented and named by +. A. 3. 5oare and is one of the best generalpurpose sorting algorithms. It is built on the ideal of partitions, and it uses divide#and# con0uer strategy. The basic algorithm for a one-dimensional array is as follows. ;. !artition Step: Select an element to place in its final position in the array. That is, all the elements to the left will be less than selected element, and all the elements to the right will be greater than the chosen element. 7e will select the first element in the array and put it in its final place in the array. Then we have one element in its proper location and two unsorted subarrays. =. 1ecursive Step: 3epeat the process on each unsorted subarray. !ach time the partition step is repeated, another element is placed in its final position in the sorted array, and two additional subarrays are created. 7hen a subarray eventually contains only one element, that subarray is sorted and the element is in its final location. Insertion Sort ,1n305 Sorting the given elements from ; to n, inserting each element into its proper position. An e&ample of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you e&tract a card, shift the remaining cards, and then insert the e&tracted card in the correct place. This process is repeated until all the cards are in the correct sequence. 4oth average and worst-case time is ,9n=:. &erge Sort

+ombing the two lists is called as merging. or e&ample A is a sorted list with r elements and 4 is a sorted list with s elements. The operation that combines the elements of A and 4 into a single sorted list + with n @ r D s elements is called merging. After combing the two lists the elements are sorted by using the following merging algorithm suppose one is given two sorted decks of cards.

ANNA %N&J7(-&'K L%7-'&)NPart E A ;. 7hat are the fundamentals steps involved in Algorithmic "roblem Solving< bF =. 7hat are the measures used to analy/e an algorithm< ;OF P. (efine Asymptotic #otations with e&amples. PNF O. 7rite a brief note on '-notation. PbF M. (efine a-notation 9Theta:. P]F N. 7rite the 1eneral plan for analy/ing the efficiency for non recursive algorithms< E3ef. #o.2 O;F b. (efine (oubly ?inked list. O]F E3ef. #o.2 E3ef. #o.2 E3ef. #o.2 E3ef. #o.2 E3ef. #o.2 E3ef. #o.2

c. (istinguish between 7orst case and 4est case efficiency. MAF ]. (ifference between sequential and parallel algorithms. M;F ;A. 7hy is an algorithm important in problem solving< M=F Part ?

E3ef. E3ef. E3ef.

#o.2 #o.2 #o.2

;. ?ist out the fundamentals steps of "roblem solving algorithm. (iscuss briefly about analy/e phase of algorithm. problem solving. P. !numerate and e&plain briefly important types of problem. 1ive e&amples. M. (escribe briefly about mathematical analysis of non recursive algorithms. of 5anoi pu//le e&ample. average case comple&ity with appropriate order notations. c. !&plain 1raph and 1eometric problems in detail. ]. (iscuss about 7orst-+ase efficiency. ;AF ;A. 7rite short notes on sorting ;;F E3ef. #o.2 E3ef. #o.2 =F E3ef. #o.2 =F E3ef. #o.2 PF E3ef. #o.2 OF E3ef. #o.2 NF E3ef. #o.2 bF E3ef. #o.2 cF E3ef. #o.2 ]F E3ef. #o.2 =. (iscuss in detail2 eatures of the top down design and bottom up design approaches in

O. 7hat is an algorithm< 7hat are the analysis frameworks in measuring the efficiency<

N. (efine 3ecursive algorithm. !&plain the mathematical analysis procedures with Towers b. 7rite an algorithm for finding ma&imum element of an array perform best, worst and