Class 9 Slides Character Input Character Input (continued) Character Output A couple of tricks And now for something completely different: analysis of algorithms Counting basic steps Comparing summing algorithms Comparing factorial algorithms Who needs basic steps? Asymptotic Complexity The Meaning of Big O More examples of demonstrating order Searching algorithms Searching an unordered array Searching an ordered array Quadratic Search Choosing jump size Binary search Binary search code Binary search time efficiency Search recap Warm-up for sorting: The Dutch East Indies Flag problem The solution in code A more difficult variation -------------------------------------------------------------------------------- Character Input For more control over the input from a file (or from the screen, for that matter), you can use character-by-character input method get. It takes one character-type output parameter, which is passed by reference. It places the next available character from the input stream into the output parameter. It reads all characters, including spaces (' '), newlines ('\n'), and tabs ('\t'). What does the following code set c1 to? char c1, c2; ifstream infile; infile.open("aFile.txt"); c1 = ' '; infile.get(c2); while (c2 != '\n') { c1 = c2; infile.get(c2); } -------------------------------------------------------------------------------- Character Input (continued) Another useful character input method is putback. It takes one argument of type char and puts that character back into the input stream. This is very useful, for instance, if you want to read an integer using >> but want to make sure that the next non-blank character is actually a digit. For example: #include "iostream.h" #include "fstream.h" #include "ctype.h" char c = ' '; ifstream infile; int num; infile.open("test.txt"); while (c == ' ' || c == '\n' || c == '\t') infile.get(c); if (isdigit(c)) { infile.putback(c); infile >> num; } else { cout >> "Error in input file: expected a number." << endl; } -------------------------------------------------------------------------------- Character Output There is a character output method, put, which is equivalent to using << with a character argument. That is, it doesn't give you anything you didn't already have! #include "fstream.h" char l = 'l'; ofstream outfile; outfile.open("testout.txt"); outfile.put('h'); outfile.put('e'); outfile.put(l); outfile << l << 'o' << "!" << endl; -------------------------------------------------------------------------------- A couple of tricks You may run into difficulty using the apstring class free method getline. For instance, in this code, what would you expect to happen on the given input? Code: int num; apstring str; cin << num; getline(cin, str); Input: 1234 Howdy! Reading in a number (or character) reads up to but not including the following space, tab, or newline character. You can call getline or the get method to 'consume' the newline after 1234, then the next getline call will start reading from the beginning of the next line. It is possible to open an output file for appending, rather than overwriting its entire contents. This is not included in the AP curriculum. But just so you know... ofstream outfile; outfile.open("testout.txt", ios::out | ios::app); -------------------------------------------------------------------------------- And now for something completely different: analysis of algorithms What's an algorithm? It's a set of steps intended to accomplish a given task. A large part of computer science is learning the algorithms that have been developed for common programming tasks, such as searching and sorting, and adapting them to your own tasks. First, we need a means of making comparisons between different algorithms, so that we can talk about how one might be 'better' than another is for a certain problem. Algorithm performance is measured along two dimensions: time (how long does it take to run) and space (how much memory does it require). Time tends to be the more critical and more difficult to characterize. Our goal: to characterize time and space usage for a particular algorithm, independent of: exact program implementation speed of the computer on which the program is run compiler/interpreter for the programming language computer's operating system any other external factors (network or ROM drive speed, for instance) Instead of measuring the actual running time of the algorithm, we characterize it in relation to the size of the input. For instance, if we're searching an array, we would characterize the time in relation to the number of elements in the array. Furthermore, we consider the worst possible input in making our characterization. This is called worst case analysis. Average case analysis is also sometimes useful, though usually more difficult to determine. What's the worst case when searching an array? What's the best case? The average case? -------------------------------------------------------------------------------- Counting basic steps We start with details, and will work up from there to generalizations. Rather than measuring actual running time, we're going to consider each statement of a program as taking one basic step to execute. All of these count as one basic step, no matter how complex: an assignment statement (x = y + 3, x *= 2, x++) an output statement (cout << x) a test of a condition (x < y, x > 0 && y % 2 == 0) In a for or while loop, you must count one basic step for each time a given statement or condition is executed. For example: i = 0; for (i = 1; i <= N; i++) { while (i < 8) { cout << i << endl; cout << i << endl; } i++; } The general formula for a for loop, where I is the number of iterations performed, is 2I + 2 + I * (# of basic steps in loop body). Try a few more: for (i = 1; i <= 2 * N; i++) for (i = 1; i <= 100; i++) cout << i << endl; { k = 10; for (i = 1; i <= N; i++) for (j = 5; j <= N; j++) for (j = 1; j <= N; j++) k += (j - i); cout << i << " " << j << endl; } -------------------------------------------------------------------------------- Comparing summing algorithms Task: Sum the numbers from 1 to N. Algorithm 1: compute the running sum using a loop sum = 0; 1 step for (ct = 1; ct <= N; ct++) { 2N + 2 steps sum += ct; N steps } = 3N + 3 steps When the total running time is proportional to N, we say it runs in linear time. Algorithm 2: use Gauss' formula sum = ((N + 1) * N) / 2; 1 step! When the running time is independent of the size of the input, it is called constant time. -------------------------------------------------------------------------------- Comparing factorial algorithms Task: Print out the factorials of the numbers from 1 to N. Algorithm 1: compute each factorial in turn (the bonehead method). for (fact = 1; fact <= N; fact++) { 2N + 2 steps product = 1; N steps for (i = 1; i <= fact; i++) ? product *= i; cout << product << endl; N steps } Counting the steps is trickier, since the inner for loop doesn't always run the same number of times. In terms of fact, the inner for loop takes 3(fact) + 2 steps. So we must compute the sum of 3(fact) + 2 as fact goes from 1 to N. Gauss' rule helps us out here; we come up with 2N + 3/2 (N + 1)(N). Multiplying out, and adding the 2N + 2 + N + N steps from above, gives a total of 3/2 N2 + 15/2 N + 2 basic steps. This is called quadratic time, since it depends on the square of the input size. Algorithm 2: compute each factorial from the previous one. product = 1; 1 step for (fact = 1; fact <= N; fact++) { 2N + 2 steps product *= fact; N steps cout << product << endl; N steps } = 4N + 3 steps Linear time! We now have concrete evidence that Algorithm 2 is more time efficient. -------------------------------------------------------------------------------- Who needs basic steps? As it turns out, counting the exact number of basic steps is really unnecessary. It's only the "biggest" term that's important. Algorithm 1's biggest term is N2; Algorithm 2's biggest term is N. The coefficients and smaller terms aren't significant in the long run. To demonstrate, suppose we have a computer which can execute a basic step in 0.0001 of a second. # steps N = 20 N = 100 N = 1000 N + 5 0.0025 s 0.0105 s 0.1005 s N + 10 0.0030 s 0.0110 s 0.1010 s 2N 0.0040 s 0.0200 s 0.2000 s N2 0.0400 s 1 s 100 s N2 + N 0.0420 s 1.01 s 100.1 s Note that constants added or multiplied don't have a particularly large impact on the execution time. But the difference between the first three and the last two is substantial, especially as N gets larger. If we expect N to be big, we care a lot whether the running time depends on N or N2. -------------------------------------------------------------------------------- Asymptotic Complexity Big scary words! Asymptotic complexity is what allows us to group together algorithms according to their largest term, ignoring constants, coefficients, and smaller terms. The most common classes, specified using "big O" notation (more on that in a sec): O(1) Constant time Rare O(log N) Logarithmic time Desirable O(N) Linear time Okay O(N2) Quadratic time Less desirable O(N3) Cubic time If necessary... O(2N) Exponential time Impractical O() Doubly-exponential time Forget it! "Big O" Notation The "O" stands for "Order". What does it mean? A function f(n) is of order g(n), O(g(n)), if there is a constant c > 0 such that for all but a finite number of values of n (which is an integer), f(n) <= c * g(n). Uh, right, but what does it mean? -------------------------------------------------------------------------------- The Meaning of Big O Basically, it means that you could slow down the machine using g(n) enough so that, past some point, the machine using f(n) would always outperform it. To illustrate, suppose f(n) = 2n + 3. Choose g(n) = n and c = 5. Now suppose f(n) = n2 and you choose g(n) = n. No matter what value you choose for c, or equivalently, how much you raise the line g(n) = n, f(n) will always exceed it past some point. In this case, f(n) is not of order g(n); that is, n2 is not O(n). -------------------------------------------------------------------------------- More examples of demonstrating order Show that 7n + 45 is O(n). f(n) = 7n + 45, g(n) = n 7n + 45 <= 12n, when n <= 9 7n + 45 <= 52n, when n >= 1 Trick: add the coefficients! Show that 2n2 + 17n + 2 is O(n2). f(n) = 2n2 + 17n + 2, g(n) = n2 2n2 + 17n + 2 <= 21 n2, when n >= 1 Show that n3 + 100n2 is O(n3). f(n) = n3 + 100n2, g(n) = n3 n3 + 100n2 <= 101 n3, when n >= 1 Note that you can argue that 7n + 45 is O(n2) or any other "higher" class, but we're always interested in the "lowest" class to which it belongs. -------------------------------------------------------------------------------- Searching Algorithms The goal: Find the position of an element x in an array (apvector) of integers. Issues: Are the array elements in order? Does x appear more than once? Does x appear at all? Where exactly does x appear? Setup for all examples: #include "apvector.h" const int N = ...; apvector Numbers(10); ...and assume all N positions of Numbers have already been filled (either manually, or through keyboard or file input). -------------------------------------------------------------------------------- Searching an unordered array Suppose the elements in Numbers are not in any particular order. There's not much choice but to start at one end or the other and examine each element in turn until we find x, or we reach the other end of the array. We write the search routine as a function that returns either the position of x in the array, or -1 if it doesn't occur. int UnorderedSearch(int x, apvector Numbers) { int pos; int N = Numbers.length(); for (pos = 0; pos < N; pos++) { if (Numbers[pos] == x) return (pos); } return -1; } This finds the first occurrence of x. To find the last occurrence, we'd change the loop to: for (pos = N - 1; pos >= 0; pos--) What can be said about the worst case time efficiency? For this algorithm, the worst case forces the for loop to run the maximum number of iterations. This happens when x doesn't occur in the array at all; the number of basic steps in that case is 3N + 3. So this is an O(n) algorithm, which is why it's commonly called linear search. -------------------------------------------------------------------------------- Searching an ordered array Suppose now that the numbers in the array are in non-decreasing order. How can we take advantage of this? 2 3 5 9 14 21 23 30 Answer: use linear search, but stop as soon as you find a number larger than the one you're looking for. The modified code: int OrderedLinearSearch(int x, apvector Numbers) { int pos; int N = Numbers.length(); for (pos = 0; pos < N; pos++) { if (Numbers[pos] == x) return pos; if (Numbers[pos] > x) return -1; } return -1; } Does this improve efficiency? It's still true that the worst case occurs when x doesn't appear in the array. For some values of x, we no longer have to search the whole array to find this out. But if x is larger than the largest value in the array, we still have to do all iterations of the for loop to find out. The running time is still O(n). -------------------------------------------------------------------------------- Quadratic Search Can't we do better? Not surprisingly, yes! Suppose instead of looking at every single element, we look at every other element... or every third element... or every fourth element... This would allow us to pinpoint a smaller region of the array in which we can then do a linear search. Algorithm sketch: choose jump size take jumps until x must lie between current position and next jump perform linear search from current position The big question is what value to use for the jump size. We'll leave that hanging for a moment and look at some code. int QuadraticSearch(int x, apvector Numbers) { int jump; jump = ?????; for (pos = 0; (pos + jump) < N && Numbers[pos + jump] <= x; pos += jump); for ( ; pos < N; pos++) { if (Numbers[pos] == x) return pos; if (Numbers[pos] > x) return -1; } return -1; } -------------------------------------------------------------------------------- Choosing jump size Effects of some jump sizes: Jump size Maximum # of jumps Maximum # of single steps N = 10 N = 1,000 N = 1,000,000 N = 1,000,000 1 10 1,000 1,000,000 1 N/10 10 10 10 100,000 log N 3 100 50,000 20 sqrt(N) 3 32 1,000 1,000 Maximum number of jumps determines how many times the first for loop may be performed; maximum number of single steps determines the potential number of iterations of the second for loop. We want the sum of those numbers to be as small as possible. We also know that: max # of jumps * max # of single steps = N The best choice, therefore, is when: max # of jumps = max # of single steps So choose jump size of . jump = int(sqrt(N)); // Don't forget to include math.h! The algorithm is O(), which is better than O(n)! -------------------------------------------------------------------------------- Binary search Suppose you're trying to find a name in the phone book. What method do you use? You open to some page in the middle, check to see whether the name you're looking for is there, then, if it's not, you restrict yourself to looking either before or after that page, as appropriate. You continue in this way until you've finally narrowed the search down to the page containing the name. This general strategy is referred to as "divide-and-conquer". You can do the same thing to search an array, where "middle" is a much more precise term. For the phone book, you use your fingers to delimit the part of the book you're currently searching. For an array, we use integer variables as markers of the left and right ends of the search area. Here's an English description of the algorithm. set left and right markers to left and right ends of array repeat these steps calculate midpoint if element at midpoint is x, return position if element at midpoint is less than x, move left marker to midpoint if element at midpoint is greater than x, move right marker to midpoint until x is found OR left and right markers meet (or cross) 2 3 5 9 14 21 23 30 56 Try searching for: 14, 3, 29 -------------------------------------------------------------------------------- Binary search code Here it is in code. Note the slight refinement: the markers aren't moved exactly to the midpoint, but one past it in the appropriate direction, since the element at the midpoint has already been compared against x and doesn't need to be included in the area remaining to be searched. This also implies that if the left and right markers are equal, the element they mark hasn't been compared against x yet. int BinarySearch(int x, apvector Numbers) { int left, right, mid; left = 1; right = Numbers.length() - 1; while (left <= right) { mid = (left + right) / 2; if (Numbers[mid] == x) { return mid; } else if (Numbers[mid] < x) { left = mid + 1; } else if (Numbers[mid] > x) { // actually unnecessary to test right = mid - 1; } return -1; } -------------------------------------------------------------------------------- Binary search time efficiency How time efficient is it? Or, restated, what is the maximum number of times the while loop might execute? The worst case is when we are forced to keep dividing the array in half until the left and right markers cross. That happens only when the element we're looking for isn't there. How many times can you divide an array in half? Math reminder: When 2k = n, then log2 n = k. Or, rephrased: k is the number of times you must double 1 to get to n. Or, equivalently: k is the number of time you can divide n by 2 before reaching 1. This algorithm takes O(log2 n) time, AKA logarithmic time, which is even better than O() time! -------------------------------------------------------------------------------- Search recap: Unordered array linear search O(n) Ordered array linear search O(n) quadratic search O() binary search O(log2 n) -------------------------------------------------------------------------------- Warm-up for sorting The Dutch East Indies Flag problem: Given an array of N elements, where each element is either 'red' or 'white', rearrange the elements so all the reds are to the left of all the whites. Restriction: the only way you may reposition an element is by swapping it with another. Start: R R W R W W R W R Finish: R R R R R W W W W Strategy: Accumulate reds from the left end of the array; accumulate whites from the right end of the array. Keep track of how many reds/whites have been correctly positioned. Intermediary state: R R R ? ? ? ? W W ­ redMarker ­ whiteMarker Algorithm: if redMarker points to a red, advance it one to the right if whiteMarker points to a white, advance it one to the left If neither of these is true, redMarker must point to a white, and whiteMarker to a red. Swap the white and the red, then advance both redMarker and whiteMarker toward the center continue this until redMarker and whiteMarker meet or cross -------------------------------------------------------------------------------- The solution in code enum Colors { RED, WHITE }; int redMarker, whiteMarker; apvector Flag(N); redMarker = 0; whiteMarker = N - 1; while (redMarker < whiteMarker) { if (Flag[redMarker] == RED) { redMarker++; } else if (Flag[whiteMarker] == WHITE) { whiteMarker--; } else { swap(Flag[redMarker], Flag[whiteMarker]); redMarker++; whiteMarker--; } } Here's the swap function, overloaded for the Colors type: void swap(Colors& c1, Colors& c2) { Colors ctemp; ctemp = c1; c1 = c2; c2 = ctemp; } -------------------------------------------------------------------------------- A more difficult variation The solution to the Dutch East Indies flag problem is O(n). (Right?) Here's a tougher variation, which can also be solved with an O(n) algorithm. See if you can solve it yourself - we'll look at the solution next lecture. The Dutch National Flag problem: The Dutch National Flag has 3 colors: red, white, and blue vertical stripes, in that order. Given an array of N elements, where each element is either 'red', 'white', or 'blue', rearrange the elements so all the red are to the left, all blues to the right, and all the whites in the middle. Again, you may only use swap operations to rearrange the elements. Hint: with two colors, we used two markers... Start: B R W B W W R B R Finish: R R R W W W B B B