Quicksort
1
Lecture 33 (Sorting 3)
CS61B, Spring 2025 @ UC Berkeley
Slides credit: Josh Hug
Quicksort
Lecture 32, CS61B, Spring 2025
Quicksort
Quicksort Runtime Analysis
Quicksort Variants
Quick Select (median finding)
Sorts So Far
| Best Case Runtime | Worst Case Runtime | Space | Demo | Notes |
Θ(N2) | Θ(N2) | Θ(1) | | ||
(in place) | Θ(N)* | Θ(N log N) | Θ(1) | Bad cache (61C) performance. | |
Θ(N log N) | Θ(N log N) | Θ(N) | Fastest of these. | ||
Insertion Sort (in place) | Θ(N) | Θ(N2) | Θ(1) | Best for small N or almost sorted. |
See this link for bonus slides on Shell's Sort, an optimization of insertion sort.
The Core Idea of Tony’s Sort: Partitioning
A partition of an array, given a pivot x, is a rearrangement of the items so that:
Which partitions are valid?
5
550
10
4
10
9
330
i
4
5
9
10
10
330
550
5
9
10
4
10
550
330
5
4
9
10
10
550
330
5
9
10
4
10
550
330
A.
C.
B.
D.
j
j
j
j
Called the ‘pivot’.
Partition Sort, a.k.a. Quicksort
Observations:
5
3
2
1
8
4
6
7
3
2
1
4
7
8
6
5
3
2
1
4
5
7
8
6
5
2
1
3
4
5
6
7
8
5
Q: How would we use this operation for sorting?
Job Interview Style Question (Partitioning)
Given an array of colors where the 0th element is white (and maybe a few more), and the remaining elements are red (less) or blue (greater), rearrange the array so that all red squares are to the left of the white square, the white squares end up together, and all blue squares are to the right. Your algorithm must complete in Θ(N) time (no space restriction).
3
1
2
4
6
8
7
3
4
1
2
6
7
8
Example of a valid output
Another example of a valid output
6
8
3
1
2
7
4
Input
6
6
6
Quick Sort
Quick sorting N items:
32
15
2
17
19
26
41
17
17
Input:
unsorted
Quick Sort
Quick sorting N items:
32
15
2
17
19
26
41
17
17
Input:
partition(32)
Quick Sort
Quick sorting N items:
15
2
17
19
26
17
17
32
41
Input:
<= 32
>= 32
in its place
partition(32)
Quick Sort
Quick sorting N items:
15
2
17
19
26
17
17
32
Input:
in its place
41
partition(32)
Quick Sort
Quick sorting N items:
partition(32)
partition(15)
partition(2)
partition(17)
partition(19)
partition(17)
partition(17)
partition(26)
x
x
x
x
x
x
x
2
15
17
17
17
19
26
32
41
Input:
in its place
in its place
in its place
in its place
in its place
in its place
in its place
in its place
x
Quick Sort
Quick sorting N items:
partition(32)
partition(15)
partition(2)
partition(17)
partition(19)
partition(17)
partition(17)
partition(26)
x
x
x
x
x
x
x
2
15
17
17
17
19
26
32
41
Input:
in its place
in its place
in its place
in its place
in its place
in its place
in its place
in its place
in its place
If you don't fully trust the recursion, see these extra slides for a complete demo.
x
Partition Sort, a.k.a. Quicksort
Quick sorting N items:
32
15
2
17
19
26
41
17
17
partition(32)
15
2
17
19
26
17
17
32
41
<= 32
>= 32
in its place
Quicksort
Quicksort was the name chosen by Tony Hoare for partition sort.
How fast is Quicksort? Need to count number and difficulty of partition operations.
Theoretical analysis:
Quicksort Runtime Analysis
Lecture 32, CS61B, Spring 2025
Quicksort
Quicksort Runtime Analysis
Quicksort Variants
Quick Select (median finding)
Best Case: Pivot Always Lands in the Middle
Only size 1 problems remain, so we’re done.
Best Case Runtime?
What is the best case runtime?
Only size 1 problems remain, so we’re done.
Best Case Runtime?
Only size 1 problems remain, so we’re done.
Total work at each level:
≈ N
≈N/2 + ≈N/2 = ≈N
≈N/4 * 4 = ≈N
Overall runtime:
Θ(NH) where H = Θ(log N)
so: Θ(N log N)
Worst Case: Pivot Always Lands at Beginning of Array
Give an example of an array that would follow the pattern to the right.
What is the runtime Θ(∙)?
Worst Case: Pivot Always Lands at Beginning of Array
Give an example of an array that would follow the pattern to the right.
What is the runtime Θ(∙)?
Quicksort Performance
Theoretical analysis:
Compare this to Mergesort.
Recall that Θ(N log N) vs. Θ(N2) is a really big deal. So how can Quicksort be the fastest sort empirically? Because on average it is Θ(N log N).
Argument #1: 10% Case
Suppose pivot always ends up at least 10% from either edge (not to scale).
Work at each level: O(N)
N
N/10
9N/10
N/100
9N/100
9N/100
81N/100
Punchline: Even if you are unlucky enough to have a pivot that never lands anywhere near the middle, but at least always 10% from the edge, runtime is still O(N log N).
Argument #2: Quicksort is BST Sort
5
3
2
1
8
4
6
7
3
2
1
4
7
8
6
2
1
4
6
8
5
3
5
7
5
3
7
2
4
6
8
1
Key idea: compareTo calls are same for BST insert and Quicksort.
Reminder: Random insertion into a BST takes O(N log N) time.
Empirical Quicksort Runtimes
For N items:
Empirical histogram for quicksort compare counts (10,000 trials with N = 1000)
Lots of arrays take 12,000ish compares to sort with Quicksort.
A very small number take 15,000ish compares to sort with Quicksort.
Chance of taking 1,000,000ish compares is effectively zero.
Quicksort Performance
Theoretical analysis:
Compare this to Mergesort.
Why is it faster than mergesort?
With extremely high probability!!
For our pivot/partitioning strategies: Sorted or close to sorted.
Sorting Summary (so far)
Listed by mechanism:
Listed by memory and runtime:
| Memory | Time | Notes |
Heapsort | Θ(1) | Θ(N log N) | Bad caching (61C) |
Insertion | Θ(1) | Θ(N2) | Θ(N) if almost sorted |
Mergesort | Θ(N) | Θ(N log N) | |
Quicksort | Θ(log N) (call stack) | Θ(N log N) expected | Fastest sort |
More Quicksort Origins
Amusingly, Quicksort was the wrong tool for the job. Two issues:
Philosophies for Avoiding Worst Case Quicksort Behavior
Lecture 32, CS61B, Spring 2025
Quicksort
Quicksort Runtime Analysis
Quicksort Variants
Quick Select (median finding)
Partition Sort, a.k.a. Quicksort
Quicksorting N items: (Demo)
32
15
2
17
19
26
41
17
17
partition(32)
15
2
17
19
26
17
17
32
41
<= 32
>= 32
in its place
Run time is Θ(N log N) in the best case, Θ(N2) in the worst case, and Θ(N log N) in the average case.
Avoiding the Worst Case: Question from Last Time
If pivot always lands somewhere “good”, Quicksort is Θ(N log N). However, the very rare Θ(N2) cases do happen in practice, e.g.
What specific array inputs yield bad Quicksort runtime?
Recall, our version of Quicksort has the following properties:
6
8
3
1
2
7
4
3
1
2
4
6
8
7
Avoiding the Worst Case: Question from Last Time
If pivot always lands somewhere “good”, Quicksort is Θ(N log N). However, the very rare Θ(N2) cases do happen in practice, e.g.
What can we do to avoid worst case behavior?
Recall, our version of Quicksort has the following properties:
6
8
3
1
2
7
4
3
1
2
4
6
8
7
Avoiding the Worst Case: My Answers
What can we do to avoid running into the worst case for QuickSort?
Four philosophies:
1. Randomness: Pick a random pivot or shuffle before sorting.
2. Smarter pivot selection: Calculate or approximate the median.
3. Introspection: Switch to a safer sort if recursion goes to deep.
4. Preprocess the array: Could analyze array to see if Quicksort will be slow. No obvious way to do this, though (can’t just check if array is sorted, almost sorted arrays are almost slow).
Philosophy 1: Randomness (My Preferred Approach)
If pivot always lands somewhere “good”, Quicksort is Θ(N log N). However, the very rare Θ(N2) cases do happen in practice, e.g.
Dealing with bad ordering:
The second strategy requires care in partitioning code to avoid Θ(N2) behavior on arrays of duplicates.
Philosophy 2a: Smarter Pivot Selection (constant time pivot pick)
Randomness is necessary for best Quicksort performance! For any pivot selection procedure that is:
Dangerous input
7
2
3
4
6
1
8
5
The resulting Quicksort has a family of dangerous inputs that an adversary could easily generate.
Philosophy 2b: Smarter Pivot Selection (linear time pivot pick)
Could calculate the actual median in linear time.
Raises interesting question though: How do you compute the median of an array? Will talk about how to do this later today.
Philosophy 3: Introspection
Can also simply watch your recursion depth.
Perfectly reasonable approach, though not super common in practice.
Sorting Summary (so far)
Listed by mechanism:
Listed by memory and runtime:
| Memory | Time | Notes |
Heapsort | Θ(1) | Θ(N log N) | Bad caching (61C) |
Insertion | Θ(1) | Θ(N2) | Θ(N) if almost sorted |
Mergesort | Θ(N) | Θ(N log N) | |
Random Quicksort | Θ(log N) expected | Θ(N log N) expected | Fastest sort |
Hoare Partitioning
Lecture 32, CS61B, Spring 2025
Quicksort
Quicksort Runtime Analysis
Quicksort Variants
Quick Select (median finding)
Quicksort Flavors
We said Quicksort is the fastest, but this is only true if we make the right decisions about:
Suppose we run a speed test of Mergesort vs. Quicksort from last time, which had:
We’ll call this Quicksort L3S
Quicksort vs. Mergesort
Quicksort didn’t do so well!
| Pivot Selection Strategy | Partition Algorithm | Worst Case Avoidance Strategy | Time to sort 1000 arrays of 10000 ints |
Mergesort | N/A | N/A | N/A | 1.3 seconds |
Quicksort L3S | Leftmost | 3-scan | Shuffle | 3.2 seconds |
Note: These are unoptimized versions of mergesort and quicksort, i.e. no switching to insertion sort for small arrays.
Tony Hoare’s In-place Partitioning Scheme
Tony originally proposed a scheme where two pointers walk towards each other.
(Note: The demo we'll show is not the exact scheme Tony used)
Using this partitioning scheme yields a very fast Quicksort.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
19
32
2
26
41
17
17
Input:
L
G
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
19
32
2
26
41
17
17
Input:
L
G
Hello, lovely 15.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
19
32
2
26
41
17
17
Input:
L
G
Grrr…… 19
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
19
32
2
26
41
17
17
Input:
L
G
I dislike 17.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
19
32
2
26
41
17
17
Input:
L
G
Time to swap.
Time to swap.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
32
2
26
41
17
19
Input:
L
G
Swapped.
Swapped.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
32
2
26
41
17
19
Input:
L
Immediate trouble.
G
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
32
2
26
41
17
19
Input:
L
G
Trouble here, too.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
32
2
26
41
17
19
Input:
L
G
Time to swap.
Time to swap.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
Swapped!
Swapped!
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
2 is cool.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
26 is grossly large.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
41 is fine.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
26 is fine.
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
2 is no good… also hi L!!
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
2 is no good… also hi L!!
Hoare Partitioning
Create L and G pointers at left and right ends.
17
15
17
17
2
26
41
32
19
Input:
L
G
Time to swap.
Time to swap.
Hoare Partitioning
Create L and G pointers at left and right ends.
2
15
17
17
17
26
41
32
19
Input:
L
G
Old pivot.
New pivot.
New pivot.
Swapped.
Swapped.
Quicksort vs. Mergesort
Using Tony Hoare’s two pointer scheme, Quicksort is better than mergesort!
| Pivot Selection Strategy | Partition Algorithm | Worst Case Avoidance Strategy | Time to sort 1000 arrays of 10000 ints |
Mergesort | N/A | N/A | N/A | 1.3 seconds |
Quicksort L3S | Leftmost | 3-scan | Shuffle | 3.2 seconds |
Quicksort LTHS | Leftmost | Tony Hoare | Shuffle | 0.9 seconds |
Note: These are unoptimized versions of mergesort and quicksort, i.e. no switching to insertion sort for small arrays.
Quick Select (median finding)
Lecture 32, CS61B, Spring 2025
Quicksort
Quicksort Runtime Analysis
Quicksort Variants
Quick Select (median finding)
What If We Don’t Want Randomness?
Our approach so far: Use randomness to avoid worst case behavior, but some people don’t like having randomness in their sorting routine.
Another approach: Use the median (or an approximation) as our pivot.
Four philosophies:
1. Randomness: Pick a random pivot or shuffle before sorting.
2. Smarter pivot selection: Calculate or approximate the median.
3. Introspection: Switch to a safer sort if recursion goes to deep.
4. Try to cheat: If the array is already sorted, don’t sort (this doesn’t work).
This is what we’ve been using.
Philosophy 2a: Smarter Pivot Selection (linear time pivot pick)
The best possible pivot is the median.
Obvious approach: Just calculate the actual median and use that as pivot.
Goal: Come up with an algorithm for finding the median of an array. Bonus points if your algorithm takes linear time.
Philosophy 2a: Smarter Pivot Selection (linear time pivot pick)
Goal: Come up with an algorithm for finding the median of an array. Bonus points if your algorithm takes linear time.
The Selection Problem
Computing the exact median would be great for picking an item to partition around. Gives us a “safe quick sort”.
However, it turns out that partitioning can be used to find the exact median.
Quick Select
Goal, find the median:
Partition, pivot lands at 2.
Now pivot lands at 6.
Partition right subproblem, median can’t be to the left!
Pivot lands at 4. Are we done?
550
14
10
330
5
4
817
913
14
10
330
550
5
4
817
913
6
5
9
550
14
10
330
817
913
9
550
14
6
10
5
330
817
913
14
10
330
5
4
4
4
5
10
14
330
5
4
4
4
5
Worst case performance?
What is the worst case performance for Quick Select? Give an array that causes this worst case (assuming we always pick leftmost item as pivot).
Worst case performance?
What is the worst case performance for Quick Select? Give an array that causes this worst case (assuming we always pick leftmost item as pivot).
Worst asymptotic performance Θ(N2) occurs if array is in sorted order.
[1 2 3 4 5 6 7 8 9 10 … N]
[1 2 3 4 5 6 7 8 9 10 … N]
[1 2 3 4 5 6 7 8 9 10 … N]
…
[1 2 3 4 5 … N/2 … N]
Expected Performance
On average, Quick Select will take Θ(N) time.
~N compares
~N/2 compares
~N/4 compares
On average, pivot ends up about halfway
N + N/2 + N/4 + … + 1 = Θ(N)
Quicksort With Quickselect?
What if we used Quickselect to find the exact median?
| Pivot Selection Strategy | Partition Algorithm | Worst Case Avoidance Strategy | Time to sort 1000 arrays of 10000 ints | Worst Case |
Mergesort | N/A | N/A | N/A | 1.3 seconds | Θ(N log N) |
Quicksort L3S | Leftmost | 3-scan | Shuffle | 3.2 seconds | Θ(N2) |
Quicksort LTHS | Leftmost | Tony Hoare | Shuffle | 0.9 seconds | Θ(N2) |
Quicksort QSTH | QuickSelect | Tony Hoare | Exact Median | 4.9 seconds | Θ(N2) |
Median Identification
Is it possible to find the median in Θ(N) time?
Historical note: The authors of this paper include FOUR Turing Award winners (and Vaughan Pratt)
Let’s see how Exact Median Quicksort performs.
Quicksort vs. Mergesort
Quicksort using PICK to find the exact median (Quicksort PickTH) is terrible!
| Pivot Selection Strategy | Partition Algorithm | Worst Case Avoidance Strategy | Time to sort 1000 arrays of 10000 ints | Worst Case |
Mergesort | N/A | N/A | N/A | 1.3 seconds | Θ(N log N) |
Quicksort L3S | Leftmost | 3-scan | Shuffle | 3.2 seconds | Θ(N2) |
Quicksort LTHS | Leftmost | Tony Hoare | Shuffle | 0.9 seconds | Θ(N2) |
Quicksort PickTH | Exact Median | Tony Hoare | Exact Median | ~15 seconds | Θ(N log N) |
Note: These are unoptimized versions of mergesort and quicksort, i.e. no switching to insertion sort for small arrays.