Lecture 34 - Priority queues & binary heaps

Logistics

  • HW6 is being graded
  • HW7 was released (due Dec 5 - no corrections)
  • Discussion tomorrow will go over HW7
  • After Thanksgiving, 4 lectures + 1 review (Dec 10)
  • Final on Dec 17, 12-2

Learning objectives

By the end of today, you should be able to:

  • Explain what a priority queue is and how binary heaps implement them
  • Analyze heap operations (insert, extract-max, peek) and their O(log n) complexity
  • Use Rust's BinaryHeap for priority-based problems
  • Understand heapsort and why it achieves O(n log n)

Motivation: Not all tasks are equal

Regular queue (FIFO): First come, first served

But what if tasks have different importance?

Hospital emergency room:

  • Patient A: Broken finger (can wait)
  • Patient B: Heart attack (urgent!)
  • Patient C: Flu symptoms (can wait)

Question: Should we serve in arrival order, or by urgency?

What is a priority queue?

Priority Queue: A data structure where each element has a priority

  • Insert: Add element with a priority
  • Extract-max (or extract-min): Remove element with highest (or lowest) priority

Not FIFO! Order depends on priority, not insertion time.

Example:

Insert (Task A, priority=5)
Insert (Task B, priority=10) 
Insert (Task C, priority=3)

Extract-max -> Task B (priority 10)
Extract-max -> Task A (priority 5)
Extract-max -> Task C (priority 3)

Naive implementations

Approach 1: Unsorted Vec

  • Insert: O(1) - just push
  • Extract-max: O(n) - must scan all elements

Approach 2: Sorted Vec

  • Insert: O(n) - must find position and shift
  • Extract-max: O(1) - just pop last element

Can we do better? Yes! O(log n) for both operations using a binary heap!

What is a binary heap?

Trees

         A         Root (top node, no parent)
        / \
       B   C       Children of A, Parents of D/E/F
      /   / \
     D   E   F     Leaves (no children)

More definitions

  • Height of a tree - how many "rows" or generations
  • Binary tree - at most 2 children per parent
    • A complete binary tree - all levels filled except possibly the last, which fills left-to-right
      • A binary heap is a complete binary tree that satisfies the heap property:
        • Parent >= both children everywhere

Today we'll focus on max-heaps (Rust's BinaryHeap is a max-heap)

Important: Two different "heaps"!

Just like with "stack", "heap" means TWO completely different things:

Heap memory:

  • Region of memory where dynamically allocated data lives
  • String, Vec, Box store their data on the heap
  • Accessed via pointers from the stack
  • Memory management concept

Heap data structure (today):

  • Binary tree with the heap property
  • Used to implement priority queues
  • Has nothing to do with memory layout!

Same word, completely different concepts! Context tells you which one.

Example: Max-heap

         42              Root is largest
        /  \
      30    25           Parents >= children
     / \    / \
   10  20  15  8
   /
  5

Parent >= children everywhere:
  42 >= 30, 25
  30 >= 10, 20
  25 >= 15, 8
  10 >= 5

Complete binary tree (all levels filled except the last, which is filled from the left)

Not a heap:

         20              Violates heap property!
        /  \
      30    25           30 > 20 (parent)

Complete binary tree?

Complete: All levels filled except possibly the last, which fills left to right

Complete (valid heap structure):

       42
      /  \
    30    25
   / \    /
  10 20  15      Last level fills left to right

Not complete:

       42
      /  \
    30    25
      \    /
      20  15      Gap on left!

Why complete? Allows efficient array representation!

A clever trick for array representation

Store heap in an array, level by level:

Tree:
         42
        /  \
      30    25
     / \    / \
   10  20  15  8

Array: [42, 30, 25, 10, 20, 15, 8]
Index:  0   1   2   3   4   5   6

Parent-child relationships:

  • Parent of node at index i: (i - 1) / 2
  • Left child of node at index i: 2*i + 1
  • Right child of node at index i: 2*i + 2

Example:

  • Node at index 1 (30): parent = (1-1)/2 = 0 (42)
  • Node at index 0 (42): left child = 2*0+1 = 1 (30)
  • Node at index 0 (42): right child = 2*0+2 = 2 (25)

No pointers needed! Just arithmetic

Practice question

Array: [50, 40, 35, 25, 30, 30, 15]

Questions:

  1. What are the children of the element at index 1 (value 40)?
  2. What is the parent of the element at index 4 (value 30)?
  3. Is this a max heap?

Heap Operation 1: Insert (push) (TC 12:35)

Goal: Add new element while maintaining heap property

Algorithm:

  1. Add element to the end (bottom-right of tree)
  2. "Bubble up" (or "sift up"): Swap with parent if larger
  3. Repeat until heap property restored

Example: Insert 35 into heap [42, 30, 25, 10, 20, 15, 8]

Step 0: Existing heap
         42
        /  \
      30    25
     / \    / \
   10  20  15  8

Step 1: Add to end
         42
        /  \
      30    25
     / \    / \
   10  20  15  8
   /
  35  

Step 2: 35 > parent (10), swap
         42
        /  \
      30    25
     / \    / \
   35  20  15  8
   /
  10

Step 3: 35 > parent (30), swap
         42
        /  \
      35    25
     / \    / \
   30  20  15  8
   /
  10

Step 4: 35 < parent (42), done!
Array: [42, 35, 25, 30, 20, 15, 8, 10]

Time complexity: O(log n) - at most height of tree (log n levels)

Operation 2: Extract-max (pop)

Goal: Remove root (max element) while maintaining heap property

Algorithm:

  1. Replace root with last element
  2. "Bubble down" (or "sift down"): Swap with larger child if smaller
  3. Repeat until heap property restored

Example: Extract-max from [42, 35, 25, 30, 20, 15, 8, 10]

Step 0: Start here:
         42
        /  \
      35    25
     / \    / \
   30  20  15  8
   /
  10
  
Step 1: Remove root (42), replace with last element (10)
         10      
        /  \
      35    25
     / \    / \
   30  20  15  8

Step 2: 10 < both children (35, 25), swap with larger (35)
         35
        /  \
      10    25
     / \    / \
   30  20  15  8

Step 3: 10 < both children (30, 20), swap with larger (30)
         35
        /  \
      30    25
     / \    / \
   10  20  15  8

Step 4: 10 has no children, done!
Array: [35, 30, 25, 10, 20, 15, 8]

Time complexity: O(log n) - at most height of tree

Operation 3: Heapify

Goal: Creating a heap from an array

Naive approach: Insert n elements one by one

  • Each insert is O(log n)
  • Total: O(n log n)

Here's a better idea:

  • Start with a complete binary heap, unsorted
  • Starting from the second-to-last row, "sift down"
Total work = n/2 * 0 + n/4 * 1 + n/8 * 2 + n/16 * 3 + ...
           = n * (1/4 + 2/8 + 3/16 + 4/32 + ...)
           = n 
           = O(n)

Why this is faster: We do less work on most nodes because we start from the bottom where most nodes live!

Rust's BinaryHeap::from(vec) uses this O(n) algorithm internally.

Complexity summary

OperationTime ComplexityWhy
Insert (push)O(log n)Bubble up at most log n levels
Extract-max (pop)O(log n)Bubble down at most log n levels
Peek (top)O(1)Just read first element
Build heap (naive)O(n log n)Insert n times
Build heap (heapify)O(n)See the last slide!

Heapsort: Why heaps make a great sorting algorithm

Key insight: A max-heap gives us elements in descending order when we repeatedly pop!

Heapsort algorithm:

  1. Build a heap from the array - O(n) time with special algorithm
  2. Repeatedly extract max and place at end - O(n log n) time

Total complexity: O(n log n)

Example:

#![allow(unused)]
fn main() {
fn heapsort(mut nums: Vec<i32>) -> Vec<i32> {
    let mut heap = BinaryHeap::from(nums);  // Build heap: O(n)
    let mut sorted = Vec::new();

    loop {                                   // n times, each O(log n)
        match heap.pop() {
            Some(max) => sorted.push(max),
            None => break,
        }
    }

    sorted.reverse();  // We got descending, reverse for ascending
    sorted
}
}

Total: O(n) + O(n log n) + O(n) = O(n log n)

Using BinaryHeap in Rust

Good news: Rust provides BinaryHeap<T> in the standard library!

We can use debug printing to see the array view:

use std::collections::BinaryHeap;

fn main() {
    let mut heap = BinaryHeap::new();

    // Insert elements
    heap.push(10);
    heap.push(30);
    heap.push(20);
    heap.push(5);

    println!("Heap: {:?}", heap);  // Order not guaranteed

    println!("Max: {:?}", heap.peek()); 

    // Extract max elements
    loop {     
        match heap.pop() {
            Some(max) => println!("Popped: {}", max),
            None => break,
        }
    }
}

Application Analysis 1: Top K elements (TC 12:45)

Problem: Find the k largest elements in a list

use std::collections::BinaryHeap;

fn top_k(nums: Vec<i32>, k: usize) -> Vec<i32> {
    let mut heap = BinaryHeap::from(nums); // O(n)
    let mut result = Vec::new();

    for _ in 0..k { // k times
        match heap.pop() {
            Some(max) => result.push(max), // O (log n)
            _ => {},
        };
    }

    result
}

fn main() {
    let nums = vec![3, 1, 4, 1, 5, 9, 2, 6];
    let top_3 = top_k(nums, 3);
    println!("Top 3: {:?}", top_3); 
}

Complexity: O(n + k log n) = O(n) for small k

Application Analysis 2: Merge K sorted lists

Problem: You have K sorted lists, merge into one sorted list

List 1: [1, 4, 7]
List 2: [2, 5, 8]
List 3: [3, 6, 9]

Result: [1, 2, 3, 4, 5, 6, 7, 8, 9]

Algorithm with min-heap:

  1. Put first element of each list in min-heap (with list index)
  2. Extract min, add next element from that list
  3. Repeat until heap empty

Complexity: O(N log K) where N = total elements, K = number of lists

Much better than repeatedly merging pairs: O(NK)!

Question: Why don't we use this for MergeSort?

Application Analysis 3: Running median

Demo at the board

Key takeaways

  • Priority Queue: Extract elements by priority, not insertion order
  • Binary Heap: Complete binary tree with heap property (parent ≥ children)
  • Array representation: Parent at (i-1)/2, children at 2i+1 and 2i+2
  • Operations: Insert and extract-max both O(log n)
  • Rust's BinaryHeap: Max-heap, use Reverse for min-heap (see appendix)
  • Applications: Top K, task scheduling, graph algorithms
NeedUse
Pop by priorityBinaryHeap
Pop oldest firstQueue (VecDeque)
Pop newest firstStack (Vec)
Binary Heap (Implementation):
         42              Max at root
        /  \
      30    25           All parents >= children
     / \    / \
   10  20  15  8

Array: [42, 30, 25, 10, 20, 15, 8]

Operations: O(log n) insert/extract, O(1) peek

Activity time!

Joey swaps in.

Appendix: Implementing Ord to define priority

use std::collections::BinaryHeap;
use std::cmp::Ordering;

#[derive(Eq, PartialEq)]
struct Task {
    name: String,
    priority: u32,
}

impl Ord for Task {
    fn cmp(&self, other: &Self) -> Ordering {
        self.priority.cmp(&other.priority)
    }
}

impl PartialOrd for Task {
    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
        Some(self.cmp(other))
    }
}

fn main() {
    let mut tasks = BinaryHeap::new();

    tasks.push(Task { name: "Low priority".to_string(), priority: 1 });
    tasks.push(Task { name: "High priority".to_string(), priority: 10 });
    tasks.push(Task { name: "Medium priority".to_string(), priority: 5 });

    while let Some(task) = tasks.pop() {
        println!("{} (priority {})", task.name, task.priority);
    }
    // Output:
    // High priority (priority 10)
    // Medium priority (priority 5)
    // Low priority (priority 1)
}

Appendix: Min-heap using Reverse

Problem: Rust's BinaryHeap is max-heap, but we want min-heap

Solution: Use Reverse wrapper to flip comparisons

use std::collections::BinaryHeap;
use std::cmp::Reverse;

fn main() {
    let mut min_heap = BinaryHeap::new();

    // Wrap values in Reverse
    min_heap.push(Reverse(10));
    min_heap.push(Reverse(30));
    min_heap.push(Reverse(20));
    min_heap.push(Reverse(5));

    // Extract minimum elements
    while let Some(Reverse(min)) = min_heap.pop() {
        println!("Popped: {}", min);
    }
    // Output: 5, 10, 20, 30 (ascending!)
}

Reverse flips the ordering: Smallest is now "largest" in heap