How do I use itertools for efficient iteration in Python?

· Category: Python Programming

Short answer

The itertools module provides fast, memory-efficient tools for creating iterators. It includes infinite iterators (count, cycle, repeat), combinatoric generators (product, permutations, combinations), and iterator algebra (chain, groupby, islice).

Steps

  1. Import itertools.
  2. Choose the iterator that fits your pattern.
  3. Consume the iterator in a loop or convert to a list if needed.
import itertools

# Infinite counter
for i in itertools.count(10):
    if i > 13:
        break
    print(i)

# Combinations
print(list(itertools.combinations(["a", "b", "c"], 2)))

# Chain iterables
combined = itertools.chain([1, 2], [3, 4])
print(list(combined))

# Groupby (requires sorted input)
data = [("a", 1), ("a", 2), ("b", 3)]
for key, group in itertools.groupby(data, key=lambda x: x[0]):
    print(key, list(group))

Tips

  • itertools functions return iterators; they are lazy and do not store the entire result in memory.
  • zip_longest is useful when combining iterables of unequal length.
  • tee splits an iterator into multiple independent iterators.
  • accumulate computes running totals or other cumulative results.

Common issues

  • Passing a generator to itertools.tee stores the generated values in memory, which can be expensive.
  • groupby only groups consecutive equal elements; sort the data first if you need global grouping.
  • Consuming an iterator in one branch of a conditional and expecting it to restart in another branch fails.