How do I use functools for partial functions, caching, and reduction?

· Category: Python Programming

Short answer

The functools module provides tools for higher-order programming. partial freezes some arguments of a function, lru_cache memoizes results, and reduce cumulatively applies a function to an iterable.

Steps

  1. Import functools.
  2. Use partial to create specialized functions.
  3. Apply @lru_cache to expensive deterministic functions.
  4. Use reduce for aggregation when a loop would be verbose.
from functools import partial, lru_cache, reduce
import operator

# partial
basetwo = partial(int, base=2)
print(basetwo("1010"))  # 10

# lru_cache
@lru_cache(maxsize=None)
def fib(n):
    if n < 2:
        return n
    return fib(n - 1) + fib(n - 2)

print(fib(100))

# reduce
product = reduce(operator.mul, [1, 2, 3, 4], 1)
print(product)  # 24

Tips

  • lru_cache keys must be hashable; convert lists to tuples if needed.
  • Use functools.wraps when writing decorators to preserve metadata.
  • reduce is often clearer than an explicit loop for simple aggregations.
  • cached_property (Python 3.8+) caches the result of a property method.

Common issues

  • An unbounded lru_cache on a function with many unique arguments can consume excessive memory.
  • partial applies arguments left-to-right; for right-side fixing, define a lambda or use keyword arguments.
  • Mutable default arguments combined with partial can lead to shared state surprises.