How do I use temporary files and in-memory buffers in Python?

· Category: Python Programming

Short answer

The tempfile module creates secure temporary files and directories that are automatically cleaned up. The io module provides StringIO and BytesIO for in-memory text and binary streams that behave like files.

Steps

  1. Use tempfile.NamedTemporaryFile or TemporaryDirectory.
  2. Use io.StringIO or io.BytesIO to avoid real disk I/O.
import tempfile
import io

# Temporary file
with tempfile.NamedTemporaryFile(mode="w+", delete=False, suffix=".txt") as tmp:
    tmp.write("Temporary data")
    tmp.seek(0)
    print(tmp.read())

# In-memory text buffer
buffer = io.StringIO()
buffer.write("Line 1
Line 2
")
buffer.seek(0)
print(buffer.read())

Tips

  • delete=False keeps the file after closing; use it when another process needs to read the file.
  • tempfile.gettempdir() returns the system's temporary directory path.
  • StringIO is useful for capturing print() output during tests.
import sys

old_stdout = sys.stdout
sys.stdout = buffer = io.StringIO()
print("captured")
sys.stdout = old_stdout
print(buffer.getvalue())

Common issues

  • Temporary files with delete=True are removed as soon as they are closed, making them inaccessible to external tools.
  • Filling BytesIO with large data still consumes memory; use real temporary files for very large buffers.
  • Forgetting to seek(0) before reading from a buffer you just wrote to results in an empty read.