🐍 Python decorators are the most elegant way to optimize your LLM apps without cluttering the main code.
Working with LLMs means dealing with slow, expensive, and unpredictable APIs. These 5 decorators solve the most common problems:
1. @lru_cache (in-memory caching) — avoids repeated API calls for the same prompt in the same session. Send the same text twice and the second call is instant.
2. @cache.memoize (disk caching with diskcache) — same as above but persists between runs, stored in SQLite. Ideal when your script runs multiple times.
3. @retry with tenacity — automatically handles network failures, timeouts and “502 Bad Gateway” with exponential backoff. Define how many retries and how long to wait between them.
4. @limits + @sleep_and_retry (rate limiting) — controls call frequency to stay within the provider’s RPM (requests per minute) limit. The 4th request automatically waits until it can proceed.
5. @prompt with magentic + Pydantic — turns a function into a direct LLM call with structured output validated by Pydantic. No boilerplate, no manual parsing.
💡 Explanation in a nutshell#
Each decorator encapsulates an infrastructure problem you’d otherwise solve inline with try/except, time.sleep(), and conditionals. The result is cleaner, more testable, and more production-ready code.
More information at the link 👇
