36.5 Guardrails: avoid confident math errors

The LLM Math Problem

If you ask an LLM "What is 1234 * 5678?", it might hallucinate the last digit. LLMs are bad at arithmetic. They are language models, not calculators.

The Code Solution

Never let the LLM do the math.

Tell the LLM to write code to do the math. Python is great at math. The LLM is great at writing Python.

• Bad: "Calculate the sum of the sales column." (Model tries to sum tokens in its head).
• Good: "Write a script to calculate the sum of the sales column." (Model writes `df['sales'].sum()`, Python executes it, result is 100% accurate).

Where to go next

• Part XIII — Expert Mode