← Field Notes

Hallucinations Are Irrational

May 27, 2025

If I ask you (a human) to pick a number between 1 and 10, most likely you'll respond with something like 2 or 3 or 7. Maybe you'll think to ask: "Can I pick 1 or 10?" Sure, go ahead.

Now if I ask you (an AI chatbot) to pick a number between 1 and 10, you'll probably play along. You'll say 2 or 3 or 7 or 10 or 1 or any of the other likely candidates. But you'd still be following the instructions if you answered with e, π, √2, 9.99999, or 5.9464326—any point on the number line between 1 and 10. And suddenly, you're hallucinating!

What we call hallucinations are better described as mismatched assumptions. You say number but mean integer; AI hears number and thinks hmm, irrational, complex, transcendental, imaginary, surreal... maybe integer?

In this simple back-and-forth, it's easy enough to ask for clarification.

But when you're programming with an AI coding assistant, things quickly get much more complex (as in problems, not numbers).

You ask the computer to write a program, and it comes up with something that plausibly fits what you requested. And it turns out that whatever it wrote probably isn't what you wanted. There are just too many mismatched assumptions that go into any program for a one-shot reply to meet your requirements.

Once you start building code on a foundation of mismatched assumptions, it becomes increasingly difficult to reach the outcome you originally envisioned.

Article content

Regular readers of this column may recall the time that I found the Goat Book in a Little Free Library in Capitol Hill. Best LFL find ever. That book entirely changed how I look at programming in a way that provided excellent training for AI-paired programming.

Based on my experience with that book, I've come up with a method, based on Test-Driven Development, for writing prompts.

Here’s the secret: Write the tests before the code.

Don't let AI write a line of code before it's written tests.

Be firm. The AI wants to write code. Don't let it.

You'll get a bunch of pytest (if you're using Python) assertions. If you're not familiar with tests and assertions, now's the time to learn.

def test_number_is_integer():
    result = generate_number()
    assert isinstance(result, int)

def test_number_is_within_bounds():
    result = generate_number()
    assert 1 <= result <= 10

def test_number_is_not_float():
    result = generate_number()
    assert not isinstance(result, float)

def test_number_is_not_complex():
    result = generate_number()
    assert not isinstance(result, complex)

def test_number_is_random():
    results = {generate_number() for _ in range(100)}
    assert len(results) > 1  # detects function repeating the same value

def test_number_is_not_pi():
    result = generate_number()
    assert result != 3.14159, "The result should not be π (or an approximation of it)"

Here's the plain-language version of these tests:

Now you're ready to ask AI to write a program that satisfies your tests.

Article content

And the fun part -- especially when you're using Agent mode -- is that AI will iterate on writing programs to your exact specifications, using your tests as a guide, until it gets the code just right.

That's pretty much all I've been doing for the past six weeks or so.

Catch me if you can.

Article content

Another way to think about tests

Suppose I ask you to draw a bicycle.

Chances are you'll end up with something like the examples you find in Velocipedia's renderings of bicycle sketches: Missing parts, misshapen frames, entirely unroadworthy.

But what if we started by having you compile a list of all the parts of a bicycle? Your drawings would probably turn out much better from that one simple change.

And suppose whatever you sketched was automatically turned into a rendering and put through simulated testing? You'd be able to watch your failures crash and burn. You'd try again. Each time, you'd fail better. Eventually, you succeed.

← Field Notes