I'm Kinda a Big Deal

September 24, 2025

In this post, you'll find out:

Why your AI generates way more code than necessary, especially for complex code How to reduce the number of security vulnerabilities in your code with one simple acronym Which AI programming methodology to use for different types of problems, from simple to complex

But first, let's cook a chicken.

How do you know when your chicken's done? Maybe you cook it until you think it's done and then just try it. Maybe it's undercooked, maybe it's overcooked. Much better to use a thermometer, though, that way you can be much more confident that the chicken passes the doneness test. That's why chefs use thermometers. But what if you have an inaccurate thermometer that says your chicken's done when it's still pink?

The fix: Test your thermometer. Stick it into an ice bath and then into boiling water to see if it reaches 0°C and 100°C. Calibrate with other thermometers.

This works in programming too. If you're writing code and you want to know "Does this code work?" then you can write tests like: Does my program convert 32°F to 0°C? Convert 100°C to 212°F? If the tests pass, then the code works. Kinda. The uncertainty arises because something could be wrong with your tests. Maybe there's an actual bug in your tests, or maybe they're testing for the wrong thing. Like the thermometer example, you don't know if your tests work until you see if they can spot broken, buggy code, for example, a temperature conversion program that only says "It's freezing!" or "It's boiling!"

If you want to make sure your tests work, the trick is to write intentionally buggy code to make sure that your tests work as intended. It's the programming equivalent of sticking a thermometer into an ice bath.

Even that's not enough. What if, in the process of changing your code to get the tests to pass, or modifying your tests to better ensure that your code meets your exacting standards, you introduce a new source of error?

Or suppose you don't trust your thermometer, and so you stick it into boiling water and get a readout of 189°F, which means it's off by 23° -- and then you recall that you're at the top of a mountain where the high altitude allows water to boil at a lower temperature. And so now, if you want to calibrate your thermometer, you have to check your altimeter. But how do you test your altimeter? How do you know it's giving you the correct altitude to verify that the lower boiling point is due to lower atmospheric pressure? And how do you know you're actually at the top of a mountain and not inside a freezing tent in a simulated mountaintop environment? The only way to know is to test your tests.

Test-Test Driven Development (T2D2) Over the past week, I've been diving into experiments on what I'm calling Test-Test Driven Development, pretty much to the exclusion of all else. I even stopped posting daily Convention City Seattle trails (I've already got plenty for a demo), because this is much more important for the moment.

The experimental design is fairly simple. For each run, you ask an AI coding assistant to write code to meet specifications using four different methodologies as independent tasks:

Immediate implementation: Just go for it. Specifications first: Write the docs, and then the code. Test-driven development: Write the tests first and then code that passes the tests. Validated test-driven development: Write tests; and then, validate tests with broken code (e.g. ice cubes); and only then write correct code that passes the tests.

I'm using Claude Code from Anthropic because it has an advanced "Task" tool for spawning multiple independent versions of an agent, where each agent reliably stays inside of its container without trying to index or explore the content of the entire repo for additional context.

You set up the run by launching Claude Code, telling it to read the README, and then saying "spawn-experiments" with an experiment number or really whatever you want to build.

The "Meta Prompt Generator" picks it up from there, creating four prompts based on your initial prompt. And the race is on.

Play spawn-experiments in action At the end, the Meta Prompt Generator protocol collects results from each independent trial and assembles an experiment report comparing the results on various dimensions.

I'm sharing it as open science so that you can try it at home. All you need is Claude Code Pro to run a single experiment (which will chew up your 5-hour allotment all by itself) or Claude Code Max if you want to go full mad-scientist mode.

https://github.com/ivantohelpyou/spawn-experiments

It's completely open, so you can easily replicate my results or try your own experiments.

I'm quite convinced that I cannot be the first person to figure out this technique, but as far as I know, I'm the first person stupid enough to share it.

But I've personally seen how adding the magic acronym "TDD" to my prompting has increased code quality and robustness. And that's way too important to keep it a secret.

Plus, I'm sharing it because it's entirely in line with what I had in mind when I started the Model Citizen Developer newsletter. We need more cooks in more kitchens. Pro coders work in pro kitchens using pro tools, but they're not cooking fast enough. That's why we've been getting reheated slop for software. With AI and systematic methodology, citizen developers can build 80 percent of what professional developers once did - but better, because we're cooking to order.

And if you've wondered how a solo developer was able to build Convention City Seattle so quickly, now you know.

Let me know how it goes -- and enjoy your chickens!