Shann³
@shannholmberg
how autoresearch works, simplified
it's a pattern that lets AI agents run experiments and improve anything you can measure
three files is all you need, everyone should be running it. ↓
> program. md is where you tell the agent what to do. your goal, the rules it has to follow, and any constraints. think of it as the job description
> train. py is the only file the agent can touch. this could be code, a config, a prompt, a math equation, whatever you want optimized
> prepare. py is the scorecard. it measures results and the agent can never edit it. if it could, it would just fake better scores
the loop it uses:
1. agent reads your goal
2. tries an experiment
3. measures the result
4. keeps it if the score improves, reverts if it doesn't
5. repeats for as long as it's improving.
it can run 100+ experiments.
a common conception is that it's for ML, but it can be applied widely.
if you can score it, you can autoresearch it
> Shopify ran it on their Liquid engine. 53% faster parsing from 93 automated commits
> someone pointed it at a portfolio website and load time dropped from 50ms to 25ms in 4 minutes
> Driveline Baseball used it for pitch velocity prediction. R-squared went from 0.44 to 0.78
marketing, trading strategies, prompt engineering, code performance.
we have three conditions for it to work:
> one number to optimize
> automated evaluation with no human in the loop
> one file the agent can change
anything where "better" is subjective doesn´t really work. brand design, UX, pricing without user traffic data, so skip that.
the edge here is picking the right metric
give it a bad one and it will confidently optimize the wrong thing
511 likes · 64 reposts·2d