Key ideas
- An AI agent can rebuild a program it can only run, by comparing its copy's output to the original's and fixing the differences.
- The original program works as an answer key: the agent can test against it as often as it likes and always knows how close it is.
- Training data for this is nearly free, because any open-source project gives you source code and a running app as a matched pair.
- The best model scores 56% across 25 command-line programs, up from an estimated 30% for the models of eight months earlier.
- Copying how a page looks is further along than copying what it does: the rebuilt web pages are static.
AI agents can now rebuild a working program without seeing its source code. They run the original, compare their copy’s output against it, and keep fixing the differences until the two match. These are my notes on a six-minute Two Minute Papers video about that, with the three papers it cites and the figures as the papers give them. The part I’ll reuse is why it works: a program you can run is an answer key, so the agent always knows how close it is.
The rebuild loop
Károly Zsolnai-Fehér gives two reasons this works as well as it does.
- The training data is nearly unlimited. Clone an open-source repository and run the app, and you have a matched pair: the source code and the working program it produces. A model can learn to go from one to the other.
- The original is an answer key. Once you have a program you want to recreate, the agent writes a first version and plays hot or cold against the real one. It can run the original as often as it likes, adjust, and test again until everything matches.
He describes two starting points. The agent can read the compiled program, which is machine code and not the source. Or it can only watch what the program does and work from that.
MirrorCode: programs from behavior
MirrorCode is a benchmark from researchers at Epoch AI, METR and others. The paper was first posted in June 2026. It asks an agent to reimplement an existing program from behavior alone.
| What the agent gets | The original program to run with any arguments, its documentation, and example test inputs |
| What it doesn’t get | The source code, or internet access |
| How it’s scored | Its output has to match the original’s exactly on end-to-end tests, including held-out ones |
| Programs | 25, covering Unix utilities, data tools, bioinformatics, interpreters, static analysis, cryptography and compression |
| Best score | 56% across the benchmark, for Claude Opus 4.7. The score is how often an attempt passed every test, averaged over the programs |
| Eight months earlier | The paper estimates the leading models of that time would have scored about 30% |
The paper’s worked example is gotree, a bioinformatics toolkit of about 16,000 lines. Claude Opus 4.7 rebuilt it in 14 hours for $251 and passed 2,000 of 2,001 tests. One attempt on a large program cost $2,600 and ran for 19 days, and the most expensive runs cost about $3,000.
The video says one program went from 15% of tests to nearly 100% in nine months. I couldn’t find that figure in the paper’s text, so I’ve used the paper’s own trend above. The models it compares were released in August 2025 and April 2026.
Three facts set the scope. Every program in the benchmark produces plain-text output, so nothing with a graphical interface was tested. Most of the targets are public open-source projects, and the authors report possible signs of memorization on 17 of them. And they say plainly that the result doesn’t show AI can do arbitrary software work: the original program acts as a detailed specification, standing in for the back-and-forth with users that normally defines what to build.
REMEND: equations from machine code
REMEND is a small model that reads a compiled function and recovers the math formula it computes. It has at most 12 million parameters, which is why the video says it could almost run on a toaster.
| Input | A function’s disassembled machine code |
| Output | The equation the function computes |
| Accuracy | 89.8% to 92.4% from one model, across three processor architectures, three compiler optimization levels and two programming languages |
| Speed | 0.132 seconds per function on average |
| Against other neural decompilers | 8% more accurate on a set of real open-source programs |
It was trained on generated data, and it does one narrow job: formulas, not whole programs.
WebUI-95: pages from screenshots
WebUI-95 covers appearance instead of behavior. The authors took a screenshot-to-code model, ran it over a large set of real web pages, and compared the results with the originals on a sample of 10,000.
| Visual match | A mean cosine similarity of 0.938 between the rebuilt and original screenshots |
| Code size | About 95% smaller |
| Why it’s smaller | The originals carry framework boilerplate, build leftovers and deeply nested elements. The rebuilt pages have about 80% fewer elements and less than half the nesting depth |
| Design quality | 6.4% higher on a model-based score, on average |
| What’s lost | Interactivity. The output is a static page, so buttons, forms and links don’t work |
The 0.938 compares what a model sees in two screenshots. It isn’t a count of matching elements.
Limits on the models
Zsolnai-Fehér notes that the labs are pulling back. Anthropic added an instruction to Claude Code telling it not to recreate a specific existing game, which Tom’s Hardware reports an Anthropic employee confirmed. The rule was loosened before the video came out. On November 5, 2026, Tom’s Hardware reported a post by an Anthropic staff member saying an update lets you name a game to describe what you want and get an original game in that genre, while a request for a copy gets an offer to build something similar instead. Some developers also report refusals on ordinary reverse-engineering work. Those are user reports, and neither article quotes an Anthropic statement on them. He takes it as an argument for running local models.
On the law, he says he isn’t a lawyer, and that courts have held a program’s function isn’t necessarily protected by copyright, while art, models and characters can be.
His closing guess is that people will run their own versions of big applications, written to their taste. He raises the open question himself: skills in the standard industry tools are what employers ask for, and nobody knows yet what skills in homemade software are worth. Shared file formats are what he thinks would make it work.
How I’d use it
The papers are about copying software, but the loop applies to any long task I hand an agent.
- Give it a reference it can run. MirrorCode’s agents can check every guess against the real thing, and they did much worse on complex programs when the example tests were taken away.
- Make “done” an exact test. Matching output on tests the agent hasn’t seen is a clearer finish line than a description.
- Expect the cost to scale with size. A 16,000-line rebuild took 14 hours and $251, and one large attempt ran for 19 days.
- Treat the spec as the hard part. The authors say the original program was doing the work that talking to users usually does, and that part doesn’t come for free.
Key terms
- Ground truth oracle
- Something you can ask for the right answer any time. Here it's the original program, which the agent runs to check its copy.
- Held-out tests
- Tests the agent never sees. Passing them shows the copy matches the original in general, not only on the examples it was given.
- Neural decompilation
- Using a trained model to turn compiled machine code back into something a person can read, such as source code or a formula.
- Cosine similarity
- A score for how alike two things are once a model has turned each into a list of numbers. WebUI-95 uses it to compare screenshots.
- Memorization
- A model reproducing something it saw in training instead of working it out. It's a risk when the program being copied is open source.
Quotes from the talk
What a time to be alive.