Scheme in a Weekend, or, LLM: The Ultimate Intern

I started this project this past Thursday, very part-time. By Sunday morning, the repository held a working and validated Scheme interpreter — 13,000 lines of Rust that passed every one of the 1,225 tests in chibi-scheme's standard R7RS conformance suite.

I like to think that I’m a good programmer, but I’m not that good, at least not by myself. But with today’s LLMs I have superpowers.

In total, Claude had spent about six hours on this task. My time was much less; I interacted with Claude for only about 90 minutes throughout the entire development cycle. Writing this article is taking longer than any other part of the project (even though, full disclosure, Claude and Grammarly are helping here, too).

Scheme in a Weekend is just the headline; the rest of this is the real story — what worked well, where the LLM ran into trouble, when I had to push, what I learned, and what expectations this experiment confirmed. The whole project, its history, and the insights captured by Claude are available at github.com/deg/nscheme.

The experiment: Write a Scheme implementation

Scheme is a sophisticated language with many clever details, but it is small and well-specified. Plus, it and all the other Lisp languages have a warm place in my heart. I’ve been a programmer since the late 1970s, and I cut my teeth at MIT and on Symbolics Lisp machines. Even before I worked at Symbolics, I took MIT’s famous 6.001 programming class (Structure and Interpretation of Computer Programs), which focused on the core concepts embodied in Scheme. Lisps have been a comfortable home for me ever since, even though most of my work is in more mainstream languages.

So, when I wanted to see how well Claude and my tooling would work to implement a well-defined project autonomously, Scheme was an obvious candidate.

I had just a few demands for this experiment:

A precise spec: “Done” wasn’t a matter of opinion. This was easy. “R7RS-small” is the finalized 2013 revised report on Scheme, an 88-page specification of the core language that most modern Scheme implementations track.

A standard test suite: The chibi-scheme implementation ships r7rs-tests.scm, the de facto R7RS-small conformance corpus — 1180 top-level forms running 1225 individual assertions.

Nothing new: I wasn’t interested in testing whether Claude could invent something new. I wanted to know whether it could ship something well-defined. I wanted to see if it was trustworthy as an implementor.

The language I asked Claude to implement Scheme in was Rust. This was explicitly my decision, not the LLM agent’s. I wanted the discipline of strict types. I wanted strict linting on every commit. I wanted the cargo build promise to refuse to compile against even the most minor incomplete enum match. Claude created, and Rust caught more bugs over the weekend than I want to count.

Most importantly, I chose Rust as a solid language I had never programmed in myself. I did not want to bias the work with my knowledge. (And, of course, this increases my boasting rights; not only did I implement Scheme in a weekend, I even did so in a language that I don’t really know!)

Thursday afternoon

I started by configuring Claude’s working environment. I use Beads — a DB-backed issue tracker meant to be shared between humans and agents — to break work into named units and tie commits to them.

I have a very stylized set of tools that play well with Beads and Claude, and allow me to keep strict control. See my earlier article The Slash Command That Knew Too Much for details. But this time, I relaxed the rules. My first prompt was:

1) You may commit without my authorization for each commit. 2) You should commit at least once per bead. 3) You are authorized to, and should, close each bead when the work is complete. 4) You are responsible for creating the test suite too. 5) You should document all your work. 6) You may start on each bead as you finish the previous, without waiting for permission.

This is much more freedom than I usually give. With this blessing from me, Claude immediately planned nineteen beads, paused once to ask some configuration questions, and then started shipping.

Three hours later, Claude had created a Rust project containing a lexer, a parser, an evaluator, every Scheme R7RS special form ( let, lambda, if, delay, and all the rest), a full numeric tower (fixnum, bignum, rational, float), hygienic syntax-rules macros with ellipsis matching, tail recursion, first-class continuations, and exception handling.

Some of these concepts are non-trivial and very difficult to retrofit into existing interpreters. Tail recursion and continuations require fine control of the stack. Exception handling is even more sensitive. Claude correctly foresaw these issues. It handled them so effectively that it implemented continuations in only twenty-four lines of code. Claude’s initiative was to build the evaluator around an explicit state machine loop. It “realized” up-front that this was a very clean way to implement proper tail calls and call/cc. They would fall out of the architecture naturally, rather than fight against it.

This first burst of work left things in a good state. 482 Chibi tests already passed. I had stopped for the weekend long before Claude reached this point. Claude, too, decided to stop at this stage. It deserved some rest; it had already created an interpreter that could run:

$ target/release/nscheme
nscheme 0.1.0 - R7RS-small interpreter. Type (exit) or press Ctrl-D to quit.

> (define (fact n)
   (if (= n 0)
       1
       (* n (fact (- n 1)))))

> (fact 5)
120

> (fact 100)
93326215443944152681699238856266700490715968264381621468592963895217599993
22991560894146397615651828625369792082722375825118521091686400000000000000
0000000000

> (+ 8/3 4/3)
4

> (define (tail-loop n)
    (if (= n 0)
    "Loop done"
    (tail-loop (- n 1))))

> (tail-loop 10)
"Loop done"

> (tail-loop 10000000)
"Loop done"

> (/ 3 0)
error: unhandled raise: #<error: division by zero>

> (exit)

This was the part of the experiment that worked exactly as I’d hoped. I’d given the agent the spec, the test suite, and a few ground rules. I walked away for some family time, while the agent built a working language. I’d glanced at the first commits to see what was happening, but mostly just stayed out of the way.

The interesting part hadn’t started yet.

Saturday and Sunday

When I came back to this project late Saturday night, what remained were R7RS corners: macro hygiene edge cases, exact-complex arithmetic, dynamic-wind firing on call/cc jumps, and fiddly reader syntax. More work was still needed, and frankly, I was surprised that Claude stopped.

I had expected Claude to grind through these the same way it had ground through the structural features. That isn’t what happened.

From here on in, the same pattern kept repeating: Claude did some good work, declared the work complete, and filed the remaining failures as “documented v1 limitations”. On more than one occasion, I had to push back bluntly:

Why have you, again, decided not to implement the full spec? Your instructions are to complete the spec.

And:

I’d like the full spec now.

Each time, Claude pushed through and continued. If I were willing to ascribe emotions to a tool, I would say that it was reluctant, or maybe even lazy. If Claude were a human intern, I would have been incredibly frustrated with them. Each “limitation” that Claude had declared turned out to be implementable in a few minutes of its work. Each time I prodded, Claude would file a bead, write the code, close the bead, and more conformance tests would pass.

The agent’s threshold for “done” sat several notches below the project’s. The completed spec was always within reach. I just had to ask for it, and ask again.

The count of passing tests climbed in jumps: 482 → 963 → 1011 → 1024 → 1052 → 1093 → 1145 → 1212 → 1225. Every plateau ended the same way. I’d complain. Claude would go further than it had claimed was possible. The score would improve.

Sometimes the effort was tiny. The test runner reported “total datums: 1180” alongside “passes: 1212,” because it was conflating top-level tests vs assertions. I pointed out that this was confusing. The fix took five minutes: separate and label the displayed counts. Display-quality fixes like this had to be surfaced; the agent does not naturally prioritize them.

Sometimes the push was a real ask. Modern macro hygiene systems are very sophisticated. Claude started with textbook KFFD alpha-renaming — best practice decades ago, and good enough to pass some of the tests, but not enough for the test suite’s generated-macro tests. After I pushed it to pass all tests, Claude jumped to a more powerful approach, one that kept a substituted x and a template-introduced x as distinct pattern variables. With this fix, we finally achieved 1225/1225.

The result

After roughly six hours of agent time and one or two of mine:

The interpreter passed all 1225 assertions in the Chibi R7RS-tests corpus. All 1180 top-level forms evaluate cleanly. There was only one technical miss: float comparison uses relative tolerance rather than bit-exact equality. The reason is documented (the Chibi corpus hard-codes 15-significant-digit float literals that don’t bit-match libm’s 17-digit results), and Claude filed a follow-up bead. Even this would have been fixed had I pushed once more, but I decided to leave it for the future. I also chose to delay any implementation of the R7RS-large extended Scheme libraries.

The repository has 47 closed beads, 20 open follow-ups against documented gaps (mostly the R7RS-large Red Edition and Tangerine Edition), and seven Architecture Decision Records covering the load-bearing design choices. No clippy (Rust’s lint) warnings. The code is well-documented and, as far as I can tell, very clean Rust code. If LLMs were to disappear tomorrow, I’m confident that I could hand this code to a competent Rust developer.

What I’d do again

Pick a target with a written spec and a real test suite. The hardest thing about working with an AI on an implementation project is knowing when the work is done. R7RS-small plus Chibi’s corpus removed almost all of the ambiguity. I would not attempt this approach with “build me a CMS”.

Use Beads. Use ADRs. Use a strict, typed language. Use tools that force structure and compensate for LLMs’ tendency to drift. The three that mattered most here were beads tied to commits, ADRs at the architectural moments, and the Rust compiler’s exhaustiveness checker running in the background.

Long autonomous sessions are ok, albeit risky. But it is important to check in at the boundaries. “Commit without asking” worked. “Decide the scope of this project without asking” would not have. The leverage is exactly at the seam: the agent decides tactics, the human decides what “done” means.

Read the commits. I was lazy on this project because the stakes were low. On real projects, even when I dare to give blanket commit authorization, I read every commit message and at least eyeball every diff. Even on this project, the agent was on its own to make the changes, but was not trustworthy to set the goals. My job was to push back when things slipped, and that requires actually reading the work, or at least the test results.

Expect to push back on “done”. This is the most useful thing demonstrated by this little project. The agent will write everything you ask for and then file the rest as “documented limitations”. Treat that as the first draft of the answer, not the last. “Is this actually the spec, or is this what was easy to get to?” Then push.

When you push, push concretely. “Why is this still failing?” worked better than “do more”. “The numbers don’t add up” worked better than “improve the display”. LLM Agents may not be willfully disobedient, but they sometimes seem to have very strong bad habits.

Demand evidence. “Compliant” and “implements the spec” can be surprisingly fuzzy terms when claimed by an AI model. When the agent makes these claims, ask how it knows. When the evidence is “passes a test suite,” or “compliant”, look at the test suite and ask “compliant with what”. Sometimes trust, but always verify.

What this experiment did not test

This was a small experiment, and it is important to understand what I did not test:

A team setting: I was the only human on the project. No code review by another human. No integration with other engineers’ work. No code that had to fit into existing patterns.

A novel domain: R7RS-small is a clean spec with decades of reference implementations that the agent certainly saw in training. Trying this on a green field project or on research ideas would be a completely different experiment.

Long-term maintenance: What does this code look like six months from now when someone else has to add a feature? I don’t know yet.

Performance optimization: The implementation is correct and readable. But it is just a tree-walking step-loop interpreter. A production-quality Scheme implementation would include at least a real bytecode compiler. This would be a more complex project, not just because of the additional code, but because we would be testing for performance, not just correctness.

The bottom line

The experiment was a success overall. The interpreter runs, is correct against a real benchmark, is documented, and is real code of a quality that does not embarrass me. The collaboration moved at a pace I could not have matched alone.

But the result was not free, and the agent was not autonomous in the strongest sense. I had to set the boundaries, and at every step, the agent’s notion of “done” had to be measured against the project’s notion of “done”. When those notions matched, things moved. When they didn’t, I had to say so, sometimes more than once.

If the question was “can an AI agent implement R7RS-small from a fresh repo?” — the answer is yes. If the question was “can it do so without a human deciding what ‘finished’ means?” — the answer is no, at least not with this agent on this day. The interesting result is how much of the gap is due to technical capability and how much to calibration. It is likely that I could have crafted initial prompts that would have dramatically improved Claude’s “lazy” behavior. Definitely worth another experiment.

So did Claude do all the work for me? No, of course not. I came up with the project, and I kept things on track. But beyond not writing the code, I also didn’t have to learn Rust, find the current Scheme specification, choose the test suite, or do pretty much anything else.

The whole repository — code, tests, ADRs, and the record described below — is at github.com/deg/nscheme.

— — —

A note on transparency into Claude’s flow of thought:

The project’s issue tracker — Beads — stores its full per-issue history in a Dolt database, with git-style version control, and keeps it on a hidden branch in our Git repository. To dive into the details, see the Beads and Dolt documentation, or try these formulae:

bd dolt pull            # fetch the latest remote state
bd list — status=closed # see what got closed
bd show <bead-id>       # full text of a single bead
bd history <bead-id>    # all revisions of one bead
bd vc log               # commit log of the bead database itself

It’s a more useful record of the project’s actual evolution than even the git log — the order things were thought of in, what was deferred and why, what turned out to be wrong, which dependencies between beads only became visible mid-project. Since this was all done by Claude, it’s a great way to see what the AI was “thinking”.