An agent with subagents, from a GPT we built

Intro

0:00 Hello and welcome back to episode twenty-one, where our model becomes an agent. It works with files, and hands parts of a task to subagents, which are copies of our model, each in a context of its own. Let's begin with a quick look at where we're going to be by the end of the episode.

0:18 By the end of today, we give our model a task, add up the numbers in three files, and it doesn't do it all on its own. It hands each file to a subagent, which reads it, adds it up with the calculator, and hands back its total. A last subagent adds the three totals, and the agent answers one thousand eight hundred and forty-nine, which is right. Each subagent is our model again, in another role, with a fresh context, and the agent's own context stays at about a hundred and thirty tokens.

0:45 On the website of the series, you can find more information about the project. The code is open source on GitHub, tagged at each step of each episode, so that you can follow along, and the docs get a page on the agent today. And each episode has a page here as well, with its chapters and its code.

1:04 Now, before we get to today's work, let's quickly go over what we did last time. We taught the model we pretrained on the web to call a calculator. The call is plain text, which the program running the model watches for, runs, and answers in the text. On questions worded the way training words them, almost all of its answers were right, and on new wordings, about half. Today, the model gets files to read and write, and a tool that starts another copy of the model.

1:33 Today, we fine-tune one model into three roles: an agent, a math role and a files role, each with tools of its own. The files live in a sandbox, a folder whose limits the program enforces, whatever the model writes. The agent hands work to the other two roles as subagents, each our model in a fresh context, which hands back its answer and keeps the rest. We generate sixty thousand traces of the three roles at work, and fine-tune for thirty minutes on the laptop. We measure each role, and then one long task two ways, with subagents and in one context. And we put the agent in a chat that shows each step as it happens. All right, let's get started with what an agent is.

Agents and subagents

2:14 ReAct: Synergizing Reasoning and Acting in Language Models, from 2022, is by Shunyu Yao and others at Princeton and Google. Its abstract proposes interleaving the two: the model writes a thought, takes an action, like a search, reads what comes back, and decides what to do next, so actions gather what the reasoning needs. Figure one shows it on a question from HotpotQA, next to reasoning alone and acting alone. ReAct searches, reads the result, and searches again with what it learned, until it can answer. That loop, act, read the result, decide, is what's meant by an agent today. Ours leaves out the written thoughts: it acts, and reads the results.

2:55 Subagents are how assistants take on bigger tasks today. Anthropic's post on its multi-agent research system, from 2025, describes a lead agent that plans, and creates subagents that search in parallel. It says subagents help by compressing: each works in a context window of its own, and hands the lead agent the most important tokens of what it found. And it's frank about what that costs: agents used about four times the tokens of a chat, and multi-agent systems about fifteen times, though on their internal research evaluation, the multi-agent system did about ninety percent better than a single agent. We're going to see both sides of that at our scale: smaller contexts, and more tokens in all.

3:37 The prompt starts with a role and its tools. The agent can write and list files, and ask the other two roles for help, but it can't read a file or calculate. The math role has the calculator, as last episode. The files role can read, write, list and calculate, so it can add up a file. All three are one model, with one set of weights, fine-tuned once on all three roles' work. What makes a subagent isn't separate weights, it's a fresh context, a role, and the tools that come with it.

4:09 When the agent writes an ask call, the program stops it, as it does for the calculator. Then it starts the model again, from an empty context, with a prompt in the files role and the task from the call, in a KV cache of its own. That run reads the file and adds it up, with its own calls. When it finishes, its answer goes back into the agent's text, as the call's result, and the file's numbers stay out of the agent's context. And the agent carries on from there, as if a tool had answered.

One model, three roles

4:39 A new file, agent dot py, holds the runtime: the program that runs the model, and the tools it calls. The comment at the top says what we're building: one model, three roles. A role is a line at the top of the prompt, with the tools it may call, and the agent delegates to the other two, each of which works in a context of its own and hands back its answer. Roles maps each role to its tools: the agent can ask math, ask files, write and list, math has the calculator, and files can read, write, list and calculate. The agent can't read a file on its own, so to know what's in one, it has to ask. Then the limits, which belong to the program, not the model. Max depth is one: a subagent can't start subagents of its own. Max calls is ten per context, so a model that loops gets stopped. Longest sum lets the calculator take a thousand characters, for adding up a whole file. Name is the shape a file name has to have: a short word of letters, digits and dashes, and dot t x t. No slash fits it, so no name leads out of the folder. A session has twenty files at most, of four hundred characters each. Number finds a number standing on its own, and not the one in a name like week one dot t x t. Prompt writes a task in our format: the role, its tools, the task, and answer. Sandbox is the files the model can see: one folder, with limits the model can't change. Names lists the files whose names fit the pattern. Read checks the name, then returns the file, or no such file. Write checks the name, refuses content that's too long or has angle brackets, which would look like a call, and refuses a new file past twenty. Each refusal is a string starting with error, which goes back to the model like any other result. List joins the names together, or says there are none. A run is one context: its role, its task, its depth, the text it wrote, its calls, each with its result and the subagent's run when there was one, and its size in tokens. Final is what it wrote after its last call's result, like the total is fifty. Answer is what a subagent hands back: the first number of its final sentence, or the sentence if it has no number. The agent class holds the model, the tokenizer and the sandbox, and an on event function, which the app uses to show each step as it happens. Call runs one call from a run, and its first word is the tool, and for ask, the tool is ask and the role. If the run's role doesn't have that tool, the answer is no such tool, and past ten calls, too many calls. Calc goes to episode twenty's calculator, list, read and write to the sandbox, and write splits the name from the content at the colon. Ask is the one that starts a subagent, and a subagent at depth one can't ask, and an ask without a task is refused. Otherwise, the role and the task go to run, one level deeper, and what comes back is the subagent's answer alone, along with its run, for the trace. Run is the loop, generalised from the one in episode twenty. It starts the run, encodes the prompt, and fills a fresh KV cache with it. Each subagent gets a cache of its own: that's what a fresh context is, here. Then a token at a time, greedily: when the text ends with two opening brackets, a call has started, and when it ends with two closing ones, it's over. The call goes to call, the result is appended with an equals sign, and the result's tokens go through the model with the last one, so the model reads them before it writes on. End of text or the token budget stops it. Show prints a run as a tree: the task, each call and its result, the subagent's run indented under the call that started it, and the final sentence, with the context's size. Main gives a task to the agent, or another role with dash dash role, in a folder, with files to start with as JSON, and prints the tree. In tools dot py, the calculator takes how long an expression may be, a hundred characters by default, as before. The tests use a scripted stand-in for the model again, now with a script per role, so a run and its subagent's each write their own. They check the sandbox's limits: a name with a slash, a capital, another extension or too many characters is refused, as is a file too long, or with a call in it, or one past twenty. A files run reads a file and adds it up. And a subagent reads a file that the agent doesn't see. The math role can't write or read, and files can't ask. And a model that keeps calling is stopped after ten calls.

9:13 All five tests pass, with no trained model involved.

Sixty thousand traces

9:19 Agent data dot py generates the traces the model learns from, sixty thousand of them, and a hundred per held-out set. Names are the words file names are made of, and the held-out sets use other words, like orchard and harbour, so the model has to copy a name it hasn't seen, and items are what notes are made of. Then the wordings for each kind of task, the ones training sees and one more each, held out. The files role saves, reads, lists, adds up a file, adds one up and saves the total, copies, and adds up several files in one context. The agent works out sums and saves them, two at once, word problems, totals of several files, and saves and lists files on its own, and share is how often each kind comes up. Trace builds an example: the prompt, then each call and its result, then the final sentence, with the prompt and the results given and the rest learned. Returned is what a subagent hands back, worked out the way agent dot py does. The maker fills in the tasks: names, notes and numbers, the math role's questions from episode twenty, the files role's tasks, one in ten of them on a file that doesn't exist, so the model learns to say so, and the agent's tasks, where each file goes to a files subagent and the totals to a math one. Build makes it all with one seed, the held-out sets, and one long task: four files of ten to fifteen numbers, for the agent and for the files role, to compare.

10:41 Agent data dot py writes sixty thousand traces, about twenty-four thousand for the files role, eighteen thousand for the agent and fifteen thousand for math, with web pages for the rest, and a hundred in each held-out set. It prints a trace of three kinds, with the given parts in square brackets. The files role reads a file, adds it up and saves the total. The agent asks a files subagent for each file's total, and a math subagent for the sum. And the agent works out two sums with math subagents, and saves both.

Fine-tuning

11:13 Agent SFT dot py follows tool SFT dot py from last episode, and takes its encode, make batch and estimate loss, with its mask. The batch is thirty-two at most, and about six thousand tokens: the traces range from forty tokens to three hundred and fifty, so a batch of long ones holds fewer of them. The learning rate is tool SFT's as well, and demo is a task the agent does at each evaluation: two files to add up. Batches shuffles the examples, sorts each stretch of fifty batches by length, and cuts it into batches of similar lengths, so little of each batch is padding. Each batch stops at thirty-two examples, or when one more would pass the token budget. Then the batches are shuffled, so the lengths come in a random order. Main loads the model we pretrained on the web, as last episode did, and that one model learns all three roles. Each evaluation measures the held-out loss on the agent's tasks, runs the demo in a scratch folder, and prints the run as a tree. In the loop, a batch comes off the queue, and the step is tool SFT's. Each fifty steps, it empties the GPU's cache: batches come in many lengths, and PyTorch keeps memory for each shape it has seen. The first time I ran it, that cache reached thirty-one gigabytes on the Mac, and the run stopped, out of memory. Each fifty steps also prints the batch's loss and the minutes so far, and logs the loss to TensorBoard. Each two hundred and fifty, it evaluates, and puts the model back in training mode after, as evaluating turns it to evaluation mode. Seen counts the examples, so the last line can say how many passes over the training set the run made. At the end, a last evaluation, the model is saved to checkpoints, agent, and it prints the steps, the examples, the passes and the minutes. Thirty minutes is the default, three times the ten we gave last episode's run, as the traces are longer, and there are three roles to learn.

13:08 Then thirty minutes on the laptop, with about two million eight hundred thousand tokens to learn. At step zero, the pretrained model treats the task as the start of a page, and writes a numbered list about the lines of a file, which goes nowhere. By the evaluations near the end, it runs the demo as we designed it: a files subagent for each file, each reading its file and adding it up, and a math subagent for the sum. The batch loss swings between near zero, for batches of traces, and about one, for batches with web pages in them. At the end, it has taken nineteen hundred steps and seen fifty-eight thousand examples, about one pass over the training set.

Measuring the agent

13:48 Agent eval dot py measures the agent and its roles. All runs collects a run and its subagents' runs, each its own context. Attempt runs one task in a folder of its own, with the files it starts with, and checks what's left: the folder has to hold the files it should, with the right contents, and no other files, and the final sentence has to have the answer. A task counts as right when both of those hold. It also counts the calls, the subagents, the tokens in the run's own context, the largest context of any run, and all of them together. Summary averages them over a set, and main runs each held-out set, and prints a line per set, with how often each kind of task is right. Then the long task, two ways: as the agent, which hands each file to a subagent, and as the files role, which reads all four files into one context. For each, how often it's right, its own context, the largest, and all the contexts together. Its own context is what the run it started holds: the agent's, with subagents, or the one files context. The largest is the biggest of any run, subagents' runs as well, which is what has to fit in the model's window. And all the contexts together is the tokens the model read and wrote across them, which is what the run costs. Each set keeps three of the traces it got wrong, and after the table, it prints a wrong one for each way of doing the long task, so we can see where it goes wrong. The results go to results, as agent, then the model's name, dot JSON, for the charts. Each task runs in a temporary folder, deleted when it's done, so no task sees another's files, the way a session in the app doesn't see another session's. Checking the folder, and not just the answer, matters for an agent: a task like saving a total is right when the file holds the total, whatever the model says it did, and a stray file left behind counts against it too.

15:41 On held-out tasks worded the way training words them, each role gets them all right: math, files and the agent, one hundred percent, with about one point four subagents per task, and about one hundred tokens in the agent's own context. On new wordings, with file names it hasn't seen, the files role gets about forty-nine percent right, and the agent about twenty-three. The names aren't the problem, as it copies them well, but the plan is. Asked to store a note, it reads the file instead. Asked to calculate two sums and store them, it does one. That's the weakness we saw last episode, a model that learned our wordings, now with more steps to get wrong.

16:23 Side by side: on the training's wordings, each role does each task right, from a sum to delegating four files. On wordings it hasn't seen, the files role does about half, and the agent about a quarter, as its tasks take more steps, and one wrong step is enough to get a task wrong.

What subagents buy

16:41 Now the long task, on one example: four files of eleven to fifteen numbers. The agent hands each file to a files subagent, each reading one file into a context of its own, of up to about a hundred and forty tokens, and a math subagent adds the four totals. The agent's context ends at a hundred and fifty-nine tokens. With dash dash role files, one context does all of it: it reads the four files, then copies fifty-three numbers into one call to the calculator. It gets the right total here too, twenty-seven thousand three hundred and fifty-two, with a context of three hundred and ninety-three tokens.

17:17 Over the hundred long tasks of the evaluation, the agent with subagents gets about ninety-five percent right, and one context about eighty. When one context goes wrong, it's in the long copy: one of its wrong ones skips most of two files on the way into the calculator. The agent's own context averages about one hundred and fifty-nine tokens, against three hundred and seventy-seven, and the largest of any of its subagents was one hundred and sixty-seven, against six hundred and eighty-seven. The price is the one Anthropic's post describes: all the contexts together come to about seven hundred and nineteen tokens, against three hundred and seventy-seven. Subagents spend more tokens, in pieces small enough to get right.

The agent in a chat

17:59 Last, agent app dot py puts the agent in a chat, with Gradio, like episode five's app, and its theme. Start is the files each session begins with, and examples are tasks to click. A lock lets one task at a time use the model. New session makes a folder for a session, and files table lists its files with their contents. Respond runs a task and shows its steps as they happen: the agent runs in a thread, and its on event function puts each step in a queue, which the chat reads. A subagent starting adds a step of its own, pending, titled with its role and task. A call adds a step under its run's, with its result, and an ask shows as the subagent's step instead. A subagent ending fills in its step with its final sentence, what it handed back and its context's size, and how long it took. At the end comes the agent's answer, with its calls and its context's size. The page has the chat, a box for the task with the examples under it, and the files next to them. The state holds the session's folder, and its delete callback removes the folder when the session ends. The value of the state is the new session function, which Gradio calls for each visitor, so each gets a folder of their own, with the starting files in it. Ask hands a task to respond, along with the model, and yields each update to the chat and the files table. Start again deletes the session's folder and makes a new one, for the button under the files. Submitting the task box runs ask, then clears the box, and loading the page fills in the files table. The model is loaded once, when the app starts, on the Mac's GPU, and all the sessions share it, which is why the lock is there: two tasks at once would interleave their tokens on one model, and each run's cache is its own anyway. Launch serves it on port seventy-eight sixty-one, next to episode five's app on seventy-eight sixty, with the series' dark theme. None of this is new to Gradio: the chat's steps are its own feature for showing an agent's work, with a title, a parent for nesting, and a status that goes from pending to done, which is what lets each subagent's calls sit inside its own step.

20:05 We run it with python agent app dot py, and open it on port seventy-eight sixty-one. Clicking an example fills in the task: work out two products and save both. The agent hands each to a math subagent, each shows as a step, and then the write, and results dot t x t appears in the files. Then the three files to add up: three files subagents and a math one, each a step, as they happen. Opening one shows its own calls, the file it read and the sum it asked for, what it handed back, and the size of its context.

20:39 Last, the docs get a page on the agent, with the roles and their tools, the sandbox's limits, how a subagent runs, the traces, and the results, with the long task both ways. It goes at the end of the navigation, after tool use.

Wrap-up

20:54 Now, let's go over what's real here, and what's scaled down. The loop is real: a model calls tools, reads the results and carries on, and starts subagents in fresh contexts that hand back their answers. So is putting the limits in the program, which decides what a call may do, so a model can ask for anything, and gets what's allowed and no more. The tasks are scaled down: a few kinds, generated from templates, where an assistant takes requests of any kind. And so is the planning, by a long way. Our agent follows the plans it saw in training. It doesn't work out a new plan for a new request, which is why new wordings trip it up, and that's what models a thousand times bigger, and trained on far more varied work, do far better.

21:38 So, today we fine-tuned our model into an agent: one model, in three roles, with the tools of each. Its files live in a sandbox whose limits the program enforces. It hands work to subagents, our model again in a fresh context, which hand back their answers alone. After thirty minutes on the laptop, it does each held-out task worded the way training words them, and about a quarter of the agent's tasks worded in new ways. On the long task, subagents got about ninety-five percent right against eighty in one context, with the agent's context less than half the size, for about twice the tokens in all. And the chat shows each step, each subagent with its own calls inside it.

22:20 If you'd give our agent one more tool, let me know in the comments which, and what you'd ask it to do with it. And if you'd like to follow along, subscribe to the channel. I hope you find this useful, and thanks for watching.