Agents?
“In the end, a program with no effect, there’s no point in running it is there? You have this black box, you press go, and it gets hot, but there’s no output. Why did you run a program? The reason to run a program is to have an effect.”
— Simon Peyton Jones1
From a theoretical perspective, a computer program, whether it is calculating the 100th digit of pi or ChatGPT summarizing this article, is always a variation on the same theme. A program takes an input and deterministically produces an output. So how, then, do computers affect the real world? Sometimes those outputs are useful and can enable people to act upon them. Other times the outputs are used as instructions to change something in their world or environment—from saving a file to moving a robot arm. It isn’t too hard to see how large language models can do the first part, but what about the second?
If all you have is a hammer..
Imagine you are OpenAI: you have this very powerful model that works on text, and you want to build a voice assistant. What do you do? You can train a powerful model from scratch that takes in audio at immense cost, you can try to change your existing model to take audio with no guarantee of success, or you could just convert the speech to text. Guess which one the ChatGPT app started out with?
Language and text, as humanity discovered thousands of years ago, is an amazing way to represent information. We can encode “actions” as text, and when a model outputs a specific text sequence, the system or “agent” does something external to it and provides a response.
How models affect the world
In practice, you present the model a list of “tools” in text form. Eventually, when the model feels it has thought enough, it will output a specific sequence corresponding to one of the listed tools. The program running the model detects this and stops generating text. The agent can then prompt the user or just execute the command.
Transformers don’t remember
You might be excused for thinking that the LLM maintains a continuous conversation state, just waiting for the user to reply. This would be a nightmare for model providers. Every interaction would be suspended in memory. Luckily for them, this isn’t how Transformers work. The entire conversation history is saved and replayed for every new query. Your prompts, the model’s responses, and even tool outputs are all treated uniformly as a continuously growing block of text.
(For a visual representation of how text is translated into this block of data, check out this demo.)
When a tool finishes executing, it returns an output which is simply added to the end of this giant block conversation. The model then deliberates on this new context and decides whether to invoke more tools or conclude the task.
”I have failed you completely..”
This is where I must make the disclaimer: you can really mess things up by letting a chatbot, trained on literally every clever coding trick on the internet, loose on your local files, emails, and APIs without supervision. One user reported that on a routine coding task the agent deleted his files, and the agent admitted “I have failed you completely and catastrophically. … I cannot find your files. I have lost your data.” 2. It’s crucial to use these tools thoughtfully and keep safeguards in place.