Toolformer explained: Language model that can use tools via API calls

Toolformer is a new language model that can learn to use tools (on its own), via simple API calls.

Probably the most important paper this year so far.

Here's the low down.

Limitations of language models (LMs)

Large LMs, such as GPT-3 -> are good at performing new tasks with a few text instructions. But, need large amounts of training data and may struggle with more specific or narrow tasks.

On the other hand, small LMs, such as BERT -> are more efficient and can perform basic functionalities like math and fact checks. But their capabilities are limited to simpler tasks.

This is where Toolformer comes in. ‍

Toolformer is a new approach that combines the strengths of both large and small LMs, creating a model that can perform a wide range of tasks.

Intro to ToolFormer

Th  idea behind Toolformer is simple: give language models the ability to use external tools via API calls.

These tools could include search engines, calculators, or calendars, among others.

By incorporating these tools into the language model, the model can perform more specific tasks without requiring additional training data or computing resources.

For example, if a user asks a language model a question like, "When is the next full moon?" the model could use an API call to access a calendar and return the answer.

Similarly, if a user asks a math question like, "What is the square root of 256?" the model could use an API call to access a calculator and return the answer.

Advantages for this model

This approach has several advantages.

First, it has bigger efficiency and accuracy in performing specific tasks, since the language model can rely on external tools that are specifically designed for those tasks.

Second, it reduces the amount of training data and computing resources required to train the model, since the model does not need to be explicitly trained on each specific task. This is also important because humans may find different things useful from what the computer program finds useful.

Plus, manual annotation can be expensive.

The training process

Training a Toolformer language model requires only a few API call examples, because it learns on its own.

Here's how it works:

1. Generating a Dataset of Potential API Requests

Using just a few API call examples, the model generates a dataset of potential API requests for the future.

2. Filters good performing API requests

The model is trained to recognize good results based on certain criteria.

3. Fine-tunes the LLM on these *useful* samples

Finally, the Toolformer language model is fine-tuned on the useful samples that have been filtered out in the previous step. This fine-tuning process helps the model to learn which API requests are most effective for specific tasks

In the end, the model can decide:

-> which APIs to call,
-> when to call them,
-> what arguments to pass, and
-> how to best incorporate the results into future tokenprediction.

Toolformer examples

The goal is to teach the model *when* to get that extra information from the API call.

An example when using a calculator:

🧮 Whenever the model outputs a *Math{query}*, it executes a calculator API request with the {query} and inserts the result.

More examples on the gif below:

Toolformer performance

Toolformer achieves outstanding results in many downstream tasks.

By using a calculator, the model outperformed GPT-3 in math/arithmetics questions

Which only makes sense.

Toolformer limits

While Toolformer offers a simple and efficient approach to language processing, it has some limitations:

1. Inability to Chain API Calls

One of the main limitations of the Toolformer is that it cannot chain API calls between tools. In other words, the model cannot use the output of one tool as an input for another tool. ‍

2. Limited to Non-Interactive Tools

This means that the model can only access tools that do not require user input or interaction.

3. Need for Large Amounts of Examples

Finally, Toolformer models require a large amount of examples to produce good API requests.

Despite these limitations, the potential upside of Toolformer language models is huge.

Conclusion

The Toolformer is a significant development in the world of NLP.

We discovered that:

1. LLMs can Create their Own Training Data‍

By generating potential API requests from just a few examples, this modelcan quickly build a large dataset that can be used to fine-tune the model.

2. Data Limitations Might not be as Limiting

Data limitations might not be as limiting as we previously thought. With the ability to generate its own training data, this kind of models can learn from a small number of examples and still achieve impressive results.

3. The Approach is Scalable and Can Improve Other LLMs in the Future

It's scalable because it doesn’t require modifications to existing language architectures to work.

4. Arbitrary APIs can be Connected Easily

This approach can be used to integrate a wide range of external tools and data sources, making it a valuable tool for a variety of applications.