# How AI Works

How a chatbot turns your words into tokens and numbers, weighs attention across layers, and picks each next word from learned odds\.

By merry\-dolphin\-13d59633

Topic: Computing, AI & Electronics

[Open the interactive Jig](https://ask.jig.com/j/58zQftrBkuKa)

## Transcript

### 1. The guessing machine

A chatbot is a kind of AI, short for artificial intelligence\. Inside, it works like a huge guessing machine: it reads your words and predicts what comes next\. Let’s follow one sentence through it\.

Chatbots like this are built on large language models, which learned from enormous amounts of text\.

### 2. Cut into tokens

First, the text is cut into small pieces called tokens\. A token can be a whole word, part of a word, or a mark like a full stop\. Each token has an ID number from the machine’s fixed list\.

GPT\-2’s list holds 50,257 tokens\. Here, “cat” is token number 3797\.

### 3. Words become numbers

Next, each token is swapped for a long list of numbers, called an embedding\. Words that are used in similar ways get similar lists\. That is how the machine can tell that “cat” is closer to “dog” than to “banana”\.

GPT\-2 small uses 768 numbers for every token, and GPT\-3 uses 12,288\. Each column here shows just 8\.

### 4. Paying attention

Now each token looks back at the words before it and weighs which ones matter most\. This is called attention\. Thicker, brighter arcs mean more attention\.

In this made\-up example, the last “the” looks hardest at “cat” and “sat”, which helps it guess a place a cat might sit\.

### 5. Layer upon layer

Attention plus a thinking step, which reworks each token’s numbers, makes one layer\. The numbers flow up through layer after layer, picking up more meaning at each one\.

GPT\-3 stacks 96 of these layers\. The tower here shows 6\.

### 6. Odds for every word

At the top, the machine gives every token in its list a chance of coming next\. Drag the temperature slider and watch the odds change\. Low temperature makes the favourite almost certain, while high temperature spreads the chances out\.

These odds are an example for this sentence, not numbers from a real model\.

### 7. Pick, add, repeat

The machine picks one word, using the odds like a weighted lucky draw, and adds it to the sentence\. Then it runs the longer sentence through again to guess the next piece\. Press the button a few times: you may not get the same word every time\.

A long answer is built this way, one token at a time\.

### 8. How it learned

Before anyone chats with it, the machine practises on huge amounts of text by guessing each next token\. After every guess, its settings are nudged a tiny bit toward the right answer\. Watch the dials turn as “mat” becomes the favourite\.

GPT\-3 has about 175 billion of these settings, called parameters, and it practised on roughly 300 billion tokens\.

### 9. What it is, and isn’t

There is no library of stored answers inside: each reply is worked out fresh from learned numbers\. That makes chatbots fluent, but they can still say things that are wrong\. They also don’t learn from your chat, because the dials stay fixed while you talk\.

Many chatbots get extra training from people who rate answers, which helps them be more helpful\.