Tech Original explainer
ChatGPT predicts the next word, one token at a time — here is the large language model behind the chatbot, in plain English.
Ask ChatGPT to draft a message to your landlord or explain photosynthesis in simple words, and it answers in seconds, as if it understood what you meant. It is not a person. Behind the chat window is a large language model, and understanding how does ChatGPT work starts with what that model is really doing.
ChatGPT is a chatbot built by OpenAI on top of a GPT model. GPT stands for Generative Pre-trained Transformer: it generates text, it was pre-trained on a huge amount of text, and it is built on a design called the transformer. The model itself does one thing, over and over: it predicts what text should come next.
A large language model, or LLM, is a very large neural network trained on huge amounts of written text — books, articles, websites, and code. It is not taught facts the way a student is taught. It is given text with the next piece hidden and asked to guess it, billions of times.
Think of it as advanced autocomplete. Your phone keyboard predicts the next word in a short message; an LLM does the same job with far more context and training. When you ask it to explain a topic, it is generating a fresh reply, one piece at a time, from patterns learned during training.
Those pieces are called tokens. ChatGPT does not read whole words the way you do: a tokenizer first splits your message into tokens — sometimes a whole short word, sometimes part of a longer word, sometimes a punctuation mark. As a rough rule of thumb from OpenAI, one token is about four characters, or about three-quarters of a word, in English. The model scores every possible next token, picks one, adds it, and repeats. That is why replies appear word by word.
The design that makes this possible is the transformer, introduced in a 2017 research paper called “Attention Is All You Need”, written by eight researchers at Google. It proposed a new kind of neural network built entirely around a mechanism called attention.
Attention is a useful name for what it does. In “Rahul left his umbrella at home because it was raining”, the word “it” only makes sense if you connect it back to the rain. A transformer lets every token look at the tokens around it and weigh which ones matter most, across many layers. GPT models are built on this design, and the same idea now sits behind most modern chatbots.
Training happens in two broad stages. The first is pre-training: the model reads an enormous collection of text and plays the same guessing game again and again, learning grammar, facts, and writing styles. It also picks up errors in that text.
A model straight out of pre-training is not yet a helpful chatbot: given a question, it might simply continue it with another question. So a second stage, fine-tuning, teaches it to follow instructions, using example answers.
One widely used fine-tuning method is RLHF — reinforcement learning from human feedback. People are shown two or more answers to the same prompt and asked which is better. Those choices train a separate reward model, which learns to predict which answers people would prefer, and the chatbot is adjusted to earn higher scores from it. OpenAI described this in its InstructGPT research, published in 2022, a key step on the way to ChatGPT.
Why, then, does it sometimes give a confident answer that is wrong? Because it is trained to produce text that looks likely, not text checked against a reliable source. A plausible sentence and a true sentence can look very similar to a system that only scores the next token.
This is often called hallucination: the model invents a detail, a date, or a reference that does not exist, and presents it in the same smooth tone as a correct answer. It does not know it is inventing anything, because it does not “know” facts the way a person does. Unless it is given a tool to look something up, it is working from patterns stored during training, which can be incomplete or out of date. Fluency is not evidence, so check anything that matters — a figure, a name, a deadline, a health or money decision — against an original source.
A search engine and ChatGPT look similar — you type a question, you get an answer — but they do opposite jobs. A search engine finds pages that already exist and points you to them, so you can judge the source for yourself. It is at its best when you need something current, specific, or verifiable.
ChatGPT, by contrast, writes a new answer for you. It can combine ideas, rephrase a difficult passage, or shape rough notes into a clear message. What it cannot do on its own is fetch a live fact or prove that what it wrote is true. Some versions can search the web, but that is an add-on: the model still writes the reply. If you need the latest train time or the price of a product today, search for it. If you need help putting thoughts into words, ask the chatbot — a writing partner that predicts text brilliantly, not a source of facts to trust on its own.
This explainer was written by The Daily Short's desk from the sources listed above. It is general information, not professional advice. Spot an error? See our Corrections Policy or tell us via the Contact page and we will correct it. Follow our official Instagram @thedaily.short.