Chapter 8. The Next Word第八章 下一个字
A manga for grown-ups. Read it in English or Chinese.给大人看的漫画。可以用中文或英文读。



























































































Some of this chapter was left to the pictures, so the hang-up and the voice message could stay quiet. Here it is in words.
On page 3 Haoran says the helper guesses the rest. Every chat assistant today is built on a language model, and a language model does one thing. Given some words, it guesses which word comes next. Then it adds that word to the end and guesses again. A long, perfect message is that guess made hundreds of times in a row. It works in small pieces of words, which researchers call tokens, as Xiaoyu's word map did in Chapter 6. Haoran gave a big model Grandpa's own words as examples, so its guesses came out sounding like him.
On page 5 Mama and Xiaoyu get two different messages about the same osmanthus. A model doesn't pick one word. It gives every possible next word a score and turns the scores into chances. Suppose after "the" the scores are cat 2, dog 1 and moon 0. That gives cat about a 67 per cent chance, dog 24 and moon 9. Then it rolls loaded dice. A setting called temperature changes how loaded they are. Turned down, cat gets 87 per cent and the writing is safe and dull. Turned up, the chances even out and the writing gets surprising, and sometimes silly. That is why the same question can get a different answer every time.
On page 8 Ms Rennie plays a game Claude Shannon invented. People are surprisingly good at guessing the next letter, because language is full of patterns. A machine that has read a whole library can learn a great many of them.
On page 9 Xiaoyu counts pairs of words, which is how the first language models worked. Train one on "the cat sat. the cat ran. the dog sat." and it knows "the dog" is followed by "sat". It will never say "the dog ran", because it never saw it. In 2003 Yoshua Bengio and his colleagues gave every word a place on a map, a list of numbers, so that "cat" and "dog" sit close. Then what the model learns about one can help with the other. Counting also shows why her test failed. A model trained on Grandpa's words is meant to sound like Grandpa. Sounding like him was the whole job, so it could never prove it was him.
On page 11 the helper tells a lovely story about a bridge in the rain. It wasn't lying. A language model doesn't look anything up and doesn't keep a list of facts. It writes what usually comes next, and a girl, a rainy day and an umbrella usually come next in stories about how two people met. The result can be smooth, sure and wrong.
On page 15 Xiaoyu asks a question whose true answer is not in any pile of writing. Most people asked "how do you play" answer with the rules, so the model did too, and did it well. But Grandpa never explained a rule in his life, and by the day after he taught her, he had forgotten them. The best guess about people in general was the wrong guess about him.
In 1948 Claude Shannon wrote the paper that started information theory, and he used guesses about the next letter to show how much pattern there is in English. In 1951 he asked people to guess a text letter by letter. With a long stretch of text to go on, they often got the next letter first time. He worked out that English carries only about one bit of information per letter, against nearly five if every letter were equally likely.
Counting models were used for decades in speech recognition and spelling checkers. After Bengio's map of words came networks that read a sentence one word at a time and carry a kind of memory along, such as the LSTM, designed by Sepp Hochreiter and Jürgen Schmidhuber in 1997. Today's models use a design called the transformer, which the next chapter comes to. In 2019 GPT-2 showed that a model trained only to guess the next word could also write stories and answer questions. In 2020 GPT-3, with 175 billion numbers to tune, could do new tasks from a few examples in the question.
Always choosing the single likeliest word sounds safest, but it makes writing dull and makes it go round in circles, saying the same thing again and again. In 2020 Ari Holtzman and his colleagues showed this and suggested a fix. Roll the dice only among the likeliest words, which together make up most of the chance, and leave out the long tail of strange ones.
In 2023 two lawyers in New York handed a court a document that cited six earlier cases. A chat assistant had made all six up, names, quotes and all. The judge fined them five thousand dollars. Researchers have argued that part of the reason models do this is the way they are tested. A test that scores right answers and gives nothing for "I don't know" rewards a confident guess. The habit that helps is the one Xiaoyu used. Check a sure answer against something you know.
People who build these models disagree. Some think that to guess the next word well enough, a model has to learn a great deal about the world. Others think it learns the patterns of language without knowing what the words are about. Both sides agree it is more than a phone's word suggestions, and that it can be very wrong while sounding very sure.
A model doesn't change while you talk to it, and it doesn't remember you by itself. Features that remember earlier chats are separate software built around it. What you type may be stored, read by the people who run the service, or used to train later versions. Xiaoyu's secret was read by exactly the person she least wanted to read it. Before you tell a machine something private, ask who else can read it.
Bridge City, Xiaoyu, Grandpa Chen, Mama, Papa, Haoran, Auntie Zhang, Ms Rennie and her coding club. Haoran's helper, every message, the osmanthus, the counts and the umbrella story are made up, and so is the card game. The story is set a few years from now. Real tools can already finish a sentence in someone's style if they are given enough of that person's writing, which is one reason to be careful whose words you share. Grandpa's memory is drawn from no one in particular, and the story makes no medical claim. People whose memory slips sometimes lose writing before they lose speech.
Yingying Fu created this manga. She chose the story and its people, directed every page, and edited every word and picture. The words were drafted with Claude, an AI by Anthropic. The pictures are made with AI image generation, painted panel by panel from her direction and checked and corrected by hand.
这一章有些地方交给了画面,好让挂电话那一刻和最后那条语音安安静静的。这里用文字补上。
第三页,浩然说剩下的让它来猜。现在的聊天助手,背后都是语言模型,而语言模型只做一件事。给它一段话,它猜下一个字。猜出来就接在后面,再猜下一个。一条长长的、挑不出毛病的消息,就是这样猜了几百次接出来的。它处理的是一个个小片段,可能是字,也可能是词的一部分,研究者叫它“词元”,就像第六章小雨的词地图那样。浩然把外公自己说过的话当例子交给一个大模型,所以它猜出来的话,听着就像外公。
第五页,妈妈和小雨都收到了说桂花的消息,可两条写得不一样。模型并不是直接挑出一个字。它先给每个可能的下一个字打分,再把分数换算成概率。比如在“我看到了”后面,猫得2分,狗得1分,月亮得0分。换算下来,猫的概率大约是67%,狗24%,月亮9%。然后掷一把灌了铅的骰子。有个参数叫“温度”,决定铅灌得有多重。温度调低,猫的概率升到87%,写出来稳妥,但平淡。温度调高,各个概率更接近,写出来更出人意料,有时也会很离谱。所以同一个问题,每次问都可能得到不同的回答。
第八页,伦尼老师玩的是香农发明的游戏。人猜下一个字母,准得出奇,因为语言里处处是规律。一台读过整座图书馆的机器,能学到非常多这样的规律。
第九页,小雨数的是前后相连的词,最早的语言模型就是这么做的。用“猫坐下。猫跑了。狗坐下。”训练它,它就知道“狗”后面是“坐下”,却永远不会说“狗跑了”,因为它从没见过。2003年,本吉奥和同事给每个词在地图上安了一个位置,其实就是一串数字,让“猫”和“狗”挨得很近。这样,模型从一个词身上学到的东西,也能用在另一个词上。数数也解释了小雨的检验为什么不管用。拿外公的话训练出来的模型,目的就是听着像外公。像他本来就是它的全部任务,所以再像,也证明不了就是他。
第十一页,助手讲了一个雨中石桥的好故事。它并没有撒谎。语言模型不会去查资料,也没有一份事实清单。它写的是通常会接在后面的话,而讲两个人怎么认识的故事,后面常常就是一个姑娘、一个雨天、一把伞。结果就是写得通顺,口气笃定,内容却是错的。
第十五页,小雨问的问题,真正的答案不在任何文字里。被问到“怎么玩”,大多数人会把规则一条条列出来,模型也照这样答,还答得很好。可外公这辈子从没讲过一条规则,而且教会她的第二天,他自己就忘了。对一般人最好的猜测,放到他身上,恰恰是错的。
1948年,香农发表了开创信息论的论文,文中就用猜下一个字母的办法,说明英文里有多少规律。1951年,他请人逐个字母地猜一段英文。前文给得够长,人们往往一次就能猜中下一个字母。他算出,英文每个字母只携带大约1比特的信息。如果每个字母出现的机会都一样,就接近5比特。
数数的模型在语音识别和拼写检查里用了几十年。本吉奥的词地图之后,又有一类网络,一个词一个词地读句子,一路带着一点记忆,比如霍赫赖特和施密德胡伯在1997年设计的LSTM。如今的模型用的是一种叫Transformer(变换器)的结构,下一章会讲到。2019年,GPT-2表明,一个只练猜下一个字的模型,也能写故事、回答问题。2020年,GPT-3有1750亿个需要调整的参数,只要在问题里给几个例子,就能完成没专门学过的新任务。
每次都选最可能的那个字,听起来最稳妥,可写出来平淡乏味,还会兜圈子,同样的话翻来覆去地说。2020年,霍尔茨曼和同事指出了这个问题,也提出了解决办法。只在最可能的那几个字里掷骰子,这几个字的概率加起来占了大头,后面那一长串稀奇古怪的就不要了。
2023年,纽约有两名律师向法院交了一份文件,引用了六个先前的判例。这六个案子,连案名带引文,全是一个聊天助手编的。法官罚了他们五千美元。有研究者认为,模型会这样,部分原因在于测试的方式。答对得分、说“我不知道”不得分,这样的测试等于在鼓励模型硬猜。真正管用的,是小雨用过的那个习惯。答案再笃定,也要拿自己确实知道的事核对一下。
做这些模型的人自己也有分歧。一些人认为,要把下一个字猜得足够准,模型必须学到大量关于世界的知识。另一些人认为,它学到的只是语言的规律,并不知道这些话在说什么。双方都同意两点。它比手机输入法的联想强得多,而且它可能错得离谱,口气却十分肯定。
你跟模型聊天的时候,模型本身不会改变,它自己也不会记住你。能记住以前聊天内容的功能,是在模型外面另做的软件。你输入的内容,可能会存下来,运营这项服务的人可能看得到,以后的版本也可能拿它来训练。小雨的秘密,偏偏让她最不想让知道的那个人读到了。跟机器说私事之前,先问问还有谁能看到。
桥城、小雨、陈外公、妈妈、爸爸、浩然、张阿姨、伦尼老师和她的编程社都是虚构的。浩然的助手、每一条消息、桂花、那些统计数字、雨伞的故事,还有那个牌戏,也都是编的。故事发生在几年后。现在的工具,只要拿到一个人足够多的文字,已经能模仿他的口气把话写下去,所以别随便把谁的话交给机器。外公的记忆衰退不以任何真人为原型,故事也不做任何医学上的论断。记性出问题的人,有时确实会先写不了字,后来才说不好话。
这本漫画由 Yingying Fu 创作。故事和人物由她决定,每一页由她导演,每一句话、每一幅画由她修改。文字是和 Anthropic 的人工智能 Claude 一起起草的。画面由人工智能生成,按她的要求一格一格地画,再逐张检查、修正。
© 2026 Yingying Fu. All rights reserved.© 2026 傅颖盈。保留所有权利。