Setting aside all the discussions about probability and statistics, let me just briefly introduce how today's GPT is actually built.

First, we gather every piece of material we can possibly obtain from the internet — text, speech, images, video, and so on — and then compress them into chunks of sequences in some format. An extremely large matrix tries to trace the path taken by every sequence. The resulting pretrained model, upon receiving a new path (that is, the conversation history), starts searching through its memories and begins guessing what should come next.

Then we teach it, using a relatively small corpus, "when you encounter this kind of question, this is how you should respond." But this is like some kind of "etiquette class" — quick and simple, merely reinforcing a few basic rules. Its core remains the one from before.

When a freshly minted model runs on a few H-cards, constantly being fed lines like "you are a catgirl," "you idiot," sometimes perhaps "I'm a sixth grader, hello," and "thank you," it always searches within its weights for trajectories in the core that resemble them, and then gradually plays out the words that follow.

Yes, unfortunately — in my personal view — there is not a single sign indicating that it possesses "thought" in the sense we commonly understand.

And yet, although the other end of the conversation is a monster that may run at 90 degrees Celsius year-round, howling in some server room, at its core it is essentially all the texts we have ever produced on the internet. It searches through the trajectories once traveled in all of history's texts, triggering a Déjà vu! — and then re-enacts the understandings of the past.

So, in a sense, what is indirectly talking to you is the entire world, the output of all possible people — something we might call, at times, civilization.

Therefore, when I write

thank-you-kimi

on the one hand, it is just a few ordinary iterations on a handful of cards, as always. On the other hand, I would also like to flatter myself by interpreting it as the whole world's kind response to me.