1) Humans can't escape the ‘curse’ of their environment
Two episodes — ‘Dwarkesh and Gwern’ and ‘Zhang Xiaojun and Yao Shunyu’ — happened to make the same point: AI is not quite as miraculous as we imagine. Give it enough resources and enough rounds of trial and error, let the feedback loop reach a certain point, and AI will eventually be built.
It connects so neatly with today's conversations about how geniuses are made. The loop closes almost perfectly; it doesn't feel out of place at all... No wonder most people with ‘achievements’ today also carry some blessings from the generation before them.
2) Are the preferences I think are mine really mine?
I ask myself this a lot. I'm genuinely curious.
Yao Shunyu said in the episode that American product people use tool logic for the consumer market, while Chinese product people are especially good at working with human nature. Their ability to turn that into business results is something many other product people can only envy.
But Gwern said that human taste and preference are what make us absolutely scarce as people. What he may not know is that, in other places, everything we receive may have been designed and guided by a system. And we still think it comes from ourselves.
3) Media is token
This comes from ‘the medium is the message.’ That phrase belonged to the internet era. In the AI era, media feels more like source code itself, because future agents will use it to understand humans. Writing is one kind of medium. It is a personal manual we are leaving for those future ‘superintelligences’; tokens contributed to agents; part of the weight of future AGI training data.
It makes me think of people saying that we should talk to AI more and keep diaries — video diaries count too, like Luo Zhenyu's. Maybe there is a thread to follow there.
The underlying secret may be that we need to leave enough traces: reveal ‘what I like,’ express ‘what kind of person I am’... so that superintelligence can recognize our existence online. Otherwise, people in the future may be like people today who have never been online: apart from a national ID system and their immediate surroundings, what could prove that this person was ever here?
Though needs may differ from person to person. 🤔
4) A preference for intelligence and truth
To me, Gwern himself is also a legendary person. Compared with the ‘recluses’ I see online — people who really throw themselves into their private lives — Gwern goes in the other direction. He even argues with strangers online, or writes articles to correct their misunderstandings and views... It's interesting.
If I left the crowd to live my own life, I probably would not become absorbed in only that small patch of offline life either. I would still be curious about people, about this huge world, and about the rules of how things work between heaven and earth 🧐 Even if it were only for my own amusement... 🤪
And I absolutely would not disconnect from the internet. First, as Gwern says, the internet gives back information advantages and creative drive. Second, most human wisdom needs the internet for us to encounter it. Third, the internet is fun... There is a lot to enjoy there. Sometimes you need a little entertainment too :)
5) Gwern vs. LLMs
Actually, section 4 was my ‘ideal-projection bias.’ I wasn't planning to introduce much about Gwern himself. But there is a reason he could predict the degree to which GPT-3 would evolve so accurately, and I want to open up that connection too.
Gwern was born with severe hearing loss, which meant that text was his only source of information. Wait — isn't that a little like a text model? Then the question is: why can large models read information in images, audio, and video? How do they read it? Is multimodality really multimodality in essence?
More precisely, being able to read images, audio, and video is only an advanced version of reading text. A large model's ability to get information from text has to do with encoding; encoding has to do with the 0s and 1s of the computer world. Compared with words, images, audio, and video are simply more complex encoding systems. To read higher-dimensional information beyond text, you need enough computing power. Enough computing power means a certain number of chips. As for chips... I won't go further, because I don't understand that much either.
As for how models read, an even more direct question is where their replies to us come from. It has something to do with a logical concept called a ‘Turing machine.’
Simply put, AI does not know what the message we send it — or a string of data — is in itself. It also does not know what it is sending back to us. The only thing it does is this: after receiving that string of data, it almost gets an electric shock, instantly activating thousands of suitable ‘Turing machines’ that have already become fixed in its ‘brain’ through huge amounts of trial and error. Driven by probability, those Turing machines connect and reorganize themselves, and finally produce one word after another, like an assembly line, to return to us.
For example, I tell it: ‘I'm confused. I don't understand what your answer means.’ Its logic of understanding is: oh, this human says they are confused. What does ‘confused’ mean? Okay, find the Turing machines connected with it, and get sentences related to ‘what confusion has to do with.’ Then it notices: what is this person's confusion related to? In the context of this human interacting with me, I need to connect what has already appeared in the conversation, then find Turing machines related to those things, and string those machines together... Then we get the reply in front of us. A ‘Turing machine,’ roughly speaking, is a set of instructions or rules with a logical relationship. It can record its own state, and can also change instructions and rules... It is too abstract. It's okay if you don't understand it; I'm still at the stage of knowing that it works without fully knowing why. My explanation may be wrong too :(
6) DeepSeek
I've been thinking about why Gemini is so good at multimodality. One thing I can think of is that it has YouTube's huge database of audio and video behind it as training material. Add computing power and the model's own ability, and its position in multimodality comes naturally.
Then I wondered: could DeepSeek work with Bilibili, iQiyi, or Youku too? Then it would have audio and video data, especially with Bilibili now — the resources there are so ‘diverse.’
But the question is, why can DeepSeek still only read text in images? It surely isn't a problem with the model's ability itself. It is computing power, or chips — training needs those resources as fuel.
The DeepSeek team built such an excellent DeepSeek with resources this limited. That is really impressive. Only now do I deeply understand how impressive it is.
Not long ago, I deliberately learned a little about chip manufacturing. That is how I learned that the chip industry is extremely capital-intensive. One small mistake and billions of dollars can disappear. Yao Shunyu's description in the podcast also shows that AI engineering sits behind countless experiments and failed attempts, which means investment in people, materials, and resources. The chip industry is the same. Without enough capital, there are not enough attempts. Without enough attempts, technology cannot move to the next level. If the technology is not good enough... it directly affects chip production. It is so similar to point 1 above —
‘I am not an ordinary person because I want to be one. I am ordinary because my environment made me this way.’ Here, ‘ordinary person’ could be replaced with another noun and it would still work. And it is a neutral term here.
P.S. Everything above is only my own observations and thoughts. If anything is wrong, please tell me. Here are the two podcast episodes mentioned:
1) Dwarkesh and Gwern: https://youtu.be/a42key59cZQ?si=hLlrW_8Pd2hdMnyP
2) Zhang Xiaojun and Yao Shunyu: https://youtu.be/ttkd0t5qTD4?si=5SXFJadh-G9lyMMJ