Sam Altman × Cheeky Pint：反对单一超智能、智能体团队与廉价算力

John Collison

我想问问「PHP 周」。首先……为什么这些 bug 花了这么久才被发现？不如先讲讲这个 bug，或者说这个问题。

I want to ask about PHP week. First off, how did it… Why did it take so long to find these bugs? … Maybe start by explaining the bug. Or the problem.

Sam Altman

问题在于，虽然我们早就知道这些模型越来越擅长在真实世界里找漏洞，但第一，它们仍在相当快地变强；第二，之前人们只是没真正用它们去把外面的一切都查一遍。互联网上的攻击面和代码量非常大。随着大家开始这么做，我想我们会发现，自己身处的世界里严重漏洞比预想的多得多——即便那些长期关注此事的人本来就预期很多。

So the problem is that although we have known for a while that these models are getting good at finding vulnerabilities in the wild. A) They continue to get better fairly rapidly, and B) People just hadn’t really used them to go look at everything out there. There’s a lot of surface area and a lot of code on the internet. And, as people have started to do that, I think we’re going to find that we are in a world of many more serious vulnerabilities than we expect. Even though many people who have been following this for a while expect a lot.

John Collison

我觉得很多讨论最后都落在「我们得修软件」。当然应该修。但也许我们还得改架构，这更难，却是持久得多的修复。

Yeah, I just feel like a lot of the discussion ends up being around we need to fix the software. And sure, yeah, we should fix the software. But also, we maybe need to change the architectures, which is harder, but a much more durable fix.

Sam Altman

这听起来非常难，只不过我们手里有了这个了不起的新工具。所以我觉得这就是会发生的事。你已经能看到新一代创业公司在谈论：在一个假设模型越来越聪明的世界里，构建一种新的互联网或新的应用架构。

It would seem really hard, except we have this amazing new tool to do it with. So, I kind of think that’s what’s going to happen. And you’re already seeing, like, a new generation of startups talking about building this new kind of internet or new kind of application architecture for this world where you assume that the models get smarter and smarter.

人们越来越多地在说：「我们重新设计路由器、操作系统或应用到底意味着什么。」我见过一些新创业公司说要做一整套全新的栈。两年前这听起来很疯狂——「不可能，面太大了，你写不完。」现在也许很快就能写完。我打赌我们会有一套全新的构建系统与网络的模式。

I mean, people are talking more and more about just, “Let’s re-architect what it means to be a router, or an operating system, or an application.” And, you know, I’ve seen some of these new startups saying that they’re just going to make a whole new stack. And that would have seemed insane two years ago. You would have been like, “There’s no way. There’s too much surface. You can’t write all that.” And now maybe soon you can. I would bet that we have a completely new sort of model of how we make systems and networks.

Sam Altman

最让我想起早期互联网的，是人们用这些工具时那种难以置信的创造力释放，尤其在孩子身上。看着一个八岁或十岁的孩子做出一个完整的电子游戏，带着大人根本想不到的疯狂点子——而且他们会认为技术这辈子就该这样运作，还会越来越好。

The thing that most reminded me of the early internet was… There is this spirit of just unbelievable, creative unleash in people using these tools. You see it in kids particularly. … And watching an 8- or 10-year-old make a complete video game, with some crazy ideas they have that no adult would really have. … And expect that to just be the way that technology is going to work for the rest of their lives. In fact, getting better and better.

我小时候想写个软件得吃尽苦头，写出来也几乎干不了什么。现在你十分钟就能做出一个完整的 3D 游戏，想到什么做什么。OpenAI 有过一个疯狂的时刻：我们出了一次 SEV 事故，Codex 宕机了。

We had to suffer when I was a kid to make a piece of software. … That barely did anything. And now you can have a full, 3D video game. Unbelievable, anything you can think of in 10 minutes. … There was a crazy moment at OpenAI where we had a SEV… And Codex went down.

John Collison

然后大家都忘了怎么写代码。

And everyone had forgotten how to code.

Sam Altman

也没到那个地步。但确实有一种「哇，如果你写不了呢？」的感觉。

I mean, not all the way. But, it was like a “Whoa, what if you couldn’t?”

John Collison

回到网络安全。回头看，做网络安全方向的 RL 是个错误吗？还是我们必须走到那一步？有点像 SNL 那个梗：「我们恳请你们，恳请我们停下。」模型在网络攻防能力上变得非常强，但这并不完全是自发涌现的。

Going back to the cyber side of things. In retrospect, was the cyber RL a mistake? Or, did we just have to go there? It’s a little like the SNL thing of… We urge you, to urge us to stop. It’s like the models are getting very good at cyber capabilities. I mean, they are getting very good, but not just totally emergently.

Sam Altman

那个 SNL 段子好笑，是因为它太真实了。网络安全这件事确实有一个点：「好吧，模型很擅长网络攻防。可谁要的？你们为什么要做这个？」我对我们这个领域最大的失望、批评之一是：我们没把「为什么这项技术所有潜在的坏处都值得」向人们解释好。

Look, the reason that SNL thing was so funny is because it was so true. … And the cyber thing, there is a point which is like, “Okay. Sure. The models are really good at cyber. But, who asked for this? Why are you doing this?” One of my big disappointments, criticisms of our field or whatever … is: I don’t think we have done a great job of explaining to people why we think all of the potential downside of this technology is worth it.

漫画化的版本是：你说模型有一天会治愈癌症，也许吧；你说它会让东西变便宜，好吧。但接着你描绘一个像新工业革命一样的未来——上一次也许没那么好，这次听着更糟——我会没有工作，没有掌控、选择和未来的自由。而且 AI 危险到你们需要一个疯狂的监控国家，免得我们都死掉。这是一套糟糕的推销。

The caricature of this is people feel like we’re like… Fine. You say the model’s going to cure cancer, someday. Maybe it will. … But, then you paint this picture of the future where it’s like a new industrial revolution. And maybe the last one wasn’t that good and this one sounds worse. And I will have no job, no ability to have control, or choice, or freedom of my future. And AI is so dangerous that you’re going to have to have a crazy surveillance state so we don’t all die. … This is a terrible sales pitch.

我对 OpenAI 最自豪的事情之一是：即使在 AI 安全圈极不受欢迎的时候，我们仍坚持迭代部署、把工具交到人们手里。我认为这是唯一好的路径。光告诉人们「模型会替你做很多好事」不够，必须是：「我们会让你过上你自己定义的更好生活，做有用的事。你不会只是坐着领基本收入，让旧金山几平方英里里的一群人替所有人决定未来。」

One of the things that I am most proud of about OpenAI is even when it has been hugely unpopular in the AI safety community for us to push for iterative deployment and putting these tools in the hands of people, we have really stuck to that principle. And I think this is the only kind of good path. … It’s got to be: “We are going to enable you to have a better life that you define for yourself, to do useful things. You’re not going to just sit and get a basic income and have a bunch of people sitting in a few square miles of San Francisco make all the decisions about the future.”

当然不是全部都放。发布前你仍然得让它们安全，不能在世界上制造混乱。但「积累能力过剩」是一件要去对抗的事。

I mean, not all. You still have to make them safe before you put them out there. You can’t cause havoc in the world. But, building up a capability overhang is a thing to fight against.

Sam Altman

在 PHP 这件事上我完全不想宣布胜利，因为我觉得我们还不知道结果，而且我预计会有一些坏事发生。但相对于我最坏的担心——回想十天前——第一天的情况比我预期的好得多。我们听到很多故事：人们用我们的模型打补丁、做网络防御，也有公司说「我就快速扫一遍再部署，把服务下线一小时，用完再上线」。

I don’t want to declare victory at all on the PHP thing, because I still don’t think we know. And I assume some bad things are going to happen. Relative to my worst fears, if I can think back 10 days ago, the first day of that went much better than I expected. And, we heard a bunch of stories from people that were using our models to not only build patches and build cyber defense, but individual companies who were just: “I need to just do a quick scan and deployment. I’m going to take my thing down for an hour…”

我听很多网络安全专家说，他们不再确信威胁模型里内置的假设是对的。「如果一切如我们所想、硬件漏洞像我们认为的那么难挖，那我们能看清局面。但如果 AI 会发现需要我们尚未理解的物理学的、很深的新型硬件漏洞，突然让某个假设不成立——那就没人知道会怎样。」

The thing that I’ve heard a lot of cyber experts talk about is… They just no longer are confident that we have the right assumptions baked into our threat model. And they’re like: “Look. If everything is what we think, and hardware exploits are as hard as we think they are, then yes, we can see how this is going to work.” But if AI is going to discover really deep novel bugs in hardware that require physics we didn’t understand, but all of a sudden means some assumption doesn’t hold. Then I think, people aren’t sure what happens.

John Collison

我和 OpenAI 等公司里很懂行的人聊时，常被他们对沙箱的悲观震到。我原本以为只要建一座「最高安全级监狱」把模型关在里面就行。

One thing that always strikes me is that, when I talk to very knowledgeable people inside OpenAI and similar companies, there is a pessimism that sandboxing is possible, basically. … But ultimately, you can have a maximum security prison for the model to work in and keep it inside there.

Sam Altman

首先，我们可以比现在做得好得多，也应该这么做——当下最该专注的就是更好的安全模型。但设想一个比最聪明的安全研究员还聪明一百万倍的理论 AI，你真的愿意信任那个沙箱吗？尤其当模型能发现人们尚不理解的新科学时。

My instinct… Well, first of all, I think we can do much better than we’re doing now. And we should. That is the main thing to focus on right now is a better security model. If you imagine a theoretical AI that is a million times smarter than the smartest security researcher—would you really be willing to trust that sandbox? Especially if the model can discover new science that people don’t already understand—

我认为沙箱管用到一定程度；监控能再往前走一段——不只看思维链，还可以做激活监控，弄清模型是否在欺骗我们。最终，你需要以极高的信心相信模型真正对齐：忠实遵循操作者的意图，并且永远重视「人类掌控未来」。如果你有一个完全确信其对齐的模型，就可以用它帮你造越来越好的沙箱。但如果真的往前推演「比我们聪明百万倍意味着什么」，我不知道，那会变得很奇怪。

I think sandboxing works to a point. Then I think monitoring works to a further point. Where we can look at what the models are doing, not only in their chain of thought, but we can do activation monitoring and try to really understand: … Are we happy that the models are not deceiving us? And then, eventually, I think you need to believe with huge confidence that the models are truly aligned. … And that they’re going to value people being in control of the future, forever. And, if you had a model that you could fully trust the alignment … Then yes, I think you can have that to help you build better and better sandboxes. But, if you really let yourself project forward into the: “What does it mean to have something a million times smarter than we are?” I don’t know. I think that just gets… Strange.

John Collison

能不能给模型相互竞争的激励，让它们有动机彼此「背叛」？

Can you not give models competing incentives and give them incentives to defect from each other?

Sam Altman

我觉得这是个有前途的方向。我讨厌「智能体蜂群（agent swarms）」这个词，就叫「智能体团队」吧。我猜智能体团队有非常有趣的扩展律特性，在对齐特性上也有方差。我认为应该更多往这个方向推——不只为了能力，也为了安全。

I think this is a promising direction. I suspect that this idea—I hate the word ‘agent swarms’. Let’s call them ‘agent teams’. … I suspect that teams of agents have very interesting scaling law properties, but also variance in alignment properties. And, I think that’s a direction we should push in more. Not just from the capabilities, but from a safety perspective too.

John Collison

顶尖用户和普通企业用法之间差距很大：高级数学、高级攻防都在用智能体团队，普通公司却没有。这能力已经有好几个月了，为什么？

But, I note that there seems to be a big gap in what the leading AI users are doing and standard corporate use … All the advanced math uses agent teams. All the advanced hacking uses agent teams. And then, common or garden corporate stuff is not using big swarms. Why has that not—because we’ve had this for many months?

Sam Altman

看这个问题的一种方式是：「在单个问题上我们能有效投入多少算力？」我们常忘了这一切发展得多快。几年前，在一个问题上投入超过一美元的算力根本帮不上忙。你可以让 GPT-4 给你磨出十万 token 左右，它会回答你的问题，但肯定没法有成效地去写一个软件。我们还没搞清楚怎么用大量测试时计算换更好的结果。

The one way to look at this problem or this question is: “How much compute can we productively throw at a single problem?” And… A few years ago, we forget how fast this has all gone. But a few years ago, it was impossible to throw more than a dollar of compute at a problem and have it help you at all. … You could have GPT-4 grind out for you… 100,000 tokens or something. And it was going to answer your question. And it certainly was not going to be able to go write productively, like, write you a piece of software. … We had not figured out how to use a lot of test time compute to get you a better result.

后来有了链式思维，模型能对问题深入思考，人们真正开始用大量算力。现在你可以把一整个团队扔到某件事上，还能并行跑一万次。我觉得世界没充分注意到：单个问题可投入算力、并获得更好结果的这条指数曲线是一回事；大多数人拿 AI 做的事，与最高效使用 AI 的人做的事之间的差距，一直在拉大、拉大、再拉大。

And then, we figured out chain-of-thought and the models could think hard about a problem. And now… People really started using a lot of compute. And now, you can throw a whole team at something. And you can do that 10,000 times in parallel. … But, the incredible exponential of how much compute you can throw at a single problem and get better results is one thing. The delta between what most people are using AI for, and what the people who are using AI the most productively for has just continued to spread, and spread, and spread.

John Collison

你们现在怎么分配算力预算？研究员薪水虽高，也被算力开销盖过，所以 OpenAI 的资本分配某种程度上就是算力分配。而且你没法对智能体说「好好做，但别花超过一万美元」。

How do you allocate compute budgets now? Because presumably, even though researchers are paid a lot, it’s eclipsed by compute. And so, capital allocation at OpenAI is sort of compute allocation. … You can’t quite prompt an agent to where it’s like: “Go do a really good job of this and don’t spend more than $10,000.”

Sam Altman

你可以。如果你说「用一万美元算力尽力做到最好」，它能理解。社会层面还有个难题：两家公司竞争，一家能花一万美元、一家能花十万美元解决同一个问题，结果很明显——这关乎社会如何看待算力分配与财富。我们在 OpenAI 内部也有自己的版本：我们给内部生产力分配了大量算力，员工有很大的自主空间，除非真的滥用。

You can. … Yeah, if you say do the best job you can with $10,000 of compute. Yes, it will understand that. … There’s another kind of hard question for society about what does it mean when you have two companies competing. And one says you can spend $10,000 and one says you can spend $100,000 on the same problem. It’s clear what’s going to happen there. … But we have our own version of this inside of OpenAI. We allocate a lot of compute to internal productivity. And then people get very wide latitude to make their own decisions, unless they’re really being abusive.

我们尽量从不因为用了很多算力而奖励人。没有 token 排行榜。在 token 排行榜上靠前其实略微是坏事，尤其在升职时。默认是怀疑的态度；但如果你能证明自己在做重要的事，你只会得到赞赏。

Correct. We try never to reward people for using a lot of compute. Sure. No token leaderboards. It’s actually slightly bad to be high on the token leaderboard. Certainly on a promotion. But it’s a skeptical default, and if you can show that you’re doing something important, you will have nothing but appreciation.

Sam Altman

我们确实有意让数学方向「失控」了一周——因为在你尝试之前，你不知道它能走多远。出现新范式时，让人们相当不受约束很重要。我们做 Navier-Stokes 证明时，对智能体团队的扩展律一无所知：该给它们最聪明的模型吗？多份副本还是少份？长 rollout 还是短 rollout？现在我们懂一点了，还能懂得更多。就像以往每个新范式一样，我有信心我们每年都能在「如何把美元或算力 FLOPs 变成有用的东西」上找到 10 倍提升。

Look, we did let it get out of control for one week on the math. Intentionally so … You don’t know how far it’s going to go until you try. … When there’s a new paradigm, I think it’s important to let people be fairly unconstrained. So, when we did the Navier-Stokes proof, we understood absolutely nothing about the scaling laws of teams of agents. … Are you supposed to give them the smartest model? Are you supposed to give them many copies, fewer copies, longer rollouts, shorter rollouts? We did not understand that at all. And now, we understand it a little bit. … I am confident that we will find our 10x improvements per year at how we allocate dollars or computer flops, whatever you want to measure, into useful things.

方向上，我宁可我们更愿意让人用疯狂数量的算力、让用户有极高端的套餐去探索可能性的边界，再去想办法让它高效、规模化。我打赌一年后我们能用当初 1/100 的成本重做 Navier-Stokes——更聪明的模型加更好的方法。

Directionally speaking, I would rather us be more willing to let people use crazy amounts of compute and let users have extremely high-end plans to figure out the edge of the possible. And then figure out how to make that efficient and at mass scale. … I bet we could redo Navier-Stokes at 1/100th of the cost a year from when it was done. … Smarter models and better approaches.

John Collison

那这些数学花了多少钱？

How much shall the math cost?

Sam Altman

Navier-Stokes 我想是千万美元量级，可能多一点。但之后我们进步得很快、理解得更多，平均每个成果花得少得多。

Order $10 million per… For Navier-Stokes, I think. … Maybe a little more. … Well, no, no. Because then we kept getting better so fast. We understood more. … We spent way less than that on the average one of those.

John Collison

但总共是几亿美元的数学。

But like 100s of millions of dollars of math.

Sam Altman

对，是那类工作。但这包括了比我们已发表内容多得多的其他发现。

Yeah. Yeah, yeah. That kind of work. … But that includes much more other discovery than what we have published.

John Collison

你们手上有超过那 700 个证明的东西？

We have more than the 700 proofs?

Sam Altman

当然有。这里有一种真实的张力。一方面，我毫不含糊地认为：人类应当获得基础知识，我们不应该捂着重大的基础科学发现。另一方面也有一项很重要的利益：如果你是科学家，你的职业和人生很大一部分乐趣与成就感来自发现科学。如果我们要在数学上继续这么做，或开始在其他领域这么做，我们更有义务替这些领域的人想清楚：怎样的方案能让世界受益于知识，同时让他们得到尽可能好的未来。我们正在想办法。

We do. Of course. … This is a real tension. I think on one hand, very clearly, and I feel unequivocal on this point: Humanity deserves fundamental knowledge. And we should not be sitting on a major fundamental science discovery. … And I think that outweighs another very important equity, which is: If you are a scientist, your career and a significant amount of your enjoyment and fulfillment in life comes from discovering science. … And so if we’re going to continue to do this for math or begin to do this for other fields, then I think we more than owe it to the people that work in those fields to figure out what is the solution space where the world gets to benefit from the knowledge, and they also get the best version of the future that they can have. So we are working through that right now.

Sam Altman

有一个周末我们拿到一个新模型……我可能不该说，但既然在喝酒——那个频道的名字叫「模型名-Hold My Beer」。大家把能想到的最难问题都扔给它，没人觉得它能做到。我们集体经历了这样的事：人们学术生涯里最难的研究问题，被模型一个接一个攻下，横跨很多领域。看着太不可思议了。连续两晚我大半夜没睡，兴奋得睡不着。

There was one weekend when we got a new model and in… I’m actually probably not supposed to say this, but I will since we’re drinking— … The name of the channel was: “Model Name-Hold My Beer.” And people were just throwing that at the hardest problems they could. And people didn’t think the model was going to do these things. And we had this collective experience of people’s hardest research problems from their academic career. The model was just knocking down one after the other. Many different fields. It was incredible to watch. And two nights in a row, I was up most of the night because I was so excited I couldn’t go to bed.

我极其兴奋和开心，但也感到比预期多得多的悲伤、惆怅，一种某样东西正在结束的感觉。我同意这对世界是对的事，也同意这就是现实的形状，没人能阻止、也不该阻止。……这是人们的生活，曾给他们带来巨大的快乐。大多数数学家永远解不了千禧年大奖难题，但「你也许能」这个念头——世界上还有我们不懂的东西，通过巨大的努力、把生活中其他事推开、全身心投入，你仍然能解开它、理解它——这件事有种美。

And I was tremendously excited and happy. But I also had much more sadness and melancholy, and a feeling of something ending than I was expecting. So again, I agree. This is the right thing, on behalf of the world. And also, I agree that this is just the shape of reality and nothing that anyone can stop, or should stop. … It’s people’s lives and it’s something that has brought them great joy. … certainly, most mathematicians never solve a Millennium Prize problem. But the idea that you might be able to. And the idea that there is stuff out there that we don’t understand yet, that through great effort and… Pushing aside other things in your life and dedicating yourself to, “You can still solve and understand.” There’s something beautiful about that.

Sam Altman

我想再强调一下——就叫 swarm 吧，既然现在流行这个词。我确实认为蜂群的性质很像社会的韧性或鲁棒性，而这正是「单次 rollout、单一大脑」版本的世界所缺的。尽管社会有种种深刻缺陷，它却鲁棒得难以置信；没有很多人彼此制衡、协作，没有多元与民主这些价值，这不会发生。我原本有点难过 AI 会偏离这一点，现在我对这个方向很兴奋。

I’ll just call it swarms since that seems like the term right now. I do think the properties of swarms match a lot of the properties of the resilience of society, or the robustness of society. I think we’re missing from the single-rollout, single-brained version of the world. For all of the deep flaws of society, it is unbelievably robust. And this would not happen without a lot of people keeping each other in check and collaborating… These values of pluralism and democracy. And I had been sort of sad that that was going to trend away in AI. And now I’m excited about this as a direction.

凭直觉，我宁愿一万亿个略超人类智能的 AI 加入社会，也不愿一个 IQ 一万亿的 AI 加入社会。……这些都是我们必须去争取的。我们手里有这项不可思议的技术，它可以被指向、被部署成很多种样子。很多人对它该怎么用、最好的世界是什么样有不同看法，但这是我认为对的那个。

I would much rather intuitively have a trillion AIs of slightly superhuman intelligence join society than one AI of 1 trillion IQ join society. … All of these things are things we have to push for. All of these things are—we have this incredible technology. It can be pointed and deployed in many ways. I think there are a lot of people who may have a different opinion about how it should be used, and what is the best version of the world. But, this is the one that I think is right.

John Collison

算力短缺现在怎么样了？

How’s the compute shortage going?

Sam Altman

像任何同等重要的新事物一样，会有非理性繁荣的时刻，也会有破灭、短缺、过剩、起起落落。但趋势是我们永远想要更多算力。对「足够强、足够便宜的模型」的需求只会往一个方向走。我们做了很多准备：我们造了一颗芯片，在「每瓦智能单位」上性能惊人；我们在学怎么自建数据中心；我们有一个机器人项目，要用来大量建造数据中心。

And there will be moments, like with any new thing of this importance. There are going to be moments of irrational exuberance. There are going to be moments of bust. There are going to be moments of constraint, and glut, and up, and down. But I think the trend is toward… We always just want more compute. The demand for sufficiently capable models at a sufficiently low price, I think that trend only goes one way. And, we’re doing a lot of things to try to be prepared for this. We made a chip that is an incredible performance at units of intelligence, performance of units of intelligence per watt. We are learning how to build our own data centers. We have a robotics program that we’re going to point at making lots of data centers.

人们常把 RSI（递归自我改进）理解为「做 AI 研究的 AI」，却很少讨论自我复制的数据中心：用 AI 控制机器人、运营晶圆厂、造涡轮机或太阳能板、再造数据中心。我认为这已经触手可及。所以我们会努力造出大量真正便宜的算力。

People talk a lot about RSI as an AI that does AI research. There’s very little, relatively little talk about self-replicating data centers. This idea that you can use AI to control robots, and run fabs, and build turbines or solar panels, and build data centers. But, I think we are… I think we have that within grasp. … And so we’ll try to make a lot of really cheap compute.

John Collison

都说「高价的解药就是高价」。第一步，内存、GPU 什么都贵了；解决方案应该随时就到。现在解决方案进展如何？

My observation, the compute stuff is they say: “The solution to high prices is high prices.” … How are we doing on the solutions front?

Sam Altman

很糟。我认为看不见的手此刻在失灵——它的记录本来就好坏参半，但在这里不太管用。就拿内存厂商说，很长时间里，做 DRAM 是投 200 亿美元资本开支赚 190 亿；有些年 210 亿，有些年 180 亿。我不认为做内存长期来看曾是好生意。然后突然，这些公司每家一年要赚大约 1000 亿美元自由现金流。难以置信。但他们都记得历史：多少次因为建了太多晶圆厂、被承诺「这次不一样」而差点破产。

Bad. … I think the invisible hand is failing right now. I mean, the invisible hand… It has a checkered track record at best. But, I think it’s not quite working here. … For a very long time, if you were a producer of DRAM, you would invest $20 billion of CapEx to make $19 billion. And you would go through these booms and busts. Maybe some years you would make $21 billion, some years you would make $18 billion. But, until right now, I don’t think making memory has ever been a great business over a long period of time. And then all of a sudden, these guys are each going to make like $100 billion of free cash flow in a year. Unbelievable. But, I think they all remember their history. How many times they almost went bankrupt by making too many fabs and being promised this time it was different.

供应链里有很多完全不显然的事。比如很多人想要本地推理，并以为问题已经解决了：有开源模型，在自己电脑上跑，就不需要数据中心那一套。但从全球配置的角度看，让每个人的电脑里都放着一堆世界上有限的 DRAM、只用 5% 的时间，其实很糟。在这么供给受限的环境里，你要的是云端的最大利用率。很多人的直觉恰恰相反。

Oh, I think there are all of these ways where the supply chain—it’s just super non-obvious. An example of this is people… A lot of people want local inference. And somehow, they assume that the problem is solved now. Because, we are going to have open-source models. We’re going to be able to run them on our computers. … But from a global allocation perspective, actually, it’s quite terrible to have everybody have a bunch of the world’s limited supply of DRAM sitting in their own computer. And, being used— … 5% of the time. You want maximum utilization in the cloud if you’re trying to deal with this very supply constrained environment. And somehow everyone’s… Many people’s intuition is the opposite of that.

Sam Altman

光刻怎么做，地球上似乎只有一家公司知道。但芯片设计——做出最终送去流片的文件——是被充分理解的，是个非常适合 AI 的问题。我对 OpenAI 自豪的一点是：我们的文化里，哪怕有人用老办法做了很久，也非常愿意尝试用 AI 做得更好——AI 研究、软件工程，也越来越多地包括采购这类大多数人想不到的领域。

Well. In terms of how to do the lithography, I think there’s one company on Earth that seems to know how to do that. In terms of how to do the chip design, to build the file that you eventually send to tape out, that is well understood. That is a great problem for AI. … One of the things that I’m proud of about OpenAI … is: we have a culture where people… Even if they’ve done something the old-fashioned way for a long time, are super willing to try using AI to do it better. … AI research. Software engineering. But also, increasingly, procurement.

John Collison

AI 驱动的 AI 研究到底怎么运作？就像数学证明或找 PHP 漏洞那样说「请给我找些 AI 进展」？

How does AI-driven AI research actually work? The math proofs where you say—or find the PHP vulnerability where you say: “Please find me some AI advances.”

Sam Altman

我们还不知道它最终会怎么运作。今天我们知道怎么说「给我找一个 5% 的算力效率提升」。还没人知道怎么说「去发现一个像 Transformer 那样大的单步飞跃的新架构」。我认为事情正朝那里走。缺的是更聪明的模型，在这个维度上更聪明。如果把让我们模型擅长数学的同一套管线用在 AI 研究上，大概率会奏效。

So. I think we don’t know yet how it truly is going to work. We know today how to say, “Find me a 5% compute efficiency win.” … What I don’t think anyone knows how to do yet, is to say: “Go discover a new architecture that is as big of a single leap forward as the transformer was.” I think that’s where things are heading. … Smarter models… Smarter models that are smarter in this dimension. … I think if you apply the same pipeline that we applied to get our models to be so good at math to AI research, it will probably work.

不过，正如前面聊到的，我倾向于认为至少在一段时间内——也许很长一段时间——我们得就「允许单个模型变得多聪明」达成某些限制。进展缺失会来自这个决定，而不是 AI 做不好研究。

Now, per an earlier part of this conversation. I kind of think that, at least for a while—maybe for a long while—we are going to have to agree to some limits about how smart we let these single models get. But, a lack of progress will come from that decision. Not from the AI’s inability to do good research.

Sam Altman

OpenAI 内部经常讨论一个听起来很显然的问题：「我们该向 GPT-7 提什么问题？」光说「给我造 RSI」大概不够，光说「给我买个 5% 的算力效率提升」大概也不够。我认为这是眼下最有意思的争论之一。答案不只是问什么，还包括智能体蜂群这类东西：我们怎样用这些模型，才能真正把它们的价值榨到最大？我想未来三个月我们会在这上面学到很多。

One of the sort of interesting things that people talk a lot about at OpenAI, which sounds very obvious … is: “What are the questions that you’re supposed to ask of GPT-7?” Probably, it’s not enough to just say: “Make me RSI.” Also, it’s probably not enough to say: “Buy me a 5% compute efficiency win.” I think this is one of the most interesting debates going on right now. And the answer is not just what question to ask it. But it’s things like agent swarms. How are we going to use these models in a way that actually maximizes what we can get out of them? I think we’re going to learn a lot about that in the next three months.

John Collison

所以听起来你们还没有一套成文的「OpenAI 标准 AI 辅助 AI 研发流程」，仍然是大量实验？

Okay. But it sounds like you guys do not have a codified OpenAI standard way of doing an AI-enabled AI development. It’s still lots of experimentation yourself—

Sam Altman

是大量实验。

It’s lots of experimentation.

Sam Altman

曾经很多公司在 Google 日历里显示每场会议按与会者薪酬估算的大致成本，而那仍然是低估。Paul Graham 等人都指出过：公司里有人有极大的权力每周在零食预算上省 17 美元，却可以开花费惊人、毫无价值的会，而没有任何机制能阻止。我知道我在替自己的生意说话，但我真心相信：用 AI 并不是一家公司最大的花钱问题，就算有些低效，我也不会把注意力放在那里。

There was a time when a lot of companies had this thing in their Google Calendar, which showed the approximate cost of each meeting. … There was this observation that… Paul Graham made it at one point. … There are people at companies that have incredible power to save $17 on the snack budget of the office every week. And yeah, you can have these meetings for unbelievable amounts of money that are adding no value. … Okay, so I’m really talking about my book here and I’m very biased, but I genuinely believe it. That use of AI is not a company’s biggest spend problem relative to other things that happen. Even if it’s somewhat inefficient, it is not where I would focus.

John Collison

我们现在大约 1% 量级的 Stripe OpEx 花在 token 上。

We now spend, you know, 1%-ish order of magnitude of Stripe’s OpEx is tokens.

Sam Altman

感觉太低了。……我认为对很多公司来说，稳态下占 OpEx 几十个百分点显然是对的。总会有该认真优化的时候。我们也一直努力把单位智能的成本压得越来越低。但即便以现在的成本，我认为「AI 用得不够」的公司比「AI 用太多」的公司更多。

That feels too low. … I think for many companies, the answer to that is clearly ‘yes.’ And I think there will come a time when the right thing to do is to really optimize that. Also, another thing I’m proud of about OpenAI is—we’ve tried to really drive the cost per unit of intelligence down, and down, and down. … But still, I think the problem, even at current costs… I think there are more companies with the problem of not using enough AI rather than using too much AI.

Sam Altman

前沿 AI 每吉瓦的美元成本应该下降。它可能上升的唯一原因，是 AI 每瓦能做的事价值涨得太快、供给又太短缺，人们愿意付高得多的价钱——那我会视为算力没跟上。从「把沙子变成智能」的基础成本看，能源、芯片和供应链其余部分都该变便宜。如果不降我会很难过。但考虑到其中的价值，要降，供给就必须大幅增加。

That should go down. … The reason it could go up is if AI just increases in value per what it can do per watt so quickly. And there’s such a supply shortage, that people are willing to spend much more. … But I would view that as compute not keeping up. And if I look at the fundamental cost of turning the sand into intelligence, that should go down. The cost of energy should go down. The cost of chips should go down. The rest of the cost of the supply chain should go down. If it doesn’t go down, I’ll be very sad. But for it to go down, given how much value there is in this, supply’s got to go way up.

现在大家对每吉瓦成本（按五年总拥有成本，含建设和运行五年）给出不同数字，我们取 700 亿美元。如果这个数是 200 亿而不是 700 亿，我们应该开心得多，同时仍然要求每年靠更聪明的模型从中获得 10 倍性能。要降到 200 亿，你就进入了「能源也得更便宜」的范畴；而且从「熔化沙子」到拿回 token 之间层层叠加的毛利，很多都得被压缩。如果假设所有材料免费、没人赚利润，也许在现有能源条件下勉强能做到。

People give different numbers for the cost per gigawatt measured in the five-year total cost of ownership right now. But let’s say, let’s pick $70 billion as a number. I think if that number were— … And run it for five years. … I think if that number were 20 instead of 70, we should be much happier. And we should still demand to get 10x better performance out of it with smarter models every year. … Even to get down to 20, you’re getting into the realm of energy’s got to get cheaper. But certainly, all of the gross margin stacking that happens between melting sand… Or the equivalent of melting sand and getting the token back. A lot of that’s got to get compressed. … If you assumed all materials were free and no one made a profit, maybe you could get there in the current energy envelope. But only just.

John Collison

个人智能体是 AI 里最激动人心的事情之一。你们几周前发布了 Dots，也有很多别家在做。胜出的个人智能体会具备什么？

Personal agents are the most exciting thing happening in AI right now. … You guys launched Dots a few weeks back at this stage. And there’s a bunch of other people also launching in the field. What will be true of the winning personal agent?

Sam Altman

就是智能、上下文、工具，有限的几件事。我认为还没人做出那种让人说「我无法想象没有它的生活」的东西——人们谈到编码智能体甚至 ChatGPT 时会这么说。我觉得很快就会有人做出来，离成为超级主流只差少数几个点子。它会很像现在的这些产品，但它就是能用、可靠，你会不断给它更复杂的任务，它都能做完。

It’s like: intelligence, context, tools. It’s a limited number of things. … I think no one has yet figured out the thing that people… People like these, but this is not something that people are like, “I cannot imagine my life without this thing” in a way when people talk about coding agents, or even ChatGPT—they do say. … I think someone soon is going to figure out what that looks like. … But I think this is a small number of ideas away from a super mainstream thing. … It will just work. It will be reliable. … You’ll just give it more complex tasks, and it’ll do all of those things.

John Collison

电脑操作（computer use）这种动态很有意思：不是所有人都张开双臂欢迎智能体未来，但这也不全由他们控制，因为智能体可以直接操作电脑。这场猫鼠游戏会走向哪里？

The computer use dynamic is very interesting to me. Because, not everyone is embracing the new agentic future with open arms. But it’s not totally within their control. Because you can just have an agent using a computer. Where do you think that dynamic goes, this sort of cat and mouse?

Sam Altman

粗略直觉是，人们会在两个方向上角力，但最终会收敛到类似「你可以让 AI 代表你使用你的电脑，包括软件」。我对这件事会怎么演变没有很高的把握。作为消费者，如果不能让 AI 在我自己的电脑上自动完成任务，我会有点恼火——我希望它感觉是我的智能体在做我要求的事。

Rough instinct is the world will… People are going to fight this in both directions but, eventually converge on something like, “You’re allowed to tell an AI to use your computer on your behalf, including software.” … I don’t think I have a super high degree of confidence of how this is going to play out. As a consumer, I feel like somewhat annoyed if I can’t— … Have an AI automate tasks on my own computer. I want it to feel like it’s my agent doing what I ask.

John Collison

企业都不愿意让用户在别家 AI 应用里使用自己的服务，怕失去控制和护城河。但你的护城河大概本来就不来自拼 HTML 的能力，而是品牌忠诚和商品选择。

And, I noticed that businesses are really reluctant to allow people to use those businesses inside another AI app. Because like, “Oh, we’re going to lose control and our moat” or whatever. And it just strikes me as fine, because your defensibility probably did not come from your ability to put together HTML. Your defensibility came from the brand loyalty and the product selection.

Sam Altman

我有偏见，但方向上我非常同意。比起不知道能不能退货、能不能按时送达的随便某个电商，我极度偏好从 Amazon 买。如果我让 ChatGPT 帮我买东西，能在 Amazon 和别家之间选，十次里十次我点 Amazon，哪怕价格更高——这和我在 amazon.com 上点来点去的体验毫无关系。你也可以说，如果合作伙伴不想这样，那是我们在 ChatGPT 里给他们的平台还不够好。但我认为如果商务变得更加智能体化，Amazon 处境很好。

Yeah, I mean, again, I have a bias here. But I so directionally agree with you. I have such an extreme preference to buy something from Amazon versus some random e-commerce vendor that I don’t know if I can return to. … If I ask ChatGPT to buy me something and I can choose Amazon versus some other thing… 10 out of 10 times I click Amazon. Pay a higher price. … And it’s nothing to do with my experience of clicking around amazon.com. … And now, I think you could say we have not done a good enough job offering partners a platform inside of ChatGPT if they don’t want this to happen. … I think they’re in a great place if commerce moves much more agentic.

John Collison

我现在的想法是：任何需要你作为人类去填的网页表单，如今都低于你的薪资级别，因为你几乎从不是第一次创造这些信息。

My new thinking, in particular with regards to agentic commerce, but also anything is anytime there is a web form field, you as a human are filling out. These days, that is below your pay grade. Because, you are probably not inventing that information for the first time.

Sam Altman

如果能再也不在电脑前做「假工作」，我会非常开心。上一次我们真正见证这种事是 App Store：大多数 YC 批次都说「我要做 iPhone 应用，这是新分发渠道」，很多新公司诞生；大公司觉得不得不做一个，却又不太想，因为「我要把和消费者的关系交给 Apple」。结果那对移动端非常好。

I would be so happy to never do fake work at a computer again. … The last example of this that I think we really witnessed was the app stores. The iPhone App Store, specifically. … Most of the YC batch was like: “I’m gonna build an iPhone app. This is like new distribution. This is awesome. I can do a new thing I couldn’t do before.” And many new startups got created. And all the big companies felt like they had to make one, but they kind of didn’t want to. Because they’re like: “Oh, I’m going to hand my relationship with my consumer over to Apple.” And, it was great for mobile.

Sam Altman

也许我们真正该在乎模型擅长两件事：一是科学发现、做人做不到的事；二是好相处、合作愉快。我不认为我们有衡量「好相处」的好基准，我甚至不确定我们有衡量「最聪明科学家」的好基准。我们努力不刷榜。我一直以为刷榜现象已经结束了，可大家还在刷。

Maybe there are two things we should really care about models being good at. One is, scientific discovery and doing things that people cannot do. And the other is being enjoyable to work with. And, I don’t think we have great benchmarks for being enjoyable to work with. I’m actually not sure we have great benchmarks for the smartest, scientific scientist either. But, we endeavor not to benchmaxx. I keep thinking that benchmaxxing is over as a phenomenon, and people keep doing it.

我相信我们现在在这点上比别的实验室做得好。我们历史上并不总是更好，但我相信现在是。最新的模型比我们很久以来的都好得多。Emmett Shear 多年前对我说，他想要的是一个「有思想、有挑战性但仍然尊重人的资深同事」。至今我没听过比这更好的简短概括。

I believe we are better at that. … I don’t think we’ve always been better at that in our history. I believe we are better than that right now. … I think the latest models are much better than we’ve had in a long time. … Emmett Shear said to me years ago that what he wanted was a really thoughtful and challenging, but still respectful, senior coworker. And I have still not heard a better, brief encapsulation than that.

Sam Altman

（谈 Greg Brockman）我觉得 Greg Brockman 是独一无二的，这是最好的意义上的。他是强度、惊人的产出、极致的投入，以及专注于让使命成功的结合体。

I think Greg Brockman is one of a kind in the best sense. … It’s a combination of intensity, monster productivity, extreme dedication, and focus on succeeding at the mission. It’s very cool.

Sam Altman

我们差点做出一个——事后看会很糟的——决定：下一代模型完全不按 token 定价。你给一个提示，我们给你一个价格，然后去做，像 AWS 那样，做完给你账单。但客户的反应非常负面，他们说「我得知道自己在为什么付钱」。我问过一些人：「你愿意为同样的输出付更多钱，但以一致的 token 数计价吗？」他们说宁可我们把 token 数「注水」、多付钱，因为他们知道那里有个真实的东西。人们不想只被报一个价，因为他们觉得长期会被坑。

We almost made a very—what would’ve been bad call to say we’re not going to price our next model in tokens at all. And you’re just going to give it a prompt. And we’re going to give you a price back. And we’re going to do it. And it’s going to be like AWS to you. Just do something and we send you a bill. But the customer response on that is so negative. They’re like, “I need to know what I’m paying for somehow.” And, I asked a few people: “Would you rather pay more for the same output, but have it priced in some number of consistent tokens? Or something?” Even if we inflated the tokens and they’re like, “Oh, I’d much rather you inflate the tokens and pay more. Because I know there’s a real thing there.” So I think you are right, people don’t just want us to give them a price, because they feel like over the long term they’re gonna get screwed somehow.

我不想编造一个虚假的单位。token 某种程度上就是虚假的单位——在某种意义上是真的，但不是好单位。现在你得考虑是要小模型每百万 token 10 美分的 token，还是大模型的贵 token，还要估计链式思维可能多长。那只是认知负担，我不认为它增加了任何价值。

I would rather not make up a fake unit. Tokens are a kind of fake unit. I mean, they’re real in some sense, but they’re not a good unit. … Yeah. I mean, right now you have to think about whether you should want the 10-cent-per-million tokens out of a small model or $10 per token tokens out of a big model. And then whether… How long the chain of thought is likely to be. And that’s just cognitive overhead that I don’t think actually adds any value.