Stefano Ermon：扩散将赢下 AI 推理（Inception / Mercury）

Sarah Guo

听众大家好，欢迎回到 No Priors。今天请来的是 Stefano Ermon——长期在 Stanford 任教，现为 Inception 联合创始人兼 CEO。

Hi listeners, welcome back to No Priors. Today I’m here with Stefano Ermon, who is a longtime Stanford professor and now co-founder and CEO of Inception.

Sarah Guo

Stefano 在生成建模上积累极广，尤其以扩散模型的奠基者之一为人熟知。我们会谈他的公司如何挑战大实验室，以及为什么速度与效率会在未来几年成为 AI 的主旋律。欢迎 Stefano。

Stefano has an extraordinarily broad body of work around generative modeling, but is especially well known as one of the fathers of diffusion. We talk about his company challenging the large labs and why speed and efficiency are going to be the name of the game in AI over the next few years. Welcome, Stefano.

Sarah Guo

Stefano，非常感谢你来。

Stefano, thanks so much for being here.

Stefano Ermon

很高兴来。

Great to be here.

Sarah Guo

我想先从你的研究背景说起，以及你是怎么走到创办公司这一步的。

I would love for us to just start with a little bit of your research background and how you ended up starting your company.

Stefano Ermon

好。我几乎整个职业生涯都在做生成模型研究。2014 年我以助理教授身份加入 Stanford，当时就在建生成模型。那会儿这个方向并不热门，模型也还不太好用。

For sure. Yeah. I’ve been doing research in generative models for basically my entire career. I started at Stanford in 2014 as an assistant professor and I was working on building generative models. Back then the research area was not particularly hot. The models were not quite working well.

Stefano Ermon

我们还在 MNIST 上做小生成模型——能生成模糊的数字图像就算很大成功。当时甚至很难发论文，你得把训练生成模型说成是从无标注数据里学特征、以便提升监督学习——因为那才是大家真正关心的事。

We were still building little generative models over MNIST and it was a big success if you could generate these grainy images of digits. It was even hard to publish papers back then on that topic, and you had to justify training a generative model as a way to learn features from unlabeled data that then could maybe help you do better at supervised learning, because that was the thing that everybody cared about.

Stefano Ermon

后来局势当然翻过来了。我算是在对的时间、对的地方做了对的事，从那以后就一直在这个空间做研究。

But then things took over, of course. I was at the right place at the right time working on the right thing, and I’ve been doing research in that space since the beginning, basically.

Sarah Guo

除了好奇心之外，2014、2015 年你个人有没有希望这些模型最终能做成什么？

Did you have, besides a curiosity in the area, a personal hope for what the models would do back in 2014 and 2015?

Stefano Ermon

我一直觉得那是从无标注数据学习的正确方式——建生成模型，才能真正理解数据里的结构。我甚至没敢梦想今天这些 LLM 的能力。

Yeah. I always felt like that was the right way to think about learning from unlabeled data—that building a generative model is really the right way to make sure you understand the structure in the data. I was not even dreaming about the kind of capabilities that these LLMs that we have today could do.

Stefano Ermon

我更多是从世界模型角度想：我当时大量做图像，想象如果我站起来走出门会发生什么——脑子里能「看见」那一幕，这对决策和模型预测控制很重要，而这需要生成能力。我感觉方向对、问题很难、够忙一整辈子——结果我错得很彻底，进展比预想快得多。

I was thinking more from a world-models perspective. I was working a lot on images, thinking: I have a world model, I can imagine what’s going to happen if I were to stand up and walk out the door—I can picture that in my mind—and that’s important to make decisions and for model predictive control. Having this kind of model of the world requires some generative capabilities. I always felt like that’s the right direction, that this is going to be very hard and keep me busy for my whole career. And then of course I was very wrong and things evolved much faster than I was expecting.

Sarah Guo

对，我觉得这几乎是普遍经验。

Yeah, I think that’s kind of universally true though.

Sarah Guo

能不能讲讲你当时研究的状态，以及它如何导向创办公司？

And sort of walk me through the state of your research and how that led you to start the company.

Stefano Ermon

我一开始做图像生成模型，先是自回归——又慢又糊——然后是 VAE，再然后 GAN 接管了。

Yeah. So I was working on generative models of images initially, working on autoregressive models which were very slow and kind of very blurry, and then VAEs, and then GANs took over.

Stefano Ermon

我们当时对图像生成的状态很不满意：GAN 能出结果，但训练极不稳定、难复现。我们想：有没有办法做到和 GAN 一样好、但更有原则？于是开始做 score-based 生成模型——基本就是后来扩散模型的前身——2019 年和我的博士生一起。

And back then we were very unhappy with the state of generative models for images. The GANs worked but they were very unstable to train, very hard to reproduce results. So we were trying to see: is there a way to build something that is as good as a GAN but more principled? And so we started working on score-based generative models, which are basically what eventually became diffusion models, back in 2019 with my PhD student.

Stefano Ermon

想法是：训练一个神经网络去给图像去噪；如果你能去噪，就说明你足够理解图像结构，就可以据此建一套生成流程。这成了现代扩散的底层技术——不是从左到右一个像素一个像素生成，而是从纯噪声出发，逐步细化，直到得到干净画面。

We came up with this idea of: let’s train a neural network to denoise images. And if you can denoise an image, then you really are understanding enough about the structure of the image that it should be possible to build a generative procedure based on these denoisers. That basically became the underlying technology of modern diffusion models, where instead of generating images left to right one pixel at a time, you start from pure noise and then gradually refine the object until you get a clean picture at the end.

Stefano Ermon

那是 2019 年在我实验室起步的，然后席卷了这个领域。直到今天，最好的图像、视频、一定程度上的音乐，以及很多蛋白质相关工作，都基于扩散。我组里做了大量各类扩散技术，包括加速采样、提升质量。

That started back in 2019 in my lab and then it took over the space. And even today the best models for image generation, video generation, music to some extent, a lot of the protein stuff—they are based on diffusion. And so my group has worked a lot on various kinds of diffusion models, techniques for accelerating them to generate samples very quickly, to improve the quality of these models.

Stefano Ermon

既然图像上跑通了，我就开始想：怎么让扩散在文本或代码生成上工作？对离散对象，有没有办法超越自回归，做得更并行、自带纠错？我在 Stanford 做了大量把扩散用于文本与代码的研究。

So since we were able to get them to work on images, I started thinking about how do we get diffusion models to work on text or code generation—and for discrete objects, is there a way to move beyond autoregressive models to something that is more parallel, with built-in error correction? And so I’ve been doing a bunch of research at Stanford on getting diffusion models to work on text and code generation.

Stefano Ermon

2024 年我们有一次突破：发表论文，首次在 GPT-2 量级上证明可以追上自回归模型的质量——参数仍不到十亿，偏学术——但我们用同样数据把 Transformer 训成扩散模型，困惑度能对齐，同样参数下拟合数据一样好；而扩散因为同时输出多个 token，文本生成大约快 10 倍。这非常令人兴奋，我想看规模化后会发生什么，于是创办 Inception，把技术放大，去做商用规模的扩散语言模型。

We had a breakthrough in 2024. We published a paper basically showing that for the first time it was possible to match the quality of an autoregressive model at the GPT-2 scale—so less than a billion parameters, still fairly academic—but we were able to train basically still a transformer model as a diffusion model on the same data. We were able to match the quality, the same perplexity, fitting the data just as well as an autoregressive model with the same number of parameters. But the diffusion model was significantly faster because you’re outputting many tokens at the same time. We were able to generate text like 10× faster compared to the autoregressive model. And so that felt very exciting. I really wanted to see what happens if you scale up, if you train bigger models. And so I started Inception, a company to basically scale up the technology and try to build commercial-scale diffusion-based language models.

Sarah Guo

大家现在都见过扩散在图像上的输出；我会说短视频生成扩散正在世界其他地方成为主流娱乐形态，这里大概也会。哪怕我跟了这个领域十多年，今天的质量仍然难以置信。图像与视频用例如此之大，甚至有人在做专门硬件。但直觉上它为什么也会在其他领域管用，或者为什么这是一条能跟当前全 Transformer、冲 AGI 的大实验室竞争的有趣路线？你能给一点直觉吗？

Everyone has now seen the outputs of diffusion models, in particular images. And I would argue that it’s increasingly a dominant form of generated short-form video—from diffusion models—as entertainment in other parts of the world, and it will likely become so here. It’s kind of unbelievable, at least to me, even having followed the field for the last decade-plus, the quality that is possible today. So that is kind of obvious, and it’s such a big use case in images and video generation that folks are even creating hardware to support better performance here. It’s not intuitive that that would work for other fields, or that this is an interesting competitive direction to the current full-transformer-focused AGI labs. Can you offer some intuition on that?

Stefano Ermon

从研究者角度看，现在世界状态很有意思：生成模型大体两种范式。一种是自回归——预测下一个 token 或像素，从左到右一次一个。另一种是扩散——粗到细、迭代去噪式的生成。

Yeah, so it’s a very interesting state of the world right now from a researcher perspective, because there are two main paradigms, two ways of building generative models. There’s autoregressive, where you have a model that predicts the next token or the next pixel and then you generate left to right one token at a time. And then there’s diffusion, which is a coarse-to-fine generation, kind of iterative denoising kind of generation.

Stefano Ermon

连续模态上，扩散主导；离散模态——文本和代码——大实验室基本都押同一套自回归架构。当我们走向更多多模态、幻想一个能处理所有模态、了解世界的模型时，它会是自回归还是扩散？没人知道，我认为还在悬而未决。Inception 押扩散，因为我们相信最终关键是推理时扩展，而扩散在推理时有根本性优势。

And as you said, we have continuous modalities where diffusion dominates. There are discrete modalities—text and code—where primarily all the big labs are betting on the same architecture: autoregressive models. And as we move toward more and more multimodal models, and this idea that maybe we’ll have a model that can handle all modalities and know everything about the world—what architecture will that be? Will it be an autoregressive model? Will it be a diffusion model? Nobody knows. I think the jury is still out. At Inception we’re betting on diffusion models because we believe that what matters eventually will be inference-time scaling, and there are fundamental reasons why diffusion models are better than autoregressive models at inference time.

Stefano Ermon

想想自回归自己的故事：2017 年从 RNN 切到 Transformer 是拐点。问题在于 RNN 必须按序一个一个处理 token，训练很慢；于是有了能并行处理多 token 的架构——Transformer——它在训练上更可扩展，支撑了 LLM 的许多成功。

Even if you think about the story of autoregressive models, there was an inflection point in 2017 when people switched from RNNs to transformers. Why was that? The problem was that RNNs had to essentially process tokens sequentially, one at a time, and training was very slow. So people came up with this idea of an architecture that allows you to process many tokens at the same time in parallel. That was a transformer, and that was the thing that scaled better for training and enabled a lot of the successes behind LLMs.

Stefano Ermon

但若谈推理、谈生成，自回归仍然是顺序的——从左到右一次一个 token。你生成不了第 10 个，除非前面都生成完了。这种负载与 GPU 映射很差，极度 memory-bound：大量时间在搬权重，算术很少。这是自回归的根本问题。

But if you think about inference now—not training, inference, generation—autoregressive models are still sequential. The computation is left to right, one token at a time. You cannot generate the 10th token until you’ve generated everything that comes before it. That kind of workload does not map well to GPUs. That kind of workload is extremely memory-bound. You’re spending most of your time moving around weights across the memory hierarchy and you’re doing very little arithmetic. And that’s a fundamental problem of autoregressive models.

Stefano Ermon

若把 RNN→Transformer 类比到推理侧，等价物就是基于扩散的模型：它在推理时天生要同时处理许多 token。扩散推理时的负载，本质上非常像训练时——并行处理多 token。

And so what’s the equivalent—if you think RNNs to transformers for autoregressive models—the equivalent at inference time is a diffusion-based model, because a diffusion model is built to have at inference time a workload where you process many tokens at the same time. And so the workload that we have at inference time in a diffusion model is basically very similar to the workload you have for training, where you’re processing many tokens at the same time in parallel.

Stefano Ermon

因此它天生让推理负载更贴合 GPU 擅长的事。我们押的是：建出在推理时扩展最好的架构与模型——因为经济账最终由每瓦特、每美元能拿到多少智能主导。

And so it’s built to essentially have an inference workload that maps really well to the kind of things GPUs do really well. And so we bet on trying to build the architecture and trying to build the kind of models that will scale best at inference time, because economics are dominated by the kind of intelligence per watt, the intelligence per dollar that you’re able to get from the models.

Stefano Ermon

推理模型的很多进展，其实是在扩展测试时算力；沿那条轴扩展得更好也会重要。RL 后训练里，瓶颈很多是生成 rollout——让模型探索、打分轨迹、再据此改进——推理再次成为关键瓶颈。若模型在推理时扩展更好，后训练扩展也会更好。所以我们押扩散 LLM：它本质上更并行；苦涩的教训是，更并行的方案最终会赢。

If you think about a lot of the advances with reasoning models, a lot of it is scaling test-time compute. Being able to scale better along that axis will also matter. And even if you think about RL post-training, a lot of the bottleneck is generating rollouts—letting the model explore, scoring the trajectories, and then improving the model based on the kind of things it finds. So inference is again a key bottleneck for RL post-training. And so if you have a model that scales better at inference time, then automatically you’re going to get better scaling during RL post-training. And so that’s why we decided to bet on a diffusion-based LLM—because it’s inherently more parallel, and the bitter lesson is that the more parallel solution is the one that is eventually going to win.

Sarah Guo

在离散与连续模态上「啃硬骨头」时，你们怎么想适用性？做过哪些实验？大家也在用新 tokenization 等方法让视频、语音适配现有主导范式；你们并不在同一套「纯 token 导向」范式里，这里怎么跑通？

How did you think about applicability, or what experiments did you run in terms of cracking the nut on discrete versus continuous modalities? Because I think people have also shaped the existing dominant paradigm through new tokenization efforts or methods to make video and voice work, for example. You’re not in the same token-oriented paradigm. How do you make it work here?

Stefano Ermon

要把一项天生紧绑连续数据结构的技术迁过去，做了大量研究。扩散是在学如何给图像去噪，对连续数据说得通——两个像素颜色之间可以插值，仍然有意义。但两个词之间未必有「中间物」，全是离散的。要把那些想法扩展到离散空间，需要很多 R&D，以及一门新的科学。

Yeah. So there was a lot of research that went into figuring out how to apply a technology that was inherently very tied to continuous structure in the data. If you think about a diffusion model, it’s learning how to denoise images, and it kind of makes sense for continuous data, because even two pixel colors—you can interpolate between them and it will still make sense. But if you think about two words, there is not necessarily something in between them. It’s all discrete. And so it required a lot of R&D and a new science that had to be developed to figure out how to extend those kinds of ideas to discrete spaces.

Sarah Guo

今天你能声称它跑得有多好？

What can you claim about how well it works today?

Stefano Ermon

我们觉得非常好。我们已经训出质量可比前沿实验室「速度优化档」模型的扩散 LLM。我们的 Mercury 在基准上与 Haiku、Flash、以及 OpenAI 的 mini / nano 等对标，同时显著更快。

We think it works really well. So we’ve been able to train diffusion-based LLMs that are comparable in quality with the speed-optimized models from frontier labs. So our Mercury models are on par with the Haiku models, Flash models, mini / nano models from OpenAI if you look at benchmarks, while being significantly faster.

Stefano Ermon

我们已经从纯研究原型走到真正被使用：今天在生产中服务这些模型。我们还解决了怎么建 serving 引擎——你没法把这些扩散 LLM 直接跑在 vLLM 或 SGLang 上，必须自建。我们能扛住真实生产负载的复杂性，端到端把这种新体验交给真实客户。

We’ve crossed from pure research prototypes to things that are actually used. We are serving these models in production today. We did all the work of figuring out how to even just build a serving engine—you cannot run these diffusion-based LLMs on vLLM or SGLang; we had to build our own serving engine. And we can handle a lot of the complexity of real production workloads. We’ve solved all these challenges and we can deliver this kind of new experience end to end to real customers today.

Sarah Guo

正好聊聊 Inception 作为公司现在到哪了：多少人、实际在服务什么、研究状态如何？

Actually a great time to just talk about where Inception is as a company—like how many people, what are you guys actually serving, sort of state of research.

Stefano Ermon

我们大约两岁，大概 50 人，大量时间仍在 R&D：怎么正确训练这些模型、怎么加速推理。对自回归来说，一次一个 token，能做的不多；对扩散，推理时在算力与质量之间有很多可交易的旋钮——就像图像/视频扩散里的蒸馏、各种微分方程求解技巧，能采得非常快。所以训练、推理、工程上都有大量研究，还有数据配比、演化、RL 后训练基础设施——要为这套新模型摸出可用配方。

Yeah, so we are about two years old, around 50 people, spending a lot of time still on R&D—figuring out what’s the right way to train these models, how to accelerate inference. If you think about an autoregressive model, it’s pretty clear there’s not a lot of things you can do there: you generate one token at a time and that’s it. In a diffusion-based model, we know there are a lot of different possibilities for trading compute for quality at inference time. Even if you think about image or video diffusion models, there’s a lot of techniques you can use to accelerate sampling—distillation or fancy differential equation solving techniques that allow you to sample very quickly from these models. And so there is a lot of research on the training, on the inference, and then engineering—thinking about data mixes, evals, RL post-training infrastructure. There is a lot of work that needs to happen to figure out how to build recipes that work for this new model.

Stefano Ermon

我们尽量复用已有东西：它仍是 Transformer 底座，不必丢掉大量架构工作；仍用 attention；仍用很多公开数据集、评测与基准。我们是创业公司，尽量精干，把力气花在能产生最高影响、也最差异化的地方——眼下是速度；未来难说，扩散语言模型也许会比自回归显著更「聪明」，没人知道。正因为如此才兴奋：我们在造很强的 AI 系统，但一切都很新。我不信我们已经发现了造这些系统的最佳方式——一定还有别的路。最终效率会非常重要：想想「AI 工厂」会怎么运转，其实没人真正清楚；能在这个空间里玩、想创造智能的替代路径，本身就很兴奋。

We try to leverage existing things as much as possible. For example, it’s still a transformer-based model, so you don’t have to throw away a lot of the work that has been done on good architectures. We still use attention. We still use a lot of the public datasets that people have created, and evals and benchmarks. We’re a startup—we try to be scrappy, use existing things as much as possible, and focus on the aspects where we can have the highest impact and where we can be the most differentiated. And right now it’s speed; in the future, who knows—it’s possible that a diffusion-based language model will be maybe significantly more intelligent than an autoregressive one. Nobody knows. That’s why I think this is very exciting, because we’re developing these really powerful AI systems but it’s all very fresh, all very new. I doubt we’ve discovered the best way of building these systems. There’s got to be alternatives, other ways of building these models. And eventually, yeah, efficiency will be very important. If you think about the AI factory—how is that going to work?—I think nobody really knows. Just being able to play in that space and thinking about alternative ways of creating intelligence—I think it’s exciting.

Sarah Guo

完全同意。而且在一个越来越被算力与供给约束的环境里，工业研究里对效率的关注过去往往是二阶的，能力扩展才是一阶。若你相信我们能把所有电力都用上、且经济上有理由这么做，那对效率的聚焦会急剧上升。很多人现在会说：AI 公司最重要的决策之一，就是手头的电力在训练与最有价值用例之间怎么分。

Absolutely. And I also think that in an increasingly fundamentally compute- and supply-constrained environment, I wouldn’t say there wasn’t very focused research on efficiency of models, but it was a second-order consideration for many of the industrial research efforts versus pure capability scaling. But if you fundamentally believe that we can use all of the power we have and there will be economic reasons to do so, then I think the focus on efficiency is going to go dramatically up. I think many people now argue one of the most important decisions for a company in the AI space is: how do we use the power we have between training and the most valuable use cases for any watt.

Sarah Guo

说到这个——这里是 2026 年 9 月——速度已经在哪里赢了？人们已经足够在乎吗？

On that topic—here in September of ‘26—where does speed win? Where do people care enough about this already?

Stefano Ermon

基本上是延迟很重要的应用。广义上每个人都在乎速度：同质量下更快，人总会选更快的。我们也看到前沿实验室也在出更快版本——人们愿意为更快模型付更多钱。一旦习惯了快模型，就很难回去，有点像宽带：越来越快，试过就回不去。

Yeah, it is basically applications where latency matters. Generally speaking I think everyone cares about speed in the sense that if you can give me the same quality but faster, people will always pick the faster solution. And then we’re seeing it with these faster versions of even the models from frontier labs. People are willing to pay more to get access to faster models. And once you get used to a fast model, it’s hard to go back. It’s kind of like broadband—get faster and faster—and if you were able to try, people cannot go back once you get the fast model.

Sarah Guo

有没有可以公开谈的客户，他们今天就很在乎这个？

Are there customers that you can talk about publicly that care about this today?

Stefano Ermon

有几个可以提。比如语音领域的 OpenCall：他们在做语音 agent。管线是 ASR → 一个要做工具调用、决定下一句说什么的 LLM（通常要推理型 LLM 才能到最高质量）→ 末端 TTS。速度极其关键。

There are a few that we can mention. Like in the voice space, for example, OpenCall is an example. They’re building voice agents. The pipeline is: you have an ASR model, you have an LLM that is doing all the tool calls and figuring out what to say next—it has to be a reasoning LLM typically to have the highest level of quality—and then there is a text-to-speech component at the end. Speed matters a lot.

Stefano Ermon

他们之前在 Cerebras 上服务 LLM，用定制芯片拿到所需速度、交付最佳体验；后来切到我们的扩散 LLM——因为用为并行而生、在软件层加速的扩散 LLM，你可以在 NVIDIA GPU 上达到接近「自回归跑在定制硬件上」的速度。GPU 虽也稀缺，但比定制芯片多得多，成本更低、质量也更高。这就是语音场景的一个例子。

They were previously serving their LLMs on Cerebras. So they were using custom chips to get to the kind of speed that they need to deliver the best experience to their customers. And then they switched over to our diffusion-based LLMs because they can essentially get the same speed as what you would get if you were to run an autoregressive model on custom hardware. If you have a diffusion-based LLM that is built to be parallel—accelerating at the software level—then you can get the same speed on NVIDIA GPUs, which means much more availability. GPUs are scarce, but there’s more of that than custom chips, and lower cost, higher quality. So that’s an example in the voice space.

Sarah Guo

我本来也想问：这怎么与硬件格局互动？我们已经看到足够多用例愿意为带大量 SRAM 的大贵芯片付钱——编码等场景也是。

Yeah, I was actually going to ask you how you think this interacts with the hardware landscape as well, given we’ve seen enough now demand from use cases who are like: yes, I want a big expensive chip with a lot of SRAM and people will pay for the outputs of that—in coding and other use cases.

Stefano Ermon

当然。硬件是加速的一条路。

Yeah, for sure. For sure. And I think hardware is one way to accelerate things.

Sarah Guo

而如果能用现有硬件，软件也许更好。

And software might be better if we can use the existing hardware.

Stefano Ermon

正是。尤其二者互补——这才是兴奋点：软件侧的收益，在某种程度上与硬件侧收益是相乘的。

Exactly. Exactly. And especially they are complementary. That’s the exciting piece—that to some extent the gains that you get from the software are multiplicative with the gains you get from the hardware.

Sarah Guo

也许有一天人们会做更贴合你们这类模型的硬件。

And maybe someday people will develop hardware that fits even better the models that you’re building.

Stefano Ermon

肯定会。

For sure. For sure. Yeah.

Sarah Guo

放大看 Inception 在行业里的位置——有一种担心（纠正我如果我错了）：今天很难投资新架构，因为一旦有架构或方法上的进展，就会被有算力资源的大玩家吸走。作为「大卫」，你们怎么想竞争？

If we just zoom out to Inception in the broader industry—I think there is a vein of concern, correct me if I’m wrong, that it’s very hard to invest in new architectures today because if there are advances in architecture or methods, it will simply be absorbed by large players with the resources to scale compute. Talk to me about how you think about going and competing as David in this situation.

Stefano Ermon

这是很正当的点，我们也时时放在心里。初期我们的护城河肯定是 IP、商业秘密、我们和研究者把模型做出来并做强的想法。

Yeah, it’s a very valid point and something that is also top of mind for us. I think initially for us, for sure, the moat is sort of the IP, the trade secrets, the ideas that we and our researchers have to build these models and make them better.

Stefano Ermon

随着公司成熟——也是我们不只做纯研究、还做出产品、有真实客户、从真实世界拿模型反馈的原因——我们在开发部署这些模型同样关键的组件，比如 serving 引擎。没有 serving，就没法在生产里真正服务。逼自己从一开始就端到端部署，我们学到大量如何服务、如何建运行这些模型所需的软件——那又成为 IP：就算你训出了扩散 LLM，没有 serving、没有 vLLM 等价物，你仍然卡住、用不了。

As we mature as a company—and one of the reasons we are not just doing pure research but we’ve also developed a product and we have real customers and we are getting feedback on the models from the real world—is that by doing that we are also developing components that are also very important to deploy these models, for example a serving engine. If you don’t have the serving engine, you can’t really serve these models in production. And so by forcing ourselves from the very beginning to go out and deploy something end to end, we’re learning a lot about how to serve these models and how to build software that is needed to run these models. And that again becomes IP. Even if you train the diffusion-based LLM, if you don’t have the serving engine, if you don’t have the vLLM equivalent to serve it, you’re still stuck and you still cannot use it.

Stefano Ermon

同样地，我们跟真实客户合作、拿反馈、搞清什么管用什么不管用，有时从他们那里收集数据、按他们看到的现象做评测——这又进入技术护城河，因为那些东西更难复制。

Along the same lines, we are working with real customers and we’re getting feedback on the models. We figure out what works, what doesn’t. We collect data sometimes from them. We create evals based on what they’re seeing. And so that again becomes part of the technical moat, because those things are a little bit harder to replicate.

Sarah Guo

如果这个问题在技术上不通，你可以告诉我。扩散在图像/视频里结构上受益于：你在复现一个世界里应有一致结构的东西；语音也类似。但 AI 迄今最有价值的一些领域——比如训练用的代码数据——非常乱，有人会说里面并没有你要找的「正确真实结构」。面对人类生成输入 vs 图像、视频、语音，你怎么想？

You can tell me if this doesn’t make sense as a question to ask technically, but one of the things that diffusion models benefit from structurally in images or video generation is: you’re replicating something where there should be some consistent structure in the world. Voice as well—whatever is really possible and most likely. There are some of the fields where AI has been most valuable to date—I’d say a lot of the input data you use to train, like code data for example—it’s very messy, and one could argue that a lot of it doesn’t actually have the correct real structure you’re looking for. How do you think about that when it’s human-generated input data versus images, video, voice?

Stefano Ermon

好问题。从根本上，无论训自回归还是某种程度上的扩散，训练生成模型时你在做的是：通过建压缩方案来识别数据中的结构。这不一定直观，但训这些模型时，你实际上是在找高效压缩数据的方式，从而识别共同结构。压缩得越多，识别的结构与模式就越多——模型就是这么工作的。惊人之处在于：光靠预测下一个词，你就在学习数据结构；扩散与自回归都一样，两者本质上都在学压缩方案。

Yeah, it’s a good question. And fundamentally, whenever you train a generative model—whether it’s an autoregressive model or to some extent even a diffusion model—you are trying to identify structure in the data by essentially building a compression scheme. That might not be obvious, but whenever you train these models you’re effectively trying to identify common structure by trying to find an efficient way of compressing the data. And so the more you can compress the data, the more structure, the more patterns you’re identifying. And that’s how these models work. The amazing thing is: just by predicting the next word, you are learning something about the structure of the data. And that’s the same whether you’re using a diffusion model or an autoregressive model. Both methods are essentially trying to learn a compression scheme.

Stefano Ermon

我提到 2024 年那篇原始论文、我们展示与自回归达到平价时，用的指标基本是困惑度——它就是一种「你在数据里识别了多少结构」的度量。所以即便不直观，我们在 GPT-2 量级上确实识别出了与自回归同等量的结构。

And when I mentioned the original 2024 paper, when we showed that we are achieving parity with autoregressive models, the metric that we’re using is basically perplexity, which is a notion of how much structure have you identified in the data. And so even though it might not seem obvious, we were actually able to identify at the GPT-2 scale the same amount of structure as an autoregressive model.

Sarah Guo

对，经验结果在那里；但直觉会是：代码数据集并不扎根于物理，噪声很多——听起来你认为这是可管理的问题。

Yes. I think that empirical result is there, but the intuition would be: well, code—the dataset you are working on—is not grounded in physics. There’s a lot of noise in there. And it sounds like you believe that’s a manageable problem.

Stefano Ermon

什么东西都有噪声。数字不会说谎：只要你能把困惑度压下去，就说明你能建出压缩方案、拿到那种压缩水平。结构一定在，模型一定挖到了它。然后更多是归纳偏置问题：Transformer 是否是识别这些模式的更好方式，还是别的；下一词预测是否是正确建模框架，还是更像去噪。这在很大程度上是经验问题——此刻我们甚至还没有足够工具去彻底理解。

Yeah, there is noise in everything. And so to the extent—the numbers don’t lie—to the extent that you’re able to drive the perplexity down, then it means that you can actually build a compression scheme and that will get you that sort of level of compression. And so the structure must be there and the model must have been able to uncover it. And then it’s more a question of an inductive bias: is a transformer a better way of identifying those patterns or something else? Is next-token prediction the right modeling framework, or is it more like denoising? And that’s very much an empirical question that I think at the moment we don’t even have the tools to understand.

Sarah Guo

因为你用了语音客户作例子——有些做 AI 产品的人认为，语音管线中间放 LLM 的好处之一，是他们更懂怎么做对齐或可控性。我猜这对扩散模型会不一样。能谈谈吗？

Can I ask a question just because you used a voice customer as the example here—one of the benefits that some people building these AI products have identified of having an LLM in the middle of this voice pipeline is they understand how to do alignment a little bit better there, or controllability. I imagine that has to look different for a diffusion-based model. Can you talk about that?

Stefano Ermon

是的，那是关键价值主张之一；他们总会看：在多大程度上，他们提供的价值是 harness，确保模型确实在做对的事。有趣的是，我们把扩散 LLM 做成向后兼容：API 一样，仍是 OpenAI 兼容、文本进文本出。而我们训出的模型擅长跟指令，也擅长输出 JSON 这类结构化输出，这些都能处理。

Yes, that’s a key value proposition, and one of the things that they always look into is: to what extent a lot of the value they provide is the harness and making sure that the models indeed are doing the right thing. And the interesting thing about a diffusion-based LLM is that we’ve built everything to be backwards compatible. So the API is the same. It’s still OpenAI compatible, text in, text out. And it so happens that the models we’ve trained are good at following instructions. They’re good at outputting—if you’re using JSONs—structured outputs. They can handle all of those things.

Stefano Ermon

而且足够好——事实上比他们之前用的模型更好——所以他们仍能用 Mercury 给客户提供同等水平的服务。

And it was good enough. It was better in fact than the models they were using before. And so they are still able to provide the kind of level of service to their customers by using Mercury.

Sarah Guo

如果接口一样、可以沿用同一套栈，就非常简单。

Well, very simple if the interfaces are the same and you can just use your same stack.

Stefano Ermon

对。但还有一点很有意思：我们知道扩散通常比自回归更容易控制。原因是：自回归你得等到生成完整对象，才知道它是否满足约束、是否对齐、是否符合你在乎的目标函数——比如生成分子、在乎溶解度——你必须等完整对象才能用奖励函数打分。

Yeah. But it could be that—I think that’s actually a very interesting point—that we know that diffusion models are typically easier to control compared to autoregressive models. And the reason is that if you think about an autoregressive model, you kind of have to wait until you’ve generated the full object to know whether or not it satisfies, let’s say, a constraint, or whether or not it’s aligned, or whatever the objective function that you care about. Maybe you’re generating a molecule and you care about solubility, and then you have to wait until you have the full object to be able to score it with some reward function.

Stefano Ermon

扩散更是粗到细生成：从一开始你就大致知道「这是不是我想要的那种对象」，可以用外部奖励或一组约束去引导生成方向。至少学术文献有大量证据表明扩散更容易控制，有些引导方式在自回归上根本不可能。那会是另一套接口——也许自回归根本提供不了。我们想很多的是：围绕自回归提供不了的新能力，正确的产品体验该是什么。

But a diffusion model is more coarse-to-fine generation. So from the very beginning you kind of know: is this object the kind of thing I want or not, and you can steer the generation in the direction provided by an external reward function or some set of constraints. And so there is a lot of evidence in the academic literature at least that diffusion models are easier to control, and there are different ways of steering them that are just not possible with autoregressive models. So that would be a different interface for the model that maybe might not even be available for autoregressive models. I think that’s something that we’ve been thinking a lot about: what’s the right product experience that we can build around new capabilities that are just not provided by autoregressive models.

Sarah Guo

除了性能，你想象 Inception 模型在规模上还会有今天模型没有的能力吗？

Are there capabilities that you imagine Inception’s models having at scale that today’s models don’t have beyond performance?

Stefano Ermon

那正是我们不知道的——它是涌现的。眼下的楔子是速度：我们知道它们快得多，那是最初的赌注，也更好测、好量，显然有价值。但让我兴奋的是：随着我们更懂怎么训这些模型，不知道会发现什么。

Yeah, that’s the thing we don’t know, right—and that’s emergent. Yes. Like right now the wedge is speed. We know they are much faster. That was the initial bet because that was easier to test. It’s also easy to measure and it’s obviously valuable. But as we—that’s why I find it so exciting—as we learn more about how to train these models, we don’t know what we’re gonna find.

Stefano Ermon

学术文献有相当证据表明，扩散模型比自回归更数据高效。直觉是：训扩散是在去噪——从一张图加噪再学去噪——等价于数据增强，同一张图被许多噪声视图增强。所以它们往往更省数据；若规模上仍然成立，在数据成为瓶颈的任务上就会更有意思。我们会看到。正因为技术如此重要、有差异化就会创造价值，才令人兴奋。

And there is a decent amount of evidence in the academic literature, for example, that diffusion-based models are more data-efficient compared to autoregressive models. And the intuition is: if you think about training a diffusion model, you’re learning by denoising. You start with an image, you add noise, and then you learn how to remove the noise. And so it’s effectively doing data augmentation in the sense that the same image is augmented by many noisy views. And so they tend to be a little bit more data-efficient. And so if that holds up at scale, then maybe we’ll get into tasks where we have less data—where data becomes the bottleneck—then it becomes more interesting. And so we’ll see. That’s why it’s so exciting, because things are changing. And this technology is so important and so valuable that having something differentiated I think will create value.

Sarah Guo

若往前看五年——在 AI 世界其实太长了——往前看两年，你觉得扩散与传统模型之间的工作负载会怎么切？

If we project out five years—that’s actually way too long in AI world—if we project out two years, what do you think is the workload split between diffusion and traditional models?

Stefano Ermon

我觉得我们还没到前沿智能水平，很多工作负载确实需要前沿智能。但按我的估算——OpenRouter 有很好的方式看各类用例：研究、对话、编码、软件工程、日志处理等，有一套不错的任务分类——我估大约 20% 到 30% 的场景延迟极其重要。至少作为下界，这些可以被「在给定延迟预算内给你最高可能质量」的模型覆盖。

I think we’re still not at the frontier level of intelligence, and I think a lot of the workloads do require frontier-level intelligence. But in my estimates—even if you just go to OpenRouter, which has this very nice way of looking at all the different use cases, and you can see research and conversational and coding and software engineering and log processing—they have a nice hard taxonomy basically of tasks—and I was doing some estimates and I think there is like between 20 and 30% where latency is really, really important. And so at the very least as a lower bound I think it could be addressable by models that, within a given latency budget, will give you the highest possible quality.

Sarah Guo

所有技术路径都有权衡。用扩散模型的挑战是什么？

And then—all technology approaches have trade-offs. What are the challenges of working with diffusion models?

Stefano Ermon

这是另一套栈，所以挑战之一是大量东西必须自建；serving 引擎、kernel 之类没有成熟生态。有一些开源模型，但不是特别好。这让部署、让客户试用更难——他们不习惯。那是挑战之一。

Yeah, it’s a different stack, and so one of the challenges is that we had to build a lot of things in-house, and there is not a mature ecosystem. If you think about the serving engine or kernels, a lot of the things had to be developed in-house. And so there is not really anything open source—or there are some open source models but they’re not particularly good—and so that makes it a little bit more difficult to deploy, to get customers to try things. They’re not used to it. So that’s been one of the challenges.

Sarah Guo

我猜这对外部也一样：在有一定成熟度、会在乎成本与性能的客户里——尽管某些用例一开始就在乎——后训练兴趣越来越大；在新架构上那就更难。

I imagine that also reflects externally, right? In a landscape where folks at some sophistication would care about cost and performance—and though for certain use cases you will care about cost and performance from the beginning—there’s an increasing amount of interest in post-training. And so I imagine in a new architecture that’s even more challenging.

Stefano Ermon

所以我们得自建做 SFT、做 RLHF、做 RL 的栈——这在某种程度上又变成 IP。我们决定不全开源，一个原因就是想把 IP 握得更近一点。但有缺点：社区贡献机会更少，更难采用，更难做本地部署。两种选择各有利弊。

So we had to build our own stack for doing SFT, for doing RLHF, doing RL. That becomes IP to some extent. So one of the reasons we decided not to open source everything was really to keep the IP a little bit closer to us and not open it. But then there are downsides: there is less opportunity for the community to contribute. It’s harder to adopt. It’s hard to do on-prem deployments. And so there are pros and cons with the two choices.

Sarah Guo

能谈谈你们自己训练的规模——当前或理想——吗？大约 50 人你们肯定还在招：为什么研究者或工程师该考虑这个方向、该来 Inception？

Can you talk about scale of your own training—current or aspirational—and then at 50 people I’m sure you’re continuing to hire: why researchers or engineers or others should consider investing in this direction or working at Inception?

Stefano Ermon

关于训练、模型大小或 FLOPS，我们不能透露太多，那是商业秘密。但我们在持续推前沿。如果你想有机会塑造这个领域的方向，Inception 是个好地方：这个子领域仍相对小，有大量东西待发明。很多人选择来 Inception 而不是别的实验室，是因为想要所有权、喜欢发明新东西、喜欢更 greener field、更少已知、更开放的空间。我们往往吸引这类人。

Yeah, so we’re not able to share much about the training, the size of the models or the FLOPS or all of that—it’s kind of a trade secret—but we are continuing to push the frontier. And Inception is a great place to be if you want to have an opportunity to shape the direction of the field. It’s still a relatively small field. There is a lot to be invented. And so a lot of the people that decide to come to Inception instead of joining one of the other labs really want to have ownership, and they like to invent new things. They like to be in a space where there’s more of a green field and more opportunities to try things. There’s less that is known or available out there. It’s a little bit more open-ended. And so we tend to attract those kinds of people.

Sarah Guo

研究圈朋友里，既兴奋又有点绝望的一点，是用模型做研究本身的递归自我改进。你们走的方向很不同，你怎么看？

I think one thing that is both exciting and causes some despair amongst research friends is the ability to use models for recursive self-improvement in the research field itself. Given you’re working on a very different direction, what is your view on this?

Stefano Ermon

这是我们在面对的事——

Yeah. I mean, it’s something that we—

Sarah Guo

听起来你明确仍觉得你和团队还有工作要做。

I mean, explicitly it sounds like you still feel there’s work for you and your team to do.

Stefano Ermon

哦对。我觉得我们还没到那一步。也许我们拿不到别人有的那些模型，但我们当然大量用模型，它加速了迭代与试想法的速度。我们用前沿实验室的模型，很棒，大大加速了开发。与此同时——至少现在，我不知道六个月或一年后会怎样——人类的巧思仍然超级重要：提出正确想法、修剪搜索空间、识别更有希望的方向，对我们一直至关重要。

Oh, yeah. Yeah. I think we’re not there yet. Maybe we don’t have access to the models that other folks have, but I feel like we use models a lot of course, and it has accelerated the speed at which we can iterate, try ideas. We use models from frontier labs and it’s been great—it has accelerated our development process a lot. At the same time, I think—at least right now; I don’t know what it’s going to be in six months or a year—but right now, human ingenuity is still super important, and the ability to come up with the right ideas and prune the space and identify directions that are more promising has been really important to us.

Sarah Guo

50 人对一个全栈——研究、serving、产品——的公司来说并不多。你们怎么组织、怎么分配资源？

50 people is not that many people for, let’s say, a full-stack research, serving, product company—or however you would think about describing it. How do you organize, and how do you think about how you allocate your resources here?

Stefano Ermon

团队小，但每个人都很强、很拼；我们也有 agent 让我们高效得多。所以我觉得人数足以做很多事；事实上我常觉得瓶颈更多是算力而不是人。

It’s a small team but everyone is very talented and they work very hard, and we have access to agents that are making us a lot more productive. And so I think the numbers are sufficient to do a lot. And in fact often I feel like the bottleneck is more compute than people.

Stefano Ermon

组织上：有一支产品团队，负责平台、帮客户用好我们的模型——本质上是服务当前最佳模型版本；另有一支在建下一版模型，覆盖训练、RL、推理——更偏研究。

But yeah, the team is organized like: there is a product team effectively that is handling the platform and working with customers to make them successful with our models—basically a team that is serving the current best version of the model—and then there is a team that is building the next version of the model, and that includes training, RL, inference—so that’s more research.

Sarah Guo

Stefano，最后一个问题。2024 那篇论文结果很有意思、也引起很大反响。你在这个领域做了很久。很多人会说，在这个可以大规模堆资源的时代，学术 AI 研究很受挑战——某种程度上确实，你也围绕它创办了商业公司。但在真正拿到那些 2024 结果之前，你怎么建立信心：自己在做的方向会有影响、有希望？

Stefano, one last question for you. The ‘24 paper was a super interesting result, made a big splash. You’ve been working in this field for a long time. A lot of folks would claim that academic AI research is very challenged in this era of being able to scale resources a great deal. This is certainly true to some degree given you started a commercial company around it as well. But how did you get confidence in the directions that you were working in having impact or being promising before you really had those ‘24 results?

Stefano Ermon

那是一堆结果的集合，是我实验室长期工作的积累。当然有早期扩散工作：在我们学术上能训的规模上，我们能打过 GAN，同时还稳定得多；然后整件事接管了世界，变成 Stable Diffusion、Midjourney——那一切都从学术、从我实验室发展出的想法起步。

Yeah. And I think it was a collection of results that I had been working on in my lab. Not necessarily—of course there’s the early diffusion work that we did in the lab. We showed that at the kind of scale of models that we could train academically, we were able to beat GANs, while being much more stable. And then the whole thing took over and became Stable Diffusion, Midjourney—sort of all of that started from ideas that were developed in academia in my lab.

Stefano Ermon

但不只这一件。我还联合指导做过 Flash Attention——又一个从学术出来、最终对工业界有巨大影响的东西。再比如 DPO：那是我组里一个轮转项目起步的算法，现在被用来对齐 LLM、扩散模型，到处都在用——完全在学术里发展出来，基于一个巧妙洞见、那个问题里有趣的数学结构，让你能用非常不同、更高效的方式做后训练与对齐。

But that’s not the only one. I was co-advisor—so I worked on Flash Attention, for example. That’s another thing that came out from academia that then eventually had a huge impact in industry. Or another example is DPO—that was another project that started out as a rotation project in my group. It’s an algorithm that is used to align LLMs and diffusion models and everywhere. And that’s again something that was developed entirely in academia, and it was just based on a clever insight—some interesting mathematical structure that you have in that problem that allows you to come up with a very different and more efficient way of post-training and aligning these models.

Stefano Ermon

所以有很多宝石，有很多机会找到解决重要问题的新的、更好的方式。学术的好处之一是允许你下这些逆势赌注。如你所说，挑战是资源可能不够——永远不够；有更多算力会更高效。但你能接触到了不起的学生，大家都想做新东西，不害怕下注——这就是为什么我认为学术如此有影响力；即便在 AI 领域，很多重要想法的根、甚至创造本身，都在学术里。

So there are gems—there are lots of opportunities for finding new and better ways of solving important problems. One of the nice things about academia is that it allows you to take these contrarian bets. As you said, there is the challenge that maybe we don’t have enough resources—there’s never enough resources—and if we had more compute we could be more efficient. But you have access to amazing students and everyone is trying to develop the new thing. People are not scared about taking bets. And that’s why academia has been so impactful, I think. Or even in the AI space, a lot of the important ideas have roots or even were created in academia.

Sarah Guo

太棒了，非常有启发。非常感谢你来，Stefano。

Awesome. Super inspirational. Thanks so much for being here, Stefano.

Stefano Ermon

非常感谢邀请我。

Thank you so much for having me.