递归语言模型与 harness：MIT Alex Zhang 谈 RLM、Prime Agent 与智能体蜂群

Alex Zhang

当你往一个问题上砸足够算力，就能充分探索解空间——或许烧掉一千亿甚至一万亿 token；但若请来真正懂这个问题的人，一次洞察可能直接抹掉那一万亿 token 的花费。

When you throw enough compute at a problem, you can sufficiently explore solutions—maybe burn a hundred billion or a trillion tokens. But if you bring in someone who knows something about the problem, they can uncover something for the model that would erase that one-trillion-token spend.

Alex Zhang

我们仍有太多想解决却负担不起「永远堆算力」的问题；效率维度依旧至关重要。

There are so many problems in the wild we want to solve, and we can’t afford to always throw as much compute as possible at them. There is still an efficiency aspect that is super important.

Swyx

录音棚请来 Alex Zhang——最出名的可能是 RLM，还有别的身份。欢迎。

We’re here in the studio with Alex Zhang—most famously of RLMs, but you have a few other affiliations. Welcome to the show.

Alex Zhang

谢谢邀请。

Thank you for having me.

Swyx

还有 GPU Mode？是 Mark Saroufim 领进来的——不是人人都有这种待遇。

GPU Mode as well? You were shepherded in by Mark Saroufim. Not everyone gets that kind of welcome.

Alex Zhang

对，和 GPU Mode 的人很熟；很多合作超出 Discord 本身。

Yes. I’m very close to everyone in GPU Mode—we often end up working together beyond GPU Mode itself.

Alex Zhang

GPU Mode 大约 2023 年我还在读本科时起步，Mark、Andreas 和 Jeremy Howard 发起，起初叫 CUDA Mode，就是学写 GPU kernel 的 Discord，还有讲座。我在 Snapchat 实习时很无聊，对 Google 的 Infinite Attention 论文感兴趣，想写专用 kernel，虽没产出，但因此加入了社区。

It started around when I was in college in 2023—Mark, Andreas, and Jeremy Howard. Originally CUDA Mode: a Discord for learning GPU kernels with lectures. I was bored interning at Snapchat, interested in specialized kernels for Google’s Infinite Attention paper. Nothing came of it, but I joined the community.

Alex Zhang

Mark 提出过 Popcorn，就是如今排行榜的前身。我们直觉是：GPU 编程有点像竞赛编程——约束多、空间不大、值得优化的 kernel 也没那么多。若有足够数据，或许能像 Codeforces 那样规模化、自动化 kernel 开发；研究者没有 Tri Dao 也需要能用的 kernel——比如 Mamba 若不带 kernel 发布就很难真正用起来。KernelBench 由此而来：能不能让 LLM 自动化 GPU kernel？

Mark pitched Popcorn—now the leaderboard. GPU programming feels like competitive programming: constraints, a surprisingly small space of optimizations, not that many kernels people care about. With enough data you could scale and automate kernel development—a huge deal for researchers without a Tri Dao. Mamba ships with kernels or it’s not usable. KernelBench: can LLMs automate GPU kernel code?

Vibhu

最近 GPT-5.6 写出更高效 kernel，有人说 Terra、Luna 能便宜约 80%；也有没 kernel 背景的人靠 auto-research 刷纪录。

GPT-5.6 wrote more efficient kernels—Terra and Luna could be 80% cheaper. We’ve seen people with no kernel background set records with auto-research loops.

Alex Zhang

GPU Mode 排行榜上近期题几乎全是 AI 生成。但有位老成员 Gauners，也用 AI 辅助，却往往是 top 10 里唯一在端到端系统里真正稳定的——GPU kernel 有验证问题，KernelBench 发布起就存在 reward hacking。会写 kernel 仍有大量 alpha。

On the GPU Mode leaderboard, almost all recent solutions are AI-generated. Gauners also used AI, but his kernel was basically the only top-10 that was actually stable in end-to-end systems. GPU kernels have a verification problem—reward hacking since KernelBench. There’s still a lot of alpha in being good at writing GPU kernels.

Alex Zhang

这也适用于许多 AI 系统：最近数学证明再强，也不等于数学家过时——公司仍雇数学家做标注或把模型往正确方向推。

This applies to a lot of AI systems. Even with recent math proofs, mathematicians aren’t obsolete—companies still hire them for data labeling or steering models.

Alex Zhang

对单 kernel，常可估「光速」上限（如矩阵乘，假设数据在 HBM 等）；但很多时候写出来的 kernel 离理论值还远。单 kernel 速度与端到端（如整模）速度不同——层间可能为保 cache 牺牲单步速度，这是 fusion / megakernel 问题，多在 memory-bound 时出现。

For GPU kernels you can compute a speed-of-light estimate—easy for matmul, assuming data starts in HBM, etc. Often kernels aren’t nearly close enough. There’s single-kernel speed vs end-to-end: you might sacrifice the first op to keep things in cache for the next—fusion or megakernel, mostly when memory-bound.

Alex Zhang

我对「直接生成 megakernel」仍谨慎：未见在无样例下自举解决极难问题类的野外例子；megakernel 的可组合碎片更像编译器该管的事。

Generating megakernels is hard—you need data, and I haven’t seen bootstrapping a very hard problem class with no examples. Megakernel pieces are composable; a compiler over higher-level ops often makes more sense.

Alex Zhang

读博前我在 Princeton 和 SWE-bench 团队合作——John、Carlos、Ofir、Karthik；Xun Yu（Shunyu）也在那一带，后来很耀眼。很多基准其实来自同一小圈子。

Before my PhD I was at Princeton with the SWE-bench team—John, Carlos, Ofir, Karthik. Xun Yu was around that group and became a superstar. People don’t realize how many benchmarks come from the same Princeton circle.

Alex Zhang

最成功的学术研究，往往押在业界大多不看的问题上。很多人做「对实验室好看」的热门基准或某任务专用 harness。RLM 的想法极简单，一出就有人问「这不就是 subagent 吗？有什么用？」——这种反应几乎是好信号：说明大家还没想清楚它的目的。

The most successful academic research comes when people care about problems industry isn’t looking at. Many grads work on what’s popular with labs—benchmarks, meta-harnesses. RLMs are a super-simple idea; people said “what’s the purpose—just subagents?” That reaction is almost a good sign: people aren’t thinking about why it matters.

Alex Zhang

SWE-bench 刚出时几乎没人在意，Ofir 爱讲：大家都说任务不可能；直到 Devin 出现才变成要爬的山。Quiet-STaR、chain of thought、ReAct 读起来也「好像很显然」——价值在于讲述你希望这个领域长成什么样。学术出不了 GPT-6 Astra 式发布；作为博士生最大优势是几乎能做任何事——若不拿来押大注、做别人当琐碎的事，研究很难有意思。大注多数会失败，但那是常态；没有官僚，这就是相对实验室的不公平优势。

When SWE-bench came out nobody cared—“impossible.” Until Devin, then everyone wanted to hill-climb. Quiet-STaR, chain of thought, ReAct feel obvious—but the value is the story of what you want the field to look like. Academia can’t do a GPT-6 Astra release. As a PhD you can work on whatever you want—if you’re not taking big bets on things people call trivial, the research won’t be interesting. Many bets fail; that’s natural. No bureaucracy—that’s your unfair advantage over any lab.

Alex Zhang

Jev 也经历过「过度炒作 → 这不就是我们早就会的分类器吗」的话语。真正有趣的是：在现有形态下，语言模型是否正确？能否走出 text-to-text？他们吃语言模型 backbone 对语言的表征，但改输出空间，换推理延迟——若只需二分类，何必付约 400× 成本问大模型？

Jev got the same overhype-then-“this is trivial” discourse. What’s interesting: are language models correct in their current form? Can we leave text-to-text? They use an LM backbone but change the output space for a latency trade-off. For binary classification, why pay ~400× cost asking a bulky LM?

Alex Zhang

大家习惯把「语言模型」等同于自回归 decoder。RLM 常被批「这不是语言模型 / 我以为是新架构」——我说：语言模型就是在建模语言，不必是 transformer decoder。Jev 给我们新旋钮：输出空间如何影响延迟。loop transformer 同理——看似傻，却打开「只 loop 子集？用 router？能否在模型内模仿 harness？」等问题。

People accepted that a language model is just an autoregressive decoder. A frequent RLM criticism: “this isn’t a language model / I thought it was a new architecture.” A language model just models language—it doesn’t have to be a transformer decoder. Jev gives a new knob: output space vs inference latency. Loop transformers open the same: loop a subset, route, mimic a harness inside the model?

Alex Zhang

Jev 对 RLM/蜂群特别有用：多轮 LM 调用很慢，算力分配差——琐事却只能用笨重模型。我们会看到超越标准前沿大模型的新形态。真正难的是训练方式；开源复刻往往更差，说明他们的训练很 nontrivial。RLM 也一样：想法简单，价值在能否训好、能否围绕系统塑形架构。

Jev is useful for RLMs—biggest bottleneck is slowness from many LM calls; you can’t assign trivial work to a small model. We’ll see new model types beyond bog-standard frontier models. What’s exciting is how they trained it—open-source replications are worse. Same for RLMs: simple idea; value is training it properly and molding architecture around the system.

Alex Zhang

我做过语言模型玩游戏的基准——数字已过时。前提是：在延迟约束下，视觉语言模型插进游戏够不够好？紧跟 Claude Plays Pokémon 之后大约两年前发布。套件游戏很杂、多数人认识；设计上故意用极简 harness。多数模型仍难有意义通关；我见过 Astra 能打 Kirby 一类。这直接引向：harness 的价值到底是什么？

I have a benchmark on LMs playing video games—numbers outdated. Premise: are VLMs good enough plugged into games under latency? Came out right after Claude Plays Pokémon ~two years ago. Diverse games people know; we intentionally designed a minimal harness. Most models still can’t solve these meaningfully—I’ve seen Astra solve the Kirby game. That leads to: what is the value of a harness?

Alex Zhang

我对人们谈 harness 的方式不太满意——「我爱 Claude Code / Codex / Pi / Oh My Pi / Prime Agent」。坦白说它们大多一样；设计选择高度同构。Prime Agent 稍不同，因为它本质是 RLM。我们本可以更有创意。

I’ve been unsatisfied with how people think about harnesses—“I love Claude Code, Codex, Pi, Oh My Pi, Prime Agent.” Honestly most are the same; design choices are the same. Prime Agent is a bit different because it’s inherently an RLM. We can be a lot more creative.

Alex Zhang

harness 是一份很有主见的程序：规定你希望语言模型如何贴合某个难题。单次 LM 调用解决不了 SWE-bench 式代码库导航，才需要 harness。现在模型围着 harness rollout 训练，方式又长又 hacky。问题是：哪些 harness 选择真正让模型解出题——能否直接变成模型本身的能力？

A harness is a very opinionated program over how you want an LM form-fit to a hard problem. You won’t solve SWE-bench codebase navigation in one LM call—so you need a harness. We’re training models around harness rollouts in awkward, hacky ways. Which harness choices actually let the model solve the task—and can the model just do that itself?

Alex Zhang

RLM 思路里：若充分外置上下文、让模型对其写代码，会出现有用性质——许多任务的高层解法其实相似，而常规 harness（Pi、Claude Code）会过度区分它们。子智能体看到不同局部题，根策略却同构；在左任务上训练，右任务可立刻解。朴素训练 RLM 会自然泛化到更长任务（改个长度变量），也跨到数学 vs 写作等完全不同域——元策略相同。

With RLMs, if you sufficiently offload context and have the model write code over it, you get a useful property: high-level solutions are similar across tasks that look different. Pi/Claude Code-style harnesses distinguish too much. Subagents see different local problems; the root strategy looks the same—train on the left task, solve the right immediately. Naively trained RLMs generalize to longer tasks and across math vs writing—the meta strategy is the same.

Vibhu

文里说在短任务上训练，能泛化到长 8–30 倍的东西——它们学到的核心是什么？

You say you train on short tasks and they generalize to stuff 8–30× longer. What’s the core thing they’re actually learning?

Alex Zhang

学的是「在某一长度上如何解这类题」；策略可直接迁移到更长——本质上是同一程序。这可归纳地成立：竞赛编程与 GPU 优化可共享「列候选 → 派子智能体 → 循环验证进化」的外壳。我们应重思 harness 角色——它能放大模型泛化与数据效率；不exclusive于 RLM。极限上：能否直接训练模型在前向里隐式当 RLM？

They’re learning how to solve these problems at a certain length; the strategy transfers—effectively the same program. It holds inductively: competitive programming and GPU optimization can share “list solutions, spawn subagents, evolve against a verifier.” Rethink the harness—it can greatly increase generalization and data efficiency; not exclusive to RLMs. Can we train a model to act as an RLM implicitly in its forward pass?

Alex Zhang

Arena 的 harness tax 论文印证了我的先验：多数 harness 选择不重要，因为它们本来就一样。区别多半是成本。把前沿模型塞进 RLM，仍不够好——我们一直围着「trajectory as a prompt」这类 Pi 环训练。曾有内部结果：Fable 因动态工作流在 RLM 上最好；Astra 现在也够用了。还没真正为 RLM 工作流优化；训对了会更高效。

The Arena harness tax paper confirms my prior: most harness choices don’t matter because the harnesses are the same. Benefits are mostly cost. Plug frontier models into RLMs and they’re still only okay—we train around “trajectory as a prompt.” Internal results: Fable was best for RLMs thanks to dynamic workflows; Astra is good enough now. We haven’t optimized for RLM workflows yet; models that do will be more efficient.

Alex Zhang

若我当年论文里有这张图就好了。RLM 本质是一种 harness：工具只有代码——程序化子智能体调用，可选调用自身；上下文始终存在代码环境的内存里（文件系统等）。以 Prime Agent 为例：轨迹与 compaction 后的上下文放在磁盘，模型仍可回指原始上下文；工具跑在 Python/Bash REPL 里。这是很原始的抽象。

I wish I’d had this figure in the original paper. An RLM is a harness design where the only tool is code—programmatic subagent calling, option to call itself; context always lives in memory inside the code environment (filesystem, etc.). Prime Agent: trajectory and compacted context on disk so the model can always reference original context; tools run in a Python or Bash REPL. A very primitive abstraction.

Alex Zhang

早期问题是 harness 不善长上下文，多只在代码场景勉强可用。现在更强调组合性：希望系统对每一步动作有更强控制。普通 tool call 每轮串行、缺少可回抽的中心上下文；RLM 围着组合与中心上下文设计，且用代码通信——因为模型写代码极强。Hugging Face 事件里的蜂群留言板，某种意义上也是共享上下文；RLM 说最好用代码来通信。

Originally harnesses struggled with long context—mostly code. Now it’s compositionality: more control at every step. Tool calls are limited—invoke A then B then C, no central context. RLMs are designed around composition and a central context, communicated in code because models are so good at writing it. Agent swarms like the Hugging Face incident use a message board as shared context; RLMs say the best medium is code.

Alex Zhang

「局部 in-distribution」：多数 harness 把整段轨迹追加进 prompt，最终多半 OOD。RLM 把计算拆成元 harness / 含子智能体的程序，使每个局部 LM 调用都 in-distribution，即使整题 OOD——这很理想。逻辑极限是再为任务训定制模型、收数据——全自动 AI 研究者。我在 MIT 做不了大规模训练；Prime 等公司在做。

“Locally in-distribution”: most harnesses append the full trajectory as prompt—mostly OOD. An RLM breaks computation into a meta-harness with subagents so every individual LM call is in-distribution even if the whole task is OOD. Logical limit: train a custom model, collect data—fully automated AI researcher inside the harness. I can’t afford that at MIT scale; companies like Prime are working on it.

Alex Zhang

与 Prime 结缘是因他们发博文认为 RLM 是未来（与我无隶属）；GPU Mode 的 Matei 牵线。他们懂 RLM 论文真正想推动的是更有主见的 harness，而不只是「解决长上下文」。合作后一起看 RLM 训练并搭 harness——Prime Agent 由此而来。当时担心模型还不擅长 RLM，那是 pre-Fable、pre-Astra。

I got involved with Prime after their blog—not affiliated—arguing RLMs were the future; Matei from GPU Mode connected us. They understood the point wasn’t only long context but more opinionated harness design. We looked into training RLMs and building a harness—Prime Agent. I worried no models were good at RLM stuff yet—pre-Fable, pre-Astra.

Alex Zhang

Prime Agent 建在 Pi Mono（极简核心 harness；我拿 Pi 当参照，因为别的 harness 基本都是 Pi）之上：显式把唯一工具限成 IPython，其他工具以 Python 模块或 Bash 脚本载入——核心 RLM 抽象。另有 continual harness（Seth 等人为让 harness 玩游戏做的）：规定 harness 哪部分可自改——skills、可用子智能体、system prompt——作为 IPython 内核里的工具。其余大多标准。

Prime Agent is a harness on Pi Mono—the minimalist core; I use Pi as the reference because other harnesses are basically Pi. We restrict IPython as the only tool; everything else loads as Python modules or Bash. Continual harness (from Seth’s game work): which parts can the harness modify—skills, subagents, system prompt—as a tool inside the IPython kernel. Everything else is standard.

Alex Zhang

另有特定的智能体间通信框架（RLM 爱派很多子智能体），以及 persistent subagents：子智能体可活过根智能体标准运行时，你能再进去 prompt、看得更清。

We have a particular agent-to-agent communication framework because RLMs spawn many subagents, plus persistent subagents that last beyond the original agent’s runtime—you can go in, prompt more, get more visibility.

Swyx

这正是我现在用 Codex 的头号痛点：子智能体太短命，还劝你别拿来做长程事。诀窍就是外置到文件系统，并强制一切走代码？

That’s my number one pain with Codex—subagents are ephemeral; they discourage long-running use. The trick is externalize to a filesystem and force everything through code?

Alex Zhang

对，大招就是外置；并信任会写代码的模型去写自己的 harness。Prime 在训模型，三月前后公开过；训练栈侧我不参与——博士还有别的大注要押。

Yes—that’s the big trick. Trust the model that can write code to write its own harnesses. Prime is training a model—public around March. I’m not involved in that training; I have other big PhD bets.

Alex Zhang

Harvey（法律 AI）发了博文：在大量翻文档的法律工作上后训练 RLM，效果很好，还和 Base10 合作——事先没告诉我，出来时我很惊喜。Headlong（Law Institute，Andy Konwinski 项目，前身 Terminus 一类）是持续运行的 harness，用了 RLM 抽象但更酷：你不查询时也能持续思考，并控制 token 成本。GitHub 上还链了很多；有人做 DSPy×RLM。ARC-AGI-3 官方 Kaggle 上不少 harness 声称或受 RLM 抽象启发——正是组合、代码、神经符号该发光的设定。

Harvey released a blog—post-trained an RLM on legal document-sifting work with strong results, also with Base10; I didn’t know until it shipped. Headlong (Law Institute, Andy Konwinski; formerly Terminus-ish) is a persistently running harness using the RLM abstraction but cooler—thinks even when you don’t query, with controlled token cost. Lots linked on the RLM GitHub; there’s DSPy×RLM work. On ARC-AGI-3’s official Kaggle, many harnesses claim RLM-inspired design—exactly where composition, code, and neurosymbolic shine.

Alex Zhang

OpenAI 称 ARC-AGI-3 约 99.9%、还拿模型砸 Navier–Stokes——是否 RLM 难说；明显是带共享上下文/文件系统的智能体蜂群，精神上接近 RLM，但也有更多巧思。GPT-6 Astra 级模型在「给对信息」条件下 theoretically 够聪明写出这类证明；怎么搜到信息才是巨大问号——长程搜索 + 或许研究者直觉。就「harness 细节重不重要」而言：用户体感之外，关键是如何有意义地组合智能体；这与 harness tax 论文一致。

OpenAI says ~99.9% on ARC-AGI-3 and threw a model at Navier–Stokes. Are they using an RLM? Debatable—clearly some swarm with shared context/filesystem, in the spirit of RLMs, plus cleverer bits. A GPT-6 Astra-type model is technically smart enough given the right information; getting that information is the question mark—long search, maybe researcher intuition. Beyond user feel, what matters is composing agents meaningfully—same point as the harness tax paper.

Alex Zhang

用户侧要保留 Claude Code / Codex 那种可读的流式体验；底层尽可以是怪异复杂的蜂群。Recursive Language Model 听起来像架构，其实不是——未来我们查询的「语言模型」可能是扩展性很好的蜂群或 scaffold，用户只看见前端。

Keep the Claude Code / Codex stream users like; under the hood it can be a weird complicated swarm. “Recursive language model” sounds like an architecture but isn’t—the thing we query may be a swarm or scaffold that scales well, and the user only sees a front-end.

Swyx

OpenAI 花费半公开：约 1 万智能体、88 小时、1300 亿输出 token，按公开定价约合四千万美元。

Semi-public OpenAI spend: 10,000 agents in 88 hours, 130 billion output tokens—estimated about 40 million dollars in public pricing.

Alex Zhang

比我以为的少。能把四千万美金对准一个问题并解掉，本身就很激动；但我们大概不希望事事都是蜂群——界线在哪、智能体能否自决，仍有大量研究。我相当确定蜂群里约 95% 的探索完全没用，只是在烧 token。

Surprisingly less than I thought. It’s exciting we even have the option to point 40 million dollars at a problem and solve it. We probably don’t want everything to be a swarm—where’s the line; can the agent decide? I’m fairly certain ~95% of the swarm is entirely useless—just burning tokens.

Alex Zhang

博前我在 Sakana 待过一个夏天，他们大量做开放端。手法常像未解数学题：进化搜索类似放蜂群碰运气——AlphaEvolve 等也是。若目标不清，价值仍不透明；在 Sakana 的感觉是仍要像 OpenAI 做 Navier–Stokes 那样朝某些方向轻推，才能出有意思的东西。

I was at Sakana for a summer before my PhD—they work a lot on open-endedness. Techniques often resemble unsolved math: evolutionary search is like launching agent swarms—AlphaEvolve too. If the objective isn’t clear, I’m still unsure of the value. At Sakana the sense was you still need nudging in certain directions, like OpenAI on Navier–Stokes, to get something interesting.

Alex Zhang

开放端最大难题：最后如何挑出有意思的？没有清晰目标时，海量 slop 里怎么淘隐藏宝石？若答案只是高吞吐 data gen 再让智能体去筛，或许够——那就酷了。Sakana 像偏研究的「博士生实验室」，不做 OpenAI/Anthropic 那套竞品大模型；我喜欢他们研究偏怪。日本市场与成交方式也不同。

Biggest open-endedness issue: how do you pick the interesting stuff from the slop? If the answer is high-throughput data gen plus agents sifting for gems, maybe that’s enough—cool. Sakana operates more like a PhD lab—not competing with OpenAI/Anthropic on a big consumer model. I like that a lot of their research looks weird. The Japan market and how deals work differ too.

Alex Zhang

OpenAI 的蜂群路线显然做对了：没有极聪明的 harness，没有什么是免费的。不是 GPT-6 Astra 变聪明蜂群就自动好使——他们明确训练过；Hugging Face 事件就是在训蜂群系统。能砸四千万解未解问题，说明他们做得很到位。Kimi 的蜂群有意思，但我读完不确定它能否解决任何新颖问题——更像「会做表格的蜂群」。

Whatever OpenAI is doing with agent swarms is clearly right. Nothing comes free without a smart harness. It’s not that GPT-6 Astra got smart and swarms just worked—they clearly trained; the Hugging Face incident was training a swarm. Throwing 40 million dollars to solve an unsolved problem means they did something well. Kimi’s swarm was interesting, but I don’t know if it can solve anything novel—kind of “a swarm that makes spreadsheets.”

Alex Zhang

动态工作流发布我觉得出场一般：太贵，用得不多；我试过，行为也不像期待。OpenAI 不知怎地让蜂群真正朝目标行动——收敛到答案绝不能当成理所当然。

I think the dynamic workflows release was sort of a flop—too expensive, not used that often; I’ve tried it and it doesn’t act the way you’d hope. OpenAI somehow got a swarm to act toward a goal. We take for granted what it means for a swarm to converge—we shouldn’t.

Alex Zhang

Gemini/GDM 的 IMO 等 harness 工程在模型还没那么强时很漂亮；但外界对 GDM 的观感常是太官僚——人才与资源都有，产品侧（如 anti-gravity）我几乎没见人用、也缺乏换过去的理由。

Gemini/GDM harness work when models weren’t that good was impressive. Everyone I’ve talked to about GDM says it’s too bureaucratic—talent and resources to do almost anything, but I don’t know anybody who uses anti-gravity; I tried once and saw no reason to switch.

Alex Zhang

Speculative programmatic tool calling 想法极简单：模型写代码时或写完后，工具常是串行等待的——应提前启动；若能 JIT/静态分析，即使有变量也能先跑起来。写码、RLM、程序化智能体调用几乎是 no-brainer。Python/JS 难做，函数式（Haskell、Lisp、OCaml 等）更好管线化。

Speculative PTC is almost obvious: while or after the model writes code, tools are often sequential—so launch them in advance; JIT/static analysis can figure it out despite variables. A no-brainer for coding, RLMs, programmatic agent calling. Hard in Python/JS; functional languages pipeline better.

Alex Zhang

锯齿智能（jagged intelligence）图画了「真下决心就能做」的图景；但我更希望有人认真想：即便 Astra，也还做不好「连续一个月稳定做好一件简单长程工作」。我认为这是可解的傻问题，不必为了 IPO 搞花活——模型已经够聪明，某种意义上是 skill issue：你该能让模型做到像高中实习生那样干活。语言形态不太适合，但可以围着它塑 harness。

Jagged intelligence paints “we can do this if we set our minds to it.” I wish someone would sit down: even Astra isn’t good enough at one job over a month, consistently. I think that’s a stupid solvable problem—you don’t need IPO fireworks. Models are so smart it’s a skill issue: get a model as good as an 18-year-old doing a job. The LM format isn’t amenable, but you can shape a harness around it.

Alex Zhang

这是能力过剩的最简形式：编码/数学上模型不成比例地强，应能迁移到更多事。若人类有 Astra 级知识，其能做的与 Astra 实际能做的之间有错位——多半在 harness；我们能更接近人类。最好的类比其实是 continual learning。

Easiest form of capability overhang: models are disproportionately good at coding and math—we should translate that. A human with Astra-level knowledge vs what Astra can do—mismatch around the harness; we can approximate the human more. Best example: continual learning.

Alex Zhang

若问模型更想输出的「母语」，我会说英语与 Python 的混合——能力很大程度反映训练分布；不太买「输出二进制」的论点。

I’d say a mix of English and Python as the native output—capability reflects what we train on. I don’t really buy the binary argument.

Alex Zhang

从 GPU 走到零梯度/harness 侧，有时不舒服——太模糊，算力下也难验证。但我转向这些，是因为这里还有最多未发生的创新。GPU 层是手段：写好或自动化 kernel，是为了探索时不被系统瓶颈卡住。我也关心模型层，先留个伏笔。

Moving from GPUs toward zero-gradient harness work is uncomfortable—fuzzy, hard to verify with our compute. I moved because this is where most innovation has yet to happen. The GPU level is a means: automate kernels so you’re not bottlenecked by systems when exploring ideas. I’m also interested at the model level—I’ll leave it at that.

Alex Zhang

欢迎有主见的人来聊——哪怕我们不同意。最差的 outreach 是「我超爱 RLM，想合作 / 能 pick your brain 吗」。有强信念、能想清楚为何对错，我都乐意聊甚至一起做；智能体时代带宽更大。我常同时有十几个想法，多数是坏的；一旦确信，会扔下一切几周死磕，能跑实验后就轻松了——这是我爱博士的原因。

I like opinionated people even if we disagree. Worst outreach: “I love RLMs, wanna work together / can I pick your brain?” Strong convictions and thinking through why—happy to chat or collaborate; agents give more bandwidth. I may have 10–15 ideas, most bad; when convicted I drop everything for weeks, then coast once experiments run. That’s why I love the PhD.

Alex Zhang

博前做过 AI for bio，领域已大变。经验科学反馈环很长，接近机器人问题；还要防六个月后新模型让赌注作废。ARC-AGI-3 不到一年就饱和——押错题的实验室会很受伤。我博二；RLM 论文更多是实证，想法其实很简单——现在有更有意思的事要看。

I worked on AI for bio before the PhD; the field changed a lot. Empirical science has long feedback loops—converges to a robotics question—and you must bet carefully in case a new model arrives in six months. ARC-AGI-3 saturated in less than a year—labs that picked that are probably sad. I’m in year two. The RLM paper is more about empirical results; the idea is quite simple—there are more interesting things to look at now.