Kohli 与 Candido：生物学的「苦涩教训」与 AlphaFold 未解之题

主持人

这是建模专场，我们都是建模者，所以自然要聊数据。我想问的是：数据能不能被合理地「规模化」？这引出了「苦涩教训」——能规模化的方法最终会赢——如果把它放到数据的框架里看，会怎样？

This is the modeling session. We are all modelers, so of course it’s natural for us to talk about data. Is there a scaling? How do you scale data properly? That brings me to the bitter lesson, the statement that methods that scale win eventually, but recast in the frame of data.

Sal Candido

当然有。首先你得有对的数据。人们对缩放定律有个误解，以为它无处不在、始终存在；其实很大一部分工作就是把那条缩放定律找出来：什么情形下，多投一点算力、多投一点数据，结果真的会更好。这种情形之所以好，是因为一旦出现，你就可以「转曲柄」，它变成了工程问题，而我喜欢工程问题。

这既和架构有关，也很大程度和数据有关。如果数据里没有解决你问题所需的信息统计特征，你就得不到具备相应能力的模型；你只能从手头的数据里提取信息，再往外泛化。

Yeah, for sure. I think that you obviously need the right data. One misconception of scaling laws is that scaling laws are everywhere, and they always exist. A lot of the work is actually finding that scaling law, right? What’s the situation where, if you put more compute into it, if you put more data into it, you actually get a better result out. And the reason that’s a great situation is then, once that happens, you can just turn the crank, it becomes an engineering problem, which is something I like.

But that has to do with architecture, but it also very much has to do with data. If you don’t have the data with the right information statistics to solve the problem that you want, you’re not going to get a model that has the capabilities. You can only really pull the information from the data that you have, and then use that to generalize beyond.

主持人

在数据采集上，你怎么看哪些数据模态最好？还有，建模者是不是倾向于去做「数据现成」的问题，而不是对转化医学影响最大的问题？

When it comes to data collection, how do you think about what types of data modalities are best? Do you think that modelers have a tendency to work on problems where the data is available, rather than the problem that has the most impact for translational medicine?

Sal Candido

当然。我不能代表所有建模者，但我比较懒，所以会用现成的。这有好的一面：做传统机器学习时你追求最干净的高质量样本，但像我这样的人会去「后屋翻别人的废料」。比如我们训练蛋白质语言模型时用的是宏基因组序列，质量并不是最高的，其中很多我敢保证甚至不是完整的蛋白质，可它确实让模型在设计真实可用蛋白质上的表现变好了。

坏的一面是，你会满足于「只扩大容易生成的数据」，我不认为这是正确的路。所以我对今天谈的这件事很兴奋：回头去问「解决这个问题到底需要什么数据」。这需要对的资源，也需要对的社区。我们在 Biohub 努力做的是开放工作、和社区合作、整体往前推进。

如果只有做模型的人，我们会偏向已有的数据；如果只有生成数据的人，他们会偏向容易生成的东西。但如果大家每一步都公开协作，就能弄清我们需要什么数据，然后找到那条缩放定律。

Oh, for sure. I wouldn’t speak for all modelers, but I’m lazy, so I’m going to work with what’s available. There’s a good side and a bad side. If you’re like me, you’re rooting around in the back room looking through people’s junk. When we train a protein language model, we train on metagenomic sequences, which are not the highest quality data. In fact, much of that data, I can guarantee you, isn’t even a real whole protein. And yet that makes the performance of the model go up for designing real proteins that work.

But that can lead you to a negative place where you say, let’s just scale up the data that we can generate easily. I don’t think that’s necessarily the way to do it. What’s exciting is going out and saying, what is the data that we need to solve the problem? It’s the right resources, but also the right community. What we try to do at Biohub is work in the open, work with the community, and move the whole community forward.

If you just have people who are building models, we’re going to lean towards the data that exists. If you just have people who generate the data, they’re going to lean towards the things that can be generated. But if you can work together as a community every step of the way in an open fashion, then you can figure out what is the data that we need, and then you can find that scaling law.

Pushmeet Kohli

我在 DeepMind 的时候，Rich 也在，我们讨论过这个「苦涩教训」。我从 Rich 在 DeepMind 的那次讲座里带走的，并不是数据到底有没有用，而是更概念性的东西：我们看问题时，常常以一种「宗教式」的方式想解决方案——我是建模的，或者我是做数据生成的。我认为这才是苦涩教训：如果带着这种心态，你可能不会成功。

问题先行，解决方案空间要灵活：需要建模就做建模，需要收集数据就收集数据。机器学习早年的做法是，「我是机器学习研究员，数据集在这儿，有训练集和测试集，我优化模型就行」。这是行不通的，因为如果最终目标是解决问题，就得同时看过程里的两个方面，而过程里不只有数据和建模，还有专长。

I was at DeepMind when Rich was with us and thinking about this idea of the bitter lesson. What I took from Rich’s original lecture at DeepMind was not about whether data is useful, but something more conceptual. Sometimes when we are looking at problems, we think about solutions in a very religious way. I’m a modeler, or I’m a data generation person. I think that is the bitter lesson: if you approach the problem with that mindset, you might not succeed.

The problem comes first and you should be flexible in terms of your solution space. If it requires modeling effort, do the modeling effort. If it requires collecting more data, then collect data. In those times people said, we are machine learning researchers, the data set is there, here’s some training data, here’s some test data, and we will just optimize the model. And that is broken, in the sense that if your eventual goal is to solve the problem, you have to look at both aspects of what goes into the process. What goes into the process is not just data or modeling, it’s also expertise.

Pushmeet Kohli

以 AlphaFold 为例，我们看的是现有数据集能做到什么，因为当时我们既没有那样的专长、坦率说也没有资源，把 PDB 数据库再扩大一个数量级。全世界的机构和科学家为构建这些数据已经投入了不可估量的心血，所以在那里你要把钱花在建模上，换最大的收益。

但如果你面对的是细胞基因组学，我们采取了同样的思路，问：用现有的细胞数据能做什么？做了不少工作之后，结论很清楚：数据还没到位，无法去追求构建虚拟细胞这个宏大的目标。这会让大家聚焦于推进科学真正面临的挑战，而不是宗教式地追随数据生成或建模的进展。

With AlphaFold, we looked at what was possible with current existing data sets, because we did not have the core expertise, or even the resources, to augment the PDB by a significant order of magnitude. The amount of investment that organizations and scientists across the world had put in to construct that data was invaluable. So there you focus on getting the biggest bang for your buck by investing in modeling.

But if you’re approaching cell genomics, we took the same approach and asked what can you do with existing cell data. After a bunch of work it was very clear that the data was not there yet to go after that grand ambition of building the virtual cell. It brings people together to focus on the actual challenges if you want to advance science, rather than religiously following advances on data generation or in modeling.

Pushmeet Kohli

我给刚入行的人的建议是：把自己当作跨学科的人。先弄清为什么做这个问题、想达成什么，再想需要什么——建模、算力扩展还是数据生成——当然还得在内部积累专长。同时要知道约束：能生成的数据量有限，能负担的训练规模也有限。理解约束，快速试错，看哪些方法在长期可行。

The advice I give to anyone starting in the area is to think of yourself as a multidisciplinary person. Understand the problem first. Why are you working on this problem? What are we trying to achieve? Then think about what will be needed, whether it’s modeling, compute scaling, or data generation. Then you have to build your expertise in-house. There are constraints: only a specific amount of data you can generate, or a specific model size you can afford to train. Understand those constraints and try to fail fast.

主持人

AlphaFold 2 几乎是一件艺术品：大量精心手工设计的特征，每个部分都经过审慎思考。现在业界的共识似乎转向更通用、可扩展的策略。在资源固定的情况下，应该先做规模，还是仍需要这种「手艺」？

AlphaFold 2 was essentially a work of art, a bunch of very carefully handcrafted features, a lot of very intentional thought to every part of the solution. The general consensus now seems to be to move to more scalable, more general strategies. With fixed resources, should we be focusing on scale first?

Pushmeet Kohli

这不是偶然。我们做了很多实验，但背后有一个愿景：来自生物物理和生物化学的科学直觉——氨基酸残基不是各自为政，而是会互相影响——那就把它内置进去。如果你从科学界学到了什么，就去用它，给模型一点「不公平的优势」。这会让模型的数据效率高很多，因为它不必把一切从头复现。

我还要说，整理好数据本身就是一门艺术。不是「多放点数据」就行；重复同样的数据哪儿也去不了。重要的不是大数据而是好数据，而弄清楚取得进展需要的数据覆盖面，其实是一个更有趣也更有挑战的问题。

It was not an accidental thing. We did a lot of experimentation, but there was a vision behind it: scientific intuition that came from biophysics and biochemistry, that amino acid residues are not just doing their own thing, they are being influenced by other residues. So let’s bake that in. If you have learned something from the scientific community, use it, and give the model that unfair advantage. It makes the model much more data efficient because it doesn’t have to replicate everything.

Curating good data is an art in itself. It’s not as if you will say, well, put more data. If you replicate the same amount of data, you’re not going anywhere. It’s not just about big data, it’s about good data. Actually understanding the coverage of what data is necessary for making progress is, I would say, a much more interesting and challenging problem in itself.

Sal Candido

我非常同意，这取决于要解决的问题。数据量较小时，模型里更多的归纳偏置肯定有帮助，小规模下你需要它才能拿到结果。数据越来越多，模型有时能发现你本来不知道的东西；如果归纳偏置不完全正确，它反而会拖住你。所以确实存在一个拐点。

我也想反驳一下这个问题：我认为「缩放」这部分同样有大量手艺。要把模型放到更多数据上训练、投入更多算力、做得更大，需要很多算法工作；无论是基础设施层面，还是让模型推理更快。

I very much agree. It really depends on the problem to be solved. If you have a smaller amount of data, then certainly having more inductive bias in the model is going to help you. You actually need that to get the results on smaller scales of data. As you get more and more data, sometimes the model can find things that you didn’t necessarily know about, and sometimes that inductive bias, if it wasn’t exactly correct, can hold you back. So there is definitely that tipping point.

I kind of object to your question too, because I think there’s a lot of craft to the scaling part of things as well. There’s a lot of algorithmic work that goes into taking these models and training them on more data, putting more compute, making them bigger. That’s from an infrastructure perspective and also making the models run inference faster.

Sal Candido

我们正在看到架构在走出标准 Transformer，变成更定制化的形式；不过我不认为我们已经处在后 Transformer 的世界，无论从哪个角度看都不是。我们是在修改这些架构，让它们更适合特定用途，即便在互联网规模的数据上也更好用。

随着更多数据进来，这个项目不仅是帮助整理数据、选择下一批真正需要的信息——这更像是「信息」而不只是「数据」——还在于在每一个规模下，找到能从数据里榨取最大价值的架构。

We’re actually seeing that things are going beyond transformers to more bespoke architectures. I don’t think we’re in a post-transformer world in any way, shape or form, but we’re modifying those architectures in order to make them more fit for purpose and actually work better even at internet scale data.

As more and more data comes in, it’s not only helping to curate that data and select what is the next batch of information that we actually need, it’s more information than data. And then at every step of the way, at every scale, what is the right architecture to get the most out of that data?

主持人

新闻说蛋白质结构预测「已经被解决」，但我的朋友们都说还有很多事要做：功能、动力学、设计都是开放问题。今年是 AlphaFold 2 发布五周年。Pushmeet，你怎么看——还有下一次大跃迁吗？是缺数据，还是缺算法和想法？

The news will say protein structure prediction has been solved. But if you talk to my friends, they will all say we have so much left to do here: function, dynamics, and design are still wide open problems. It’s been five years since AlphaFold 2 was announced. Is there another big leap being made here? Do we not have the right data yet, the right algorithms or ideas?

Pushmeet Kohli

科学的运作方式，基本就是先隔离出一个问题，再一步一步取得进展。所以当人们说蛋白质折叠问题被解决了——在概念层面，是的，有了一些进展。但当你想到「蛋白质是积木」这个叙事……我总在说蛋白质是积木，可我其实不相信。蛋白质极其复杂：它们有无序区域，形状会随环境改变。

我和 John Jumper 常讨论：我们到底在解决什么？我们并不知道蛋白质真正的基态是什么，也不知道它们结构的真实分布是什么。我们做的是：有人解出了一个结构，存进 PDB，我们试着复现同样的结构。它碰巧很有用，但这并不意味着我们已经理解了全部蛋白质动力学。

在顶层，说「我们取得了进步」更容易沟通；但在座的科学家都清楚：还有大量工作要做。有很多值得庆祝的，但别停止对蛋白质结构预测和蛋白质动力学的资助，因为我们才刚刚开始。

Science operates basically by isolating something and then making progress step by step. When people say the protein folding problem has been solved, at a conceptual level, yes, there have been some advances. But when you think about that narrative of proteins being the building blocks — I say proteins are the building blocks all the time, but actually I don’t believe in it. Proteins are extremely complex. They are disordered. Their shape might change depending on the context.

John Jumper and I used to discuss this: what are we trying to solve? We don’t know the actual true ground state that proteins take, or the actual distribution of structures that proteins take. What we did is that someone got a structure, deposited it in the PDB, and we tried to replicate and get the same structure. It just so happens that it’s useful. But that does not mean we have understood all of protein dynamics.

At the top level, it’s easier to communicate that we have made progress, but the scientists among us know that a lot of work needs to be done. We have made progress, and there’s a lot to celebrate, but let’s not stop the funding of protein structure prediction and protein dynamics, because we are just getting started.

主持人

如果能挥一下魔杖，多出一样东西来加速功能、动力学或设计，你会要什么？

If there’s one thing you could wave a magic wand and say we have more of, that would accelerate one of those things — function, dynamics, or design — what would you wave into existence?

Pushmeet Kohli

我的背景相当杂：最初做安全研究，然后是计算机视觉、贝叶斯理论，最后是判别式机器学习和深度学习，再到科学。所以你问这个问题，我心里的计算机视觉研究员会对冷冻电镜显微图像特别兴奋：我想，PDB 数据是什么？我应该在源头工作，直接看冷冻电镜图像，因为那些结构一定漏掉了很多数据。

如果能有模型在这个层面上、用合适数量的数据进行扩展，并提取出其中捕捉到的所有动力学和分布信息，那将是了不起的事。我试过，但需要更多工作，我仍在继续。也许会有比我更厉害的人来尝试，然后我们会有所收获。

My background is quite eclectic. I started as a security researcher, then went into computer vision, Bayesian theory, and finally got into discriminative machine learning and deep learning, and then science. When you ask me that question, the computer vision researcher in me was super excited about cryo-EM micrographs. What is this PDB data? I should be working at the source. I should be looking at the cryo-EM micrographs. I don’t want those structures. They must be missing out on a lot of data.

Getting models that can scale at that level with the right amount of data and can extract all the dynamics and distributional information that is captured there would be an amazing thing. I tried it, but it requires more work. I still work on it. Maybe at some point people better than me will take a stab at it and we’ll get somewhere.

Sal Candido

这些模型很有用，但并不是大多数人想解决的问题本身，它们解决的是一个很具体的用途，你再拿它们去做别的，比如设计新蛋白。我把我们现在建的模型想成：有人想弄懂自行车怎么工作，而你在给其中一根辐条建模。这些模型可以越来越好，但真正需要的是从「辐条的模型」走向「车轮」，再走向「整辆自行车」，因为那才是人们想理解的。

随着我们把这些模型放进它们所处的具体背景里，就能在更广的层面上学到这些相互作用，这是接下来的方向。不过我同意：我们应该继续做折叠模型，因为它们还会越来越好。

These models are quite useful, but they’re not exactly the problem that most people want to solve. They solve a very specific purpose and then you can also use them to design new proteins. I think of the models that we build right now as someone trying to understand how a bicycle works and you’re modeling a spoke on it. Those models can get better and better over time, but you need to move from models of spokes to wheels to maybe whole bicycles, because that’s actually what people want to understand.

As we put these models more into the context in which they’re in, we’ll be able to learn more about these interactions on a broader sense, and that’s where things are going. But I agree with you. We should keep trying to work on folding models because they’re going to keep getting better and better.

主持人

费曼有句名言：我不能创造的东西，我就不理解。现在我们处在一个不理解也很容易创造东西的世界。你怎么看，让模型帮助人类理解，与得到一个能「一发命中」皮摩尔级结合体的魔法黑箱，哪个更重要？

There’s this famous Feynman quote, that which I cannot create, I do not understand. Now we’re in a world where it’s really easy to create things without understanding them at all. How important is it to have models which can help humans understand things versus magical black boxes which can effectively one-shot a picomolar binder?

Sal Candido

在 AI 出现之前，我们就一直在用魔法黑箱设计东西，所以从某种意义上它仍然有用。但我认为理解很重要，这也是我对 AI 想得很多的一点：智能越来越便宜、越来越容易获得，你可以部署它去学习世界上越来越多的东西。可问题是，如何把这些知识从机器里「拿出来」，变成我们能理解的东西？也许这只是我个人的好奇心，但我认为很多科学家确实想理解，而终点也许没那么重要——不过我们在这里是为了解决转化医学问题。

We were designing things with magical black boxes long before AI came around, so in some sense that’s still useful. But I think the understanding is really important. That’s one of the big things I think about a lot with AI. As intelligence is getting cheaper and cheaper and more available, you can deploy it to learn more and more things about what’s going on in the world. But then how do you actually pull that knowledge out of the machine so that I can understand it? Maybe that’s just my esoteric curiosity. I think lots of scientists really want to understand things and the endpoints are maybe not as important, but then we are here to solve translational medicine as a problem.

Sal Candido

我认为这些模型里的信息比我们知道的要多得多。我们对自己的模型做了很多可解释性工作，发现里面有很多信息：人们知道蛋白质语言模型会在表征里学到某种结构的概念，但我们还发现了关于功能的信息，关于运动的信息。所以我认为，即使是现有的模型，里面也还有很多可以释放出来的东西。

归根结底，你期待在把所有这些信息压缩进模型的过程中出现一个世界模型。模型怎么完成工作？它怎么设计一个蛋白质？它把演化的全部信息压缩进了这个模型。所以除了它能给我们吐出有用的结果之外，里面一定还有值得学习的东西。

I think the models actually have a lot more information in them than we know. We’ve worked a lot on interpretability for our models, and you find a lot of information in there. People know that protein language models learn some notion of structure within their representations. But we find information about function. We find information about motions. I think there’s a lot in there still to be unlocked, even from the models that we have now.

At the end of the day, you expect a world model to come out of trying to compress all this information into the model. How does it design a protein? It’s compressed all the information from evolution into this model. In addition to it being able to spit out something which is useful to us, there’s certainly something to learn about what’s inside.

Pushmeet Kohli

我的看法有些不同：我认为某种程度的理解是必要的，我来说是哪种程度。AlphaFold 并不完美，AlphaFold 2 在当时那个数据集上大约是 90 的 GDT。但即使它是 95，如果它的置信度分数完全没有校准，谁会信它？它会神奇地给出好答案，但也会非常自信地告诉你一个答案，你为此忙上一年才发现它完全错了。所以不确定性度量的校准极其重要。在这个意义上，我们确实在理解 AlphaFold 2 的行为方面取得了很多进展。

这不同于「它是怎么找到解的」。在这一点上，我同意 Sal，存在可解释性的概念：它为什么给出这个答案？AlphaFold 2 在最高层面上是可解释的，体现在它的泛化能力上。而且这是我们发布前没发现的：我们发布并开放权重之后，人们发现它是个很好的无序蛋白预测器，能判断蛋白的哪些部分是无序的。

I have a different take. I think some level of understanding is necessary. AlphaFold is not perfect. AlphaFold 2 was 90 GDT at that time on that set. But even if it was 95 GDT, if the confidence score was completely uncalibrated, who would trust it? It would just magically give good answers, but suddenly tell you “here’s the answer” very, very confidently, and you would be working on it for the next year and find out it was completely wrong. The calibration of the uncertainty measure was extremely important. In that sense, we did make a lot of progress in understanding how AlphaFold 2 behaves.

That’s different from how it found the solution. There I agree with Sal that there’s a notion of interpretability. Why did it give this answer? I think AlphaFold 2 was, at the highest level, interpretable in terms of its ability to generalize, and we did not discover it before launching. When we launched AlphaFold 2 and made the weights available, people found out that it was a great disordered protein predictor. It could figure out which elements of the protein are disordered.

Pushmeet Kohli

可解释性要问：对谁可解释？如果是指一个受人脑认知与计算限制的理性系统，那么 AlphaFold 2 不可解释。但如果问的是，对一个更庞大、更复杂的模型来说它是否可解释，也许是，只是我们看不懂。不过作为 AlphaFold 2 的用户，我们需要了解它能做什么、不能做什么，也就是对模型强项和局限的行为刻画。没有这种刻画就使用这些模型，它非但没有帮助，还会伤害我们。

Interpretability asks the question: interpretable by whom? If you’re saying interpretable by a human rational system with the cognitive and computational limitations of the human brain, then no, AlphaFold 2 is not interpretable. But if you’re asking is it interpretable to a much larger and more sophisticated model, maybe it is, we just don’t get it. As users of AlphaFold 2, we need to understand what it is able to do and what it is not able to do. Understanding the behavioral characterization and the strengths and limitations of the model is extremely important. We can’t just be using these models without that, because otherwise rather than being helpful, it will harm us.

主持人

所以你的看法是，严格意义上的可解释性不是必须的，重要的是确保模型值得信任，让人能做出可执行的决策？

So your take is that interpretability, strictly speaking, isn’t necessary, but a way of ensuring the model is trustworthy so humans can make actionable decisions is the thing people should be focusing on?

Pushmeet Kohli

没错。我也认为可解释性取决于旁观者的眼睛。如果是人类科学家想弄懂模型怎样做出预测，那是一回事；而如果你把未来的大型前沿模型接到激活层上，问它「你能预测 AlphaFold 会做什么吗？」也许它们能预测，还会提出一套关于 AlphaFold 2 如何解读并产生这些结果的理论。

Exactly, and I also think interpretability is in the eye of the beholder. If it is a human scientist trying to interpret how the model goes about making this prediction, that’s a different question from another much larger LLM. If you give some of these large frontier models of the future access to the activation layers and say, can you predict what AlphaFold will do? Maybe they will be able to predict, and they will come up with a theory of how AlphaFold 2 was interpreting and producing these results.

主持人

AI 很有可能在不久的将来大幅改善人类健康。我喜欢量化预测：你的最佳估计是，多久我们会看到 AI 的成果进入临床？

There’s a real chance that AI will make dramatic improvements in human health in the immediate future. I like quantitative predictions. Best guess, how long until we start seeing AI results in the clinic?

Pushmeet Kohli

「AI 成果进入临床」这个问题本身不严谨：AI 今天已经用在药物发现的每一个环节。如果你问的是什么时候会看到 10 倍或 100 倍的时间线加速，那就要问：我们在加速什么？是先导化合物优化、靶点发现，还是临床前或毒理工作？未来几年我们需要攻克生物学里一些难题，这也是我们今天宣布的这项工作如此重要的原因，只有这样才能得到真正的加速。

AI 会一直出现在临床里，影响着进入临床的各种东西；但更大的加速，只有对这项工作试图构建的生物学模型有了更好的理解之后才会解锁。

AI results in the clinic is ill-posed. AI is being used today already in every part of the drug discovery process. If you’re asking when we will see that 10x acceleration or 100x acceleration in the timelines, then the question is, what are we accelerating? Is it lead optimization? Is it target discovery? Is it the preclinical or the tox work? Over the next few years, which is why this effort that we announced today is extremely important, we need to tackle some of these hard challenges of biology. Only then will we be able to get these true unlocks of acceleration that dramatically transform how drug discovery will happen.

AI will be there in the clinic and will be impacting things that go into the clinic all the time. But the larger acceleration will be unlocked only with a better understanding of the biological models that this effort is trying to create.

Sal Candido

我很想戴上生物学的帽子来回答，但重要的是看我们到底希望看到什么结果。什么时候会有一种完全由 AI 做出来的药？很难说，但我认为进展会非常快，因为所有这些工具都已经在被使用。

我很早以前在 Google 学到的一点是，有时候去想「怎样做到 10 倍提升」比想「怎样提升 10%」更容易解决问题。这并不是因为前者路径更容易，而是它让你拥有更宽的视野，看到原本没有考虑的方案，然后回到第一性原理去想：我们要怎么做到？10% 和 10 倍两种方法都有价值，应该两者都做。我很高兴在 Biohub 有一句美好而宏大的使命——治愈所有疾病；要做到这一点，你确实需要找到 10 倍的路径。

I’m tempted to put on my bio hat to answer this question, but what we really want to see is those outcomes affected. When will it be that a drug was made entirely by an AI? It’s hard to know, but I do think we’re going to see rapid progress very quickly because all these tools are being used.

One thing I learned a long time ago in my career, back at Google, is sometimes it’s easier to approach a problem by looking at what it’s going to take to make a 10x improvement rather than a 10% improvement. That’s not because that’s necessarily an easier path, but because it allows you to take a broader view and see some solutions that you haven’t been approaching. You go back and look from first principles: how are we going to really push this? Both approaches, the 10% and the 10x, are valuable and we should do both. I’m very happy that at Biohub we have a very beautiful and lofty mission statement to cure all disease. If you want to do that, you really need to figure out what is the 10x approach.