生物安全是一场 AI 军备竞赛：Eric Nguyen 谈生成式基因组学

欢迎来到 Latent Space。我是 Brandon，在 Atomic AI 构建 RNA 疗法。和我一起的是我的联合主持人 R.J. Honiki，Mirror OMIX 的 CTO 和联合创始人。今天很荣幸邀请到 Radical Numerics 的 CEO 和联合创始人 Eric Nguyen。Eric 在 Chris Ré 的组里开始攻读博士。在长上下文或基因组模型流行之前，他就花了很多时间思考如何做长上下文基因组模型。他是 Evo 生成模型的第一作者，基本上是这个模型的远见者，Evo 是最早的生成式基因组平台之一，后来开发了 Evo 2，这自然引出了 Radical Numeric。感谢你的到来。我漏掉了什么吗？

Welcome to Latent Space. I'm Brandon. I build RNA therapeutics at Atomic AI. I'm joined by my co-host R.J. Honiki, CTO and co-founder of Mirror OMIX. Today it's a pleasure to have with us Eric Nguyen, CEO and co-founder of Radical Numerics. Eric started his PhD in Chris Ré's group. He spent a lot of time thinking about how to do long context genomic models before long context or genomic models were cool. He was the first author and basically the visionary behind the Evo generative model, one of the first generative genomics platforms, developed Evo 2, which naturally led into Radical Numerics. Thank you for being here. Did I miss anything?

听起来很棒。

That sounds great.

是的，欢迎。谢谢。

Yeah, welcome. Thank you.

谢谢。

Thank you.

那么，Eric，我们来谈谈 Omni 和你们关于基准测试的博客文章，但我想先听听，好吧，什么是基因语言模型？我为什么要在意？它做什么？然后我们再谈谈博客文章中的主要结果。

So, Eric, let's talk about Omni and the blog post that you guys did about the benchmarking, but I want to hear first, okay, what is a genetic language model? Why do I care? What does it do? And then let's talk about the topline results from the blog post.

基因组语言模型（GLM）是一种在 DNA 序列上训练的大型语言模型。它非常像你看到的自然语言和聊天机器人，但不是用单词或自然语言训练的，而是用生命的原始材料——构成 DNA 的字母序列——训练的。我们公司和我们团队以创建了最早的生成式基因组模型而闻名，这些模型在 DNA 上训练，不仅能读取，还能写入，也就是能够生成新的 DNA 序列。我们觉得这是一个被忽视的领域，如果 AI 能读写 DNA，它就能极大地改变科学发现、对人类健康的理解以及治疗方法。所以我们觉得在基因组上训练 AI 是一个巨大的机会。

So a genome language model or GLM is a large language model trained on DNA sequences. So very much like natural language and chatbots you see but not trained on words or natural language but on the raw fabric of life which is these sequence of letters that make up DNA. And we ourselves, our company and our team is known for creating the first generative genomics models which are models trained on DNA not just to read but also write, meaning able to generate new sequences of DNA. And we felt this was an area that was overlooked and that if AI could read and write DNA, it could change a lot, you know, scientific discovery and understanding of human health and how to treat it. And so we felt there was a big opportunity to train AI on the genome.

用这样的模型，你潜在能做哪些事情？

What kind of things can you potentially do with a model like this?

对我们来说，一开始就很好。当我们最初开始研究 DNA 模型时，我们研究了一个叫 hyena DNA 的模型，它是一个大型语言模型，但使用了卷积而不是注意力机制。所以，技术细节多一点。DNA 有一个特性，就是它非常长，对吧？当时，这些大型语言模型在上下文方面有有限的限制，对吧？能够容纳长序列。所以我们寻找一种更高效的算法来处理像 DNA 这样的东西。于是我们提出了所谓的 hyena 算子，使用卷积。长话短说，它让我们能处理更长的序列。在这种情况下，长达一百万，当时是语言模型最大的上下文。我们用它来读取 DNA 并预测功能。所以给定一个 DNA 字符串序列，我们预测它的调控功能，它对基因组的影响。这对科学家来说很有趣，因为人体中的很多 DNA，也许人们不太了解，但实际上我们对基因组的了解并不多。我们知道它编码了让我们成为我们的信息，以及所有不同的复杂性和潜在疾病，但同时，关于这些字母组合如何形成、它们编码什么以及如何编码功能和性状的语法规则，还没有完全理解。所以希望是使用这些 DNA 模型语言模型，能够从原始 DNA 序列中映射出一些功能。所以我们的第一代模型能够证明，是的，我们可以训练 AI 来读取和理解 DNA 序列到某种程度，尤其是我们所说的长程相互作用，意思是在序列上，你知道，如果你使用聊天机器人，比如如果你听说过“上下文腐烂”这个短语，你输入语言模型的输入越长，它就开始退化。所以能够捕捉长程信息和模式、基序、长序列上的语法，是我们试图实现的目标。所以我们在 hyena DNA 的第一代中展示了这在超过一百万的上下文上是可能的。然后真正开启现在称为生成式基因组学领域的，是叫做 Evo 的模型。在 Evo 中，我们试图展示的不仅仅是读取 DNA，还能生成它。所以我们想加速生物学家和科学家从生物学，特别是基因组学中学习的方式。我们觉得生成式 AI 应用于语言这个整个领域极大地加速了我们对自然语言的理解和操作能力。但这里有另一种语言——DNA，基因组，我们不了解它，而且几乎没有应用 AI。在我们几年前看到的模型中，它们真的很小，上下文很短，所以它们只能捕捉小模式和有限的上下文，而且没有一个生成 DNA，所以它们都只是读取。我们觉得生成的想法在自然语言中是如此强大和具有变革性，如果我们能把它带到生物学和 DNA 中会怎样？

Great to start for us. When we first started working on DNA models, we worked on this model called hyena DNA, which is a large language model, but it used a convolution instead of attention. So, a little more technical details. DNA has this property that well, it's very long, right? At the time, these large language models had limited constraints on context, right? Being able to fit long sequences. And so we were looking for a more efficient algorithm to be able to handle something like DNA. And so we came up with this what we call the hyena operator use convolutions. Long story short, it let us process longer sequences. In this case up to a million and at the time was the largest context for a language model. And what we did with it was essentially used it to read DNA predict function. So given a sequence of DNA string of characters, we predict its regulatory function, its effect on a genome. And you know this is interesting to scientists because a lot of the DNA in our bodies, you know perhaps people are less aware but actually we don't know a lot that much about our genome. It's we know it obviously it encodes the information for making us us and how you know all the different complexities and potential diseases at the same time the grammar sort of grammar rules about how the combination of those letters are sort of formed what they encode and how they encode function and traits is not fully understood and so the hope was using these DNA models language models to be able to map some function from the raw DNA sequence and so our first generation models was able to show that yes, we can train AI to be able to read and understand to some degree DNA sequences and especially what we call the longer range interactions, meaning over sequences, you know, if you use chat bots for example, like if you've heard the phrase a context rot, you know, the longer the input you put into a language model, it starts to deteriorate. And so being able to pick up long range information and sort of patterns motifs, grammar over long sequences was what we were trying to accomplish. And so we showcased in that first generation of hyena DNA that that was possible over a million context. And then really what started the field now known as generative genomics was the model called Evo. And Evo we tried to showcase there was this idea of not just reading DNA but being able to generate it. And so we wanted to accelerate essentially how biologists and scientists had learned from biology and in particular genomics. And we felt like this whole field of generative AI being applied to language great accelerated the you know obviously our understanding and ability to manipulate the natural language. But here's this other language DNA the genome that we don't understand and it's barely being applied with AI. in our at the time a few years ago models that we saw were really small short context so they could only pick up small patterns and limited context and none of them generated DNA so they all just would read and we felt that the idea of generation was so powerful and transformative in natural language what if we could bring that to biology and and DNA particular

你能完成哪些在实验室里做不到的事情？所以如果你能做得非常好，这会为你解锁什么？

What can you accomplish that you can't do in a lab right so what does that unlock for you if you could do that very well

我认为我们展示的第一件事，让人们对此潜力感兴趣的是 CRISPR-Cas 系统。所以它是一种能够切割 DNA 本身的酶。我认为 Evo 模型特别实现的是能够生成不仅仅是一种模态或一种类型的序列，而是跨越多种模态和多种尺度。所以 CRISPR-Cas 是一个由 RNA、DNA 和蛋白质组成的分子。所以当时你还没有真正看到能够生成多种模态的模型。他们有蛋白质语言模型可以生成蛋白质。

I think one of the first things that we showcased that got folks sort of intrigued by the potential of this was a CRISPR-Cas system. So it's an enzyme that's able to cut DNA itself. And I think what was particularly enabled by the Evo models was the ability to generate over not just one modality or one type of sequence but spanning multiple modalities and spanning multiple scales. So CRISPR-Cas it's a molecule made up of both RNA and DNA and proteins. And so at the time you hadn't really seen models that can generate multiple modalities. They had protein language models that can generate proteins.

过去你有能生成 RNA 的 RNA 模型，但没有一个统一的系统来做协同设计。我们展示了单个 DNA 模型是这两者的基础：从 DNA 出发，你可以得到 RNA 和蛋白质，我们可以设计一个统一的系统来生成，并且能在真实世界中发挥作用。于是我们问 Evo——我们在一批天然的 CRISPR-Cas 系统上做了展示，然后问它：你能造一个新的吗？我们能够从训练好的那个模型里采样，确实展示了 Evo 能够发现一个新的 CRISPR-Cas 系统。大家对此很感兴趣，它登上了《科学》杂志封面，后来我还有机会就此做一个 TED 演讲，这很有意思，因为显然观众非常大众化。所以要努力解释什么是 CRISPR-Cas 系统、什么是 DNA、你怎么生成它？你为什么要生成它？各种有趣的话题。

Sometimes you had RNA models that could generate RNA, but you didn't have a single system to co-design. We showcased that a single DNA model is sort of the foundation of both of those: right from DNA you can get RNA and proteins, and we can design a single system to generate and also function in the real world. So we asked Evo — we showcased it on a bunch of natural CRISPR-Cas systems and essentially asked it, can you make a new one? And we were able to sample from that model that we trained, and indeed we showcased that Evo was able to discover a new CRISPR-Cas system. Folks were intrigued by it, it was on the cover of Science magazine, and later I got to give a TED talk about the work, which is interesting because obviously the audience is very general. So trying to explain what is a CRISPR-Cas system, what is DNA, and how do you generate it? Why would you generate it? All sorts of fun topics.

而我认为更引人入胜、更令人兴奋的是，科学家们最终——就在去年——展示的生成式 DNA 模型能做到的事情：用 AI 从零生成第一个基因组。这是人类做不到的事情，对吧？人类通常是——你可以把它想成复制粘贴其他基因组或其他 DNA 的片段，放进别的东西里。但他们只是取出他们知道功能的小基序，他们理解规则，但从零、从头构建，在基因组层面此前从未做到过。结果 Evo 能够生成一个功能性基因组，在这个例子里就是所谓的噬菌体，也叫病毒。我认为这对科学界、对我们 Radical Numerics 这家公司来说都是一个关键的转折点，因为我们觉得这是一个极具指示性的潜力体现——对吧，创造一种自然界不存在的完整生物体——但同时也意味着潜在的危害。对吧，如果你能控制、能操纵生命本身的构造，控制它的功能，那意味着什么样的影响？你在把什么释放到这个世界里？所以我们实际上收到了很多反馈、很多评论、很多联系，来自既兴奋又担忧的人们，他们担忧这种能力、这类能力，以及它的发展轨迹，对吧？这还只是早期阶段，人们能做的第一件事就是去推测、去想象这可能导致什么。

And I think the even more intriguing, exciting thing that scientists eventually showcased — just last year — about what you can do with a generative DNA model was to generate the first genome from scratch using AI. So this is something not possible by humans, right? Humans usually — you can think of it like copy and paste parts of other genomes or other DNA, put it into something else. But they would just take out small motifs that they know the function of, they understand the rules, but to build something from scratch, from the ground up, had not been done before at the genome level. And so Evo turns out was able to generate a functional genome, and in this case it was what's known as a bacteriophage, also known as a virus. And this was a key turning point, I think, for the scientific community and for us as a company at Radical Numerics, because we felt this was such an indicative manifestation of the potential — right, to create a whole organism not existing in nature — but also the potential harm that that means as well. Right, if you can control, if you can manipulate the fabric of life itself, control its function, what kind of implications does that mean? What are you enabling into the world? And so we actually got a lot of feedback, a lot of comments, a lot of outreach from folks both excited and concerned about this capability, this kind of capability, and just the trajectory, right? This is the early stages, the first thing one can sort of project and imagine what this could lead to.

所以我们作为一家公司觉得，重要的不仅是推进这些模型在生物设计上的能力，还有把它们用作防御工具的能力，来应对可能出现的滥用和生物风险。我认为确实有很多公司、很多前沿实验室也在担忧这个正在浮现的风险：AI 模型有能力设计生物序列。与此同时，目前主要是从自然语言层面来防御——比如各种防护措施，像 Claude，你知道，如果你谈论病毒，它就会直接把你拦下来。这很好，我认为在某种程度上你确实需要自然语言层面的防护，但我认为你显然还需要生物序列层面的防护。所以你需要模型不仅能理解语言和你对话的脉络，还要理解底层的物质本身，这是下一步，也是终极的边界。对吧，如果你能让模型理解某个序列是否是致病的、是否是病毒，那才是你想要的防御能力水平。然后能够把它推广到监测系统、国家安全，能够监控环境中新出现的序列——这就是这类能力所能实现的。然后我们认为，把这项前沿技术带给那个群体也同样重要，和把它用于人类健康一样重要，而后者是我们主要关注的方向。

And so we felt as a company it was important to not only push on the biological design capabilities of these models, but also the ability to use them as defensive tools for the potential of misuse and biological risk that emerges. And I think indeed a lot of companies, a lot of frontier labs, are also concerned about this emerging risk of AI models being capable of designing biological sequences. And at the same time it's being mostly attacked from a natural language standpoint — like safeguards and things like Claude, you know, if you talk about viruses it'll just shut you down. Which is great, I think, to some degree you need safeguards at the natural language level, but I think what you also need clearly is safeguards at the biological sequence level too. So you need models that not just can understand language and the trajectory of your chat, but to understand the substrate itself is the next step and ultimate limit. Right, if you can have models that can understand whether the sequence is pathogenic or a virus, that's the level of defensive capabilities that you want. And then being able to push that out into surveillance systems, national security, being able to monitor emerging sequences in the environment — this is what that kind of capability makes possible. And then we think bringing this frontier technology to that community as well is also important, just as important as using this for human health, which is what we primarily focus on.

你提到了这样一个演进过程——有 HyenaDNA，然后是 Evo、Evo 2，现在是 Omni。你能稍微谈谈 Evo 和 Evo 2 吗？它们在很多任务上都难以击败那些专用模型，而在博客文章里你谈到，现在 Omni 在广泛的任务上实际上能够超越它们。所以我们能不能稍微聊聊这个？

There was this evolution that you mentioned — there's HyenaDNA, and then there's Evo, Evo 2, and now Omni. Can you just talk a little bit about Evo and Evo 2? They struggled to beat sort of specialized models across many tasks, whereas in the blog post you talk about how across a wide range of tasks Omni is actually able to outperform them now. So can we just talk a little bit about that?

是的，Evo 在很多方面都让大家很感兴趣。它展示了应用于多种模态的潜力，但它在很多方面仍然不如专用的 DNA 模型，尤其是在人类遗传学或基因组学上。所以尽管 Evo 有竞争力，它仍然不是最先进的，或者说没能推动那个指针。所以社区里有些人就想，既然我能用这些更小、更专用的模型，为什么还要用一个巨大的 LLM 呢？所以我们想用 Omni 做的事情，在众多目标之中，最先要做的之一就是展示中间训练和后训练，或者广义上的对齐，这个理念。我们看待生物领域、尤其是基因组学领域的语言模型的方式是，你之前大多只见过训练好的基础模型。所以它们是预训练过的，但基本上没有对齐。所以在自然语言的类比空间里，就好比你把预训练全做了，但要让它真正在现实世界中有用、回答用户真正想要的问题、并以他们觉得真正有信息量的形式呈现，还需要做一堆对齐、后训练和中间训练，才能让模型达到可上生产、真正有用的状态。所以我们觉得 Evo 只是展示了预训练的潜力，而 Omni 是朝着真正让它对科学家这样的人有用迈出的一步。所以我们花了很多时间在对齐以及中间训练和后训练上，本质上就是把任务以人们通常想要理解基因组学的那种形式展示出来。你知道，给定一个野生型和一段突变序列，帮我找出因果变异，对吧？这类问题和人们可能想分析基因组学的形式，并不一定会自然而然地从预训练中轻易涌现出来。预训练就是这个下一个词预测任务，或者填充任务。基本上我把它想成，你在预训练里教给它的那种原始的造模式能力，但接着要把那些学到的嵌入或特征拿过来，指向特定任务，或者其实是一堆任务，并对齐它——意思是让它以对你而言有意义的方式向你展示输出——这需要一点调教和操作。到目前为止，在前沿文献里，是那些实验室在推动自然语言社区里的这项研究。

So yeah, Evo was intriguing to folks in many ways. It showcased the potential for applying to multiple types of modalities, but it still in many ways underperformed the specialist DNA models, especially on human genetics or genomics. And so although Evo was competitive, it still wasn't state-of-the-art or kind of pushing the needle. And so some parts of the community thought, why use a giant LLM when I can use these smaller, more specialized models? And so what we wanted to do with Omni was, amongst many things, but one of the first things was to showcase this idea of mid-training and post-training, or broadly alignment. So the way we think about language models in bio, and genomics in particular, is that mostly you've only seen base models trained. So they're pre-trained, but they're basically unaligned. So in the analogous space for natural language, it's like you're doing all the pre-training, but to make it actually useful in the real world and answer questions that users actually want, and in the form factor they find actually informative, there's a bunch of alignment and post-training and mid-training done to get the models to be production-ready and actually useful. And so we felt Evo was just showcasing the potential of that pre-training, but Omni is a step of actually making it useful for folks like scientists. And so we spent a lot of time on alignment and mid- and post-training, which is essentially showcasing tasks in the form that people generally would want to understand about genomics. You know, given a wild type and a mutation sequence, help me find the causal variant, right? These types of questions and form factors for how you might want to analyze genomics don't just emerge necessarily easily on its own from pre-training. Pre-training is this next-token prediction task or infilling. And basically I think of it as the raw pattern-making ability that you're teaching it is in the pre-training, but then taking those learned embeddings or features and pointing at specific tasks, or a bunch of tasks really, and aligning it — meaning have it show you the output in a way that is meaningful to you — takes a little bit of teasing and manipulating. So far, the frontier literature labs are the ones that drive that research in the natural language community.

所以我们想把大量这类研究以及更多的东西带到基因组学里。我认为 Omni 其实只是一个预览，用来展示这种潜力。哪怕你只做到一点点，我们也很惊讶地发现它开始达到最先进的水平，并且在推动边界——不只是像 Evo 那样在多个任务上广泛地有效，而是真正在推动每一个领域的前沿：变异效应预测、疾病的因果突变。它开始真正对人类基因组学有用了。我们非常兴奋能把这个分享给大家。从这个意义上说，它只是一个预览，因为我们当时还在积极训练，还在把额外的技术融入模型，比如额外的模态。但我觉得我们当时实在太兴奋了，想更快地把它交到大家手里。我们早期收到的一些反馈和兴趣——很多医院系统、非营利组织，它们拥有海量的基因数据——比如它们知道患者有某种病症或症状，但搞不清 DNA 的哪些部分导致了它。所以这些 VUS，也就是意义不明的变异，我们非常兴奋能把这类模型应用上去，真正帮助诊断很多这样的患者。这是一个真实应用场景的例子。

So we wanted to bring a lot of that research and more to genomics. I think Omni is really just a preview to showcase that potential. Once you do that even just a little bit, we were surprised that it did start being state-of-the-art and pushing the boundaries — not just being effective at multiple tasks broadly like Evo was, but actually pushing the frontier of each of those areas: variant effect prediction, causal mutations for disease. It could start actually being useful for human genomics. We're really excited to share that with folks. It was just a preview in that sense because we were still actively training and incorporating additional techniques into the model, like additional modalities. But I think we were basically too excited and we wanted to get this into the hands of folks faster. Some of the feedback we got in early interest — lots of hospital systems, nonprofits that have tons of genetic data — for example, they know there's some kind of condition or symptom for the patient, but they can't figure out which parts of the DNA are causing it. So these VUSes, or variants of unknown significance — we're extremely excited to apply these models to and actually help diagnose a lot of these patients. That's one example of a real use application.

我想待会儿再聊更多应用，但我很好奇，从技术角度来说，对齐一个基因组模型是什么样子、意味着什么？我能想象对于大语言模型，有一种自然的思维链，你知道，强化学习，这里有一个清晰的范式。我很好奇对于基因组模型它是什么样子，也许更贴切的描述是像用另一种语言做微调。

I'd like to talk more about the applications in a bit, but I am curious just from a technical standpoint: what does it look like and what does it mean to align a genomics model? I can imagine with a large language model there's a sort of natural chain of thought, you know, RL, there's a clear paradigm here. I'm curious what it looks like for a genomic model, which is maybe better described as something like fine-tuning in a different language.

是的。我的意思是，在很多方面可以把它描述为微调，但接着要引入——我会说关键组成部分是输入的正确结构。也就是按某种序列把它们喂进去，让模型意识到有某个任务被要求完成。所以会混合一些特殊 token——你可以把它想成，如果你要做疾病 A 的预测，就期待这个特殊 token。有点像一种提示它的方式。如果你要做设计，就用另一个特殊 token，然后像思维链那样展示示例，意思是展示一串期望的输出及其轨迹。这说得有点模糊，是故意的，因为这是我们仍在开发的秘方的一部分，我想随着时间推移我们会展示越来越多。但在很多方面它确实模仿了自然语言社区的很多做法。很大一部分就是微调。但真正来说，它也是挑出你希望它专注的特定数据集，然后以特定方式组织问题或任务。这与预训练相对——预训练其实就是把它喂进去，把所有东西喂进去，只做下一个词预测，或者如果你做掩码语言建模就做掩码填充，它没有这种问答式结构的概念，即你有一个问题、一个提示，然后一个输出。你可以把中期训练和后训练看作开始展示：给定这类输入，我期待这类输出，无论是分数、预测分数还是设计。这大体上就是中期和后训练。然后后训练也包括强化学习之类的东西。但我认为更大的步骤是引入问答的结构。

Yeah. I mean, in many ways one could describe it as fine-tuning, but then introducing — I'd say the key components are the right structure of the inputs. So feeding them in a certain sequence so that the model is aware that a certain task is being asked of it. So there's a mix of special tokens — you can think of it as, if you're going to do disease prediction for disease A, expect this special token. Just kind of like a way to prompt it. If you're expected to do design, have another special token, and then showcase the examples kind of like in a chain of thought manner, meaning showcase a sequence of desired outputs and the trajectory of it. This is a little vague, sort of intentionally, because it's part of our secret sauce that we're still developing, and I think over time we want to showcase more and more of it. But in many ways it does mimic a lot of the natural language community. A lot of it is fine-tuning. But really it's also carving out specific data sets that you want it to focus on and then structuring the questions or tasks in specific ways. As opposed to pre-training — pre-training is really just feeding it in, everything in, and just doing next-token prediction or mask infilling if you're doing masked language modeling, and it has no sense of this Q&A type structure where you have a question, a prompt, and then an output. You can think of mid and post training as starting to showcase: given this type of input, I expect this type of output, whether it's a score, a prediction score, or a design. That's largely the mid and post training. Then post-training also includes things like reinforcement learning too. But I think the bigger steps are introducing structure of question and answering.

这些模型是有多个头、针对特定任务的，还是你在训练一组头之类的，能同时回答多个问题，只是改变输入 token 之类的？

Do the models have multiple heads that you are task-specific, or are you training one set of heads or whatever, that can answer multiple questions at the same time, just change the input tokens or whatever?

是的，大体上有点——我觉得我们在这一点上是灵活的。有时你可以用不同的头，但对我们来说，理念是更多地统一。所以我想在早期实验里我们确实用了不同的头，但在某些情况下不同的头表现更好；在某些情况下统一的、你知道，单个头、单个模型表现更好。所以我觉得我们在这一点上是灵活的，但大体上我们前进的方向是单一模型。这么做的原因、特别的动机，是我们试图解锁大量的模态、大量的泛化，而我认为我们越能让这些模型统一，就越能看到更多涌现能力出现。这在很大程度上正是推动 DNA 工作的原因。我们觉得很多模型都专门针对 DNA 更下游的其他模态，比如 RNA、蛋白质或分子。我们在很大程度上觉得 DNA 是基础，从 DNA 你可以学到很多其他模态——可能是全部。我把其他模态看作你展示给模型的某种额外上下文。这是我在脑子里从哲学上的看法。但没错，单一模型跨模态统一并扩展，这正是实验室所构建并努力追求的方向。

Yeah, broadly it's a little — I think we're flexible on this. Sometimes you can use different heads, but the idea for us is to unify more. So I think in early experiments we did have different heads, but in some cases the different heads do better; in some cases the unified, you know, a single head, a single model does better. So I think we're flexible on that, but broadly the direction that we are moving toward is a single [model]. The reason, the special motivation for that, is that we're trying to unlock a lot of modality, a lot of generalization, and I think the more unifying we're able to make these models, I think that's when you see more emerging capabilities happen. In large part that's what motivated the DNA work. We felt like a lot of models were specialized into other modalities a little more downstream from DNA, like RNA or proteins or molecules. We largely felt DNA is the foundation, and that from DNA you can learn a good deal of other modalities — potentially all of them. And I think of other modalities as sort of additional context that you're showing the model. That's how I kind of view it philosophically in my head. But yeah, the idea that single models unifying across modalities scales is what the lab builds and builds toward.

你能不能带我们过一遍你在博客里谈到的几个任务，解释一下——记得为只听音频的听众做点讲述——但谈谈其中一些最重要的结果，我们也许可以稍微深入一点。

Can you walk through a few of the tasks that you talk about in the blog and just explain — and remembering to narrate for the listener-only audience — but talk about some of these topline results, and we can maybe dig into them a little bit.

当然。我们认为在这次 Omni 的发布中，一个非常值得展示的领域是理解变异及其效应。本质上，在 DNA 里，就是某个位置发生了变化——你知道，把那个字母换成你基因组里另外三个字母之一。在 DNA 里，有时它会致病，有时它什么都不做——实际上很多时候它什么都不做。但你基因组里有特定的区域，如果你在那里有一个不同的变异或不同的字母，它就可能致病。而在很多情况下，因为基因组中这些变化在三十多亿个字母上的组合实在太多，对临床医生和科学家来说，我们实际上只知道其中很小一部分变异是否致病。所以有一些来自收集变异的人——不同的医院系统和诊所——的基准，有些是已知的，有些仍然未知。人们从 ClinVar 或 TraitGym 创建了一些基准。基本思路是：给定 DNA 中的一个突变，你能判断它是否会致病吗？这对 DNA 模型来说是一个非常好的设定，因为它们是概率性的。所以当你做出一个改变时，它基本上能改变自己的置信度，或者说预测下一个字母的概率。

Sure. One of the areas that we thought was really interesting for showcasing in this particular release for Omni was on understanding variants and their effects, which are essentially, in DNA, a change in a position — you know, changing the letter to one of the other three letters in your genome. In DNA, sometimes it can cause a disease and sometimes it doesn't do anything — actually many times it doesn't do anything. But there are specific areas in your genome that if you have a different variant or a different letter there, it can cause disease. And in many cases, because the combinations of these changes in the genome over three billion letters is so vast, for clinicians and scientists we actually know only a very small portion of which variants are causal to disease or not. So there are these benchmarks from folks who collect variants — different hospital systems and clinics — and some are known and some are still unknown. Folks have created some benchmarks from ClinVar or TraitGym. Basically the idea is: given a mutation in DNA, can you tell if it's going to cause a disease or not? So this is a very good setup for DNA models because they're probabilistic. And so when you do make a change, it basically can modify its confidence or the probability of predicting a next letter.

在这个案例中，我们利用了这些模型的可预测性。它们本质上已经见过并接受了大量 DNA 的训练，尤其是人类 DNA，因此它们能够理解什么是常见的，而通常在不同人群中常见或保守的，往往意味着更健康。所以如果它不那么常见，你可以这样理解：如果它不那么常见，模型就能捕捉到它，并预测它可能具有致病性或导致疾病。

In this case, we've leveraged the predictability of these models. They've essentially seen and been trained on so much DNA, particularly human DNA, that they kind of understand what's common in cis and usually common or conserved across different other folks typically means healthier. And so if it's less common, you can think of it this way: if it's less common, the model can pick it up and predict that it's potentially pathogenic or disease-causing.

我们采用了一些这样的基准测试，当你引入一个变异或突变时——变异中有不同类型的突变。有时你可以完全删除一个字母，你可以直接翻转它，你可以移除大段 DNA。但这些通常是单变异。在这种情况下，这正是之前的 DNA 模型真正挣扎的地方，尤其是在人类身上。所以我们展示了它不仅对人类具有竞争力或能力，而且在许多情况下达到了最先进水平，实际上在大多数这些案例中都是最先进的。

We've taken some of these benchmarks, and when you introduce a variant or a mutation—there are different types of mutations in variants. Sometimes you can delete a letter altogether, you can just flip it, you can remove big portions of the DNA. But these are generally single variants. And in this case, this is where previous DNA models really struggled, especially on humans. And so we showcased that not only is it competitive or capable for humans, but in many cases it's state-of-the-art, and actually in most of these cases it's state-of-the-art.

我认为令人兴奋的部分是，其他目前处于前沿的模型在基因组中的某些区域仍然落后相当多。所以在基因组中有编码区和非编码区，比如蛋白质区域——蛋白质区域指的是实际上编码蛋白质结构的区域，而其他区域则做其他事情，比如调控哪些基因被表达。

And I think the exciting part is that areas where other models that were currently on the frontier still lagged behind quite a bit in terms of where in the genome. So in the genome there's coding and non-coding regions, like protein areas—protein regions meaning areas that are actually the coding the structure of a protein, versus other areas that do other things like regulate what genes are expressed.

是的。

Yeah.

所以这些非编码区主要是调控性的。它们控制某个基因的使用量或使用时机，或者开启它们。在许多情况下，这些非编码区、这些调控区中的变异、突变，更难预测它们是否导致疾病。我认为我们用 Omni 构建的新一代模型令人兴奋的地方在于，这正是我们擅长的领域，尤其是模型能够捕捉突变，并能够区分在这些非编码区尤其是长程区域中它是否致病。

And so these non-coding regions are largely regulatory. They kind of control how much or when to use a certain gene or turn them on. And in many cases these non-coding regions, these regulatory regions, variants there, mutations there, are much harder to predict if they cause disease or not. And I think what's exciting about the new generation of models we're building with Omni is that that's where we shine, especially the models are able to pick up mutations and be able to distinguish if it's disease-causing in these non-coding and especially long-range areas.

所以我认为这对许多遗传学家来说特别令人兴奋，他们一直难以使用传统的生物信息学工具或统计方法，因为这些方法主要关注编码区，而编码区只占基因组的约 1.5% 到 2%。而事实证明，许多甚至大多数疾病都位于这些非编码区。因此，人们一直真正渴望构建能够实际捕捉这些非编码区中致病变异的模型。

So I think this is particularly exciting to a lot of geneticists that have struggled to use traditional bioinformatic tools or statistical methods because they've largely focused on coding regions, which is only about one and a half to 2% of the genome. And it turns out many if not most of the diseases are in these non-coding regions. And so there's been a real desire to build models that can actually pick up these variants of disease-causing variants in these non-coding regions.

我有几个问题。第一个，趁我们在这里，为听众们，博客文章中这个基准图表上有一列叫做 Borzoi 参考编号四。我想也许为了提供一些历史背景，你能谈谈这一列代表什么吗？也许这也有助于为右边的 Omni 列提供背景。

I have several questions. So this first one, just while we're here, for the listeners, there's this column on this benchmark chart called Borzoi reference number four in the blog post. I think for maybe some historical context, could you talk about what this column represents? And maybe this also helped give context for the Omni column on the right.

是的，说得好。我们在这个基准测试中展示的，实际上是选取了深度学习方面以及传统方法中一些代表性的模型或最强的模型。所以我们有 Evo 2，这是我们团队之前研究的最新基因组模型。然后 Borzoi 也是一个 DNA 模型，但类型非常不同。本质上它是一个监督模型，从 DNA 预测功能性基因组轨迹。所以它本质上也是多模态的，但它不是语言模型。所以它不像下一个词预测那样预测。它从 DNA 序列直接到功能性基因组轨迹，比如染色质柔性或基因表达。

Yeah, great point. So what we show in this benchmark here is really taking some of the representative models or the strongest models in the deep learning side and also in the traditional methods. So we have Evo 2, which is the latest previous genomic model that our team had worked on. And then Borzoi is also a DNA model but a very different kind. Essentially it's a supervised model that predicts from DNA functional genomic tracks. So it too is inherently multimodal, but it's not a language model. So it doesn't predict like a next-token prediction. It goes from a DNA sequence directly to a functional genomic track like chromatin flexibility or gene expression.

而这些轨迹已经被人们通过写论文等方式进行了广泛的标注。

And these tracks have been annotated extensively by people writing their dissertations and whatever.

是的，是的，对。所以那里的巨大区别在于它是一个监督任务，对吧？所以它需要标注的输出。而在我们的案例中，这些语言模型不需要标注的输出。所以你在未标注的序列上进行原始预训练。那么为什么这令人向往呢？嗯，有更多的数据，更多的基因组数据没有被标注。实际上，大部分——在很多方面几乎全部，几乎所有——都没有被标注。所以能够以无监督的方式学习是非常令人向往的。对我们来说，我们想展示在原始基因组序列上进行预训练的好处，并将其与 DNA 其他领域的最先进模型进行比较。

Yeah. Yeah, right. So the big difference there is that it's a supervised task, right? So it requires labeled outputs. And in our case, these language models, they do not require labeled outputs. So you're doing raw pre-training on unannotated sequences. So why is that desirable? Well, there's a lot more data, a lot more genomic data that's not annotated. Actually, most of it—pretty much in many ways all of it, almost all of it—is not annotated. And so being able to learn from an unsupervised manner is hugely desirable. For us, we wanted to showcase the benefit of pre-training on raw genomic sequences and comparing it to state-of-the-art models in other spaces in DNA.

所以你的预测是——当你说它是无监督的，无监督的特性是如何运作的？比如你如何将你模型的输出转换为可操作的排名分数之类的？我之前谈到的是——

And so your prediction is—when you say it's unsupervised, how does the unsupervised property work? Like how do you convert the output of whatever your model is to an actionable ranker score, whatever? What I talked about before is—

训练 Omni 模型时，它是预训练的，所以是无监督的。但当你将它指向特定任务时，有一个监督步骤。所以只是取一个较小的、有标注的数据集。但本质上我们做的是使用似然分数。也就是语言模型的原始输出，你可以把它想成预测下一个字母的概率。我们基本上可以展示突变相对于野生型的概率分数、似然分数。也就是在这个特定案例中参考基因组中的序列相对于突变。然后我们会得到两个分数。然后你可以把它想成使用两者的比率来展示基本上这个突变与正常或基线有多不同。然后那就是——你可以把它想成模型能够使用和利用的一种惊讶因子，然后我们可以用它来基本上为是否患病打分一个实际的预测。这说得通吗？

To train the Omni model, it's pre-trained, so it's unsupervised. But when you're pointing it at a specific task, there is a supervised step. So it's just taking a smaller data set that is labeled. But essentially what we're doing is using the likelihood scores. So the raw outputs of the language model, which basically you can think of it like a probability for predicting what the next letter is. We can essentially showcase the probability score, the likelihood score, for the mutation versus the wild type. So the sequence in the reference genome versus the mutation in this particular case. And then we'll have two scores. And then you can think of it as using a ratio of the two to showcase basically how different—how different is this mutation from a normal or baseline, basically. And then that's—you can think of it as like a surprise factor that the model is able to use and leverage, and then we can use that to essentially score an actual prediction for disease or not. Does that make sense?

是的。所以 Omni 是自回归的吗，还是扩散模型，或者是你不能告诉我的其他东西？

Yeah. So Omni autoregressive is that, or is it diffusion or something else that you can't tell me?

是的，这次我们不描述确切的构成。

Yeah, this time we're not describing the exact makeup.

Evo 是自回归的，我认为它是第一个大规模自回归的。

Evo was autoregressive, and I think it was the first large-scale autoregressive.

所以对我们来说，我们不把自己束缚在特定的训练目标上。我们基本上使用工具箱里的每一个工具。

And so for us, we don't tie ourselves down to a specific training objective. We use every tool in the toolbox, essentially.

好的，但对于这个特定的基准测试，你只是使用 token 的似然分布，有些 token——模型认为这些不太可能，这可能是由于某种进化约束，比如这并不经常出现，因为它不经常出现在基因组中，它可能会导致问题，人们将无法存活，等等。

Okay, but for this specific benchmark, you're going along and you're just using the likelihood distribution of the tokens, and some tokens are—the model thinks these are unlikely, and that is probably because some evolutionary constraint, like this doesn't show up often, and because it doesn't show often across genomes, it is probably going to cause problems, and people will not survive, so on.

所以你觉得那基本上就是你们的排序指标之类的？

And so you think that is basically your ranking metric or something?

对，这是对模型如何思考它的一种解读，和自然语言里你可以描述的那套范式非常相似。

Yeah, it's one interpretation of how the model's thinking about it, and very similar to in natural language you can describe the same kind of paradigm.

而且在中期训练和后训练里还有额外的处理来利用更多东西，我想，因为我们可以教它特定的结构和基准，让它能在你刚描述的那套——也就是自然界里常见的东西——之上继续构建。但同时也因为疾病变异预测是一个特定任务，模型在中期训练期间被引入了额外的训练，本质上是为了增加额外的学习能力。

And there is additional case in mid and post training to leverage more than that, I suppose, because we can teach it specific structure and benchmarks so that it can build on top of what you just described, which is like what's common in nature. But also because it's a specific task for disease variant prediction, the model has additional training introduced during mid-training to add additional learning power, essentially.

能举些例子吗？这大概在某种程度上是秘方，但你能大致讲讲那是什么样的吗？你会往里放哪些东西？

What are some examples of that? Again, probably secret sauce to some extent, but can you give just a gist of what that looks like? What are the kinds of things you would throw in there?

我们实际上会把分数本身放进去。所以，就像我提到的，一个比值，野生型对突变的比值。我会说那更像是一种零样本方法，你甚至不需要做任何中期训练，这正是 Evo 2 在这一列里所做的。所以 Evo 2 本质上没有经过微调。有一个 EV 列，人们基本上是在用 Evo 2 的分数做微调。那是 Goodfire 做的，也是我们非常尊敬的人，他们展示了这些模型在你能微调它们时也能达到最先进水平，而不只是零样本。然后我们的模型引入了比那更高一层的做法，不只是微调，还把结构——我说的结构是指这些基准的格式——引入模型本身，也就是中期训练期间那种问答式的格式，这正是给我们带来额外提升的原因。

We would actually throw in the score itself. So, you know, like I mentioned, a ratio that's a ratio of wild type versus mutation. I would say that's more of a zero-shot method where you don't even have to do any mid-training, and that's what Evo 2 is doing in particular in this column. So Evo 2 is not fine-tuned, essentially. There's an EV column which people are basically fine-tuning using the scores from Evo 2. That's from Goodfire, and that's also folks that we greatly respect, and they kind of showcase that these models are able to be state-of-the-art when you can fine-tune them as well, not just zero-shot. And then our model is introducing sort of a step above that, not just fine-tuning but also introducing structure — by structure I mean the format of these benchmarks — into the model itself, that Q&A-style formatting during mid-training, which is what gives us an extra boost.

我看到了额外的提升，但我觉得另一个好处——我们在博客里没有太强调，但我认为它对科学家的实际使用非常方便——就是你做这件事时不需要取出嵌入、再在上面接一个头、然后做一些快速回归，那是额外的一步，是一道额外的门槛。你能想象 ChatGPT 如果每次你问一个问题都得为某个特定领域微调它吗？我们做到的是让模型在中期训练期间足够灵活，能同时被训练在许多任务上。所以那个微调、也就是训练嵌入的最后一步，就不必做了。到那个阶段它就是开箱即用的。你只要用某种格式去提示，那种格式会告诉它你要做哪个任务，然后它就会以期望的格式输出答案，基本上就是这样。

I see an extra boost, but I think the other benefit too, that we didn't emphasize too much in the blog, but I think it's really convenient for practical use for scientists, is that you're doing this without taking the embeddings and then tapping on a head and then doing some quick regression, which is an extra step. It's an extra hurdle. Can you imagine ChatGPT if every time you ask a question you had to fine-tune it for a certain domain? We've done it so that the model is flexible during mid-training to be trained on many tasks at once. And so that fine-tuning, that last step of training the embeddings, doesn't have to be done. It's out of the box at that point. You just prompt in a certain format, and that certain format tells it which task you're going to do, and then it'll output the answers in the desired format, basically.

你们在设计这套后训练方案时，有多小心去避免与 ClinVar 训练集之类的数据集发生数据泄漏？你有多确信没有数据泄漏，无论是意外的还是上游的？因为如果这些序列中有很多也出现在你们的训练数据里，哪怕是以无监督的方式，我也不会感到意外。

How careful were you in designing this post-training scheme to avoid kind of data leakage with the ClinVar training and so on, like these data sets? Like how confident are you that there is no data leakage, either accidental or something upstream? Because I would not be surprised if a lot of these sequences showed up also in your training data, even in an unsupervised sort of way.

对，简短的回答是我们非常清楚数据泄漏的风险，而且要做到不误导、要小心是极其困难的。所以我们有生物信息学家，他们基本上能梳理数据、做整理、去重、比对序列，确保那些可能与基准里内容相似的东西不在里面，如果确实在，我们就把它移除。所以我们实际上有步骤对数据做相当广泛的质控。

Yeah, the short answer is we're extremely cognizant of the risk of data leakage, and it's extremely hard to not mislead or to be careful. And so we have bioinformaticians that are able to basically comb through the data and curate and dedup and align sequences to make sure that things that are similar potentially to what's in the benchmark are not there, and if they are there, we remove it. And so we actually have steps to QC the data quite extensively.

对。太酷了。我想也许在我们继续之前，看到有这些监督方法真的很酷——之前其中好几个数字，怎么说呢，要么在误差范围内，要么就是直接超过了之前的东西。而现在你们又显著地改进了它。

Yeah. Cool. I guess maybe before we move on, I think it's really cool seeing that there are these supervised methods which previously several of these numbers were, let's say, within the error bars if not just straight up beating what came before them. And now you have significantly improved upon that.

对，我们超级兴奋。我觉得很长一段时间里，有这么一个领域，或者说另一种叫 CAD 的方法，它一直是最先进的，而且它最先进是有原因的，因为它在设计上会取最好的方法来做集成，对吧？所以他们会拿——即使最好的方法是另一个之前的模型——他们会把它和比如 SVM 混在一起，把能用的全用上。所以你就能明白为什么它会是最好的，对吧？所以那就是我们的门槛。我们想，如果他们要把能用的全用上，我们就不能只挑一个模型然后说我们比那个好。我们得打败现在人类所能达到的极限，在所有方面都是。所以，对，我们的研究人员把目标定在了那上面，看他们是否真的能在每一种方法上都提升性能。

Yeah, we're super excited. And I think for the longest time there's this area or this other method called CAD, which has been state-of-the-art, and state-of-the-art for a reason, which is that by design they'll take the best methods and kind of do an ensemble, right? So they'll take — even if the best method is another previous model — they'll mix it with like an SVM and just throw the kitchen sink at it. And so you can see why it would be the best, right? And so that was the bar for us. We're like, if they're going to throw the kitchen sink at it, we're not going to cherry-pick one model and say we're better than that. We need to beat what's possible, humanly possible now, like across everything. And so, yeah, our researchers were setting their sights on that to see if they can actually improve performance across every method.

我很想看看——里面有一些关于思维链的讨论，我第一次听说的时候有点烧脑。我真的很想听听那到底是什么意思。我得仔细翻博客文章才能真正理解。

I'd be interested to see — there's some discussion of chain of thought, and that broke my brain a little bit when I was first hearing about that. I'd be really interested to hear about what that even means. I had to pore through the blog post to really understand that.

对。对。我觉得这真的只是让我们一窥我们认为设计能力可以走向何方、并且对人们来说更易用。所以思维链源自自然语言领域。我相信是 OpenAI 的 Jason Wei 展示了最早的例子，那里的突破实际上是展示了这些语言模型在你把过程写出来、也就是展示你如何得出结论或论证的步骤时，表现会更好。结果发现即使只是简单的步骤，它也只是给了模型一个机会——你知道，也许这有点哲学意味，谁知道它到底为什么有效——但本质上就是喂进更多的 token，给它更多打草稿的空间去思考。所以人们就认为这是这些语言模型推理的开端，这种通过自己对自己思考来得到答案的能力。所以它在语言上似乎相当成功，非常成功。这就是为什么你有很多智能体只是花掉海量的 token，对吧？就是把过程写出来，对吧？在很多方面它都源自这套思维链范式。在生物学里，我们看到的很少。

Yeah. Yeah. I think this is really just a taste of where we think the design capabilities can move toward and be way more usable for folks. So chain of thought stems from the natural language community. I believe Jason Wei at OpenAI showcased the first examples, and really what the breakthrough there was, was showcasing that these language models performed better when you just show your work, essentially, or showed the steps of how you came to a conclusion or an argument. And it turns out even if they were like simple steps, it just gave the model a chance — you know, maybe it's sort of philosophical, who knows exactly why it works — but essentially feeding more tokens in and giving it more scratch space to think. And so people just think this is the beginning of reasoning for these language models, this ability to get to an answer by thinking to itself, by itself. And so it seemed quite successful in language, very successful. And that's why you have a lot of agents that just spend tons of tokens, right? Just showing its work, right? In many ways it stems from this chain of thought paradigm. In biology, we saw very little of that.

我们在蛋白质设计里已经开始看到一些苗头，所以想推进并展示这一点。首先探索：这在 DNA 里可能吗？在基因组学里这到底意味着什么？因为你其实没有词能描述它的思考。那么如何把同样的范式引入 DNA 语言模型？我们做的是一个在很多方面更简单的版本。我们有一个 RNA 适配体的数据集。就把它想成这些期望的序列，带有某种适应度分数。所以我们取了这个大型数据集，有 RNA 输入和适应度分数。适应度分数：高就是好，对吧？最简单的版本。这是一个大数据集。我们想展示的是：如果我们按一系列步骤向模型展示逐步更好的 RNA 及其分数——先给低分，然后逐渐往上走——模型能否自己延续这个轨迹？然后在最后一步，它能否自我优化到它能得到的最佳分数？这就是实验。我们能做吗？所以我们取了一个大型适配体数据集，留出了一部分表现最好的，只给它看较低的那些，但我们对它进行了排序。所以我们展示了较低分数的 RNA 适配体，然后逐步提高，再让模型继续这个模式。结果它能够重现一些我们尚未展示给它的更高分数。我们目前正在湿实验室验证。所以这里没能展示，但我们想知道：它能否不仅在硅基上做到——它能。它展示了能够延续这个轨迹，并设计出具有更高适应度分数的合理适配体。现在我们认为，如果这在实验室里可行，这显然是一个极其有价值的范式，几乎可以应用于所有其他类型的电子序列。通常你有的就是：序列，以及某种适应度分数或期望输出。如果我们能让模型最终学会这种结构，基本上只需展示一系列逐步更强的序列，模型就能预测其余部分。这是一个非常强大的范式。

We started to see some of that in protein design a little, and so we wanted to push that and showcase that. First, explore: is that possible in DNA? What does that even mean in genomics? Because you don't really have words that describe its thinking. So how do you take that same paradigm and introduce it to a DNA language model? What we did was a simpler version in many ways. We had this data set of RNA aptamers. Just think of it as these desired sequences with some kind of fitness score associated with them. So we took this large data set that had RNA input and a fitness score. The fitness score: high is good, right? The simplest version. It's a big data set. What we wanted to showcase was: if we show the model progressively better RNAs in a series of steps with its score—so you have low scores first and then you gradually move up the chain—can the model continue that trajectory on its own? And then in the final step, does it self-optimize to a point where it's like the best score it can get? That was the experiment. Can we do that? So we took a large data set of aptamers, we held out a portion of the best performing ones, and we showed it only the lower ones but then we ranked it. So we showcased lower scores with the RNA aptamer and then progressively got higher, and then asked the model to continue with that pattern. It turns out it was able to recapitulate some of those higher scores that we had not shown it yet. We're actually in the process of validating in the wet lab right now. So we didn't get to show it here, but we wanted to know: can it not just do this in silico, which it can? It showcased that it was able to continue this trajectory and create design plausible aptamers with higher fitness scores. Now we think this is—obviously if this works in the lab—a hugely valuable paradigm that can be applied to pretty much every other type of electrical sequence. That's what you usually have: you have sequence, you have some kind of fitness score or desired output. And if we can get models to eventually learn that structure and basically just show a series of progressively stronger sequences, the model can then predict the rest. That's a very powerful paradigm.

老实说，我对此有点惊讶，尤其是这个任务看到了强劲的提升。也许是我个人的偏见，但 RNA 在约束结构方面缺乏良好的共进化数据，这有点臭名昭著，对吧？对于病毒基因组，通常有很强的进化压力，但对于哺乳动物，通常 RNA 没有进化压力。我认为社区已经非常清楚地看到了这一点：主要的压力是 RNA 会编码——会携带信息，编码信息。

Honestly a bit surprised about this, that specifically this task saw strong improvement. Maybe my personal bias is coming in here, but RNA is somewhat notorious for not having good co-evolution data in terms of constraining structures, right? For viral genomes, often times there is strong evolutionary pressure, but for mammalian, usually RNA does not have evolutionary pressure. And I think the community has seen that very clearly: the predominant pressure is like RNA will code—will carry information, coding information.

是的。

Yeah.

所以我在想：基因组携带很多不同的信息。它们编码蛋白质。它们有调控元件，不同类型的基因组有不同的结构。所以我想知道，你认为这种能力可能是在这个语言模型的哪里涌现出来的？

So I'm wondering: genomes carry lots of different information. They code for proteins. They have regulatory elements, and different types of genomes have different types of structure. So I'm wondering where do you think this capability might have emerged in this language model?

这是个好问题，老实说我也不确定。我们也很惊讶。第一，因为模型是在 DNA 上预训练的，而它只是在 RNA 上非常非常小规模地中期训练。是的，我认为你关于 DNA 有很多进化效应或信息的直觉可能就是原因。所以我认为这暗示了为什么我们认为先在 DNA 基因组上预训练如此重要，然后再在上面添加其他模态，因为你会获得大量迁移，而模态泛化正是我们努力的方向。你知道我没谈到的是，对公司而言，也是这个想法：朝着通用生物智能构建，在那里我们统一许多不同的所谓生物语言或模态。归根结底它们都源于 DNA，我认为人们没有充分利用这个事实。通常是非常专门的领域特定模型或模态特定模型，没有利用其他模态中许多固有的共享结构。一个例子是，当人们谈论虚拟细胞时——这有点跑题——但他们往往只关注 RNA 和转录本，并希望泛化到描述整个细胞，但显然细胞远不止这些。在我看来，如果你想学习一个系统，你想从那个世界或系统的所有信号或传感器中学习。如果是一个细胞，你想融合，你想理解 DNA，你想理解代谢组学、表观基因组学、蛋白质组学，那时你才更接近所谓的虚拟细胞。在我们看来，我们甚至不想止步于细胞，而是想融合整个生物学的所有这些传感器。这会把我们带到理解生物学每一个细节的超级智能吗？谁知道呢？但我确信这种范式会让我们比现在走得远得多。这就是我的标准：你能做出比现在有用得多的东西吗？

That's a good question, and honestly I'm not sure. We were surprised too. One, because the model is pre-trained on DNA and it's really just like mid-trained on RNA very, very—you know—in a small way. Yeah, I think your intuition about DNA having a lot of evolutionary effects or information is probably where. And so I think this is hinting at the idea of why we think it's so important to pre-train on DNA genomes first, and then sort of add additional modalities on top, because you get a lot of transfer and the modality generalization is the thing that we're working toward. You know what I didn't talk about is a company for the company as well is this idea of building toward general biological intelligence where we are unifying a lot of the different so-called languages or modalities of biology. At the end of the day they stem from DNA, and I think people have not exploited that fact as much. It's usually really specialized domain-specific models or modality-specific models and not leveraging a lot of the inherent shared structure from other modalities. And so one example of that is like when people talk about virtual cells—this is a little bit tangent—but they tend to focus on just RNA and transcripts and they want to generalize to describing an entire cell, but obviously a cell is a lot more things than that. In my mind, if you want to learn a system, you want to learn from all the signals or sensors of that world or that system. If it's a cell, you want to fuse, you want to understand the DNA, you want to understand the metabolomics, the epigenomics, the proteomics, and that's when you get closer to like quote-unquote a virtual cell. And in our minds, we don't even want to stop at just the cell, but we want to fuse all of these sensors across all of biology. Will it get us to a superintelligence that understands every component of minutia biology? Who knows? But I am confident that this type of paradigm will get us a hell of a lot further than we are now. That's my bar: can you make something far more useful than now?

对。所以我很好奇：你们是专注于真核细胞吗？是专注于人类基因组吗？你们有没有做到病毒基因组？我的意思是，有很多 DNA 病毒，但似乎 RNA 也有可能——外面有很多 RNA 序列病毒序列，我不确定在训练方面它们会有多大不同。我很好奇，如果可以谈的话，范围是什么？

Right. So I'm curious: are you focusing on eukaryotic cells? Are you focusing on like human genomes? Have you gone so far as to do viral genomes? I mean there's a lot of DNA viruses, but it seems like plausible that RNA—there's a lot of RNA sequence virus sequences out there and I'm not sure fundamentally they would be much different in terms of training. I'm curious like what's the scope of that if you can talk about it.

是的，当然。我们对所有生命领域都感兴趣。所以这里我们特别关注人类，因为我们认为这是之前模型 Evo 和 Evo 2 不太强的领域，并且收到了很多反馈，人们问这些模型有什么用？它们无法理解人类基因组，因为语法和规则太复杂，DNA 也太嘈杂，有太多重复字符等等。所以我们想展示我们认为这实际上是有用的，可以应用于人类，而且在某些方面是最复杂中的最复杂。但我认为有机会将这些模型普遍应用于每一种生命形式。所以我们绝对对原核生物和病毒感兴趣。

Yeah, absolutely. We are interested in all domains of life. So here we focused on humans in particular because we thought this was an area that previous models Evo and Evo 2 were not as strong and sort of got a lot of feedback from folks asking, you know, what are these models useful for? They can't understand human genomics because it's too complex of grammar and rules and DNA too—it's too noisy, there's too many repeat characters and all that stuff. So we wanted to showcase we think this is actually useful and it can be applied to humans and it's sort of the most complex of the complex in some ways. But I think there's opportunity to apply these models generally to every form of life. So we absolutely are interested in prokaryotes and viral.

我们特别关注病毒基因组，尤其是生物防御和生物安全方面。我认为我们还可以从微生物生命中学到很多治疗应用。显然，对一些人来说，他们提到有人用它生成了第一个 AI 基因组——一种噬菌体。事实证明，你可以将噬菌体用于 AMR，即抗微生物药物耐药性。如果你有超级细菌感染，全球仍有约 200 万人死于细菌感染。使用病毒——设计病毒——来靶向特定细菌的想法已经存在很长时间了，尤其是在东欧。我们有可能制造出一类新的抗微生物药物，它们不像抗生素但非常相似，可以像抗生素一样使用。所以我认为我们正倾向于那些高影响力、有可能拯救生命的事情。我们不会只停留在一种基因组上。我认为我们对任何对人类有益的东西都感兴趣。

We care about viral genomes in particular, especially for biodefense and biosecurity. I think there are also lots of therapeutic applications that we can learn from microbial life. Obviously, for some folks, in particular, they mentioned that folks had used it to generate the first AI genome, a bacteriophage. It turns out you can use bacteriophages potentially for AMR, or antimicrobial resistance. If you have a superbug bacterial infection, around the world there are about 2 million deaths from bacterial infections still. The idea of using viruses—designed viruses—to target specific bacteria has been done for a long time, particularly in Eastern Europe. There's a potential to make a new class of antimicrobials that are not like antibiotics but very similar, that can be used just like them. So I think we're gravitating toward things that are high impact and have the potential to save lives. We don't stop at just one type of genome. I think we're interested in anything that's beneficial to us humans.

也许回到我关于 Solex 和 RNA 的问题，预测 RNA 进化的一种思维链。我很好奇，这个模型是在 RNA 序列上训练的，还是在可能对 RNA 结构有进化压力的序列上训练的？

Maybe going back to my question about Solex and RNA, predicting a kind of chain of thought of RNA evolution. I'm curious, was this model trained on RNA sequences or sequences which might have evolutionary pressure on RNA structure?

只在中期训练时。预训练全是 DNA 基因组。

Only during mid-training. Pre-training is all just genomes in DNA.

所以我们引入 RNA 的唯一一次就是为了这个特定任务，而且只用了这个数据集里的 RNA，甚至没有用外部的？

And so the only time we introduced RNA was for this specific task, and only RNA from this data set, so not even outside?

我明白了。所以这真的是——这个模型在某种程度上编码了 RNA 结构，也许？

I see. So this really was something along the lines of—there is something encoding RNA structure in this model to some degree, maybe?

或者要么是隐式地，也许。

Or either that or implicitly, maybe.

是的，隐式地。我会说是隐式地。

Yeah, implicitly. I would say implicitly.

我的意思是，因为这很酷。

I mean, because that's cool.

是的。因为我想说，你知道，序列正如我们从蛋白质中了解到的——应该从序列本身隐式地学习结构。

Yeah. Because I'd say, you know, sequences as we know from proteins—implicitly should learn structure from just the sequence.

是的，所以我们现在不加入 2D 或 3D 信息，但我们绝对计划这样做。

Yeah, and so we don't add in 2D or 3D information at this point, but we absolutely plan to.

是的。

Yeah.

所以为了让我理解，首先，思维链的想法——这是一个演示，但想法是，任何你能获得某种训练集，其中某种测量值依次变好，也许就是这种技术的候选。

So just so I understand, first of all, the chain of thought ideas—this is a demonstration of it, but the idea is that anything that you can get sort of a training set that has a sequentially better measurement of some sort is maybe a candidate for this technique.

是的。

Yeah.

那么你能描述一下这个特定实验吗，这样我们就能理解我们如何将思维链映射到这个数据集，以及数据集必须是什么样子？你能描述一下——对于这个，我知道这不是你的数据集，但数据是如何收集的，以便你能从适应度或其他东西准确地映射到数据集的特定阶段或部分？

And so can you just describe for this particular experiment, just so we can understand how we're mapping chain of thought to this data set and what the data sets have to look like? Can you just describe how—for this, I know this wasn't your data set, but how was the data collected in such a way that you could map accurately from sort of fitness or whatever to a particular phase of or part of the data set?

是的。所以我不太熟悉数据实际上是如何发送给我们的，或者从实验角度是如何生成的，但它们在收集时是在现实世界的湿实验室中验证的。所以我相信它有一些适应度分数。

Yeah. So I'm less familiar with how the data was actually sent to us or generated from the experimental point of view, but they are validated from a wet lab in the real world when it was collected. So it has some kind of fitness score, I believe.

如果你愿意，我可以提供一些背景。我的意思是，这里的想法是你生成一堆随机序列，然后你取这些序列——所以你有类似适配体结构，本质上像 RNA 开关，当某物结合时，它会做类似切割序列的事情，你可以使用 NGS 如下一代测序读数，非常高通量。所以你创建许多这些不同的序列。你——我猜在这种情况下，它们靶向特定的 HIV 样蛋白或基因组或其他东西，如果它结合，你基本上可以得到信号，比如你得到更多的读数。所以时间越多，漂浮的序列越多，有点像适应度越高，结合的可能性越大。然后你取那些，然后再次突变它们，你在这个上迭代，对吧？所以你有这个迭代实验，你逐步使用培养皿来基本上识别最适应的东西。所以然后你在这里做的是，你基本上在硅中做同样的实验，他们实际上在做非常高的通量，比如我认为有 10 的 11 次方或什么的序列——一些非常高的数字，在这个实验中并行探索。但关键是实验的生物学实际上在做过滤，对吧？

I can maybe provide a bit of context on this if you want. I mean, so the idea here is you just generate a bunch of random sequences and then you take those sequences and—so you have like an aptamer structure which is essentially like a switch with RNA which sort of when something binds to it, it will do something like cleave off a sequence and you can use an NGS like next generation sequencing readout very high throughput. So you create lots of these different sequences. You—I guess in this case they were targeting a specific HIV like protein or genome or something and if it binds you can basically get the signal of like you get more reads of that. And so the more time the more sequences which are floating around kind of like the more fitness the more likely it is to bind. And then you take those and then you mutate them again and you kind of iterate on this, right? So you have this iterative experiment where you're progressively using the petri dish to basically identify the most fit things. So and then what you're doing here is you're basically doing this same experiment in silico and they're actually doing very high throughput like I think there's like 10 to the 11 or something sequence—some like really high number of kind of sequences explored in parallel for this experiment. But the key is the biology of the experiment is actually doing the filtering, right?

是的。是的。是的。对。

Yes. Yes. Yeah. Right.

所以，你可以想象其他类型的实验，你可以应用同样的想法，你正在做这种渐进式改进，这在生物实验室工作中非常常见，所以如果你捕捉那些中间状态，那么你也许可以将它们输入模型并做那种事情。

So, and so you can imagine other types of experiments where you could apply the same kind of idea where you're doing this like progressive refinement of something which is very common in biological lab work and so if you're capturing those intermediate states then you can maybe feed them into the model and do that kind of thing.

绝对。是的。所以另一个例子是抗微生物药物耐药性。我认为那也是我们非常感兴趣的事情。以及选择性靶向特定细菌菌株或杀死细菌菌株的能力，这基本上用 0 到 1 的分数来衡量其有效性。所以展示逐渐更有效的噬菌体基因组及其相关的有效性分数非常符合这个范式。我们还有一个设计任务，正在与国家实验室合作，在一个对我们来说相当不同的领域，但它是关于设计蛋白质来提取稀土矿物。所以事实证明，你不仅关心能结合某物的蛋白质，还希望它具有选择性。你希望它结合一种稀土矿物。所以你有与每种矿物的亲和力或结合亲和力相关的分数。我们想基本上做类似的想法——这个类似的练习，用稀土矿物，并展示一系列逐渐理想的分数，不仅是为了结合这个，而且对其他矿物的结合分数较低，这样你可以选择性地做到。所以我认为，你可以展示序列并设计具有某种适应度或功能输出的理想序列的创造力，是一种相对直观的方式，让人们设计，你知道，基本上只是通过提示工程，我认为这非常令人兴奋，值得进一步探索。

Absolutely. Yeah. So another example is for the antimicrobial resistance. I think that's something we're very interested in as well. And the ability to selectively target specific bacteria strains or kill bacteria strains yet that's measured basically a score zero to one of how effectively that is done. And so showing progressively more effective phage genomes and their associated scores for effectiveness fits that paradigm very well. There's another design task that we're working with a national lab to do this with as well in an area that is quite different for us, but it's on designing proteins to extract rare earth minerals. So and so it turns out that you don't just care about proteins that could bind to something, but you want it to be selective. You want it to bind to one type of rare earth mineral. And so you have scores associated with how much affinity or binding affinity for each type of mineral. We want to essentially do the similar thought—this similar exercise with rare earth minerals and show it a series of progressively desirable scores not just for binding to this but like lower binding scores for other ones so you can selectively do it. So I think there's the creativity in which you can showcase sequence and design desirable sequence with some kind of fitness or functional output is a relatively intuitive way for people to design, you know, just by prompt engineering essentially, which I think is very exciting to explore more.

接着稀土矿物提取的话题，我觉得这是一个有趣的用例。事实上，也许是一个你可能不会比其他技术有比较优势的领域，因为看起来你的优势可能在于更大规模的设计，跨越整个生物体，但专注于单个蛋白质可能更像是一个结构任务。

Following up on the rare earth mineral extraction, I find that is an interesting use case for this. In fact, maybe one where you probably wouldn't have a comparative advantage compared to some other techniques because it seems like your strengths are probably in larger scale design across like entire organisms, but focusing on individual proteins that may be much more of a like a structural task.

我很好奇，说到 ClinVar，这里的论点是，模型现在能很好地统计表示哪些类型的突变常见或不常见。

And I'm curious, going to talk about ClinVar, the argument here is that the model now is a really good statistical representation of what type of mutations are common or not common.

是的。

Yeah.

我认为，为了进行设计，如果你想设计结构，你需要理解结构。如果你想理解疾病，对许多疾病来说，从群体基因组学的角度来理解更自然。所以我很好奇，在使用这种策略时，你认为你最强的竞争优势在哪里？你认为你的模型除了功能之外，还理解结构吗？

And I think in order to do design, if you want to design structures, I think you want to understand structure. If you want to understand disease, I think that is more natural for many diseases in a population genomic sort of way. So I'm curious, where do you think your strongest competitive advantage is in using this strategy? And do you think that your model understands structure in addition to function?

是的。我想说，吸引我们关注稀土这一特定应用以及我们总体优势的原因，我们认为它可能适合，有两点。一是，我认为在上下文重要的领域。所以，你说得对。对于只是设计结构的蛋白质设计任务，可能不是我们自然看到自己有竞争优势的地方。但在稀土案例中，我们的假设是上下文很重要，这里可能重要的是某些微生物，其蛋白质具有我们想要的功能。我们可以潜在地提示并提供上下文，告诉 Omni 这是你应该搜索新蛋白质的基因组或微生物邻域。所以这真的是一种挖掘练习，我们能向模型展示其他人无法为蛋白质结构模型展示的是，我们可以输入感兴趣基因或蛋白质之前的非编码区域，然后要求模型提供变异。所以就像突变这个蛋白质，但知道你在这种微生物中，但给我展示你在自然界中见过的不同变异，或结合你在自然界中见过的不同东西，给定这个上下文。我认为这是我们能制造非常进化多样序列和潜在阶段的一个原因。人们已经使用 Evo 模型以类似技术对毒素-抗毒素做了这个。他们提示感兴趣蛋白质上游的东西，然后要求模型生成一堆合理的其他东西。我认为在稀土案例中这非常令人兴奋，因为然后我们可以提出合理的变异，然后相对简单地测试它们结合什么和选择性结合什么，我认为这对这个合作伙伴非常有趣，例如关心为美国战略性地确保稀土供应链。所以我们认为这绝对值得支持。

Yeah. I'd say what drew us to this particular application of rare earths and our strengths in general, why we thought it might be suited for it, is two things. One is, I think in areas where context matters. So yeah, you're right. So a protein design task where you're just designing structure, maybe not naturally where we see ourselves competitively advantaged. But in the rare earths case, our hypothesis is that context matters, and what it matters here possibly is certain microorganisms with proteins that have the function that we desire. We can potentially prompt and provide us context for this is the neighborhood in which you know tell Omni this is the neighborhood of genomes or microorganisms that you should search for new proteins. So it's really a mining exercise, and what we can showcase to our models that other folks can't for protein structure models is that we can feed in non-coding regions before that gene of interest or that protein of interest and then ask for the model to provide variance essentially. So like mutate this protein but know that you're in this microorganism but show me different variants that you've seen in nature or combine different things that you've seen in nature given this context. And I think that is one reason why we can make very evolutionary diverse sequences and potentially phases. Folks have used Evo models to do this with toxin antitoxins in a similar technique. They've prompted on things upstream from the proteins of interest and then asked the model to generate a bunch of plausible other ones. And I think in this case for rare earths that's super exciting because then we can come up with plausible variants and then test them relatively simply for what they bind to and selectively bind to, which I think is really interesting for this partner who cares, for example, about securing the supply chain of rare earths for the US strategically. And so we thought that's absolutely worth something that is worth supporting.

所以你的观点不只是你在设计蛋白质，而是你在设计一个产生蛋白质的生物体，这个蛋白质有作用，但或者说是设计这个的方式是你需要理解蛋白质与稀土或某些矿物的相互作用在生命树中是如何发生的。

So your point is not just you're designing a protein but you're designing an organism which generates a protein and this protein has an action but or is it that the way to design this is you need to understand how through the tree of life interactions of proteins with rare earths have or with certain minerals have occurred.

是的，我想说在这个特定案例中，对于稀土，我们并不真正感兴趣于生物体，比如设计整个生物体，但我认为生物体确实告诉我们哪些蛋白质是合理的，以及它们的选择性不是或它们进化的方式，不仅仅是蛋白质本身，还有周围的调控元件。它帮助我们缩小范围并提供额外的上下文。我想我把基因组或 DNA 广义上视为物理世界在 DNA 中的印记。所以 DNA 中有我们想要的部分，但我们想要的部分周围的东西也告诉我们一些关于它来自哪里的上下文以及它如何形成其功能。所以我就这么想，你知道，回到那个词，但上下文。我认为上下文在许多这些应用中很重要，或至少在其中的一些中。

Yeah, I'd say in this particular case for like the rare earths, we're not really interested in the organism like designing the whole organism, but I think the organism does tell us about what proteins are plausible and their selectivity is not or the way they evolve, not just around the protein itself, but also the regulatory elements around it. And it helps us narrow and provide additional context. I guess I think of the genome or DNA broadly as sort of the imprint of the physical world into DNA. And so there's parts of the DNA that we want, but the things around those parts that we want also tell us a little bit about the context of where it came from and how it came to be in its function. And so I just think of it, you know, like going back to that word, but context. I think context matters in many of these applications or at least in some of them.

是的，我认为上下文在生物学中非常重要。我认为也许我们的一大瓶颈是缺乏上下文，以及我们人类大多如何以非常工程化的方式处理生物学，比如让我们隔离单个系统，而系统生物学方法很难获得任何有意义的定量预测能力。所以也许跑题了，但我很好奇，说到上下文，思考上下文如何扩展到生物体。你现在说的是 200 万长度的上下文，对吧？

Yeah, I think context matters a lot in biology. I think maybe one of our big bottlenecks is the lack of context and how we as humans mostly approach biology in terms of a very engineering like let's isolate individual systems and a systems biology approach is very hard to get any sort of meaningful quantitative predictive power. So maybe going off on a tangent, but I'm curious, going to context and thinking about how context scales to an organism. You're talking about right now 2 million length context, right?

是的。

Yeah.

你知道，我认为人类基因组大约长一千倍。所以或者但即使很多，比如说细菌基因组，如果你试图工程化它们，也比那长得多。那么你如何利用一个长但仍有限的上下文，在大型生物体中进行合成生物学？

You know, I think the human genome is roughly a thousand times longer. So or but even like a lot of, you know, say bacterial genomes if you're trying to engineer them are quite a bit longer than that. So how do you leverage something which has a long but still finite context compared in a to do synthetic biology across like large organisms?

也许实际上也许为了上下文，Evo 2 细菌噬菌体设计是如何工作的，那可能也远超过 200 万。

Maybe actually maybe for context how did the Evo 2 bacteria phage design work which was probably much more than 2 million for that as well.

是的。所以我可以稍微谈谈 Evo 细菌噬菌体，因为实际上是一个单独的团队做的，但那个上下文实际上相当短。他们从噬菌体开始的原因是它是最短的基因组之一，所以我相信大约是 6000 个碱基对。

Yeah. So I could I can speak to the Evo bacteria phase just a little bit because it's actually a separate group that worked on that, but that context was actually pretty short actually. The reason why they started with phases is because it's amongst the shortest genomes and so I believe it was something around 6,000 base pairs.

那真是 是的。是的。所以极短是疯狂的。它们非常高效。它们在那里影响很大。

That's really Yeah. Yeah. So extremely short are insane. They're incredibly efficient. They impact a lot in there.

是的。是的。嗯，但是的，不，我认为你关于如何用更小的东西获得更长上下文的问题是一个非常关键的问题，我认为整个 AI 社区一直在尝试解决并发挥创意，所以我的意思是，我认为这是我们作为一家公司首先是一个 AI 研究实验室的重要原因，因为我们认为创新需要不断推动。这不是一个我们可以直接抓取开源模型并期望我们关心的许多任务就会得到解决的空间。我们想要不断推动边界。所以上下文是我们推动的关键研究领域之一。我想这可能就是我们得名的原因，因为我们在长上下文成为一件事之前就研究了它，我想是 2023 年、2022 年。

Yeah. Yeah. Um but yeah, no I think your question about how how do you get longer context with something smaller is a very key question that we I think broadly the AI community is constantly trying to fix and be creative about and so I mean I think that's large part why we as a company are an AI research lab first because we we think the innovation needs to be constantly pushed. It's not a space where we can just grab open source models and and expect that, you know, many of the tasks that we care about are just going to be solved. Uh we we want to continually push the envelope. And so context is one of the key research areas that we we drive. And I would say that's probably how we got our name cuz we worked on long context before it was a was it was a thing I guess like 2023 2022.

长上下文。你和你的团队，我的意思是你的合作者，在这些状态空间模型方面有很长的历史，你知道，推动上下文，嗯，就像当时看起来疯狂的那样。

Long context. you and your team, I mean your your collaborators have a long history of these spa space state space models doing you know pushing context um like what seemed insane at the time.

是的。我的意思是现在我认为是常规的书籍大块，但当时是数量级更长。

Yeah. I mean now I think routine of books the big slabs but but at the time was like orders of magnitude longer.

我不知道你是否想稍微谈谈这个。另外，我觉得有一件事非常吸引人，就是对于这个模型和其他模型，你们真的以第一性原理的方式深入 GPU 的架构，并设计出利用 GPU 架构的模型。我的意思是，这不仅仅是“哦，我们正在构建更长的”，而是“在算力约束下，我们能做什么特别的事情来推动边界？”

I don't know if you want to talk about that a bit. And also, I think one thing which I think is really fascinating is how, for both this model and other ones, you really go in a first-principles way, like diving into the architectures of how GPUs work and designing models which are exploiting the architecture of GPU. In addition to—I mean, it's not just, oh, we're building longer. It's like, what can we do special given the compute constraints we have to push the boundary?

是的，绝对如此。如果你想分散我们在 Radical Numerics 的研究人员的注意力，这就是如何用极客话题狙击他们。你提起长上下文和内核、GPU，然后他们就会说：“等等，有人提到内核了吗？”然后他们就开始尝试弄清楚如何让事情变快。是的。长上下文在我心中有一个特殊的位置，因为那是我在斯坦福读博士时专注的领域。这也是我们开始思考 DNA 的方式。稍微回顾一下，当时我们在研究语言模型，然后我们注意到我们的模型在长上下文上表现很好。所以这就是我们开始进入这个领域的进展——就像，哦，这些模型在长上下文上似乎非常高效。然后和我的实验室伙伴 Michael Py 一起，他参与了 Pyina 的首次设计，这是一种卷积架构。然后我们开始思考，让我们进一步推动这个，看看如果我们真正倾向于长上下文，会开启哪些新应用。我们问，最长的序列是什么？最终，不出所料，我们落在了 DNA 上。我们想，DNA 肯定是最长的——三十亿个碱基对。然后我们开始思考，好吧，那里正在做什么？人们在做多长的上下文？他们做得超级短——一次只有一两千个碱基对或 token。这比人们想要的 DNA 上下文要小得多。

Yeah, absolutely. If you want to distract our researchers at Radical Numerics, this is how you nerd snipe them. You bring up long context and kernels, GPUs, then they're like, "Wait, did someone say kernels?" And then they start trying to figure out how to make things fast. Yeah. Long context has a special place in my heart because that's what I focused on in my PhD at Stanford. And that's how we started thinking about DNA. Going back a little bit for fun, we were working on language models in general, and then we noticed our models were good at long context. So that's the progression of how we started working in the space—like, oh, these models seem to be really efficient on long context. And then with Michael Py, my lab mate, he worked on the first design of Pyina, this convolutional architecture. And then we started thinking, let's push this further, let's see what new applications open up if we really lean into long context. And we asked, what's the longest sequence out there? And eventually, unsurprisingly, we landed on DNA. We're like, DNA's got to be the longest—three billion base pairs. And we started thinking, okay, what's being done there? What kind of context lengths are people doing? They were doing super short—like one or two thousand base pairs or tokens at a time. It's way smaller than what one would want for DNA.

我的意思是，大多数人类转录本大约 3K 左右。所以，这甚至不是——如你所知，大多数——这甚至不是表示一个蛋白质或大多数时候所需要的东西。

I mean, most human transcripts are like 3K or so. So, that's not even—as you know, most—that's not even what you need to represent like a protein or most of the time.

所以，那里显然有需求，却被忽视了。我们最初把它看作一种方式——就像，让我们看看能否做一些我们不知道是否会成功、也不知道谁会想要的事情。于是，我们开始捣鼓，结果发现开箱即用，相对开箱即用，它在读取 DNA 方面做得相当不错。然后它引向了——好吧，让我们——人们不断询问 DNA。他们不太关心我们的语言模型，他们会说，你能做更长的吗？你能——你能用它做什么？他们总是问你能用它做什么？我们真的不知道。然后不知为何，我们落在了这个想法上——嗯，没人在写 DNA。我们能让它写 DNA 吗？这是一个非常简单的问题，但事后看来几乎显而易见，是的，你会想要写 DNA 并设计它。但当我最初向人们推销 Evo 的想法时，我花了六个月——我想在生物领域这不算长，但我花了六个月四处说，嘿，如果我生成 DNA，你觉得有用吗？你会用它做什么？你会支持我们吗？你想参与其中吗？疯狂的是，我在斯坦福交谈过的几乎每一位科学家都认为这是个愚蠢的想法。我当时想，这太酷了——比如生成 DNA。我们会加速这个领域多少？然后人们会说，你会用它做什么？我说，我不知道。然后我会得到这样的评论：那不可能。就像我们人类都不理解规则。你怎么能期望 AI 学会它？你甚至无法判断它是对是错。就像你无法告诉 AI，是的，那是对的或错的。你怎么能期望它学会？或者有太多重复字符，或者 DNA 分布太嘈杂——就像没有真正的规则，只是一堆垃圾。我听到了所有理由，但我就是如此固执。从能够生成 DNA 中一定有一个用例。我只是——感觉是对的，但我不知道那是什么。所以这真的是一次实验，就像，会发生什么？所以当我们第一次训练 Evo 时，我记得我们不知道它是否会成功。我们不知道。我们不知道会出现什么。我们只是想，让我们训练一个大的，这有点荒谬。但不知何故，Arc 的人说，当然。

And so, it's clearly a need there and overlooked. And we saw it as a way to initially—like, let's see if we can do something that we had no idea if it was going to work and no idea who would want it. And so, we just started tinkering around, and it turns out out of the box, relatively out of the box, it was doing pretty well at reading DNA. And then it led to—okay, let's—people keep asking about DNA. They didn't really care about our language stuff as much, and they would say, can you do longer? Can you—what can you do with it? They always ask what can you do with it? We didn't know really. And then for some reason we landed on this idea of—well, no one's writing DNA. Can we get it to write DNA? And that was a really simple question, but in hindsight almost seems obvious that yeah, you would want to write DNA and design it. But when I was first pitching the idea of Evo to folks, I spent six months—which I guess in bio words is not that long, but I spent six months going around saying, hey, if I generate DNA, would you find that useful? What would you do with it? Would you back us? Would you want to be a part of this? And crazy enough, almost every scientist at Stanford I talked to thought it was a stupid idea. I was like, this would be so cool—like generating DNA. How much would we accelerate the field? And then people would say, what would you do with it? I'm like, I don't know. And then I would get comments like, that's not possible. Like we as humans don't understand the rules. How could you expect an AI to learn it? You can't even tell if it's right or wrong. Like you can't tell the AI, yes, that's right or wrong. How can you expect it to learn it? Or there's too many repeat characters or DNA is too noisy of a distribution—like there's no real rules and there's just a bunch of junk in there. I heard all the reasons and I was just so stubborn about it. There's got to be a use case from being able to generate DNA. I just—it just feels right and I didn't know what it was. So it really was an experiment of like, what happens? And so when we first trained Evo, I remember we had no idea if it was going to work. We had no idea. We didn't know what thing was going to emerge. We just thought, let's just train a big one, which is kind of ridiculous. But somehow, folks at Arc were like, sure.

是的，为什么不呢？

Yeah, why not?

为什么不呢？让我们看看会发生什么。让我们为 GPU 付点钱，让这些疯狂的孩子训练一个模型。我记得当第一个结果回来时，它让我们不寒而栗。我们想，哦，也许有什么事情在发生。有人拿了模型检查点，第一个，扔给了 Protein Gym，一个蛋白质基准测试，结果发现它与蛋白质专用模型竞争。我们想，好吧，这相当令人惊讶，因为我们从未告诉模型什么是蛋白质，什么不是。实际上——还有蛋白质和 DNA，对吧？所以那是一个部分。我认为只是令人惊讶的是，它实际上与蛋白质专用模型竞争。这就像是第一次经历。然后它们一个接一个地出现——就像，哦，这个在这里有竞争力，哦，它在 RNA 和 DNA 上是 state of the art。我们开始看到，哦，它可以跨不同模态学习，不仅仅是 DNA。然后我们觉得，好吧，那里有东西。所以它就从那里开始了。然后 NVIDIA 的人说，让我们支持 Evo 2，让它更大。然后 OpenAI 的 Greg Brockman 说，我要从 OpenAI 休息一下，休四个月的假，帮助这些疯狂的孩子。然后我们在 Slack 上给 Greg 发消息——凌晨 3 点把我叫起来调试代码，这太疯狂了。是的。所以它只是——轨迹在许多方面非常令人惊讶，但与此同时，我们仍然得到很多反馈，比如，这些模型有什么好处？它们在现实世界中有用吗？所以，这就是真正激励我们创办公司的原因。我们认为我们展示的只是学术风味的一小部分。类似地，当语言模型首次出现——如果自然语言出现——人们问类似的问题，比如，这些东西有什么好处？哦，酷，它可以为我写一些笑话。这会导向一个可以自动化一切的 AGI 吗？他们并不这么认为，对吧？他们认为它像个玩具。它有能力涌现，他们会推断其潜力。

Why not? Let's see what happens. Let's pay some money for the GPUs and let these crazy kids train a model. And I remember when the first result came back and it kind of gave us chills. We're like, oh, maybe there's something going on. Somebody took the model checkpoint, the first one, and threw it at Protein Gym, one of the protein benchmarks, and it turned out to be competitive with protein-specific models. And we were like, okay, that is pretty surprising because we never told the model what's proteins versus not. And there's actually—it was—there's also proteins and DNA, right? So that was one piece. I think it was just surprising that it was actually competitive with protein-specific models. And this was like the first experience. And then they just kind of kept on coming one after another—like, oh, this is competitive here, oh, it's state of the art on RNA and DNA. And we started seeing like, oh, it can learn across different modalities, not just DNA. And then we felt like, okay, there's something there. And so it kind of went from there. And then folks at NVIDIA were like, let's back Evo 2, let's make this even bigger. And then Greg Brockman from OpenAI is like, I'll take a break from OpenAI and take a four-month sabbatical and help these crazy kids out. And then we're Slack messaging Greg—brought me at 3:00 a.m. trying to debug our code, which is wild. Yeah. So it just—the trajectory was very surprising in many ways, but at the same time, we still got a lot of feedback like, what are these models good for? What are they useful for in the real world? And so, that's what really motivated us to start a company. We thought what we showed was just a taste from an academic flavor. In a similar way, when language models first came out—if natural language came out—people asked similar questions like, what are these things good for? Oh, cool, it can write some jokes for me. Is this going to lead to an AGI that can automate everything? They did not think that, right? They thought it's like a toy. It's got emerging capabilities and they would extrapolate the potential.

从很多方面看，我认为我们在这里看到的甚至更令人兴奋，或者让人想起 DNA 的那条轨迹。但 Evo 2 甚至展示了细菌噬菌体，那并不具有说服力——我的意思是，这几乎是一个可怕的例子，对吧，正如你提到的，所以人们没有看到那一点，那并不是一个让人恍然大悟的时刻。

And in many ways, I think what we're seeing here is even more exciting or reminiscent of that trajectory for DNA. But is Evo 2 even was showing the bacteria phage and that wasn't a persuasive—I mean, it's almost a scary example, right, as you mentioned, so that people didn't see that like that isn't a light bulb moment for people.

是的，我的意思是，有些人，对吧，我认为要么是公平的批评，要么是粗糙的批评，你甚至可以扮演魔鬼代言人说，它生成的东西相当接近自然，你只是在重述一个小的变异。所以我认为有很多方法可以批评并贬低其潜力，这可能是一个公平的论点。所以我认为在这一点上，我们认为更重要的不仅仅是推进科学，比如“酷，这可以做到”，然后把它留作一种思想实验，而是我们如何实际利用它来增进人类对疾病的理解，改善治疗，制造更好的稀土矿物提取器。我们如何真正让它变得有用，这是我们作为公司现在关心的，除了这些更科学的问题之外。

Yeah, so I mean, and some people, right, I think either fair or rough critiques, you can even play devil's advocate and say what it's generating is kind of pretty close to nature and you're kind of recapitulating just a small variance. And so there's, I think there's a lot of ways to critique and minimize the potential, which can be a fair argument. So I think at this point we think what is more important is not just pushing on the science and like cool this can be done and leave it there as a sort of thought experiment, but like how do we actually use this to improve human understanding of disease, improve treatments, make better rare earth mineral extractors. How do we actually make this useful is what we care about now as a company in addition to some of these more scientific questions.

所以也许这是一个很好的过渡。这让我想到了两件事。一是博客文章末尾的机制部分，然后还有生物安全部分。我认为这些都是重要的应用。那么我们可以先做机制部分吗？你能稍微谈谈吗？这实际上是建立在 Evo 所做的工作之上的。我记得在 Evo 论文中有一些机制工作，他们讨论了使用模型反转问题，并利用模型提取关于生物学的见解，对吧？所以这实际上已经成为一个常见的主题，我认为可能生物学比其他任何领域都更常见，因为这是一个科学问题，它在学习世界的模式。你实际上可以说，好吧，你学到了什么模式？

So maybe that's a good segue. There's that brings up two things for me. One is the mechan stuff that's in the end of the blog post and then also the biosafety stuff. Those are both I think important applications. So can we let's do mechan first. Can you talk a little bit about um and this is actually kind of building on the work that Evo did as well. I think um in I remember in Evo paper there was some mechan work in which they were discussing using the mod reversing the question and using the model to extract insights about biology, right? And so so and this has actually become a common theme with I think maybe biology more than any other domain is that people because it's a scientific question and it's learning patterns about the world. You can actually say okay well what patterns did you learn?

是的。所以我认为机制可解释性是生物学中一个新兴领域，我们对此非常兴奋，而且承认我们处于早期阶段。所以我们正在组建那个团队，但到目前为止，我们展示并感到兴奋的是分析模型中的嵌入和一些激活。所以生物学中的机制可解释性这个想法目前大量借鉴了自然语言社区。你可以认为模型正在做的是压缩它见过的一堆信息，对吧？在这种压缩中，它基本上将其提炼成关键组件、关键模式，帮助它理解数据或学习数据的分布。所以我们试图做的是探测模型，看看模型将什么提炼到其权重和激活中，或者类似输出中。所以对我们来说，我们从模型的许多输出开始。所以激活，我们想看看能看到什么样的结构，什么样的可视化，帮助我们理解 DNA 的一些复杂性，这有很多，对吧，所以其中一些复杂性可能围绕 GC 含量、某些转录因子基序，有一堆调控类型的模式和基序，我们相信模型必须捕捉这些才能完成其任务，以理解疾病是否由变异引起，通常它会压缩所有这些不同的基序，并将它们提炼到模型权重中。所以我们的工作是找到这些，看看我们是否能提炼出一个模式或结构，可以推广到我们不理解模式的其他情况。对。所以这有点像是。

Yeah. So I think mechan is an emerging field in bio that we're extremely excited about and admittedly we are on the early side I'd say. So we're building up that team but so far what we've showcased and been excited about is looking really analyzing the embeddings and some of the activations in the model. And so this idea of mechan for bio is borrowing a lot from the natural language community currently. You could think of what the models are doing is compressing a bunch of information it's seen, right? And so in this compression, it's really basically distilling it down into the key components, key patterns that helps it understand the data or learns the distribution of the data. And so what we're trying to do is probe the models to see what did the model distill into its weights and its activations or sort of like the outputs. And so for us, we started off with a lot of the outputs of the model. So the activations and we wanted to see what kind of structure what kind of visualizations can we see that help us understand some of the complexities of DNA which was a ton right and so some of these complexities can range around GC content certain motifs of transcription factors there's a bunch of regulatory types of patterns and motifs that the model we believe has to pick up to be able to do its tasks right to understand whether a disease is caused by a variant and generally it's going to compress all these different motifs and distill them into the model weights. And so our job is to then find these and see if we can distill a pattern or structure that can be generalized to other cases where we don't understand the patterns. Right. So that's sort of laral.

所以在这种情况下，GC 意味着两个核苷酸，嗯，它们在序列中的比例。

So in this case GC means the two nucleotides are um the fraction of those in a sequence.

没错。是的。所以基因组中 G 和 C 字母的比例，我想这是比较简单的事情之一，但你也知道，简单的事情如重复序列、逆转录因子数量，我认为转录因子或 TF 基序是另一个特别有趣的，因为最终我们会达到一个可以设计转录因子的情况，意味着我们将通过像 ChIP-seq 这样的模态拥有某些转录因子模式。

Exactly. Yeah. So the fraction of the G and C letters in a genome which are I guess amongst the more simpler things but also you know simple things as repeats number of retif I think transcription factor or TF motifs is another one that is especially interesting for folks because I eventually we're going to get to a case where we can design transcription factors meaning we'll have certain transcription factor patterns via like chipse modalities

转录因子是，嗯，你继续。

transcription factors are are well you go ahead

转录因子是能够结合 DNA 的分子，它们会改变基因表达模式。所以它具有这种调节作用，不修改 DNA 本身，但可以修改 DNA 的效果和 DNA 产生的产物。它可以对你身体中几乎一切产生影响。所以它可以改变疾病状态。它可以改变，我想有很多与衰老相关的研究都是围绕转录因子设计的。所以我们认为，能够通过 DNA 以及其他模态理解一些基序，最终将让我们能够设计转录载体模式。所以我认为这非常令人兴奋，而且在许多方面，学习这些转录因子的组合空间，比如什么结合、在哪里结合以及引起什么效果，实在太广阔，无法手动完成。所以我们想采用数据驱动的方法来学习一些这些基序。这里的复杂性部分是因为转录因子本身是编码基因，所以它们可以相互调节，这就是组合效应的地方。

transcription factors are uh molecules that can bind to DNA and they would alter essentially the gene expression pattern. Um and so it has this regulatory effect that doesn't modify the DNA itself but can modify sort of the effects of DNA and the products that DNA makes. Um and it can have a lot of implications on well pretty much everything in your body. So it can modify disease states. It can modify I guess there a lot of aging related research is around transcription factor um design. And so we think being able to understand some of the motifs via uh DNA but also additional modalities will eventually let us be able to design transcription vector patterns as well. And so I think this is very exciting and in many ways the combinatoral space of learning these transcription factors like what binds and where they bind and what effects it causes is just far too vast to be able to do this um in a manual way. And so we want to take a datadriven approach to to learn some of these motifs. The complexity here is partly because the transcription factors are themselves um coding genes and so that you can reg those can regulate each other and so that you have this that's where that combinatorial effect.

是的。所以就像只为听众叙述一下，我们正在从 GC 含量开始，逐步处理这些越来越复杂和更高层次的因素。我们现在正在研究疾病，这可能是你能想到的最复杂的事情，或者你在这个分析中指出的。

Yeah. So like just narrating for the listeners only we're kind of marching through these increasingly complex and higher level factors all the way from GC content. we started now we're looking at disease which is maybe the most complex thing you could or you were uh pointing towards in this analysis.

是的。所以我认为，我们预览这个的第一个想法，以及我们广泛努力的方向，是映射疾病流形的想法。对吧？所以流形就像是模型对疾病在输出分数或嵌入方面的表示空间。我们相信，通过理解一些这些输出，可以收集到更多的结构。所以你知道，映射这个流形或疾病结构的景观，显然有很多疾病，所以我认为这将是一个大的令人兴奋的研究领域，对我们来说，实际上驱动了我在生物学中的动力。

Yeah. So I think um what we our first idea for previewing this was and and what we're working toward this broadly is um this idea of mapping the manifold of disease. Right? So manifold there's like the sort of representation space of what disease looks like to a model in terms of the output scores or embeddings. We believe that there's a lot more structure that can be gleaned from understanding some of these outputs. Um and so you know mapping this manifold or the landscape of what the structure of disease and obviously there's many diseases and so um I think would be a big exciting area of research for us um to to actually drive motivation for me in bio.

我觉得我们真的只是触及了表面，因为如果你能从 DNA 中获取这么多干净的洞见——在我看来，DNA 只是你最终想要融合到生物建模中的传感器之一——那么能够在所有模态上做类似的事情，就像……哦，我说的所有模态是指蛋白质、RNA、表观基因组学，你知道，ATAC、染色质可及性和甲基化模式，所有这些围绕 DNA 的其他不同类型的分子表型。这展现了一个巨大的机会，就摆在我们面前，全是空白领域，绿野。不，我还没见过有人把这种级别的复杂技术——从机器学习、深度学习——应用到我认为将是 AI 最重要、最具影响力的理解和应用领域。这就是我们兴奋的地方。我想我们只是展示了一个预览，一个非常简单的 DNA 模型预览。但在我们的下一代模型中，将越来越具有多模态性，我们说的是几十种模态。对我们来说，这是一个非常激动人心的时刻。

I think we're just really scratching the surface, because if you can get this much clean insight potentially from just DNA—which is in my mind just one of the sensors that you want to ultimately fuse into modeling biology—then being able to do something similar across all modalities is something like... oh, by all modalities I mean protein, RNA, epigenomics, you know, ATAC, chromatin accessibility, and methylation patterns, all these other different types of molecular phenotypes around DNA. Just presents such a huge opportunity that this kind of laid in front of us that is all green space, green field. No, I haven't seen anybody do this level of sophisticated techniques from machine learning, deep learning into what I think is going to be the most important, impactful area of understanding and applications for AI. That's what we're excited about. And I think we're just showing a preview, very simple preview from the DNA-only models. But in our next generation of models, which will be increasingly multimodal, we're talking dozens. It's a very exciting moment for us.

顺便问一下，语言，自然语言，是最……之一吗？

Sidebar on that. Is language, natural language, one of the most...?

还不是。

Not yet.

是的。但将来会是。

Yeah. But it will be.

所以我会说，关于如何将语言与生物融合，有很多问题，但我实际上认为这将会相对直接。我认为对我们来说更棘手的部分实际上是如何更多地融合生物信号。所以融合语言——有很多例子，比如图像和视频领域。所以我们对此感觉相当不错，我们已经用语言做了一些早期实验，我认为当你能将它与语言连接起来时，这会让人们更容易接触。但我认为人们仍在努力弄清楚的配方是，如何跨生物模态做到这一点。

So I'd say there's a lot of questions about how to fuse that with bio, but I actually think it will be relatively straightforward. I think the more tricky part for us is actually how to fuse the biological signals more. So fusing language—there's a lot of examples with that, with like image and video space. So we feel pretty good about that, and we've done some early experiments with language, and I think that will make it extra accessible for folks when you can connect it to language. But I think the part, the recipe that folks still are trying to figure out, is how to do this across biological modalities.

这似乎是一种能够进行思维链的自然方式。对吧。

It just seems like a natural way to be able to do chain of thought. Right.

正是。是的。

Exactly. Yeah.

绝对。而且我认为，你知道，特别是当你开始从编排工具中获得思维链时，比如云科学，我认为融入语言的想法已经在很多人的脑海中。如果你没有问题，让我们谈谈……

Absolutely. And I think, you know, especially when you start having chain of thoughts from orchestrating tools, things like cloud science, I think the idea of incorporating language is already in a lot of people's minds. If you don't have any questions, let's talk about the...

对我们来说，我们作为一家公司认为拥有双重使命非常重要——我们称之为双重使命。这个理念本质上是意识到并感到责任，或者想要感到责任，对我们正在设计方面赋能的能力负责。所以如果我们要创建能够设计序列功能的模型，我们相信并看到公司在保护这项技术并确保其被负责任地使用方面存在差距。越来越多——我的意思是，人们有——我昨晚刚参加了一个小组讨论，关于 AI 科学家、能够进行科学发现的智能体，最后一个问题是：AI 学习科学的最大风险或末日场景是什么？他们每个人都谈到了生物武器。同时，我很好奇，因为我想，好吧，那么这些人对此做了什么？基本上我没听到任何相关的内容。这些公司——我不会说他们的名字——但我看看这些公司，他们在这些领域没有大的努力。所以无论如何，我们觉得作为一个实验室，在实验室里，一个既构建设计能力的团队实际上也最适合构建防御能力，因为它们基本上是相同的模型。一个擅长生成的模型，结果也擅长区分或预测一个序列是否致病。所以我们觉得，不仅在原则立场上，有必要同时从事生物安全和设计，而且在战略上也是合理的。所以我们觉得这引起了我们团队以及更广泛社区的共鸣，美国政府内外的人们，甚至国外，都存在明显的差距，需要一种类型的实体来做这件事。所以是的，我们觉得有必要将其纳入我们的使命。

For us, we as a company thought it was very important to have a dual mandate—what we call a dual mandate. It's this idea of essentially being cognizant and feeling responsible, or wanting to feel responsible, for the capabilities that we're enabling on the design side. So if we're going to create models that can design function into sequences, we believe and see a gap in companies being able to safeguard that technology and make sure that it's used responsibly. Increasingly more—I mean, folks have—I was just at a panel last night, a panel on AI scientists, agents that can do scientific discovery, and one of the last questions was: what are some of the biggest risks or doomsday scenarios with AI learning about science? And every one of them talked about biological weapons. And at the same time, I was curious because I was like, okay, so then what are any of these folks doing about that? And basically I didn't hear anything about that. And these companies—I won't say their names—but I look at the companies, they don't have big efforts in those spaces. So anyways, we felt it was important as a lab, in the lab, that a team that was both building the design capabilities is actually best also best suited for building the defense capabilities, because they're basically the same models. A model that is good at generating turns out is also very good at discriminating or predicting whether a sequence is pathogenic or not. So we felt it was not just from a principal standpoint necessary to work on both biosecurity and design, but that it was strategically it just made sense as well. And so we felt this resonated with our team but also the broader community, folks in the US government and abroad even, that there was a clear gap in and need for a type of entity to exist to do this. And so yeah, we felt it was necessary to build it into our mission.

所以对我们来说，我们已经做的和计划做的——当你审视它时，或者我应该说，生物防御、生物安全大致有三到四个不同的战略支柱。第一个是检测和监测，对吧？你知道，广泛地说，你能从环境中检测出序列中是否含有病原体，某种能引起疾病的东西——比如来自机场的拭子、鼻拭子，或污水系统，对吧？收集样本。下一个是归因，即：一旦你检测到危险，你能弄清楚它来自哪里吗？是自然的吗？来自国外的某个随机国家？是由某个国家特定实验室的人类工程改造的吗？这有助于你弄清楚该怎么处理，对吧？所以下一个想法是关于对策。所以一旦你检测到、弄清楚了它来自哪里，你该怎么办？你能制造对抗剂吗？你能制造抗病毒或抗菌药物吗？然后第四个通常是关于威慑，但那是政府层面的事情。但是的，所以我们主要关注前三个。所以我们创建工具，给定一个序列，可以检测它是否致病，但也……所以我们觉得社区缺失的不仅仅是广泛检测是否致病，而是对其进行彻底表征，意思是：序列的哪些部分危险？例如哪些基因？归因它来自哪里——能够不仅查看其序列并将其与数据库匹配，还能归因其签名。然后还有一个重要组成部分，使其首先有效：社区，生物防御社区广泛关注序列匹配。所以他们会取一个序列，基本上将其与已知数据库比对，看看，说，我以前见过这个吗？它是否匹配这个已知病原体列表？但我认为正在出现且让许多实验室担忧的是：第一，新东西，对吧？如果它不在你的列表上。第二，被故意混淆以在序列空间——意味着字母完全匹配——以及功能空间中不被检测到的东西。

And so for us, what we have done and plan to do—when you look at it, or I should say biodefense, biosecurity sort of has three or four different pillars of strategy for biodefense. The first one is around detection and surveillance, right? You know, broadly, can you detect from the environment if a sequence has a pathogen in it, something that can cause disease—let's say from swabs at an airport, a nasal swab, or a sewage system, right? Collecting samples. The next is attribution, which is: once you detect a danger, can you figure out where it came from? Is it natural? Is it from a random country abroad? Is it engineered by a human from a specific lab in a country? And that helps you figure out what to do about it, right? So this next idea is around countermeasures. So once you've detected, figured out where it's from, what do you do about it? Can you make a counter agent? Can you make an antiviral or antimicrobial? And then the fourth one's generally around deterrence, but that's more of a government-level thing. But yeah, so we focus primarily on the first three. So we create tools that can, given a sequence, detect if it's pathogenic, but also... So what we felt was missing from the community was not just detect broadly if it's pathogenic, but characterize the heck out of it, meaning: what parts of the sequence are dangerous? What genes, for example? Attributing where it came from—being able to not just look at its sequence and match it to a database, but be able to attribute its signatures. And then also a big component to make this effective in the first place: the community, the biodefense community broadly focuses on sequence matching. So they'll take a sequence and they'll basically align it to a known database and see, say, have I seen this before? Does it match this list of known pathogens? But I think what's emerging and a concern for a lot of labs is: well, one, new stuff, right? If it's not on your list. And two, things that were intentionally obfuscated to not be detected in sequence space—meaning the letters matching up exactly—but also function space.

是的。

Yeah.

对。因为基本上我们现在赋能的模型，它们将能够功能感知或结构感知。所以这意味着，具体来说，你可以有一个序列，具有相同的功能，比如病原体，但在字母方面实际上看起来不同。你可以想象，基于我们屏幕上仍然显示的这个 smack interp 的东西，模型内部构建了这种流形，它在编码这些功能，以及其他东西。

Right. Because basically models that we're enabling now, they will be able to be function-aware or structure-aware. And so that means, concretely, you can have a sequence that has the same function, like a pathogen, but actually look different in terms of the letters. And you can imagine based actually what we have on the screen here still is the smack interp thing and there's this sort of manifold that the model constructs internally that is kind of coding for these, among other things, functions.

所以你可以想象它能够说：“哦，那个，你知道，也许基因上相当不同，或者至少有些不同，但仍然具有相似的功能。”

And so that you can imagine how it would be able to say, "Oh, well that's, you know, maybe genetically quite different or at least somewhat different, but still has a similar function."

确实。确实。所以我们在防御博客中讨论的是，功能相似的东西可以在序列上开始出现差异，同时保持相同的功能。对吧？这在自然界中可以自然发生，但这些 AI 模型也允许你有意地做到这一点。所以具有相同的功能能力，但有不同的字母，基本上是不同的拼写，但描述的是同一件事。在这种情况下，微软的这项工作叫做“paraphrases”。所以这就像是奖励机制，他们展示了你可以使用蛋白质语言模型来基本上保持相同的结构，结构意味着相似的功能，但然后改变拼写，对吧？不仅如此，他们还想测试如果你有这个能力，你把它发送通过现有的检测系统，它会破坏系统吗？它会检测到它吗？我认为一个有趣的事实，也许普通公众，但大多数生物学家知道的是，有这些 DNA 合成公司，对吧？你可以基本上发送一个序列设计，然后拿回一个 DNA 分子，就像亚马逊包裹一样。你把它发出去，他们就会把物理 DNA 发给你。他们会为你制造它。这支撑着科学界，对吧？这是允许人们做研究、理解生物学、制造药物等等的管道。所以它很普遍，而且是公开的。所以我认为人们的一个担忧，也是我们刚开始研究时的一个担忧是，在一个例如智能体在网上泛滥的世界里，可能有数十亿、数万亿的智能体在互联网上构建和采取各种行动。它们没有任何工具来基本上告诉它是否在制造危险的东西，这太疯狂了。所以这实际上是我们最初的动机，就像我们应该制造一些能够检测某物是否危险的东西。对吧。

Exactly. Exactly. So what we talk about in our defense blog is this idea of things that can function similarly, they can start having separation in terms of what the sequence looks like while maintaining the same functionality. Right? So this can happen in nature sort of naturally but also what these AI models allow you to do is also intentionally do that as well. So have the same functional capability but have diverse letters basically diverse spelling but describe the same thing basically. And so in this case this work from Microsoft called paraphrases. So it's like you know kind of rewarding things where they showcase that you can for example use protein language models to essentially keep the same structure which structure implies similar function but then change the spelling right and not just that they wanted to test that if you have this capability and you send this through existing detection systems would it break the system like would it actually detect it or not and um I think one of the interesting facts that maybe the general public, but most biologists know is that there's these DNA synthesis companies, right, where you can basically send a design of sequences and get back a DNA, you know, DNA molecule like it's an Amazon package. Like you just send it off and they'll send you the physical DNA of the sign. Um, they'll manufacture it for you. And this is this runs the scientific community, right? It's the pipeline that allows people to do research and understand biology and make drugs and everything. So it's prevalent and it's public. And so I think a con one of the concerns for folks is well and one of the concerns for us when we first started working on this was in a world for example where agents are prolific online presumably billions and trillions of agents building and taking all sorts of actions on the internet. It's wild that they don't have any tools to basically tell it if it's making anything dangerous or not. And so that was literally our first motivation of like we should probably make something that can detect if something is dangerous or not. Right.

所以你们为这些制造商过滤，他们可以说我在这里制造什么？这危险吗？之类的？是的。也许你需要一个，你可以有一个例外，如果你是某个有执照的实验室之类的，我在做危险的事情。我知道我在做。请让我做吧。之类的。

So you guys filter in the if for these manufacturers that they could say what am I making here? Is this dangerous or whatever? Yeah. And maybe you need a you you could have an exception if you're like some licensed lab or something and I'm doing something dangerous. I know I'm doing it. Please let me do it anyway. Or something.

各种情况。所以那是一种场景。公平地说，许多 DNA 合成公司有检测工具。但我会强烈假设它们不是基于 AI 的，而且公平地说，它们可能不够稳健。是的，它们大多都是模式匹配。

All sorts of cases. So that's that's one scenario. To be fair, the many of the DNA synthesis companies have detection tools. But I would I would uh strongly hypothesize that they're not AI based and and fairly they're probably not robust. Yeah, they're all pattern matching mostly.

抱歉，也许也许我应该稍后问这个问题。我只是好奇，你知道，我们所有在生物领域工作的人都试过使用 Fable，然后你输入的任何东西都会立刻被挑出来

Sorry, maybe maybe this I should ask this question later. I I'm just curious about you know, we've all everyone who is working in bio has tried to use Fable and instantly got picked out on literally everything you type in

比如我的网站。

my website for example.

是的。是的。你知道，在 Fable 里你什么都做不了，但是就科学家探索合成生物学、创建新序列和设计新序列而言，似乎很难获得一个你可以接受的 ROC 曲线，而不会阻碍具有合法目的的新科学研究。你如何避免，即使 F1 分数达到 0.9，而你现在还远未达到。我认为，你知道，如果每天或至少每年有数十亿个序列生成，那仍然很容易。我的意思是，我认为你可能会遇到很多，这似乎是一个非常困难的平衡

Yeah. Yeah. you you you know you can't do anything in Fable without um but in terms of people like scientists exploring synthetic biology and creating new sequences and designing new sequences it seems like it it'd be very hard to get you know to to have a an ROC curve which you can you can live on that doesn't like impede novel scientific research which will legitimate purposes how do you avoid you know even with an F1 of point you 9 which you're not even close to right now. I think you know that that still could easily if there are billions of sequences you know sequence generated a day or at least like a year. I mean I I think you could really have a lot of it seems like a very hard balance to

是的，这是个难题，对吧？所以我认为，从广义上看，我们的看法是，如果我们考虑如何 100% 阻止危险设计，我认为这是一个更难的问题。我认为我们问的问题是，在能力设计方面，有很多人在推动前沿，当我们看防御方面时，我们是否看到前沿技术被应用在那里？答案是没有，对吧？所以我们看到了这个巨大的差距，我们想提供帮助，来加强防御，给它一个推动，对吧？所以从这个角度来看，对我们来说很容易选择说让我们推动，让它更好地与设计方面正面竞争。它会解决一切吗？嗯，我认为那是我们渴望成为的，但现实地说，总会有一些情况它可以绕过，对吧？我认为这是一个很大的动机，为什么我们认为语言模型和聊天机器人的安全是一层，对吧？但你也说得对，可能有些东西就是会传播出去，无论如何都会绕过。那么，对于那些已经通过聊天机器人，已经帮助某人制造了危险序列的情况，你该怎么办？它已经在那里了。嗯，我认为我们的工具很酷的一点是，我们为那些关心已经存在于环境中的东西的人构建工具，现在有一个完整的生态系统，我们想要构建到其中，进行监控、归因、对策，我们想要增强那个社区，对吧？为那个领域构建更强大的工具。它必须完美才能在那里有用吗？我不这么认为。我认为我们可以提供帮助，推动生物防御社区向前发展，将 AI 技术和 AI 前沿技术带到他们的援助中，而不必完美，所以这有点

yeah it's a tough question right so I think broadly the way we look at it is you know if we thought about like how do we 100% stop the dangerous design I think it's a it's it's a harder question to ask I think the question we asked is on the capabilities design side there's plenty of folks pushing the frontier of that when we look at the defense side do we see frontier technology being applied there and the answer to that was no right so we we we saw this huge gap and we wanted to sort of aid come to its defense I said come to its um its aid to give it a boost right so I think that perspective it's it's easy choice for us to say let's push let's bring that to a better you know head-to-head match against the design side is it going to solve everything well I think that's what we're going to aspire to be but but u realistically there's always going to be cases where it can get get around, right? And I think that's one that's a big motivation for why we think, you know, safety for language models and chat bots is one one layer, right? But also, you're right, there may be things that kind of just get out there, get past that anyways. And so, what do you do about those cases is it's already passed, you know, the chat bots, right? It's already it's already aided someone into making dangerous sequences, so it's out there. Um, I think the cool thing about our tools is that we what we're building is tools for the folks that care about things that that's already out there out there in the environment that's made it somewhere and now there's this whole ecosystem that we want to build into that does the surveillance that does the attribution that does the countermeasures we want to boost that community right and build stronger tools for that space. Does it have to be perfect to be useful there? I don't think so. I think we can we can be helpful and and move the biod defense community forward in terms of you know bringing AI technology and AI frontier technology to their aid without being perfect and so that's kind of

也许另一个说法是威胁模型，在这一点上甚至不清楚你实际试图防御的威胁模型是什么，但能力目前根本不存在。所以你知道，你提供了一些东西，现在可以在更大的监管框架或政府或非营利组织等更大的框架下进行工作，现在为你提供了一个社区可以构建的工具。

maybe another say so threat model that it's not even clear what threat model you're actually trying to defend against at this point but the capabilities don't currently exist at all. So you know you provide something and now this can be worked on with uh in terms of a larger regulatory framework or um government or you know nonprofit whatever larger framework and now provides you a tool that the community can build upon.

嗯，即使它不完美，它也提供了一个起点，如果你没有工具，那么你什么都做不了。

Um even if it's not perfect it's it provides a starting point and if you don't have a tool then you can't do anything.

是的，标准是改进，对吧。对。所以标准是，你现在在哪里？我们能推动指针吗？

Yeah the bar is the improvement right. Right. So the bar is like where you at now? Can we move the needle?

我们能超越基于序列的匹配、对齐、匹配吗？绝对可以。我认为有很多，而且我认为兴趣只会增长。

Can we move beyond sequence-based matching, alignment, match matching? Absolutely. I think there's tons and I think um the interest is is only growing.

我觉得在监管方面，我们听到很多议论，来自不同的政界人士和智库的人。看起来大家确实在动员起来。所以我们很乐观，这件事能传播出去，让大家更重视，真正采取行动，而不只是空谈。因为现在确实感觉有很多空谈。我们之所以觉得应该构建工具、把它发布出去、让更多人能用，其中一个原因就是，很多 AI 公司一直在说我们应该做生物防御，但然后呢？那到底意味着什么？你打算怎么做？所以这就是我们的思路。

I think we're hearing a lot of chatter on the regulatory side, from different politicians and folks in think tanks. It does seem like folks are mobilizing. So we're optimistic that this gets out and is more top of mind for folks to actually take action as opposed to just talking about it. Because right now it does feel like there's a lot of talk. One of the reasons why we felt like let's build tools and put it out and give access to people is because there's been a lot of talk about AI companies saying we should do bio defense, but then okay, what does that mean? What are you going to do about it? So that was our approach.

那我先来替反方观点辩护一下。你能切到另一张图吗？就是有那个——对，如果你点击它们，就会显示其他类似的化合物或类似的基因组。所以可以说，替反方观点辩护一下：如果你处在最前沿——看起来你确实是，而且你肯定也在努力做到——那你就是在同时推进攻击和防御的前沿。所以就像现在，你做的事情显然有帮助，但也许如果你处在最前沿，那从长远来看并不真正重要。所以你怎么看这个问题？我的意思是，我猜你只能去削弱人们正在做的事情之类的。

So I can just steelman this for a minute though. Can you go to the other diagram where you have the—yeah, and if you click on them, then they show other similar compounds or similar genomes. So arguably, just steelmanning the opposing viewpoint: if you are on the frontier, which it seems like you are and you certainly strive to be, you're moving the frontier of the attack and the defense at the same time. So like right now, what you're doing clearly helps, but maybe if you're on the frontier, then that doesn't really matter in the long term. So how do you think about that? I mean, I guess you just have to nerf what people are doing or something.

是的。我们希望让人们开始思考的心态，不是‘哦，我们做个生物防御工具就完事了’。这基本上是一场军备竞赛，对吧？类似于网络安全领域。你会做出更好的技术，尤其是像 Fable 这样可能具有攻击性的东西。这意味着会有来回的较量。设计方会变得更强。防御方需要努力领先，对吧？然后这又会激励其他人在设计方那边超越，对吧？所以我认为本质上存在这种军备竞赛式的动态。在我们看来，防御方一直远远落后。所以我们想做的就是把防御方拉到接近同等水平。所以这就是我想象的心态，或者说我思考的框架。

Yeah. The mindset of what we were hoping to get folks to start thinking about is less of a 'oh, let's make a bio defense tool and call it a day.' It's basically an arms race, right? It's similar to the cyber security community. You're going to make better technology, especially with something like Fable that can potentially attack. And that means you're going to have this back and forth. The design side is going to get more capable. The defensive side needs to try to get ahead, right? And then that just motivates other folks to do past that too, right? On the design side. So I think inherently there is this arms race style dynamic. The way it appears to us is that the defensive side has been far, far lagging. And so what we want to do is bring the defensive side closer to par, essentially. So that's the mindset I picture or the framing I think about it.

网络安全的类比很有意思，但我认为它在一些关键点上有所不同。首先，我认为在网络安全方面，有了足够强大的模型，你也许真的能堵住所有非社会性的漏洞。有些漏洞总是很难，总有绕过的方法，但原则上你可以抓住每一个漏洞利用，我认为这在未来可能实现。然后你可以打补丁，只要人们保持更新，你就安全了。我们有固定的基因组，对吧？你不能给人类打补丁。我的意思是，一旦某个东西——所以我认为在某种意义上，攻击面以及防御的方式要难得多。但反过来，也许有人以同样程度发动攻击的动机似乎要小得多。而且成功的门槛——即使你能打印出任意 DNA，从那里到制造出成功的病毒，尤其是不会杀死设计者本人的病毒，这个过程实际上相当大。所以我很好奇：在你看来，我们实际面临的最大威胁是什么？

The cyber security analogy is interesting, but I think it differs in some key points. So first of all, I think that with cyber security, with a sufficiently strong model, you might actually be able to close all loopholes which are not sociological. There are certain ones which will always be hard, always be ways of getting around things, but you could in principle catch every single exploit, and I think that might be possible in the future. And then you can patch them, and as long as people stay updated, you know, you're secure. We have fixed genomes, right? So you can't patch a human. I mean, so once something—so I think that in some sense, the attack surfaces and the way that you defend against that is much higher or much harder. But then maybe the converse is that it seems much less likely that someone would have the incentive to go on offense to the same degree. And also the barrier to entry to success—even if you can print out arbitrary DNA, the process of going from that to making a successful virus, especially one which doesn't kill the person designing it, is actually quite large. So I guess I'm curious: what is the single, from your opinion, what is the single biggest threat that we actually have?

有什么事情让你睡不着觉，或者让你夜不能寐？是有什么特别的事情，还是你只是觉得这是我们需要构建的东西，那就去构建它？

What would be the thing which keeps you asleep or keeps you awake at night? Is there something in particular, or is this like you just think this is something we need to build and let's build it?

第一，很难进入一个想要设计生物武器的人的思维模式。

One, it's hard to get in the mindset of a person wanting to design a bioweapon.

是的。

Yeah.

所以我们并不一定要去设想所有潜在的人或场景，那些坏人可能会做什么。我大体上担心的是，我们正在降低所需专业知识的门槛，以及人们工程化这些东西的速度。这意味着数量在某个时候会指数级增长。所以混合了——是的，有故意的担忧，因为肯定有国家行为体拥有生物战项目，非常非常庞大。我们公司的一位顾问曾亲眼见过这些设施并参与拆除。所以我们听到很多关于国家确实有动机制造这种武器的故事。这是一个担忧。我认为在和平时期没那么可怕；在战时尤其可怕。我认为无意的那些，在我看来近期可能更有可能，人们确实尝试生成东西和控制功能，但不小心——也许他们想制造某种东西来理解和研究，但它泄漏了，对吧？因为这些东西可能很难控制，例如，会有泄漏。所以我认为这些场景似乎最有可能。这让我夜不能寐吗？不一定。我总体上更乐观。我认为构建这类技术最终是一场权衡利弊、成本和收益的游戏。我认为好处远远超过潜在的危害。这就是为什么我们研究它，因为最终我们确实认为它将成为发现和人类健康改善的引擎。但与此同时，我们只是觉得防御方在这场军备竞赛中有点落后。所以我们也研究它，并努力推动前沿。但总的来说，我认为作为一个研究者社区，以及政策方面的人，我认为社区有足够的韧性，有能力动员起来领先于它。所以我对此非常乐观。

So we're not trying to necessarily think of all the potential people or scenarios that bad actors might work on. What I think broadly what I worry about is we're lowering the bar for how much expertise is needed and the speed at which folks can engineer these kind of things. And that means the volume is going to just exponentially increase at some point. And so there's a mix of—yes, there's intentional worries because there's certainly state actors that have had biological war programs, very, very, very large ones. And one of our advisers on a company has physically seen these facilities and decommissioned them. And so we've heard a lot of stories about states actually being motivated to create such weapons. That is one concern. And I think during peacetime it's less scary; during wartime it's particularly scary. I think the unintentional ones are also things that in my mind potentially more likely in the near term, where folks do try to generate things and control function and inadvertently—maybe they tried to make a certain thing to understand and to study, but it got out, right? Because these things can be hard to contain, for example, and there's a leakage. So I think those scenarios potentially seem the most likely. Does it keep me up at night? Not necessarily. I'm much more of an overall optimist. I think building this type of technology is ultimately a game that you weigh out the pros and cons and the cost and benefits. I think the benefits far outweigh the potential harm. And so that's why we work on it, because ultimately we do think it's going to be an engine for discovery and human health improvement. But at the same time, we just felt like the defensive side was sort of losing this arms race. And so we work on it as well and try to push the frontier. But I'd say overall, I think as a community of researchers, but also folks on the policy side, I think the community is resilient enough and has the ability to mobilize to get ahead of it. So I'm very optimistic about it.

我们有两个问题喜欢问每位嘉宾。第一个是：如果你能通过法令消除一个对你很重要的瓶颈，那会是什么？

We have two questions that we like to ask every guest. So the first one is: if you could by fiat remove a bottleneck that is important to you, what would that be?

有意思。这个我能给两个答案吗？

Interesting. Could I get two answers on this one?

当然。

Sure.

我觉得第一个有点像是逃避，因为每个 AI 实验室都这么说：GPU。

I think the first one is kind of a copout because every AI lab says this: GPUs.

GPU。这个答案我们听过几次了。我们可以用 GPU。

GPUs. We've done that answer a few times. We can use GPUs.

另一个我觉得在这个领域更偏哲学层面的问题，也是我们试图、渴望去做的事，就是重塑科学家在这个领域的工作方式。我认为我们遇到的一个障碍是——你在很多领域都能看到——你在某个领域越专业，就越容易对那个领域感到悲观。我觉得这在生物领域尤其明显：你对某个疾病领域或模态了如指掌，然后有人引入新东西，你就会说，哦，但这个怎么办，那个怎么办，他们非常悲观，而且可能确实有道理。但我也注意到，在公司里，我们努力去做、努力引进的人，是既懂那个领域的专家，又仍然是梦想家，也就是说他们仍然有那种想象力，有改变做事方式的渴望。我觉得这个障碍在领域里很常见。如果我们能更多地拥抱这一点，就能看到更多进步和阶跃式变化，那是我很想看到的。

The other one, which I think is a little more philosophical in this space, and what we're trying to do, aspire to do, is reinvent how scientists do their work in the space. And I think one hurdle we run into is folks who — and you see this in many domains — the more expertise you have in something, the more pessimistic you become about that space. And I think you especially see this in bio, where you know a disease area or modality so well, and then someone introduces something else new, and you're like, oh but what about this and this and that, and they're very pessimistic, and probably rightfully so. But also I think what I've noticed at the company, what we strive to do, and the folks that we try to bring in, are domain experts that do know that field but also are still dreamers, meaning they still do have that imagination and desire to change how things are done. I think that barrier we see a lot in the field. I think if we embrace that more, we can see a lot more progress and step change that I would love to see.

这引出了最后一个问题：有没有什么你想让听众记住的，一个核心信息？

Brings us to the last question, which is: is there something you want the audience to take away, a single message?

是的，我觉得一个核心信息是：我认为人们，尤其是在 AI 研究社区，常常觉得有时必须做出选择——要么在前沿 AI 技术上工作，而那意味着消费级应用或企业级应用，他们只能做聊天机器人，那才是尖端技术；要么去做——你知道，当人们思考 AI 的真正潜力时，我认为很多都在于改善人类健康、理解我们的生物学。但人们觉得他们必须二选一，比如如果我在那个领域工作，我就不能在前沿 AI 工作。而我希望人们记住的是：你不必选择。我认为我们可以既做你真正关心、你认为能推动人类进步的事，又做尖端技术。这正是我们在 Radical Numerics 试图构建的。

Yeah, I think one message is: I think folks have had, especially in the AI research community, felt like there was a choice they had to make sometimes to either work on the frontier of AI technology — and that was like consumer-related apps or enterprise-related apps, and they just had to work on chatbots, and that's the cutting-edge technology — but also doing, you know, I think when people think about the true potential of what AI can do, and I think a lot of it is about improving human health, understanding our biology. But people have felt like they have had to choose, like if I work in that space I can't work in the frontier of AI. And what I would like people to take away is that you don't have to choose. I think we can work on things you truly care about, that you think will push humanity, and work on cutting-edge technology. And that's what we're trying to build at Radical Numerics.

是的，而且很明显——我鼓励大家去读那些博客文章、架构、机制——在构建这些模型的过程中有很多创新，我坚信生物学确实处在 AI 的最前沿。

Yeah, and I mean clearly — and I encourage people to read the blog posts, the architecture, the mechan — there's a lot of innovation that is going into building these models, and I firmly believe that biology is really on the forefront of AI.

太棒了。是的，很高兴你有同感。太好了。

Awesome. Yeah, glad to feel that way. Amazing.

谢谢你来聊天。

Thanks for chatting.

是的，谢谢你跑这么远——就几个街区而已。

Yeah, thank you for making the long trip — like blocks away.

随时。

Anytime.

感谢邀请。我玩得很开心。

Appreciate the invite. I had a blast.

太好了。谢谢。

Great. Thank you.