生物神经网络与人工神经网络的神秘差异

对你来说，生物神经网络与人工神经网络之间最神秘、最迷人、最深刻的差异是什么？

What difference between biological neural networks and artificial neural networks is most mysterious, captivating, and profound for you?

首先，我们对生物神经网络知之甚少，这非常神秘且迷人，因为也许它掌握着改进人工神经网络的关键。我最近研究的一件事是，我们不知道生物神经网络如何做到，但对人工神经网络却非常有用：在极长的时间跨度内进行信用分配。我们原则上可以用人工神经网络做到一些事情，但不太方便，也不符合生物学合理性。这种不匹配可能是一个有趣的研究方向，以便更好地理解大脑如何做到这些，因为我们没有与人工神经网络对应的好理论，并且可能提供新想法，探索大脑不同的运作方式，并将其融入人工神经网络。

First of all, there's so much we don't know about biological neural networks, and that's very mysterious and captivating because maybe it holds the key to improving our artificial neural networks. One of the things I studied recently is something we don't know how biological neural networks do but would be really useful for artificial ones: the ability to do credit assignment through very long time spans. There are things we can in principle do with artificial neural nets, but it's not very convenient and it's not biologically plausible. This mismatch may be an interesting thing to study, to understand better how brains might do these things, because we don't have good corresponding theories with artificial neural nets, and maybe provide new ideas that we could explore about things that brains do differently and that we could incorporate in artificial neural nets.

我们来分解一下信用分配。这是个很美的技术术语，但可以涵盖很多方面。它更偏向 RNN 记忆那一侧吗？还是关于随时间积累常识知识？或者更偏向强化学习，随时间获取奖励以实现特定目标？

Let's break credit assignment up a little bit. It's a beautifully technical term, but it could incorporate so many things. Is it more on the RNN memory side, thinking like that? Or is it something about knowledge building up common sense knowledge over time? Or is it more in the reinforcement learning sense, picking up rewards over time to achieve certain goals?

我更倾向于前两种含义，即我们在大脑中存储各种记忆，包括情景记忆，之后可以访问这些记忆，帮助我们推断当前观察到的事物的原因，并为之前存储记忆时做出的决策或解释分配信用。然后我们可以改变过去反应或解释事物的方式，这就是用于学习的信用分配。

I was thinking more about the first two meanings, whereby we store all kinds of memories, episodic memories, in our brain, which we can access later to help us both infer causes of things we are observing now and assign credit to decisions or interpretations we came up with a while ago when those memories were stored. Then we can change the way we would have reacted or interpreted things in the past, and that's credit assignment used for learning.

你认为人工神经网络，比如当前的 LSTM 和架构，在哪些方面无法捕捉这一点？大概你考虑的是非常长的时间跨度。

In which way do you think artificial neural networks, the current LSTM, the current architectures, are not able to capture that? Presumably you're thinking of very long term.

是的。当前的循环网络对于几十或几百个时间步的序列表现不错，但随着时间跨度变长，难度越来越大。而人类似乎能够在任意长的时间跨度内进行信用分配。例如，我记得去年做的一件事，现在因为看到新证据，我会改变去年思考的方式，希望不再犯同样的错误。

Yes. Current recurrent nets do a fairly good job for sequences with dozens or say hundreds of time steps, and then it gets harder and harder as you consider longer durations. Whereas humans seem to be able to do credit assignment through essentially arbitrary times. For example, I could remember something I did last year, and then now because I see some new evidence, I'm going to change my mind about the way I was thinking last year and hopefully not do the same mistake again.

我认为很大一部分原因可能是遗忘。你只记住真正重要的事情。高效的遗忘。

I think a big part of that is probably forgetting. You're only remembering the really important things. It's very efficient forgetting.

是的，所以我们对记忆有所选择。我认为这与高级认知有很酷的联系，涉及意识、决策和情感，比如决定什么进入意识、什么存入记忆，这些都不简单。

Yes, so there's a selection of what we remember. And I think there are really cool connections to higher-level cognition here, regarding consciousness, deciding, and emotions, like sort of deciding what comes to consciousness and what gets stored in memory, which are not trivial either.

你一直处于前沿，展示了神经网络、深度神经网络在 AI 中能做的一些惊人事情，广泛应用于各种领域。但你认为，深度神经网络表示世界的方式中最薄弱的方面是什么？缺少了什么？

You've been at the forefront showing some of the amazing things that neural networks, deep neural networks, can do in AI, broadly in all kinds of applications. But what, in your view, is the weakest aspect of the way deep neural networks represent the world? What is missing?

目前，在大量图像或文本上训练的最先进神经网络，对解释这些数据集有一定程度的理解，但非常基础、非常低级，远不如我们的理解那样鲁棒和抽象。这并没有告诉我们如何修复，但我认为它鼓励我们思考如何以不同方式训练神经网络，例如让它们专注于因果解释，这是当前神经网络训练没有做到的。另外，我下午演讲中会提到：我们不应分别从图像/视频和文本中学习，而需要更好地联合学习语言及其所指的世界，这样双方可以互相帮助。我们需要在神经网络中拥有良好的世界模型，才能让它们真正理解描述世界状态的句子。而且我认为需要语言输入来提供线索，告诉神经网络顶层应该表示哪些高级概念，比如语义概念。事实上，有证据表明，纯粹无监督的表示学习不会产生像监督学习那样强大的高级表示。所以仅凭标签（甚至不是句子）提供的线索就已经非常强大了。

Currently, state-of-the-art neural nets trained on large quantities of images or texts have some level of understanding of what explains those datasets, but it's very basic, very low-level, and not nearly as robust and abstract in general as our understanding. That doesn't tell us how to fix things, but I think it encourages us to think about how we can maybe train our neural nets differently, so that they would focus, for example, on causal explanations, something we don't do currently with neural net training. Also, one thing I'll talk about in my talk this afternoon is: instead of learning separately from images and videos on one hand and from text on the other, we need to do a better job of jointly learning about language and about the world to which it refers, so that both sides can help each other. We need to have good world models in our neural nets for them to really understand sentences which talk about what's going on in the world. And I think we need language input to help provide clues about what high-level concepts, like semantic concepts, should be represented at the top levels of these neural nets. In fact, there is evidence that purely unsupervised learning of representations doesn't give rise to high-level representations as powerful as those from supervised learning. So the clues we get just with labels, not even sentences, are already very powerful.

你认为这是架构挑战还是数据集挑战？

Do you think that's an architecture challenge or a dataset challenge?

都不是。我差点就想直接说“都不是”了。当然，数据集和架构总是需要调整的，但我认为关键更多在于训练目标和训练框架。例如，从被动观察数据转向主动智能体，通过干预世界来学习因果关系。那种能让最高层解释从学习中涌现的目标函数，我认为我们现在还没有。那种可以用来奖励探索、正确探索的目标函数。所以这些问题既不在数据集中，也不在架构中，而更多在于我们如何学习、在什么目标下学习等等。

Neither. I'm tempted to just end it there. Of course, datasets and architectures are something you always want to play with, but I think the crucial thing is more the training objectives, the training frameworks. For example, going from passive observation of data to more active agents which learn by intervening in the world, the relationships between causes and effects. The sort of objective functions which could be important to allow the highest level explanations to rise from learning, which I don't think we have now. The kinds of objective functions which could be used to reward exploration, the right kind of exploration. So these kinds of questions are neither in the dataset nor in the architecture, but more in how we learn, under what objectives, and so on.

你在多个场合提到儿童学习的方式：他们与世界中的物体互动。这很迷人，因为除了强化学习中的一些情况，这个想法并不属于人工神经网络的学习过程。你是否设想一种目标函数，比如“如果你以这种方式戳这个物体，将有助于我进一步学习”？

You mentioned in several contexts the idea of the way children learn: they interact with objects of the world. It seems fascinating because, except in some cases in reinforcement learning, that idea is not part of the learning process in artificial neural networks. Do you envision something like an objective function saying, 'You know what, if you poke this object in this kind of way, it would be really helpful for me to further learn'?

是的，有点像引导学习的某些方面。一小时前我和 Rebecca Saxe 交谈，她谈到大量证据表明婴儿似乎以有方向的方式关注他们感兴趣的事物。所以他们不是被动的学习者；他们专注于世界上那些最有趣、以非平凡方式令人惊讶的方面，这促使他们改变对世界的理论。所以这是对未来进展的一个迷人看法。

Yes, sort of almost guiding some aspect of learning. I was talking to Rebecca Saxe just an hour ago, and she was talking about lots of evidence for infants seeming to clearly take what interests them in a directed way. So they're not passive learners; they focus their attention on aspects of the world which are most interesting, surprising in a non-trivial way, that makes them change their theories of the world. So that's a fascinating view of future progress.

但一个可能更无聊的问题：你认为仅仅通过更深更大、增加规模，就能达到目标吗？

But a more maybe boring question: do you think going deeper and larger, just increasing the size of the things, will get us there?

我认为 Scaling（规模扩张）很重要，但还不够。我们需要新想法，特别是关于训练目标以及如何融入因果学习和主动互动。

I think scaling is important, but it's not sufficient. We need new ideas, especially about training objectives and how to incorporate causal learning and active interaction.

你提到的一些表征问题——它们在某种意义上有些浅薄，在抽象意义上更高。它们并没有获得某些……我不认为增加网络深度，比如从一百层变成一万层，就能解决我们的问题。你不这么认为吗？这对你来说显而易见吗？

Some of the representational issues you mentioned—they're kind of shallow in some sense, higher in a sense of abstraction. They're not getting some... I don't think that having more depth in the network, like instead of a hundred layers we have ten thousand, is going to solve our problem. You don't think so? Is that obvious to you?

是的。我清楚的是，工程师、公司、实验室和研究生们会继续调整架构，探索各种微调，让当前的技术水平略微提升。但我认为这远远不够。我们需要在学习方式上做出相当剧烈的改变，才能实现这些学习者真正深刻理解他们观察和行动的环境这一目标。

Yes. What is clear to me is that engineers, companies, labs, and grad students will continue to tune architectures and explore all kinds of tweaks to make the current state of the art ever so slightly better. But I don't think that's going to be nearly enough. I think we need some fairly drastic changes in the way we're considering learning to achieve the goal that these learners actually understand in a deep way the environment in which they are observing and acting.

但我想问一个比单纯增加层数更有趣的问题：基本上，一旦你找到一种通过交互来学习的方法，需要多少参数来存储这些信息？我认为我们的大脑比大多数神经网络要大得多，对吧？

But I guess I was trying to ask a question that is more interesting than just more layers: basically, once you figure out a way to learn through interacting, how many parameters does it take to store that information? I think our brain is quite bigger than most neural networks, right?

对，对。哦，我明白你的意思了。我同意你的看法。所以我同意，要构建具有典型成年人那样广泛世界知识的神经网络，我们目前的算力可能是不够的。好消息是，有硬件公司在制造神经网络芯片，所以情况会好转。然而，好消息在某种程度上也是坏消息：即使我们最先进的深度学习方法，也无法学习理解非常简单的环境，比如我们构建的一些网格世界。即使是这些相当简单的环境——当然，如果你用足够多的例子训练它们，最终它们也能学会，但就像人类可能只需要几十个例子，这些东西却需要数百万个例子来完成非常非常简单的任务。所以我认为，对于那些没有像谷歌那样算力的学者来说，有一个机会去做真正重要且激动人心的研究，以推进训练框架、学习模型、智能体学习在即使是合成且看似简单但当前机器学习却失败的环境中的技术水平。

Right, right. Oh, I see what you mean. I'm with you there. So I agree that in order to build neural nets with the kind of broad knowledge of the world that typical adult humans have, probably the kind of computing power we have now is going to be insufficient. So the good news is there are hardware companies building neural net chips, so it's going to get better. However, the good news in a way is also bad news: even our state-of-the-art deep learning methods fail to learn models that understand even very simple environments, like some grid worlds we have built. Even these fairly simple environments—of course, if you train them with enough examples, eventually they get it, but it's just like instead of what humans might need—just dozens of examples—these things will need millions for very, very simple tasks. So I think there's an opportunity for academics who don't have the kind of computing power that, say, Google has, to do really important and exciting research to advance the state of the art in training frameworks, learning models, agent learning in even simple environments that are synthetic, that seem trivial, but yet current machine learning fails on.

我们讨论过先验知识和常识知识。似乎我们人类把很多知识视为理所当然。那么你对这些先验知识、形成这种广泛的世界观、信息的积累，以及我们如何教神经网络或学习系统获取这些知识有什么看法？

We've talked about priors and common-sense knowledge. It seems like we humans take a lot of knowledge for granted. So what's your view of these priors, of forming this broad view of the world, this accumulation of information, and how we can teach neural networks or learning systems to pick that knowledge up?

知识——在人工智能领域有一段时间，也许是 80 年代，有过知识表示、知识获取、专家系统的时代。我的意思是，尽管符号 AI 是一个有趣的问题集，但它似乎被搁置了，因为它行不通。确实行不通。但那些目标仍然重要。你认为这些目标如何能够实现？

So knowledge—for a while in artificial intelligence, maybe in the 80s, there was a time of knowledge representation, knowledge acquisition, expert systems. I mean, though the symbolic AI was an interesting problem set to solve, and it was kind of put on hold a little bit, it seems like because it doesn't work. It doesn't work, that's right. But the goals of that remain important. How do you think those goals can be addressed?

对。

Right.

所以首先，我认为经典专家系统方法失败的一个原因是，我们拥有的很多知识——你谈到的常识直觉——有很多这样的知识是无法有意识获取的。我们做出的很多决定我们无法真正解释，即使有时我们会编造一个故事。而这些知识对于机器做出好的决策也是必要的，并且这些知识很难在专家系统、基于规则的系统以及 Prolog 等形式中编码。当然，旧 AI 还有其他问题，比如没有真正好的处理不确定性的方法。我想说一些更微妙的东西，我们现在理解得更好了，但我认为在人们心中仍然不够：分布式表征带来了非常强大的东西。真正让神经网络工作得这么好的东西，很难在符号世界中复制这种力量。专家系统中的知识被很好地分解成一系列规则，而如果你考虑神经网络，情况正好相反：你有一个巨大的参数块，它们紧密地协同工作来表示网络知道的一切，并且没有充分分解。所以我认为这是当前神经网络的弱点之一，我们必须从经典 AI 中吸取教训，以引入另一种组合性，这在语言和这些规则中很常见，但对神经网络来说并不自然。

So first of all, I believe that one reason why the classical expert systems approach failed is because a lot of the knowledge we have—you talked about common sense intuition—there's a lot of knowledge like this which is not consciously accessible. Lots of decisions we're taking that we can't really explain, even if sometimes we make up a story. And that knowledge is also necessary for machines to take good decisions, and that knowledge is hard to codify in expert systems, rule-based systems, and you know, Prolog or something. There are other issues, of course, with the old AI, like not really good ways of handling uncertainty. I would say something more subtle, which we understand better now, but I think still isn't enough in the minds of people: there is something really powerful that comes from distributed representations. The thing that really makes neural nets work so well, and it's hard to replicate that kind of power in a symbolic world. The knowledge in expert systems and so on is nicely decomposed into like a bunch of rules, whereas if you think about a neural net, it's the opposite: you have this big blob of parameters which work intensely together to represent everything the network knows, and it's not sufficiently factorized. So I think this is one of the weaknesses of current neural nets, that we have to take lessons from classical AI in order to bring in another kind of compositionality, which is common in language for example, and in these rules, but that isn't so native to neural nets.

沿着这个思路，解耦表征？

And on that line of thinking, disentangled representations?

是的，完全正确。所以多年来我一直认为，并且仍然相信，我们提出学习算法——无论是无监督、有监督还是强化学习等——来构建表征非常重要，在这些表征中，重要的因素（希望是因果因素）被很好地分离，并且容易从表征中提取。这就是解耦表征的思想：它说将数据转换到一个空间，在那里一切都变得容易，我们也许只需要用线性模型来学习我们关心的事情。我仍然认为这很重要，但我认为这遗漏了一个非常重要的成分，经典 AI 系统可以提醒我们这一点。假设我们有这些解耦表征；你仍然需要学习变量之间的关系，那些高级语义变量。它们不会是独立的——我的意思是，这假设性太强了。它们会有一些有趣的关系，可以预测未来，解释过去。在经典 AI 系统中，关于这些关系的知识被编码在规则中。就像一个规则就是一小块知识，说：‘哦，我有这两个、三个、四个变量以这种有趣的方式联系在一起；那么给定其他几个，我就可以对其中一两个做出推断。’对吧？除了解耦表征的元素（类似于基于规则的系统中的变量），你还需要解耦那些将这些变量联系在一起的机制，就像规则一样。所以规则被整齐地分离，每个规则独立存在，当我因为学习而改变一个规则时，它不需要破坏其他规则。而当前的神经网络，例如，对所谓的灾难性遗忘非常敏感，在我学了一些东西之后，再学新东西，它们可能会破坏我学过的旧东西。对吧？如果知识被更好地分解和分离……

Yes, exactly. So for many years I've thought, and I still believe, that it's really important that we come up with learning algorithms—either unsupervised or supervised or reinforcement, whatever—that build representations in which the important factors, hopefully causal factors, are nicely separated and easy to pick up from the representation. So that's the idea of disentangled representations: it says transform the data into a space where everything becomes easy, we can maybe just learn with linear models about the things we care about. And I still think this is important, but I think this is missing out on a very important ingredient which classical AI systems can remind us of. So let's say we have these disentangled representations; you still need to learn about the relationships between the variables, those high-level semantic variables. They're not going to be independent—I mean, this is like too much of an assumption. They're going to have some interesting relationships that allow to predict things in the future, to explain what happened in the past. The kind of knowledge about those relationships in a classical AI system is encoded in the rules. Like a rule is just like a little piece of knowledge that says, 'Oh, I have these two, three, four variables that are linked in this interesting way; then I can say something about one or two of them given a couple of others.' Right? In addition to disentangling the elements of the representation, which are like the variables in a rule-based system, you also need to disentangle the mechanisms that relate those variables to each other, like the rules. So the rules are neatly separated, like each rule is living on its own, and when I change a rule because I'm learning, it doesn't need to break other rules. Whereas current neural nets, for example, are very sensitive to what's called catastrophic forgetting, where after I've learned some things and then I learn new things, they can destroy the old things that I had learned. Right? If the knowledge was better factorized and separated...

如果做到解耦，就能避免很多问题。在感官域（比如像素空间）你做不到这一点。但我的想法是，当你把数据投影到正确的语义空间时，就有可能表示出从输入到表示这一转换之外的额外知识——也就是表示之间如何相互作用、如何预测未来等，而且可以做到干净地解耦。这样一来，规则之间是解耦的，而不仅仅是变量之间。

If you disentangle, you would avoid a lot of that. Now you can't do this in the sensory domain, like pixel space. But my idea is that when you project the data into the right semantic space, it becomes possible to represent extra knowledge beyond the transformation from input to representations. That is how representations act on each other and predict the future, in a way that can be neatly disentangled. So now the rules are disentangled from each other, not just the variables.

你区分了语义空间和像素空间。这需要架构上的差异吗？

You draw a distinction between semantic space and pixel space. Is there an architectural difference needed?

是的。感官空间（比如像素）里一切都是纠缠的。信息、变量以非常复杂的方式完全相互依赖。计算——不仅是变量，还有它们之间的关系——全都交织在一起。但我假设，在正确的高层表示空间中，变量以及它们之间的关系都可以解耦。这将提供很强的泛化能力。

Yes. There's the sensory space, like pixels, where everything is tangled. The information, the variables, are completely interdependent in very complicated ways. Computation, not just variables, but how they relate to each other, is all intertwined. But I hypothesize that in the right high-level representation space, both the variables and how they relate to each other can be disentangled. That will provide a lot of generalization power.

泛化能力？

Generalization power?

测试集的分布被假设与训练集的分布相同，这正是当前机器学习太弱的地方。它无法告诉我们如何泛化到新分布。人们可能会想，如果我们不知道新分布是什么，那就没什么可说的。但事实是，人类能够泛化到新分布。我们是怎么做到的？因为即使这些新分布看起来与训练情境非常不同，它们也有共同点。举个具体例子：你读一本科幻小说，它把你带到另一个星球，表面上一切都很不同，但物理定律是一样的。所以你能读懂这本书，理解发生了什么。分布非常不同，但因为你从地球带去了大量关于因果关系、物理机制甚至社会互动的知识，你就能理解这个视觉上完全不同的星球上发生了什么。

The distribution of the test set is assumed to be the same as the distribution of the training set. That's where current machine learning is too weak. It doesn't tell us anything about how we will generalize to a new distribution. People may think there's nothing we can say if we don't know what the new distribution will be. But the truth is, humans are able to generalize to new distributions. How are we able to do that? Because even though these new distributions could look very different from the training situations, they have things in common. Let me give you a concrete example: you read a science fiction novel. It brings you to some other planet where things look very different on the surface, but it's still the same laws of physics. You can read the book and understand what's going on. The distribution is very different, but because you can transport a lot of knowledge from Earth about underlying cause-effect relationships, physical mechanisms, and maybe even social interactions, you can make sense of what's happening on this planet where visually things are totally different.

把这个类比再延伸一下，我们进入一个科幻世界，比如《2001 太空漫游》，这大概是我最喜欢的 AI 电影之一。还有一部很多人喜欢、但可能有点超出 AI 社区的电影是《机械姬》。你看过吗？你对那部电影有什么看法？喜欢什么，讨厌什么？结合我想问的一个问题：有一个来自不同背景的庞大群体，通常不是 AI 领域的，他们担心人工智能的生存威胁。你看到这个群体在发展。你认为最好的方式是什么——在 AI 社区内外讨论安全、思考安全、进行对话？而且《机械姬》是公众了解 AI 的主要信息来源之一。

Taking that analogy further, let's enter a science fiction world like 2001: A Space Odyssey, which is probably one of my favorite AI movies. And then there's another one that a lot of people love, maybe a bit outside the AI community: Ex Machina. I don't know if you've seen it. What are your views on that movie? Things you like and things you hate. In the context of a question I want to ask: there's a large community of people from different backgrounds, often outside AI, who are concerned about existential threat from AI. You've seen this community develop over time. What do you think is the best way to talk about safety, to think about it, to have discourse about it within the AI community and outside, grounded in the fact that Ex Machina is one of the main sources of information for the general public about AI?

你说得对。我们在 AI 社区内部应该进行的讨论，与对公众真正重要的讨论之间有很大区别。《终结者》那种 AI 失控杀人、超级智能无论如何都会毁灭我们的画面，对公众讨论没什么用。对公众来说，真正重要的是 AI 对社会可能造成的短期和中期负面影响：安全方面，比如人脸识别带来的老大哥场景、杀人机器人；就业市场的影响；权力集中；歧视；各种社会问题。其中一些甚至可能威胁到民主。

I think you're putting it right. There's a big difference between the discussion we ought to have within the AI community and the discussion that really matters for the general public. The picture of Terminator, AI losing and killing people, superintelligence that will destroy us no matter what we try, isn't really useful for public discussion. For the public, the things that really matter are the short-term and medium-term likely negative impacts of AI on society: security issues like Big Brother scenarios with face recognition, killer robots, impact on the job market, concentration of power, discrimination, all kinds of social issues. Some of these could really threaten democracy, for example.

澄清一下，你说的杀人机器人是指自主武器吗？

Just to clarify, when you said killer robots, you mean autonomous weapons?

是的，武器系统。我不是说《终结者》。所以我认为这些短期和中期担忧应该成为公众辩论的重要部分。生存风险对我来说是一个非常不可能的情况，但仍然值得学术研究。就像你可以说，我们应该研究如果小行星撞击地球会怎样。我认为这不太可能发生，或者在可预见的未来发生。AI 失控的场景与我对至少当前机器学习和神经网络的认知相悖。我觉得这不可信。但当然，我没有水晶球，谁知道五十年后 AI 会怎样。所以我认为科学家研究这些问题是有价值的，但就我而言，这不是一个紧迫的问题。

Yes, weapon systems. I do not mean Terminator. So I think these short- and medium-term concerns should be an important part of the public debate. Existential risk, for me, is a very unlikely consideration, but still worth academic investigation. In the same way, you could say we should study what could happen if a meteorite came to Earth and destroyed it. I think it's very unlikely that this will happen, or happen in a reasonable future. The scenario of an AI getting loose goes against my understanding of at least current machine learning and current neural nets. It's not plausible to me. But of course, I don't have a crystal ball, and who knows what AI will be in fifty years. So I think it is worth scientists studying those problems, but it's not a pressing question as far as I'm concerned.

在继续之前，还有几个问题。但你对《机械姬》这部电影喜欢什么、不喜欢什么？我看了第二遍，挺喜欢的。第一次看很讨厌，但第二次看的时候学会了接受某些部分，就更喜欢了。这是一部概念电影。你的体验如何？

Before continuing, a few questions there. But what do you like and not like about Ex Machina as a movie? I watched it for the second time and enjoyed it. I hated it the first time, but enjoyed it more the second time when I learned to accept certain pieces of it. It's a concept movie. What was your experience?

负面的是它对科学的描绘。完全错误。科学，尤其是 AI，不是在一个隐秘的地方由一个非常聪明的人完成的。一个人？这完全不现实。不是这样的。即使是一个团队在某个孤立的地方也做不到。科学是通过大量人互动合作的社区，一步步前进的。所有领域的专家都知道发生了什么，即使在工业实验室里。信息会流动和泄露。它的精神与这部电影描绘的科学方式截然不同。

The negative is the picture it paints of science. It's totally wrong. Science in general, and AI in particular, is not happening in some hidden place by one really smart guy. One person? This is totally unrealistic. This is not how it happens. Even a team of people in some isolated place will not make it. Science moves by small steps thanks to the collaboration and community of a large number of people interacting. All the scientists who are expert in their field know what is going on, even in industrial labs. Information flows and leaks. The spirit of it is very different from the way science is painted in this movie.

到目前为止确实如此。即使研究发生在谷歌或 Facebook 内部，在公司内部，它还是会流出来。

It's been the case to this point. Even if research happens inside Google or Facebook, inside companies, it still comes out.

绝对如此。我认为永远都会是这样。那么，有没有可能把想法封存起来，以至于一系列突破完全不被整个研究社区发现？有可能，但不太可能。现在不是这样做的，我也无法预见会这样。

Absolutely. I think that will always be the case. So is it possible to bottle ideas to the point where a set of breakthroughs go completely undiscovered by the general research community? It's possible, but unlikely. It's not how it is done now, and it's not how I can foresee.

在可预见的未来，但当然我没有水晶球，谁知道呢——这毕竟是科幻。但通常那种不祥的感觉，比如讨论中灯灭了，问题又来了……电影是一回事，你可以想象各种科幻。对我来说，问题可能类似于关于存在风险的问题：这类电影描绘了与实际科学及其进展如此错误的画面，以至于可能对人们理解当前科学产生不幸的影响。这有点可悲。

In the foreseeable future, but of course I don't have a crystal ball, so who knows—this is science fiction after all. But usually the ominous thing, like the lights went off during that discussion, the problem again... There's one thing about the movie: you could imagine all kinds of science fiction. The problem for me may be similar to the question about existential risk: this kind of movie paints such a wrong picture of what the actual science is and how it's going on that it can have unfortunate effects on people's understanding of current science. That's kind of sad.

研究中的一个重要原则是多样性。换句话说，研究是探索，是在思想空间中的探索，不同的人会专注于不同的方向。这不仅是好的，而且是必要的。所以我完全接受人们探索与我相反或正交的方向。我不仅接受，而且认为这很重要。我和我的朋友并不声称我们拥有关于未来会发生什么的普遍真理。话虽如此，我们有我们的直觉，并据此行动，在我们认为最能发挥作用以及社会得失最大的地方。我们应该进行这些辩论，而不是最终陷入一个只有一种声音和一种思维方式的社会。在研究领域，资金是分散的，所以分歧是好的研究和好的科学的标志。

It is an important principle in research: diversity. In other words, research is exploration, exploration in the space of ideas, and different people will focus on different directions. This is not just good, it's essential. So I'm totally fine with people exploring directions that are contrary to mine or orthogonal to mine. I am more than fine; I think it's important. I and my friends don't claim we have universal truth about what will happen in the future. That being said, we have our intuitions and we act accordingly, where we think we can be most useful and where society has the most to gain or lose. We should have those debates and not end up in a society where there's only one voice and one way of thinking. In research, money is spread out, so disagreement is a sign of good research, good science.

是的，关于人类意义上的偏见。你如何看待在机器学习中灌输与人类价值观一致的东西，特别是在偏见方面？我们直觉上知道偏见意味着什么，以及对他人基本尊重意味着什么。但我们如何将其灌输到机器学习系统中呢？

Yes, the idea of bias in the human sense. How do you think about instilling in machine learning something that's aligned with human values in terms of bias? We intuitively have a concept of what bias means, of what fundamental respect for other human beings means. But how do we instill that into machine learning systems?

我认为有短期已经在发生的事情，也有长期我们需要做的事情。短期来看，有一些已经被提出的技术，我认为会继续改进，也许会出现替代方案。对于我们知道存在偏见的数据集——我们可以衡量它。几乎任何观察人类做决策的数据集都会有某种偏见，对特定群体的歧视等等。我们可以使用机器学习技术来构建偏差更小的预测器或分类器。例如，我们可以使用对抗性方法使我们的系统对这些我们不应敏感的变量不那么敏感。这些都是清晰、定义明确的解决问题的方法。也许它们有弱点，需要更多研究，但我认为它们已经足够成熟，政府应该开始监管相关公司，比如保险公司，强制它们使用这些技术。因为这些技术会减少偏见，但代价是——例如，它们的预测可能不那么准确——而公司不会主动去做，除非你强迫它们。所以这是短期。

I think there are short-term things that are already happening, and then there are long-term things that we need to do. In the short term, there are techniques that have been proposed and I think will continue to be improved, and maybe alternatives will come up. To take datasets in which we know there is bias—we can measure it. Pretty much any dataset where humans are being observed taking decisions will have some sort of bias, discrimination against particular groups, and so on. We can use machine learning techniques to try to build predictors or classifiers that are going to be less biased. We can do it, for example, using adversarial methods to make our systems less sensitive to these variables we should not be sensitive to. These are clear, well-defined ways of trying to address the problem. Maybe they have weaknesses and more research is needed, but I think they are sufficiently mature that governments should start regulating companies where it matters, say like insurance companies, so that they use those techniques. Because those techniques will reduce bias, but at a cost—for example, their predictions might be less accurate—and companies will not do it until you force them. So this is short term.

长期来看，我真的很感兴趣如何将道德价值观灌输给计算机。显然，这不是我们在未来五到十年内能实现的。我们如何……已经有关于检测情绪的工作，例如在图像、声音和文本中，以及研究不同智能体以不同方式交互如何可能对应不公正的模式，从而引发愤怒。这些是我们在中期可以做的事情。最终，我们可以训练计算机模拟人类的情感反应。我想说最简单的是不公平的情境会引发愤怒——这是人类与其他动物共有的最基本情感之一。我认为在未来几年内这是相当可行的。因此，我们可以构建能够检测这类事物的系统，前提是它们对周围世界有足够理解——这还很遥远——但也许我们可以在虚拟环境中先做。你可以想象一个视频游戏，智能体以某种方式交互，然后某些情境触发情绪。我认为我们可以训练机器检测这些情境，并预测如果人类扮演其中一个角色，可能会感受到特定的情绪。

Long term, I'm really interested in thinking of how we can instill moral values into computers. Obviously, this is not something we'll achieve in the next five or ten years. How can we... There's already work in detecting emotions, for example in images, sounds, and texts, and also studying how different agents interacting in different ways may correspond to patterns of injustice, which could trigger anger. These are things we can do in the medium term. Eventually, we can train computers to model how humans react emotionally. I would say the simplest thing is unfair situations which trigger anger—this is one of the most basic emotions that we share with other animals. I think it's quite feasible within the next few years. So we can build systems that can detect these kinds of things, to the extent that they understand enough about the world around us—which is a long time away—but maybe we can initially do this in virtual environments. You can imagine a video game where agents interact in some ways, and then some situations trigger an emotion. I think we could train machines to detect those situations and predict that a particular emotion would likely be felt if a human was playing one of the characters.

你表现出热情，并在监督学习方面做了很多出色的工作。但在超级细菌上……监督学习取得了很大成功。是的，是的。我真正热衷的事情之一是人类和机器人如何协同工作。在监督学习的背景下，这意味着标注过程。你考虑过标注的问题吗？或者换一种更有趣的说法，是人类在教机器吗？

You have shown excitement and done a lot of excellent work with supervised learning. But on a superbug... there's been a lot of success with supervised learning. Yes, yes. And one of the things I'm really passionate about is how humans and robots work together. In the context of supervised learning, that means the process of annotation. Do you think about the problem of annotation? Or put in a more interesting way, is humans teaching machines?

是的，我认为这是一个重要的主题。将其简化为标注可能对明天构建系统的人有用，但从长远来看，教学过程我认为值得机器学习社区更多关注。所以有人创造了“机器教学”这个术语。教授一个学习智能体的好策略是什么？我们能否设计或训练一个系统，使其成为一个好老师？在我的团队中，我们有一个名为“婴儿 AI”或“婴儿 AI 游戏”的项目，其中有一个游戏或场景，包含一个学习智能体和一个教学智能体。教学智能体最终可能是人类，但我们还没到那一步。教师的角色是利用其对环境的知识——它可以通过任何方式获取，比如暴力搜索——来帮助学习者尽可能快地学习。学习者会尝试通过探索等方式学习，但教师可以选择影响与学习者的互动，以引导学习者，也许教给学习者最困难的东西，或者正好在其已知和未知的边界上，等等。这些想法在其他领域有传统，比如辅导系统。当然，人文学科的人一直在思考这些问题。但我认为现在是机器学习领域的人关注这个问题的时候了，因为未来我们将有越来越多的人机交互，人类参与其中。理解如何使其更好地工作——围绕这个的所有问题——都非常有趣，且尚未得到充分解决。

Yes, I think it's an important subject. Reducing it to annotation may be useful for somebody building a system tomorrow, but longer-term, the process of teaching, I think, is something that deserves a lot more attention from the machine learning community. So people have coined the term 'machine teaching'. What are good strategies for teaching a learning agent? And can we design or train a system that is going to be a good teacher? In my group, we have a project called the Baby AI or Baby AI Game, where there is a game or scenario with a learning agent and a teaching agent. Presumably, the teaching agent would eventually be a human, but we're not there yet. The role of the teacher is to use its knowledge of the environment—which it can acquire using whatever way, brute force—to help the learner learn as quickly as possible. The learner is going to try to learn by maybe using some exploration and whatever, but the teacher can choose to have an influence on the interaction with the learner so as to guide the learner, maybe teach it the things that the learner has most trouble with, or just at the boundary between what it knows and doesn't know, and so on. There is a tradition of these ideas from other fields, like tutorial systems, for example. And of course, people in the humanities have been thinking about these questions. But I think it's time that machine learning people look at this, because in the future we'll have more and more human-machine interaction with a human in the loop. Understanding how to make this work better—all the problems around that—are very interesting and not sufficiently addressed.

你在语言方面做了很多工作。传统图灵测试——一个测试自然语言理解和生成的测试——在你看来哪个方面最难？就对话而言，在你看来，机器最难解决的部分是什么？

You've done a lot of work with language. What aspect of the traditionally formulated Turing test—a test of natural language understanding and generation—in your eyes is the most difficult? Of conversation, but in your eyes, what is the hardest part of conversation to solve for machines?

我会说，一切与非语言知识有关的东西，这些知识是你隐含需要的，以便……

I would say it's everything having to do with the non-linguistic knowledge which implicitly you need in order to...

理解像 Winograd 模式这样的句子。这些句子在语义上是模糊的；你需要对世界有足够的了解才能解释它们。我认为这些是机器学习的有趣挑战，因为它们指向构建理解世界运作方式、世界因果关系，并将这些知识与语言表达（无论是阅读还是写作）相关联的系统。

Make sense of sentences like the Winograd schemas. These sentences are semantically ambiguous; you need to understand enough about the world to interpret them. I think these are interesting challenges for machine learning because they point toward building systems that understand how the world works, its causal relationships, and associate that knowledge with language expression, either for reading or writing.

你说法语，对吧？那是你的母语，是罗曼语之一。你认为通过图灵测试及其背后的挑战依赖于语言吗？在法语中是否更容易，还是与语言无关？

You speak French, yes? It's your mother tongue, one of the Romance languages. Do you think passing the Turing test and the underlying challenges depend on language? Might it be easier in French, or is it independent of language?

我认为这与语言无关。我希望构建的系统能够使用相同的原理和学习机制，从人类智能体那里学习，无论他们使用什么语言。当然，人类可以用诗歌更优美地表达。俄语诗歌可能比英语更容易传达复杂思想，但这是我的偏见。有些人可能对法语也有同感。最终，人脑可以利用任何语言作为工具来传达意义。存在差异，但在理解大脑如何工作和语言的大框架下，这些差异是细微的。

I think it's independent of language. I would like to build systems that use the same principles and learning mechanisms to learn from human agents, whatever their language. Certainly, humans can speak more beautifully in poetry. Russian poetry might convey complex ideas more easily than English, but that's my bias. Some might say the same about French. Ultimately, the human brain can use any language as a tool to convey meaning. There are differences, but in the grand scheme of understanding how the brain works and language, these differences are minutiae.

你经历过 AI 寒冬。你是如何保持热情并继续前行的？

You've lived through an AI winter. How did you stay warm and continue?

和朋友一起保持温暖。有朋友很重要。我学到的是：倾听你内心的声音。不要试图取悦大众或追随潮流。如果你对某件事有强烈的直觉，且没有与事实证据相矛盾，就去做。它可能会被人反对，但不会被你基于所知的一切的直觉所反对。当然，当实验与你的信念相矛盾时，你必须调整信念，但除此之外要坚持信念。正是这一点让我度过了那些年，并在那些需要时间成熟并结出果实的方向上坚持下去。

Stay warm with friends. It's important to have friends. What I learned: listen to your inner voice. Don't try to please the crowds or follow fashion. If you have a strong intuition about something not contradicted by actual evidence, go for it. It might be contradicted by people, but not by your own instinct based on everything you know. Of course, you must adapt your beliefs when experiments contradict them, but stick to your beliefs otherwise. That's what allowed me to persist through those years and in directions that took time to mature and bear fruit.

AI 的历史以技术突破和像 AlphaGo 击败世界冠军这样的标志性事件为标志。你认为下一个这样的时刻可能是什么？

History of AI is marked by technical breakthroughs and seminal events like AlphaGo beating the world champion. What do you think the next such moment might be?

首先，我认为这些标志性事件被高估了。科学是通过小步前进的。一小步可以填满桶，产生巨大后果，比如实现新功能或降低解决方案成本，从而打开新市场。在科学本身，进步是非常渐进的。目前，我所在研究所的两个热门话题是 GAN 和强化学习。在蒙特利尔，强化学习在两三年前还几乎不存在，但现在引起了很大兴趣。我认为我们会看到更多进展，尽管除了 AlphaGo 之外，它还没有产生太多工业影响。从长远来看，强化学习和更一般的智能体学习非常重要。

First, I think these seminal events are overrated. Science moves by small steps. One small step can fill the bucket and have drastic consequences, like enabling something new or making a solution cheaper, opening a new market. In science itself, progress is very gradual. Currently, two hot topics at my institute are GANs and reinforcement learning. In Montreal, reinforcement learning was absent just two or three years ago, but now there's big interest. I think we'll see more progress there, even though it hasn't yet had much industrial impact beyond AlphaGo. Over the long term, reinforcement learning and more generally agent learning are really important.

你对强化学习感到兴奋。你认为 GANs 能带来一个标志性时刻吗？

You're excited about reinforcement learning. Do you think GANs could provide a seminal moment?

GAN 或其他生成模型将成为构建理解世界的智能体的关键要素。过去强化学习的许多成功都使用了策略梯度，学习策略而不学习世界模型。但这存在问题，我们还不知道如何做好基于模型的强化学习。我认为这是我们需要前进的方向，以构建更快、更好地泛化的模型，捕捉潜在的因果机制。

GANs or other generative models will be crucial ingredients in building agents that understand the world. Many past successes in reinforcement learning used policy gradient, learning a policy without a world model. But there are issues, and we don't know how to do model-based RL well yet. I think that's where we need to go to build models that generalize faster and better, capturing underlying causal mechanisms.

是什么让你爱上了人工智能？你第一次对人类心智或人工心智着迷是什么时候？

What made you fall in love with artificial intelligence? When was the first moment you were fascinated by the human mind or the artificial mind?

当我不上课的时候，我读了很多书。我开始读科幻小说。那就是我着迷的地方。然后我有了第一批个人电脑之一，并迷上了编程。从科幻开始，然后把它变成现实。

When I wasn't in class, I was reading a lot. I started reading science fiction. That's where I got hooked. Then I had one of the first personal computers and got hooked on programming. It started with fiction and then making it a reality.

Yoshua，非常感谢你接受我的采访。

Yoshua, thank you so much for talking to me.

我的荣幸。

My pleasure.