04.srt

1
00:00:01,775 --> 00:00:04,847
字幕组：赵含霖 谢鑫鑫

2
00:00:05,556 --> 00:00:07,732
Hello,大家好,我是 ZOMI 酱

3
00:00:07,732 --> 00:00:09,732
欢迎来到 ZOMI 的课堂

4
00:00:10,000 --> 00:00:14,150
上一节课真的是不管是你听的还是我讲的

5
00:00:14,150 --> 00:00:16,150
都是脑瓜子嗡嗡的

6
00:00:18,792 --> 00:00:22,000
这也是所有的分享里面最难的了

7
00:00:22,000 --> 00:00:27,000
那今天呢我想给大家一起去看一看自动微分是怎么实现的

8
00:00:27,000 --> 00:00:29,000
就是它的具体实现方式了

9
00:00:29,000 --> 00:00:33,000
自动微分的具体实现方式呢分三个

10
00:00:33,000 --> 00:00:36,000
第一个是基于表达式或者基于图的

11
00:00:36,000 --> 00:00:39,000
那第二个呢就是操作符重载这个概念

12
00:00:39,000 --> 00:00:41,000
基本上只有在高级语言里面才会有的

13
00:00:41,000 --> 00:00:44,000
那第三个呢就是源码转换

14
00:00:44,000 --> 00:00:47,000
源码转换也是最难实现最复杂的

15
00:00:47,000 --> 00:00:51,000
所以呢会在下面两节分享里面呢

16
00:00:51,000 --> 00:00:55,000
去讲讲基于表达式或者操作符里面去怎么去实现的

17
00:00:55,000 --> 00:00:58,000
基于源码转换啊说实话你放过我吧

18
00:00:58,000 --> 00:01:00,000
这些实现起来太麻烦了

19
00:01:00,000 --> 00:01:02,000
了解基本概念就好了

20
00:01:02,000 --> 00:01:06,000
基于表达式或者表达方法去实现的自动微分

21
00:01:06,000 --> 00:01:10,000
就有应该在 12 年推出的 AutoDiff

22
00:01:10,000 --> 00:01:13,000
那 AutoDiff 呢它基于表达式的几种实现方式呢

23
00:01:13,000 --> 00:01:17,000
主要是它封装了基本的表达作为一个库函数

24
00:01:17,000 --> 00:01:20,000
这是我的加减乘除的每一次的导数

25
00:01:20,000 --> 00:01:23,000
我都要去手工的写一遍

26
00:01:23,000 --> 00:01:28,000
运行的时候呢大家会去记录基本表达式的一个相关的组合关系

27
00:01:28,000 --> 00:01:31,000
就我的加减乘除这些都要记下来

28
00:01:31,000 --> 00:01:33,000
我加我要算一遍加

29
00:01:33,000 --> 00:01:36,000
然后加的导数我要记下来

30
00:01:36,000 --> 00:01:38,000
然后去不是人工的记啊

31
00:01:38,000 --> 00:01:40,000
是那个写程序的时候记下来啊

32
00:01:40,000 --> 00:01:43,000
然后最后再进行一个组合

33
00:01:43,000 --> 00:01:46,000
就实现的过程非常之复杂

34
00:01:46,000 --> 00:01:50,000
第二种实现方式呢就是操作符重载

35
00:01:51,000 --> 00:01:54,000
一看到这个 logo 呢我就不困了

36
00:01:57,000 --> 00:01:59,000
这个呢就是 PyTorch

37
00:01:59,000 --> 00:02:01,000
其余操作符存在除了 PyTorch 呢

38
00:02:01,000 --> 00:02:04,000
其实它没有一个非常之明显的界限

39
00:02:04,000 --> 00:02:07,000
TensorFlow 也是基于操作符重载的方式去实现的

40
00:02:07,000 --> 00:02:10,000
只是可能细节上呢有些不一样

41
00:02:11,000 --> 00:02:14,000
那操作符重载呢就是利用语言的多态性

42
00:02:14,000 --> 00:02:18,000
用操作符去重载基本的运算表达式

43
00:02:18,000 --> 00:02:21,000
也就是我的加减乘除求根号这些

44
00:02:21,000 --> 00:02:22,000
它都重载了一遍

45
00:02:22,000 --> 00:02:26,000
那重载的过程当中呢就用了一个记录器

46
00:02:26,000 --> 00:02:28,000
把这些计算记录下来

47
00:02:28,000 --> 00:02:31,000
然后呢根据链式求导法则组合

48
00:02:31,000 --> 00:02:34,000
那第三种呢就是源码转换啊

49
00:02:34,000 --> 00:02:36,000
源码转换最典型的代表呢

50
00:02:36,000 --> 00:02:39,000
就是华为自研的一个 MindSpore

51
00:02:39,000 --> 00:02:41,000
就叫做头孢啊

52
00:02:41,000 --> 00:02:45,000
简单的 Mind 就是头嘛 Spore 就是孢子嘛

53
00:02:45,000 --> 00:02:46,000
对简单的头孢

54
00:02:46,000 --> 00:02:48,000
实际上呢它不叫头孢

55
00:02:48,000 --> 00:02:51,000
它有它自己的中文名叫做昇思

56
00:02:51,000 --> 00:02:52,000
对吧声腾的声

57
00:02:52,000 --> 00:02:54,000
昇腾产业线里面的昇

58
00:02:54,000 --> 00:02:55,000
思考的思

59
00:02:55,000 --> 00:02:56,000
是 Spore 嘛

60
00:02:56,000 --> 00:02:57,000
思考啊

61
00:02:57,000 --> 00:03:00,000
最主要的是基于一个语言的预处理

62
00:03:00,000 --> 00:03:03,000
或者编译器或者解析器去进行一个扩展的

63
00:03:03,000 --> 00:03:06,000
把 Python 的源码进行转换

64
00:03:06,000 --> 00:03:07,000
转换完之后呢

65
00:03:07,000 --> 00:03:12,000
再对它这些计算呢进行一个组织整理

66
00:03:12,000 --> 00:03:14,000
然后最后通过链式求导法则

67
00:03:14,000 --> 00:03:15,000
然后进行组合的

68
00:03:15,000 --> 00:03:17,000
所以呢它会比较复杂

69
00:03:17,000 --> 00:03:19,000
它需要去写一个编译器啊

70
00:03:19,000 --> 00:03:21,000
或者对源码的解析器啊

71
00:03:21,000 --> 00:03:24,000
这里面呢就不会展开太多

72
00:03:24,000 --> 00:03:27,000
又回到这一篇文章了

73
00:03:27,000 --> 00:03:30,000
这篇文章呢还是非常的经典的

74
00:03:30,000 --> 00:03:31,000
应该是 18 年的时候出的

75
00:03:31,000 --> 00:03:34,000
这里面呢就总结了非常多的 AI 框架

76
00:03:34,000 --> 00:03:37,000
然后包括刚才说的 Tangent 啊 PyTorch 啊

77
00:03:37,000 --> 00:03:40,000
还有那个 Autogrid 啊

78
00:03:40,000 --> 00:03:42,000
不同好多框架

79
00:03:42,000 --> 00:03:43,000
但只是到 18 年了

80
00:03:43,000 --> 00:03:44,000
MindSpore 还没有推出

81
00:03:45,000 --> 00:03:47,000
TensorFlow 它没有统计在这里面

82
00:03:47,000 --> 00:03:49,000
它就总结了就三种模式

83
00:03:49,000 --> 00:03:51,000
这里面以这三种模式来展开

84
00:03:51,000 --> 00:03:53,000
那第一种呢就是 Library

85
00:03:53,000 --> 00:03:55,000
看一下基于库的一个表达方式

86
00:03:55,000 --> 00:03:58,000
以一个 f (x ,y ,z) 

87
00:03:58,000 --> 00:04:02,000
这一个简单的表达去展开介绍一下

88
00:04:02,000 --> 00:04:04,000
手工呢去展开一下

89
00:04:04,000 --> 00:04:06,000
首先把 x+y 展开

90
00:04:06,000 --> 00:04:10,000
然后呢再把(x+y)/z 展开成为 b

91
00:04:10,000 --> 00:04:13,000
所以呢就变成下面两套公式

92
00:04:13,000 --> 00:04:14,000
那有了下面两套公式

93
00:04:14,000 --> 00:04:16,000
我对 a 进行求导的时候

94
00:04:16,000 --> 00:04:18,000
我需要人工的去算一遍

95
00:04:18,000 --> 00:04:19,000
我需要人工的去算一遍

96
00:04:19,000 --> 00:04:20,000
我算 da 的导数

97
00:04:20,000 --> 00:04:23,000
那是不是算 dx 加 dy 啊

98
00:04:23,000 --> 00:04:24,000
那我算除的导数呢

99
00:04:24,000 --> 00:04:26,000
可能会稍微复杂一点啊

100
00:04:26,000 --> 00:04:27,000
除了就更复杂了

101
00:04:27,000 --> 00:04:28,000
就不再展开

102
00:04:28,000 --> 00:04:31,000
基本就是求导法则去展开的

103
00:04:31,000 --> 00:04:33,000
优点就是实现很简单

104
00:04:33,000 --> 00:04:35,000
我可以用任何语言就可以去实现了

105
00:04:35,000 --> 00:04:39,000
你想想我去实现 a 等于 x 加 y

106
00:04:39,000 --> 00:04:40,000
然后 da 的导数

107
00:04:40,000 --> 00:04:42,000
这个很好实现呢

108
00:04:42,000 --> 00:04:46,000
缺点就是我需要去了解库函数

109
00:04:46,000 --> 00:04:49,000
然后去使用库函数的底层进行编码

110
00:04:49,000 --> 00:04:52,000
没办法去用原语言进行编码

111
00:04:52,000 --> 00:04:54,000
TensorFlow 觉得很别扭

112
00:04:54,000 --> 00:04:56,000
你必须去记它的规则

113
00:04:56,000 --> 00:04:59,000
就因为它部分的采用了这种思想

114
00:04:59,000 --> 00:05:03,000
那下面来看看一些实际上的代码

115
00:05:03,000 --> 00:05:04,000
大家不要慌

116
00:05:04,000 --> 00:05:06,000
看到代码千万不要慌

117
00:05:06,000 --> 00:05:07,000
这边很简单

118
00:05:07,000 --> 00:05:08,000
我的 adadd

119
00:05:08,000 --> 00:05:09,000
我的输入有很多个啊

120
00:05:09,525 --> 00:05:14,000
x，还有我的 y,还有我的 z

121
00:05:14,000 --> 00:05:16,000
分别都是一对的一对的 pair 的

122
00:05:16,000 --> 00:05:17,000
为啥呢

123
00:05:17,000 --> 00:05:19,000
因为我输入的时候我要算它的导数啊

124
00:05:19,000 --> 00:05:20,000
我要取它的导数啊

125
00:05:20,000 --> 00:05:21,000
对吧

126
00:05:21,000 --> 00:05:22,000
我输入等于输出

127
00:05:22,000 --> 00:05:24,000
如果它是在 C++里面

128
00:05:24,000 --> 00:05:25,000
它是一个取地址的话

129
00:05:25,000 --> 00:05:28,000
那我可以拿到它的自己的值嘛

130
00:05:28,000 --> 00:05:29,000
那这个时候呢

131
00:05:29,000 --> 00:05:31,000
我的 z 就等于 x 加 y

132
00:05:31,000 --> 00:05:33,000
我的 dz 就是我的输出啊

133
00:05:33,000 --> 00:05:34,000
等于 dx 加 dy

134
00:05:34,000 --> 00:05:35,000
那实际上呢

135
00:05:35,000 --> 00:05:37,000
就对应刚才的两条公式

136
00:05:37,000 --> 00:05:39,000
那一个除的一个方式呢

137
00:05:39,000 --> 00:05:41,000
同样我的输入呢

138
00:05:41,000 --> 00:05:42,000
也是一对一对的

139
00:05:42,000 --> 00:05:44,000
里面的具体的计算呢

140
00:05:44,000 --> 00:05:46,000
还是那个计算

141
00:05:46,000 --> 00:05:47,000
那怎么去用啊

142
00:05:48,000 --> 00:05:49,000
用很简单的是吧

143
00:05:49,000 --> 00:05:51,000
我的库都已经实现好了

144
00:05:51,000 --> 00:05:52,000
用的时候呢

145
00:05:52,000 --> 00:05:55,650
你看一下我直接去调这个 ADAdd

146
00:05:56,000 --> 00:05:57,000
然后把数据输进来

147
00:05:57,000 --> 00:05:58,000
不就可以用了吗

148
00:05:58,000 --> 00:06:01,000
然后我去调我的 AddDiv

149
00:06:01,000 --> 00:06:02,000
就可以输出来

150
00:06:02,000 --> 00:06:03,000
直接可以用了

151
00:06:03,000 --> 00:06:06,000
我对这个公式进行使用的时候

152
00:06:06,000 --> 00:06:08,000
我调两次不就得了吗

153
00:06:08,000 --> 00:06:09,000
哎对

154
00:06:09,000 --> 00:06:10,000
麻烦吧

155
00:06:10,000 --> 00:06:11,000
麻烦

156
00:06:11,000 --> 00:06:13,000
现在很少用这种方式了

157
00:06:13,000 --> 00:06:15,000
但是十年前这种方式啊

158
00:06:15,000 --> 00:06:16,000
是非常普遍的

159
00:06:16,000 --> 00:06:17,000
那个时候自动微分

160
00:06:17,000 --> 00:06:19,000
还真的没有像今天的自动微分

161
00:06:19,000 --> 00:06:20,000
这么方便

162
00:06:20,000 --> 00:06:21,000
因为自动微分

163
00:06:21,000 --> 00:06:24,000
它依赖于一个这么方便的

164
00:06:24,000 --> 00:06:28,859
就是有了高级语言的操作符重载

165
00:06:30,109 --> 00:06:31,000
大二的时候

166
00:06:31,000 --> 00:06:33,000
刚学这个 c++的多态性

167
00:06:33,000 --> 00:06:34,000
我就特别苦恼

168
00:06:34,000 --> 00:06:35,000
简直没法理解

169
00:06:35,000 --> 00:06:37,000
但现在其实等一下

170
00:06:37,000 --> 00:06:39,000
下一节课讲的时候

171
00:06:39,000 --> 00:06:40,000
其实它变得非常简单了

172
00:06:40,000 --> 00:06:42,000
然后利用语言的多态性

173
00:06:42,000 --> 00:06:44,000
把每一个计算

174
00:06:44,000 --> 00:06:45,000
我都记录成

175
00:06:45,000 --> 00:06:47,000
记录在我的 Tape 里面

176
00:06:47,000 --> 00:06:48,000
把每个计算都记下来

177
00:06:48,000 --> 00:06:50,000
例如我现在算我的 op

178
00:06:50,000 --> 00:06:51,000
就是我第一个操作

179
00:06:51,000 --> 00:06:53,000
我算假设是乘

180
00:06:53,000 --> 00:06:56,000
我通过 Tape 去把我的这个记下来

181
00:06:56,000 --> 00:06:58,000
然后我算第二个的时候

182
00:06:58,000 --> 00:07:00,000
我又通过我的 Tape 去记下来

183
00:07:00,000 --> 00:07:01,000
记下来之后

184
00:07:01,000 --> 00:07:03,000
我这里面用户是不感知的

185
00:07:03,000 --> 00:07:05,000
因为我已经操作符重载了

186
00:07:05,000 --> 00:07:06,000
我可以随便的去改

187
00:07:06,000 --> 00:07:07,000
我里面的内容是吧

188
00:07:07,000 --> 00:07:09,000
计算机你随便让我记

189
00:07:09,000 --> 00:07:11,000
我就记录下来

190
00:07:11,000 --> 00:07:14,000
然后通过 Tape 对 Tape 进行便利

191
00:07:14,000 --> 00:07:16,000
对里面的一些基本操作

192
00:07:16,000 --> 00:07:18,000
去计算我的微分

193
00:07:18,000 --> 00:07:22,000
这样用户用起来就比较简单了

194
00:07:22,000 --> 00:07:24,000
所以说它的应用性很高

195
00:07:24,000 --> 00:07:26,000
非常贴合原生语言

196
00:07:26,000 --> 00:07:27,000
听不懂也没关系

197
00:07:27,000 --> 00:07:29,000
你会用这个 AI 框架就好了

198
00:07:29,000 --> 00:07:31,000
现在还在讲自动微分

199
00:07:31,000 --> 00:07:32,000
还没讲到 AI 框架

200
00:07:33,000 --> 00:07:35,000
那缺点就是我要显示地

201
00:07:35,000 --> 00:07:37,000
构造 Tape 数据结构

202
00:07:37,000 --> 00:07:39,000
和对 Tape 进行读写和遍历

203
00:07:39,000 --> 00:07:40,000
这个就非常麻烦了

204
00:07:40,000 --> 00:07:43,000
Tape 就对程序员来说就要求很高了

205
00:07:43,000 --> 00:07:45,000
对用户来说要求很少了

206
00:07:45,000 --> 00:07:48,000
另外还要有一个额外的数据结构

207
00:07:48,000 --> 00:07:51,000
引入操作不利于高阶微分

208
00:07:51,000 --> 00:07:53,000
这里面只是记录一个 Tape

209
00:07:53,000 --> 00:07:56,000
然后对它进行一次求导还好

210
00:07:56,000 --> 00:07:58,000
二阶求导呢

211
00:07:58,000 --> 00:07:59,000
三阶求导呢

212
00:07:59,000 --> 00:08:00,000
更高阶的求导呢

213
00:08:00,000 --> 00:08:02,000
哎呀这个操作就非常难了

214
00:08:02,000 --> 00:08:03,000
那另外还有呢

215
00:08:03,000 --> 00:08:05,000
它的 if else while 这种呢

216
00:08:05,000 --> 00:08:09,000
表达式是非常难以通过操作符重载去计算的

217
00:08:09,000 --> 00:08:11,000
也就是 if else while

218
00:08:11,000 --> 00:08:13,000
怎么放在我的操作符里面呢

219
00:08:13,000 --> 00:08:14,000
怎么变成一个图呢

220
00:08:14,000 --> 00:08:18,000
操作符重载是有它自己的毛病的

221
00:08:18,000 --> 00:08:20,000
那看一下操作符重载

222
00:08:20,000 --> 00:08:21,000
其实很简单的

223
00:08:21,000 --> 00:08:23,000
在这个是 python 的伪代码了

224
00:08:23,000 --> 00:08:25,000
去把这个 mul 还有 add

225
00:08:25,000 --> 00:08:28,000
通过那个高级语言特性的

226
00:08:28,000 --> 00:08:29,000
去把它重载下来

227
00:08:29,000 --> 00:08:30,000
然后呢

228
00:08:30,000 --> 00:08:31,000
假设这里面呢

229
00:08:31,000 --> 00:08:32,000
有个 ops_add

230
00:08:32,000 --> 00:08:34,000
就是实现的一个功能了

231
00:08:34,000 --> 00:08:35,000
然后第一步呢

232
00:08:35,000 --> 00:08:36,000
是这么做的

233
00:08:36,000 --> 00:08:37,000
那第二步呢

234
00:08:37,000 --> 00:08:39,000
就是实现微分了

235
00:08:39,000 --> 00:08:41,000
每一个微分都要去遍历

236
00:08:41,000 --> 00:08:43,000
我的那个 Tape

237
00:08:43,000 --> 00:08:46,000
就是把每一个刚才的计算都遍历一遍

238
00:08:46,000 --> 00:08:48,000
遍历完之后呢

239
00:08:48,000 --> 00:08:50,000
我就需要对我的这个 Tape 啊

240
00:08:50,000 --> 00:08:52,000
进行一个累积啊

241
00:08:52,000 --> 00:08:56,000
我下面这个计算是依赖于上面的

242
00:08:56,000 --> 00:08:57,000
所以我需要累积啊

243
00:08:57,000 --> 00:08:59,000
每一次计算都要累积的

244
00:08:59,000 --> 00:09:00,000
在反向的时候

245
00:09:00,000 --> 00:09:02,000
刚才上一集已经讲过了

246
00:09:02,000 --> 00:09:03,000
那再接下来呢

247
00:09:03,000 --> 00:09:06,000
就是讲基于源码转换的一种方式啊

248
00:09:06,000 --> 00:09:08,000
源码转换的方式呢

249
00:09:08,000 --> 00:09:09,000
没有图

250
00:09:09,000 --> 00:09:10,000
现在还没图

251
00:09:10,000 --> 00:09:11,000
等一下还会出现一个图

252
00:09:11,000 --> 00:09:12,000
就是需要去分析

253
00:09:12,000 --> 00:09:14,000
原语言程序的表达形式

254
00:09:14,000 --> 00:09:15,000
Parse 呢

255
00:09:15,000 --> 00:09:16,000
就是解析器

256
00:09:16,000 --> 00:09:18,000
都要原语言进行解析

257
00:09:18,000 --> 00:09:20,000
然后基于原语言呢

258
00:09:20,000 --> 00:09:22,000
进行一个操作的一个分解

259
00:09:22,000 --> 00:09:23,000
分解完之后呢

260
00:09:23,000 --> 00:09:25,000
还要搞清楚它的依赖关系

261
00:09:26,000 --> 00:09:27,000
搞清楚它的依赖关系呢

262
00:09:27,000 --> 00:09:29,000
才可以应用链式求导法则

263
00:09:29,000 --> 00:09:30,000
进行一个自动微分

264
00:09:30,000 --> 00:09:32,000
实现起来了

265
00:09:32,000 --> 00:09:33,000
问题呢

266
00:09:33,000 --> 00:09:34,000
就是代码非常难理解

267
00:09:34,000 --> 00:09:36,000
包括写代码的人也很痛苦

268
00:09:36,000 --> 00:09:39,000
要搞很多计算机底层编译的问题

269
00:09:39,000 --> 00:09:41,000
但是这个才是学计算机人的

270
00:09:41,000 --> 00:09:43,000
最重要的概念呢

271
00:09:43,000 --> 00:09:45,000
大三大四都学啥呀

272
00:09:45,000 --> 00:09:47,000
不就学这个编译原理吗

273
00:09:47,000 --> 00:09:49,000
那问题就是复杂度很高啊

274
00:09:49,000 --> 00:09:51,000
需要学那个编译器和解析器

275
00:09:51,000 --> 00:09:54,000
对于 ZOMI 来说也是个很头痛的事情

276
00:09:54,000 --> 00:09:57,000
因为大学的时候我就没有好好上这门课

277
00:10:00,000 --> 00:10:02,000
那它的优点就是没有额外的 Tape

278
00:10:02,000 --> 00:10:04,000
所以非常容易实现高阶的微分

279
00:10:04,000 --> 00:10:07,000
因为我已经得到了整个链式求导法则

280
00:10:07,000 --> 00:10:09,000
或者整个表达式的依赖关系

281
00:10:09,000 --> 00:10:10,000
然后呢

282
00:10:10,000 --> 00:10:13,000
微分结果也是以代码的形式去存在的

283
00:10:13,000 --> 00:10:16,000
非常方便分布式系统去计算啊

284
00:10:16,000 --> 00:10:18,000
可能大家对分布式系统感知还不太大

285
00:10:18,000 --> 00:10:21,000
后面讲到的并行操作的时候

286
00:10:21,000 --> 00:10:23,000
大模型你就会感受很深

287
00:10:24,000 --> 00:10:27,000
那这个呢就是 AST 的过程

288
00:10:27,000 --> 00:10:29,000
首先分开编译的过程

289
00:10:29,000 --> 00:10:30,000
还有 Runtime 的过程

290
00:10:30,000 --> 00:10:31,000
Runtime 就是执行

291
00:10:31,000 --> 00:10:32,000
这个就是编译

292
00:10:32,000 --> 00:10:34,000
需要拿到源码

293
00:10:34,000 --> 00:10:36,000
原生的一个 python 的代码

294
00:10:36,000 --> 00:10:38,000
对它进行 Parser 解析

295
00:10:38,000 --> 00:10:40,000
然后变成图的 IR

296
00:10:40,000 --> 00:10:41,000
然后去进行宏展开

297
00:10:41,000 --> 00:10:42,000
然后宏推理

298
00:10:42,000 --> 00:10:43,000
宏编译

299
00:10:43,000 --> 00:10:46,000
然后类型推导那个 IR 优化

300
00:10:46,000 --> 00:10:48,000
最后再编译成机器语言

301
00:10:49,000 --> 00:10:52,000
机器能够识别的一些代码

302
00:10:52,000 --> 00:10:54,000
就丢过来去执行的过程当中

303
00:10:54,000 --> 00:10:56,000
通过执行的时候呢

304
00:10:56,000 --> 00:10:58,000
对代码进行执行

305
00:10:58,000 --> 00:11:00,000
然后对代码的进行展开

306
00:11:00,000 --> 00:11:02,000
最后调用函数

307
00:11:03,000 --> 00:11:04,000
那过程非常复杂

308
00:11:04,000 --> 00:11:05,000
非常冗余是吧

309
00:11:05,000 --> 00:11:06,000
实现很难

310
00:11:06,000 --> 00:11:07,000
但是呢

311
00:11:07,000 --> 00:11:09,000
可能可以对做高阶微分很方便

312
00:11:09,000 --> 00:11:12,000
对做一些分布式很方便

313
00:11:13,000 --> 00:11:15,000
总结一下自动微分的实现方式有三种

314
00:11:15,000 --> 00:11:16,000
第一种呢

315
00:11:16,000 --> 00:11:18,000
就是表达式或图

316
00:11:18,000 --> 00:11:19,000
第二种呢

317
00:11:19,000 --> 00:11:20,000
就是操作符号重载

318
00:11:20,000 --> 00:11:22,000
这三种不就源码转换吗

319
00:11:22,000 --> 00:11:23,000
刚才讲完

320
00:11:23,000 --> 00:11:25,000
那基于表达式的实现呢

321
00:11:25,000 --> 00:11:27,000
主要依赖于基础

322
00:11:27,000 --> 00:11:28,000
基于操作附存载呢

323
00:11:28,000 --> 00:11:30,000
主要依赖于语言的多态性

324
00:11:30,000 --> 00:11:32,000
基于源码转换呢

325
00:11:32,000 --> 00:11:35,000
主要依赖于整个编译的流程了

326
00:11:35,000 --> 00:11:38,000
这个就是很复杂很漫长了