04.srt

1
00:00:00,000 --> 00:00:04,500
字幕生成: BLACK 字幕校对: 杨绎

2
00:00:05,300 --> 00:00:07,480
Hello,大家好,我是 ZOMI

3
00:00:07,480 --> 00:00:09,520
最近好冷啊,冷得直哆嗦

4
00:00:09,520 --> 00:00:13,200
那今天呢,还是回到 AI 编译器系列里面的

5
00:00:13,200 --> 00:00:15,280
拍 Torch 里面最重要的一个特性

6
00:00:15,280 --> 00:00:16,920
Torch Dynamo

7
00:00:16,920 --> 00:00:20,000
那今天是真正的来到 Torch Dynamo 里面

8
00:00:20,000 --> 00:00:23,440
去讲讲 Torch Dynamo 的一个具体的原理

9
00:00:23,440 --> 00:00:26,360
那在讲原理之前呢,还是有点内容的

10
00:00:26,360 --> 00:00:27,880
就是不要那么着急

11
00:00:28,360 --> 00:00:33,080
先来回顾一下拍 Torch 对于图模式的一个尝试

12
00:00:33,080 --> 00:00:34,400
那总结一下

13
00:00:34,400 --> 00:00:35,600
其实图模式呢

14
00:00:35,600 --> 00:00:40,120
最主要的还是希望能够把这些 Python 写的一些代码呢

15
00:00:40,120 --> 00:00:41,760
变成一个图的结构

16
00:00:41,760 --> 00:00:43,200
变成一个计算图

17
00:00:43,200 --> 00:00:43,720
最后呢

18
00:00:43,720 --> 00:00:46,960
这个计算图作为编译器的一个输入

19
00:00:46,960 --> 00:00:48,480
对它进行一个编译优化

20
00:00:48,480 --> 00:00:51,880
就自行在具体的生成的硬件上面

21
00:00:51,880 --> 00:00:53,200
使它跑得更快

22
00:00:54,200 --> 00:00:55,880
那现在呢

23
00:00:55,880 --> 00:00:58,760
来看看各个特性之间的一个差别

24
00:00:58,760 --> 00:01:00,320
首先是 Torch Script

25
00:01:00,320 --> 00:01:01,080
Torch Script 呢

26
00:01:01,080 --> 00:01:04,800
它主要的用途其实是用在模型的部署方面

27
00:01:04,800 --> 00:01:05,520
拍 Torch 呢

28
00:01:05,520 --> 00:01:07,720
它是一个灵活结构的框架

29
00:01:07,720 --> 00:01:08,200
所以呢

30
00:01:08,200 --> 00:01:11,800
到时候可以用 Torch Dynamo 去拼接 Torch Script

31
00:01:11,800 --> 00:01:14,120
就用 Torch Dynamo 作为前端

32
00:01:14,120 --> 00:01:17,280
用 Torch Script 作为后端进行去执行的

33
00:01:17,280 --> 00:01:18,120
这也是可以的

34
00:01:18,120 --> 00:01:18,920
所以说呢

35
00:01:18,920 --> 00:01:22,640
它的使用场景主要是聚焦于推理和部署的

36
00:01:22,640 --> 00:01:23,720
它的技术方案呢

37
00:01:23,720 --> 00:01:24,560
可以用 AST

38
00:01:24,560 --> 00:01:26,840
也可以用 Trace 这两种方式

39
00:01:26,840 --> 00:01:28,080
像 Torch FX 呢

40
00:01:28,080 --> 00:01:29,000
它最主要呢

41
00:01:29,000 --> 00:01:31,920
是使用 Python to Python 的一种翻译

42
00:01:31,920 --> 00:01:33,720
既然是 Python to Python 的翻译

43
00:01:33,720 --> 00:01:36,400
它就没有了一个后向的反向传播的

44
00:01:36,400 --> 00:01:38,440
或者叫做反向图

45
00:01:38,440 --> 00:01:39,160
那这里面呢

46
00:01:39,160 --> 00:01:40,560
主要是用在量化

47
00:01:40,560 --> 00:01:42,640
或者做一些大双子的替换里面

48
00:01:42,640 --> 00:01:45,440
适用的场景就是简单的去修改一下

49
00:01:45,440 --> 00:01:47,600
正向的一个 forward 图

50
00:01:47,600 --> 00:01:48,840
就是正向图

51
00:01:48,840 --> 00:01:49,440
里面呢

52
00:01:49,440 --> 00:01:50,520
最主要的方式呢

53
00:01:50,520 --> 00:01:51,960
是基于 Basing Trace 的

54
00:01:51,960 --> 00:01:53,760
就是基于跟踪的方式

55
00:01:53,760 --> 00:01:54,640
那第三种呢

56
00:01:54,640 --> 00:01:56,400
就是 Lazy Tensor

57
00:01:56,400 --> 00:01:58,240
Lazy Tensor 的主要作用呢

58
00:01:58,240 --> 00:02:00,480
是使用 PyTorch 的前端

59
00:02:00,480 --> 00:02:02,280
然后后端或者硬件呢

60
00:02:02,280 --> 00:02:05,120
使用谷歌的 NPU 进行加速

61
00:02:05,120 --> 00:02:06,480
Lazy Tensor 的方式呢

62
00:02:06,480 --> 00:02:08,680
会比 Torch FX 做得更好一点

63
00:02:08,680 --> 00:02:11,240
但是它们之间有本质的区别的

64
00:02:11,240 --> 00:02:13,520
就是主要它是一个隐式的构图

65
00:02:13,520 --> 00:02:16,040
加上一些简单的编译优化

66
00:02:16,040 --> 00:02:17,080
那这个编译优化呢

67
00:02:17,080 --> 00:02:18,680
主要是靠 XLA

68
00:02:18,680 --> 00:02:21,840
所以它说是针对一个 NPU 的子图的编译

69
00:02:21,960 --> 00:02:24,080
记住有一个子图的概念

70
00:02:24,080 --> 00:02:25,520
它不是整图的编译

71
00:02:25,520 --> 00:02:27,000
这是很大的区别的

72
00:02:27,000 --> 00:02:27,920
而这种方式呢

73
00:02:27,920 --> 00:02:29,840
同样是基于 Basing Trace 的

74
00:02:30,600 --> 00:02:33,000
今天主角就是 Torch Dynamo

75
00:02:33,000 --> 00:02:33,600
Dynamo 呢

76
00:02:33,600 --> 00:02:36,560
是 PyTorch 2.0 最新的一个特性

77
00:02:36,560 --> 00:02:38,120
最重要的就是这句话

78
00:02:38,120 --> 00:02:40,280
修改的时机是在 Cpython 的

79
00:02:40,280 --> 00:02:41,880
Bytecode 执行之前

80
00:02:41,880 --> 00:02:44,600
也就是在 Python 生成字节码之前

81
00:02:44,600 --> 00:02:46,440
把这个字节码给改掉了

82
00:02:46,440 --> 00:02:48,760
然后适用的场景非常多

83
00:02:48,760 --> 00:02:50,320
就可以做一些动静统一

84
00:02:50,320 --> 00:02:52,720
还有一些编译优化的工作

85
00:02:52,720 --> 00:02:53,720
那技术方式呢

86
00:02:53,720 --> 00:02:55,720
可能跟 Basing Trace 和 AST

87
00:02:55,720 --> 00:02:57,120
源码转换不太一样

88
00:02:57,120 --> 00:02:59,680
它更多的是基于 Python 的解析器

89
00:02:59,680 --> 00:03:00,600
去进行修改的

90
00:03:00,600 --> 00:03:01,760
如果非得去归类

91
00:03:01,760 --> 00:03:03,560
我觉得更多的是属于 AST

92
00:03:03,560 --> 00:03:04,720
就是源码转换

93
00:03:04,720 --> 00:03:06,120
因为 Python 的 Bytecode 呢

94
00:03:06,120 --> 00:03:08,400
也是对 Python 的代码进行源码转换

95
00:03:08,400 --> 00:03:10,200
成为 Bytecode 字节码

96
00:03:10,200 --> 00:03:12,200
关于 PyTorch 在图模式的尝试

97
00:03:12,200 --> 00:03:13,800
了解完这四种之后呢

98
00:03:13,800 --> 00:03:15,480
去看看 Torch Dynamo 的

99
00:03:15,480 --> 00:03:17,120
一个最重要的原理

100
00:03:18,120 --> 00:03:20,680
下面呢有部分的图呢

101
00:03:20,680 --> 00:03:22,640
我还是在至于 PyTorch Conference

102
00:03:22,640 --> 00:03:23,560
里面的一些

103
00:03:23,560 --> 00:03:24,400
那现在来看看

104
00:03:24,400 --> 00:03:26,200
为什么 Torch Dynamo 呢

105
00:03:26,200 --> 00:03:27,880
听起来这么的牛逼

106
00:03:27,880 --> 00:03:28,680
那这个呢

107
00:03:28,680 --> 00:03:30,720
这么牛逼我就不再解释了

108
00:03:30,720 --> 00:03:33,800
来看看它现在的一个具体的情况

109
00:03:33,920 --> 00:03:35,400
可以看到他们说

110
00:03:35,400 --> 00:03:36,880
现在 GitHub 里面的 model

111
00:03:36,880 --> 00:03:38,960
有 7K 加就 7000 多的 GitHub 的

112
00:03:38,960 --> 00:03:40,480
模型呢是已经支持的

113
00:03:40,480 --> 00:03:42,440
而且那个推理的后端呢

114
00:03:42,440 --> 00:03:43,200
有 20 多个

115
00:03:43,200 --> 00:03:45,000
那这个后端为什么会有 20 多个

116
00:03:45,000 --> 00:03:46,920
其实我是不明白的

117
00:03:47,440 --> 00:03:48,240
不过也没关系

118
00:03:48,240 --> 00:03:50,200
它支持的后端确实很多

119
00:03:50,440 --> 00:03:51,560
训练的后端呢

120
00:03:51,560 --> 00:03:52,560
支持一个以上

121
00:03:52,680 --> 00:03:54,880
那其中一个我也不知道是哪一个

122
00:03:54,880 --> 00:03:56,560
反正我觉得最重要的

123
00:03:56,560 --> 00:03:58,960
就是它确实能行

124
00:03:59,960 --> 00:04:00,960
那现在呢看看

125
00:04:00,960 --> 00:04:02,920
PyTorch 的一个整体的编译模式

126
00:04:02,920 --> 00:04:04,560
那它不仅是 Dynamo

127
00:04:04,560 --> 00:04:05,400
它是整体的

128
00:04:05,400 --> 00:04:07,920
包括整个编译的后端

129
00:04:07,920 --> 00:04:10,280
现在呢从左往右开始看

130
00:04:10,280 --> 00:04:11,400
首先第一个就是使用

131
00:04:11,400 --> 00:04:13,480
PyTorch Python 的 API

132
00:04:13,480 --> 00:04:15,520
然后去做一个写代码嘛

133
00:04:15,520 --> 00:04:16,560
写完代码之后呢

134
00:04:16,560 --> 00:04:18,040
在图获取的情况下呢

135
00:04:18,040 --> 00:04:19,240
使用 Torch Dynamo

136
00:04:19,240 --> 00:04:20,360
但是 Torch Dynamo 呢

137
00:04:20,360 --> 00:04:21,960
主要是改前向的图

138
00:04:21,960 --> 00:04:24,920
因为写代码只是写前向的图

139
00:04:24,920 --> 00:04:27,000
反向的图其实是没有写的

140
00:04:27,000 --> 00:04:27,920
Torch Dynamo 呢

141
00:04:27,920 --> 00:04:29,200
是改 Python 的

142
00:04:29,200 --> 00:04:31,360
Bitcore 自然不涉及到

143
00:04:31,360 --> 00:04:33,200
反向传播和反向图

144
00:04:33,200 --> 00:04:33,920
那在这时候呢

145
00:04:33,920 --> 00:04:35,760
就引入了另外一个新的特性

146
00:04:35,760 --> 00:04:37,520
AoT Ahead of Time

147
00:04:37,520 --> 00:04:38,640
AutoGrade

148
00:04:38,640 --> 00:04:39,520
引入这个之后呢

149
00:04:39,520 --> 00:04:41,360
得到全图的信息

150
00:04:41,360 --> 00:04:42,840
引入了 AoT AutoGrade 呢

151
00:04:42,840 --> 00:04:44,760
就得到了一个全图的信息

152
00:04:44,760 --> 00:04:46,240
包括正向的反向的

153
00:04:46,240 --> 00:04:46,760
接着呢

154
00:04:46,760 --> 00:04:48,240
把它变成一个 Atom 的

155
00:04:48,240 --> 00:04:49,720
ARCore Permit AR

156
00:04:49,720 --> 00:04:51,240
这个 AR 最有意思的就是

157
00:04:51,240 --> 00:04:53,520
它是慢慢的往底层靠了

158
00:04:53,520 --> 00:04:54,160
最后呢

159
00:04:54,160 --> 00:04:56,720
就是给图编译器去执行的

160
00:04:56,720 --> 00:04:57,640
那图编译器呢

161
00:04:57,640 --> 00:04:59,560
后端可以有非常多啊

162
00:04:59,560 --> 00:05:00,440
有 MVFuser

163
00:05:00,440 --> 00:05:01,080
有 TVM

164
00:05:01,080 --> 00:05:01,920
有 XLA

165
00:05:01,920 --> 00:05:02,800
还有 TensorRT

166
00:05:02,800 --> 00:05:05,040
有非常多的图编译的后端

167
00:05:05,040 --> 00:05:06,600
那真正的让这些后端

168
00:05:06,600 --> 00:05:08,320
跟硬件打交道

169
00:05:08,320 --> 00:05:09,600
那这个图呢

170
00:05:09,600 --> 00:05:11,840
就是 PyTorch 编译的一个最重要的流程

171
00:05:11,840 --> 00:05:13,880
可以看到 Torch Dynamo 呢

172
00:05:13,880 --> 00:05:16,160
只是最前端的前端

173
00:05:16,160 --> 00:05:17,760
把动态的正向图

174
00:05:17,760 --> 00:05:19,840
变成一个静态的正向图

175
00:05:19,840 --> 00:05:21,320
而且非常强调的是

176
00:05:21,320 --> 00:05:22,880
正向图的这个概念

177
00:05:22,880 --> 00:05:23,440
现在呢

178
00:05:23,440 --> 00:05:26,800
来看看 PyTorch Dynamo 的一个编译模式

179
00:05:26,800 --> 00:05:27,360
刚才呢

180
00:05:27,360 --> 00:05:28,680
也不仅是 Dynamo 啊

181
00:05:28,680 --> 00:05:30,520
还是包括端到端的

182
00:05:30,520 --> 00:05:31,920
那在前端呢

183
00:05:31,920 --> 00:05:35,040
主要是通过 Dynamo 去做一个解析

184
00:05:35,040 --> 00:05:37,120
得到正向的图之后呢

185
00:05:37,120 --> 00:05:39,240
再做一个 AOT 的 AutoGrade

186
00:05:39,240 --> 00:05:40,920
然后得到一个全图

187
00:05:40,920 --> 00:05:41,720
全图之后呢

188
00:05:41,720 --> 00:05:43,640
我往下发变成一个 Atom

189
00:05:43,640 --> 00:05:44,800
或 Permitive 的 AR

190
00:05:44,800 --> 00:05:47,280
就把算子的 AR 引进来

191
00:05:47,280 --> 00:05:48,120
那引进来之后呢

192
00:05:48,120 --> 00:05:50,280
可以使用那个 Triton 的后端

193
00:05:50,280 --> 00:05:52,040
使用 OpenMP 的后端

194
00:05:52,040 --> 00:05:54,520
或者其他后端去生成代码

195
00:05:54,520 --> 00:05:56,280
这个呢就是 CoreGene 的阶段

196
00:05:56,280 --> 00:05:58,240
这里面呢也是 Low-Level 的 IR

197
00:05:58,240 --> 00:05:59,160
而往上走呢

198
00:05:59,160 --> 00:06:00,320
因为有图的信息

199
00:06:00,320 --> 00:06:02,160
所以叫做 High-Level 的 AR

200
00:06:02,160 --> 00:06:04,080
那这个图的优化和图的信息呢

201
00:06:04,080 --> 00:06:06,560
将会在后面详细的展开

202
00:06:06,560 --> 00:06:07,480
这个也不用急

203
00:06:10,320 --> 00:06:13,760
接下来去看看一个画外题

204
00:06:13,800 --> 00:06:16,560
就是 Python 的执行机制

205
00:06:16,560 --> 00:06:18,200
哎呀真的好冷啊

206
00:06:18,200 --> 00:06:21,480
现在来看看 Python 的执行机制

207
00:06:21,480 --> 00:06:22,240
那 Python 呢

208
00:06:22,240 --> 00:06:26,120
它是一个混合了解析和编译的一个

209
00:06:26,120 --> 00:06:28,320
这读解析还是解释啊

210
00:06:28,320 --> 00:06:29,480
反正我分不清啊

211
00:06:29,480 --> 00:06:31,320
大家看着差不多就行了

212
00:06:31,320 --> 00:06:32,840
就给 Python 的解释器呢

213
00:06:32,840 --> 00:06:35,360
然后去做一个解析编译的过程

214
00:06:35,360 --> 00:06:35,960
然后首先呢

215
00:06:35,960 --> 00:06:38,040
会对它这些代码进行一个编译

216
00:06:38,040 --> 00:06:39,640
要把它转换成为一个 Bytecode

217
00:06:39,640 --> 00:06:40,720
就是自解码

218
00:06:40,720 --> 00:06:41,760
自解码之后呢

219
00:06:41,760 --> 00:06:42,920
给 Python

220
00:06:42,920 --> 00:06:44,760
不过值得注意的一点就是

221
00:06:44,760 --> 00:06:46,200
cPython 这个虚拟机呢

222
00:06:46,200 --> 00:06:47,080
实际上内部呢

223
00:06:47,080 --> 00:06:48,200
是有一个 while 循环的

224
00:06:48,200 --> 00:06:50,120
不断的去匹配这些自解码

225
00:06:50,120 --> 00:06:51,440
并对自解码的指令呢

226
00:06:51,440 --> 00:06:53,320
分成非常多条 c 的函数

227
00:06:53,320 --> 00:06:54,400
然后去执行的

228
00:06:54,400 --> 00:06:57,240
所以它是一个 while 或者 loop 的一个过程

229
00:06:57,240 --> 00:07:00,040
这也不是一下子就把所有代码都执行完毕了

230
00:07:00,720 --> 00:07:02,320
那了解到这个过程呢

231
00:07:02,320 --> 00:07:04,880
再看看 Python 执行机制里面的

232
00:07:04,880 --> 00:07:06,080
两个重要的概念

233
00:07:06,080 --> 00:07:06,840
第一个是呢

234
00:07:06,840 --> 00:07:08,040
PyThinkObject

235
00:07:08,040 --> 00:07:10,320
另外一个是 PyCoreObject

236
00:07:10,360 --> 00:07:12,880
那可以知道其实每个 PyThinkObject 呢

237
00:07:12,880 --> 00:07:15,240
都对应一个 PyCoreObject

238
00:07:15,240 --> 00:07:17,680
而每一个 PyThinkObject 对应呢

239
00:07:17,680 --> 00:07:20,000
是 Python 源码当中的一段 Core

240
00:07:20,000 --> 00:07:22,560
就是跟代码是相对应的

241
00:07:22,560 --> 00:07:24,760
这是 Python 实现的内部一个机制

242
00:07:24,760 --> 00:07:25,720
而这里面呢

243
00:07:25,720 --> 00:07:26,880
可能概念稍微多一点

244
00:07:26,880 --> 00:07:28,960
大家简单的去理解一下就好了

245
00:07:28,960 --> 00:07:30,080
因为后面呢

246
00:07:30,080 --> 00:07:31,880
会讲到的

247
00:07:31,880 --> 00:07:32,840
Python 的 CoreObject 呢

248
00:07:32,840 --> 00:07:34,880
其实是包含程序的静态信息

249
00:07:34,880 --> 00:07:36,360
但是动态的信息怎么办呢

250
00:07:36,360 --> 00:07:37,400
程序的自行环境

251
00:07:37,400 --> 00:07:38,800
到底由谁来执行呢

252
00:07:38,800 --> 00:07:39,600
而这个时候呢

253
00:07:39,600 --> 00:07:40,800
就由 Python 的虚拟机

254
00:07:40,800 --> 00:07:42,680
另外一个层面的对象

255
00:07:42,680 --> 00:07:44,640
使用的是 PyThinkObject

256
00:07:44,640 --> 00:07:46,320
那在 Python 实际执行的时候呢

257
00:07:46,320 --> 00:07:48,040
就会产生非常多的

258
00:07:48,040 --> 00:07:49,880
PyThinkObject 这些对象

259
00:07:49,880 --> 00:07:50,640
这些对象呢

260
00:07:50,640 --> 00:07:51,520
就会连接起来

261
00:07:51,520 --> 00:07:53,840
跟虚拟机去执行

262
00:07:53,840 --> 00:07:55,400
ZOMI 老师你好啊

263
00:07:55,400 --> 00:07:56,240
我想问一下

264
00:07:56,240 --> 00:07:57,280
PyThinkObject 呢

265
00:07:57,280 --> 00:07:59,800
除了执行执行环境之外

266
00:07:59,800 --> 00:08:01,800
它还有哪些信息呢

267
00:08:02,800 --> 00:08:04,120
这位蜡笔小新同学

268
00:08:04,120 --> 00:08:05,920
你问的这个问题非常好啊

269
00:08:05,920 --> 00:08:07,080
就是第五点了

270
00:08:07,120 --> 00:08:10,240
除了对应的程序的静态信息呢

271
00:08:10,240 --> 00:08:13,000
是给 PyCodeObject 之外

272
00:08:13,000 --> 00:08:14,720
PyThinkObject 呢

273
00:08:14,720 --> 00:08:17,000
主要可能会有一些内存的空间

274
00:08:17,280 --> 00:08:18,800
计算器的地址指针呢

275
00:08:18,800 --> 00:08:20,000
还有运行站的顺序

276
00:08:20,000 --> 00:08:21,240
然后执行环境等

277
00:08:21,240 --> 00:08:23,000
非常多其他额外的信息

278
00:08:23,000 --> 00:08:25,880
都会包含在 PyThinkObject 里面

279
00:08:25,880 --> 00:08:27,720
所以简单的理解为

280
00:08:27,720 --> 00:08:29,360
它是跟程序一对应

281
00:08:29,360 --> 00:08:30,120
这个内容呢

282
00:08:30,120 --> 00:08:32,040
还是有点偏小的

283
00:08:32,040 --> 00:08:33,800
那现在呢看看

284
00:08:33,800 --> 00:08:34,800
现在呢来看看

285
00:08:34,800 --> 00:08:35,920
具体原始的时候

286
00:08:35,960 --> 00:08:37,480
Python 是怎么去执行的

287
00:08:37,480 --> 00:08:39,280
首先拿到一个附的函数

288
00:08:39,280 --> 00:08:40,640
就这个 demo 函数

289
00:08:40,640 --> 00:08:42,600
然后呢执行的 PyThinkObject 呢

290
00:08:42,600 --> 00:08:43,280
PyThinkObject 呢

291
00:08:43,280 --> 00:08:45,760
其实是跟 PyCodeObject 一一对应的

292
00:08:45,760 --> 00:08:47,040
那实际上程序执行

293
00:08:47,040 --> 00:08:48,400
是对应到真正的 code

294
00:08:48,400 --> 00:08:48,960
而这里面呢

295
00:08:48,960 --> 00:08:50,000
会有非常多的额外

296
00:08:50,000 --> 00:08:51,200
刚才讲到的信息

297
00:08:51,200 --> 00:08:52,480
最后由核心

298
00:08:52,480 --> 00:08:55,640
PyEvaluateThinkDefault 去执行的

299
00:08:55,640 --> 00:08:56,360
而这个呢

300
00:08:56,360 --> 00:08:58,800
是解析器程序的核心函数

301
00:08:58,800 --> 00:08:59,680
就是这一个了

302
00:08:59,680 --> 00:09:02,680
就它底层的执行的一些对象

303
00:09:02,680 --> 00:09:04,000
那看看

304
00:09:04,040 --> 00:09:04,680
Dynamo 呢

305
00:09:04,680 --> 00:09:06,200
它是怎么实现的

306
00:09:06,200 --> 00:09:08,480
简单的总结一句就是

307
00:09:08,480 --> 00:09:10,200
在 Dynamo 里面呢

308
00:09:10,200 --> 00:09:11,520
主要是使用

309
00:09:11,520 --> 00:09:12,520
Python 解析器的时候

310
00:09:12,520 --> 00:09:13,840
自动的把 bytecode

311
00:09:13,840 --> 00:09:14,760
就是字节码

312
00:09:14,760 --> 00:09:15,400
进行捕捉

313
00:09:15,400 --> 00:09:18,120
转换成为 TouchFX 的 IR

314
00:09:18,120 --> 00:09:20,200
之所以之前花了那么多精力

315
00:09:20,200 --> 00:09:22,040
去讲那些 TouchFX 啊

316
00:09:22,040 --> 00:09:23,320
TouchScript 啊

317
00:09:23,320 --> 00:09:24,320
这些是有用的

318
00:09:24,320 --> 00:09:25,680
后面去把它转成

319
00:09:25,680 --> 00:09:26,960
TouchFX 的 IR

320
00:09:26,960 --> 00:09:28,360
而 TouchFX 的 IR 呢

321
00:09:28,360 --> 00:09:29,680
是因为 TouchScript 的 IR

322
00:09:29,680 --> 00:09:30,800
确实太复杂了

323
00:09:30,800 --> 00:09:32,920
所以它在 Python 层建立了一个 IR

324
00:09:32,920 --> 00:09:34,640
既然是 Python 层建的 IR

325
00:09:34,640 --> 00:09:36,160
所以就非常方便

326
00:09:36,160 --> 00:09:37,400
在 Python 层面

327
00:09:37,400 --> 00:09:39,560
做一个解析的功能

328
00:09:39,560 --> 00:09:42,080
那现在看看右边的这个图

329
00:09:42,080 --> 00:09:43,040
附的这个呢

330
00:09:43,040 --> 00:09:44,880
就是一个 Dynamo 函数

331
00:09:44,880 --> 00:09:45,720
然后在

332
00:09:45,720 --> 00:09:47,120
PythonFimObject 的时候

333
00:09:47,120 --> 00:09:48,960
它自动的会对应到

334
00:09:48,960 --> 00:09:50,120
Python 的 CoreObject

335
00:09:50,120 --> 00:09:51,480
拿到的 CoreObject 之后呢

336
00:09:51,480 --> 00:09:53,280
会对它进行一个

337
00:09:53,280 --> 00:09:54,880
动态的解析和分析

338
00:09:54,880 --> 00:09:55,560
然后呢

339
00:09:55,560 --> 00:09:56,560
分两条分支

340
00:09:56,560 --> 00:09:57,720
第一条分支呢

341
00:09:57,720 --> 00:09:59,640
就是把它转换成为一个

342
00:09:59,640 --> 00:10:00,640
Python 的 CoreObject

343
00:10:00,640 --> 00:10:01,360
那这个时候呢

344
00:10:01,360 --> 00:10:03,240
是没有 Python 相关的内容的

345
00:10:03,240 --> 00:10:03,720
第二个呢

346
00:10:03,720 --> 00:10:06,680
就是把它转换成为一个 FS 的图

347
00:10:06,680 --> 00:10:07,800
那这个 FS 的图呢

348
00:10:07,800 --> 00:10:09,320
在上一节里面去讲过一下

349
00:10:09,320 --> 00:10:10,680
我会稍微简单一下

350
00:10:10,680 --> 00:10:11,200
接着呢

351
00:10:11,200 --> 00:10:12,720
使用一个用户定义的

352
00:10:12,720 --> 00:10:14,600
一个简单的小的编译器

353
00:10:14,600 --> 00:10:16,880
然后编译成想要的一些函数

354
00:10:16,880 --> 00:10:17,640
那这个时候呢

355
00:10:17,640 --> 00:10:18,960
如果走另外一条分支呢

356
00:10:18,960 --> 00:10:21,080
只是简单的一个调用就行了

357
00:10:21,080 --> 00:10:21,560
最后呢

358
00:10:21,560 --> 00:10:22,640
打了一个 Patch

359
00:10:22,640 --> 00:10:24,720
在 PyFimObject 里面

360
00:10:24,720 --> 00:10:27,000
真正执行还是回到

361
00:10:27,000 --> 00:10:29,000
PyEvaluateFimDevolute

362
00:10:29,000 --> 00:10:30,920
这个核心函数里面可以看到

363
00:10:30,920 --> 00:10:32,280
这个时候可以看到

364
00:10:32,280 --> 00:10:34,040
在 Python 的解析器的时候呢

365
00:10:34,040 --> 00:10:36,320
把动态的 ByCore 进行捕捉

366
00:10:36,320 --> 00:10:37,920
然后最重要的就是这一步

367
00:10:37,920 --> 00:10:40,720
把它转换成为 ToshFX 的 IR

368
00:10:40,720 --> 00:10:42,080
通过这种方式呢

369
00:10:42,080 --> 00:10:44,400
去把前端 PyTosh 的动态图

370
00:10:44,400 --> 00:10:48,200
转换成为 ToshFXIR 这个静态图

371
00:10:48,200 --> 00:10:48,680
接着呢

372
00:10:48,680 --> 00:10:50,480
还是回到上一节的内容

373
00:10:50,480 --> 00:10:51,560
其实已经讲了

374
00:10:51,560 --> 00:10:54,040
ToshFX 的一个 IR 的一个概念

375
00:10:54,040 --> 00:10:56,280
这些就是 ToshFX 的 IR 的一些

376
00:10:56,280 --> 00:10:57,600
主要的 op code

377
00:10:57,600 --> 00:10:58,320
那这里面呢

378
00:10:58,320 --> 00:11:00,040
这些就是刚才讲到的

379
00:11:00,040 --> 00:11:01,600
六个重要的 OB code 里面的

380
00:11:01,600 --> 00:11:02,400
具体的含义

381
00:11:02,400 --> 00:11:03,920
就不一一带过了

382
00:11:03,920 --> 00:11:05,600
现在来具体的看看

383
00:11:05,600 --> 00:11:07,880
Tosh Dynamo 是怎么实现的

384
00:11:08,880 --> 00:11:09,440
首先呢

385
00:11:09,440 --> 00:11:10,600
已经写好了一段

386
00:11:10,600 --> 00:11:11,360
那这里面呢

387
00:11:11,360 --> 00:11:13,960
首先关注于一个图

388
00:11:13,960 --> 00:11:16,200
或者需要解析的一个函数

389
00:11:16,200 --> 00:11:16,920
输入呢

390
00:11:16,920 --> 00:11:17,440
有两个

391
00:11:17,440 --> 00:11:18,400
一个是 A 和 B

392
00:11:18,400 --> 00:11:19,000
那 A 呢

393
00:11:19,000 --> 00:11:21,520
先进行一个简单的操作

394
00:11:21,520 --> 00:11:22,640
然后得到 X

395
00:11:22,640 --> 00:11:23,240
接着呢

396
00:11:23,240 --> 00:11:24,760
加一个循环

397
00:11:24,760 --> 00:11:25,640
就控制流

398
00:11:25,640 --> 00:11:27,840
因为控制流对来说是很重要的

399
00:11:27,840 --> 00:11:28,720
很多静态图

400
00:11:28,720 --> 00:11:31,160
其实是没有办法表示控制流的

401
00:11:31,160 --> 00:11:32,000
特别是 FX

402
00:11:32,000 --> 00:11:33,640
它其实表示不了控制流的

403
00:11:33,640 --> 00:11:35,800
if B.sum 小于 0 的话

404
00:11:35,800 --> 00:11:36,320
B 呢

405
00:11:36,320 --> 00:11:37,520
就执行另外一个操作

406
00:11:37,520 --> 00:11:37,880
最后呢

407
00:11:37,880 --> 00:11:39,280
返回 S 乘以 B

408
00:11:39,280 --> 00:11:40,440
这个函数呢

409
00:11:40,440 --> 00:11:41,200
很简单

410
00:11:41,200 --> 00:11:42,600
就是把图打印出来嘛

411
00:11:42,600 --> 00:11:43,240
然后接着呢

412
00:11:43,240 --> 00:11:44,680
去调用这个

413
00:11:44,680 --> 00:11:47,040
然后去真正的把 foo 打印出来

414
00:11:47,040 --> 00:11:47,960
那打印出来之前呢

415
00:11:47,960 --> 00:11:49,280
加了一个装饰器

416
00:11:49,280 --> 00:11:52,120
就是 Dynamo 里面的 optimize My Compiler

417
00:11:52,120 --> 00:11:54,920
然后尝试的去执行这一段函数

418
00:11:57,520 --> 00:11:58,120
经过这个呢

419
00:11:58,120 --> 00:11:59,640
就把那个话打印出来了

420
00:11:59,640 --> 00:12:01,440
My Compiler Core with FX

421
00:12:01,440 --> 00:12:03,240
My Compiler called with FX

422
00:12:03,240 --> 00:12:03,400
哎

423
00:12:03,400 --> 00:12:05,000
为啥有两个呢

424
00:12:05,000 --> 00:12:06,960
这是一个 FX 的图

425
00:12:06,960 --> 00:12:08,440
这其实也是一个 FX 的图

426
00:12:08,440 --> 00:12:10,440
可以看到 OP core 有 placeholder

427
00:12:10,440 --> 00:12:11,080
有 core function

428
00:12:11,080 --> 00:12:11,840
core method

429
00:12:11,840 --> 00:12:12,520
还有个 output

430
00:12:12,520 --> 00:12:14,440
这边也有一个 OP core 的

431
00:12:14,440 --> 00:12:16,040
为什么会有两个图呢

432
00:12:16,040 --> 00:12:17,520
在计算图介绍的时候

433
00:12:17,520 --> 00:12:18,240
已经明确说了

434
00:12:18,240 --> 00:12:20,800
其实有一些 AI 框架或者有一些办法呢

435
00:12:20,800 --> 00:12:24,480
就是把 if 这些把它展开成为子图

436
00:12:24,480 --> 00:12:25,120
那这里面呢

437
00:12:25,120 --> 00:12:26,000
有两个子图

438
00:12:26,000 --> 00:12:26,800
第一个子图呢

439
00:12:26,800 --> 00:12:28,760
就是 b.sum 小于 0 的时候呢

440
00:12:28,760 --> 00:12:30,600
就是执行这一句话

441
00:12:30,600 --> 00:12:31,760
要在执行这一句话

442
00:12:31,760 --> 00:12:32,800
然后再返回

443
00:12:32,800 --> 00:12:33,560
第 2 个子图呢

444
00:12:33,560 --> 00:12:34,480
就执行 x

445
00:12:34,480 --> 00:12:36,880
然后直接返回 x×b

446
00:12:36,880 --> 00:12:39,400
主要是执行两个子图

447
00:12:39,400 --> 00:12:42,680
在 Dynamo 转换成为 FX 的图之前呢

448
00:12:42,680 --> 00:12:46,120
它其实是它利用了 PyTorch 的一个 bytecore

449
00:12:46,120 --> 00:12:47,240
就是字节码

450
00:12:47,240 --> 00:12:48,600
刚才的代码负呢

451
00:12:48,600 --> 00:12:51,080
实际上对应的 Python 的 bytecore 呢

452
00:12:51,080 --> 00:12:52,400
是下面这几条

453
00:12:52,400 --> 00:12:53,080
但是呢

454
00:12:53,080 --> 00:12:54,840
看一看下面这个图呢

455
00:12:54,840 --> 00:12:56,560
这个图就是 Torch Dynamo

456
00:12:56,560 --> 00:12:59,400
动态的去把 bytecore 改写之后啊

457
00:12:59,400 --> 00:13:00,960
对 0 可能会改写

458
00:13:00,960 --> 00:13:03,680
然后把帮刚才的第 2 个图改写

459
00:13:03,680 --> 00:13:05,400
就里面出现了两个图

460
00:13:05,400 --> 00:13:06,280
第 1 个图

461
00:13:06,280 --> 00:13:07,160
第 2 个图

462
00:13:07,160 --> 00:13:07,880
两个图

463
00:13:07,880 --> 00:13:10,000
最后呢返回真正的值

464
00:13:10,000 --> 00:13:11,000
这种方式呢

465
00:13:11,000 --> 00:13:15,160
就是真正的直接基于 bytecore 字节码去改动了

466
00:13:15,160 --> 00:13:18,440
所以说这种方式做的非常的彻底

467
00:13:18,440 --> 00:13:19,760
有了这种方式之后呢

468
00:13:19,760 --> 00:13:21,920
可以看到 Torch.compile 呢

469
00:13:21,920 --> 00:13:25,240
在一些目标的模型里面呢

470
00:13:25,240 --> 00:13:27,800
有 93%的模型是能够跑通的

471
00:13:27,800 --> 00:13:29,440
然后有 43%的模型呢

472
00:13:29,440 --> 00:13:30,600
是有性能优势的

473
00:13:30,600 --> 00:13:31,680
对比起 ego model

474
00:13:31,680 --> 00:13:34,920
最后来看看它的一个 points and cons

475
00:13:34,920 --> 00:13:35,520
cons 呢

476
00:13:35,520 --> 00:13:37,160
其实我觉得这一个新特性呢

477
00:13:37,160 --> 00:13:39,480
没有必要马上对它进行定义

478
00:13:39,480 --> 00:13:41,120
那看看它的 points

479
00:13:41,120 --> 00:13:44,600
就是 PyTorch 自己总结的有哪些优点

480
00:13:44,600 --> 00:13:45,280
第 1 个优点呢

481
00:13:45,280 --> 00:13:48,080
就是支持所有的 python 的表达

482
00:13:48,080 --> 00:13:51,680
因为在 python 的一个解析器里面去直接改掉了

483
00:13:51,680 --> 00:13:54,200
那这个时候肯定支持 python 的大部分的表达

484
00:13:54,200 --> 00:13:55,880
所以说它这种方式呢

485
00:13:55,880 --> 00:13:56,720
做的非常彻底

486
00:13:56,720 --> 00:13:58,040
做的非常好

487
00:13:58,040 --> 00:14:00,520
而且动静态图的转换

488
00:14:00,520 --> 00:14:02,200
基本上用户就是无感了

489
00:14:02,200 --> 00:14:05,080
加个装饰器或者加一句话包一包就行了

490
00:14:05,080 --> 00:14:05,960
第 2 个优点呢

491
00:14:05,960 --> 00:14:07,600
就是开销非常的小

492
00:14:07,600 --> 00:14:08,400
第 3 个优点呢

493
00:14:08,400 --> 00:14:10,280
就是减少了一些延迟

494
00:14:10,280 --> 00:14:11,480
它不会延迟的太多

495
00:14:11,480 --> 00:14:14,280
不像有一些像 fsr 或者 git 这种方式

496
00:14:14,280 --> 00:14:15,520
它有大量的延迟

497
00:14:15,520 --> 00:14:16,960
所以说它这种方式呢

498
00:14:16,960 --> 00:14:18,120
做的非常彻底

499
00:14:18,120 --> 00:14:21,680
直接基于 cpython 的一个解析的编译器里面呢

500
00:14:21,680 --> 00:14:23,600
直接去改掉了 bytecode

501
00:14:23,600 --> 00:14:25,840
然后基于 bytecode 进行一个改写的

502
00:14:25,840 --> 00:14:28,000
这种方式真的非常的棒

503
00:14:29,120 --> 00:14:31,280
已经讲完了 python 最新的一个特性

504
00:14:31,280 --> 00:14:32,000
touchdynamo

505
00:14:32,000 --> 00:14:32,600
好了

506
00:14:32,600 --> 00:14:33,400
谢谢各位

507
00:14:33,400 --> 00:14:34,120
摆了个掰

508
00:14:35,600 --> 00:14:37,240
卷的不行了卷的不行了

509
00:14:37,240 --> 00:14:39,080
记得一键三连加关注哦

510
00:14:39,080 --> 00:14:42,680
所有的内容都会开源在下面这条链接里面

511
00:14:42,680 --> 00:14:43,400
摆了个掰