Papers

[zhangjun]

accelerator

Modeling Deep Learning Accelerator Enabled GPUs

[zhangjun]

Papaer Lists

Mlsys

Bolt

• 论文阅读 [精读]-BOLT: BRIDGING THE GAP BETWEEN AUTO-TUNERS AND HARDWARE-NATIVE PERFORMANCE
  
• BOLT：弥合自动调优和硬件原生性能之间的差距

[zhangjun]

量化

• Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
  https://blog.csdn.net/qq_19784349/article/details/82883271
  $r=S(q-Z)$ => $q=round(\frac{r}{S}+Z)$; S - scale, Z - zero-point
  $\Large{S=\frac{val_{max}-val_{min}}{2_{bit_length}-1}}$
  $\Large{Z=round(-\frac{val_{min}}{S})}$
  

[zhangjun]

训练

PS

• Accelerating Collective Communication in Data Parallel Training across Deep Learning Frameworks

parallel training

• PipeDream: Generalized Pipeline Parallelism for DNN Training
  https://insujang.github.io/2022-06-11/parallelism-in-distributed-deep-learning/

communication

• Compressed Communication for Distributed Deep Learning: Survey and Quantitative Evaluation
• Efficient Sparse Collective Communication and its application to Accelerate Distributed Deep Learning