16-VisualizeGD

Practice 16 - Visualize Gradient Descent in TensorFlow

This practice is refer to the following resources credited to Morvan.

• 莫烦PYTHON - 有趣的机器学习: 神经网络 梯度下降
• 莫烦PYTHON - TensorFlow: 用 Tensorflow 可视化梯度下降

---

Execution

1. Run main.py with a argument

   # Make sure your current directory is in this folder
   # Using linear function as target and training function and learning rate 0.1
   $ python3 main.py 1 0.1
   
   # Using non-linear as target and training function and learning rate 0.1
   $ python3 main.py 2 0.1
   
   # Using sin/cos as target and training function and learning rate 0.1
   % python3 main.py 3 0.1

2. If succeed, you will get the following result

   # Training with "ReLU" as the activation function
   [INFO] Using linear function as target and training function
   [INFO] Using 0.1 learning rate
   Weight =  1.2083511 bias =  2.5011578
   
   # Training with "tanh" as the activation function
   [INFO] Using non-linear function as target and training function
   [INFO] Using 0.1 learning rate
   Weight =  1.2017697 bias =  2.485243
   
   # Using sin/cos function as target and training function
   [INFO] Using sin/cos function as target and training function
   [INFO] Using 0.1 learning rate
   Weight =  1.2080404 bias =  2.4960968

3. Besides, you will see the following figures (you can also try to use 1.0 as the learning rate)

   Function	Learning rate = 0.1	Learning rate = 1.0
   Linear
   Non-linear
   sin/cos

---

References

• TensorFlow Official - Tutorial
• GitHub - tensorFlow/tensorflow
• 莫烦PYTHON - TensorFlow