nn_Sequential_、_nn_ModuleList_、_nn_ModuleDict.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyNndqhWE21m6/eWpVoqgH4M",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/zzqnot996/learn_deeplearning/blob/main/nn_Sequential_%E3%80%81_nn_ModuleList_%E3%80%81_nn_ModuleDict.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "`1、nn.Sequential` 、 `nn.ModuleList` 、 `nn.ModuleDict` 类都继承自 `Module` 类。\n",
        "\n",
        "  \n",
        "\n",
        "![](//i0.hdslb.com/bfs/article/4aa545dccf7de8d4a93c2b2b8e3265ac0a26d216.png@progressive.webp)\n",
        "\n",
        "`2、nn.Sequential、nn.ModuleList` 和 `nn.ModuleDict`语法\n"
      ],
      "metadata": {
        "id": "lxxLYWY0oLHL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "\n",
        "net = nn.Sequential(nn.Linear(32, 64), nn.ReLU())\n",
        "\n",
        "net = nn.ModuleList([nn.Linear(32, 64), nn.ReLU()])\n",
        "\n",
        "net = nn.ModuleDict({'linear': nn.Linear(32, 64), 'act': nn.ReLU()})"
      ],
      "metadata": {
        "id": "tDviYUVhobGF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "`3、Sequential` 、`ModuleDict`、 `ModuleList` 的区别\n",
        "\n",
        " `1）``ModuleList` 仅仅是一个储存各种模块的列表，这些模块之间没有联系也没有顺序（所以不用保证相邻层的输入输出维度匹配），而且没有实现 `forward` 功能需要自己实现\n",
        "\n",
        " 2）和 `ModuleList` 一样， `ModuleDict` 实例仅仅是存放了一些模块的字典，并没有定义 `forward` 函数需要自己定义  但是字典形式可以使用自己命名，在微调权重参数时可以直接根据名字复制权重\n",
        "\n",
        " 3）而 `Sequential` 内的模块需要按照顺序排列，要保证相邻层的输入输出大小相匹配，内部 `forward` 功能已经实现，所以，直接如下写模型，是可以直接调用的，不再需要写forward，sequential 内部已经有 forward"
      ],
      "metadata": {
        "id": "iA0MuCJ0ogrN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "net1 = nn.Sequential(nn.Linear(32, 64), nn.ReLU())\n",
        "net2 = nn.ModuleList([nn.Linear(32, 64), nn.ReLU()])\n",
        "net3 = nn.ModuleDict({'linear': nn.Linear(32, 64), 'act': nn.ReLU()})\n",
        "\n",
        "# print(net1)\n",
        "# print(net2)\n",
        "# print(net3)\n",
        "\n",
        "x = torch.randn(8, 3, 32)\n",
        "print(net1(x).shape)\n",
        "# print(net2(x).shape)  # 会报错，提示缺少forward\n",
        "# print(net3(x).shape)   # 会报错，提示缺少forward # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4-pbRhCJpACi",
        "outputId": "23a312ad-b45d-4334-a3fd-7fdbcd3929ea"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([8, 3, 64])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "为 nn.ModuleList 写 forward 函数"
      ],
      "metadata": {
        "id": "OaquVnOIpJCQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "\n",
        "class My_Model(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(My_Model, self).__init__()\n",
        "        self.layers = nn.ModuleList([nn.Linear(32, 64),nn.ReLU()])\n",
        "\n",
        "    def forward(self, x):\n",
        "        for layer in self.layers:\n",
        "            x = layer(x)\n",
        "        return x\n",
        "\n",
        "net = My_Model()\n",
        "\n",
        "x = torch.randn(8, 3, 32)\n",
        "out = net(x)\n",
        "print(out.shape) # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TMHzW6DppNRe",
        "outputId": "91977764-a112-45ff-ce53-2d93a93df05c"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([8, 3, 64])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "AoQH0577qp0f"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "为 nn.ModuleDict 写 forward 函数 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "id": "Ndbu_xgepUJa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "\n",
        "class My_Model(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(My_Model, self).__init__()\n",
        "        self.layers = nn.ModuleDict({'linear': nn.Linear(32, 64), 'act': nn.ReLU()})\n",
        "\n",
        "    def forward(self, x):\n",
        "        for layer in self.layers.values():\n",
        "            x = layer(x)\n",
        "        return x\n",
        "\n",
        "net = My_Model()\n",
        "x = torch.randn(8, 3, 32)\n",
        "out = net(x)\n",
        "print(out.shape) # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8woKgD0IpVYc",
        "outputId": "3269760b-172d-422c-96b8-4a6b06d2060f"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([8, 3, 64])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "将 nn.ModuleList 转换成 nn.Sequential"
      ],
      "metadata": {
        "id": "9Aii-k8Qpaep"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "module_list = nn.ModuleList([nn.Linear(32, 64), nn.ReLU()])\n",
        "net = nn.Sequential(*module_list)\n",
        "x = torch.randn(8, 3, 32)\n",
        "print(net(x).shape) # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "id": "HmWt44Tbpb45"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "将 nn.ModuleDict 转换成 nn.Sequentia"
      ],
      "metadata": {
        "id": "Ah0WdC3jqraV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "module_dict = nn.ModuleDict({'linear': nn.Linear(32, 64), 'act': nn.ReLU()})\n",
        "net = nn.Sequential(*module_dict.values())\n",
        "x = torch.randn(8, 3, 32)\n",
        "print(net(x).shape) # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "id": "TNHFO8eqqujb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "`4、ModuleDict`、 `ModuleList` 的区别\n",
        "\n",
        " `1）ModuleDict` 可以给每个层定义名字，`ModuleList` 不会\n",
        "\n",
        " `2）ModuleList` 可以通过索引读取，并且使用 append 添加元素"
      ],
      "metadata": {
        "id": "lrAvPiVsq08u"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "\n",
        "net = nn.ModuleList([nn.Linear(32, 64), nn.ReLU()])\n",
        "net.append(nn.Linear(64, 10))\n",
        "print(net) # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TEII8gkVq23c",
        "outputId": "e0d11489-0ca4-4d59-af2c-e9a09897a320"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ModuleList(\n",
            "  (0): Linear(in_features=32, out_features=64, bias=True)\n",
            "  (1): ReLU()\n",
            "  (2): Linear(in_features=64, out_features=10, bias=True)\n",
            ")\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " 3）ModuleDict 可以通过 key 读取，并且可以像 字典一样添加元素 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "id": "wG_AnRi-q7iW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "\n",
        "net = nn.ModuleDict({'linear1': nn.Linear(32, 64), 'act': nn.ReLU()})\n",
        "net['linear2'] = nn.Linear(64, 128)\n",
        "print(net) # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PYJkaUpkq8Ge",
        "outputId": "178e0919-d6a5-4c8e-a84c-2a06c202bbe2"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ModuleDict(\n",
            "  (linear1): Linear(in_features=32, out_features=64, bias=True)\n",
            "  (act): ReLU()\n",
            "  (linear2): Linear(in_features=64, out_features=128, bias=True)\n",
            ")\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "5、`nn.ModuleList` 、 `nn.ModuleDict` 与 Python `list、``Dict` 的区别\n",
        "\n",
        "`加入到 ModuleList` 、`ModuleDict` 里面的所有模块的参数会被自动添加到整个网络中。"
      ],
      "metadata": {
        "id": "lpkzNSEFrC-j"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "\n",
        "net = nn.ModuleList([nn.Linear(32, 64), nn.ReLU()])\n",
        "\n",
        "for name, param in net.named_parameters():\n",
        "    print(name, param.size()) # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hEfsxM9ZrEGD",
        "outputId": "ba246839-19bb-4312-b6c2-885b3beb39c0"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.weight torch.Size([64, 32])\n",
            "0.bias torch.Size([64])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "\n",
        "net = nn.ModuleDict({'linear': nn.Linear(32, 64), 'act': nn.ReLU()})\n",
        "\n",
        "for name, param in net.named_parameters():\n",
        "    print(name, param.size())\n",
        " # 作者：Enzo_Mi https://www.bilibili.com/read/cv25687486 出处：bilibili"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ORsc3eO6rGO4",
        "outputId": "3c3b4b5d-2ec0-41cb-b1c8-7d7d29ddb913"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "linear.weight torch.Size([64, 32])\n",
            "linear.bias torch.Size([64])\n"
          ]
        }
      ]
    }
  ]
}