Object_Detection_TensorFlow_Hub.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Object Detection: TensorFlow Hub.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pc36yXvvvp9m"
      },
      "source": [
        "# **Object Detection: Using TensorFlow Hub**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zBpW9D3MwYJ-"
      },
      "source": [
        "#### `Code by @Om Mule`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6CUrXQwHwc31"
      },
      "source": [
        "`Jupyter Notebook` contains code for `Object Detection` using `TensorFlow Hub`\n",
        "\n",
        "1. Explore `TensorFlow Hub` for models\n",
        "2. Import model into workspace\n",
        "3. Preprocess image for getting `predictions`.\n",
        "4. Feed the preprocessed images to model to get `predictions`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kQ9jPD3yxUii"
      },
      "source": [
        "## Import Libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MAdeKG9AvYaX"
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import sklearn\n",
        "import seaborn as sns\n",
        "\n",
        "import tensorflow_hub as hub\n",
        "from PIL import Image\n",
        "from PIL import ImageOps\n",
        "import tempfile\n",
        "from six.moves.urllib.request import urlopen\n",
        "from six import BytesIO\n",
        "%matplotlib inline"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CML32F5Mzjhk"
      },
      "source": [
        "`Tensorflow Hub` is a repository of trained `Machine Learning models` which you can reuse in your own projects. \n",
        "\n",
        "\n",
        "Link for `TensorFlow Hub`:  [TensorFlow Hub](https://www.tensorflow.org/hub)\n",
        "\n",
        "Link for `Model`:   [Model URL](https://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1) \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "InFZGk8Tyi6E"
      },
      "source": [
        "# Importing the model from TensorFlow Hub\n",
        "\n",
        "module_handle = \"https://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1\""
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FAb_UUBXdWVg"
      },
      "source": [
        "### Load the model\n",
        "\n",
        "We will load the `model` using the `module handle` specified"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dmXlIJcydVfK",
        "outputId": "628063c7-9a52-4995-a148-7e49070bf884"
      },
      "source": [
        "model = hub.load(module_handle)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mYcJfgabdwLk"
      },
      "source": [
        "### Choosing the `signatures`\n",
        "\n",
        "Some models in the Tensorflow hub can be used for different tasks. So each model's documentation should show what `signature` to use when running the model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TjVTTGgNdj8k"
      },
      "source": [
        "# Get all signatures\n",
        "\n",
        "model.signatures.keys()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MB7XklpbeF8H"
      },
      "source": [
        "Please choose the `'default'` signature for your `object detector.`\n",
        "- For `object detection` models, its 'default' signature will accept a batch of `image tensors` and output a `dictionary` describing the objects detected, which is what you'll want here."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f48fBxU9eBE5"
      },
      "source": [
        "detector = model.signatures['default']"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bzt7srK6f0Bm"
      },
      "source": [
        "### Download & Resize image\n",
        "\n",
        "This function `downloads` an image from a `URL` & `preprocesses` it & then `saves` it to the disk.\n",
        "\n",
        "-`tempfile package` in Python is used to create a temporary file of the `image.`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ec-4xyikeWJW"
      },
      "source": [
        "def download_and_resize_image(url, new_width=256, new_height=256):\n",
        "  # Fetch an image online, resizes & saves it locally\n",
        "\n",
        "  # Create a JPG file instance\n",
        "  _, filename = tempfile.mkstemp(suffix=\".jpg\")\n",
        "  response = urlopen(url)\n",
        "  image_data = response.read()\n",
        "\n",
        "  # Store the image data in memory buffer\n",
        "  image_data = BytesIO(image_data)\n",
        "  pil_image = Image.open(image_data)\n",
        "\n",
        "  # Resize the image using Pillow (if aspect ratio is different)\n",
        "  pil_image = ImageOps.fit(pil_image, (new_width, new_height), Image.ANTIALIAS)\n",
        "  pil_image_rgb = pil_image.convert(\"RGB\")\n",
        "\n",
        "  # Save image to temporary .jpg file created earlier\n",
        "  pil_image_rgb.save(filename, format=\"JPEG\", quality=90)\n",
        "  print(\"Image downloaded to %s.\"% filename)\n",
        "  return filename"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GfFSH7CBH2kL",
        "outputId": "4289325b-48fc-4687-abde-4d80a1c1a331"
      },
      "source": [
        "# Download and preprocess the image\n",
        "url = \"https://upload.wikimedia.org/wikipedia/commons/f/fb/20130807_dublin014.JPG\"\n",
        "\n",
        "# Using original height & width\n",
        "downloaded_image_path = download_and_resize_image(url, 3872, 2592)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Image downloaded to /tmp/tmpxdrb73ju.jpg.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NaafIGp5IvIl"
      },
      "source": [
        "## Run the detector\n",
        "\n",
        "Run the detector with `preprocessing functions` & using the `model` "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MP-mYaLvImHD"
      },
      "source": [
        "def load_img(path):\n",
        "  # Load a JPEG image & convert it to a tensor\n",
        "  img = tf.io.read_file(path)\n",
        "\n",
        "  # Conversion to a tensor\n",
        "  img = tf.image.decode_jpeg(img, channels=3)\n",
        "  return img\n",
        "\n",
        "\n",
        "def run_detector(detector, path):\n",
        "  img = load_img(path)\n",
        "  # Add a batch dimension in front of tensor\n",
        "  converted_img  = tf.image.convert_image_dtype(img, tf.float32)[tf.newaxis, ...]\n",
        "\n",
        "  # Get Inference\n",
        "  result = detector(converted_img)\n",
        "  # Save results\n",
        "  result = {key:value.numpy() for key,value in result.items()}\n",
        "  # print results\n",
        "  print(\"Found %d objects.\" % len(result[\"detection_scores\"]))\n",
        "\n",
        "  print(result[\"detection_scores\"])\n",
        "  print(result[\"detection_class_entities\"])\n",
        "  print(result[\"detection_boxes\"])"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l8uJRpnkKlR-"
      },
      "source": [
        "## Run the detector"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gWYraUWVJ3Cv",
        "outputId": "10575bfe-409f-4d93-ef62-89f205d01d77"
      },
      "source": [
        "run_detector(detector, downloaded_image_path)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 100 objects.\n",
            "[0.6532174  0.61050487 0.60152495 0.59255666 0.5917778  0.58154964\n",
            " 0.550532   0.4957557  0.47424877 0.47322133 0.44066477 0.4051141\n",
            " 0.39803803 0.3940655  0.3714858  0.36155996 0.3615076  0.34689182\n",
            " 0.33362356 0.31252867 0.2887799  0.25758353 0.25748897 0.25196075\n",
            " 0.24782148 0.23412406 0.20432054 0.20324662 0.17988308 0.17964761\n",
            " 0.1737451  0.16431497 0.16031103 0.15895295 0.15620388 0.15468779\n",
            " 0.14754449 0.13622597 0.12740111 0.12555613 0.12102704 0.11813033\n",
            " 0.11387308 0.11229108 0.11129158 0.09718708 0.09137249 0.08976003\n",
            " 0.0888024  0.08633561 0.08337442 0.08095147 0.07988831 0.0774143\n",
            " 0.0773216  0.07631025 0.07507852 0.07386057 0.07233316 0.07204045\n",
            " 0.07110182 0.06935571 0.06825587 0.06427855 0.0624855  0.06226387\n",
            " 0.06211156 0.05940022 0.05798778 0.05784462 0.05725535 0.05346682\n",
            " 0.05304348 0.05242869 0.04892596 0.04813037 0.04578473 0.04424176\n",
            " 0.04337559 0.04277798 0.04262636 0.04163509 0.04081756 0.03976011\n",
            " 0.03945961 0.03944291 0.03863972 0.03769058 0.03759596 0.03566739\n",
            " 0.03359376 0.03333249 0.03275878 0.03231584 0.03134292 0.02978723\n",
            " 0.02858371 0.02856135 0.02822309 0.02787886]\n",
            "[b'Person' b'Person' b'Person' b'Person' b'Footwear' b'Person' b'Building'\n",
            " b'Bicycle' b'Window' b'Building' b'Person' b'Wheel' b'Building'\n",
            " b'Building' b'Person' b'Wheel' b'Building' b'Window' b'Window'\n",
            " b'Building' b'Person' b'Van' b'Person' b'Bicycle wheel' b'Person'\n",
            " b'Window' b'Window' b'Bicycle' b'Building' b'Window' b'Window' b'Man'\n",
            " b'Person' b'Person' b'Woman' b'Clothing' b'Bicycle wheel' b'Window'\n",
            " b'Person' b'Window' b'Land vehicle' b'Land vehicle' b'Clothing'\n",
            " b'Bicycle' b'Window' b'House' b'Land vehicle' b'Land vehicle' b'House'\n",
            " b'Man' b'Window' b'Clothing' b'Footwear' b'Person' b'Window' b'Man'\n",
            " b'Man' b'House' b'Person' b'Building' b'Clothing' b'Window' b'Person'\n",
            " b'Jeans' b'Man' b'Furniture' b'Person' b'Person' b'Person'\n",
            " b'Land vehicle' b'Person' b'Window' b'House' b'Woman' b'Window' b'Man'\n",
            " b'Person' b'Man' b'Clothing' b'Bicycle' b'Man' b'Person' b'Window'\n",
            " b'Person' b'Car' b'Man' b'Car' b'Chair' b'House' b'Window' b'Clothing'\n",
            " b'Tire' b'Clothing' b'Window' b'Land vehicle' b'Window' b'Man' b'Window'\n",
            " b'Bus' b'Clothing']\n",
            "[[5.12787819e-01 5.29258847e-01 6.01622522e-01 5.52077651e-01]\n",
            " [5.19631088e-01 6.01512671e-01 6.46177113e-01 6.34626806e-01]\n",
            " [5.05507112e-01 5.00440836e-01 6.01288378e-01 5.23084342e-01]\n",
            " [4.86331582e-01 4.12729561e-01 6.78829789e-01 4.59919751e-01]\n",
            " [8.15191388e-01 9.56122279e-01 8.42702985e-01 9.87146139e-01]\n",
            " [4.95409906e-01 9.23548341e-01 8.35687816e-01 9.99051571e-01]\n",
            " [1.14792129e-02 1.22223590e-02 7.38669753e-01 4.24632907e-01]\n",
            " [5.77677429e-01 3.66453379e-01 7.12771595e-01 4.83375698e-01]\n",
            " [0.00000000e+00 1.19262360e-01 2.23896131e-01 1.83930472e-01]\n",
            " [7.74121135e-02 4.12998110e-01 5.79539895e-01 5.60446203e-01]\n",
            " [5.13818145e-01 7.48031437e-01 5.91993213e-01 7.66611218e-01]\n",
            " [6.32137895e-01 3.59925479e-01 7.03870296e-01 4.11826164e-01]\n",
            " [0.00000000e+00 7.97051966e-01 6.73368454e-01 1.00000000e+00]\n",
            " [1.60234068e-02 6.84869528e-01 5.58761537e-01 8.11168015e-01]\n",
            " [5.00276923e-01 3.76966298e-01 6.33272946e-01 4.14501339e-01]\n",
            " [6.40540302e-01 4.45089400e-01 7.02983618e-01 4.83437657e-01]\n",
            " [0.00000000e+00 2.19054237e-01 6.60400987e-01 4.33263451e-01]\n",
            " [1.93080271e-03 0.00000000e+00 1.39376864e-01 2.62956936e-02]\n",
            " [2.57196440e-03 9.66668665e-01 1.53728694e-01 1.00000000e+00]\n",
            " [5.57276420e-04 1.52055710e-03 7.65210569e-01 2.69977152e-01]\n",
            " [5.04524827e-01 3.61187756e-01 6.34731770e-01 3.85342211e-01]\n",
            " [4.83405888e-01 6.19650841e-01 5.62705696e-01 6.61556125e-01]\n",
            " [4.98067319e-01 3.64576191e-01 6.61239445e-01 4.04972345e-01]\n",
            " [6.31278813e-01 3.60364169e-01 7.04153776e-01 4.11501348e-01]\n",
            " [5.21813869e-01 5.77647567e-01 5.87599933e-01 6.00718856e-01]\n",
            " [2.19569609e-01 3.48744750e-01 3.38372618e-01 3.77075404e-01]\n",
            " [1.24863066e-01 2.50912935e-01 2.79940903e-01 2.81580776e-01]\n",
            " [5.77186048e-01 3.62296611e-01 7.07020819e-01 4.41811115e-01]\n",
            " [2.57474422e-01 5.67561567e-01 5.31102896e-01 6.87727273e-01]\n",
            " [4.20639627e-02 8.74773145e-01 2.52773494e-01 9.13028836e-01]\n",
            " [1.56351045e-01 4.43401158e-01 2.22213224e-01 4.75786090e-01]\n",
            " [5.01968026e-01 9.21486855e-01 8.36406708e-01 1.00000000e+00]\n",
            " [5.23622572e-01 5.70259511e-01 5.84519446e-01 5.91583431e-01]\n",
            " [5.13246417e-01 6.79276407e-01 5.50994515e-01 6.92579925e-01]\n",
            " [5.19120276e-01 5.99985361e-01 6.46378398e-01 6.34036422e-01]\n",
            " [5.24297774e-01 9.24962044e-01 8.10777128e-01 9.97999549e-01]\n",
            " [6.38187289e-01 4.42918181e-01 7.01653838e-01 4.84097779e-01]\n",
            " [3.42190564e-02 3.55574757e-01 1.62255093e-01 3.74921203e-01]\n",
            " [4.88476783e-01 4.53496963e-01 6.21795475e-01 4.79725718e-01]\n",
            " [9.28812078e-04 3.07699054e-01 1.06533393e-01 3.32059622e-01]\n",
            " [4.83008981e-01 6.19908273e-01 5.64775169e-01 6.60696924e-01]\n",
            " [5.82192898e-01 3.64929765e-01 7.13880658e-01 4.84707862e-01]\n",
            " [5.23547709e-01 7.49199331e-01 5.85378110e-01 7.65317559e-01]\n",
            " [6.09156787e-01 4.26705897e-01 7.05165207e-01 4.87089008e-01]\n",
            " [3.51368606e-01 9.74856079e-01 5.53130627e-01 9.98878717e-01]\n",
            " [0.00000000e+00 8.11223328e-01 6.86410844e-01 9.97151256e-01]\n",
            " [5.76297641e-01 3.57461751e-01 7.04812348e-01 4.40279901e-01]\n",
            " [5.64892411e-01 3.63023102e-01 7.08650351e-01 4.16036338e-01]\n",
            " [1.09374998e-02 2.33155154e-02 7.26522923e-01 4.21747833e-01]\n",
            " [4.84686643e-01 4.10686046e-01 6.94686472e-01 4.63092834e-01]\n",
            " [8.09777379e-02 3.84715289e-01 2.07808718e-01 4.11746383e-01]\n",
            " [5.38284421e-01 6.03573740e-01 6.34776115e-01 6.34408653e-01]\n",
            " [6.29844606e-01 6.14971519e-01 6.44933462e-01 6.25384450e-01]\n",
            " [5.02758026e-01 3.82395953e-01 5.96146226e-01 4.12722319e-01]\n",
            " [0.00000000e+00 1.24522885e-02 1.40193507e-01 2.47382112e-02]\n",
            " [5.14441311e-01 7.47791588e-01 5.91985822e-01 7.66827345e-01]\n",
            " [5.06182134e-01 5.00406921e-01 6.00681305e-01 5.23312032e-01]\n",
            " [0.00000000e+00 2.11283579e-01 6.50794089e-01 4.34300780e-01]\n",
            " [4.89451557e-01 4.54391301e-01 5.72340131e-01 4.76470768e-01]\n",
            " [0.00000000e+00 7.06215978e-01 6.16998792e-01 8.66189659e-01]\n",
            " [5.09172916e-01 4.16281193e-01 6.69304490e-01 4.59598720e-01]\n",
            " [4.65173740e-03 8.03094208e-01 1.59853578e-01 8.40397060e-01]\n",
            " [5.26151001e-01 5.68352938e-01 5.79440355e-01 5.82810223e-01]\n",
            " [6.71924829e-01 9.40277696e-01 8.21276009e-01 9.89250779e-01]\n",
            " [5.02770185e-01 3.73883098e-01 6.46991491e-01 4.12972301e-01]\n",
            " [5.74243903e-01 2.67400861e-01 6.57769084e-01 3.20318550e-01]\n",
            " [4.86056775e-01 4.44508791e-01 6.24788880e-01 4.73503351e-01]\n",
            " [5.17248929e-01 7.56969213e-01 5.88517189e-01 7.71465480e-01]\n",
            " [5.23374975e-01 5.57850122e-01 5.79139531e-01 5.73541582e-01]\n",
            " [6.12461030e-01 4.27332461e-01 7.06080198e-01 4.88251865e-01]\n",
            " [5.24124146e-01 5.61553180e-01 5.78385353e-01 5.80475152e-01]\n",
            " [0.00000000e+00 2.44231746e-01 6.07754774e-02 2.93613434e-01]\n",
            " [1.48921711e-02 2.14740215e-03 7.45441973e-01 2.59790719e-01]\n",
            " [4.93236154e-01 9.23950195e-01 8.37110877e-01 9.97755051e-01]\n",
            " [8.37683585e-03 2.42165715e-01 4.97285351e-02 2.83162564e-01]\n",
            " [5.05334914e-01 3.60175252e-01 6.43561006e-01 3.91461760e-01]\n",
            " [5.13099134e-01 5.23794115e-01 6.00504339e-01 5.42967975e-01]\n",
            " [5.20421326e-01 6.00978673e-01 6.46124125e-01 6.34366393e-01]\n",
            " [5.18224835e-01 5.03395557e-01 5.97548664e-01 5.22683859e-01]\n",
            " [5.94199121e-01 3.61327976e-01 7.05465913e-01 4.15853351e-01]\n",
            " [5.13256431e-01 6.79316938e-01 5.50533950e-01 6.92482173e-01]\n",
            " [5.22302687e-01 5.36195457e-01 5.97564995e-01 5.53163290e-01]\n",
            " [4.29876357e-01 8.28702271e-01 5.89928269e-01 8.64323139e-01]\n",
            " [5.04884601e-01 3.89426976e-01 6.15080714e-01 4.19936091e-01]\n",
            " [5.26588559e-01 6.27176881e-01 5.63299775e-01 6.53728902e-01]\n",
            " [5.01304924e-01 3.64189029e-01 6.59964740e-01 4.03793275e-01]\n",
            " [5.15171111e-01 6.24104798e-01 5.63795388e-01 6.58002079e-01]\n",
            " [5.73137939e-01 2.66902655e-01 6.66162014e-01 3.18640232e-01]\n",
            " [8.34235623e-02 4.07414347e-01 5.84092379e-01 5.58522940e-01]\n",
            " [2.88196921e-01 4.77982452e-04 4.14364636e-01 3.65995839e-02]\n",
            " [4.97272849e-01 4.55296665e-01 5.83817124e-01 4.77936029e-01]\n",
            " [6.27168000e-01 3.61024052e-01 7.05996811e-01 4.09780174e-01]\n",
            " [5.15861034e-01 3.80056977e-01 5.96893847e-01 4.11758274e-01]\n",
            " [1.18098035e-02 3.08121800e-01 9.72859487e-02 3.25038970e-01]\n",
            " [5.12501717e-01 6.23653352e-01 5.62422156e-01 6.57641888e-01]\n",
            " [4.01003152e-01 8.85088801e-01 5.81281602e-01 9.39214468e-01]\n",
            " [5.13853133e-01 5.29484570e-01 6.02009714e-01 5.52362800e-01]\n",
            " [0.00000000e+00 1.00606047e-02 1.36156827e-01 3.16007212e-02]\n",
            " [4.80426341e-01 6.20422781e-01 5.65284550e-01 6.60150290e-01]\n",
            " [5.19355476e-01 3.61840397e-01 6.24995410e-01 3.84919673e-01]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9jhWfqaYKr6r"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}