{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "877efa3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2021 NVIDIA Corporation. All Rights Reserved.\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     http://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "# =============================================================================="
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44b821d9",
   "metadata": {},
   "source": [
    "<img src=\"http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png\" style=\"width: 90px; float: center;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc433756",
   "metadata": {},
   "source": [
    "# Optimize Object Detection with EfficientDet and TensorRT 8"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b0f88f5",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "This notebook will show how to optimize a pre-trained TensorFlow EfficientDet model checkpoint using TensorRT 8.0.1. NVIDIA TensorRT is a platform for high-performance deep learning inference. It includes a deep learning inference optimizer and runtime that delivers low latency and high-throughput for deep learning inference applications. After optimizing the pre-trained TF EfficientDet-D0 model for object detection with NVIDIA TensorRT, inference throughput increased by up to 2x to 3x over native Tensorflow depending on the batch size and precision used for TensorRT conversion.\n",
    "\n",
    "One of the most important problems in computer vision is object detection, where objects of interest like cars, people, obstacles, etc., need to be detected. Since the inception of deep learning algorithms, there has been a lot of research in developing model architectures that can help detect and classify such objects in photos or videos. [EfficientDet](https://arxiv.org/abs/1911.09070) is state of the art object detector which is efficient and accurate while requiring less computational resources. Such a network with accuracy, low compute, and memory requirement is perfect in robotics and driverless car systems.  "
   ]
  },
  {
   "attachments": {
    "efficientdet.png": {
     "image/png": "[encoded data, 356640 characters]"
    }
   },
   "cell_type": "markdown",
   "id": "370302a7",
   "metadata": {},
   "source": [
    "## Model architecture\n",
    "\n",
    "\n",
    "Recently, the Google Brain team released their own ConvNet model called **[EfficientNet](https://arxiv.org/abs/1905.11946)**. EfficientNet forms the backbone of the **EfficientDet**. The model seeks to optimize downstream performance (eg. Object detection) given free range over depth, width, and resolution while staying within the constraints of target memory and target FLOPs. The new model architecture is discovered through neural architecture search where it optimizes for accuracy, given a certain number of FLOPS, and results in the creation of a baseline ConvNet called EfficientNet-D0. This baseline model is scaled up using compound scaling, which jointly scales up all dimensions to create a family of EfficientDet models from baseline EfficientDet-D0 through EfficientDet-D7.\n",
    "\n",
    "![efficientdet.png](attachment:efficientdet.png)\n",
    "\n",
    "[EfficientDet](https://arxiv.org/abs/1911.09070) architecture – It employs [EfficientNet](https://arxiv.org/abs/1905.11946) as the backbone network, BiFPN as the feature network, and shared class/box prediction network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "488b3faa",
   "metadata": {},
   "source": [
    "## TensorRT model conversion pipeline\n",
    "\n",
    "TensorRT provides a lot of options for model optimization like reduced precision, batching, layer fusion, etc. In particular, for EfficientDet model, additional optimizations are performed in addition to the ones mentioned previously:\n",
    "* Fusion of Convolution+Swish layers (used throughout the EfficientNet backbone)\n",
    "* Improved INT8 Global Average Pooling (used in the SE blocks of EfficientNet backbone)\n",
    "\n",
    "To run a model with TensorRT, we'll follow these steps: \n",
    "* Download and save pre-trained TensorFlow model checkpoint in saved model format\n",
    "* Export the checkpoint an ONNX model\n",
    "* Build TensorRT engine from the ONNX model and serialize to TensorRT plan file\n",
    "\n",
    "The final TensorRT engine of EfficientDet can then be launched for inference. Note that TensorRT engine is being runtime optimized before serialization. TensorRT tries a vast set of options to find the strategy that performs best on user’s GPU (depends on the type of underlying GPU) - so it takes a few minutes. After the TensorRT plan file is created, it can be reused."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f77ab56",
   "metadata": {},
   "source": [
    "## Requirements\n",
    "\n",
    "* Nvidia GPU (Check TensorRT 8 for GPU requirements)\n",
    "* Nvidia driver 465 with CUDA toolkit \n",
    "* TensorRT >= 8.0.1 as per [TensorRT installation](https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html) guide\n",
    "* TensorFlow >= 2.4.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "408ac33a",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Before running this notebook, please check whether NVIDIA driver, CUDA, TensorRT are installed using the following commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b59a2d16",
   "metadata": {},
   "outputs": [],
   "source": [
    "# NVIDIA driver and CUDA version\n",
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f12a26fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TensorRT version\n",
    "!python3 -c 'import tensorrt; print(\"TensorRT version: {}\".format(tensorrt.__version__))'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6b01d8d",
   "metadata": {},
   "source": [
    "You will need to make sure the Python bindings for TensorRT are also installed correctly, these are available by installing the `python3-libnvinfer` and `python3-libnvinfer-dev` packages on your TensorRT download."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11e4a6b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "!dpkg -l | grep TensorRT\n",
    "\n",
    "# Check 'Python 3 bindings for TensorRT'\n",
    "# Check 'Python 3 development package for TensorRT'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34066147",
   "metadata": {},
   "source": [
    "## Install dependencies for EfficientDet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3dd17c7",
   "metadata": {},
   "source": [
    "### 1. Install requirements and dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf687a52-d705-403e-bf3f-df2ff3bab596",
   "metadata": {},
   "outputs": [],
   "source": [
    "# EfficientDet sample is present at this location\n",
    "!ls -ltr $TRT_OSSPATH/samples/python/efficientdet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a5074ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install the dependencies\n",
    "!pip3 install -r $TRT_OSSPATH/samples/python/efficientdet/requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b16d2c6e",
   "metadata": {},
   "source": [
    "### 2. Clone [AutoML github repository](https://github.com/google/automl) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02343be0",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/google/automl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0ad7c98",
   "metadata": {},
   "source": [
    "### 3. Install requirements for AutoML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b05ca377",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install matplotlib>=3.0.3 PyYAML>=5.1 tensorflow-model-optimization>=0.5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "855e8150-6ef7-4cc1-b734-8087bfc38bb3",
   "metadata": {},
   "source": [
    "The full list of requirements for AutoML is present at `automl/efficientdet/requirements.txt`, but we only need the above for this example."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c514b30",
   "metadata": {},
   "source": [
    "### 4. Install onnx_graphsurgeon \n",
    "\n",
    "You will also need the latest onnx_graphsurgeon python module. If not already installed by TensorRT, you can install it manually by running:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41d09474",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install 'git+https://github.com/NVIDIA/TensorRT#subdirectory=tools/onnx-graphsurgeon'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "659c26fc",
   "metadata": {},
   "source": [
    "# Model conversion\n",
    "\n",
    "## 1. TensorFlow Saved Model\n",
    "\n",
    "The first step in TensorRT pipeline for EfficientDet is downloading pre-trained TensorFlow checkpoint and converting it into TensorFlow saved model as follows:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3863495",
   "metadata": {},
   "outputs": [],
   "source": [
    "![ ! -d \"tf_checkpoint\" ] && mkdir tf_checkpoint\n",
    "!wget https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco2/efficientdet-d0.tar.gz -P tf_checkpoint\n",
    "!tar -xvf tf_checkpoint/efficientdet-d0.tar.gz -C tf_checkpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15a8d061",
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls tf_checkpoint/efficientdet-d0/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c25a4a77",
   "metadata": {},
   "source": [
    "## 2. Export a TensorFlow saved model\n",
    "\n",
    "The extracted TensorFlow checkpoint now be converted into saved model format using the automl/efficientdet/model_inspect.py script:\n",
    "```\n",
    "* --runmode is passed as saved_model\n",
    "* --model_name supports any one of the model from efficientdet-d0 to efficientdet-d7x\n",
    "* --ckpt_path /path/to/tf_checkpoint\n",
    "* --saved_model_dir /path/to/tf_model with protobuf graph and other related files inside.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1286b00a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create a directory to store TensorFlow saved model \n",
    "![ ! -d \"tf_model\" ] && mkdir tf_model\n",
    "\n",
    "# Export TF model\n",
    "!python3 ./automl/efficientdet/model_inspect.py \\\n",
    "    --runmode saved_model \\\n",
    "    --model_name efficientdet-d0 \\\n",
    "    --ckpt_path ./tf_checkpoint/efficientdet-d0/ \\\n",
    "    --saved_model_dir ./tf_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fb9cd58",
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls tf_model/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab756d32",
   "metadata": {},
   "source": [
    "## 3. Create ONNX Graph\n",
    "\n",
    "To generate an ONNX model file, first find the input shape that corresponds to the model you're converting:\n",
    "\n",
    "| **Model**           | **Input Size** |\n",
    "| --------------------|----------------|\n",
    "| efficientdet-d0     | 512,512        |\n",
    "| efficientdet-d1     | 640,640        |\n",
    "| efficientdet-d2     | 768,768        |\n",
    "| efficientdet-d3     | 896,896        |\n",
    "| efficientdet-d4     | 1024,1024      |\n",
    "| efficientdet-d5     | 1280,1280      |\n",
    "| efficientdet-d6     | 1280,1280      |\n",
    "| efficientdet-d7     | 1536,1536      |\n",
    "| efficientdet-d7x    | 1536,1536      |\n",
    "| efficientdet-lite0  | 320,320        |\n",
    "| efficientdet-lite1  | 384,384        |\n",
    "| efficientdet-lite2  | 448,448        |\n",
    "| efficientdet-lite3  | 512,512        |\n",
    "| efficientdet-lite3x | 640,640        |\n",
    "| efficientdet-lite4  | 640,640        |\n",
    "\n",
    "To create the ONNX graph, execute efficientdet/create_onnx.py script which takes the following arguments:\n",
    "```\n",
    "* --saved_model /path/to/tf_model \n",
    "* --onnx /path/to/onnx.model\n",
    "* --input_size One of the input shapes corresponding to the model mentioned previously\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0f0f987",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create directory for onnx_model\n",
    "![ ! -d \"onnx_model\" ] && mkdir onnx_model\n",
    "\n",
    "# Export TF to ONNX\n",
    "!python3 $TRT_OSSPATH/samples/python/efficientdet/create_onnx.py \\\n",
    "    --saved_model ./tf_model/ \\\n",
    "    --onnx ./onnx_model/model.onnx \\\n",
    "    --input_size '512,512'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb9ebeca",
   "metadata": {},
   "source": [
    "This will create the file `model.onnx` which is ready to be converted to TensorRT. \n",
    "\n",
    "You can visualize the resulting file with a tool such as [Netron](https://netron.app/).\n",
    "\n",
    "The script has a few additional arguments:\n",
    "\n",
    "* `--nms_threshold` allows overriding the NMS score threshold value. The runtime latency of the EfficientNMS plugin is sensitive to the score threshold used, so it's a good practice to set this value as high as possible, while still fulfilling your application requirements, to reduce latency as much as possible.\n",
    "* `--preprocessor [imagenet,scale_range]` allows switching between two possible image preprocessing methods. Most EfficientDet models use the `imagenet` method, which this argument defaults to, and corresponds to standard ImageNet mean subtraction and standard deviation normalization. The `scale_range` method instead normalizes the image to a range of [-1,+1]. Please use this method only when converting the **AdvProp** pre-trained checkpoints, as they were created with this preprocessor operation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07945320",
   "metadata": {},
   "source": [
    "## 4. Build TensorRT engine\n",
    "\n",
    "Final step is to convert the exported ONNX model into TensorRT using the efficientdet/build_engine.py script\n",
    "```\n",
    "* --onnx /path/to/model.onnx\n",
    "* --engine /path/to/trt_output\n",
    "* --precision (fp32,fp16,int8)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5257225c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create directory for exported TensorRT engine\n",
    "![ ! -d \"trt_engine\" ] && mkdir trt_engine\n",
    "\n",
    "# Build engine with FP32 precision\n",
    "!python3 $TRT_OSSPATH/samples/python/efficientdet/build_engine.py \\\n",
    "    --onnx ./onnx_model/model.onnx \\\n",
    "    --engine ./trt_engine/engine.trt \\\n",
    "    --precision fp32\n",
    "\n",
    "## To build TensorRT engine with INT8 precision run the following after setting path to 'calib_input' and 'calib_cache':\n",
    "# python $TRT_OSSPATH/samples/python/efficientdet/build_engine.py \\\n",
    "#     --onnx ./onnx_model/model.onnx \\\n",
    "#     --engine ./trt_engine/engine.trt \\\n",
    "#     --precision int8 \\\n",
    "#     --calib_input /path/to/calibration/images \\\n",
    "#     --calib_cache /path/to/calibration.cache\n",
    "\n",
    "# Where --calib_input points to a directory with several thousands of images. \n",
    "# For example, this could be a subset of the training or validation datasets that were used for the model.\n",
    "# It is important that this data represents the runtime data distribution relatively well, therefore,\n",
    "# the more images that are used for calibration, the better accuracy that will be achieved in INT8 precision. \n",
    "# For models trained for the COCO dataset, we have found that 5,000 images gives a good result.\n",
    "\n",
    "# The --calib_cache controls where the calibration cache file will be written to.\n",
    "# This is useful to keep a cached copy of the calibration results. \n",
    "# Next time you need to build the engine for the same network, if this file exists, \n",
    "# it will skip the calibration step and use the cached values instead.\n",
    "\n",
    "# Run python build_engine.py --help for additional calibration options."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59256876",
   "metadata": {},
   "source": [
    "The file `engine.trt` will be created, which can now be used to infer with TensorRT.\n",
    "For best results, make sure no other processes are using the GPU during engine build, as it may affect the optimal tactic selection process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10a9e312",
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls trt_engine/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "953ff4d4",
   "metadata": {},
   "source": [
    "# Benchmarking TensorRT Engine\n",
    "\n",
    "Optionally, you can obtain execution timing information for the built engine by using the trtexec utility, as:\n",
    "\n",
    "`NOTE:` After a succesful TensorRT OSS build, the `trtexec` binary should have been created in the `out/` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8763b4f9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!/workspace/TensorRT/build/out/trtexec \\\n",
    "    --loadEngine=trt_engine/engine.trt \\\n",
    "    --useCudaGraph --noDataTransfers \\\n",
    "    --iterations=100 --avgRuns=100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05af5387",
   "metadata": {},
   "source": [
    "The step above should generate a Performance summary. For instance: <br>\n",
    "```\n",
    "GPU Compute Time: min = 3.58606 ms, max = 4.67763 ms, mean = 3.71858 ms, median = 3.6167 ms, percentile(99%) = 4.56601 ms\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea40c5da",
   "metadata": {},
   "source": [
    "# Inference\n",
    "\n",
    "Now let's check inference of our TensorRT engine and compare with TensorFlow predictions and ground truth on COCO validation 2017 dataset as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04e6c30a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Download the validation dataset images\n",
    "!wget http://images.cocodataset.org/zips/val2017.zip\n",
    "\n",
    "# Unzip the archive\n",
    "!unzip val2017.zip "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7421869b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download the annotations (Ground truth)\n",
    "!wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip\n",
    "    \n",
    "# Unzip the annotations\n",
    "!unzip annotations_trainval2017.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43abe1a2",
   "metadata": {},
   "source": [
    "To check how the TensorRT results look in comparison to the original TensorFlow model and ground truth, you can run efficientdet/compare_tf.py:\n",
    "\n",
    "```\n",
    "* --engine /path/to/trt_engine\n",
    "* --saved_model /path/to/tf_saved_model\n",
    "* --input /path/to/input_image\n",
    "* --annotations /path/to/downloaded_annotations/annotations.json\n",
    "* --labels /path/to/labels\n",
    "* --output /path/to/output directory\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d025d09f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create directory for exported TensorRT engine\n",
    "![ ! -d \"output_imgs\" ] && mkdir output_imgs\n",
    "\n",
    "# Run inference and compare the outputs\n",
    "!python3 $TRT_OSSPATH/samples/python/efficientdet/compare_tf.py \\\n",
    "    --engine ./trt_engine/engine.trt \\\n",
    "    --saved_model ./tf_model/ \\\n",
    "    --input ./val2017 \\\n",
    "    --annotations ./annotations/instances_val2017.json \\\n",
    "    --labels $TRT_OSSPATH/samples/python/efficientdet/labels_coco.txt \\\n",
    "    --output ./output_imgs"
   ]
  },
  {
   "attachments": {
    "000000002153.compare.png": {
     "image/png": "[encoded data, 843208 characters]"
    }
   },
   "cell_type": "markdown",
   "id": "37c32c51",
   "metadata": {},
   "source": [
    "![000000002153.compare.png](attachment:000000002153.compare.png)\n",
    "\n",
    "The inference on validation image from [COCO dataset](http://images.cocodataset.org/zips/val2017.zip) using TensorRT engine of EfficientDet shows detection of people and baseball bat \n",
    "\n",
    "The predictions on left are by TensorFlow saved model, in the center are TensorRT predictions and the ones on the right are ground truth. As we can see, the accuracy of EfficientDet TensorRT engine predictions remain same as the original TensorRT model. \n",
    "\n",
    "The TensorRT engine built with this process can also be deployed at scale with either [Triton Inference Server](https://developer.nvidia.com/nvidia-triton-inference-server) or [DeepStream SDK](https://developer.nvidia.com/deepstream-sdk)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf70d4ba",
   "metadata": {},
   "source": [
    "## Validation on entire dataset\n",
    "\n",
    "Given a validation dataset (such as [COCO val2017 data](http://images.cocodataset.org/zips/val2017.zip)) and ground truth annotations (such as [COCO instances_val2017.json](http://images.cocodataset.org/annotations/annotations_trainval2017.zip)), you can get the mAP metrics for the built TensorRT engine. This will use the mAP metrics calculation script from the AutoML EfficientDet repository on `https://github.com/google/automl`.\n",
    "\n",
    "```\n",
    "python eval_coco.py \\\n",
    "    --engine /path/to/engine.trt \\\n",
    "    --input /path/to/coco/val2017 \\\n",
    "    --annotations /path/to/coco/annotations/instances_val2017.json \\\n",
    "    --automl_path /path/to/automl\n",
    "```\n",
    "\n",
    "Where the `--automl_path` argument points to the root of the AutoML repository.\n",
    "\n",
    "**NOTE:** mAP metrics are highly sensitive to NMS threshold. Using a high threshold will obviously reduce the mAP value. Ideally, this should run with a threshold of 0.00 or 0.01, but such a low threshold will impact the runtime performance of the EfficientNMS plugin. So you may need to build separate TensorRT engines for different purposes, one with a low threshold (like 0.01) dedicated for validation, and one with your application specific threshold (like 0.4) for deployment inference to minimimze latency. This is why we keep the NMS threshold as a configurable parameter in the TensorRT conversion script.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
