nvidia--tensorrt/quickstart/IntroNotebooks/0. Running This Guide.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running This Guide:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This guide is presented as a series of Jupyter notebooks covering both Tensorflow and PyTorch using a Python runtime.\n",
    "\n",
    "If you would like to run this code yourself, you can do so using the following steps:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Setup__:\n",
    "\n",
    "The basic prerequisites you will need to use TensorRT are:\n",
    "\n",
    "- a [supported NVIDIA GPU](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#hardware-precision-matrix) (preferred compute capability 7.0 or higher - which includes INT8 precision support) \n",
    "- latest [NVIDIA GPU drivers](https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html)\n",
    "- a [supported CUDA and cuDNN](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#platform-matrix) installation\n",
    "\n",
    "You can make sure your GPU environment is properly configured and check which GPU and CUDA version you are using with nvidia-smi:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For some of the examples you will also need cuda-python, skimage, matplotlib, and onnx:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install cuda-python onnx scikit-image matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__PyTorch__:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using PyTorch to walk through the basic steps of deploying a TensorRT model by [exporting the model in ONNX format](https://pytorch.org/docs/stable/onnx.html).\n",
    "\n",
    "You can find PyTorch installation instructions [here](https://pytorch.org/get-started/locally/), or use one of NVIDIA's NGC containers [here](https://ngc.nvidia.com/catalog/containers/nvidia:pytorch). \n",
    "\n",
    "You will also need torchvision:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Using Colab:__\n",
    "\n",
    "You can also test these notebooks out using [Google Colab](https://colab.research.google.com/), which includes PyTorch, as well as supported NVIDIA drivers. __Make sure to select a GPU hardware accelerator__ in the runtime options. Just note that TensorRT performance is best on newer gpus, Colab often has trouble with reduced precision inference, and you will have to use an older version of TensorRT."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TensorRT Support:__\n",
    "\n",
    "TensorRT support for NVIDIA GPUs is determined by their __compute capability__. You can check the compute cabapility of your card on the [NVIDIA website](https://developer.nvidia.com/cuda-gpus).\n",
    "\n",
    "TensorRT supports different feautures depending on your compute capability. Higher compute capabilities allow additional TensorRT optimizations, like reduced precision inference. You can check which TensorRT features are supported by your compute capability in the [TensorRT documentation](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#hardware-precision-matrix).\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Next Steps:__\n",
    "\n",
    "Now, start by opening [1. Introduction.ipynb](./1.%20Introduction.ipynb) and proceed through the notebook!\n",
    "\n",
    "The notebooks included with this guide are:\n",
    "- [1. Introduction.ipynb](./1.%20Introduction.ipynb)\n",
    "- [2. Using PyTorch through ONNX.ipynb](./2.%20Using%20PyTorch%20through%20ONNX.ipynb)\n",
    "- [3. Understanding TensorRT Runtimes.ipynb](./3.%20Understanding%20TensorRT%20Runtimes.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}