{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "00_pytorch_fundamentals_exercise_solutions.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# 00. PyTorch Fundamentals Exercise Solutions\n",
        "\n",
        "### 1. Documentation reading \n",
        "\n",
        "A big part of deep learning (and learning to code in general) is getting familiar with the documentation of a certain framework you're using. We'll be using the PyTorch documentation a lot throughout the rest of this course. So I'd recommend spending 10-minutes reading the following (it's okay if you don't get some things for now, the focus is not yet full understanding, it's awareness):\n",
        "  * The documentation on [`torch.Tensor`](https://pytorch.org/docs/stable/tensors.html#torch-tensor).\n",
        "  * The documentation on [`torch.cuda`](https://pytorch.org/docs/master/notes/cuda.html#cuda-semantics).\n",
        "\n"
      ],
      "metadata": {
        "id": "AzDBM_v4iMe7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# No code solution (reading)"
      ],
      "metadata": {
        "id": "bGD0oD8Kizak"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2. Create a random tensor with shape `(7, 7)`.\n"
      ],
      "metadata": {
        "id": "__iXqqz-ioUJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import torch\n",
        "import torch \n",
        "\n",
        "# Create random tensor\n",
        "X = torch.rand(size=(7, 7))\n",
        "X, X.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6pUq9Dc8i2L7",
        "outputId": "0fefb85a-c3f7-4f8f-8ff7-cff485f7cc8d"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([[0.5656, 0.4012, 0.1987, 0.2464, 0.6861, 0.4953, 0.3433],\n",
              "         [0.0032, 0.0228, 0.9020, 0.1267, 0.8009, 0.5274, 0.7453],\n",
              "         [0.9123, 0.8138, 0.1667, 0.5998, 0.4657, 0.4473, 0.8367],\n",
              "         [0.5302, 0.2213, 0.4747, 0.6485, 0.4770, 0.8675, 0.3054],\n",
              "         [0.4226, 0.1398, 0.4495, 0.6974, 0.1808, 0.5872, 0.6931],\n",
              "         [0.2153, 0.7517, 0.3505, 0.3815, 0.3244, 0.2511, 0.4269],\n",
              "         [0.1158, 0.6696, 0.3733, 0.2633, 0.4102, 0.1101, 0.1613]]),\n",
              " torch.Size([7, 7]))"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 3. Perform a matrix multiplication on the tensor from 2 with another random tensor with shape `(1, 7)` (hint: you may have to transpose the second tensor)."
      ],
      "metadata": {
        "id": "9-XxvRLfiqkR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create another random tensor\n",
        "Y = torch.rand(size=(1, 7))\n",
        "# Z = torch.matmul(X, Y) # will error because of shape issues\n",
        "Z = torch.matmul(X, Y.T) # no error because of transpose\n",
        "Z, Z.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NcLqR0Sbi_vT",
        "outputId": "c1bcc64c-5192-474f-dbb3-0ba7d8516a2e"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([[1.0888],\n",
              "         [1.7506],\n",
              "         [1.8468],\n",
              "         [1.7496],\n",
              "         [1.9022],\n",
              "         [1.2684],\n",
              "         [0.8617]]), torch.Size([7, 1]))"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4. Set the random seed to `0` and do 2 & 3 over again.\n",
        "\n",
        "The output should be:\n",
        "```\n",
        "(tensor([[1.8542],\n",
        "         [1.9611],\n",
        "         [2.2884],\n",
        "         [3.0481],\n",
        "         [1.7067],\n",
        "         [2.5290],\n",
        "         [1.7989]]), torch.Size([7, 1]))\n",
        "```"
      ],
      "metadata": {
        "id": "eiutdKUFiryU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set manual seed\n",
        "torch.manual_seed(0)\n",
        "\n",
        "# Create two random tensors\n",
        "X = torch.rand(size=(7, 7))\n",
        "Y = torch.rand(size=(1, 7))\n",
        "\n",
        "# Matrix multiply tensors\n",
        "Z = torch.matmul(X, Y.T)\n",
        "Z, Z.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D-lOWI_1jRMm",
        "outputId": "7486a251-be91-4946-a31c-68e87a08ac86"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([[1.8542],\n",
              "         [1.9611],\n",
              "         [2.2884],\n",
              "         [3.0481],\n",
              "         [1.7067],\n",
              "         [2.5290],\n",
              "         [1.7989]]), torch.Size([7, 1]))"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 5. Speaking of random seeds, we saw how to set it with `torch.manual_seed()` but is there a GPU equivalent? (hint: you'll need to look into the documentation for `torch.cuda` for this one)\n",
        "  * If there is, set the GPU random seed to `1234`."
      ],
      "metadata": {
        "id": "ezY6ks9Cis37"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set random seed on the GPU\n",
        "torch.cuda.manual_seed(1234)"
      ],
      "metadata": {
        "id": "_LKWcfSTjp00"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "### 6. Create two random tensors of shape `(2, 3)` and send them both to the GPU (you'll need access to a GPU for this). Set `torch.manual_seed(1234)` when creating the tensors (this doesn't have to be the GPU random seed). The output should be something like:\n",
        "\n",
        "```\n",
        "Device: cuda\n",
        "(tensor([[0.0290, 0.4019, 0.2598],\n",
        "         [0.3666, 0.0583, 0.7006]], device='cuda:0'),\n",
        " tensor([[0.0518, 0.4681, 0.6738],\n",
        "         [0.3315, 0.7837, 0.5631]], device='cuda:0'))\n",
        "```"
      ],
      "metadata": {
        "id": "Ir9qSaj6it4n"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set random seed\n",
        "torch.manual_seed(1234)\n",
        "\n",
        "# Check for access to GPU\n",
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "print(f\"Device: {device}\")\n",
        "\n",
        "# Create two random tensors on GPU\n",
        "tensor_A = torch.rand(size=(2,3)).to(device)\n",
        "tensor_B = torch.rand(size=(2,3)).to(device)\n",
        "tensor_A, tensor_B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "azXExiFZj5nm",
        "outputId": "12d8b85e-efc4-4541-f309-b2d1528ade05"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Device: cuda\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([[0.0290, 0.4019, 0.2598],\n",
              "         [0.3666, 0.0583, 0.7006]], device='cuda:0'),\n",
              " tensor([[0.0518, 0.4681, 0.6738],\n",
              "         [0.3315, 0.7837, 0.5631]], device='cuda:0'))"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "### 7. Perform a matrix multiplication on the tensors you created in 6 (again, you may have to adjust the shapes of one of the tensors).\n",
        "\n",
        "The output should look like:\n",
        "```\n",
        "(tensor([[0.3647, 0.4709],\n",
        "         [0.5184, 0.5617]], device='cuda:0'), torch.Size([2, 2]))\n",
        "```"
      ],
      "metadata": {
        "id": "5TlAxeiSiu1y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Perform matmul on tensor_A and tensor_B\n",
        "# tensor_C = torch.matmul(tensor_A, tensor_B) # won't work because of shape error\n",
        "tensor_C = torch.matmul(tensor_A, tensor_B.T)\n",
        "tensor_C, tensor_C.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fAeG7ox0lHEO",
        "outputId": "dec0db78-f280-4160-e45f-e85c80a9241b"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([[0.3647, 0.4709],\n",
              "         [0.5184, 0.5617]], device='cuda:0'), torch.Size([2, 2]))"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 8. Find the maximum and minimum values of the output of 7."
      ],
      "metadata": {
        "id": "G7qfa5CSivwg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Find max\n",
        "max = torch.max(tensor_C)\n",
        "\n",
        "# Find min\n",
        "min = torch.min(tensor_C)\n",
        "max, min"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Fu8_3mZpllOd",
        "outputId": "3379333d-4661-411b-9426-fe587e1579f0"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0.5617, device='cuda:0'), tensor(0.3647, device='cuda:0'))"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 9. Find the maximum and minimum index values of the output of 7."
      ],
      "metadata": {
        "id": "wrTj5FgNiw47"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Find arg max\n",
        "arg_max = torch.argmax(tensor_C)\n",
        "\n",
        "# Find arg min\n",
        "arg_min = torch.argmin(tensor_C)\n",
        "arg_max, arg_min"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CCEKt4K2lsfQ",
        "outputId": "dc8049b1-5686-4157-dc63-d30b94100a12"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(3, device='cuda:0'), tensor(0, device='cuda:0'))"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "### 10. Make a random tensor with shape `(1, 1, 1, 10)` and then create a new tensor with all the `1` dimensions removed to be left with a tensor of shape `(10)`. Set the seed to `7` when you create it and print out the first tensor and it's shape as well as the second tensor and it's shape.\n",
        "\n",
        "The output should look like:\n",
        "\n",
        "```\n",
        "tensor([[[[0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297,\n",
        "           0.3653, 0.8513]]]]) torch.Size([1, 1, 1, 10])\n",
        "tensor([0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297, 0.3653,\n",
        "        0.8513]) torch.Size([10])\n",
        "```"
      ],
      "metadata": {
        "id": "hmeybz4uixy7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set seed\n",
        "torch.manual_seed(7)\n",
        "\n",
        "# Create random tensor\n",
        "tensor_D = torch.rand(size=(1, 1, 1, 10))\n",
        "\n",
        "# Remove single dimensions\n",
        "tensor_E = tensor_D.squeeze()\n",
        "\n",
        "# Print out tensors\n",
        "print(tensor_D, tensor_D.shape)\n",
        "print(tensor_E, tensor_E.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TQ9zbRzVl1jV",
        "outputId": "230bbe23-7662-464e-e485-d8aa9a5bad7f"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[[[0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297,\n",
            "           0.3653, 0.8513]]]]) torch.Size([1, 1, 1, 10])\n",
            "tensor([0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297, 0.3653,\n",
            "        0.8513]) torch.Size([10])\n"
          ]
        }
      ]
    }
  ]
}