chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,299 @@
|
||||
# Introduction to classification
|
||||
|
||||
In these four lessons, you will explore a fundamental focus of classic machine learning - _classification_. We will walk through using various classification algorithms with a dataset about all the brilliant cuisines of Asia and India. Hope you're hungry!
|
||||
|
||||
|
||||
|
||||
> Celebrate pan-Asian cuisines in these lessons! Image by [Jen Looper](https://twitter.com/jenlooper)
|
||||
|
||||
Classification is a form of [supervised learning](https://wikipedia.org/wiki/Supervised_learning) that bears a lot in common with regression techniques. If machine learning is all about predicting values or names to things by using datasets, then classification generally falls into two groups: _binary classification_ and _multiclass classification_.
|
||||
|
||||
[](https://youtu.be/eg8DJYwdMyg "Introduction to classification")
|
||||
|
||||
> 🎥 Click the image above for a video: MIT's John Guttag introduces classification
|
||||
|
||||
Remember:
|
||||
|
||||
- **Linear regression** helped you predict relationships between variables and make accurate predictions on where a new datapoint would fall in relationship to that line. So, you could predict _what price a pumpkin would be in September vs. December_, for example.
|
||||
- **Logistic regression** helped you discover "binary categories": at this price point, _is this pumpkin orange or not-orange_?
|
||||
|
||||
Classification uses various algorithms to determine other ways of determining a data point's label or class. Let's work with this cuisine data to see whether, by observing a group of ingredients, we can determine its cuisine of origin.
|
||||
|
||||
## [Pre-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
|
||||
> ### [This lesson is available in R!](./solution/R/lesson_10.html)
|
||||
|
||||
### Introduction
|
||||
|
||||
Classification is one of the fundamental activities of the machine learning researcher and data scientist. From basic classification of a binary value ("is this email spam or not?"), to complex image classification and segmentation using computer vision, it's always useful to be able to sort data into classes and ask questions of it.
|
||||
|
||||
To state the process in a more scientific way, your classification method creates a predictive model that enables you to map the relationship between input variables to output variables.
|
||||
|
||||
|
||||
|
||||
> Binary vs. multiclass problems for classification algorithms to handle. Infographic by [Jen Looper](https://twitter.com/jenlooper)
|
||||
|
||||
Before starting the process of cleaning our data, visualizing it, and prepping it for our ML tasks, let's learn a bit about the various ways machine learning can be leveraged to classify data.
|
||||
|
||||
Derived from [statistics](https://wikipedia.org/wiki/Statistical_classification), classification using classic machine learning uses features, such as `smoker`, `weight`, and `age` to determine _likelihood of developing X disease_. As a supervised learning technique similar to the regression exercises you performed earlier, your data is labeled and the ML algorithms use those labels to classify and predict classes (or 'features') of a dataset and assign them to a group or outcome.
|
||||
|
||||
✅ Take a moment to imagine a dataset about cuisines. What would a multiclass model be able to answer? What would a binary model be able to answer? What if you wanted to determine whether a given cuisine was likely to use fenugreek? What if you wanted to see if, given a present of a grocery bag full of star anise, artichokes, cauliflower, and horseradish, you could create a typical Indian dish?
|
||||
|
||||
[](https://youtu.be/GuTeDbaNoEU "Crazy mystery baskets")
|
||||
|
||||
> 🎥 Click the image above for a video.The whole premise of the show 'Chopped' is the 'mystery basket' where chefs have to make some dish out of a random choice of ingredients. Surely a ML model would have helped!
|
||||
|
||||
## Hello 'classifier'
|
||||
|
||||
The question we want to ask of this cuisine dataset is actually a **multiclass question**, as we have several potential national cuisines to work with. Given a batch of ingredients, which of these many classes will the data fit?
|
||||
|
||||
Scikit-learn offers several different algorithms to use to classify data, depending on the kind of problem you want to solve. In the next two lessons, you'll learn about several of these algorithms.
|
||||
|
||||
## Exercise - clean and balance your data
|
||||
|
||||
The first task at hand, before starting this project, is to clean and **balance** your data to get better results. Start with the blank _notebook.ipynb_ file in the root of this folder.
|
||||
|
||||
The first thing to install is [imblearn](https://imbalanced-learn.org/stable/). This is a Scikit-learn package that will allow you to better balance the data (you will learn more about this task in a minute).
|
||||
|
||||
1. To install `imblearn`, run `pip install`, like so:
|
||||
|
||||
```python
|
||||
pip install imblearn
|
||||
```
|
||||
|
||||
1. Import the packages you need to import your data and visualize it, also import `SMOTE` from `imblearn`.
|
||||
|
||||
```python
|
||||
import pandas as pd
|
||||
import matplotlib.pyplot as plt
|
||||
import matplotlib as mpl
|
||||
import numpy as np
|
||||
from imblearn.over_sampling import SMOTE
|
||||
```
|
||||
|
||||
Now you are set up to read import the data next.
|
||||
|
||||
1. The next task will be to import the data:
|
||||
|
||||
```python
|
||||
df = pd.read_csv('../data/cuisines.csv')
|
||||
```
|
||||
|
||||
Using `read_csv()` will read the content of the csv file _cusines.csv_ and place it in the variable `df`.
|
||||
|
||||
1. Check the data's shape:
|
||||
|
||||
```python
|
||||
df.head()
|
||||
```
|
||||
|
||||
The first five rows look like this:
|
||||
|
||||
```output
|
||||
| | Unnamed: 0 | cuisine | almond | angelica | anise | anise_seed | apple | apple_brandy | apricot | armagnac | ... | whiskey | white_bread | white_wine | whole_grain_wheat_flour | wine | wood | yam | yeast | yogurt | zucchini |
|
||||
| --- | ---------- | ------- | ------ | -------- | ----- | ---------- | ----- | ------------ | ------- | -------- | --- | ------- | ----------- | ---------- | ----------------------- | ---- | ---- | --- | ----- | ------ | -------- |
|
||||
| 0 | 65 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 1 | 66 | indian | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 2 | 67 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 3 | 68 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 4 | 69 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
|
||||
```
|
||||
|
||||
1. Get info about this data by calling `info()`:
|
||||
|
||||
```python
|
||||
df.info()
|
||||
```
|
||||
|
||||
Your out resembles:
|
||||
|
||||
```output
|
||||
<class 'pandas.core.frame.DataFrame'>
|
||||
RangeIndex: 2448 entries, 0 to 2447
|
||||
Columns: 385 entries, Unnamed: 0 to zucchini
|
||||
dtypes: int64(384), object(1)
|
||||
memory usage: 7.2+ MB
|
||||
```
|
||||
|
||||
## Exercise - learning about cuisines
|
||||
|
||||
Now the work starts to become more interesting. Let's discover the distribution of data, per cuisine
|
||||
|
||||
1. Plot the data as bars by calling `barh()`:
|
||||
|
||||
```python
|
||||
df.cuisine.value_counts().plot.barh()
|
||||
```
|
||||
|
||||
|
||||
|
||||
There are a finite number of cuisines, but the distribution of data is uneven. You can fix that! Before doing so, explore a little more.
|
||||
|
||||
1. Find out how much data is available per cuisine and print it out:
|
||||
|
||||
```python
|
||||
thai_df = df[(df.cuisine == "thai")]
|
||||
japanese_df = df[(df.cuisine == "japanese")]
|
||||
chinese_df = df[(df.cuisine == "chinese")]
|
||||
indian_df = df[(df.cuisine == "indian")]
|
||||
korean_df = df[(df.cuisine == "korean")]
|
||||
|
||||
print(f'thai df: {thai_df.shape}')
|
||||
print(f'japanese df: {japanese_df.shape}')
|
||||
print(f'chinese df: {chinese_df.shape}')
|
||||
print(f'indian df: {indian_df.shape}')
|
||||
print(f'korean df: {korean_df.shape}')
|
||||
```
|
||||
|
||||
the output looks like so:
|
||||
|
||||
```output
|
||||
thai df: (289, 385)
|
||||
japanese df: (320, 385)
|
||||
chinese df: (442, 385)
|
||||
indian df: (598, 385)
|
||||
korean df: (799, 385)
|
||||
```
|
||||
|
||||
## Discovering ingredients
|
||||
|
||||
Now you can dig deeper into the data and learn what are the typical ingredients per cuisine. You should clean out recurrent data that creates confusion between cuisines, so let's learn about this problem.
|
||||
|
||||
1. Create a function `create_ingredient()` in Python to create an ingredient dataframe. This function will start by dropping an unhelpful column and sort through ingredients by their count:
|
||||
|
||||
```python
|
||||
def create_ingredient_df(df):
|
||||
ingredient_df = df.T.drop(['cuisine','Unnamed: 0']).sum(axis=1).to_frame('value')
|
||||
ingredient_df = ingredient_df[(ingredient_df.T != 0).any()]
|
||||
ingredient_df = ingredient_df.sort_values(by='value', ascending=False,
|
||||
inplace=False)
|
||||
return ingredient_df
|
||||
```
|
||||
|
||||
Now you can use that function to get an idea of top ten most popular ingredients by cuisine.
|
||||
|
||||
1. Call `create_ingredient()` and plot it calling `barh()`:
|
||||
|
||||
```python
|
||||
thai_ingredient_df = create_ingredient_df(thai_df)
|
||||
thai_ingredient_df.head(10).plot.barh()
|
||||
```
|
||||
|
||||
|
||||
|
||||
1. Do the same for the japanese data:
|
||||
|
||||
```python
|
||||
japanese_ingredient_df = create_ingredient_df(japanese_df)
|
||||
japanese_ingredient_df.head(10).plot.barh()
|
||||
```
|
||||
|
||||
|
||||
|
||||
1. Now for the chinese ingredients:
|
||||
|
||||
```python
|
||||
chinese_ingredient_df = create_ingredient_df(chinese_df)
|
||||
chinese_ingredient_df.head(10).plot.barh()
|
||||
```
|
||||
|
||||
|
||||
|
||||
1. Plot the indian ingredients:
|
||||
|
||||
```python
|
||||
indian_ingredient_df = create_ingredient_df(indian_df)
|
||||
indian_ingredient_df.head(10).plot.barh()
|
||||
```
|
||||
|
||||
|
||||
|
||||
1. Finally, plot the korean ingredients:
|
||||
|
||||
```python
|
||||
korean_ingredient_df = create_ingredient_df(korean_df)
|
||||
korean_ingredient_df.head(10).plot.barh()
|
||||
```
|
||||
|
||||
|
||||
|
||||
1. Now, drop the most common ingredients that create confusion between distinct cuisines, by calling `drop()`:
|
||||
|
||||
Everyone loves rice, garlic and ginger!
|
||||
|
||||
```python
|
||||
feature_df= df.drop(['cuisine','Unnamed: 0','rice','garlic','ginger'], axis=1)
|
||||
labels_df = df.cuisine #.unique()
|
||||
feature_df.head()
|
||||
```
|
||||
|
||||
## Balance the dataset
|
||||
|
||||
Now that you have cleaned the data, use [SMOTE](https://imbalanced-learn.org/dev/references/generated/imblearn.over_sampling.SMOTE.html) - "Synthetic Minority Over-sampling Technique" - to balance it.
|
||||
|
||||
1. Call `fit_resample()`, this strategy generates new samples by interpolation.
|
||||
|
||||
```python
|
||||
oversample = SMOTE()
|
||||
transformed_feature_df, transformed_label_df = oversample.fit_resample(feature_df, labels_df)
|
||||
```
|
||||
|
||||
By balancing your data, you'll have better results when classifying it. Think about a binary classification. If most of your data is one class, a ML model is going to predict that class more frequently, just because there is more data for it. Balancing the data takes any skewed data and helps remove this imbalance.
|
||||
|
||||
1. Now you can check the numbers of labels per ingredient:
|
||||
|
||||
```python
|
||||
print(f'new label count: {transformed_label_df.value_counts()}')
|
||||
print(f'old label count: {df.cuisine.value_counts()}')
|
||||
```
|
||||
|
||||
Your output looks like so:
|
||||
|
||||
```output
|
||||
new label count: korean 799
|
||||
chinese 799
|
||||
indian 799
|
||||
japanese 799
|
||||
thai 799
|
||||
Name: cuisine, dtype: int64
|
||||
old label count: korean 799
|
||||
indian 598
|
||||
chinese 442
|
||||
japanese 320
|
||||
thai 289
|
||||
Name: cuisine, dtype: int64
|
||||
```
|
||||
|
||||
The data is nice and clean, balanced, and very delicious!
|
||||
|
||||
1. The last step is to save your balanced data, including labels and features, into a new dataframe that can be exported into a file:
|
||||
|
||||
```python
|
||||
transformed_df = pd.concat([transformed_label_df,transformed_feature_df],axis=1, join='outer')
|
||||
```
|
||||
|
||||
1. You can take one more look at the data using `transformed_df.head()` and `transformed_df.info()`. Save a copy of this data for use in future lessons:
|
||||
|
||||
```python
|
||||
transformed_df.head()
|
||||
transformed_df.info()
|
||||
transformed_df.to_csv("../data/cleaned_cuisines.csv")
|
||||
```
|
||||
|
||||
This fresh CSV can now be found in the root data folder.
|
||||
|
||||
---
|
||||
|
||||
## 🚀Challenge
|
||||
|
||||
This curriculum contains several interesting datasets. Dig through the `data` folders and see if any contain datasets that would be appropriate for binary or multi-class classification? What questions would you ask of this dataset?
|
||||
|
||||
## [Post-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
|
||||
## Review & Self Study
|
||||
|
||||
Explore SMOTE's API. What use cases is it best used for? What problems does it solve?
|
||||
|
||||
## Assignment
|
||||
|
||||
[Explore classification methods](assignment.md)
|
||||
@@ -0,0 +1,11 @@
|
||||
# Explore classification methods
|
||||
|
||||
## Instructions
|
||||
|
||||
In [Scikit-learn documentation](https://scikit-learn.org/stable/supervised_learning.html) you'll find a large list of ways to classify data. Do a little scavenger hunt in these docs: your goals is to look for classification methods and match a dataset in this curriculum, a question you can ask of it, and a technique of classification. Create a spreadsheet or table in a .doc file and explain how the dataset would work with the classification algorithm.
|
||||
|
||||
## Rubric
|
||||
|
||||
| Criteria | Exemplary | Adequate | Needs Improvement |
|
||||
| -------- | ----------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| | a document is presented overviewing 5 algorithms alongside a classification technique. The overview is well-explained and detailed. | a document is presented overviewing 3 algorithms alongside a classification technique. The overview is well-explained and detailed. | a document is presented overviewing fewer than three algorithms alongside a classification technique and the overview is neither well-explained nor detailed. |
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 240 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 8.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.8 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 624 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 8.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 7.6 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 8.9 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 393 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 6.3 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 7.7 KiB |
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": 3
|
||||
},
|
||||
"orig_nbformat": 2
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2,
|
||||
"cells": [
|
||||
{
|
||||
"source": [
|
||||
"# Delicious Asian and Indian Cuisines "
|
||||
],
|
||||
"cell_type": "markdown",
|
||||
"metadata": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
This is a temporary placeholder
|
||||
@@ -0,0 +1,721 @@
|
||||
{
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "lesson_10-R.ipynb",
|
||||
"provenance": [],
|
||||
"collapsed_sections": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"name": "ir",
|
||||
"display_name": "R"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "R"
|
||||
}
|
||||
},
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# Build a classification model: Delicious Asian and Indian Cuisines"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ItETB4tSFprR"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Introduction to classification: Clean, prep, and visualize your data\n",
|
||||
"\n",
|
||||
"In these four lessons, you will explore a fundamental focus of classic machine learning - *classification*. We will walk through using various classification algorithms with a dataset about all the brilliant cuisines of Asia and India. Hope you're hungry!\n",
|
||||
"\n",
|
||||
"<p >\n",
|
||||
" <img src=\"../../images/pinch.png\"\n",
|
||||
" width=\"600\"/>\n",
|
||||
" <figcaption>Celebrate pan-Asian cuisines in these lessons! Image by Jen Looper</figcaption>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!---->\n",
|
||||
"\n",
|
||||
"Classification is a form of [supervised learning](https://wikipedia.org/wiki/Supervised_learning) that bears a lot in common with regression techniques. In classification, you train a model to predict which `category` an item belongs to. If machine learning is all about predicting values or names to things by using datasets, then classification generally falls into two groups: *binary classification* and *multiclass classification*.\n",
|
||||
"\n",
|
||||
"Remember:\n",
|
||||
"\n",
|
||||
"- **Linear regression** helped you predict relationships between variables and make accurate predictions on where a new datapoint would fall in relationship to that line. So, you could predict a numeric values such as *what price a pumpkin would be in September vs. December*, for example.\n",
|
||||
"\n",
|
||||
"- **Logistic regression** helped you discover \"binary categories\": at this price point, *is this pumpkin orange or not-orange*?\n",
|
||||
"\n",
|
||||
"Classification uses various algorithms to determine other ways of determining a data point's label or class. Let's work with this cuisine data to see whether, by observing a group of ingredients, we can determine its cuisine of origin.\n",
|
||||
"\n",
|
||||
"### [**Pre-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/19/)\n",
|
||||
"\n",
|
||||
"### **Introduction**\n",
|
||||
"\n",
|
||||
"Classification is one of the fundamental activities of the machine learning researcher and data scientist. From basic classification of a binary value (\"is this email spam or not?\"), to complex image classification and segmentation using computer vision, it's always useful to be able to sort data into classes and ask questions of it.\n",
|
||||
"\n",
|
||||
"To state the process in a more scientific way, your classification method creates a predictive model that enables you to map the relationship between input variables to output variables.\n",
|
||||
"\n",
|
||||
"<p >\n",
|
||||
" <img src=\"../../images/binary-multiclass.png\"\n",
|
||||
" width=\"600\"/>\n",
|
||||
" <figcaption>Binary vs. multiclass problems for classification algorithms to handle. Infographic by Jen Looper</figcaption>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Before starting the process of cleaning our data, visualizing it, and prepping it for our ML tasks, let's learn a bit about the various ways machine learning can be leveraged to classify data.\n",
|
||||
"\n",
|
||||
"Derived from [statistics](https://wikipedia.org/wiki/Statistical_classification), classification using classic machine learning uses features, such as `smoker`, `weight`, and `age` to determine *likelihood of developing X disease*. As a supervised learning technique similar to the regression exercises you performed earlier, your data is labeled and the ML algorithms use those labels to classify and predict classes (or 'features') of a dataset and assign them to a group or outcome.\n",
|
||||
"\n",
|
||||
"✅ Take a moment to imagine a dataset about cuisines. What would a multiclass model be able to answer? What would a binary model be able to answer? What if you wanted to determine whether a given cuisine was likely to use fenugreek? What if you wanted to see if, given a present of a grocery bag full of star anise, artichokes, cauliflower, and horseradish, you could create a typical Indian dish?\n",
|
||||
"\n",
|
||||
"### **Hello 'classifier'**\n",
|
||||
"\n",
|
||||
"The question we want to ask of this cuisine dataset is actually a **multiclass question**, as we have several potential national cuisines to work with. Given a batch of ingredients, which of these many classes will the data fit?\n",
|
||||
"\n",
|
||||
"Tidymodels offers several different algorithms to use to classify data, depending on the kind of problem you want to solve. In the next two lessons, you'll learn about several of these algorithms.\n",
|
||||
"\n",
|
||||
"#### **Prerequisite**\n",
|
||||
"\n",
|
||||
"For this lesson, we'll require the following packages to clean, prep and visualize our data:\n",
|
||||
"\n",
|
||||
"- `tidyverse`: The [tidyverse](https://www.tidyverse.org/) is a [collection of R packages](https://www.tidyverse.org/packages) designed to makes data science faster, easier and more fun!\n",
|
||||
"\n",
|
||||
"- `tidymodels`: The [tidymodels](https://www.tidymodels.org/) framework is a [collection of packages](https://www.tidymodels.org/packages/) for modeling and machine learning.\n",
|
||||
"\n",
|
||||
"- `DataExplorer`: The [DataExplorer package](https://cran.r-project.org/web/packages/DataExplorer/vignettes/dataexplorer-intro.html) is meant to simplify and automate EDA process and report generation.\n",
|
||||
"\n",
|
||||
"- `themis`: The [themis package](https://themis.tidymodels.org/) provides Extra Recipes Steps for Dealing with Unbalanced Data.\n",
|
||||
"\n",
|
||||
"You can have them installed as:\n",
|
||||
"\n",
|
||||
"`install.packages(c(\"tidyverse\", \"tidymodels\", \"DataExplorer\", \"here\"))`\n",
|
||||
"\n",
|
||||
"Alternately, the script below checks whether you have the packages required to complete this module and installs them for you in case they are missing."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ri5bQxZ-Fz_0"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"suppressWarnings(if (!require(\"pacman\"))install.packages(\"pacman\"))\r\n",
|
||||
"\r\n",
|
||||
"pacman::p_load(tidyverse, tidymodels, DataExplorer, themis, here)"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "KIPxa4elGAPI"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We'll later load these awesome packages and make them available in our current R session. (This is for mere illustration, `pacman::p_load()` already did that for you)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "YkKAxOJvGD4C"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Exercise - clean and balance your data\n",
|
||||
"\n",
|
||||
"The first task at hand, before starting this project, is to clean and **balance** your data to get better results\n",
|
||||
"\n",
|
||||
"Let's meet the data!🕵️"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "PFkQDlk0GN5O"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Import data\r\n",
|
||||
"df <- read_csv(file = \"https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/4-Classification/data/cuisines.csv\")\r\n",
|
||||
"\r\n",
|
||||
"# View the first 5 rows\r\n",
|
||||
"df %>% \r\n",
|
||||
" slice_head(n = 5)\r\n"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "Qccw7okxGT0S"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Interesting! From the looks of it, the first column is a kind of `id` column. Let's get a little more information about the data."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "XrWnlgSrGVmR"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Basic information about the data\r\n",
|
||||
"df %>%\r\n",
|
||||
" introduce()\r\n",
|
||||
"\r\n",
|
||||
"# Visualize basic information above\r\n",
|
||||
"df %>% \r\n",
|
||||
" plot_intro(ggtheme = theme_light())"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "4UcGmxRxGieA"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"From the output, we can immediately see that we have `2448` rows and `385` columns and `0` missing values. We also have 1 discrete column, *cuisine*.\n",
|
||||
"\n",
|
||||
"## Exercise - learning about cuisines\n",
|
||||
"\n",
|
||||
"Now the work starts to become more interesting. Let's discover the distribution of data, per cuisine."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "AaPubl__GmH5"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Count observations per cuisine\r\n",
|
||||
"df %>% \r\n",
|
||||
" count(cuisine) %>% \r\n",
|
||||
" arrange(n)\r\n",
|
||||
"\r\n",
|
||||
"# Plot the distribution\r\n",
|
||||
"theme_set(theme_light())\r\n",
|
||||
"df %>% \r\n",
|
||||
" count(cuisine) %>% \r\n",
|
||||
" ggplot(mapping = aes(x = n, y = reorder(cuisine, -n))) +\r\n",
|
||||
" geom_col(fill = \"midnightblue\", alpha = 0.7) +\r\n",
|
||||
" ylab(\"cuisine\")"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "FRsBVy5eGrrv"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"There are a finite number of cuisines, but the distribution of data is uneven. You can fix that! Before doing so, explore a little more.\r\n",
|
||||
"\r\n",
|
||||
"Next, let's assign each cuisine into its individual tibble and find out how much data is available (rows, columns) per cuisine.\r\n",
|
||||
"\r\n",
|
||||
"> A [tibble](https://tibble.tidyverse.org/) is a modern data frame.\r\n",
|
||||
"\r\n",
|
||||
"<p >\r\n",
|
||||
" <img src=\"../../images/dplyr_filter.jpg\"\r\n",
|
||||
" width=\"600\"/>\r\n",
|
||||
" <figcaption>Artwork by @allison_horst</figcaption>\r\n",
|
||||
"\r\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "vVvyDb1kG2in"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Create individual tibble for the cuisines\r\n",
|
||||
"thai_df <- df %>% \r\n",
|
||||
" filter(cuisine == \"thai\")\r\n",
|
||||
"japanese_df <- df %>% \r\n",
|
||||
" filter(cuisine == \"japanese\")\r\n",
|
||||
"chinese_df <- df %>% \r\n",
|
||||
" filter(cuisine == \"chinese\")\r\n",
|
||||
"indian_df <- df %>% \r\n",
|
||||
" filter(cuisine == \"indian\")\r\n",
|
||||
"korean_df <- df %>% \r\n",
|
||||
" filter(cuisine == \"korean\")\r\n",
|
||||
"\r\n",
|
||||
"\r\n",
|
||||
"# Find out how much data is available per cuisine\r\n",
|
||||
"cat(\" thai df:\", dim(thai_df), \"\\n\",\r\n",
|
||||
" \"japanese df:\", dim(japanese_df), \"\\n\",\r\n",
|
||||
" \"chinese_df:\", dim(chinese_df), \"\\n\",\r\n",
|
||||
" \"indian_df:\", dim(indian_df), \"\\n\",\r\n",
|
||||
" \"korean_df:\", dim(korean_df))"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "0TvXUxD3G8Bk"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Perfect!😋\n",
|
||||
"\n",
|
||||
"## **Exercise - Discovering top ingredients by cuisine using dplyr**\n",
|
||||
"\n",
|
||||
"Now you can dig deeper into the data and learn what are the typical ingredients per cuisine. You should clean out recurrent data that creates confusion between cuisines, so let's learn about this problem.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Create a function `create_ingredient()` in R that returns an ingredient dataframe. This function will start by dropping an unhelpful column and sort through ingredients by their count.\n",
|
||||
"\n",
|
||||
"The basic structure of a function in R is:\n",
|
||||
"\n",
|
||||
"`myFunction <- function(arglist){`\n",
|
||||
"\n",
|
||||
"**`...`**\n",
|
||||
"\n",
|
||||
"**`return`**`(value)`\n",
|
||||
"\n",
|
||||
"`}`\n",
|
||||
"\n",
|
||||
"A tidy introduction to R functions can be found [here](https://skirmer.github.io/presentations/functions_with_r.html#1).\n",
|
||||
"\n",
|
||||
"Let's get right into it! We'll make use of [dplyr verbs](https://dplyr.tidyverse.org/) which we have been learning in our previous lessons. As a recap:\n",
|
||||
"\n",
|
||||
"- `dplyr::select()`: help you pick which **columns** to keep or exclude.\n",
|
||||
"\n",
|
||||
"- `dplyr::pivot_longer()`: helps you to \"lengthen\" data, increasing the number of rows and decreasing the number of columns.\n",
|
||||
"\n",
|
||||
"- `dplyr::group_by()` and `dplyr::summarise()`: helps you to find find summary statistics for different groups, and put them in a nice table.\n",
|
||||
"\n",
|
||||
"- `dplyr::filter()`: creates a subset of the data only containing rows that satisfy your conditions.\n",
|
||||
"\n",
|
||||
"- `dplyr::mutate()`: helps you to create or modify columns.\n",
|
||||
"\n",
|
||||
"Check out this [*art*-filled learnr tutorial](https://allisonhorst.shinyapps.io/dplyr-learnr/#section-welcome) by Allison Horst, that introduces some useful data wrangling functions in dplyr *(part of the Tidyverse)*"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "K3RF5bSCHC76"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Creates a functions that returns the top ingredients by class\r\n",
|
||||
"\r\n",
|
||||
"create_ingredient <- function(df){\r\n",
|
||||
" \r\n",
|
||||
" # Drop the id column which is the first colum\r\n",
|
||||
" ingredient_df = df %>% select(-1) %>% \r\n",
|
||||
" # Transpose data to a long format\r\n",
|
||||
" pivot_longer(!cuisine, names_to = \"ingredients\", values_to = \"count\") %>% \r\n",
|
||||
" # Find the top most ingredients for a particular cuisine\r\n",
|
||||
" group_by(ingredients) %>% \r\n",
|
||||
" summarise(n_instances = sum(count)) %>% \r\n",
|
||||
" filter(n_instances != 0) %>% \r\n",
|
||||
" # Arrange by descending order\r\n",
|
||||
" arrange(desc(n_instances)) %>% \r\n",
|
||||
" mutate(ingredients = factor(ingredients) %>% fct_inorder())\r\n",
|
||||
" \r\n",
|
||||
" \r\n",
|
||||
" return(ingredient_df)\r\n",
|
||||
"} # End of function"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "uB_0JR82HTPa"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Now we can use the function to get an idea of top ten most popular ingredient by cuisine. Let's take it out for a spin with `thai_df`\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "h9794WF8HWmc"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Call create_ingredient and display popular ingredients\r\n",
|
||||
"thai_ingredient_df <- create_ingredient(df = thai_df)\r\n",
|
||||
"\r\n",
|
||||
"thai_ingredient_df %>% \r\n",
|
||||
" slice_head(n = 10)"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "agQ-1HrcHaEA"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"In the previous section, we used `geom_col()`, let's see how you can use `geom_bar` too, to create bar charts. Use `?geom_bar` for further reading."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "kHu9ffGjHdcX"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Make a bar chart for popular thai cuisines\r\n",
|
||||
"thai_ingredient_df %>% \r\n",
|
||||
" slice_head(n = 10) %>% \r\n",
|
||||
" ggplot(aes(x = n_instances, y = ingredients)) +\r\n",
|
||||
" geom_bar(stat = \"identity\", width = 0.5, fill = \"steelblue\") +\r\n",
|
||||
" xlab(\"\") + ylab(\"\")"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "fb3Bx_3DHj6e"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Let's do the same for the Japanese data"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "RHP_xgdkHnvM"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Get popular ingredients for Japanese cuisines and make bar chart\r\n",
|
||||
"create_ingredient(df = japanese_df) %>% \r\n",
|
||||
" slice_head(n = 10) %>%\r\n",
|
||||
" ggplot(aes(x = n_instances, y = ingredients)) +\r\n",
|
||||
" geom_bar(stat = \"identity\", width = 0.5, fill = \"darkorange\", alpha = 0.8) +\r\n",
|
||||
" xlab(\"\") + ylab(\"\")\r\n"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "019v8F0XHrRU"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"What about the Chinese cuisines?\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "iIGM7vO8Hu3v"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Get popular ingredients for Chinese cuisines and make bar chart\r\n",
|
||||
"create_ingredient(df = chinese_df) %>% \r\n",
|
||||
" slice_head(n = 10) %>%\r\n",
|
||||
" ggplot(aes(x = n_instances, y = ingredients)) +\r\n",
|
||||
" geom_bar(stat = \"identity\", width = 0.5, fill = \"cyan4\", alpha = 0.8) +\r\n",
|
||||
" xlab(\"\") + ylab(\"\")"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "lHd9_gd2HyzU"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Let's take a look at the Indian cuisines 🌶️."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ir8qyQbNH1c7"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Get popular ingredients for Indian cuisines and make bar chart\r\n",
|
||||
"create_ingredient(df = indian_df) %>% \r\n",
|
||||
" slice_head(n = 10) %>%\r\n",
|
||||
" ggplot(aes(x = n_instances, y = ingredients)) +\r\n",
|
||||
" geom_bar(stat = \"identity\", width = 0.5, fill = \"#041E42FF\", alpha = 0.8) +\r\n",
|
||||
" xlab(\"\") + ylab(\"\")"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "ApukQtKjH5FO"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Finally, plot the Korean ingredients."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "qv30cwY1H-FM"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Get popular ingredients for Korean cuisines and make bar chart\r\n",
|
||||
"create_ingredient(df = korean_df) %>% \r\n",
|
||||
" slice_head(n = 10) %>%\r\n",
|
||||
" ggplot(aes(x = n_instances, y = ingredients)) +\r\n",
|
||||
" geom_bar(stat = \"identity\", width = 0.5, fill = \"#852419FF\", alpha = 0.8) +\r\n",
|
||||
" xlab(\"\") + ylab(\"\")"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "lumgk9cHIBie"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"From the data visualizations, we can now drop the most common ingredients that create confusion between distinct cuisines, using `dplyr::select()`.\n",
|
||||
"\n",
|
||||
"Everyone loves rice, garlic and ginger!"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "iO4veMXuIEta"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Drop id column, rice, garlic and ginger from our original data set\r\n",
|
||||
"df_select <- df %>% \r\n",
|
||||
" select(-c(1, rice, garlic, ginger))\r\n",
|
||||
"\r\n",
|
||||
"# Display new data set\r\n",
|
||||
"df_select %>% \r\n",
|
||||
" slice_head(n = 5)"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "iHJPiG6rIUcK"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Preprocessing data using recipes 👩🍳👨🍳 - Dealing with imbalanced data ⚖️\r\n",
|
||||
"\r\n",
|
||||
"<p >\r\n",
|
||||
" <img src=\"../../images/recipes.png\"\r\n",
|
||||
" width=\"600\"/>\r\n",
|
||||
" <figcaption>Artwork by @allison_horst</figcaption>\r\n",
|
||||
"\r\n",
|
||||
"Given that this lesson is about cuisines, we have to put `recipes` into context .\r\n",
|
||||
"\r\n",
|
||||
"Tidymodels provides yet another neat package: `recipes`- a package for preprocessing data.\r\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "kkFd-JxdIaL6"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Let's take a look at the distribution of our cuisines again.\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "6l2ubtTPJAhY"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Distribution of cuisines\r\n",
|
||||
"old_label_count <- df_select %>% \r\n",
|
||||
" count(cuisine) %>% \r\n",
|
||||
" arrange(desc(n))\r\n",
|
||||
"\r\n",
|
||||
"old_label_count"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "1e-E9cb7JDVi"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"\n",
|
||||
"As you can see, there is quite an unequal distribution in the number of cuisines. Korean cuisines are almost 3 times Thai cuisines. Imbalanced data often has negative effects on the model performance. Think about a binary classification. If most of your data is one class, a ML model is going to predict that class more frequently, just because there is more data for it. Balancing the data takes any skewed data and helps remove this imbalance. Many models perform best when the number of observations is equal and, thus, tend to struggle with unbalanced data.\n",
|
||||
"\n",
|
||||
"There are majorly two ways of dealing with imbalanced data sets:\n",
|
||||
"\n",
|
||||
"- adding observations to the minority class: `Over-sampling` e.g using a SMOTE algorithm\n",
|
||||
"\n",
|
||||
"- removing observations from majority class: `Under-sampling`\n",
|
||||
"\n",
|
||||
"Let's now demonstrate how to deal with imbalanced data sets using a `recipe`. A recipe can be thought of as a blueprint that describes what steps should be applied to a data set in order to get it ready for data analysis."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "soAw6826JKx9"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Load themis package for dealing with imbalanced data\r\n",
|
||||
"library(themis)\r\n",
|
||||
"\r\n",
|
||||
"# Create a recipe for preprocessing data\r\n",
|
||||
"cuisines_recipe <- recipe(cuisine ~ ., data = df_select) %>% \r\n",
|
||||
" step_smote(cuisine)\r\n",
|
||||
"\r\n",
|
||||
"cuisines_recipe"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "HS41brUIJVJy"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Let's break down our preprocessing steps.\n",
|
||||
"\n",
|
||||
"- The call to `recipe()` with a formula tells the recipe the *roles* of the variables using `df_select` data as the reference. For instance the `cuisine` column has been assigned an `outcome` role while the rest of the columns have been assigned a `predictor` role.\n",
|
||||
"\n",
|
||||
"- [`step_smote(cuisine)`](https://themis.tidymodels.org/reference/step_smote.html) creates a *specification* of a recipe step that synthetically generates new examples of the minority class using nearest neighbors of these cases.\n",
|
||||
"\n",
|
||||
"Now, if we wanted to see the preprocessed data, we'd have to [**`prep()`**](https://recipes.tidymodels.org/reference/prep.html) and [**`bake()`**](https://recipes.tidymodels.org/reference/bake.html) our recipe.\n",
|
||||
"\n",
|
||||
"`prep()`: estimates the required parameters from a training set that can be later applied to other data sets.\n",
|
||||
"\n",
|
||||
"`bake()`: takes a prepped recipe and applies the operations to any data set.\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "Yb-7t7XcJaC8"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Prep and bake the recipe\r\n",
|
||||
"preprocessed_df <- cuisines_recipe %>% \r\n",
|
||||
" prep() %>% \r\n",
|
||||
" bake(new_data = NULL) %>% \r\n",
|
||||
" relocate(cuisine)\r\n",
|
||||
"\r\n",
|
||||
"# Display data\r\n",
|
||||
"preprocessed_df %>% \r\n",
|
||||
" slice_head(n = 5)\r\n",
|
||||
"\r\n",
|
||||
"# Quick summary stats\r\n",
|
||||
"preprocessed_df %>% \r\n",
|
||||
" introduce()"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "9QhSgdpxJl44"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Let's now check the distribution of our cuisines and compare them with the imbalanced data."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "dmidELh_LdV7"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Distribution of cuisines\r\n",
|
||||
"new_label_count <- preprocessed_df %>% \r\n",
|
||||
" count(cuisine) %>% \r\n",
|
||||
" arrange(desc(n))\r\n",
|
||||
"\r\n",
|
||||
"list(new_label_count = new_label_count,\r\n",
|
||||
" old_label_count = old_label_count)"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "aSh23klBLwDz"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Yum! The data is nice and clean, balanced, and very delicious 😋!\n",
|
||||
"\n",
|
||||
"> Normally, a recipe is usually used as a preprocessor for modelling where it defines what steps should be applied to a data set in order to get it ready for modelling. In that case, a `workflow()` is typically used (as we have already seen in our previous lessons) instead of manually estimating a recipe\n",
|
||||
">\n",
|
||||
"> As such, you don't typically need to **`prep()`** and **`bake()`** recipes when you use tidymodels, but they are helpful functions to have in your toolkit for confirming that recipes are doing what you expect like in our case.\n",
|
||||
">\n",
|
||||
"> When you **`bake()`** a prepped recipe with **`new_data = NULL`**, you get the data that you provided when defining the recipe back, but having undergone the preprocessing steps.\n",
|
||||
"\n",
|
||||
"Let's now save a copy of this data for use in future lessons:\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "HEu80HZ8L7ae"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"source": [
|
||||
"# Save preprocessed data\r\n",
|
||||
"write_csv(preprocessed_df, \"../../../data/cleaned_cuisines_R.csv\")"
|
||||
],
|
||||
"outputs": [],
|
||||
"metadata": {
|
||||
"id": "cBmCbIgrMOI6"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"This fresh CSV can now be found in the root data folder.\r\n",
|
||||
"\r\n",
|
||||
"**🚀Challenge**\r\n",
|
||||
"\r\n",
|
||||
"This curriculum contains several interesting datasets. Dig through the `data` folders and see if any contain datasets that would be appropriate for binary or multi-class classification? What questions would you ask of this dataset?\r\n",
|
||||
"\r\n",
|
||||
"## [**Post-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/20/)\r\n",
|
||||
"\r\n",
|
||||
"## **Review & Self Study**\r\n",
|
||||
"\r\n",
|
||||
"- Check out [package themis](https://github.com/tidymodels/themis). What other techniques could we use to deal with imbalanced data?\r\n",
|
||||
"\r\n",
|
||||
"- Tidy models [reference website](https://www.tidymodels.org/start/).\r\n",
|
||||
"\r\n",
|
||||
"- H. Wickham and G. Grolemund, [*R for Data Science: Visualize, Model, Transform, Tidy, and Import Data*](https://r4ds.had.co.nz/).\r\n",
|
||||
"\r\n",
|
||||
"#### THANK YOU TO:\r\n",
|
||||
"\r\n",
|
||||
"[`Allison Horst`](https://twitter.com/allison_horst/) for creating the amazing illustrations that make R more welcoming and engaging. Find more illustrations at her [gallery](https://www.google.com/url?q=https://github.com/allisonhorst/stats-illustrations&sa=D&source=editors&ust=1626380772530000&usg=AOvVaw3zcfyCizFQZpkSLzxiiQEM).\r\n",
|
||||
"\r\n",
|
||||
"[Cassie Breviu](https://www.twitter.com/cassieview) and [Jen Looper](https://www.twitter.com/jenlooper) for creating the original Python version of this module ♥️\r\n",
|
||||
"\r\n",
|
||||
"<p >\r\n",
|
||||
" <img src=\"../../images/r_learners_sm.jpeg\"\r\n",
|
||||
" width=\"600\"/>\r\n",
|
||||
" <figcaption>Artwork by @allison_horst</figcaption>\r\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "WQs5621pMGwf"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,422 @@
|
||||
---
|
||||
title: 'Build a classification model: Delicious Asian and Indian Cuisines'
|
||||
output:
|
||||
html_document:
|
||||
df_print: paged
|
||||
theme: flatly
|
||||
highlight: breezedark
|
||||
toc: yes
|
||||
toc_float: yes
|
||||
code_download: yes
|
||||
---
|
||||
|
||||
## Introduction to classification: Clean, prep, and visualize your data
|
||||
|
||||
In these four lessons, you will explore a fundamental focus of classic machine learning - *classification*. We will walk through using various classification algorithms with a dataset about all the brilliant cuisines of Asia and India. Hope you're hungry!
|
||||
|
||||
|
||||
|
||||
Classification is a form of [supervised learning](https://wikipedia.org/wiki/Supervised_learning) that bears a lot in common with regression techniques. In classification, you train a model to predict which `category` an item belongs to. If machine learning is all about predicting values or names to things by using datasets, then classification generally falls into two groups: *binary classification* and *multiclass classification*.
|
||||
|
||||
Remember:
|
||||
|
||||
- **Linear regression** helped you predict relationships between variables and make accurate predictions on where a new datapoint would fall in relationship to that line. So, you could predict a numeric values such as *what price a pumpkin would be in September vs. December*, for example.
|
||||
|
||||
- **Logistic regression** helped you discover "binary categories": at this price point, *is this pumpkin orange or not-orange*?
|
||||
|
||||
Classification uses various algorithms to determine other ways of determining a data point's label or class. Let's work with this cuisine data to see whether, by observing a group of ingredients, we can determine its cuisine of origin.
|
||||
|
||||
### [**Pre-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/19/)
|
||||
|
||||
### **Introduction**
|
||||
|
||||
Classification is one of the fundamental activities of the machine learning researcher and data scientist. From basic classification of a binary value ("is this email spam or not?"), to complex image classification and segmentation using computer vision, it's always useful to be able to sort data into classes and ask questions of it.
|
||||
|
||||
To state the process in a more scientific way, your classification method creates a predictive model that enables you to map the relationship between input variables to output variables.
|
||||
|
||||
{width="500"}
|
||||
|
||||
Before starting the process of cleaning our data, visualizing it, and prepping it for our ML tasks, let's learn a bit about the various ways machine learning can be leveraged to classify data.
|
||||
|
||||
Derived from [statistics](https://wikipedia.org/wiki/Statistical_classification), classification using classic machine learning uses features, such as `smoker`, `weight`, and `age` to determine *likelihood of developing X disease*. As a supervised learning technique similar to the regression exercises you performed earlier, your data is labeled and the ML algorithms use those labels to classify and predict classes (or 'features') of a dataset and assign them to a group or outcome.
|
||||
|
||||
✅ Take a moment to imagine a dataset about cuisines. What would a multiclass model be able to answer? What would a binary model be able to answer? What if you wanted to determine whether a given cuisine was likely to use fenugreek? What if you wanted to see if, given a present of a grocery bag full of star anise, artichokes, cauliflower, and horseradish, you could create a typical Indian dish?
|
||||
|
||||
### **Hello 'classifier'**
|
||||
|
||||
The question we want to ask of this cuisine dataset is actually a **multiclass question**, as we have several potential national cuisines to work with. Given a batch of ingredients, which of these many classes will the data fit?
|
||||
|
||||
Tidymodels offers several different algorithms to use to classify data, depending on the kind of problem you want to solve. In the next two lessons, you'll learn about several of these algorithms.
|
||||
|
||||
#### **Prerequisite**
|
||||
|
||||
For this lesson, we'll require the following packages to clean, prep and visualize our data:
|
||||
|
||||
- `tidyverse`: The [tidyverse](https://www.tidyverse.org/) is a [collection of R packages](https://www.tidyverse.org/packages) designed to makes data science faster, easier and more fun!
|
||||
|
||||
- `tidymodels`: The [tidymodels](https://www.tidymodels.org/) framework is a [collection of packages](https://www.tidymodels.org/packages/) for modeling and machine learning.
|
||||
|
||||
- `DataExplorer`: The [DataExplorer package](https://cran.r-project.org/web/packages/DataExplorer/vignettes/dataexplorer-intro.html) is meant to simplify and automate EDA process and report generation.
|
||||
|
||||
- `themis`: The [themis package](https://themis.tidymodels.org/) provides Extra Recipes Steps for Dealing with Unbalanced Data.
|
||||
|
||||
You can have them installed as:
|
||||
|
||||
`install.packages(c("tidyverse", "tidymodels", "DataExplorer", "here"))`
|
||||
|
||||
Alternately, the script below checks whether you have the packages required to complete this module and installs them for you in case they are missing.
|
||||
|
||||
```{r, message=F, warning=F}
|
||||
suppressWarnings(if (!require("pacman"))install.packages("pacman"))
|
||||
|
||||
pacman::p_load(tidyverse, tidymodels, DataExplorer, themis, here)
|
||||
```
|
||||
|
||||
We'll later load these awesome packages and make them available in our current R session. (This is for mere illustration, `pacman::p_load()` already did that for you)
|
||||
|
||||
## Exercise - clean and balance your data
|
||||
|
||||
The first task at hand, before starting this project, is to clean and **balance** your data to get better results
|
||||
|
||||
Let's meet the data!🕵️
|
||||
|
||||
```{r import_data}
|
||||
# Import data
|
||||
df <- read_csv(file = "https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/4-Classification/data/cuisines.csv")
|
||||
|
||||
# View the first 5 rows
|
||||
df %>%
|
||||
slice_head(n = 5)
|
||||
|
||||
```
|
||||
|
||||
Interesting! From the looks of it, the first column is a kind of `id` column. Let's get a little more information about the data.
|
||||
|
||||
```{r info}
|
||||
# Basic information about the data
|
||||
df %>%
|
||||
introduce()
|
||||
|
||||
# Visualize basic information above
|
||||
df %>%
|
||||
plot_intro(ggtheme = theme_light())
|
||||
```
|
||||
|
||||
From the output, we can immediately see that we have `2448` rows and `385` columns and `0` missing values. We also have 1 discrete column, *cuisine*.
|
||||
|
||||
## Exercise - learning about cuisines
|
||||
|
||||
1. Now the work starts to become more interesting. Let's discover the distribution of data, per cuisine.
|
||||
|
||||
```{r filter_cuisine}
|
||||
# Count observations per cuisine
|
||||
df %>%
|
||||
count(cuisine) %>%
|
||||
arrange(n)
|
||||
|
||||
# Plot the distribution
|
||||
theme_set(theme_light())
|
||||
df %>%
|
||||
count(cuisine) %>%
|
||||
ggplot(mapping = aes(x = n, y = reorder(cuisine, -n))) +
|
||||
geom_col(fill = "midnightblue", alpha = 0.7) +
|
||||
ylab("cuisine")
|
||||
```
|
||||
|
||||
There are a finite number of cuisines, but the distribution of data is uneven. You can fix that! Before doing so, explore a little more.
|
||||
|
||||
2. Next, let's assign each cuisine into it's individual tibble and find out how much data is available (rows, columns) per cuisine.
|
||||
|
||||
> A tibble, or tbl_df, is a modern reimagining of the data.frame, keeping what time has proven to be effective, and throwing out what is not.
|
||||
|
||||
|
||||
|
||||
```{r cuisine_df}
|
||||
# Create individual tibbles for the cuisines
|
||||
thai_df <- df %>%
|
||||
filter(cuisine == "thai")
|
||||
japanese_df <- df %>%
|
||||
filter(cuisine == "japanese")
|
||||
chinese_df <- df %>%
|
||||
filter(cuisine == "chinese")
|
||||
indian_df <- df %>%
|
||||
filter(cuisine == "indian")
|
||||
korean_df <- df %>%
|
||||
filter(cuisine == "korean")
|
||||
|
||||
|
||||
# Find out how much data is available per cuisine
|
||||
cat(" thai df:", dim(thai_df), "\n",
|
||||
"japanese df:", dim(japanese_df), "\n",
|
||||
"chinese_df:", dim(chinese_df), "\n",
|
||||
"indian_df:", dim(indian_df), "\n",
|
||||
"korean_df:", dim(korean_df))
|
||||
```
|
||||
|
||||
Perfect!😋
|
||||
|
||||
## **Exercise - Discovering top ingredients by cuisine using dplyr**
|
||||
|
||||
Now you can dig deeper into the data and learn what are the typical ingredients per cuisine. You should clean out recurrent data that creates confusion between cuisines, so let's learn about this problem.
|
||||
|
||||
1. Create a function `create_ingredient()` in R that returns an ingredient dataframe. This function will start by dropping an unhelpful column and sort through ingredients by their count.
|
||||
|
||||
The basic structure of a function in R is:
|
||||
|
||||
`myFunction <- function(arglist){`
|
||||
|
||||
**`...`**
|
||||
|
||||
**`return`**`(value)`
|
||||
|
||||
`}`
|
||||
|
||||
A tidy introduction to R functions can be found [here](https://skirmer.github.io/presentations/functions_with_r.html#1).
|
||||
|
||||
Let's get right into it! We'll make use of [dplyr verbs](https://dplyr.tidyverse.org/) which we have been learning in our previous lessons. As a recap:
|
||||
|
||||
- `dplyr::select()`: help you pick which **columns** to keep or exclude.
|
||||
|
||||
- `dplyr::pivot_longer()`: helps you to "lengthen" data, increasing the number of rows and decreasing the number of columns.
|
||||
|
||||
- `dplyr::group_by()` and `dplyr::summarise()`: helps you to find find summary statistics for different groups, and put them in a nice table.
|
||||
|
||||
- `dplyr::filter()`: creates a subset of the data only containing rows that satisfy your conditions.
|
||||
|
||||
- `dplyr::mutate()`: helps you to create or modify columns.
|
||||
|
||||
Check out this [*art*-filled learnr tutorial](https://allisonhorst.shinyapps.io/dplyr-learnr/#section-welcome) by Allison Horst, that introduces some useful data wrangling functions in dplyr *(part of the Tidyverse)*
|
||||
|
||||
```{r create_ingredient}
|
||||
# Creates a functions that returns the top ingredients by class
|
||||
|
||||
create_ingredient <- function(df){
|
||||
|
||||
# Drop the id column which is the first colum
|
||||
ingredient_df = df %>% select(-1) %>%
|
||||
# Transpose data to a long format
|
||||
pivot_longer(!cuisine, names_to = "ingredients", values_to = "count") %>%
|
||||
# Find the top most ingredients for a particular cuisine
|
||||
group_by(ingredients) %>%
|
||||
summarise(n_instances = sum(count)) %>%
|
||||
filter(n_instances != 0) %>%
|
||||
# Arrange by descending order
|
||||
arrange(desc(n_instances)) %>%
|
||||
mutate(ingredients = factor(ingredients) %>% fct_inorder())
|
||||
|
||||
|
||||
return(ingredient_df)
|
||||
} # End of function
|
||||
|
||||
```
|
||||
|
||||
2. Now we can use the function to get an idea of top ten most popular ingredient by cuisine. Let's take it out for a spin with `thai_df`
|
||||
|
||||
```{r thai_ingredient_df}
|
||||
# Call create_ingredient and display popular ingredients
|
||||
thai_ingredient_df <- create_ingredient(df = thai_df)
|
||||
|
||||
thai_ingredient_df %>%
|
||||
slice_head(n = 10)
|
||||
|
||||
```
|
||||
|
||||
In the previous section, we used `geom_col()`, let's see how you can use `geom_bar` too, to create bar charts. Use `?geom_bar` for further reading.
|
||||
|
||||
```{r thai_chart}
|
||||
# Make a bar chart for popular thai cuisines
|
||||
thai_ingredient_df %>%
|
||||
slice_head(n = 10) %>%
|
||||
ggplot(aes(x = n_instances, y = ingredients)) +
|
||||
geom_bar(stat = "identity", width = 0.5, fill = "steelblue") +
|
||||
xlab("") + ylab("")
|
||||
|
||||
```
|
||||
|
||||
3. Let's do the same for the Japanese data
|
||||
|
||||
```{r japanese_ingredient_df}
|
||||
# Get popular ingredients for Japanese cuisines and make bar chart
|
||||
create_ingredient(df = japanese_df) %>%
|
||||
slice_head(n = 10) %>%
|
||||
ggplot(aes(x = n_instances, y = ingredients)) +
|
||||
geom_bar(stat = "identity", width = 0.5, fill = "darkorange", alpha = 0.8) +
|
||||
xlab("") + ylab("")
|
||||
|
||||
|
||||
|
||||
```
|
||||
|
||||
4. What about the Chinese cuisines?
|
||||
|
||||
```{r chinese_ingredient_df}
|
||||
# Get popular ingredients for Chinese cuisines and make bar chart
|
||||
create_ingredient(df = chinese_df) %>%
|
||||
slice_head(n = 10) %>%
|
||||
ggplot(aes(x = n_instances, y = ingredients)) +
|
||||
geom_bar(stat = "identity", width = 0.5, fill = "cyan4", alpha = 0.8) +
|
||||
xlab("") + ylab("")
|
||||
|
||||
|
||||
```
|
||||
|
||||
5. Let's take a look at the Indian cuisines 🌶️.
|
||||
|
||||
```{r indian_ingredient_df }
|
||||
# Get popular ingredients for Indian cuisines and make bar chart
|
||||
create_ingredient(df = indian_df) %>%
|
||||
slice_head(n = 10) %>%
|
||||
ggplot(aes(x = n_instances, y = ingredients)) +
|
||||
geom_bar(stat = "identity", width = 0.5, fill = "#041E42FF", alpha = 0.8) +
|
||||
xlab("") + ylab("")
|
||||
```
|
||||
|
||||
6. Finally, plot the Korean ingredients.
|
||||
|
||||
```{r korean_ingredient_df }
|
||||
# Get popular ingredients for Korean cuisines and make bar chart
|
||||
create_ingredient(df = korean_df) %>%
|
||||
slice_head(n = 10) %>%
|
||||
ggplot(aes(x = n_instances, y = ingredients)) +
|
||||
geom_bar(stat = "identity", width = 0.5, fill = "#852419FF", alpha = 0.8) +
|
||||
xlab("") + ylab("")
|
||||
```
|
||||
|
||||
7. From the data visualizations, we can now drop the most common ingredients that create confusion between distinct cuisines, using `dplyr::select()`.
|
||||
|
||||
Everyone loves rice, garlic and ginger!
|
||||
|
||||
```{r df_select}
|
||||
# Drop rice, garlic and ginger from our original data set
|
||||
df_select <- df %>%
|
||||
select(-c(1, rice, garlic, ginger))
|
||||
|
||||
# Display new data set
|
||||
df_select %>%
|
||||
slice_head(n = 5)
|
||||
|
||||
```
|
||||
|
||||
## Preprocessing data using recipes 👩🍳👨🍳 - Dealing with imbalanced data ⚖️
|
||||
|
||||
|
||||
|
||||
Given that this lesson is about cuisines, we have to put `recipes` into context .
|
||||
|
||||
Tidymodels provides yet another neat package: `recipes`- a package for preprocessing data.
|
||||
|
||||
Now we are on the same page 😅.
|
||||
|
||||
Let's take a look at the distribution of our cuisines again.
|
||||
|
||||
```{r df_select_n}
|
||||
# Distribution of cuisines
|
||||
old_label_count <- df_select %>%
|
||||
count(cuisine) %>%
|
||||
arrange(desc(n))
|
||||
|
||||
old_label_count
|
||||
```
|
||||
|
||||
As you can see, there is quite an unequal distribution in the number of cuisines. Korean cuisines are almost 3 times Thai cuisines. Imbalanced data often has negative effects on the model performance. Think about a binary classification. If most of your data is one class, a ML model is going to predict that class more frequently, just because there is more data for it. Balancing the data takes any skewed data and helps remove this imbalance. Many models perform best when the number of observations is equal and, thus, tend to struggle with unbalanced data.
|
||||
|
||||
There are majorly two ways of dealing with imbalanced data sets:
|
||||
|
||||
- adding observations to the minority class: `Over-sampling` e.g using a SMOTE algorithm
|
||||
|
||||
- removing observations from majority class: `Under-sampling`
|
||||
|
||||
Let's now demonstrate how to deal with imbalanced data sets using a `recipe`. A recipe can be thought of as a blueprint that describes what steps should be applied to a data set in order to get it ready for data analysis.
|
||||
|
||||
```{r recipe}
|
||||
# Load themis package for dealing with imbalanced data
|
||||
library(themis)
|
||||
|
||||
# Create a recipe for preprocessing data
|
||||
cuisines_recipe <- recipe(cuisine ~ ., data = df_select) %>%
|
||||
step_smote(cuisine)
|
||||
|
||||
cuisines_recipe
|
||||
```
|
||||
|
||||
Let's break down our preprocessing steps.
|
||||
|
||||
- The call to `recipe()` with a formula tells the recipe the *roles* of the variables using `df_select` data as the reference. For instance the `cuisine` column has been assigned an `outcome` role while the rest of the columns have been assigned a `predictor` role.
|
||||
|
||||
- [`step_smote(cuisine)`](https://themis.tidymodels.org/reference/step_smote.html) creates a *specification* of a recipe step that synthetically generates new examples of the minority class using nearest neighbors of these cases.
|
||||
|
||||
Now, if we wanted to see the preprocessed data, we'd have to [**`prep()`**](https://recipes.tidymodels.org/reference/prep.html) and [**`bake()`**](https://recipes.tidymodels.org/reference/bake.html) our recipe.
|
||||
|
||||
`prep()`: estimates the required parameters from a training set that can be later applied to other data sets.
|
||||
|
||||
`bake()`: takes a prepped recipe and applies the operations to any data set.
|
||||
|
||||
```{r prep_bake}
|
||||
# Prep and bake the recipe
|
||||
preprocessed_df <- cuisines_recipe %>%
|
||||
prep() %>%
|
||||
bake(new_data = NULL) %>%
|
||||
relocate(cuisine)
|
||||
|
||||
# Display data
|
||||
preprocessed_df %>%
|
||||
slice_head(n = 5)
|
||||
|
||||
# Quick summary stats
|
||||
preprocessed_df %>%
|
||||
introduce()
|
||||
|
||||
```
|
||||
|
||||
Let's now check the distribution of our cuisines and compare them with the imbalanced data.
|
||||
|
||||
```{r prep_cuisines}
|
||||
# Distribution of cuisines
|
||||
new_label_count <- preprocessed_df %>%
|
||||
count(cuisine) %>%
|
||||
arrange(desc(n))
|
||||
|
||||
list(new_label_count = new_label_count,
|
||||
old_label_count = old_label_count)
|
||||
|
||||
```
|
||||
|
||||
Yum! The data is nice and clean, balanced, and very delicious 😋!
|
||||
|
||||
> Normally, a recipe is usually used as a preprocessor for modelling where it defines what steps should be applied to a data set in order to get it ready for modelling. In that case, a `workflow()` is typically used (as we have already seen in our previous lessons) instead of manually estimating a recipe
|
||||
>
|
||||
> As such, you don't typically need to **`prep()`** and **`bake()`** recipes when you use tidymodels, but they are helpful functions to have in your toolkit for confirming that recipes are doing what you expect like in our case.
|
||||
>
|
||||
> When you **`bake()`** a prepped recipe with **`new_data = NULL`**, you get the data that you provided when defining the recipe back, but having undergone the preprocessing steps.
|
||||
|
||||
Let's now save a copy of this data for use in future lessons:
|
||||
|
||||
```{r save_preproc_data}
|
||||
# Save preprocessed data
|
||||
write_csv(preprocessed_df, "../../../data/cleaned_cuisines_R.csv")
|
||||
|
||||
```
|
||||
|
||||
This fresh CSV can now be found in the root data folder.
|
||||
|
||||
**🚀Challenge**
|
||||
|
||||
This curriculum contains several interesting datasets. Dig through the `data` folders and see if any contain datasets that would be appropriate for binary or multi-class classification? What questions would you ask of this dataset?
|
||||
|
||||
## [**Post-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/20/)
|
||||
|
||||
## **Review & Self Study**
|
||||
|
||||
- Check out [package themis](https://github.com/tidymodels/themis). What other techniques could we use to deal with imbalanced data?
|
||||
|
||||
- Tidy models [reference website](https://www.tidymodels.org/start/).
|
||||
|
||||
- H. Wickham and G. Grolemund, [*R for Data Science: Visualize, Model, Transform, Tidy, and Import Data*](https://r4ds.had.co.nz/).
|
||||
|
||||
#### THANK YOU TO:
|
||||
|
||||
[`Allison Horst`](https://twitter.com/allison_horst/) for creating the amazing illustrations that make R more welcoming and engaging. Find more illustrations at her [gallery](https://www.google.com/url?q=https://github.com/allisonhorst/stats-illustrations&sa=D&source=editors&ust=1626380772530000&usg=AOvVaw3zcfyCizFQZpkSLzxiiQEM).
|
||||
|
||||
[Cassie Breviu](https://www.twitter.com/cassieview) and [Jen Looper](https://www.twitter.com/jenlooper) for creating the original Python version of this module ♥️
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,241 @@
|
||||
# Cuisine classifiers 1
|
||||
|
||||
In this lesson, you will use the dataset you saved from the last lesson full of balanced, clean data all about cuisines.
|
||||
|
||||
You will use this dataset with a variety of classifiers to _predict a given national cuisine based on a group of ingredients_. While doing so, you'll learn more about some of the ways that algorithms can be leveraged for classification tasks.
|
||||
|
||||
## [Pre-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
# Preparation
|
||||
|
||||
Assuming you completed [Lesson 1](../1-Introduction/README.md), make sure that a _cleaned_cuisines.csv_ file exists in the root `/data` folder for these four lessons.
|
||||
|
||||
## Exercise - predict a national cuisine
|
||||
|
||||
1. Working in this lesson's _notebook.ipynb_ folder, import that file along with the Pandas library:
|
||||
|
||||
```python
|
||||
import pandas as pd
|
||||
cuisines_df = pd.read_csv("../data/cleaned_cuisines.csv")
|
||||
cuisines_df.head()
|
||||
```
|
||||
|
||||
The data looks like this:
|
||||
|
||||
| | Unnamed: 0 | cuisine | almond | angelica | anise | anise_seed | apple | apple_brandy | apricot | armagnac | ... | whiskey | white_bread | white_wine | whole_grain_wheat_flour | wine | wood | yam | yeast | yogurt | zucchini |
|
||||
| --- | ---------- | ------- | ------ | -------- | ----- | ---------- | ----- | ------------ | ------- | -------- | --- | ------- | ----------- | ---------- | ----------------------- | ---- | ---- | --- | ----- | ------ | -------- |
|
||||
| 0 | 0 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 1 | 1 | indian | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 2 | 2 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 3 | 3 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 4 | 4 | indian | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
|
||||
|
||||
|
||||
1. Now, import several more libraries:
|
||||
|
||||
```python
|
||||
from sklearn.linear_model import LogisticRegression
|
||||
from sklearn.model_selection import train_test_split, cross_val_score
|
||||
from sklearn.metrics import accuracy_score,precision_score,confusion_matrix,classification_report, precision_recall_curve
|
||||
from sklearn.svm import SVC
|
||||
import numpy as np
|
||||
```
|
||||
|
||||
1. Divide the X and y coordinates into two dataframes for training. `cuisine` can be the labels dataframe:
|
||||
|
||||
```python
|
||||
cuisines_label_df = cuisines_df['cuisine']
|
||||
cuisines_label_df.head()
|
||||
```
|
||||
|
||||
It will look like this:
|
||||
|
||||
```output
|
||||
0 indian
|
||||
1 indian
|
||||
2 indian
|
||||
3 indian
|
||||
4 indian
|
||||
Name: cuisine, dtype: object
|
||||
```
|
||||
|
||||
1. Drop that `Unnamed: 0` column and the `cuisine` column, calling `drop()`. Save the rest of the data as trainable features:
|
||||
|
||||
```python
|
||||
cuisines_feature_df = cuisines_df.drop(['Unnamed: 0', 'cuisine'], axis=1)
|
||||
cuisines_feature_df.head()
|
||||
```
|
||||
|
||||
Your features look like this:
|
||||
|
||||
| | almond | angelica | anise | anise_seed | apple | apple_brandy | apricot | armagnac | artemisia | artichoke | ... | whiskey | white_bread | white_wine | whole_grain_wheat_flour | wine | wood | yam | yeast | yogurt | zucchini |
|
||||
| ---: | -----: | -------: | ----: | ---------: | ----: | -----------: | ------: | -------: | --------: | --------: | ---: | ------: | ----------: | ---------: | ----------------------: | ---: | ---: | ---: | ----: | -----: | -------: |
|
||||
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
|
||||
|
||||
Now you are ready to train your model!
|
||||
|
||||
## Choosing your classifier
|
||||
|
||||
Now that your data is clean and ready for training, you have to decide which algorithm to use for the job.
|
||||
|
||||
Scikit-learn groups classification under Supervised Learning, and in that category you will find many ways to classify. [The variety](https://scikit-learn.org/stable/supervised_learning.html) is quite bewildering at first sight. The following methods all include classification techniques:
|
||||
|
||||
- Linear Models
|
||||
- Support Vector Machines
|
||||
- Stochastic Gradient Descent
|
||||
- Nearest Neighbors
|
||||
- Gaussian Processes
|
||||
- Decision Trees
|
||||
- Ensemble methods (voting Classifier)
|
||||
- Multiclass and multioutput algorithms (multiclass and multilabel classification, multiclass-multioutput classification)
|
||||
|
||||
> You can also use [neural networks to classify data](https://scikit-learn.org/stable/modules/neural_networks_supervised.html#classification), but that is outside the scope of this lesson.
|
||||
|
||||
### What classifier to go with?
|
||||
|
||||
So, which classifier should you choose? Often, running through several and looking for a good result is a way to test. Scikit-learn offers a [side-by-side comparison](https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html) on a created dataset, comparing KNeighbors, SVC two ways, GaussianProcessClassifier, DecisionTreeClassifier, RandomForestClassifier, MLPClassifier, AdaBoostClassifier, GaussianNB and QuadraticDiscrinationAnalysis, showing the results visualized:
|
||||
|
||||
|
||||
> Plots generated on Scikit-learn's documentation
|
||||
|
||||
> AutoML solves this problem neatly by running these comparisons in the cloud, allowing you to choose the best algorithm for your data. Try it [here](https://docs.microsoft.com/learn/modules/automate-model-selection-with-azure-automl/?WT.mc_id=academic-77952-leestott)
|
||||
|
||||
### A better approach
|
||||
|
||||
A better way than wildly guessing, however, is to follow the ideas on this downloadable [ML Cheat sheet](https://docs.microsoft.com/azure/machine-learning/algorithm-cheat-sheet?WT.mc_id=academic-77952-leestott). Here, we discover that, for our multiclass problem, we have some choices:
|
||||
|
||||
|
||||
> A section of Microsoft's Algorithm Cheat Sheet, detailing multiclass classification options
|
||||
|
||||
✅ Download this cheat sheet, print it out, and hang it on your wall!
|
||||
|
||||
### Reasoning
|
||||
|
||||
Let's see if we can reason our way through different approaches given the constraints we have:
|
||||
|
||||
- **Neural networks are too heavy**. Given our clean, but minimal dataset, and the fact that we are running training locally via notebooks, neural networks are too heavyweight for this task.
|
||||
- **No two-class classifier**. We do not use a two-class classifier, so that rules out one-vs-all.
|
||||
- **Decision tree or logistic regression could work**. A decision tree might work, or logistic regression for multiclass data.
|
||||
- **Multiclass Boosted Decision Trees solve a different problem**. The multiclass boosted decision tree is most suitable for nonparametric tasks, e.g. tasks designed to build rankings, so it is not useful for us.
|
||||
|
||||
### Using Scikit-learn
|
||||
|
||||
We will be using Scikit-learn to analyze our data. However, there are many ways to use logistic regression in Scikit-learn. Take a look at the [parameters to pass](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html?highlight=logistic%20regressio#sklearn.linear_model.LogisticRegression).
|
||||
|
||||
Essentially there are two important parameters - `multi_class` and `solver` - that we need to specify, when we ask Scikit-learn to perform a logistic regression. The `multi_class` value applies a certain behavior. The value of the solver is what algorithm to use. Not all solvers can be paired with all `multi_class` values.
|
||||
|
||||
According to the docs, in the multiclass case, the training algorithm:
|
||||
|
||||
- **Uses the one-vs-rest (OvR) scheme**, if the `multi_class` option is set to `ovr`
|
||||
- **Uses the cross-entropy loss**, if the `multi_class` option is set to `multinomial`. (Currently the `multinomial` option is supported only by the ‘lbfgs’, ‘sag’, ‘saga’ and ‘newton-cg’ solvers.)"
|
||||
|
||||
> 🎓 The 'scheme' here can either be 'ovr' (one-vs-rest) or 'multinomial'. Since logistic regression is really designed to support binary classification, these schemes allow it to better handle multiclass classification tasks. [source](https://machinelearningmastery.com/one-vs-rest-and-one-vs-one-for-multi-class-classification/)
|
||||
|
||||
> 🎓 The 'solver' is defined as "the algorithm to use in the optimization problem". [source](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html?highlight=logistic%20regressio#sklearn.linear_model.LogisticRegression).
|
||||
|
||||
Scikit-learn offers this table to explain how solvers handle different challenges presented by different kinds of data structures:
|
||||
|
||||
|
||||
|
||||
## Exercise - split the data
|
||||
|
||||
We can focus on logistic regression for our first training trial since you recently learned about the latter in a previous lesson.
|
||||
Split your data into training and testing groups by calling `train_test_split()`:
|
||||
|
||||
```python
|
||||
X_train, X_test, y_train, y_test = train_test_split(cuisines_feature_df, cuisines_label_df, test_size=0.3)
|
||||
```
|
||||
|
||||
## Exercise - apply logistic regression
|
||||
|
||||
Since you are using the multiclass case, you need to choose what _scheme_ to use and what _solver_ to set. Use LogisticRegression with a multiclass setting and the **liblinear** solver to train.
|
||||
|
||||
1. Create a logistic regression with multi_class set to `ovr` and the solver set to `liblinear`:
|
||||
|
||||
```python
|
||||
lr = LogisticRegression(multi_class='ovr',solver='liblinear')
|
||||
model = lr.fit(X_train, np.ravel(y_train))
|
||||
|
||||
accuracy = model.score(X_test, y_test)
|
||||
print ("Accuracy is {}".format(accuracy))
|
||||
```
|
||||
|
||||
✅ Try a different solver like `lbfgs`, which is often set as default
|
||||
|
||||
> Note, use Pandas [`ravel`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.ravel.html) function to flatten your data when needed.
|
||||
|
||||
The accuracy is good at over **80%**!
|
||||
|
||||
1. You can see this model in action by testing one row of data (#50):
|
||||
|
||||
```python
|
||||
print(f'ingredients: {X_test.iloc[50][X_test.iloc[50]!=0].keys()}')
|
||||
print(f'cuisine: {y_test.iloc[50]}')
|
||||
```
|
||||
|
||||
The result is printed:
|
||||
|
||||
```output
|
||||
ingredients: Index(['cilantro', 'onion', 'pea', 'potato', 'tomato', 'vegetable_oil'], dtype='object')
|
||||
cuisine: indian
|
||||
```
|
||||
|
||||
✅ Try a different row number and check the results
|
||||
|
||||
1. Digging deeper, you can check for the accuracy of this prediction:
|
||||
|
||||
```python
|
||||
test= X_test.iloc[50].values.reshape(-1, 1).T
|
||||
proba = model.predict_proba(test)
|
||||
classes = model.classes_
|
||||
resultdf = pd.DataFrame(data=proba, columns=classes)
|
||||
|
||||
topPrediction = resultdf.T.sort_values(by=[0], ascending = [False])
|
||||
topPrediction.head()
|
||||
```
|
||||
|
||||
The result is printed - Indian cuisine is its best guess, with good probability:
|
||||
|
||||
| | 0 |
|
||||
| -------: | -------: |
|
||||
| indian | 0.715851 |
|
||||
| chinese | 0.229475 |
|
||||
| japanese | 0.029763 |
|
||||
| korean | 0.017277 |
|
||||
| thai | 0.007634 |
|
||||
|
||||
✅ Can you explain why the model is pretty sure this is an Indian cuisine?
|
||||
|
||||
1. Get more detail by printing a classification report, as you did in the regression lessons:
|
||||
|
||||
```python
|
||||
y_pred = model.predict(X_test)
|
||||
print(classification_report(y_test,y_pred))
|
||||
```
|
||||
|
||||
| | precision | recall | f1-score | support |
|
||||
| ------------ | --------- | ------ | -------- | ------- |
|
||||
| chinese | 0.73 | 0.71 | 0.72 | 229 |
|
||||
| indian | 0.91 | 0.93 | 0.92 | 254 |
|
||||
| japanese | 0.70 | 0.75 | 0.72 | 220 |
|
||||
| korean | 0.86 | 0.76 | 0.81 | 242 |
|
||||
| thai | 0.79 | 0.85 | 0.82 | 254 |
|
||||
| accuracy | | | 0.80 | 1199 |
|
||||
| macro avg | 0.80 | 0.80 | 0.80 | 1199 |
|
||||
| weighted avg | 0.80 | 0.80 | 0.80 | 1199 |
|
||||
|
||||
## 🚀Challenge
|
||||
|
||||
In this lesson, you used your cleaned data to build a machine learning model that can predict a national cuisine based on a series of ingredients. Take some time to read through the many options Scikit-learn provides to classify data. Dig deeper into the concept of 'solver' to understand what goes on behind the scenes.
|
||||
|
||||
## [Post-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
|
||||
## Review & Self Study
|
||||
|
||||
Dig a little more into the math behind logistic regression in [this lesson](https://people.eecs.berkeley.edu/~russell/classes/cs194/f11/lectures/CS194%20Fall%202011%20Lecture%2006.pdf)
|
||||
## Assignment
|
||||
|
||||
[Study the solvers](assignment.md)
|
||||
@@ -0,0 +1,9 @@
|
||||
# Study the solvers
|
||||
## Instructions
|
||||
|
||||
In this lesson you learned about the various solvers that pair algorithms with a machine learning process to create an accurate model. Walk through the solvers listed in the lesson and pick two. In your own words, compare and contrast these two solvers. What kind of problem do they address? How do they work with various data structures? Why would you pick one over another?
|
||||
## Rubric
|
||||
|
||||
| Criteria | Exemplary | Adequate | Needs Improvement |
|
||||
| -------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------ | ---------------------------- |
|
||||
| | A .doc file is presented with two paragraphs, one on each solver, comparing them thoughtfully. | A .doc file is presented with only one paragraph | The assignment is incomplete |
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 715 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 580 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 132 KiB |
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": 3
|
||||
},
|
||||
"orig_nbformat": 2
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2,
|
||||
"cells": [
|
||||
{
|
||||
"source": [
|
||||
"# Build Classification Models"
|
||||
],
|
||||
"cell_type": "markdown",
|
||||
"metadata": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
This is a temporary placeholder
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,349 @@
|
||||
---
|
||||
title: 'Build a classification model: Delicious Asian and Indian Cuisines'
|
||||
output:
|
||||
html_document:
|
||||
df_print: paged
|
||||
theme: flatly
|
||||
highlight: breezedark
|
||||
toc: yes
|
||||
toc_float: yes
|
||||
code_download: yes
|
||||
---
|
||||
|
||||
## Cuisine classifiers 1
|
||||
|
||||
In this lesson, we'll explore a variety of classifiers to *predict a given national cuisine based on a group of ingredients.* While doing so, we'll learn more about some of the ways that algorithms can be leveraged for classification tasks.
|
||||
|
||||
### [**Pre-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/21/)
|
||||
|
||||
### **Preparation**
|
||||
|
||||
This lesson builds up on our [previous lesson](https://github.com/microsoft/ML-For-Beginners/blob/main/4-Classification/1-Introduction/solution/R/lesson_10.html) where we:
|
||||
|
||||
- Made a gentle introduction to classifications using a dataset about all the brilliant cuisines of Asia and India 😋.
|
||||
|
||||
- Explored some [dplyr verbs](https://dplyr.tidyverse.org/) to prep and clean our data.
|
||||
|
||||
- Made beautiful visualizations using ggplot2.
|
||||
|
||||
- Demonstrated how to deal with imbalanced data by preprocessing it using [recipes](https://recipes.tidymodels.org/articles/Simple_Example.html).
|
||||
|
||||
- Demonstrated how to `prep` and `bake` our recipe to confirm that it will work as supposed to.
|
||||
|
||||
#### **Prerequisite**
|
||||
|
||||
For this lesson, we'll require the following packages to clean, prep and visualize our data:
|
||||
|
||||
- `tidyverse`: The [tidyverse](https://www.tidyverse.org/) is a [collection of R packages](https://www.tidyverse.org/packages) designed to makes data science faster, easier and more fun!
|
||||
|
||||
- `tidymodels`: The [tidymodels](https://www.tidymodels.org/) framework is a [collection of packages](https://www.tidymodels.org/packages/) for modeling and machine learning.
|
||||
|
||||
- `DataExplorer`: The [DataExplorer package](https://cran.r-project.org/web/packages/DataExplorer/vignettes/dataexplorer-intro.html) is meant to simplify and automate EDA process and report generation.
|
||||
|
||||
- `themis`: The [themis package](https://themis.tidymodels.org/) provides Extra Recipes Steps for Dealing with Unbalanced Data.
|
||||
|
||||
- `nnet`: The [nnet package](https://cran.r-project.org/web/packages/nnet/nnet.pdf) provides functions for estimating feed-forward neural networks with a single hidden layer, and for multinomial logistic regression models.
|
||||
|
||||
You can have them installed as:
|
||||
|
||||
`install.packages(c("tidyverse", "tidymodels", "DataExplorer", "here"))`
|
||||
|
||||
Alternatively, the script below checks whether you have the packages required to complete this module and installs them for you in case they are missing.
|
||||
|
||||
```{r, message=F, warning=F}
|
||||
suppressWarnings(if (!require("pacman"))install.packages("pacman"))
|
||||
|
||||
pacman::p_load(tidyverse, tidymodels, DataExplorer, themis, here)
|
||||
```
|
||||
|
||||
Now, let's hit the ground running!
|
||||
|
||||
## 1. Split the data into training and test sets.
|
||||
|
||||
We'll start by picking a few steps from our previous lesson.
|
||||
|
||||
### Drop the most common ingredients that create confusion between distinct cuisines, using `dplyr::select()`.
|
||||
|
||||
Everyone loves rice, garlic and ginger!
|
||||
|
||||
```{r recap_drop}
|
||||
# Load the original cuisines data
|
||||
df <- read_csv(file = "https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/4-Classification/data/cuisines.csv")
|
||||
|
||||
# Drop id column, rice, garlic and ginger from our original data set
|
||||
df_select <- df %>%
|
||||
select(-c(1, rice, garlic, ginger)) %>%
|
||||
# Encode cuisine column as categorical
|
||||
mutate(cuisine = factor(cuisine))
|
||||
|
||||
# Display new data set
|
||||
df_select %>%
|
||||
slice_head(n = 5)
|
||||
|
||||
# Display distribution of cuisines
|
||||
df_select %>%
|
||||
count(cuisine) %>%
|
||||
arrange(desc(n))
|
||||
```
|
||||
|
||||
Perfect! Now, time to split the data such that 70% of the data goes to training and 30% goes to testing. We'll also apply a `stratification` technique when splitting the data to `maintain the proportion of each cuisine` in the training and validation datasets.
|
||||
|
||||
[rsample](https://rsample.tidymodels.org/), a package in Tidymodels, provides infrastructure for efficient data splitting and resampling:
|
||||
|
||||
```{r data_split}
|
||||
# Load the core Tidymodels packages into R session
|
||||
library(tidymodels)
|
||||
|
||||
# Create split specification
|
||||
set.seed(2056)
|
||||
cuisines_split <- initial_split(data = df_select,
|
||||
strata = cuisine,
|
||||
prop = 0.7)
|
||||
|
||||
# Extract the data in each split
|
||||
cuisines_train <- training(cuisines_split)
|
||||
cuisines_test <- testing(cuisines_split)
|
||||
|
||||
# Print the number of cases in each split
|
||||
cat("Training cases: ", nrow(cuisines_train), "\n",
|
||||
"Test cases: ", nrow(cuisines_test), sep = "")
|
||||
|
||||
# Display the first few rows of the training set
|
||||
cuisines_train %>%
|
||||
slice_head(n = 5)
|
||||
|
||||
|
||||
# Display distribution of cuisines in the training set
|
||||
cuisines_train %>%
|
||||
count(cuisine) %>%
|
||||
arrange(desc(n))
|
||||
|
||||
|
||||
```
|
||||
|
||||
## 2. Deal with imbalanced data
|
||||
|
||||
As you might have noticed in the original data set as well as in our training set, there is quite an unequal distribution in the number of cuisines. Korean cuisines are *almost* 3 times Thai cuisines. Imbalanced data often has negative effects on the model performance. Many models perform best when the number of observations is equal and, thus, tend to struggle with unbalanced data.
|
||||
|
||||
There are majorly two ways of dealing with imbalanced data sets:
|
||||
|
||||
- adding observations to the minority class: `Over-sampling` e.g using a SMOTE algorithm which synthetically generates new examples of the minority class using nearest neighbors of these cases.
|
||||
|
||||
- removing observations from majority class: `Under-sampling`
|
||||
|
||||
In our previous lesson, we demonstrated how to deal with imbalanced data sets using a `recipe`. A recipe can be thought of as a blueprint that describes what steps should be applied to a data set in order to get it ready for data analysis. In our case, we want to have an equal distribution in the number of our cuisines for our `training set`. Let's get right into it.
|
||||
|
||||
```{r recap_balance}
|
||||
# Load themis package for dealing with imbalanced data
|
||||
library(themis)
|
||||
|
||||
# Create a recipe for preprocessing training data
|
||||
cuisines_recipe <- recipe(cuisine ~ ., data = cuisines_train) %>%
|
||||
step_smote(cuisine)
|
||||
|
||||
# Print recipe
|
||||
cuisines_recipe
|
||||
|
||||
```
|
||||
|
||||
You can of course go ahead and confirm (using prep+bake) that the recipe will work as you expect it - all the cuisine labels having `559` observations.
|
||||
|
||||
Since we'll be using this recipe as a preprocessor for modeling, a `workflow()` will do all the prep and bake for us, so we won't have to manually estimate the recipe.
|
||||
|
||||
Now we are ready to train a model 👩💻👨💻!
|
||||
|
||||
## 3. Choosing your classifier
|
||||
|
||||
{width="600"}
|
||||
|
||||
Now we have to decide which algorithm to use for the job 🤔.
|
||||
|
||||
In Tidymodels, the [`parsnip package`](https://parsnip.tidymodels.org/index.html) provides consistent interface for working with models across different engines (packages). Please see the parsnip documentation to explore [model types & engines](https://www.tidymodels.org/find/parsnip/#models) and their corresponding [model arguments](https://www.tidymodels.org/find/parsnip/#model-args). The variety is quite bewildering at first sight. For instance, the following methods all include classification techniques:
|
||||
|
||||
- C5.0 Rule-Based Classification Models
|
||||
|
||||
- Flexible Discriminant Models
|
||||
|
||||
- Linear Discriminant Models
|
||||
|
||||
- Regularized Discriminant Models
|
||||
|
||||
- Logistic Regression Models
|
||||
|
||||
- Multinomial Regression Models
|
||||
|
||||
- Naive Bayes Models
|
||||
|
||||
- Support Vector Machines
|
||||
|
||||
- Nearest Neighbors
|
||||
|
||||
- Decision Trees
|
||||
|
||||
- Ensemble methods
|
||||
|
||||
- Neural Networks
|
||||
|
||||
The list goes on!
|
||||
|
||||
### **What classifier to go with?**
|
||||
|
||||
So, which classifier should you choose? Often, running through several and looking for a good result is a way to test.
|
||||
|
||||
> AutoML solves this problem neatly by running these comparisons in the cloud, allowing you to choose the best algorithm for your data. Try it [here](https://docs.microsoft.com/learn/modules/automate-model-selection-with-azure-automl/?WT.mc_id=academic-77952-leestott)
|
||||
|
||||
Also the choice of classifier depends on our problem. For instance, when the outcome can be categorized into `more than two classes`, like in our case, you must use a `multiclass classification algorithm` as opposed to `binary classification.`
|
||||
|
||||
### **A better approach**
|
||||
|
||||
A better way than wildly guessing, however, is to follow the ideas on this downloadable [ML Cheat sheet](https://docs.microsoft.com/azure/machine-learning/algorithm-cheat-sheet?WT.mc_id=academic-77952-leestott). Here, we discover that, for our multiclass problem, we have some choices:
|
||||
|
||||
{width="500"}
|
||||
|
||||
### **Reasoning**
|
||||
|
||||
Let's see if we can reason our way through different approaches given the constraints we have:
|
||||
|
||||
- **Deep Neural networks are too heavy**. Given our clean, but minimal dataset, and the fact that we are running training locally via notebooks, deep neural networks are too heavyweight for this task.
|
||||
|
||||
- **No two-class classifier**. We do not use a two-class classifier, so that rules out one-vs-all.
|
||||
|
||||
- **Decision tree or logistic regression could work**. A decision tree might work, or multinomial regression/multiclass logistic regression for multiclass data.
|
||||
|
||||
- **Multiclass Boosted Decision Trees solve a different problem**. The multiclass boosted decision tree is most suitable for nonparametric tasks, e.g. tasks designed to build rankings, so it is not useful for us.
|
||||
|
||||
Also, normally before embarking on more complex machine learning models e.g ensemble methods, it's a good idea to build the simplest possible model to get an idea of what is going on. So for this lesson, we'll start with a `multinomial logistic regression` model.
|
||||
|
||||
> Logistic regression is a technique used when the outcome variable is categorical (or nominal). For Binary logistic regression the number of outcome variables is two, whereas the number of outcome variables for multinomial logistic regression is more than two. See [Advanced Regression Methods](https://bookdown.org/chua/ber642_advanced_regression/multinomial-logistic-regression.html) for further reading.
|
||||
|
||||
## 4. Train and evaluate a Multinomial logistic regression model.
|
||||
|
||||
In Tidymodels, `parsnip::multinom_reg()`, defines a model that uses linear predictors to predict multiclass data using the multinomial distribution. See `?multinom_reg()` for the different ways/engines you can use to fit this model.
|
||||
|
||||
For this example, we'll fit a Multinomial regression model via the default [nnet](https://cran.r-project.org/web/packages/nnet/nnet.pdf) engine.
|
||||
|
||||
> I picked a value for `penalty` sort of randomly. There are better ways to choose this value that is, by using `resampling` and `tuning` the model which we'll discuss later.
|
||||
>
|
||||
> See [Tidymodels: Get Started](https://www.tidymodels.org/start/tuning/) in case you want to learn more on how to tune model hyperparameters.
|
||||
|
||||
```{r multinorm_reg}
|
||||
# Create a multinomial regression model specification
|
||||
mr_spec <- multinom_reg(penalty = 1) %>%
|
||||
set_engine("nnet", MaxNWts = 2086) %>%
|
||||
set_mode("classification")
|
||||
|
||||
# Print model specification
|
||||
mr_spec
|
||||
|
||||
```
|
||||
|
||||
Great job 🥳! Now that we have a recipe and a model specification, we need to find a way of bundling them together into an object that will first preprocess the data then fit the model on the preprocessed data and also allow for potential post-processing activities. In Tidymodels, this convenient object is called a [`workflow`](https://workflows.tidymodels.org/) and conveniently holds your modeling components! This is what we'd call *pipelines* in *Python*.
|
||||
|
||||
So let's bundle everything up into a workflow!📦
|
||||
|
||||
```{r workflow}
|
||||
# Bundle recipe and model specification
|
||||
mr_wf <- workflow() %>%
|
||||
add_recipe(cuisines_recipe) %>%
|
||||
add_model(mr_spec)
|
||||
|
||||
# Print out workflow
|
||||
mr_wf
|
||||
|
||||
```
|
||||
|
||||
Workflows 👌👌! A **`workflow()`** can be fit in much the same way a model can. So, time to train a model!
|
||||
|
||||
```{r train}
|
||||
# Train a multinomial regression model
|
||||
mr_fit <- fit(object = mr_wf, data = cuisines_train)
|
||||
|
||||
mr_fit
|
||||
```
|
||||
|
||||
The output shows the coefficients that the model learned during training.
|
||||
|
||||
### Evaluate the Trained Model
|
||||
|
||||
It's time to see how the model performed 📏 by evaluating it on a test set! Let's begin by making predictions on the test set.
|
||||
|
||||
```{r test}
|
||||
# Make predictions on the test set
|
||||
results <- cuisines_test %>% select(cuisine) %>%
|
||||
bind_cols(mr_fit %>% predict(new_data = cuisines_test))
|
||||
|
||||
# Print out results
|
||||
results %>%
|
||||
slice_head(n = 5)
|
||||
|
||||
```
|
||||
|
||||
Great job! In Tidymodels, evaluating model performance can be done using [yardstick](https://yardstick.tidymodels.org/) - a package used to measure the effectiveness of models using performance metrics. As we did in our logistic regression lesson, let's begin by computing a confusion matrix.
|
||||
|
||||
```{r conf_mat}
|
||||
# Confusion matrix for categorical data
|
||||
conf_mat(data = results, truth = cuisine, estimate = .pred_class)
|
||||
|
||||
|
||||
```
|
||||
|
||||
When dealing with multiple classes, it's generally more intuitive to visualize this as a heat map, like this:
|
||||
|
||||
```{r conf_viz}
|
||||
update_geom_defaults(geom = "tile", new = list(color = "black", alpha = 0.7))
|
||||
# Visualize confusion matrix
|
||||
results %>%
|
||||
conf_mat(cuisine, .pred_class) %>%
|
||||
autoplot(type = "heatmap")
|
||||
```
|
||||
|
||||
The darker squares in the confusion matrix plot indicate high numbers of cases, and you can hopefully see a diagonal line of darker squares indicating cases where the predicted and actual label are the same.
|
||||
|
||||
Let's now calculate summary statistics for the confusion matrix.
|
||||
|
||||
```{r conf_stats}
|
||||
# Summary stats for confusion matrix
|
||||
conf_mat(data = results, truth = cuisine, estimate = .pred_class) %>% summary()
|
||||
```
|
||||
|
||||
If we narrow down to some metrics such as accuracy, sensitivity, ppv, we are not badly off for a start 🥳!
|
||||
|
||||
## 4. Digging Deeper
|
||||
|
||||
Let's ask one subtle question: What criteria is used to settle for a given type of cuisine as the predicted outcome?
|
||||
|
||||
Well, Statistical machine learning algorithms, like logistic regression, are based on `probability`; so what actually gets predicted by a classifier is a probability distribution over a set of possible outcomes. The class with the highest probability is then chosen as the most likely outcome for the given observations.
|
||||
|
||||
Let's see this in action by making both hard class predictions and probabilities.
|
||||
|
||||
```{r pred_prob}
|
||||
# Make hard class prediction and probabilities
|
||||
results_prob <- cuisines_test %>%
|
||||
select(cuisine) %>%
|
||||
bind_cols(mr_fit %>% predict(new_data = cuisines_test)) %>%
|
||||
bind_cols(mr_fit %>% predict(new_data = cuisines_test, type = "prob"))
|
||||
|
||||
# Print out results
|
||||
results_prob %>%
|
||||
slice_head(n = 5)
|
||||
|
||||
|
||||
```
|
||||
|
||||
Much better!
|
||||
|
||||
✅ Can you explain why the model is pretty sure that the first observation is Thai?
|
||||
|
||||
## **🚀Challenge**
|
||||
|
||||
In this lesson, you used your cleaned data to build a machine learning model that can predict a national cuisine based on a series of ingredients. Take some time to read through the [many options](https://www.tidymodels.org/find/parsnip/#models) Tidymodels provides to classify data and [other ways](https://parsnip.tidymodels.org/articles/articles/Examples.html#multinom_reg-models) to fit multinomial regression.
|
||||
|
||||
#### THANK YOU TO:
|
||||
|
||||
[`Allison Horst`](https://twitter.com/allison_horst/) for creating the amazing illustrations that make R more welcoming and engaging. Find more illustrations at her [gallery](https://www.google.com/url?q=https://github.com/allisonhorst/stats-illustrations&sa=D&source=editors&ust=1626380772530000&usg=AOvVaw3zcfyCizFQZpkSLzxiiQEM).
|
||||
|
||||
[Cassie Breviu](https://www.twitter.com/cassieview) and [Jen Looper](https://www.twitter.com/jenlooper) for creating the original Python version of this module ♥️
|
||||
|
||||
Happy Learning,
|
||||
|
||||
[Eric](https://twitter.com/ericntay), Gold Microsoft Learn Student Ambassador.
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,268 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"source": [
|
||||
"# Build Classification Models"
|
||||
],
|
||||
"cell_type": "markdown",
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" Unnamed: 0 cuisine almond angelica anise anise_seed apple \\\n",
|
||||
"0 0 indian 0 0 0 0 0 \n",
|
||||
"1 1 indian 1 0 0 0 0 \n",
|
||||
"2 2 indian 0 0 0 0 0 \n",
|
||||
"3 3 indian 0 0 0 0 0 \n",
|
||||
"4 4 indian 0 0 0 0 0 \n",
|
||||
"\n",
|
||||
" apple_brandy apricot armagnac ... whiskey white_bread white_wine \\\n",
|
||||
"0 0 0 0 ... 0 0 0 \n",
|
||||
"1 0 0 0 ... 0 0 0 \n",
|
||||
"2 0 0 0 ... 0 0 0 \n",
|
||||
"3 0 0 0 ... 0 0 0 \n",
|
||||
"4 0 0 0 ... 0 0 0 \n",
|
||||
"\n",
|
||||
" whole_grain_wheat_flour wine wood yam yeast yogurt zucchini \n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 0 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 1 0 \n",
|
||||
"\n",
|
||||
"[5 rows x 382 columns]"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>Unnamed: 0</th>\n <th>cuisine</th>\n <th>almond</th>\n <th>angelica</th>\n <th>anise</th>\n <th>anise_seed</th>\n <th>apple</th>\n <th>apple_brandy</th>\n <th>apricot</th>\n <th>armagnac</th>\n <th>...</th>\n <th>whiskey</th>\n <th>white_bread</th>\n <th>white_wine</th>\n <th>whole_grain_wheat_flour</th>\n <th>wine</th>\n <th>wood</th>\n <th>yam</th>\n <th>yeast</th>\n <th>yogurt</th>\n <th>zucchini</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>indian</td>\n <td>1</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>2</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>3</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>4</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>1</td>\n <td>0</td>\n </tr>\n </tbody>\n</table>\n<p>5 rows × 382 columns</p>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 1
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"cuisines_df = pd.read_csv(\"../../data/cleaned_cuisines.csv\")\n",
|
||||
"cuisines_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.linear_model import LogisticRegression\n",
|
||||
"from sklearn.model_selection import train_test_split, cross_val_score\n",
|
||||
"from sklearn.metrics import accuracy_score,precision_score,confusion_matrix,classification_report, precision_recall_curve\n",
|
||||
"from sklearn.svm import SVC\n",
|
||||
"import numpy as np"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0 indian\n",
|
||||
"1 indian\n",
|
||||
"2 indian\n",
|
||||
"3 indian\n",
|
||||
"4 indian\n",
|
||||
"Name: cuisine, dtype: object"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"cuisines_label_df = cuisines_df['cuisine']\n",
|
||||
"cuisines_label_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" almond angelica anise anise_seed apple apple_brandy apricot \\\n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 1 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 0 0 \n",
|
||||
"\n",
|
||||
" armagnac artemisia artichoke ... whiskey white_bread white_wine \\\n",
|
||||
"0 0 0 0 ... 0 0 0 \n",
|
||||
"1 0 0 0 ... 0 0 0 \n",
|
||||
"2 0 0 0 ... 0 0 0 \n",
|
||||
"3 0 0 0 ... 0 0 0 \n",
|
||||
"4 0 0 0 ... 0 0 0 \n",
|
||||
"\n",
|
||||
" whole_grain_wheat_flour wine wood yam yeast yogurt zucchini \n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 0 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 1 0 \n",
|
||||
"\n",
|
||||
"[5 rows x 380 columns]"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>almond</th>\n <th>angelica</th>\n <th>anise</th>\n <th>anise_seed</th>\n <th>apple</th>\n <th>apple_brandy</th>\n <th>apricot</th>\n <th>armagnac</th>\n <th>artemisia</th>\n <th>artichoke</th>\n <th>...</th>\n <th>whiskey</th>\n <th>white_bread</th>\n <th>white_wine</th>\n <th>whole_grain_wheat_flour</th>\n <th>wine</th>\n <th>wood</th>\n <th>yam</th>\n <th>yeast</th>\n <th>yogurt</th>\n <th>zucchini</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>1</td>\n <td>0</td>\n </tr>\n </tbody>\n</table>\n<p>5 rows × 380 columns</p>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 4
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"cuisines_feature_df = cuisines_df.drop(['Unnamed: 0', 'cuisine'], axis=1)\n",
|
||||
"cuisines_feature_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"X_train, X_test, y_train, y_test = train_test_split(cuisines_feature_df, cuisines_label_df, test_size=0.3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Accuracy is 0.8181818181818182\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"lr = LogisticRegression(multi_class='ovr',solver='liblinear')\n",
|
||||
"model = lr.fit(X_train, np.ravel(y_train))\n",
|
||||
"\n",
|
||||
"accuracy = model.score(X_test, y_test)\n",
|
||||
"print (\"Accuracy is {}\".format(accuracy))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"ingredients: Index(['artemisia', 'black_pepper', 'mushroom', 'shiitake', 'soy_sauce',\n 'vegetable_oil'],\n dtype='object')\ncuisine: korean\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# test an item\n",
|
||||
"print(f'ingredients: {X_test.iloc[50][X_test.iloc[50]!=0].keys()}')\n",
|
||||
"print(f'cuisine: {y_test.iloc[50]}')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0\n",
|
||||
"korean 0.392231\n",
|
||||
"chinese 0.372872\n",
|
||||
"japanese 0.218825\n",
|
||||
"thai 0.013427\n",
|
||||
"indian 0.002645"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>0</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>korean</th>\n <td>0.392231</td>\n </tr>\n <tr>\n <th>chinese</th>\n <td>0.372872</td>\n </tr>\n <tr>\n <th>japanese</th>\n <td>0.218825</td>\n </tr>\n <tr>\n <th>thai</th>\n <td>0.013427</td>\n </tr>\n <tr>\n <th>indian</th>\n <td>0.002645</td>\n </tr>\n </tbody>\n</table>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 8
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"#rehsape to 2d array and transpose\n",
|
||||
"test= X_test.iloc[50].values.reshape(-1, 1).T\n",
|
||||
"# predict with score\n",
|
||||
"proba = model.predict_proba(test)\n",
|
||||
"classes = model.classes_\n",
|
||||
"# create df with classes and scores\n",
|
||||
"resultdf = pd.DataFrame(data=proba, columns=classes)\n",
|
||||
"\n",
|
||||
"# create df to show results\n",
|
||||
"topPrediction = resultdf.T.sort_values(by=[0], ascending = [False])\n",
|
||||
"topPrediction.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
" precision recall f1-score support\n\n chinese 0.75 0.73 0.74 223\n indian 0.93 0.88 0.90 255\n japanese 0.78 0.78 0.78 253\n korean 0.87 0.86 0.86 236\n thai 0.76 0.84 0.80 232\n\n accuracy 0.82 1199\n macro avg 0.82 0.82 0.82 1199\nweighted avg 0.82 0.82 0.82 1199\n\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"y_pred = model.predict(X_test)\r\n",
|
||||
"print(classification_report(y_test,y_pred))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
},
|
||||
"kernelspec": {
|
||||
"name": "python3",
|
||||
"display_name": "Python 3.7.0 64-bit ('3.7')"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.0"
|
||||
},
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,235 @@
|
||||
# Cuisine classifiers 2
|
||||
|
||||
In this second classification lesson, you will explore more ways to classify numeric data. You will also learn about the ramifications for choosing one classifier over the other.
|
||||
|
||||
## [Pre-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
|
||||
### Prerequisite
|
||||
|
||||
We assume that you have completed the previous lessons and have a cleaned dataset in your `data` folder called _cleaned_cuisines.csv_ in the root of this 4-lesson folder.
|
||||
|
||||
### Preparation
|
||||
|
||||
We have loaded your _notebook.ipynb_ file with the cleaned dataset and have divided it into X and y dataframes, ready for the model building process.
|
||||
|
||||
## A classification map
|
||||
|
||||
Previously, you learned about the various options you have when classifying data using Microsoft's cheat sheet. Scikit-learn offers a similar, but more granular cheat sheet that can further help narrow down your estimators (another term for classifiers):
|
||||
|
||||
|
||||
> Tip: [visit this map online](https://scikit-learn.org/stable/tutorial/machine_learning_map/) and click along the path to read documentation.
|
||||
|
||||
### The plan
|
||||
|
||||
This map is very helpful once you have a clear grasp of your data, as you can 'walk' along its paths to a decision:
|
||||
|
||||
- We have >50 samples
|
||||
- We want to predict a category
|
||||
- We have labeled data
|
||||
- We have fewer than 100K samples
|
||||
- ✨ We can choose a Linear SVC
|
||||
- If that doesn't work, since we have numeric data
|
||||
- We can try a ✨ KNeighbors Classifier
|
||||
- If that doesn't work, try ✨ SVC and ✨ Ensemble Classifiers
|
||||
|
||||
This is a very helpful trail to follow.
|
||||
|
||||
## Exercise - split the data
|
||||
|
||||
Following this path, we should start by importing some libraries to use.
|
||||
|
||||
1. Import the needed libraries:
|
||||
|
||||
```python
|
||||
from sklearn.neighbors import KNeighborsClassifier
|
||||
from sklearn.linear_model import LogisticRegression
|
||||
from sklearn.svm import SVC
|
||||
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier
|
||||
from sklearn.model_selection import train_test_split, cross_val_score
|
||||
from sklearn.metrics import accuracy_score,precision_score,confusion_matrix,classification_report, precision_recall_curve
|
||||
import numpy as np
|
||||
```
|
||||
|
||||
1. Split your training and test data:
|
||||
|
||||
```python
|
||||
X_train, X_test, y_train, y_test = train_test_split(cuisines_features_df, cuisines_label_df, test_size=0.3)
|
||||
```
|
||||
|
||||
## Linear SVC classifier
|
||||
|
||||
Support-Vector clustering (SVC) is a child of the Support-Vector machines family of ML techniques (learn more about these below). In this method, you can choose a 'kernel' to decide how to cluster the labels. The 'C' parameter refers to 'regularization' which regulates the influence of parameters. The kernel can be one of [several](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC); here we set it to 'linear' to ensure that we leverage linear SVC. Probability defaults to 'false'; here we set it to 'true' to gather probability estimates. We set the random state to '0' to shuffle the data to get probabilities.
|
||||
|
||||
### Exercise - apply a linear SVC
|
||||
|
||||
Start by creating an array of classifiers. You will add progressively to this array as we test.
|
||||
|
||||
1. Start with a Linear SVC:
|
||||
|
||||
```python
|
||||
C = 10
|
||||
# Create different classifiers.
|
||||
classifiers = {
|
||||
'Linear SVC': SVC(kernel='linear', C=C, probability=True,random_state=0)
|
||||
}
|
||||
```
|
||||
|
||||
2. Train your model using the Linear SVC and print out a report:
|
||||
|
||||
```python
|
||||
n_classifiers = len(classifiers)
|
||||
|
||||
for index, (name, classifier) in enumerate(classifiers.items()):
|
||||
classifier.fit(X_train, np.ravel(y_train))
|
||||
|
||||
y_pred = classifier.predict(X_test)
|
||||
accuracy = accuracy_score(y_test, y_pred)
|
||||
print("Accuracy (train) for %s: %0.1f%% " % (name, accuracy * 100))
|
||||
print(classification_report(y_test,y_pred))
|
||||
```
|
||||
|
||||
The result is pretty good:
|
||||
|
||||
```output
|
||||
Accuracy (train) for Linear SVC: 78.6%
|
||||
precision recall f1-score support
|
||||
|
||||
chinese 0.71 0.67 0.69 242
|
||||
indian 0.88 0.86 0.87 234
|
||||
japanese 0.79 0.74 0.76 254
|
||||
korean 0.85 0.81 0.83 242
|
||||
thai 0.71 0.86 0.78 227
|
||||
|
||||
accuracy 0.79 1199
|
||||
macro avg 0.79 0.79 0.79 1199
|
||||
weighted avg 0.79 0.79 0.79 1199
|
||||
```
|
||||
|
||||
## K-Neighbors classifier
|
||||
|
||||
K-Neighbors is part of the "neighbors" family of ML methods, which can be used for both supervised and unsupervised learning. In this method, a predefined number of points is created and data are gathered around these points such that generalized labels can be predicted for the data.
|
||||
|
||||
### Exercise - apply the K-Neighbors classifier
|
||||
|
||||
The previous classifier was good, and worked well with the data, but maybe we can get better accuracy. Try a K-Neighbors classifier.
|
||||
|
||||
1. Add a line to your classifier array (add a comma after the Linear SVC item):
|
||||
|
||||
```python
|
||||
'KNN classifier': KNeighborsClassifier(C),
|
||||
```
|
||||
|
||||
The result is a little worse:
|
||||
|
||||
```output
|
||||
Accuracy (train) for KNN classifier: 73.8%
|
||||
precision recall f1-score support
|
||||
|
||||
chinese 0.64 0.67 0.66 242
|
||||
indian 0.86 0.78 0.82 234
|
||||
japanese 0.66 0.83 0.74 254
|
||||
korean 0.94 0.58 0.72 242
|
||||
thai 0.71 0.82 0.76 227
|
||||
|
||||
accuracy 0.74 1199
|
||||
macro avg 0.76 0.74 0.74 1199
|
||||
weighted avg 0.76 0.74 0.74 1199
|
||||
```
|
||||
|
||||
✅ Learn about [K-Neighbors](https://scikit-learn.org/stable/modules/neighbors.html#neighbors)
|
||||
|
||||
## Support Vector Classifier
|
||||
|
||||
Support-Vector classifiers are part of the [Support-Vector Machine](https://wikipedia.org/wiki/Support-vector_machine) family of ML methods that are used for classification and regression tasks. SVMs "map training examples to points in space" to maximize the distance between two categories. Subsequent data is mapped into this space so their category can be predicted.
|
||||
|
||||
### Exercise - apply a Support Vector Classifier
|
||||
|
||||
Let's try for a little better accuracy with a Support Vector Classifier.
|
||||
|
||||
1. Add a comma after the K-Neighbors item, and then add this line:
|
||||
|
||||
```python
|
||||
'SVC': SVC(),
|
||||
```
|
||||
|
||||
The result is quite good!
|
||||
|
||||
```output
|
||||
Accuracy (train) for SVC: 83.2%
|
||||
precision recall f1-score support
|
||||
|
||||
chinese 0.79 0.74 0.76 242
|
||||
indian 0.88 0.90 0.89 234
|
||||
japanese 0.87 0.81 0.84 254
|
||||
korean 0.91 0.82 0.86 242
|
||||
thai 0.74 0.90 0.81 227
|
||||
|
||||
accuracy 0.83 1199
|
||||
macro avg 0.84 0.83 0.83 1199
|
||||
weighted avg 0.84 0.83 0.83 1199
|
||||
```
|
||||
|
||||
✅ Learn about [Support-Vectors](https://scikit-learn.org/stable/modules/svm.html#svm)
|
||||
|
||||
## Ensemble Classifiers
|
||||
|
||||
Let's follow the path to the very end, even though the previous test was quite good. Let's try some 'Ensemble Classifiers, specifically Random Forest and AdaBoost:
|
||||
|
||||
```python
|
||||
'RFST': RandomForestClassifier(n_estimators=100),
|
||||
'ADA': AdaBoostClassifier(n_estimators=100)
|
||||
```
|
||||
|
||||
The result is very good, especially for Random Forest:
|
||||
|
||||
```output
|
||||
Accuracy (train) for RFST: 84.5%
|
||||
precision recall f1-score support
|
||||
|
||||
chinese 0.80 0.77 0.78 242
|
||||
indian 0.89 0.92 0.90 234
|
||||
japanese 0.86 0.84 0.85 254
|
||||
korean 0.88 0.83 0.85 242
|
||||
thai 0.80 0.87 0.83 227
|
||||
|
||||
accuracy 0.84 1199
|
||||
macro avg 0.85 0.85 0.84 1199
|
||||
weighted avg 0.85 0.84 0.84 1199
|
||||
|
||||
Accuracy (train) for ADA: 72.4%
|
||||
precision recall f1-score support
|
||||
|
||||
chinese 0.64 0.49 0.56 242
|
||||
indian 0.91 0.83 0.87 234
|
||||
japanese 0.68 0.69 0.69 254
|
||||
korean 0.73 0.79 0.76 242
|
||||
thai 0.67 0.83 0.74 227
|
||||
|
||||
accuracy 0.72 1199
|
||||
macro avg 0.73 0.73 0.72 1199
|
||||
weighted avg 0.73 0.72 0.72 1199
|
||||
```
|
||||
|
||||
✅ Learn about [Ensemble Classifiers](https://scikit-learn.org/stable/modules/ensemble.html)
|
||||
|
||||
This method of Machine Learning "combines the predictions of several base estimators" to improve the model's quality. In our example, we used Random Trees and AdaBoost.
|
||||
|
||||
- [Random Forest](https://scikit-learn.org/stable/modules/ensemble.html#forest), an averaging method, builds a 'forest' of 'decision trees' infused with randomness to avoid overfitting. The n_estimators parameter is set to the number of trees.
|
||||
|
||||
- [AdaBoost](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html) fits a classifier to a dataset and then fits copies of that classifier to the same dataset. It focuses on the weights of incorrectly classified items and adjusts the fit for the next classifier to correct.
|
||||
|
||||
---
|
||||
|
||||
## 🚀Challenge
|
||||
|
||||
Each of these techniques has a large number of parameters that you can tweak. Research each one's default parameters and think about what tweaking these parameters would mean for the model's quality.
|
||||
|
||||
## [Post-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
|
||||
## Review & Self Study
|
||||
|
||||
There's a lot of jargon in these lessons, so take a minute to review [this list](https://docs.microsoft.com/dotnet/machine-learning/resources/glossary?WT.mc_id=academic-77952-leestott) of useful terminology!
|
||||
|
||||
## Assignment
|
||||
|
||||
[Parameter play](assignment.md)
|
||||
@@ -0,0 +1,11 @@
|
||||
# Parameter Play
|
||||
|
||||
## Instructions
|
||||
|
||||
There are a lot of parameters that are set by default when working with these classifiers. Intellisense in VS Code can help you dig into them. Adopt one of the ML Classification Techniques in this lesson and retrain models tweaking various parameter values. Build a notebook explaining why some changes help the model quality while others degrade it. Be detailed in your answer.
|
||||
|
||||
## Rubric
|
||||
|
||||
| Criteria | Exemplary | Adequate | Needs Improvement |
|
||||
| -------- | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ----------------------------- |
|
||||
| | A notebook is presented with a classifier fully built up and its parameters tweaked and changes explained in textboxes | A notebook is partially presented or poorly explained | A notebook is buggy or flawed |
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 743 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
@@ -0,0 +1,152 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Build Classification Model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" Unnamed: 0 cuisine almond angelica anise anise_seed apple \\\n",
|
||||
"0 0 indian 0 0 0 0 0 \n",
|
||||
"1 1 indian 1 0 0 0 0 \n",
|
||||
"2 2 indian 0 0 0 0 0 \n",
|
||||
"3 3 indian 0 0 0 0 0 \n",
|
||||
"4 4 indian 0 0 0 0 0 \n",
|
||||
"\n",
|
||||
" apple_brandy apricot armagnac ... whiskey white_bread white_wine \\\n",
|
||||
"0 0 0 0 ... 0 0 0 \n",
|
||||
"1 0 0 0 ... 0 0 0 \n",
|
||||
"2 0 0 0 ... 0 0 0 \n",
|
||||
"3 0 0 0 ... 0 0 0 \n",
|
||||
"4 0 0 0 ... 0 0 0 \n",
|
||||
"\n",
|
||||
" whole_grain_wheat_flour wine wood yam yeast yogurt zucchini \n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 0 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 1 0 \n",
|
||||
"\n",
|
||||
"[5 rows x 382 columns]"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>Unnamed: 0</th>\n <th>cuisine</th>\n <th>almond</th>\n <th>angelica</th>\n <th>anise</th>\n <th>anise_seed</th>\n <th>apple</th>\n <th>apple_brandy</th>\n <th>apricot</th>\n <th>armagnac</th>\n <th>...</th>\n <th>whiskey</th>\n <th>white_bread</th>\n <th>white_wine</th>\n <th>whole_grain_wheat_flour</th>\n <th>wine</th>\n <th>wood</th>\n <th>yam</th>\n <th>yeast</th>\n <th>yogurt</th>\n <th>zucchini</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>indian</td>\n <td>1</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>2</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>3</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>4</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>1</td>\n <td>0</td>\n </tr>\n </tbody>\n</table>\n<p>5 rows × 382 columns</p>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 9
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"cuisines_df = pd.read_csv(\"../data/cleaned_cuisines.csv\")\n",
|
||||
"cuisines_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0 indian\n",
|
||||
"1 indian\n",
|
||||
"2 indian\n",
|
||||
"3 indian\n",
|
||||
"4 indian\n",
|
||||
"Name: cuisine, dtype: object"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 10
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"cuisines_label_df = cuisines_df['cuisine']\n",
|
||||
"cuisines_label_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" almond angelica anise anise_seed apple apple_brandy apricot \\\n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 1 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 0 0 \n",
|
||||
"\n",
|
||||
" armagnac artemisia artichoke ... whiskey white_bread white_wine \\\n",
|
||||
"0 0 0 0 ... 0 0 0 \n",
|
||||
"1 0 0 0 ... 0 0 0 \n",
|
||||
"2 0 0 0 ... 0 0 0 \n",
|
||||
"3 0 0 0 ... 0 0 0 \n",
|
||||
"4 0 0 0 ... 0 0 0 \n",
|
||||
"\n",
|
||||
" whole_grain_wheat_flour wine wood yam yeast yogurt zucchini \n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 0 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 1 0 \n",
|
||||
"\n",
|
||||
"[5 rows x 380 columns]"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>almond</th>\n <th>angelica</th>\n <th>anise</th>\n <th>anise_seed</th>\n <th>apple</th>\n <th>apple_brandy</th>\n <th>apricot</th>\n <th>armagnac</th>\n <th>artemisia</th>\n <th>artichoke</th>\n <th>...</th>\n <th>whiskey</th>\n <th>white_bread</th>\n <th>white_wine</th>\n <th>whole_grain_wheat_flour</th>\n <th>wine</th>\n <th>wood</th>\n <th>yam</th>\n <th>yeast</th>\n <th>yogurt</th>\n <th>zucchini</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>1</td>\n <td>0</td>\n </tr>\n </tbody>\n</table>\n<p>5 rows × 380 columns</p>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 11
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"cuisines_features_df = cuisines_df.drop(['Unnamed: 0', 'cuisine'], axis=1)\n",
|
||||
"cuisines_features_df.head()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
},
|
||||
"kernelspec": {
|
||||
"name": "python3",
|
||||
"display_name": "Python 3.7.0 64-bit ('3.7')"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.0"
|
||||
},
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
This is a temporary placeholder
|
||||
@@ -0,0 +1,645 @@
|
||||
{
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "lesson_12-R.ipynb",
|
||||
"provenance": [],
|
||||
"collapsed_sections": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"name": "ir",
|
||||
"display_name": "R"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "R"
|
||||
}
|
||||
},
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "jsFutf_ygqSx"
|
||||
},
|
||||
"source": [
|
||||
"# Build a classification model: Delicious Asian and Indian Cuisines"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "HD54bEefgtNO"
|
||||
},
|
||||
"source": [
|
||||
"## Cuisine classifiers 2\n",
|
||||
"\n",
|
||||
"In this second classification lesson, we will explore `more ways` to classify categorical data. We will also learn about the ramifications for choosing one classifier over the other.\n",
|
||||
"\n",
|
||||
"### [**Pre-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/23/)\n",
|
||||
"\n",
|
||||
"### **Prerequisite**\n",
|
||||
"\n",
|
||||
"We assume that you have completed the previous lessons since we will be carrying forward some concepts we learned before.\n",
|
||||
"\n",
|
||||
"For this lesson, we'll require the following packages:\n",
|
||||
"\n",
|
||||
"- `tidyverse`: The [tidyverse](https://www.tidyverse.org/) is a [collection of R packages](https://www.tidyverse.org/packages) designed to makes data science faster, easier and more fun!\n",
|
||||
"\n",
|
||||
"- `tidymodels`: The [tidymodels](https://www.tidymodels.org/) framework is a [collection of packages](https://www.tidymodels.org/packages/) for modeling and machine learning.\n",
|
||||
"\n",
|
||||
"- `themis`: The [themis package](https://themis.tidymodels.org/) provides Extra Recipes Steps for Dealing with Unbalanced Data.\n",
|
||||
"\n",
|
||||
"You can have them installed as:\n",
|
||||
"\n",
|
||||
"`install.packages(c(\"tidyverse\", \"tidymodels\", \"kernlab\", \"themis\", \"ranger\", \"xgboost\", \"kknn\"))`\n",
|
||||
"\n",
|
||||
"Alternatively, the script below checks whether you have the packages required to complete this module and installs them for you in case they are missing."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "vZ57IuUxgyQt"
|
||||
},
|
||||
"source": [
|
||||
"suppressWarnings(if (!require(\"pacman\"))install.packages(\"pacman\"))\n",
|
||||
"\n",
|
||||
"pacman::p_load(tidyverse, tidymodels, themis, kernlab, ranger, xgboost, kknn)"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "z22M-pj4g07x"
|
||||
},
|
||||
"source": [
|
||||
"Now, let's hit the ground running!\n",
|
||||
"\n",
|
||||
"## **1. A classification map**\n",
|
||||
"\n",
|
||||
"In our [previous lesson](https://github.com/microsoft/ML-For-Beginners/tree/main/4-Classification/2-Classifiers-1), we tried to address the question: how do we choose between multiple models? To a great extent, it depends on the characteristics of the data and the type of problem we want to solve (for instance classification or regression?)\n",
|
||||
"\n",
|
||||
"Previously, we learned about the various options you have when classifying data using Microsoft's cheat sheet. Python's Machine Learning framework, Scikit-learn, offers a similar but more granular cheat sheet that can further help narrow down your estimators (another term for classifiers):\n",
|
||||
"\n",
|
||||
"<p >\n",
|
||||
" <img src=\"../../images/map.png\"\n",
|
||||
" width=\"700\"/>\n",
|
||||
" <figcaption></figcaption>\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "u1i3xRIVg7vG"
|
||||
},
|
||||
"source": [
|
||||
"> Tip: [visit this map online](https://scikit-learn.org/stable/tutorial/machine_learning_map/) and click along the path to read documentation.\n",
|
||||
">\n",
|
||||
"> The [Tidymodels reference site](https://www.tidymodels.org/find/parsnip/#models) also provides an excellent documentation about different types of model.\n",
|
||||
"\n",
|
||||
"### **The plan** 🗺️\n",
|
||||
"\n",
|
||||
"This map is very helpful once you have a clear grasp of your data, as you can 'walk' along its paths to a decision:\n",
|
||||
"\n",
|
||||
"- We have \\>50 samples\n",
|
||||
"\n",
|
||||
"- We want to predict a category\n",
|
||||
"\n",
|
||||
"- We have labeled data\n",
|
||||
"\n",
|
||||
"- We have fewer than 100K samples\n",
|
||||
"\n",
|
||||
"- ✨ We can choose a Linear SVC\n",
|
||||
"\n",
|
||||
"- If that doesn't work, since we have numeric data\n",
|
||||
"\n",
|
||||
" - We can try a ✨ KNeighbors Classifier\n",
|
||||
"\n",
|
||||
" - If that doesn't work, try ✨ SVC and ✨ Ensemble Classifiers\n",
|
||||
"\n",
|
||||
"This is a very helpful trail to follow. Now, let's get right into it using the [tidymodels](https://www.tidymodels.org/) modelling framework: a consistent and flexible collection of R packages developed to encourage good statistical practice 😊.\n",
|
||||
"\n",
|
||||
"## 2. Split the data and deal with imbalanced data set.\n",
|
||||
"\n",
|
||||
"From our previous lessons, we learnt that there were a set of common ingredients across our cuisines. Also, there was quite an unequal distribution in the number of cuisines.\n",
|
||||
"\n",
|
||||
"We'll deal with these by\n",
|
||||
"\n",
|
||||
"- Dropping the most common ingredients that create confusion between distinct cuisines, using `dplyr::select()`.\n",
|
||||
"\n",
|
||||
"- Use a `recipe` that preprocesses the data to get it ready for modelling by applying an `over-sampling` algorithm.\n",
|
||||
"\n",
|
||||
"We already looked at the above in the previous lesson so this should be a breeze 🥳!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "6tj_rN00hClA"
|
||||
},
|
||||
"source": [
|
||||
"# Load the core Tidyverse and Tidymodels packages\n",
|
||||
"library(tidyverse)\n",
|
||||
"library(tidymodels)\n",
|
||||
"\n",
|
||||
"# Load the original cuisines data\n",
|
||||
"df <- read_csv(file = \"https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/4-Classification/data/cuisines.csv\")\n",
|
||||
"\n",
|
||||
"# Drop id column, rice, garlic and ginger from our original data set\n",
|
||||
"df_select <- df %>% \n",
|
||||
" select(-c(1, rice, garlic, ginger)) %>%\n",
|
||||
" # Encode cuisine column as categorical\n",
|
||||
" mutate(cuisine = factor(cuisine))\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Create data split specification\n",
|
||||
"set.seed(2056)\n",
|
||||
"cuisines_split <- initial_split(data = df_select,\n",
|
||||
" strata = cuisine,\n",
|
||||
" prop = 0.7)\n",
|
||||
"\n",
|
||||
"# Extract the data in each split\n",
|
||||
"cuisines_train <- training(cuisines_split)\n",
|
||||
"cuisines_test <- testing(cuisines_split)\n",
|
||||
"\n",
|
||||
"# Display distribution of cuisines in the training set\n",
|
||||
"cuisines_train %>% \n",
|
||||
" count(cuisine) %>% \n",
|
||||
" arrange(desc(n))"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "zFin5yw3hHb1"
|
||||
},
|
||||
"source": [
|
||||
"### Deal with imbalanced data\n",
|
||||
"\n",
|
||||
"Imbalanced data often has negative effects on the model performance. Many models perform best when the number of observations is equal and, thus, tend to struggle with unbalanced data.\n",
|
||||
"\n",
|
||||
"There are majorly two ways of dealing with imbalanced data sets:\n",
|
||||
"\n",
|
||||
"- adding observations to the minority class: `Over-sampling` e.g using a SMOTE algorithm which synthetically generates new examples of the minority class using nearest neighbors of these cases.\n",
|
||||
"\n",
|
||||
"- removing observations from majority class: `Under-sampling`\n",
|
||||
"\n",
|
||||
"In our previous lesson, we demonstrated how to deal with imbalanced data sets using a `recipe`. A recipe can be thought of as a blueprint that describes what steps should be applied to a data set in order to get it ready for data analysis. In our case, we want to have an equal distribution in the number of our cuisines for our `training set`. Let's get right into it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "cRzTnHolhLWd"
|
||||
},
|
||||
"source": [
|
||||
"# Load themis package for dealing with imbalanced data\n",
|
||||
"library(themis)\n",
|
||||
"\n",
|
||||
"# Create a recipe for preprocessing training data\n",
|
||||
"cuisines_recipe <- recipe(cuisine ~ ., data = cuisines_train) %>%\n",
|
||||
" step_smote(cuisine) \n",
|
||||
"\n",
|
||||
"# Print recipe\n",
|
||||
"cuisines_recipe"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KxOQ2ORhhO81"
|
||||
},
|
||||
"source": [
|
||||
"Now we are ready to train models 👩💻👨💻!\n",
|
||||
"\n",
|
||||
"## 3. Beyond multinomial regression models\n",
|
||||
"\n",
|
||||
"In our previous lesson, we looked at multinomial regression models. Let's explore some more flexible models for classification.\n",
|
||||
"\n",
|
||||
"### Support Vector Machines.\n",
|
||||
"\n",
|
||||
"In the context of classification, `Support Vector Machines` is a machine learning technique that tries to find a *hyperplane* that \"best\" separates the classes. Let's look at a simple example:\n",
|
||||
"\n",
|
||||
"<p >\n",
|
||||
" <img src=\"../../images/svm.png\"\n",
|
||||
" width=\"300\"/>\n",
|
||||
" <figcaption>https://commons.wikimedia.org/w/index.php?curid=22877598</figcaption>\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "C4Wsd0vZhXYu"
|
||||
},
|
||||
"source": [
|
||||
"H1~ does not separate the classes. H2~ does, but only with a small margin. H3~ separates them with the maximal margin.\n",
|
||||
"\n",
|
||||
"#### Linear Support Vector Classifier\n",
|
||||
"\n",
|
||||
"Support-Vector clustering (SVC) is a child of the Support-Vector machines family of ML techniques. In SVC, the hyperplane is chosen to correctly separate `most` of the training observations, but `may misclassify` a few observations. By allowing some points to be on the wrong side, the SVM becomes more robust to outliers hence better generalization to new data. The parameter that regulates this violation is referred to as `cost` which has a default value of 1 (see `help(\"svm_poly\")`).\n",
|
||||
"\n",
|
||||
"Let's create a linear SVC by setting `degree = 1` in a polynomial SVM model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "vJpp6nuChlBz"
|
||||
},
|
||||
"source": [
|
||||
"# Make a linear SVC specification\n",
|
||||
"svc_linear_spec <- svm_poly(degree = 1) %>% \n",
|
||||
" set_engine(\"kernlab\") %>% \n",
|
||||
" set_mode(\"classification\")\n",
|
||||
"\n",
|
||||
"# Bundle specification and recipe into a worklow\n",
|
||||
"svc_linear_wf <- workflow() %>% \n",
|
||||
" add_recipe(cuisines_recipe) %>% \n",
|
||||
" add_model(svc_linear_spec)\n",
|
||||
"\n",
|
||||
"# Print out workflow\n",
|
||||
"svc_linear_wf"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "rDs8cWNkhoqu"
|
||||
},
|
||||
"source": [
|
||||
"Now that we have captured the preprocessing steps and model specification into a *workflow*, we can go ahead and train the linear SVC and evaluate results while at it. For performance metrics, let's create a metric set that will evaluate: `accuracy`, `sensitivity`, `Positive Predicted Value` and `F Measure`\n",
|
||||
"\n",
|
||||
"> `augment()` will add column(s) for predictions to the given data."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "81wiqcwuhrnq"
|
||||
},
|
||||
"source": [
|
||||
"# Train a linear SVC model\n",
|
||||
"svc_linear_fit <- svc_linear_wf %>% \n",
|
||||
" fit(data = cuisines_train)\n",
|
||||
"\n",
|
||||
"# Create a metric set\n",
|
||||
"eval_metrics <- metric_set(ppv, sens, accuracy, f_meas)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Make predictions and Evaluate model performance\n",
|
||||
"svc_linear_fit %>% \n",
|
||||
" augment(new_data = cuisines_test) %>% \n",
|
||||
" eval_metrics(truth = cuisine, estimate = .pred_class)"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0UFQvHf-huo3"
|
||||
},
|
||||
"source": [
|
||||
"#### Support Vector Machine\n",
|
||||
"\n",
|
||||
"The support vector machine (SVM) is an extension of the support vector classifier in order to accommodate a non-linear boundary between the classes. In essence, SVMs use the *kernel trick* to enlarge the feature space to adapt to nonlinear relationships between classes. One popular and extremely flexible kernel function used by SVMs is the *Radial basis function.* Let's see how it will perform on our data."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "-KX4S8mzhzmp"
|
||||
},
|
||||
"source": [
|
||||
"set.seed(2056)\n",
|
||||
"\n",
|
||||
"# Make an RBF SVM specification\n",
|
||||
"svm_rbf_spec <- svm_rbf() %>% \n",
|
||||
" set_engine(\"kernlab\") %>% \n",
|
||||
" set_mode(\"classification\")\n",
|
||||
"\n",
|
||||
"# Bundle specification and recipe into a worklow\n",
|
||||
"svm_rbf_wf <- workflow() %>% \n",
|
||||
" add_recipe(cuisines_recipe) %>% \n",
|
||||
" add_model(svm_rbf_spec)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Train an RBF model\n",
|
||||
"svm_rbf_fit <- svm_rbf_wf %>% \n",
|
||||
" fit(data = cuisines_train)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Make predictions and Evaluate model performance\n",
|
||||
"svm_rbf_fit %>% \n",
|
||||
" augment(new_data = cuisines_test) %>% \n",
|
||||
" eval_metrics(truth = cuisine, estimate = .pred_class)"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "QBFSa7WSh4HQ"
|
||||
},
|
||||
"source": [
|
||||
"Much better 🤩!\n",
|
||||
"\n",
|
||||
"> ✅ Please see:\n",
|
||||
">\n",
|
||||
"> - [*Support Vector Machines*](https://bradleyboehmke.github.io/HOML/svm.html), Hands-on Machine Learning with R\n",
|
||||
">\n",
|
||||
"> - [*Support Vector Machines*](https://www.statlearning.com/), An Introduction to Statistical Learning with Applications in R\n",
|
||||
">\n",
|
||||
"> for further reading.\n",
|
||||
"\n",
|
||||
"### Nearest Neighbor classifiers\n",
|
||||
"\n",
|
||||
"*K*-nearest neighbor (KNN) is an algorithm in which each observation is predicted based on its *similarity* to other observations.\n",
|
||||
"\n",
|
||||
"Let's fit one to our data."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "k4BxxBcdh9Ka"
|
||||
},
|
||||
"source": [
|
||||
"# Make a KNN specification\n",
|
||||
"knn_spec <- nearest_neighbor() %>% \n",
|
||||
" set_engine(\"kknn\") %>% \n",
|
||||
" set_mode(\"classification\")\n",
|
||||
"\n",
|
||||
"# Bundle recipe and model specification into a workflow\n",
|
||||
"knn_wf <- workflow() %>% \n",
|
||||
" add_recipe(cuisines_recipe) %>% \n",
|
||||
" add_model(knn_spec)\n",
|
||||
"\n",
|
||||
"# Train a boosted tree model\n",
|
||||
"knn_wf_fit <- knn_wf %>% \n",
|
||||
" fit(data = cuisines_train)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Make predictions and Evaluate model performance\n",
|
||||
"knn_wf_fit %>% \n",
|
||||
" augment(new_data = cuisines_test) %>% \n",
|
||||
" eval_metrics(truth = cuisine, estimate = .pred_class)"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "HaegQseriAcj"
|
||||
},
|
||||
"source": [
|
||||
"It appears that this model is not performing that well. Probably changing the model's arguments (see `help(\"nearest_neighbor\")` will improve model performance. Be sure to try it out.\n",
|
||||
"\n",
|
||||
"> ✅ Please see:\n",
|
||||
">\n",
|
||||
"> - [Hands-on Machine Learning with R](https://bradleyboehmke.github.io/HOML/)\n",
|
||||
">\n",
|
||||
"> - [An Introduction to Statistical Learning with Applications in R](https://www.statlearning.com/)\n",
|
||||
">\n",
|
||||
"> to learn more about *K*-Nearest Neighbors classifiers.\n",
|
||||
"\n",
|
||||
"### Ensemble classifiers\n",
|
||||
"\n",
|
||||
"Ensemble algorithms work by combining multiple base estimators to produce an optimal model either by:\n",
|
||||
"\n",
|
||||
"`bagging`: applying an *averaging function* to a collection of base models\n",
|
||||
"\n",
|
||||
"`boosting`: building a sequence of models that build on one another to improve predictive performance.\n",
|
||||
"\n",
|
||||
"Let's start by trying out a Random Forest model, which builds a large collection of decision trees then applies an averaging function to for a better overall model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "49DPoVs6iK1M"
|
||||
},
|
||||
"source": [
|
||||
"# Make a random forest specification\n",
|
||||
"rf_spec <- rand_forest() %>% \n",
|
||||
" set_engine(\"ranger\") %>% \n",
|
||||
" set_mode(\"classification\")\n",
|
||||
"\n",
|
||||
"# Bundle recipe and model specification into a workflow\n",
|
||||
"rf_wf <- workflow() %>% \n",
|
||||
" add_recipe(cuisines_recipe) %>% \n",
|
||||
" add_model(rf_spec)\n",
|
||||
"\n",
|
||||
"# Train a random forest model\n",
|
||||
"rf_wf_fit <- rf_wf %>% \n",
|
||||
" fit(data = cuisines_train)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Make predictions and Evaluate model performance\n",
|
||||
"rf_wf_fit %>% \n",
|
||||
" augment(new_data = cuisines_test) %>% \n",
|
||||
" eval_metrics(truth = cuisine, estimate = .pred_class)"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "RGVYwC_aiUWc"
|
||||
},
|
||||
"source": [
|
||||
"Good job 👏!\n",
|
||||
"\n",
|
||||
"Let's also experiment with a Boosted Tree model.\n",
|
||||
"\n",
|
||||
"Boosted Tree defines an ensemble method that creates a series of sequential decision trees where each tree depends on the results of previous trees in an attempt to incrementally reduce the error. It focuses on the weights of incorrectly classified items and adjusts the fit for the next classifier to correct.\n",
|
||||
"\n",
|
||||
"There are different ways to fit this model (see `help(\"boost_tree\")`). In this example, we'll fit Boosted trees via `xgboost` engine."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "Py1YWo-micWs"
|
||||
},
|
||||
"source": [
|
||||
"# Make a boosted tree specification\n",
|
||||
"boost_spec <- boost_tree(trees = 200) %>% \n",
|
||||
" set_engine(\"xgboost\") %>% \n",
|
||||
" set_mode(\"classification\")\n",
|
||||
"\n",
|
||||
"# Bundle recipe and model specification into a workflow\n",
|
||||
"boost_wf <- workflow() %>% \n",
|
||||
" add_recipe(cuisines_recipe) %>% \n",
|
||||
" add_model(boost_spec)\n",
|
||||
"\n",
|
||||
"# Train a boosted tree model\n",
|
||||
"boost_wf_fit <- boost_wf %>% \n",
|
||||
" fit(data = cuisines_train)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Make predictions and Evaluate model performance\n",
|
||||
"boost_wf_fit %>% \n",
|
||||
" augment(new_data = cuisines_test) %>% \n",
|
||||
" eval_metrics(truth = cuisine, estimate = .pred_class)"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "zNQnbuejigZM"
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"> ✅ Please see:\n",
|
||||
">\n",
|
||||
"> - [Machine Learning for Social Scientists](https://cimentadaj.github.io/ml_socsci/tree-based-methods.html#random-forests)\n",
|
||||
">\n",
|
||||
"> - [Hands-on Machine Learning with R](https://bradleyboehmke.github.io/HOML/)\n",
|
||||
">\n",
|
||||
"> - [An Introduction to Statistical Learning with Applications in R](https://www.statlearning.com/)\n",
|
||||
">\n",
|
||||
"> - <https://algotech.netlify.app/blog/xgboost/> - Explores the AdaBoost model which is a good alternative to xgboost.\n",
|
||||
">\n",
|
||||
"> to learn more about Ensemble classifiers.\n",
|
||||
"\n",
|
||||
"## 4. Extra - comparing multiple models\n",
|
||||
"\n",
|
||||
"We have fitted quite a number of models in this lab 🙌. It can become tedious or onerous to create a lot of workflows from different sets of preprocessors and/or model specifications and then calculate the performance metrics one by one.\n",
|
||||
"\n",
|
||||
"Let's see if we can address this by creating a function that fits a list of workflows on the training set then returns the performance metrics based on the test set. We'll get to use `map()` and `map_dfr()` from the [purrr](https://purrr.tidyverse.org/) package to apply functions to each element in list.\n",
|
||||
"\n",
|
||||
"> [`map()`](https://purrr.tidyverse.org/reference/map.html) functions allow you to replace many for loops with code that is both more succinct and easier to read. The best place to learn about the [`map()`](https://purrr.tidyverse.org/reference/map.html) functions is the [iteration chapter](http://r4ds.had.co.nz/iteration.html) in R for data science."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "Qzb7LyZnimd2"
|
||||
},
|
||||
"source": [
|
||||
"set.seed(2056)\n",
|
||||
"\n",
|
||||
"# Create a metric set\n",
|
||||
"eval_metrics <- metric_set(ppv, sens, accuracy, f_meas)\n",
|
||||
"\n",
|
||||
"# Define a function that returns performance metrics\n",
|
||||
"compare_models <- function(workflow_list, train_set, test_set){\n",
|
||||
" \n",
|
||||
" suppressWarnings(\n",
|
||||
" # Fit each model to the train_set\n",
|
||||
" map(workflow_list, fit, data = train_set) %>% \n",
|
||||
" # Make predictions on the test set\n",
|
||||
" map_dfr(augment, new_data = test_set, .id = \"model\") %>%\n",
|
||||
" # Select desired columns\n",
|
||||
" select(model, cuisine, .pred_class) %>% \n",
|
||||
" # Evaluate model performance\n",
|
||||
" group_by(model) %>% \n",
|
||||
" eval_metrics(truth = cuisine, estimate = .pred_class) %>% \n",
|
||||
" ungroup()\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
"} # End of function"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Fwa712sNisDA"
|
||||
},
|
||||
"source": [
|
||||
"Let's call our function and compare the accuracy across the models."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"id": "3i4VJOi2iu-a"
|
||||
},
|
||||
"source": [
|
||||
"# Make a list of workflows\n",
|
||||
"workflow_list <- list(\n",
|
||||
" \"svc\" = svc_linear_wf,\n",
|
||||
" \"svm\" = svm_rbf_wf,\n",
|
||||
" \"knn\" = knn_wf,\n",
|
||||
" \"random_forest\" = rf_wf,\n",
|
||||
" \"xgboost\" = boost_wf)\n",
|
||||
"\n",
|
||||
"# Call the function\n",
|
||||
"set.seed(2056)\n",
|
||||
"perf_metrics <- compare_models(workflow_list = workflow_list, train_set = cuisines_train, test_set = cuisines_test)\n",
|
||||
"\n",
|
||||
"# Print out performance metrics\n",
|
||||
"perf_metrics %>% \n",
|
||||
" group_by(.metric) %>% \n",
|
||||
" arrange(desc(.estimate)) %>% \n",
|
||||
" slice_head(n=7)\n",
|
||||
"\n",
|
||||
"# Compare accuracy\n",
|
||||
"perf_metrics %>% \n",
|
||||
" filter(.metric == \"accuracy\") %>% \n",
|
||||
" arrange(desc(.estimate))\n"
|
||||
],
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KuWK_lEli4nW"
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"[**workflowset**](https://workflowsets.tidymodels.org/) package allow users to create and easily fit a large number of models but is mostly designed to work with resampling techniques such as `cross-validation`, an approach we are yet to cover.\n",
|
||||
"\n",
|
||||
"## **🚀Challenge**\n",
|
||||
"\n",
|
||||
"Each of these techniques has a large number of parameters that you can tweak for instance `cost` in SVMs, `neighbors` in KNN, `mtry` (Randomly Selected Predictors) in Random Forest.\n",
|
||||
"\n",
|
||||
"Research each one's default parameters and think about what tweaking these parameters would mean for the model's quality.\n",
|
||||
"\n",
|
||||
"To find out more about a particular model and its parameters, use: `help(\"model\")` e.g `help(\"rand_forest\")`\n",
|
||||
"\n",
|
||||
"> In practice, we usually *estimate* the *best values* for these by training many models on a `simulated data set` and measuring how well all these models perform. This process is called **tuning**.\n",
|
||||
"\n",
|
||||
"### [**Post-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/24/)\n",
|
||||
"\n",
|
||||
"### **Review & Self Study**\n",
|
||||
"\n",
|
||||
"There's a lot of jargon in these lessons, so take a minute to review [this list](https://docs.microsoft.com/dotnet/machine-learning/resources/glossary?WT.mc_id=academic-77952-leestott) of useful terminology!\n",
|
||||
"\n",
|
||||
"#### THANK YOU TO:\n",
|
||||
"\n",
|
||||
"[`Allison Horst`](https://twitter.com/allison_horst/) for creating the amazing illustrations that make R more welcoming and engaging. Find more illustrations at her [gallery](https://www.google.com/url?q=https://github.com/allisonhorst/stats-illustrations&sa=D&source=editors&ust=1626380772530000&usg=AOvVaw3zcfyCizFQZpkSLzxiiQEM).\n",
|
||||
"\n",
|
||||
"[Cassie Breviu](https://www.twitter.com/cassieview) and [Jen Looper](https://www.twitter.com/jenlooper) for creating the original Python version of this module ♥️\n",
|
||||
"\n",
|
||||
"Happy Learning,\n",
|
||||
"\n",
|
||||
"[Eric](https://twitter.com/ericntay), Gold Microsoft Learn Student Ambassador.\n",
|
||||
"\n",
|
||||
"<p >\n",
|
||||
" <img src=\"../../images/r_learners_sm.jpeg\"\n",
|
||||
" width=\"569\"/>\n",
|
||||
" <figcaption>Artwork by @allison_horst</figcaption>\n"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,452 @@
|
||||
---
|
||||
title: 'Build a classification model: Delicious Asian and Indian Cuisines'
|
||||
output:
|
||||
html_document:
|
||||
df_print: paged
|
||||
theme: flatly
|
||||
highlight: breezedark
|
||||
toc: yes
|
||||
toc_float: yes
|
||||
code_download: yes
|
||||
---
|
||||
|
||||
## Cuisine classifiers 2
|
||||
|
||||
In this second classification lesson, we will explore `more ways` to classify categorical data. We will also learn about the ramifications for choosing one classifier over the other.
|
||||
|
||||
### [**Pre-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/23/)
|
||||
|
||||
### **Prerequisite**
|
||||
|
||||
We assume that you have completed the previous lessons since we will be carrying forward some concepts we learned before.
|
||||
|
||||
For this lesson, we'll require the following packages:
|
||||
|
||||
- `tidyverse`: The [tidyverse](https://www.tidyverse.org/) is a [collection of R packages](https://www.tidyverse.org/packages) designed to makes data science faster, easier and more fun!
|
||||
|
||||
- `tidymodels`: The [tidymodels](https://www.tidymodels.org/) framework is a [collection of packages](https://www.tidymodels.org/packages/) for modeling and machine learning.
|
||||
|
||||
- `themis`: The [themis package](https://themis.tidymodels.org/) provides Extra Recipes Steps for Dealing with Unbalanced Data.
|
||||
|
||||
You can have them installed as:
|
||||
|
||||
`install.packages(c("tidyverse", "tidymodels", "kernlab", "themis", "ranger", "xgboost", "kknn"))`
|
||||
|
||||
Alternatively, the script below checks whether you have the packages required to complete this module and installs them for you in case they are missing.
|
||||
|
||||
```{r, message=F, warning=F}
|
||||
suppressWarnings(if (!require("pacman"))install.packages("pacman"))
|
||||
|
||||
pacman::p_load(tidyverse, tidymodels, themis, kernlab, ranger, xgboost, kknn)
|
||||
```
|
||||
|
||||
Now, let's hit the ground running!
|
||||
|
||||
## **1. A classification map**
|
||||
|
||||
In our [previous lesson](https://github.com/microsoft/ML-For-Beginners/tree/main/4-Classification/2-Classifiers-1), we tried to address the question: how do we choose between multiple models? To a great extent, it depends on the characteristics of the data and the type of problem we want to solve (for instance classification or regression?)
|
||||
|
||||
Previously, we learned about the various options you have when classifying data using Microsoft's cheat sheet. Python's Machine Learning framework, Scikit-learn, offers a similar but more granular cheat sheet that can further help narrow down your estimators (another term for classifiers):
|
||||
|
||||
{width="650"}\
|
||||
|
||||
> Tip: [visit this map online](https://scikit-learn.org/stable/tutorial/machine_learning_map/) and click along the path to read documentation.
|
||||
>
|
||||
> The [Tidymodels reference site](https://www.tidymodels.org/find/parsnip/#models) also provides an excellent documentation about different types of model.
|
||||
|
||||
### **The plan** 🗺️
|
||||
|
||||
This map is very helpful once you have a clear grasp of your data, as you can 'walk' along its paths to a decision:
|
||||
|
||||
- We have \>50 samples
|
||||
|
||||
- We want to predict a category
|
||||
|
||||
- We have labeled data
|
||||
|
||||
- We have fewer than 100K samples
|
||||
|
||||
- ✨ We can choose a Linear SVC
|
||||
|
||||
- If that doesn't work, since we have numeric data
|
||||
|
||||
- We can try a ✨ KNeighbors Classifier
|
||||
|
||||
- If that doesn't work, try ✨ SVC and ✨ Ensemble Classifiers
|
||||
|
||||
This is a very helpful trail to follow. Now, let's get right into it using the [tidymodels](https://www.tidymodels.org/) modelling framework: a consistent and flexible collection of R packages developed to encourage good statistical practice 😊.
|
||||
|
||||
## 2. Split the data and deal with imbalanced data set.
|
||||
|
||||
From our previous lessons, we learnt that there were a set of common ingredients across our cuisines. Also, there was quite an unequal distribution in the number of cuisines.
|
||||
|
||||
We'll deal with these by
|
||||
|
||||
- Dropping the most common ingredients that create confusion between distinct cuisines, using `dplyr::select()`.
|
||||
|
||||
- Use a `recipe` that preprocesses the data to get it ready for modelling by applying an `over-sampling` algorithm.
|
||||
|
||||
We already looked at the above in the previous lesson so this should be a breeze 🥳!
|
||||
|
||||
```{r clean_imbalance}
|
||||
# Load the core Tidyverse and Tidymodels packages
|
||||
library(tidyverse)
|
||||
library(tidymodels)
|
||||
|
||||
# Load the original cuisines data
|
||||
df <- read_csv(file = "https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/4-Classification/data/cuisines.csv")
|
||||
|
||||
# Drop id column, rice, garlic and ginger from our original data set
|
||||
df_select <- df %>%
|
||||
select(-c(1, rice, garlic, ginger)) %>%
|
||||
# Encode cuisine column as categorical
|
||||
mutate(cuisine = factor(cuisine))
|
||||
|
||||
|
||||
# Create data split specification
|
||||
set.seed(2056)
|
||||
cuisines_split <- initial_split(data = df_select,
|
||||
strata = cuisine,
|
||||
prop = 0.7)
|
||||
|
||||
# Extract the data in each split
|
||||
cuisines_train <- training(cuisines_split)
|
||||
cuisines_test <- testing(cuisines_split)
|
||||
|
||||
# Display distribution of cuisines in the training set
|
||||
cuisines_train %>%
|
||||
count(cuisine) %>%
|
||||
arrange(desc(n))
|
||||
|
||||
|
||||
```
|
||||
|
||||
### Deal with imbalanced data
|
||||
|
||||
Imbalanced data often has negative effects on the model performance. Many models perform best when the number of observations is equal and, thus, tend to struggle with unbalanced data.
|
||||
|
||||
There are majorly two ways of dealing with imbalanced data sets:
|
||||
|
||||
- adding observations to the minority class: `Over-sampling` e.g using a SMOTE algorithm which synthetically generates new examples of the minority class using nearest neighbors of these cases.
|
||||
|
||||
- removing observations from majority class: `Under-sampling`
|
||||
|
||||
In our previous lesson, we demonstrated how to deal with imbalanced data sets using a `recipe`. A recipe can be thought of as a blueprint that describes what steps should be applied to a data set in order to get it ready for data analysis. In our case, we want to have an equal distribution in the number of our cuisines for our `training set`. Let's get right into it.
|
||||
|
||||
```{r recap_balance}
|
||||
# Load themis package for dealing with imbalanced data
|
||||
library(themis)
|
||||
|
||||
# Create a recipe for preprocessing training data
|
||||
cuisines_recipe <- recipe(cuisine ~ ., data = cuisines_train) %>%
|
||||
step_smote(cuisine)
|
||||
|
||||
# Print recipe
|
||||
cuisines_recipe
|
||||
|
||||
```
|
||||
|
||||
Now we are ready to train models 👩💻👨💻!
|
||||
|
||||
## 3. Beyond multinomial regression models
|
||||
|
||||
In our previous lesson, we looked at multinomial regression models. Let's explore some more flexible models for classification.
|
||||
|
||||
### Support Vector Machines.
|
||||
|
||||
In the context of classification, `Support Vector Machines` is a machine learning technique that tries to find a *hyperplane* that "best" separates the classes. Let's look at a simple example:
|
||||
|
||||
{width="300"}
|
||||
|
||||
H~1~ does not separate the classes. H~2~ does, but only with a small margin. H~3~ separates them with the maximal margin.
|
||||
|
||||
#### Linear Support Vector Classifier
|
||||
|
||||
Support-Vector clustering (SVC) is a child of the Support-Vector machines family of ML techniques. In SVC, the hyperplane is chosen to correctly separate `most` of the training observations, but `may misclassify` a few observations. By allowing some points to be on the wrong side, the SVM becomes more robust to outliers hence better generalization to new data. The parameter that regulates this violation is referred to as `cost` which has a default value of 1 (see `help("svm_poly")`).
|
||||
|
||||
Let's create a linear SVC by setting `degree = 1` in a polynomial SVM model.
|
||||
|
||||
```{r svc_spec}
|
||||
# Make a linear SVC specification
|
||||
svc_linear_spec <- svm_poly(degree = 1) %>%
|
||||
set_engine("kernlab") %>%
|
||||
set_mode("classification")
|
||||
|
||||
# Bundle specification and recipe into a worklow
|
||||
svc_linear_wf <- workflow() %>%
|
||||
add_recipe(cuisines_recipe) %>%
|
||||
add_model(svc_linear_spec)
|
||||
|
||||
# Print out workflow
|
||||
svc_linear_wf
|
||||
```
|
||||
|
||||
Now that we have captured the preprocessing steps and model specification into a *workflow*, we can go ahead and train the linear SVC and evaluate results while at it. For performance metrics, let's create a metric set that will evaluate: `accuracy`, `sensitivity`, `Positive Predicted Value` and `F Measure`
|
||||
|
||||
> `augment()` will add column(s) for predictions to the given data.
|
||||
|
||||
```{r svc_train}
|
||||
# Train a linear SVC model
|
||||
svc_linear_fit <- svc_linear_wf %>%
|
||||
fit(data = cuisines_train)
|
||||
|
||||
# Create a metric set
|
||||
eval_metrics <- metric_set(ppv, sens, accuracy, f_meas)
|
||||
|
||||
|
||||
# Make predictions and Evaluate model performance
|
||||
svc_linear_fit %>%
|
||||
augment(new_data = cuisines_test) %>%
|
||||
eval_metrics(truth = cuisine, estimate = .pred_class)
|
||||
|
||||
|
||||
|
||||
```
|
||||
|
||||
####
|
||||
|
||||
#### Support Vector Machine
|
||||
|
||||
The support vector machine (SVM) is an extension of the support vector classifier in order to accommodate a non-linear boundary between the classes. In essence, SVMs use the *kernel trick* to enlarge the feature space to adapt to nonlinear relationships between classes. One popular and extremely flexible kernel function used by SVMs is the *Radial basis function.* Let's see how it will perform on our data.
|
||||
|
||||
```{r svm_rbf}
|
||||
set.seed(2056)
|
||||
|
||||
# Make an RBF SVM specification
|
||||
svm_rbf_spec <- svm_rbf() %>%
|
||||
set_engine("kernlab") %>%
|
||||
set_mode("classification")
|
||||
|
||||
# Bundle specification and recipe into a worklow
|
||||
svm_rbf_wf <- workflow() %>%
|
||||
add_recipe(cuisines_recipe) %>%
|
||||
add_model(svm_rbf_spec)
|
||||
|
||||
|
||||
# Train an RBF model
|
||||
svm_rbf_fit <- svm_rbf_wf %>%
|
||||
fit(data = cuisines_train)
|
||||
|
||||
|
||||
# Make predictions and Evaluate model performance
|
||||
svm_rbf_fit %>%
|
||||
augment(new_data = cuisines_test) %>%
|
||||
eval_metrics(truth = cuisine, estimate = .pred_class)
|
||||
```
|
||||
|
||||
Much better 🤩!
|
||||
|
||||
> ✅ Please see:
|
||||
>
|
||||
> - [*Support Vector Machines*](https://bradleyboehmke.github.io/HOML/svm.html), Hands-on Machine Learning with R
|
||||
>
|
||||
> - [*Support Vector Machines*](https://www.statlearning.com/), An Introduction to Statistical Learning with Applications in R
|
||||
>
|
||||
> for further reading.
|
||||
|
||||
### Nearest Neighbor classifiers
|
||||
|
||||
*K*-nearest neighbor (KNN) is an algorithm in which each observation is predicted based on its *similarity* to other observations.
|
||||
|
||||
Let's fit one to our data.
|
||||
|
||||
```{r knn}
|
||||
# Make a KNN specification
|
||||
knn_spec <- nearest_neighbor() %>%
|
||||
set_engine("kknn") %>%
|
||||
set_mode("classification")
|
||||
|
||||
# Bundle recipe and model specification into a workflow
|
||||
knn_wf <- workflow() %>%
|
||||
add_recipe(cuisines_recipe) %>%
|
||||
add_model(knn_spec)
|
||||
|
||||
# Train a boosted tree model
|
||||
knn_wf_fit <- knn_wf %>%
|
||||
fit(data = cuisines_train)
|
||||
|
||||
|
||||
# Make predictions and Evaluate model performance
|
||||
knn_wf_fit %>%
|
||||
augment(new_data = cuisines_test) %>%
|
||||
eval_metrics(truth = cuisine, estimate = .pred_class)
|
||||
```
|
||||
|
||||
It appears that this model is not performing that well. Probably changing the model's arguments (see `help("nearest_neighbor")` will improve model performance. Be sure to try it out.
|
||||
|
||||
> ✅ Please see:
|
||||
>
|
||||
> - [Hands-on Machine Learning with R](https://bradleyboehmke.github.io/HOML/)
|
||||
>
|
||||
> - [An Introduction to Statistical Learning with Applications in R](https://www.statlearning.com/)
|
||||
>
|
||||
> to learn more about *K*-Nearest Neighbors classifiers.
|
||||
|
||||
### Ensemble classifiers
|
||||
|
||||
Ensemble algorithms work by combining multiple base estimators to produce an optimal model either by:
|
||||
|
||||
`bagging`: applying an *averaging function* to a collection of base models
|
||||
|
||||
`boosting`: building a sequence of models that build on one another to improve predictive performance.
|
||||
|
||||
Let's start by trying out a Random Forest model, which builds a large collection of decision trees then applies an averaging function to for a better overall model.
|
||||
|
||||
```{r rf}
|
||||
# Make a random forest specification
|
||||
rf_spec <- rand_forest() %>%
|
||||
set_engine("ranger") %>%
|
||||
set_mode("classification")
|
||||
|
||||
# Bundle recipe and model specification into a workflow
|
||||
rf_wf <- workflow() %>%
|
||||
add_recipe(cuisines_recipe) %>%
|
||||
add_model(rf_spec)
|
||||
|
||||
# Train a random forest model
|
||||
rf_wf_fit <- rf_wf %>%
|
||||
fit(data = cuisines_train)
|
||||
|
||||
|
||||
# Make predictions and Evaluate model performance
|
||||
rf_wf_fit %>%
|
||||
augment(new_data = cuisines_test) %>%
|
||||
eval_metrics(truth = cuisine, estimate = .pred_class)
|
||||
|
||||
|
||||
```
|
||||
|
||||
Good job 👏!
|
||||
|
||||
Let's also experiment with a Boosted Tree model.
|
||||
|
||||
Boosted Tree defines an ensemble method that creates a series of sequential decision trees where each tree depends on the results of previous trees in an attempt to incrementally reduce the error. It focuses on the weights of incorrectly classified items and adjusts the fit for the next classifier to correct.
|
||||
|
||||
There are different ways to fit this model (see `help("boost_tree")`). In this example, we'll fit Boosted trees via `xgboost` engine.
|
||||
|
||||
```{r boosted_tree}
|
||||
# Make a boosted tree specification
|
||||
boost_spec <- boost_tree(trees = 200) %>%
|
||||
set_engine("xgboost") %>%
|
||||
set_mode("classification")
|
||||
|
||||
# Bundle recipe and model specification into a workflow
|
||||
boost_wf <- workflow() %>%
|
||||
add_recipe(cuisines_recipe) %>%
|
||||
add_model(boost_spec)
|
||||
|
||||
# Train a boosted tree model
|
||||
boost_wf_fit <- boost_wf %>%
|
||||
fit(data = cuisines_train)
|
||||
|
||||
|
||||
# Make predictions and Evaluate model performance
|
||||
boost_wf_fit %>%
|
||||
augment(new_data = cuisines_test) %>%
|
||||
eval_metrics(truth = cuisine, estimate = .pred_class)
|
||||
```
|
||||
|
||||
> ✅ Please see:
|
||||
>
|
||||
> - [Machine Learning for Social Scientists](https://cimentadaj.github.io/ml_socsci/tree-based-methods.html#random-forests)
|
||||
>
|
||||
> - [Hands-on Machine Learning with R](https://bradleyboehmke.github.io/HOML/)
|
||||
>
|
||||
> - [An Introduction to Statistical Learning with Applications in R](https://www.statlearning.com/)
|
||||
>
|
||||
> - <https://algotech.netlify.app/blog/xgboost/> - Explores the AdaBoost model which is a good alternative to xgboost.
|
||||
>
|
||||
> to learn more about Ensemble classifiers.
|
||||
|
||||
## 4. Extra - comparing multiple models
|
||||
|
||||
We have fitted quite a number of models in this lab 🙌. It can become tedious or onerous to create a lot of workflows from different sets of preprocessors and/or model specifications and then calculate the performance metrics one by one.
|
||||
|
||||
Let's see if we can address this by creating a function that fits a list of workflows on the training set then returns the performance metrics based on the test set. We'll get to use `map()` and `map_dfr()` from the [purrr](https://purrr.tidyverse.org/) package to apply functions to each element in list.
|
||||
|
||||
> [`map()`](https://purrr.tidyverse.org/reference/map.html) functions allow you to replace many for loops with code that is both more succinct and easier to read. The best place to learn about the [`map()`](https://purrr.tidyverse.org/reference/map.html) functions is the [iteration chapter](http://r4ds.had.co.nz/iteration.html) in R for data science.
|
||||
|
||||
```{r compare_models}
|
||||
set.seed(2056)
|
||||
|
||||
# Create a metric set
|
||||
eval_metrics <- metric_set(ppv, sens, accuracy, f_meas)
|
||||
|
||||
# Define a function that returns performance metrics
|
||||
compare_models <- function(workflow_list, train_set, test_set){
|
||||
|
||||
suppressWarnings(
|
||||
# Fit each model to the train_set
|
||||
map(workflow_list, fit, data = train_set) %>%
|
||||
# Make predictions on the test set
|
||||
map_dfr(augment, new_data = test_set, .id = "model") %>%
|
||||
# Select desired columns
|
||||
select(model, cuisine, .pred_class) %>%
|
||||
# Evaluate model performance
|
||||
group_by(model) %>%
|
||||
eval_metrics(truth = cuisine, estimate = .pred_class) %>%
|
||||
ungroup()
|
||||
)
|
||||
|
||||
} # End of function
|
||||
|
||||
|
||||
```
|
||||
|
||||
Let's call our function and compare the accuracy across the models.
|
||||
|
||||
```{r call_fn}
|
||||
# Make a list of workflows
|
||||
workflow_list <- list(
|
||||
"svc" = svc_linear_wf,
|
||||
"svm" = svm_rbf_wf,
|
||||
"knn" = knn_wf,
|
||||
"random_forest" = rf_wf,
|
||||
"xgboost" = boost_wf)
|
||||
|
||||
# Call the function
|
||||
set.seed(2056)
|
||||
perf_metrics <- compare_models(workflow_list = workflow_list, train_set = cuisines_train, test_set = cuisines_test)
|
||||
|
||||
# Print out performance metrics
|
||||
perf_metrics %>%
|
||||
group_by(.metric) %>%
|
||||
arrange(desc(.estimate)) %>%
|
||||
slice_head(n=7)
|
||||
|
||||
# Compare accuracy
|
||||
perf_metrics %>%
|
||||
filter(.metric == "accuracy") %>%
|
||||
arrange(desc(.estimate))
|
||||
|
||||
```
|
||||
|
||||
[**workflowset**](https://workflowsets.tidymodels.org/) package allow users to create and easily fit a large number of models but is mostly designed to work with resampling techniques such as `cross-validation`, an approach we are yet to cover.
|
||||
|
||||
## **🚀Challenge**
|
||||
|
||||
Each of these techniques has a large number of parameters that you can tweak for instance `cost` in SVMs, `neighbors` in KNN, `mtry` (Randomly Selected Predictors) in Random Forest.
|
||||
|
||||
Research each one's default parameters and think about what tweaking these parameters would mean for the model's quality.
|
||||
|
||||
To find out more about a particular model and its parameters, use: `help("model")` e.g `help("rand_forest")`
|
||||
|
||||
> In practice, we usually *estimate* the *best values* for these by training many models on a `simulated data set` and measuring how well all these models perform. This process is called **tuning**.
|
||||
|
||||
### [**Post-lecture quiz**](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/24/)
|
||||
|
||||
### **Review & Self Study**
|
||||
|
||||
There's a lot of jargon in these lessons, so take a minute to review [this list](https://docs.microsoft.com/dotnet/machine-learning/resources/glossary?WT.mc_id=academic-77952-leestott) of useful terminology!
|
||||
|
||||
#### THANK YOU TO:
|
||||
|
||||
[`Allison Horst`](https://twitter.com/allison_horst/) for creating the amazing illustrations that make R more welcoming and engaging. Find more illustrations at her [gallery](https://www.google.com/url?q=https://github.com/allisonhorst/stats-illustrations&sa=D&source=editors&ust=1626380772530000&usg=AOvVaw3zcfyCizFQZpkSLzxiiQEM).
|
||||
|
||||
[Cassie Breviu](https://www.twitter.com/cassieview) and [Jen Looper](https://www.twitter.com/jenlooper) for creating the original Python version of this module ♥️
|
||||
|
||||
Happy Learning,
|
||||
|
||||
[Eric](https://twitter.com/ericntay), Gold Microsoft Learn Student Ambassador.
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,268 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"source": [
|
||||
"# Build More Classification Models"
|
||||
],
|
||||
"cell_type": "markdown",
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Dataset Overview\n",
|
||||
"This dataset contains individual samples (for example, recipes) labeled by cuisine.\n",
|
||||
"Each row corresponds to a single sample/record, and the columns represent ingredients or other attributes used for classification, including the `cuisine` label."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"# Load dataset containing cuisine features\n",
|
||||
"cuisines_df = pd.read_csv(\"../../data/cleaned_cuisines.csv\")\n",
|
||||
"cuisines_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0 indian\n",
|
||||
"1 indian\n",
|
||||
"2 indian\n",
|
||||
"3 indian\n",
|
||||
"4 indian\n",
|
||||
"Name: cuisine, dtype: object"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 2
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"cuisines_label_df = cuisines_df['cuisine']\n",
|
||||
"cuisines_label_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" almond angelica anise anise_seed apple apple_brandy apricot \\\n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 1 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 0 0 \n",
|
||||
"\n",
|
||||
" armagnac artemisia artichoke ... whiskey white_bread white_wine \\\n",
|
||||
"0 0 0 0 ... 0 0 0 \n",
|
||||
"1 0 0 0 ... 0 0 0 \n",
|
||||
"2 0 0 0 ... 0 0 0 \n",
|
||||
"3 0 0 0 ... 0 0 0 \n",
|
||||
"4 0 0 0 ... 0 0 0 \n",
|
||||
"\n",
|
||||
" whole_grain_wheat_flour wine wood yam yeast yogurt zucchini \n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 0 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 1 0 \n",
|
||||
"\n",
|
||||
"[5 rows x 380 columns]"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>almond</th>\n <th>angelica</th>\n <th>anise</th>\n <th>anise_seed</th>\n <th>apple</th>\n <th>apple_brandy</th>\n <th>apricot</th>\n <th>armagnac</th>\n <th>artemisia</th>\n <th>artichoke</th>\n <th>...</th>\n <th>whiskey</th>\n <th>white_bread</th>\n <th>white_wine</th>\n <th>whole_grain_wheat_flour</th>\n <th>wine</th>\n <th>wood</th>\n <th>yam</th>\n <th>yeast</th>\n <th>yogurt</th>\n <th>zucchini</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>1</td>\n <td>0</td>\n </tr>\n </tbody>\n</table>\n<p>5 rows × 380 columns</p>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"cuisines_features_df = cuisines_df.drop(['Unnamed: 0', 'cuisine'], axis=1)\n",
|
||||
"cuisines_features_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Try different classifiers"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.neighbors import KNeighborsClassifier\n",
|
||||
"from sklearn.linear_model import LogisticRegression\n",
|
||||
"from sklearn.svm import SVC\n",
|
||||
"from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier\n",
|
||||
"from sklearn.model_selection import train_test_split, cross_val_score\n",
|
||||
"from sklearn.metrics import accuracy_score,precision_score,confusion_matrix,classification_report, precision_recall_curve\n",
|
||||
"import numpy as np"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"X_train, X_test, y_train, y_test = train_test_split(cuisines_features_df, cuisines_label_df, test_size=0.3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\n",
|
||||
"C = 10\n",
|
||||
"# Create different classifiers.\n",
|
||||
"classifiers = {\n",
|
||||
" 'Linear SVC': SVC(kernel='linear', C=C, probability=True,random_state=0),\n",
|
||||
" 'KNN classifier': KNeighborsClassifier(C),\n",
|
||||
" 'SVC': SVC(),\n",
|
||||
" 'RFST': RandomForestClassifier(n_estimators=100),\n",
|
||||
" 'ADA': AdaBoostClassifier(n_estimators=100)\n",
|
||||
" \n",
|
||||
"}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Accuracy (train) for Linear SVC: 76.4% \n",
|
||||
" precision recall f1-score support\n",
|
||||
"\n",
|
||||
" chinese 0.64 0.66 0.65 242\n",
|
||||
" indian 0.91 0.86 0.89 236\n",
|
||||
" japanese 0.72 0.73 0.73 245\n",
|
||||
" korean 0.83 0.75 0.79 234\n",
|
||||
" thai 0.75 0.82 0.78 242\n",
|
||||
"\n",
|
||||
" accuracy 0.76 1199\n",
|
||||
" macro avg 0.77 0.76 0.77 1199\n",
|
||||
"weighted avg 0.77 0.76 0.77 1199\n",
|
||||
"\n",
|
||||
"Accuracy (train) for KNN classifier: 70.7% \n",
|
||||
" precision recall f1-score support\n",
|
||||
"\n",
|
||||
" chinese 0.65 0.63 0.64 242\n",
|
||||
" indian 0.84 0.81 0.82 236\n",
|
||||
" japanese 0.60 0.81 0.69 245\n",
|
||||
" korean 0.89 0.53 0.67 234\n",
|
||||
" thai 0.69 0.75 0.72 242\n",
|
||||
"\n",
|
||||
" accuracy 0.71 1199\n",
|
||||
" macro avg 0.73 0.71 0.71 1199\n",
|
||||
"weighted avg 0.73 0.71 0.71 1199\n",
|
||||
"\n",
|
||||
"Accuracy (train) for SVC: 80.1% \n",
|
||||
" precision recall f1-score support\n",
|
||||
"\n",
|
||||
" chinese 0.71 0.69 0.70 242\n",
|
||||
" indian 0.92 0.92 0.92 236\n",
|
||||
" japanese 0.77 0.78 0.77 245\n",
|
||||
" korean 0.87 0.77 0.82 234\n",
|
||||
" thai 0.75 0.86 0.80 242\n",
|
||||
"\n",
|
||||
" accuracy 0.80 1199\n",
|
||||
" macro avg 0.80 0.80 0.80 1199\n",
|
||||
"weighted avg 0.80 0.80 0.80 1199\n",
|
||||
"\n",
|
||||
"Accuracy (train) for RFST: 82.8% \n",
|
||||
" precision recall f1-score support\n",
|
||||
"\n",
|
||||
" chinese 0.80 0.75 0.77 242\n",
|
||||
" indian 0.90 0.91 0.90 236\n",
|
||||
" japanese 0.82 0.78 0.80 245\n",
|
||||
" korean 0.85 0.82 0.83 234\n",
|
||||
" thai 0.78 0.89 0.83 242\n",
|
||||
"\n",
|
||||
" accuracy 0.83 1199\n",
|
||||
" macro avg 0.83 0.83 0.83 1199\n",
|
||||
"weighted avg 0.83 0.83 0.83 1199\n",
|
||||
"\n",
|
||||
"Accuracy (train) for ADA: 71.1% \n",
|
||||
" precision recall f1-score support\n",
|
||||
"\n",
|
||||
" chinese 0.60 0.57 0.58 242\n",
|
||||
" indian 0.87 0.84 0.86 236\n",
|
||||
" japanese 0.71 0.60 0.65 245\n",
|
||||
" korean 0.68 0.78 0.72 234\n",
|
||||
" thai 0.70 0.78 0.74 242\n",
|
||||
"\n",
|
||||
" accuracy 0.71 1199\n",
|
||||
" macro avg 0.71 0.71 0.71 1199\n",
|
||||
"weighted avg 0.71 0.71 0.71 1199\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"n_classifiers = len(classifiers)\n",
|
||||
"\n",
|
||||
"for index, (name, classifier) in enumerate(classifiers.items()):\n",
|
||||
" classifier.fit(X_train, np.ravel(y_train))\n",
|
||||
"\n",
|
||||
" y_pred = classifier.predict(X_test)\n",
|
||||
" accuracy = accuracy_score(y_test, y_pred)\n",
|
||||
" print(\"Accuracy (train) for %s: %0.1f%% \" % (name, accuracy * 100))\n",
|
||||
" print(classification_report(y_test,y_pred))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
},
|
||||
"kernelspec": {
|
||||
"name": "python3",
|
||||
"display_name": "Python 3.7.0 64-bit ('3.7')"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.0"
|
||||
},
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,314 @@
|
||||
# Build a Cuisine Recommender Web App
|
||||
|
||||
In this lesson, you will build a classification model using some of the techniques you have learned in previous lessons and with the delicious cuisine dataset used throughout this series. In addition, you will build a small web app to use a saved model, leveraging Onnx's web runtime.
|
||||
|
||||
One of the most useful practical uses of machine learning is building recommendation systems, and you can take the first step in that direction today!
|
||||
|
||||
[](https://youtu.be/17wdM9AHMfg "Applied ML")
|
||||
|
||||
> 🎥 Click the image above for a video: Jen Looper builds a web app using classified cuisine data
|
||||
|
||||
## [Pre-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
|
||||
In this lesson you will learn:
|
||||
|
||||
- How to build a model and save it as an Onnx model
|
||||
- How to use Netron to inspect the model
|
||||
- How to use your model in a web app for inference
|
||||
|
||||
## Build your model
|
||||
|
||||
Building applied ML systems is an important part of leveraging these technologies for your business systems. You can use models within your web applications (and thus use them in an offline context if needed) by using Onnx.
|
||||
|
||||
In a [previous lesson](../../3-Web-App/1-Web-App/README.md), you built a Regression model about UFO sightings, "pickled" it, and used it in a Flask app. While this architecture is very useful to know, it is a full-stack Python app, and your requirements may include the use of a JavaScript application.
|
||||
|
||||
In this lesson, you can build a basic JavaScript-based system for inference. First, however, you need to train a model and convert it for use with Onnx.
|
||||
|
||||
## Exercise - train classification model
|
||||
|
||||
First, train a classification model using the cleaned cuisines dataset we used.
|
||||
|
||||
1. Start by importing useful libraries:
|
||||
|
||||
```python
|
||||
!pip install skl2onnx
|
||||
import pandas as pd
|
||||
```
|
||||
|
||||
You need '[skl2onnx](https://onnx.ai/sklearn-onnx/)' to help convert your Scikit-learn model to Onnx format.
|
||||
|
||||
1. Then, work with your data in the same way you did in previous lessons, by reading a CSV file using `read_csv()`:
|
||||
|
||||
```python
|
||||
data = pd.read_csv('../data/cleaned_cuisines.csv')
|
||||
data.head()
|
||||
```
|
||||
|
||||
1. Remove the first two unnecessary columns and save the remaining data as 'X':
|
||||
|
||||
```python
|
||||
X = data.iloc[:,2:]
|
||||
X.head()
|
||||
```
|
||||
|
||||
1. Save the labels as 'y':
|
||||
|
||||
```python
|
||||
y = data[['cuisine']]
|
||||
y.head()
|
||||
|
||||
```
|
||||
|
||||
### Commence the training routine
|
||||
|
||||
We will use the 'SVC' library which has good accuracy.
|
||||
|
||||
1. Import the appropriate libraries from Scikit-learn:
|
||||
|
||||
```python
|
||||
from sklearn.model_selection import train_test_split
|
||||
from sklearn.svm import SVC
|
||||
from sklearn.model_selection import cross_val_score
|
||||
from sklearn.metrics import accuracy_score,precision_score,confusion_matrix,classification_report
|
||||
```
|
||||
|
||||
1. Separate training and test sets:
|
||||
|
||||
```python
|
||||
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3)
|
||||
```
|
||||
|
||||
1. Build an SVC Classification model as you did in the previous lesson:
|
||||
|
||||
```python
|
||||
model = SVC(kernel='linear', C=10, probability=True,random_state=0)
|
||||
model.fit(X_train,y_train.values.ravel())
|
||||
```
|
||||
|
||||
1. Now, test your model, calling `predict()`:
|
||||
|
||||
```python
|
||||
y_pred = model.predict(X_test)
|
||||
```
|
||||
|
||||
1. Print out a classification report to check the model's quality:
|
||||
|
||||
```python
|
||||
print(classification_report(y_test,y_pred))
|
||||
```
|
||||
|
||||
As we saw before, the accuracy is good:
|
||||
|
||||
```output
|
||||
precision recall f1-score support
|
||||
|
||||
chinese 0.72 0.69 0.70 257
|
||||
indian 0.91 0.87 0.89 243
|
||||
japanese 0.79 0.77 0.78 239
|
||||
korean 0.83 0.79 0.81 236
|
||||
thai 0.72 0.84 0.78 224
|
||||
|
||||
accuracy 0.79 1199
|
||||
macro avg 0.79 0.79 0.79 1199
|
||||
weighted avg 0.79 0.79 0.79 1199
|
||||
```
|
||||
|
||||
### Convert your model to Onnx
|
||||
|
||||
Make sure to do the conversion with the proper Tensor number. This dataset has 380 ingredients listed, so you need to notate that number in `FloatTensorType`:
|
||||
|
||||
1. Convert using a tensor number of 380.
|
||||
|
||||
```python
|
||||
from skl2onnx import convert_sklearn
|
||||
from skl2onnx.common.data_types import FloatTensorType
|
||||
|
||||
initial_type = [('float_input', FloatTensorType([None, 380]))]
|
||||
options = {id(model): {'nocl': True, 'zipmap': False}}
|
||||
```
|
||||
|
||||
1. Create the onx and store as a file **model.onnx**:
|
||||
|
||||
```python
|
||||
onx = convert_sklearn(model, initial_types=initial_type, options=options)
|
||||
with open("./model.onnx", "wb") as f:
|
||||
f.write(onx.SerializeToString())
|
||||
```
|
||||
|
||||
> Note, you can pass in [options](https://onnx.ai/sklearn-onnx/parameterized.html) in your conversion script. In this case, we passed in 'nocl' to be True and 'zipmap' to be False. Since this is a classification model, you have the option to remove ZipMap which produces a list of dictionaries (not necessary). `nocl` refers to class information being included in the model. Reduce your model's size by setting `nocl` to 'True'.
|
||||
|
||||
Running the entire notebook will now build an Onnx model and save it to this folder.
|
||||
|
||||
## View your model
|
||||
|
||||
Onnx models are not very visible in Visual Studio code, but there's a very good free software that many researchers use to visualize the model to ensure that it is properly built. Download [Netron](https://github.com/lutzroeder/Netron) and open your model.onnx file. You can see your simple model visualized, with its 380 inputs and classifier listed:
|
||||
|
||||
|
||||
|
||||
Netron is a helpful tool to view your models.
|
||||
|
||||
Now you are ready to use this neat model in a web app. Let's build an app that will come in handy when you look in your refrigerator and try to figure out which combination of your leftover ingredients you can use to cook a given cuisine, as determined by your model.
|
||||
|
||||
## Build a recommender web application
|
||||
|
||||
You can use your model directly in a web app. This architecture also allows you to run it locally and even offline if needed. Start by creating an `index.html` file in the same folder where you stored your `model.onnx` file.
|
||||
|
||||
1. In this file _index.html_, add the following markup:
|
||||
|
||||
```html
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<header>
|
||||
<title>Cuisine Matcher</title>
|
||||
</header>
|
||||
<body>
|
||||
...
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
|
||||
1. Now, working within the `body` tags, add a little markup to show a list of checkboxes reflecting some ingredients:
|
||||
|
||||
```html
|
||||
<h1>Check your refrigerator. What can you create?</h1>
|
||||
<div id="wrapper">
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="4" class="checkbox">
|
||||
<label>apple</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="247" class="checkbox">
|
||||
<label>pear</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="77" class="checkbox">
|
||||
<label>cherry</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="126" class="checkbox">
|
||||
<label>fenugreek</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="302" class="checkbox">
|
||||
<label>sake</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="327" class="checkbox">
|
||||
<label>soy sauce</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="112" class="checkbox">
|
||||
<label>cumin</label>
|
||||
</div>
|
||||
</div>
|
||||
<div style="padding-top:10px">
|
||||
<button onClick="startInference()">What kind of cuisine can you make?</button>
|
||||
</div>
|
||||
```
|
||||
|
||||
Notice that each checkbox is given a value. This reflects the index where the ingredient is found according to the dataset. Apple, for example, in this alphabetic list, occupies the fifth column, so its value is '4' since we start counting at 0. You can consult the [ingredients spreadsheet](../data/ingredient_indexes.csv) to discover a given ingredient's index.
|
||||
|
||||
Continuing your work in the index.html file, add a script block where the model is called after the final closing `</div>`.
|
||||
|
||||
1. First, import the [Onnx Runtime](https://www.onnxruntime.ai/):
|
||||
|
||||
```html
|
||||
<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web@1.9.0/dist/ort.min.js"></script>
|
||||
```
|
||||
|
||||
> Onnx Runtime is used to enable running your Onnx models across a wide range of hardware platforms, including optimizations and an API to use.
|
||||
|
||||
1. Once the Runtime is in place, you can call it:
|
||||
|
||||
```html
|
||||
<script>
|
||||
const ingredients = Array(380).fill(0);
|
||||
|
||||
const checks = [...document.querySelectorAll('.checkbox')];
|
||||
|
||||
checks.forEach(check => {
|
||||
check.addEventListener('change', function() {
|
||||
// toggle the state of the ingredient
|
||||
// based on the checkbox's value (1 or 0)
|
||||
ingredients[check.value] = check.checked ? 1 : 0;
|
||||
});
|
||||
});
|
||||
|
||||
function testCheckboxes() {
|
||||
// validate if at least one checkbox is checked
|
||||
return checks.some(check => check.checked);
|
||||
}
|
||||
|
||||
async function startInference() {
|
||||
|
||||
let atLeastOneChecked = testCheckboxes()
|
||||
|
||||
if (!atLeastOneChecked) {
|
||||
alert('Please select at least one ingredient.');
|
||||
return;
|
||||
}
|
||||
try {
|
||||
// create a new session and load the model.
|
||||
|
||||
const session = await ort.InferenceSession.create('./model.onnx');
|
||||
|
||||
const input = new ort.Tensor(new Float32Array(ingredients), [1, 380]);
|
||||
const feeds = { float_input: input };
|
||||
|
||||
// feed inputs and run
|
||||
const results = await session.run(feeds);
|
||||
|
||||
// read from results
|
||||
alert('You can enjoy ' + results.label.data[0] + ' cuisine today!')
|
||||
|
||||
} catch (e) {
|
||||
console.log(`failed to inference ONNX model`);
|
||||
console.error(e);
|
||||
}
|
||||
}
|
||||
|
||||
</script>
|
||||
```
|
||||
|
||||
In this code, there are several things happening:
|
||||
|
||||
1. You created an array of 380 possible values (1 or 0) to be set and sent to the model for inference, depending on whether an ingredient checkbox is checked.
|
||||
2. You created an array of checkboxes and a way to determine whether they were checked in an `init` function that is called when the application starts. When a checkbox is checked, the `ingredients` array is altered to reflect the chosen ingredient.
|
||||
3. You created a `testCheckboxes` function that checks whether any checkbox was checked.
|
||||
4. You use `startInference` function when the button is pressed and, if any checkbox is checked, you start inference.
|
||||
5. The inference routine includes:
|
||||
1. Setting up an asynchronous load of the model
|
||||
2. Creating a Tensor structure to send to the model
|
||||
3. Creating 'feeds' that reflects the `float_input` input that you created when training your model (you can use Netron to verify that name)
|
||||
4. Sending these 'feeds' to the model and waiting for a response
|
||||
|
||||
## Test your application
|
||||
|
||||
Open a terminal session in Visual Studio Code in the folder where your index.html file resides. Ensure that you have [http-server](https://www.npmjs.com/package/http-server) installed globally, and type `http-server` at the prompt. A localhost should open and you can view your web app. Check what cuisine is recommended based on various ingredients:
|
||||
|
||||
|
||||
|
||||
Congratulations, you have created a 'recommendation' web app with a few fields. Take some time to build out this system!
|
||||
## 🚀Challenge
|
||||
|
||||
Your web app is very minimal, so continue to build it out using ingredients and their indexes from the [ingredient_indexes](../data/ingredient_indexes.csv) data. What flavor combinations work to create a given national dish?
|
||||
|
||||
## [Post-lecture quiz](https://ff-quizzes.netlify.app/en/ml/)
|
||||
|
||||
## Review & Self Study
|
||||
|
||||
While this lesson just touched on the utility of creating a recommendation system for food ingredients, this area of ML applications is very rich in examples. Read some more about how these systems are built:
|
||||
|
||||
- https://www.sciencedirect.com/topics/computer-science/recommendation-engine
|
||||
- https://www.technologyreview.com/2014/08/25/171547/the-ultimate-challenge-for-recommendation-engines/
|
||||
- https://www.technologyreview.com/2015/03/23/168831/everything-is-a-recommendation/
|
||||
|
||||
## Assignment
|
||||
|
||||
[Build a new recommender](assignment.md)
|
||||
@@ -0,0 +1,11 @@
|
||||
# Build a recommender
|
||||
|
||||
## Instructions
|
||||
|
||||
Given your exercises in this lesson, you now know how to build JavaScript-based web app using Onnx Runtime and a converted Onnx model. Experiment with building a new recommender using data from these lessons or sourced elsewhere (give credit, please). You might create a pet recommender given various personality attributes, or a music genre recommender based on a person's mood. Be creative!
|
||||
|
||||
## Rubric
|
||||
|
||||
| Criteria | Exemplary | Adequate | Needs Improvement |
|
||||
| -------- | ---------------------------------------------------------------------- | ------------------------------------- | --------------------------------- |
|
||||
| | A web app and notebook are presented, both well documented and running | One of those two is missing or flawed | Both are either missing or flawed |
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 147 KiB |
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": 3
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2,
|
||||
"cells": [
|
||||
{
|
||||
"source": [
|
||||
"# Build a cuisine recommender"
|
||||
],
|
||||
"cell_type": "markdown",
|
||||
"metadata": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<header>
|
||||
<title>Cuisine Matcher</title>
|
||||
</header>
|
||||
<body>
|
||||
<h1>Check your refrigerator. What can you create?</h1>
|
||||
<div id="wrapper">
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="4" class="checkbox">
|
||||
<label>apple</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="247" class="checkbox">
|
||||
<label>pear</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="77" class="checkbox">
|
||||
<label>cherry</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="126" class="checkbox">
|
||||
<label>fenugreek</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="302" class="checkbox">
|
||||
<label>sake</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="327" class="checkbox">
|
||||
<label>soy sauce</label>
|
||||
</div>
|
||||
|
||||
<div class="boxCont">
|
||||
<input type="checkbox" value="112" class="checkbox">
|
||||
<label>cumin</label>
|
||||
</div>
|
||||
</div>
|
||||
<div style="padding-top:10px">
|
||||
<button onClick="startInference()">What kind of cuisine can you make?</button>
|
||||
</div>
|
||||
<!-- import ONNXRuntime Web from CDN -->
|
||||
<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web@1.9.0/dist/ort.min.js"></script>
|
||||
<script>
|
||||
const ingredients = Array(380).fill(0);
|
||||
|
||||
const checks = [...document.querySelectorAll('.checkbox')];
|
||||
|
||||
checks.forEach(check => {
|
||||
check.addEventListener('change', function() {
|
||||
// toggle the state of the ingredient
|
||||
// based on the checkbox's value (1 or 0)
|
||||
ingredients[check.value] = check.checked ? 1 : 0;
|
||||
});
|
||||
});
|
||||
|
||||
function testCheckboxes() {
|
||||
// validate if at least one checkbox is checked
|
||||
return checks.some(check => check.checked);
|
||||
}
|
||||
|
||||
async function startInference() {
|
||||
|
||||
let atLeastOneChecked = testCheckboxes()
|
||||
|
||||
if (!atLeastOneChecked) {
|
||||
alert('Please select at least one ingredient.');
|
||||
return;
|
||||
}
|
||||
try {
|
||||
// create a new session and load the model.
|
||||
|
||||
const session = await ort.InferenceSession.create('./model.onnx');
|
||||
|
||||
const input = new ort.Tensor(new Float32Array(ingredients), [1, 380]);
|
||||
const feeds = { float_input: input };
|
||||
|
||||
// feed inputs and run
|
||||
const results = await session.run(feeds);
|
||||
|
||||
// read from results
|
||||
alert('You can enjoy ' + results.label.data[0] + ' cuisine today!')
|
||||
|
||||
} catch (e) {
|
||||
console.log(`failed to inference ONNX model`);
|
||||
console.error(e);
|
||||
}
|
||||
}
|
||||
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
@@ -0,0 +1,279 @@
|
||||
{
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.0"
|
||||
},
|
||||
"orig_nbformat": 2,
|
||||
"kernelspec": {
|
||||
"name": "python3",
|
||||
"display_name": "Python 3.7.0 64-bit ('3.7')"
|
||||
},
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
}
|
||||
},
|
||||
"interpreter": {
|
||||
"hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2,
|
||||
"cells": [
|
||||
{
|
||||
"source": [
|
||||
"# Build a cuisine recommender"
|
||||
],
|
||||
"cell_type": "markdown",
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 58,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Requirement already satisfied: skl2onnx in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (1.8.0)\n",
|
||||
"Requirement already satisfied: protobuf in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from skl2onnx) (3.8.0)\n",
|
||||
"Requirement already satisfied: numpy>=1.15 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from skl2onnx) (1.19.2)\n",
|
||||
"Requirement already satisfied: onnx>=1.2.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from skl2onnx) (1.9.0)\n",
|
||||
"Requirement already satisfied: six in /Users/jenlooper/Library/Python/3.7/lib/python/site-packages (from skl2onnx) (1.12.0)\n",
|
||||
"Requirement already satisfied: onnxconverter-common<1.9,>=1.6.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from skl2onnx) (1.8.1)\n",
|
||||
"Requirement already satisfied: scikit-learn>=0.19 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from skl2onnx) (0.24.2)\n",
|
||||
"Requirement already satisfied: scipy>=1.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from skl2onnx) (1.4.1)\n",
|
||||
"Requirement already satisfied: setuptools in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from protobuf->skl2onnx) (45.1.0)\n",
|
||||
"Requirement already satisfied: typing-extensions>=3.6.2.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from onnx>=1.2.1->skl2onnx) (3.10.0.0)\n",
|
||||
"Requirement already satisfied: threadpoolctl>=2.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from scikit-learn>=0.19->skl2onnx) (2.1.0)\n",
|
||||
"Requirement already satisfied: joblib>=0.11 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from scikit-learn>=0.19->skl2onnx) (0.16.0)\n",
|
||||
"\u001b[33mWARNING: You are using pip version 20.2.3; however, version 21.1.2 is available.\n",
|
||||
"You should consider upgrading via the '/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 -m pip install --upgrade pip' command.\u001b[0m\n",
|
||||
"Note: you may need to restart the kernel to use updated packages.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!pip install skl2onnx"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" Unnamed: 0 cuisine almond angelica anise anise_seed apple \\\n",
|
||||
"0 0 indian 0 0 0 0 0 \n",
|
||||
"1 1 indian 1 0 0 0 0 \n",
|
||||
"2 2 indian 0 0 0 0 0 \n",
|
||||
"3 3 indian 0 0 0 0 0 \n",
|
||||
"4 4 indian 0 0 0 0 0 \n",
|
||||
"\n",
|
||||
" apple_brandy apricot armagnac ... whiskey white_bread white_wine \\\n",
|
||||
"0 0 0 0 ... 0 0 0 \n",
|
||||
"1 0 0 0 ... 0 0 0 \n",
|
||||
"2 0 0 0 ... 0 0 0 \n",
|
||||
"3 0 0 0 ... 0 0 0 \n",
|
||||
"4 0 0 0 ... 0 0 0 \n",
|
||||
"\n",
|
||||
" whole_grain_wheat_flour wine wood yam yeast yogurt zucchini \n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 0 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 1 0 \n",
|
||||
"\n",
|
||||
"[5 rows x 382 columns]"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>Unnamed: 0</th>\n <th>cuisine</th>\n <th>almond</th>\n <th>angelica</th>\n <th>anise</th>\n <th>anise_seed</th>\n <th>apple</th>\n <th>apple_brandy</th>\n <th>apricot</th>\n <th>armagnac</th>\n <th>...</th>\n <th>whiskey</th>\n <th>white_bread</th>\n <th>white_wine</th>\n <th>whole_grain_wheat_flour</th>\n <th>wine</th>\n <th>wood</th>\n <th>yam</th>\n <th>yeast</th>\n <th>yogurt</th>\n <th>zucchini</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>indian</td>\n <td>1</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>2</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>3</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>4</td>\n <td>indian</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>1</td>\n <td>0</td>\n </tr>\n </tbody>\n</table>\n<p>5 rows × 382 columns</p>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 60
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"data = pd.read_csv('../../data/cleaned_cuisines.csv')\n",
|
||||
"data.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 61,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" almond angelica anise anise_seed apple apple_brandy apricot \\\n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 1 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 0 0 \n",
|
||||
"\n",
|
||||
" armagnac artemisia artichoke ... whiskey white_bread white_wine \\\n",
|
||||
"0 0 0 0 ... 0 0 0 \n",
|
||||
"1 0 0 0 ... 0 0 0 \n",
|
||||
"2 0 0 0 ... 0 0 0 \n",
|
||||
"3 0 0 0 ... 0 0 0 \n",
|
||||
"4 0 0 0 ... 0 0 0 \n",
|
||||
"\n",
|
||||
" whole_grain_wheat_flour wine wood yam yeast yogurt zucchini \n",
|
||||
"0 0 0 0 0 0 0 0 \n",
|
||||
"1 0 0 0 0 0 0 0 \n",
|
||||
"2 0 0 0 0 0 0 0 \n",
|
||||
"3 0 0 0 0 0 0 0 \n",
|
||||
"4 0 0 0 0 0 1 0 \n",
|
||||
"\n",
|
||||
"[5 rows x 380 columns]"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>almond</th>\n <th>angelica</th>\n <th>anise</th>\n <th>anise_seed</th>\n <th>apple</th>\n <th>apple_brandy</th>\n <th>apricot</th>\n <th>armagnac</th>\n <th>artemisia</th>\n <th>artichoke</th>\n <th>...</th>\n <th>whiskey</th>\n <th>white_bread</th>\n <th>white_wine</th>\n <th>whole_grain_wheat_flour</th>\n <th>wine</th>\n <th>wood</th>\n <th>yam</th>\n <th>yeast</th>\n <th>yogurt</th>\n <th>zucchini</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>...</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>0</td>\n <td>1</td>\n <td>0</td>\n </tr>\n </tbody>\n</table>\n<p>5 rows × 380 columns</p>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 61
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"X = data.iloc[:,2:]\n",
|
||||
"X.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 62,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" cuisine\n",
|
||||
"0 indian\n",
|
||||
"1 indian\n",
|
||||
"2 indian\n",
|
||||
"3 indian\n",
|
||||
"4 indian"
|
||||
],
|
||||
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>cuisine</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>indian</td>\n </tr>\n <tr>\n <th>1</th>\n <td>indian</td>\n </tr>\n <tr>\n <th>2</th>\n <td>indian</td>\n </tr>\n <tr>\n <th>3</th>\n <td>indian</td>\n </tr>\n <tr>\n <th>4</th>\n <td>indian</td>\n </tr>\n </tbody>\n</table>\n</div>"
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 62
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"y = data[['cuisine']]\n",
|
||||
"y.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 63,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.svm import SVC\n",
|
||||
"from sklearn.model_selection import cross_val_score\n",
|
||||
"from sklearn.metrics import accuracy_score,precision_score,confusion_matrix,classification_report"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 64,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 65,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"SVC(C=10, kernel='linear', probability=True, random_state=0)"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 65
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model = SVC(kernel='linear', C=10, probability=True,random_state=0)\n",
|
||||
"model.fit(X_train,y_train.values.ravel())\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 66,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"y_pred = model.predict(X_test)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
" precision recall f1-score support\n\n chinese 0.72 0.70 0.71 236\n indian 0.91 0.88 0.89 243\n japanese 0.80 0.75 0.77 240\n korean 0.80 0.81 0.81 230\n thai 0.76 0.85 0.80 250\n\n accuracy 0.80 1199\n macro avg 0.80 0.80 0.80 1199\nweighted avg 0.80 0.80 0.80 1199\n\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(classification_report(y_test,y_pred))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from skl2onnx import convert_sklearn\n",
|
||||
"from skl2onnx.common.data_types import FloatTensorType\n",
|
||||
"\n",
|
||||
"initial_type = [('float_input', FloatTensorType([None, 380]))]\n",
|
||||
"options = {id(model): {'nocl': True, 'zipmap': False}}\n",
|
||||
"onx = convert_sklearn(model, initial_types=initial_type, options=options)\n",
|
||||
"with open(\"./model.onnx\", \"wb\") as f:\n",
|
||||
" f.write(onx.SerializeToString())\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
# Getting started with classification
|
||||
|
||||
## Regional topic: Delicious Asian and Indian Cuisines 🍜
|
||||
|
||||
In Asia and India, food traditions are extremely diverse, and very delicious! Let's look at data about regional cuisines to try to understand their ingredients.
|
||||
|
||||
|
||||
> Photo by <a href="https://unsplash.com/@changlisheng?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Lisheng Chang</a> on <a href="https://unsplash.com/s/photos/asian-food?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
|
||||
|
||||
## What you will learn
|
||||
|
||||
In this section, you will build on your earlier study of Regression and learn about other classifiers that you can use to better understand the data.
|
||||
|
||||
> There are useful low-code tools that can help you learn about working with classification models. Try [Azure ML for this task](https://docs.microsoft.com/learn/modules/create-classification-model-azure-machine-learning-designer/?WT.mc_id=academic-77952-leestott)
|
||||
|
||||
## Lessons
|
||||
|
||||
1. [Introduction to classification](1-Introduction/README.md)
|
||||
2. [More classifiers](2-Classifiers-1/README.md)
|
||||
3. [Yet other classifiers](3-Classifiers-2/README.md)
|
||||
4. [Applied ML: build a web app](4-Applied/README.md)
|
||||
|
||||
## Credits
|
||||
|
||||
"Getting started with classification" was written with ♥️ by [Cassie Breviu](https://www.twitter.com/cassiebreviu) and [Jen Looper](https://www.twitter.com/jenlooper)
|
||||
|
||||
The delicious cuisines dataset was sourced from [Kaggle](https://www.kaggle.com/hoandan/asian-and-indian-cuisines).
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,2 @@
|
||||
almond,angelica,anise,anise_seed,apple,apple_brandy,apricot,armagnac,artemisia,artichoke,asparagus,avocado,bacon,baked_potato,balm,banana,barley,bartlett_pear,basil,bay,bean,beech,beef,beef_broth,beef_liver,beer,beet,bell_pepper,bergamot,berry,bitter_orange,black_bean,black_currant,black_mustard_seed_oil,black_pepper,black_raspberry,black_sesame_seed,black_tea,blackberry,blackberry_brandy,blue_cheese,blueberry,bone_oil,bourbon_whiskey,brandy,brassica,bread,broccoli,brown_rice,brussels_sprout,buckwheat,butter,buttermilk,cabbage,cabernet_sauvignon_wine,cacao,camembert_cheese,cane_molasses,caraway,cardamom,carnation,carob,carrot,cashew,cassava,catfish,cauliflower,caviar,cayenne,celery,celery_oil,cereal,chamomile,champagne_wine,chayote,cheddar_cheese,cheese,cherry,cherry_brandy,chervil,chicken,chicken_broth,chicken_liver,chickpea,chicory,chinese_cabbage,chive,cider,cilantro,cinnamon,citrus,citrus_peel,clam,clove,cocoa,coconut,coconut_oil,cod,coffee,cognac,concord_grape,condiment,coriander,corn,corn_flake,corn_grit,cottage_cheese,crab,cranberry,cream,cream_cheese,cucumber,cumin,cured_pork,currant,date,dill,durian,eel,egg,egg_noodle,elderberry,emmental_cheese,endive,enokidake,fennel,fenugreek,feta_cheese,fig,fish,flower,frankfurter,fruit,galanga,gardenia,gelatin,geranium,gin,goat_cheese,grape,grape_brandy,grape_juice,grapefruit,green_bell_pepper,green_tea,gruyere_cheese,guava,haddock,ham,hazelnut,herring,holy_basil,honey,hop,horseradish,huckleberry,jamaican_rum,japanese_plum,jasmine,jasmine_tea,juniper_berry,kaffir_lime,kale,katsuobushi,kelp,kidney_bean,kiwi,kohlrabi,kumquat,lamb,lard,laurel,lavender,leaf,leek,lemon,lemon_juice,lemon_peel,lemongrass,lentil,lettuce,licorice,lilac_flower_oil,lima_bean,lime,lime_juice,lime_peel_oil,lingonberry,litchi,liver,lobster,long_pepper,lovage,macadamia_nut,macaroni,mace,mackerel,malt,mandarin,mandarin_peel,mango,maple_syrup,marjoram,mate,matsutake,meat,melon,milk,milk_fat,mint,mozzarella_cheese,mung_bean,munster_cheese,muscat_grape,mushroom,mussel,mustard,mutton,nectarine,nira,nut,nutmeg,oat,oatmeal,octopus,okra,olive,olive_oil,onion,orange,orange_flower,orange_juice,orange_peel,oregano,ouzo,oyster,palm,papaya,parmesan_cheese,parsley,parsnip,passion_fruit,pea,peach,peanut,peanut_butter,peanut_oil,pear,pear_brandy,pecan,pelargonium,pepper,peppermint,peppermint_oil,pimenta,pimento,pineapple,pistachio,plum,popcorn,porcini,pork,pork_liver,pork_sausage,port_wine,potato,potato_chip,prawn,prickly_pear,provolone_cheese,pumpkin,quince,radish,raisin,rapeseed,raspberry,raw_beef,red_algae,red_bean,red_kidney_bean,red_wine,rhubarb,roasted_almond,roasted_beef,roasted_hazelnut,roasted_meat,roasted_nut,roasted_peanut,roasted_pecan,roasted_pork,roasted_sesame_seed,romano_cheese,root,roquefort_cheese,rose,rosemary,rum,rutabaga,rye_bread,rye_flour,saffron,sage,sake,salmon,salmon_roe,sassafras,sauerkraut,savory,scallion,scallop,sea_algae,seaweed,seed,sesame_oil,sesame_seed,shallot,sheep_cheese,shellfish,sherry,shiitake,shrimp,smoke,smoked_fish,smoked_salmon,smoked_sausage,sour_cherry,sour_milk,soy_sauce,soybean,soybean_oil,spearmint,squash,squid,star_anise,starch,strawberry,strawberry_jam,strawberry_juice,sturgeon_caviar,sumac,sunflower_oil,sweet_potato,swiss_cheese,tabasco_pepper,tamarind,tangerine,tarragon,tea,tequila,thai_pepper,thyme,tomato,tomato_juice,truffle,tuna,turkey,turmeric,turnip,vanilla,veal,vegetable,vegetable_oil,vinegar,violet,walnut,wasabi,watercress,watermelon,wheat,wheat_bread,whiskey,white_bread,white_wine,whole_grain_wheat_flour,wine,wood,yam,yeast,yogurt,zucchini
|
||||
0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379
|
||||
|
Binary file not shown.
|
After Width: | Height: | Size: 114 KiB |
Reference in New Issue
Block a user