chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,74 @@
|
||||
name: Bug report
|
||||
description: Create a report to help us improve
|
||||
title: "[bug] "
|
||||
labels: ["bug", "help wanted"]
|
||||
assignees: ["ageron"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for helping us improve this project!
|
||||
|
||||
Before you create this issue, please:
|
||||
|
||||
* search for existing issues (both open and closed), as your question may already have been answered: https://github.com/ageron/handson-ml3/issues?q=is%3Aissue
|
||||
* make sure you are using the latest updated code and libraries: https://github.com/ageron/handson-ml3/blob/main/INSTALL.md#update-this-project-and-its-libraries
|
||||
* read the FAQ: https://github.com/ageron/handson-ml3#faq
|
||||
- type: input
|
||||
id: chapter
|
||||
attributes:
|
||||
label: Enter the chapter number
|
||||
placeholder: E.g., Chapter 2, end to end project
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: page
|
||||
attributes:
|
||||
label: Enter the page number
|
||||
placeholder: E.g., Page 123, second code example
|
||||
- type: input
|
||||
id: cell
|
||||
attributes:
|
||||
label: What is the cell's number in the notebook
|
||||
placeholder: E.g., Cell 123, in gradient descent section
|
||||
- type: dropdown
|
||||
id: environment
|
||||
attributes:
|
||||
label: Enter the environment you are using to run the notebook
|
||||
options:
|
||||
- Colab
|
||||
- Kaggle
|
||||
- Jupyter on Windows
|
||||
- Jupyter on Linux
|
||||
- Jupyter on MacOS
|
||||
- Other (specify in description)
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Please describe the problem and copy/paste the failing code using code blocks like this:
|
||||
|
||||
````
|
||||
```python
|
||||
def inverse(x):
|
||||
return 1 / x
|
||||
|
||||
result = inverse(0)
|
||||
```
|
||||
````
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Describe your issue
|
||||
placeholder: I got an error when running...
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: expected
|
||||
attributes:
|
||||
label: Enter what you expected to happen
|
||||
placeholder: I expected...
|
||||
- type: textarea
|
||||
id: workaround
|
||||
attributes:
|
||||
label: If you found a workaround, describe it here
|
||||
placeholder: I worked around the issue by...
|
||||
@@ -0,0 +1,64 @@
|
||||
name: Clarification request
|
||||
description: Not a bug, but something is not clear
|
||||
title: "[question] "
|
||||
labels: ["question"]
|
||||
assignees: ["ageron"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for helping us improve this project!
|
||||
|
||||
Before you create this issue, please:
|
||||
|
||||
* search for existing issues (both open and closed), as your question may already have been answered: https://github.com/ageron/handson-ml3/issues?q=is%3Aissue
|
||||
* make sure you are using the latest updated code and libraries: https://github.com/ageron/handson-ml3/blob/main/INSTALL.md#update-this-project-and-its-libraries
|
||||
* read the FAQ: https://github.com/ageron/handson-ml3#faq
|
||||
- type: input
|
||||
id: chapter
|
||||
attributes:
|
||||
label: Enter the chapter number
|
||||
placeholder: E.g., Chapter 2, end to end project
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: page
|
||||
attributes:
|
||||
label: Enter the page number
|
||||
placeholder: E.g., Page 123, second code example
|
||||
- type: input
|
||||
id: cell
|
||||
attributes:
|
||||
label: What is the cell's number in the notebook
|
||||
placeholder: E.g., Cell 123, in gradient descent section
|
||||
- type: dropdown
|
||||
id: environment
|
||||
attributes:
|
||||
label: Enter the environment you are using to run the notebook
|
||||
options:
|
||||
- Colab
|
||||
- Kaggle
|
||||
- Jupyter on Windows
|
||||
- Jupyter on Linux
|
||||
- Jupyter on MacOS
|
||||
- Other (specify in description)
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Please ask your question below. If you want to include some code, please use code blocks like this:
|
||||
|
||||
````
|
||||
```python
|
||||
def inverse(x):
|
||||
return 1 / x
|
||||
|
||||
result = inverse(0)
|
||||
```
|
||||
````
|
||||
- type: textarea
|
||||
id: question
|
||||
attributes:
|
||||
label: Question
|
||||
placeholder: Ask your question or describe what's unclear
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1 @@
|
||||
blank_issues_enabled: false
|
||||
@@ -0,0 +1,69 @@
|
||||
name: Feature or Improvement request
|
||||
description: Suggest an idea for this project
|
||||
title: "[feature] "
|
||||
labels: ["feature request"]
|
||||
assignees: ["ageron"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for helping us improve this project!
|
||||
|
||||
Before you submit this feature request, please:
|
||||
|
||||
* search for existing issues (both open and closed), as your feature request may already have been submitted: https://github.com/ageron/handson-ml3/issues?q=is%3Aissue
|
||||
* make sure you are using the latest updated code and libraries: https://github.com/ageron/handson-ml3/blob/main/INSTALL.md#update-this-project-and-its-libraries
|
||||
* read the FAQ: https://github.com/ageron/handson-ml3#faq
|
||||
- type: input
|
||||
id: chapter
|
||||
attributes:
|
||||
label: Enter the chapter number
|
||||
placeholder: E.g., Chapter 2, end to end project
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: page
|
||||
attributes:
|
||||
label: Enter the page number
|
||||
placeholder: E.g., Page 123, second code example
|
||||
- type: input
|
||||
id: cell
|
||||
attributes:
|
||||
label: What is the cell's number in the notebook
|
||||
placeholder: E.g., Cell 123, in gradient descent section
|
||||
- type: dropdown
|
||||
id: environment
|
||||
attributes:
|
||||
label: Enter the environment you are using to run the notebook
|
||||
options:
|
||||
- Colab
|
||||
- Kaggle
|
||||
- Jupyter on Windows
|
||||
- Jupyter on Linux
|
||||
- Jupyter on MacOS
|
||||
- Other (specify in description)
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Please describe your idea below. If you want to include some code, please use code blocks like this:
|
||||
|
||||
````
|
||||
```python
|
||||
def inverse(x):
|
||||
return 1 / x
|
||||
|
||||
result = inverse(0)
|
||||
```
|
||||
````
|
||||
- type: textarea
|
||||
id: idea
|
||||
attributes:
|
||||
label: Feature or improvement request
|
||||
placeholder: It would be useful to have...
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: motivation
|
||||
attributes:
|
||||
label: Why is this needed?
|
||||
placeholder: Describe the problem or limitation this feature would address
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
*.bak
|
||||
*.bak.*
|
||||
*.ckpt
|
||||
*.old
|
||||
*.pyc
|
||||
.DS_Store
|
||||
.ipynb_checkpoints/
|
||||
.vscode/
|
||||
checkpoint
|
||||
/logs
|
||||
/tf_logs
|
||||
/images
|
||||
my_*
|
||||
/person.proto
|
||||
/person.desc
|
||||
/person_pb2.py
|
||||
/datasets
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
+11
@@ -0,0 +1,11 @@
|
||||
# Changes in the third edition
|
||||
|
||||
Below are the changes in the book between the second (2019) and the third edition (2022). For the changes between the first (2017) and the second edition (2019), please check out [this other page](https://homl.info/changes2).
|
||||
|
||||
* All the code was updated to the latest library versions. In particular, this third edition introduces many new additions to Scikit-Learn (e.g., feature name tracking, histogram-based gradient boosting, label propagation, and more). It also introduces the _Keras Tuner_ library for hyperparameter tuning, Hugging Face's _Transformers_ library for natural language processing, and Keras's new preprocessing and data augmentation layers.
|
||||
* Several vision models were added (ResNeXt, DenseNet, MobileNet, CSPNet, and EfficientNet), as well as guidelines for choosing the right one.
|
||||
* Chapter 15 on sequence processing using RNNs and CNNs now analyzes the Chicago bus and rail ridership data instead of generated time series, and it introduces the ARMA model and its variants.
|
||||
* Chapter 16 on natural language processing now builds an English-to-Spanish translation model, first using an encoder–decoder RNN, then using a transformer model. The chapter also covers language models such as Switch Transformers, DistilBERT, T5, and PaLM (with chain-of-thought prompting). In addition, it introduces vision transformers (ViTs) and gives an overview of a few transformer-based visual models, such as data-efficient image transformers (DeiTs), Perceiver, and DINO, as well as a brief overview of some large multimodal models, including CLIP, DALL·E, Flamingo, and GATO.
|
||||
* Chapter 17 on generative learning now introduces diffusion models, and shows how to implement a denoising diffusion probabilistic model (DDPM) from scratch.
|
||||
* Chapter 19 migrated from Google Cloud AI Platform to Google Vertex AI, and uses distributed Keras Tuner for large-scale hyperparameter search. The chapter now includes TensorFlow.js code that you can experiment with online. It also introduces additional distributed training techniques, including PipeDream and Pathways.
|
||||
* To allow for all the new content, some sections have been moved online, including [installation instructions](INSTALL.md), and appendix A (the exercise solutions, now available at the end of each notebook). Other sections have been removed and will be added to this repository within the next few weeks, including kernel principal component analysis (kPCA), mathematical details of Bayesian Gaussian mixtures, TF Agents, and former appendices C (support vector machine math), and E (extra neural net architectures).
|
||||
+94
@@ -0,0 +1,94 @@
|
||||
# Installation
|
||||
|
||||
## Download this repository
|
||||
To install this repository and run the Jupyter notebooks on your machine, you will first need git, which you may already have. Open a terminal and type `git` to check. If you do not have git, you can download it from [git-scm.com](https://git-scm.com/).
|
||||
|
||||
Next, clone this repository by opening a terminal and typing the following commands (do not type the first `$` on each line, it's just a convention to show that this is a terminal prompt, not something else like Python code):
|
||||
|
||||
$ cd $HOME # or any other development directory you prefer
|
||||
$ git clone https://github.com/ageron/handson-ml3.git
|
||||
$ cd handson-ml3
|
||||
|
||||
If you do not want to install git, you can instead download [main.zip](https://github.com/ageron/handson-ml3/archive/main.zip), unzip it, rename the resulting directory to `handson-ml3` and move it to your development directory.
|
||||
|
||||
## Install Anaconda
|
||||
Next, you will need Python 3 and a bunch of Python libraries. The simplest way to install these is to [download and install Anaconda](https://www.anaconda.com/distribution/), which is a great cross-platform Python distribution for scientific computing. It comes bundled with many scientific libraries, including NumPy, Pandas, Matplotlib, Scikit-Learn and much more, so it's quite a large installation. If you prefer a lighter weight Anaconda distribution, you can [install Miniconda](https://docs.conda.io/en/latest/miniconda.html), which contains the bare minimum to run the `conda` packaging tool. You should install the latest version of Anaconda (or Miniconda) available.
|
||||
|
||||
During the installation on MacOSX and Linux, you will be asked whether to initialize Anaconda by running `conda init`: you should accept, as it will update your shell script to ensure that `conda` is available whenever you open a terminal. After the installation, you must close your terminal and open a new one for the changes to take effect.
|
||||
|
||||
During the installation on Windows, you will be asked whether you want the installer to update the `PATH` environment variable. This is not recommended as it may interfere with other software. Instead, after the installation you should open the Start Menu and launch an Anaconda Shell whenever you want to use Anaconda.
|
||||
|
||||
Once Anaconda (or Miniconda) is installed, run the following command to update the `conda` packaging tool to the latest version:
|
||||
|
||||
$ conda update -n base -c defaults conda
|
||||
|
||||
> **Note**: if you don't like Anaconda for some reason, then you can install Python 3 and use pip to install the required libraries manually (this is not recommended, unless you really know what you are doing). I recommend using Python 3.7, since some libs don't support Python 3.8 or 3.9 yet.
|
||||
|
||||
|
||||
## Install the GPU Driver and Libraries
|
||||
If you have a TensorFlow-compatible GPU card (NVidia card with Compute Capability ≥ 3.5), and you want TensorFlow to use it, then you should download the latest driver for your card from [nvidia.com](https://www.nvidia.com/Download/index.aspx?lang=en-us) and install it. You will also need NVidia's CUDA and cuDNN libraries, but the good news is that they will be installed automatically when you install the tensorflow-gpu package from Anaconda. However, if you don't use Anaconda, you will have to install them manually. If you hit any roadblock, see TensorFlow's [GPU installation instructions](https://tensorflow.org/install/gpu) for more details.
|
||||
|
||||
## Create the `homl3` Environment
|
||||
Next, make sure you're in the `handson-ml3` directory and run the following command. It will create a new `conda` environment containing every library you will need to run all the notebooks (by default, the environment will be named `homl3`, but you can choose another name using the `-n` option):
|
||||
|
||||
$ conda env create -f environment.yml
|
||||
|
||||
Next, activate the new environment:
|
||||
|
||||
$ conda activate homl3
|
||||
|
||||
|
||||
## Start Jupyter
|
||||
You're almost there! You just need to register the `homl3` conda environment to Jupyter. The notebooks in this project will default to the environment named `python3`, so it's best to register this environment using the name `python3` (if you prefer to use another name, you will have to select it in the "Kernel > Change kernel..." menu in Jupyter every time you open a notebook):
|
||||
|
||||
$ python3 -m ipykernel install --user --name=python3
|
||||
|
||||
And that's it! You can now start Jupyter like this:
|
||||
|
||||
$ jupyter notebook
|
||||
|
||||
This should open up your browser, and you should see Jupyter's tree view, with the contents of the current directory. If your browser does not open automatically, visit [localhost:8888](http://localhost:8888/tree). Click on `index.ipynb` to get started.
|
||||
|
||||
Congrats! You are ready to learn Machine Learning, hands on!
|
||||
|
||||
When you're done with Jupyter, you can close it by typing Ctrl-C in the Terminal window where you started it. Every time you want to work on this project, you will need to open a Terminal, and run:
|
||||
|
||||
$ cd $HOME # or whatever development directory you chose earlier
|
||||
$ cd handson-ml3
|
||||
$ conda activate homl3
|
||||
$ jupyter notebook
|
||||
|
||||
## Update This Project and its Libraries
|
||||
I regularly update the notebooks to fix issues and add support for new libraries. So make sure you update this project regularly.
|
||||
|
||||
For this, open a terminal, and run:
|
||||
|
||||
$ cd $HOME # or whatever development directory you chose earlier
|
||||
$ cd handson-ml3 # go to this project's directory
|
||||
$ git pull
|
||||
|
||||
If you get an error, it's probably because you modified a notebook. In this case, before running `git pull` you will first need to commit your changes. I recommend doing this in your own branch, or else you may get conflicts:
|
||||
|
||||
$ git checkout -b my_branch # you can use another branch name if you want
|
||||
$ git add -u
|
||||
$ git commit -m "describe your changes here"
|
||||
$ git checkout main
|
||||
$ git pull
|
||||
|
||||
Next, let's update the libraries. First, let's update `conda` itself:
|
||||
|
||||
$ conda update -c defaults -n base conda
|
||||
|
||||
Then we'll delete this project's `homl3` environment:
|
||||
|
||||
$ conda activate base
|
||||
$ conda env remove -n homl3
|
||||
|
||||
And recreate the environment:
|
||||
|
||||
$ conda env create -f environment.yml
|
||||
|
||||
Lastly, we reactivate the environment and start Jupyter:
|
||||
|
||||
$ conda activate homl3
|
||||
$ jupyter notebook
|
||||
@@ -0,0 +1,178 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
Machine Learning Notebooks, 3rd edition
|
||||
=================================
|
||||
|
||||
This project aims at teaching you the fundamentals of Machine Learning in
|
||||
python. It contains the example code and solutions to the exercises in the third edition of my O'Reilly book [Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow (3rd edition)](https://homl.info/er3):
|
||||
|
||||
<a href="https://homl.info/er3"><img src="https://learning.oreilly.com/library/cover/9781098125967/300w/" title="book" width="150" border="0" /></a>
|
||||
|
||||
**Note**: If you are looking for the second edition notebooks, check out [ageron/handson-ml2](https://github.com/ageron/handson-ml2). For the first edition, see [ageron/handson-ml](https://github.com/ageron/handson-ml).
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Want to play with these notebooks online without having to install anything?
|
||||
|
||||
* <a href="https://colab.research.google.com/github/ageron/handson-ml3/blob/main/" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> (recommended)
|
||||
|
||||
⚠ _Colab provides a temporary environment: anything you do will be deleted after a while, so make sure you download any data you care about._
|
||||
|
||||
<details>
|
||||
|
||||
Other services may work as well, but I have not fully tested them:
|
||||
|
||||
* <a href="https://homl.info/kaggle3/"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open in Kaggle" /></a>
|
||||
|
||||
* <a href="https://mybinder.org/v2/gh/ageron/handson-ml3/HEAD?filepath=%2Findex.ipynb"><img src="https://mybinder.org/badge_logo.svg" alt="Launch binder" /></a>
|
||||
|
||||
* <a href="https://homl.info/deepnote3/"><img src="https://deepnote.com/buttons/launch-in-deepnote-small.svg" alt="Launch in Deepnote" /></a>
|
||||
|
||||
</details>
|
||||
|
||||
### Just want to quickly look at some notebooks, without executing any code?
|
||||
|
||||
* <a href="https://nbviewer.jupyter.org/github/ageron/handson-ml3/blob/main/index.ipynb"><img src="https://raw.githubusercontent.com/jupyter/design/master/logos/Badges/nbviewer_badge.svg" alt="Render nbviewer" /></a>
|
||||
|
||||
* [github.com's notebook viewer](https://github.com/ageron/handson-ml3/blob/main/index.ipynb) also works but it's not ideal: it's slower, the math equations are not always displayed correctly, and large notebooks often fail to open.
|
||||
|
||||
### Want to run this project using a Docker image?
|
||||
Read the [Docker instructions](https://github.com/ageron/handson-ml3/tree/main/docker).
|
||||
|
||||
### Want to install this project on your own machine?
|
||||
|
||||
Start by installing [Anaconda](https://www.anaconda.com/products/distribution) (or [Miniconda](https://docs.conda.io/en/latest/miniconda.html)), [git](https://git-scm.com/downloads), and if you have a TensorFlow-compatible GPU, install the [GPU driver](https://www.nvidia.com/Download/index.aspx), as well as the appropriate version of CUDA and cuDNN (see TensorFlow's documentation for more details).
|
||||
|
||||
Next, clone this project by opening a terminal and typing the following commands (do not type the first `$` signs on each line, they just indicate that these are terminal commands):
|
||||
|
||||
$ git clone https://github.com/ageron/handson-ml3.git
|
||||
$ cd handson-ml3
|
||||
|
||||
Next, run the following commands:
|
||||
|
||||
$ conda env create -f environment.yml
|
||||
$ conda activate homl3
|
||||
$ python -m ipykernel install --user --name=python3
|
||||
|
||||
Finally, start Jupyter:
|
||||
|
||||
$ jupyter notebook
|
||||
|
||||
If you need further instructions, read the [detailed installation instructions](INSTALL.md).
|
||||
|
||||
# FAQ
|
||||
|
||||
**Which Python version should I use?**
|
||||
|
||||
I recommend Python 3.10. If you follow the installation instructions above, that's the version you will get. Any version ≥3.7 should work as well.
|
||||
|
||||
**I'm getting an error when I call `load_housing_data()`**
|
||||
|
||||
If you're getting an HTTP error, make sure you're running the exact same code as in the notebook (copy/paste it if needed). If the problem persists, please check your network configuration. If it's an SSL error, see the next question.
|
||||
|
||||
**I'm getting an SSL error on MacOSX**
|
||||
|
||||
You probably need to install the SSL certificates (see this [StackOverflow question](https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error)). If you downloaded Python from the official website, then run `/Applications/Python\ 3.10/Install\ Certificates.command` in a terminal (change `3.10` to whatever version you installed). If you installed Python using MacPorts, run `sudo port install curl-ca-bundle` in a terminal.
|
||||
|
||||
**I've installed this project locally. How do I update it to the latest version?**
|
||||
|
||||
See [INSTALL.md](INSTALL.md)
|
||||
|
||||
**How do I update my Python libraries to the latest versions, when using Anaconda?**
|
||||
|
||||
See [INSTALL.md](INSTALL.md)
|
||||
|
||||
## Contributors
|
||||
I would like to thank everyone [who contributed to this project](https://github.com/ageron/handson-ml3/graphs/contributors), either by providing useful feedback, filing issues or submitting Pull Requests. Special thanks go to Haesun Park and Ian Beauregard who reviewed every notebook and submitted many PRs, including help on some of the exercise solutions. Thanks as well to Steven Bunkley and Ziembla who created the `docker` directory, and to github user SuperYorio who helped on some exercise solutions. Thanks a lot to Victor Khaustov who submitted plenty of excellent PRs, fixing many errors. And lastly, thanks to Google ML Developer Programs team who supported this work by providing Google Cloud Credit.
|
||||
@@ -0,0 +1,7 @@
|
||||
# WeHub 来源说明
|
||||
|
||||
- 原始项目：`ageron/handson-ml3`
|
||||
- 原始仓库：https://github.com/ageron/handson-ml3
|
||||
- 导入方式：上游默认分支的最新快照
|
||||
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
|
||||
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
|
||||
@@ -0,0 +1,14 @@
|
||||
build-essential
|
||||
cmake
|
||||
ffmpeg
|
||||
git
|
||||
libboost-all-dev
|
||||
libjpeg-dev
|
||||
libpq-dev
|
||||
libsdl2-dev
|
||||
sudo
|
||||
swig
|
||||
unzip
|
||||
xorg-dev
|
||||
zip
|
||||
zlib1g-dev
|
||||
Binary file not shown.
@@ -0,0 +1 @@
|
||||
COMPOSE_PROJECT_NAME=handson-ml3
|
||||
@@ -0,0 +1,78 @@
|
||||
FROM continuumio/miniconda3:latest
|
||||
|
||||
RUN apt-get update && apt-get install -y \
|
||||
protobuf-compiler \
|
||||
sudo \
|
||||
&& apt clean \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY environment.yml /tmp/
|
||||
RUN conda env create -f /tmp/environment.yml \
|
||||
&& conda clean -afy \
|
||||
&& find /opt/conda/ -follow -type f -name '*.a' -delete \
|
||||
&& find /opt/conda/ -follow -type f -name '*.pyc' -delete \
|
||||
&& find /opt/conda/ -follow -type f -name '*.js.map' -delete \
|
||||
&& rm /tmp/environment.yml
|
||||
|
||||
ARG username
|
||||
ARG userid
|
||||
|
||||
ARG home=/home/${username}
|
||||
ARG workdir=${home}/handson-ml3
|
||||
|
||||
RUN adduser ${username} --uid ${userid} --gecos '' --disabled-password \
|
||||
&& echo "${username} ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/${username} \
|
||||
&& chmod 0440 /etc/sudoers.d/${username}
|
||||
|
||||
WORKDIR ${workdir}
|
||||
RUN chown ${username}:${username} ${workdir}
|
||||
|
||||
USER ${username}
|
||||
WORKDIR ${workdir}
|
||||
|
||||
ENV PATH /opt/conda/envs/homl3/bin:$PATH
|
||||
|
||||
# The config below enables diffing notebooks with nbdiff (and nbdiff support
|
||||
# in git diff command) after connecting to the container by "make exec" (or
|
||||
# "docker-compose exec handson-ml3 bash")
|
||||
# You may also try running:
|
||||
# nbdiff NOTEBOOK_NAME.ipynb
|
||||
# to get nbdiff between checkpointed version and current version of the
|
||||
# given notebook.
|
||||
|
||||
RUN git-nbdiffdriver config --enable --global
|
||||
|
||||
# INFO: Optionally uncomment any (one) of the following RUN commands below to ignore either
|
||||
# metadata or details in nbdiff within git diff
|
||||
#RUN git config --global diff.jupyternotebook.command 'git-nbdiffdriver diff --ignore-metadata'
|
||||
RUN git config --global diff.jupyternotebook.command 'git-nbdiffdriver diff --ignore-details'
|
||||
|
||||
|
||||
COPY docker/bashrc.bash /tmp/
|
||||
RUN cat /tmp/bashrc.bash >> ${home}/.bashrc \
|
||||
&& echo "export PATH=\"${workdir}/docker/bin:$PATH\"" >> ${home}/.bashrc \
|
||||
&& sudo rm /tmp/bashrc.bash
|
||||
|
||||
|
||||
# INFO: Uncomment lines below to enable automatic save of python-only and html-only
|
||||
# exports alongside the notebook
|
||||
#COPY docker/jupyter_notebook_config.py /tmp/
|
||||
#RUN cat /tmp/jupyter_notebook_config.py >> ${home}/.jupyter/jupyter_notebook_config.py
|
||||
#RUN sudo rm /tmp/jupyter_notebook_config.py
|
||||
|
||||
|
||||
# INFO: Uncomment the RUN command below to disable git diff paging
|
||||
#RUN git config --global core.pager ''
|
||||
|
||||
|
||||
# INFO: Uncomment the RUN command below for easy and constant notebook URL (just localhost:8888)
|
||||
# That will switch Jupyter to using empty password instead of a token.
|
||||
# To avoid making a security hole you SHOULD in fact not only uncomment but
|
||||
# regenerate the hash for your own non-empty password and replace the hash below.
|
||||
# You can compute a password hash in any notebook, just run the code:
|
||||
# from notebook.auth import passwd
|
||||
# passwd()
|
||||
# and take the hash from the output
|
||||
#RUN mkdir -p ${home}/.jupyter && \
|
||||
# echo 'c.NotebookApp.password = u"sha1:c6bbcba2d04b:f969e403db876dcfbe26f47affe41909bd53392e"' \
|
||||
# >> ${home}/.jupyter/jupyter_notebook_config.py
|
||||
@@ -0,0 +1,204 @@
|
||||
# This Dockerfile includes sections from tensorflow/tensorflow:latest-gpu's Dockerfile:
|
||||
# https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/dockerfiles/dockerfiles/gpu.Dockerfile
|
||||
# and sections from continuumio/miniconda3:latest's Dockerfile:
|
||||
# https://github.com/ContinuumIO/docker-images/blob/master/miniconda3/debian/Dockerfile
|
||||
|
||||
|
||||
# First we need CUDA and everything else needed to support GPUs
|
||||
|
||||
###############################################
|
||||
#### FROM tensorflow/tensorflow:latest-gpu ####
|
||||
###############################################
|
||||
ARG UBUNTU_VERSION=20.04
|
||||
|
||||
ARG ARCH=
|
||||
ARG CUDA_BASE=11.2.2
|
||||
FROM nvidia/cuda:${CUDA_BASE}-base-ubuntu${UBUNTU_VERSION} as base
|
||||
# ARCH and CUDA are specified again because the FROM directive resets ARGs
|
||||
# (but their default value is retained if set previously)
|
||||
ARG ARCH
|
||||
ARG CUDA=11.2
|
||||
ARG CUDNN=8.1.0.77-1
|
||||
ARG CUDNN_MAJOR_VERSION=8
|
||||
ARG LIB_DIR_PREFIX=x86_64
|
||||
ARG LIBNVINFER=7.2.2-1
|
||||
ARG LIBNVINFER_MAJOR_VERSION=7
|
||||
|
||||
# Let us install tzdata painlessly
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# Needed for string substitution
|
||||
SHELL ["/bin/bash", "-c"]
|
||||
# Pick up some TF dependencies
|
||||
# [HOML3] Tweaked for handson-ml3: added all the libs before build-essentials
|
||||
# and call apt clean + delete apt cache files.
|
||||
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub && \
|
||||
apt-get update && apt-get install -y --no-install-recommends \
|
||||
git \
|
||||
protobuf-compiler \
|
||||
sudo \
|
||||
wget \
|
||||
build-essential \
|
||||
cuda-command-line-tools-${CUDA/./-} \
|
||||
libcublas-${CUDA/./-} \
|
||||
cuda-nvrtc-${CUDA/./-} \
|
||||
libcufft-${CUDA/./-} \
|
||||
libcurand-${CUDA/./-} \
|
||||
libcusolver-${CUDA/./-} \
|
||||
libcusparse-${CUDA/./-} \
|
||||
curl \
|
||||
libcudnn8=${CUDNN}+cuda${CUDA} \
|
||||
libfreetype6-dev \
|
||||
libhdf5-serial-dev \
|
||||
libzmq3-dev \
|
||||
pkg-config \
|
||||
software-properties-common \
|
||||
unzip \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install TensorRT if not building for PowerPC
|
||||
# NOTE: libnvinfer uses cuda11.1 versions
|
||||
RUN [[ "${ARCH}" = "ppc64le" ]] || { apt-get update && \
|
||||
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub && \
|
||||
echo "deb https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/tensorRT.list && \
|
||||
apt-get update && \
|
||||
apt-get install -y --no-install-recommends libnvinfer${LIBNVINFER_MAJOR_VERSION}=${LIBNVINFER}+cuda11.0 \
|
||||
libnvinfer-plugin${LIBNVINFER_MAJOR_VERSION}=${LIBNVINFER}+cuda11.0 \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/*; }
|
||||
|
||||
# For CUDA profiling, TensorFlow requires CUPTI.
|
||||
ENV LD_LIBRARY_PATH /usr/local/cuda-11.0/targets/x86_64-linux/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64:$LD_LIBRARY_PATH
|
||||
|
||||
# Link the libcuda stub to the location where tensorflow is searching for it and reconfigure
|
||||
# dynamic linker run-time bindings
|
||||
RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/libcuda.so.1 \
|
||||
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
|
||||
&& ldconfig
|
||||
|
||||
# [HOML3] Tweaked for handson-ml3: removed Python3 & TensorFlow installation using pip
|
||||
|
||||
#################################################
|
||||
#### End of tensorflow/tensorflow:latest-gpu ####
|
||||
#################################################
|
||||
|
||||
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
|
||||
ENV PATH /opt/conda/bin:/opt/conda/envs/homl3/bin:$PATH
|
||||
ENV CONDA_DEFAULT_ENV=homl3
|
||||
|
||||
# Next we need to install miniconda
|
||||
|
||||
############################################
|
||||
#### FROM continuumio/miniconda3:latest ####
|
||||
############################################
|
||||
|
||||
# [HOML2] Tweaked for handson-ml3: removed the beginning of the Dockerfile
|
||||
CMD [ "/bin/bash" ]
|
||||
|
||||
# Leave these args here to better use the Docker build cache
|
||||
ARG CONDA_VERSION=py39_4.12.0
|
||||
|
||||
RUN set -x && \
|
||||
UNAME_M="$(uname -m)" && \
|
||||
if [ "${UNAME_M}" = "x86_64" ]; then \
|
||||
MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-x86_64.sh"; \
|
||||
SHA256SUM="78f39f9bae971ec1ae7969f0516017f2413f17796670f7040725dd83fcff5689"; \
|
||||
elif [ "${UNAME_M}" = "s390x" ]; then \
|
||||
MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-s390x.sh"; \
|
||||
SHA256SUM="ff6fdad3068ab5b15939c6f422ac329fa005d56ee0876c985e22e622d930e424"; \
|
||||
elif [ "${UNAME_M}" = "aarch64" ]; then \
|
||||
MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-aarch64.sh"; \
|
||||
SHA256SUM="5f4f865812101fdc747cea5b820806f678bb50fe0a61f19dc8aa369c52c4e513"; \
|
||||
elif [ "${UNAME_M}" = "ppc64le" ]; then \
|
||||
MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-ppc64le.sh"; \
|
||||
SHA256SUM="1fe3305d0ccc9e55b336b051ae12d82f33af408af4b560625674fa7ad915102b"; \
|
||||
fi && \
|
||||
wget "${MINICONDA_URL}" -O miniconda.sh -q && \
|
||||
echo "${SHA256SUM} miniconda.sh" > shasum && \
|
||||
if [ "${CONDA_VERSION}" != "latest" ]; then sha256sum --check --status shasum; fi && \
|
||||
mkdir -p /opt && \
|
||||
sh miniconda.sh -b -p /opt/conda && \
|
||||
rm miniconda.sh shasum && \
|
||||
ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
|
||||
echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
|
||||
echo "conda activate base" >> ~/.bashrc && \
|
||||
find /opt/conda/ -follow -type f -name '*.a' -delete && \
|
||||
find /opt/conda/ -follow -type f -name '*.js.map' -delete && \
|
||||
/opt/conda/bin/conda clean -afy
|
||||
|
||||
##############################################
|
||||
#### End of continuumio/miniconda3:latest ####
|
||||
##############################################
|
||||
|
||||
# Now we're ready to create our conda environment
|
||||
|
||||
COPY environment.yml /tmp/
|
||||
RUN conda env create -f /tmp/environment.yml \
|
||||
&& conda clean -afy \
|
||||
&& find /opt/conda/ -follow -type f -name '*.a' -delete \
|
||||
&& find /opt/conda/ -follow -type f -name '*.pyc' -delete \
|
||||
&& find /opt/conda/ -follow -type f -name '*.js.map' -delete \
|
||||
&& rm /tmp/environment.yml
|
||||
|
||||
ARG username
|
||||
ARG userid
|
||||
|
||||
ARG home=/home/${username}
|
||||
ARG workdir=${home}/handson-ml3
|
||||
|
||||
RUN adduser ${username} --uid ${userid} --gecos '' --disabled-password \
|
||||
&& echo "${username} ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/${username} \
|
||||
&& chmod 0440 /etc/sudoers.d/${username}
|
||||
|
||||
WORKDIR ${workdir}
|
||||
RUN chown ${username}:${username} ${workdir}
|
||||
|
||||
USER ${username}
|
||||
WORKDIR ${workdir}
|
||||
|
||||
|
||||
# The config below enables diffing notebooks with nbdiff (and nbdiff support
|
||||
# in git diff command) after connecting to the container by "make exec" (or
|
||||
# "docker-compose exec handson-ml3 bash")
|
||||
# You may also try running:
|
||||
# nbdiff NOTEBOOK_NAME.ipynb
|
||||
# to get nbdiff between checkpointed version and current version of the
|
||||
# given notebook.
|
||||
|
||||
RUN git-nbdiffdriver config --enable --global
|
||||
|
||||
# INFO: Optionally uncomment any (one) of the following RUN commands below to ignore either
|
||||
# metadata or details in nbdiff within git diff
|
||||
#RUN git config --global diff.jupyternotebook.command 'git-nbdiffdriver diff --ignore-metadata'
|
||||
RUN git config --global diff.jupyternotebook.command 'git-nbdiffdriver diff --ignore-details'
|
||||
|
||||
|
||||
COPY docker/bashrc.bash /tmp/
|
||||
RUN cat /tmp/bashrc.bash >> ${home}/.bashrc \
|
||||
&& echo "export PATH=\"${workdir}/docker/bin:$PATH\"" >> ${home}/.bashrc \
|
||||
&& sudo rm /tmp/bashrc.bash
|
||||
|
||||
|
||||
# INFO: Uncomment lines below to enable automatic save of python-only and html-only
|
||||
# exports alongside the notebook
|
||||
#COPY docker/jupyter_notebook_config.py /tmp/
|
||||
#RUN cat /tmp/jupyter_notebook_config.py >> ${home}/.jupyter/jupyter_notebook_config.py
|
||||
#RUN sudo rm /tmp/jupyter_notebook_config.py
|
||||
|
||||
|
||||
# INFO: Uncomment the RUN command below to disable git diff paging
|
||||
#RUN git config --global core.pager ''
|
||||
|
||||
|
||||
# INFO: Uncomment the RUN command below for easy and constant notebook URL (just localhost:8888)
|
||||
# That will switch Jupyter to using empty password instead of a token.
|
||||
# To avoid making a security hole you SHOULD in fact not only uncomment but
|
||||
# regenerate the hash for your own non-empty password and replace the hash below.
|
||||
# You can compute a password hash in any notebook, just run the code:
|
||||
# from notebook.auth import passwd
|
||||
# passwd()
|
||||
# and take the hash from the output
|
||||
#RUN mkdir -p ${home}/.jupyter && \
|
||||
# echo 'c.NotebookApp.password = u"sha1:c6bbcba2d04b:f969e403db876dcfbe26f47affe41909bd53392e"' \
|
||||
# >> ${home}/.jupyter/jupyter_notebook_config.py
|
||||
@@ -0,0 +1,14 @@
|
||||
|
||||
help:
|
||||
cat Makefile
|
||||
run:
|
||||
docker-compose up
|
||||
exec:
|
||||
docker-compose exec handson-ml3 bash
|
||||
build: stop .FORCE
|
||||
docker-compose build
|
||||
rebuild: stop .FORCE
|
||||
docker-compose build --no-cache
|
||||
stop:
|
||||
docker stop handson-ml3 || true; docker rm handson-ml3 || true;
|
||||
.FORCE:
|
||||
@@ -0,0 +1,151 @@
|
||||
|
||||
# Hands-on Machine Learning in Docker
|
||||
|
||||
This is the Docker configuration which allows you to run and tweak the book's notebooks without installing any dependencies on your machine!<br/>OK, any except `docker` and `docker-compose`.<br />And optionally `make`.<br />And a few more things if you want GPU support (see below for details).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Follow the instructions on [Install Docker](https://docs.docker.com/engine/installation/) and [Install Docker Compose](https://docs.docker.com/compose/install/) for your environment if you haven't got `docker` and `docker-compose` already.
|
||||
|
||||
Some general knowledge about `docker` infrastructure might be useful (that's an interesting topic on its own) but is not strictly *required* to just run the notebooks.
|
||||
|
||||
## Usage
|
||||
|
||||
### Prepare the image (once)
|
||||
|
||||
The first option is to pull the image from Docker Hub (this will download about 1.9 GB of compressed data):
|
||||
|
||||
```bash
|
||||
$ docker pull ageron/handson-ml3
|
||||
```
|
||||
|
||||
**Note**: this is the CPU-only image. For GPU support, read the GPU section below.
|
||||
|
||||
Alternatively, you can build the image yourself. This will be slower, but it will ensure the image is up to date, with the latest libraries. For this, assuming you already downloaded this project into the directory `/path/to/project/handson-ml3`:
|
||||
|
||||
```bash
|
||||
$ cd /path/to/project/handson-ml3/docker
|
||||
$ docker-compose build
|
||||
```
|
||||
|
||||
This will take quite a while, but is only required once.
|
||||
|
||||
After the process is finished you have an `ageron/handson-ml3:latest` image, that will be the base for your experiments. You can confirm that by running the following command:
|
||||
|
||||
```bash
|
||||
$ docker images
|
||||
REPOSITORY TAG IMAGE ID CREATED SIZE
|
||||
ageron/handson-ml3 latest 3ebafebc604a 2 minutes ago 4.87GB
|
||||
```
|
||||
|
||||
### Run the notebooks
|
||||
|
||||
Still assuming you already downloaded this project into the directory `/path/to/project/handson-ml3`, run the following commands to start the Jupyter server inside the container, which is named `handson-ml3`:
|
||||
|
||||
```bash
|
||||
$ cd /path/to/project/handson-ml3/docker
|
||||
$ docker-compose up
|
||||
```
|
||||
|
||||
Next, just point your browser to the URL printed on the screen (or go to <http://localhost:8888> if you enabled password authentication inside the `jupyter_notebook_config.py` file, before building the image) and you're ready to play with the book's code!
|
||||
|
||||
The server runs in the directory containing the notebooks, and the changes you make from the browser will be persisted there.
|
||||
|
||||
You can close the server just by pressing `Ctrl-C` in the terminal window.
|
||||
|
||||
### Using `make` (optional)
|
||||
|
||||
If you have `make` installed on your computer, you can use it as a thin layer to run `docker-compose` commands. For example, executing `make rebuild` will actually run `docker-compose build --no-cache`, which will rebuild the image without using the cache. This ensures that your image is based on the latest version of the `continuumio/miniconda3` image which the `ageron/handson-ml3` image is based on.
|
||||
|
||||
If you don't have `make` (and you don't want to install it), just examine the contents of `Makefile` to see which `docker-compose` commands you can run instead.
|
||||
|
||||
### Run additional commands in the container
|
||||
|
||||
Run `make exec` (or `docker-compose exec handson-ml3 bash`) while the server is running to run an additional `bash` shell inside the `handson-ml3` container. Now you're inside the environment prepared within the image.
|
||||
|
||||
One of the useful things that can be done there would be starting TensorBoard (for example with simple `tb` command, see bashrc file).
|
||||
|
||||
Another one may be comparing versions of the notebooks using the `nbdiff` command if you haven't got `nbdime` installed locally (it is **way** better than plain `diff` for notebooks). See [Tools for diffing and merging of Jupyter notebooks](https://github.com/jupyter/nbdime) for more details.
|
||||
|
||||
You can see changes you made relative to the version in git using `git diff` which is integrated with `nbdiff`.
|
||||
|
||||
You may also try `nbd NOTEBOOK_NAME.ipynb` command (custom, see bashrc file) to compare one of your notebooks with its `checkpointed` version.<br/>
|
||||
To be precise, the output will tell you *what modifications should be re-played on the **manually saved** version of the notebook (located in `.ipynb_checkpoints` subdirectory) to update it to the **current** i.e. **auto-saved** version (given as command's argument - located in working directory)*.
|
||||
|
||||
## GPU Support on Linux (experimental)
|
||||
|
||||
### Prerequisites
|
||||
|
||||
If you're running on Linux, and you have a TensorFlow-compatible GPU card (NVidia card with Compute Capability ≥ 3.5) that you would like TensorFlow to use inside the Docker container, then you should download and install the latest driver for your card from [nvidia.com](https://www.nvidia.com/Download/index.aspx?lang=en-us). You will also need to install [NVidia Docker support](https://github.com/NVIDIA/nvidia-docker): if you are using Docker 19.03 or above, you must install the `nvidia-container-toolkit` package, and for earlier versions, you must install `nvidia-docker2`.
|
||||
|
||||
Next, edit the `docker-compose.yml` file:
|
||||
|
||||
```bash
|
||||
$ cd /path/to/project/handson-ml3/docker
|
||||
$ edit docker-compose.yml # use your favorite editor
|
||||
```
|
||||
|
||||
* Replace `dockerfile: ./docker/Dockerfile` with `dockerfile: ./docker/Dockerfile.gpu`
|
||||
* Replace `image: ageron/handson-ml3:latest` with `image: ageron/handson-ml3:latest-gpu`
|
||||
* If you want to use `docker-compose`, you will need version 1.28 or above for GPU support, and you must uncomment the whole `deploy` section in `docker-compose.yml`.
|
||||
|
||||
### Prepare the image (once)
|
||||
|
||||
If you want to pull the prebuilt image from Docker Hub (this will download over 3.5 GB of compressed data):
|
||||
|
||||
```bash
|
||||
$ docker pull ageron/handson-ml3:latest-gpu
|
||||
```
|
||||
|
||||
If you prefer to build the image yourself:
|
||||
|
||||
```bash
|
||||
$ cd /path/to/project/handson-ml3/docker
|
||||
$ docker-compose build
|
||||
```
|
||||
|
||||
### Run the notebooks with `docker-compose` (version 1.28 or above)
|
||||
|
||||
If you have `docker-compose` version 1.28 or above, that's great! You can simply run:
|
||||
|
||||
```bash
|
||||
$ cd /path/to/project/handson-ml3/docker
|
||||
$ docker-compose up
|
||||
[...]
|
||||
or http://127.0.0.1:8888/?token=[...]
|
||||
```
|
||||
|
||||
Then point your browser to the URL and Jupyter should appear. If you then open or create a notebook and execute the following code, a list containing your GPU device(s) should be displayed (success!):
|
||||
|
||||
```python
|
||||
import tensorflow as tf
|
||||
|
||||
tf.config.list_physical_devices("GPU")
|
||||
```
|
||||
|
||||
To stop the server, just press Ctrl-C.
|
||||
|
||||
### Run the notebooks without `docker-compose`
|
||||
|
||||
If you have a version of `docker-compose` earlier than 1.28, you will have to use `docker run` directly.
|
||||
|
||||
If you are using Docker 19.03 or above, you can run:
|
||||
|
||||
```bash
|
||||
$ cd /path/to/project/handson-ml3
|
||||
$ docker run --name handson-ml3 --gpus all -p 8888:8888 -p 6006:6006 --log-opt mode=non-blocking --log-opt max-buffer-size=50m -v `pwd`:/home/devel/handson-ml3 ageron/handson-ml3:latest-gpu /opt/conda/envs/homl3/bin/jupyter notebook --ip='0.0.0.0' --port=8888 --no-browser
|
||||
```
|
||||
|
||||
If you are using an older version of Docker, then replace `--gpus all` with `--runtime=nvidia`.
|
||||
|
||||
Now point your browser to the displayed URL: Jupyter should appear, and you can open a notebook and run `import tensorflow as tf` and `tf.config.list_physical_devices("GPU")` as above to confirm that TensorFlow does indeed see your GPU device(s).
|
||||
|
||||
Lastly, to interrupt the server, press Ctrl-C, then run:
|
||||
|
||||
```bash
|
||||
$ docker rm handson-ml3
|
||||
```
|
||||
|
||||
This will remove the container so you can start a new one later (but it will not remove the image or the notebooks, don't worry!).
|
||||
|
||||
Have fun!
|
||||
@@ -0,0 +1,3 @@
|
||||
alias ll="ls -alF"
|
||||
alias nbd="nbdiff_checkpoint"
|
||||
alias tb="tensorboard --logdir=tf_logs"
|
||||
Executable
+116
@@ -0,0 +1,116 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import collections
|
||||
import glob
|
||||
import hashlib
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
|
||||
class NotebookAnalyser:
|
||||
|
||||
def __init__(self, dry_run=False, verbose=False, colorful=False):
|
||||
self._dry_run = dry_run
|
||||
self._verbose = verbose
|
||||
self._colors = collections.defaultdict(lambda: "")
|
||||
if colorful:
|
||||
for color in [
|
||||
NotebookAnalyser.COLOR_WHITE,
|
||||
NotebookAnalyser.COLOR_RED,
|
||||
NotebookAnalyser.COLOR_GREEN,
|
||||
NotebookAnalyser.COLOR_YELLOW,
|
||||
]:
|
||||
self._colors[color] = "\033[{}m".format(color)
|
||||
|
||||
NOTEBOOK_SUFFIX = ".ipynb"
|
||||
CHECKPOINT_DIR = NOTEBOOK_SUFFIX + "_checkpoints"
|
||||
CHECKPOINT_MASK = "*-checkpoint" + NOTEBOOK_SUFFIX
|
||||
CHECKPOINT_MASK_LEN = len(CHECKPOINT_MASK) - 1
|
||||
|
||||
@staticmethod
|
||||
def get_hash(file_path):
|
||||
with open(file_path, "rb") as input:
|
||||
hash = hashlib.md5()
|
||||
for chunk in iter(lambda: input.read(4096), b""):
|
||||
hash.update(chunk)
|
||||
return hash.hexdigest()
|
||||
|
||||
MESSAGE_ORPHANED = "missing "
|
||||
MESSAGE_MODIFIED = "modified"
|
||||
MESSAGE_DELETED = "DELETING"
|
||||
|
||||
COLOR_WHITE = "0"
|
||||
COLOR_RED = "31"
|
||||
COLOR_GREEN = "32"
|
||||
COLOR_YELLOW = "33"
|
||||
|
||||
def log(self, message, file, color=COLOR_WHITE):
|
||||
color_on = self._colors[color]
|
||||
color_off = self._colors[NotebookAnalyser.COLOR_WHITE]
|
||||
print("{}{}{}: {}".format(color_on, message, color_off, file))
|
||||
|
||||
def clean_checkpoints(self, directory):
|
||||
for checkpoint_path in sorted(glob.glob(os.path.join(directory, NotebookAnalyser.CHECKPOINT_MASK))):
|
||||
|
||||
workfile_dir = os.path.dirname(os.path.dirname(checkpoint_path))
|
||||
workfile_name = os.path.basename(checkpoint_path)[:-NotebookAnalyser.CHECKPOINT_MASK_LEN] + NotebookAnalyser.NOTEBOOK_SUFFIX
|
||||
workfile_path = os.path.join(workfile_dir, workfile_name)
|
||||
|
||||
status = ""
|
||||
if not os.path.isfile(workfile_path):
|
||||
if self._verbose:
|
||||
self.log(NotebookAnalyser.MESSAGE_ORPHANED, workfile_path, NotebookAnalyser.COLOR_RED)
|
||||
else:
|
||||
checkpoint_stat = os.stat(checkpoint_path)
|
||||
workfile_stat = os.stat(workfile_path)
|
||||
|
||||
modified = workfile_stat.st_size != checkpoint_stat.st_size
|
||||
|
||||
if not modified:
|
||||
checkpoint_hash = NotebookAnalyser.get_hash(checkpoint_path)
|
||||
workfile_hash = NotebookAnalyser.get_hash(workfile_path)
|
||||
modified = checkpoint_hash != workfile_hash
|
||||
|
||||
if modified:
|
||||
if self._verbose:
|
||||
self.log(NotebookAnalyser.MESSAGE_MODIFIED, workfile_path, NotebookAnalyser.COLOR_YELLOW)
|
||||
else:
|
||||
self.log(NotebookAnalyser.MESSAGE_DELETED, checkpoint_path, NotebookAnalyser.COLOR_GREEN)
|
||||
if not self._dry_run:
|
||||
os.remove(checkpoint_path)
|
||||
|
||||
if not self._dry_run and not os.listdir(directory):
|
||||
self.log(NotebookAnalyser.MESSAGE_DELETED, directory, NotebookAnalyser.COLOR_GREEN)
|
||||
os.rmdir(directory)
|
||||
|
||||
def clean_checkpoints_recursively(self, directory):
|
||||
for (root, subdirs, files) in os.walk(directory):
|
||||
subdirs.sort() # INFO: traverse alphabetically
|
||||
if NotebookAnalyser.CHECKPOINT_DIR in subdirs:
|
||||
subdirs.remove(NotebookAnalyser.CHECKPOINT_DIR) # INFO: don't recurse there
|
||||
self.clean_checkpoints(os.path.join(root, NotebookAnalyser.CHECKPOINT_DIR))
|
||||
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description="Remove checkpointed versions of those jupyter notebooks that are identical to their working copies.",
|
||||
epilog="""Notebooks will be reported as either
|
||||
"DELETED" if the working copy and checkpointed version are identical
|
||||
(checkpoint will be deleted),
|
||||
"missing" if there is a checkpoint but no corresponding working file can be found
|
||||
or "modified" if notebook and the checkpoint are not byte-to-byte identical.
|
||||
If removal of checkpoints results in empty ".ipynb_checkpoints" directory
|
||||
that directory is also deleted.
|
||||
""") #, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("dirs", metavar="DIR", type=str, nargs="*", default=".", help="directories to search")
|
||||
parser.add_argument("-d", "--dry-run", action="store_true", help="only print messages, don't perform any removals")
|
||||
parser.add_argument("-v", "--verbose", action="store_true", help="verbose mode")
|
||||
parser.add_argument("-c", "--color", action="store_true", help="colorful mode")
|
||||
args = parser.parse_args()
|
||||
|
||||
analyser = NotebookAnalyser(args.dry_run, args.verbose, args.color)
|
||||
for directory in args.dirs:
|
||||
analyser.clean_checkpoints_recursively(directory)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+17
@@ -0,0 +1,17 @@
|
||||
#!/bin/bash
|
||||
if [[ "$#" -lt 1 || "$1" =~ ^((-h)|(--help))$ ]] ; then
|
||||
echo "usage: nbdiff_checkpoint NOTEBOOK.ipynb"
|
||||
echo
|
||||
echo "Show differences between given jupyter notebook and its checkpointed version (in .ipynb_checkpoints subdirectory)"
|
||||
exit
|
||||
fi
|
||||
|
||||
DIRNAME=$(dirname "$1")
|
||||
BASENAME=$(basename "$1" .ipynb)
|
||||
shift
|
||||
|
||||
WORKING_COPY=$DIRNAME/$BASENAME.ipynb
|
||||
CHECKPOINT_COPY=$DIRNAME/.ipynb_checkpoints/$BASENAME-checkpoint.ipynb
|
||||
|
||||
echo "----- Analysing how to change $CHECKPOINT_COPY into $WORKING_COPY -----"
|
||||
nbdiff "$CHECKPOINT_COPY" "$WORKING_COPY" --ignore-details "$@"
|
||||
Executable
+54
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import os
|
||||
|
||||
def remove_empty_directories(initial_dir,
|
||||
allow_initial_delete=False, ignore_nonexistant_initial=False,
|
||||
dry_run=False, quiet=False):
|
||||
|
||||
FORBIDDEN_SUBDIRS = set([".git"])
|
||||
|
||||
if not os.path.isdir(initial_dir) and not ignore_nonexistant_initial:
|
||||
raise RuntimeError("Initial directory '{}' not found!".format(initial_dir))
|
||||
|
||||
message = "removed"
|
||||
if dry_run:
|
||||
message = "to be " + message
|
||||
|
||||
deleted = set()
|
||||
|
||||
for (directory, subdirs, files) in os.walk(initial_dir, topdown=False):
|
||||
forbidden = False
|
||||
parent = directory
|
||||
while parent:
|
||||
parent, dirname = os.path.split(parent)
|
||||
if dirname in FORBIDDEN_SUBDIRS:
|
||||
forbidden = True
|
||||
break
|
||||
if forbidden:
|
||||
continue
|
||||
|
||||
is_empty = len(files) < 1 and len(set([os.path.join(directory, s) for s in subdirs]) - deleted) < 1
|
||||
|
||||
if is_empty and (initial_dir != directory or allow_initial_delete):
|
||||
if not quiet:
|
||||
print("{}: {}".format(message, directory))
|
||||
deleted.add(directory)
|
||||
if not dry_run:
|
||||
os.rmdir(directory)
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description="Remove empty directories recursively in subtree.")
|
||||
parser.add_argument("dir", metavar="DIR", type=str, nargs="+", help="directory to be searched")
|
||||
parser.add_argument("-r", "--allow-dir-removal", action="store_true", help="allow deletion of DIR itself")
|
||||
parser.add_argument("-i", "--ignore-nonexistent-dir", action="store_true", help="don't throw an error if DIR doesn't exist")
|
||||
parser.add_argument("-d", "--dry-run", action="store_true", help="only print messages, don't perform any removals")
|
||||
parser.add_argument("-q", "--quiet", action="store_true", help="don't print names of directories being removed")
|
||||
args = parser.parse_args()
|
||||
for directory in args.dir:
|
||||
remove_empty_directories(directory, args.allow_dir_removal, args.ignore_nonexistent_dir,
|
||||
args.dry_run, args.quiet)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+2
@@ -0,0 +1,2 @@
|
||||
#!/bin/bash
|
||||
python -m tensorboard.main "$@"
|
||||
@@ -0,0 +1,27 @@
|
||||
version: "3"
|
||||
services:
|
||||
handson-ml3:
|
||||
build:
|
||||
context: ../
|
||||
dockerfile: ./docker/Dockerfile #Dockerfile.gpu
|
||||
args:
|
||||
- username=devel
|
||||
- userid=1000
|
||||
container_name: handson-ml3
|
||||
image: ageron/handson-ml3:latest #latest-gpu
|
||||
restart: unless-stopped
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: 50m
|
||||
ports:
|
||||
- "8888:8888"
|
||||
- "6006:6006"
|
||||
volumes:
|
||||
- ../:/home/devel/handson-ml3
|
||||
command: /opt/conda/envs/homl3/bin/jupyter lab --ip='0.0.0.0' --port=8888 --no-browser
|
||||
#deploy:
|
||||
# resources:
|
||||
# reservations:
|
||||
# devices:
|
||||
# - capabilities: [gpu]
|
||||
@@ -0,0 +1,15 @@
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
def export_script_and_view(model, os_path, contents_manager):
|
||||
if model["type"] != "notebook":
|
||||
return
|
||||
dir_name, file_name = os.path.split(os_path)
|
||||
file_base, file_ext = os.path.splitext(file_name)
|
||||
if file_base.startswith("Untitled"):
|
||||
return
|
||||
export_name = file_base if file_ext == ".ipynb" else file_name
|
||||
subprocess.check_call(["jupyter", "nbconvert", "--to", "script", file_name, "--output", export_name + "_script"], cwd=dir_name)
|
||||
subprocess.check_call(["jupyter", "nbconvert", "--to", "html", file_name, "--output", export_name + "_view"], cwd=dir_name)
|
||||
|
||||
c.FileContentsManager.post_save_hook = export_script_and_view
|
||||
@@ -0,0 +1,45 @@
|
||||
name: homl3
|
||||
channels:
|
||||
- conda-forge
|
||||
dependencies:
|
||||
- box2d-py=2.3 # used only in chapter 18, exercise 8
|
||||
- ffmpeg=6.1 # used only in the matplotlib tutorial to generate animations
|
||||
- graphviz # used only in chapter 6 for dot files
|
||||
- python-graphviz # used only in chapter 6 for dot files
|
||||
- ipython=8.17 # a powerful Python shell
|
||||
- ipywidgets=8.1 # optionally used only in chapter 12 for tqdm in Jupyter
|
||||
- joblib=1.3 # used only in chapter 2 to save/load Scikit-Learn models
|
||||
- jupyterlab=4.0 # to edit and run Jupyter notebooks
|
||||
- jupyter=1.1 # to edit and run Jupyter notebooks (classical UI)
|
||||
- matplotlib=3.8 # beautiful plots. See tutorial tools_matplotlib.ipynb
|
||||
- nbdime=3.2 # optional tool to diff Jupyter notebooks
|
||||
- nltk=3.8 # optionally used in chapter 3, exercise 4
|
||||
- numexpr=2.8 # used only in the Pandas tutorial for numerical expressions
|
||||
- numpy=1.26 # Powerful n-dimensional arrays and numerical computing tools
|
||||
- pandas=2.1 # data analysis and manipulation tool
|
||||
- pillow=10.1 # image manipulation library, (used by matplotlib.image.imread)
|
||||
- pip # Python's package-management system
|
||||
- py-xgboost=1.7 # used only in chapter 6 for optimized Gradient Boosting
|
||||
- pydot=1.4 # used only for in chapter 10 for tf.keras.utils.plot_model()
|
||||
- python=3.10 # your beloved programming language! :)
|
||||
- requests=2.31 # used only in chapter 19 for REST API queries
|
||||
- scikit-learn=1.3 # machine learning library
|
||||
- scipy=1.11 # scientific/technical computing library
|
||||
- statsmodels=0.14 # used only in chapter 15 for time series analysis
|
||||
- tqdm=4.66 # used only in chapter 12 to display nice progress bars
|
||||
- wheel # built-package format for pip
|
||||
- widgetsnbextension=4.0 # interactive HTML widgets for Jupyter notebooks
|
||||
- pip:
|
||||
- keras-core # used in chapter 10
|
||||
- keras-tuner~=1.4.6 # used in chapters 10 and 19 for hyperparameter tuning
|
||||
- tensorboard-plugin-profile~=2.14.0 # profiling plugin for TensorBoard
|
||||
- tensorboard~=2.14.1 # TensorFlow's visualization toolkit
|
||||
- tensorflow-datasets~=4.9.3 # datasets repository, ready to use
|
||||
- tensorflow-hub~=0.15.0 # trained ML models repository, ready to use
|
||||
- tensorflow-serving-api~=2.14.0 # or tensorflow-serving-api-gpu if gpu
|
||||
- tensorflow~=2.14.0 # Deep Learning library
|
||||
- transformers~=4.35.0 # Natural Language Processing lib for TF or PyTorch
|
||||
- urlextract~=1.8.0 # optionally used in chapter 3, exercise 4
|
||||
- gymnasium[classic_control,atari] # used only in ch18
|
||||
- google-cloud-aiplatform~=1.36.2 # used only in chapter 19
|
||||
- google-cloud-storage~=2.13.0 # used only in chapter 19
|
||||
@@ -0,0 +1,244 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6a3bb8d7-dd99-4784-b1e7-73e8392506b6",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# A Few Extra ANN Architectures"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0bcf9ed3-d166-4b31-a770-7dee78ba7faa",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In this notebook I will give a quick overview of a few historically important neural network architectures that are much less used today than deep Multilayer Perceptrons (chapter 10), convolutional neural networks (chapter 14), recurrent neural networks (chapter 15), attention networks (chapter 16), autoencoders, generative adversarial networks, or diffusion models (chapter 17). They are often mentioned in the literature, and some are still used in a range of applications, so it is worth knowing about them. Additionally, we will discuss _deep belief nets_, which were the state of the art in Deep Learning until the early 2010s. They are still the subject of active research, so they may well come back with a vengeance in the future."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2c0aa7cf-4b8c-470e-bda1-7a7356e256e1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Hopfield Networks\n",
|
||||
"_Hopfield networks_ were first introduced by W. A. Little in 1974, then popularized by J. Hopfield in 1982. They are _associative memory_ networks: you first teach them some patterns, and then when they see a new pattern they (hopefully) output the closest learned pattern. This made them useful for character recognition, in particular, before they were outperformed by other approaches: you first train the network by showing it examples of character images (each binary pixel maps to one neuron), and then when you show it a new character image, after a few iterations it outputs the closest learned character.\n",
|
||||
"\n",
|
||||
"Hopfield networks are fully connected graphs (see figure 1 below); that is, every neuron is connected to every other neuron. Note that in the diagram the images are 6 × 6 pixels, so the neural network on the left should contain 36 neurons (and 630 connections), but for visual clarity a much smaller network is represented."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"id": "dca9121c-5a23-4029-bc6a-8155bb9935b7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Figure 1**: _Hopfield network_\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "53434a2d-a9ea-4840-906e-55e5562148f2",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The training algorithm works by using Hebb's rule (see the section about Perceptrons in chapter 10): for each training image, the weight between two neurons is increased if the corresponding pixels are both on or both off, but decreased if one pixel is on and the other is off.\n",
|
||||
"\n",
|
||||
"To show a new image to the network, you just activate the neurons that correspond to active pixels. The network then computes the output of every neuron, and this gives you a new image. You can then take this new image and repeat the whole process. After a while, the network reaches a stable state. Generally, this corresponds to the training image that most resembles the input image.\n",
|
||||
"\n",
|
||||
"A so-called _energy function_ is associated with Hopfield nets. At each iteration, the energy decreases, so the network is guaranteed to eventually stabilize to a low-energy state. The training algorithm tweaks the weights in a way that decreases the energy level of the training patterns, so the network is likely to stabilize in one of these low-energy configurations. Unfortunately, some patterns that were not in the training set also end up with low energy, so the network sometimes stabilizes in a configuration that was not learned. These are called _spurious patterns_.\n",
|
||||
"\n",
|
||||
"Another major flaw with Hopfield nets is that they don't scale very well–their memory capacity is roughly equal to 14% of the number of neurons. For example, to classify 28 × 28–pixel images, you would need a Hopfield net with 784 fully connected neurons and 306,936 weights. Such a network would only be able to learn about 110 different characters (14% of 784). That's a lot of parameters for such a small memory."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1e769531-a97c-4c60-acb8-32b97b681011",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Boltzmann Machines\n",
|
||||
"_Boltzmann machines_ were invented in 1985 by Geoffrey Hinton and Terrence Sejnowski. Just like Hopfield nets, they are fully connected ANNs, but they are based on _stochastic neurons_: instead of using a deterministic step function to decide what value to output, these neurons output 1 with some probability, and 0 otherwise. The probability function that these ANNs use is based on the Boltzmann distribution (used in statistical mechanics), hence their name. See equation 1.\n",
|
||||
"\n",
|
||||
"**Equation 1**: _probability that a particular neuron will output 1_\n",
|
||||
"$$P(s_i^\\text{next step}=1) = \\sigma\\left(\\dfrac{\\sum_{j=1}^N{w_{i,j}s_j + b_i}}{T}\\right)$$\n",
|
||||
"\n",
|
||||
"* $s_j$ is the $j^\\text{th}$ neuron's state (0 or 1).\n",
|
||||
"* $w_{i,j}$ is the connection weight between the $i^\\text{th}$ and $j^\\text{th}$ neurons. Note that $w_{i,i}$ = 0.\n",
|
||||
"* $b_i$ is the $i^\\text{th}$ neuron's bias term. We can implement this term by adding a bias neuron to the network.\n",
|
||||
"* _N_ is the number of neurons in the network.\n",
|
||||
"* _T_ is a number called the network's _temperature_; the higher the temperature, the more random the output is (i.e., the more the probability approaches 50%).\n",
|
||||
"* $\\sigma$ is the logistic function.\n",
|
||||
"\n",
|
||||
"Neurons in Boltzmann machines are separated into two groups: _visible units_ and _hidden units_ (see figure 2). All neurons work in the same stochastic way, but the visible units are the ones that receive the inputs and from which outputs are read."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9d2d1d0c-5dc8-4da7-904b-0928900e4511",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Figure 2**: Boltzmann machine\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "be861bf1-0611-4b64-8f8e-830ea201ebc0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Because of its stochastic nature, a Boltzmann machine will never stabilize into a fixed configuration; instead, it will keep switching between many configurations. If it is left running for a sufficiently long time, the probability of observing a particular configuration will only be a function of the connection weights and bias terms, not of the original configuration (similarly, after you shuffle a deck of cards for long enough, the configuration of the deck does not depend on the initial state). When the network reaches this state where the original configuration is \"forgotten,\" it is said to be in _thermal equilibrium_ (although its configuration keeps changing all the time). By setting the network parameters appropriately, letting the network reach thermal equilibrium, and then observing its state, we can simulate a wide range of probability distributions. This is a type of _generative model_.\n",
|
||||
"\n",
|
||||
"Training a Boltzmann machine means finding the parameters that will make the network approximate the training set's probability distribution. For example, if there are three visible neurons and the training set contains 75% (0, 1, 1) triplets, 10% (0, 0, 1) triplets, and 15% (1, 1, 1) triplets, then after training a Boltzmann machine, you could use it to generate random binary triplets with about the same probability distribution. For example, about 75% of the time it would output the (0, 1, 1) triplet.\n",
|
||||
"\n",
|
||||
"Such a generative model can be used in a variety of ways. For example, if it is trained on images, and you provide an incomplete or noisy image to the network, it will automatically \"repair\" the image in a reasonable way. You can also use a generative model for classification. Just add a few visible neurons to encode the training image's class (e.g., add 10 visible neurons and turn on only the fifth neuron when the training image represents a 5). Then, when given a new image, the network will automatically turn on the appropriate visible neurons, indicating the image's class (e.g., it will turn on the fifth visible neuron if the image represents a 5).\n",
|
||||
"\n",
|
||||
"Unfortunately, there is no efficient technique to train Boltzmann machines. However, fairly efficient algorithms have been developed to train _restricted Boltzmann machines_ (RBMs)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "08878949-d5bf-4862-b8c9-71041522a10a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Restricted Boltzmann Machines\n",
|
||||
"An RBM is simply a Boltzmann machine in which there are no connections between visible units or between hidden units, only between visible and hidden units. For example, figure 3 represents an RBM with three visible units and four hidden units."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7a22c8c4-b186-411d-88ca-9c1bf89f1a5b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Figure 3**: _Restricted Boltzmann machine_\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "56ff7321-7891-4c9e-bc63-c7097332d3c5",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"A very efficient training algorithm called [Contrastive Divergence](https://homl.info/135) was introduced in 2005 by Miguel Á. Carreira-Perpiñán and Geoffrey Hinton.\n",
|
||||
"\n",
|
||||
"> Reference: Miguel Á. Carreira-Perpiñán and Geoffrey E. Hinton, \"On Contrastive Divergence Learning,\" _Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics_ (2005): 59–66.\n",
|
||||
"\n",
|
||||
"Here is how it works: for each training instance **x**, the algorithm starts by feeding it to the network by setting the state of the visible units to _x_<sub>1</sub>, _x_<sub>2</sub>, ..., _x_<sub>_n_</sub>. Then you compute the state of the hidden units by applying the stochastic equation described above. This gives you a hidden vector **h** (where _h_<sub>_i_</sub> is equal to the state of the _i_<sup>th</sup> unit). Next you compute the state of the visible units, by applying the same stochastic equation. This gives you a vector **xʹ**. Then once again you compute the state of the hidden units, which gives you a vector **hʹ**. Now you can update each connection weight by applying the rule in equation 2, where _η_ is the learning rate."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1f67389d-cb84-42af-a29e-e6537e55cf2a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Equation 2**: _Contrastive divergence weight update_\n",
|
||||
"$$w_{i,j} \\leftarrow w_{i,j} + \\eta(\\mathbf{xh}^\\intercal - \\mathbf{x'h'}^\\intercal)$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1b02d62a-44e3-4d4c-bd87-304b89b4a783",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The great benefit of this algorithm is that it does not require waiting for the network to reach thermal equilibrium: it just goes forward, backward, and forward again, and that's it. This makes it incomparably more efficient than previous algorithms, and it was a key ingredient to the first success of Deep Learning based on multiple stacked RBMs."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "593cf35c-dfa7-4335-9014-38427c58cd58",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Deep Belief Nets\n",
|
||||
"Several layers of RBMs can be stacked; the hidden units of the first-level RBM serve as the visible units for the second-layer RBM, and so on. Such an RBM stack is called a _deep belief net_ (DBN).\n",
|
||||
"\n",
|
||||
"Yee-Whye Teh, one of Geoffrey Hinton's students, observed that it was possible to train DBNs one layer at a time using Contrastive Divergence, starting with the lower layers and then gradually moving up to the top layers. This led to the [groundbreaking article that kickstarted the Deep Learning tsunami in 2006](https://homl.info/136).\n",
|
||||
"\n",
|
||||
"> Reference: Geoffrey E. Hinton et al., \"A Fast Learning Algorithm for Deep Belief Nets,\" _Neural Computation_ 18 (2006): 1527–1554.\n",
|
||||
"\n",
|
||||
"Just like RBMs, DBNs learn to reproduce the probability distribution of their inputs, without any supervision. However, they are much better at it, for the same reason that deep neural networks are more powerful than shallow ones: real-world data is often organized in hierarchical patterns, and DBNs take advantage of that. Their lower layers learn low-level features in the input data, while higher layers learn high-level features.\n",
|
||||
"\n",
|
||||
"Just like RBMs, DBNs are fundamentally unsupervised, but you can also train them in a supervised manner by adding some visible units to represent the labels. Moreover, one great feature of DBNs is that they can be trained in a semisupervised fashion. Figure 4 represents such a DBN configured for semisupervised learning."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b0d4a41d-7c6d-4c2b-a2ac-cf5a4db97a53",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Figure 4**: _A deep belief network configured for semisupervised learning_\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "40defe73-5ea2-4f4b-947a-8657138556a4",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"First, RBM 1 is trained without supervision. It learns low-level features in the training data. Then RBM 2 is trained with RBM 1's hidden units as inputs, again without supervision: it learns higher-level features (note that RBM 2's hidden units include only the three rightmost units, not the label units). Several more RBMs could be stacked this way, but you get the idea. So far, training was 100% unsupervised. Lastly, RBM 3 is trained using RBM 2's hidden units as inputs, as well as extra visible units used to represent the target labels (e.g., a one-hot vector representing the instance class). It learns to associate high-level features with training labels. This is the supervised step.\n",
|
||||
"\n",
|
||||
"At the end of training, if you feed RBM 1 a new instance, the signal will propagate up to RBM 2, then up to the top of RBM 3, and then back down to the label units; hopefully, the appropriate label will light up. This is how a DBN can be used for classification.\n",
|
||||
"\n",
|
||||
"One great benefit of this semisupervised approach is that you don't need much labeled training data. If the unsupervised RBMs do a good enough job, then only a small amount of labeled training instances per class will be necessary. Similarly, a baby learns to recognize objects without supervision, so when you point to a chair and say \"chair,\" the baby can associate the word \"chair\" with the class of objects it has already learned to recognize on its own. You don't need to point to every single chair and say \"chair\"; only a few examples will suffice (just enough so the baby can be sure that you are indeed referring to the chair, not to its color or one of the chair's parts).\n",
|
||||
"\n",
|
||||
"Quite amazingly, DBNs can also work in reverse. If you activate one of the label units, the signal will propagate up to the hidden units of RBM 3, then down to RBM 2, and then RBM 1, and a new instance will be output by the visible units of RBM 1. This new instance will usually look like a regular instance of the class whose label unit you activated. This generative capability of DBNs is quite powerful. For example, it has been used to automatically generate captions for images, and vice versa: first a DBN is trained (without supervision) to learn features in images, and another DBN is trained (again without supervision) to learn features in sets of captions (e.g., \"car\" often comes with \"automobile\"). Then an RBM is stacked on top of both DBNs and trained with a set of images along with their captions; it learns to associate high-level features in images with high-level features in captions. Next, if you feed the image DBN an image of a car, the signal will propagate through the network, up to the top-level RBM, and back down to the bottom of the caption DBN, producing a caption. Due to the stochastic nature of RBMs and DBNs, the caption will keep changing randomly, but it will generally be appropriate for the image. If you generate a few hundred captions, the most frequently generated ones will likely be a good description of the image (see [this video](https://homl.info/137) by Geoffrey Hinton for more details and a demo)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "771cc450-0a02-4bec-ac85-ce0de87bfb84",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Self-Organizing Maps\n",
|
||||
"_Self-organizing maps_ (SOMs) are quite different from all the other types of neural networks we have discussed so far. They are used to produce a low-dimensional representation of a high-dimensional dataset, generally for visualization, clustering, or classification. The neurons are spread across a map (typically 2D for visualization, but it can be any number of dimensions you want), as shown in figure 5, and each neuron has a weighted connection to every input (note that the diagram shows just two inputs, but there are typically a very large number, since the whole point of SOMs is to reduce dimensionality)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5dab2cb8-133f-48f3-bcff-598113cf390e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Figure 5**: _Self-organizing map_\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0eb0405b-bbf0-4294-b9ae-7a0fc4461c8b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Once the network is trained, you can feed it a new instance and this will activate only one neuron (i.e., one point on the map): the neuron whose weight vector is closest to the input vector. In general, instances that are nearby in the original input space will activate neurons that are nearby on the map. This makes SOMs useful not only for visualization (in particular, you can easily identify clusters on the map), but also for applications like speech recognition. For example, if each instance represents an audio recording of a person pronouncing a vowel, then different pronunciations of the vowel \"a\" will activate neurons in the same area of the map, while instances of the vowel \"e\" will activate neurons in another area, and intermediate sounds will generally activate intermediate neurons on the map.\n",
|
||||
"\n",
|
||||
"> **Note**: One important difference from the other dimensionality reduction techniques discussed in chapter 8 is that all instances get mapped to a discrete number of points in the low-dimensional space (one point per neuron). When there are very few neurons, this technique is better described as clustering rather than dimensionality reduction.\n",
|
||||
"\n",
|
||||
"The training algorithm is unsupervised. It works by having all the neurons compete against each other. First, all the weights are initialized randomly. Then a training instance is picked randomly and fed to the network. All neurons compute the distance between their weight vector and the input vector (this is very different from the artificial neurons we have seen so far). The neuron that measures the smallest distance wins and tweaks its weight vector to be slightly closer to the input vector, making it more likely to win future competitions for other inputs similar to this one. It also recruits its neighboring neurons, and they too update their weight vectors to be slightly closer to the input vector (but they don't update their weights as much as the winning neuron). Then the algorithm picks another training instance and repeats the process, again and again. This algorithm tends to make nearby neurons gradually specialize in similar inputs.\n",
|
||||
"\n",
|
||||
"Here's an anology: imagine a class of young children with roughly similar skills. One child happens to be slightly better at basketball. This motivates her to practice more, especially with her friends. After a while, this group of friends gets so good at basketball that other kids cannot compete. But that's okay, because the other kids specialize in other areas. After a while, the class is full of little specialized groups.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.13"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
+148
@@ -0,0 +1,148 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Machine Learning Notebooks\n",
|
||||
"\n",
|
||||
"*Welcome to the Machine Learning Notebooks!*\n",
|
||||
"\n",
|
||||
"[Prerequisites](#Prerequisites) (see below)\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/ageron/handson-ml3/blob/main/index.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a target=\"_blank\" href=\"https://homl.info/kaggle3/\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" /></a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Notebooks\n",
|
||||
"1. [The Machine Learning landscape](01_the_machine_learning_landscape.ipynb)\n",
|
||||
"2. [End-to-end Machine Learning project](02_end_to_end_machine_learning_project.ipynb)\n",
|
||||
"3. [Classification](03_classification.ipynb)\n",
|
||||
"4. [Training Models](04_training_linear_models.ipynb)\n",
|
||||
"5. [Support Vector Machines](05_support_vector_machines.ipynb)\n",
|
||||
"6. [Decision Trees](06_decision_trees.ipynb)\n",
|
||||
"7. [Ensemble Learning and Random Forests](07_ensemble_learning_and_random_forests.ipynb)\n",
|
||||
"8. [Dimensionality Reduction](08_dimensionality_reduction.ipynb)\n",
|
||||
"9. [Unsupervised Learning Techniques](09_unsupervised_learning.ipynb)\n",
|
||||
"10. [Artificial Neural Nets with Keras](10_neural_nets_with_keras.ipynb)\n",
|
||||
"11. [Training Deep Neural Networks](11_training_deep_neural_networks.ipynb)\n",
|
||||
"12. [Custom Models and Training with TensorFlow](12_custom_models_and_training_with_tensorflow.ipynb)\n",
|
||||
"13. [Loading and Preprocessing Data](13_loading_and_preprocessing_data.ipynb)\n",
|
||||
"14. [Deep Computer Vision Using Convolutional Neural Networks](14_deep_computer_vision_with_cnns.ipynb)\n",
|
||||
"15. [Processing Sequences Using RNNs and CNNs](15_processing_sequences_using_rnns_and_cnns.ipynb)\n",
|
||||
"16. [Natural Language Processing with RNNs and Attention](16_nlp_with_rnns_and_attention.ipynb)\n",
|
||||
"17. [Autoencoders, GANs, and Diffusion Models](17_autoencoders_gans_and_diffusion_models.ipynb)\n",
|
||||
"18. [Reinforcement Learning](18_reinforcement_learning.ipynb)\n",
|
||||
"19. [Training and Deploying TensorFlow Models at Scale](19_training_and_deploying_at_scale.ipynb)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Scientific Python tutorials\n",
|
||||
"* [NumPy](tools_numpy.ipynb)\n",
|
||||
"* [Matplotlib](tools_matplotlib.ipynb)\n",
|
||||
"* [Pandas](tools_pandas.ipynb)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Math Tutorials\n",
|
||||
"* [Linear Algebra](math_linear_algebra.ipynb)\n",
|
||||
"* [Differential Calculus](math_differential_calculus.ipynb)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Extra Material\n",
|
||||
"* [Auto-differentiation](extra_autodiff.ipynb)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Misc.\n",
|
||||
"* [Equations](book_equations.pdf) (list of equations in the book)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Prerequisites"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### To understand\n",
|
||||
"* **Python** – you don't need to be an expert python programmer, but you do need to know the basics. If you don't, the official [Python tutorial](https://docs.python.org/3/tutorial/) is a good place to start.\n",
|
||||
"* **Scientific Python** – We will be using a few popular python libraries, in particular NumPy, matplotlib and pandas. If you are not familiar with these libraries, you should probably start by going through the tutorials in the Tools section (especially NumPy).\n",
|
||||
"* **Math** – We will also use some notions of Linear Algebra, Calculus, Statistics and Probability theory. You should be able to follow along if you learned these in the past as it won't be very advanced, but if you don't know about these topics or you need a refresher then go through the appropriate introduction in the Math section."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### To run the examples\n",
|
||||
"* **Jupyter** – These notebooks are based on Jupyter. You can run these notebooks in just one click using a hosted platform such as Binder, Deepnote or Colaboratory (no installation required), or you can just view them using Jupyter.org's viewer, or you can install everything on your machine, as you prefer. Check out the [home page](https://github.com/ageron/handson-ml3/) for more details."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.6"
|
||||
},
|
||||
"nav_menu": {},
|
||||
"toc": {
|
||||
"navigate_menu": true,
|
||||
"number_sections": true,
|
||||
"sideBar": true,
|
||||
"threshold": 6,
|
||||
"toc_cell": false,
|
||||
"toc_section_display": "block",
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,129 @@
|
||||
This checklist can guide you through your Machine Learning projects. There are eight main steps:
|
||||
|
||||
1. Frame the problem and look at the big picture.
|
||||
2. Get the data.
|
||||
3. Explore the data to gain insights.
|
||||
4. Prepare the data to better expose the underlying data patterns to Machine Learning algorithms.
|
||||
5. Explore many different models and short-list the best ones.
|
||||
6. Fine-tune your models and combine them into a great solution.
|
||||
7. Present your solution.
|
||||
8. Launch, monitor, and maintain your system.
|
||||
|
||||
Obviously, you should feel free to adapt this checklist to your needs.
|
||||
|
||||
# Frame the problem and look at the big picture
|
||||
1. Define the objective in business terms.
|
||||
2. How will your solution be used?
|
||||
3. What are the current solutions/workarounds (if any)?
|
||||
4. How should you frame this problem (supervised/unsupervised, online/offline, etc.)
|
||||
5. How should performance be measured?
|
||||
6. Is the performance measure aligned with the business objective?
|
||||
7. What would be the minimum performance needed to reach the business objective?
|
||||
8. What are comparable problems? Can you reuse experience or tools?
|
||||
9. Is human expertise available?
|
||||
10. How would you solve the problem manually?
|
||||
11. List the assumptions you or others have made so far.
|
||||
12. Verify assumptions if possible.
|
||||
|
||||
# Get the data
|
||||
Note: automate as much as possible so you can easily get fresh data.
|
||||
|
||||
1. List the data you need and how much you need.
|
||||
2. Find and document where you can get that data.
|
||||
3. Check how much space it will take.
|
||||
4. Check legal obligations, and get the authorization if necessary.
|
||||
5. Get access authorizations.
|
||||
6. Create a workspace (with enough storage space).
|
||||
7. Get the data.
|
||||
8. Convert the data to a format you can easily manipulate (without changing the data itself).
|
||||
9. Ensure sensitive information is deleted or protected (e.g., anonymized).
|
||||
10. Check the size and type of data (time series, sample, geographical, etc.).
|
||||
11. Sample a test set, put it aside, and never look at it (no data snooping!).
|
||||
|
||||
# Explore the data
|
||||
Note: try to get insights from a field expert for these steps.
|
||||
|
||||
1. Create a copy of the data for exploration (sampling it down to a manageable size if necessary).
|
||||
2. Create a Jupyter notebook to keep record of your data exploration.
|
||||
3. Study each attribute and its characteristics:
|
||||
- Name
|
||||
- Type (categorical, int/float, bounded/unbounded, text, structured, etc.)
|
||||
- % of missing values
|
||||
- Noisiness and type of noise (stochastic, outliers, rounding errors, etc.)
|
||||
- Possibly useful for the task?
|
||||
- Type of distribution (Gaussian, uniform, logarithmic, etc.)
|
||||
4. For supervised learning tasks, identify the target attribute(s).
|
||||
5. Visualize the data.
|
||||
6. Study the correlations between attributes.
|
||||
7. Study how you would solve the problem manually.
|
||||
8. Identify the promising transformations you may want to apply.
|
||||
9. Identify extra data that would be useful (go back to "Get the Data" on page 502).
|
||||
10. Document what you have learned.
|
||||
|
||||
# Prepare the data
|
||||
Notes:
|
||||
- Work on copies of the data (keep the original dataset intact).
|
||||
- Write functions for all data transformations you apply, for five reasons:
|
||||
- So you can easily prepare the data the next time you get a fresh dataset
|
||||
- So you can apply these transformations in future projects
|
||||
- To clean and prepare the test set
|
||||
- To clean and prepare new data instances
|
||||
- To make it easy to treat your preparation choices as hyperparameters
|
||||
|
||||
1. Data cleaning:
|
||||
- Fix or remove outliers (optional).
|
||||
- Fill in missing values (e.g., with zero, mean, median...) or drop their rows (or columns).
|
||||
2. Feature selection (optional):
|
||||
- Drop the attributes that provide no useful information for the task.
|
||||
3. Feature engineering, where appropriate:
|
||||
- Discretize continuous features.
|
||||
- Decompose features (e.g., categorical, date/time, etc.).
|
||||
- Add promising transformations of features (e.g., log(x), sqrt(x), x^2, etc.).
|
||||
- Aggregate features into promising new features.
|
||||
4. Feature scaling: standardize or normalize features.
|
||||
|
||||
# Short-list promising models
|
||||
Notes:
|
||||
- If the data is huge, you may want to sample smaller training sets so you can train many different models in a reasonable time (be aware that this penalizes complex models such as large neural nets or Random Forests).
|
||||
- Once again, try to automate these steps as much as possible.
|
||||
|
||||
1. Train many quick and dirty models from different categories (e.g., linear, naive, Bayes, SVM, Random Forests, neural net, etc.) using standard parameters.
|
||||
2. Measure and compare their performance.
|
||||
- For each model, use N-fold cross-validation and compute the mean and standard deviation of their performance.
|
||||
3. Analyze the most significant variables for each algorithm.
|
||||
4. Analyze the types of errors the models make.
|
||||
- What data would a human have used to avoid these errors?
|
||||
5. Have a quick round of feature selection and engineering.
|
||||
6. Have one or two more quick iterations of the five previous steps.
|
||||
7. Short-list the top three to five most promising models, preferring models that make different types of errors.
|
||||
|
||||
# Fine-Tune the System
|
||||
Notes:
|
||||
- You will want to use as much data as possible for this step, especially as you move toward the end of fine-tuning.
|
||||
- As always automate what you can.
|
||||
|
||||
1. Fine-tune the hyperparameters using cross-validation.
|
||||
- Treat your data transformation choices as hyperparameters, especially when you are not sure about them (e.g., should I replace missing values with zero or the median value? Or just drop the rows?).
|
||||
- Unless there are very few hyperparameter values to explore, prefer random search over grid search. If training is very long, you may prefer a Bayesian optimization approach (e.g., using a Gaussian process priors, as described by Jasper Snoek, Hugo Larochelle, and Ryan Adams ([https://goo.gl/PEFfGr](https://goo.gl/PEFfGr)))
|
||||
2. Try Ensemble methods. Combining your best models will often perform better than running them individually.
|
||||
3. Once you are confident about your final model, measure its performance on the test set to estimate the generalization error.
|
||||
|
||||
> Don't tweak your model after measuring the generalization error: you would just start overfitting the test set.
|
||||
|
||||
# Present your solution
|
||||
1. Document what you have done.
|
||||
2. Create a nice presentation.
|
||||
- Make sure you highlight the big picture first.
|
||||
3. Explain why your solution achieves the business objective.
|
||||
4. Don't forget to present interesting points you noticed along the way.
|
||||
- Describe what worked and what did not.
|
||||
- List your assumptions and your system's limitations.
|
||||
5. Ensure your key findings are communicated through beautiful visualizations or easy-to-remember statements (e.g., "the median income is the number-one predictor of housing prices").
|
||||
|
||||
# Launch!
|
||||
1. Get your solution ready for production (plug into production data inputs, write unit tests, etc.).
|
||||
2. Write monitoring code to check your system's live performance at regular intervals and trigger alerts when it drops.
|
||||
- Beware of slow degradation too: models tend to "rot" as data evolves.
|
||||
- Measuring performance may require a human pipeline (e.g., via a crowdsourcing service).
|
||||
- Also monitor your inputs' quality (e.g., a malfunctioning sensor sending random values, or another team's output becoming stale). This is particularly important for online learning systems.
|
||||
3. Retrain your models on a regular basis on fresh data (automate as much as possible).
|
||||
@@ -0,0 +1,92 @@
|
||||
# TensorFlow is much easier to install using Anaconda, especially
|
||||
# on Windows or when using a GPU. Please see the installation
|
||||
# instructions in INSTALL.md
|
||||
|
||||
|
||||
##### Core scientific packages
|
||||
jupyterlab~=4.0.8
|
||||
matplotlib~=3.8.1
|
||||
numpy~=1.26.2
|
||||
pandas~=2.1.3
|
||||
scipy~=1.11.3
|
||||
|
||||
##### Machine Learning packages
|
||||
scikit-learn~=1.3.2
|
||||
|
||||
# Optional: the XGBoost library is only used in chapter 7
|
||||
xgboost~=2.0.2
|
||||
|
||||
# Optional: the transformers library is only used in chapter 16
|
||||
transformers~=4.35.0
|
||||
|
||||
##### TensorFlow-related packages
|
||||
|
||||
# If you have a TF-compatible GPU and you want to enable GPU support, then
|
||||
# replace tensorflow-serving-api with tensorflow-serving-api-gpu.
|
||||
# Your GPU must have CUDA Compute Capability 3.5 or higher support, and
|
||||
# you must install CUDA, cuDNN and more: see tensorflow.org for the detailed
|
||||
# installation instructions.
|
||||
|
||||
tensorflow~=2.14.0
|
||||
|
||||
# Optional: the TF Serving API library is just needed for chapter 18.
|
||||
tensorflow-serving-api~=2.14.0 # or tensorflow-serving-api-gpu if gpu
|
||||
|
||||
tensorboard~=2.14.1
|
||||
tensorboard-plugin-profile~=2.14.0
|
||||
tensorflow-datasets~=4.9.3
|
||||
tensorflow-hub~=0.15.0
|
||||
|
||||
# Used in chapter 10 and 19 for hyperparameter tuning
|
||||
keras-tuner~=1.4.6
|
||||
|
||||
##### Reinforcement Learning library (chapter 18)
|
||||
|
||||
# There are a few dependencies you need to install first, check out:
|
||||
# https://github.com/Farama-Foundation/Gymnasium
|
||||
swig~=4.1.1
|
||||
gymnasium[Box2D,atari,accept-rom-license]~=0.29.1
|
||||
# WARNING: on Windows, installing Box2D this way requires:
|
||||
# * Swig: http://www.swig.org/download.html
|
||||
# * Microsoft C++ Build Tools:
|
||||
# https://visualstudio.microsoft.com/visual-cpp-build-tools/
|
||||
# It's much easier to use Anaconda instead.
|
||||
|
||||
##### Image manipulation
|
||||
Pillow~=10.1.0
|
||||
graphviz~=0.20.1
|
||||
|
||||
##### Google Cloud Platform - used only in chapter 19
|
||||
google-cloud-aiplatform~=1.36.2
|
||||
google-cloud-storage~=2.13.0
|
||||
|
||||
##### Additional utilities
|
||||
|
||||
# Efficient jobs (caching, parallelism, persistence)
|
||||
joblib~=1.3.2
|
||||
|
||||
# Easy http requests
|
||||
requests~=2.31.0
|
||||
|
||||
# Nice utility to diff Jupyter Notebooks.
|
||||
nbdime~=3.2.1
|
||||
|
||||
# May be useful with Pandas for complex "where" clauses (e.g., Pandas
|
||||
# tutorial).
|
||||
numexpr~=2.8.7
|
||||
|
||||
# Optional: these libraries can be useful in chapter 3, exercise 4.
|
||||
nltk~=3.8.1
|
||||
urlextract~=1.8.0
|
||||
|
||||
# Optional: tqdm displays nice progress bars, ipywidgets for tqdm's notebook
|
||||
# support
|
||||
tqdm~=4.66.1
|
||||
ipywidgets~=8.1.1
|
||||
|
||||
# Optional: pydot is only used in chapter 10 for tf.keras.utils.plot_model()
|
||||
pydot~=1.4.2
|
||||
|
||||
# Optional: statsmodels is only used in chapter 15 for time series analysis
|
||||
statsmodels~=0.14.0
|
||||
|
||||
File diff suppressed because one or more lines are too long
+5073
File diff suppressed because one or more lines are too long
+10652
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user