chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
+71
@@ -0,0 +1,71 @@
|
||||
docs/equations/*.aux
|
||||
docs/equations/*.log
|
||||
docs/equations/*.out
|
||||
docs/equations/*.synctex.gz
|
||||
code/ch12/mnist
|
||||
code/datasets/mnist/t10k-images-idx3-ubyte
|
||||
code/datasets/mnist/t10k-labels-idx1-ubyte
|
||||
code/datasets/mnist/train-images-idx3-ubyte
|
||||
code/datasets/mnist/train-labels-idx1-ubyte
|
||||
|
||||
.ipynb_checkpoints
|
||||
.DS_Store
|
||||
code/datasets/movie/aclImdb_v1.tar.gz
|
||||
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
env/
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
|
||||
# PyInstaller
|
||||
# Usually these files are written by a python script from a template
|
||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
||||
*.manifest
|
||||
*.spec
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
htmlcov/
|
||||
.tox/
|
||||
.coverage
|
||||
.coverage.*
|
||||
.cache
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
*,cover
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Django stuff:
|
||||
*.log
|
||||
|
||||
# Sphinx documentation
|
||||
docs/_build/
|
||||
|
||||
# PyBuilder
|
||||
target/
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2015, 2016 SEBASTIAN RASCHKA (mail@sebastianraschka.com)
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,440 @@
|
||||
# Python Machine Learning book code repository
|
||||
|
||||
|
||||
[](https://groups.google.com/forum/#!forum/python-machine-learning-reader-discussion-board)
|
||||
|
||||
---
|
||||
|
||||
#### IMPORTANT NOTE (09/21/2017):
|
||||
|
||||
This GitHub repository contains the code examples of the **1st Edition** of Python Machine Learning book. If you are looking for the code examples of the **2nd Edition**, please refer to [this](https://github.com/rasbt/python-machine-learning-book-2nd-edition#whats-new-in-the-second-edition-from-the-first-edition) repository instead.
|
||||
|
||||
---
|
||||
|
||||
What you can expect are 400 pages rich in useful material just about everything you need to know to get started with machine learning ... from theory to the actual code that you can directly put into action! This is not yet just another "this is how scikit-learn works" book. I aim to explain all the underlying concepts, tell you everything you need to know in terms of best practices and caveats, and
|
||||
we will put those concepts into action mainly using NumPy, scikit-learn, and Theano.
|
||||
|
||||
You are not sure if this book is for you? Please checkout the excerpts from the [Foreword](./docs/foreword_ro.pdf) and [Preface](./docs/preface_sr.pdf), or take a look at the [FAQ](#faq) section for further information.
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
[](https://www.amazon.com/Python-Machine-Learning-Sebastian-Raschka/dp/1783555130/ref=sr_1_1?ie=UTF8&qid=1470882464&sr=8-1&keywords=python+machine+learning)
|
||||
|
||||
1st edition, published September 23rd 2015<br>
|
||||
Paperback: 454 pages<br>
|
||||
Publisher: Packt Publishing<br>
|
||||
Language: English<br>
|
||||
ISBN-10: 1783555130<br>
|
||||
ISBN-13: 978-1783555130<br>
|
||||
Kindle ASIN: B00YSILNL0<br>
|
||||
|
||||
<br>
|
||||
|
||||
[](http://www.computingreviews.com/recommend/bestof/notableitems.cfm?bestYear=2016)
|
||||
|
||||
<br>
|
||||
|
||||
German ISBN-13: 978-3958454224<br>
|
||||
Japanese ISBN-13: 978-4844380603<br>
|
||||
Italian ISBN-13: 978-8850333974<br>
|
||||
Chinese (traditional) ISBN-13: 978-9864341405<br>
|
||||
Chinese (mainland) ISBN-13: 978-7111558804<br>
|
||||
Korean ISBN-13: 979-1187497035<br>
|
||||
Russian ISBN-13: 978-5970604090<br>
|
||||
|
||||
|
||||
|
||||
## Table of Contents and Code Notebooks
|
||||
|
||||
|
||||
Simply click on the `ipynb`/`nbviewer` links next to the chapter headlines to view the code examples (currently, the internal document links are only supported by the NbViewer version).
|
||||
**Please note that these are just the code examples accompanying the book, which I uploaded for your convenience; be aware that these notebooks may not be useful without the formulae and descriptive text.**
|
||||
|
||||
|
||||
- Excerpts from the [Foreword](./docs/foreword_ro.pdf) and [Preface](./docs/preface_sr.pdf)
|
||||
- [Instructions for setting up Python and the Jupiter Notebook](./code/ch01/README.md)
|
||||
|
||||
<br>
|
||||
|
||||
1. Machine Learning - Giving Computers the Ability to Learn from Data [[dir](./code/ch01)] [[ipynb](./code/ch01/ch01.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch01/ch01.ipynb)]
|
||||
2. Training Machine Learning Algorithms for Classification [[dir](./code/ch02)] [[ipynb](./code/ch02/ch02.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch02/ch02.ipynb)]
|
||||
3. A Tour of Machine Learning Classifiers Using Scikit-Learn [[dir](./code/ch03)] [[ipynb](./code/ch03/ch03.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch03/ch03.ipynb)]
|
||||
4. Building Good Training Sets – Data Pre-Processing [[dir](./code/ch04)] [[ipynb](./code/ch04/ch04.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch04/ch04.ipynb)]
|
||||
5. Compressing Data via Dimensionality Reduction [[dir](./code/ch05)] [[ipynb](./code/ch05/ch05.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch05/ch05.ipynb)]
|
||||
6. Learning Best Practices for Model Evaluation and Hyperparameter Optimization [[dir](./code/ch06)] [[ipynb](./code/ch06/ch06.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch06/ch06.ipynb)]
|
||||
7. Combining Different Models for Ensemble Learning [[dir](./code/ch07)] [[ipynb](./code/ch07/ch07.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch07/ch07.ipynb)]
|
||||
8. Applying Machine Learning to Sentiment Analysis [[dir](./code/ch08)] [[ipynb](./code/ch08/ch08.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch08/ch08.ipynb)]
|
||||
9. Embedding a Machine Learning Model into a Web Application [[dir](./code/ch09)] [[ipynb](./code/ch09/ch09.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch09/ch09.ipynb)]
|
||||
10. Predicting Continuous Target Variables with Regression Analysis [[dir](./code/ch10)] [[ipynb](./code/ch10/ch10.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch10/ch10.ipynb)]
|
||||
11. Working with Unlabeled Data – Clustering Analysis [[dir](./code/ch11)] [[ipynb](./code/ch11/ch11.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch11/ch11.ipynb)]
|
||||
12. Training Artificial Neural Networks for Image Recognition [[dir](./code/ch12)] [[ipynb](./code/ch12/ch12.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch12/ch12.ipynb)]
|
||||
13. Parallelizing Neural Network Training via Theano [[dir](./code/ch13)] [[ipynb](./code/ch13/ch13.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch13/ch13.ipynb)]
|
||||
|
||||
<br>
|
||||
|
||||
#### Equation Reference
|
||||
|
||||
<a href="https://github.com/rasbt/python-machine-learning-book/tree/master/docs/equations"><img src="images/equation-ref-logo.png" width="200" height="200" /></a>
|
||||
|
||||
[[PDF](./docs/equations/pymle-equations.pdf)] [[TEX](./docs/equations/pymle-equations.tex)]
|
||||
|
||||
#### Slides for Teaching
|
||||
|
||||
A big thanks to [Dmitriy Dligach](dmitriydligach) for sharing his slides from his machine learning course that is currently offered at [Loyola University Chicago](http://www.luc.edu/cs/).
|
||||
|
||||
- [https://github.com/dmitriydligach/PyMLSlides](https://github.com/dmitriydligach/PyMLSlides)
|
||||
-
|
||||
|
||||
|
||||
|
||||
#### Additional Math and NumPy Resources
|
||||
|
||||
Some readers were asking about Math and NumPy primers, since they were not included due to length limitations. However, I recently put together such resources for another book, but I made these *chapters* freely available online in hope that they also serve as helpful background material for this book:
|
||||
|
||||
|
||||
- Algebra Basics [[PDF](https://sebastianraschka.com/pdf/books/dlb/appendix_b_algebra.pdf)] [[EPUB](https://sebastianraschka.com/pdf/books/dlb/appendix_b_algebra.epub)]
|
||||
|
||||
- A Calculus and Differentiation Primer [[PDF](https://sebastianraschka.com/pdf/books/dlb/appendix_d_calculus.pdf)] [[EPUB](https://sebastianraschka.com/pdf/books/dlb/appendix_d_calculus.epub)]
|
||||
|
||||
- Introduction to NumPy [[PDF](https://sebastianraschka.com/pdf/books/dlb/appendix_f_numpy-intro.pdf)] [[EPUB](https://sebastianraschka.com/pdf/books/dlb/appendix_f_numpy-intro.epub)] [[Code Notebook](https://github.com/rasbt/deep-learning-book/blob/master/code/appendix_f_numpy-intro/appendix_f_numpy-intro.ipynb)]
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
#### Citing this Book
|
||||
|
||||
You are very welcome to re-use the code snippets or other contents from this book
|
||||
in scientific publications and other works;
|
||||
in this case, I would appreciate citations to the original source:
|
||||
|
||||
**BibTeX**:
|
||||
|
||||
```
|
||||
@Book{raschka2015python,
|
||||
author = {Raschka, Sebastian},
|
||||
title = {Python Machine Learning},
|
||||
publisher = {Packt Publishing},
|
||||
year = {2015},
|
||||
address = {Birmingham, UK},
|
||||
isbn = {1783555130}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
**MLA**:
|
||||
|
||||
|
||||
Raschka, Sebastian. *Python machine learning*. Birmingham, UK: Packt Publishing, 2015. Print.
|
||||
|
||||
---
|
||||
|
||||
### [Feedback & Reviews](./docs/feedback.md)
|
||||
|
||||
#### [Short review snippets](./docs/feedback.md)
|
||||
|
||||
[](https://www.amazon.com/Python-Machine-Learning-Sebastian-Raschka/dp/1783555130/ref=sr_1_1?ie=UTF8&qid=1472342570&sr=8-1&keywords=sebastian+raschka)
|
||||
|
||||
---
|
||||
> *Sebastian Raschka’s new book, Python Machine Learning, has just been released. I got a chance to read a review copy and it’s just as I expected - really great! It’s well organized, super easy to follow, and it not only offers a good foundation for smart, non-experts, practitioners will get some ideas and learn new tricks here as well.*
|
||||
– Lon Riesberg at [Data Elixir](http://dataelixir.com/issues/55#start)
|
||||
|
||||
> *Superb job! Thus far, for me it seems to have hit the right balance of theory and practice…math and code!*
|
||||
– [Brian Thomas](http://sebastianraschka.com/blog/2015/writing-pymle.html#comment-2295668894)
|
||||
|
||||
> *I've read (virtually) every Machine Learning title based around Scikit-learn and this is hands-down the best one out there.*
|
||||
– [Jason Wolosonovich](https://www.linkedin.com/pulse/python-machine-learning-sebastian-raschka-review-jason-wolosonovich?trk=prof-post)
|
||||
|
||||
> *The best book I've seen to come out of PACKT Publishing. This is a very well written introduction to machine learning with Python. As others have noted, a perfect mixture of theory and application.*
|
||||
– [Josh D.](https://www.amazon.com/gp/customer-reviews/R27WB1GWTNGIR2/ref=cm_cr_getr_d_rvw_ttl?ie=UTF8&ASIN=1783555130)
|
||||
|
||||
> *A book with a blend of qualities that is hard to come by: combines the needed mathematics to control the theory with the applied coding in Python. Also great to see it doesn't waste paper in giving a primer on Python as many other books do just to appeal to the greater audience. You can tell it's been written by knowledgeable writers and not just DIY geeks.*
|
||||
– [Amazon Customer](https://www.amazon.com/gp/customer-reviews/RZWY4TF66Z6V0/ref=cm_cr_getr_d_rvw_ttl?ie=UTF8&ASIN=1783555130)
|
||||
|
||||
> *Sebastian Raschka created an amazing machine learning tutorial which combines theory with practice. The book explains machine learning from a theoretical perspective and has tons of coded examples to show how you would actually use the machine learning technique. It can be read by a beginner or advanced programmer.*
|
||||
- William P. Ross, [7 Must Read Python Books](http://williampross.com/7-must-read-python-books/)
|
||||
|
||||
#### Longer reviews
|
||||
|
||||
If you need help to decide whether this book is for you, check out some of the "longer" reviews linked below. (If you wrote a review, please let me know, and I'd be happy to add it to the list).
|
||||
|
||||
- [Python Machine Learning Review](http://www.bcs.org/content/conWebDoc/55586) by Patrick Hill at the Chartered Institute for IT
|
||||
- [Book Review: Python Machine Learning by Sebastian Raschka](http://whatpixel.com/python-machine-learning-book-review/) by Alex Turner at WhatPixel
|
||||
|
||||
---
|
||||
|
||||
## Links
|
||||
|
||||
- ebook and paperback at [Amazon.com](http://www.amazon.com/Python-Machine-Learning-Sebastian-Raschka/dp/1783555130/ref=sr_1_2?ie=UTF8&qid=1437754343&sr=8-2&keywords=python+machine+learning+essentials), [Amazon.co.uk](http://www.amazon.co.uk/Python-Machine-Learning-Sebastian-Raschka/dp/1783555130), [Amazon.de](http://www.amazon.de/s/ref=nb_sb_noss_2?__mk_de_DE=ÅMÅŽÕÑ&url=search-alias%3Daps&field-keywords=python+machine+learning)
|
||||
- [ebook and paperback](https://www.packtpub.com/big-data-and-business-intelligence/python-machine-learning) from Packt (the publisher)
|
||||
- at other book stores: [Google Books](https://books.google.com/books?id=GOVOCwAAQBAJ&source=gbs_slider_cls_metadata_7_mylibrary), [O'Reilly](http://shop.oreilly.com/product/9781783555130.do), [Safari](https://www.safaribooksonline.com/library/view/python-machine-learning/9781783555130/), [Barnes & Noble](http://www.barnesandnoble.com/w/python-machine-learning-essentials-sebastian-raschka/1121999969?ean=9781783555130), [Apple iBooks](https://itunes.apple.com/us/book/python-machine-learning/id1028207310?mt=11), ...
|
||||
- social platforms: [Goodreads](https://www.goodreads.com/book/show/25545994-python-machine-learning)
|
||||
|
||||
#### Translations
|
||||
|
||||
- [Italian translation](https://www.amazon.it/learning-Costruire-algoritmi-generare-conoscenza/dp/8850333978/) via "Apogeo"
|
||||
- [German translation](https://www.amazon.de/Machine-Learning-Python-mitp-Professional/dp/3958454224/) via "mitp Verlag"
|
||||
- [Japanese translation](http://www.amazon.co.jp/gp/product/4844380605/) via "Impress Top Gear"
|
||||
- [Chinese translation (traditional Chinese)](https://taiwan.kinokuniya.com/bw/9789864341405)
|
||||
- [Chinese translation (simple Chinese)](https://book.douban.com/subject/27000110/)
|
||||
- [Korean translation](http://www.kyobobook.co.kr/product/detailViewKor.laf?mallGb=KOR&ejkGb=KOR&barcode=9791187497035) via "Kyobo"
|
||||
- [Polish translation](https://www.amazon.de/Python-Uczenie-maszynowe-Sebastian-Raschka/dp/8328336138/ref=sr_1_11?ie=UTF8&qid=1513601461&sr=8-11&keywords=sebastian+raschka) via "Helion"
|
||||
|
||||
---
|
||||
|
||||
### [Literature References & Further Reading Resources](./docs/references.md)
|
||||
|
||||
### [Errata](./docs/errata.md)
|
||||
|
||||
|
||||
---
|
||||
|
||||
### Bonus Notebooks (not in the book)
|
||||
|
||||
- Logistic Regression Implementation [[dir](./code/bonus)] [[ipynb](./code/bonus/logistic_regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/logistic_regression.ipynb)]
|
||||
- A Basic Pipeline and Grid Search Setup [[dir](./code/bonus)] [[ipynb](./code/bonus/svm_iris_pipeline_and_gridsearch.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/svm_iris_pipeline_and_gridsearch.ipynb)]
|
||||
- An Extended Nested Cross-Validation Example [[dir](./code/bonus)] [[ipynb](./code/bonus/nested_cross_validation.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/nested_cross_validation.ipynb)]
|
||||
- A Simple Barebones Flask Webapp Template [[view directory](./code/bonus/flask_webapp_ex01)][[download as zip-file](https://github.com/rasbt/python-machine-learning-book/raw/master/code/bonus/flask_webapp_ex01/flask_webapp_ex01.zip)]
|
||||
- Reading handwritten digits from MNIST into NumPy arrays [[GitHub ipynb](./code/bonus/reading_mnist.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/reading_mnist.ipynb)]
|
||||
- Scikit-learn Model Persistence using JSON [[GitHub ipynb](./code/bonus/scikit-model-to-json.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/scikit-model-to-json.ipynb)]
|
||||
- Multinomial logistic regression / softmax regression [[GitHub ipynb](./code/bonus/softmax-regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/softmax-regression.ipynb)]
|
||||
|
||||
<hr>
|
||||
|
||||
**"Related Content" (not in the book)**
|
||||
|
||||
- [Model evaluation, model selection, and algorithm selection in machine learning - Part I](http://sebastianraschka.com/blog/2016/model-evaluation-selection-part1.html)
|
||||
- [Model evaluation, model selection, and algorithm selection in machine learning - Part II](http://sebastianraschka.com/blog/2016/model-evaluation-selection-part2.html)
|
||||
- [Model evaluation, model selection, and algorithm selection in machine learning - Part III](http://sebastianraschka.com/blog/2016/model-evaluation-selection-part3.html)
|
||||
|
||||
---
|
||||
|
||||
#### SciPy 2016
|
||||
|
||||
We had such a great time at [SciPy 2016](http://scipy2016.scipy.org/ehome/index.php?eventid=146062&tabid=332930&) in Austin! It was a real pleasure to meet and chat with so many readers of my book. Thanks so much for all the nice words and feedback! And in case you missed it, Andreas Mueller and I gave an **Introduction to Machine Learning with Scikit-learn**; if you are interested, the video recordings of [Part I](https://www.youtube.com/watch?v=OB1reY6IX-o&index=91&list=PLYx7XA2nY5Gf37zYZMw6OqGFRPjB1jCy6) and [Part II](https://www.youtube.com/watch?v=Cte8FYCpylk&list=PLYx7XA2nY5Gf37zYZMw6OqGFRPjB1jCy6&index=90) are now online!
|
||||
|
||||
[](https://www.youtube.com/watch?v=OB1reY6IX-o&index=91&list=PLYx7XA2nY5Gf37zYZMw6OqGFRPjB1jCy6)
|
||||
|
||||
#### PyData Chicago 2016
|
||||
|
||||
I attempted the rather challenging task of introducing scikit-learn & machine learning in *just* 90 minutes at PyData Chicago 2016. The slides and tutorial material are available at "[Learning scikit-learn -- An Introduction to Machine Learning in Python](https://github.com/rasbt/pydata-chicago2016-ml-tutorial)."
|
||||
|
||||
|
||||
---
|
||||
|
||||
**Note**
|
||||
|
||||
I have set up a separate library, [`mlxtend`](http://rasbt.github.io/mlxtend/), containing additional implementations of machine learning (and general "data science") algorithms. I also added implementations from this book (for example, the decision region plot, the artificial neural network, and sequential feature selection algorithms) with additional functionality.
|
||||
|
||||
[](http://rasbt.github.io/mlxtend/)
|
||||
|
||||
|
||||
<br>
|
||||
|
||||
<hr>
|
||||
|
||||
### Translations
|
||||
|
||||
[](https://www.amazon.it/learning-Costruire-algoritmi-generare-conoscenza/dp/8850333978/)
|
||||
[](https://www.amazon.de/Machine-Learning-Python-mitp-Professional/dp/3958454224/)
|
||||
[](http://www.amazon.co.jp/gp/product/4844380605/)
|
||||
[](https://taiwan.kinokuniya.com/bw/9789864341405)
|
||||
[](https://book.douban.com/subject/27000110/)
|
||||
[](http://www.kyobobook.co.kr/product/detailViewKor.laf?ejkGb=KOR&mallGb=KOR&barcode=9791187497035&orderClick=LEA&Kc=)
|
||||
[](http://www.ozon.ru/context/detail/id/140152222/)
|
||||
[](https://www.amazon.de/Python-Uczenie-maszynowe-Sebastian-Raschka/dp/8328336138/ref=sr_1_11?ie=UTF8&qid=1513601461&sr=8-11&keywords=sebastian+raschka)
|
||||
|
||||
<hr>
|
||||
|
||||
---
|
||||
|
||||
***Dear readers***,
|
||||
first of all, I want to thank all of you for the great support! I am really happy about all the great feedback you sent me so far, and I am glad that the book has been so useful to a broad audience.
|
||||
|
||||
Over the last couple of months, I received hundreds of emails, and I tried to answer as many as possible in the available time I have. To make them useful to other readers as well, I collected many of my answers in the FAQ section (below).
|
||||
|
||||
In addition, some of you asked me about a platform for readers to discuss the contents of the book. I hope that this would provide an opportunity for you to discuss and share your knowledge with other readers:
|
||||
|
||||
#### [Google Groups Discussion Board](https://groups.google.com/forum/#!forum/python-machine-learning-reader-discussion-board)
|
||||
|
||||
(And I will try my best to answer questions myself if time allows! :))
|
||||
|
||||
> The only thing to do with good advice is to pass it on. It is never of any use to oneself.
|
||||
— Oscar Wilde
|
||||
|
||||
---
|
||||
|
||||
## Examples and Applications by Readers
|
||||
|
||||
Once again, I have to say (big!) THANKS for all the nice feedback about the book. I've received many emails from readers, who
|
||||
put the concepts and examples from this book out into the real world and make good use of them in their projects. In this section, I am
|
||||
starting to gather some of these great applications, and I'd be more than happy to add your project to this list -- just shoot me a quick mail!
|
||||
|
||||
- [40 scripts on Optical Character Recognition](https://github.com/rrlyman/PythonMachineLearingExamples) by [Richard Lyman](https://github.com/rrlyman)
|
||||
- [Code experiments](https://github.com/jeremyn/python-machine-learning-book) by [Jeremy Nation](https://github.com/jeremyn)
|
||||
- [What I Learned Implementing a Classifier from Scratch in Python](http://www.jeannicholashould.com) by [Jean-Nicholas Hould](http://www.jeannicholashould.com)
|
||||
|
||||
## FAQ
|
||||
|
||||
### General Questions
|
||||
|
||||
- [What are machine learning and data science?](./faq/datascience-ml.md)
|
||||
- [Why do you and other people sometimes implement machine learning algorithms from scratch?](./faq/implementing-from-scratch.md)
|
||||
- [What learning path/discipline in data science I should focus on?](./faq/data-science-career.md)
|
||||
- [At what point should one start contributing to open source?](./faq/open-source.md)
|
||||
- [How important do you think having a mentor is to the learning process?](./faq/mentor.md)
|
||||
- [Where are the best online communities centered around data science/machine learning or python?](./faq/ml-python-communities.md)
|
||||
- [How would you explain machine learning to a software engineer?](./faq/ml-to-a-programmer.md)
|
||||
- [How would your curriculum for a machine learning beginner look like?](./faq/ml-curriculum.md)
|
||||
- [What is the Definition of Data Science?](./faq/definition_data-science.md)
|
||||
- [How do Data Scientists perform model selection? Is it different from Kaggle?](./faq/model-selection-in-datascience.md)
|
||||
|
||||
### Questions about the Machine Learning Field
|
||||
|
||||
- [How are Artificial Intelligence and Machine Learning related?](./faq/ai-and-ml.md)
|
||||
- [What are some real-world examples of applications of machine learning in the field?](./faq/ml-examples.md)
|
||||
- [What are the different fields of study in data mining?](./faq/datamining-overview.md)
|
||||
- [What are differences in research nature between the two fields: machine learning & data mining?](./faq/datamining-vs-ml.md)
|
||||
- [How do I know if the problem is solvable through machine learning?](./faq/ml-solvable.md)
|
||||
- [What are the origins of machine learning?](./faq/ml-origins.md)
|
||||
- [How was classification, as a learning machine, developed?](./faq/classifier-history.md)
|
||||
- [Which machine learning algorithms can be considered as among the best?](./faq/best-ml-algo.md)
|
||||
- [What are the broad categories of classifiers?](./faq/classifier-categories.md)
|
||||
- [What is the difference between a classifier and a model?](./faq/difference_classifier_model.md)
|
||||
- [What is the difference between a parametric learning algorithm and a nonparametric learning algorithm?](./faq/parametric_vs_nonparametric.md)
|
||||
- [What is the difference between a cost function and a loss function in machine learning?](./faq/cost-vs-loss.md)
|
||||
|
||||
### Questions about ML Concepts and Statistics
|
||||
|
||||
##### Cost Functions and Optimization
|
||||
|
||||
- [Fitting a model via closed-form equations vs. Gradient Descent vs Stochastic Gradient Descent vs Mini-Batch Learning -- what is the difference?](./faq/closed-form-vs-gd.md)
|
||||
- [How do you derive the Gradient Descent rule for Linear Regression and Adaline?](./faq/linear-gradient-derivative.md)
|
||||
|
||||
##### Regression Analysis
|
||||
|
||||
- [What is the difference between Pearson R and Simple Linear Regression?](./faq/pearson-r-vs-linear-regr.md)
|
||||
|
||||
##### Tree models
|
||||
|
||||
- [How does the random forest model work? How is it different from bagging and boosting in ensemble models?](./faq/bagging-boosting-rf.md)
|
||||
- [What are the disadvantages of using classic decision tree algorithm for a large dataset?](./faq/decision-tree-disadvantages.md)
|
||||
- [Why are implementations of decision tree algorithms usually binary, and what are the advantages of the different impurity metrics?](./faq/decision-tree-binary.md)
|
||||
- [Why are we growing decision trees via entropy instead of the classification error?](./faq/decisiontree-error-vs-entropy.md)
|
||||
- [When can a random forest perform terribly?](./faq/random-forest-perform-terribly.md)
|
||||
|
||||
##### Model evaluation
|
||||
|
||||
- [What is overfitting?](./faq/overfitting.md)
|
||||
- [How can I avoid overfitting?](./faq/avoid-overfitting.md)
|
||||
- [Is it always better to have the largest possible number of folds when performing cross validation?](./faq/number-of-kfolds.md)
|
||||
- [When training an SVM classifier, is it better to have a large or small number of support vectors?](./faq/num-support-vectors.md)
|
||||
- [How do I evaluate a model?](./faq/evaluate-a-model.md)
|
||||
- [What is the best validation metric for multi-class classification?](./faq/multiclass-metric.md)
|
||||
- [What factors should I consider when choosing a predictive model technique?](./faq/choosing-technique.md)
|
||||
- [What are the best toy datasets to help visualize and understand classifier behavior?](./faq/clf-behavior-data.md)
|
||||
- [How do I select SVM kernels?](./faq/select_svm_kernels.md)
|
||||
- [Interlude: Comparing and Computing Performance Metrics in Cross-Validation -- Imbalanced Class Problems and 3 Different Ways to Compute the F1 Score](./faq/computing-the-f1-score.md)
|
||||
|
||||
##### Logistic Regression
|
||||
|
||||
- [What is Softmax regression and how is it related to Logistic regression?](./faq/softmax_regression.md)
|
||||
- [Why is logistic regression considered a linear model?](./faq/logistic_regression_linear.md)
|
||||
- [What is the probabilistic interpretation of regularized logistic regression?](./faq/probablistic-logistic-regression.md)
|
||||
- [Does regularization in logistic regression always results in better fit and better generalization?](./faq/regularized-logistic-regression-performance.md)
|
||||
- [What is the major difference between naive Bayes and logistic regression?](./faq/naive-bayes-vs-logistic-regression.md)
|
||||
- [What exactly is the "softmax and the multinomial logistic loss" in the context of machine learning?](./faq/softmax.md)
|
||||
- [What is the relation between Logistic Regression and Neural Networks and when to use which?](./faq/logisticregr-neuralnet.md)
|
||||
- [Logistic Regression: Why sigmoid function?](./faq/logistic-why-sigmoid.md)
|
||||
- [Is there an analytical solution to Logistic Regression similar to the Normal Equation for Linear Regression?](./faq/logistic-analytical.md)
|
||||
|
||||
|
||||
##### Neural Networks and Deep Learning
|
||||
|
||||
- [What is the difference between deep learning and usual machine learning?](./faq/difference-deep-and-normal-learning.md)
|
||||
- [Can you give a visual explanation for the back propagation algorithm for neural networks?](./faq/visual-backpropagation.md)
|
||||
- [Why did it take so long for deep networks to be invented?](./faq/inventing-deeplearning.md)
|
||||
- [What are some good books/papers for learning deep learning?](./faq/deep-learning-resources.md)
|
||||
- [Why are there so many deep learning libraries?](./faq/many-deeplearning-libs.md)
|
||||
- [Why do some people hate neural networks/deep learning?](./faq/deeplearning-criticism.md)
|
||||
- [How can I know if Deep Learning works better for a specific problem than SVM or random forest?](./faq/deeplearn-vs-svm-randomforest.md)
|
||||
- [What is wrong when my neural network's error increases?](./faq/neuralnet-error.md)
|
||||
- [How do I debug an artificial neural network algorithm?](./faq/nnet-debugging-checklist.md)
|
||||
- [What is the difference between a Perceptron, Adaline, and neural network model?](./faq/diff-perceptron-adaline-neuralnet.md)
|
||||
- [What is the basic idea behind the dropout technique?](./faq/dropout.md)
|
||||
|
||||
|
||||
##### Other Algorithms for Supervised Learning
|
||||
|
||||
- [Why is Nearest Neighbor a Lazy Algorithm?](./faq/lazy-knn.md)
|
||||
|
||||
##### Unsupervised Learning
|
||||
|
||||
- [What are some of the issues with clustering?](./faq/issues-with-clustering.md)
|
||||
|
||||
##### Semi-Supervised Learning
|
||||
|
||||
- [What are the advantages of semi-supervised learning over supervised and unsupervised learning?](./faq/semi-vs-supervised.md)
|
||||
|
||||
##### Ensemble Methods
|
||||
|
||||
- [Is Combining Classifiers with Stacking Better than Selecting the Best One?](./faq/logistic-boosting.md)
|
||||
|
||||
##### Preprocessing, Feature Selection and Extraction
|
||||
|
||||
- [Why do we need to re-use training parameters to transform test data?](./faq/scale-training-test.md)
|
||||
- [What are the different dimensionality reduction methods in machine learning?](./faq/dimensionality-reduction.md)
|
||||
- [What is the difference between LDA and PCA for dimensionality reduction?](./faq/lda-vs-pca.md)
|
||||
- [When should I apply data normalization/standardization?](./faq/when-to-standardize.md)
|
||||
- [Does mean centering or feature scaling affect a Principal Component Analysis?](./faq/pca-scaling.md)
|
||||
- [How do you attack a machine learning problem with a large number of features?](./faq/large-num-features.md)
|
||||
- [What are some common approaches for dealing with missing data?](./faq/missing-data.md)
|
||||
- [What is the difference between filter, wrapper, and embedded methods for feature selection?](./faq/feature_sele_categories.md)
|
||||
- [Should data preparation/pre-processing step be considered one part of feature engineering? Why or why not?](./faq/dataprep-vs-dataengin.md)
|
||||
- [Is a bag of words feature representation for text classification considered as a sparse matrix?](./faq/bag-of-words-sparsity.md)
|
||||
|
||||
##### Naive Bayes
|
||||
|
||||
- [Why is the Naive Bayes Classifier naive?](./faq/naive-naive-bayes.md)
|
||||
- [What is the decision boundary for Naive Bayes?](./faq/naive-bayes-boundary.md)
|
||||
- [Can I use Naive Bayes classifiers for mixed variable types?](./faq/naive-bayes-vartypes.md)
|
||||
- [Is it possible to mix different variable types in Naive Bayes, for example, binary and continues features?](./naive-bayes-vartypes.md)
|
||||
|
||||
##### Other
|
||||
|
||||
- [What is Euclidean distance in terms of machine learning?](./faq/euclidean-distance.md)
|
||||
- [When should one use median, as opposed to the mean or average?](./faq/median-vs-mean.md)
|
||||
|
||||
##### Programming Languages and Libraries for Data Science and Machine Learning
|
||||
|
||||
- [Is R used extensively today in data science?](./faq/r-in-datascience.md)
|
||||
- [What is the main difference between TensorFlow and scikit-learn?](./faq/tensorflow-vs-scikitlearn.md)
|
||||
|
||||
<br>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Questions about the Book
|
||||
|
||||
- [Can I use paragraphs and images from the book in presentations or my blog?](./faq/copyright.md)
|
||||
- [How is this different from other machine learning books?](./faq/different.md)
|
||||
- [Which version of Python was used in the code examples?](./faq/py2py3.md)
|
||||
- [Which technologies and libraries are being used?](./faq/technologies.md)
|
||||
- [Which book version/format would you recommend?](./faq/version.md)
|
||||
- [Why did you choose Python for machine learning?](./faq/why-python.md)
|
||||
- [Why do you use so many leading and trailing underscores in the code examples?](./faq/underscore-convention.md)
|
||||
- [What is the purpose of the `return self` idioms in your code examples?](./faq/return_self_idiom.md)
|
||||
- [Are there any prerequisites and recommended pre-readings?](./faq/prerequisites.md)
|
||||
- [How can I apply SVM to categorical data?](./faq/svm_for_categorical.md)
|
||||
|
||||
|
||||
## Contact
|
||||
|
||||
I am happy to answer questions! Just write me an [email](mailto:mail@sebastianraschka.com)
|
||||
or consider asking the question on the [Google Groups Email List](https://groups.google.com/forum/#!forum/python-machine-learning-book).
|
||||
|
||||
If you are interested in keeping in touch, I have quite a lively twitter stream ([@rasbt](https://twitter.com/rasbt)) all about data science and machine learning. I also maintain a [blog](http://sebastianraschka.com/articles.html) where I post all of the things I am particularly excited about.
|
||||
@@ -0,0 +1,7 @@
|
||||
# WeHub 来源说明
|
||||
|
||||
- 原始项目：`rasbt/python-machine-learning-book`
|
||||
- 原始仓库：https://github.com/rasbt/python-machine-learning-book
|
||||
- 导入方式：上游默认分支的最新快照
|
||||
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
|
||||
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
|
||||
@@ -0,0 +1,59 @@
|
||||
## Resources for setting up your coding environment
|
||||
|
||||
- [Instructions for setting up Python and the Jupyter Notebook](./ch01/README.md)
|
||||
|
||||
***If you need help with opening the Jupyter notebooks, I made a short [step by step guide](../docs/running_jupyter_nb.pdf) that illustrates this process***
|
||||
|
||||
- A [quick and great NumPy refresher](https://docs.scipy.org/doc/numpy-dev/user/quickstart.html) that covers everything (and more) you'd need for this book
|
||||
|
||||
- Recommended! To check your coding environment, open the `check_environment.ipynb` (it can be found in this directory) in Jupyter Notebook and execute the code cell:
|
||||
|
||||
|
||||
|
||||
|
||||
## Table of contents and code notebooks
|
||||
|
||||
|
||||
Simply click on the `ipynb`/`nbviewer` links next to the chapter headlines to view the code examples (currently, the internal document links are only supported by the NbViewer version).
|
||||
**Please note that these are just the code examples accompanying the book, which I uploaded for your convenience; be aware that these notebooks may not be useful without the formulae and descriptive text.**
|
||||
|
||||
|
||||
|
||||
<br>
|
||||
|
||||
|
||||
1. Machine Learning - Giving Computers the Ability to Learn from Data [[dir](./ch01)] [[ipynb](./ch01/ch01.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch01/ch01.ipynb)]
|
||||
2. Training Machine Learning Algorithms for Classification [[dir](./ch02)] [[ipynb](./ch02/ch02.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch02/ch02.ipynb)]
|
||||
3. A Tour of Machine Learning Classifiers Using Scikit-Learn [[dir](./ch03)] [[ipynb](./ch03/ch03.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch03/ch03.ipynb)]
|
||||
4. Building Good Training Sets – Data Pre-Processing [[dir](./ch04)] [[ipynb](./ch04/ch04.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch04/ch04.ipynb)]
|
||||
5. Compressing Data via Dimensionality Reduction [[dir](./ch05)] [[ipynb](./ch05/ch05.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch05/ch05.ipynb)]
|
||||
6. Learning Best Practices for Model Evaluation and Hyperparameter Optimization [[dir](./ch06)] [[ipynb](./ch06/ch06.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch06/ch06.ipynb)]
|
||||
7. Combining Different Models for Ensemble Learning [[dir](./ch07)] [[ipynb](./ch07/ch07.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch07/ch07.ipynb)]
|
||||
8. Applying Machine Learning to Sentiment Analysis [[dir](./ch08)] [[ipynb](./ch08/ch08.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch08/ch08.ipynb)]
|
||||
9. Embedding a Machine Learning Model into a Web Application [[dir](./ch09)] [[ipynb](./ch09/ch09.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch09/ch09.ipynb)]
|
||||
10. Predicting Continuous Target Variables with Regression Analysis [[dir](./ch10)] [[ipynb](./ch10/ch10.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch10/ch10.ipynb)]
|
||||
11. Working with Unlabeled Data – Clustering Analysis [[dir](./ch11)] [[ipynb](./ch11/ch11.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch11/ch11.ipynb)]
|
||||
12. Training Artificial Neural Networks for Image Recognition [[dir](./ch12)] [[ipynb](./ch12/ch12.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch12/ch12.ipynb)]
|
||||
13. Parallelizing Neural Network Training via Theano [[dir](./ch13)] [[ipynb](./ch13/ch13.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch13/ch13.ipynb)]
|
||||
|
||||
<br>
|
||||
|
||||
**Bonus Notebooks (not in the book)**
|
||||
|
||||
- Logistic Regression Implementation [[dir](./bonus)] [[ipynb](./bonus/logistic_regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/logistic_regression.ipynb)]
|
||||
- A Basic Pipeline and Grid Search Setup [[dir](./bonus)] [[ipynb](./bonus/svm_iris_pipeline_and_gridsearch.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/svm_iris_pipeline_and_gridsearch.ipynb)]
|
||||
- An Extended Nested Cross-Validation Example [[dir](./bonus)] [[ipynb](./bonus/nested_cross_validation.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/nested_cross_validation.ipynb)]
|
||||
- A Simple(r) Barebones Flask Webapp Template [[view directory](./bonus/flask_webapp_ex01)][[download as zip-file](https://github.com/rasbt/python-machine-learning-book/raw/master/code/bonus/flask_webapp_ex01/flask_webapp_ex01.zip)]
|
||||
- Reading handwritten digits from MNIST into NumPy arrays [[GitHub ipynb](./bonus/reading_mnist.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/reading_mnist.ipynb)]
|
||||
- Scikit-learn Model Persistence using JSON [[GitHub ipynb](./bonus/scikit-model-to-json.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/scikit-model-to-json.ipynb)]
|
||||
- Multinomial logistic regression / softmax regression [[GitHub ipynb](./bonus/softmax-regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/softmax-regression.ipynb)]
|
||||
|
||||
|
||||
|
||||
|
||||
## Contact
|
||||
|
||||
I am happy to answer questions! Just write me an [email](mailto:mail@sebastianraschka.com)
|
||||
or consider asking the question on the [Google Groups Email List](https://groups.google.com/forum/#!forum/python-machine-learning-book).
|
||||
|
||||
If you are interested in keeping in touch, I have quite a lively twitter stream ([@rasbt](https://twitter.com/rasbt)) all about data science and machine learning. I also maintain a [blog](http://sebastianraschka.com/articles.html) where I post all of the things I am particularly excited about.
|
||||
@@ -0,0 +1,17 @@
|
||||
Building intelligent machines to transform data into knowledge
|
||||
The three different types of machine learning
|
||||
Making predictions about the future with supervised learning
|
||||
Classification for predicting class labels
|
||||
Regression for predicting continuous outcomes
|
||||
Solving interactive problems with reinforcement learning
|
||||
Discovering hidden structures with unsupervised learning
|
||||
Finding subgroups with clustering
|
||||
Dimensionality reduction for data compression
|
||||
An introduction to the basic terminology and notations
|
||||
A roadmap for building machine learning systems
|
||||
Preprocessing – getting data into shape
|
||||
Training and selecting a predictive model
|
||||
Evaluating models and predicting unseen data instances
|
||||
Using Python for machine learning
|
||||
Installing Python packages
|
||||
Summary
|
||||
@@ -0,0 +1,9 @@
|
||||
Artificial neurons - a brief glimpse into the early history
|
||||
of machine learning
|
||||
Implementing a perceptron learning algorithm in Python
|
||||
Training a perceptron model on the Iris dataset
|
||||
Adaptive linear neurons and the convergence of learning
|
||||
Minimizing cost functions with gradient descent
|
||||
Implementing an Adaptive Linear Neuron in Python
|
||||
Large scale machine learning and stochastic gradient descent
|
||||
Summary
|
||||
@@ -0,0 +1,20 @@
|
||||
Choosing a classification algorithm
|
||||
First steps with scikit-learn
|
||||
Training a perceptron via scikit-learn
|
||||
Modeling class probabilities via logistic regression
|
||||
Logistic regression intuition and conditional probabilities
|
||||
Learning the weights of the logistic cost function
|
||||
Training a logistic regression model with scikit-learn
|
||||
Tackling overfitting via regularization
|
||||
Maximum margin classification with support vector machines
|
||||
Maximum margin intuition
|
||||
Dealing with the nonlinearly separable case using slack variables
|
||||
Alternative implementations in scikit-learn
|
||||
Solving nonlinear problems using a kernel SVM
|
||||
Using the kernel trick to find separating hyperplanes in higher dimensional space
|
||||
Decision tree learning
|
||||
Maximizing information gain – getting the most bang for the buck
|
||||
Building a decision tree
|
||||
Combining weak to strong learners via random forests
|
||||
K-nearest neighbors – a lazy learning algorithm
|
||||
Summary
|
||||
@@ -0,0 +1,15 @@
|
||||
Dealing with missing data
|
||||
Eliminating samples or features with missing values
|
||||
Imputing missing values
|
||||
Understanding the scikit-learn estimator API
|
||||
Handling categorical data
|
||||
Mapping ordinal features
|
||||
Encoding class labels
|
||||
Performing one-hot encoding on nominal features
|
||||
Partitioning a dataset in training and test sets
|
||||
Bringing features onto the same scale
|
||||
Selecting meaningful features
|
||||
Sparse solutions with L1 regularization
|
||||
Sequential feature selection algorithms
|
||||
Assessing feature importance with random forests
|
||||
Summary
|
||||
@@ -0,0 +1,17 @@
|
||||
Unsupervised dimensionality reduction via principal component analysis 128
|
||||
Total and explained variance
|
||||
Feature transformation
|
||||
Principal component analysis in scikit-learn
|
||||
Supervised data compression via linear discriminant analysis
|
||||
Computing the scatter matrices
|
||||
Selecting linear discriminants for the new feature subspace
|
||||
Projecting samples onto the new feature space
|
||||
LDA via scikit-learn
|
||||
Using kernel principal component analysis for nonlinear mappings
|
||||
Kernel functions and the kernel trick
|
||||
Implementing a kernel principal component analysis in Python
|
||||
Example 1 – separating half-moon shapes
|
||||
Example 2 – separating concentric circles
|
||||
Projecting new data points
|
||||
Kernel principal component analysis in scikit-learn
|
||||
Summary
|
||||
@@ -0,0 +1,4 @@
|
||||
Streamlining workflows with pipelines
|
||||
Loading the Breast Cancer Wisconsin dataset
|
||||
Combining transformers and estimators in a pipeline
|
||||
Using k-fold cross-validation to assess model performance
|
||||
@@ -0,0 +1,6 @@
|
||||
Learning with ensembles
|
||||
Implementing a simple majority vote classifier
|
||||
Combining different algorithms for classification with majority vote
|
||||
Evaluating and tuning the ensemble classifier
|
||||
Bagging – building an ensemble of classifiers from bootstrap samples
|
||||
Leveraging weak learners via adaptive boosting
|
||||
@@ -0,0 +1,7 @@
|
||||
Obtaining the IMDb movie review dataset
|
||||
Introducing the bag-of-words model
|
||||
Transforming words into feature vectors
|
||||
Assessing word relevancy via term frequency-inverse document frequency
|
||||
Cleaning text data
|
||||
Processing documents into tokens
|
||||
Training a logistic regression model for document classification
|
||||
@@ -0,0 +1,6 @@
|
||||
Serializing fitted scikit-learn estimators
|
||||
Setting up a SQLite database for data storage
|
||||
Developing a web application with Flask
|
||||
Our first Flask web application
|
||||
Form validation and rendering
|
||||
Turning the movie classifier into a web application
|
||||
@@ -0,0 +1,12 @@
|
||||
Introducing a simple linear regression model
|
||||
Exploring the Housing Dataset
|
||||
Visualizing the important characteristics of a dataset
|
||||
Implementing an ordinary least squares linear regression model
|
||||
Solving regression for regression parameters with gradient descent
|
||||
Estimating the coefficient of a regression model via scikit-learn
|
||||
Fitting a robust regression model using RANSAC
|
||||
Evaluating the performance of linear regression models
|
||||
Using regularized methods for regression
|
||||
Turning a linear regression model into a curve – polynomial regression
|
||||
Modeling nonlinear relationships in the Housing Dataset
|
||||
Dealing with nonlinear relationships using random forests
|
||||
@@ -0,0 +1,11 @@
|
||||
Grouping objects by similarity using k-means
|
||||
K-means++
|
||||
Hard versus soft clustering
|
||||
Using the elbow method to find the optimal number of clusters
|
||||
Quantifying the quality of clustering via silhouette plots
|
||||
Organizing clusters as a hierarchical tree
|
||||
Performing hierarchical clustering on a distance matrix
|
||||
Attaching dendrograms to a heat map
|
||||
Applying agglomerative clustering via scikit-learn
|
||||
Locating regions of high density via DBSCAN
|
||||
Summary
|
||||
@@ -0,0 +1,9 @@
|
||||
Modeling complex functions with artificial neural networks
|
||||
Single-layer neural network recap
|
||||
Introducing the multi-layer neural network architecture
|
||||
Activating a neural network via forward propagation
|
||||
Classifying handwritten digits
|
||||
Obtaining the MNIST dataset
|
||||
Implementing a multi-layer perceptron
|
||||
Training an artificial neural network
|
||||
Computing the logistic cost function
|
||||
@@ -0,0 +1,9 @@
|
||||
Building, compiling, and running expressions with Theano
|
||||
What is Theano?
|
||||
First steps with Theano
|
||||
Configuring Theano
|
||||
Working with array structures
|
||||
Wrapping things up – a linear regression example
|
||||
Choosing activation functions for feedforward neural networks
|
||||
Logistic function recap
|
||||
Estimating probabilities in multi-class classification via the softmax function
|
||||
@@ -0,0 +1,14 @@
|
||||
# Sebastian Raschka, 2015
|
||||
# convenience function for myself to add internal links to IPython toc
|
||||
# use as `python ipynb_toc_links.py /blank_tocs/ch01.toc`
|
||||
|
||||
import sys
|
||||
|
||||
ipynb = sys.argv[1]
|
||||
with open(ipynb, 'r') as f:
|
||||
for line in f:
|
||||
out_str = ' ' * (len(line) - len(line.lstrip()))
|
||||
line = line.strip()
|
||||
out_str += '- [%s' % line
|
||||
out_str += '](#%s)' % line.replace(' ', '-')
|
||||
print(out_str)
|
||||
@@ -0,0 +1,13 @@
|
||||
# Sebastian Raschka, 2015
|
||||
# convenience function for myself to create nested TOC lists
|
||||
# use as `python md_toc.py /blank_tocs/ch01.toc`
|
||||
|
||||
import sys
|
||||
|
||||
ipynb = sys.argv[1]
|
||||
with open(ipynb, 'r') as f:
|
||||
for line in f:
|
||||
out_str = ' ' * (len(line) - len(line.lstrip()))
|
||||
line = line.strip()
|
||||
out_str += '- %s' % line
|
||||
print(out_str)
|
||||
@@ -0,0 +1,20 @@
|
||||
Sebastian Raschka, 2015
|
||||
|
||||
Python Machine Learning - Code Examples
|
||||
|
||||
## Bonus Material
|
||||
|
||||
A collection of additional notebooks and code examples to clarify and explain concepts based on reader feedback.
|
||||
|
||||
|
||||
- A Basic Pipeline and Grid Search Setup [[GitHub ipynb](./svm_iris_pipeline_and_gridsearch.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/svm_iris_pipeline_and_gridsearch.ipynb)]
|
||||
|
||||
- An Extended Nested Cross-Validation Example [[GitHub ipynb](./nested_cross_validation.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/nested_cross_validation.ipynb)]
|
||||
|
||||
- A Simple(r) Barebones Flask Webapp Template [[view directory](./flask_webapp_ex01)][[download as zip-file](https://github.com/rasbt/python-machine-learning-book/raw/master/code/bonus/flask_webapp_ex01/flask_webapp_ex01.zip)]
|
||||
|
||||
- Reading handwritten digits from MNIST into NumPy arrays [[GitHub ipynb](./reading_mnist.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/reading_mnist.ipynb)]
|
||||
|
||||
- Scikit-learn Model Persistence using JSON [[GitHub ipynb](./scikit-model-to-json.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/scikit-model-to-json.ipynb)]
|
||||
|
||||
- Multinomial logistic regression / softmax regression [[GitHub ipynb](./softmax-regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/softmax-regression.ipynb)]
|
||||
@@ -0,0 +1,16 @@
|
||||
Sebastian Raschka, 2015
|
||||
|
||||
Python Machine Learning - Code Examples (Bonus Material)
|
||||
|
||||
|
||||
# A Simple(r) Barebones Flask Webapp Template
|
||||
|
||||
A simple Flask app that calculates the sum of two numbers entered in the respective input fields.
|
||||
|
||||
You can run the app locally by executing `python app.py` within this directory.
|
||||
|
||||
<hr>
|
||||
|
||||
|
||||
|
||||
Click [here](https://github.com/rasbt/python-machine-learning-book/raw/master/code/bonus/flask_webapp_ex01/flask_webapp_ex01.zip) to download this example as zip-file.
|
||||
@@ -0,0 +1,31 @@
|
||||
from flask import Flask, render_template, request
|
||||
from wtforms import Form, DecimalField, validators
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
|
||||
class EntryForm(Form):
|
||||
x_entry = DecimalField('x:',
|
||||
places=10,
|
||||
validators=[validators.NumberRange(-1e10, 1e10)])
|
||||
y_entry = DecimalField('y:',
|
||||
places=10,
|
||||
validators=[validators.NumberRange(-1e10, 1e10)])
|
||||
|
||||
@app.route('/')
|
||||
def index():
|
||||
form = EntryForm(request.form)
|
||||
return render_template('entry.html', form=form, z='')
|
||||
|
||||
@app.route('/results', methods=['POST'])
|
||||
def results():
|
||||
form = EntryForm(request.form)
|
||||
z = ''
|
||||
if request.method == 'POST' and form.validate():
|
||||
x = request.form['x_entry']
|
||||
y = request.form['y_entry']
|
||||
z = float(x) + float(y)
|
||||
return render_template('entry.html', form=form, z=z)
|
||||
|
||||
if __name__ == '__main__':
|
||||
app.run(debug=True)
|
||||
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 6.1 KiB |
@@ -0,0 +1,7 @@
|
||||
body{
|
||||
width:600px;
|
||||
}
|
||||
|
||||
#button{
|
||||
padding-top: 20px;
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
{% macro render_field(field) %}
|
||||
<dt>{{ field.label }}
|
||||
<dd>{{ field(**kwargs)|safe }}
|
||||
{% if field.errors %}
|
||||
<ul class=errors>
|
||||
{% for error in field.errors %}
|
||||
<li>{{ error }}</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% endif %}
|
||||
</dd>
|
||||
{% endmacro %}
|
||||
@@ -0,0 +1,26 @@
|
||||
<!doctype html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Webapp Ex 1</title>
|
||||
<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
|
||||
</head>
|
||||
<body>
|
||||
|
||||
{% from "_formhelpers.html" import render_field %}
|
||||
|
||||
<form method=post action="/results">
|
||||
<dl>
|
||||
{{ render_field(form.x_entry, cols='1', rows='1') }}
|
||||
{{ render_field(form.y_entry, cols='1', rows='1') }}
|
||||
</dl>
|
||||
<div>
|
||||
<input type=submit value='Submit' name='submit_btn'>
|
||||
</div>
|
||||
|
||||
<dl>
|
||||
x + y = {{ z }}
|
||||
</dl>
|
||||
</form>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 59 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 76 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 106 KiB |
File diff suppressed because one or more lines are too long
@@ -0,0 +1,311 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"[Sebastian Raschka](http://sebastianraschka.com), 2015\n",
|
||||
"\n",
|
||||
"https://github.com/rasbt/python-machine-learning-book"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Python Machine Learning - Code Examples"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Bonus Material - An Extended Nested Cross-Validation Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"For an explanation of nested cross-validation, please see:\n",
|
||||
" \n",
|
||||
"- Chapter 6, section \"Algorithm-selection-with-nested-cross-validation\" (open the code example via [nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch06/ch06.ipynb#Algorithm-selection-with-nested-cross-validation))\n",
|
||||
"- FAQ, section: [How do I evaluate a model?](https://github.com/rasbt/python-machine-learning-book/blob/master/faq/evaluate-a-model.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<br>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Sebastian Raschka \n",
|
||||
"Last updated: 11/30/2015 \n",
|
||||
"\n",
|
||||
"CPython 3.5.0\n",
|
||||
"IPython 4.0.0\n",
|
||||
"\n",
|
||||
"numpy 1.10.1\n",
|
||||
"pandas 0.17.1\n",
|
||||
"matplotlib 1.5.0\n",
|
||||
"scikit-learn 0.17\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%load_ext watermark\n",
|
||||
"%watermark -a 'Sebastian Raschka' -u -d -v -p numpy,pandas,matplotlib,scikit-learn"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Dataset and Estimator Setup"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.grid_search import GridSearchCV\n",
|
||||
"from sklearn.pipeline import Pipeline\n",
|
||||
"from sklearn.preprocessing import StandardScaler\n",
|
||||
"from sklearn.svm import SVC\n",
|
||||
"from sklearn.datasets import load_iris\n",
|
||||
"from sklearn.cross_validation import train_test_split\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# load and split data\n",
|
||||
"iris = load_iris()\n",
|
||||
"X, y = iris.data, iris.target\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)\n",
|
||||
"\n",
|
||||
"# pipeline setup\n",
|
||||
"cls = SVC(C=10.0, kernel='rbf', gamma=0.1, decision_function_shape='ovr')\n",
|
||||
"kernel_svm = Pipeline([('std', StandardScaler()), \n",
|
||||
" ('svc', cls)])\n",
|
||||
"\n",
|
||||
"# gridsearch setup\n",
|
||||
"param_grid = [\n",
|
||||
" {'svc__C': [1, 10, 100, 1000], \n",
|
||||
" 'svc__gamma': [0.001, 0.0001], \n",
|
||||
" 'svc__kernel': ['rbf']},\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# setup multiple GridSearchCV objects, 1 for each algorithm\n",
|
||||
"\n",
|
||||
"gs_svm = GridSearchCV(estimator=kernel_svm, \n",
|
||||
" param_grid=param_grid, \n",
|
||||
" scoring='accuracy', \n",
|
||||
" n_jobs=-1, \n",
|
||||
" cv=5, \n",
|
||||
" verbose=0, \n",
|
||||
" refit=True,\n",
|
||||
" pre_dispatch='2*n_jobs')\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## A. Nested Crossvalidation - Quick Version"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here, the `cross_val_function` runs the 5 outer loops, and the the `GridSearch` object (`gs`) peforms the hyperparameter optimization during the 5 inner loops."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"Average Accuracy 0.95 +/- 0.06\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import numpy as np \n",
|
||||
"\n",
|
||||
"from sklearn.cross_validation import cross_val_score\n",
|
||||
"scores = cross_val_score(gs_svm, X_train, y_train, scoring='accuracy', cv=5)\n",
|
||||
"print('\\nAverage Accuracy %.2f +/- %.2f' % (np.mean(scores), np.std(scores)))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## B. Nested Crossvalidation - Manual Approach Printing the Model Parameters"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.cross_validation import StratifiedKFold\n",
|
||||
"from sklearn.metrics import accuracy_score\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"params = []\n",
|
||||
"scores = []\n",
|
||||
"\n",
|
||||
"skfold = StratifiedKFold(y=y_train, n_folds=5, shuffle=False, random_state=1)\n",
|
||||
"for train_idx, test_idx in skfold:\n",
|
||||
" gs_svm.fit(X_train[train_idx], y_train[train_idx])\n",
|
||||
" y_pred = gs_svm.predict(X_train[test_idx])\n",
|
||||
" acc = accuracy_score(y_true=y_train[test_idx], y_pred=y_pred)\n",
|
||||
" params.append(gs_svm.best_params_)\n",
|
||||
" scores.append(acc)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"SVM models:\n",
|
||||
"1. Acc: 0.96 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
|
||||
"2. Acc: 1.00 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
|
||||
"3. Acc: 0.83 Params: {'svc__C': 1000, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
|
||||
"4. Acc: 1.00 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
|
||||
"5. Acc: 0.96 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
|
||||
"\n",
|
||||
"Average Accuracy 0.95 +/- 0.06\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print('SVM models:')\n",
|
||||
"for idx, m in enumerate(zip(params, scores)):\n",
|
||||
" print('%s. Acc: %.2f Params: %s' % (idx+1, m[1], m[0]))\n",
|
||||
"print('\\nAverage Accuracy %.2f +/- %.2f' % (np.mean(scores), np.std(scores)))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Regular K-fold CV to Optimize the Model on the Complete Training Set"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Repeat the nested cross-validation for different algorithms. Then, pick the \"best\" algorithm (not the best model!). Next, use the complete training set to tune the best algorithm via grid search:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Best parameters {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"gs_svm.fit(X_train, y_train)\n",
|
||||
"print('Best parameters %s' % gs_svm.best_params_)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training accuracy: 0.97\n",
|
||||
"Test accuracy: 0.97\n",
|
||||
"Parameters: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"train_acc = accuracy_score(y_true=y_train, y_pred=gs_svm.predict(X_train))\n",
|
||||
"test_acc = accuracy_score(y_true=y_test, y_pred=gs_svm.predict(X_test))\n",
|
||||
"print('Training accuracy: %.2f' % train_acc)\n",
|
||||
"print('Test accuracy: %.2f' % test_acc)\n",
|
||||
"print('Parameters: %s' % gs_svm.best_params_)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.5.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"classes_":[
|
||||
0,
|
||||
1,
|
||||
2
|
||||
],
|
||||
"coef_":[
|
||||
[
|
||||
0.42625236403173844,
|
||||
-8.557501546363858
|
||||
],
|
||||
[
|
||||
1.5644231337040186,
|
||||
-1.6783659020502222
|
||||
],
|
||||
[
|
||||
-1.990675497337773,
|
||||
10.235867448186507
|
||||
]
|
||||
],
|
||||
"intercept_":[
|
||||
27.533384852155145,
|
||||
4.18509910962595,
|
||||
-31.71848396177913
|
||||
],
|
||||
"n_iter_":[
|
||||
27
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"dual": false, "max_iter": 100, "warm_start": false, "verbose": 0, "C": 100.0, "class_weight": null, "random_state": 1, "fit_intercept": true, "multi_class": "multinomial", "intercept_scaling": 1, "penalty": "l2", "solver": "newton-cg", "n_jobs": 1, "tol": 0.0001}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,355 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"[Sebastian Raschka](http://sebastianraschka.com), 2015\n",
|
||||
"\n",
|
||||
"https://github.com/rasbt/python-machine-learning-book"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Python Machine Learning - Code Examples"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Bonus Material - A Basic Pipeline and Grid Search Setup"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Sebastian Raschka \n",
|
||||
"Last updated: 01/20/2016 \n",
|
||||
"\n",
|
||||
"CPython 3.5.1\n",
|
||||
"IPython 4.0.1\n",
|
||||
"\n",
|
||||
"numpy 1.10.1\n",
|
||||
"pandas 0.17.1\n",
|
||||
"matplotlib 1.5.0\n",
|
||||
"scikit-learn 0.17\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%load_ext watermark\n",
|
||||
"%watermark -a 'Sebastian Raschka' -u -d -v -p numpy,pandas,matplotlib,scikit-learn"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Fitting 5 folds for each of 8 candidates, totalling 40 fits\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[Parallel(n_jobs=-1)]: Done 40 out of 40 | elapsed: 0.2s finished\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"GridSearchCV(cv=5, error_score='raise',\n",
|
||||
" estimator=Pipeline(steps=[('std', StandardScaler(copy=True, with_mean=True, with_std=True)), ('svc', SVC(C=10.0, cache_size=200, class_weight=None, coef0=0.0,\n",
|
||||
" decision_function_shape='ovr', degree=3, gamma=0.1, kernel='rbf',\n",
|
||||
" max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
|
||||
" tol=0.001, verbose=False))]),\n",
|
||||
" fit_params={}, iid=True, n_jobs=-1,\n",
|
||||
" param_grid=[{'svc__kernel': ['rbf'], 'svc__C': [1, 10, 100, 1000], 'svc__gamma': [0.001, 0.0001]}],\n",
|
||||
" pre_dispatch='2*n_jobs', refit=True, scoring='accuracy', verbose=1)"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.grid_search import GridSearchCV\n",
|
||||
"from sklearn.pipeline import Pipeline\n",
|
||||
"from sklearn.preprocessing import StandardScaler\n",
|
||||
"from sklearn.svm import SVC\n",
|
||||
"from sklearn.datasets import load_iris\n",
|
||||
"from sklearn.cross_validation import train_test_split\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# load and split data\n",
|
||||
"iris = load_iris()\n",
|
||||
"X, y = iris.data, iris.target\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)\n",
|
||||
"\n",
|
||||
"# pipeline setup\n",
|
||||
"cls = SVC(C=10.0, \n",
|
||||
" kernel='rbf', \n",
|
||||
" gamma=0.1, \n",
|
||||
" decision_function_shape='ovr')\n",
|
||||
"\n",
|
||||
"kernel_svm = Pipeline([('std', StandardScaler()), \n",
|
||||
" ('svc', cls)])\n",
|
||||
"\n",
|
||||
"# gridsearch setup\n",
|
||||
"param_grid = [\n",
|
||||
" {'svc__C': [1, 10, 100, 1000], \n",
|
||||
" 'svc__gamma': [0.001, 0.0001], \n",
|
||||
" 'svc__kernel': ['rbf']},\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
"gs = GridSearchCV(estimator=kernel_svm, \n",
|
||||
" param_grid=param_grid, \n",
|
||||
" scoring='accuracy', \n",
|
||||
" n_jobs=-1, \n",
|
||||
" cv=5, \n",
|
||||
" verbose=1, \n",
|
||||
" refit=True,\n",
|
||||
" pre_dispatch='2*n_jobs')\n",
|
||||
"\n",
|
||||
"# run gridearch\n",
|
||||
"gs.fit(X_train, y_train)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Best GS Score 0.96\n",
|
||||
"best GS Params {'svc__kernel': 'rbf', 'svc__C': 100, 'svc__gamma': 0.001}\n",
|
||||
"\n",
|
||||
"Train Accuracy: 0.97\n",
|
||||
"\n",
|
||||
"Test Accuracy: 0.97\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print('Best GS Score %.2f' % gs.best_score_)\n",
|
||||
"print('best GS Params %s' % gs.best_params_)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# prediction on the training set\n",
|
||||
"y_pred = gs.predict(X_train)\n",
|
||||
"train_acc = (y_train == y_pred).sum()/len(y_train)\n",
|
||||
"print('\\nTrain Accuracy: %.2f' % (train_acc))\n",
|
||||
"\n",
|
||||
"# evaluation on the test set\n",
|
||||
"y_pred = gs.predict(X_test)\n",
|
||||
"test_acc = (y_test == y_pred).sum()/len(y_test)\n",
|
||||
"print('\\nTest Accuracy: %.2f' % (test_acc))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"### A Note about `GridSearchCV`'s `best_score_` attribute"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Please note that `gs.best_score_` is the average k-fold cross-validation score. I.e., if we have a `GridSearchCV` object with 5-fold cross-validation (like the one above), the `best_score_` attribute returns the average score over the 5-folds of the best model. To illustrate this with an example:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array([ 0.6, 0.4, 0.6, 0.2, 0.6])"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.cross_validation import StratifiedKFold, cross_val_score\n",
|
||||
"from sklearn.linear_model import LogisticRegression\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"np.random.seed(0)\n",
|
||||
"np.set_printoptions(precision=6)\n",
|
||||
"y = [np.random.randint(3) for i in range(25)]\n",
|
||||
"X = (y + np.random.randn(25)).reshape(-1, 1)\n",
|
||||
"\n",
|
||||
"cv5_idx = list(StratifiedKFold(y, n_folds=5, shuffle=False, random_state=0))\n",
|
||||
"cross_val_score(LogisticRegression(random_state=123), X, y, cv=cv5_idx)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"By executing the code above, we created a simple data set of random integers that shall represent our class labels. Next, we fed the indices of 5 cross-validation folds (`cv3_idx`) to the `cross_val_score` scorer, which returned 5 accuracy scores -- these are the 5 accuracy values for the 5 test folds. \n",
|
||||
"\n",
|
||||
"Next, let us use the `GridSearchCV` object and feed it the same 5 cross-validation sets (via the pre-generated `cv3_idx` indices):"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Fitting 5 folds for each of 1 candidates, totalling 5 fits\n",
|
||||
"[CV] ................................................................\n",
|
||||
"[CV] ....................................... , score=0.600000 - 0.0s\n",
|
||||
"[CV] ................................................................\n",
|
||||
"[CV] ....................................... , score=0.400000 - 0.0s\n",
|
||||
"[CV] ................................................................\n",
|
||||
"[CV] ....................................... , score=0.600000 - 0.0s\n",
|
||||
"[CV] ................................................................\n",
|
||||
"[CV] ....................................... , score=0.200000 - 0.0s\n",
|
||||
"[CV] ................................................................\n",
|
||||
"[CV] ....................................... , score=0.600000 - 0.0s\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[Parallel(n_jobs=1)]: Done 5 out of 5 | elapsed: 0.0s finished\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.grid_search import GridSearchCV\n",
|
||||
"gs = GridSearchCV(LogisticRegression(), {}, cv=cv5_idx, verbose=3).fit(X, y) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"As we can see, the scores for the 5 folds are exactly the same as the ones from `cross_val_score` earlier. \n",
|
||||
"Now, the best_score_ attribute of the `GridSearchCV` object, which becomes available after `fit`ting, returns the average accuracy score of the best model:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0.47999999999999998"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"gs.best_score_"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"As we can see, the result above is consistent with the average score computed the `cross_val_score`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0.47999999999999998"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"cross_val_score(LogisticRegression(), X, y, cv=cv5_idx).mean()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.5.1"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
Sebastian Raschka, 2015
|
||||
|
||||
Python Machine Learning - Code Examples
|
||||
|
||||
|
||||
## Chapter 1 - Giving Computers the Ability to Learn from Data
|
||||
|
||||
- Building intelligent machines to transform data into knowledge
|
||||
- The three different types of machine learning
|
||||
- Making predictions about the future with supervised learning
|
||||
- Classification for predicting class labels
|
||||
- Regression for predicting continuous outcomes
|
||||
- Solving interactive problems with reinforcement learning
|
||||
- Discovering hidden structures with unsupervised learning
|
||||
- Finding subgroups with clustering
|
||||
- Dimensionality reduction for data compression
|
||||
- An introduction to the basic terminology and notations
|
||||
- A roadmap for building machine learning systems
|
||||
- Preprocessing – getting data into shape
|
||||
- Training and selecting a predictive model
|
||||
- Evaluating models and predicting unseen data instances
|
||||
- Using Python for machine learning
|
||||
- Installing Python packages
|
||||
- Summary
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
**Chapter 1 does not contain any code examples.**
|
||||
|
||||
---
|
||||
|
||||
## Installing Python packages
|
||||
|
||||
Python is available for all three major operating systems — Microsoft Windows, Mac OS X, and Linux — and the installer, as well as the documentation, can be downloaded from the official Python website: https://www.python.org.
|
||||
|
||||
This book is written for Python version `>= 3.4.3`, and it is recommended
|
||||
you use the most recent version of Python 3 that is currently available,
|
||||
although most of the code examples may also be compatible with Python `>= 2.7.10`. If you decide to use Python 2.7 to execute the code examples, please make sure that you know about the major differences between the two Python versions. A good summary about the differences between Python 3.4 and 2.7 can be found at https://wiki.python.org/moin/Python2orPython3.
|
||||
|
||||
**Note**
|
||||
|
||||
You can check your current default version of Python by executing
|
||||
|
||||
$ python -V
|
||||
|
||||
In my case, it returns
|
||||
|
||||
Python 3.5.1 :: Continuum Analytics, Inc.
|
||||
|
||||
|
||||
#### Pip
|
||||
|
||||
The additional packages that we will be using throughout this book can be installed via the `pip` installer program, which has been part of the Python standard library since Python 3.3. More information about pip can be found at https://docs.python.org/3/installing/index.html.
|
||||
|
||||
After we have successfully installed Python, we can execute pip from the command line terminal to install additional Python packages:
|
||||
|
||||
pip install SomePackage
|
||||
|
||||
|
||||
(where `SomePackage` is a placeholder for numpy, pandas, matplotlib, scikit-learn, and so forth).
|
||||
|
||||
Already installed packages can be updated via the `--upgrade` flag:
|
||||
|
||||
pip install SomePackage --upgrade
|
||||
|
||||
|
||||
#### Anaconda
|
||||
|
||||
A highly recommended alternative Python distribution for scientific computing
|
||||
is Anaconda by Continuum Analytics. Anaconda is a free—including commercial use—enterprise-ready Python distribution that bundles all the essential Python packages for data science, math, and engineering in one user-friendly cross-platform distribution. The Anaconda installer can be downloaded at http://continuum.io/downloads#py34, and an Anaconda quick start-guide is available at https://store.continuum.io/static/img/Anaconda-Quickstart.pdf.
|
||||
|
||||
After successfully installing Anaconda, we can install new Python packages using the following command:
|
||||
|
||||
conda install SomePackage
|
||||
|
||||
Existing packages can be updated using the following command:
|
||||
|
||||
conda update SomePackage
|
||||
|
||||
Throughout this book, we will mainly use NumPy's multi-dimensional arrays to store and manipulate data. Occasionally, we will make use of pandas, which is a library built on top of NumPy that provides additional higher level data manipulation tools that make working with tabular data even more convenient. To augment our learning experience and visualize quantitative data, which is often extremely useful to intuitively make sense of it, we will use the very customizable matplotlib library.
|
||||
|
||||
#### Core packages
|
||||
|
||||
The version numbers of the major Python packages that were used for writing this book are listed below. Please make sure that the version numbers of your installed packages are equal to, or greater than, those version numbers to ensure the code examples run correctly:
|
||||
|
||||
- [NumPy](http://www.numpy.org) 1.9.1
|
||||
- [SciPy](http://www.scipy.org) 0.14.0
|
||||
- [scikit-learn](http://scikit-learn.org/stable/) 0.15.2
|
||||
- [matplotlib](http://matplotlib.org) 1.4.0
|
||||
- [pandas](http://pandas.pydata.org) 0.15.2
|
||||
|
||||
## Python/Jupyter Notebook
|
||||
|
||||
Some readers were wondering about the `.ipynb` of the code files -- these files are IPython notebooks. I chose IPython notebooks over plain Python `.py` scripts, because I think that they are just great for data analysis projects! IPython notebooks allow us to have everything in one place: Our code, the results from executing the code, plots of our data, and documentation that supports the handy Markdown and powerful LaTeX syntax!
|
||||
|
||||
|
||||
|
||||
**Side Note:**
|
||||
|
||||
"IPython Notebook" recently became the "[Jupyter Notebook](<http://jupyter.org>)"; Jupyter is an umbrella project that aims to support other languages in addition to Python including Julia, R, and many more. Don't worry, though, for a Python user, there's only a difference in terminology (we say "Jupyter Notebook" now instead of "IPython Notebook").
|
||||
|
||||
The Jupyter notebook can be installed as usually via pip.
|
||||
|
||||
$ pip install jupyter notebook
|
||||
|
||||
Alternatively, we can use the Conda installer if we have Anaconda or Miniconda installed:
|
||||
|
||||
$ conda install jupyter notebook
|
||||
|
||||
To open a Jupyter notebook, we `cd` to the directory that contains your code examples, e.g,.
|
||||
|
||||
$ cd ~/code/python-machine-learning-book
|
||||
|
||||
and launch `jupyter notebook` by executing
|
||||
|
||||
$ jupyter notebook
|
||||
|
||||
Jupyter will start in our default browser (typically running at [http://localhost:8888/](http://localhost:8888/)). Now, we can simply select the notebook you wish to open from the Jupyter menu.
|
||||
|
||||
|
||||
|
||||
For more information about the Jupyter notebook, I recommend the [Jupyter Beginner Guide](http://jupyter-notebook-beginner-guide.readthedocs.org/en/latest/what_is_jupyter.html).
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
After Width: | Height: | Size: 76 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 159 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 41 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 181 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 74 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 304 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 86 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 72 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 23 KiB |
@@ -0,0 +1,28 @@
|
||||
import unittest
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
import watermark
|
||||
import nbformat
|
||||
|
||||
|
||||
def run_ipynb(path):
|
||||
error_cells = []
|
||||
with tempfile.NamedTemporaryFile(suffix=".ipynb") as fout:
|
||||
args = ["python", "-m", "nbconvert", "--to",
|
||||
"notebook", "--execute", "--output",
|
||||
"--ExecutePreprocessor.kernel_name", "python"
|
||||
fout.name, path]
|
||||
subprocess.check_output(args)
|
||||
|
||||
|
||||
class TestNotebooks(unittest.TestCase):
|
||||
|
||||
def test_appendix_g_tensorflow_basics(self):
|
||||
this_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
run_ipynb(os.path.join(this_dir,
|
||||
'../appendix_g_tensorflow-basics.ipynb'))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,15 @@
|
||||
Sebastian Raschka, 2015
|
||||
|
||||
Python Machine Learning - Code Examples
|
||||
|
||||
## Chapter 2 - Training Machine Learning Algorithms for Classification
|
||||
|
||||
- Artificial neurons - a brief glimpse into the early history
|
||||
- of machine learning
|
||||
- Implementing a perceptron learning algorithm in Python
|
||||
- Training a perceptron model on the Iris dataset
|
||||
- Adaptive linear neurons and the convergence of learning
|
||||
- Minimizing cost functions with gradient descent
|
||||
- Implementing an Adaptive Linear Neuron in Python
|
||||
- Large scale machine learning and stochastic gradient descent
|
||||
- Summary
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
After Width: | Height: | Size: 623 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 272 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 70 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 226 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 940 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 73 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 109 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 60 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 246 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 197 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 246 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 273 KiB |
@@ -0,0 +1,27 @@
|
||||
Sebastian Raschka, 2015
|
||||
|
||||
Python Machine Learning - Code Examples
|
||||
|
||||
|
||||
## Chapter 3 - A Tour of Machine Learning Classifiers Using Scikit-learn
|
||||
|
||||
- Choosing a classification algorithm
|
||||
- First steps with scikit-learn
|
||||
- Training a perceptron via scikit-learn
|
||||
- Modeling class probabilities via logistic regression
|
||||
- Logistic regression intuition and conditional probabilities
|
||||
- Learning the weights of the logistic cost function
|
||||
- Training a logistic regression model with scikit-learn
|
||||
- Tackling overfitting via regularization
|
||||
- Maximum margin classification with support vector machines
|
||||
- Maximum margin intuition
|
||||
- Dealing with the nonlinearly separable case using slack variables
|
||||
- Alternative implementations in scikit-learn
|
||||
- Solving nonlinear problems using a kernel SVM
|
||||
- Using the kernel trick to find separating hyperplanes in higher dimensional space
|
||||
- Decision tree learning
|
||||
- Maximizing information gain – getting the most bang for the buck
|
||||
- Building a decision tree
|
||||
- Combining weak to strong learners via random forests
|
||||
- K-nearest neighbors – a lazy learning algorithm
|
||||
- Summary
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
After Width: | Height: | Size: 148 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 80 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 76 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 148 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 93 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 93 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 78 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 148 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 56 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 525 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 73 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 149 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 143 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 253 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 160 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 131 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 133 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 202 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 150 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 30 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 65 KiB |
@@ -0,0 +1,20 @@
|
||||
digraph Tree {
|
||||
node [shape=box] ;
|
||||
0 [label="petal width <= 0.75\nentropy = 1.5799\nsamples = 105\nvalue = [34, 32, 39]"] ;
|
||||
1 [label="entropy = 0.0\nsamples = 34\nvalue = [34, 0, 0]"] ;
|
||||
0 -> 1 [labeldistance=2.5, labelangle=45, headlabel="True"] ;
|
||||
2 [label="petal length <= 4.95\nentropy = 0.993\nsamples = 71\nvalue = [0, 32, 39]"] ;
|
||||
0 -> 2 [labeldistance=2.5, labelangle=-45, headlabel="False"] ;
|
||||
3 [label="petal width <= 1.65\nentropy = 0.4306\nsamples = 34\nvalue = [0, 31, 3]"] ;
|
||||
2 -> 3 ;
|
||||
4 [label="entropy = 0.0\nsamples = 30\nvalue = [0, 30, 0]"] ;
|
||||
3 -> 4 ;
|
||||
5 [label="entropy = 0.8113\nsamples = 4\nvalue = [0, 1, 3]"] ;
|
||||
3 -> 5 ;
|
||||
6 [label="petal length <= 5.05\nentropy = 0.1793\nsamples = 37\nvalue = [0, 1, 36]"] ;
|
||||
2 -> 6 ;
|
||||
7 [label="entropy = 0.8113\nsamples = 4\nvalue = [0, 1, 3]"] ;
|
||||
6 -> 7 ;
|
||||
8 [label="entropy = 0.0\nsamples = 33\nvalue = [0, 0, 33]"] ;
|
||||
6 -> 8 ;
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
Sebastian Raschka, 2015
|
||||
|
||||
Python Machine Learning - Code Examples
|
||||
|
||||
## Chapter 4 - Building Good Training Sets – Data Preprocessing
|
||||
|
||||
- Dealing with missing data
|
||||
- Eliminating samples or features with missing values
|
||||
- Imputing missing values
|
||||
- Understanding the scikit-learn estimator API
|
||||
- Handling categorical data
|
||||
- Mapping ordinal features
|
||||
- Encoding class labels
|
||||
- Performing one-hot encoding on nominal features
|
||||
- Partitioning a dataset in training and test sets
|
||||
- Bringing features onto the same scale
|
||||
- Selecting meaningful features
|
||||
- Sparse solutions with L1 regularization
|
||||
- Sequential feature selection algorithms
|
||||
- Assessing feature importance with random forests
|
||||
- Summary
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 8.5 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 9.9 KiB |
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user