chore: import upstream snapshot with attribution

code/README.md

@@ -0,0 +1,59 @@
+## Resources for setting up your coding environment
+
+- [Instructions for setting up Python and the Jupyter Notebook](./ch01/README.md)  
+
+***If you need help with opening the Jupyter notebooks, I made a short [step by step guide](../docs/running_jupyter_nb.pdf) that illustrates this process***
+
+- A [quick and great NumPy refresher](https://docs.scipy.org/doc/numpy-dev/user/quickstart.html) that covers everything (and more) you'd need for this book
+
+- Recommended! To check your coding environment, open the `check_environment.ipynb` (it can be found in this directory) in Jupyter Notebook and execute the code cell:
+
+![](../images/check_env.png)
+
+
+## Table of contents and code notebooks
+
+
+Simply click on the `ipynb`/`nbviewer` links next to the chapter headlines to view the code examples (currently, the internal document links are only supported by the NbViewer version).
+**Please note that these are just the code examples accompanying the book, which I uploaded for your convenience; be aware that these notebooks may not be useful without the formulae and descriptive text.**   
+
+
+
+<br>
+
+
+1. Machine Learning - Giving Computers the Ability to Learn from Data [[dir](./ch01)] [[ipynb](./ch01/ch01.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch01/ch01.ipynb)]
+2. Training Machine Learning Algorithms for Classification [[dir](./ch02)] [[ipynb](./ch02/ch02.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch02/ch02.ipynb)]
+3. A Tour of Machine Learning Classifiers Using Scikit-Learn [[dir](./ch03)] [[ipynb](./ch03/ch03.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch03/ch03.ipynb)]
+4. Building Good Training Sets – Data Pre-Processing [[dir](./ch04)] [[ipynb](./ch04/ch04.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch04/ch04.ipynb)]
+5. Compressing Data via Dimensionality Reduction [[dir](./ch05)] [[ipynb](./ch05/ch05.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch05/ch05.ipynb)]
+6. Learning Best Practices for Model Evaluation and Hyperparameter Optimization [[dir](./ch06)] [[ipynb](./ch06/ch06.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch06/ch06.ipynb)]
+7. Combining Different Models for Ensemble Learning [[dir](./ch07)] [[ipynb](./ch07/ch07.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch07/ch07.ipynb)]
+8. Applying Machine Learning to Sentiment Analysis [[dir](./ch08)] [[ipynb](./ch08/ch08.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch08/ch08.ipynb)]
+9. Embedding a Machine Learning Model into a Web Application [[dir](./ch09)] [[ipynb](./ch09/ch09.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch09/ch09.ipynb)]
+10. Predicting Continuous Target Variables with Regression Analysis [[dir](./ch10)] [[ipynb](./ch10/ch10.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch10/ch10.ipynb)]
+11. Working with Unlabeled Data – Clustering Analysis [[dir](./ch11)] [[ipynb](./ch11/ch11.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch11/ch11.ipynb)]
+12. Training Artificial Neural Networks for Image Recognition [[dir](./ch12)] [[ipynb](./ch12/ch12.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch12/ch12.ipynb)]
+13. Parallelizing Neural Network Training via Theano [[dir](./ch13)] [[ipynb](./ch13/ch13.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch13/ch13.ipynb)]
+
+<br>
+
+**Bonus Notebooks (not in the book)**
+
+- Logistic Regression Implementation [[dir](./bonus)] [[ipynb](./bonus/logistic_regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/logistic_regression.ipynb)]
+- A Basic Pipeline and Grid Search Setup [[dir](./bonus)] [[ipynb](./bonus/svm_iris_pipeline_and_gridsearch.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/svm_iris_pipeline_and_gridsearch.ipynb)]
+- An Extended Nested Cross-Validation Example [[dir](./bonus)] [[ipynb](./bonus/nested_cross_validation.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/nested_cross_validation.ipynb)]
+- A Simple(r) Barebones Flask Webapp Template [[view directory](./bonus/flask_webapp_ex01)][[download as zip-file](https://github.com/rasbt/python-machine-learning-book/raw/master/code/bonus/flask_webapp_ex01/flask_webapp_ex01.zip)]
+- Reading handwritten digits from MNIST into NumPy arrays [[GitHub ipynb](./bonus/reading_mnist.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/reading_mnist.ipynb)]
+- Scikit-learn Model Persistence using JSON [[GitHub ipynb](./bonus/scikit-model-to-json.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/scikit-model-to-json.ipynb)]
+- Multinomial logistic regression / softmax regression [[GitHub ipynb](./bonus/softmax-regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/softmax-regression.ipynb)]
+
+
+
+
+## Contact
+
+I am happy to answer questions! Just write me an [email](mailto:mail@sebastianraschka.com)
+or consider asking the question on the [Google Groups Email List](https://groups.google.com/forum/#!forum/python-machine-learning-book).
+
+If you are interested in keeping in touch, I have quite a lively twitter stream ([@rasbt](https://twitter.com/rasbt)) all about data science and machine learning. I also maintain a [blog](http://sebastianraschka.com/articles.html) where I post all of the things I am particularly excited about.

code/_convenience_scripts/blank_tocs/ch01.toc

@@ -0,0 +1,17 @@
+Building intelligent machines to transform data into knowledge
+The three different types of machine learning
+  Making predictions about the future with supervised learning
+    Classification for predicting class labels
+    Regression for predicting continuous outcomes
+  Solving interactive problems with reinforcement learning
+  Discovering hidden structures with unsupervised learning
+    Finding subgroups with clustering
+    Dimensionality reduction for data compression
+An introduction to the basic terminology and notations
+A roadmap for building machine learning systems
+  Preprocessing – getting data into shape
+  Training and selecting a predictive model
+  Evaluating models and predicting unseen data instances
+Using Python for machine learning
+  Installing Python packages
+Summary

code/_convenience_scripts/blank_tocs/ch02.toc

@@ -0,0 +1,9 @@
+Artificial neurons - a brief glimpse into the early history
+of machine learning
+Implementing a perceptron learning algorithm in Python
+  Training a perceptron model on the Iris dataset
+Adaptive linear neurons and the convergence of learning
+  Minimizing cost functions with gradient descent
+  Implementing an Adaptive Linear Neuron in Python
+  Large scale machine learning and stochastic gradient descent
+Summary

code/_convenience_scripts/blank_tocs/ch03.toc

@@ -0,0 +1,20 @@
+Choosing a classification algorithm
+First steps with scikit-learn
+    Training a perceptron via scikit-learn
+Modeling class probabilities via logistic regression
+    Logistic regression intuition and conditional probabilities
+    Learning the weights of the logistic cost function
+    Training a logistic regression model with scikit-learn
+    Tackling overfitting via regularization
+Maximum margin classification with support vector machines
+    Maximum margin intuition
+    Dealing with the nonlinearly separable case using slack variables
+    Alternative implementations in scikit-learn
+Solving nonlinear problems using a kernel SVM
+    Using the kernel trick to find separating hyperplanes in higher dimensional space
+Decision tree learning
+    Maximizing information gain – getting the most bang for the buck
+    Building a decision tree
+    Combining weak to strong learners via random forests
+K-nearest neighbors – a lazy learning algorithm
+Summary

code/_convenience_scripts/blank_tocs/ch04.toc

@@ -0,0 +1,15 @@
+Dealing with missing data
+  Eliminating samples or features with missing values
+  Imputing missing values
+  Understanding the scikit-learn estimator API
+Handling categorical data
+  Mapping ordinal features
+  Encoding class labels
+  Performing one-hot encoding on nominal features
+Partitioning a dataset in training and test sets
+Bringing features onto the same scale
+Selecting meaningful features
+  Sparse solutions with L1 regularization
+  Sequential feature selection algorithms
+Assessing feature importance with random forests
+Summary

code/_convenience_scripts/blank_tocs/ch05.toc

@@ -0,0 +1,17 @@
+Unsupervised dimensionality reduction via principal component analysis 128
+  Total and explained variance
+  Feature transformation
+  Principal component analysis in scikit-learn
+Supervised data compression via linear discriminant analysis
+  Computing the scatter matrices
+  Selecting linear discriminants for the new feature subspace
+  Projecting samples onto the new feature space
+  LDA via scikit-learn
+Using kernel principal component analysis for nonlinear mappings
+  Kernel functions and the kernel trick
+  Implementing a kernel principal component analysis in Python
+    Example 1 – separating half-moon shapes
+    Example 2 – separating concentric circles
+  Projecting new data points
+  Kernel principal component analysis in scikit-learn
+Summary

code/_convenience_scripts/blank_tocs/ch06.toc

@@ -0,0 +1,4 @@
+Streamlining workflows with pipelines
+  Loading the Breast Cancer Wisconsin dataset
+  Combining transformers and estimators in a pipeline
+Using k-fold cross-validation to assess model performance

code/_convenience_scripts/blank_tocs/ch07.toc

@@ -0,0 +1,6 @@
+Learning with ensembles
+Implementing a simple majority vote classifier
+  Combining different algorithms for classification with majority vote
+Evaluating and tuning the ensemble classifier
+Bagging – building an ensemble of classifiers from bootstrap samples
+Leveraging weak learners via adaptive boosting

code/_convenience_scripts/blank_tocs/ch08.toc

@@ -0,0 +1,7 @@
+Obtaining the IMDb movie review dataset
+Introducing the bag-of-words model
+  Transforming words into feature vectors
+  Assessing word relevancy via term frequency-inverse document frequency 
+  Cleaning text data
+  Processing documents into tokens
+Training a logistic regression model for document classification

code/_convenience_scripts/blank_tocs/ch09.toc

@@ -0,0 +1,6 @@
+Serializing fitted scikit-learn estimators
+Setting up a SQLite database for data storage 
+Developing a web application with Flask
+Our first Flask web application
+  Form validation and rendering
+  Turning the movie classifier into a web application

code/_convenience_scripts/blank_tocs/ch10.toc

@@ -0,0 +1,12 @@
+Introducing a simple linear regression model
+Exploring the Housing Dataset
+  Visualizing the important characteristics of a dataset
+Implementing an ordinary least squares linear regression model
+  Solving regression for regression parameters with gradient descent
+  Estimating the coefficient of a regression model via scikit-learn
+Fitting a robust regression model using RANSAC
+Evaluating the performance of linear regression models
+Using regularized methods for regression
+Turning a linear regression model into a curve – polynomial regression
+  Modeling nonlinear relationships in the Housing Dataset
+  Dealing with nonlinear relationships using random forests

code/_convenience_scripts/blank_tocs/ch11.toc

@@ -0,0 +1,11 @@
+Grouping objects by similarity using k-means
+  K-means++
+  Hard versus soft clustering
+  Using the elbow method to find the optimal number of clusters
+  Quantifying the quality of clustering via silhouette plots
+Organizing clusters as a hierarchical tree
+  Performing hierarchical clustering on a distance matrix
+  Attaching dendrograms to a heat map
+  Applying agglomerative clustering via scikit-learn
+Locating regions of high density via DBSCAN
+Summary

code/_convenience_scripts/blank_tocs/ch12.toc

@@ -0,0 +1,9 @@
+Modeling complex functions with artificial neural networks
+  Single-layer neural network recap
+  Introducing the multi-layer neural network architecture
+  Activating a neural network via forward propagation
+Classifying handwritten digits
+  Obtaining the MNIST dataset
+  Implementing a multi-layer perceptron
+Training an artificial neural network
+  Computing the logistic cost function

code/_convenience_scripts/blank_tocs/ch13.toc

@@ -0,0 +1,9 @@
+Building, compiling, and running expressions with Theano
+  What is Theano?
+  First steps with Theano
+  Configuring Theano
+  Working with array structures
+  Wrapping things up – a linear regression example
+Choosing activation functions for feedforward neural networks
+  Logistic function recap
+  Estimating probabilities in multi-class classification via the softmax function

code/_convenience_scripts/ipynb_toc_links.py

@@ -0,0 +1,14 @@
+# Sebastian Raschka, 2015
+# convenience function for myself to add internal links to IPython toc
+# use as `python ipynb_toc_links.py /blank_tocs/ch01.toc`
+
+import sys
+
+ipynb = sys.argv[1]
+with open(ipynb, 'r') as f:
+    for line in f:
+        out_str = ' ' * (len(line) - len(line.lstrip()))
+        line = line.strip()
+        out_str += '- [%s' % line
+        out_str += '](#%s)' % line.replace(' ', '-')
+        print(out_str)

code/_convenience_scripts/md_toc.py

@@ -0,0 +1,13 @@
+# Sebastian Raschka, 2015
+# convenience function for myself to create nested TOC lists
+# use as `python md_toc.py /blank_tocs/ch01.toc`
+
+import sys
+
+ipynb = sys.argv[1]
+with open(ipynb, 'r') as f:
+    for line in f:
+        out_str = ' ' * (len(line) - len(line.lstrip()))
+        line = line.strip()
+        out_str += '- %s' % line
+        print(out_str)

code/bonus/README.md

@@ -0,0 +1,20 @@
+Sebastian Raschka, 2015
+
+Python Machine Learning - Code Examples
+
+## Bonus Material
+
+A collection of additional notebooks and code examples to clarify and explain concepts based on reader feedback.
+
+
+- A Basic Pipeline and Grid Search Setup [[GitHub ipynb](./svm_iris_pipeline_and_gridsearch.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/svm_iris_pipeline_and_gridsearch.ipynb)]
+
+- An Extended Nested Cross-Validation Example [[GitHub ipynb](./nested_cross_validation.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/nested_cross_validation.ipynb)]
+
+- A Simple(r) Barebones Flask Webapp Template [[view directory](./flask_webapp_ex01)][[download as zip-file](https://github.com/rasbt/python-machine-learning-book/raw/master/code/bonus/flask_webapp_ex01/flask_webapp_ex01.zip)]
+
+- Reading handwritten digits from MNIST into NumPy arrays [[GitHub ipynb](./reading_mnist.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/reading_mnist.ipynb)]
+
+- Scikit-learn Model Persistence using JSON [[GitHub ipynb](./scikit-model-to-json.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/scikit-model-to-json.ipynb)]
+
+- Multinomial logistic regression / softmax regression [[GitHub ipynb](./softmax-regression.ipynb)] [[nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/bonus/softmax-regression.ipynb)]

code/bonus/flask_webapp_ex01/README.md

@@ -0,0 +1,16 @@
+Sebastian Raschka, 2015
+
+Python Machine Learning - Code Examples (Bonus Material)
+
+
+# A Simple(r) Barebones Flask Webapp Template
+
+A simple Flask app that calculates the sum of two numbers entered in the respective input fields.
+
+You can run the app locally by executing `python app.py` within this directory.
+
+<hr>
+
+![](./img/img_1.png)
+
+Click [here](https://github.com/rasbt/python-machine-learning-book/raw/master/code/bonus/flask_webapp_ex01/flask_webapp_ex01.zip) to download this example as zip-file.

code/bonus/flask_webapp_ex01/app.py

@@ -0,0 +1,31 @@
+from flask import Flask, render_template, request
+from wtforms import Form, DecimalField, validators
+
+app = Flask(__name__)
+
+
+class EntryForm(Form):
+    x_entry = DecimalField('x:',
+                           places=10,
+                           validators=[validators.NumberRange(-1e10, 1e10)])
+    y_entry = DecimalField('y:',
+                           places=10,
+                           validators=[validators.NumberRange(-1e10, 1e10)])
+
+@app.route('/')
+def index():
+    form = EntryForm(request.form)
+    return render_template('entry.html', form=form, z='')
+
+@app.route('/results', methods=['POST'])
+def results():
+    form = EntryForm(request.form)
+    z = ''
+    if request.method == 'POST' and form.validate():
+        x = request.form['x_entry']
+        y = request.form['y_entry']
+        z = float(x) + float(y)
+    return render_template('entry.html', form=form, z=z)
+
+if __name__ == '__main__':
+    app.run(debug=True)

code/bonus/flask_webapp_ex01/static/style.css

@@ -0,0 +1,7 @@
+body{
+	width:600px;
+}
+
+#button{
+	padding-top: 20px;
+}

code/bonus/flask_webapp_ex01/templates/_formhelpers.html

@@ -0,0 +1,12 @@
+{% macro render_field(field) %}
+  <dt>{{ field.label }}
+  <dd>{{ field(**kwargs)|safe }}
+  {% if field.errors %}
+    <ul class=errors>
+    {% for error in field.errors %}
+      <li>{{ error }}</li>
+    {% endfor %}
+    </ul>
+  {% endif %}
+  </dd>
+{% endmacro %}

code/bonus/flask_webapp_ex01/templates/entry.html

@@ -0,0 +1,26 @@
+<!doctype html>
+<html>
+  <head>
+    <title>Webapp Ex 1</title>
+	<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
+  </head>
+  <body>
+
+{% from "_formhelpers.html" import render_field %}
+
+<form method=post action="/results">
+  <dl>
+	{{ render_field(form.x_entry, cols='1', rows='1') }}
+    {{ render_field(form.y_entry, cols='1', rows='1') }}
+  </dl>
+  <div>
+	  <input type=submit value='Submit' name='submit_btn'>
+  </div>
+
+  <dl>
+	x + y = {{ z }}
+  </dl>
+</form>
+
+  </body>
+</html>

code/bonus/nested_cross_validation.ipynb

@@ -0,0 +1,311 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "[Sebastian Raschka](http://sebastianraschka.com), 2015\n",
+    "\n",
+    "https://github.com/rasbt/python-machine-learning-book"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Python Machine Learning - Code Examples"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Bonus Material - An Extended Nested Cross-Validation Example"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For an explanation of nested cross-validation, please see:\n",
+    "    \n",
+    "- Chapter 6, section \"Algorithm-selection-with-nested-cross-validation\" (open the code example via [nbviewer](http://nbviewer.ipython.org/github/rasbt/python-machine-learning-book/blob/master/code/ch06/ch06.ipynb#Algorithm-selection-with-nested-cross-validation))\n",
+    "- FAQ, section: [How do I evaluate a model?](https://github.com/rasbt/python-machine-learning-book/blob/master/faq/evaluate-a-model.md)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<br>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Sebastian Raschka \n",
+      "Last updated: 11/30/2015 \n",
+      "\n",
+      "CPython 3.5.0\n",
+      "IPython 4.0.0\n",
+      "\n",
+      "numpy 1.10.1\n",
+      "pandas 0.17.1\n",
+      "matplotlib 1.5.0\n",
+      "scikit-learn 0.17\n"
+     ]
+    }
+   ],
+   "source": [
+    "%load_ext watermark\n",
+    "%watermark -a 'Sebastian Raschka' -u -d -v -p numpy,pandas,matplotlib,scikit-learn"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Dataset and Estimator Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "from sklearn.grid_search import GridSearchCV\n",
+    "from sklearn.pipeline import Pipeline\n",
+    "from sklearn.preprocessing import StandardScaler\n",
+    "from sklearn.svm import SVC\n",
+    "from sklearn.datasets import load_iris\n",
+    "from sklearn.cross_validation import train_test_split\n",
+    "\n",
+    "\n",
+    "# load and split data\n",
+    "iris = load_iris()\n",
+    "X, y = iris.data, iris.target\n",
+    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)\n",
+    "\n",
+    "# pipeline setup\n",
+    "cls = SVC(C=10.0, kernel='rbf', gamma=0.1, decision_function_shape='ovr')\n",
+    "kernel_svm = Pipeline([('std', StandardScaler()), \n",
+    "                       ('svc', cls)])\n",
+    "\n",
+    "# gridsearch setup\n",
+    "param_grid = [\n",
+    "  {'svc__C': [1, 10, 100, 1000], \n",
+    "   'svc__gamma': [0.001, 0.0001], \n",
+    "   'svc__kernel': ['rbf']},\n",
+    " ]\n",
+    "\n",
+    "\n",
+    "# setup multiple GridSearchCV objects, 1 for each algorithm\n",
+    "\n",
+    "gs_svm = GridSearchCV(estimator=kernel_svm, \n",
+    "                       param_grid=param_grid, \n",
+    "                       scoring='accuracy', \n",
+    "                       n_jobs=-1, \n",
+    "                       cv=5, \n",
+    "                       verbose=0, \n",
+    "                       refit=True,\n",
+    "                       pre_dispatch='2*n_jobs')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## A. Nested Crossvalidation - Quick Version"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here, the `cross_val_function` runs the 5 outer loops, and the the `GridSearch` object (`gs`) peforms the hyperparameter optimization during the 5 inner loops."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Average Accuracy 0.95 +/- 0.06\n"
+     ]
+    }
+   ],
+   "source": [
+    "import numpy as np \n",
+    "\n",
+    "from sklearn.cross_validation import cross_val_score\n",
+    "scores = cross_val_score(gs_svm, X_train, y_train, scoring='accuracy', cv=5)\n",
+    "print('\\nAverage Accuracy %.2f +/- %.2f' % (np.mean(scores), np.std(scores)))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## B. Nested Crossvalidation - Manual Approach Printing the Model Parameters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "from sklearn.cross_validation import StratifiedKFold\n",
+    "from sklearn.metrics import accuracy_score\n",
+    "import numpy as np\n",
+    "\n",
+    "params = []\n",
+    "scores = []\n",
+    "\n",
+    "skfold = StratifiedKFold(y=y_train, n_folds=5, shuffle=False, random_state=1)\n",
+    "for train_idx, test_idx in skfold:\n",
+    "    gs_svm.fit(X_train[train_idx], y_train[train_idx])\n",
+    "    y_pred = gs_svm.predict(X_train[test_idx])\n",
+    "    acc = accuracy_score(y_true=y_train[test_idx], y_pred=y_pred)\n",
+    "    params.append(gs_svm.best_params_)\n",
+    "    scores.append(acc)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "SVM models:\n",
+      "1. Acc: 0.96 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
+      "2. Acc: 1.00 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
+      "3. Acc: 0.83 Params: {'svc__C': 1000, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
+      "4. Acc: 1.00 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
+      "5. Acc: 0.96 Params: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n",
+      "\n",
+      "Average Accuracy 0.95 +/- 0.06\n"
+     ]
+    }
+   ],
+   "source": [
+    "print('SVM models:')\n",
+    "for idx, m in enumerate(zip(params, scores)):\n",
+    "    print('%s. Acc: %.2f Params: %s' % (idx+1, m[1], m[0]))\n",
+    "print('\\nAverage Accuracy %.2f +/- %.2f' % (np.mean(scores), np.std(scores)))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Regular K-fold CV to Optimize the Model on the Complete Training Set"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Repeat the nested cross-validation for different algorithms. Then, pick the \"best\" algorithm (not the best model!). Next, use the complete training set to tune the best algorithm via grid search:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Best parameters {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n"
+     ]
+    }
+   ],
+   "source": [
+    "gs_svm.fit(X_train, y_train)\n",
+    "print('Best parameters %s' % gs_svm.best_params_)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Training accuracy: 0.97\n",
+      "Test accuracy: 0.97\n",
+      "Parameters: {'svc__C': 100, 'svc__gamma': 0.001, 'svc__kernel': 'rbf'}\n"
+     ]
+    }
+   ],
+   "source": [
+    "train_acc = accuracy_score(y_true=y_train, y_pred=gs_svm.predict(X_train))\n",
+    "test_acc = accuracy_score(y_true=y_test, y_pred=gs_svm.predict(X_test))\n",
+    "print('Training accuracy: %.2f' % train_acc)\n",
+    "print('Test accuracy: %.2f' % test_acc)\n",
+    "print('Parameters: %s' % gs_svm.best_params_)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

code/bonus/sckit-model-to-json/attributes.json

@@ -0,0 +1,29 @@
+{
+    "classes_":[
+        0,
+        1,
+        2
+    ],
+    "coef_":[
+        [
+            0.42625236403173844,
+            -8.557501546363858
+        ],
+        [
+            1.5644231337040186,
+            -1.6783659020502222
+        ],
+        [
+            -1.990675497337773,
+            10.235867448186507
+        ]
+    ],
+    "intercept_":[
+        27.533384852155145,
+        4.18509910962595,
+        -31.71848396177913
+    ],
+    "n_iter_":[
+        27
+    ]
+}

code/bonus/sckit-model-to-json/params.json

@@ -0,0 +1 @@
+{"dual": false, "max_iter": 100, "warm_start": false, "verbose": 0, "C": 100.0, "class_weight": null, "random_state": 1, "fit_intercept": true, "multi_class": "multinomial", "intercept_scaling": 1, "penalty": "l2", "solver": "newton-cg", "n_jobs": 1, "tol": 0.0001}

code/bonus/svm_iris_pipeline_and_gridsearch.ipynb

@@ -0,0 +1,355 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "[Sebastian Raschka](http://sebastianraschka.com), 2015\n",
+    "\n",
+    "https://github.com/rasbt/python-machine-learning-book"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Python Machine Learning - Code Examples"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Bonus Material - A Basic Pipeline and Grid Search Setup"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Sebastian Raschka \n",
+      "Last updated: 01/20/2016 \n",
+      "\n",
+      "CPython 3.5.1\n",
+      "IPython 4.0.1\n",
+      "\n",
+      "numpy 1.10.1\n",
+      "pandas 0.17.1\n",
+      "matplotlib 1.5.0\n",
+      "scikit-learn 0.17\n"
+     ]
+    }
+   ],
+   "source": [
+    "%load_ext watermark\n",
+    "%watermark -a 'Sebastian Raschka' -u -d -v -p numpy,pandas,matplotlib,scikit-learn"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Fitting 5 folds for each of 8 candidates, totalling 40 fits\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[Parallel(n_jobs=-1)]: Done  40 out of  40 | elapsed:    0.2s finished\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "GridSearchCV(cv=5, error_score='raise',\n",
+       "       estimator=Pipeline(steps=[('std', StandardScaler(copy=True, with_mean=True, with_std=True)), ('svc', SVC(C=10.0, cache_size=200, class_weight=None, coef0=0.0,\n",
+       "  decision_function_shape='ovr', degree=3, gamma=0.1, kernel='rbf',\n",
+       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
+       "  tol=0.001, verbose=False))]),\n",
+       "       fit_params={}, iid=True, n_jobs=-1,\n",
+       "       param_grid=[{'svc__kernel': ['rbf'], 'svc__C': [1, 10, 100, 1000], 'svc__gamma': [0.001, 0.0001]}],\n",
+       "       pre_dispatch='2*n_jobs', refit=True, scoring='accuracy', verbose=1)"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from sklearn.grid_search import GridSearchCV\n",
+    "from sklearn.pipeline import Pipeline\n",
+    "from sklearn.preprocessing import StandardScaler\n",
+    "from sklearn.svm import SVC\n",
+    "from sklearn.datasets import load_iris\n",
+    "from sklearn.cross_validation import train_test_split\n",
+    "\n",
+    "\n",
+    "# load and split data\n",
+    "iris = load_iris()\n",
+    "X, y = iris.data, iris.target\n",
+    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)\n",
+    "\n",
+    "# pipeline setup\n",
+    "cls = SVC(C=10.0, \n",
+    "          kernel='rbf', \n",
+    "          gamma=0.1, \n",
+    "          decision_function_shape='ovr')\n",
+    "\n",
+    "kernel_svm = Pipeline([('std', StandardScaler()), \n",
+    "                       ('svc', cls)])\n",
+    "\n",
+    "# gridsearch setup\n",
+    "param_grid = [\n",
+    "  {'svc__C': [1, 10, 100, 1000], \n",
+    "   'svc__gamma': [0.001, 0.0001], \n",
+    "   'svc__kernel': ['rbf']},\n",
+    " ]\n",
+    "\n",
+    "gs = GridSearchCV(estimator=kernel_svm, \n",
+    "                  param_grid=param_grid, \n",
+    "                  scoring='accuracy', \n",
+    "                  n_jobs=-1, \n",
+    "                  cv=5, \n",
+    "                  verbose=1, \n",
+    "                  refit=True,\n",
+    "                  pre_dispatch='2*n_jobs')\n",
+    "\n",
+    "# run gridearch\n",
+    "gs.fit(X_train, y_train)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Best GS Score 0.96\n",
+      "best GS Params {'svc__kernel': 'rbf', 'svc__C': 100, 'svc__gamma': 0.001}\n",
+      "\n",
+      "Train Accuracy: 0.97\n",
+      "\n",
+      "Test Accuracy: 0.97\n"
+     ]
+    }
+   ],
+   "source": [
+    "print('Best GS Score %.2f' % gs.best_score_)\n",
+    "print('best GS Params %s' % gs.best_params_)\n",
+    "\n",
+    "\n",
+    "# prediction on the training set\n",
+    "y_pred = gs.predict(X_train)\n",
+    "train_acc = (y_train == y_pred).sum()/len(y_train)\n",
+    "print('\\nTrain Accuracy: %.2f' % (train_acc))\n",
+    "\n",
+    "# evaluation on the test set\n",
+    "y_pred = gs.predict(X_test)\n",
+    "test_acc = (y_test == y_pred).sum()/len(y_test)\n",
+    "print('\\nTest Accuracy: %.2f' % (test_acc))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "### A Note about `GridSearchCV`'s `best_score_` attribute"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Please note that `gs.best_score_` is the average k-fold cross-validation score. I.e., if we have a `GridSearchCV` object with 5-fold cross-validation (like the one above), the `best_score_` attribute returns the average score over the 5-folds of the best model. To illustrate this with an example:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([ 0.6,  0.4,  0.6,  0.2,  0.6])"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from sklearn.cross_validation import StratifiedKFold, cross_val_score\n",
+    "from sklearn.linear_model import LogisticRegression\n",
+    "import numpy as np\n",
+    "\n",
+    "np.random.seed(0)\n",
+    "np.set_printoptions(precision=6)\n",
+    "y = [np.random.randint(3) for i in range(25)]\n",
+    "X = (y + np.random.randn(25)).reshape(-1, 1)\n",
+    "\n",
+    "cv5_idx = list(StratifiedKFold(y, n_folds=5, shuffle=False, random_state=0))\n",
+    "cross_val_score(LogisticRegression(random_state=123), X, y, cv=cv5_idx)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "By executing the code above, we created a simple data set of random integers that shall represent our class labels. Next, we fed the indices of 5 cross-validation folds (`cv3_idx`) to the `cross_val_score` scorer, which returned 5 accuracy scores -- these are the 5 accuracy values for the 5 test folds.  \n",
+    "\n",
+    "Next, let us use the `GridSearchCV` object and feed it the same 5 cross-validation sets (via the pre-generated `cv3_idx` indices):"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Fitting 5 folds for each of 1 candidates, totalling 5 fits\n",
+      "[CV]  ................................................................\n",
+      "[CV] ....................................... , score=0.600000 -   0.0s\n",
+      "[CV]  ................................................................\n",
+      "[CV] ....................................... , score=0.400000 -   0.0s\n",
+      "[CV]  ................................................................\n",
+      "[CV] ....................................... , score=0.600000 -   0.0s\n",
+      "[CV]  ................................................................\n",
+      "[CV] ....................................... , score=0.200000 -   0.0s\n",
+      "[CV]  ................................................................\n",
+      "[CV] ....................................... , score=0.600000 -   0.0s\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:    0.0s finished\n"
+     ]
+    }
+   ],
+   "source": [
+    "from sklearn.grid_search import GridSearchCV\n",
+    "gs = GridSearchCV(LogisticRegression(), {}, cv=cv5_idx, verbose=3).fit(X, y) "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As we can see, the scores for the 5 folds are exactly the same as the ones from `cross_val_score` earlier.  \n",
+    "Now, the best_score_ attribute of the `GridSearchCV` object, which becomes available after `fit`ting, returns the average accuracy score of the best model:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "0.47999999999999998"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "gs.best_score_"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As we can see, the result above is consistent with the average score computed the `cross_val_score`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "0.47999999999999998"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "cross_val_score(LogisticRegression(), X, y, cv=cv5_idx).mean()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

code/ch01/README.md

@@ -0,0 +1,124 @@
+Sebastian Raschka, 2015
+
+Python Machine Learning - Code Examples
+
+
+##  Chapter 1 - Giving Computers the Ability to Learn from Data
+
+- Building intelligent machines to transform data into knowledge
+- The three different types of machine learning
+  - Making predictions about the future with supervised learning
+    - Classification for predicting class labels
+    - Regression for predicting continuous outcomes
+  - Solving interactive problems with reinforcement learning
+  - Discovering hidden structures with unsupervised learning
+    - Finding subgroups with clustering
+    - Dimensionality reduction for data compression
+- An introduction to the basic terminology and notations
+- A roadmap for building machine learning systems
+  - Preprocessing – getting data into shape
+  - Training and selecting a predictive model
+  - Evaluating models and predicting unseen data instances
+- Using Python for machine learning
+  - Installing Python packages
+- Summary
+
+
+
+---
+
+**Chapter 1 does not contain any code examples.**
+
+---
+
+## Installing Python packages
+
+Python is available for all three major operating systems — Microsoft Windows, Mac OS X, and Linux — and the installer, as well as the documentation, can be downloaded from the official Python website: https://www.python.org.
+
+This book is written for Python version `>= 3.4.3`, and it is recommended
+you use the most recent version of Python 3 that is currently available,
+although most of the code examples may also be compatible with Python `>= 2.7.10`. If you decide to use Python 2.7 to execute the code examples, please make sure that you know about the major differences between the two Python versions. A good summary about the differences between Python 3.4 and 2.7 can be found at https://wiki.python.org/moin/Python2orPython3.
+
+**Note**
+
+You can check your current default version of Python by executing
+
+    $ python -V
+
+In my case, it returns
+
+    Python 3.5.1 :: Continuum Analytics, Inc.
+
+
+#### Pip
+
+The additional packages that we will be using throughout this book can be installed via the `pip` installer program, which has been part of the Python standard library since Python 3.3. More information about pip can be found at https://docs.python.org/3/installing/index.html.
+
+After we have successfully installed Python, we can execute pip from the command line terminal to install additional Python packages:
+
+    pip install SomePackage
+
+
+(where `SomePackage` is a placeholder for numpy, pandas, matplotlib, scikit-learn, and so forth).
+
+Already installed packages can be updated via the `--upgrade` flag:
+
+    pip install SomePackage --upgrade
+
+
+#### Anaconda
+
+A highly recommended alternative Python distribution for scientific computing
+is Anaconda by Continuum Analytics. Anaconda is a free—including commercial use—enterprise-ready Python distribution that bundles all the essential Python packages for data science, math, and engineering in one user-friendly cross-platform distribution. The Anaconda installer can be downloaded at http://continuum.io/downloads#py34, and an Anaconda quick start-guide is available at https://store.continuum.io/static/img/Anaconda-Quickstart.pdf.
+
+After successfully installing Anaconda, we can install new Python packages using the following command:
+
+    conda install SomePackage
+
+Existing packages can be updated using the following command:
+
+    conda update SomePackage
+
+Throughout this book, we will mainly use NumPy's multi-dimensional arrays to store and manipulate data. Occasionally, we will make use of pandas, which is a library built on top of NumPy that provides additional higher level data manipulation tools that make working with tabular data even more convenient. To augment our learning experience and visualize quantitative data, which is often extremely useful to intuitively make sense of it, we will use the very customizable matplotlib library.
+
+#### Core packages
+
+The version numbers of the major Python packages that were used for writing this book are listed below. Please make sure that the version numbers of your installed packages are equal to, or greater than, those version numbers to ensure the code examples run correctly:
+
+- [NumPy](http://www.numpy.org) 1.9.1
+- [SciPy](http://www.scipy.org) 0.14.0
+- [scikit-learn](http://scikit-learn.org/stable/) 0.15.2
+- [matplotlib](http://matplotlib.org) 1.4.0
+- [pandas](http://pandas.pydata.org) 0.15.2
+
+## Python/Jupyter Notebook
+
+Some readers were wondering about the `.ipynb` of the code files -- these files are IPython notebooks. I chose IPython notebooks over plain Python `.py` scripts, because I think that they are just great for data analysis projects! IPython notebooks allow us to have everything in one place: Our code, the results from executing the code, plots of our data, and documentation that supports the handy Markdown and powerful LaTeX syntax!
+
+![](./images/ipynb_ex1.png)
+
+**Side Note:**  
+
+"IPython Notebook" recently became the "[Jupyter Notebook](<http://jupyter.org>)"; Jupyter is an umbrella project that aims to support other languages in addition to Python including Julia, R, and many more. Don't worry, though, for a Python user, there's only a difference in terminology (we say "Jupyter Notebook" now instead of "IPython Notebook").
+
+The Jupyter notebook can be installed as usually via pip.
+
+    $ pip install jupyter notebook
+
+Alternatively, we can use the Conda installer if we have Anaconda or Miniconda installed:
+
+    $ conda install jupyter notebook
+
+To open a Jupyter notebook, we `cd` to the directory that contains your code examples, e.g,.
+
+    $ cd ~/code/python-machine-learning-book
+
+and launch `jupyter notebook` by executing
+
+    $ jupyter notebook
+
+Jupyter will start in our default browser (typically running at [http://localhost:8888/](http://localhost:8888/)). Now, we can simply select the notebook you wish to open from the Jupyter menu.
+
+![](./images/ipynb_ex2.png)
+
+For more information about the Jupyter notebook, I recommend the [Jupyter Beginner Guide](http://jupyter-notebook-beginner-guide.readthedocs.org/en/latest/what_is_jupyter.html).

code/ch01/tests/test_notebooks.py

@@ -0,0 +1,28 @@
+import unittest
+import os
+import subprocess
+import tempfile
+import watermark
+import nbformat
+
+
+def run_ipynb(path):
+    error_cells = []
+    with tempfile.NamedTemporaryFile(suffix=".ipynb") as fout:
+        args = ["python", "-m", "nbconvert", "--to",
+                "notebook", "--execute", "--output",
+                "--ExecutePreprocessor.kernel_name", "python"
+                fout.name, path]
+        subprocess.check_output(args)
+
+
+class TestNotebooks(unittest.TestCase):
+
+    def test_appendix_g_tensorflow_basics(self):
+        this_dir = os.path.dirname(os.path.abspath(__file__))
+        run_ipynb(os.path.join(this_dir,
+                               '../appendix_g_tensorflow-basics.ipynb'))
+
+
+if __name__ == '__main__':
+    unittest.main()

code/ch02/README.md

@@ -0,0 +1,15 @@
+Sebastian Raschka, 2015
+
+Python Machine Learning - Code Examples
+
+##  Chapter 2 - Training Machine Learning Algorithms for Classification
+
+- Artificial neurons - a brief glimpse into the early history
+- of machine learning
+- Implementing a perceptron learning algorithm in Python
+  - Training a perceptron model on the Iris dataset
+- Adaptive linear neurons and the convergence of learning
+  - Minimizing cost functions with gradient descent
+  - Implementing an Adaptive Linear Neuron in Python
+  - Large scale machine learning and stochastic gradient descent
+- Summary

code/ch03/README.md

@@ -0,0 +1,27 @@
+Sebastian Raschka, 2015
+
+Python Machine Learning - Code Examples
+
+
+## Chapter 3 - A Tour of Machine Learning Classifiers Using Scikit-learn
+
+- Choosing a classification algorithm
+- First steps with scikit-learn
+    - Training a perceptron via scikit-learn
+- Modeling class probabilities via logistic regression
+    - Logistic regression intuition and conditional probabilities
+    - Learning the weights of the logistic cost function
+    - Training a logistic regression model with scikit-learn
+    - Tackling overfitting via regularization
+- Maximum margin classification with support vector machines
+    - Maximum margin intuition
+    - Dealing with the nonlinearly separable case using slack variables
+    - Alternative implementations in scikit-learn
+- Solving nonlinear problems using a kernel SVM
+    - Using the kernel trick to find separating hyperplanes in higher dimensional space
+- Decision tree learning
+    - Maximizing information gain – getting the most bang for the buck
+    - Building a decision tree
+    - Combining weak to strong learners via random forests
+- K-nearest neighbors – a lazy learning algorithm
+- Summary

code/ch03/tree.dot

@@ -0,0 +1,20 @@
+digraph Tree {
+node [shape=box] ;
+0 [label="petal width <= 0.75\nentropy = 1.5799\nsamples = 105\nvalue = [34, 32, 39]"] ;
+1 [label="entropy = 0.0\nsamples = 34\nvalue = [34, 0, 0]"] ;
+0 -> 1 [labeldistance=2.5, labelangle=45, headlabel="True"] ;
+2 [label="petal length <= 4.95\nentropy = 0.993\nsamples = 71\nvalue = [0, 32, 39]"] ;
+0 -> 2 [labeldistance=2.5, labelangle=-45, headlabel="False"] ;
+3 [label="petal width <= 1.65\nentropy = 0.4306\nsamples = 34\nvalue = [0, 31, 3]"] ;
+2 -> 3 ;
+4 [label="entropy = 0.0\nsamples = 30\nvalue = [0, 30, 0]"] ;
+3 -> 4 ;
+5 [label="entropy = 0.8113\nsamples = 4\nvalue = [0, 1, 3]"] ;
+3 -> 5 ;
+6 [label="petal length <= 5.05\nentropy = 0.1793\nsamples = 37\nvalue = [0, 1, 36]"] ;
+2 -> 6 ;
+7 [label="entropy = 0.8113\nsamples = 4\nvalue = [0, 1, 3]"] ;
+6 -> 7 ;
+8 [label="entropy = 0.0\nsamples = 33\nvalue = [0, 0, 33]"] ;
+6 -> 8 ;
+}

code/ch04/README.md

@@ -0,0 +1,21 @@
+Sebastian Raschka, 2015
+
+Python Machine Learning - Code Examples
+
+## Chapter 4 - Building Good Training Sets – Data Preprocessing
+
+- Dealing with missing data
+  - Eliminating samples or features with missing values
+  - Imputing missing values
+  - Understanding the scikit-learn estimator API
+- Handling categorical data
+  - Mapping ordinal features
+  - Encoding class labels
+  - Performing one-hot encoding on nominal features
+- Partitioning a dataset in training and test sets
+- Bringing features onto the same scale
+- Selecting meaningful features
+  - Sparse solutions with L1 regularization
+  - Sequential feature selection algorithms
+- Assessing feature importance with random forests
+- Summary