wehub-resource-sync
710 lines
69 KiB
Plaintext
710 lines
69 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Using TensorFlow backend.\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"'2.0.8'"
|
|
]
|
|
},
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import keras\n",
|
|
"keras.__version__"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Predicting house prices: a regression example\n",
|
|
"\n",
|
|
"This notebook contains the code samples found in Chapter 3, Section 6 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
|
|
"\n",
|
|
"----\n",
|
|
"\n",
|
|
"\n",
|
|
"In our two previous examples, we were considering classification problems, where the goal was to predict a single discrete label of an \n",
|
|
"input data point. Another common type of machine learning problem is \"regression\", which consists of predicting a continuous value instead \n",
|
|
"of a discrete label. For instance, predicting the temperature tomorrow, given meteorological data, or predicting the time that a \n",
|
|
"software project will take to complete, given its specifications.\n",
|
|
"\n",
|
|
"Do not mix up \"regression\" with the algorithm \"logistic regression\": confusingly, \"logistic regression\" is not a regression algorithm, \n",
|
|
"it is a classification algorithm."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## The Boston Housing Price dataset\n",
|
|
"\n",
|
|
"\n",
|
|
"We will be attempting to predict the median price of homes in a given Boston suburb in the mid-1970s, given a few data points about the \n",
|
|
"suburb at the time, such as the crime rate, the local property tax rate, etc.\n",
|
|
"\n",
|
|
"The dataset we will be using has another interesting difference from our two previous examples: it has very few data points, only 506 in \n",
|
|
"total, split between 404 training samples and 102 test samples, and each \"feature\" in the input data (e.g. the crime rate is a feature) has \n",
|
|
"a different scale. For instance some values are proportions, which take a values between 0 and 1, others take values between 1 and 12, \n",
|
|
"others between 0 and 100...\n",
|
|
"\n",
|
|
"Let's take a look at the data:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from keras.datasets import boston_housing\n",
|
|
"\n",
|
|
"(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"(404, 13)"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"train_data.shape"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"(102, 13)"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"test_data.shape"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"As you can see, we have 404 training samples and 102 test samples. The data comprises 13 features. The 13 features in the input data are as \n",
|
|
"follow:\n",
|
|
"\n",
|
|
"1. Per capita crime rate.\n",
|
|
"2. Proportion of residential land zoned for lots over 25,000 square feet.\n",
|
|
"3. Proportion of non-retail business acres per town.\n",
|
|
"4. Charles River dummy variable (= 1 if tract bounds river; 0 otherwise).\n",
|
|
"5. Nitric oxides concentration (parts per 10 million).\n",
|
|
"6. Average number of rooms per dwelling.\n",
|
|
"7. Proportion of owner-occupied units built prior to 1940.\n",
|
|
"8. Weighted distances to five Boston employment centres.\n",
|
|
"9. Index of accessibility to radial highways.\n",
|
|
"10. Full-value property-tax rate per $10,000.\n",
|
|
"11. Pupil-teacher ratio by town.\n",
|
|
"12. 1000 * (Bk - 0.63) ** 2 where Bk is the proportion of Black people by town.\n",
|
|
"13. % lower status of the population.\n",
|
|
"\n",
|
|
"The targets are the median values of owner-occupied homes, in thousands of dollars:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"array([ 15.2, 42.3, 50. , 21.1, 17.7, 18.5, 11.3, 15.6, 15.6,\n",
|
|
" 14.4, 12.1, 17.9, 23.1, 19.9, 15.7, 8.8, 50. , 22.5,\n",
|
|
" 24.1, 27.5, 10.9, 30.8, 32.9, 24. , 18.5, 13.3, 22.9,\n",
|
|
" 34.7, 16.6, 17.5, 22.3, 16.1, 14.9, 23.1, 34.9, 25. ,\n",
|
|
" 13.9, 13.1, 20.4, 20. , 15.2, 24.7, 22.2, 16.7, 12.7,\n",
|
|
" 15.6, 18.4, 21. , 30.1, 15.1, 18.7, 9.6, 31.5, 24.8,\n",
|
|
" 19.1, 22. , 14.5, 11. , 32. , 29.4, 20.3, 24.4, 14.6,\n",
|
|
" 19.5, 14.1, 14.3, 15.6, 10.5, 6.3, 19.3, 19.3, 13.4,\n",
|
|
" 36.4, 17.8, 13.5, 16.5, 8.3, 14.3, 16. , 13.4, 28.6,\n",
|
|
" 43.5, 20.2, 22. , 23. , 20.7, 12.5, 48.5, 14.6, 13.4,\n",
|
|
" 23.7, 50. , 21.7, 39.8, 38.7, 22.2, 34.9, 22.5, 31.1,\n",
|
|
" 28.7, 46. , 41.7, 21. , 26.6, 15. , 24.4, 13.3, 21.2,\n",
|
|
" 11.7, 21.7, 19.4, 50. , 22.8, 19.7, 24.7, 36.2, 14.2,\n",
|
|
" 18.9, 18.3, 20.6, 24.6, 18.2, 8.7, 44. , 10.4, 13.2,\n",
|
|
" 21.2, 37. , 30.7, 22.9, 20. , 19.3, 31.7, 32. , 23.1,\n",
|
|
" 18.8, 10.9, 50. , 19.6, 5. , 14.4, 19.8, 13.8, 19.6,\n",
|
|
" 23.9, 24.5, 25. , 19.9, 17.2, 24.6, 13.5, 26.6, 21.4,\n",
|
|
" 11.9, 22.6, 19.6, 8.5, 23.7, 23.1, 22.4, 20.5, 23.6,\n",
|
|
" 18.4, 35.2, 23.1, 27.9, 20.6, 23.7, 28. , 13.6, 27.1,\n",
|
|
" 23.6, 20.6, 18.2, 21.7, 17.1, 8.4, 25.3, 13.8, 22.2,\n",
|
|
" 18.4, 20.7, 31.6, 30.5, 20.3, 8.8, 19.2, 19.4, 23.1,\n",
|
|
" 23. , 14.8, 48.8, 22.6, 33.4, 21.1, 13.6, 32.2, 13.1,\n",
|
|
" 23.4, 18.9, 23.9, 11.8, 23.3, 22.8, 19.6, 16.7, 13.4,\n",
|
|
" 22.2, 20.4, 21.8, 26.4, 14.9, 24.1, 23.8, 12.3, 29.1,\n",
|
|
" 21. , 19.5, 23.3, 23.8, 17.8, 11.5, 21.7, 19.9, 25. ,\n",
|
|
" 33.4, 28.5, 21.4, 24.3, 27.5, 33.1, 16.2, 23.3, 48.3,\n",
|
|
" 22.9, 22.8, 13.1, 12.7, 22.6, 15. , 15.3, 10.5, 24. ,\n",
|
|
" 18.5, 21.7, 19.5, 33.2, 23.2, 5. , 19.1, 12.7, 22.3,\n",
|
|
" 10.2, 13.9, 16.3, 17. , 20.1, 29.9, 17.2, 37.3, 45.4,\n",
|
|
" 17.8, 23.2, 29. , 22. , 18. , 17.4, 34.6, 20.1, 25. ,\n",
|
|
" 15.6, 24.8, 28.2, 21.2, 21.4, 23.8, 31. , 26.2, 17.4,\n",
|
|
" 37.9, 17.5, 20. , 8.3, 23.9, 8.4, 13.8, 7.2, 11.7,\n",
|
|
" 17.1, 21.6, 50. , 16.1, 20.4, 20.6, 21.4, 20.6, 36.5,\n",
|
|
" 8.5, 24.8, 10.8, 21.9, 17.3, 18.9, 36.2, 14.9, 18.2,\n",
|
|
" 33.3, 21.8, 19.7, 31.6, 24.8, 19.4, 22.8, 7.5, 44.8,\n",
|
|
" 16.8, 18.7, 50. , 50. , 19.5, 20.1, 50. , 17.2, 20.8,\n",
|
|
" 19.3, 41.3, 20.4, 20.5, 13.8, 16.5, 23.9, 20.6, 31.5,\n",
|
|
" 23.3, 16.8, 14. , 33.8, 36.1, 12.8, 18.3, 18.7, 19.1,\n",
|
|
" 29. , 30.1, 50. , 50. , 22. , 11.9, 37.6, 50. , 22.7,\n",
|
|
" 20.8, 23.5, 27.9, 50. , 19.3, 23.9, 22.6, 15.2, 21.7,\n",
|
|
" 19.2, 43.8, 20.3, 33.2, 19.9, 22.5, 32.7, 22. , 17.1,\n",
|
|
" 19. , 15. , 16.1, 25.1, 23.7, 28.7, 37.2, 22.6, 16.4,\n",
|
|
" 25. , 29.8, 22.1, 17.4, 18.1, 30.3, 17.5, 24.7, 12.6,\n",
|
|
" 26.5, 28.7, 13.3, 10.4, 24.4, 23. , 20. , 17.8, 7. ,\n",
|
|
" 11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"train_targets"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"The prices are typically between \\$10,000 and \\$50,000. If that sounds cheap, remember this was the mid-1970s, and these prices are not \n",
|
|
"inflation-adjusted."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Preparing the data\n",
|
|
"\n",
|
|
"\n",
|
|
"It would be problematic to feed into a neural network values that all take wildly different ranges. The network might be able to \n",
|
|
"automatically adapt to such heterogeneous data, but it would definitely make learning more difficult. A widespread best practice to deal \n",
|
|
"with such data is to do feature-wise normalization: for each feature in the input data (a column in the input data matrix), we \n",
|
|
"will subtract the mean of the feature and divide by the standard deviation, so that the feature will be centered around 0 and will have a \n",
|
|
"unit standard deviation. This is easily done in Numpy:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"mean = train_data.mean(axis=0)\n",
|
|
"train_data -= mean\n",
|
|
"std = train_data.std(axis=0)\n",
|
|
"train_data /= std\n",
|
|
"\n",
|
|
"test_data -= mean\n",
|
|
"test_data /= std"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"Note that the quantities that we use for normalizing the test data have been computed using the training data. We should never use in our \n",
|
|
"workflow any quantity computed on the test data, even for something as simple as data normalization."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Building our network\n",
|
|
"\n",
|
|
"\n",
|
|
"Because so few samples are available, we will be using a very small network with two \n",
|
|
"hidden layers, each with 64 units. In general, the less training data you have, the worse overfitting will be, and using \n",
|
|
"a small network is one way to mitigate overfitting."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from keras import models\n",
|
|
"from keras import layers\n",
|
|
"\n",
|
|
"def build_model():\n",
|
|
" # Because we will need to instantiate\n",
|
|
" # the same model multiple times,\n",
|
|
" # we use a function to construct it.\n",
|
|
" model = models.Sequential()\n",
|
|
" model.add(layers.Dense(64, activation='relu',\n",
|
|
" input_shape=(train_data.shape[1],)))\n",
|
|
" model.add(layers.Dense(64, activation='relu'))\n",
|
|
" model.add(layers.Dense(1))\n",
|
|
" model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
|
|
" return model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"Our network ends with a single unit, and no activation (i.e. it will be linear layer). \n",
|
|
"This is a typical setup for scalar regression (i.e. regression where we are trying to predict a single continuous value). \n",
|
|
"Applying an activation function would constrain the range that the output can take; for instance if \n",
|
|
"we applied a `sigmoid` activation function to our last layer, the network could only learn to predict values between 0 and 1. Here, because \n",
|
|
"the last layer is purely linear, the network is free to learn to predict values in any range.\n",
|
|
"\n",
|
|
"Note that we are compiling the network with the `mse` loss function -- Mean Squared Error, the square of the difference between the \n",
|
|
"predictions and the targets, a widely used loss function for regression problems.\n",
|
|
"\n",
|
|
"We are also monitoring a new metric during training: `mae`. This stands for Mean Absolute Error. It is simply the absolute value of the \n",
|
|
"difference between the predictions and the targets. For instance, a MAE of 0.5 on this problem would mean that our predictions are off by \n",
|
|
"\\$500 on average."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Validating our approach using K-fold validation\n",
|
|
"\n",
|
|
"\n",
|
|
"To evaluate our network while we keep adjusting its parameters (such as the number of epochs used for training), we could simply split the \n",
|
|
"data into a training set and a validation set, as we were doing in our previous examples. However, because we have so few data points, the \n",
|
|
"validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot \n",
|
|
"depending on _which_ data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a \n",
|
|
"high _variance_ with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
|
|
"\n",
|
|
"The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions \n",
|
|
"(typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining \n",
|
|
"partition. The validation score for the model used would then be the average of the K validation scores obtained."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In terms of code, this is straightforward:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"processing fold # 0\n",
|
|
"processing fold # 1\n",
|
|
"processing fold # 2\n",
|
|
"processing fold # 3\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"k = 4\n",
|
|
"num_val_samples = len(train_data) // k\n",
|
|
"num_epochs = 100\n",
|
|
"all_scores = []\n",
|
|
"for i in range(k):\n",
|
|
" print('processing fold #', i)\n",
|
|
" # Prepare the validation data: data from partition # k\n",
|
|
" val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
|
|
" val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
|
|
"\n",
|
|
" # Prepare the training data: data from all other partitions\n",
|
|
" partial_train_data = np.concatenate(\n",
|
|
" [train_data[:i * num_val_samples],\n",
|
|
" train_data[(i + 1) * num_val_samples:]],\n",
|
|
" axis=0)\n",
|
|
" partial_train_targets = np.concatenate(\n",
|
|
" [train_targets[:i * num_val_samples],\n",
|
|
" train_targets[(i + 1) * num_val_samples:]],\n",
|
|
" axis=0)\n",
|
|
"\n",
|
|
" # Build the Keras model (already compiled)\n",
|
|
" model = build_model()\n",
|
|
" # Train the model (in silent mode, verbose=0)\n",
|
|
" model.fit(partial_train_data, partial_train_targets,\n",
|
|
" epochs=num_epochs, batch_size=1, verbose=0)\n",
|
|
" # Evaluate the model on the validation data\n",
|
|
" val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
|
|
" all_scores.append(val_mae)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[2.0750808349930412, 2.117215852926273, 2.9140411863232605, 2.4288365227161068]"
|
|
]
|
|
},
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"all_scores"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"2.3837935992396706"
|
|
]
|
|
},
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"np.mean(all_scores)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"As you can notice, the different runs do indeed show rather different validation scores, from 2.1 to 2.9. Their average (2.4) is a much more \n",
|
|
"reliable metric than any single of these scores -- that's the entire point of K-fold cross-validation. In this case, we are off by \\$2,400 on \n",
|
|
"average, which is still significant considering that the prices range from \\$10,000 to \\$50,000. \n",
|
|
"\n",
|
|
"Let's try training the network for a bit longer: 500 epochs. To keep a record of how well the model did at each epoch, we will modify our training loop \n",
|
|
"to save the per-epoch validation score log:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 21,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from keras import backend as K\n",
|
|
"\n",
|
|
"# Some memory clean-up\n",
|
|
"K.clear_session()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 22,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"processing fold # 0\n",
|
|
"processing fold # 1\n",
|
|
"processing fold # 2\n",
|
|
"processing fold # 3\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"num_epochs = 500\n",
|
|
"all_mae_histories = []\n",
|
|
"for i in range(k):\n",
|
|
" print('processing fold #', i)\n",
|
|
" # Prepare the validation data: data from partition # k\n",
|
|
" val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
|
|
" val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
|
|
"\n",
|
|
" # Prepare the training data: data from all other partitions\n",
|
|
" partial_train_data = np.concatenate(\n",
|
|
" [train_data[:i * num_val_samples],\n",
|
|
" train_data[(i + 1) * num_val_samples:]],\n",
|
|
" axis=0)\n",
|
|
" partial_train_targets = np.concatenate(\n",
|
|
" [train_targets[:i * num_val_samples],\n",
|
|
" train_targets[(i + 1) * num_val_samples:]],\n",
|
|
" axis=0)\n",
|
|
"\n",
|
|
" # Build the Keras model (already compiled)\n",
|
|
" model = build_model()\n",
|
|
" # Train the model (in silent mode, verbose=0)\n",
|
|
" history = model.fit(partial_train_data, partial_train_targets,\n",
|
|
" validation_data=(val_data, val_targets),\n",
|
|
" epochs=num_epochs, batch_size=1, verbose=0)\n",
|
|
" mae_history = history.history['val_mean_absolute_error']\n",
|
|
" all_mae_histories.append(mae_history)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can then compute the average of the per-epoch MAE scores for all folds:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 23,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"average_mae_history = [\n",
|
|
" np.mean([x[i] for x in all_mae_histories]) for i in range(num_epochs)]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's plot this:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 26,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVOXVwPHfmdnZvstSll4WBAGRJigW7ESxG0ussURf\nooktiTGa15hoYkk0tpjEWGKMLSrR14INu1hAiiBKlV53gS3ssn2e949b9s7MnZmlzC4w5/v57Gdn\n7r0z89wt99ynnUeMMSillFIAgfYugFJKqd2HBgWllFIuDQpKKaVcGhSUUkq5NCgopZRyaVBQSinl\n0qCglFLKpUFBKaWUS4OCUkopV0Z7F2B7denSxZSUlLR3MZRSao8ya9asTcaY4mTH7XFBoaSkhJkz\nZ7Z3MZRSao8iIitbc5w2HymllHJpUFBKKeXSoKCUUsqlQUEppZRLg4JSSilXyoOCiARFZI6IvO6z\n7xIRKRORr+yvy1NdHqWUUvG1xZDUa4EFQGGc/c8bY65qg3IopZRKIqU1BRHpDZwEPJbKz2mNRRu2\ncu87i9hUXd/eRVFKqd1WqpuP7gduAMIJjjlTROaJyGQR6ZOqgiwtrebB95eypaYhVR+hlFJ7vJQF\nBRE5GSg1xsxKcNhrQIkxZgQwFXgyzntNEpGZIjKzrKxsB8tjfQ8bs0OvV0qpdJDKmsJhwKkisgL4\nD3CMiDztPcAYs9kY47TnPAaM8XsjY8wjxpixxpixxcVJU3f4CojzXjv0cqWUSgspCwrGmJuMMb2N\nMSXAucD7xpgLvceISA/P01OxOqRTQuyqgtYUlFIqvjZPiCcitwEzjTGvAteIyKlAE7AFuCRln2t/\n15iglFLxtUlQMMZ8CHxoP77Fs/0m4Ka2KEPAriloUFBKqfjSZkazdjQrpVRyaRMU3JpCO5dDKaV2\nZ2kTFNCaglJKJZU2QaGlT0GDglJKxZNGQcH6rjFBKaXiS5ugIDjzFNq5IEoptRtLm6DQUlPQqKCU\nUvGkTVBo6Whu32IopdTuLG2CQsuQVI0KSikVT/oFBY0JSikVV9oEBZ3RrJRSyaVNUNAhqUoplVza\nBAXQ1NlKKZVM2gQFt6bQvsVQSqndWtoEBdE0F0oplVTaBAXtU1BKqeTSKChomgullEombYKCQzua\nlVIqvrQJCjp5TSmlkkuboCCaEE8ppZJKm6Cgy3EqpVRyaRMUNM2FUkollzZBQYekKqVUcmkTFEQ0\nzYVSSiWTPkHB/q4xQSml4kuboKCL7CilVHJpExTcjuZw+5ZDKaV2Z2kTFHRIqlJKJZc2QUGHpCql\nVHJpFBQ0dbZSSiWTNkFB5ykopVRyaRMUBE2drZRSyaRNUGhZjlOjglJKxZM2QQG3o7l9i6GUUruz\ntAkKAdFOBaWUSiblQUFEgiIyR0Re99mXJSLPi8hSEZkuIiWpKocux6mUUsm1RU3hWmBBnH2XAeXG\nmIHAfcAfU1UIJ/eRzlNQSqn4UhoURKQ3cBLwWJxDTgOetB9PBo4VZ0LBLqbLcSqlVHKprincD9wA\nxMs41AtYDWCMaQIqgc4pKYnOaFZKqaRSFhRE5GSg1Bgzaxe81yQRmSkiM8vKynboPQIpqX8opdTe\nJZU1hcOAU0VkBfAf4BgReTrqmLVAHwARyQA6AJuj38gY84gxZqwxZmxxcfEOFUYX2VFKqeRSFhSM\nMTcZY3obY0qAc4H3jTEXRh32KnCx/fgs+5iUXLU1zYVSSiWX0dYfKCK3ATONMa8CjwNPichSYAtW\n8EgJHZKqlFLJtUlQMMZ8CHxoP77Fs70OOLstyuDQ5iOllIov/WY0K6WUiittgkLLcpxaU1BKqXjS\nJijocpxKKZVcGgUF67v2KSilVHxpExRERx8ppVRSaRMUwO5X0JqCUkrFlV5BAa0pKKVUImkVFAIi\nuhynUkolkFZBQURrCkoplUiaBQXRLgWllEogrYJCQCBF+faUUmqvkFZBQRCdp6CUUgmkVVCwagrt\nXQqllNp9xQ0KInKD5/HZUfvuSGWhUkVEtKNZKaUSSFRT8K5tcFPUvokpKEvKiaBDUpVSKoFEQUHi\nPPZ7vkfQCc1KKZVYoqBg4jz2e75HCARERx8ppVQCiVZeGykiVVg32Dn2Y+zn2SkvWQoEtE9BKaUS\nihsUjDHBtixIW7ByH2lUUEqpeLZrSKqI5InIhSIyJVUFSiUR2TPbvZRSqo0kDQoikiki3xeRF4H1\nwLHAwykvWQqIzmhWSqmE4jYfichxwHnAccAHwL+BA40xl7ZR2XY5nbymlFKJJaopvAUMAMYbYy40\nxrwGhNumWKmhaS6UUiqxRKOPDsCawPauiCwD/gPs0Z3PAU2drZRSCcWtKRhjvjLG3GiM2Qf4LTAK\nCInImyIyqc1KuAtp6myllEqsVaOPjDGfGWOuBnoD9wEHp7RUKaIdzUoplViijuYD4uzaBDyUmuKk\nVkCHpCqlVEKJ+hRmAvOxggBE5jsywDGpKlSqWMtxalhQSql4EgWFnwNnAbVYncwvG2Oq26RUKRLQ\nPgWllEooUUfz/caY8cDVQB/gPRF5QURGtVnpdjGtKSilVGJJO5qNMcuAV4B3gIOAfVNdqFTR1NlK\nKZVYoo7mAVjzFE4DVmM1Id1hjKlto7LtclZHs0YFpZSKJ1GfwlJgHlYtoQroC1wpYvU3G2PuTXnp\ndjERCO/Rc7KVUiq1EgWF22hZTCe/DcqSclpTUEqpxBKtp/C7nXljEckGPgay7M+ZbIz5bdQxlwB3\nA2vtTQ8ZYx7bmc9NRtNcKKVUfIlqCjurHjjGGFMtIiFgmoi8aYz5Iuq4540xV6WwHC4dkqqUUoml\nLCgYK5+EM68hZH+16yU5ENA0F0oplch2rby2vUQkKCJfAaXAVGPMdJ/DzhSReSIyWUT6pLQ8mjpb\nKaUSSlpTEJEs4EygxHu8Mea2ZK81xjQDo0SkCHhZRPY3xsz3HPIa8Jwxpl5Efgw8iU/6DDsr6ySA\nvn37JvvYuALSzlUVpZTazbWmpvAK1lyFJqDG89VqxpgKrNXbJkZt32yMqbefPgaMifP6R4wxY40x\nY4uLi7fnoyOJaEezUkol0Jo+hd7GmInJD4skIsVAozGmQkRygO8Bf4w6pocxZr399FRgwfZ+zvYI\naOpspZRKqDVB4TMRGW6M+Xo737sH8KSIBLFqJC8YY14XkduAmcaYV4FrRORUrFrIFuCS7fyM7aJp\nLpRSKrHWBIXxwCUishxrmKl1bTVmRKIXGWPmAaN9tt/ieXwTcNN2lXgn6OQ1pZRKrDVB4YSUl6KN\nBEQ0zYVSSiXQmiypK4Ei4BT7q8jetufR1NlKKZVQ0qAgItcCzwBd7a+nReTqVBcsFXRIqlJKJdaa\n5qPLgHHGmBoAEfkj8Dnwl1QWLBUEwRhtP1JKqXhaM09BgGbP82Yi12veY1hpLtq7FEoptftqTU3h\nCWC6iLxsPz8deDx1RUqdgAjNGhWUUiqupEHBGHOviHyINTQV4FJjzJyUlipFggEhrFOalVIqrkTL\ncRYaY6pEpBOwwv5y9nUyxmxJffF2rVAwQEOzBgWllIonUU3hWeBkYBaRg3bEfj4gheVKicxggIam\n5uQHKqVUmkq08trJ9vf+bVec1MrMCNCoNQWllIqrNfMU3mvNtj1BKCg0NuuQVKWUiidRn0I2kAt0\nEZGOtAxDLQR6tUHZdrlQMEBDkwYFpZSKJ1Gfwo+B64CeWP0KTlCoAh5KcblSIjMjQIPWFJRSKq5E\nfQoPAA+IyNXGmD1u9rKfzGBAm4+UUiqB1sxT+IuI7A/sB2R7tv87lQVLBW0+UkqpxFqzRvNvgaOw\ngsIbWKm0pwF7XFDIzAgQNtAcNgQDe2SmDqWUSqnW5D46CzgW2GCMuRQYCXRIaalSJBS0TldrC0op\n5a81QaHWWKlFm0SkECgF+qS2WKmRmWEHBe1XUEopX61JiDdTRIqAR7FGIVVjpc7e42QGrSYj7WxW\nSil/relo/on98GEReQsotNdf3uNo85FSSiWWaPLaAYn2GWNmp6ZIqeM0H2lNQSml/CWqKfzZ/p4N\njAXmYk1gGwHMBA5JbdF2PaemoEFBKaX8xe1oNsYcbYw5GlgPHGCMGWuMGQOMBta2VQF3JSco1Gvz\nkVJK+WrN6KPBxpivnSfGmPnA0NQVKXWy3OYjzZSqlFJ+WjP6aJ6IPAY8bT+/ANijO5q1+Ugppfy1\nJihcClwJXGs//xj4e8pKlELuPAVtPlJKKV+tGZJaB9xnf+3RQvY8BZ28ppRS/hINSX3BGPMDEfma\nyOU4ATDGjEhpyVLAbT7SmoJSSvlKVFNwmotObouCtIUsTXOhlFIJJVpPYb39fWXbFSe1tKNZKaUS\nS9R8tBWfZiOsCWzGGFOYslKlSMgZktqkQ1KVUspPoppCQVsWpC1kOpPXtKaglFK+WjMkFQAR6Urk\nymurUlKiFMrUjmallEoo6YxmETlVRJYAy4GPgBXAmykuV0qEMjR1tlJKJdKaNBe/Bw4GFhtj+mOt\nwvZFSkuVIpmaOlsppRJqTVBoNMZsBgIiEjDGfICVNTUhEckWkRkiMldEvhGRW32OyRKR50VkqYhM\nF5GS7T6D7RAMCCJaU1BKqXha06dQISL5WOktnhGRUqCmFa+rB44xxlSLSAiYJiJvGmO8tYzLgHJj\nzEARORf4I3DOdp5Dq4kImcEADZoQTymlfLWmpnAaUAv8DHgL+A44JdmLjKXafhqyv6KvxqcBT9qP\nJwPHioi0okw7LDMY0OYjpZSKI25QEJG/ishhxpgaY0yzMabJGPOkMeZBuzkpKREJishXQCkw1Rgz\nPeqQXsBqAGNME1AJdN6xU2mdUEZAm4+UUiqORDWFxcA9IrJCRP4kIqO3983tYDIK6A0cJCL770gh\nRWSSiMwUkZllZWU78hYurSkopVR8iVZee8AYcwhwJLAZ+KeILBSR34rIvtvzIcaYCuADYGLUrrVA\nHwARyQA62J8V/fpH7JXfxhYXF2/PR8cIZYjWFJRSKo6kfQrGmJXGmD8aY0YD5wGnAwuSvU5EikWk\nyH6cA3wPWBh12KvAxfbjs4D3jTEp7QUOBQOaEE8ppeJozeS1DBE5RUSewZq0tgg4oxXv3QP4QETm\nAV9i9Sm8LiK3icip9jGPA51FZCnwc+DGHTqL7aDNR0opFV+ihHjfw6oZnAjMAP4DTDLGtGY4KsaY\neUBMP4Qx5hbP4zrg7O0s807J1I5mpZSKK9E8hZuAZ4FfGGPK26g8KRcKBmjUeQpKKeUrUZbUY9qy\nIG1Fm4+UUiq+1kxe26uEMrSjWSml4km7oJAZ1D4FtWfa1tDElyu2xGw/4PdT+fuH37VDidLTPr9+\ng4v/OaO9i5Ey6RcUMkSbj1RCxhg+WVJGikdHb7cfPzWLsx/+nKq6RndbOGzYUtPAH9+KHu2tdsYH\ni0q54w3/kffNYcNHi3d8Em3p1jr++sHS3e7vy5F2QSGkNQWVxIuz1vDDx2fw8py17V2UCJ8s2QRA\neU2Du62moSnmuFkry5lmH7u7m7VyCxXbGpIf2MYufeJLHvl4WUre+xcvzOXutxcxf21VSt5/Z6Vd\nUMjU0UcqibKt9QAs3lgdsX1TdT1DfvMms1a272C88m0tNYXq+sigcO/UxZz598+48PHoNGOp19gc\n5t6pi9nqqckk0hw2nPn3z7ngsZ0v61erK/h06Y4HwlWbt7m/dy9vq8L8tZV8thOf4dhcbQXBcCtr\nCv/+fAWvfNV2NyitXo5zbxHKCFCvzUcqgbzMIAA1URfcGcu3UNcY5h8ffccjF7UsKbKsrJqtdU2M\n7FOUkvK8OHM1xQVZ7nPvnbW3jMYYHnxvifu8rrGZ7FAwJWXy8+pX63jwvSVU1zVxyyn7+R5jjOGV\nr9Zx6MDObtm+Wbfzd8yn//VTAFbcddIOvf6Iuz9ABJbfGfn6rXWNdM63fvYn/2XazhXS5rRUbGto\nTnicMYZfvzyf52ZYKx+fNqrXLvn8ZNK0pqBBQVkmz1rDizNXR2zLiRMUMgJWVvfmcOQd3jF//ojT\n7ItStF9NnscD7y6J2378wszVrK+sjdlujKG0qg6AX06exyVPfOnu+8kzs6m0awtb61rKuNluVhrZ\nuwMA73y7kZIbp7BgffyL7opNNVTWNrKuopYz//6Z+5nRahuaY847mtOUtaGqlvom/wveszNWcd3z\nX/HYJ8uprott+mpPfr+iMX94l7UVsb8fx3MzVvHW/PWtev+qukZ++uxsNtg/4+haXrTybY1uQGhL\n6RcUdEbzHm3OqnJKbpzC0tKtu+T9rn9xLr+cPC+i6cBZhCneP21jnIvj5urI5ofmsOH5mau5793F\nfLEsdtTQlpoGbpg8j8ufnBmz75npqzjojvf4+Qtfxezb1tDMpKdmMvLWd9jouYivsy9eRw3uCsA1\nz80B4I2v/S9a05Zs4qh7PuSix6czZ1UFs1aWc/sbCzj+vo8j+i0Aht7yFtf8x3q/Gcu3MG3JJuav\nrYw4xmlqeePrDfz0mdnudu//20eLrA7apmYTE3SdY2997RtWb9nmW+ZkknXehsMm5pjo59EDUeas\nit9ceNNLX3PF07Pj7vd65at1TJm33g3kyZrZSrf6B+hUS7ugEAoK2xqamb6sVUtCqN3MS7OtttV4\nHanLN9Vw/qNftLpd23Hg7e+6j+sbrbvcd77d6Nup2xTnpuJrz0UyHDb87YOl7vOX56yJOX5LjRVE\nNlbFtmW/Oncd0HK+0aYv30JlbWPEZy5cbwXKI/aNzCTcFCeIOYF17ppK9wL0ylfrWLRxK3e/syji\nXACmzLOCyw/+8TkXPj49pjnFO//n3QWlALy3YCOD/vdNlmy0Pquy1vq9bNxax5aa2A7maUs38cSn\nK7jkiRkc++cPtzs41DXGv+EzxjDg129wyyvfRGz3NuM0NofZFBXc4zXzJLpoG2P46wdL+WzpJhZt\n2MqnSzeRnRF5uU1WUyitir3JaAtpFxTG9OsIwAszY/9J1e7P+QfNzfTvDrt36mI++24z79kXpe3h\n3NHWNbZcBB6w2+jrGpt58+sNgHWX62dpaUvH9JzV5fx56mL3+aINkTWbusZmNlRa//ShoNUsNX3Z\nZvdCGX1himdLTUvwu+G/8wDo3yWPe84e6W5fsnErb369PqaGvL6y5aIW3a7/7PRVzFtTAcA2z88j\nXrMQQGNT7M/FGcH15QrrbtsJClPmreecR76IOX6NHQS+K6vhu7IaXrCb9owxTPr3TD72GQo6eVbL\n/3JFbWSgqWtsprymgbrGZjbZHbxPfbEy4piK2pafYenWelZsikzvtrGyjm0+o7wOuv0997FT26iu\nb2L5phrWVdZx99uLOP+x6Rx//8dc8Nj0mJrVVp/ms6e+WMldby5k9qrymDkpbTVKK+06mo8Z0o19\nu+X7/pKVv8raRjrkhNq7GEDLBTuU0bJqazhsKN/WQOf8LIrsckb/A93z9iIGFOdxxgG94753TX0T\nRbmZEXebzt3Zra99yzvfbgSgMdyy33uhrfL8k6/e0tIOHQqKezF03nPIb95yn2cEhXDYcM4jXzCo\naz5Tf34km3xGwvhZUx55J50ZDNAxNxTx+3p3QSnvLijlkkNL+N2pw9zt3qDwhafmPKhrPktKq3nn\nm40sK6vhkH1aFkN0LqzecwkGBGNMxPwJh/Pz2WD3m3h/Dl51jc089flKno1qQ5++3LowVtc38c63\nG+nVMSeiJmSM4foX57rPK7Y10qNDjvv8/Ee/YPYqK7gdNrDlPGrqm8jLsi5/lZ7RXOsqapmzuiKi\nDOsqa31rNV4V2xrpmJfJRY9PZ/aqCt689vCYY7zBC6ygsHBDFaVV9Tzy8TJ+NXEIb369nq9WV/Dw\nR7GTEbfUNLid3qmUdjUFsO4ya5L0/CvLtCWbGHnrO7tkKN6u4ATz2oaWi/FTX6xkzB/eZfmmGjrm\nWhdD77BNgIc+WMrPX5hLtIBnRXDnzq2usZlQUOiQE6J8WwPPzVjFt+ta7vLmrKpwR/mUe4JPleeC\n5+2c7N0xlxWbt3Hl07MouXEK+/z6jYgyhAIBFto1iSWl1fYF1v+m5eELx0Q8X1se2QnaITeEiFCU\nGxvEoy926ytr6dc5F4A1nvcZ3qsDvYpyeOiDpVz3/FfMWN5yxxo9bHPYb9+isTnME5+u4PFpy2M+\nc+VmK2hNXVDKU5+viBsU5q+t5PY3FrA86i595eYalpZuZYldC9tQGdnstDnqYn3CA5/w4aJS987d\nCQgAny5tCXzDf/c2Vz83hwsfm84qTxPVh4tKmbOqnE55me6252asjtsvkx2yLqFO57HzeX41vehr\nTmVtIxPv/4SL/jmDaUs3cdmTX7K5uiFuc9X8dZW+23e1tAwKeVlB304uFWu23cn26Xf+QeGjxWU7\n3Cm4I5x/GG9Nb67dzDFt6SYC9lXeW1OIdyFyLhwDu+YDLW28dU3NFGSH6Nspl9fnreeml75m7prI\nf8iH3rf6C7wXKG9zgPcOvndH6871zfkbfMuxbFMNJz74ifvcr1nFMWFoV4b1LPR8TmRQcGpK0TW7\nwd0KWFdRy1vz17tlW19Zx4jeRYgdGHsVWeXsWZTDCHsEE8AHC1ua4qJHJ9U1Wm3wU3wumo3NYb4r\nsy7mC9ZX8ZtXvvG94GVmBLhhstX0VZCVQY49VDU/K4NN1Q1MuPdjzvjbZwDMXFnOAb+fym/+bz7G\nmJigCHDJE1/yr89WxGz3Cht4be46pi3dxD8+tu7Ku+Rn8eb8DSwtrebAko4Rxz//5eqY9zh+WDee\n+5+DAfg66u8julzeIDNhaDd6FeXENCeVbq1n0cb4AyjueXtx3H27UloGhdzMDGatLGf0be+0d1F2\ne84/qPfO3Ovif85g4v0f++4Lhw0/fWY2n8UJKDuittEJCi0Xl/6d8wD4dl0ltfb2DZ6Llzdoedv2\n65vChA0U21Xy8m0NNDSFqWsMk50RoKPnHzna0B4FQGRQqKprZG1FLbdP+TYiADhBobW8d+bRMoIB\nOua2lCs6uaPTJBIdFA4e0ImyrfVc8fRsLnniS8Jhw8aqOnp3zKGzfZ6H7NOZs8f05rhh3bjxhCE8\ncO4ouhdmu81mAGU+d8BlW+vdGprXlyu20NhsOHhAp4Tn++ezR7LMriH84rh9ybWHBO/XszCmc9Wp\nqTz1xUqe/mJl3OGi73yz0Xe7nzn23f2YfkVsqKxjbUUtA4qtG4WBXfMZ3beI78pil5HpkBNieK8O\nDO/VgT+9vTDibyE6WPcsygZgSPcCHrt4LEW5oYhBAsmceUBv1lbUJu2c3hXSMig4k5OimxhUrGz7\nZ1XbGHuH57QXx2uKW765hilfr+dnz8cOq4w2a2U5r89bF7Ht589/xS9fjGzyqbB/Z/dOXczCDVbn\nqDO6ZvbKCjdYeJs5vP+gx9//MY3NYd79dqPbBu5MDDv/0el8776PmDxrDdmhIJlB/3+PXkU57jmX\n2x29HXJCTP12I4fd9T6PfrLcLSdYzUe7UqJgFbRrStFBYVjPljv/0qo6NtXU09hs6Nkh2w0kJZ1z\nufvskYzoXUS/znmcNqoXJ4/oEXEhuvW1b2M+89Olm90Lq9f5j1ozlScdMSBi++i+kZP8xnmCRn52\nyJ1cur+nzH4+WbLJvSO/YeLgiODz+bLNEX0NAP0657J/r8KIbd4aUUnnPLY1NNPYbOjdMYcp14xn\n8hWHMLK3Vd7C7Mgu2I65mWQEA/zprBFsrmngqc9bOrCj+3qcQO4060WPEAMY5TP5ccLQbjx7+TiO\nHmId3xa18vQMCllp17++w7LsC2OdT1DwGz3h5VSPnWaJeCbe/zFn/v0zrnp2TsT2l+as5cVZa5hw\n70fuNu8F6kp7fHidPSJm0cat7gggb03CGXq5T7FVo7jwselc/u+Z7uiRrp7Zwk4beFPYxB3tMWFo\nV5aWVvPYJ8vctuOSzvEv/NtbU/AzuFuB+3O8cFxfzh7T0mGe5Rnq6HSReGcyXzdhED09v4Ouhdms\nr7BqUt075Li/2752jcsrehatXzLJP761MKZt3+uwgV348n8nuM+vOXYQy+880X1elNMS5AqyM9wR\nTtEXcK8DSzqyYEMVyzbVUJidwZVH7sP1xw2OOCa6Y/ejXx7N8F6RgeZoe04HQPcO2e7j3h1zGdaz\nA0W5mZw+2voZbGtodicwHrFvMT85aiAAQ3sU0jkviw1VLTcf0TUYp/moc571t3bNMYOYdMQA/nTm\nCG7//v785uT93GO65Lf8PLoWZnHowC70sW8sNCikiDcoGGNiOrdUC6d5otanNuA3FyAcNvzihbl8\nvaaSeXY7a7fC7IhjjDERzQILNySeiLbU7nyF2LQOAPWe0UKf26NonKAQtieQHVTSiSnXHE5GQNwR\nLY6uhbEjOsq3tXT4RV9InBEgf5iywG277tMpflDoEjVixNu5fecZwyP2xQugL/3kUD690Vr3atyA\nztx99ki3b+GYIS0XNpHY1143YV9G9Gk5h9zMoDvyqEeHbPd328/nHAZ1y4/Zds7YPqy46ySevmxc\nxPb8qJut356yH89POpisjCDFBVn84fT9GdK9gP16FCIivHXd4fzjh2PI9AS1gqwMt5YzLEFN4eAB\nnVm9pZbnZqxi/14d4nauRwtE/YAOG9gFsH4m3td7A/moPkX8+IgBPHzhGPfC/dD5o+ngOb5TXoiy\nrZHNR85QY2iZLT3crpnkZAb59YlD+cGBfbhgXD8uG9/f/d2fd1BftznOqa06f1+rffpQdrW0DApO\nmyXA299s4Oh7PnQn5rS15rD/zM7dhXNn+NY3GyISjoXDxremsLmmgf/OXsOpf53mVu2du/v1lbXU\nNTbzx7cWsc+v3/CdXQpWraQyqmmvrjFMU3M4Im+VczGpa2ymW2FWxN260xG9cWsdq7fUcsrIHmSH\ngvTyuWv35hVybK1r4p6zR3LugX24+NASd/srPz0M72VluX2n2tmnScfZFn2dFvvCdOcZwzk96k7c\n6biMluOTw+j5Hx/Ct7cdH9FUJJ6L3s0nDeVRO0dTYXaIab86ml5FOWypaXBTa/TokE2h/fp+PrWd\n6NxJxQVYdbOVAAAcZUlEQVRZ3HzyUADGD+oS0ZfgtJsDXDCuLxcdUsK4AS3DQC88uB9vXXeEe5Mw\npHshxw/rHvH++dkZ/GfSwdxy8n4MKM7j8EFdOG1Uz6gyBThpRA/3uVPTK2zFsOlfHDc4YmjqvnbQ\nO21Ur4ifY3RwvunEoUzYrxvP/s/BXH/cvhRkxTYlLd/UMk9lfWUdBdnW+x07pCsrNls3nt7mqmjX\nHz+Yy8f356dHD+THR+4DtKRW6ZgbIj8ro01mOadlUAh6/nGc9APOCJa29ssX5zLst2+3y2e3hvci\n/LtXrZmgHy8uY8Cv32CmZ3KNe9duV/2NaemEraptJBw2HHLn+0x6ahZP2nfXR93zYcRYebCC0APv\nLeHMhz+L2F5d3xQxiQqsNOjQkvjN207r3OU7ZSguyI54jZf3Tv4v5412H+/Xs5C7zhzh3omfOLw7\nI/sUcfFhJdx80lAmDO0GWE0DzgXgxOEtF7knLj2QFXed5I5ucrQ08QTIyQzyW0/yuPzsDLdfwLHi\nrpPcUVVe+VkZ5GZm8OuThnKsXUbvUZcfPoDv7dfNfd67Yy4T9+/OmvJabn3tWzIzAnTKy+Rflx7E\nzScNpSg3fl+F45JDS9xzBSLmBHhvEq6dMCjmPFqjIDvEwK4F/Gh8f0LBAE9dNi4mcHbMzWRI90Le\n/8WRQEsTV2vm0nTKy+Sv5x8AWDWZotxMPrj+KG49dZj7+uKCrLiJBAd2zeeqYwZFBF+nTNGd0flZ\nGSy740Qeu3gsvzhuMH065TC6T+SoJq8OOSFuPnk/skNBt3bqBFURYebNE7jphKFJz3FnpWVQ8E44\nau+79JfsGZ/hNprCnkjZ1npKbpzCB4tahiB6Z7A6FzdnotPvPJ2OTke0t+9hnX03WlXXxEb7Dufj\nxWX0sO8oV23ZFvFZYDVTLS2tjmk73dbQxLb6yKCQ4QaFMNkZQUo8beLbGprdBWigpU3XL12xd3a0\n9yLq6JSXyWtXjefOM0YA1l335YcP4FB7UlenvEz6ds6lICuDO88YwZvXHs7JI3owpLvVvNO1MJtv\nbj3efT/nepKdYV14Lj2sv7svLyvI29cdzn3ntMxITqYwO8SD541mWM9C/vekxBcNb1NNjw7ZiAgD\nu+Zz+eED4r7G2yzkrWUD9C9u+Zl39rSF+9VsWiO6CQpgmKdvIScUdC/eA4rz+e6OEznaDohZGf6f\n2SU/i1s9k/aKcjNZcvsJ7s+9f5c8MjMC7vvuSB+Q0/l/+KAu7s+oY26IQEAQEY7ct5hPbjjGTbaY\nzGEDuzD918dG/D22VcbbtOxxrfcEBedOtd6nI7UtNYbDZAXaLs2xHyetwZOfrXA74Lwdi85FOLpd\nFqw78iuenh1xt+aM+qmsbXQ7cAG6F2azzL6rim6CqmloonRrfUx68/lrq4i+8Zy7uoKVm2uoa2om\nOxSIaRp6ZsYqVtnVdicoXH/cYK5+bk5En0YwINx/zigGdy+I+4833Kfaf1D/Tu57n3VAb44f1p0O\nOdZs4ofsu1FH9MUU/P/JszKCDOxawMCuBeSEgr6jvvzkZWUw5ZrYWbTRvLXkHh2yExzZYurPj+Ds\nhz9nTXltzHlMHNadKfPWc8rIntx0whAOvet9YMeDQkF27CWpa0E2t39/f5aV1fDBwtKIIbnxaiMX\nHtyXp79YxSEDOvPcpNgmOb8aY6EbFLZ/tJjT2T+mX0dOGdGTdZW1bk1yR0X3xbWVtAwKeG4Wl9mT\na0pbmVYgVRqbDe09KMqpQX24qIz3F27kmCHdqG8Kkx0KMKBLvjsap3xbA1lR61JUbGtk3pqKmPZz\nsJqPVnmCgjdArI8apbGtoYkyn/TNP322JRNl30657izUI+/+kIMHdCIrFIxpB/7N/813HztB4cTh\nPThxeA9Kbpzi7ivMznBHmAA8etHYpCOmwBp10iEnRNfCbAIBSdh84TQ3jOnX0Z7oZMgKJa6oT9y/\nR8L9O+LKo/bhkyVlzF1T6XtX7qdHhxwOLOnEmvK15GdFnuMpI3sypHsBg7oVRGzPiDOcN5msDP/X\nXTCuH2D9zBOV+9nLx1FckMWgbgX8/rT9t+uzO+SECAj02YGagjMKrVdRDmeP7bPdr9+dpGXz0ZVH\n7eMOd1tn1xQ2xskj31biZd5sS96cPz/6l5XOuaEpTG5mBt0Ks9hS04Axhoraxpi78pr6JrY1NMfM\n/ejdMYemsGHBhpaEa9bkIKvZIbpPobq+2XeClNddZwxnwtCWETeVtU1kh4Jxq/0i8dub/3vloe5E\nJcf39uvGfj3jD4d0BAPCc/9zMNdNGJT0WIBZN0/gmctbRux4awpTf3YEj/xwjN/Ldqm8rAx+NN5q\nNtmeJYJvPGEI54/ry4T9usbsiw4IO+LhC8dw1pjeMW310c4a05uJ+3ePu//QgV3c8ohI0vfzysoI\n8uhFY7nksJJWv8bh5Ica3Td+n8GeIi1rCkW5mfzlvAM4+p4P3W3JLkSpFj0ztT34jcuvb2omMxiw\nOuQWlXHZkzNpaApTlBOiX+dc966/qq7Jdwz7kO4FrCmvjcn42K3AakKKHs+9trw26XKpuVkZ7nhv\nsFIo9OnYzb3w52YGI+YpFGTFdt46Ru/kammtCR6O6GRm3rviQd1i77ZTxWna8eu8jqdbYTZ3fH94\n0uMmX3GIOxR5e0zcv3vCi31bOXYHm3zOP6gvE4d1b5OEdamWljUFiO3QSpSHvS3sDutG+83wbmgK\nkxUKuGOu319YSvm2BjrmZvLqVeP52wVW23l0orQLD+7LwK75bmfe/LVVDOnectFzJgpFp2x2hu4l\nkpsZ5OaTh/LAuaPcbdmhICLCgtsm8tD5oyOO97v4OW3X23Nh3GWcjuY2XCrT64h9i/n+6F7ccrL/\nkpk7Y2xJJ7cmkk5EZK8ICJCmNQUgJldLu3c0t8O60VV1jUxbsokOOSE+XlIWM7oHrCGpmcEAKzz9\nABXbGhncvYAOOSEOsKvL0UHhyqMG8ofTczDGSqWwrrKOvp1y2VBVR8W2xohZxMft141LDi3h/Mem\nc/fbi0gmNzNIQXaIU0b05IbJ89x+D7AmBXnbvS85tMQdneI19WdHJlxmsS20V1DIDgW575xRyQ9U\naSltg4K3I6ykc25EArX20BRu+6Dwyxfn8vY3G+lVlBP3AunUFM46oDczlm8hJxSkwq4pgDWuHqCs\nOvLn5zRRiAgj+xSxrnIDXQuz6JSXScW2xoiJRueP6xvTrp9IXmbLXX6/zrks3lgdcYF1Rsj07ZQb\nsX6AV/cO2RFpDdqSO08hTqeqUu0prf8qnTHb/bvkUd8UTrq+ayo1+KxaVbmtMaVZEZ2FYLLjjIIx\nxrg1hR8c2IfLx/entrGZmoZm904/174YR9cUvEMSnU7l/KwQA+2Lv3fkUqe8THKzkt81O7M7vWO9\n+9lzE7zJ65zhpt4cMrsTp+8zq51qCkolktZBwcmZPqA4H2Pat10/eqlEgJG3vcPBd7znc3TrhMOG\nxQnyszudr/EWdHE6j51JQd7snPvanaKBgLh57728nahOp3BlbQM3TBwCwGGe1bw65WVSkJXhm2J5\n8hWHAHDG6F5MueZwbpg4OKJWcILdOekdzz+sZyHnj+vLA+dG9i3sLu45eyQDuuTt8Fh+pVIpbZuP\nAP52/hhmryp3M2vWNzVHzPj8Zl0l5TWNXPj4dKb+7AgGdStI2dKUfkEBki/uncijnyzjzjcX8tpV\n430nXzmdrPHWA67Y1kB9U7Ob/sCbMGxfT6dxXlaQWSvL3efZoUBEB+4pI3vy7IxV/Oiw/gzsms/y\nO0+MGCrYKS8TEeHJHx3E4Jtblqm03ivIsjtORMRqihrcPXKEzvdH90IEDurfEmQygoFWjZRpLyeP\n6MnJI3omP1CpdpDWNYUOuSGOHtLVnUTkHVK5cEMVJz04jQsft3LCf7S4zF2a8tMULE3p1FKMMXy5\nYkurm7KMMTz43hLfTK8z7Qv16nL/dLtOEsd4H/VdWTX1TWH3rt87k7Snpz1+Y1VkUPFLovbuz4+M\nGD/u5aSZ8KYpcIJzKBhwUwX4ERG+P7p3qyabKaWSS+ug4HAuet52biffvKOqrokZ9lj76Z5FzncV\np6bw2rz1nP3w5/x39tpWvW5TdQP3Tl3Mxf+cEbPPudj75fsB/3QVl4/vz8s/OZS8zCBTvy21O5qt\ni7XTlt+rKCfuRTojIDvVLHLBuL788vjB/PlsK/dPe3UGK5WuUtZ8JCJ9gH8D3bASSzxijHkg6pij\ngFcAZ8Xvl4wxt6WqTPFk+gSFpqgEdVW1je6dcnOCu/jahmZO+ssn3HXGCDc3Tms0NoepqW/irflW\nCu8lpfH7AuqbmjHGuiN3Ri35rW3gqPCZf1DX2OzbNNWvSx6j+3bkyMHFvLdgI4aWzuTRfYoYP7AL\nt57mP6IHrBnMfsHGz+BuBTFr0t7uafY5ZaQ2sSjV1lJZU2gCfmGM2Q84GPipiPjNlvnEGDPK/mrz\ngAAtzRart2yj/01TmLF8S0zaia11TTgDXBJNPv52fRXLymq4440FMfvCYePmsY/W2Bzmmufm8MbX\n1tq+9Qkm0x37548Y8hur7d2ZdOefZNXaWO5ZFevz7zZTU9/EmX//zHdxG2f+xoSh3SjdWk/Z1nq6\n2XfrRbmZPH35OPaJGj76+tXj3cc/PKSEcw5sXe6XKdeMZ/EfTmjVsUqptpGyoGCMWW+MmW0/3gos\nAHolflX7cJqPPl+2GWPgbx8ujUiTANZEr7CnOeZXk+fxl/eWRBwTDhvmrLLa8f0myt7/7mIOufN9\nHp+2PCYlRGOzcZunIPFSl941h2s9K4xFc0YVbbHTVyzfVMN5j37Bvz9fGTOT2OGkcz7Ks0xhzyRN\nOPt7ViY798A+7gIhyWQEAxEd+0qp9tcmo49EpAQYDUz32X2IiMwF1gHXG2O+aYsyeTkXpspaq5ll\nU3V9TNNKVW2ju/ZCTX0Tz89cDcDVx1rJ0BZt2Mq7Cza6M3L9cu28u8BaO+D3r3/Lqs013OrJ4tjY\nHI7IMOqUJRl3HYOmZpqawxGT8py1BN75ZiPQkop37ur4Cwo55e7kGX7aoxWduAf0LWL2qgodZqnU\nHi7lQUFE8oH/AtcZY6JvT2cD/Ywx1SJyIvB/QEzKSRGZBEwC6Nu37y4vo9N89Oz0VYCVpycvM3J5\nzvJtDe6dd0XUBXvu6gpO++un0WWO+Rxvh+/HSyJHMEUPSa3y9BE0h41vkFmwvooz//6Z/XrD6X/7\nlNevPpzmsOHRT5a56arXVtTyxKcr3Nd9vTY2YdnFh/SjorbRzfbolaymAPDkjw5i5eZt7ZNLSCm1\ny6S07i4iIayA8Iwx5qXo/caYKmNMtf34DSAkIl18jnvEGDPWGDO2uLg4evdO82vCiF7cvbK20e3M\njV4DYJFP23zAHe5p3KYib1BYvqkmYthpQ3Nsx7Yj3kIrJzzwScTz+WurqK5v4lf/ncddby6koTmM\nSGSep5xQ0DelxeDuhTxw7uiI4aRDe1gZQFtTUyjIDkU0Iyml9kwpCwpi3So/Diwwxtwb55ju9nGI\nyEF2eXb9eM8kohf2uMSzULtjY1U9r8+zag/Ra7H6NfU4d/Z/+/A79r35TWrqm2I6g70L+zQ1hyNq\nF5s9ncO1Da1P1nf7lG+ZPGuN+/zus0Yy55bj3LWHTx/tP6Inw+cO/98/OohHfjim1YuxKKX2fKms\nKRwG/BA4RkS+sr9OFJErROQK+5izgPl2n8KDwLmmHRIQRQeFowYnro1EBwG/oBAQwRjj9jFM+Xq9\nO3Pacf+7i93H0c1H3lxC05aWccSfPuDz7zYnndS2pSYy3YQzqeuUkT2ZdfMEjtw3NmMoRK616ygu\nyOK4Ye2f414p1XZSdgtojJkGvqszeo95CHgoVWVoLW/z0d8uOICDB8S2q8djjIkbFLwzfW+YPC/m\nmOdmrHYfJ8q79MmSTazaso1bX/uGV646LGF5OuVF5nT3rkbWOT8rIlUFWE1L0351DHlaG1BKoTOa\ngcj0CicO70F2KMj4gTFdG0Ds4jz1TeGYDKFg9SkkmoDmXU8A8F21zOGsbra1rsl3zQOvvKiF1aNn\nBEcHhe4dcjQgKKVcGhTAdwH1py8fx+mjItvfO+VlcvdZIyK2fbiolHlrYod4Nhvc5qJuhbErMjkr\nfzlenbsubtOQk9doY1Vd0gR53lrL9cftSyhqAfWinJahpqeN6sm/Lj0w4fsppdKL3iLSkos/uq81\nx07Ulp+VQXV9E1kZAboWRt55X/H0bN/3rGtsZklpNUW5Ifp3yYtJGufcnffokM36yjrfhHYOp5+g\nKWxYtcU/uZ3DWSzoX5ceGDEBzeGtKVx0SIk7d0EppUBrCoDV0fzjIwfwyk/HR2x3JmI5qbKzMgK+\nd/0AfTpFDtusb2xmaWk1A4vzfVNtO6uDtXaxcudivqys2nf/2WN6A1ZtYkj3At+AAJEZTPfr0fpF\n55VS6UGDAtZEs5tOGBqz5kCnPOtC3LtjDhOGduOBc0fTtcC6sz58UEufw+MXj3WXiHTUN4X5rrSa\ngV3zKcj2CwrW8dGvi8dZC9kZDnvzSUOZdMQAd//x9iihDZV1bsBJJqeVxyml0ocGhQScyVtrymt5\n7OKxjOxTRGZGgBV3ncT1xw0G4K4zhnPs0G4xSysu3LCVzTUNDOyaT6EdFC4b39/d72QSzcvK4KWf\nHMohSUY8HWrPNF64wZoUPq5/Z3594lB3v7PmcVVdU9KO46uOHsifzhyR8BilVHrSPoUEhvW0ag5+\nM4BH9inikxuOpk+nXKBlEfZBXfNZX9nSIdy/S56bHiMnFORnE/YlLyvIF8usGdN5WUEO6NuRhy8c\nwy8nz2VsSUdWb6nlqS9WRnzeQf07EQoKXyzbQqe8TAZ1szKVvvvzIyjf1siI3h0oyM5ga11TzLyL\naNcfP3hHfyRKqb2c1hQScPoPjh/WzXe/ExCgZf2FH4ztw+i+RQAM6V7A4YOKCdk92AbDtRMGcfnh\nLc0+TvNRh9wQj1w0lklH7MP+vWLb+vcpzneT1J0xupfbNzCwawEHlnQiKyPIzSdZNQe/9ROUUqo1\ntKaQgIgw73fHuemkEym301N375DNOrtmcccZw8nMaFmv2G/NA7/2f29n8FVHD+S/s9eQl5VBjT1H\nwW+9ZYB+na1ZyfHWXFZKqWQ0KCRR6NNJ7Me5O+9ZlO0ug9nfvkg7KY381jzwy37qTKYTsZp6nOYe\np0kq3qihfp2tmsum6gbf/UoplYw2H+0iLTWFHB65aCw3nzSUjnZzT4Hd8ZsbMdIoflqLbHsyXXS4\nOGaINcy0f5fYPEUA3QqyCQaEGyZqn4FSasdoTWEXyQgIjc2GrgVZhIIBBnZtWbLynAP7UrGtkf/x\nDCF1moiiZxyDt6YQGRYeOn80m6sbIhbS8QoEhO/uOHGnz0Uplb40KOwiL//kMGYs3+J7kc/MCLgr\ntDluPXUYvTrmRMx3cDg1heiWpdzMDHI76a9MKZU6eoXZRfbv1WG7FpnpnJ/FTScM9d3n1CL8Vm9T\nSqlU0j6F3ZAbFNq5HEqp9KNBYTfkdjRrVFBKtTENCrshp6M5oFFBKdXGNCjshpw0FdFrLiilVKrp\nVWc3lJeVwQ0TB7uZT5VSqq1oUNhN/eSoge1dBKVUGtLmI6WUUi4NCkoppVwaFJRSSrk0KCillHJp\nUFBKKeXSoKCUUsqlQUEppZRLg4JSSimXGBN/BbDdkYiUASt38OVdgE27sDh7Aj3n9KDnnB525pz7\nGWOKkx20xwWFnSEiM40xY9u7HG1Jzzk96Dmnh7Y4Z20+Ukop5dKgoJRSypVuQeGR9i5AO9BzTg96\nzukh5eecVn0KSimlEku3moJSSqkE0iYoiMhEEVkkIktF5Mb2Ls+uIiL/FJFSEZnv2dZJRKaKyBL7\ne0d7u4jIg/bPYJ6IHNB+Jd9xItJHRD4QkW9F5BsRudbevteet4hki8gMEZlrn/Ot9vb+IjLdPrfn\nRSTT3p5lP19q7y9pz/LvKBEJisgcEXndfr5Xny+AiKwQka9F5CsRmWlva7O/7bQICiISBP4KnADs\nB5wnIvu1b6l2mX8BE6O23Qi8Z4wZBLxnPwfr/AfZX5OAv7dRGXe1JuAXxpj9gIOBn9q/z735vOuB\nY4wxI4FRwEQRORj4I3CfMWYgUA5cZh9/GVBub7/PPm5PdC2wwPN8bz9fx9HGmFGe4adt97dtjNnr\nv4BDgLc9z28Cbmrvcu3C8ysB5nueLwJ62I97AIvsx/8AzvM7bk/+Al4Bvpcu5w3kArOBcVgTmTLs\n7e7fOfA2cIj9OMM+Ttq77Nt5nr3tC+AxwOuA7M3n6znvFUCXqG1t9redFjUFoBew2vN8jb1tb9XN\nGLPefrwB6GY/3ut+DnYzwWhgOnv5edtNKV8BpcBU4DugwhjTZB/iPS/3nO39lUDnti3xTrsfuAEI\n2887s3efr8MA74jILBGZZG9rs79tXaN5L2eMMSKyVw4xE5F84L/AdcaYKhFx9+2N522MaQZGiUgR\n8DIwpJ2LlDIicjJQaoyZJSJHtXd52th4Y8xaEekKTBWRhd6dqf7bTpeawlqgj+d5b3vb3mqjiPQA\nsL+X2tv3mp+DiISwAsIzxpiX7M17/XkDGGMqgA+wmk+KRMS5ufOel3vO9v4OwOY2LurOOAw4VURW\nAP/BakJ6gL33fF3GmLX291Ks4H8Qbfi3nS5B4UtgkD1yIRM4F3i1ncuUSq8CF9uPL8Zqc3e2X2SP\nWDgYqPRUSfcYYlUJHgcWGGPu9ezaa89bRIrtGgIikoPVh7IAKzicZR8Wfc7Oz+Is4H1jNzrvCYwx\nNxljehtjSrD+X983xlzAXnq+DhHJE5EC5zFwHDCftvzbbu9OlTbsvDkRWIzVDvu/7V2eXXhezwHr\ngUas9sTLsNpS3wOWAO8CnexjBWsU1nfA18DY9i7/Dp7zeKx213nAV/bXiXvzeQMjgDn2Oc8HbrG3\nDwBmAEuBF4Ese3u2/XypvX9Ae5/DTpz7UcDr6XC+9vnNtb++ca5Vbfm3rTOalVJKudKl+UgppVQr\naFBQSinl0qCglFLKpUFBKaWUS4OCUkoplwYFpWwi0mxnpnS+dlk2XREpEU8mW6V2V5rmQqkWtcaY\nUe1dCKXak9YUlErCzm//JzvH/QwRGWhvLxGR9+089u+JSF97ezcRedle+2CuiBxqv1VQRB6110N4\nx56ZjIhcI9baEPNE5D/tdJpKARoUlPLKiWo+Osezr9IYMxx4CCt7J8BfgCeNMSOAZ4AH7e0PAh8Z\na+2DA7BmpoKV8/6vxphhQAVwpr39RmC0/T5XpOrklGoNndGslE1Eqo0x+T7bV2AtcLPMTsS3wRjT\nWUQ2YeWub7S3rzfGdBGRMqC3Mabe8x4lwFRjLZKCiPwKCBlj/iAibwHVwP8B/2eMqU7xqSoVl9YU\nlGodE+fx9qj3PG6mpU/vJKz8NQcAX3qygCrV5jQoKNU653i+f24//gwrgyfABcAn9uP3gCvBXRin\nQ7w3FZEA0McY8wHwK6yUzzG1FaXait6RKNUix17ZzPGWMcYZltpRROZh3e2fZ2+7GnhCRH4JlAGX\n2tuvBR4RkcuwagRXYmWy9RMEnrYDhwAPGmu9BKXahfYpKJWE3acw1hizqb3LolSqafORUkopl9YU\nlFJKubSmoJRSyqVBQSmllEuDglJKKZcGBaWUUi4NCkoppVwaFJRSSrn+H7Bzz028X5LpAAAAAElF\nTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x7f5078ac7518>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n",
|
|
"plt.xlabel('Epochs')\n",
|
|
"plt.ylabel('Validation MAE')\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"It may be a bit hard to see the plot due to scaling issues and relatively high variance. Let's:\n",
|
|
"\n",
|
|
"* Omit the first 10 data points, which are on a different scale from the rest of the curve.\n",
|
|
"* Replace each point with an exponential moving average of the previous points, to obtain a smooth curve."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 38,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecXHW9+P/Xe3vvm92UTTYJ6T0sSSCIISCEJiqgoCAi\nmouiF+5FpXi/FxX1pyIoCoooiF6ahSo9YOiQSnrvyZZs7333/fvjnJmd7bPZnZ3s7vv5eMxjz3zO\n58x8zmYz7/l0UVWMMcaY3oQEuwDGGGOGBgsYxhhj/GIBwxhjjF8sYBhjjPGLBQxjjDF+sYBhjDHG\nLxYwjDHG+MUChjHGGL9YwDDGGOOXsGAXYCClpaVpdnZ2sIthjDFDxoYNG4pVNd2fvMMqYGRnZ7N+\n/fpgF8MYY4YMETnsb15rkjLGGOMXCxjGGGP8YgHDGGOMXyxgGGOM8YsFDGOMMX4JWMAQkSwRWS0i\nO0Rku4jc1EWeRBH5l4hsdvNc53PuWhHZ6z6uDVQ5jTHG+CeQw2qbgVtUdaOIxAMbRGSVqu7wyXMj\nsENVLxGRdGC3iDwOxAF3AjmAute+oKplASyvMcaYHgSshqGq+aq60T2uAnYCYztmA+JFRHCCRClO\noDkfWKWqpW6QWAWsCFRZjRnuCirqeX5TbrCLYYa4QZm4JyLZwAJgTYdT9wMvAHlAPPAFVW0VkbHA\nUZ98x+gcbIwxfrr16S28vaeIT05NJykmItjFMUNUwDu9RSQOeBq4WVUrO5w+H9gEjAHmA/eLSEIf\nX3+liKwXkfVFRUUDUmZjhpMP95fw9h7n/8Ydz26lvqklyCUyQ1VAA4aIhOMEi8dV9ZkuslwHPKOO\nfcBBYDqQC2T55BvnpnWiqg+pao6q5qSn+7UcijEjyp/ePUBMRCgAL28t4P8+9HslCGPaCeQoKQEe\nBnaq6r3dZDsCnOPmzwCmAQeA14DzRCRZRJKB89w0Y0wf7cyvZPn0Ud7nh0trglgaM5QFsoaxFLgG\nWC4im9zHhSJyg4jc4Oa5CzhDRLYCbwK3qmqxqpa659a5jx+5acaYPqhuaCavop4ZoxNIigkH4LGP\njjDtf17hBy9sD3Lphq/SmkZaWjXYxRhwAev0VtX3AOklTx5O7aGrc48AjwSgaMaMGPsLqwGYnB7H\nuu+fyxs7jvPUuqN8sL+YF7fkceclM3EaA8xAqW9qYeFdq7hgdia/v/rUYBdnQNlMb2OGsOaWVo6W\n1nZ7fp8bME4ZFUd4aAgXzBnNX766iO9fOIPi6kYKKusHq6jDWkur8sN/bedgcQ0bjzjTxV7ZVsCR\nku7/bYYiCxjGDGG/fH0Pn/jFagoquv7g31tYTXiokJ0a0y59zrhEALbldhy4aE7EoZIa/vz+Ic7+\n5Vt88/GN3vSn1h0JYqkGngUMY4awl7bmAfCP9UfZfLScmoZmsm97iWc2HgOcGkZ2aixhoe3/q88c\nnUiIwNbcikEv83BUWNngPY4OD+WWT03l7GnpPPdxLqrDpy/DAoYxQ1RFXRMVtU0A3LNqD5c+8D6H\nSpwRUD99eRcA+4uqmZIR1+na6IhQThkVxzY/A0ZhVb3N3+jGnuNVXPXHj7zPn/nmGXz7nClcOGc0\neRX1bDk2fIKyBQxjhqgfv7iDyvpmpmXEe9Pe3VsMQHF1A6t2HOdwSQ2npHcOGACzxyay5VhFr6N5\nymsbOfvutzjnnrf54zsHKKlu6DH/SPM/z27zHm/6308xOjEagE9OdeaFbTpaHpRyBYIFDGOGqD2F\n1Zw+KZX/Pm+qN+217QXe46//dT2tCqf4BBRf587IoLi6gb98cKjH93n0g0PUNLaQW17HT17eyf88\nt63H/P1RVd/Exb99l0/evZoP9hd3Ol9a00h+RV3A3t8fqkphVT3H3QED+ZVt5fFddiUtLpKwEKGw\nyr+BBbWNzfzp3QN+1+Re317Akp++ya/f2EPrIA3htYBhzBBVWFnPmKRozpuZwSNfySE1NoKPj3T+\nNttdDeOC2ZlMGRXH+/s6fzB7qCovbMrjjMmpfHT7OZw/K4NXthX0ODKrP3bmV7Ett5LDJbV8+4mP\nqahr8p4rr21k4V2r+MwD7wfkvf115wvbWfSTNznrF6s5VFzD0dKuA1hIiJAWF8nxSv9qZK9uK+DH\nL+3kh//yb37M+/uKKais59dv7OXNXYV+l78/LGAYMwS1tCqFVQ1kJkYiIiyfnsGnZmYAEBoifCGn\nbWWdSemxXb6GiDAhNZbc8u6/se8vquFAcQ0XzBlNZmIU379wJtC+JjOQPIHoR5fOoqSmka3HKjhU\nXEP2bS9x01ObADhe2UBVfVNPLxNQ7+4tZnpmPA3NrfziNaevaNm0dO66dFanvBkJkRRW+Rcwytz+\nqCfXHvWrlnGwpJapGXGEhQgfHxmcnR8sYBgzBJVUN9DSqmQmRHnTLp47BoAQgcWTUgBnxE5UeGi3\nrzMuOZrcsrpuR/JsOOwssLB0cioA41NjmJoRx1u7i9p9+x8I97y+m1v+sRmAM9z3y6+o8y6c6PkJ\nwRvdtfloOQeLa7hg9mgWTUzh5a1O4PzOedO45vTsTvnT46Mo9HOuS75P4H74vYPc+fw2tudVUNvY\n3ClvYVU9H+4vZmpGPNMy4wft9zEoy5sbYwaWZ8Jdhk/AOHNKGn/+ymmEh4YwJslJ//nlc3t8nbFJ\n0VQ1NFNZ10yiu3SIx6HiGm59eisxEaFMTGurpUzNiOfFLfnM++HrHPrZRf2+l4q6Jv7zyY/bBYRx\nyc68kYKKeg75TH67esl4nlhzhPf3FXPG5LR+v3dfXeo2h80YHU9keAhrDzoBNTut61pcRkKkdyJf\nb/Ir65mYFktYiHD3a7sB+MuHh4mLDOMnn53NpfPbdnj4j//bQFOL84UhLjKMV7YVoKoBn7VvAcOY\nISjfnajnGZHjcbbPIoN7fnwBEWE9NyKMTXauP1ZeS2JMYrtz96/eB8B5MzPafRBN8vlwrKpvIj6q\nfaDpqzd3Hm8XLACiwkNJiY3gtR0F7D1e7U3/znnTyC2r44HV+4mNDOOby07p13v3RXVD2zf907JT\nmJIRz89f3cWUUXHERXb9UZqZEEVpTSN1jS1ER3Rf0wOnhjE6MYrLTx3HnS9sJyU2gsMltTS3tnLT\nU5u47429zB+fxNLJad6+qvNmZZIeH8nXPjFp4G60BxYwjBlE9U0tvLHzOBfNGd2vb4Oetv6slOhu\n8/QWLACyU50P/wNFNcwa0xYwGptbeXlrPlecOo6fX9a+lpKV0jZrPL+ivt8B44h7L1eelsWXFk8g\nPMz5vYSFCNtyK8lKiebpG84gJjKMuMgwvrX8FFbvLuKFTXmDGjA8s+l//YX5JMdGkBwbwa67VhDS\nw7/jKaOcAQf7Cqu9s+u70tjcyuGSWj45LZ3PLRzHpfPHUt3QzPHKejYdKed7T2/hQLHTn/TMRmen\nh5vPncKiiSkDeIe9sz4MYwbRT17aybee+JgNh/vXSXm4pJaEqLB+7543eVQsoSHCrgJniZDmlla+\n9pd1/OSlHdQ2tnDmlDRCQtp/IGYmtjWD5bnt7sXVDVzx4Aes3l1IXWPfJvjtL6ohKyWan102lznj\nEpme6eyh5un4ffS6RYxym14ATp2QwtfOnMjB4pqArwhbVtPIn949wPHKem9fhG8zYGRYKOGh3X+M\nTnGHNO8+XtXl+T3Hq6hrbOGXr++mpKaRi+eOBpyBC4nR4UzNiOd0tz+no0ndjH4LJKthGDOIPJO4\n/r2rkAXjkwkN6Xst49Vt+fzfR4eZObpPm1N2KTIslMnpsewucD7Qnt54jDd2tg3R9HxD9nXmKWnc\neclMfvivHeSVOx+iq3cVsu5QGdf9eR1XLxnPjz8zx+8yHCiqZnIXH36PfnURBRX1XZ6bkhFHQ3Mr\nx8pqmZDadf/BQPjm4xv58EAJNQ0tjE91anMZCZF+X5+dGkNEaAh7uggYpTWNnPerd5gzNpGtuRV8\nISeL5dMzOuXLSonh0etOY0FWMr97ex9/ePsAAJO7Gf0WSFbDMKYfnvs4l7d2+zcGXlW9Q1h/99Z+\n73pPfVFZ38QNjzmL2/k7Iaw30zIT2OP2E6w50H7bma4+rEWEa5ZMIESc5qQP95fw3X9u8Z5/Z0/3\n8zo8Gppb2F1QxZ7jVRwoqunyfRaOT+bCOaO7vN7zzX2PT/9GV3LL6/j6X9f7PVLJ1+6CKj48UALA\nsbJa73wK3xpGb8JCQ5iSEcfD7x3kqbXtFyL0TEz0jHC69ozsbl9n2bRRJMaEc+v50wkLEWaOThiQ\nLwx9ZTUMY07Qn949wI9f2gng12ihkppGSmsavc935Pd9pdgdec418ZFh/L+LZ/b5+q6MSYrite31\nqCp7C9s+gNPjI7sdkhsWGsIZk9N47KPDrDlY4k2/bmk2j685QmNza7d9KAeLa7jwvnep85lr0N1c\nke54aj5f/+t6Hrx6IStmdw4sb+w4ztf+uh5wOu6vyMnqlAegtVVpbGntdK9bjjm1wbS4CI6V1REZ\nHkJ8VBix3XRwd2deVhLb8yq57ZmtfD4ny9vE13HC5MRuRlr5CgkR1txxDonR4UHZxySQW7Rmichq\nEdkhIttF5KYu8nzXZze+bSLSIiIp7rlDIrLVPbc+UOU05kSU1TR6gwU4s5A9Wlu1y3kNh4qdhQEf\nvjaHsUnRFFY18Oq2fB5wRyP5w9NB/NJ/fqLdMMv+yIiPorG5lbLaJvYVVnPt6RP4wzWn8s8bTu/x\nuh98eibVDc3eETuPfCWHnAkpNDa3eptg3ttbzINv72933W/f3NsuWEDXNZmeJPh0tN/w2MYuJ/K9\n77O0yLGy7icn/vL13Uz/f6/y33/bRKXP63hqg4smpnCsvJaPj5Qzt4eO6+5M9Gky8/z7qap33S+P\n3kZReaTGRXZafXiwBPJdm4FbVHUmsAS4UUTafSVS1btVdb6qzgduB97usBXr2e75nACW05g+y3PX\nM7r+zIkAvLW7bVjoGT/7N199dF2naw64AeOUUXFMSo/lpS353PDYRu+Y+6aWVoq6mRX8j/VHWX7P\nW3zPbfoZneR/s0hvRrlt8puOllHX1MK0zATOn5XZa9/AKaPimeE2i1yzZALLp2d4V8bdX+TUVK5+\neA0/e2UXD769n6aWVgA2HyvnUzMzOPDTC73t8H0NGB1t7WJF2LzyOqZmxDEuOZr73tzLOx2G7nr8\n6d2DADzzcS6vbM3nH+uPcun97/HvXYVkJEQyKS2Oo6V1bM+r5NTxyX0u2+cWjmVsktP/saugiobm\nFtYeLOVYWR13XjKThKgw7yz9k13AAoaq5qvqRve4CtgJ9PSV6CrgyUCVx5j+UlXe2l1IQ3ML+W5n\n70VzRzMpPZY/vnsAVaWusYWCynpW7+784XSwuIbwUGFsUjSVHWZJt7Yq972xl9N+8gbHyjqv0/Td\nf27hQFGN93lPI3P6ytMm/95ep2mpq47u7vzsc3OYl5XkHd0zITWGEGnbGtab75VdPLsxl9rGZg4U\n1zBrTAIhIcL3L5rBpfPHkBbX99FeX/Fp89+RX8ljHx1ut0RGXnk9oxOjKal2an9ffmRtl7/b+Ki2\nJqa3dhdx69Nb2Hysgi3HKhibFM2sMW19BUu6GbHUk9S4SN74708iArsKKvnBC9v5wkPOcuifnJrO\nuv85l99/aWGfXzcYBqVeIyLZwAJgTTfnY4AVwNM+yQq8LiIbRGRloMtoTHeaWlr59P3v8Y3HNvKV\nP6/j3tf3sN4dFjsuKZobzprM9rxK3t1b3G6JhsoOzSQHi2rISokhLDSEbyw7hemZ8fzXuVO9eT9y\nO1h/++Y+aho6LwcRKBnxTsDwtKlP6UPAmJeVxPM3LmXxJOeDNDIslAmpsezI7zwq6Ndv7OEbj21E\nFWa7cz6WT8/gvisXnFB7/J2XzGTfTy4gNET48UvOKrq+ne955XWMSYrmjgunk+AGhY77f+RX1FFS\n08g3l01m8cQUXtlWQKs6S6YAjE2O4Vz3239CVBinT+p7wACnuSk7NZZd+VW8sCnPmz4xLZbIsNCg\nNTH1VcBLKSJxOIHgZlXtrpfvEuD9Ds1RZ6rqQuACnOass7p5/ZUisl5E1hcVdV3lNKY/dhdUseVY\nBa+6C+794Z0D3nb51LhILl0whpTYCJ79OLddwNhytO1YVTlYXOOdJb1idiav3nyWd6hmaU0jjW6T\nzd/WH+WsX6wGnAlfr2zNb1eet76zbEDvz9Mktft4FanuhLT+WDg+mTe6mL2dmRjlTTtzSv+X9RAR\nwkJDvHMxTstOZl9hNTvzKympbqCkppGxSVFcc3o2a+44lxChXSBrbG7lf5/fTojAFTlZXLfUaV6c\nmhHHL6+YxyempHHpvDGEh4bw/m3LefOWZf3qaJ6eGc+ugkoi3c7182dlBKXjuj8COkpKRMJxgsXj\nqvpMD1mvpENzlKrmuj8LReRZYBHwTscLVfUh4CGAnJyc4bMXojlp+O6Y9sXFzlpGHqEhQmhIKIsn\nprDuUKk7egViI8L41pMbef7GpUxIjeUzD7zP7uNVnDW1/QelZ+JdbnkdB4trEAFVZ0TVVQ995B3W\n6XHTOVO6XbfoREWFh7ofZlVM7WbvjL646zOzeHlrPtc+srZd+tPfOIP73tzLrDGJPS6I2Fe3rpjO\nsbJabjlvGot+8gbPbcr1zoRfNNGpEURHhJKdFsuj7x/kqkVZjE6M5rGPDrNqx3GuPX0CE9NiyU6N\n4Y4Lp3PujAwmpcexxKc24emD6I/pmQm8ss350vHd86fx9UFazmMgBXKUlAAPAztV9d4e8iUCnwSe\n90mLFZF4zzFwHhC4XVuM6cHmo+UkxYSz664V3HTOFG/6PVfM8x7nZKdwrKyOj4+UMWVUHFeelkV5\nbRM/f3UXtY3NbHaDTsdv7yluwLjm4bVU1TfznfOmMT3T+dDuGCze/u6ydu8/kMa7y31cPK/reQ99\nERMR5l380OOmc6YgItx87tQB7+D9xrLJ/OSzc0iJjWCZu4/2u3uLuWzhuHZLZ3x5yQQq65t53m0S\n+mB/MRNSY/jhpbMBp8ay8qzJAZtBPdOnL2TxxBS/lm452QSyhrEUuAbYKiKb3LQ7gPEAqvqgm/ZZ\n4HVVrfG5NgN41q2uhQFPqOqrASyrMd3afKycueOSiHKXCv/cwrEsnZzGZaeO8+bxDLfcfKyCs6el\nc8eFMzhe1cC/Nufx4f62D/7lPosDAiT7LO0xJjGKS+ePYcmkVC77/QcAPPDFhdz4hDNRL5Azmr+3\nYhqxkWF8bsG43jP7wTNX4a5LZ/GlxRM6LS8SKJ9ZMNY7U31xh3WWvrJ0Ig+9c4C3dxex4XAZb+ws\n5MrTup6bEQjzs5K8xzOCMOluIAQsYKjqe0CvfyWq+ijwaIe0A8C8rvIbM5hqG5vZW1jNeT7fiu/9\n/PxO+XyHhY5JiiYkRPjamRP51+Y878Y4q/7rLO8MZY+k2Lb5BO9872zCQkOIjXD+W45JjOKiuaOJ\nCMthQmoMgXTKqHh+9YXO93Wi7rliHi9vLeDqJRMGtZ3+3BkZhIcKTS3KaV0szDdrbCKrdhz3Pvfs\nGzIY0uPblhTp6+S/k8XQLLUxg2RnfiUtrcrccUk95kvxaWry7OUwLyuJp1Yu4Up3CGVXM3njI8OI\nCg/h28uneEfKJMdG8OPPzOYTbsfwUBmj72tKRjw3DUB/SF9FhYey9o5zKatt7PL3PT8rqX3AmHhi\no55O1H1Xzu9xdduTnQUMY3rg2XfCd0nv3lwwO9N7fOqEZC5bOI6rl4zvcuikiLDrrgs6pV+9ZMIJ\nlNYA3qXHu3LZwnHeiZKhIcKYAejM7ouBmp0fLBYwjOmBZ+2nFD+Gmj52/WJKahrajWIKDw3hns9b\n6+rJIjMxige+uJCxydHewQXGfxYwjOmBZ5ZwckzvmwQNxNwCE3gXze3/SLCRauiN6zImgB5YvY/L\nf/8BFbVNVNQ1UVrTSFJM+JCZiWtMIFkNwxgfnvbteT963ZsW6BFKxgwV9rXJmF4Ud7OCrDEjjQUM\nY1wtrerdMvXBqxd612yq6eMe1cYMV9YkZYwrv6KOllbl//vcHFbMHu3dBOmibrYJNWaksYBhjOtI\nibNgnWddJWeOxIoB3XvCmKHMAoYxLs9+1r4bCA3kqqrGDHX21ckY1+7jVSREhTHKZ80fY0ybER8w\nmlta+cf6o2w4XNp7ZjOs7T3u7Acx1Da1MWawjPiAERoi/OhfO3ju47zeM5th7XBJLZPSA7eEuDFD\n3YjvwxARpmXGs7ug8x7EZmRQVVrdXe5GxUf1foExI9SIr2EATB8dz86CSu8wSjNyPL8pl4m3v8y+\nwmpaWrXdngXGmPYCuUVrloisFpEdIrJdRG7qIs93RWST+9gmIi0ikuKeWyEiu0Vkn4jcFqhyAkzL\nTKCqvtm7lLUZ/uqbWvjLB4e46SlnM8h/73J2abOAYUz3AlnDaAZuUdWZwBLgRhGZ6ZtBVe9W1fmq\nOh+4HXhbVUtFJBR4ALgAmAlc1fHageQZFeNZytoMf3969wB3vrDd+/znr+4CLGAY05OABQxVzVfV\nje5xFbAT6Gn3kKuAJ93jRcA+VT2gqo3AU8ClgSprTIQz1r6uyZaAGAlUlSfWHAEgrsNWmelxFjCM\n6c6g9GGISDawAFjTzfkYYAXwtJs0Fjjqk+UYPQebfvEEjFpbM2hEOFpaR15FPT/89CzW3HEOkWFt\n/w3SrIZhTLcCPkpKROJwAsHNqlrZTbZLgPdVtc+TIURkJbASYPz48SdUxuhw59dQZwFj2Fl3qJRZ\nYxKIiWj7U//oYAkAp09OJTYyjH9/Zxk78yrZVVDZqcZhjGkT0BqGiITjBIvHVfWZHrJeSVtzFEAu\nkOXzfJyb1omqPqSqOaqak56efkLljPY2STWf0PXm5HS4pIYrHvyQu17c0S59zYFSUmIjmOIuATI2\nKZpzZ2bwreVTglFMY4aMQI6SEuBhYKeq3ttDvkTgk8DzPsnrgCkiMlFEInACyguBKqs1SQ1PW3Mr\nANh7vLpd+kcHSliUnWIzuo3po0DWv5cC1wBbRWSTm3YHMB5AVR900z4LvK6qNZ4LVbVZRL4FvAaE\nAo+oatuQlgHmrWFYwBhWNh8tByA+qu3PvLCqntzyOr565sRgFcuYIStgAUNV3wN6/Qqnqo8Cj3aR\n/jLw8oAXrAvR4RYwhqNd7uz9ouq2HfOOldUBMDHNtl01pq9spjcQHhpCeKhQa8Nqh5Uid2vVvPK2\nCZl55U7AGJMUHZQyGTOUWcBwRYeHWg1jmCl2axalNY2UVHuChwUMY06UBQxXdIQFjOGkpVUprWnk\nvJkZALy4JR9wahvxkWEkRIUHs3jGDEkWMFwxEWHWJDWM/G71PloVPjEljXnjErn7td0cLqnhWFkd\no5NsRVpjToQFDJfTJGXzMIaDj4+Ucc+qPYCzNtRvrlpAdUMzr20vYEdeBVMz4oNcQmOGJgsYrpiI\nUFtLaph4cu0R73FCVDgTUmOZlBbLC5vzyKuoZ8H45CCWzpihywKGKzoilJoGCxjDwbpDZcwak8Bl\nC8d5g8PiSSlsy3VWppmflRTM4hkzZHUbMETkez7HV3Q499NAFioY4qPCqG6wJqmhZnteBX/54JD3\neVFVAweLa/j0vDHc8/l53kmZiyemevPMGZs42MU0ZljoqYZxpc/x7R3OrQhAWYIqMTqcyrqmYBfD\n9NGNj2/kzhe2e7fY9fzsGBQWT0rxHkeEWcXamBPR0/8c6ea4q+dDXkJUOBUWMIacFndb3RX3vcPB\n4hoOFjvrRk1Kj2uXb3RiNDeePZm/rVwy6GU0ZrjoaWkQ7ea4q+dDXkJ0OA3NrdQ3tRDlLhViTm6q\nSllNExNSYzhcUssf3z1AVFgo0eGhZCR03tfiu+dPD0IpjRk+egoY80SkEqc2Ee0e4z4fdgPZE6Kd\niVxV9c0WMIaIJ9cepbqhmVsvmM6aAyWs3lXI9Mx4stNibSVaYwKg24ChqiPqUzPBXdG0oq7J9nUe\nIv764SEiwkK4aM5oKuuaeHFLPqowL8s6tY0JhD71/olIrIhcLSIvBapAweKpYVTWWz/GUFFR18Sl\n88aQEhvhnYxXUFlPRsKwqwAbc1LoNWCISISIfFZE/gHkA+cAD/Zy2ZCT6AkY1vE9ZFTUNZEU4/y7\neXbPAxhlNURjAqLbJikROQ+4CjgPWA38FThNVa8bpLINKs9idDZSamhoaG6htrGFpJgIALJSYogI\nC6GxuZVR8VbDMCYQeqphvApMAs5U1atV9V9Aq78vLCJZIrJaRHaIyHYRuambfMtEZJOb522f9EMi\nstU9t97f9z1RCdFO7Kyst8l7wfTqtgKyb3uJgor6HvN5ArunZhgaIkx2h9JaH5QxgdFTwFgIfAi8\nISKrROR6nO1S/dUM3KKqM4ElwI0iMtM3g4gkAb8DPq2qs4ArOrzG2ao6X1Vz+vC+J8RTw7AmqcFX\nUdeEuvMpnnDXgdpwuKzna2qdfydPkxS0NUtZwDAmMLoNGKq6SVVvU9XJwJ3AfCBcRF4RkZW9vbCq\n5qvqRve4CtgJjO2Q7YvAM6p6xM1XeIL30W9R4aFEhoVYwBhkR0trmffD13lsjRMo4iKd7yT7Cqt7\nvK7c/XdKio7wpk3LdDq+rdPbmMDwa5SUqn6gqt8GxgG/wqkx+E1EsoEFwJoOp6YCySLylohsEJEv\n+74t8Lqb3muAGggJ0eE2SmqQfXigBIB39hRR39TC/sIaALbmlvd4XXkXNYyrF0/gwasXWg3DmADp\nqdN7YTenioH7/X0DEYkDngZuVtXKDqfDgFNxRl5FAx+KyEequgen7yRXREYBq0Rkl6q+08XrrwRW\nAowfP97fYnUpISrMOr0D5Ib/20BtUwu/vWoBLa3KL1/fzX9/aiprD5YCTl/Ej17cwe7jzlpQ7+8r\nobaxmZiIrv9Ey2sbvdd5JMaEs2L26ADfiTEjV08zvdcD23ACBLRfP0qB5b29uIiE4wSLx1X1mS6y\nHANKVLUGqBGRd4B5wB5VzQWnmUpEngUWAZ0Chqo+BDwEkJOT068lS5wFCK3Te6A1NLfw6vYCAP62\n7ggbD5eIA6m2AAAgAElEQVTz6vYC4iLDeMndOvV4ZT17jzvNUBfMzuSVbQW8s6eYFbMzAWhtVX7/\n9n6+uGg8ybERFLl7dKfGRXTxjsaYQOipSeq/gUqgDvgzcImqnu0+/AkWAjwM7FTVe7vJ9jxwpoiE\niUgMsBjY6U4QjHdfJxZnaO82v+/qBFmTVGAUVjZ4j5/9OI/39znfQR565wB1TS1MTIslr7yOEIGL\n547m3s/PB2Dv8SpaW53vAOsPl3H3a7v5/nNbAcgrryM5JrzbGogxZuD11On9a1U9E/g2kAW8KSJ/\nF5H5fr72UuAaYLk7NHaTiFwoIjeIyA3ue+zEGb67BVgL/ElVtwEZwHsistlNf0lVXz3Rm/SXrVgb\nGAWVzhDZ2WMT2JlfSVVDM9Huel0RoSF8cmo6ueV15FfWMzk9jugIZ/HAZzflMumOl9l0tJxit0bx\n8tYC5v/odR776AijE6ODdk/GjES9fj1T1QMi8jxOH8M1OB3Vm/y47j38WAZdVe8G7u74njhNU4PK\n9sQIDM+ciuXTRnl3vbtuaTa/e2s/Ta2tnHlKGo+6myCNT4kBYEJqrLd/468fHuJFt+kK2jq8rTnK\nmMHV0457k0TkDhFZA/wQ2AzMUNW/D1rpBllCdBiV9c3eOQFmYBx3axjLZ2R4065a5AxQCA8J4dyZ\nGdx35XwyEiKZ526fOsENHADPbMylsdmZM5qVEk3OhOR2r2uMGRw91TD24TQVPY/TlzEe+IZn2ege\n+iWGrOSYCFpalcq6ZhJ9hmua3r2zp4j73tzLE19fTGRY+/mdBRX1RIWHMG9cIr/70kIWjk8mMzGK\nlWdN4pzpowC4dP5YLp3fNk1n4YRk/rHhWKf3efd7y6msb2LuD17nwjk2IsqYwdRTwPgRbRslxfWQ\nb9jITHQmfOVX1lnA6KMvP7IWgEPFtd4JdB5Hy2rJSo5BRNp9yN9x4YxuX+/K07Ioq23kpS35bM9z\nmrHu/bzTSpkQFc6WH5xHnHV4GzOoetoP4weDWI6TwmhPwCivZ3pmQpBLM3Q0tbQtMXasrHPAOFJa\n5+2b8JeI8M1lp5ASE8Ftz2xlbFI0n1s4znves5SLMWbw9Gk/jOHOM+omv5eF70x7nhFMAMfK6tqd\nU1WOlNSQ1ceA4eHZm7um0ebHGBNsFjB8jIqPJEQgv6Ku98zGq6S60Xt8tLS23bnSmkZqGluYkHqi\nASMWsBqFMScDawT2ERYaQkZCFHnlVsPoC98axhGfgNHSqvzv89sBmDH6xJr4UmMj+J+LZrBs2qj+\nFdIY02+9BgwRiQQuA7J986vqjwJXrOAZFR/pXXbC+MdTw5gxOoGdBW3Lhf1zw1Fe2prP7RdMZ/HE\nlBN6bRHha5+YNCDlNMb0jz9NUs8Dl+Lsb1Hj8xiW0uIiKaqygNEXJTXO7+vsaekcLa3zLgy48XA5\nqbERrDxrEp7h2MaYocufJqlxqroi4CU5SaTHR7IltyLYxRhSSqobiQwLYcmkVH731n6251Wy9JQ0\ndhVUMn10vAULY4YJf2oYH4jInICX5CSRFhdJaU0jLa0229tfRVUNpMVFMtnd8e5IaS0trcru41VM\ny7DhycYMF/7UMM4EviIiB4EGnPWhVFXnBrRkQZIeH0lLq1JW20hanG3E05vaxmb+vbuQJRNTGRUf\niQg893Eu45KjqW9qZfKo2GAX0RgzQPwJGBcEvBQnEU+QKK5usIDhh3f2FFNe28SXz5hAeGgIyTER\nrDlYypqHnZnfY5JsRVljhotem6RU9TCQBFziPpLctGEpzV0BtbiqsZecBmB7XgWhIcLC8c6CgLUd\nJth5Zs8bY4a+XgOGiNwEPA6Mch+Pici3A12wYPHsB11UbXMx/LEjr5LJ6bFEuftb1De1tjs/OsFq\nGMYMF/40SV0PLHa3UUVEfg58CPw2kAULljQ3YFgNwz878ys5zWeOxfTMeHYVVHmfJ0Tb3FBjhgt/\nRkkJ0OLzvAU/NkYSkSwRWS0iO0Rku1tT6SrfMnc3vu0i8rZP+goR2S0i+0TkNj/KOSDiI8OICAtp\nN3vZdK2usYW8inpOSW9bzPixry3muRuXep/bkFpjhg9/vv79GVgjIs+6zz+Ds1d3b5qBW1R1o7s/\n9wYRWaWqOzwZRCQJ+B2wQlWPiMgoNz0UeAD4FHAMWCciL/heGygiQrpN3vPLoRJn/mZ2WttIqLS4\nSNLiInn5Pz9BdYMtGGjMcOLPFq33ishbOMNrAa5T1Y/9uC4fyHePq0RkJzAW8P3Q/yLwjKoecfMV\nuumLgH3uVq2IyFM4s80DHjDAaZay5UF6d6jYCRgT0zoPnZ05xuZfGDPcdBswRCRBVStFJAU45D48\n51JUtdTfNxGRbGABsKbDqalAuBuQ4oH7VPWvOIHlqE++Y8Bif9+vv9LjIsi1BQh7dbCLGoYxZvjq\nqYbxBHAxsIG2nffAnbgH+LUinIjEAU8DN6tqZYfTYcCpwDlANPChiHzkX9G9r78SWAkwfvz4vlza\nrfT4SDYdteVBenO0tI6U2AjiIq1j25iRoKcd9y52f0480RcXkXCcYPG4qj7TRZZjQIk7AqtGRN4B\n5rnpWT75xgG53ZTzIeAhgJycnAFZz8NZHqSBllYlNMQ6bTv6cH8JM0cnUFBRZ/MsjBlB/JmH8aY/\naV3kEZzO8Z2qem832Z4HzhSRMBGJwWl22gmsA6aIyEQRiQCuBF7o7T0HSlpcJK0KZbU2tLaj4uoG\nrvrjR3zryY3kV9R7dyk0xgx/PfVhRAExQJqIJNM2lDYBp4+hN0uBa4CtIrLJTbsDGA+gqg+q6k4R\neRXYArQCf1LVbe77fwt4DQgFHlHV7X29uRPlnbxXZcuDdLT+kNN19e7eYhKiwlh0gvtcGGOGnp4a\nn/8DuBkYg9OP4QkYlcD9vb2wqr6HH/M1VPVu4O4u0l8GXu7t+kDwXU/KtPfRgbaxDpX1zVbDMGYE\n6akP4z7gPhH5tqoOy1nd3fGuJ2UBo52ymkb+sf4ok9JjOVDkjJDKSrGAYcxI4c88jN+KyGxgJhDl\nk/7XQBYsmHybpEyb9YfLqGls4eeXzeWP7xygsaWV82ZmBrtYxphB4s+e3ncCy3ACxss4y52/Bwzb\ngBEXGUZkWAjF1dbp7et4pTM3JSs5hj9cc6ot+2HMCOPPWlKX48yTKFDV63CGvSYGtFRBJiKkxUVS\nbDWMdgqrGhBxmuwsWBgz8vgTMOpUtRVoFpEEoJD2cySGpXRbHqSToqp6UmMjCQv158/GGDPc+DNF\nd727SOAfcUZLVeMsbz6spcVFcqysNtjFOKkUVjYwKt6GGRszUvnT6f1N9/BBd85EgqpuCWyxgi89\nPoJNR8uDXYyTyvGqekYlWMAwZqTqaeLewp7OqerGwBTp5JBuy4N0UljZwMzRtgqtMSNVTzWMe9yf\nUUAOsBlnIt5cYD1wemCLFlxp8c7yINvzKpg7LinYxQm6llaluLqBjARbO8qYkarb3ktVPVtVz8bZ\n02Khquao6qk4y5R3uRDgcLJkUioAv3x9T5BLcnIoqWmgVbE+DGNGMH+Gu0xT1a2eJ+5aTzMCV6ST\nw9SMeFbMyiSvvC7YRTkpFFY6I8bS462GYcxI5c8oqS0i8ifgMff5l3AWCxz2kmPDqTjSFOxinBQK\nq5xJe9bpbczI5U/AuA74BnCT+/wd4PcBK9FJJCE6nIraJlR1xE9U89QwrEnKmJHLn2G19cCv3MeI\nkhgdTmNLK/VNrURHhAa7OEFVWOVpkrKAYcxI1dOw2r+r6udFZCvtt2gFQFXnBrRkJ4HE6HAAKuqa\nRnzAqKxrIiYilMiwkf17MGYk66mG4WmCungwCnIy8g0YmSN8K9KaxmZibe9uY0a0nvbDyHd/Hj6R\nFxaRLJwVbTNwaigPuXts+OZZhrNN60E36RlV/ZF77hBQBbQAzaqacyLl6A/fgDHSVTe0EGcBw5gR\nracmqSq6aIrCmbynqtrblN9m4BZV3Sgi8cAGEVmlqjs65HtXVburxZytqsW9vE/AJEU7GylZwIDq\n+iZiI605ypiRrKcaRnx/XtitoXhqKVUishNnL/COAeOkZTWMNjUNLcRGWA3DmJHM73WqRWSUiIz3\nPPryJiKSjTNDfE0Xp08Xkc0i8oqIzPJJV+B1EdkgIiv78n4DxQJGm+qGZmuSMmaE82fHvU/jrCs1\nBmcvjAnATmBWT9f5XB8HPA3crKqVHU5vBCaoarWIXAg8B0xxz52pqrkiMgpYJSK7VPWdLl5/JbAS\nYPz4PsWxXsVHhSECFbUjc+e9sppGjlfVMy0j3jq9jTF+1TDuApYAe1R1Is7uex/58+IiEo4TLB5X\n1Wc6nlfVSlWtdo9fBsJFJM19nuv+LASeBRZ19R6q+pC7zlVOenq6P8XyW0iIEB8ZNiJrGKrK1/66\nnhW/fpeH3ztITYMFDGNGOn8CRpOqlgAhIhKiqqtxVq/tkThTox8Gdqrqvd3kyXTzISKL3PKUiEis\n21GOiMQC5wHb/LqjAZYYEz4iA8bGI2VsOFwGwO/e2k9JTSNx1ultzIjmz1fGcrdZ6R3gcREpBGr8\nuG4pcA2wVUQ2uWl3AOMBVPVBnP3CvyEizUAdcKWqqohkAM+6sSQMeEJVX+3DfQ2YxOiRGTCOlTmL\nLn73/Gnc/dpuAOIiw4NZJGNMkPkTMC4F6oH/wll4MBH4UW8Xqep7OENwe8pzP3B/F+kHgHl+lC3g\nRmrAKHKXAjl3RoY3YNiwWmNGtm6bpETkARFZqqo1qtqiqs2q+hdV/Y3bRDUiJEVHsPFIOesOlQa7\nKIOqqKqBiLAQpmbEedNslJQxI1tPfRh7gF+KyCER+YWILBisQp1MPNuzXvHgh0EuyeAqqmogPS4S\nEeHrn5jIgvFJnDE5LdjFMsYEUU8T9+4D7hORCcCVwCMiEg08CTypqiNiK7o9x6u8x8cr60fMFqWF\nVQ3evS++f9HMIJfGGHMy6HWUlKoeVtWfq+oC4CrgMzjzMEaE/3fxTNLinCVCtudVBLk0g6ewqp70\nOFvK3BjTpteAISJhInKJiDwOvALsBj4X8JKdJJaeksaz31wKQHHVyJjAl1dex97CamaO6W25MGPM\nSNLT4oOfwqlRXAisBZ4CVqqqP0Nqh5U095t2UXVDkEsyOF7ZVoAqfHbB2GAXxRhzEulp2MvtwBM4\nK86WDVJ5TkrREaHERYZRPEICRn55HTERoUxIjQ12UYwxJ5GeOr2XD2ZBTnZpcREUV4+MJqnyuiaS\nom2SnjGmPb9Xqx3p0uIiKaqqD3YxBkV5bROJMRHBLoYx5iRjAcNPaXGRI6aGUVHXaDUMY0wnFjD8\nlJ0Wy+GSGuqbWoJdlIArr20iOdYChjGmPQsYfsqZkExTi7L5aHmwixJw5XVNJEZbk5Qxpj0LGH46\ndUIyAGsPltLc0hrk0gSOqlJe20hSjNUwjDHtWcDwU3JsBGOTorln1R4uuf/9YBcnYGobW2hqUevD\nMMZ0YgGjDyalO/MSduZX0tg8PGsZpTVOx36yjZIyxnRgAaMPWlW9x1tzh2dfxtHSWgDGJUcHuSTG\nmJNNwAKGiGSJyGoR2SEi20Xkpi7yLBORChHZ5D7+1+fcChHZLSL7ROS2QJWzLy6YPdp7vCO/qoec\nQ9dhN2BkpcQEuSTGmJNNIHfEacZZVmSjuz/3BhFZpao7OuR7V1Uv9k0QkVDgAeBTwDFgnYi80MW1\ng+pLi8dzwexMFv30TY5XDM9JfIdLagkPFcYkWQ3DGNNewGoYqpqvqhvd4yqcJdH9Xc1uEbBPVQ+o\naiPOwoeXBqak/hMRUuMiyYiPJH8YBowjJbU8+PZ+xiRFezeOMsYYj0HpwxCRbGABsKaL06eLyGYR\neUVEZrlpY4GjPnmO4X+wCbjMxCgKKuuCXYwB9/aeQgDOnjYqyCUxxpyMAh4wRCQOeBq4WVUrO5ze\nCExQ1XnAb4HnTuD1V4rIehFZX1RU1P8C+2F0YvSwrGEUVjUQIs6mUcYY01FAA4aIhOMEi8dV9ZmO\n51W1UlWr3eOXgXARSQNygSyfrOPctE5U9SFVzVHVnPT09AG/h65kJkaRX16P+oyaGg4KKxtIjYu0\n5ihjTJcCOUpKgIeBnap6bzd5Mt18iMgitzwlwDpgiohMFJEInD3FXwhUWftqUnosdU0tHCsbXs1S\nhVX1jIq3bVmNMV0LZA1jKXANsNxn2OyFInKDiNzg5rkc2CYim4HfAFeqoxn4FvAaTmf531V1ewDL\n2iezxiQC8IlfrB5WmyoVVjVYwDDGdCtgw2pV9T2gx7YNVb0fuL+bcy8DLwegaP02PTPee7z2YCkX\nzhndQ+6ho7CqgdluMDTGmI5spvcJiAoP5dwZzkiikmFSw2hpVUqqGxiVYDUMY0zXLGCcoIeuySEs\nRIbNaKmS6gZaFWuSMsZ0ywLGCQoJETISooZNwCiscmpK6fFRQS6JMeZkZQGjH8YkRZFfMTxGShW6\n+5Vbk5QxpjsWMPphdGL0sBhaW9fYNkTYmqSMMd0J5OKDw960zHhe2JxHZX0TCVFDd8Ohqx9ew4bD\nZQCkW8AwxnTDahj9MGtMAgA78jqueDK0eIIFQGRYaBBLYow5mVnA6AfPBL7tQyxgHCur5fMPfshL\nW/Jpcvcnv2juaB75Sk6QS2aMOZlZk1Q/pMdHkhIbwb7CobWZ0r2r9rD2UCkJ0WGclp0MwJJJqSyf\nnhHkkhljTmZWw+inU9Lj2FdYHexi9MmaA6UAFFc3Ulzt7OGdFmt7eBtjemYBo58mjxpaAeNoaS25\n5c6IqCOltTy/yVkEOMUChjGmFxYw+umUUXGU1TZRVDU0lgh5Yu0RRJztZktrGvnDOwcASI2z0VHG\nmJ5ZwOineeOcju+Pj5T1kjP4Xtmaz+/f2s+nZmSwrMOueqlWwzDG9MI6vftp9thEIkJDeGZjLo0t\nrVw8d0ywi9QlVeW+N/cyKT2We78wn2Z3dBTA5aeOIylm6M4jMcYMDqth9FNUeCjzs5J4dXsB33ri\nY1paT85d+LblVrKroIqvnTmJuMgwkmLaahS/vGIe7j5WxhjTLathDIDzZ2ey9pAz8qiyronkk7B5\n56Wt+YSGCBfMzvSm/etbZ9LY0hLEUhljhpJAbtGaJSKrRWSHiGwXkZt6yHuaiDSLyOU+aS0+O/Wd\nNNuzduXyheO8xyU1jUEsSddUlVe25XPG5NR2wWzOuEROnZASxJIZY4aSQDZJNQO3qOpMYAlwo4jM\n7JhJREKBnwOvdzhVp6rz3cenA1jOfkuMCeevX10EQFlt3wPGr9/Yw5KfvjnQxfLanlfJ4ZJaLhom\nOwMaY4IjYAFDVfNVdaN7XIWzN/fYLrJ+G3gaKAxUWQaDZx5DSfWJBIy9FFTWU1HbNNDFAuCVbU5z\n1HmzMnvPbIwx3RiUTm8RyQYWAGs6pI8FPgv8vovLokRkvYh8JCKfCXgh+8kTME6khhEb4Sz4tz2/\nYkDL5PHKtgJOn5Rqk/OMMf0S8IAhInE4NYibVbXjKn2/Bm5V1dbOVzJBVXOALwK/FpHJ3bz+Sjew\nrC8qKhrQsveF58O4tKaRPceruPWfW9oNXe1JdlosANtznV+PqrLmQAl1jf3vkG5pVQ4V17BgfFK/\nX8sYM7IFNGCISDhOsHhcVZ/pIksO8JSIHAIuB37nqU2oaq778wDwFk4NpRNVfUhVc1Q1Jz09feBv\nwk9R4aHERISyM7+Si37zLn9bf5SDxTV+XRsd7tQwXt6WD8DGI+V84aGPuOg37/a7XKU1jbQqpNlM\nbmNMPwVylJQADwM7VfXervKo6kRVzVbVbOCfwDdV9TkRSRaRSPd10oClwI5AlXWgjE6M4sUt+TS1\nOHMx/B0xVevWJD4+Us7hkhqOlDqB5oCfAacnxdXOkiUWMIwx/RXIGsZS4Bpguc/w2AtF5AYRuaGX\na2cA60VkM7Aa+JmqnvQBY8qo+HbPj1fW+3VdbWMzp4yKA+CpdUe55e+bveda+zARsKCinvf2FrdL\nawsY1n9hjOmfgE3cU9X3AL+nD6vqV3yOPwDmBKBYAZWR0P5bfEGFvwGjhU9MSeFISS2/f2t/u3Mf\nHSwhJiKM+Vm990Fc8YcPOFpax667VhDlNnN5Rm2l2darxph+sqVBBlBiTPtv8QU91DAqapv42l/W\nsfloObWNLSTFhDMtM75Tvi/+cQ2feeB978543bnn9d0cLXWWLd+R3za2wFvDiLWAYYzpHwsYA+g/\nzprELZ+ayk3nTEGk5yapH/5rO2/sLOTFLXnUNDYTExHK7LGJ3eZfvavnaSq//fc+7/G23LbhuUVV\nDYSHCgnRtgqMMaZ/7FNkAMVGhvHtc6YAzof2nuNdb6zU3NLKm24AqKpvRhViIsKYPTbKmyczIapd\nDWVXQZXfE++2HmsLGLsKqpiUFmeLCxpj+s1qGAEye2wi+4uqqWlo7nRu09FyKuqcWd2HSpyRULGR\nocwe49Qwbl0xnd9fvbDdNZ58XaltdN7jeyum8cmp6Wx1axiqyuZj5X71fxhjTG8sYATInLGJqDr9\nCR/sK+bmpz5mz/Eqiqsb+Mf6YwBMTIvlcEkt4MzFmDUmgeuWZnPhnEzio9rvT3G4pJa39xTx2EeH\n26U3tbRy2o/fACA9LpK54xLZW1hNfVMLtz29lfLaJuZmdd/UZYwx/rImqQCZ4+7Et/VYBX9ff5Rd\nBVVMTIvjlW357CqoIiIshFljEnhxizNZLzYyjLDQEO68ZBYAhVVtzVFR4SHsyKvk2kfWAnDZwnFE\nu8uJ7DleRY07jyM9PpL4qHBaWpVdBVW8vDWftLhILp5zcm7qZIwZWqyGESAZCVGkx0fy1p4idhVU\nAZBfUec9bmxuJTSkrV9hdGJUu+sTfGoYSyenUdfUtkzIRwdLvMe+Hdzp8ZGMT4kB4IP9xVQ1NPOd\n86aSaLvpGWMGgAWMAJozNpF39jjrW4UI5FfUExnW9iv3fLg/dv1iFoxPbnetb74vLRnvPQ4NETYe\nbts/fKtPwMhMiGJscjQAv3h1N0CPI6+MMaYvLGAE0LJpztpW6fGRnDsjg7f3FNHQ7MynuPvyudx4\n9im8ecsnOXNKWqdrRYRvLHPWW5w9NpGffnYOT359CZkJUeSW1XnzbcutZFF2Ch/evpzUuEgSotpa\nGVNjI7qc22GMMSfC+jAC6JolE6hvamHRxFT+9O4Bb/r3VkzjipwsACanx3V7/a0rpnP9mRNJi4vk\ni4udWsbYpGiOldfR0qrc/swWNh0t5/ozJzI60alZ+A6fffE/zyQ81L4TGGMGhgWMABIRVp7l1BJ8\nF/87a4r/q+p2XDRwbHI0aw+W8vf1R/m7O9pqeje1CE8QMcaYgWABY5B85/xpXDR3NKdl928P7TFJ\nzoS+TUfKAfjElDTOnj6qXZ7V31lGdX3n+R/GGNMfFjAGSVxkWL+DBUBWcgwtrcrGI2XMGJ3A/12/\nuFOeie6GTMYYM5CsgXuImTvOmbW9t7C60+q4xhgTSBYwhphpmfHERToVw4z4qF5yG2PMwLGAMcSE\nhggLJzhzNkZZDcMYM4gCuUVrloisFpEdIrJdRG7qIe9pItIsIpf7pF0rInvdx7WBKudQdLY7v6Ox\nlz0yjDFmIAWy07sZuEVVN4pIPLBBRFZ13GpVREKBnwOv+6SlAHcCOYC6176gqmUYrlo0noKKer66\ndGKwi2KMGUECVsNQ1XxV3egeVwE7gbFdZP028DTgu0PQ+cAqVS11g8QqYEWgyjrURIWHcvuFM8hI\nsD4MY8zgGZQ+DBHJBhYAazqkjwU+C/y+wyVjgaM+z4/RdbAxxhgzSAIeMEQkDqcGcbOqVnY4/Wvg\nVlU94cZ4EVkpIutFZH1RUVF/imqMMaYHAZ24JyLhOMHicVV9possOcBT7vpHacCFItIM5ALLfPKN\nA97q6j1U9SHgIYCcnBwdqLIbY4xpL2ABQ5wo8DCwU1Xv7SqPqk70yf8o8KKqPud2ev9URDxrfp8H\n3B6oshpjjOldIGsYS4FrgK0isslNuwMYD6CqD3Z3oaqWishdwDo36UeqWhrAshpjjOlFwAKGqr4H\nSK8Z2/J/pcPzR4BHBrhYxhhjTpDN9DbGGOMXCxjGGGP8IqrDZ2CRiBQBh0/w8jSgeACLM1TYfY88\nI/Xe7b67NkFV/drVbVgFjP4QkfWqmhPscgw2u++RZ6Teu913/1mTlDHGGL9YwDDGGOMXCxhtHgp2\nAYLE7nvkGan3bvfdT9aHYYwxxi9WwzDGGOOXER8wRGSFiOwWkX0icluwyzPQROQRESkUkW0+aSki\nssrdzXCVZ80ucfzG/V1sEZGFwSt5/3S34+Nwv3cRiRKRtSKy2b3vH7rpE0VkjXt/fxORCDc90n2+\nzz2fHczy95eIhIrIxyLyovt82N+3iBwSka0isklE1rtpAfk7H9EBw93t7wHgAmAmcJWIzAxuqQbc\no3TefOo24E1VnQK86T4H5/cwxX2spPM+JUOJZ8fHmcAS4Eb333a433sDsFxV5wHzgRUisgRnV8tf\nqeopQBlwvZv/eqDMTf+Vm28ouwlnszaPkXLfZ6vqfJ/hs4H5O1fVEfsATgde83l+O3B7sMsVgPvM\nBrb5PN8NjHaPRwO73eM/AFd1lW+oP4DngU+NpHsHYoCNwGKciVthbrr37x54DTjdPQ5z80mwy36C\n9zvO/XBcDryIs5bdSLjvQ0Bah7SA/J2P6BoGI3dnvwxVzXePC4AM93hY/j467Pg47O/dbZbZhLPt\n8SpgP1Cuqs1uFt978963e74CSB3cEg+YXwPfAzwbsqUyMu5bgddFZIOIrHTTAvJ3HtANlMzJT1VV\nRIbtULmOOz66m3UBw/feVbUFmC8iScCzwPQgFyngRORioFBVN4jIsmCXZ5Cdqaq5IjIKWCUiu3xP\nDuTf+UivYeQCWT7Px7lpw91xERkN4P4sdNOH1e+jmx0fR8S9A6hqObAapykmSUQ8XxB978173+75\nREJP9RwAAAMbSURBVKBkkIs6EJYCnxaRQ8BTOM1S9zH87xtVzXV/FuJ8QVhEgP7OR3rAWAdMcUdS\nRABXAi8EuUyD4QXgWvf4Wpz2fU/6l92RFEuACp9q7ZAi0u2Oj8P63kUk3a1ZICLROP02O3ECx+Vu\nto737fl9XA78W93G7aFEVW9X1XGqmo3z//jfqvolhvl9i0isiMR7jnF2J91GoP7Og91hE+wHcCGw\nB6ed9/vBLk8A7u9JIB9owmmvvB6nrfZNYC/wBpDi5hWcUWP7ga1ATrDL34/7PhOnbXcLsMl9XDjc\n7x2YC3zs3vc24H/d9EnAWmAf8A8g0k2Pcp/vc89PCvY9DMDvYBnOds/D/r7d+9vsPrZ7PsMC9Xdu\nM72NMcb4ZaQ3SRljjPGTBQxjjDF+sYBhjDHGLxYwjDHG+MUChjHGGL9YwDCmFyLS4q4E6nkM2KrG\nIpItPisJG3Mys6VBjOldnarOD3YhjAk2q2EYc4LcfQh+4e5FsFZETnHTs0Xk3+5+A2+KyHg3PUNE\nnnX3qtgsIme4LxUqIn9096943Z2hjYj8pzj7eWwRkaeCdJvGeFnAMKZ30R2apL7gc65CVecA9+Os\nlgrwW+AvqjoXeBz4jZv+G+BtdfaqWIgzMxecvQkeUNVZQDlwmZt+G7DAfZ0bAnVzxvjLZnob0wsR\nqVbVuC7SD+FsVnTAXeiwQFVTRaQYZ4+BJjc9X1XTRKQIGKeqDT6vkQ2sUmejG0TkViBcVX8sIq8C\n1cBzwHOqWh3gWzWmR1bDMKZ/tJvjvmjwOW6hrW/xIpx1fxYC63xWXTUmKCxgGNM/X/D5+aF7/AHO\niqkAXwLedY/fBL4B3k2OErt7UREJAbJUdTVwK87y251qOcYMJvvGYkzvot0d7DxeVVXP0NpkEdmC\nU0u4yk37NvBnEfkuUARc56bfBDwkItfj1CS+gbOScFdCgcfcoCLAb9TZ38KYoLE+DGNOkNuHkaOq\nxcEuizGDwZqkjDHG+MVqGMYYY/xiNQxjjDF+sYBhjDHGLxYwjDHG+MUChjHGGL9YwDDGGOMXCxjG\nGGP88v8DYqN7v3gUkLQAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x7f5091305860>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"def smooth_curve(points, factor=0.9):\n",
|
|
" smoothed_points = []\n",
|
|
" for point in points:\n",
|
|
" if smoothed_points:\n",
|
|
" previous = smoothed_points[-1]\n",
|
|
" smoothed_points.append(previous * factor + point * (1 - factor))\n",
|
|
" else:\n",
|
|
" smoothed_points.append(point)\n",
|
|
" return smoothed_points\n",
|
|
"\n",
|
|
"smooth_mae_history = smooth_curve(average_mae_history[10:])\n",
|
|
"\n",
|
|
"plt.plot(range(1, len(smooth_mae_history) + 1), smooth_mae_history)\n",
|
|
"plt.xlabel('Epochs')\n",
|
|
"plt.ylabel('Validation MAE')\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"According to this plot, it seems that validation MAE stops improving significantly after 80 epochs. Past that point, we start overfitting.\n",
|
|
"\n",
|
|
"Once we are done tuning other parameters of our model (besides the number of epochs, we could also adjust the size of the hidden layers), we \n",
|
|
"can train a final \"production\" model on all of the training data, with the best parameters, then look at its performance on the test data:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 28,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\r",
|
|
" 32/102 [========>.....................] - ETA: 0s"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Get a fresh, compiled model.\n",
|
|
"model = build_model()\n",
|
|
"# Train it on the entirety of the data.\n",
|
|
"model.fit(train_data, train_targets,\n",
|
|
" epochs=80, batch_size=16, verbose=0)\n",
|
|
"test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 29,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"2.5532484335057877"
|
|
]
|
|
},
|
|
"execution_count": 29,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"test_mae_score"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We are still off by about \\$2,550."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Wrapping up\n",
|
|
"\n",
|
|
"\n",
|
|
"Here's what you should take away from this example:\n",
|
|
"\n",
|
|
"* Regression is done using different loss functions from classification; Mean Squared Error (MSE) is a commonly used loss function for \n",
|
|
"regression.\n",
|
|
"* Similarly, evaluation metrics to be used for regression differ from those used for classification; naturally the concept of \"accuracy\" \n",
|
|
"does not apply for regression. A common regression metric is Mean Absolute Error (MAE).\n",
|
|
"* When features in the input data have values in different ranges, each feature should be scaled independently as a preprocessing step.\n",
|
|
"* When there is little data available, using K-Fold validation is a great way to reliably evaluate a model.\n",
|
|
"* When little training data is available, it is preferable to use a small network with very few hidden layers (typically only one or two), \n",
|
|
"in order to avoid severe overfitting.\n",
|
|
"\n",
|
|
"This example concludes our series of three introductory practical examples. You are now able to handle common types of problems with vector data input:\n",
|
|
"\n",
|
|
"* Binary (2-class) classification.\n",
|
|
"* Multi-class, single-label classification.\n",
|
|
"* Scalar regression.\n",
|
|
"\n",
|
|
"In the next chapter, you will acquire a more formal understanding of some of the concepts you have encountered in these first examples, \n",
|
|
"such as data preprocessing, model evaluation, and overfitting."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.5.2"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|