wehub-resource-sync
1871 lines
89 KiB
Plaintext
1871 lines
89 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "986bcaab",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Running Tune experiments with BOHB\n",
|
||
"\n",
|
||
"<a id=\"try-anyscale-quickstart-ray-tune-bohb_example\" href=\"https://console.anyscale.com/register/ha?render_flow=ray&utm_source=ray_docs&utm_medium=docs&utm_campaign=ray-tune-bohb_example\">\n",
|
||
" <img src=\"../../_static/img/run-on-anyscale.svg\" alt=\"try-anyscale-quickstart\">\n",
|
||
"</a>\n",
|
||
"<br></br>\n",
|
||
"\n",
|
||
"In this tutorial we introduce BOHB, while running a simple Ray Tune experiment.\n",
|
||
"Tune’s Search Algorithms integrate with BOHB and, as a result,\n",
|
||
"allow you to seamlessly scale up a BOHB optimization\n",
|
||
"process - without sacrificing performance.\n",
|
||
"\n",
|
||
"Bayesian Optimization HyperBand (BOHB) combines the benefits of Bayesian optimization\n",
|
||
"together with Bandit-based methods (e.g. HyperBand). BOHB does not rely on\n",
|
||
"the gradient of the objective function,\n",
|
||
"but instead, learns from samples of the search space.\n",
|
||
"It is suitable for optimizing functions that are non-differentiable,\n",
|
||
"with many local minima, or even unknown but only testable.\n",
|
||
"Therefore, this approach belongs to the domain of\n",
|
||
"\"derivative-free optimization\" and \"black-box optimization\".\n",
|
||
"\n",
|
||
"In this example we minimize a simple objective to briefly demonstrate the usage of\n",
|
||
"BOHB with Ray Tune via `BOHBSearch`. It's useful to keep in mind that despite\n",
|
||
"the emphasis on machine learning experiments, Ray Tune optimizes any implicit\n",
|
||
"or explicit objective. Here we assume `ConfigSpace==0.4.18` and `hpbandster==0.7.4`\n",
|
||
"libraries are installed. To learn more, please refer to the\n",
|
||
"[BOHB website](https://github.com/automl/HpBandSter)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "d12bd979",
|
||
"metadata": {
|
||
"tags": [
|
||
"remove-cell"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Collecting ConfigSpace==0.4.18\n",
|
||
" Using cached ConfigSpace-0.4.18-cp37-cp37m-macosx_10_15_x86_64.whl\n",
|
||
"Requirement already satisfied: pyparsing in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from ConfigSpace==0.4.18) (2.4.7)\n",
|
||
"Requirement already satisfied: numpy in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from ConfigSpace==0.4.18) (1.21.6)\n",
|
||
"Requirement already satisfied: cython in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from ConfigSpace==0.4.18) (0.29.30)\n",
|
||
"Installing collected packages: ConfigSpace\n",
|
||
" Attempting uninstall: ConfigSpace\n",
|
||
" Found existing installation: ConfigSpace 0.4.21\n",
|
||
" Uninstalling ConfigSpace-0.4.21:\n",
|
||
" Successfully uninstalled ConfigSpace-0.4.21\n",
|
||
"Successfully installed ConfigSpace-0.4.18\n",
|
||
"\u001b[33mWARNING: There was an error checking the latest version of pip.\u001b[0m\u001b[33m\n",
|
||
"\u001b[0mRequirement already satisfied: hpbandster==0.7.4 in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (0.7.4)\n",
|
||
"Requirement already satisfied: netifaces in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from hpbandster==0.7.4) (0.11.0)\n",
|
||
"Requirement already satisfied: scipy in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from hpbandster==0.7.4) (1.4.1)\n",
|
||
"Requirement already satisfied: Pyro4 in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from hpbandster==0.7.4) (4.82)\n",
|
||
"Requirement already satisfied: serpent in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from hpbandster==0.7.4) (1.40)\n",
|
||
"Requirement already satisfied: statsmodels in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from hpbandster==0.7.4) (0.13.2)\n",
|
||
"Requirement already satisfied: ConfigSpace in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from hpbandster==0.7.4) (0.4.18)\n",
|
||
"Requirement already satisfied: numpy in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from hpbandster==0.7.4) (1.21.6)\n",
|
||
"Requirement already satisfied: cython in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from ConfigSpace->hpbandster==0.7.4) (0.29.30)\n",
|
||
"Requirement already satisfied: pyparsing in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from ConfigSpace->hpbandster==0.7.4) (2.4.7)\n",
|
||
"Requirement already satisfied: pandas>=0.25 in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from statsmodels->hpbandster==0.7.4) (1.3.5)\n",
|
||
"Requirement already satisfied: packaging>=21.3 in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from statsmodels->hpbandster==0.7.4) (21.3)\n",
|
||
"Requirement already satisfied: patsy>=0.5.2 in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from statsmodels->hpbandster==0.7.4) (0.5.2)\n",
|
||
"Requirement already satisfied: python-dateutil>=2.7.3 in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from pandas>=0.25->statsmodels->hpbandster==0.7.4) (2.8.2)\n",
|
||
"Requirement already satisfied: pytz>=2017.3 in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from pandas>=0.25->statsmodels->hpbandster==0.7.4) (2022.1)\n",
|
||
"Requirement already satisfied: six in ~/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from patsy>=0.5.2->statsmodels->hpbandster==0.7.4) (1.16.0)\n",
|
||
"\u001b[33mWARNING: There was an error checking the latest version of pip.\u001b[0m\u001b[33m\n",
|
||
"\u001b[0m"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"!pip install ray[tune]\n",
|
||
"!pip install ConfigSpace==0.4.18\n",
|
||
"!pip install hpbandster==0.7.4"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "96641e94",
|
||
"metadata": {},
|
||
"source": [
|
||
"Click below to see all the imports we need for this example."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "0e65ccdb",
|
||
"metadata": {
|
||
"tags": [
|
||
"hide-input"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"import tempfile\n",
|
||
"import time\n",
|
||
"from pathlib import Path\n",
|
||
"\n",
|
||
"import ray\n",
|
||
"from ray import tune\n",
|
||
"from ray.tune.schedulers.hb_bohb import HyperBandForBOHB\n",
|
||
"from ray.tune.search.bohb import TuneBOHB\n",
|
||
"import ConfigSpace as CS"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "edba942a",
|
||
"metadata": {},
|
||
"source": [
|
||
"Let's start by defining a simple evaluation function.\n",
|
||
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
|
||
"This setup assumes that we're running multiple `step`s of an experiment and try to tune\n",
|
||
"two hyperparameters, namely `width` and `height`, and `activation`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "af512205",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def evaluate(step, width, height, activation):\n",
|
||
" time.sleep(0.1)\n",
|
||
" activation_boost = 10 if activation==\"relu\" else 1\n",
|
||
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "c073ea21",
|
||
"metadata": {},
|
||
"source": [
|
||
"Next, our `objective` function takes a Tune `config`, evaluates the `score` of your\n",
|
||
"experiment in a training loop, and uses `tune.report` to report the `score` back to Tune.\n",
|
||
"\n",
|
||
"BOHB will interrupt our trials often, so we also need to {ref}`save and restore checkpoints <train-checkpointing>`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "8a086e87",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def objective(config):\n",
|
||
" start = 0\n",
|
||
" if tune.get_checkpoint():\n",
|
||
" with tune.get_checkpoint().as_directory() as checkpoint_dir:\n",
|
||
" start = int((Path(checkpoint_dir) / \"data.ckpt\").read_text())\n",
|
||
"\n",
|
||
" for step in range(start, config[\"steps\"]):\n",
|
||
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
|
||
" with tempfile.TemporaryDirectory() as checkpoint_dir:\n",
|
||
" (Path(checkpoint_dir) / \"data.ckpt\").write_text(str(step))\n",
|
||
" tune.report(\n",
|
||
" {\"iterations\": step, \"mean_loss\": score},\n",
|
||
" checkpoint=tune.Checkpoint.from_directory(checkpoint_dir)\n",
|
||
" )"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "05d07329",
|
||
"metadata": {
|
||
"tags": [
|
||
"remove-cell"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"ray.init(configure_logging=False)"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "32ee1ba7",
|
||
"metadata": {},
|
||
"source": [
|
||
"Next we define a search space. The critical assumption is that the optimal\n",
|
||
"hyperparameters live within this space. Yet, if the space is very large,\n",
|
||
"then those hyperparameters may be difficult to find in a short amount of time."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "21598e54",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"search_space = {\n",
|
||
" \"steps\": 100,\n",
|
||
" \"width\": tune.uniform(0, 20),\n",
|
||
" \"height\": tune.uniform(-100, 100),\n",
|
||
" \"activation\": tune.choice([\"relu\", \"tanh\"]),\n",
|
||
"}"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "def82932",
|
||
"metadata": {},
|
||
"source": [
|
||
"Next we define the search algorithm built from `TuneBOHB`, constrained\n",
|
||
"to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`.\n",
|
||
"Below `algo` will take care of the BO (Bayesian optimization) part of BOHB,\n",
|
||
"while scheduler will take care the HB (HyperBand) part."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "e847b5b6",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"algo = TuneBOHB()\n",
|
||
"algo = tune.search.ConcurrencyLimiter(algo, max_concurrent=4)\n",
|
||
"scheduler = HyperBandForBOHB(\n",
|
||
" time_attr=\"training_iteration\",\n",
|
||
" max_t=100,\n",
|
||
" reduction_factor=4,\n",
|
||
" stop_last_trials=False,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "1787a842",
|
||
"metadata": {},
|
||
"source": [
|
||
"The number of samples is the number of hyperparameter combinations\n",
|
||
"that will be tried out. This Tune run is set to `1000` samples.\n",
|
||
"(you can decrease this if it takes too long on your machine)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "702eb3d4",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"num_samples = 1000"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "dfb3ecad",
|
||
"metadata": {
|
||
"pycharm": {
|
||
"name": "#%%\n"
|
||
},
|
||
"tags": [
|
||
"remove-cell"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"num_samples = 10"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "aa5936df",
|
||
"metadata": {},
|
||
"source": [
|
||
"Finally, we run the experiment to `min`imize the \"mean_loss\" of the `objective`\n",
|
||
"by searching within `\"steps\": 100` via `algo`, `num_samples` times. This previous\n",
|
||
"sentence is fully characterizes the search problem we aim to solve.\n",
|
||
"With this in mind, notice how efficient it is to execute `tuner.fit()`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "4bdfb12d",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"== Status ==<br>Current time: 2022-07-22 15:07:54 (running for 00:00:26.41)<br>Memory usage on this node: 9.9/16.0 GiB<br>Using HyperBand: num_stopped=9 total_brackets=1\n",
|
||
"Round #0:\n",
|
||
" Bracket(Max Size (n)=1, Milestone (r)=64, completed=66.8%): {TERMINATED: 10} <br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.95 GiB heap, 0.0/2.0 GiB objects<br>Current best trial: a0c11456 with mean_loss=-4.53376204004117 and parameters={'steps': 100, 'width': 3.7250202606878258, 'height': -57.97769618290691, 'activation': 'tanh'}<br>Result logdir: ~/ray_results/bohb_exp<br>Number of trials: 10/10 (10 TERMINATED)<br><table>\n",
|
||
"<thead>\n",
|
||
"<tr><th>Trial name </th><th>status </th><th>loc </th><th>activation </th><th style=\"text-align: right;\"> height</th><th style=\"text-align: right;\"> width</th><th style=\"text-align: right;\"> loss</th><th style=\"text-align: right;\"> iter</th><th style=\"text-align: right;\"> total time (s)</th><th style=\"text-align: right;\"> ts</th><th style=\"text-align: right;\"> iterations</th><th style=\"text-align: right;\"> neg_mean_loss</th></tr>\n",
|
||
"</thead>\n",
|
||
"<tbody>\n",
|
||
"<tr><td>objective_9e8d8b06</td><td>TERMINATED</td><td>127.0.0.1:45117</td><td>tanh </td><td style=\"text-align: right;\"> 37.6516 </td><td style=\"text-align: right;\">12.2188 </td><td style=\"text-align: right;\"> 5.28254</td><td style=\"text-align: right;\"> 16</td><td style=\"text-align: right;\"> 2.23943 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 15</td><td style=\"text-align: right;\"> -5.28254</td></tr>\n",
|
||
"<tr><td>objective_a052a214</td><td>TERMINATED</td><td>127.0.0.1:45150</td><td>relu </td><td style=\"text-align: right;\"> -4.10627</td><td style=\"text-align: right;\">17.9931 </td><td style=\"text-align: right;\">11.1524 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.531915</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -11.1524 </td></tr>\n",
|
||
"<tr><td>objective_a06180d6</td><td>TERMINATED</td><td>127.0.0.1:45151</td><td>tanh </td><td style=\"text-align: right;\"> 89.5711 </td><td style=\"text-align: right;\"> 8.05512</td><td style=\"text-align: right;\">12.884 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.534212</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -12.884 </td></tr>\n",
|
||
"<tr><td>objective_a077899e</td><td>TERMINATED</td><td>127.0.0.1:45152</td><td>relu </td><td style=\"text-align: right;\"> 67.3538 </td><td style=\"text-align: right;\">13.6388 </td><td style=\"text-align: right;\">18.6994 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.538702</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -18.6994 </td></tr>\n",
|
||
"<tr><td>objective_a0865c76</td><td>TERMINATED</td><td>127.0.0.1:45153</td><td>relu </td><td style=\"text-align: right;\"> 25.9876 </td><td style=\"text-align: right;\"> 9.57103</td><td style=\"text-align: right;\">15.1819 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.559531</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -15.1819 </td></tr>\n",
|
||
"<tr><td>objective_a0a42d1e</td><td>TERMINATED</td><td>127.0.0.1:45154</td><td>relu </td><td style=\"text-align: right;\"> 80.9133 </td><td style=\"text-align: right;\">13.0972 </td><td style=\"text-align: right;\">20.1201 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.538588</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -20.1201 </td></tr>\n",
|
||
"<tr><td>objective_a0c11456</td><td>TERMINATED</td><td>127.0.0.1:45117</td><td>tanh </td><td style=\"text-align: right;\">-57.9777 </td><td style=\"text-align: right;\"> 3.72502</td><td style=\"text-align: right;\">-4.53376</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 13.0563 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 4.53376</td></tr>\n",
|
||
"<tr><td>objective_a0dce442</td><td>TERMINATED</td><td>127.0.0.1:45200</td><td>tanh </td><td style=\"text-align: right;\"> -1.68715</td><td style=\"text-align: right;\"> 1.87185</td><td style=\"text-align: right;\"> 3.4575 </td><td style=\"text-align: right;\"> 16</td><td style=\"text-align: right;\"> 2.27704 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 15</td><td style=\"text-align: right;\"> -3.4575 </td></tr>\n",
|
||
"<tr><td>objective_a0f84156</td><td>TERMINATED</td><td>127.0.0.1:45157</td><td>tanh </td><td style=\"text-align: right;\"> 65.9879 </td><td style=\"text-align: right;\"> 5.41575</td><td style=\"text-align: right;\">11.4087 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.548028</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -11.4087 </td></tr>\n",
|
||
"<tr><td>objective_a1142416</td><td>TERMINATED</td><td>127.0.0.1:45201</td><td>tanh </td><td style=\"text-align: right;\"> 5.35569</td><td style=\"text-align: right;\"> 4.85644</td><td style=\"text-align: right;\"> 2.74262</td><td style=\"text-align: right;\"> 16</td><td style=\"text-align: right;\"> 2.27749 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 15</td><td style=\"text-align: right;\"> -2.74262</td></tr>\n",
|
||
"</tbody>\n",
|
||
"</table><br><br>"
|
||
],
|
||
"text/plain": [
|
||
"<IPython.core.display.HTML object>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_9e8d8b06:\n",
|
||
" date: 2022-07-22_15-07-31\n",
|
||
" done: false\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.765164520733162\n",
|
||
" neg_mean_loss: -14.765164520733162\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10084700584411621\n",
|
||
" time_this_iter_s: 0.10084700584411621\n",
|
||
" time_total_s: 0.10084700584411621\n",
|
||
" timestamp: 1658498851\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 9e8d8b06\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a06180d6:\n",
|
||
" date: 2022-07-22_15-07-31\n",
|
||
" done: false\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 19.957107852020386\n",
|
||
" neg_mean_loss: -19.957107852020386\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10333395004272461\n",
|
||
" time_this_iter_s: 0.10333395004272461\n",
|
||
" time_total_s: 0.10333395004272461\n",
|
||
" timestamp: 1658498851\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a06180d6\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a0a42d1e:\n",
|
||
" date: 2022-07-22_15-07-31\n",
|
||
" done: false\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 28.091327935768636\n",
|
||
" neg_mean_loss: -28.091327935768636\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10419368743896484\n",
|
||
" time_this_iter_s: 0.10419368743896484\n",
|
||
" time_total_s: 0.10419368743896484\n",
|
||
" timestamp: 1658498851\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0a42d1e\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a0c11456:\n",
|
||
" date: 2022-07-22_15-07-31\n",
|
||
" done: false\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 5.202230381709309\n",
|
||
" neg_mean_loss: -5.202230381709309\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10060286521911621\n",
|
||
" time_this_iter_s: 0.10060286521911621\n",
|
||
" time_total_s: 0.10060286521911621\n",
|
||
" timestamp: 1658498851\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0c11456\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a0dce442:\n",
|
||
" date: 2022-07-22_15-07-32\n",
|
||
" done: false\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 10.831285273000042\n",
|
||
" neg_mean_loss: -10.831285273000042\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10183191299438477\n",
|
||
" time_this_iter_s: 0.10183191299438477\n",
|
||
" time_total_s: 0.10183191299438477\n",
|
||
" timestamp: 1658498852\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0dce442\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a0f84156:\n",
|
||
" date: 2022-07-22_15-07-32\n",
|
||
" done: false\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 17.598785421495204\n",
|
||
" neg_mean_loss: -17.598785421495204\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10328507423400879\n",
|
||
" time_this_iter_s: 0.10328507423400879\n",
|
||
" time_total_s: 0.10328507423400879\n",
|
||
" timestamp: 1658498852\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0f84156\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a1142416:\n",
|
||
" date: 2022-07-22_15-07-32\n",
|
||
" done: false\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 11.53556906389516\n",
|
||
" neg_mean_loss: -11.53556906389516\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10475611686706543\n",
|
||
" time_this_iter_s: 0.10475611686706543\n",
|
||
" time_total_s: 0.10475611686706543\n",
|
||
" timestamp: 1658498852\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a1142416\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a052a214:\n",
|
||
" date: 2022-07-22_15-07-34\n",
|
||
" done: false\n",
|
||
" experiment_id: 0fb96edfd1b34561b337e7146a3c64aa\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 19.5893732640325\n",
|
||
" neg_mean_loss: -19.5893732640325\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45126\n",
|
||
" time_since_restore: 0.10437989234924316\n",
|
||
" time_this_iter_s: 0.10437989234924316\n",
|
||
" time_total_s: 0.10437989234924316\n",
|
||
" timestamp: 1658498854\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a052a214\n",
|
||
" warmup_time: 0.003122091293334961\n",
|
||
" \n",
|
||
"Result for objective_a077899e:\n",
|
||
" date: 2022-07-22_15-07-34\n",
|
||
" done: false\n",
|
||
" experiment_id: 008f8a44fb0c4475a3e9dad32f1bd2c9\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 26.735381559909975\n",
|
||
" neg_mean_loss: -26.735381559909975\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45129\n",
|
||
" time_since_restore: 0.10449695587158203\n",
|
||
" time_this_iter_s: 0.10449695587158203\n",
|
||
" time_total_s: 0.10449695587158203\n",
|
||
" timestamp: 1658498854\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a077899e\n",
|
||
" warmup_time: 0.0031490325927734375\n",
|
||
" \n",
|
||
"Result for objective_a0865c76:\n",
|
||
" date: 2022-07-22_15-07-34\n",
|
||
" done: false\n",
|
||
" experiment_id: 06a194a4b7784ca5932a66e44e3cd815\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 22.598763488408313\n",
|
||
" neg_mean_loss: -22.598763488408313\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45132\n",
|
||
" time_since_restore: 0.10512685775756836\n",
|
||
" time_this_iter_s: 0.10512685775756836\n",
|
||
" time_total_s: 0.10512685775756836\n",
|
||
" timestamp: 1658498854\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0865c76\n",
|
||
" warmup_time: 0.00397801399230957\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45117)\u001b[0m 2022-07-22 15:07:34,468\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_9e8d8b06_1_activation=tanh,height=37.6516,steps=100,width=12.2188_2022-07-22_15-07-28/checkpoint_tmpf674c6\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45117)\u001b[0m 2022-07-22 15:07:34,469\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10084700584411621, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45154)\u001b[0m 2022-07-22 15:07:37,958\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0a42d1e_6_activation=relu,height=80.9133,steps=100,width=13.0972_2022-07-22_15-07-31/checkpoint_tmp30fafa\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45154)\u001b[0m 2022-07-22 15:07:37,958\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10419368743896484, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45155)\u001b[0m 2022-07-22 15:07:37,952\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0c11456_7_activation=tanh,height=-57.9777,steps=100,width=3.7250_2022-07-22_15-07-31/checkpoint_tmp7c225a\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45155)\u001b[0m 2022-07-22 15:07:37,952\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10060286521911621, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45151)\u001b[0m 2022-07-22 15:07:37,933\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a06180d6_3_activation=tanh,height=89.5711,steps=100,width=8.0551_2022-07-22_15-07-31/checkpoint_tmpcde58c\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45151)\u001b[0m 2022-07-22 15:07:37,933\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10333395004272461, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45150)\u001b[0m 2022-07-22 15:07:37,929\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a052a214_2_activation=relu,height=-4.1063,steps=100,width=17.9931_2022-07-22_15-07-31/checkpoint_tmp24072a\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45150)\u001b[0m 2022-07-22 15:07:37,929\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10437989234924316, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45153)\u001b[0m 2022-07-22 15:07:38,010\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0865c76_5_activation=relu,height=25.9876,steps=100,width=9.5710_2022-07-22_15-07-31/checkpoint_tmpad3367\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45153)\u001b[0m 2022-07-22 15:07:38,010\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10512685775756836, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45158)\u001b[0m 2022-07-22 15:07:38,010\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a1142416_10_activation=tanh,height=5.3557,steps=100,width=4.8564_2022-07-22_15-07-32/checkpoint_tmpe18b15\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45158)\u001b[0m 2022-07-22 15:07:38,010\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10475611686706543, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45156)\u001b[0m 2022-07-22 15:07:38,008\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0dce442_8_activation=tanh,height=-1.6871,steps=100,width=1.8718_2022-07-22_15-07-32/checkpoint_tmp01ad4f\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45156)\u001b[0m 2022-07-22 15:07:38,008\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10183191299438477, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45157)\u001b[0m 2022-07-22 15:07:38,025\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0f84156_9_activation=tanh,height=65.9879,steps=100,width=5.4158_2022-07-22_15-07-32/checkpoint_tmp57fedd\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45157)\u001b[0m 2022-07-22 15:07:38,025\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10328507423400879, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45152)\u001b[0m 2022-07-22 15:07:38,022\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a077899e_4_activation=relu,height=67.3538,steps=100,width=13.6388_2022-07-22_15-07-31/checkpoint_tmpfdb6b4\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45152)\u001b[0m 2022-07-22 15:07:38,022\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10449695587158203, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_a06180d6:\n",
|
||
" date: 2022-07-22_15-07-38\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 19.957107852020386\n",
|
||
" neg_mean_loss: -19.957107852020386\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45151\n",
|
||
" time_since_restore: 0.10456109046936035\n",
|
||
" time_this_iter_s: 0.10456109046936035\n",
|
||
" time_total_s: 0.20789504051208496\n",
|
||
" timestamp: 1658498858\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a06180d6\n",
|
||
" warmup_time: 0.010712862014770508\n",
|
||
" \n",
|
||
"Result for objective_a0c11456:\n",
|
||
" date: 2022-07-22_15-07-38\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 5.202230381709309\n",
|
||
" neg_mean_loss: -5.202230381709309\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45155\n",
|
||
" time_since_restore: 0.10466408729553223\n",
|
||
" time_this_iter_s: 0.10466408729553223\n",
|
||
" time_total_s: 0.20526695251464844\n",
|
||
" timestamp: 1658498858\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0c11456\n",
|
||
" warmup_time: 0.007361888885498047\n",
|
||
" \n",
|
||
"Result for objective_a0a42d1e:\n",
|
||
" date: 2022-07-22_15-07-38\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 28.091327935768636\n",
|
||
" neg_mean_loss: -28.091327935768636\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45154\n",
|
||
" time_since_restore: 0.10032892227172852\n",
|
||
" time_this_iter_s: 0.10032892227172852\n",
|
||
" time_total_s: 0.20452260971069336\n",
|
||
" timestamp: 1658498858\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0a42d1e\n",
|
||
" warmup_time: 0.008682966232299805\n",
|
||
" \n",
|
||
"Result for objective_a0dce442:\n",
|
||
" date: 2022-07-22_15-07-38\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 10.831285273000042\n",
|
||
" neg_mean_loss: -10.831285273000042\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45156\n",
|
||
" time_since_restore: 0.10120797157287598\n",
|
||
" time_this_iter_s: 0.10120797157287598\n",
|
||
" time_total_s: 0.20303988456726074\n",
|
||
" timestamp: 1658498858\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0dce442\n",
|
||
" warmup_time: 0.0077381134033203125\n",
|
||
" \n",
|
||
"Result for objective_a1142416:\n",
|
||
" date: 2022-07-22_15-07-38\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 11.53556906389516\n",
|
||
" neg_mean_loss: -11.53556906389516\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45158\n",
|
||
" time_since_restore: 0.10349392890930176\n",
|
||
" time_this_iter_s: 0.10349392890930176\n",
|
||
" time_total_s: 0.2082500457763672\n",
|
||
" timestamp: 1658498858\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a1142416\n",
|
||
" warmup_time: 0.007157087326049805\n",
|
||
" \n",
|
||
"Result for objective_a0f84156:\n",
|
||
" date: 2022-07-22_15-07-38\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 17.598785421495204\n",
|
||
" neg_mean_loss: -17.598785421495204\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45157\n",
|
||
" time_since_restore: 0.10368680953979492\n",
|
||
" time_this_iter_s: 0.10368680953979492\n",
|
||
" time_total_s: 0.2069718837738037\n",
|
||
" timestamp: 1658498858\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: a0f84156\n",
|
||
" warmup_time: 0.006359100341796875\n",
|
||
" \n",
|
||
"Result for objective_a052a214:\n",
|
||
" date: 2022-07-22_15-07-38\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 0fb96edfd1b34561b337e7146a3c64aa\n",
|
||
" experiment_tag: 2_activation=relu,height=-4.1063,steps=100,width=17.9931\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 3\n",
|
||
" iterations_since_restore: 4\n",
|
||
" mean_loss: 11.152378612292932\n",
|
||
" neg_mean_loss: -11.152378612292932\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45150\n",
|
||
" time_since_restore: 0.4275352954864502\n",
|
||
" time_this_iter_s: 0.1124732494354248\n",
|
||
" time_total_s: 0.5319151878356934\n",
|
||
" timestamp: 1658498858\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 4\n",
|
||
" trial_id: a052a214\n",
|
||
" warmup_time: 0.010692834854125977\n",
|
||
" \n",
|
||
"Result for objective_9e8d8b06:\n",
|
||
" date: 2022-07-22_15-07-39\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.765164520733162\n",
|
||
" neg_mean_loss: -14.765164520733162\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 0.10145044326782227\n",
|
||
" time_this_iter_s: 0.10145044326782227\n",
|
||
" time_total_s: 0.6308252811431885\n",
|
||
" timestamp: 1658498859\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 9e8d8b06\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45117)\u001b[0m 2022-07-22 15:07:39,222\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_9e8d8b06_1_activation=tanh,height=37.6516,steps=100,width=12.2188_2022-07-22_15-07-28/checkpoint_tmpacb5ff\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45117)\u001b[0m 2022-07-22 15:07:39,223\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.5293748378753662, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45199)\u001b[0m 2022-07-22 15:07:41,833\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0c11456_7_activation=tanh,height=-57.9777,steps=100,width=3.7250_2022-07-22_15-07-31/checkpoint_tmpdf2071\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45199)\u001b[0m 2022-07-22 15:07:41,833\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.53450608253479, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45200)\u001b[0m 2022-07-22 15:07:41,833\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0dce442_8_activation=tanh,height=-1.6871,steps=100,width=1.8718_2022-07-22_15-07-32/checkpoint_tmp9cd0fc\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45200)\u001b[0m 2022-07-22 15:07:41,833\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.5349018573760986, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45201)\u001b[0m 2022-07-22 15:07:41,882\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a1142416_10_activation=tanh,height=5.3557,steps=100,width=4.8564_2022-07-22_15-07-32/checkpoint_tmp7bd297\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45201)\u001b[0m 2022-07-22 15:07:41,882\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.5445291996002197, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_a0c11456:\n",
|
||
" date: 2022-07-22_15-07-43\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 11\n",
|
||
" iterations_since_restore: 12\n",
|
||
" mean_loss: -2.8360322412948014\n",
|
||
" neg_mean_loss: 2.8360322412948014\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45199\n",
|
||
" time_since_restore: 1.3028512001037598\n",
|
||
" time_this_iter_s: 0.10884499549865723\n",
|
||
" time_total_s: 1.8373572826385498\n",
|
||
" timestamp: 1658498863\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 12\n",
|
||
" trial_id: a0c11456\n",
|
||
" warmup_time: 0.0074689388275146484\n",
|
||
" \n",
|
||
"Result for objective_a0dce442:\n",
|
||
" date: 2022-07-22_15-07-43\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 11\n",
|
||
" iterations_since_restore: 12\n",
|
||
" mean_loss: 4.100295137851358\n",
|
||
" neg_mean_loss: -4.100295137851358\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45200\n",
|
||
" time_since_restore: 1.3082969188690186\n",
|
||
" time_this_iter_s: 0.10707712173461914\n",
|
||
" time_total_s: 1.8431987762451172\n",
|
||
" timestamp: 1658498863\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 12\n",
|
||
" trial_id: a0dce442\n",
|
||
" warmup_time: 0.007519960403442383\n",
|
||
" \n",
|
||
"Result for objective_a1142416:\n",
|
||
" date: 2022-07-22_15-07-43\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 11\n",
|
||
" iterations_since_restore: 12\n",
|
||
" mean_loss: 3.112339173810433\n",
|
||
" neg_mean_loss: -3.112339173810433\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45201\n",
|
||
" time_since_restore: 1.2954580783843994\n",
|
||
" time_this_iter_s: 0.1081690788269043\n",
|
||
" time_total_s: 1.8399872779846191\n",
|
||
" timestamp: 1658498863\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 12\n",
|
||
" trial_id: a1142416\n",
|
||
" warmup_time: 0.006086826324462891\n",
|
||
" \n",
|
||
"Result for objective_a1142416:\n",
|
||
" date: 2022-07-22_15-07-43\n",
|
||
" done: true\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 15\n",
|
||
" iterations_since_restore: 16\n",
|
||
" mean_loss: 2.7426202715773025\n",
|
||
" neg_mean_loss: -2.7426202715773025\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45201\n",
|
||
" time_since_restore: 1.7329609394073486\n",
|
||
" time_this_iter_s: 0.1072549819946289\n",
|
||
" time_total_s: 2.2774901390075684\n",
|
||
" timestamp: 1658498863\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 16\n",
|
||
" trial_id: a1142416\n",
|
||
" warmup_time: 0.006086826324462891\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45117)\u001b[0m 2022-07-22 15:07:43,765\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp/objective_a0c11456_7_activation=tanh,height=-57.9777,steps=100,width=3.7250_2022-07-22_15-07-31/checkpoint_tmp3f09eb\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45117)\u001b[0m 2022-07-22 15:07:43,765\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 2.273958206176758, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_a0c11456:\n",
|
||
" date: 2022-07-22_15-07-48\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 40\n",
|
||
" iterations_since_restore: 41\n",
|
||
" mean_loss: -4.168842006342518\n",
|
||
" neg_mean_loss: 4.168842006342518\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 4.4259912967681885\n",
|
||
" time_this_iter_s: 0.10874629020690918\n",
|
||
" time_total_s: 6.699949502944946\n",
|
||
" timestamp: 1658498868\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 41\n",
|
||
" trial_id: a0c11456\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a0c11456:\n",
|
||
" date: 2022-07-22_15-07-53\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 87\n",
|
||
" iterations_since_restore: 88\n",
|
||
" mean_loss: -4.498437215767879\n",
|
||
" neg_mean_loss: 4.498437215767879\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 9.488422155380249\n",
|
||
" time_this_iter_s: 0.10667800903320312\n",
|
||
" time_total_s: 11.762380361557007\n",
|
||
" timestamp: 1658498873\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 88\n",
|
||
" trial_id: a0c11456\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n",
|
||
"Result for objective_a0c11456:\n",
|
||
" date: 2022-07-22_15-07-54\n",
|
||
" done: true\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: c1cb9895c3f04e73b7cce9435cd92c68\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 99\n",
|
||
" iterations_since_restore: 100\n",
|
||
" mean_loss: -4.53376204004117\n",
|
||
" neg_mean_loss: 4.53376204004117\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45117\n",
|
||
" time_since_restore: 10.782363176345825\n",
|
||
" time_this_iter_s: 0.1065070629119873\n",
|
||
" time_total_s: 13.056321382522583\n",
|
||
" timestamp: 1658498874\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 100\n",
|
||
" trial_id: a0c11456\n",
|
||
" warmup_time: 0.005752086639404297\n",
|
||
" \n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"tuner = tune.Tuner(\n",
|
||
" objective,\n",
|
||
" tune_config=tune.TuneConfig(\n",
|
||
" metric=\"mean_loss\",\n",
|
||
" mode=\"min\",\n",
|
||
" search_alg=algo,\n",
|
||
" scheduler=scheduler,\n",
|
||
" num_samples=num_samples,\n",
|
||
" ),\n",
|
||
" run_config=tune.RunConfig(\n",
|
||
" name=\"bohb_exp\",\n",
|
||
" stop={\"training_iteration\": 100},\n",
|
||
" ),\n",
|
||
" param_space=search_space,\n",
|
||
")\n",
|
||
"results = tuner.fit()"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "3e89853c",
|
||
"metadata": {},
|
||
"source": [
|
||
"Here are the hyperparameters found to minimize the mean loss of the defined objective."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "4be691d5",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Best hyperparameters found were: {'steps': 100, 'width': 3.7250202606878258, 'height': -57.97769618290691, 'activation': 'tanh'}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(\"Best hyperparameters found were: \", results.get_best_result().config)"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "800a19d9",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Optional: Passing the search space via the TuneBOHB algorithm\n",
|
||
"\n",
|
||
"We can define the hyperparameter search space using `ConfigSpace`,\n",
|
||
"which is the format accepted by BOHB."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "b96cb496",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"~/.pyenv/versions/3.7.7/lib/python3.7/site-packages/ipykernel_launcher.py:3: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
|
||
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
|
||
" This is separate from the ipykernel package so we can avoid doing imports until\n",
|
||
"~/.pyenv/versions/3.7.7/lib/python3.7/site-packages/ipykernel_launcher.py:6: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
|
||
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
|
||
" \n",
|
||
"~/.pyenv/versions/3.7.7/lib/python3.7/site-packages/ipykernel_launcher.py:9: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
|
||
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
|
||
" if __name__ == \"__main__\":\n",
|
||
"~/.pyenv/versions/3.7.7/lib/python3.7/site-packages/ipykernel_launcher.py:13: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
|
||
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
|
||
" del sys.path[0]\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"activation, Type: Categorical, Choices: {relu, tanh}, Default: relu"
|
||
]
|
||
},
|
||
"execution_count": 12,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"config_space = CS.ConfigurationSpace()\n",
|
||
"config_space.add_hyperparameter(\n",
|
||
" CS.Constant(\"steps\", 100)\n",
|
||
")\n",
|
||
"config_space.add_hyperparameter(\n",
|
||
" CS.UniformFloatHyperparameter(\"width\", lower=0, upper=20)\n",
|
||
")\n",
|
||
"config_space.add_hyperparameter(\n",
|
||
" CS.UniformFloatHyperparameter(\"height\", lower=-100, upper=100)\n",
|
||
")\n",
|
||
"config_space.add_hyperparameter(\n",
|
||
" CS.CategoricalHyperparameter(\n",
|
||
" \"activation\", choices=[\"relu\", \"tanh\"]\n",
|
||
" )\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "9cb77270",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# As we are passing config space directly to the searcher,\n",
|
||
"# we need to define metric and mode in it as well, in addition\n",
|
||
"# to Tuner()\n",
|
||
"algo = TuneBOHB(\n",
|
||
" space=config_space,\n",
|
||
" metric=\"mean_loss\",\n",
|
||
" mode=\"max\",\n",
|
||
")\n",
|
||
"algo = tune.search.ConcurrencyLimiter(algo, max_concurrent=4)\n",
|
||
"scheduler = HyperBandForBOHB(\n",
|
||
" time_attr=\"training_iteration\",\n",
|
||
" max_t=100,\n",
|
||
" reduction_factor=4,\n",
|
||
" stop_last_trials=False,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "8305c975",
|
||
"metadata": {
|
||
"pycharm": {
|
||
"name": "#%%\n"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"== Status ==<br>Current time: 2022-07-22 15:11:40 (running for 00:00:29.52)<br>Memory usage on this node: 10.3/16.0 GiB<br>Using HyperBand: num_stopped=9 total_brackets=1\n",
|
||
"Round #0:\n",
|
||
" Bracket(Max Size (n)=1, Milestone (r)=64, completed=66.8%): {TERMINATED: 10} <br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.95 GiB heap, 0.0/2.0 GiB objects<br>Current best trial: 25b64488 with mean_loss=-3.74634537130406 and parameters={'activation': 'tanh', 'height': -48.451797714080236, 'steps': 100, 'width': 10.119125894538891}<br>Result logdir: ~/ray_results/bohb_exp_2<br>Number of trials: 10/10 (10 TERMINATED)<br><table>\n",
|
||
"<thead>\n",
|
||
"<tr><th>Trial name </th><th>status </th><th>loc </th><th>activation </th><th style=\"text-align: right;\"> height</th><th style=\"text-align: right;\"> steps</th><th style=\"text-align: right;\"> width</th><th style=\"text-align: right;\"> loss</th><th style=\"text-align: right;\"> iter</th><th style=\"text-align: right;\"> total time (s)</th><th style=\"text-align: right;\"> ts</th><th style=\"text-align: right;\"> iterations</th><th style=\"text-align: right;\"> neg_mean_loss</th></tr>\n",
|
||
"</thead>\n",
|
||
"<tbody>\n",
|
||
"<tr><td>objective_2397442c</td><td>TERMINATED</td><td>127.0.0.1:45401</td><td>tanh </td><td style=\"text-align: right;\"> 32.8422</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">12.1847 </td><td style=\"text-align: right;\"> 4.80297</td><td style=\"text-align: right;\"> 16</td><td style=\"text-align: right;\"> 2.25951 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 15</td><td style=\"text-align: right;\"> -4.80297</td></tr>\n",
|
||
"<tr><td>objective_25b4a998</td><td>TERMINATED</td><td>127.0.0.1:45401</td><td>relu </td><td style=\"text-align: right;\"> 20.2852</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 2.08202 </td><td style=\"text-align: right;\">18.1839 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.535426</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -18.1839 </td></tr>\n",
|
||
"<tr><td>objective_25b64488</td><td>TERMINATED</td><td>127.0.0.1:45453</td><td>tanh </td><td style=\"text-align: right;\">-48.4518</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">10.1191 </td><td style=\"text-align: right;\">-3.74635</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.5319 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 3.74635</td></tr>\n",
|
||
"<tr><td>objective_25b7dfe6</td><td>TERMINATED</td><td>127.0.0.1:45403</td><td>relu </td><td style=\"text-align: right;\">-18.8439</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">19.1277 </td><td style=\"text-align: right;\"> 9.59966</td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.581903</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -9.59966</td></tr>\n",
|
||
"<tr><td>objective_25cfab4e</td><td>TERMINATED</td><td>127.0.0.1:45404</td><td>relu </td><td style=\"text-align: right;\"> 17.2057</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 0.317083</td><td style=\"text-align: right;\">20.8519 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.59468 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -20.8519 </td></tr>\n",
|
||
"<tr><td>objective_278eba4c</td><td>TERMINATED</td><td>127.0.0.1:45454</td><td>relu </td><td style=\"text-align: right;\">-27.0179</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">13.577 </td><td style=\"text-align: right;\"> 7.76626</td><td style=\"text-align: right;\"> 16</td><td style=\"text-align: right;\"> 2.31198 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 15</td><td style=\"text-align: right;\"> -7.76626</td></tr>\n",
|
||
"<tr><td>objective_279d01a6</td><td>TERMINATED</td><td>127.0.0.1:45407</td><td>relu </td><td style=\"text-align: right;\"> 59.1103</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 2.4466 </td><td style=\"text-align: right;\">21.6781 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.575556</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -21.6781 </td></tr>\n",
|
||
"<tr><td>objective_27aa31e6</td><td>TERMINATED</td><td>127.0.0.1:45409</td><td>relu </td><td style=\"text-align: right;\"> 50.058 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">17.3776 </td><td style=\"text-align: right;\">16.6153 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.537561</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -16.6153 </td></tr>\n",
|
||
"<tr><td>objective_27b7e2be</td><td>TERMINATED</td><td>127.0.0.1:45455</td><td>relu </td><td style=\"text-align: right;\">-51.2093</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 8.94948 </td><td style=\"text-align: right;\"> 5.57235</td><td style=\"text-align: right;\"> 16</td><td style=\"text-align: right;\"> 2.64238 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 15</td><td style=\"text-align: right;\"> -5.57235</td></tr>\n",
|
||
"<tr><td>objective_27c59a80</td><td>TERMINATED</td><td>127.0.0.1:45446</td><td>relu </td><td style=\"text-align: right;\"> 29.165 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 4.26995 </td><td style=\"text-align: right;\">17.3006 </td><td style=\"text-align: right;\"> 4</td><td style=\"text-align: right;\"> 0.539177</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> -17.3006 </td></tr>\n",
|
||
"</tbody>\n",
|
||
"</table><br><br>"
|
||
],
|
||
"text/plain": [
|
||
"<IPython.core.display.HTML object>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_2397442c:\n",
|
||
" date: 2022-07-22_15-11-15\n",
|
||
" done: false\n",
|
||
" experiment_id: 1a4ebf62df50443492dc6df792fcb67a\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.284216630043918\n",
|
||
" neg_mean_loss: -14.284216630043918\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45353\n",
|
||
" time_since_restore: 0.10108494758605957\n",
|
||
" time_this_iter_s: 0.10108494758605957\n",
|
||
" time_total_s: 0.10108494758605957\n",
|
||
" timestamp: 1658499075\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 2397442c\n",
|
||
" warmup_time: 0.002635955810546875\n",
|
||
" \n",
|
||
"Result for objective_25b7dfe6:\n",
|
||
" date: 2022-07-22_15-11-17\n",
|
||
" done: false\n",
|
||
" experiment_id: 0fd6607aa9674cb5a05b6ce63e474fd3\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 18.115606120430186\n",
|
||
" neg_mean_loss: -18.115606120430186\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45371\n",
|
||
" time_since_restore: 0.10365676879882812\n",
|
||
" time_this_iter_s: 0.10365676879882812\n",
|
||
" time_total_s: 0.10365676879882812\n",
|
||
" timestamp: 1658499077\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25b7dfe6\n",
|
||
" warmup_time: 0.005063056945800781\n",
|
||
" \n",
|
||
"Result for objective_25b4a998:\n",
|
||
" date: 2022-07-22_15-11-17\n",
|
||
" done: false\n",
|
||
" experiment_id: 20a5d76dc18749e4b1c9f15c5d8b43cf\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 22.028519616352035\n",
|
||
" neg_mean_loss: -22.028519616352035\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45369\n",
|
||
" time_since_restore: 0.10431909561157227\n",
|
||
" time_this_iter_s: 0.10431909561157227\n",
|
||
" time_total_s: 0.10431909561157227\n",
|
||
" timestamp: 1658499077\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25b4a998\n",
|
||
" warmup_time: 0.004379987716674805\n",
|
||
" \n",
|
||
"Result for objective_25b64488:\n",
|
||
" date: 2022-07-22_15-11-17\n",
|
||
" done: false\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 6.154820228591976\n",
|
||
" neg_mean_loss: -6.154820228591976\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45370\n",
|
||
" time_since_restore: 0.10407018661499023\n",
|
||
" time_this_iter_s: 0.10407018661499023\n",
|
||
" time_total_s: 0.10407018661499023\n",
|
||
" timestamp: 1658499077\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25b64488\n",
|
||
" warmup_time: 0.003113985061645508\n",
|
||
" \n",
|
||
"Result for objective_25cfab4e:\n",
|
||
" date: 2022-07-22_15-11-18\n",
|
||
" done: false\n",
|
||
" experiment_id: fdc43ca37ed44cde857ca150a8f1e84f\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 21.72056884033249\n",
|
||
" neg_mean_loss: -21.72056884033249\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45378\n",
|
||
" time_since_restore: 0.10431408882141113\n",
|
||
" time_this_iter_s: 0.10431408882141113\n",
|
||
" time_total_s: 0.10431408882141113\n",
|
||
" timestamp: 1658499078\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25cfab4e\n",
|
||
" warmup_time: 0.0029649734497070312\n",
|
||
" \n",
|
||
"Result for objective_279d01a6:\n",
|
||
" date: 2022-07-22_15-11-18\n",
|
||
" done: false\n",
|
||
" experiment_id: 20a5d76dc18749e4b1c9f15c5d8b43cf\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 25.911032294938884\n",
|
||
" neg_mean_loss: -25.911032294938884\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45369\n",
|
||
" time_since_restore: 0.10361599922180176\n",
|
||
" time_this_iter_s: 0.10361599922180176\n",
|
||
" time_total_s: 0.10361599922180176\n",
|
||
" timestamp: 1658499078\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 279d01a6\n",
|
||
" warmup_time: 0.004379987716674805\n",
|
||
" \n",
|
||
"Result for objective_27aa31e6:\n",
|
||
" date: 2022-07-22_15-11-18\n",
|
||
" done: false\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 25.005798678308594\n",
|
||
" neg_mean_loss: -25.005798678308594\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45370\n",
|
||
" time_since_restore: 0.10430693626403809\n",
|
||
" time_this_iter_s: 0.10430693626403809\n",
|
||
" time_total_s: 0.10430693626403809\n",
|
||
" timestamp: 1658499078\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 27aa31e6\n",
|
||
" warmup_time: 0.003113985061645508\n",
|
||
" \n",
|
||
"Result for objective_27b7e2be:\n",
|
||
" date: 2022-07-22_15-11-18\n",
|
||
" done: false\n",
|
||
" experiment_id: fdc43ca37ed44cde857ca150a8f1e84f\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.879072389639937\n",
|
||
" neg_mean_loss: -14.879072389639937\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45378\n",
|
||
" time_since_restore: 0.10371994972229004\n",
|
||
" time_this_iter_s: 0.10371994972229004\n",
|
||
" time_total_s: 0.10371994972229004\n",
|
||
" timestamp: 1658499078\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 27b7e2be\n",
|
||
" warmup_time: 0.0029649734497070312\n",
|
||
" \n",
|
||
"Result for objective_27c59a80:\n",
|
||
" date: 2022-07-22_15-11-18\n",
|
||
" done: false\n",
|
||
" experiment_id: 20a5d76dc18749e4b1c9f15c5d8b43cf\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 22.916501000037474\n",
|
||
" neg_mean_loss: -22.916501000037474\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45369\n",
|
||
" time_since_restore: 0.10353732109069824\n",
|
||
" time_this_iter_s: 0.10353732109069824\n",
|
||
" time_total_s: 0.10353732109069824\n",
|
||
" timestamp: 1658499078\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 27c59a80\n",
|
||
" warmup_time: 0.004379987716674805\n",
|
||
" \n",
|
||
"Result for objective_278eba4c:\n",
|
||
" date: 2022-07-22_15-11-20\n",
|
||
" done: false\n",
|
||
" experiment_id: 90186993d7ff42698c4615640d47d896\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 17.29821256734647\n",
|
||
" neg_mean_loss: -17.29821256734647\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45393\n",
|
||
" time_since_restore: 0.10304784774780273\n",
|
||
" time_this_iter_s: 0.10304784774780273\n",
|
||
" time_total_s: 0.10304784774780273\n",
|
||
" timestamp: 1658499080\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 278eba4c\n",
|
||
" warmup_time: 0.0027189254760742188\n",
|
||
" \n",
|
||
"Result for objective_2397442c:\n",
|
||
" date: 2022-07-22_15-11-20\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 1a4ebf62df50443492dc6df792fcb67a\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.284216630043918\n",
|
||
" neg_mean_loss: -14.284216630043918\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45370\n",
|
||
" time_since_restore: 0.10411787033081055\n",
|
||
" time_this_iter_s: 0.10411787033081055\n",
|
||
" time_total_s: 0.20520281791687012\n",
|
||
" timestamp: 1658499080\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 2397442c\n",
|
||
" warmup_time: 0.003113985061645508\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45370)\u001b[0m 2022-07-22 15:11:20,826\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_2397442c_1_activation=tanh,height=32.8422,steps=100,width=12.1847_2022-07-22_15-11-11/checkpoint_tmpf4b290\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45370)\u001b[0m 2022-07-22 15:11:20,826\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10108494758605957, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_25b4a998:\n",
|
||
" date: 2022-07-22_15-11-24\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 20a5d76dc18749e4b1c9f15c5d8b43cf\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 22.028519616352035\n",
|
||
" neg_mean_loss: -22.028519616352035\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45401\n",
|
||
" time_since_restore: 0.10445284843444824\n",
|
||
" time_this_iter_s: 0.10445284843444824\n",
|
||
" time_total_s: 0.2087719440460205\n",
|
||
" timestamp: 1658499084\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25b4a998\n",
|
||
" warmup_time: 0.010488033294677734\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45405)\u001b[0m 2022-07-22 15:11:23,963\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_278eba4c_6_activation=relu,height=-27.0179,steps=100,width=13.5770_2022-07-22_15-11-18/checkpoint_tmpd366c5\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45405)\u001b[0m 2022-07-22 15:11:23,964\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10304784774780273, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45401)\u001b[0m 2022-07-22 15:11:23,959\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_25b4a998_2_activation=relu,height=20.2852,steps=100,width=2.0820_2022-07-22_15-11-14/checkpoint_tmpc96cdb\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45401)\u001b[0m 2022-07-22 15:11:23,959\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10431909561157227, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45402)\u001b[0m 2022-07-22 15:11:23,966\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_25b64488_3_activation=tanh,height=-48.4518,steps=100,width=10.1191_2022-07-22_15-11-14/checkpoint_tmp5ce175\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45402)\u001b[0m 2022-07-22 15:11:23,966\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10407018661499023, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45407)\u001b[0m 2022-07-22 15:11:23,966\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_279d01a6_7_activation=relu,height=59.1103,steps=100,width=2.4466_2022-07-22_15-11-18/checkpoint_tmp942eac\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45407)\u001b[0m 2022-07-22 15:11:23,966\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10361599922180176, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45404)\u001b[0m 2022-07-22 15:11:23,960\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_25cfab4e_5_activation=relu,height=17.2057,steps=100,width=0.3171_2022-07-22_15-11-15/checkpoint_tmpc8ceee\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45404)\u001b[0m 2022-07-22 15:11:23,960\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10431408882141113, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45403)\u001b[0m 2022-07-22 15:11:23,966\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_25b7dfe6_4_activation=relu,height=-18.8439,steps=100,width=19.1277_2022-07-22_15-11-14/checkpoint_tmp24108b\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45403)\u001b[0m 2022-07-22 15:11:23,966\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10365676879882812, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45409)\u001b[0m 2022-07-22 15:11:23,997\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_27aa31e6_8_activation=relu,height=50.0580,steps=100,width=17.3776_2022-07-22_15-11-18/checkpoint_tmp4218e5\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45409)\u001b[0m 2022-07-22 15:11:23,997\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10430693626403809, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_27aa31e6:\n",
|
||
" date: 2022-07-22_15-11-24\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 25.005798678308594\n",
|
||
" neg_mean_loss: -25.005798678308594\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45409\n",
|
||
" time_since_restore: 0.10203695297241211\n",
|
||
" time_this_iter_s: 0.10203695297241211\n",
|
||
" time_total_s: 0.2063438892364502\n",
|
||
" timestamp: 1658499084\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 27aa31e6\n",
|
||
" warmup_time: 0.0062160491943359375\n",
|
||
" \n",
|
||
"Result for objective_279d01a6:\n",
|
||
" date: 2022-07-22_15-11-24\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 20a5d76dc18749e4b1c9f15c5d8b43cf\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 25.911032294938884\n",
|
||
" neg_mean_loss: -25.911032294938884\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45407\n",
|
||
" time_since_restore: 0.10443115234375\n",
|
||
" time_this_iter_s: 0.10443115234375\n",
|
||
" time_total_s: 0.20804715156555176\n",
|
||
" timestamp: 1658499084\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 279d01a6\n",
|
||
" warmup_time: 0.010073184967041016\n",
|
||
" \n",
|
||
"Result for objective_25b7dfe6:\n",
|
||
" date: 2022-07-22_15-11-24\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 0fd6607aa9674cb5a05b6ce63e474fd3\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 18.115606120430186\n",
|
||
" neg_mean_loss: -18.115606120430186\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45403\n",
|
||
" time_since_restore: 0.1044917106628418\n",
|
||
" time_this_iter_s: 0.1044917106628418\n",
|
||
" time_total_s: 0.20814847946166992\n",
|
||
" timestamp: 1658499084\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25b7dfe6\n",
|
||
" warmup_time: 0.011971235275268555\n",
|
||
" \n",
|
||
"Result for objective_25cfab4e:\n",
|
||
" date: 2022-07-22_15-11-24\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: fdc43ca37ed44cde857ca150a8f1e84f\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 21.72056884033249\n",
|
||
" neg_mean_loss: -21.72056884033249\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45404\n",
|
||
" time_since_restore: 0.10379600524902344\n",
|
||
" time_this_iter_s: 0.10379600524902344\n",
|
||
" time_total_s: 0.20811009407043457\n",
|
||
" timestamp: 1658499084\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25cfab4e\n",
|
||
" warmup_time: 0.009023904800415039\n",
|
||
" \n",
|
||
"Result for objective_25b64488:\n",
|
||
" date: 2022-07-22_15-11-24\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 6.154820228591976\n",
|
||
" neg_mean_loss: -6.154820228591976\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45402\n",
|
||
" time_since_restore: 0.10440897941589355\n",
|
||
" time_this_iter_s: 0.10440897941589355\n",
|
||
" time_total_s: 0.2084791660308838\n",
|
||
" timestamp: 1658499084\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25b64488\n",
|
||
" warmup_time: 0.011686325073242188\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45424)\u001b[0m 2022-07-22 15:11:24,383\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_27b7e2be_9_activation=relu,height=-51.2093,steps=100,width=8.9495_2022-07-22_15-11-18/checkpoint_tmp996dec\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45424)\u001b[0m 2022-07-22 15:11:24,384\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10371994972229004, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_27b7e2be:\n",
|
||
" date: 2022-07-22_15-11-24\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: fdc43ca37ed44cde857ca150a8f1e84f\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.879072389639937\n",
|
||
" neg_mean_loss: -14.879072389639937\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45424\n",
|
||
" time_since_restore: 0.1031639575958252\n",
|
||
" time_this_iter_s: 0.1031639575958252\n",
|
||
" time_total_s: 0.20688390731811523\n",
|
||
" timestamp: 1658499084\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 27b7e2be\n",
|
||
" warmup_time: 0.0069200992584228516\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45446)\u001b[0m 2022-07-22 15:11:26,749\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_27c59a80_10_activation=relu,height=29.1650,steps=100,width=4.2700_2022-07-22_15-11-18/checkpoint_tmp49d063\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45446)\u001b[0m 2022-07-22 15:11:26,749\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.10353732109069824, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_27c59a80:\n",
|
||
" date: 2022-07-22_15-11-26\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 20a5d76dc18749e4b1c9f15c5d8b43cf\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 22.916501000037474\n",
|
||
" neg_mean_loss: -22.916501000037474\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45446\n",
|
||
" time_since_restore: 0.10502910614013672\n",
|
||
" time_this_iter_s: 0.10502910614013672\n",
|
||
" time_total_s: 0.20856642723083496\n",
|
||
" timestamp: 1658499086\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 27c59a80\n",
|
||
" warmup_time: 0.007359027862548828\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45401)\u001b[0m 2022-07-22 15:11:27,287\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_2397442c_1_activation=tanh,height=32.8422,steps=100,width=12.1847_2022-07-22_15-11-11/checkpoint_tmp2e0d09\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45401)\u001b[0m 2022-07-22 15:11:27,287\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.535153865814209, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_2397442c:\n",
|
||
" date: 2022-07-22_15-11-27\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 1a4ebf62df50443492dc6df792fcb67a\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.284216630043918\n",
|
||
" neg_mean_loss: -14.284216630043918\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45401\n",
|
||
" time_since_restore: 0.1044008731842041\n",
|
||
" time_this_iter_s: 0.1044008731842041\n",
|
||
" time_total_s: 0.6395547389984131\n",
|
||
" timestamp: 1658499087\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 2397442c\n",
|
||
" warmup_time: 0.010488033294677734\n",
|
||
" \n",
|
||
"Result for objective_278eba4c:\n",
|
||
" date: 2022-07-22_15-11-29\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 90186993d7ff42698c4615640d47d896\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 17.29821256734647\n",
|
||
" neg_mean_loss: -17.29821256734647\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45454\n",
|
||
" time_since_restore: 0.1037449836730957\n",
|
||
" time_this_iter_s: 0.1037449836730957\n",
|
||
" time_total_s: 0.6844677925109863\n",
|
||
" timestamp: 1658499089\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 278eba4c\n",
|
||
" warmup_time: 0.006754875183105469\n",
|
||
" \n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\u001b[2m\u001b[36m(objective pid=45454)\u001b[0m 2022-07-22 15:11:29,879\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_278eba4c_6_activation=relu,height=-27.0179,steps=100,width=13.5770_2022-07-22_15-11-18/checkpoint_tmp2835d4\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45454)\u001b[0m 2022-07-22 15:11:29,879\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.5807228088378906, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45455)\u001b[0m 2022-07-22 15:11:29,909\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_27b7e2be_9_activation=relu,height=-51.2093,steps=100,width=8.9495_2022-07-22_15-11-18/checkpoint_tmpd7ea63\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45455)\u001b[0m 2022-07-22 15:11:29,910\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.9150340557098389, '_episodes_total': 0}\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45453)\u001b[0m 2022-07-22 15:11:29,930\tINFO trainable.py:655 -- Restored on 127.0.0.1 from checkpoint: ~/ray_results/bohb_exp_2/objective_25b64488_3_activation=tanh,height=-48.4518,steps=100,width=10.1191_2022-07-22_15-11-14/checkpoint_tmp11824e\n",
|
||
"\u001b[2m\u001b[36m(objective pid=45453)\u001b[0m 2022-07-22 15:11:29,930\tINFO trainable.py:663 -- Current state after restoring: {'_iteration': 0, '_timesteps_total': 0, '_time_total': 0.5960800647735596, '_episodes_total': 0}\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Result for objective_27b7e2be:\n",
|
||
" date: 2022-07-22_15-11-30\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: fdc43ca37ed44cde857ca150a8f1e84f\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 14.879072389639937\n",
|
||
" neg_mean_loss: -14.879072389639937\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45455\n",
|
||
" time_since_restore: 0.10332393646240234\n",
|
||
" time_this_iter_s: 0.10332393646240234\n",
|
||
" time_total_s: 1.0183579921722412\n",
|
||
" timestamp: 1658499090\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 27b7e2be\n",
|
||
" warmup_time: 0.006285190582275391\n",
|
||
" \n",
|
||
"Result for objective_25b64488:\n",
|
||
" date: 2022-07-22_15-11-30\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 0\n",
|
||
" iterations_since_restore: 1\n",
|
||
" mean_loss: 6.154820228591976\n",
|
||
" neg_mean_loss: -6.154820228591976\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45453\n",
|
||
" time_since_restore: 0.1026160717010498\n",
|
||
" time_this_iter_s: 0.1026160717010498\n",
|
||
" time_total_s: 0.6986961364746094\n",
|
||
" timestamp: 1658499090\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 1\n",
|
||
" trial_id: 25b64488\n",
|
||
" warmup_time: 0.006460905075073242\n",
|
||
" \n",
|
||
"Result for objective_25b64488:\n",
|
||
" date: 2022-07-22_15-11-35\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 46\n",
|
||
" iterations_since_restore: 47\n",
|
||
" mean_loss: -3.634865890857194\n",
|
||
" neg_mean_loss: 3.634865890857194\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45453\n",
|
||
" time_since_restore: 5.1935131549835205\n",
|
||
" time_this_iter_s: 0.10786604881286621\n",
|
||
" time_total_s: 5.78959321975708\n",
|
||
" timestamp: 1658499095\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 47\n",
|
||
" trial_id: 25b64488\n",
|
||
" warmup_time: 0.006460905075073242\n",
|
||
" \n",
|
||
"Result for objective_25b64488:\n",
|
||
" date: 2022-07-22_15-11-40\n",
|
||
" done: false\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 93\n",
|
||
" iterations_since_restore: 94\n",
|
||
" mean_loss: -3.740036002402735\n",
|
||
" neg_mean_loss: 3.740036002402735\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45453\n",
|
||
" time_since_restore: 10.256795167922974\n",
|
||
" time_this_iter_s: 0.10682511329650879\n",
|
||
" time_total_s: 10.852875232696533\n",
|
||
" timestamp: 1658499100\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 94\n",
|
||
" trial_id: 25b64488\n",
|
||
" warmup_time: 0.006460905075073242\n",
|
||
" \n",
|
||
"Result for objective_25b64488:\n",
|
||
" date: 2022-07-22_15-11-40\n",
|
||
" done: true\n",
|
||
" episodes_total: 0\n",
|
||
" experiment_id: 75e7c1ad20a2495cac29630df6c3c782\n",
|
||
" hostname: Kais-MacBook-Pro.local\n",
|
||
" iterations: 99\n",
|
||
" iterations_since_restore: 100\n",
|
||
" mean_loss: -3.74634537130406\n",
|
||
" neg_mean_loss: 3.74634537130406\n",
|
||
" node_ip: 127.0.0.1\n",
|
||
" pid: 45453\n",
|
||
" time_since_restore: 10.935801029205322\n",
|
||
" time_this_iter_s: 0.10489487648010254\n",
|
||
" time_total_s: 11.531881093978882\n",
|
||
" timestamp: 1658499100\n",
|
||
" timesteps_since_restore: 0\n",
|
||
" timesteps_total: 0\n",
|
||
" training_iteration: 100\n",
|
||
" trial_id: 25b64488\n",
|
||
" warmup_time: 0.006460905075073242\n",
|
||
" \n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"tuner = tune.Tuner(\n",
|
||
" objective,\n",
|
||
" tune_config=tune.TuneConfig(\n",
|
||
" metric=\"mean_loss\",\n",
|
||
" mode=\"min\",\n",
|
||
" search_alg=algo,\n",
|
||
" scheduler=scheduler,\n",
|
||
" num_samples=num_samples,\n",
|
||
" ),\n",
|
||
" run_config=tune.RunConfig(\n",
|
||
" name=\"bohb_exp_2\",\n",
|
||
" stop={\"training_iteration\": 100},\n",
|
||
" ),\n",
|
||
")\n",
|
||
"results = tuner.fit()"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "5bf0dd87",
|
||
"metadata": {},
|
||
"source": [
|
||
"Here again are the hyperparameters found to minimize the mean loss of the\n",
|
||
"defined objective."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 18,
|
||
"id": "1ae613e4",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Best hyperparameters found were: {'activation': 'tanh', 'height': -48.451797714080236, 'steps': 100, 'width': 10.119125894538891}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(\"Best hyperparameters found were: \", results.get_best_result().config)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 19,
|
||
"id": "6b83ef6d",
|
||
"metadata": {
|
||
"tags": [
|
||
"remove-cell"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"ray.shutdown()"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.7.7"
|
||
},
|
||
"orphan": true,
|
||
"vscode": {
|
||
"interpreter": {
|
||
"hash": "3c0d54d489a08ae47a06eae2fd00ff032d6cddb527c382959b7b2575f6a8167f"
|
||
}
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|