wehub-resource-sync
593b94c120
pytest / Unit Tests (push) Waiting to run
pytest / Integration (integration_tests_a) (push) Waiting to run
pytest / Integration (integration_tests_b) (push) Waiting to run
pytest / Integration (integration_tests_c) (push) Waiting to run
pytest / Integration (integration_tests_d) (push) Waiting to run
pytest / Integration (integration_tests_e) (push) Waiting to run
pytest / Integration (integration_tests_f) (push) Waiting to run
pytest / Integration (integration_tests_g) (push) Waiting to run
pytest / Integration (integration_tests_h) (push) Waiting to run
pytest / Integration (integration_tests_i) (push) Waiting to run
pytest / Integration (integration_tests_j) (push) Waiting to run
pytest / Distributed (distributed_a) (push) Waiting to run
pytest / Distributed (distributed_b) (push) Waiting to run
pytest / Distributed (distributed_c) (push) Waiting to run
pytest / Distributed (distributed_d) (push) Waiting to run
pytest / Distributed (distributed_e) (push) Waiting to run
pytest / Distributed (distributed_f) (push) Waiting to run
pytest / Minimal Install (push) Waiting to run
pytest / Event File (push) Waiting to run
pytest (slow) / py-slow (push) Waiting to run
Publish JSON Schema / publish-schema (push) Waiting to run
428 lines
16 KiB
Plaintext
428 lines
16 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0001-0000-0000-000000000001",
|
|
"metadata": {},
|
|
"source": [
|
|
"# HyperNetworkCombiner: Conditional Feature Processing\n",
|
|
"\n",
|
|
"[](https://colab.research.google.com/github/ludwig-ai/ludwig/blob/main/examples/hypernetwork/hypernetwork.ipynb)\n",
|
|
"\n",
|
|
"> **Note:** This notebook requires **Ludwig >= 0.14** (PR #4092). The `hypernetwork`\n",
|
|
"> combiner type is not available in earlier releases. Install with:\n",
|
|
"> `pip install \"ludwig>=0.14\"`\n",
|
|
"\n",
|
|
"This notebook demonstrates the `HyperNetworkCombiner`, which lets one feature\n",
|
|
"(**the conditioning feature**) generate the weights of the layers that process all\n",
|
|
"other features — rather than simply concatenating everyone together.\n",
|
|
"\n",
|
|
"Based on **HyperFusion** ([arXiv 2403.13319](https://arxiv.org/abs/2403.13319), 2024).\n",
|
|
"\n",
|
|
"**What we cover:**\n",
|
|
"\n",
|
|
"1. Why concatenation is not always enough\n",
|
|
"2. Generating a synthetic multi-modal sensor dataset\n",
|
|
"3. Baseline: concat combiner\n",
|
|
"4. HyperNetworkCombiner\n",
|
|
"5. Comparing results and understanding why hypernetwork wins"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a1b2c3d4-0002-0000-0000-000000000002",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install \"ludwig>=0.14\" --quiet"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0003-0000-0000-000000000003",
|
|
"metadata": {},
|
|
"source": [
|
|
"## The problem: context-dependent features\n",
|
|
"\n",
|
|
"Imagine a network of industrial sensors. Each sensor reports three readings\n",
|
|
"(`sensor_a`, `sensor_b`, `sensor_c`), but every sensor belongs to one of three\n",
|
|
"measurement types: **temperature**, **pressure**, or **humidity**.\n",
|
|
"\n",
|
|
"The catch: **the same numerical reading means something completely different**\n",
|
|
"depending on the sensor type.\n",
|
|
"\n",
|
|
"- For a **temperature** sensor, `sensor_a = 3.0` is an anomaly (overheating).\n",
|
|
"- For a **pressure** sensor, `sensor_a = 3.0` is perfectly normal.\n",
|
|
"- For a **humidity** sensor, the anomaly rule involves the *sum* of all three readings.\n",
|
|
"\n",
|
|
"A concat combiner encodes all four features independently and then stitches them\n",
|
|
"together. The network has to learn — **after** the concatenation — to undo the mixing\n",
|
|
"and apply type-specific logic. This is hard because the critical signal (`sensor_type`)\n",
|
|
"is buried in a shared representation alongside the numerical readings.\n",
|
|
"\n",
|
|
"The `hypernetwork` combiner solves this directly:\n",
|
|
"\n",
|
|
"```\n",
|
|
"sensor_type ──► HyperNetwork ──► generates weights W, b\n",
|
|
" │\n",
|
|
"sensor_a ────────────────────► FC(W, b) ──►\n",
|
|
"sensor_b ────────────────────► FC(W, b) ──► combined repr.\n",
|
|
"sensor_c ────────────────────► FC(W, b) ──►\n",
|
|
"```\n",
|
|
"\n",
|
|
"`sensor_type` does not contribute a feature vector — it **rewrites the entire\n",
|
|
"transformation** applied to the numerical sensors."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0004-0000-0000-000000000004",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Dataset\n",
|
|
"\n",
|
|
"We generate a synthetic dataset with three sensor types. Each type has its own\n",
|
|
"normal operating range and its own anomaly rule, making the sensor type an\n",
|
|
"essential piece of context for correct classification."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a1b2c3d4-0005-0000-0000-000000000005",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import pandas as pd\n",
|
|
"\n",
|
|
"RNG = np.random.default_rng(42)\n",
|
|
"\n",
|
|
"N_PER_TYPE = 600\n",
|
|
"SENSOR_TYPES = [\"temperature\", \"pressure\", \"humidity\"]\n",
|
|
"\n",
|
|
"\n",
|
|
"def make_samples(sensor_type: str, n: int, rng: np.random.Generator) -> pd.DataFrame:\n",
|
|
" \"\"\"Generate n samples for a single sensor type with type-specific anomaly rules.\"\"\"\n",
|
|
" if sensor_type == \"temperature\":\n",
|
|
" # Normal: readings near 0; anomaly: sensor_a > 2.5 (overheating)\n",
|
|
" sensor_a = rng.normal(0.0, 1.0, n)\n",
|
|
" sensor_b = rng.normal(0.0, 1.0, n)\n",
|
|
" sensor_c = rng.normal(0.0, 1.0, n)\n",
|
|
" anomaly = (sensor_a > 2.5).astype(int)\n",
|
|
" elif sensor_type == \"pressure\":\n",
|
|
" # Normal: readings near 1; anomaly: sensor_b drops below -0.5 (leak)\n",
|
|
" sensor_a = rng.normal(1.0, 0.8, n)\n",
|
|
" sensor_b = rng.normal(1.0, 0.8, n)\n",
|
|
" sensor_c = rng.normal(1.0, 0.8, n)\n",
|
|
" anomaly = (sensor_b < -0.5).astype(int)\n",
|
|
" else: # humidity\n",
|
|
" # Normal: readings near -1; anomaly: combined level exceeds threshold\n",
|
|
" sensor_a = rng.normal(-1.0, 0.9, n)\n",
|
|
" sensor_b = rng.normal(-1.0, 0.9, n)\n",
|
|
" sensor_c = rng.normal(-1.0, 0.9, n)\n",
|
|
" anomaly = ((sensor_a + sensor_b + sensor_c) > 0).astype(int)\n",
|
|
"\n",
|
|
" return pd.DataFrame(\n",
|
|
" {\n",
|
|
" \"sensor_a\": sensor_a,\n",
|
|
" \"sensor_b\": sensor_b,\n",
|
|
" \"sensor_c\": sensor_c,\n",
|
|
" \"sensor_type\": sensor_type,\n",
|
|
" \"anomaly\": anomaly,\n",
|
|
" }\n",
|
|
" )\n",
|
|
"\n",
|
|
"\n",
|
|
"frames = [make_samples(t, N_PER_TYPE, RNG) for t in SENSOR_TYPES]\n",
|
|
"df = pd.concat(frames, ignore_index=True).sample(frac=1, random_state=42).reset_index(drop=True)\n",
|
|
"\n",
|
|
"# Train / validation / test split (70 / 15 / 15)\n",
|
|
"n = len(df)\n",
|
|
"split = np.full(n, 2, dtype=int) # default: test\n",
|
|
"idx = np.arange(n)\n",
|
|
"RNG.shuffle(idx)\n",
|
|
"split[idx[: int(0.70 * n)]] = 0\n",
|
|
"split[idx[int(0.70 * n) : int(0.85 * n)]] = 1\n",
|
|
"df[\"split\"] = split\n",
|
|
"\n",
|
|
"print(f\"Total rows: {n}\")\n",
|
|
"print(f\"Overall anomaly rate: {df['anomaly'].mean():.1%}\")\n",
|
|
"print()\n",
|
|
"print(\"Anomaly rate by sensor type:\")\n",
|
|
"print(df.groupby(\"sensor_type\")[\"anomaly\"].mean().rename(\"anomaly_rate\"))\n",
|
|
"print()\n",
|
|
"df.head(10)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0006-0000-0000-000000000006",
|
|
"metadata": {},
|
|
"source": [
|
|
"Notice that the three sensor types have **different anomaly rates** and, more\n",
|
|
"importantly, the anomaly rules are structurally different. The same value of\n",
|
|
"`sensor_a = 3.0` triggers an anomaly for `temperature` but not for `pressure`.\n",
|
|
"A model that treats all features symmetrically will struggle with this."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0007-0000-0000-000000000007",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Baseline: concat combiner\n",
|
|
"\n",
|
|
"The default Ludwig combiner concatenates all encoder outputs and passes them\n",
|
|
"through fully-connected layers. `sensor_type` is just another input — its\n",
|
|
"embedding is concatenated alongside the numerical sensor values."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a1b2c3d4-0008-0000-0000-000000000008",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import yaml\n",
|
|
"\n",
|
|
"from ludwig.api import LudwigModel\n",
|
|
"\n",
|
|
"config_concat_str = \"\"\"\n",
|
|
"model_type: ecd\n",
|
|
"\n",
|
|
"input_features:\n",
|
|
" - name: sensor_a\n",
|
|
" type: number\n",
|
|
" preprocessing:\n",
|
|
" normalization: zscore\n",
|
|
" - name: sensor_b\n",
|
|
" type: number\n",
|
|
" preprocessing:\n",
|
|
" normalization: zscore\n",
|
|
" - name: sensor_c\n",
|
|
" type: number\n",
|
|
" preprocessing:\n",
|
|
" normalization: zscore\n",
|
|
" - name: sensor_type\n",
|
|
" type: category\n",
|
|
"\n",
|
|
"output_features:\n",
|
|
" - name: anomaly\n",
|
|
" type: binary\n",
|
|
"\n",
|
|
"combiner:\n",
|
|
" type: concat\n",
|
|
" fc_layers:\n",
|
|
" - output_size: 128\n",
|
|
" - output_size: 64\n",
|
|
"\n",
|
|
"trainer:\n",
|
|
" epochs: 30\n",
|
|
" learning_rate: 0.001\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"config_concat = yaml.safe_load(config_concat_str)\n",
|
|
"\n",
|
|
"model_concat = LudwigModel(config_concat, logging_level=30)\n",
|
|
"model_concat.train(dataset=df)\n",
|
|
"\n",
|
|
"test_df = df[df[\"split\"] == 2].copy()\n",
|
|
"preds_concat, _ = model_concat.predict(dataset=test_df)\n",
|
|
"\n",
|
|
"acc_concat = (preds_concat[\"anomaly_predictions\"].values == test_df[\"anomaly\"].values).mean()\n",
|
|
"print(f\"Concat combiner — test accuracy: {acc_concat:.4f}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0009-0000-0000-000000000009",
|
|
"metadata": {},
|
|
"source": [
|
|
"## HyperNetworkCombiner\n",
|
|
"\n",
|
|
"Now we switch to `type: hypernetwork`. The combiner reads `sensor_type` (the last\n",
|
|
"feature listed in `input_features`) through a hyper-network and uses the output to\n",
|
|
"generate the weight matrix and bias of the layer that processes `sensor_a`,\n",
|
|
"`sensor_b`, and `sensor_c`.\n",
|
|
"\n",
|
|
"Key parameters:\n",
|
|
"\n",
|
|
"| Parameter | Role |\n",
|
|
"|---|---|\n",
|
|
"| `hidden_size` | Size of the main processing layer |\n",
|
|
"| `hyper_hidden_size` | Hidden size of the hyper-network itself |\n",
|
|
"| `output_size` | Dimension of the combined representation passed to decoders |"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a1b2c3d4-0010-0000-0000-000000000010",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"config_hypernetwork_str = \"\"\"\n",
|
|
"model_type: ecd\n",
|
|
"\n",
|
|
"input_features:\n",
|
|
" - name: sensor_a\n",
|
|
" type: number\n",
|
|
" preprocessing:\n",
|
|
" normalization: zscore\n",
|
|
" - name: sensor_b\n",
|
|
" type: number\n",
|
|
" preprocessing:\n",
|
|
" normalization: zscore\n",
|
|
" - name: sensor_c\n",
|
|
" type: number\n",
|
|
" preprocessing:\n",
|
|
" normalization: zscore\n",
|
|
" - name: sensor_type\n",
|
|
" type: category\n",
|
|
"\n",
|
|
"output_features:\n",
|
|
" - name: anomaly\n",
|
|
" type: binary\n",
|
|
"\n",
|
|
"combiner:\n",
|
|
" type: hypernetwork\n",
|
|
" hidden_size: 128\n",
|
|
" hyper_hidden_size: 64\n",
|
|
" output_size: 128\n",
|
|
"\n",
|
|
"trainer:\n",
|
|
" epochs: 30\n",
|
|
" learning_rate: 0.001\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"config_hypernetwork = yaml.safe_load(config_hypernetwork_str)\n",
|
|
"\n",
|
|
"model_hypernetwork = LudwigModel(config_hypernetwork, logging_level=30)\n",
|
|
"model_hypernetwork.train(dataset=df)\n",
|
|
"\n",
|
|
"preds_hyper, _ = model_hypernetwork.predict(dataset=test_df)\n",
|
|
"\n",
|
|
"acc_hyper = (preds_hyper[\"anomaly_predictions\"].values == test_df[\"anomaly\"].values).mean()\n",
|
|
"print(f\"HyperNetworkCombiner — test accuracy: {acc_hyper:.4f}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0011-0000-0000-000000000011",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Comparison"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a1b2c3d4-0012-0000-0000-000000000012",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"# Per-type breakdown\n",
|
|
"rows = []\n",
|
|
"for stype in SENSOR_TYPES:\n",
|
|
" mask = test_df[\"sensor_type\"] == stype\n",
|
|
" true = test_df.loc[mask, \"anomaly\"].values\n",
|
|
"\n",
|
|
" acc_c = (preds_concat.loc[mask, \"anomaly_predictions\"].values == true).mean()\n",
|
|
" acc_h = (preds_hyper.loc[mask, \"anomaly_predictions\"].values == true).mean()\n",
|
|
" rows.append({\"sensor_type\": stype, \"concat\": round(acc_c, 4), \"hypernetwork\": round(acc_h, 4)})\n",
|
|
"\n",
|
|
"rows.append({\"sensor_type\": \"OVERALL\", \"concat\": round(acc_concat, 4), \"hypernetwork\": round(acc_hyper, 4)})\n",
|
|
"\n",
|
|
"results_df = pd.DataFrame(rows)\n",
|
|
"print(results_df.to_string(index=False))\n",
|
|
"\n",
|
|
"# Bar chart\n",
|
|
"fig, ax = plt.subplots(figsize=(8, 4))\n",
|
|
"x = np.arange(len(rows))\n",
|
|
"width = 0.35\n",
|
|
"ax.bar(x - width / 2, results_df[\"concat\"], width, label=\"Concat\", color=\"steelblue\")\n",
|
|
"ax.bar(x + width / 2, results_df[\"hypernetwork\"], width, label=\"HyperNetwork\", color=\"darkorange\")\n",
|
|
"ax.set_xticks(x)\n",
|
|
"ax.set_xticklabels(results_df[\"sensor_type\"])\n",
|
|
"ax.set_ylabel(\"Test accuracy\")\n",
|
|
"ax.set_ylim(0.5, 1.0)\n",
|
|
"ax.set_title(\"Sensor anomaly detection: concat vs HyperNetworkCombiner\")\n",
|
|
"ax.legend()\n",
|
|
"plt.tight_layout()\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1b2c3d4-0013-0000-0000-000000000013",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Why hypernetwork wins\n",
|
|
"\n",
|
|
"### The problem with concatenation\n",
|
|
"\n",
|
|
"When we use `concat`, the model receives a vector like:\n",
|
|
"\n",
|
|
"```\n",
|
|
"[enc(sensor_a), enc(sensor_b), enc(sensor_c), enc(sensor_type)]\n",
|
|
"```\n",
|
|
"\n",
|
|
"The fully-connected layers after the concat have to learn — from scratch — that\n",
|
|
"`enc(sensor_type)` should *gate* the interpretation of the numerical sensors.\n",
|
|
"Effectively the network must implement a conditional logic as a series of\n",
|
|
"multiplications and additions over the entire concatenated vector. This is\n",
|
|
"theoretically possible but inefficient: many capacity-bearing parameters in the\n",
|
|
"FC layers end up implementing the routing rather than the actual anomaly detection.\n",
|
|
"\n",
|
|
"### What hypernetwork does instead\n",
|
|
"\n",
|
|
"The `HyperNetworkCombiner` separates the two roles explicitly:\n",
|
|
"\n",
|
|
"1. **Hyper-network** — a small MLP that reads `sensor_type` and emits a vector\n",
|
|
" of weights `W` and biases `b`.\n",
|
|
"2. **Main network** — a linear layer `FC(W, b)` applied to the concatenation of\n",
|
|
" `[sensor_a, sensor_b, sensor_c]` using the *dynamically generated* `W` and `b`.\n",
|
|
"\n",
|
|
"Because `W` and `b` are different for each sensor type, the transformation\n",
|
|
"applied to the numerical sensors is literally different per context. For\n",
|
|
"`temperature`, the generated `W` learns to make `sensor_a` highly predictive;\n",
|
|
"for `pressure`, the generated `W` shifts attention to `sensor_b`; for\n",
|
|
"`humidity`, it learns to combine all three.\n",
|
|
"\n",
|
|
"### When to use it\n",
|
|
"\n",
|
|
"- The conditioning feature is a **type, class, mode, or context** that changes\n",
|
|
" the semantics of other features qualitatively.\n",
|
|
"- You have enough samples per conditioning category (roughly 200+ per class) to\n",
|
|
" learn meaningful per-context transformations.\n",
|
|
"- The target signal requires different logic for different contexts, not just\n",
|
|
" different magnitudes.\n",
|
|
"\n",
|
|
"### When to stick with concat\n",
|
|
"\n",
|
|
"- All features contribute on equal footing with no hierarchical conditioning.\n",
|
|
"- The dataset is very small — the hyper-network adds parameters.\n",
|
|
"- The extra complexity is not warranted (always start simple)."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"name": "python",
|
|
"version": "3.12.0"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|