wehub/deeplearning4j--deeplearning4j
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:47:05 +08:00
commit 4f3b7da785
7394 changed files with 2005594 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/developerReadme.md
+51
View File
@@ -0,0 +1,51 @@
## Developer Readme
Some things to know regarding the deeplearning4j-ui-components module:
- This module is designed with the following goals in mind:
- Providing easy Java/JavaScript interoperability for reusable user interface components (charts, tables etc)
- Maintainability via the use of TypeScript and proper OO design
- Rapid application development, rather than providing the greatest amount of power/flexibility
The directory/package and file structure is mirrored (as much as possible) on both the java and typescript sides.
Some design principles here:
- Content and style are separate
- The goal is to have style information as an *optional* component on the Java side
- In principle styling can be done entirely in Java, entirely in JavaScript/CSS, or via a mixture of both (style options defined in Java will override those defined in CSS)
- In both the Java and TypeScript, there are Style objects; these define properties such as colors or line widths
- Java objects for UI components are created; these are converted to JSON using Jackson (typically to be posted to a web server)
- Jackson is configured to ignore null elements when encoding JSON (and the TypeScript code should handle missing values)
- JSON is converted back into JavaScript/TypeScript objects using the Component.getComponent(jsonStr: string) method, or using the TS object constructors directly (i.e., these parse the JSON)
- Consequently, any changes made to the Java UI component classes should be mirrored in both the Java and TS code
- After the JS/TS versions of the UI objects are created, they can be added to an existing component using the .render(addToObject: JQuery) method
- For charts, d3.js is used
- At present: behaviour (on click events, etc) cannot be defined in Java (this has to be done in TS/JS)
**Setup for developing TypeScript code**
- Install [node.js](https://nodejs.org/en/) and [TypeScript compiler](https://www.typescriptlang.org/#download-links)
- Then, in IntelliJ
- Ensure auto-compilation to TypeScript is disabled (untick "Enable TypeScript Compiler" in File -> Settings -> Languages & Frameworks -> TypeScript)
- Install the File Watchers plugin (File -> Settings -> Plugins). This will enable auto compilation to a *single* javascript library any time you make a change to the .ts files (next step)
- Set up a new file watcher for compilation
- File -> Settings -> Tools -> File Watchers; (+) -> TypeScript
- Program: point to typescript compiler (on Windows, for example: "C:\Users\ *UserName* \AppData\Roaming\npm\tsc.cmd")
- Arguments: empty (empty: use tsconfig.json)
After the above setup, all TypeScript files will be compiled to a single javascript file.
(Make sure your TypeScript compiler version is up to date; tsconfig.json is only suported by version 1.5 or above)
- Location: /deeplearning4j-ui-components/src/main/resources/assets/dl4j-ui.js
- A source map file (dl4j-ui.js.map) will also be produced; this aids with debugging (can see/debug original typescript code, even after it is converted to JavaScript)
- File name, location, and other options are defined in the tsconfig.json
**Note:** A rendering test is available in *test.java.org.deeplearning4j.ui.TestRendering.java* that generates a HTML file with some sample graphs.
It is recommended to open this via IntelliJ (right click -> Open In Browser)
**Note 2:** The *src/main/typescript/typedefs/* folder contains type definitions for some javascript libraries, from [https://github.com/DefinitelyTyped/DefinitelyTyped](DefinitelyTyped).
These allow us to use javascript libraries in typescript, whilst having type information.
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/pom.xml
+97
View File
@@ -0,0 +1,97 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ /* ******************************************************************************
~ *
~ *
~ * This program and the accompanying materials are made available under the
~ * terms of the Apache License, Version 2.0 which is available at
~ * https://www.apache.org/licenses/LICENSE-2.0.
~ *
~ * See the NOTICE file distributed with this work for additional
~ * information regarding copyright ownership.
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
~ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
~ * License for the specific language governing permissions and limitations
~ * under the License.
~ *
~ * SPDX-License-Identifier: Apache-2.0
~ ******************************************************************************/
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>deeplearning4j-ui-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>deeplearning4j-ui-components</artifactId>
<properties>
<module.name>deeplearning4j.ui.components</module.name>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.moditect</groupId>
<artifactId>moditect-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<dependencies>
<!-- ND4J Shaded Jackson Dependency -->
<dependency>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>jackson</artifactId>
<version>${nd4j.version}</version>
</dependency>
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
<version>${freemarker.version}</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-launcher</artifactId>
<version>${junit.platform.launcher.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>${commonsio.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>nd4j-common</artifactId>
<version>${nd4j.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>deeplearning4j-common-tests</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/api/Component.java
+56
View File
@@ -0,0 +1,56 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.api;
import lombok.Data;
import org.deeplearning4j.ui.components.chart.*;
import org.deeplearning4j.ui.components.component.ComponentDiv;
import org.deeplearning4j.ui.components.decorator.DecoratorAccordion;
import org.deeplearning4j.ui.components.table.ComponentTable;
import org.deeplearning4j.ui.components.text.ComponentText;
import org.nd4j.shade.jackson.annotation.JsonSubTypes;
import org.nd4j.shade.jackson.annotation.JsonTypeInfo;
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.WRAPPER_OBJECT)
@JsonSubTypes(value = {@JsonSubTypes.Type(value = ChartHistogram.class, name = "ChartHistogram"),
@JsonSubTypes.Type(value = ChartHorizontalBar.class, name = "ChartHorizontalBar"),
@JsonSubTypes.Type(value = ChartLine.class, name = "ChartLine"),
@JsonSubTypes.Type(value = ChartScatter.class, name = "ChartScatter"),
@JsonSubTypes.Type(value = ChartStackedArea.class, name = "ChartStackedArea"),
@JsonSubTypes.Type(value = ChartTimeline.class, name = "ChartTimeline"),
@JsonSubTypes.Type(value = ComponentDiv.class, name = "ComponentDiv"),
@JsonSubTypes.Type(value = DecoratorAccordion.class, name = "DecoratorAccordion"),
@JsonSubTypes.Type(value = ComponentTable.class, name = "ComponentTable"),
@JsonSubTypes.Type(value = ComponentText.class, name = "ComponentText")})
@Data
public abstract class Component {
/** Component type: used by the Arbiter UI to determine how to decode and render the object which is
* represented by the JSON representation of this object*/
protected final String componentType;
protected final Style style;
public Component(String componentType, Style style) {
this.componentType = componentType;
this.style = style;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/api/LengthUnit.java
+25
View File
@@ -0,0 +1,25 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.api;
public enum LengthUnit {
Px, Percent, CM, MM, In
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/api/Style.java
+129
View File
@@ -0,0 +1,129 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.api;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.deeplearning4j.ui.components.component.style.StyleDiv;
import org.deeplearning4j.ui.components.decorator.style.StyleAccordion;
import org.deeplearning4j.ui.components.table.style.StyleTable;
import org.deeplearning4j.ui.components.text.style.StyleText;
import org.nd4j.shade.jackson.annotation.JsonSubTypes;
import org.nd4j.shade.jackson.annotation.JsonTypeInfo;
import java.awt.*;
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.WRAPPER_OBJECT)
@JsonSubTypes(value = {@JsonSubTypes.Type(value = StyleChart.class, name = "StyleChart"),
@JsonSubTypes.Type(value = StyleTable.class, name = "StyleTable"),
@JsonSubTypes.Type(value = StyleText.class, name = "StyleText"),
@JsonSubTypes.Type(value = StyleAccordion.class, name = "StyleAccordion"),
@JsonSubTypes.Type(value = StyleDiv.class, name = "StyleDiv")})
@Data
@AllArgsConstructor
@NoArgsConstructor
public abstract class Style {
private Double width;
private Double height;
private LengthUnit widthUnit;
private LengthUnit heightUnit;
protected LengthUnit marginUnit;
protected Double marginTop;
protected Double marginBottom;
protected Double marginLeft;
protected Double marginRight;
protected String backgroundColor;
public Style(Builder b) {
this.width = b.width;
this.height = b.height;
this.widthUnit = b.widthUnit;
this.heightUnit = b.heightUnit;
this.marginUnit = b.marginUnit;
this.marginTop = b.marginTop;
this.marginBottom = b.marginBottom;
this.marginLeft = b.marginLeft;
this.marginRight = b.marginRight;
this.backgroundColor = b.backgroundColor;
}
@SuppressWarnings("unchecked")
public static abstract class Builder<T extends Builder<T>> {
protected Double width;
protected Double height;
protected LengthUnit widthUnit;
protected LengthUnit heightUnit;
protected LengthUnit marginUnit;
protected Double marginTop;
protected Double marginBottom;
protected Double marginLeft;
protected Double marginRight;
protected String backgroundColor;
public T width(double width, LengthUnit widthUnit) {
this.width = width;
this.widthUnit = widthUnit;
return (T) this;
}
public T height(double height, LengthUnit heightUnit) {
this.height = height;
this.heightUnit = heightUnit;
return (T) this;
}
public T margin(LengthUnit unit, Integer marginTop, Integer marginBottom, Integer marginLeft,
Integer marginRight) {
return margin(unit, (marginTop != null ? marginTop.doubleValue() : null),
(marginBottom != null ? marginBottom.doubleValue() : null),
(marginLeft != null ? marginLeft.doubleValue() : null),
(marginRight != null ? marginRight.doubleValue() : null));
}
public T margin(LengthUnit unit, Double marginTop, Double marginBottom, Double marginLeft, Double marginRight) {
this.marginUnit = unit;
this.marginTop = marginTop;
this.marginBottom = marginBottom;
this.marginLeft = marginLeft;
this.marginRight = marginRight;
return (T) this;
}
public T backgroundColor(Color color) {
return backgroundColor(Utils.colorToHex(color));
}
public T backgroundColor(String color) {
this.backgroundColor = color;
return (T) this;
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/api/Utils.java
+34
View File
@@ -0,0 +1,34 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.api;
import java.awt.*;
public class Utils {
private Utils() {}
/** Convert an AWT color to a hex color string, such as #000000 */
public static String colorToHex(Color color) {
return String.format("#%02x%02x%02x", color.getRed(), color.getGreen(), color.getBlue());
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/Chart.java
+179
View File
@@ -0,0 +1,179 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import org.deeplearning4j.ui.api.Component;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.nd4j.shade.jackson.annotation.JsonInclude;
@Data
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public abstract class Chart extends Component {
private String title;
private Boolean suppressAxisHorizontal;
private Boolean suppressAxisVertical;
private boolean showLegend;
private Double setXMin;
private Double setXMax;
private Double setYMin;
private Double setYMax;
private Double gridVerticalStrokeWidth;
private Double gridHorizontalStrokeWidth;
public Chart(String componentType) {
super(componentType, null);
}
public Chart(String componentType, Builder builder) {
super(componentType, builder.getStyle());
this.title = builder.title;
this.suppressAxisHorizontal = builder.suppressAxisHorizontal;
this.suppressAxisVertical = builder.suppressAxisVertical;
this.showLegend = builder.showLegend;
this.setXMin = builder.setXMin;
this.setXMax = builder.setXMax;
this.setYMin = builder.setYMin;
this.setYMax = builder.setYMax;
this.gridVerticalStrokeWidth = builder.gridVerticalStrokeWidth;
this.gridHorizontalStrokeWidth = builder.gridHorizontalStrokeWidth;
}
@Getter
@SuppressWarnings("unchecked")
public static abstract class Builder<T extends Builder<T>> {
private String title;
private StyleChart style;
private Boolean suppressAxisHorizontal;
private Boolean suppressAxisVertical;
private boolean showLegend;
private Double setXMin;
private Double setXMax;
private Double setYMin;
private Double setYMax;
private Double gridVerticalStrokeWidth;
private Double gridHorizontalStrokeWidth;
/**
* @param title Title for the chart (may be null)
* @param style Style for the chart (may be null)
*/
public Builder(String title, StyleChart style) {
this.title = title;
this.style = style;
}
/**
* @param suppressAxisHorizontal if true: don't show the horizontal axis (default: show)
*/
public T suppressAxisHorizontal(boolean suppressAxisHorizontal) {
this.suppressAxisHorizontal = suppressAxisHorizontal;
return (T) this;
}
/**
* @param suppressAxisVertical if true: don't show the vertical axis (default: show)
*/
public T suppressAxisVertical(boolean suppressAxisVertical) {
this.suppressAxisVertical = suppressAxisVertical;
return (T) this;
}
/**
* @param showLegend if true: show the legend. (default: no legend)
*/
public T showLegend(boolean showLegend) {
this.showLegend = showLegend;
return (T) this;
}
/**
* Used to override/set the minimum value for the x axis. If this is not set, x axis minimum is set based on the data
* @param xMin Minimum value to use for the x axis
*/
public T setXMin(Double xMin) {
this.setXMin = xMin;
return (T) this;
}
/**
* Used to override/set the maximum value for the x axis. If this is not set, x axis maximum is set based on the data
* @param xMax Maximum value to use for the x axis
*/
public T setXMax(Double xMax) {
this.setXMax = xMax;
return (T) this;
}
/**
* Used to override/set the minimum value for the y axis. If this is not set, y axis minimum is set based on the data
* @param yMin Minimum value to use for the y axis
*/
public T setYMin(Double yMin) {
this.setYMin = yMin;
return (T) this;
}
/**
* Used to override/set the maximum value for the y axis. If this is not set, y axis minimum is set based on the data
* @param yMax Minimum value to use for the y axis
*/
public T setYMax(Double yMax) {
this.setYMax = yMax;
return (T) this;
}
/**
* Set the grid lines to be enabled, and if enabled: set the grid.
* @param gridVerticalStrokeWidth If null (or 0): show no vertical grid. Otherwise: width in px
* @param gridHorizontalStrokeWidth If null (or 0): show no horizontal grid. Otherwise: width in px
*/
public T setGridWidth(Double gridVerticalStrokeWidth, Double gridHorizontalStrokeWidth) {
this.gridVerticalStrokeWidth = gridVerticalStrokeWidth;
this.gridHorizontalStrokeWidth = gridHorizontalStrokeWidth;
return (T) this;
}
/**
* Set the grid lines to be enabled, and if enabled: set the grid.
* @param gridVerticalStrokeWidth If null (or 0): show no vertical grid. Otherwise: width in px
* @param gridHorizontalStrokeWidth If null (or 0): show no horizontal grid. Otherwise: width in px
*/
public T setGridWidth(Integer gridVerticalStrokeWidth, Integer gridHorizontalStrokeWidth) {
return setGridWidth((gridVerticalStrokeWidth != null ? gridVerticalStrokeWidth.doubleValue() : null),
(gridHorizontalStrokeWidth != null ? gridHorizontalStrokeWidth.doubleValue() : null));
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/ChartHistogram.java
+107
View File
@@ -0,0 +1,107 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.util.ArrayList;
import java.util.List;
@EqualsAndHashCode(callSuper = true)
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ChartHistogram extends Chart {
public static final String COMPONENT_TYPE = "ChartHistogram";
private List<Double> lowerBounds = new ArrayList<>();
private List<Double> upperBounds = new ArrayList<>();
private List<Double> yValues = new ArrayList<>();
public ChartHistogram(Builder builder) {
super(COMPONENT_TYPE, builder);
this.lowerBounds = builder.lowerBounds;
this.upperBounds = builder.upperBounds;
this.yValues = builder.yValues;
}
//No arg constructor for Jackson
public ChartHistogram() {
super(COMPONENT_TYPE);
}
public static class Builder extends Chart.Builder<Builder> {
private List<Double> lowerBounds = new ArrayList<>();
private List<Double> upperBounds = new ArrayList<>();
private List<Double> yValues = new ArrayList<>();
public Builder(String title, StyleChart style) {
super(title, style);
}
/**
* Add a single bin
*
* @param lower Lower (minimum/left) value for the bin (x axis)
* @param upper Upper (maximum/right) value for the bin (x axis)
* @param yValue The height of the bin
*/
public Builder addBin(double lower, double upper, double yValue) {
lowerBounds.add(lower);
upperBounds.add(upper);
yValues.add(yValue);
return this;
}
public ChartHistogram build() {
return new ChartHistogram(this);
}
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("ChartHistogram(lowerBounds=");
if (lowerBounds != null) {
sb.append(lowerBounds);
} else {
sb.append("[]");
}
sb.append(",upperBounds=");
if (upperBounds != null) {
sb.append(upperBounds);
} else {
sb.append("[]");
}
sb.append(",yValues=");
if (yValues != null) {
sb.append(yValues);
} else {
sb.append("[]");
}
sb.append(")");
return sb.toString();
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/ChartHorizontalBar.java
+126
View File
@@ -0,0 +1,126 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.util.ArrayList;
import java.util.List;
@EqualsAndHashCode(callSuper = true)
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ChartHorizontalBar extends Chart {
public static final String COMPONENT_TYPE = "ChartHorizontalBar";
private List<String> labels = new ArrayList<>();
private List<Double> values = new ArrayList<>();
private Double xmin;
private Double xmax;
private ChartHorizontalBar(Builder builder) {
super(COMPONENT_TYPE, builder);
labels = builder.labels;
values = builder.values;
this.xmin = builder.xMin;
this.xmax = builder.xMax;
}
public ChartHorizontalBar() {
super(COMPONENT_TYPE, null);
//no-arg constructor for Jackson
}
public static class Builder extends Chart.Builder<Builder> {
private List<String> labels = new ArrayList<>();
private List<Double> values = new ArrayList<>();
private Double xMin;
private Double xMax;
public Builder(String title, StyleChart style) {
super(title, style);
}
public Builder addValue(String name, double value) {
labels.add(name);
values.add(value);
return this;
}
public Builder addValues(List<String> names, double[] values) {
for (int i = 0; i < names.size(); i++) {
addValue(names.get(i), values[i]);
}
return this;
}
public Builder xMin(double xMin) {
this.xMin = xMin;
return this;
}
public Builder xMax(double xMax) {
this.xMax = xMax;
return this;
}
public Builder addValues(List<String> names, float[] values) {
for (int i = 0; i < names.size(); i++) {
addValue(names.get(i), values[i]);
}
return this;
}
public ChartHorizontalBar build() {
return new ChartHorizontalBar(this);
}
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("ChartHorizontalBar(labels=");
if (labels != null) {
sb.append(labels);
} else {
sb.append("[]");
}
sb.append(",values=");
if (values != null) {
sb.append(values);
} else {
sb.append("[]");
}
if (xmin != null)
sb.append(",xMin=").append(xmin);
if (xmax != null)
sb.append(",xMax=").append(xmax);
sb.append(")");
return sb.toString();
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/ChartLine.java
+109
View File
@@ -0,0 +1,109 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
@Data
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ChartLine extends Chart {
public static final String COMPONENT_TYPE = "ChartLine";
private List<double[]> x;
private List<double[]> y;
private List<String> seriesNames;
private ChartLine(Builder builder) {
super(COMPONENT_TYPE, builder);
x = builder.x;
y = builder.y;
seriesNames = builder.seriesNames;
}
public ChartLine() {
super(COMPONENT_TYPE);
//no-arg constructor for Jackson
}
public static class Builder extends Chart.Builder<Builder> {
private List<double[]> x = new ArrayList<>();
private List<double[]> y = new ArrayList<>();
private List<String> seriesNames = new ArrayList<>();
private boolean showLegend = true;
public Builder(String title, StyleChart style) {
super(title, style);
}
public Builder addSeries(String seriesName, double[] xValues, double[] yValues) {
x.add(xValues);
y.add(yValues);
seriesNames.add(seriesName);
return this;
}
public ChartLine build() {
return new ChartLine(this);
}
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("ChartLine(x=[");
boolean first = true;
if (x != null) {
for (double[] d : x) {
if (!first)
sb.append(",");
sb.append(Arrays.toString(d));
first = false;
}
}
sb.append("],y=[");
first = true;
if (y != null) {
for (double[] d : y) {
if (!first)
sb.append(",");
sb.append(Arrays.toString(d));
first = false;
}
}
sb.append("],seriesNames=");
if (seriesNames != null)
sb.append(seriesNames);
sb.append(")");
return sb.toString();
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/ChartScatter.java
+114
View File
@@ -0,0 +1,114 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
@EqualsAndHashCode(callSuper = true)
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ChartScatter extends Chart {
public static final String COMPONENT_TYPE = "ChartScatter";
private List<double[]> x;
private List<double[]> y;
private List<String> seriesNames;
private ChartScatter(Builder builder) {
super(COMPONENT_TYPE, builder);
x = builder.x;
y = builder.y;
seriesNames = builder.seriesNames;
}
public ChartScatter() {
super(COMPONENT_TYPE);
//no-arg constructor for Jackson
}
public static class Builder extends Chart.Builder<Builder> {
private List<double[]> x = new ArrayList<>();
private List<double[]> y = new ArrayList<>();
private List<String> seriesNames = new ArrayList<>();
public Builder(String title, StyleChart style) {
super(title, style);
}
/**
*
* @param seriesName Name of the series
* @param xValues Array of x values
* @param yValues Array of y values (such that a single point i has coordinates (x[i],y[i]))
* @return
*/
public Builder addSeries(String seriesName, double[] xValues, double[] yValues) {
x.add(xValues);
y.add(yValues);
seriesNames.add(seriesName);
return this;
}
public ChartScatter build() {
return new ChartScatter(this);
}
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("ChartScatter(x=[");
boolean first = true;
if (x != null) {
for (double[] d : x) {
if (!first)
sb.append(",");
sb.append(Arrays.toString(d));
first = false;
}
}
sb.append("],y=[");
first = true;
if (y != null) {
for (double[] d : y) {
if (!first)
sb.append(",");
sb.append(Arrays.toString(d));
first = false;
}
}
sb.append("],seriesNames=");
if (seriesNames != null)
sb.append(seriesNames);
sb.append(")");
return sb.toString();
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/ChartStackedArea.java
+115
View File
@@ -0,0 +1,115 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
@EqualsAndHashCode(callSuper = true)
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ChartStackedArea extends Chart {
public static final String COMPONENT_TYPE = "ChartStackedArea";
private double[] x = new double[0];
private List<double[]> y = new ArrayList<>();
private List<String> labels = new ArrayList<>();
public ChartStackedArea() {
super(COMPONENT_TYPE);
}
public ChartStackedArea(Builder builder) {
super(COMPONENT_TYPE, builder);
this.x = builder.x;
this.y = builder.y;
this.labels = builder.seriesNames;
}
public static class Builder extends Chart.Builder<Builder> {
private double[] x;
private List<double[]> y = new ArrayList<>();
private List<String> seriesNames = new ArrayList<>();
public Builder(String title, StyleChart style) {
super(title, style);
}
/**
* Set the x-axis values
*/
public Builder setXValues(double[] x) {
this.x = x;
return this;
}
/**
* Add a single series.
*
* @param seriesName Name of the series
* @param yValues length of the yValues array must be same as the x-values array
*/
public Builder addSeries(String seriesName, double[] yValues) {
y.add(yValues);
seriesNames.add(seriesName);
return this;
}
public ChartStackedArea build() {
return new ChartStackedArea(this);
}
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("ChartStackedArea(x=");
if (x != null) {
sb.append(Arrays.toString(x));
} else {
sb.append("[]");
}
sb.append(",y=[");
boolean first = true;
if (y != null) {
for (double[] d : y) {
if (!first)
sb.append(",");
sb.append(Arrays.toString(d));
first = false;
}
}
sb.append("],labels=");
if (labels != null)
sb.append(labels);
sb.append(")");
return sb.toString();
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/ChartTimeline.java
+103
View File
@@ -0,0 +1,103 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.NoArgsConstructor;
import org.deeplearning4j.ui.api.Utils;
import org.deeplearning4j.ui.components.chart.style.StyleChart;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.awt.*;
import java.util.ArrayList;
import java.util.List;
@Data
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ChartTimeline extends Chart {
public static final String COMPONENT_TYPE = "ChartTimeline";
private List<String> laneNames = new ArrayList<>();
private List<List<TimelineEntry>> laneData = new ArrayList<>();
public ChartTimeline() {
super(COMPONENT_TYPE);
//no-arg constructor for Jackson
}
private ChartTimeline(Builder builder) {
super(COMPONENT_TYPE, builder);
this.laneNames = builder.laneNames;
this.laneData = builder.laneData;
}
public static class Builder extends Chart.Builder<Builder> {
private List<String> laneNames = new ArrayList<>();
private List<List<TimelineEntry>> laneData = new ArrayList<>();
public Builder(String title, StyleChart style) {
super(title, style);
}
public Builder addLane(String name, List<TimelineEntry> data) {
laneNames.add(name);
laneData.add(data);
return this;
}
public ChartTimeline build() {
return new ChartTimeline(this);
}
}
@Data
@NoArgsConstructor
@AllArgsConstructor
@JsonInclude(JsonInclude.Include.NON_NULL)
public static class TimelineEntry {
private String entryLabel;
private long startTimeMs;
private long endTimeMs;
private String color;
public TimelineEntry(String entryLabel, long startTimeMs, long endTimeMs) {
this.entryLabel = entryLabel;
this.startTimeMs = startTimeMs;
this.endTimeMs = endTimeMs;
}
public TimelineEntry(String entryLabel, long startTimeMs, long endTimeMs, Color color) {
this.entryLabel = entryLabel;
this.startTimeMs = startTimeMs;
this.endTimeMs = endTimeMs;
this.color = Utils.colorToHex(color);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/chart/style/StyleChart.java
+116
View File
@@ -0,0 +1,116 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.chart.style;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.NoArgsConstructor;
import org.deeplearning4j.ui.api.Style;
import org.deeplearning4j.ui.api.Utils;
import org.deeplearning4j.ui.components.text.style.StyleText;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.awt.*;
@AllArgsConstructor
@Data
@NoArgsConstructor
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class StyleChart extends Style {
public static final Double DEFAULT_CHART_MARGIN_TOP = 60.0;
public static final Double DEFAULT_CHART_MARGIN_BOTTOM = 20.0;
public static final Double DEFAULT_CHART_MARGIN_LEFT = 60.0;
public static final Double DEFAULT_CHART_MARGIN_RIGHT = 20.0;
protected Double strokeWidth;
protected Double pointSize;
protected String[] seriesColors;
protected Double axisStrokeWidth;
protected StyleText titleStyle;
private StyleChart(Builder b) {
super(b);
this.strokeWidth = b.strokeWidth;
this.pointSize = b.pointSize;
this.seriesColors = b.seriesColors;
this.axisStrokeWidth = b.axisStrokeWidth;
this.titleStyle = b.titleStyle;
}
public static class Builder extends Style.Builder<Builder> {
protected Double strokeWidth;
protected Double pointSize;
protected String[] seriesColors;
protected Double axisStrokeWidth;
protected StyleText titleStyle;
public Builder() {
super.marginTop = DEFAULT_CHART_MARGIN_TOP;
super.marginBottom = DEFAULT_CHART_MARGIN_BOTTOM;
super.marginLeft = DEFAULT_CHART_MARGIN_LEFT;
super.marginRight = DEFAULT_CHART_MARGIN_RIGHT;
}
public Builder strokeWidth(double strokeWidth) {
this.strokeWidth = strokeWidth;
return this;
}
/** Point size, for scatter plot etc */
public Builder pointSize(double pointSize) {
this.pointSize = pointSize;
return this;
}
public Builder seriesColors(Color... colors) {
String[] str = new String[colors.length];
for (int i = 0; i < str.length; i++)
str[i] = Utils.colorToHex(colors[i]);
return seriesColors(str);
}
public Builder seriesColors(String... colors) {
this.seriesColors = colors;
return this;
}
public Builder axisStrokeWidth(double axisStrokeWidth) {
this.axisStrokeWidth = axisStrokeWidth;
return this;
}
public Builder titleStyle(StyleText style) {
this.titleStyle = style;
return this;
}
public StyleChart build() {
return new StyleChart(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/component/ComponentDiv.java
+53
View File
@@ -0,0 +1,53 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.component;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.api.Component;
import org.deeplearning4j.ui.api.Style;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.util.Collection;
@Data
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ComponentDiv extends Component {
public static final String COMPONENT_TYPE = "ComponentDiv";
private Component[] components;
public ComponentDiv() {
super(COMPONENT_TYPE, null);
}
public ComponentDiv(Style style, Component... components) {
super(COMPONENT_TYPE, style);
this.components = components;
}
public ComponentDiv(Style style, Collection<Component> componentCollection) {
this(style, (componentCollection == null ? null
: componentCollection.toArray(new Component[componentCollection.size()])));
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/component/style/StyleDiv.java
+63
View File
@@ -0,0 +1,63 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.component.style;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.NoArgsConstructor;
import org.deeplearning4j.ui.api.Style;
import org.nd4j.shade.jackson.annotation.JsonInclude;
@NoArgsConstructor
@Data
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class StyleDiv extends Style {
/** Enumeration: possible values for float style option */
public enum FloatValue {
non, left, right, initial, inherit
}
private FloatValue floatValue;
private StyleDiv(Builder builder) {
super(builder);
this.floatValue = builder.floatValue;
}
public static class Builder extends Style.Builder<Builder> {
private FloatValue floatValue;
/** CSS float styling option */
public Builder floatValue(FloatValue floatValue) {
this.floatValue = floatValue;
return this;
}
public StyleDiv build() {
return new StyleDiv(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/decorator/DecoratorAccordion.java
+99
View File
@@ -0,0 +1,99 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.decorator;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.api.Component;
import org.deeplearning4j.ui.components.decorator.style.StyleAccordion;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
@EqualsAndHashCode(callSuper = true)
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public class DecoratorAccordion extends Component {
public static final String COMPONENT_TYPE = "DecoratorAccordion";
private String title;
private boolean defaultCollapsed;
private Component[] innerComponents;
public DecoratorAccordion() {
super(COMPONENT_TYPE, null);
//No arg constructor for Jackson
}
private DecoratorAccordion(Builder builder) {
super(COMPONENT_TYPE, builder.style);
this.title = builder.title;
this.defaultCollapsed = builder.defaultCollapsed;
this.innerComponents = builder.innerComponents.toArray(new Component[builder.innerComponents.size()]);
}
public static class Builder {
private StyleAccordion style;
private String title;
private List<Component> innerComponents = new ArrayList<>();
private boolean defaultCollapsed;
public Builder(StyleAccordion style) {
this(null, style);
}
public Builder(String title, StyleAccordion style) {
this.title = title;
this.style = style;
}
public Builder title(String title) {
this.title = title;
return this;
}
/**
* Components to show internally in the accordion element
*/
public Builder addComponents(Component... innerComponents) {
Collections.addAll(this.innerComponents, innerComponents);
return this;
}
/**
* Set the default collapsed/expanded state
*
* @param defaultCollapsed If true: default to collapsed
*/
public Builder setDefaultCollapsed(boolean defaultCollapsed) {
this.defaultCollapsed = defaultCollapsed;
return this;
}
public DecoratorAccordion build() {
return new DecoratorAccordion(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/decorator/style/StyleAccordion.java
+50
View File
@@ -0,0 +1,50 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.decorator.style;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.NoArgsConstructor;
import org.deeplearning4j.ui.api.Style;
import org.nd4j.shade.jackson.annotation.JsonInclude;
@NoArgsConstructor
@Data
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class StyleAccordion extends Style {
private StyleAccordion(Builder builder) {
super(builder);
}
public static class Builder extends Style.Builder<Builder> {
public StyleAccordion build() {
return new StyleAccordion(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/table/ComponentTable.java
+90
View File
@@ -0,0 +1,90 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.table;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.api.Component;
import org.deeplearning4j.ui.components.table.style.StyleTable;
import org.nd4j.shade.jackson.annotation.JsonInclude;
@EqualsAndHashCode(callSuper = true)
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ComponentTable extends Component {
public static final String COMPONENT_TYPE = "ComponentTable";
private String title;
private String[] header;
private String[][] content;
public ComponentTable() {
super(COMPONENT_TYPE, null);
//No arg constructor for Jackson
}
public ComponentTable(Builder builder) {
super(COMPONENT_TYPE, builder.style);
this.header = builder.header;
this.content = builder.content;
}
public ComponentTable(String[] header, String[][] table, StyleTable style) {
super(COMPONENT_TYPE, style);
this.header = header;
this.content = table;
}
public static class Builder {
private StyleTable style;
private String[] header;
private String[][] content;
public Builder(StyleTable style) {
this.style = style;
}
/**
* @param header Header values for the table
*/
public Builder header(String... header) {
this.header = header;
return this;
}
/**
* Content for the table, as 2d String[]
*/
public Builder content(String[][] content) {
this.content = content;
return this;
}
public ComponentTable build() {
return new ComponentTable(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/table/style/StyleTable.java
+143
View File
@@ -0,0 +1,143 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.table.style;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.api.LengthUnit;
import org.deeplearning4j.ui.api.Style;
import org.deeplearning4j.ui.api.Utils;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.awt.*;
@Data
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class StyleTable extends Style {
private double[] columnWidths;
private LengthUnit columnWidthUnit;
private Integer borderWidthPx;
private String headerColor;
private String backgroundColor;
private String whitespaceMode;
private StyleTable(Builder builder) {
super(builder);
this.columnWidths = builder.columnWidths;
this.columnWidthUnit = builder.columnWidthUnit;
this.borderWidthPx = builder.borderWidthPx;
this.headerColor = builder.headerColor;
this.backgroundColor = builder.backgroundColor;
this.whitespaceMode = builder.whitespaceMode;
}
//No arg constructor for Jackson
private StyleTable() {
}
public static class Builder extends Style.Builder<Builder> {
private double[] columnWidths;
private LengthUnit columnWidthUnit;
private Integer borderWidthPx;
private String headerColor;
private String backgroundColor;
private String whitespaceMode;
/**
* Specify the widths for the columns
*
* @param unit Unit that the widths are specified in
* @param widths Width values for the columns
*/
public Builder columnWidths(LengthUnit unit, double... widths) {
this.columnWidthUnit = unit;
this.columnWidths = widths;
return this;
}
/**
* @param borderWidthPx Width of the border, in px
*/
public Builder borderWidth(int borderWidthPx) {
this.borderWidthPx = borderWidthPx;
return this;
}
/**
* @param color Background color for the header row
*/
public Builder headerColor(Color color) {
String hex = Utils.colorToHex(color);
return headerColor(hex);
}
/**
* @param color Background color for the header row
*/
public Builder headerColor(String color) {
if (!color.toLowerCase().matches("#[a-f0-9]{6}"))
throw new IllegalArgumentException("Invalid color: must be hex format. Got: " + color);
this.headerColor = color;
return this;
}
/**
* @param color Background color for the table cells (ex. header row)
*/
public Builder backgroundColor(Color color) {
String hex = Utils.colorToHex(color);
return backgroundColor(hex);
}
/**
* @param color Background color for the table cells (ex. header row)
*/
public Builder backgroundColor(String color) {
if (!color.toLowerCase().matches("#[a-f0-9]{6}"))
throw new IllegalArgumentException("Invalid color: must be hex format. Got: " + color);
this.backgroundColor = color;
return this;
}
/**
* Set the whitespace mode (CSS style tag). For example, "pre" to maintain current formatting with no wrapping,
* "pre-wrap" to wrap (but otherwise take into account new line characters in text, etc)
*
* @param whitespaceMode CSS whitespace mode
*/
public Builder whitespaceMode(String whitespaceMode) {
this.whitespaceMode = whitespaceMode;
return this;
}
public StyleTable build() {
return new StyleTable(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/text/ComponentText.java
+72
View File
@@ -0,0 +1,72 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.text;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.deeplearning4j.ui.api.Component;
import org.deeplearning4j.ui.components.text.style.StyleText;
import org.nd4j.shade.jackson.annotation.JsonInclude;
@EqualsAndHashCode(callSuper = true)
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public class ComponentText extends Component {
public static final String COMPONENT_TYPE = "ComponentText";
private String text;
public ComponentText() {
super(COMPONENT_TYPE, null);
//No arg constructor for Jackson deserialization
text = null;
}
public ComponentText(String text, StyleText style) {
super(COMPONENT_TYPE, style);
this.text = text;
}
private ComponentText(Builder builder) {
this(builder.text, builder.style);
}
@Override
public String toString() {
return "ComponentText(" + text + ")";
}
public static class Builder {
private StyleText style;
private String text;
public Builder(String text, StyleText style) {
this.text = text;
this.style = style;
}
public ComponentText build() {
return new ComponentText(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/components/text/style/StyleText.java
+111
View File
@@ -0,0 +1,111 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.components.text.style;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.NoArgsConstructor;
import org.deeplearning4j.ui.api.Style;
import org.deeplearning4j.ui.api.Utils;
import org.nd4j.shade.jackson.annotation.JsonInclude;
import java.awt.*;
/**
* Style for text
*/
@Data
@NoArgsConstructor
@EqualsAndHashCode(callSuper = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class StyleText extends Style {
private String font;
private Double fontSize;
private Boolean underline;
private String color;
private Boolean whitespacePre;
private StyleText(Builder builder) {
super(builder);
this.font = builder.font;
this.fontSize = builder.fontSize;
this.underline = builder.underline;
this.color = builder.color;
this.whitespacePre = builder.whitespacePre;
}
public static class Builder extends Style.Builder<Builder> {
private String font;
private Double fontSize;
private Boolean underline;
private String color;
private Boolean whitespacePre;
/** Specify the font to be used for the text */
public Builder font(String font) {
this.font = font;
return this;
}
/** Size of the font (pt) */
public Builder fontSize(double size) {
this.fontSize = size;
return this;
}
/** If true: text should be underlined (default: not) */
public Builder underline(boolean underline) {
this.underline = underline;
return this;
}
/** Color for the text */
public Builder color(Color color) {
return color(Utils.colorToHex(color));
}
/** Color for the text */
public Builder color(String color) {
this.color = color;
return this;
}
/**
* If set to true: add a "white-space: pre" to the style.
* In effect, this stops the representation from compressing the whitespace characters, and messing up/removing
* text that contains newlines, tabs, etc.
*/
public Builder whitespacePre(boolean whitespacePre) {
this.whitespacePre = whitespacePre;
return this;
}
public StyleText build() {
return new StyleText(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/standalone/ComponentObject.java
+33
View File
@@ -0,0 +1,33 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.standalone;
import lombok.AllArgsConstructor;
import lombok.Data;
@AllArgsConstructor
@Data
public class ComponentObject {
public final String id;
public final String content;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java/org/deeplearning4j/ui/standalone/StaticPageUtil.java
+129
View File
@@ -0,0 +1,129 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.standalone;
import freemarker.template.Configuration;
import freemarker.template.Template;
import freemarker.template.TemplateExceptionHandler;
import freemarker.template.Version;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.deeplearning4j.ui.api.Component;
import org.nd4j.common.io.ClassPathResource;
import org.nd4j.shade.jackson.databind.DeserializationFeature;
import org.nd4j.shade.jackson.databind.MapperFeature;
import org.nd4j.shade.jackson.databind.ObjectMapper;
import org.nd4j.shade.jackson.databind.SerializationFeature;
import java.io.File;
import java.io.IOException;
import java.io.StringWriter;
import java.io.Writer;
import java.util.*;
public class StaticPageUtil {
private StaticPageUtil() {}
/**
* Given the specified components, render them to a stand-alone HTML page (which is returned as a String)
*
* @param components Components to render
* @return Stand-alone HTML page, as a String
*/
public static String renderHTML(Collection<Component> components) {
return renderHTML(components.toArray(new Component[components.size()]));
}
/**
* Given the specified components, render them to a stand-alone HTML page (which is returned as a String)
*
* @param components Components to render
* @return Stand-alone HTML page, as a String
*/
public static String renderHTML(Component... components) {
try {
return renderHTMLContent(components);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public static String renderHTMLContent(Component... components) throws Exception {
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
mapper.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true);
mapper.enable(SerializationFeature.INDENT_OUTPUT);
Configuration cfg = new Configuration(new Version(2, 3, 23));
// Where do we load the templates from:
cfg.setClassForTemplateLoading(StaticPageUtil.class, "");
// Some other recommended settings:
cfg.setIncompatibleImprovements(new Version(2, 3, 23));
cfg.setDefaultEncoding("UTF-8");
cfg.setLocale(Locale.US);
cfg.setTemplateExceptionHandler(TemplateExceptionHandler.RETHROW_HANDLER);
ClassPathResource cpr = new ClassPathResource("assets/dl4j-ui.js");
String scriptContents = IOUtils.toString(cpr.getInputStream(), "UTF-8");
Map<String, Object> pageElements = new HashMap<>();
List<ComponentObject> list = new ArrayList<>();
int i = 0;
for (Component c : components) {
list.add(new ComponentObject(String.valueOf(i), mapper.writeValueAsString(c)));
i++;
}
pageElements.put("components", list);
pageElements.put("scriptcontent", scriptContents);
Template template = cfg.getTemplate("staticpage.ftl");
Writer stringWriter = new StringWriter();
template.process(pageElements, stringWriter);
return stringWriter.toString();
}
/**
* A version of {@link #renderHTML(Component...)} that exports the resulting HTML to the specified path.
*
* @param outputPath Output path
* @param components Components to render
*/
public static void saveHTMLFile(String outputPath, Component... components) throws IOException {
saveHTMLFile(new File(outputPath));
}
/**
* A version of {@link #renderHTML(Component...)} that exports the resulting HTML to the specified File.
*
* @param outputFile Output path
* @param components Components to render
*/
public static void saveHTMLFile(File outputFile, Component... components) throws IOException {
FileUtils.writeStringToFile(outputFile, renderHTML(components));
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/java9/module-info.java
+19
View File
@@ -0,0 +1,19 @@
open module deeplearning4j.ui.components {
requires commons.io;
requires freemarker;
requires jackson;
requires nd4j.common;
requires java.desktop;
exports org.deeplearning4j.ui.api;
exports org.deeplearning4j.ui.components.chart;
exports org.deeplearning4j.ui.components.chart.style;
exports org.deeplearning4j.ui.components.component;
exports org.deeplearning4j.ui.components.component.style;
exports org.deeplearning4j.ui.components.decorator;
exports org.deeplearning4j.ui.components.decorator.style;
exports org.deeplearning4j.ui.components.table;
exports org.deeplearning4j.ui.components.table.style;
exports org.deeplearning4j.ui.components.text;
exports org.deeplearning4j.ui.components.text.style;
exports org.deeplearning4j.ui.standalone;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/resources/META-INF/native-image/reflect-config.json
+130
View File
@@ -0,0 +1,130 @@
[
{
"name":"org.deeplearning4j.ui.components.chart.ChartHistogram",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.chart.Chart",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.chart.ChartHorizontalBar",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.chart.ChartLine",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.chart.ChartScatter",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.chart.ChartStackedArea",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.chart.ChartTimeline",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.api.Style",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.component.ComponentDiv",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.decorator.style.StyleAccordion",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.decorator.DecoratorAccordion",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.table.style.StyleTable",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.table.ComponentTable",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.text.style.StyleText",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.components.text.ComponentText",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
},
{
"name":"org.deeplearning4j.ui.api.Style",
"allDeclaredFields":true,
"allDeclaredMethods":true,
"allDeclaredConstructors":true,
"allDeclaredClasses" : true,
"allPublicClasses" : true
}
]
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/resources/assets/dl4j-ui.d.ts
Vendored
+241
View File
@@ -0,0 +1,241 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
declare abstract class Style {
private width;
private height;
private widthUnit;
private heightUnit;
private marginTop;
private marginBottom;
private marginLeft;
private marginRight;
private backgroundColor;
constructor(jsonObj: any);
getWidth: () => number;
getHeight: () => number;
getWidthUnit: () => string;
getHeightUnit: () => string;
getMarginTop: () => number;
getMarginBottom: () => number;
getMarginLeft: () => number;
getMarginRight: () => number;
getBackgroundColor: () => string;
static getMargins(s: Style): Margin;
}
declare enum ComponentType {
ComponentText = 0,
ComponentTable = 1,
ComponentDiv = 2,
ChartHistogram = 3,
ChartHorizontalBar = 4,
ChartLine = 5,
ChartScatter = 6,
ChartStackedArea = 7,
ChartTimeline = 8,
DecoratorAccordion = 9,
}
declare abstract class Component {
private componentType;
constructor(componentType: ComponentType);
getComponentType(): ComponentType;
static getComponent(jsonStr: string): Renderable;
}
declare class ChartConstants {
static DEFAULT_CHART_STROKE_WIDTH: number;
static DEFAULT_CHART_POINT_SIZE: number;
static DEFAULT_AXIS_STROKE_WIDTH: number;
static DEFAULT_TITLE_COLOR: string;
}
interface Margin {
top: number;
right: number;
bottom: number;
left: number;
widthExMargins: number;
heightExMargins: number;
}
interface Renderable {
render: (addToObject: JQuery) => void;
}
declare class TSUtils {
static max(input: number[][]): number;
static min(input: number[][]): number;
static normalizeLengthUnit(input: string): string;
}
declare abstract class Chart extends Component {
protected style: StyleChart;
protected title: string;
protected suppressAxisHorizontal: boolean;
protected suppressAxisVertical: boolean;
protected showLegend: boolean;
protected setXMin: number;
protected setXMax: number;
protected setYMin: number;
protected setYMax: number;
protected gridVerticalStrokeWidth: number;
protected gridHorizontalStrokeWidth: number;
constructor(componentType: ComponentType, jsonStr: string);
getStyle(): StyleChart;
protected static appendTitle(svg: any, title: string, margin: Margin, titleStyle: StyleText): void;
}
declare class ChartHistogram extends Chart implements Renderable {
private lowerBounds;
private upperBounds;
private yValues;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class ChartLine extends Chart implements Renderable {
private xData;
private yData;
private seriesNames;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class ChartScatter extends Chart implements Renderable {
private xData;
private yData;
private seriesNames;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class Legend {
private static offsetX;
private static offsetY;
private static padding;
private static separation;
private static boxSize;
private static fillColor;
private static legendOpacity;
private static borderStrokeColor;
static legendFn: (g: any) => void;
}
declare class ChartStackedArea extends Chart implements Renderable {
private xData;
private yData;
private labels;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class ChartTimeline extends Chart implements Renderable {
private laneNames;
private laneData;
private lanes;
private itemData;
private mainView;
private miniView;
private brush;
private x;
private x1;
private xTimeAxis;
private y1;
private y2;
private itemRects;
private rect;
private static MINI_LANE_HEIGHT_PX;
private static ENTRY_LANE_HEIGHT_OFFSET_FRACTION;
private static ENTRY_LANE_HEIGHT_TOTAL_FRACTION;
private static MILLISEC_PER_MINUTE;
private static MILLISEC_PER_HOUR;
private static MILLISEC_PER_DAY;
private static MILLISEC_PER_WEEK;
private static DEFAULT_COLOR;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
renderChart: () => void;
moveBrush: () => void;
getMiniViewPaths: (items: any) => any[];
}
declare class StyleChart extends Style {
protected strokeWidth: number;
protected pointSize: number;
protected seriesColors: string[];
protected axisStrokeWidth: number;
protected titleStyle: StyleText;
constructor(jsonObj: any);
getStrokeWidth: () => number;
getPointSize: () => number;
getSeriesColors: () => string[];
getSeriesColor: (idx: number) => string;
getAxisStrokeWidth: () => number;
getTitleStyle: () => StyleText;
}
declare class ComponentDiv extends Component implements Renderable {
private style;
private components;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class StyleDiv extends Style {
protected floatValue: string;
constructor(jsonObj: any);
getFloatValue: () => string;
}
declare class DecoratorAccordion extends Component implements Renderable {
private style;
private title;
private defaultCollapsed;
private innerComponents;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class StyleAccordion extends Style {
constructor(jsonObj: any);
}
declare class ComponentTable extends Component implements Renderable {
private header;
private content;
private style;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class StyleTable extends Style {
private columnWidths;
private columnWidthUnit;
private borderWidthPx;
private headerColor;
private whitespaceMode;
constructor(jsonObj: any);
getColumnWidths: () => number[];
getColumnWidthUnit: () => string;
getBorderWidthPx: () => number;
getHeaderColor: () => string;
getWhitespaceMode: () => string;
}
declare class ComponentText extends Component implements Renderable {
private text;
private style;
constructor(jsonStr: string);
render: (appendToObject: JQuery) => void;
}
declare class StyleText extends Style {
private font;
private fontSize;
private underline;
private color;
private whitespacePre;
constructor(jsonObj: any);
getFont: () => string;
getFontSize: () => number;
getUnderline: () => boolean;
getColor: () => string;
getWhitespacePre: () => boolean;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/resources/assets/dl4j-ui.js
+1353
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/resources/assets/dl4j-ui.js.map
+1
View File
File diff suppressed because one or more lines are too long
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/resources/org/deeplearning4j/ui/standalone/staticpage.ftl
+72
View File
@@ -0,0 +1,72 @@
<html>
<head>
<style type="text/css">
html, body {
width: 100%;
height: 100%;
}
h1 {
font-family: Georgia, Times, 'Times New Roman', serif;
font-size: 28px;
font-style: bold;
font-variant: normal;
font-weight: 500;
line-height: 26.4px;
}
h3 {
font-family: Georgia, Times, 'Times New Roman', serif;
font-size: 16px;
font-style: normal;
font-variant: normal;
font-weight: 500;
line-height: 26.4px;
}
</style>
<title>Data Analysis</title>
</head>
<body style="padding: 0px; margin: 0px" onload="generateContent()">
<#--<meta name="viewport" content="width=device-width, initial-scale=1">-->
<link href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js"></script>
<link href="http://code.jquery.com/ui/1.11.4/themes/smoothness/jquery-ui.css">
<script src="http://code.jquery.com/jquery-1.10.2.js"></script>
<script src="http://code.jquery.com/ui/1.11.4/jquery-ui.js"></script>
<script src="http://cdnjs.cloudflare.com/ajax/libs/d3/3.5.5/d3.min.js"></script>
<script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
<script>
function generateContent(){
var mainDiv = $('#outerdiv');
<#list components as c>
var div_${c.id} = $('#${c.id}');
var html_${c.id} = div_${c.id}.html();
var component = Component.getComponent(html_${c.id});
component.render(mainDiv);
</#list>
}
</script>
<script>
${scriptcontent}
</script>
<div style="width:1400px; margin:0 auto; border:0px" id="outerdiv">
</div>
<#list components as c>
<div id="${c.id}" style="display:none">
${c.content}
</div>
</#list>
</body>
</html>
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/api/Component.ts
+86
View File
@@ -0,0 +1,86 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
abstract class Component {
private componentType: ComponentType;
constructor(componentType: ComponentType){
this.componentType = componentType;
}
public getComponentType(){
return this.componentType;
}
/* Parse the JSON string that represents a component, and build that component
* Assumption here: the format is like '{"ComponentTable": {...}}' - i.e., as generated by Jackson for the Java objects
* */
public static getComponent(jsonStr: string): Renderable {
var json: any = JSON.parse(jsonStr);
var key: string;
if(json["componentType"]) key = json["componentType"]; //No wrapper object case...
else key = Object.keys(json)[0]; //Typical wrapper object case
switch(key){
case ComponentType[ComponentType.ComponentText]:
return new ComponentText(jsonStr);
case ComponentType[ComponentType.ComponentTable]:
return new ComponentTable(jsonStr);
case ComponentType[ComponentType.ChartHistogram]:
return new ChartHistogram(jsonStr);
case ComponentType[ComponentType.ChartHorizontalBar]:
throw new Error("Horizontal bar chart: not yet implemented");
case ComponentType[ComponentType.ChartLine]:
return new ChartLine(jsonStr);
case ComponentType[ComponentType.ChartScatter]:
return new ChartScatter(jsonStr);
case ComponentType[ComponentType.ChartStackedArea]:
return new ChartStackedArea(jsonStr);
case ComponentType[ComponentType.ChartTimeline]:
return new ChartTimeline(jsonStr);
case ComponentType[ComponentType.DecoratorAccordion]:
return new DecoratorAccordion(jsonStr);
case ComponentType[ComponentType.ComponentDiv]:
return new ComponentDiv(jsonStr);
default:
throw new Error("Unknown component type \"" + key + "\" or invalid JSON: \"" + jsonStr + "\"");
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/api/ComponentType.ts
+32
View File
@@ -0,0 +1,32 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
enum ComponentType {
ComponentText,
ComponentTable,
ComponentDiv,
ChartHistogram,
ChartHorizontalBar,
ChartLine,
ChartScatter,
ChartStackedArea,
ChartTimeline,
DecoratorAccordion
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/api/Constants.ts
+27
View File
@@ -0,0 +1,27 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ChartConstants {
static DEFAULT_CHART_STROKE_WIDTH = 1.0;
static DEFAULT_CHART_POINT_SIZE = 3.0;
static DEFAULT_AXIS_STROKE_WIDTH = 1.0;
static DEFAULT_TITLE_COLOR = "#000000";
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/api/Margin.ts
+28
View File
@@ -0,0 +1,28 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
interface Margin {
top: number;
right: number;
bottom: number;
left: number;
widthExMargins: number;
heightExMargins: number;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/api/Renderable.ts
+28
View File
@@ -0,0 +1,28 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
interface Renderable {
render: (addToObject: JQuery) => void;
//TODO: Implement an update method. Can be used to update an existing chart, table etc with new data,
// without redrawing the whole thing from scratch
//update: (jsonObj: any) => void;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/api/Style.ts
+72
View File
@@ -0,0 +1,72 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
abstract class Style {
private width: number;
private height: number;
private widthUnit: string;
private heightUnit: string;
private marginTop: number;
private marginBottom: number;
private marginLeft: number;
private marginRight: number;
private backgroundColor: string;
constructor( jsonObj: any){
this.width = jsonObj['width'];
this.height = jsonObj['height'];
this.widthUnit = TSUtils.normalizeLengthUnit(jsonObj['widthUnit']);
this.heightUnit = TSUtils.normalizeLengthUnit(jsonObj['heightUnit']);
this.marginTop = jsonObj['marginTop'];
this.marginBottom = jsonObj['marginBottom'];
this.marginLeft = jsonObj['marginLeft'];
this.marginRight = jsonObj['marginRight'];
this.backgroundColor = jsonObj['backgroundColor'];
}
getWidth = () => this.width;
getHeight = () => this.height;
getWidthUnit = () => this.widthUnit;
getHeightUnit = () => this.heightUnit;
getMarginTop = () => this.marginTop;
getMarginBottom = () => this.marginBottom;
getMarginLeft = () => this.marginLeft;
getMarginRight = () => this.marginRight;
getBackgroundColor = () => this.backgroundColor;
static getMargins(s: Style): Margin{
var mTop: number = (s ? s.getMarginTop() : 0);
var mBottom: number = (s ? s.getMarginBottom() : 0);
var mLeft: number = (s ? s.getMarginLeft() : 0);
var mRight: number = (s ? s.getMarginRight() : 0);
// Set the dimensions of the canvas / graph
return {top: mTop,
right: mRight,
bottom: mBottom,
left: mLeft,
widthExMargins: s.getWidth() - mLeft - mRight,
heightExMargins: s.getHeight() - mTop - mBottom};
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/Chart.ts
+83
View File
@@ -0,0 +1,83 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
abstract class Chart extends Component {
protected style: StyleChart;
protected title: string;
protected suppressAxisHorizontal: boolean;
protected suppressAxisVertical: boolean;
protected showLegend: boolean;
protected setXMin: number;
protected setXMax: number;
protected setYMin: number;
protected setYMax: number;
protected gridVerticalStrokeWidth: number;
protected gridHorizontalStrokeWidth: number;
constructor(componentType: ComponentType, jsonStr: string){
super(componentType);
var jsonOrig: any = JSON.parse(jsonStr);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[componentType]]; //Usually: expect type as wrapper object
this.suppressAxisHorizontal = json['suppressAxisHorizontal'];
this.suppressAxisVertical = json['suppressAxisVertical'];
this.showLegend = json['showLegend'];
this.title = json['title'];
this.setXMin = json['setXMin'];
this.setXMax = json['setXMax'];
this.setYMin = json['setYMin'];
this.setYMax = json['setYMax'];
this.gridVerticalStrokeWidth = json['gridVerticalStrokeWidth'];
this.gridHorizontalStrokeWidth = json['gridHorizontalStrokeWidth'];
if(json['style']) this.style = new StyleChart(json['style']);
}
getStyle(): StyleChart {
return this.style;
}
protected static appendTitle(svg: any, title: string, margin: Margin, titleStyle: StyleText): void{
var text = svg.append("text")
.text(title)
.attr("x", (margin.widthExMargins / 2))
.attr("y", 0 - ((margin.top - 30) / 2))
.attr("text-anchor", "middle");
if(titleStyle){
if(titleStyle.getFont()) text.attr("font-family",titleStyle.getFont);
if(titleStyle.getFontSize() != null) text.attr("font-size",titleStyle.getFontSize()+"pt");
if(titleStyle.getUnderline() != null) text.style("text-decoration", "underline");
if(titleStyle.getColor()) text.style("fill",titleStyle.getColor);
else text.style("fill",ChartConstants.DEFAULT_TITLE_COLOR);
} else {
text.style("text-decoration", "underline");
text.style("fill",ChartConstants.DEFAULT_TITLE_COLOR);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/ChartHistogram.ts
+147
View File
@@ -0,0 +1,147 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ChartHistogram extends Chart implements Renderable {
private lowerBounds: number[];
private upperBounds: number[];
private yValues: number[];
constructor(jsonStr: string){
super(ComponentType.ChartHistogram, jsonStr);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.ChartHistogram]];
this.lowerBounds = json['lowerBounds'];
this.upperBounds = json['upperBounds'];
this.yValues = json['yvalues'];
}
render = (appendToObject: JQuery) => {
var s: StyleChart = this.getStyle();
var margin: Margin = Style.getMargins(s);
// Add the bins.
var xMin: number;
var xMax: number;
var yMin: number;
var yMax: number;
if(this.setXMin) xMin = this.setXMin;
else xMin = (this.lowerBounds ? d3.min(this.lowerBounds) : 0);
if(this.setXMax) xMax = this.setXMax;
else xMax = (this.upperBounds ? d3.max(this.upperBounds) : 1);
if(this.setYMin) yMin = this.setYMin;
else yMin = 0;
if(this.setYMax) yMax = this.setYMax;
else yMax = (this.yValues ? d3.max(this.yValues) : 1);
//// Define the axes
var xScale: any = d3.scale.linear()
.domain([xMin, xMax])
.range([0, margin.widthExMargins]);
var xAxis: any = d3.svg.axis().scale(xScale)
.orient("bottom").ticks(5);
if(this.gridVerticalStrokeWidth && this.gridVerticalStrokeWidth > 0){
xAxis.innerTickSize(-margin.heightExMargins); //used as grid line
}
var yScale: any = d3.scale.linear()
.domain([0, yMax])
.range([margin.heightExMargins, 0]);
var yAxis: any = d3.svg.axis().scale(yScale)
.orient("left").ticks(5);
if(this.gridHorizontalStrokeWidth && this.gridHorizontalStrokeWidth > 0){
yAxis.innerTickSize(-margin.widthExMargins); //used as grid line
}
if(this.suppressAxisHorizontal === true) xAxis.tickValues([]);
if(this.suppressAxisVertical === true) yAxis.tickValues([]);
// Set up the data:
var lowerBounds: number[] = this.lowerBounds;
var upperBounds: number[] = this.upperBounds;
var yValues: number[] = this.yValues;
var data: any = lowerBounds.map(function (d, i) {
return {'width': upperBounds[i] - lowerBounds[i], 'height': yValues[i], 'offset': lowerBounds[i]};
});
// Adds the svg canvas
var svg = d3.select("#" + appendToObject.attr("id"))
.append("svg")
.style("fill", "none")
.attr("width", s.getWidth())
.attr("height", s.getHeight())
.attr("padding", "20px")
.append("g")
.attr("transform",
"translate(" + margin.left + "," + margin.top + ")");
svg.selectAll(".bin")
.data(data)
.enter().append("rect")
.attr("class", "bin")
.style("fill","steelblue")
.attr("x", function(d: any) { return xScale(d.offset); })
.attr("width", function(d: any) { return xScale(xMin + d.width) - 1; })
.attr("y", function(d: any) { return yScale(d.height); })
.attr("height", function(d: any) { return margin.heightExMargins - yScale(d.height); });
// Add the X Axis
var xAxisNode = svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + margin.heightExMargins + ")")
.style("stroke","#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill","none")
.call(xAxis);
xAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
if(this.gridVerticalStrokeWidth != null) xAxisNode.selectAll('.axis line').style({'stroke-width': this.gridVerticalStrokeWidth});
// Add the Y Axis
var yAxisNode = svg.append("g")
.attr("class", "y axis")
.style("stroke","#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill","none")
.call(yAxis);
yAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
if(this.gridHorizontalStrokeWidth != null) yAxisNode.selectAll('.axis line').style({'stroke-width': this.gridHorizontalStrokeWidth});
//Add title (if present)
if (this.title) {
var titleStyle: StyleText;
if(this.style) titleStyle = this.style.getTitleStyle();
Chart.appendTitle(svg, this.title, margin, titleStyle);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/ChartLine.ts
+167
View File
@@ -0,0 +1,167 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ChartLine extends Chart implements Renderable {
private xData: number[][];
private yData: number[][];
private seriesNames: string[];
constructor(jsonStr: string){
super(ComponentType.ChartLine, jsonStr);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.ChartLine]];
this.xData = json['x'];
this.yData = json['y'];
this.seriesNames = json['seriesNames'];
}
render = (appendToObject: JQuery) => {
var nSeries: number = (!this.xData ? 0 : this.xData.length);
var s: StyleChart = this.getStyle();
var margin: Margin = Style.getMargins(s);
// Set the ranges
var xScale: d3.scale.Linear<number,number> = d3.scale.linear().range([0, margin.widthExMargins]);
var yScale: d3.scale.Linear<number,number> = d3.scale.linear().range([margin.heightExMargins, 0]);
// Define the axes
var xAxis = d3.svg.axis().scale(xScale)
.orient("bottom").ticks(5);
if(this.gridVerticalStrokeWidth != null && this.gridVerticalStrokeWidth > 0){
xAxis.innerTickSize(-margin.heightExMargins); //used as grid line
}
var yAxis = d3.svg.axis().scale(yScale)
.orient("left").ticks(5);
if(this.gridHorizontalStrokeWidth != null && this.gridHorizontalStrokeWidth > 0){
yAxis.innerTickSize(-margin.widthExMargins); //used as grid line
}
if(this.suppressAxisHorizontal === true) xAxis.tickValues([]);
if(this.suppressAxisVertical === true) yAxis.tickValues([]);
// Define the line
var valueline = d3.svg.line()
.x(function (d: any) {
return xScale(d.xPos);
})
.y(function (d: any) {
return yScale(d.yPos);
});
// Adds the svg canvas
//TODO don't hardcode these colors/attributes...
var svg = d3.select("#" + appendToObject.attr("id"))
.append("svg")
.style("stroke-width", ( s && s.getStrokeWidth() ? s.getStrokeWidth() : ChartConstants.DEFAULT_CHART_STROKE_WIDTH))
.style("fill", "none")
.attr("width", s.getWidth())
.attr("height", s.getHeight())
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
// Scale the range of the chart
var xMin: number;
var xMax: number;
var yMin: number;
var yMax: number;
if(this.setXMin != null) xMin = this.setXMin;
else xMin = (this.xData ? TSUtils.min(this.xData) : 0);
if(this.setXMax != null) xMax = this.setXMax;
else xMax = (this.xData ? TSUtils.max(this.xData) : 1);
if(this.setYMin != null) yMin = this.setYMin;
else yMin = (this.yData ? TSUtils.min(this.yData) : 0);
if(this.setYMax != null) yMax = this.setYMax;
else yMax = (this.yData ? TSUtils.max(this.yData) : 1);
xScale.domain([xMin, xMax]);
yScale.domain([yMin, yMax]);
// Add the valueline path.
var defaultColor: Ordinal<string,string> = d3.scale.category10();
for (var i = 0; i < nSeries; i++) {
var xVals: number[] = this.xData[i];
var yVals: number[] = this.yData[i];
var data: any[] = xVals.map(function (d, i) {
return {'xPos': xVals[i], 'yPos': yVals[i]};
});
svg.append("path")
.attr("class", "line")
.style("stroke", (s && s.getSeriesColor(i) ? s.getSeriesColor(i) : defaultColor(String(i))))
.attr("d", valueline(data));
}
// Add the X Axis
var xAxisNode = svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + margin.heightExMargins + ")")
.style("stroke","#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill","none")
.call(xAxis);
xAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
if(this.gridVerticalStrokeWidth != null) xAxisNode.selectAll('.axis line').style({'stroke-width': this.gridVerticalStrokeWidth});
// Add the Y Axis
var yAxisNode = svg.append("g")
.attr("class", "y axis")
.style("stroke","#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill","none")
.call(yAxis);
yAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
if(this.gridHorizontalStrokeWidth != null) yAxisNode.selectAll('.axis line').style({'stroke-width': this.gridHorizontalStrokeWidth});
//Add legend (if present)
if (this.seriesNames && this.showLegend === true) {
var legendSpace = margin.widthExMargins / i;
for (var i = 0; i < nSeries; i++) {
var values = this.xData[i];
var yValues = this.yData[i];
var lastX = values[values.length - 1];
var lastY = yValues[yValues.length - 1];
var toDisplay = this.seriesNames[i];
svg.append("text")
.attr("x", (legendSpace / 2) + i * legendSpace) // spacing
.attr("y", margin.heightExMargins + (margin.bottom / 2) + 5)
.attr("class", "legend") // style the legend
.style("fill", (s && s.getSeriesColor(i) ? s.getSeriesColor(i) : defaultColor(String(i))))
.text(toDisplay);
}
}
//Add title (if present)
if (this.title) {
var titleStyle: StyleText;
if(this.style) titleStyle = this.style.getTitleStyle();
Chart.appendTitle(svg, this.title, margin, titleStyle);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/ChartScatter.ts
+165
View File
@@ -0,0 +1,165 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ChartScatter extends Chart implements Renderable {
private xData:number[][];
private yData:number[][];
private seriesNames:string[];
constructor(jsonStr:string) {
super(ComponentType.ChartScatter, jsonStr);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.ChartScatter]];
this.xData = json['x'];
this.yData = json['y'];
this.seriesNames = json['seriesNames'];
}
render = (appendToObject:JQuery) => {
var nSeries:number = (!this.xData ? 0 : this.xData.length);
var s:StyleChart = this.getStyle();
var margin:Margin = Style.getMargins(s);
// Set the ranges
var xScale:d3.scale.Linear<number,number> = d3.scale.linear().range([0, margin.widthExMargins]);
var yScale:d3.scale.Linear<number,number> = d3.scale.linear().range([margin.heightExMargins, 0]);
// Define the axes
var xAxis = d3.svg.axis().scale(xScale)
.innerTickSize(-margin.heightExMargins) //used as grid line
.orient("bottom").ticks(5);
var yAxis = d3.svg.axis().scale(yScale)
.innerTickSize(-margin.widthExMargins) //used as grid line
.orient("left").ticks(5);
if (this.suppressAxisHorizontal === true) xAxis.tickValues([]);
if (this.suppressAxisVertical === true) yAxis.tickValues([]);
// Adds the svg canvas
//TODO don't hardcode these colors/attributes...
var svg = d3.select("#" + appendToObject.attr("id"))
.append("svg")
.style("stroke-width", ( s && s.getStrokeWidth() ? s.getStrokeWidth() : 1))
.style("fill", "none")
.attr("width", s.getWidth())
.attr("height", s.getHeight())
.attr("padding", "20px")
.append("g")
.attr("transform",
"translate(" + margin.left + "," + margin.top + ")");
// Scale the range of the chart
var xMin:number;
var xMax:number;
var yMin:number;
var yMax:number;
if (this.setXMin) xMin = this.setXMin;
else xMin = (this.xData ? TSUtils.min(this.xData) : 0);
if (this.setXMax) xMax = this.setXMax;
else xMax = (this.xData ? TSUtils.max(this.xData) : 1);
if (this.setYMin) yMin = this.setYMin;
else yMin = (this.yData ? TSUtils.min(this.yData) : 0);
if (this.setYMax) yMax = this.setYMax;
else yMax = (this.yData ? TSUtils.max(this.yData) : 1);
xScale.domain([xMin, xMax]);
yScale.domain([yMin, yMax]);
// Add the valueline path.
var defaultColor:Ordinal<string,string> = d3.scale.category10();
for (var i = 0; i < nSeries; i++) {
var xVals = this.xData[i];
var yVals = this.yData[i];
var data = xVals.map(function (d, i) {
return {'xPos': xVals[i], 'yPos': yVals[i]};
});
svg.selectAll("circle")
.data(data)
.enter()
.append("circle")
.style("fill", (s && s.getSeriesColor(i) ? s.getSeriesColor(i) : defaultColor(String(i))))
.attr("r", (s && s.getPointSize() ? s.getPointSize() : ChartConstants.DEFAULT_CHART_POINT_SIZE))
.attr("cx", function (d) {
return xScale(d['xPos']);
})
.attr("cy", function (d) {
return yScale(d['yPos']);
});
}
// Add the X Axis
var xAxisNode = svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + margin.heightExMargins + ")")
.style("stroke", "#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill", "none")
.call(xAxis);
xAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
if (this.gridVerticalStrokeWidth != null) xAxisNode.selectAll('.axis line').style({'stroke-width': this.gridVerticalStrokeWidth});
// Add the Y Axis
var yAxisNode = svg.append("g")
.attr("class", "y axis")
.style("stroke", "#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill", "none")
.call(yAxis);
yAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
if (this.gridHorizontalStrokeWidth != null) yAxisNode.selectAll('.axis line').style({'stroke-width': this.gridHorizontalStrokeWidth});
//Add legend (if present)
if (this.seriesNames && this.showLegend === true) {
var legendSpace = margin.widthExMargins / i;
for (var i = 0; i < nSeries; i++) {
var values = this.xData[i];
var yValues = this.yData[i];
var lastX = values[values.length - 1];
var lastY = yValues[yValues.length - 1];
var toDisplay;
if (!lastX || !lastY) toDisplay = this.seriesNames[i] + " (no data)";
else toDisplay = this.seriesNames[i] + " (" + lastX.toPrecision(5) + "," + lastY.toPrecision(5) + ")";
svg.append("text")
.attr("x", (legendSpace / 2) + i * legendSpace) // spacing
.attr("y", margin.heightExMargins + (margin.bottom / 2) + 5)
.attr("class", "legend") // style the legend
.style("fill", (s && s.getSeriesColor(i) ? s.getSeriesColor(i) : defaultColor(String(i))))
.text(toDisplay);
}
}
//Add title (if present)
if (this.title) {
var titleStyle: StyleText;
if(this.style) titleStyle = this.style.getTitleStyle();
Chart.appendTitle(svg, this.title, margin, titleStyle);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/ChartStackedArea.ts
+176
View File
@@ -0,0 +1,176 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ChartStackedArea extends Chart implements Renderable {
private xData: number[];
private yData: number[][];
private labels: string[];
constructor(jsonStr: string){
super(ComponentType.ChartStackedArea, jsonStr);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.ChartStackedArea]];
this.xData = json['x'];
this.yData = json['y'];
this.labels = json['labels'];
}
render = (appendToObject: JQuery) => {
var nSeries: number = (!this.xData ? 0 : this.xData.length);
var s: StyleChart = this.getStyle();
var margin: Margin = Style.getMargins(s);
// Set the ranges
var xScale: d3.scale.Linear<number,number> = d3.scale.linear().range([0, margin.widthExMargins]);
var yScale: d3.scale.Linear<number,number> = d3.scale.linear().range([margin.heightExMargins, 0]);
// Define the axes
var xAxis = d3.svg.axis().scale(xScale)
.orient("bottom").ticks(5);
if(this.gridVerticalStrokeWidth != null && this.gridVerticalStrokeWidth > 0){
xAxis.innerTickSize(-margin.heightExMargins); //used as grid line
}
var yAxis = d3.svg.axis().scale(yScale)
.orient("left").ticks(5);
if(this.gridHorizontalStrokeWidth != null && this.gridHorizontalStrokeWidth > 0){
yAxis.innerTickSize(-margin.widthExMargins); //used as grid line
}
if(this.suppressAxisHorizontal === true) xAxis.tickValues([]);
if(this.suppressAxisVertical === true) yAxis.tickValues([]);
var data: any[] = [];
for(var i=0; i<this.xData.length; i++ ){
var obj = {};
for( var j=0; j<this.labels.length; j++ ){
obj[this.labels[j]] = this.yData[j][i];
obj['xValue'] = this.xData[i];
}
data.push(obj);
}
var area = d3.svg.area()
.x(function(d: any) { return xScale(d.xValue); })
.y0(function(d: any) { return yScale(d.y0); })
.y1(function(d: any) { return yScale(d.y0 + d.y); });
var stack = d3.layout.stack()
.values(function(d: any) { return d.values; });
var svg = d3.select("#" + appendToObject.attr("id")).append("svg")
.attr("width", margin.widthExMargins + margin.left + margin.right)
.attr("height", margin.heightExMargins + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
var color: any = d3.scale.category20();
color.domain(d3.keys(data[0]).filter(function (key) {
return key !== "xValue";
}));
var browsers = stack(color.domain().map(function (name) {
return {
name: name,
values: data.map(function (d) {
return {xValue: d.xValue, y: d[name] * 1};
})
};
}));
// Find the value of the day with highest total value
var maxX = d3.max(data, function (d) {
var vals = d3.keys(d).map(function (key) {
return key !== "xValue" ? d[key] : 0
});
return d3.sum(vals);
});
// Set domains for axes
xScale.domain(d3.extent(data, function (d) {
return d.xValue;
}));
yScale.domain([0, maxX]);
var browser = svg.selectAll(".browser")
.data(browsers)
.enter().append("g")
.attr("class", "browser");
var tempLabels = this.labels;
var defaultColor: Ordinal<string,string> = d3.scale.category20();
browser.append("path")
.attr("class", "area")
.attr("data-legend",function(d: any) { return d.name})
.attr("d", function (d: any) {
return area(d.values);
})
.style("fill", function(d: any){
if(s && s.getSeriesColor(tempLabels.indexOf(d.name))){
return s.getSeriesColor(tempLabels.indexOf(d.name));
} else{
return defaultColor(String(tempLabels.indexOf(d.name)))
}
})
.style({"stroke-width": "0px"});
//Add the x axis:
var xAxisNode = svg.append("g")
.attr("class", "x axis")
.style("stroke","#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill","none")
.attr("transform", "translate(0," + margin.heightExMargins + ")")
.call(xAxis);
xAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
//Add the y axis
var yAxisNode = svg.append("g")
.attr("class", "y axis")
.style("stroke","#000")
.style("stroke-width", (s != null && s.getAxisStrokeWidth() != null ? s.getAxisStrokeWidth() : ChartConstants.DEFAULT_AXIS_STROKE_WIDTH))
.style("fill","none")
.call(yAxis);
yAxisNode.selectAll('text').style("stroke-width",0).style("fill","#000000");
//Add title (if present)
if (this.title) {
var titleStyle: StyleText;
if(this.style) titleStyle = this.style.getTitleStyle();
Chart.appendTitle(svg, this.title, margin, titleStyle);
}
//Append the legend
var legend: any = svg.append("g")
.attr("class","legend")
.attr("transform","translate(40,40)")
.style("font-size","12px")
.call(Legend.legendFn);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/ChartTimeline.ts
+388
View File
@@ -0,0 +1,388 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ChartTimeline extends Chart implements Renderable {
private laneNames:string[];
private laneData:any[][];
private lanes:any;
private itemData:any;
private mainView:any;
private miniView:any;
private brush:any;
private x:any;
private x1:any;
private xTimeAxis:any;
private y1:any;
private y2:any;
private itemRects:any;
private rect:any;
private static MINI_LANE_HEIGHT_PX = 12;
private static ENTRY_LANE_HEIGHT_OFFSET_FRACTION:number = 0.05;
private static ENTRY_LANE_HEIGHT_TOTAL_FRACTION:number = 0.90;
private static MILLISEC_PER_MINUTE:number = 60 * 1000;
private static MILLISEC_PER_HOUR:number = 60 * ChartTimeline.MILLISEC_PER_MINUTE;
private static MILLISEC_PER_DAY:number = 24 * ChartTimeline.MILLISEC_PER_HOUR;
private static MILLISEC_PER_WEEK:number = 7 * ChartTimeline.MILLISEC_PER_DAY;
private static DEFAULT_COLOR = "LightGrey";
constructor(jsonStr:string) {
super(ComponentType.ChartTimeline, jsonStr);
var json = JSON.parse(jsonStr);
if (!json["componentType"]) json = json[ComponentType[ComponentType.ChartTimeline]];
this.laneNames = json['laneNames'];
this.laneData = json['laneData'];
}
render = (appendToObject:JQuery) => {
var instance = this;
var s:StyleChart = this.getStyle();
var margin:Margin = Style.getMargins(s);
//Format data
this.itemData = [];
var count = 0;
for (var i = 0; i < this.laneData.length; i++) {
for (var j = 0; j < this.laneData[i].length; j++) {
var obj = {};
obj["start"] = this.laneData[i][j]["startTimeMs"];
obj["end"] = this.laneData[i][j]["endTimeMs"];
obj["id"] = count++;
obj["lane"] = i;
obj["color"] = this.laneData[i][j]["color"];
obj["label"] = this.laneData[i][j]["entryLabel"];
this.itemData.push(obj);
}
}
this.lanes = [];
for (var i = 0; i < this.laneNames.length; i++) {
var obj = {};
obj["label"] = this.laneNames[i];
obj["id"] = i;
this.lanes.push(obj);
}
// Adds the svg canvas
//TODO don't hardcode these colors/attributes...
var svg = d3.select("#" + appendToObject.attr("id"))
.append("svg")
.style("stroke-width", ( s && s.getStrokeWidth() ? s.getStrokeWidth() : ChartConstants.DEFAULT_CHART_STROKE_WIDTH))
.style("fill", "none")
.attr("width", s.getWidth())
.attr("height", s.getHeight())
.append("g");
var heightExMargins = s.getHeight() - margin.top - margin.bottom;
var widthExMargins = s.getWidth() - margin.left - margin.right;
var miniHeight = this.laneNames.length * ChartTimeline.MINI_LANE_HEIGHT_PX;
var mainHeight = s.getHeight() - miniHeight - margin.top - margin.bottom - 25;
var minTime:number = d3.min(this.itemData, function (d:any) { return d.start; });
var maxTime:number = d3.max(this.itemData, function (d:any) { return d.end; });
this.x = d3.time.scale()
.domain([minTime, maxTime])
.range([0, widthExMargins]);
this.x1 = d3.time.scale().range([0, widthExMargins]);
this.y1 = d3.scale.linear().domain([0, this.laneNames.length]).range([0, mainHeight]);
this.y2 = d3.scale.linear().domain([0, this.laneNames.length]).range([0, miniHeight]);
//Add a rectangle for clipping the elements in each swimlane
this.rect = svg.append('defs').append('clipPath')
.attr('id', 'clip')
.append('rect')
.attr('width', widthExMargins)
.attr('height', s.getHeight() - 100);
this.mainView = svg.append('g')
.attr('transform', 'translate(' + margin.left + ',' + margin.top + ')')
.attr('width', widthExMargins)
.attr('height', mainHeight)
.attr('font-size', '12px')
.attr('font', 'sans-serif');
this.miniView = svg.append('g')
.attr('transform', 'translate(' + margin.left + ',' + (mainHeight + margin.top + 25) + ')') //25 being space for ticks/time label
.attr('width', widthExMargins)
.attr('height', miniHeight)
.attr('font-size', '10px')
.attr('font', 'sans-serif');
// Horizontal lane divider lines for mainView chart:
this.mainView.append('g').selectAll('.laneLines')
.data(this.lanes)
.enter().append('line')
.attr('x1', 0)
.attr('y1', function (d:any) {
return d3.round(instance.y1(d.id)) + 0.5;
})
.attr('x2', widthExMargins)
.attr('y2', function (d:any) {
return d3.round(instance.y1(d.id)) + 0.5;
})
.attr('stroke', 'lightgray')
.attr('stroke-width', 1);
//Add labels for lane text
this.mainView.append('g').selectAll('.laneText')
.data(this.lanes)
.enter().append('text')
.text(function (d:any) {
if(d.label) return d.label;
return "";
})
.attr('x', -10)
.attr('y', function (d:any) {
return instance.y1(d.id + .5);
})
.attr('text-anchor', 'end')
.attr("font","8pt sans-serif")
.attr('fill', 'black');
// Divider lines for miniView chart
this.miniView.append('g').selectAll('.laneLines')
.data(this.lanes)
.enter().append('line')
.attr('x1', 0)
.attr('y1', function (d:any) { return d3.round(instance.y2(d.id)) + 0.5; })
.attr('x2', widthExMargins)
.attr('y2', function (d:any) { return d3.round(instance.y2(d.id)) + 0.5; })
.attr('stroke', 'gray')
.attr('stroke-width', 1.0);
//Text for mini view
this.miniView.append('g').selectAll('.laneText')
.data(this.lanes)
.enter().append('text')
.text(function (d:any) {
if(d.label) return d.label;
return "";
})
.attr('x', -10)
.attr('y', function (d:any) {
return instance.y2(d.id + .5);
})
.attr('dy', '0.5ex')
.attr('text-anchor', 'end')
.attr('fill', 'black');
// Render time axis
this.xTimeAxis = d3.svg.axis()
.scale(this.x1)
.orient('bottom')
.ticks(d3.time.days, 1)
.tickFormat(d3.time.format('%a %d'))
.tickSize(6, 0);
//Time axis
var temp:any = this.mainView.append('g')
.attr('transform', 'translate(0,' + mainHeight + ')')
// .attr('class', 'mainView axis time')
.attr('class', 'timeAxis')
.attr('fill', 'black')
.style("stroke", "black").style("stroke-width", 1.0).style("fill", "black")
.attr("font", "10px sans-serif")
.call(this.xTimeAxis);
temp.selectAll('text').style("stroke-width", 0.0).attr('stroke-width', 0.0);
// draw the itemData
this.itemRects = this.mainView.append('g')
.attr('clip-path', 'url(#clip)');
//Entries for miniView chart
this.miniView.append('g').selectAll('miniItems')
.data(this.getMiniViewPaths(this.itemData))
.enter().append('path')
.attr('class', function (d:any) {
return 'miniItem ' + d.class;
})
.attr('d', function (d:any) {
return d.path;
})
.attr('stroke', 'black')
.attr('stroke-width', 'black');
// Draw the brush selection area (default - set extent to all data)
this.miniView.append('rect')
.attr('pointer-events', 'painted')
.attr('width', widthExMargins)
.attr('height', miniHeight)
.attr('visibility', 'hidden')
.on('mouseup', this.moveBrush);
this.brush = d3.svg.brush()
.x(this.x)
.extent([minTime, maxTime])
.on("brush", this.renderChart);
this.miniView.append('g')
.attr('class', 'x brush')
.call(this.brush)
.selectAll('rect')
.attr('y', 1)
.attr('height', miniHeight - 1)
.style('fill','gray')
.style('fill-opacity','0.2')
.style('stroke','DarkSlateGray')
.style('stroke-width',1);
this.miniView.selectAll('rect.background').remove();
this.renderChart();
//Add title (if present)
if (this.title) {
var titleStyle:StyleText;
if (this.style) titleStyle = this.style.getTitleStyle();
var text = svg.append("text")
.text(this.title)
.attr("x", (s.getWidth() / 2))
.attr("y", ((margin.top - 30) / 2))
.attr("text-anchor", "middle");
if (titleStyle) {
if (titleStyle.getFont()) text.attr("font-family", titleStyle.getFont);
if (titleStyle.getFontSize() != null) text.attr("font-size", titleStyle.getFontSize() + "pt");
if (titleStyle.getUnderline() != null) text.style("text-decoration", "underline");
if (titleStyle.getColor()) text.style("fill", titleStyle.getColor);
else text.style("fill", ChartConstants.DEFAULT_TITLE_COLOR);
} else {
text.style("text-decoration", "underline");
text.style("fill", ChartConstants.DEFAULT_TITLE_COLOR);
}
}
};
renderChart = () => {
var instance:any = this;
var extent:number[] = this.brush.extent();
var minExtent:number = extent[0];
var maxExtent:number = extent[1];
var visibleItems:any = this.itemData.filter(function (d) {
return d.start < maxExtent && d.end > minExtent
});
this.miniView.select('.brush').call(this.brush.extent([minExtent, maxExtent]));
this.x1.domain([minExtent, maxExtent]);
//https://github.com/d3/d3-time-format#timeFormat
var range = maxExtent - minExtent;
if (range > 2 * ChartTimeline.MILLISEC_PER_WEEK) {
this.xTimeAxis.ticks(d3.time.mondays, 1).tickFormat(d3.time.format('%a %d'));
} else if (range > 2 * ChartTimeline.MILLISEC_PER_DAY) {
this.xTimeAxis.ticks(d3.time.days, 1).tickFormat(d3.time.format('%a %d'));
} else if (range > 2 * ChartTimeline.MILLISEC_PER_HOUR) {
this.xTimeAxis.ticks(d3.time.hours, 4).tickFormat(d3.time.format('%H %p'));
} else if (range > 2 * ChartTimeline.MILLISEC_PER_MINUTE) {
this.xTimeAxis.ticks(d3.time.minutes, 1).tickFormat(d3.time.format('%H:%M'));
} else if (range >= 30000) {
this.xTimeAxis.ticks(d3.time.seconds, 10).tickFormat(d3.time.format('%H:%M:%S'));
} else {
this.xTimeAxis.ticks(d3.time.seconds, 1).tickFormat(d3.time.format('%H:%M:%S'));
} //no d3.time.milliseconds, so ticks below 1 second are not possible? (or, at least not using same approach as here)
// Update the axis
this.mainView.select('.timeAxis').call(this.xTimeAxis);
// Update the rectangles
var rects:any = this.itemRects.selectAll('rect')
.data(visibleItems, function (d) { return d.id; })
.attr('x', function (d) { return instance.x1(d.start); })
.attr('width', function (d) { return instance.x1(d.end) - instance.x1(d.start); });
//Set attributes for mainView swimlane rectangles
rects.enter().append('rect')
.attr('x', function (d) { return instance.x1(d.start); })
.attr('y', function (d) { return instance.y1(d.lane) + ChartTimeline.ENTRY_LANE_HEIGHT_OFFSET_FRACTION * instance.y1(1) + 0.5; })
.attr('width', function (d) { return instance.x1(d.end) - instance.x1(d.start); })
.attr('height', function (d) { return ChartTimeline.ENTRY_LANE_HEIGHT_TOTAL_FRACTION * instance.y1(1); })
.attr('stroke', 'black')
.attr('fill', function(d){
if(d.color) return d.color;
return ChartTimeline.DEFAULT_COLOR;
})
.attr('stroke-width', 1);
rects.exit().remove();
// Update the item labels
var labels:any = this.itemRects.selectAll('text')
.data(visibleItems, function (d) {
return d.id;
})
.attr('x', function (d) {
return instance.x1(Math.max(d.start, minExtent)) + 2;
})
.attr('fill', 'black');
labels.enter().append('text')
.text(function (d) {
if(instance.x1(d.end) - instance.x1(d.start) <= 30) return "";
if(d.label) return d.label;
return "";
})
.attr('x', function (d) {
return instance.x1(Math.max(d.start, minExtent)) + 2;
})
.attr('y', function (d) {
return instance.y1(d.lane) + .4 * instance.y1(1) + 0.5;
})
.attr('text-anchor', 'start')
.attr('class', 'itemLabel')
.attr('fill', 'black');
labels.exit().remove();
};
moveBrush = () => {
var origin:any = d3.mouse(this.rect[0]);
var time: any = this.x.invert(origin[0]).getTime();
var halfExtent: number = (this.brush.extent()[1].getTime() - this.brush.extent()[0].getTime()) / 2;
this.brush.extent([new Date(time - halfExtent), new Date(time + halfExtent)]);
this.renderChart();
};
getMiniViewPaths = (items:any) => {
var paths = {}, d, offset = .5 * this.y2(1) + 0.5, result = [];
for (var i = 0; i < items.length; i++) {
d = items[i];
if (!paths[d.class]) paths[d.class] = '';
paths[d.class] += ['M', this.x(d.start), (this.y2(d.lane) + offset), 'H', this.x(d.end)].join(' ');
}
for (var className in paths) {
result.push({class: className, path: paths[className]});
}
return result;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/Legend.ts
+86
View File
@@ -0,0 +1,86 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class Legend {
//TODO: make these configurable...
private static offsetX: number = 15;
private static offsetY: number = 15;
private static padding: number = 8;
private static separation: number = 12;
private static boxSize: number = 10;
private static fillColor: string = "#FFFFFF";
private static legendOpacity: number = 0.75;
private static borderStrokeColor: string = "#000000";
static legendFn = (function(g: any) {
//Get SVG and legend box/items:
var svg = d3.select(g.property("nearestViewportElement"));
var legendBox = g.selectAll(".outerRect").data([true]);
var legendItems = g.selectAll(".legendElement").data([true]);
legendBox.enter().append("rect").attr("class","outerRect");
legendItems.enter().append("g").attr("class","legendElement");
var legendElements: any[] = [];
svg.selectAll("[data-legend]").each(function() {
var thisVar = d3.select(this);
legendElements.push({
label: thisVar.attr("data-legend"),
color: thisVar.style("fill")
});
});
//Add rectangles for color
legendItems.selectAll("rect")
.data(legendElements,function(d) { return d.label})
.call(function(d) { d.enter().append("rect")})
.call(function(d) { d.exit().remove()})
.attr("x",0)
.attr("y",function(d,i) { return i*Legend.separation-Legend.boxSize+"px"})
.attr("width",Legend.boxSize)
.attr("height",Legend.boxSize)
//.style("fill",function(d) { return d.value.color});
.style("fill",function(d) { return d.color});
//Add labels
legendItems.selectAll("text")
.data(legendElements,function(d) { return d.label})
.call(function(d) { d.enter().append("text")})
.call(function(d) { d.exit().remove()})
.attr("y",function(d,i) { return i*Legend.separation + "px"})
.attr("x",(Legend.padding + Legend.boxSize) + "px")
.text(function(d) { return d.label});
//Add the outer box
var legendBoundingBox: any = legendItems[0][0].getBBox();
legendBox.attr("x",(legendBoundingBox.x-Legend.padding))
.attr("y",(legendBoundingBox.y-Legend.padding))
.attr("height",(legendBoundingBox.height+2*Legend.padding))
.attr("width",(legendBoundingBox.width+2*Legend.padding))
.style("fill",Legend.fillColor)
.style("stroke",Legend.borderStrokeColor)
.style("opacity",Legend.legendOpacity);
svg.selectAll(".legend").attr("transform","translate(" + Legend.offsetX + "," + Legend.offsetY + ")");
});
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/chart/style/StyleChart.ts
+53
View File
@@ -0,0 +1,53 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class StyleChart extends Style {
protected strokeWidth: number;
protected pointSize: number;
protected seriesColors: string[];
protected axisStrokeWidth: number;
protected titleStyle: StyleText;
constructor( jsonObj: any ){
super(jsonObj['StyleChart']);
var style: any = jsonObj['StyleChart'];
if(style){
this.strokeWidth = style['strokeWidth'];
this.pointSize = style['pointSize'];
this.seriesColors = style['seriesColors'];
if(style['titleStyle']) this.titleStyle = new StyleText(style['titleStyle']);
}
}
getStrokeWidth = () => this.strokeWidth;
getPointSize = () => this.pointSize;
getSeriesColors = () => this.seriesColors;
getSeriesColor = (idx: number) => {
if(!this.seriesColors || idx < 0 || idx > this.seriesColors.length) return null;
return this.seriesColors[idx];
};
getAxisStrokeWidth = () => this.axisStrokeWidth;
getTitleStyle = () => this.titleStyle;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/component/ComponentDiv.ts
+77
View File
@@ -0,0 +1,77 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ComponentDiv extends Component implements Renderable {
private style: StyleDiv;
private components: Renderable[];
constructor(jsonStr: string){
super(ComponentType.ComponentDiv);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.ComponentDiv]];
var components: any[] = json['components'];
if(components){
this.components = [];
for( var i=0; i<components.length; i++ ){
var asStr: string = JSON.stringify(components[i]);
this.components.push(Component.getComponent(asStr));
}
}
if(json['style']) this.style = new StyleDiv(json['style']);
}
render = (appendToObject: JQuery) => {
var newDiv: JQuery = $('<div></div>');
newDiv.uniqueId();
if(this.style){
if(this.style.getWidth()){
var unit: string = this.style.getWidthUnit();
newDiv.width(this.style.getWidth() + (unit ? unit : ""));
}
if(this.style.getHeight()){
var unit: string = this.style.getHeightUnit();
newDiv.height(this.style.getHeight() + (unit ? unit : ""));
}
if(this.style.getBackgroundColor()) newDiv.css("background-color",this.style.getBackgroundColor());
if(this.style.getFloatValue()) newDiv.css("float", this.style.getFloatValue());
}
//Adding the div before adding the sub-components seems to be important for charts/d3 to work properly
appendToObject.append(newDiv);
//now, add the sub-components:
if(this.components){
for( var i=0; i<this.components.length; i++ ){
this.components[i].render(newDiv);
}
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/component/style/StyleDiv.ts
+35
View File
@@ -0,0 +1,35 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class StyleDiv extends Style {
protected floatValue: string;
constructor( jsonObj: any){
super(jsonObj['StyleDiv']);
if(jsonObj && jsonObj['StyleDiv']) this.floatValue = jsonObj['StyleDiv']['floatValue'];
}
getFloatValue = () => this.floatValue;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/decorator/DecoratorAccordion.ts
+90
View File
@@ -0,0 +1,90 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class DecoratorAccordion extends Component implements Renderable {
private style: StyleAccordion;
private title: string;
private defaultCollapsed: boolean;
private innerComponents: Renderable[];
constructor(jsonStr: string){
super(ComponentType.DecoratorAccordion);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.DecoratorAccordion]];
this.title = json['title'];
this.defaultCollapsed = json['defaultCollapsed'];
var innerCs: any[] = json['innerComponents'];
if(innerCs){
this.innerComponents = [];
for( var i=0; i<innerCs.length; i++ ){
var asStr: string = JSON.stringify(innerCs[i]);
this.innerComponents.push(Component.getComponent(asStr));
}
}
if(json['style']) this.style = new StyleAccordion(json['style']);
}
render = (appendToObject: JQuery) => {
var s:StyleAccordion = this.style;
var outerDiv: JQuery = $('<div></div>');
outerDiv.uniqueId();
var titleDiv: JQuery;
if(this.title) titleDiv = $('<div>' + this.title + '</div>');
else titleDiv = $('<div></div>');
titleDiv.uniqueId();
outerDiv.append(titleDiv);
var innerDiv: JQuery = $('<div></div>');
innerDiv.uniqueId();
outerDiv.append(innerDiv);
//Add the inner components:
if (this.innerComponents) {
for (var i = 0; i < this.innerComponents.length; i++) {
//this.innerComponents[i].render(outerDiv);
this.innerComponents[i].render(innerDiv);
}
}
appendToObject.append(outerDiv);
if(this.defaultCollapsed) outerDiv.accordion({collapsible: true, heightStyle: "content", active: false});
else outerDiv.accordion({collapsible: true, heightStyle: "content"});
//$(function(){outerDiv.accordion({collapsible: true, heightStyle: "content"})});
//elementdiv.accordion();
//$(function(){elementdiv.accordion()});
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/decorator/style/StyleAccordion.ts
+29
View File
@@ -0,0 +1,29 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class StyleAccordion extends Style {
constructor( jsonObj: any){
super(jsonObj['StyleAccordion']);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/table/ComponentTable.ts
+106
View File
@@ -0,0 +1,106 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ComponentTable extends Component implements Renderable {
private header: string[];
private content: string[][];
private style: StyleTable;
constructor(jsonStr: string){
super(ComponentType.ComponentTable);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.ComponentTable]];
this.header = json['header'];
this.content = json['content'];
if(json['style']) this.style = new StyleTable(json['style']);
}
render = (appendToObject: JQuery) => {
var s: StyleTable = this.style;
var margin: Margin = Style.getMargins(s);
var tbl = document.createElement('table');
//TODO allow setting of table width/height
tbl.style.width = '100%';
if(s && s.getBorderWidthPx() != null ) tbl.setAttribute('border', String(s.getBorderWidthPx()));
if(s && s.getBackgroundColor()) tbl.style.backgroundColor = s.getBackgroundColor();
if(s && s.getWhitespaceMode()) tbl.style.whiteSpace = s.getWhitespaceMode();
if (s && s.getColumnWidths()) {
//TODO allow other than percentage
var colWidths: number[] = s.getColumnWidths();
var unit: string = TSUtils.normalizeLengthUnit(s.getColumnWidthUnit());
for (var i = 0; i < colWidths.length; i++) {
var col = document.createElement('col');
col.setAttribute('width', colWidths[i] + unit);
tbl.appendChild(col);
}
}
//TODO: don't hardcode
var padTop = 1;
var padRight = 1;
var padBottom = 1;
var padLeft = 1;
if (this.header) {
var theader = document.createElement('thead');
var headerRow = document.createElement('tr');
if(s && s.getHeaderColor()) headerRow.style.backgroundColor = s.getHeaderColor();
for (var i = 0; i < this.header.length; i++) {
var headerd = document.createElement('th');
headerd.style.padding = padTop + 'px ' + padRight + 'px ' + padBottom + 'px ' + padLeft + 'px';
headerd.appendChild(document.createTextNode(this.header[i]));
headerRow.appendChild(headerd);
}
tbl.appendChild(headerRow);
}
//Add content:
if (this.content) {
var tbdy = document.createElement('tbody');
for (var i = 0; i < this.content.length; i++) {
var tr = document.createElement('tr');
for (var j = 0; j < this.content[i].length; j++) {
var td = document.createElement('td');
td.style.padding = padTop + 'px ' + padRight + 'px ' + padBottom + 'px ' + padLeft + 'px';
td.appendChild(document.createTextNode(this.content[i][j]));
tr.appendChild(td);
}
tbdy.appendChild(tr);
}
tbl.appendChild(tbdy);
}
appendToObject.append(tbl);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/table/style/StyleTable.ts
+47
View File
@@ -0,0 +1,47 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class StyleTable extends Style {
private columnWidths: number[];
private columnWidthUnit: string;
private borderWidthPx: number;
private headerColor: string;
private whitespaceMode: string;
constructor( jsonObj: any ){
super(jsonObj['StyleTable']);
var style: any = jsonObj['StyleTable'];
if(style){
this.columnWidths = jsonObj['StyleTable']['columnWidths'];
this.borderWidthPx = jsonObj['StyleTable']['borderWidthPx'];
this.headerColor = jsonObj['StyleTable']['headerColor'];
this.columnWidthUnit = jsonObj['StyleTable']['columnWidthUnit'];
this.whitespaceMode = jsonObj['StyleTable']['whitespaceMode'];
}
}
getColumnWidths = () => this.columnWidths;
getColumnWidthUnit = () => this.columnWidthUnit;
getBorderWidthPx = () => this.borderWidthPx;
getHeaderColor = () => this.headerColor;
getWhitespaceMode = () => this.whitespaceMode;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/text/ComponentText.ts
+61
View File
@@ -0,0 +1,61 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class ComponentText extends Component implements Renderable {
private text: string;
private style: StyleText;
constructor(jsonStr: string){
super(ComponentType.ComponentText);
var json = JSON.parse(jsonStr);
if(!json["componentType"]) json = json[ComponentType[ComponentType.ComponentText]];
this.text = json['text'];
if(json['style']) this.style = new StyleText(json['style']);
}
render = (appendToObject: JQuery) => {
var textNode: Text = document.createTextNode(this.text);
if(this.style){
var newSpan: HTMLSpanElement = document.createElement('span');
if(this.style.getFont()) newSpan.style.font = this.style.getFont();
if(this.style.getFontSize() != null) newSpan.style.fontSize = this.style.getFontSize() + "pt";
if(this.style.getUnderline() != null) newSpan.style.textDecoration='underline';
if(this.style.getColor()) newSpan.style.color = this.style.getColor();
if(this.style.getMarginTop()) newSpan.style.marginTop = this.style.getMarginTop() + "px";
if(this.style.getMarginBottom()) newSpan.style.marginBottom = this.style.getMarginBottom() + "px";
if(this.style.getMarginLeft()) newSpan.style.marginLeft = this.style.getMarginLeft() + "px";
if(this.style.getMarginRight()) newSpan.style.marginRight = this.style.getMarginRight() + "px";
if(this.style.getWhitespacePre()) newSpan.style.whiteSpace = 'pre';
newSpan.appendChild(textNode);
appendToObject.append(newSpan);
} else {
var newSpan: HTMLSpanElement = document.createElement('span');
//appendToObject.append(textNode);
newSpan.appendChild(textNode);
appendToObject.append(newSpan);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/components/text/style/StyleText.ts
+47
View File
@@ -0,0 +1,47 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class StyleText extends Style {
private font: string;
private fontSize: number;
private underline: boolean;
private color: string;
private whitespacePre: boolean;
constructor( jsonObj: any){
super(jsonObj['StyleText']);
var style: any = jsonObj['StyleText'];
if(style){
this.font = style['font'];
this.fontSize = style['fontSize'];
this.underline = style['underline'];
this.color = style['color'];
this.whitespacePre = style['whitespacePre'];
}
}
getFont = () => this.font;
getFontSize = () => this.fontSize;
getUnderline = () => this.underline;
getColor = () => this.color;
getWhitespacePre = () => this.whitespacePre;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/tsconfig.json
+11
View File
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"outFile": "../../../../resources/assets/dl4j-ui.js",
"removeComments": true,
"sourceMap": true,
"declaration": true
},
"filesGlob": [
"**/*.ts"
]
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/typedefs/d3.d.ts
Vendored
+3373
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/typedefs/jquery.d.ts
Vendored
+3209
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/typedefs/jqueryui.d.ts
Vendored
+1855
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-components/src/main/typescript/org/deeplearning4j/ui/util/TSUtils.ts
+66
View File
@@ -0,0 +1,66 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
class TSUtils {
//Get the maximum value
static max(input: number[][]): number {
var max: number = -Number.MAX_VALUE;
for (var i = 0; i < input.length; i++) {
for( var j=0; j<input[i].length; j++ ){
max = Math.max(max,input[i][j]);
}
}
return max;
}
//Get the minimum value
static min(input: number[][]): number {
var min: number = Number.MAX_VALUE;
for (var i = 0; i < input.length; i++) {
for( var j=0; j<input[i].length; j++ ){
min = Math.min(min,input[i][j]);
}
}
return min;
}
//Normalize the length unit (for example, parse the LengthUnit enum values)
static normalizeLengthUnit(input: string): string{
if(input == null) return input;
switch(input.toLowerCase()){
case "px":
return "px";
case "percent":
case "%":
return "%";
case "cm":
return "cm";
case "mm":
return "mm";
case "in":
return "in";
default:
return input;
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/pom.xml
+132
View File
@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ /* ******************************************************************************
~ *
~ *
~ * This program and the accompanying materials are made available under the
~ * terms of the Apache License, Version 2.0 which is available at
~ * https://www.apache.org/licenses/LICENSE-2.0.
~ *
~ * See the NOTICE file distributed with this work for additional
~ * information regarding copyright ownership.
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
~ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
~ * License for the specific language governing permissions and limitations
~ * under the License.
~ *
~ * SPDX-License-Identifier: Apache-2.0
~ ******************************************************************************/
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>deeplearning4j-ui-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>deeplearning4j-ui-model</artifactId>
<name>deeplearning4j-ui-model</name>
<properties>
<module.name>deeplearning4j.ui.model</module.name>
</properties>
<build>
<plugins>
<!-- <plugin>
<groupId>org.moditect</groupId>
<artifactId>moditect-maven-plugin</artifactId>
</plugin>-->
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.lz4</groupId>
<artifactId>lz4-pure-java</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>${project.version}</version>
</dependency>
<!-- ND4J APIs -->
<dependency>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>nd4j-api</artifactId>
<version>${nd4j.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>nd4j-native-api</artifactId>
<version>${nd4j.version}</version>
</dependency>
<!-- MapDB - Used for StatsListener storage -->
<dependency>
<groupId>org.mapdb</groupId>
<artifactId>mapdb</artifactId>
<version>${mapdb.version}</version>
</dependency>
<!-- SQLite - Used only for Java-7 Compatible StatsListener Storage -->
<dependency>
<groupId>org.xerial</groupId>
<artifactId>sqlite-jdbc</artifactId>
<version>${sqlite.version}</version>
</dependency>
<!-- Workaround for Java 9+ -->
<dependency>
<groupId>javax.annotation</groupId>
<artifactId>javax.annotation-api</artifactId>
<version>${javax.annotation-api.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-launcher</artifactId>
<version>${junit.platform.launcher.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.eclipse.deeplearning4j</groupId>
<artifactId>deeplearning4j-common-tests</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.aeron</groupId>
<artifactId>aeron-all</artifactId>
<version>1.39.0</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/activation/PathUpdate.java
+35
View File
@@ -0,0 +1,35 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.activation;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.io.Serializable;
/**
* @author Adam Gibson
*/
public @Data @NoArgsConstructor class PathUpdate implements Serializable {
private String path;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/nearestneighbors/word2vec/NearestNeighborsQuery.java
+76
View File
@@ -0,0 +1,76 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.nearestneighbors.word2vec;
import java.io.Serializable;
/**
* @author Adam Gibson
*/
public class NearestNeighborsQuery implements Serializable {
private String word;
private int numWords;
public NearestNeighborsQuery(String word, int numWords) {
this.word = word;
this.numWords = numWords;
}
public NearestNeighborsQuery() {}
public String getWord() {
return word;
}
public void setWord(String word) {
this.word = word;
}
public int getNumWords() {
return numWords;
}
public void setNumWords(int numWords) {
this.numWords = numWords;
}
@Override
public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
NearestNeighborsQuery that = (NearestNeighborsQuery) o;
if (numWords != that.numWords)
return false;
return !(word != null ? !word.equals(that.word) : that.word != null);
}
@Override
public int hashCode() {
int result = word != null ? word.hashCode() : 0;
result = 31 * result + numWords;
return result;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/renders/PathUpdate.java
+35
View File
@@ -0,0 +1,35 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.renders;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.io.Serializable;
/**
* @author Adam Gibson
*/
public @Data @NoArgsConstructor class PathUpdate implements Serializable {
private String path;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/BaseStatsListener.java
+789
View File
@@ -0,0 +1,789 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.IOUtils;
import org.bytedeco.javacpp.Pointer;
import org.deeplearning4j.config.DL4JClassLoading;
import org.deeplearning4j.core.storage.StatsStorageRouter;
import org.deeplearning4j.core.storage.StorageMetaData;
import org.deeplearning4j.core.storage.listener.RoutingIterationListener;
import org.deeplearning4j.nn.api.Layer;
import org.deeplearning4j.nn.api.Model;
import org.deeplearning4j.nn.conf.NeuralNetConfiguration;
import org.deeplearning4j.nn.gradient.Gradient;
import org.deeplearning4j.nn.graph.ComputationGraph;
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork;
import org.deeplearning4j.ui.model.stats.api.*;
import org.deeplearning4j.ui.model.storage.FileStatsStorage;
import org.deeplearning4j.ui.model.storage.InMemoryStatsStorage;
import org.deeplearning4j.ui.model.stats.impl.DefaultStatsInitializationConfiguration;
import org.deeplearning4j.ui.model.stats.impl.DefaultStatsUpdateConfiguration;
import org.deeplearning4j.core.util.UIDProvider;
import org.nd4j.linalg.api.buffer.util.DataTypeUtil;
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.factory.Nd4j;
import org.nd4j.common.primitives.Pair;
import org.nd4j.nativeblas.NativeOps;
import org.nd4j.nativeblas.NativeOpsHolder;
import java.io.InputStream;
import java.io.Serializable;
import java.lang.management.GarbageCollectorMXBean;
import java.lang.management.ManagementFactory;
import java.lang.management.OperatingSystemMXBean;
import java.lang.management.RuntimeMXBean;
import java.util.*;
@Slf4j
public abstract class BaseStatsListener implements RoutingIterationListener {
public static final String TYPE_ID = "StatsListener";
private enum StatType {
Mean, Stdev, MeanMagnitude
}
private StatsStorageRouter router;
private final StatsInitializationConfiguration initConfig;
private StatsUpdateConfiguration updateConfig;
private String sessionID;
private String workerID;
private transient List<GarbageCollectorMXBean> gcBeans;
private Map<String, Pair<Long, Long>> gcStatsAtLastReport;
//NOTE: may have multiple models, due to multiple pretrain layers all using the same StatsListener
private List<ModelInfo> modelInfos = new ArrayList<>();
private Map<String, Histogram> activationHistograms;
private Map<String, Double> meanActivations; //TODO replace with Eclipse collections primitive maps...
private Map<String, Double> stdevActivations;
private Map<String, Double> meanMagActivations;
private Map<String, Histogram> gradientHistograms;
private Map<String, Double> meanGradients; //TODO replace with Eclipse collections primitive maps...
private Map<String, Double> stdevGradient;
private Map<String, Double> meanMagGradients;
private static class ModelInfo implements Serializable {
private final Model model;
private long initTime;
private long lastReportTime = -1;
private int lastReportIteration = -1;
private int examplesSinceLastReport = 0;
private int minibatchesSinceLastReport = 0;
private long totalExamples = 0;
private long totalMinibatches = 0;
private int iterCount = 0;
private ModelInfo(Model model) {
this.model = model;
}
}
private ModelInfo getModelInfo(Model model) {
ModelInfo mi = null;
for (ModelInfo m : modelInfos) {
if (m.model == model) {
mi = m;
break;
}
}
if (mi == null) {
mi = new ModelInfo(model);
modelInfos.add(mi);
}
return mi;
}
/**
* Create a StatsListener with network information collected at every iteration.
*
* @param router Where/how to store the calculated stats. For example, {@link InMemoryStatsStorage} or
* {@link FileStatsStorage}
*/
public BaseStatsListener(StatsStorageRouter router) {
this(router, null, null, null, null);
}
/**
* Create a StatsListener with network information collected every n >= 1 time steps
*
* @param router Where/how to store the calculated stats. For example, {@link InMemoryStatsStorage} or
* {@link FileStatsStorage}
* @param listenerFrequency Frequency with which to collect stats information
*/
public BaseStatsListener(StatsStorageRouter router, int listenerFrequency) {
this(router, null, new DefaultStatsUpdateConfiguration.Builder().reportingFrequency(listenerFrequency).build(),
null, null);
}
public BaseStatsListener(StatsStorageRouter router, StatsInitializationConfiguration initConfig,
StatsUpdateConfiguration updateConfig, String sessionID, String workerID) {
this.router = router;
if (initConfig == null) {
this.initConfig = new DefaultStatsInitializationConfiguration(true, true, true);
} else {
this.initConfig = initConfig;
}
if (updateConfig == null) {
this.updateConfig = new DefaultStatsUpdateConfiguration.Builder().build();
} else {
this.updateConfig = updateConfig;
}
if (sessionID == null) {
//TODO handle syncing session IDs across different listeners in the same model...
this.sessionID = UUID.randomUUID().toString();
} else {
this.sessionID = sessionID;
}
if (workerID == null) {
this.workerID = UIDProvider.getJVMUID() + "_" + Thread.currentThread().getId();
} else {
this.workerID = workerID;
}
}
public abstract StatsInitializationReport getNewInitializationReport();
public abstract StatsReport getNewStatsReport();
// public abstract StorageMetaData getNewStorageMetaData();
public abstract StorageMetaData getNewStorageMetaData(long initTime, String sessionID, String workerID);
// Class<? extends StatsInitializationReport> initializationReportClass,
// Class<? extends StatsReport> statsReportClass);
//new SbeStorageMetaData(initTime, getSessionID(model), TYPE_ID, workerID, SbeStatsInitializationReport.class, SbeStatsReport.class);
public StatsInitializationConfiguration getInitConfig() {
return initConfig;
}
public StatsUpdateConfiguration getUpdateConfig() {
return updateConfig;
}
public void setUpdateConfig(StatsUpdateConfiguration newConfig) {
this.updateConfig = newConfig;
}
@Override
public void setStorageRouter(StatsStorageRouter router) {
this.router = router;
}
@Override
public StatsStorageRouter getStorageRouter() {
return router;
}
@Override
public void setWorkerID(String workerID) {
this.workerID = workerID;
}
@Override
public String getWorkerID() {
return workerID;
}
@Override
public void setSessionID(String sessionID) {
this.sessionID = sessionID;
}
@Override
public String getSessionID() {
return sessionID;
}
private String getSessionID(Model model) {
if (model instanceof MultiLayerNetwork || model instanceof ComputationGraph)
return sessionID;
if (model instanceof Layer) {
//Keep in mind MultiLayerNetwork implements Layer also...
Layer l = (Layer) model;
int layerIdx = l.getIndex();
return sessionID + "_layer" + layerIdx;
}
return sessionID; //Should never happen
}
@Override
public void onEpochStart(Model model) {
}
@Override
public void onEpochEnd(Model model) {
}
@Override
public void onForwardPass(Model model, List<INDArray> activations) {
int iterCount = getModelInfo(model).iterCount;
if (calcFromActivations() && (iterCount == 0 || iterCount % updateConfig.reportingFrequency() == 0)) {
//Assumption: we have input, layer 0, layer 1, ...
Map<String, INDArray> activationsMap = new HashMap<>();
int count = 0;
for (INDArray arr : activations) {
String layerName = (count == 0 ? "input" : String.valueOf(count - 1));
activationsMap.put(layerName, arr);
count++;
}
onForwardPass(model, activationsMap);
}
}
@Override
public void onForwardPass(Model model, Map<String, INDArray> activations) {
int iterCount = getModelInfo(model).iterCount;
if (calcFromActivations() && updateConfig.reportingFrequency() > 0
&& (iterCount == 0 || iterCount % updateConfig.reportingFrequency() == 0)) {
if (updateConfig.collectHistograms(StatsType.Activations)) {
activationHistograms = getHistograms(activations, updateConfig.numHistogramBins(StatsType.Activations));
}
if (updateConfig.collectMean(StatsType.Activations)) {
meanActivations = calculateSummaryStats(activations, StatType.Mean);
}
if (updateConfig.collectStdev(StatsType.Activations)) {
stdevActivations = calculateSummaryStats(activations, StatType.Stdev);
}
if (updateConfig.collectMeanMagnitudes(StatsType.Activations)) {
meanMagActivations = calculateSummaryStats(activations, StatType.MeanMagnitude);
}
}
}
@Override
public void onGradientCalculation(Model model) {
int iterCount = getModelInfo(model).iterCount;
if (calcFromGradients() && updateConfig.reportingFrequency() > 0
&& (iterCount == 0 || iterCount % updateConfig.reportingFrequency() == 0)) {
Gradient g = model.gradient();
if (updateConfig.collectHistograms(StatsType.Gradients)) {
gradientHistograms = getHistograms(g.gradientForVariable(), updateConfig.numHistogramBins(StatsType.Gradients));
}
if (updateConfig.collectMean(StatsType.Gradients)) {
meanGradients = calculateSummaryStats(g.gradientForVariable(), StatType.Mean);
}
if (updateConfig.collectStdev(StatsType.Gradients)) {
stdevGradient = calculateSummaryStats(g.gradientForVariable(), StatType.Stdev);
}
if (updateConfig.collectMeanMagnitudes(StatsType.Gradients)) {
meanMagGradients = calculateSummaryStats(g.gradientForVariable(), StatType.MeanMagnitude);
}
}
}
private boolean calcFromActivations() {
return updateConfig.collectMean(StatsType.Activations) || updateConfig.collectStdev(StatsType.Activations)
|| updateConfig.collectMeanMagnitudes(StatsType.Activations)
|| updateConfig.collectHistograms(StatsType.Activations);
}
private boolean calcFromGradients() {
return updateConfig.collectMean(StatsType.Gradients) || updateConfig.collectStdev(StatsType.Gradients)
|| updateConfig.collectMeanMagnitudes(StatsType.Gradients)
|| updateConfig.collectHistograms(StatsType.Gradients);
}
@Override
public void onBackwardPass(Model model) {
//No op
}
@Override
public void iterationDone(Model model, int iteration, int epoch) {
ModelInfo modelInfo = getModelInfo(model);
boolean backpropParamsOnly = backpropParamsOnly(model);
long currentTime = getTime();
if (modelInfo.iterCount == 0) {
modelInfo.initTime = currentTime;
doInit(model);
}
if (updateConfig.collectPerformanceStats()) {
updateExamplesMinibatchesCounts(model);
}
if (updateConfig.reportingFrequency() > 1 && (iteration == 0 || iteration % updateConfig.reportingFrequency() != 0)) {
modelInfo.iterCount = iteration;
return;
}
StatsReport report = getNewStatsReport();
report.reportIDs(getSessionID(model), TYPE_ID, workerID, System.currentTimeMillis()); //TODO support NTP time
//--- Performance and System Stats ---
if (updateConfig.collectPerformanceStats()) {
//Stats to collect: total runtime, total examples, total minibatches, iterations/second, examples/second
double examplesPerSecond;
double minibatchesPerSecond;
if (modelInfo.iterCount == 0) {
//Not possible to work out perf/second: first iteration...
examplesPerSecond = 0.0;
minibatchesPerSecond = 0.0;
} else {
long deltaTimeMS = currentTime - modelInfo.lastReportTime;
examplesPerSecond = 1000.0 * modelInfo.examplesSinceLastReport / deltaTimeMS;
minibatchesPerSecond = 1000.0 * modelInfo.minibatchesSinceLastReport / deltaTimeMS;
}
long totalRuntimeMS = currentTime - modelInfo.initTime;
report.reportPerformance(totalRuntimeMS, modelInfo.totalExamples, modelInfo.totalMinibatches,
examplesPerSecond, minibatchesPerSecond);
modelInfo.examplesSinceLastReport = 0;
modelInfo.minibatchesSinceLastReport = 0;
}
if (updateConfig.collectMemoryStats()) {
Runtime runtime = Runtime.getRuntime();
long jvmTotal = runtime.totalMemory();
long jvmMax = runtime.maxMemory();
//Off-heap memory
long offheapTotal = Pointer.totalBytes();
long offheapMax = Pointer.maxBytes();
//GPU
long[] gpuCurrentBytes = null;
long[] gpuMaxBytes = null;
NativeOps nativeOps =Nd4j.getNativeOps();
int nDevices = nativeOps.getAvailableDevices();
if (nDevices > 0) {
gpuCurrentBytes = new long[nDevices];
gpuMaxBytes = new long[nDevices];
for (int i = 0; i < nDevices; i++) {
try {
gpuMaxBytes[i] = nativeOps.getDeviceTotalMemory(0);
gpuCurrentBytes[i] = gpuMaxBytes[i] - nativeOps.getDeviceFreeMemory(0);
} catch (Exception e) {
log.error("",e);
}
}
}
report.reportMemoryUse(jvmTotal, jvmMax, offheapTotal, offheapMax, gpuCurrentBytes, gpuMaxBytes);
}
if (updateConfig.collectGarbageCollectionStats()) {
if (modelInfo.lastReportIteration == -1 || gcBeans == null) {
//Haven't reported GC stats before...
gcBeans = ManagementFactory.getGarbageCollectorMXBeans();
gcStatsAtLastReport = new HashMap<>();
for (GarbageCollectorMXBean bean : gcBeans) {
long count = bean.getCollectionCount();
long timeMs = bean.getCollectionTime();
gcStatsAtLastReport.put(bean.getName(), new Pair<>(count, timeMs));
}
} else {
for (GarbageCollectorMXBean bean : gcBeans) {
long count = bean.getCollectionCount();
long timeMs = bean.getCollectionTime();
Pair<Long, Long> lastStats = gcStatsAtLastReport.get(bean.getName());
long deltaGCCount = count - lastStats.getFirst();
long deltaGCTime = timeMs - lastStats.getSecond();
lastStats.setFirst(count);
lastStats.setSecond(timeMs);
report.reportGarbageCollection(bean.getName(), (int) deltaGCCount, (int) deltaGCTime);
}
}
}
//--- General ---
report.reportScore(model.score()); //Always report score
if (updateConfig.collectLearningRates()) {
Map<String, Double> lrs = new HashMap<>();
if (model instanceof MultiLayerNetwork) {
//Need to append "0_", "1_" etc to param names from layers...
int layerIdx = 0;
for (Layer l : ((MultiLayerNetwork) model).getLayers()) {
NeuralNetConfiguration conf = l.conf();
List<String> paramkeys = l.conf().getLayer().initializer().paramKeys(l.conf().getLayer());
for (String s : paramkeys) {
double lr = conf.getLayer().getUpdaterByParam(s).getLearningRate(l.getIterationCount(), l.getEpochCount());
if (Double.isNaN(lr)) {
//Edge case: No-Op updater, AdaDelta etc - don't have a LR hence return NaN for IUpdater.getLearningRate
lr = 0.0;
}
lrs.put(layerIdx + "_" + s, lr);
}
layerIdx++;
}
} else if (model instanceof ComputationGraph) {
for (Layer l : ((ComputationGraph) model).getLayers()) {
NeuralNetConfiguration conf = l.conf();
String layerName = conf.getLayer().getLayerName();
List<String> paramkeys = l.conf().getLayer().initializer().paramKeys(l.conf().getLayer());
for (String s : paramkeys) {
double lr = conf.getLayer().getUpdaterByParam(s).getLearningRate(l.getIterationCount(), l.getEpochCount());
if (Double.isNaN(lr)) {
//Edge case: No-Op updater, AdaDelta etc - don't have a LR hence return NaN for IUpdater.getLearningRate
lr = 0.0;
}
lrs.put(layerName + "_" + s, lr);
}
}
} else if (model instanceof Layer) {
Layer l = (Layer) model;
List<String> paramkeys = l.conf().getLayer().initializer().paramKeys(l.conf().getLayer());
for (String s : paramkeys) {
double lr = l.conf().getLayer().getUpdaterByParam(s).getLearningRate(l.getIterationCount(), l.getEpochCount());
lrs.put(s, lr);
}
}
report.reportLearningRates(lrs);
}
//--- Histograms ---
if (updateConfig.collectHistograms(StatsType.Parameters)) {
Map<String, Histogram> paramHistograms = getHistograms(model.paramTable(backpropParamsOnly),
updateConfig.numHistogramBins(StatsType.Parameters));
report.reportHistograms(StatsType.Parameters, paramHistograms);
}
if (updateConfig.collectHistograms(StatsType.Gradients)) {
report.reportHistograms(StatsType.Gradients, gradientHistograms);
}
if (updateConfig.collectHistograms(StatsType.Updates)) {
Map<String, Histogram> updateHistograms = getHistograms(model.gradient().gradientForVariable(),
updateConfig.numHistogramBins(StatsType.Updates));
report.reportHistograms(StatsType.Updates, updateHistograms);
}
if (updateConfig.collectHistograms(StatsType.Activations)) {
report.reportHistograms(StatsType.Activations, activationHistograms);
}
//--- Summary Stats: Mean, Variance, Mean Magnitudes ---
if (updateConfig.collectMean(StatsType.Parameters)) {
Map<String, Double> meanParams = calculateSummaryStats(model.paramTable(backpropParamsOnly), StatType.Mean);
report.reportMean(StatsType.Parameters, meanParams);
}
if (updateConfig.collectMean(StatsType.Gradients)) {
report.reportMean(StatsType.Gradients, meanGradients);
}
if (updateConfig.collectMean(StatsType.Updates)) {
Map<String, Double> meanUpdates =
calculateSummaryStats(model.gradient().gradientForVariable(), StatType.Mean);
report.reportMean(StatsType.Updates, meanUpdates);
}
if (updateConfig.collectMean(StatsType.Activations)) {
report.reportMean(StatsType.Activations, meanActivations);
}
if (updateConfig.collectStdev(StatsType.Parameters)) {
Map<String, Double> stdevParams =
calculateSummaryStats(model.paramTable(backpropParamsOnly), StatType.Stdev);
report.reportStdev(StatsType.Parameters, stdevParams);
}
if (updateConfig.collectStdev(StatsType.Gradients)) {
report.reportStdev(StatsType.Gradients, stdevGradient);
}
if (updateConfig.collectStdev(StatsType.Updates)) {
Map<String, Double> stdevUpdates =
calculateSummaryStats(model.gradient().gradientForVariable(), StatType.Stdev);
report.reportStdev(StatsType.Updates, stdevUpdates);
}
if (updateConfig.collectStdev(StatsType.Activations)) {
report.reportStdev(StatsType.Activations, stdevActivations);
}
if (updateConfig.collectMeanMagnitudes(StatsType.Parameters)) {
Map<String, Double> meanMagParams =
calculateSummaryStats(model.paramTable(backpropParamsOnly), StatType.MeanMagnitude);
report.reportMeanMagnitudes(StatsType.Parameters, meanMagParams);
}
if (updateConfig.collectMeanMagnitudes(StatsType.Gradients)) {
report.reportMeanMagnitudes(StatsType.Gradients, meanMagGradients);
}
if (updateConfig.collectMeanMagnitudes(StatsType.Updates)) {
Map<String, Double> meanMagUpdates =
calculateSummaryStats(model.gradient().gradientForVariable(), StatType.MeanMagnitude);
report.reportMeanMagnitudes(StatsType.Updates, meanMagUpdates);
}
if (updateConfig.collectMeanMagnitudes(StatsType.Activations)) {
report.reportMeanMagnitudes(StatsType.Activations, meanMagActivations);
}
long endTime = getTime();
report.reportStatsCollectionDurationMS((int) (endTime - currentTime)); //Amount of time required to alculate all histograms, means etc.
modelInfo.lastReportTime = currentTime;
modelInfo.lastReportIteration = iteration;
report.reportIterationCount(iteration);
this.router.putUpdate(report);
modelInfo.iterCount = iteration;
activationHistograms = null;
meanActivations = null;
stdevActivations = null;
meanMagActivations = null;
gradientHistograms = null;
meanGradients = null;
stdevGradient = null;
meanMagGradients = null;
}
private long getTime() {
//Abstraction to allow NTP to be plugged in later...
return System.currentTimeMillis();
}
private void doInit(Model model) {
boolean backpropParamsOnly = backpropParamsOnly(model);
long initTime = System.currentTimeMillis(); //TODO support NTP
StatsInitializationReport initReport = getNewInitializationReport();
initReport.reportIDs(getSessionID(model), TYPE_ID, workerID, initTime);
if (initConfig.collectSoftwareInfo()) {
OperatingSystemMXBean osBean = ManagementFactory.getOperatingSystemMXBean();
RuntimeMXBean runtime = ManagementFactory.getRuntimeMXBean();
String arch = osBean.getArch();
String osName = osBean.getName();
String jvmName = runtime.getVmName();
String jvmVersion = System.getProperty("java.version");
String jvmSpecVersion = runtime.getSpecVersion();
String nd4jBackendClass = Nd4j.getNDArrayFactory().getClass().getName();
String nd4jDataTypeName = DataTypeUtil.getDtypeFromContext().name();
String hostname = System.getenv("COMPUTERNAME");
if (hostname == null || hostname.isEmpty()) {
try {
Process proc = Runtime.getRuntime().exec("hostname");
try (InputStream stream = proc.getInputStream()) {
hostname = IOUtils.toString(stream);
}
} catch (Exception e) {
}
}
Properties p = Nd4j.getExecutioner().getEnvironmentInformation();
Map<String, String> envInfo = new HashMap<>();
for (Map.Entry<Object, Object> e : p.entrySet()) {
Object v = e.getValue();
String value = (v == null ? "" : v.toString());
envInfo.put(e.getKey().toString(), value);
}
initReport.reportSoftwareInfo(arch, osName, jvmName, jvmVersion, jvmSpecVersion, nd4jBackendClass,
nd4jDataTypeName, hostname, UIDProvider.getJVMUID(), envInfo);
}
if (initConfig.collectHardwareInfo()) {
int availableProcessors = Runtime.getRuntime().availableProcessors();
NativeOps nativeOps =Nd4j.getNativeOps();
int nDevices = nativeOps.getAvailableDevices();
long[] deviceTotalMem = null;
String[] deviceDescription = null; //TODO
if (nDevices > 0) {
deviceTotalMem = new long[nDevices];
deviceDescription = new String[nDevices];
for (int i = 0; i < nDevices; i++) {
try {
deviceTotalMem[i] = nativeOps.getDeviceTotalMemory(i);
deviceDescription[i] = nativeOps.getDeviceName(i);
if (nDevices > 1) {
deviceDescription[i] = deviceDescription[i] + " (" + i + ")";
}
} catch (Exception e) {
log.debug("Error getting device info", e);
}
}
}
long jvmMaxMemory = Runtime.getRuntime().maxMemory();
long offheapMaxMemory = Pointer.maxBytes();
initReport.reportHardwareInfo(availableProcessors, nDevices, jvmMaxMemory, offheapMaxMemory, deviceTotalMem,
deviceDescription, UIDProvider.getHardwareUID());
}
if (initConfig.collectModelInfo()) {
String jsonConf;
int numLayers;
long numParams;
if (model instanceof MultiLayerNetwork) {
MultiLayerNetwork net = ((MultiLayerNetwork) model);
jsonConf = net.getLayerWiseConfigurations().toJson();
numLayers = net.getnLayers();
numParams = net.numParams();
} else if (model instanceof ComputationGraph) {
ComputationGraph cg = ((ComputationGraph) model);
jsonConf = cg.getConfiguration().toJson();
numLayers = cg.getNumLayers();
numParams = cg.numParams();
} else if (model instanceof Layer) {
Layer l = (Layer) model;
jsonConf = l.conf().toJson();
numLayers = 1;
numParams = l.numParams();
} else {
throw new RuntimeException("Invalid model: Expected MultiLayerNetwork or ComputationGraph. Got: "
+ (model == null ? null : model.getClass()));
}
Map<String, INDArray> paramMap = model.paramTable(backpropParamsOnly);
String[] paramNames = new String[paramMap.size()];
int i = 0;
for (String s : paramMap.keySet()) { //Assuming sensible iteration order - LinkedHashMaps are used in MLN/CG for example
paramNames[i++] = s;
}
initReport.reportModelInfo(model.getClass().getName(), jsonConf, paramNames, numLayers, numParams);
}
StorageMetaData meta = getNewStorageMetaData(initTime, getSessionID(model), workerID);
router.putStorageMetaData(meta);
router.putStaticInfo(initReport); //TODO error handling
}
private Map<Integer, Pointer> devPointers = new HashMap<>();
private synchronized Pointer getDevicePointer(int device) {
if (devPointers.containsKey(device)) {
return devPointers.get(device);
}
try {
Pointer pointer = DL4JClassLoading.createNewInstance(
"org.nd4j.jita.allocator.pointers.CudaPointer",
Pointer.class,
new Class[] { long.class },
new Object[]{(long) device});
devPointers.put(device, pointer);
return pointer;
} catch (Throwable t) {
devPointers.put(device, null); //Stops attempting the failure again later...
return null;
}
}
private void updateExamplesMinibatchesCounts(Model model) {
ModelInfo modelInfo = getModelInfo(model);
int examplesThisMinibatch = 0;
if (model instanceof MultiLayerNetwork) {
examplesThisMinibatch = model.batchSize();
} else if (model instanceof ComputationGraph) {
examplesThisMinibatch = model.batchSize();
} else if (model instanceof Layer) {
examplesThisMinibatch = ((Layer) model).getInputMiniBatchSize();
}
modelInfo.examplesSinceLastReport += examplesThisMinibatch;
modelInfo.totalExamples += examplesThisMinibatch;
modelInfo.minibatchesSinceLastReport++;
modelInfo.totalMinibatches++;
}
private boolean backpropParamsOnly(Model model) {
//For pretrain layers (VAE, AE) we *do* want pretrain params also; for MLN and CG we only want backprop params
// as we only have backprop gradients
return model instanceof MultiLayerNetwork || model instanceof ComputationGraph;
}
private static Map<String, Double> calculateSummaryStats(Map<String, INDArray> source, StatType statType) {
Map<String, Double> out = new LinkedHashMap<>();
if (source == null)
return out;
for (Map.Entry<String, INDArray> entry : source.entrySet()) {
String name = entry.getKey();
double value;
switch (statType) {
case Mean:
value = entry.getValue().meanNumber().doubleValue();
break;
case Stdev:
value = entry.getValue().stdNumber().doubleValue();
break;
case MeanMagnitude:
value = entry.getValue().norm1Number().doubleValue() / entry.getValue().length();
break;
default:
throw new RuntimeException(); //Should never happen
}
out.put(name, value);
}
return out;
}
private static Map<String, Histogram> getHistograms(Map<String, INDArray> map, int nBins) {
Map<String, Histogram> out = new LinkedHashMap<>();
if (map == null)
return out;
for (Map.Entry<String, INDArray> entry : map.entrySet()) {
org.nd4j.linalg.api.ops.impl.transforms.Histogram hOp =
new org.nd4j.linalg.api.ops.impl.transforms.Histogram(entry.getValue(), nBins);
Nd4j.exec(hOp);
INDArray bins = hOp.getOutputArgument(0);
int[] count = new int[nBins];
for (int i = 0; i < bins.length(); i++) {
count[i] = (int) bins.getDouble(i);
}
double min = entry.getValue().minNumber().doubleValue();
double max = entry.getValue().maxNumber().doubleValue();
Histogram h = new Histogram(min, max, nBins, count);
out.put(entry.getKey(), h);
}
return out;
}
@Override
public abstract BaseStatsListener clone();
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/J7StatsListener.java
+88
View File
@@ -0,0 +1,88 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats;
import lombok.extern.slf4j.Slf4j;
import org.deeplearning4j.core.storage.StatsStorageRouter;
import org.deeplearning4j.core.storage.StorageMetaData;
import org.deeplearning4j.ui.model.stats.api.StatsInitializationConfiguration;
import org.deeplearning4j.ui.model.stats.api.StatsInitializationReport;
import org.deeplearning4j.ui.model.stats.api.StatsReport;
import org.deeplearning4j.ui.model.stats.api.StatsUpdateConfiguration;
import org.deeplearning4j.ui.model.storage.FileStatsStorage;
import org.deeplearning4j.ui.model.storage.InMemoryStatsStorage;
import org.deeplearning4j.ui.model.storage.impl.JavaStorageMetaData;
import org.deeplearning4j.ui.model.stats.impl.DefaultStatsUpdateConfiguration;
import org.deeplearning4j.ui.model.stats.impl.java.JavaStatsInitializationReport;
import org.deeplearning4j.ui.model.stats.impl.java.JavaStatsReport;
@Slf4j
public class J7StatsListener extends BaseStatsListener {
/**
* Create a StatsListener with network information collected at every iteration. Equivalent to {@link #J7StatsListener(StatsStorageRouter, int)}
* with {@code listenerFrequency == 1}
*
* @param router Where/how to store the calculated stats. For example, {@link InMemoryStatsStorage} or
* {@link FileStatsStorage}
*/
public J7StatsListener(StatsStorageRouter router) {
this(router, null, null, null, null);
}
/**
* Create a StatsListener with network information collected every n >= 1 time steps
*
* @param router Where/how to store the calculated stats. For example, {@link InMemoryStatsStorage} or
* {@link FileStatsStorage}
* @param listenerFrequency Frequency with which to collect stats information
*/
public J7StatsListener(StatsStorageRouter router, int listenerFrequency) {
this(router, null, new DefaultStatsUpdateConfiguration.Builder().reportingFrequency(listenerFrequency).build(),
null, null);
}
public J7StatsListener(StatsStorageRouter router, StatsInitializationConfiguration initConfig,
StatsUpdateConfiguration updateConfig, String sessionID, String workerID) {
super(router, initConfig, updateConfig, sessionID, workerID);
}
@Override
public StatsInitializationReport getNewInitializationReport() {
return new JavaStatsInitializationReport();
}
@Override
public StatsReport getNewStatsReport() {
return new JavaStatsReport();
}
@Override
public StorageMetaData getNewStorageMetaData(long initTime, String sessionID, String workerID) {
return new JavaStorageMetaData(initTime, sessionID, TYPE_ID, workerID,
JavaStatsInitializationReport.class, JavaStatsReport.class);
}
@Override
public J7StatsListener clone() {
return new J7StatsListener(this.getStorageRouter(), this.getInitConfig(), this.getUpdateConfig(), null, null);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/StatsListener.java
+99
View File
@@ -0,0 +1,99 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats;
import lombok.extern.slf4j.Slf4j;
import org.deeplearning4j.core.storage.StatsStorageRouter;
import org.deeplearning4j.core.storage.StorageMetaData;
import org.deeplearning4j.ui.model.stats.api.StatsInitializationConfiguration;
import org.deeplearning4j.ui.model.stats.api.StatsInitializationReport;
import org.deeplearning4j.ui.model.stats.api.StatsReport;
import org.deeplearning4j.ui.model.stats.api.StatsUpdateConfiguration;
import org.deeplearning4j.ui.model.storage.FileStatsStorage;
import org.deeplearning4j.ui.model.storage.InMemoryStatsStorage;
import org.deeplearning4j.ui.model.storage.impl.SbeStorageMetaData;
import org.deeplearning4j.ui.model.stats.impl.DefaultStatsUpdateConfiguration;
import org.deeplearning4j.ui.model.stats.impl.SbeStatsInitializationReport;
import org.deeplearning4j.ui.model.stats.impl.SbeStatsReport;
@Slf4j
public class StatsListener extends BaseStatsListener {
/**
* Create a StatsListener with network information collected at every iteration. Equivalent to {@link #StatsListener(StatsStorageRouter, int)}
* with {@code listenerFrequency == 1}
*
* @param router Where/how to store the calculated stats. For example, {@link InMemoryStatsStorage} or
* {@link FileStatsStorage}
*/
public StatsListener(StatsStorageRouter router) {
this(router, null, null, null, null);
}
/**
* Create a StatsListener with network information collected every n >= 1 time steps
*
* @param router Where/how to store the calculated stats. For example, {@link InMemoryStatsStorage} or
* {@link FileStatsStorage}
* @param listenerFrequency Frequency with which to collect stats information
*/
public StatsListener(StatsStorageRouter router, int listenerFrequency) {
this(router, listenerFrequency, null);
}
/**
* Create a StatsListener with network information collected every n >= 1 time steps
*
* @param router Where/how to store the calculated stats. For example, {@link InMemoryStatsStorage} or
* {@link FileStatsStorage}
* @param listenerFrequency Frequency with which to collect stats information
* @param sessionId The Session ID for storing the stats, optional (may be null)
*/
public StatsListener(StatsStorageRouter router, int listenerFrequency, String sessionId) {
this(router, null, new DefaultStatsUpdateConfiguration.Builder().reportingFrequency(listenerFrequency).build(),
sessionId, null);
}
public StatsListener(StatsStorageRouter router, StatsInitializationConfiguration initConfig,
StatsUpdateConfiguration updateConfig, String sessionID, String workerID) {
super(router, initConfig, updateConfig, sessionID, workerID);
}
public StatsListener clone() {
return new StatsListener(this.getStorageRouter(), this.getInitConfig(), this.getUpdateConfig(), null, null);
}
@Override
public StatsInitializationReport getNewInitializationReport() {
return new SbeStatsInitializationReport();
}
@Override
public StatsReport getNewStatsReport() {
return new SbeStatsReport();
}
@Override
public StorageMetaData getNewStorageMetaData(long initTime, String sessionID, String workerID) {
return new SbeStorageMetaData(initTime, sessionID, BaseStatsListener.TYPE_ID, workerID,
SbeStatsInitializationReport.class, SbeStatsReport.class);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/api/Histogram.java
+37
View File
@@ -0,0 +1,37 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.api;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.io.Serializable;
@AllArgsConstructor
@NoArgsConstructor
@Data
public class Histogram implements Serializable {
private double min;
private double max;
private int nBins;
private int[] binCounts;
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/api/StatsInitializationConfiguration.java
+50
View File
@@ -0,0 +1,50 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.api;
import org.deeplearning4j.ui.model.stats.StatsListener;
import java.io.Serializable;
public interface StatsInitializationConfiguration extends Serializable {
/**
* Should software configuration information be collected? For example, OS, JVM, and ND4J backend details
*
* @return true if software information should be collected; false if not
*/
boolean collectSoftwareInfo();
/**
* Should hardware configuration information be collected? JVM available processors, number of devices, total memory for each device
*
* @return true if hardware information should be collected
*/
boolean collectHardwareInfo();
/**
* Should model information be collected? Model class, configuration (JSON), number of layers, number of parameters, etc.
*
* @return true if model information should be collected
*/
boolean collectModelInfo();
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/api/StatsInitializationReport.java
+126
View File
@@ -0,0 +1,126 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.api;
import org.deeplearning4j.core.storage.Persistable;
import org.deeplearning4j.ui.model.stats.StatsListener;
import java.util.Map;
public interface StatsInitializationReport extends Persistable {
void reportIDs(String sessionID, String typeID, String workerID, long timestamp);
/**
* @param arch Operating system architecture, as reported by JVM
* @param osName Operating system name
* @param jvmName JVM name
* @param jvmVersion JVM version
* @param jvmSpecVersion JVM Specification version (for example, 1.8)
* @param nd4jBackendClass ND4J backend Factory class
* @param nd4jDataTypeName ND4J datatype name
* @param hostname Hostname for the machine, if available
* @param jvmUID A unique identified for the current JVM. Should be shared by all instances in the same JVM.
* Should vary for different JVMs on the same machine.
* @param swEnvironmentInfo Environment information: Usually from Nd4j.getExecutioner().getEnvironmentInformation()
*/
void reportSoftwareInfo(String arch, String osName, String jvmName, String jvmVersion, String jvmSpecVersion,
String nd4jBackendClass, String nd4jDataTypeName, String hostname, String jvmUID,
Map<String, String> swEnvironmentInfo);
/**
* @param jvmAvailableProcessors Number of available processor cores according to the JVM
* @param numDevices Number of compute devices (GPUs)
* @param jvmMaxMemory Maximum memory for the JVM
* @param offHeapMaxMemory Maximum off-heap memory
* @param deviceTotalMemory GPU memory by device: same length as numDevices. May be null, if numDevices is 0
* @param deviceDescription Description of each device. May be null, if numDevices is 0
* @param hardwareUID A unique identifier for the machine. Should be shared by all instances running on
* the same machine, including in different JVMs
*
*/
void reportHardwareInfo(int jvmAvailableProcessors, int numDevices, long jvmMaxMemory, long offHeapMaxMemory,
long[] deviceTotalMemory, String[] deviceDescription, String hardwareUID);
/**
* Report the model information
*
* @param modelClassName Model class name: i.e., type of model
* @param modelConfigJson Model configuration, as JSON string
* @param numLayers Number of layers in the model
* @param numParams Number of parameters in the model
*/
void reportModelInfo(String modelClassName, String modelConfigJson, String[] paramNames, int numLayers,
long numParams);
boolean hasSoftwareInfo();
boolean hasHardwareInfo();
boolean hasModelInfo();
String getSwArch();
String getSwOsName();
String getSwJvmName();
String getSwJvmVersion();
String getSwJvmSpecVersion();
String getSwNd4jBackendClass();
String getSwNd4jDataTypeName();
String getSwHostName();
String getSwJvmUID();
Map<String, String> getSwEnvironmentInfo();
int getHwJvmAvailableProcessors();
int getHwNumDevices();
long getHwJvmMaxMemory();
long getHwOffHeapMaxMemory();
long[] getHwDeviceTotalMemory();
String[] getHwDeviceDescription();
String getHwHardwareUID();
String getModelClassName();
String getModelConfigJson();
String[] getModelParamNames();
int getModelNumLayers();
long getModelNumParams();
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/api/StatsReport.java
+322
View File
@@ -0,0 +1,322 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.api;
import org.deeplearning4j.core.storage.Persistable;
import org.deeplearning4j.ui.model.stats.StatsListener;
import org.nd4j.common.primitives.Pair;
import java.io.Serializable;
import java.util.List;
import java.util.Map;
public interface StatsReport extends Persistable {
void reportIDs(String sessionID, String typeID, String workerID, long timestamp);
/**
* Report the current iteration number
*/
void reportIterationCount(int iterationCount);
/**
* Get the current iteration number
*/
int getIterationCount();
/**
* Report the number of milliseconds required to calculate all of the stats. This is effectively the
* amount of listener overhead
*/
void reportStatsCollectionDurationMS(int statsCollectionDurationMS);
/**
* Get the number of millisecons required to calculate al of the stats. This is effectively the amount of
* listener overhead.
*/
int getStatsCollectionDurationMs();
/**
* Report model score at the current iteration
*/
void reportScore(double currentScore);
/**
* Get the score at the current iteration
*/
double getScore();
/**
* Report the learning rates by parameter
*/
void reportLearningRates(Map<String, Double> learningRatesByParam);
/**
* Get the learning rates by parameter
*/
Map<String, Double> getLearningRates();
//--- Performance and System Stats ---
/**
* Report the memory stats at this iteration
*
* @param jvmCurrentBytes Current bytes used by the JVM
* @param jvmMaxBytes Max bytes usable by the JVM (heap)
* @param offHeapCurrentBytes Current off-heap bytes used
* @param offHeapMaxBytes Maximum off-heap bytes
* @param deviceCurrentBytes Current bytes used by each device (GPU, etc). May be null if no devices are present
* @param deviceMaxBytes Maximum bytes for each device (GPU, etc). May be null if no devices are present
*/
void reportMemoryUse(long jvmCurrentBytes, long jvmMaxBytes, long offHeapCurrentBytes, long offHeapMaxBytes,
long[] deviceCurrentBytes, long[] deviceMaxBytes);
/**
* Get JVM memory - current bytes used
*/
long getJvmCurrentBytes();
/**
* Get JVM memory - max available bytes
*/
long getJvmMaxBytes();
/**
* Get off-heap memory - current bytes used
*/
long getOffHeapCurrentBytes();
/**
* Get off-heap memory - max available bytes
*/
long getOffHeapMaxBytes();
/**
* Get device (GPU, etc) current bytes - may be null if no compute devices are present in the system
*/
long[] getDeviceCurrentBytes();
/**
* Get device (GPU, etc) maximum bytes - may be null if no compute devices are present in the system
*/
long[] getDeviceMaxBytes();
/**
* Report the performance stats (since the last report)
*
* @param totalRuntimeMs Overall runtime since initialization
* @param totalExamples Total examples processed since initialization
* @param totalMinibatches Total number of minibatches (iterations) since initialization
* @param examplesPerSecond Examples per second since last report
* @param minibatchesPerSecond Minibatches per second since last report
*/
void reportPerformance(long totalRuntimeMs, long totalExamples, long totalMinibatches, double examplesPerSecond,
double minibatchesPerSecond);
/**
* Get the total runtime since listener/model initialization
*/
long getTotalRuntimeMs();
/**
* Get total number of examples that have been processed since initialization
*/
long getTotalExamples();
/**
* Get the total number of minibatches that have been processed since initialization
*/
long getTotalMinibatches();
/**
* Get examples per second since the last report
*/
double getExamplesPerSecond();
/**
* Get the number of minibatches per second, since the last report
*/
double getMinibatchesPerSecond();
/**
* Report Garbage collection stats
*
* @param gcName Garbage collector name
* @param deltaGCCount Change in the total number of garbage collections, since last report
* @param deltaGCTime Change in the amount of time (milliseconds) for garbage collection, since last report
*/
void reportGarbageCollection(String gcName, int deltaGCCount, int deltaGCTime);
/**
* Get the garbage collection stats: Pair contains GC name and the delta count/time values
*/
List<Pair<String, int[]>> getGarbageCollectionStats();
//--- Histograms ---
/**
* Report histograms for all parameters, for a given {@link StatsType}
*
* @param statsType StatsType: Parameters, Updates, Activations
* @param histogram Histogram values for all parameters
*/
void reportHistograms(StatsType statsType, Map<String, Histogram> histogram);
/**
* Get the histograms for all parameters, for a given StatsType (Parameters/Updates/Activations)
*
* @param statsType Stats type (Params/updatse/activations) to get histograms for
* @return Histograms by parameter name, or null if not available
*/
Map<String, Histogram> getHistograms(StatsType statsType);
//--- Summary Stats: Mean, Variance, Mean Magnitudes ---
/**
* Report the mean values for each parameter, the given StatsType (Parameters/Updates/Activations)
*
* @param statsType Stats type to report
* @param mean Map of mean values, by parameter
*/
void reportMean(StatsType statsType, Map<String, Double> mean);
/**
* Get the mean values for each parameter for the given StatsType (Parameters/Updates/Activations)
*
* @param statsType Stats type to get mean values for
* @return Map of mean values by parameter
*/
Map<String, Double> getMean(StatsType statsType);
/**
* Report the standard deviation values for each parameter for the given StatsType (Parameters/Updates/Activations)
*
* @param statsType Stats type to report std. dev values for
* @param stdev Map of std dev values by parameter
*/
void reportStdev(StatsType statsType, Map<String, Double> stdev);
/**
* Get the standard deviation values for each parameter for the given StatsType (Parameters/Updates/Activations)
*
* @param statsType Stats type to get std dev values for
* @return Map of stdev values by parameter
*/
Map<String, Double> getStdev(StatsType statsType);
/**
* Report the mean magnitude values for each parameter for the given StatsType (Parameters/Updates/Activations)
*
* @param statsType Stats type to report mean magnitude values for
* @param meanMagnitudes Map of mean magnitude values by parameter
*/
void reportMeanMagnitudes(StatsType statsType, Map<String, Double> meanMagnitudes);
/**
* Report any metadata for the DataSet
*
* @param dataSetMetaData MetaData for the DataSet
* @param metaDataClass Class of the metadata. Can be later retieved using {@link #getDataSetMetaDataClassName()}
*/
void reportDataSetMetaData(List<Serializable> dataSetMetaData, Class<?> metaDataClass);
/**
* Report any metadata for the DataSet
*
* @param dataSetMetaData MetaData for the DataSet
* @param metaDataClass Class of the metadata. Can be later retieved using {@link #getDataSetMetaDataClassName()}
*/
void reportDataSetMetaData(List<Serializable> dataSetMetaData, String metaDataClass);
/**
* Get the mean magnitude values for each parameter for the given StatsType (Parameters/Updates/Activations)
*
* @param statsType Stats type to get mean magnitude values for
* @return Map of mean magnitude values by parameter
*/
Map<String, Double> getMeanMagnitudes(StatsType statsType);
/**
* Get the DataSet metadata, if any (null otherwise).
* Note: due to serialization issues, this may in principle throw an unchecked exception related
* to class availability, serialization etc.
*
* @return List of DataSet metadata, if any.
*/
List<Serializable> getDataSetMetaData();
/**
* Get the class
*
* @return
*/
String getDataSetMetaDataClassName();
/**
* Return whether the score is present (has been reported)
*/
boolean hasScore();
/**
* Return whether the learning rates are present (have been reported)
*/
boolean hasLearningRates();
/**
* Return whether memory use has been reported
*/
boolean hasMemoryUse();
/**
* Return whether performance stats (total time, total examples etc) have been reported
*/
boolean hasPerformance();
/**
* Return whether garbage collection information has been reported
*/
boolean hasGarbageCollection();
/**
* Return whether histograms have been reported, for the given stats type (Parameters, Updates, Activations)
*
* @param statsType Stats type
*/
boolean hasHistograms(StatsType statsType);
/**
* Return whether the summary stats (mean, standard deviation, mean magnitudes) have been reported for the
* given stats type (Parameters, Updates, Activations)
*
* @param statsType stats type (Parameters, Updates, Activations)
* @param summaryType Summary statistic type (mean, stdev, mean magnitude)
*/
boolean hasSummaryStats(StatsType statsType, SummaryType summaryType);
/**
* Return whether any DataSet metadata is present or not
*
* @return True if DataSet metadata is present
*/
boolean hasDataSetMetaData();
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/api/StatsType.java
+29
View File
@@ -0,0 +1,29 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.api;
import org.deeplearning4j.ui.model.stats.StatsListener;
public enum StatsType {
Parameters, Gradients, Updates, Activations
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/api/StatsUpdateConfiguration.java
+102
View File
@@ -0,0 +1,102 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.api;
import java.io.Serializable;
public interface StatsUpdateConfiguration extends Serializable {
/**
* Get the reporting frequency, in terms of listener calls
*/
int reportingFrequency();
//TODO
//boolean useNTPTimeSource();
//--- Performance and System Stats ---
/**
* Should performance stats be collected/reported?
* Total time, total examples, total batches, Minibatches/second, examples/second
*/
boolean collectPerformanceStats();
/**
* Should JVM, off-heap and memory stats be collected/reported?
*/
boolean collectMemoryStats();
/**
* Should garbage collection stats be collected and reported?
*/
boolean collectGarbageCollectionStats();
//TODO
// boolean collectDataSetMetaData();
//--- General ---
/**
* Should per-parameter type learning rates be collected and reported?
*/
boolean collectLearningRates();
//--- Histograms ---
/**
* Should histograms (per parameter type, or per layer for activations) of the given type be collected?
*
* @param type Stats type: Parameters, Updates, Activations
*/
boolean collectHistograms(StatsType type);
/**
* Get the number of histogram bins to use for the given type (for use with {@link #collectHistograms(StatsType)}
*
* @param type Stats type: Parameters, Updates, Activatinos
*/
int numHistogramBins(StatsType type);
//--- Summary Stats: Mean, Variance, Mean Magnitudes ---
/**
* Should the mean values (per parameter type, or per layer for activations) be collected?
*
* @param type Stats type: Parameters, Updates, Activations
*/
boolean collectMean(StatsType type);
/**
* Should the standard devication values (per parameter type, or per layer for activations) be collected?
*
* @param type Stats type: Parameters, Updates, Activations
*/
boolean collectStdev(StatsType type);
/**
* Should the mean magnitude values (per parameter type, or per layer for activations) be collected?
*
* @param type Stats type: Parameters, Updates, Activations
*/
boolean collectMeanMagnitudes(StatsType type);
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/api/SummaryType.java
+27
View File
@@ -0,0 +1,27 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.api;
import org.deeplearning4j.ui.model.stats.StatsListener;
public enum SummaryType {
Mean, Stdev, MeanMagnitudes
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/impl/DefaultStatsInitializationConfiguration.java
+47
View File
@@ -0,0 +1,47 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.impl;
import lombok.AllArgsConstructor;
import org.deeplearning4j.ui.model.stats.api.StatsInitializationConfiguration;
@AllArgsConstructor
public class DefaultStatsInitializationConfiguration implements StatsInitializationConfiguration {
private final boolean collectSoftwareInfo;
private final boolean collectHardwareInfo;
private final boolean collectModelInfo;
@Override
public boolean collectSoftwareInfo() {
return collectSoftwareInfo;
}
@Override
public boolean collectHardwareInfo() {
return collectHardwareInfo;
}
@Override
public boolean collectModelInfo() {
return collectModelInfo;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/impl/DefaultStatsUpdateConfiguration.java
+308
View File
@@ -0,0 +1,308 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.impl;
import lombok.AllArgsConstructor;
import org.deeplearning4j.ui.model.stats.api.StatsType;
import org.deeplearning4j.ui.model.stats.api.StatsUpdateConfiguration;
@AllArgsConstructor
public class DefaultStatsUpdateConfiguration implements StatsUpdateConfiguration {
public static final int DEFAULT_REPORTING_FREQUENCY = 10;
private int reportingFrequency = DEFAULT_REPORTING_FREQUENCY;
private boolean collectPerformanceStats = true;
private boolean collectMemoryStats = true;
private boolean collectGarbageCollectionStats = true;
private boolean collectLearningRates = true;
private boolean collectHistogramsParameters = true;
private boolean collectHistogramsGradients = true;
private boolean collectHistogramsUpdates = true;
private boolean collectHistogramsActivations = true;
private int numHistogramBins = 20;
private boolean collectMeanParameters = true;
private boolean collectMeanGradients = true;
private boolean collectMeanUpdates = true;
private boolean collectMeanActivations = true;
private boolean collectStdevParameters = true;
private boolean collectStdevGradients = true;
private boolean collectStdevUpdates = true;
private boolean collectStdevActivations = true;
private boolean collectMeanMagnitudesParameters = true;
private boolean collectMeanMagnitudesGradients = true;
private boolean collectMeanMagnitudesUpdates = true;
private boolean collectMeanMagnitudesActivations = true;
private DefaultStatsUpdateConfiguration(Builder b) {
this.reportingFrequency = b.reportingFrequency;
this.collectPerformanceStats = b.collectPerformanceStats;
this.collectMemoryStats = b.collectMemoryStats;
this.collectGarbageCollectionStats = b.collectGarbageCollectionStats;
this.collectLearningRates = b.collectLearningRates;
this.collectHistogramsParameters = b.collectHistogramsParameters;
this.collectHistogramsGradients = b.collectHistogramsGradients;
this.collectHistogramsUpdates = b.collectHistogramsUpdates;
this.collectHistogramsActivations = b.collectHistogramsActivations;
this.numHistogramBins = b.numHistogramBins;
this.collectMeanParameters = b.collectMeanParameters;
this.collectMeanGradients = b.collectMeanGradients;
this.collectMeanUpdates = b.collectMeanUpdates;
this.collectMeanActivations = b.collectMeanActivations;
this.collectStdevParameters = b.collectStdevParameters;
this.collectStdevGradients = b.collectStdevGradients;
this.collectStdevUpdates = b.collectStdevUpdates;
this.collectStdevActivations = b.collectStdevActivations;
this.collectMeanMagnitudesParameters = b.collectMeanMagnitudesParameters;
this.collectMeanMagnitudesGradients = b.collectMeanMagnitudesGradients;
this.collectMeanMagnitudesUpdates = b.collectMeanMagnitudesUpdates;
this.collectMeanMagnitudesActivations = b.collectMeanMagnitudesActivations;
}
@Override
public int reportingFrequency() {
return reportingFrequency;
}
@Override
public boolean collectPerformanceStats() {
return collectPerformanceStats;
}
@Override
public boolean collectMemoryStats() {
return collectMemoryStats;
}
@Override
public boolean collectGarbageCollectionStats() {
return collectGarbageCollectionStats;
}
@Override
public boolean collectLearningRates() {
return collectLearningRates;
}
@Override
public boolean collectHistograms(StatsType type) {
switch (type) {
case Parameters:
return collectHistogramsParameters;
case Gradients:
return collectStdevGradients;
case Updates:
return collectHistogramsUpdates;
case Activations:
return collectHistogramsActivations;
}
return false;
}
@Override
public int numHistogramBins(StatsType type) {
return numHistogramBins;
}
@Override
public boolean collectMean(StatsType type) {
switch (type) {
case Parameters:
return collectMeanParameters;
case Gradients:
return collectMeanGradients;
case Updates:
return collectMeanUpdates;
case Activations:
return collectMeanActivations;
}
return false;
}
@Override
public boolean collectStdev(StatsType type) {
switch (type) {
case Parameters:
return collectStdevParameters;
case Gradients:
return collectStdevGradients;
case Updates:
return collectStdevUpdates;
case Activations:
return collectStdevActivations;
}
return false;
}
@Override
public boolean collectMeanMagnitudes(StatsType type) {
switch (type) {
case Parameters:
return collectMeanMagnitudesParameters;
case Gradients:
return collectMeanMagnitudesGradients;
case Updates:
return collectMeanMagnitudesUpdates;
case Activations:
return collectMeanMagnitudesActivations;
}
return false;
}
public static class Builder {
private int reportingFrequency = DEFAULT_REPORTING_FREQUENCY;
private boolean collectPerformanceStats = true;
private boolean collectMemoryStats = true;
private boolean collectGarbageCollectionStats = true;
private boolean collectLearningRates = true;
private boolean collectHistogramsParameters = true;
private boolean collectHistogramsGradients = true;
private boolean collectHistogramsUpdates = true;
private boolean collectHistogramsActivations = true;
private int numHistogramBins = 20;
private boolean collectMeanParameters = true;
private boolean collectMeanGradients = true;
private boolean collectMeanUpdates = true;
private boolean collectMeanActivations = true;
private boolean collectStdevParameters = true;
private boolean collectStdevGradients = true;
private boolean collectStdevUpdates = true;
private boolean collectStdevActivations = true;
private boolean collectMeanMagnitudesParameters = true;
private boolean collectMeanMagnitudesGradients = true;
private boolean collectMeanMagnitudesUpdates = true;
private boolean collectMeanMagnitudesActivations = true;
public Builder reportingFrequency(int reportingFrequency) {
this.reportingFrequency = reportingFrequency;
return this;
}
public Builder collectPerformanceStats(boolean collectPerformanceStats) {
this.collectPerformanceStats = collectPerformanceStats;
return this;
}
public Builder collectMemoryStats(boolean collectMemoryStats) {
this.collectMemoryStats = collectMemoryStats;
return this;
}
public Builder collectGarbageCollectionStats(boolean collectGarbageCollectionStats) {
this.collectGarbageCollectionStats = collectGarbageCollectionStats;
return this;
}
public Builder collectLearningRates(boolean collectLearningRates) {
this.collectLearningRates = collectLearningRates;
return this;
}
public Builder collectHistogramsParameters(boolean collectHistogramsParameters) {
this.collectHistogramsParameters = collectHistogramsParameters;
return this;
}
public Builder collectHistogramsGradients(boolean collectHistogramsGradients) {
this.collectHistogramsGradients = collectHistogramsGradients;
return this;
}
public Builder collectHistogramsUpdates(boolean collectHistogramsUpdates) {
this.collectHistogramsUpdates = collectHistogramsUpdates;
return this;
}
public Builder collectHistogramsActivations(boolean isCollectHistogramsActivations) {
this.collectHistogramsActivations = isCollectHistogramsActivations;
return this;
}
public Builder numHistogramBins(int numHistogramBins) {
this.numHistogramBins = numHistogramBins;
return this;
}
public Builder collectMeanParameters(boolean collectMeanParameters) {
this.collectMeanParameters = collectMeanParameters;
return this;
}
public Builder collectMeanGradients(boolean collectMeanGradients) {
this.collectMeanGradients = collectMeanGradients;
return this;
}
public Builder collectMeanUpdates(boolean collectMeanUpdates) {
this.collectMeanUpdates = collectMeanUpdates;
return this;
}
public Builder collectMeanActivations(boolean collectMeanActivations) {
this.collectMeanActivations = collectMeanActivations;
return this;
}
public Builder collectStdevParameters(boolean collectStdevParameters) {
this.collectStdevParameters = collectStdevParameters;
return this;
}
public Builder collectStdevGradients(boolean collectStdevGradients) {
this.collectStdevGradients = collectStdevGradients;
return this;
}
public Builder collectStdevUpdates(boolean collectStdevUpdates) {
this.collectStdevUpdates = collectStdevUpdates;
return this;
}
public Builder collectStdevActivations(boolean collectStdevActivations) {
this.collectStdevActivations = collectStdevActivations;
return this;
}
public Builder collectMeanMagnitudesParameters(boolean collectMeanMagnitudesParameters) {
this.collectMeanMagnitudesParameters = collectMeanMagnitudesParameters;
return this;
}
public Builder collectMeanMagnitudesGradients(boolean collectMeanMagnitudesGradients) {
this.collectMeanMagnitudesGradients = collectMeanMagnitudesGradients;
return this;
}
public Builder collectMeanMagnitudesUpdates(boolean collectMeanMagnitudesUpdates) {
this.collectMeanMagnitudesUpdates = collectMeanMagnitudesUpdates;
return this;
}
public Builder collectMeanMagnitudesActivations(boolean collectMeanMagnitudesActivations) {
this.collectMeanMagnitudesActivations = collectMeanMagnitudesActivations;
return this;
}
public DefaultStatsUpdateConfiguration build() {
return new DefaultStatsUpdateConfiguration(this);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/impl/SbeStatsInitializationReport.java
+468
View File
@@ -0,0 +1,468 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.impl;
import lombok.Data;
import org.agrona.DirectBuffer;
import org.agrona.MutableDirectBuffer;
import org.agrona.concurrent.UnsafeBuffer;
import org.apache.commons.io.IOUtils;
import org.deeplearning4j.ui.model.stats.api.StatsInitializationReport;
import org.deeplearning4j.ui.model.stats.sbe.*;
import org.deeplearning4j.ui.model.storage.AgronaPersistable;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.util.HashMap;
import java.util.Map;
@Data
public class SbeStatsInitializationReport implements StatsInitializationReport, AgronaPersistable {
private String sessionID;
private String typeID;
private String workerID;
private long timeStamp;
private boolean hasSoftwareInfo;
private boolean hasHardwareInfo;
private boolean hasModelInfo;
private String swArch;
private String swOsName;
private String swJvmName;
private String swJvmVersion;
private String swJvmSpecVersion;
private String swNd4jBackendClass;
private String swNd4jDataTypeName;
private String swHostName;
private String swJvmUID;
private Map<String, String> swEnvironmentInfo;
private int hwJvmAvailableProcessors;
private int hwNumDevices;
private long hwJvmMaxMemory;
private long hwOffHeapMaxMemory;
private long[] hwDeviceTotalMemory;
private String[] hwDeviceDescription;
private String hwHardwareUID;
private String modelClassName;
private String modelConfigJson;
private String[] modelParamNames;
private int modelNumLayers;
private long modelNumParams;
@Override
public void reportIDs(String sessionID, String typeID, String workerID, long timeStamp) {
this.sessionID = sessionID;
this.typeID = typeID;
this.workerID = workerID;
this.timeStamp = timeStamp;
}
@Override
public void reportSoftwareInfo(String arch, String osName, String jvmName, String jvmVersion, String jvmSpecVersion,
String nd4jBackendClass, String nd4jDataTypeName, String hostname, String jvmUid,
Map<String, String> swEnvironmentInfo) {
this.swArch = arch;
this.swOsName = osName;
this.swJvmName = jvmName;
this.swJvmVersion = jvmVersion;
this.swJvmSpecVersion = jvmSpecVersion;
this.swNd4jBackendClass = nd4jBackendClass;
this.swNd4jDataTypeName = nd4jDataTypeName;
this.swHostName = hostname;
this.swJvmUID = jvmUid;
this.swEnvironmentInfo = swEnvironmentInfo;
hasSoftwareInfo = true;
}
@Override
public void reportHardwareInfo(int jvmAvailableProcessors, int numDevices, long jvmMaxMemory, long offHeapMaxMemory,
long[] deviceTotalMemory, String[] deviceDescription, String hardwareUID) {
this.hwJvmAvailableProcessors = jvmAvailableProcessors;
this.hwNumDevices = numDevices;
this.hwJvmMaxMemory = jvmMaxMemory;
this.hwOffHeapMaxMemory = offHeapMaxMemory;
this.hwDeviceTotalMemory = deviceTotalMemory;
this.hwDeviceDescription = deviceDescription;
this.hwHardwareUID = hardwareUID;
hasHardwareInfo = true;
}
@Override
public void reportModelInfo(String modelClassName, String modelConfigJson, String[] modelParamNames, int numLayers,
long numParams) {
this.modelClassName = modelClassName;
this.modelConfigJson = modelConfigJson;
this.modelParamNames = modelParamNames;
this.modelNumLayers = numLayers;
this.modelNumParams = numParams;
hasModelInfo = true;
}
@Override
public boolean hasSoftwareInfo() {
return hasSoftwareInfo;
}
@Override
public boolean hasHardwareInfo() {
return hasHardwareInfo;
}
@Override
public boolean hasModelInfo() {
return hasModelInfo;
}
private void clearHwFields() {
hwDeviceTotalMemory = null;
hwDeviceDescription = null;
hwHardwareUID = null;
}
private void clearSwFields() {
swArch = null;
swOsName = null;
swJvmName = null;
swJvmVersion = null;
swJvmSpecVersion = null;
swNd4jBackendClass = null;
swNd4jDataTypeName = null;
swHostName = null;
swJvmUID = null;
}
private void clearModelFields() {
modelClassName = null;
modelConfigJson = null;
modelParamNames = null;
}
@Override
public String getSessionID() {
return sessionID;
}
@Override
public String getTypeID() {
return typeID;
}
@Override
public String getWorkerID() {
return workerID;
}
@Override
public long getTimeStamp() {
return timeStamp;
}
@Override
public int encodingLengthBytes() {
//TODO reuse the byte[]s here, to avoid converting them twice...
//First: need to determine how large a buffer to use.
//Buffer is composed of:
//(a) Header: 8 bytes (4x uint16 = 8 bytes)
//(b) Fixed length entries length (sie.BlockLength())
//(c) Group 1: Hardware devices (GPUs) max memory: 4 bytes header + nEntries * 8 (int64) + nEntries * variable length Strings (header + content) = 4 + 8*n + content
//(d) Group 2: Software device info: 4 bytes header + 2x variable length Strings for each
//(d) Group 3: Parameter names: 4 bytes header + nEntries * variable length strings (header + content) = 4 + content
//(e) Variable length fields: 15 String length fields. Size: 4 bytes header, plus content. 60 bytes header
//Fixed length + repeating groups + variable length...
StaticInfoEncoder sie = new StaticInfoEncoder();
int bufferSize = 8 + sie.sbeBlockLength() + 4 + 4 + 60; //header + fixed values + group headers + variable length headers
//For variable length field lengths: easist way is simply to convert to UTF-8
//Of course, it is possible to calculate it first - but we might as well convert (1 pass), rather than count then convert (2 passes)
byte[] bSessionId = SbeUtil.toBytes(true, sessionID);
byte[] bTypeId = SbeUtil.toBytes(true, typeID);
byte[] bWorkerId = SbeUtil.toBytes(true, workerID);
byte[] bswArch = SbeUtil.toBytes(hasSoftwareInfo, swArch);
byte[] bswOsName = SbeUtil.toBytes(hasSoftwareInfo, swOsName);
byte[] bswJvmName = SbeUtil.toBytes(hasSoftwareInfo, swJvmName);
byte[] bswJvmVersion = SbeUtil.toBytes(hasSoftwareInfo, swJvmVersion);
byte[] bswJvmSpecVersion = SbeUtil.toBytes(hasSoftwareInfo, swJvmSpecVersion);
byte[] bswNd4jBackendClass = SbeUtil.toBytes(hasSoftwareInfo, swNd4jBackendClass);
byte[] bswNd4jDataTypeName = SbeUtil.toBytes(hasSoftwareInfo, swNd4jDataTypeName);
byte[] bswHostname = SbeUtil.toBytes(hasSoftwareInfo, swHostName);
byte[] bswJvmUID = SbeUtil.toBytes(hasSoftwareInfo, swJvmUID);
byte[] bHwHardwareUID = SbeUtil.toBytes(hasHardwareInfo, hwHardwareUID);
byte[] bmodelConfigClass = SbeUtil.toBytes(hasModelInfo, modelClassName);
byte[] bmodelConfigJson = SbeUtil.toBytes(hasModelInfo, modelConfigJson);
byte[][] bhwDeviceDescription = SbeUtil.toBytes(hasHardwareInfo, hwDeviceDescription);
byte[][][] bswEnvInfo = SbeUtil.toBytes(swEnvironmentInfo);
byte[][] bModelParamNames = SbeUtil.toBytes(hasModelInfo, modelParamNames);
bufferSize += bSessionId.length + bTypeId.length + bWorkerId.length;
bufferSize += 4; //swEnvironmentInfo group header (always present)
if (hasSoftwareInfo) {
bufferSize += SbeUtil.length(bswArch);
bufferSize += SbeUtil.length(bswOsName);
bufferSize += SbeUtil.length(bswJvmName);
bufferSize += SbeUtil.length(bswJvmVersion);
bufferSize += SbeUtil.length(bswJvmSpecVersion);
bufferSize += SbeUtil.length(bswNd4jBackendClass);
bufferSize += SbeUtil.length(bswNd4jDataTypeName);
bufferSize += SbeUtil.length(bswHostname);
bufferSize += SbeUtil.length(bswJvmUID);
//For each entry: 2 variable-length headers (2x4 bytes each) + content
int envCount = (bswEnvInfo != null ? bswEnvInfo.length : 0);
bufferSize += envCount * 8;
bufferSize += SbeUtil.length(bswEnvInfo);
}
int nHWDeviceStats = hwNumDevices;
if (!hasHardwareInfo)
nHWDeviceStats = 0;
if (hasHardwareInfo) {
//Device info group:
bufferSize += hwNumDevices * 8; //fixed content in group: int64 -> 8 bytes. Encode an entry, even if hwDeviceTotalMemory is null
bufferSize += hwNumDevices * 4; //uint32: 4 bytes per entry for var length header...; as above
bufferSize += SbeUtil.length(bhwDeviceDescription);
bufferSize += SbeUtil.length(bHwHardwareUID);
}
if (hasModelInfo) {
bufferSize += SbeUtil.length(bmodelConfigClass);
bufferSize += SbeUtil.length(bmodelConfigJson);
bufferSize += SbeUtil.length(bModelParamNames);
bufferSize += (bModelParamNames == null ? 0 : bModelParamNames.length * 4); //uint32: 4 bytes per entry for var length header...
}
return bufferSize;
}
@Override
public byte[] encode() {
byte[] bytes = new byte[encodingLengthBytes()];
MutableDirectBuffer buffer = new UnsafeBuffer(bytes);
encode(buffer);
return bytes;
}
@Override
public void encode(ByteBuffer buffer) {
encode(new UnsafeBuffer(buffer));
}
@Override
public void encode(MutableDirectBuffer buffer) {
MessageHeaderEncoder enc = new MessageHeaderEncoder();
StaticInfoEncoder sie = new StaticInfoEncoder();
byte[] bSessionId = SbeUtil.toBytes(true, sessionID);
byte[] bTypeId = SbeUtil.toBytes(true, typeID);
byte[] bWorkerId = SbeUtil.toBytes(true, workerID);
byte[] bswArch = SbeUtil.toBytes(hasSoftwareInfo, swArch);
byte[] bswOsName = SbeUtil.toBytes(hasSoftwareInfo, swOsName);
byte[] bswJvmName = SbeUtil.toBytes(hasSoftwareInfo, swJvmName);
byte[] bswJvmVersion = SbeUtil.toBytes(hasSoftwareInfo, swJvmVersion);
byte[] bswJvmSpecVersion = SbeUtil.toBytes(hasSoftwareInfo, swJvmSpecVersion);
byte[] bswNd4jBackendClass = SbeUtil.toBytes(hasSoftwareInfo, swNd4jBackendClass);
byte[] bswNd4jDataTypeName = SbeUtil.toBytes(hasSoftwareInfo, swNd4jDataTypeName);
byte[] bswHostname = SbeUtil.toBytes(hasSoftwareInfo, swHostName);
byte[] bswJvmUID = SbeUtil.toBytes(hasSoftwareInfo, swJvmUID);
byte[] bHwHardwareUID = SbeUtil.toBytes(hasHardwareInfo, hwHardwareUID);
byte[] bmodelConfigClass = SbeUtil.toBytes(hasModelInfo, modelClassName);
byte[] bmodelConfigJson = SbeUtil.toBytes(hasModelInfo, modelConfigJson);
byte[][] bhwDeviceDescription = SbeUtil.toBytes(hasHardwareInfo, hwDeviceDescription);
byte[][][] bswEnvInfo = SbeUtil.toBytes(swEnvironmentInfo);
byte[][] bModelParamNames = SbeUtil.toBytes(hasModelInfo, modelParamNames);
enc.wrap(buffer, 0).blockLength(sie.sbeBlockLength()).templateId(sie.sbeTemplateId())
.schemaId(sie.sbeSchemaId()).version(sie.sbeSchemaVersion());
int offset = enc.encodedLength(); //Expect 8 bytes...
//Fixed length fields: always encoded, whether present or not.
sie.wrap(buffer, offset).time(timeStamp).fieldsPresent().softwareInfo(hasSoftwareInfo)
.hardwareInfo(hasHardwareInfo).modelInfo(hasModelInfo);
sie.hwJvmProcessors(hwJvmAvailableProcessors).hwNumDevices((short) hwNumDevices).hwJvmMaxMemory(hwJvmMaxMemory)
.hwOffheapMaxMemory(hwOffHeapMaxMemory).modelNumLayers(modelNumLayers)
.modelNumParams(modelNumParams);
//Device info group...
StaticInfoEncoder.HwDeviceInfoGroupEncoder hwdEnc = sie.hwDeviceInfoGroupCount(hwNumDevices);
int nHWDeviceStats = (hasHardwareInfo ? hwNumDevices : 0);
for (int i = 0; i < nHWDeviceStats; i++) {
long maxMem = hwDeviceTotalMemory == null || hwDeviceTotalMemory.length <= i ? 0 : hwDeviceTotalMemory[i];
byte[] descr = bhwDeviceDescription == null || bhwDeviceDescription.length <= i ? SbeUtil.EMPTY_BYTES
: bhwDeviceDescription[i];
if (descr == null)
descr = SbeUtil.EMPTY_BYTES;
hwdEnc.next().deviceMemoryMax(maxMem).putDeviceDescription(descr, 0, descr.length);
}
//Environment info group
int numEnvValues = (hasSoftwareInfo && swEnvironmentInfo != null ? swEnvironmentInfo.size() : 0);
StaticInfoEncoder.SwEnvironmentInfoEncoder swEnv = sie.swEnvironmentInfoCount(numEnvValues);
if (numEnvValues > 0) {
byte[][][] mapAsBytes = SbeUtil.toBytes(swEnvironmentInfo);
for (byte[][] entryBytes : mapAsBytes) {
swEnv.next().putEnvKey(entryBytes[0], 0, entryBytes[0].length).putEnvValue(entryBytes[1], 0,
entryBytes[1].length);
}
}
int nParamNames = modelParamNames == null ? 0 : modelParamNames.length;
StaticInfoEncoder.ModelParamNamesEncoder mpnEnc = sie.modelParamNamesCount(nParamNames);
for (int i = 0; i < nParamNames; i++) {
mpnEnc.next().putModelParamNames(bModelParamNames[i], 0, bModelParamNames[i].length);
}
//In the case of !hasSoftwareInfo: these will all be empty byte arrays... still need to encode them (for 0 length) however
sie.putSessionID(bSessionId, 0, bSessionId.length).putTypeID(bTypeId, 0, bTypeId.length)
.putWorkerID(bWorkerId, 0, bWorkerId.length).putSwArch(bswArch, 0, bswArch.length)
.putSwOsName(bswOsName, 0, bswOsName.length).putSwJvmName(bswJvmName, 0, bswJvmName.length)
.putSwJvmVersion(bswJvmVersion, 0, bswJvmVersion.length)
.putSwJvmSpecVersion(bswJvmSpecVersion, 0, bswJvmSpecVersion.length)
.putSwNd4jBackendClass(bswNd4jBackendClass, 0, bswNd4jBackendClass.length)
.putSwNd4jDataTypeName(bswNd4jDataTypeName, 0, bswNd4jDataTypeName.length)
.putSwHostName(bswHostname, 0, bswHostname.length).putSwJvmUID(bswJvmUID, 0, bswJvmUID.length)
.putHwHardwareUID(bHwHardwareUID, 0, bHwHardwareUID.length);
//Similar: !hasModelInfo -> empty byte[]
sie.putModelConfigClassName(bmodelConfigClass, 0, bmodelConfigClass.length).putModelConfigJson(bmodelConfigJson,
0, bmodelConfigJson.length);
}
@Override
public void encode(OutputStream outputStream) throws IOException {
//TODO there may be more efficient way of doing this
outputStream.write(encode());
}
@Override
public void decode(byte[] decode) {
MutableDirectBuffer buffer = new UnsafeBuffer(decode);
decode(buffer);
}
@Override
public void decode(ByteBuffer buffer) {
decode(new UnsafeBuffer(buffer));
}
@Override
public void decode(DirectBuffer buffer) {
//TODO we could do this much more efficiently, with buffer re-use, etc.
MessageHeaderDecoder dec = new MessageHeaderDecoder();
StaticInfoDecoder sid = new StaticInfoDecoder();
dec.wrap(buffer, 0);
final int blockLength = dec.blockLength();
final int version = dec.version();
final int headerLength = dec.encodedLength();
//TODO: in general, we should check the header, version, schema etc. But we don't have any other versions yet.
sid.wrap(buffer, headerLength, blockLength, version);
timeStamp = sid.time();
InitFieldsPresentDecoder fields = sid.fieldsPresent();
hasSoftwareInfo = fields.softwareInfo();
hasHardwareInfo = fields.hardwareInfo();
hasModelInfo = fields.modelInfo();
//These fields: always present, even if !hasHardwareInfo
hwJvmAvailableProcessors = sid.hwJvmProcessors();
hwNumDevices = sid.hwNumDevices();
hwJvmMaxMemory = sid.hwJvmMaxMemory();
hwOffHeapMaxMemory = sid.hwOffheapMaxMemory();
modelNumLayers = sid.modelNumLayers();
modelNumParams = sid.modelNumParams();
//Hardware device info group
StaticInfoDecoder.HwDeviceInfoGroupDecoder hwDeviceInfoGroupDecoder = sid.hwDeviceInfoGroup();
int count = hwDeviceInfoGroupDecoder.count();
if (count > 0) {
hwDeviceTotalMemory = new long[count];
hwDeviceDescription = new String[count];
}
int i = 0;
for (StaticInfoDecoder.HwDeviceInfoGroupDecoder hw : hwDeviceInfoGroupDecoder) {
hwDeviceTotalMemory[i] = hw.deviceMemoryMax();
hwDeviceDescription[i++] = hw.deviceDescription();
}
//Environment info group
i = 0;
StaticInfoDecoder.SwEnvironmentInfoDecoder swEnvDecoder = sid.swEnvironmentInfo();
if (swEnvDecoder.count() > 0) {
swEnvironmentInfo = new HashMap<>();
}
for (StaticInfoDecoder.SwEnvironmentInfoDecoder env : swEnvDecoder) {
String key = env.envKey();
String value = env.envValue();
swEnvironmentInfo.put(key, value);
}
i = 0;
StaticInfoDecoder.ModelParamNamesDecoder mpdec = sid.modelParamNames();
int mpnCount = mpdec.count();
modelParamNames = new String[mpnCount];
for (StaticInfoDecoder.ModelParamNamesDecoder mp : mpdec) {
modelParamNames[i++] = mp.modelParamNames();
}
//Variable length data. Even if it is missing: still needs to be read, to advance buffer
//Again, the exact order of these calls matters here
sessionID = sid.sessionID();
typeID = sid.typeID();
workerID = sid.workerID();
swArch = sid.swArch();
swOsName = sid.swOsName();
swJvmName = sid.swJvmName();
swJvmVersion = sid.swJvmVersion();
swJvmSpecVersion = sid.swJvmSpecVersion();
swNd4jBackendClass = sid.swNd4jBackendClass();
swNd4jDataTypeName = sid.swNd4jDataTypeName();
swHostName = sid.swHostName();
swJvmUID = sid.swJvmUID();
if (!hasSoftwareInfo)
clearSwFields();
hwHardwareUID = sid.hwHardwareUID();
if (!hasHardwareInfo)
clearHwFields();
modelClassName = sid.modelConfigClassName();
modelConfigJson = sid.modelConfigJson();
if (!hasModelInfo)
clearModelFields();
}
@Override
public void decode(InputStream inputStream) throws IOException {
byte[] bytes = IOUtils.toByteArray(inputStream);
decode(bytes);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/impl/SbeStatsReport.java
+1181
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/impl/SbeUtil.java
+131
View File
@@ -0,0 +1,131 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.impl;
import java.io.*;
import java.nio.charset.Charset;
import java.util.Map;
public class SbeUtil {
public static final Charset UTF8 = Charset.forName("UTF-8");
public static final byte[] EMPTY_BYTES = new byte[0]; //Also equivalent to "".getBytes(UTF8);
private SbeUtil() {}
public static int length(byte[] bytes) {
if (bytes == null)
return 0;
return bytes.length;
}
public static int length(byte[][] bytes) {
if (bytes == null)
return 0;
int count = 0;
for (int i = 0; i < bytes.length; i++) {
if (bytes[i] != null)
count += bytes[i].length;
}
return count;
}
public static int length(byte[][][] bytes) {
if (bytes == null)
return 0;
int count = 0;
for (byte[][] arr : bytes) {
count += length(arr);
}
return count;
}
public static int length(String str) {
if (str == null)
return 0;
return str.length();
}
public static int length(String[] arr) {
if (arr == null || arr.length == 0)
return 0;
int sum = 0;
for (String s : arr)
sum += length(s);
return sum;
}
public static byte[] toBytes(boolean present, String str) {
if (!present || str == null)
return EMPTY_BYTES;
return str.getBytes(UTF8);
}
public static byte[][] toBytes(boolean present, String[] str) {
if (str == null)
return null;
byte[][] b = new byte[str.length][0];
for (int i = 0; i < str.length; i++) {
if (str[i] == null)
continue;
b[i] = toBytes(present, str[i]);
}
return b;
}
public static byte[][][] toBytes(Map<String, String> map) {
if (map == null)
return null;
byte[][][] b = new byte[map.size()][2][0];
int i = 0;
for (Map.Entry<String, String> entry : map.entrySet()) {
b[i][0] = toBytes(true, entry.getKey());
b[i][1] = toBytes(true, entry.getValue());
i++;
}
return b;
}
public static byte[] toBytesSerializable(Serializable serializable) {
if (serializable == null)
return EMPTY_BYTES;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ObjectOutputStream oos = new ObjectOutputStream(baos)) {
oos.writeObject(serializable);
} catch (IOException e) {
throw new RuntimeException("Unexpected IOException during serialization", e);
}
return baos.toByteArray();
}
public static Serializable fromBytesSerializable(byte[] bytes) {
if (bytes == null || bytes.length == 0)
return null;
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
try (ObjectInputStream ois = new ObjectInputStream(bais)) {
return (Serializable) ois.readObject();
} catch (IOException e) {
throw new RuntimeException("Unexpected IOException during deserialization", e);
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
}
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/impl/java/JavaStatsInitializationReport.java
+195
View File
@@ -0,0 +1,195 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.impl.java;
import lombok.Data;
import org.apache.commons.compress.utils.IOUtils;
import org.deeplearning4j.ui.model.stats.api.StatsInitializationReport;
import java.io.*;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.util.Map;
@Data
public class JavaStatsInitializationReport implements StatsInitializationReport {
private String sessionID;
private String typeID;
private String workerID;
private long timeStamp;
private boolean hasSoftwareInfo;
private boolean hasHardwareInfo;
private boolean hasModelInfo;
private String swArch;
private String swOsName;
private String swJvmName;
private String swJvmVersion;
private String swJvmSpecVersion;
private String swNd4jBackendClass;
private String swNd4jDataTypeName;
private String swHostName;
private String swJvmUID;
private Map<String, String> swEnvironmentInfo;
private int hwJvmAvailableProcessors;
private int hwNumDevices;
private long hwJvmMaxMemory;
private long hwOffHeapMaxMemory;
private long[] hwDeviceTotalMemory;
private String[] hwDeviceDescription;
private String hwHardwareUID;
private String modelClassName;
private String modelConfigJson;
private String[] modelParamNames;
private int modelNumLayers;
private long modelNumParams;
@Override
public void reportIDs(String sessionID, String typeID, String workerID, long timeStamp) {
this.sessionID = sessionID;
this.typeID = typeID;
this.workerID = workerID;
this.timeStamp = timeStamp;
}
@Override
public void reportSoftwareInfo(String arch, String osName, String jvmName, String jvmVersion, String jvmSpecVersion,
String nd4jBackendClass, String nd4jDataTypeName, String hostname, String jvmUid,
Map<String, String> swEnvironmentInfo) {
this.swArch = arch;
this.swOsName = osName;
this.swJvmName = jvmName;
this.swJvmVersion = jvmVersion;
this.swJvmSpecVersion = jvmSpecVersion;
this.swNd4jBackendClass = nd4jBackendClass;
this.swNd4jDataTypeName = nd4jDataTypeName;
this.swHostName = hostname;
this.swJvmUID = jvmUid;
this.swEnvironmentInfo = swEnvironmentInfo;
hasSoftwareInfo = true;
}
@Override
public void reportHardwareInfo(int jvmAvailableProcessors, int numDevices, long jvmMaxMemory, long offHeapMaxMemory,
long[] deviceTotalMemory, String[] deviceDescription, String hardwareUID) {
this.hwJvmAvailableProcessors = jvmAvailableProcessors;
this.hwNumDevices = numDevices;
this.hwJvmMaxMemory = jvmMaxMemory;
this.hwOffHeapMaxMemory = offHeapMaxMemory;
this.hwDeviceTotalMemory = deviceTotalMemory;
this.hwDeviceDescription = deviceDescription;
this.hwHardwareUID = hardwareUID;
hasHardwareInfo = true;
}
@Override
public void reportModelInfo(String modelClassName, String modelConfigJson, String[] modelParamNames, int numLayers,
long numParams) {
this.modelClassName = modelClassName;
this.modelConfigJson = modelConfigJson;
this.modelParamNames = modelParamNames;
this.modelNumLayers = numLayers;
this.modelNumParams = numParams;
hasModelInfo = true;
}
@Override
public boolean hasSoftwareInfo() {
return hasSoftwareInfo;
}
@Override
public boolean hasHardwareInfo() {
return hasHardwareInfo;
}
@Override
public boolean hasModelInfo() {
return hasModelInfo;
}
@Override
public int encodingLengthBytes() {
//TODO - presumably a more efficient way to do this
byte[] encoded = encode();
return encoded.length;
}
@Override
public byte[] encode() {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ObjectOutputStream oos = new ObjectOutputStream(baos)) {
oos.writeObject(this);
} catch (IOException e) {
throw new RuntimeException(e); //Should never happen
}
return baos.toByteArray();
}
@Override
public void encode(ByteBuffer buffer) {
buffer.put(encode());
}
@Override
public void encode(OutputStream outputStream) throws IOException {
try (ObjectOutputStream oos = new ObjectOutputStream(outputStream)) {
oos.writeObject(this);
}
}
@Override
public void decode(byte[] decode) {
JavaStatsInitializationReport r;
try (ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(decode))) {
r = (JavaStatsInitializationReport) ois.readObject();
} catch (IOException | ClassNotFoundException e) {
throw new RuntimeException(e); //Should never happen
}
Field[] fields = JavaStatsInitializationReport.class.getDeclaredFields();
for (Field f : fields) {
f.setAccessible(true);
try {
f.set(this, f.get(r));
} catch (IllegalAccessException e) {
throw new RuntimeException(e); //Should never happen
}
}
}
@Override
public void decode(ByteBuffer buffer) {
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);
decode(bytes);
}
@Override
public void decode(InputStream inputStream) throws IOException {
decode(IOUtils.toByteArray(inputStream));
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/impl/java/JavaStatsReport.java
+390
View File
@@ -0,0 +1,390 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.impl.java;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.ToString;
import org.apache.commons.compress.utils.IOUtils;
import org.deeplearning4j.ui.model.stats.api.Histogram;
import org.deeplearning4j.ui.model.stats.api.StatsReport;
import org.deeplearning4j.ui.model.stats.api.StatsType;
import org.deeplearning4j.ui.model.stats.api.SummaryType;
import org.nd4j.common.primitives.Pair;
import java.io.*;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@EqualsAndHashCode
@ToString
@Data
public class JavaStatsReport implements StatsReport {
private String sessionID;
private String typeID;
private String workerID;
private long timeStamp;
private int iterationCount;
private int statsCollectionDurationMs;
private double score;
private long jvmCurrentBytes;
private long jvmMaxBytes;
private long offHeapCurrentBytes;
private long offHeapMaxBytes;
private long[] deviceCurrentBytes;
private long[] deviceMaxBytes;
private long totalRuntimeMs;
private long totalExamples;
private long totalMinibatches;
private double examplesPerSecond;
private double minibatchesPerSecond;
private List<GCStats> gcStats;
private Map<String, Double> learningRatesByParam;
private Map<StatsType, Map<String, Histogram>> histograms;
private Map<StatsType, Map<String, Double>> meanValues;
private Map<StatsType, Map<String, Double>> stdevValues;
private Map<StatsType, Map<String, Double>> meanMagnitudeValues;
private String metaDataClassName;
//Store in serialized form; deserialize iff required. Might save us some class not found (or, version) errors, if
// metadata is saved but is never used
private List<byte[]> dataSetMetaData;
private boolean scorePresent;
private boolean memoryUsePresent;
private boolean performanceStatsPresent;
public JavaStatsReport() {
//No-Arg constructor only for deserialization
}
@Override
public void reportIDs(String sessionID, String typeID, String workerID, long timeStamp) {
this.sessionID = sessionID;
this.typeID = typeID;
this.workerID = workerID;
this.timeStamp = timeStamp;
}
@Override
public void reportIterationCount(int iterationCount) {
this.iterationCount = iterationCount;
}
@Override
public void reportStatsCollectionDurationMS(int statsCollectionDurationMS) {
this.statsCollectionDurationMs = statsCollectionDurationMS;
}
@Override
public void reportScore(double currentScore) {
this.score = currentScore;
this.scorePresent = true;
}
@Override
public void reportLearningRates(Map<String, Double> learningRatesByParam) {
this.learningRatesByParam = learningRatesByParam;
}
@Override
public void reportMemoryUse(long jvmCurrentBytes, long jvmMaxBytes, long offHeapCurrentBytes, long offHeapMaxBytes,
long[] deviceCurrentBytes, long[] deviceMaxBytes) {
this.jvmCurrentBytes = jvmCurrentBytes;
this.jvmMaxBytes = jvmMaxBytes;
this.offHeapCurrentBytes = offHeapCurrentBytes;
this.offHeapMaxBytes = offHeapMaxBytes;
this.deviceCurrentBytes = deviceCurrentBytes;
this.deviceMaxBytes = deviceMaxBytes;
this.memoryUsePresent = true;
}
@Override
public void reportPerformance(long totalRuntimeMs, long totalExamples, long totalMinibatches,
double examplesPerSecond, double minibatchesPerSecond) {
this.totalRuntimeMs = totalRuntimeMs;
this.totalExamples = totalExamples;
this.totalMinibatches = totalMinibatches;
this.examplesPerSecond = examplesPerSecond;
this.minibatchesPerSecond = minibatchesPerSecond;
this.performanceStatsPresent = true;
}
@Override
public void reportGarbageCollection(String gcName, int deltaGCCount, int deltaGCTime) {
if (gcStats == null)
gcStats = new ArrayList<>();
gcStats.add(new GCStats(gcName, deltaGCCount, deltaGCTime));
}
@Override
public List<Pair<String, int[]>> getGarbageCollectionStats() {
if (gcStats == null)
return null;
List<Pair<String, int[]>> temp = new ArrayList<>();
for (GCStats g : gcStats) {
temp.add(new Pair<>(g.gcName, new int[] {g.getDeltaGCCount(), g.getDeltaGCTime()}));
}
return temp;
}
@Override
public void reportHistograms(StatsType statsType, Map<String, Histogram> histogram) {
if (this.histograms == null)
this.histograms = new HashMap<>();
this.histograms.put(statsType, histogram);
}
@Override
public Map<String, Histogram> getHistograms(StatsType statsType) {
if (histograms == null)
return null;
return histograms.get(statsType);
}
@Override
public void reportMean(StatsType statsType, Map<String, Double> mean) {
if (this.meanValues == null)
this.meanValues = new HashMap<>();
this.meanValues.put(statsType, mean);
}
@Override
public Map<String, Double> getMean(StatsType statsType) {
if (this.meanValues == null)
return null;
return meanValues.get(statsType);
}
@Override
public void reportStdev(StatsType statsType, Map<String, Double> stdev) {
if (this.stdevValues == null)
this.stdevValues = new HashMap<>();
this.stdevValues.put(statsType, stdev);
}
@Override
public Map<String, Double> getStdev(StatsType statsType) {
if (this.stdevValues == null)
return null;
return stdevValues.get(statsType);
}
@Override
public void reportMeanMagnitudes(StatsType statsType, Map<String, Double> meanMagnitudes) {
if (this.meanMagnitudeValues == null)
this.meanMagnitudeValues = new HashMap<>();
this.meanMagnitudeValues.put(statsType, meanMagnitudes);
}
@Override
public void reportDataSetMetaData(List<Serializable> dataSetMetaData, Class<?> metaDataClass) {
reportDataSetMetaData(dataSetMetaData, (metaDataClass == null ? null : metaDataClass.getName()));
}
@Override
public void reportDataSetMetaData(List<Serializable> dataSetMetaData, String metaDataClass) {
if (dataSetMetaData != null) {
this.dataSetMetaData = new ArrayList<>();
for (Serializable s : dataSetMetaData) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ObjectOutputStream oos = new ObjectOutputStream(baos)) {
oos.writeObject(s);
oos.flush();
oos.close();
} catch (IOException e) {
throw new RuntimeException("Unexpected IOException from ByteArrayOutputStream", e);
}
byte[] b = baos.toByteArray();
this.dataSetMetaData.add(b);
}
} else {
this.dataSetMetaData = null;
}
this.metaDataClassName = metaDataClass;
}
@Override
public Map<String, Double> getMeanMagnitudes(StatsType statsType) {
if (this.meanMagnitudeValues == null)
return null;
return this.meanMagnitudeValues.get(statsType);
}
@Override
public List<Serializable> getDataSetMetaData() {
if (dataSetMetaData == null || dataSetMetaData.isEmpty())
return null;
List<Serializable> l = new ArrayList<>();
for (byte[] b : dataSetMetaData) {
try (ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(b))) {
l.add((Serializable) ois.readObject());
} catch (IOException | ClassNotFoundException e) {
throw new RuntimeException(e);
}
}
return l;
}
@Override
public String getDataSetMetaDataClassName() {
return metaDataClassName;
}
@Override
public Map<String, Double> getLearningRates() {
return this.learningRatesByParam;
}
@Override
public boolean hasScore() {
return scorePresent;
}
@Override
public boolean hasLearningRates() {
return learningRatesByParam != null;
}
@Override
public boolean hasMemoryUse() {
return memoryUsePresent;
}
@Override
public boolean hasPerformance() {
return performanceStatsPresent;
}
@Override
public boolean hasGarbageCollection() {
return gcStats != null && !gcStats.isEmpty();
}
@Override
public boolean hasHistograms(StatsType statsType) {
if (histograms == null)
return false;
return histograms.containsKey(statsType);
}
@Override
public boolean hasSummaryStats(StatsType statsType, SummaryType summaryType) {
switch (summaryType) {
case Mean:
return meanValues != null && meanValues.containsKey(statsType);
case Stdev:
return stdevValues != null && stdevValues.containsKey(statsType);
case MeanMagnitudes:
return meanMagnitudeValues != null && meanMagnitudeValues.containsKey(statsType);
}
return false;
}
@Override
public boolean hasDataSetMetaData() {
return dataSetMetaData != null || metaDataClassName != null;
}
@AllArgsConstructor
@Data
private static class GCStats implements Serializable {
private String gcName;
private int deltaGCCount;
private int deltaGCTime;
}
@Override
public int encodingLengthBytes() {
//TODO - presumably a more efficient way to do this
byte[] encoded = encode();
return encoded.length;
}
@Override
public byte[] encode() {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ObjectOutputStream oos = new ObjectOutputStream(baos)) {
oos.writeObject(this);
} catch (IOException e) {
throw new RuntimeException(e); //Should never happen
}
return baos.toByteArray();
}
@Override
public void encode(ByteBuffer buffer) {
buffer.put(encode());
}
@Override
public void encode(OutputStream outputStream) throws IOException {
try (ObjectOutputStream oos = new ObjectOutputStream(outputStream)) {
oos.writeObject(this);
}
}
@Override
public void decode(byte[] decode) {
JavaStatsReport r;
try (ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(decode))) {
r = (JavaStatsReport) ois.readObject();
} catch (IOException | ClassNotFoundException e) {
throw new RuntimeException(e); //Should never happen
}
Field[] fields = JavaStatsReport.class.getDeclaredFields();
for (Field f : fields) {
f.setAccessible(true);
try {
f.set(this, f.get(r));
} catch (IllegalAccessException e) {
throw new RuntimeException(e); //Should never happen
}
}
}
@Override
public void decode(ByteBuffer buffer) {
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);
decode(bytes);
}
@Override
public void decode(InputStream inputStream) throws IOException {
decode(IOUtils.toByteArray(inputStream));
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/GroupSizeEncodingDecoder.java
+93
View File
@@ -0,0 +1,93 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.DirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.GroupSizeEncodingDecoder"})
@SuppressWarnings("all")
public class GroupSizeEncodingDecoder {
public static final int ENCODED_LENGTH = 4;
private DirectBuffer buffer;
private int offset;
public GroupSizeEncodingDecoder wrap(final DirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public static int blockLengthNullValue() {
return 65535;
}
public static int blockLengthMinValue() {
return 0;
}
public static int blockLengthMaxValue() {
return 65534;
}
public int blockLength() {
return (buffer.getShort(offset + 0, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF);
}
public static int numInGroupNullValue() {
return 65535;
}
public static int numInGroupMinValue() {
return 0;
}
public static int numInGroupMaxValue() {
return 65534;
}
public int numInGroup() {
return (buffer.getShort(offset + 2, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF);
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
builder.append('(');
//Token{signal=ENCODING, name='blockLength', description='Extra metadata bytes', id=-1, version=0, encodedLength=2, offset=0, componentTokenCount=1, encoding=Encoding{presence=REQUIRED, primitiveType=UINT16, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='UTF-8', epoch='null', timeUnit=null, semanticType='null'}}
builder.append("blockLength=");
builder.append(blockLength());
builder.append('|');
//Token{signal=ENCODING, name='numInGroup', description='Extra metadata bytes', id=-1, version=0, encodedLength=2, offset=2, componentTokenCount=1, encoding=Encoding{presence=REQUIRED, primitiveType=UINT16, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='UTF-8', epoch='null', timeUnit=null, semanticType='null'}}
builder.append("numInGroup=");
builder.append(numInGroup());
builder.append(')');
return builder;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/GroupSizeEncodingEncoder.java
+88
View File
@@ -0,0 +1,88 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.MutableDirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.GroupSizeEncodingEncoder"})
@SuppressWarnings("all")
public class GroupSizeEncodingEncoder {
public static final int ENCODED_LENGTH = 4;
private MutableDirectBuffer buffer;
private int offset;
public GroupSizeEncodingEncoder wrap(final MutableDirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public static int blockLengthNullValue() {
return 65535;
}
public static int blockLengthMinValue() {
return 0;
}
public static int blockLengthMaxValue() {
return 65534;
}
public GroupSizeEncodingEncoder blockLength(final int value) {
buffer.putShort(offset + 0, (short) value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public static int numInGroupNullValue() {
return 65535;
}
public static int numInGroupMinValue() {
return 0;
}
public static int numInGroupMaxValue() {
return 65534;
}
public GroupSizeEncodingEncoder numInGroup(final int value) {
buffer.putShort(offset + 2, (short) value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
GroupSizeEncodingDecoder writer = new GroupSizeEncodingDecoder();
writer.wrap(buffer, offset);
return writer.appendTo(builder);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/InitFieldsPresentDecoder.java
+87
View File
@@ -0,0 +1,87 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.DirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.InitFieldsPresentDecoder"})
@SuppressWarnings("all")
public class InitFieldsPresentDecoder {
public static final int ENCODED_LENGTH = 1;
private DirectBuffer buffer;
private int offset;
public InitFieldsPresentDecoder wrap(final DirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public boolean softwareInfo() {
return 0 != (buffer.getByte(offset) & (1 << 0));
}
public boolean hardwareInfo() {
return 0 != (buffer.getByte(offset) & (1 << 1));
}
public boolean modelInfo() {
return 0 != (buffer.getByte(offset) & (1 << 2));
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
builder.append('{');
boolean atLeastOne = false;
if (softwareInfo()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("softwareInfo");
atLeastOne = true;
}
if (hardwareInfo()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("hardwareInfo");
atLeastOne = true;
}
if (modelInfo()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("modelInfo");
atLeastOne = true;
}
builder.append('}');
return builder;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/InitFieldsPresentEncoder.java
+68
View File
@@ -0,0 +1,68 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.MutableDirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.InitFieldsPresentEncoder"})
@SuppressWarnings("all")
public class InitFieldsPresentEncoder {
public static final int ENCODED_LENGTH = 1;
private MutableDirectBuffer buffer;
private int offset;
public InitFieldsPresentEncoder wrap(final MutableDirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public InitFieldsPresentEncoder clear() {
buffer.putByte(offset, (byte) (short) 0);
return this;
}
public InitFieldsPresentEncoder softwareInfo(final boolean value) {
byte bits = buffer.getByte(offset);
bits = (byte) (value ? bits | (1 << 0) : bits & ~(1 << 0));
buffer.putByte(offset, bits);
return this;
}
public InitFieldsPresentEncoder hardwareInfo(final boolean value) {
byte bits = buffer.getByte(offset);
bits = (byte) (value ? bits | (1 << 1) : bits & ~(1 << 1));
buffer.putByte(offset, bits);
return this;
}
public InitFieldsPresentEncoder modelInfo(final boolean value) {
byte bits = buffer.getByte(offset);
bits = (byte) (value ? bits | (1 << 2) : bits & ~(1 << 2));
buffer.putByte(offset, bits);
return this;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/MemoryType.java
+60
View File
@@ -0,0 +1,60 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.MemoryType"})
public enum MemoryType {
JvmCurrent((short) 0), JvmMax((short) 1), OffHeapCurrent((short) 2), OffHeapMax((short) 3), DeviceCurrent(
(short) 4), DeviceMax((short) 5), NULL_VAL((short) 255);
private final short value;
MemoryType(final short value) {
this.value = value;
}
public short value() {
return value;
}
public static MemoryType get(final short value) {
switch (value) {
case 0:
return JvmCurrent;
case 1:
return JvmMax;
case 2:
return OffHeapCurrent;
case 3:
return OffHeapMax;
case 4:
return DeviceCurrent;
case 5:
return DeviceMax;
}
if ((short) 255 == value) {
return NULL_VAL;
}
throw new IllegalArgumentException("Unknown value: " + value);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/MessageHeaderDecoder.java
+110
View File
@@ -0,0 +1,110 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.DirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.MessageHeaderDecoder"})
@SuppressWarnings("all")
public class MessageHeaderDecoder {
public static final int ENCODED_LENGTH = 8;
private DirectBuffer buffer;
private int offset;
public MessageHeaderDecoder wrap(final DirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public static int blockLengthNullValue() {
return 65535;
}
public static int blockLengthMinValue() {
return 0;
}
public static int blockLengthMaxValue() {
return 65534;
}
public int blockLength() {
return (buffer.getShort(offset + 0, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF);
}
public static int templateIdNullValue() {
return 65535;
}
public static int templateIdMinValue() {
return 0;
}
public static int templateIdMaxValue() {
return 65534;
}
public int templateId() {
return (buffer.getShort(offset + 2, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF);
}
public static int schemaIdNullValue() {
return 65535;
}
public static int schemaIdMinValue() {
return 0;
}
public static int schemaIdMaxValue() {
return 65534;
}
public int schemaId() {
return (buffer.getShort(offset + 4, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF);
}
public static int versionNullValue() {
return 65535;
}
public static int versionMinValue() {
return 0;
}
public static int versionMaxValue() {
return 65534;
}
public int version() {
return (buffer.getShort(offset + 6, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/MessageHeaderEncoder.java
+114
View File
@@ -0,0 +1,114 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.MutableDirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.MessageHeaderEncoder"})
@SuppressWarnings("all")
public class MessageHeaderEncoder {
public static final int ENCODED_LENGTH = 8;
private MutableDirectBuffer buffer;
private int offset;
public MessageHeaderEncoder wrap(final MutableDirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public static int blockLengthNullValue() {
return 65535;
}
public static int blockLengthMinValue() {
return 0;
}
public static int blockLengthMaxValue() {
return 65534;
}
public MessageHeaderEncoder blockLength(final int value) {
buffer.putShort(offset + 0, (short) value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public static int templateIdNullValue() {
return 65535;
}
public static int templateIdMinValue() {
return 0;
}
public static int templateIdMaxValue() {
return 65534;
}
public MessageHeaderEncoder templateId(final int value) {
buffer.putShort(offset + 2, (short) value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public static int schemaIdNullValue() {
return 65535;
}
public static int schemaIdMinValue() {
return 0;
}
public static int schemaIdMaxValue() {
return 65534;
}
public MessageHeaderEncoder schemaId(final int value) {
buffer.putShort(offset + 4, (short) value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public static int versionNullValue() {
return 65535;
}
public static int versionMinValue() {
return 0;
}
public static int versionMaxValue() {
return 65534;
}
public MessageHeaderEncoder version(final int value) {
buffer.putShort(offset + 6, (short) value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/MetaAttribute.java
+26
View File
@@ -0,0 +1,26 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.MetaAttribute"})
public enum MetaAttribute {
EPOCH, TIME_UNIT, SEMANTIC_TYPE
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/StatSource.java
+53
View File
@@ -0,0 +1,53 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
@javax.annotation.Generated(value = {"StatSource"})
public enum StatSource {
Parameters((short) 0), Updates((short) 1), Activations((short) 2), NULL_VAL((short) 255);
private final short value;
StatSource(final short value) {
this.value = value;
}
public short value() {
return value;
}
public static StatSource get(final short value) {
switch (value) {
case 0:
return Parameters;
case 1:
return Updates;
case 2:
return Activations;
}
if ((short) 255 == value) {
return NULL_VAL;
}
throw new IllegalArgumentException("Unknown value: " + value);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/StatType.java
+53
View File
@@ -0,0 +1,53 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
@javax.annotation.Generated(value = {"StatType"})
public enum StatType {
Mean((short) 0), Stdev((short) 1), MeanMagnitude((short) 2), NULL_VAL((short) 255);
private final short value;
StatType(final short value) {
this.value = value;
}
public short value() {
return value;
}
public static StatType get(final short value) {
switch (value) {
case 0:
return Mean;
case 1:
return Stdev;
case 2:
return MeanMagnitude;
}
if ((short) 255 == value) {
return NULL_VAL;
}
throw new IllegalArgumentException("Unknown value: " + value);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/StaticInfoDecoder.java
+2148
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/StaticInfoEncoder.java
+1853
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/StatsType.java
+55
View File
@@ -0,0 +1,55 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.StatsType"})
public enum StatsType {
Parameters((short) 0), Gradients((short) 1), Updates((short) 2), Activations((short) 3), NULL_VAL((short) 255);
private final short value;
StatsType(final short value) {
this.value = value;
}
public short value() {
return value;
}
public static StatsType get(final short value) {
switch (value) {
case 0:
return Parameters;
case 1:
return Gradients;
case 2:
return Updates;
case 3:
return Activations;
}
if ((short) 255 == value) {
return NULL_VAL;
}
throw new IllegalArgumentException("Unknown value: " + value);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/StorageMetaDataDecoder.java
+659
View File
@@ -0,0 +1,659 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.DirectBuffer;
import org.agrona.MutableDirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.StorageMetaDataDecoder"})
@SuppressWarnings("all")
public class StorageMetaDataDecoder {
public static final int BLOCK_LENGTH = 8;
public static final int TEMPLATE_ID = 3;
public static final int SCHEMA_ID = 1;
public static final int SCHEMA_VERSION = 0;
private final StorageMetaDataDecoder parentMessage = this;
private DirectBuffer buffer;
protected int offset;
protected int limit;
protected int actingBlockLength;
protected int actingVersion;
public int sbeBlockLength() {
return BLOCK_LENGTH;
}
public int sbeTemplateId() {
return TEMPLATE_ID;
}
public int sbeSchemaId() {
return SCHEMA_ID;
}
public int sbeSchemaVersion() {
return SCHEMA_VERSION;
}
public String sbeSemanticType() {
return "";
}
public int offset() {
return offset;
}
public StorageMetaDataDecoder wrap(final DirectBuffer buffer, final int offset, final int actingBlockLength,
final int actingVersion) {
this.buffer = buffer;
this.offset = offset;
this.actingBlockLength = actingBlockLength;
this.actingVersion = actingVersion;
limit(offset + actingBlockLength);
return this;
}
public int encodedLength() {
return limit - offset;
}
public int limit() {
return limit;
}
public void limit(final int limit) {
this.limit = limit;
}
public static int timeStampId() {
return 1;
}
public static String timeStampMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static long timeStampNullValue() {
return -9223372036854775808L;
}
public static long timeStampMinValue() {
return -9223372036854775807L;
}
public static long timeStampMaxValue() {
return 9223372036854775807L;
}
public long timeStamp() {
return buffer.getLong(offset + 0, java.nio.ByteOrder.LITTLE_ENDIAN);
}
private final ExtraMetaDataBytesDecoder extraMetaDataBytes = new ExtraMetaDataBytesDecoder();
public static long extraMetaDataBytesDecoderId() {
return 2;
}
public ExtraMetaDataBytesDecoder extraMetaDataBytes() {
extraMetaDataBytes.wrap(parentMessage, buffer);
return extraMetaDataBytes;
}
public static class ExtraMetaDataBytesDecoder
implements Iterable<ExtraMetaDataBytesDecoder>, java.util.Iterator<ExtraMetaDataBytesDecoder> {
private static final int HEADER_SIZE = 4;
private final GroupSizeEncodingDecoder dimensions = new GroupSizeEncodingDecoder();
private StorageMetaDataDecoder parentMessage;
private DirectBuffer buffer;
private int blockLength;
private int actingVersion;
private int count;
private int index;
private int offset;
public void wrap(final StorageMetaDataDecoder parentMessage, final DirectBuffer buffer) {
this.parentMessage = parentMessage;
this.buffer = buffer;
dimensions.wrap(buffer, parentMessage.limit());
blockLength = dimensions.blockLength();
count = dimensions.numInGroup();
index = -1;
parentMessage.limit(parentMessage.limit() + HEADER_SIZE);
}
public static int sbeHeaderSize() {
return HEADER_SIZE;
}
public static int sbeBlockLength() {
return 1;
}
public int actingBlockLength() {
return blockLength;
}
public int count() {
return count;
}
public java.util.Iterator<ExtraMetaDataBytesDecoder> iterator() {
return this;
}
public void remove() {
throw new UnsupportedOperationException();
}
public boolean hasNext() {
return (index + 1) < count;
}
public ExtraMetaDataBytesDecoder next() {
if (index + 1 >= count) {
throw new java.util.NoSuchElementException();
}
offset = parentMessage.limit();
parentMessage.limit(offset + blockLength);
++index;
return this;
}
public static int bytesId() {
return 3;
}
public static String bytesMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static byte bytesNullValue() {
return (byte) -128;
}
public static byte bytesMinValue() {
return (byte) -127;
}
public static byte bytesMaxValue() {
return (byte) 127;
}
public byte bytes() {
return buffer.getByte(offset + 0);
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
builder.append('(');
//Token{signal=BEGIN_FIELD, name='bytes', description='null', id=3, version=0, encodedLength=0, offset=0, componentTokenCount=3, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
//Token{signal=ENCODING, name='int8', description='null', id=-1, version=0, encodedLength=1, offset=0, componentTokenCount=1, encoding=Encoding{presence=REQUIRED, primitiveType=INT8, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("bytes=");
builder.append(bytes());
builder.append(')');
return builder;
}
}
public static int sessionIDId() {
return 4;
}
public static String sessionIDCharacterEncoding() {
return "UTF-8";
}
public static String sessionIDMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int sessionIDHeaderLength() {
return 4;
}
public int sessionIDLength() {
final int limit = parentMessage.limit();
return (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
}
public int getSessionID(final MutableDirectBuffer dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public int getSessionID(final byte[] dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public String sessionID() {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
parentMessage.limit(limit + headerLength + dataLength);
final byte[] tmp = new byte[dataLength];
buffer.getBytes(limit + headerLength, tmp, 0, dataLength);
final String value;
try {
value = new String(tmp, "UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
return value;
}
public static int typeIDId() {
return 5;
}
public static String typeIDCharacterEncoding() {
return "UTF-8";
}
public static String typeIDMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int typeIDHeaderLength() {
return 4;
}
public int typeIDLength() {
final int limit = parentMessage.limit();
return (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
}
public int getTypeID(final MutableDirectBuffer dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public int getTypeID(final byte[] dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public String typeID() {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
parentMessage.limit(limit + headerLength + dataLength);
final byte[] tmp = new byte[dataLength];
buffer.getBytes(limit + headerLength, tmp, 0, dataLength);
final String value;
try {
value = new String(tmp, "UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
return value;
}
public static int workerIDId() {
return 6;
}
public static String workerIDCharacterEncoding() {
return "UTF-8";
}
public static String workerIDMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int workerIDHeaderLength() {
return 4;
}
public int workerIDLength() {
final int limit = parentMessage.limit();
return (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
}
public int getWorkerID(final MutableDirectBuffer dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public int getWorkerID(final byte[] dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public String workerID() {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
parentMessage.limit(limit + headerLength + dataLength);
final byte[] tmp = new byte[dataLength];
buffer.getBytes(limit + headerLength, tmp, 0, dataLength);
final String value;
try {
value = new String(tmp, "UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
return value;
}
public static int initTypeClassId() {
return 7;
}
public static String initTypeClassCharacterEncoding() {
return "UTF-8";
}
public static String initTypeClassMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int initTypeClassHeaderLength() {
return 4;
}
public int initTypeClassLength() {
final int limit = parentMessage.limit();
return (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
}
public int getInitTypeClass(final MutableDirectBuffer dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public int getInitTypeClass(final byte[] dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public String initTypeClass() {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
parentMessage.limit(limit + headerLength + dataLength);
final byte[] tmp = new byte[dataLength];
buffer.getBytes(limit + headerLength, tmp, 0, dataLength);
final String value;
try {
value = new String(tmp, "UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
return value;
}
public static int updateTypeClassId() {
return 8;
}
public static String updateTypeClassCharacterEncoding() {
return "UTF-8";
}
public static String updateTypeClassMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int updateTypeClassHeaderLength() {
return 4;
}
public int updateTypeClassLength() {
final int limit = parentMessage.limit();
return (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
}
public int getUpdateTypeClass(final MutableDirectBuffer dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public int getUpdateTypeClass(final byte[] dst, final int dstOffset, final int length) {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
final int bytesCopied = Math.min(length, dataLength);
parentMessage.limit(limit + headerLength + dataLength);
buffer.getBytes(limit + headerLength, dst, dstOffset, bytesCopied);
return bytesCopied;
}
public String updateTypeClass() {
final int headerLength = 4;
final int limit = parentMessage.limit();
final int dataLength = (int) (buffer.getInt(limit, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
parentMessage.limit(limit + headerLength + dataLength);
final byte[] tmp = new byte[dataLength];
buffer.getBytes(limit + headerLength, tmp, 0, dataLength);
final String value;
try {
value = new String(tmp, "UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
return value;
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
final int originalLimit = limit();
limit(offset + actingBlockLength);
builder.append("[StorageMetaData](sbeTemplateId=");
builder.append(TEMPLATE_ID);
builder.append("|sbeSchemaId=");
builder.append(SCHEMA_ID);
builder.append("|sbeSchemaVersion=");
if (actingVersion != SCHEMA_VERSION) {
builder.append(actingVersion);
builder.append('/');
}
builder.append(SCHEMA_VERSION);
builder.append("|sbeBlockLength=");
if (actingBlockLength != BLOCK_LENGTH) {
builder.append(actingBlockLength);
builder.append('/');
}
builder.append(BLOCK_LENGTH);
builder.append("):");
//Token{signal=BEGIN_FIELD, name='timeStamp', description='null', id=1, version=0, encodedLength=0, offset=0, componentTokenCount=3, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
//Token{signal=ENCODING, name='int64', description='null', id=-1, version=0, encodedLength=8, offset=0, componentTokenCount=1, encoding=Encoding{presence=REQUIRED, primitiveType=INT64, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("timeStamp=");
builder.append(timeStamp());
builder.append('|');
//Token{signal=BEGIN_GROUP, name='extraMetaDataBytes', description='Extra metadata bytes', id=2, version=0, encodedLength=1, offset=8, componentTokenCount=9, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='null', timeUnit=null, semanticType='null'}}
builder.append("extraMetaDataBytes=[");
ExtraMetaDataBytesDecoder extraMetaDataBytes = extraMetaDataBytes();
if (extraMetaDataBytes.count() > 0) {
while (extraMetaDataBytes.hasNext()) {
extraMetaDataBytes.next().appendTo(builder);
builder.append(',');
}
builder.setLength(builder.length() - 1);
}
builder.append(']');
builder.append('|');
//Token{signal=BEGIN_VAR_DATA, name='sessionID', description='null', id=4, version=0, encodedLength=0, offset=-1, componentTokenCount=6, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("sessionID=");
builder.append(sessionID());
builder.append('|');
//Token{signal=BEGIN_VAR_DATA, name='typeID', description='null', id=5, version=0, encodedLength=0, offset=-1, componentTokenCount=6, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("typeID=");
builder.append(typeID());
builder.append('|');
//Token{signal=BEGIN_VAR_DATA, name='workerID', description='null', id=6, version=0, encodedLength=0, offset=-1, componentTokenCount=6, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("workerID=");
builder.append(workerID());
builder.append('|');
//Token{signal=BEGIN_VAR_DATA, name='initTypeClass', description='null', id=7, version=0, encodedLength=0, offset=-1, componentTokenCount=6, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("initTypeClass=");
builder.append(initTypeClass());
builder.append('|');
//Token{signal=BEGIN_VAR_DATA, name='updateTypeClass', description='null', id=8, version=0, encodedLength=0, offset=-1, componentTokenCount=6, encoding=Encoding{presence=REQUIRED, primitiveType=null, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='null', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("updateTypeClass=");
builder.append(updateTypeClass());
limit(originalLimit);
return builder;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/StorageMetaDataEncoder.java
+567
View File
@@ -0,0 +1,567 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.DirectBuffer;
import org.agrona.MutableDirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.StorageMetaDataEncoder"})
@SuppressWarnings("all")
public class StorageMetaDataEncoder {
public static final int BLOCK_LENGTH = 8;
public static final int TEMPLATE_ID = 3;
public static final int SCHEMA_ID = 1;
public static final int SCHEMA_VERSION = 0;
private final StorageMetaDataEncoder parentMessage = this;
private MutableDirectBuffer buffer;
protected int offset;
protected int limit;
protected int actingBlockLength;
protected int actingVersion;
public int sbeBlockLength() {
return BLOCK_LENGTH;
}
public int sbeTemplateId() {
return TEMPLATE_ID;
}
public int sbeSchemaId() {
return SCHEMA_ID;
}
public int sbeSchemaVersion() {
return SCHEMA_VERSION;
}
public String sbeSemanticType() {
return "";
}
public int offset() {
return offset;
}
public StorageMetaDataEncoder wrap(final MutableDirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
limit(offset + BLOCK_LENGTH);
return this;
}
public int encodedLength() {
return limit - offset;
}
public int limit() {
return limit;
}
public void limit(final int limit) {
this.limit = limit;
}
public static long timeStampNullValue() {
return -9223372036854775808L;
}
public static long timeStampMinValue() {
return -9223372036854775807L;
}
public static long timeStampMaxValue() {
return 9223372036854775807L;
}
public StorageMetaDataEncoder timeStamp(final long value) {
buffer.putLong(offset + 0, value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
private final ExtraMetaDataBytesEncoder extraMetaDataBytes = new ExtraMetaDataBytesEncoder();
public static long extraMetaDataBytesId() {
return 2;
}
public ExtraMetaDataBytesEncoder extraMetaDataBytesCount(final int count) {
extraMetaDataBytes.wrap(parentMessage, buffer, count);
return extraMetaDataBytes;
}
public static class ExtraMetaDataBytesEncoder {
private static final int HEADER_SIZE = 4;
private final GroupSizeEncodingEncoder dimensions = new GroupSizeEncodingEncoder();
private StorageMetaDataEncoder parentMessage;
private MutableDirectBuffer buffer;
private int blockLength;
private int actingVersion;
private int count;
private int index;
private int offset;
public void wrap(final StorageMetaDataEncoder parentMessage, final MutableDirectBuffer buffer,
final int count) {
if (count < 0 || count > 65534) {
throw new IllegalArgumentException("count outside allowed range: count=" + count);
}
this.parentMessage = parentMessage;
this.buffer = buffer;
actingVersion = SCHEMA_VERSION;
dimensions.wrap(buffer, parentMessage.limit());
dimensions.blockLength((int) 1);
dimensions.numInGroup((int) count);
index = -1;
this.count = count;
blockLength = 1;
parentMessage.limit(parentMessage.limit() + HEADER_SIZE);
}
public static int sbeHeaderSize() {
return HEADER_SIZE;
}
public static int sbeBlockLength() {
return 1;
}
public ExtraMetaDataBytesEncoder next() {
if (index + 1 >= count) {
throw new java.util.NoSuchElementException();
}
offset = parentMessage.limit();
parentMessage.limit(offset + blockLength);
++index;
return this;
}
public static byte bytesNullValue() {
return (byte) -128;
}
public static byte bytesMinValue() {
return (byte) -127;
}
public static byte bytesMaxValue() {
return (byte) 127;
}
public ExtraMetaDataBytesEncoder bytes(final byte value) {
buffer.putByte(offset + 0, value);
return this;
}
}
public static int sessionIDId() {
return 4;
}
public static String sessionIDCharacterEncoding() {
return "UTF-8";
}
public static String sessionIDMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int sessionIDHeaderLength() {
return 4;
}
public StorageMetaDataEncoder putSessionID(final DirectBuffer src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder putSessionID(final byte[] src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder sessionID(final String value) {
final byte[] bytes;
try {
bytes = value.getBytes("UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
final int length = bytes.length;
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, bytes, 0, length);
return this;
}
public static int typeIDId() {
return 5;
}
public static String typeIDCharacterEncoding() {
return "UTF-8";
}
public static String typeIDMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int typeIDHeaderLength() {
return 4;
}
public StorageMetaDataEncoder putTypeID(final DirectBuffer src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder putTypeID(final byte[] src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder typeID(final String value) {
final byte[] bytes;
try {
bytes = value.getBytes("UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
final int length = bytes.length;
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, bytes, 0, length);
return this;
}
public static int workerIDId() {
return 6;
}
public static String workerIDCharacterEncoding() {
return "UTF-8";
}
public static String workerIDMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int workerIDHeaderLength() {
return 4;
}
public StorageMetaDataEncoder putWorkerID(final DirectBuffer src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder putWorkerID(final byte[] src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder workerID(final String value) {
final byte[] bytes;
try {
bytes = value.getBytes("UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
final int length = bytes.length;
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, bytes, 0, length);
return this;
}
public static int initTypeClassId() {
return 7;
}
public static String initTypeClassCharacterEncoding() {
return "UTF-8";
}
public static String initTypeClassMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int initTypeClassHeaderLength() {
return 4;
}
public StorageMetaDataEncoder putInitTypeClass(final DirectBuffer src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder putInitTypeClass(final byte[] src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder initTypeClass(final String value) {
final byte[] bytes;
try {
bytes = value.getBytes("UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
final int length = bytes.length;
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, bytes, 0, length);
return this;
}
public static int updateTypeClassId() {
return 8;
}
public static String updateTypeClassCharacterEncoding() {
return "UTF-8";
}
public static String updateTypeClassMetaAttribute(final MetaAttribute metaAttribute) {
switch (metaAttribute) {
case EPOCH:
return "unix";
case TIME_UNIT:
return "nanosecond";
case SEMANTIC_TYPE:
return "";
}
return "";
}
public static int updateTypeClassHeaderLength() {
return 4;
}
public StorageMetaDataEncoder putUpdateTypeClass(final DirectBuffer src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder putUpdateTypeClass(final byte[] src, final int srcOffset, final int length) {
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, src, srcOffset, length);
return this;
}
public StorageMetaDataEncoder updateTypeClass(final String value) {
final byte[] bytes;
try {
bytes = value.getBytes("UTF-8");
} catch (final java.io.UnsupportedEncodingException ex) {
throw new RuntimeException(ex);
}
final int length = bytes.length;
if (length > 1073741824) {
throw new IllegalArgumentException("length > max value for type: " + length);
}
final int headerLength = 4;
final int limit = parentMessage.limit();
parentMessage.limit(limit + headerLength + length);
buffer.putInt(limit, (int) length, java.nio.ByteOrder.LITTLE_ENDIAN);
buffer.putBytes(limit + headerLength, bytes, 0, length);
return this;
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
StorageMetaDataDecoder writer = new StorageMetaDataDecoder();
writer.wrap(buffer, offset, BLOCK_LENGTH, SCHEMA_VERSION);
return writer.appendTo(builder);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/SummaryType.java
+53
View File
@@ -0,0 +1,53 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.SummaryType"})
public enum SummaryType {
Mean((short) 0), Stdev((short) 1), MeanMagnitude((short) 2), NULL_VAL((short) 255);
private final short value;
SummaryType(final short value) {
this.value = value;
}
public short value() {
return value;
}
public static SummaryType get(final short value) {
switch (value) {
case 0:
return Mean;
case 1:
return Stdev;
case 2:
return MeanMagnitude;
}
if ((short) 255 == value) {
return NULL_VAL;
}
throw new IllegalArgumentException("Unknown value: " + value);
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/UpdateDecoder.java
+2594
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/UpdateEncoder.java
+1667
View File
File diff suppressed because it is too large Load Diff
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/UpdateFieldsPresentDecoder.java
+296
View File
@@ -0,0 +1,296 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.DirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.UpdateFieldsPresentDecoder"})
@SuppressWarnings("all")
public class UpdateFieldsPresentDecoder {
public static final int ENCODED_LENGTH = 4;
private DirectBuffer buffer;
private int offset;
public UpdateFieldsPresentDecoder wrap(final DirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public boolean score() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 0));
}
public boolean memoryUse() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 1));
}
public boolean performance() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 2));
}
public boolean garbageCollection() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 3));
}
public boolean histogramParameters() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 4));
}
public boolean histogramGradients() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 5));
}
public boolean histogramUpdates() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 6));
}
public boolean histogramActivations() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 7));
}
public boolean meanParameters() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 8));
}
public boolean meanGradients() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 9));
}
public boolean meanUpdates() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 10));
}
public boolean meanActivations() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 11));
}
public boolean stdevParameters() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 12));
}
public boolean stdevGradients() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 13));
}
public boolean stdevUpdates() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 14));
}
public boolean stdevActivations() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 15));
}
public boolean meanMagnitudeParameters() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 16));
}
public boolean meanMagnitudeGradients() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 17));
}
public boolean meanMagnitudeUpdates() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 18));
}
public boolean meanMagnitudeActivations() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 19));
}
public boolean learningRatesPresent() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 20));
}
public boolean dataSetMetaDataPresent() {
return 0 != (buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN) & (1 << 21));
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
builder.append('{');
boolean atLeastOne = false;
if (score()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("score");
atLeastOne = true;
}
if (memoryUse()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("memoryUse");
atLeastOne = true;
}
if (performance()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("performance");
atLeastOne = true;
}
if (garbageCollection()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("garbageCollection");
atLeastOne = true;
}
if (histogramParameters()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("histogramParameters");
atLeastOne = true;
}
if (histogramGradients()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("histogramGradients");
atLeastOne = true;
}
if (histogramUpdates()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("histogramUpdates");
atLeastOne = true;
}
if (histogramActivations()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("histogramActivations");
atLeastOne = true;
}
if (meanParameters()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanParameters");
atLeastOne = true;
}
if (meanGradients()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanGradients");
atLeastOne = true;
}
if (meanUpdates()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanUpdates");
atLeastOne = true;
}
if (meanActivations()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanActivations");
atLeastOne = true;
}
if (stdevParameters()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("stdevParameters");
atLeastOne = true;
}
if (stdevGradients()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("stdevGradients");
atLeastOne = true;
}
if (stdevUpdates()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("stdevUpdates");
atLeastOne = true;
}
if (stdevActivations()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("stdevActivations");
atLeastOne = true;
}
if (meanMagnitudeParameters()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanMagnitudeParameters");
atLeastOne = true;
}
if (meanMagnitudeGradients()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanMagnitudeGradients");
atLeastOne = true;
}
if (meanMagnitudeUpdates()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanMagnitudeUpdates");
atLeastOne = true;
}
if (meanMagnitudeActivations()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("meanMagnitudeActivations");
atLeastOne = true;
}
if (learningRatesPresent()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("learningRatesPresent");
atLeastOne = true;
}
if (dataSetMetaDataPresent()) {
if (atLeastOne) {
builder.append(',');
}
builder.append("dataSetMetaDataPresent");
atLeastOne = true;
}
builder.append('}');
return builder;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/UpdateFieldsPresentEncoder.java
+201
View File
@@ -0,0 +1,201 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.MutableDirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.UpdateFieldsPresentEncoder"})
@SuppressWarnings("all")
public class UpdateFieldsPresentEncoder {
public static final int ENCODED_LENGTH = 4;
private MutableDirectBuffer buffer;
private int offset;
public UpdateFieldsPresentEncoder wrap(final MutableDirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public UpdateFieldsPresentEncoder clear() {
buffer.putInt(offset, (int) 0L, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder score(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 0) : bits & ~(1 << 0);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder memoryUse(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 1) : bits & ~(1 << 1);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder performance(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 2) : bits & ~(1 << 2);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder garbageCollection(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 3) : bits & ~(1 << 3);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder histogramParameters(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 4) : bits & ~(1 << 4);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder histogramGradients(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 5) : bits & ~(1 << 5);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder histogramUpdates(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 6) : bits & ~(1 << 6);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder histogramActivations(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 7) : bits & ~(1 << 7);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanParameters(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 8) : bits & ~(1 << 8);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanGradients(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 9) : bits & ~(1 << 9);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanUpdates(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 10) : bits & ~(1 << 10);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanActivations(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 11) : bits & ~(1 << 11);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder stdevParameters(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 12) : bits & ~(1 << 12);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder stdevGradients(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 13) : bits & ~(1 << 13);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder stdevUpdates(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 14) : bits & ~(1 << 14);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder stdevActivations(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 15) : bits & ~(1 << 15);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanMagnitudeParameters(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 16) : bits & ~(1 << 16);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanMagnitudeGradients(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 17) : bits & ~(1 << 17);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanMagnitudeUpdates(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 18) : bits & ~(1 << 18);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder meanMagnitudeActivations(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 19) : bits & ~(1 << 19);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder learningRatesPresent(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 20) : bits & ~(1 << 20);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public UpdateFieldsPresentEncoder dataSetMetaDataPresent(final boolean value) {
int bits = buffer.getInt(offset, java.nio.ByteOrder.LITTLE_ENDIAN);
bits = value ? bits | (1 << 21) : bits & ~(1 << 21);
buffer.putInt(offset, bits, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/VarDataUTF8Decoder.java
+87
View File
@@ -0,0 +1,87 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.DirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.VarDataUTF8Decoder"})
@SuppressWarnings("all")
public class VarDataUTF8Decoder {
public static final int ENCODED_LENGTH = -1;
private DirectBuffer buffer;
private int offset;
public VarDataUTF8Decoder wrap(final DirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public static long lengthNullValue() {
return 4294967294L;
}
public static long lengthMinValue() {
return 0L;
}
public static long lengthMaxValue() {
return 1073741824L;
}
public long length() {
return (buffer.getInt(offset + 0, java.nio.ByteOrder.LITTLE_ENDIAN) & 0xFFFF_FFFFL);
}
public static short varDataNullValue() {
return (short) 255;
}
public static short varDataMinValue() {
return (short) 0;
}
public static short varDataMaxValue() {
return (short) 254;
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
builder.append('(');
//Token{signal=ENCODING, name='length', description='null', id=-1, version=0, encodedLength=4, offset=0, componentTokenCount=1, encoding=Encoding{presence=REQUIRED, primitiveType=UINT32, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=1073741824, nullValue=null, constValue=null, characterEncoding='UTF-8', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append("length=");
builder.append(length());
builder.append('|');
//Token{signal=ENCODING, name='varData', description='null', id=-1, version=0, encodedLength=-1, offset=4, componentTokenCount=1, encoding=Encoding{presence=REQUIRED, primitiveType=UINT8, byteOrder=LITTLE_ENDIAN, minValue=null, maxValue=null, nullValue=null, constValue=null, characterEncoding='UTF-8', epoch='unix', timeUnit=nanosecond, semanticType='null'}}
builder.append(')');
return builder;
}
}
deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-model/src/main/java/org/deeplearning4j/ui/model/stats/sbe/VarDataUTF8Encoder.java
+83
View File
@@ -0,0 +1,83 @@
/*
* ******************************************************************************
* *
* *
* * This program and the accompanying materials are made available under the
* * terms of the Apache License, Version 2.0 which is available at
* * https://www.apache.org/licenses/LICENSE-2.0.
* *
* * See the NOTICE file distributed with this work for additional
* * information regarding copyright ownership.
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* * License for the specific language governing permissions and limitations
* * under the License.
* *
* * SPDX-License-Identifier: Apache-2.0
* *****************************************************************************
*/
package org.deeplearning4j.ui.model.stats.sbe;
import org.agrona.MutableDirectBuffer;
@javax.annotation.Generated(value = {"org.deeplearning4j.ui.stats.sbe.VarDataUTF8Encoder"})
@SuppressWarnings("all")
public class VarDataUTF8Encoder {
public static final int ENCODED_LENGTH = -1;
private MutableDirectBuffer buffer;
private int offset;
public VarDataUTF8Encoder wrap(final MutableDirectBuffer buffer, final int offset) {
this.buffer = buffer;
this.offset = offset;
return this;
}
public int encodedLength() {
return ENCODED_LENGTH;
}
public static long lengthNullValue() {
return 4294967294L;
}
public static long lengthMinValue() {
return 0L;
}
public static long lengthMaxValue() {
return 1073741824L;
}
public VarDataUTF8Encoder length(final long value) {
buffer.putInt(offset + 0, (int) value, java.nio.ByteOrder.LITTLE_ENDIAN);
return this;
}
public static short varDataNullValue() {
return (short) 255;
}
public static short varDataMinValue() {
return (short) 0;
}
public static short varDataMaxValue() {
return (short) 254;
}
public String toString() {
return appendTo(new StringBuilder(100)).toString();
}
public StringBuilder appendTo(final StringBuilder builder) {
VarDataUTF8Decoder writer = new VarDataUTF8Decoder();
writer.wrap(buffer, offset);
return writer.appendTo(builder);
}
}

Some files were not shown because too many files have changed in this diff Show More