chore: import upstream snapshot with attribution

codegen/op-codegen/adr/0001-kotlin_dsl_as_source_of_truth.md

@@ -0,0 +1,49 @@
+# Use Kotlin-based DSL as the source of truth 
+
+## Status
+
+ACCEPTED
+
+Discussed by: Paul Dubs, Alex Black, raver119 on 19. October 2019
+
+
+## Context
+
+This code generation experiment is meant to be our starting point for both the API unification for ND4J and SameDiff, 
+and the multi-language support. For this reason we have to define ops, or their interface, in a language neutral way.
+
+The initial idea was to use a Language Workbench like MPS. This had to be discarded because of bugs and limitations
+encountered while trying to define a language that would work for a few simple examples.
+
+The next idea was to use Ops defined in JSON files. This would have allowed us to define Ops as human readable data and
+read and write those files from any programming language. However, the drawback with this approach is that writing json
+manually invites many problems if written manually (e.g. typos, bad structuring, having to look up the proper keys,...).
+In order to rectify that drawback, we would have to create custom tooling, that we would have to maintain and that
+contributors would have to use. 
+
+Using a Java builder pattern based approach is very verbose. 
+
+## Decision
+
+We use a Kotlin-based DSL to define Ops.
+
+## Consequences
+
+Using a full programming language as the platform to build our DSL has both advantages and drawbacks.
+
+### Drawbacks
+* Contributors will have to install a JVM in order to be able to run code generation or get a serialized op graph
+* Serialization is a one way road, we only output to JSON, but don't read from it
+* Contributors to Op definitions have to learn about our DSL and maybe some Kotlin
+* Contributors to the DSL will have to learn about Kotlin 
+
+
+### Advantages
+* We can utilize the Java knowledge of the existing team for writing code generators as Kotlin is two-way interoperable
+  with Java and other JVM languages. 
+* We can utilize IntelliJ (or other IDEs supporting Kotlin) as an existing editor for Ops definitions. This provides us 
+  with benefits like code completion, error highlighting
+* We get compile time checks, freeing us from trivial errors like typos
+* We can utilize some base features of Kotlin, like variable assignment to simplify the implementation
+* Kotlin has first class DSL definition support, allowing us to make Op definitions almost as easy to read as a full
+  language workbench would have allowed

codegen/op-codegen/adr/0002-separate_object_graph_for_serialization.md

@@ -0,0 +1,40 @@
+# Splitting Object Graph for Code Gen and Serialization
+
+## Status
+
+ACCEPTED
+
+Discussed by: Paul Dubs & Alex Black on 07. November 2019
+
+
+## Context
+
+Serialization and Code Generation have different needs when it comes to how the object graph should be laid out. When
+generating code, it is a lot easier to be able to directly access referenced objects and traverse through the graph.
+However, if the same object graph is used for serialization, the same object will appear in multiple places.
+
+This becomes very apparent when defining constraints. A single constraint might be referring to an input multiple times,
+and then there can also be multiple constraints that refer to multiple inputs.
+
+The main reason why we want to keep serialization in mind, is that we want to keep code generators in other languages as
+a viable option. Just serializing the graph that is meant to be used during runtime in code generation however, would
+can easily become a problem when object identity is required for equality comparisons. 
+
+An implementer in a different language would therefore need work through that graph and find identical objects AND would
+have to know where that identity is a coincidence and where it is meant to be that way. By creating an object graph that
+makes this explicit, we make that work easier.
+
+## Decision
+
+We use two distinct object graphs. One is used for code generation, and the other is used for serialization.  
+
+
+## Consequences
+
+### Advantages
+* Easier to work with object graphs that are aligned with their use-case
+* Less error prone when access is direct
+
+### Disadvantages
+* We have to explicitly transform one object graph into another
+* If we want to support reading JSON back, we will have to also define the backwards transformation

codegen/op-codegen/adr/0003-dealing_with_inconsistencies_in_java_naming.md

@@ -0,0 +1,33 @@
+# Dealing with Inconsistencies in Java Naming
+
+## Status
+
+ACCEPTED
+
+Discussed by: Paul Dubs, Alex Black, raver119 on 22. November 2019
+
+
+## Context
+
+There are slight inconsistencies in naming between existing op class definitions and factory methods. For example a 
+factory method called `bernoulli` in the `random` namespace with a corresponding op class called 
+`BernoulliDistribution`.
+
+Two possible solutions where suggested:
+1. Add an additional property that provides us with the correct class name
+2. Rename classes in ND4J to ensure consistency and provide backwards compatibility via deprecated subclasses   
+
+## Decision
+
+For now we will introduce a `javaOpClass` property which in cases of inconsistency provides us with the correct class
+name.
+
+## Consequences
+
+### Advantages
+* We can start using this property immediately
+* No need to change anything of the existing ND4J / SameDiff API
+
+### Disadvantages
+* Inconsistency continues to exist within the Java codebase
+* We have to take extra care to add the new property where needed

codegen/op-codegen/adr/0004-auto_initialization_for_inplace_operations.md

@@ -0,0 +1,44 @@
+# Auto initialization for in-place operations
+
+## Status
+
+REJECTED
+
+Discussed by: Paul Dubs, Alex Black, raver119 on 25. November 2019
+
+
+## Context
+
+Some operations work in-place on the inputs that they are given in ND4J, but in SameDiff the same operations will
+generate an array from a given shape. Examples for this include `BernoulliDistribution`, and other random ops, that
+effectively initialize the array that they are given.
+
+From a consistency point of view, it would be nice if both API's would support both ways of using those ops.
+
+
+## Decision
+
+We introduce an option to mark inputs as `inPlace = true` to make it clear that this input is going to be changed
+in-place. In addition we introduce an option `supportsInPlaceInit = true` to mark an input as initialize-able. If the
+`supportsInPlaceInit` option is enabled, two signatures for the Op will be created, one that takes an input, and one
+that takes the appropriate shape and data type information in its stead.  
+
+
+## Consequences
+
+### Advantages
+* We get support for both in-place and initialization use-cases
+* Support for this use-case is explicitly defined in the DSL instead of implicit support by the code generator
+
+### Disadvantages
+* Codegen becomes more complex, as it requires us to generate more signatures
+
+
+## Reasons for Rejection
+This feature would only come in handy on legacy ops. However, those ops will not be exposed to frontend languages 
+anymore starting from the next release. 
+
+For the ops that replace them ("Custom Ops"), it is *always* possible to pass an output array, when used in NDArray
+programming mode. Thereby making the in-place initialization support a side-effect of that general design.
+
+For support of both `op(out)` and `op(dataType, shape)` see ADR 0005 "Optional parameters and signatures".

codegen/op-codegen/adr/0005-optional_parameters_and_signatures.md

@@ -0,0 +1,108 @@
+# Optional parameters and signatures
+
+## Status
+
+ACCEPTED
+
+Discussed by: Alex Black, raver 119 and Paul Dubs on 25. November 2019
+
+## Context
+
+Not all inputs or args (= parameters) are always required. 
+
+Often there are sensible defaults available. We want to be able to make those defaults explicit where possible.
+
+Even though some parameters may be optional, they might become required in the presence of other optional parameters.
+
+We need a way to explicitly define what combinations are possible.
+
+
+## Decision
+We drop the `optional` property on parameters. Instead, parameters get an additional property `defaultValue`. It can be 
+set to either a fixed literal value (e.g. `7`, `"something"`, `null`), an Arg, or it may reference the specific methods 
+`shape()` and `dataType()` on inputs and outputs. Parameters with `defaultValue` specified are treated as optional.
+
+To be able to deal with languages that do not support default values for arguments, Signatures will be specified.
+Signatures are specified using a `Signature(a,b,c){ "signature specific documentation" }` section for each signature.
+With the signature specific documentation being optional. 
+
+Signatures making use of outputs will only be generated for NDArray programming mode, not in SameDiff mode. This also
+means that parameters with a `defaultValue` based on an output will be treated as required in SameDiff mode.
+
+If signatures are specified, only the specified signatures will be generated.
+
+If no signatures are explicitly specified, only the "all-arg" and "no-optional-arg" signatures will be generated. In
+NDArray programming mode, the default signatures also include a variant that includes the output.
+
+
+## Examples
+### BatchNorm with all otherwise auto generated signatures stated explicitly
+ 
+```kotlin
+Op("batchNorm") {
+    val input    = Input(NUMERIC, "input") { description = "Input variable" }
+    val mean     = Input(NUMERIC, "mean") { description = "Mean value. For 1d axis, this should match input.size(axis)" }
+    val variance = Input(NUMERIC, "variance") { description = "Variance value. For 1d axis, this should match input.size(axis)" }
+    val gamma    = Input(NUMERIC, "gamma") { description = "Gamma value. For 1d axis, this should match input.size(axis)" }
+    val beta     = Input(NUMERIC, "beta") { description = "Beta value. For 1d axis, this should match input.size(axis)" }
+
+    val applyGamma = Arg(BOOL, "applyGamma") { description = ""; defaultValue = true}
+    val applyBeta  = Arg(BOOL, "applyBeta") { description = ""; defaultValue = true}
+    val axis       = Arg(INT, "axis"){
+        count = AtLeast(1) 
+        description = """
+        For 2d CNN activations: 1 for NCHW format activations, or 3 for NHWC format activations.
+        For 3d CNN activations: 1 for NCDHW format, 4 for NDHWC
+        For 1d/RNN activations: 1 for NCW format, 2 for NWC
+        """.trimIndent()
+    }
+
+    val out = Output(INT, "output"){ description = "Output variable for batch normalization" }
+
+    Doc(Language.ANY, DocScope.ALL){
+        """
+        Neural network batch normalization operation.
+        For details, see <a href="https://arxiv.org/abs/1502.03167">https://arxiv.org/abs/1502.03167</a>
+        """.trimIndent()
+    }
+  
+    Signature(input, mean, variance, gamma, beta, axis)
+    Signature(input, mean, variance, gamma, beta, applyGamma, applyBeta, axis)
+    Signature(out, input, mean, variance, gamma, beta, axis)
+    Signature(out, input, mean, variance, gamma, beta, applyGamma, applyBeta, axis)
+}
+```
+
+### Random Uniform initialization with support for (dataType, shape) and (out) invocation
+```kotlin
+Op("uniform") {
+    val out = Output(NUMERIC, "output") { description = "new random %INPUT_TYPE%, where values are randomly sampled according to a uniform distribution" }    
+    
+    val min = Arg(FLOATING_POINT, "min") { description = "Minimum value" }
+    val max = Arg(FLOATING_POINT, "max") { description = "Maximum value." }
+    val dataType = Arg(DATA_TYPE, "dataType") { description = "Data Type of the output array"; defaultValue = out.dataType() }
+    val shape = Arg(INT, "shape") { count = AtLeast(1); description = "Shape of the new random %INPUT_TYPE%, as a 1D array"; defaultValue = out.dataType() }
+
+    Doc(Language.ANY, DocScope.ALL) {
+        """
+        Generate a new random %INPUT_TYPE%, where values are randomly sampled according to a uniform distribution,
+        U(min,max)
+        """.trimIndent()
+    }
+
+    Signature(min, max, dataType, shape)
+    Signature(out, min, max)
+}
+```
+
+
+## Consequences
+
+### Advantages
+* We get to explicitly define edge cases
+* We can make Signatures compatible with existing code
+* Even in languages with default value support, the added signatures may become useful parts of the documentation
+
+### Disadvantages
+* The order of definitions within the op changes to Output first, if inputs need to reference it
+

codegen/op-codegen/adr/0006-op_specific_enums.md

@@ -0,0 +1,40 @@
+# Op specific enums
+
+## Status
+
+ACCEPTED
+
+Discussed by: Alex Black, Robert Altena and Paul Dubs on 26. November 2019
+
+## Context
+Some ops have an ordinal parameter which switches between a few possible modes. Giving those modes a proper name
+makes usage and documentation easier. 
+
+
+## Decision
+We allow `Arg` sections to have an `ENUM` data type and add a `possibleValues` property to define the possible values
+for this arg. The ordinal number of the enum is the same as its position within the `possibleValues` list starting from 
+`0`. 
+
+A runtime check on op construction, will ensure that each enum arg has one or more possible values, and that default
+values match one of the possible values (if applicable).
+
+On code generation, an appropriate representation of this enum will be generated in the target language. The name of 
+the generated enum will be derived from the name of the arg.
+
+### Example
+```kotlin
+Arg(ENUM, "padMode"){ 
+  possibleValues = listOf("CONSTANT", "REFLECT", "SYMMETRIC")
+  description = "padding mode"  
+}
+```
+
+## Consequences
+
+### Advantages
+* We get easily understandable names for otherwise in-transparent ordinal mode modifiers
+
+### Disadvantages
+* The defined enum can only be used for a single op
+* The defined enum is only usable with a single arg

codegen/op-codegen/adr/0007-configuration_objects.md

@@ -0,0 +1,81 @@
+# Configuration Objects
+
+## Status
+
+ACCEPTED
+
+Discussed by Alex Black, raver119 and Paul Dubs on 27. November 2019.
+
+## Context
+Some Ops (esp. convolution) have many parameters. Many of them can have reasonable defaults, but even then creating
+signatures for evey reasonable configuration may be impossible, as those signatures would require different naming in
+order to be actually distinguishable from each other.
+
+In other cases, an op may have a lot of same typed parameters that are required (e.g. GRU, LSTM, SRU) but it is very
+easy to mix them up. 
+
+For both of those cases (many optional parameters, easily mixed up required parameters) it is reasonable to use a
+config holder with builder pattern in languages that do not support named or default parameters. 
+
+In our current codebase those configurations are often used across several related ops. 
+
+
+## Decision
+We add a `Config("name"){ ... }` section to the namespace context. It supports `Input` and `Arg` definitions in the same
+way that `Op` does.
+
+Ops that want to use that config can use `useConfig(conf)`. As configs are often reused across related objects, this
+will have the effect of a mixin: All inputs and args defined in that config will also be automatically defined on that
+Op. If there is a naming conflict, an exception will be thrown at construction time.
+
+For default signatures, configs will be passed at the end, in the order that they were added to the Op.
+
+If other signatures are desired, configs, like regular inputs and args, can be passed to `Signature`.
+
+In languages that do not support default or named parameters, a config holder will be created, that will take the
+parameters of the config using a builder pattern. For languages with default and named parameters, no additional config
+holder will be created, and the parameters of the config will be treated as if they were directly configured on the Op.
+
+### Example
+This example shows a very simple case in order to highlight how this feature would be used. 
+```kotlin
+fun RNN() = Namespace("RNN"){
+    val sruWeights = Config("SRUWeights"){
+        Input(FLOATING_POINT, "weights"){ description = "Weights, with shape [inSize, 3*inSize]" }
+        Input(FLOATING_POINT, "bias"){ description = "Biases, with shape [2*inSize]" }
+    }
+
+    Op("SRU"){
+        Input(FLOATING_POINT, "x"){ description = "..." }
+        Input(FLOATING_POINT, "initialC"){ description = "..." }
+        Input(FLOATING_POINT, "mask"){ description = "..." }
+        
+        useConfig(sruWeights)
+    
+        Output(FLOATING_POINT, "out"){ description = "..." }
+    }
+    
+    Op("SRUCell"){
+        val x = Input(FLOATING_POINT, "x"){ description = "..." }
+        val cLast = Input(FLOATING_POINT, "cLast"){ description = "..." }
+        
+        val conf = useConfig(sruWeights)
+    
+        Output(FLOATING_POINT, "out"){ description = "..." }
+    
+        // Just for demonstration purposes
+        Signature(x, cLast, conf)
+    }
+}
+```
+ 
+## Consequences
+
+### Advantages
+* Ops that share parameters can make that sharing explicit
+* Easier definition of related ops with common parameters
+* Simplifies usage of complex ops in languages without named parameters
+
+### Disadvantages
+* Not all parameters are defined directly within an op anymore
+* Configs are not shareable across namespaces

codegen/op-codegen/adr/0008-inheritance.md

@@ -0,0 +1,89 @@
+# Inheritance 
+
+## Status
+ACCEPTED
+
+Discussed by Alex Black and Paul Dubs on 29. November 2019 and 2. December 2019. 
+
+## Context
+In many cases ops have a similar interface. For example all transform ops take a single input, but some of them take
+additional arguments; all pairwise ops take two inputs, and so on. The documentation of those ops is often the result
+of copy & paste with just a few little modifications, and changing anything later on suddenly becomes a huge undertaking
+because what should effectively be a change in a single place, has to be changed in many places.
+
+Another issue that copy & paste based definitions bring to the table is that this practice effectively makes any
+relationship between those ops implicit.
+
+When defining ops with the DSL we prefer to make things as explicit as possible, while also reducing repetition and
+boilerplate.
+
+The existing inheritance mechanism, added without a formal proposal at the beginning of the project, allows the
+definition of an abstract base op. The new op based on it, can copy any of the given parts before it gets to define its
+own properties. This approach has the problem that we can't inherit from multiple base ops, and that we do not get any
+direct access to its fields for ease of use.
+
+# Decision
+We introduce an explicit mixin mechanism `Mixin("name") {...}` which can define any parts of any op, but isn't an Op
+definition on its own. Mixins can be defined at top-level, thereby being usable across namespaces.
+
+A mixin is mixed into an op with `useMixin(mixinReference, ...options...)` within an op context. It will add all (if 
+not otherwise configured) definitions of the mixin to the current op as if they were copied into its place. 
+If `useMixin(ref)` is used as the first thing within an op definition, then it will behave exactly like the old 
+inheritance mechanism.
+
+`useMixin(ref)` returns a holder object, that can be used to reference its parameters.
+
+The available options on `useMixin` are `keepInputs`, `keepArgs`, `keepOutputs`, `keepSignatures`, `keepDoc`,
+`keepConstraints`. They default to `true`.
+
+If there is a naming conflict between mixins or between mixin and op definition, the last definition wins.
+
+### Example
+```kotlin
+
+val indexAccum = Mixin("indexAccum"){
+    legacy = true
+    javaPackage = "org.nd4j.linalg.api.ops.impl.indexaccum"
+    val input = Input(NUMERIC, "in") { description = "Input variable" }
+    val keepDims = Arg(BOOL, "keepDims") { description = "If true: keep the dimensions that are reduced on (as length 1). False: remove the reduction dimensions"; defaultValue = false }
+    val dims = Arg(INT, "dimensions"){ count = AtLeast(1); description = "Dimensions to reduce over. If dimensions are not specified, full array reduction is performed" }
+    Output(NUMERIC, "output"){ description = "Reduced array of rank (input rank - num dimensions)" }
+
+    Signature(input, dims)
+    AllParamSignature(withOutput = false)
+}
+
+
+Namespace("math"){
+    Op("firstIndex") {
+        val idxAccum = useMixin(indexAccum, keepSignatures=false)
+        var c = Arg(CONDITION, "condition") { description = "Condition to check on input variable" }
+        Signature(idxAccum.input("in"), c, idxAccum.arg("dimensions"))
+        Signature(idxAccum.input("in"), c, idxAccum.arg("keepDims"), idxAccum.arg("dimensions"))
+
+        Doc(Language.ANY, DocScope.ALL){
+            """
+                First index reduction operation.
+                Returns a variable that contains the index of the first element that matches the specified condition (for each
+                slice along the specified dimensions)
+                Note that if keepDims = true, the output variable has the same rank as the input variable,
+                with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting
+                the mean along a dimension).
+                Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:
+                keepDims = true: [a,1,c]
+                keepDims = false: [a,c]
+            """.trimIndent()
+        }
+    }
+}
+```
+ 
+  
+## Consequences
+### Advantages
+* We can have multiple inheritance 
+* We can share op similarities across namespaces
+* We get explicit access to parameters defined in mixins
+
+### Disadvantages
+* We have to adapt our current usage

codegen/op-codegen/adr/0009-aliasing.md

@@ -0,0 +1,72 @@
+# Aliasing
+
+## Status
+ACCEPTED
+
+Discussed by Alex Black and Paul Dubs on 05. March 2020. 
+ 
+
+## Context
+The API surface area is very large over all those namespaces. For consistency with our previous manually created API we want to be able to alias ops into different namespaces. Those aliases are meant as a convenience for users, so they can find ops more easily. 
+
+# Decision
+We introduce an aliasing mechanism `Alias(NamespaceName().op("opName"))` at the Namespace level, which can reference an op directly.
+
+When an op is aliased like that, all of that ops signatures will be available in the referencing namespace. However, they will get an additional note in their documentation saying that it is an alias to the original op. In addition, the implementation of an alias signature, is a direct call of the same signature in the original namespace.  
+
+It is not allowed to alias an op from a namespace that only has it because it has aliased it itself. 
+
+When the requested op is not part of the given namespace, trying to alias it will throw an OpNotFoundException. 
+
+### Example
+#### Definitions
+```kotlin
+fun BaseOps() = Namespace("BaseOps"){
+    Op("mmul") {
+        javaPackage = "org.nd4j.linalg.api.ops.impl.reduce"
+        Input(NUMERIC, "x") { description = "First input variable" }
+        Input(NUMERIC, "y") { description = "Second input variable" }
+        Arg(BOOL, "transposeX") { description = "Transpose x (first argument)"; defaultValue=false }
+        Arg(BOOL, "transposeY") { description = "Transpose y (second argument)"; defaultValue=false }
+        Arg(BOOL, "transposeZ") { description = "Transpose result array"; defaultValue=false }
+        Output(NUMERIC, "output"){ description = "" }
+        Doc(Language.ANY, DocScope.ALL){
+            """
+                Matrix multiplication: out = mmul(x,y)
+                Supports specifying transpose argument to perform operation such as mmul(a^T, b), etc.
+            """.trimIndent()
+        }
+    }
+}
+
+fun Linalg() = Namespace("Linalg"){
+    Alias(BaseOps(), "mmul")
+}
+```
+
+#### Output
+This output is meant as a possible example. It is not what it will look like exactly once this feature is implemented.
+
+```java
+public class Linalg{
+    /**
+     * Matrix multiplication: out = mmul(x,y)<br>
+     * Supports specifying transpose argument to perform operation such as mmul(a^T, b), etc. <br>
+     * 
+     * Alias of basepackage.baseOps.mmul(x, y)<br>
+     * 
+     * @param x First input variable
+     * @param y Second input variable
+     */
+    public static INDArray mmul(INDArray x, INDArray y){
+        return basepakage.baseOps.mmul(x,y);
+    }
+}
+```
+  
+## Consequences
+### Advantages
+* We can make the API more flexible by allowing ops to be available from other namespaces
+
+### Disadvantages
+* The API becomes more crowded

codegen/op-codegen/adr/0010-ir-codegen.md

@@ -0,0 +1,283 @@
+###Interpreter
+
+An interpreter takes  a tensorflow or pytorch model and figure out how to map
+various ops. Their attributes and op names are mapped to libnd4j
+using information from the above op descriptor.
+
+An interpreter can take in an individual op from tensorflow, onnx or
+another framework and translate it to an equivalent op in libnd4j represented
+as the equivalent op descriptor.
+
+The usage is as follows:
+
+
+## Op def files
+
+For each framework in tensorflow/onnx, we have inbuilt definition files
+for each tensorflow and pytorch.
+
+For onnx, we have an onnx.pbtxt generated by the dl4j-dev tools submodule 
+onnx-defs. This definition file has each op serialized as an [onnx NodeProto](https://github.com/onnx/onnx/blob/25fd2c332cf854fd38a92aa8f60d232530ab0065/onnx/onnx-ml.proto#L193)
+For tensorflow, we have an ops.proto pulled from tensorflow's official repo.
+
+We  use these files to map operation attributes serialized by 
+nd4j's generated operation definition tool found in dl4j-dev-tools
+to their equivalents in tensorflow and pytorch.
+
+An interpreter has 2 methods:
+
+
+```java
+Interpreter interpreter = ...;
+
+OpDescriptor descriptor = interpreter.interpretTensorflow(nodeFromOtherFramework);
+OpDescriptor descriptor = interpreter.interpretOnnx(nodeFromOtherFramework);
+
+//proceed to use descriptor to map for model import...
+```  
+
+##Interpreter file format
+
+An interpreter is language neutral. We have a mini syntax
+for mapping attributes from one format to another.
+
+Through indexing every attribute and input/output in libnd4j,
+we can maintain an index of operation names and attributes
+with a mapping syntax. If we want to map a trivial operation like say:
+Abs, let's compare tensorflow, onnx and the descriptor in nd4j.
+
+Tensorflow:
+```prototext
+op {
+  name: "Floor"
+  input_arg {
+    name: "x"
+    type_attr: "T"
+  }
+  output_arg {
+    name: "y"
+    type_attr: "T"
+  }
+  attr {
+    name: "T"
+    type: "type"
+    allowed_values {
+      list {
+        type: DT_BFLOAT16
+        type: DT_HALF
+        type: DT_FLOAT
+        type: DT_DOUBLE
+      }
+    }
+  }
+}
+```
+
+Onnx:
+```prototext
+input: "X"
+output: "Y"
+name: "Floor"
+op_type: "Floor"
+attribute {
+  name: "X-types"
+  strings: "float"
+  strings: "float16"
+  strings: "double"
+  type: STRINGS
+}
+doc_string: "\nFloor takes one input data (Tensor<T>) and produces one output data\n(Tensor<T>) where the floor is, y = floor(x), is applied to\nthe tensor elementwise.\n"
+```
+
+The op descriptor for libnd4j is:
+```
+OpDeclarationDescriptor(name=Floor, nIn=1, nOut=1, tArgs=0, iArgs=0, inplaceAble=true, inArgNames=[first], outArgNames=[z], tArgNames=[], iArgNames=[], bArgNames=[], opDeclarationType=OP_IMPL)
+```
+
+Floor is a fairly simple op with 1 input and 1 output.
+Inputs and outputs are implicitly tensors.
+This is true for both onnx and tensorflow.
+
+Tensorflow has an attribute defined for valid types.
+The way we generated the onnx schema proto, we have something equivalent
+that allows for a list of types presented as a string.
+
+
+Mapping a descriptor happens based on attribute.
+An example of abs below:
+
+```prototext
+floor {
+  tensorflow_mapping: {
+     input_mappings: {
+       input_mapping {
+        first: "x"
+     }
+         
+     }
+     output_mappings: {
+        z: "y"
+     }
+  
+      attribute_mapping_functions: {
+
+     }
+
+  }
+  onnx_mapping {
+    input_mappings {
+       first: "X"
+    }
+    output_mappings {
+        z: "Y"
+    }
+
+     attribute_mapping_functions {
+
+     }
+  }
+}
+```
+
+Now we can compare this to Convolution.
+In tensorflow, the convolution op is represented as:
+```prototext
+op {
+  name: "Conv2D"
+  input_arg {
+    name: "input"
+    type_attr: "T"
+  }
+  input_arg {
+    name: "filter"
+    type_attr: "T"
+  }
+  output_arg {
+    name: "output"
+    type_attr: "T"
+  }
+  attr {
+    name: "T"
+    type: "type"
+    allowed_values {
+      list {
+        type: DT_HALF
+        type: DT_BFLOAT16
+        type: DT_FLOAT
+        type: DT_DOUBLE
+      }
+    }
+  }
+  attr {
+    name: "strides"
+    type: "list(int)"
+  }
+  attr {
+    name: "use_cudnn_on_gpu"
+    type: "bool"
+    default_value {
+      b: true
+    }
+  }
+  attr {
+    name: "padding"
+    type: "string"
+    allowed_values {
+      list {
+        s: "SAME"
+        s: "VALID"
+      }
+    }
+  }
+  attr {
+    name: "data_format"
+    type: "string"
+    default_value {
+      s: "NHWC"
+    }
+    allowed_values {
+      list {
+        s: "NHWC"
+        s: "NCHW"
+      }
+    }
+  }
+  attr {
+    name: "dilations"
+    type: "list(int)"
+    default_value {
+      list {
+        i: 1
+        i: 1
+        i: 1
+        i: 1
+      }
+    }
+  }
+}
+```
+In onnx, it's represented as:
+```prototext
+input: "X"
+input: "W"
+input: "B"
+output: "Y"
+name: "Conv"
+op_type: "Conv"
+attribute {
+  name: "auto_pad"
+  s: "NOTSET"
+  type: STRING
+}
+attribute {
+  name: "dilations"
+  s: ""
+  type: INTS
+}
+attribute {
+  name: "group"
+  i: 1
+  type: INT
+}
+attribute {
+  name: "kernel_shape"
+  s: ""
+  type: INTS
+}
+attribute {
+  name: "pads"
+  s: ""
+  type: INTS
+}
+attribute {
+  name: "strides"
+  s: ""
+  type: INTS
+}
+attribute {
+  name: "X-types"
+  strings: "double"
+  strings: "float"
+  strings: "float16"
+  type: STRINGS
+}
+attribute {
+  name: "W-types"
+  strings: "double"
+  strings: "float"
+  strings: "float16"
+  type: STRINGS
+}
+attribute {
+  name: "B-types"
+  strings: "double"
+  strings: "float"
+  strings: "float16"
+  type: STRINGS
+}
+doc_string: "\nThe convolution operator consumes an input tensor and a filter, and\ncomputes the output."
+```
+
+
+
+