wehub-resource-sync
502 lines
16 KiB
Plaintext
502 lines
16 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "g_nWetWWd_ns",
|
|
"metadata": {
|
|
"id": "g_nWetWWd_ns"
|
|
},
|
|
"source": [
|
|
"##### Copyright 2024 The AI Edge Authors."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2pHVBk_seED1",
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "2pHVBk_seED1"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
|
"# you may not use this file except in compliance with the License.\n",
|
|
"# You may obtain a copy of the License at\n",
|
|
"#\n",
|
|
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
|
"#\n",
|
|
"# Unless required by applicable law or agreed to in writing, software\n",
|
|
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
|
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
|
"# See the License for the specific language governing permissions and\n",
|
|
"# limitations under the License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "M7vSdG6sAIQn",
|
|
"metadata": {
|
|
"id": "M7vSdG6sAIQn"
|
|
},
|
|
"source": [
|
|
"# Signatures in TensorFlow Lite"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fwc5GKHBASdc",
|
|
"metadata": {
|
|
"id": "fwc5GKHBASdc"
|
|
},
|
|
"source": [
|
|
"<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
|
|
" <td>\n",
|
|
" <a target=\"_blank\" href=\"https://www.tensorflow.org/lite/guide/signatures\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
|
|
" </td>\n",
|
|
" <td>\n",
|
|
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/guide/signatures.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
|
|
" </td>\n",
|
|
" <td>\n",
|
|
" <a target=\"_blank\" href=\"https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/guide/signatures.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
|
|
" </td>\n",
|
|
" <td>\n",
|
|
" <a href=\"https://storage.googleapis.com/tensorflow_docs/tensorflow/tensorflow/lite/g3doc/guide/signatures.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
|
|
" </td>\n",
|
|
"</table>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9ee074e4",
|
|
"metadata": {
|
|
"id": "9ee074e4"
|
|
},
|
|
"source": [
|
|
"TensorFlow Lite supports converting TensorFlow model's input/output\n",
|
|
"specifications to TensorFlow Lite models. The input/output specifications are\n",
|
|
"called \"signatures\". Signatures can be specified when building a SavedModel or\n",
|
|
"creating concrete functions.\n",
|
|
"\n",
|
|
"Signatures in TensorFlow Lite provide the following features:\n",
|
|
"\n",
|
|
"* They specify inputs and outputs of the converted TensorFlow Lite model by\n",
|
|
" respecting the TensorFlow model's signatures.\n",
|
|
"* Allow a single TensorFlow Lite model to support multiple entry points.\n",
|
|
"\n",
|
|
"The signature is composed of three pieces:\n",
|
|
"\n",
|
|
"* Inputs: Map for inputs from input name in the signature to an input tensor.\n",
|
|
"* Outputs: Map for output mapping from output name in signature to an output\n",
|
|
" tensor.\n",
|
|
"* Signature Key: Name that identifies an entry point of the graph.\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "UaWdLA3fQDK2",
|
|
"metadata": {
|
|
"id": "UaWdLA3fQDK2"
|
|
},
|
|
"source": [
|
|
"## Setup"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9j4MGqyKQEo4",
|
|
"metadata": {
|
|
"id": "9j4MGqyKQEo4"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import tensorflow as tf"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "FN2N6hPEP-Ay",
|
|
"metadata": {
|
|
"id": "FN2N6hPEP-Ay"
|
|
},
|
|
"source": [
|
|
"## Example model\n",
|
|
"\n",
|
|
"Let's say we have two tasks, e.g., encoding and decoding, as a TensorFlow model:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "d8577c80",
|
|
"metadata": {
|
|
"id": "d8577c80"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"class Model(tf.Module):\n",
|
|
"\n",
|
|
" @tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.float32)])\n",
|
|
" def encode(self, x):\n",
|
|
" result = tf.strings.as_string(x)\n",
|
|
" return {\n",
|
|
" \"encoded_result\": result\n",
|
|
" }\n",
|
|
"\n",
|
|
" @tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.string)])\n",
|
|
" def decode(self, x):\n",
|
|
" result = tf.strings.to_number(x)\n",
|
|
" return {\n",
|
|
" \"decoded_result\": result\n",
|
|
" }"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9c814c6e",
|
|
"metadata": {
|
|
"id": "9c814c6e"
|
|
},
|
|
"source": [
|
|
"In the signature wise, the above TensorFlow model can be summarized as follows:\n",
|
|
"\n",
|
|
"* Signature\n",
|
|
"\n",
|
|
" - Key: encode\n",
|
|
" - Inputs: {\"x\"}\n",
|
|
" - Output: {\"encoded_result\"}\n",
|
|
"\n",
|
|
"* Signature\n",
|
|
"\n",
|
|
" - Key: decode\n",
|
|
" - Inputs: {\"x\"}\n",
|
|
" - Output: {\"decoded_result\"}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c4099f20",
|
|
"metadata": {
|
|
"id": "c4099f20"
|
|
},
|
|
"source": [
|
|
"## Convert a model with Signatures\n",
|
|
"\n",
|
|
"TensorFlow Lite converter APIs will bring the above signature information into\n",
|
|
"the converted TensorFlow Lite model.\n",
|
|
"\n",
|
|
"This conversion functionality is available on all the converter APIs starting\n",
|
|
"from TensorFlow version 2.7.0. See example usages.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "Qv0WwFQkQgnO",
|
|
"metadata": {
|
|
"id": "Qv0WwFQkQgnO"
|
|
},
|
|
"source": [
|
|
"\n",
|
|
"### From Saved Model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "96c8fc79",
|
|
"metadata": {
|
|
"id": "96c8fc79"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"model = Model()\n",
|
|
"\n",
|
|
"# Save the model\n",
|
|
"SAVED_MODEL_PATH = 'content/saved_models/coding'\n",
|
|
"\n",
|
|
"tf.saved_model.save(\n",
|
|
" model, SAVED_MODEL_PATH,\n",
|
|
" signatures={\n",
|
|
" 'encode': model.encode.get_concrete_function(),\n",
|
|
" 'decode': model.decode.get_concrete_function()\n",
|
|
" })\n",
|
|
"\n",
|
|
"# Convert the saved model using TFLiteConverter\n",
|
|
"converter = tf.lite.TFLiteConverter.from_saved_model(SAVED_MODEL_PATH)\n",
|
|
"converter.target_spec.supported_ops = [\n",
|
|
" tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n",
|
|
" tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n",
|
|
"]\n",
|
|
"tflite_model = converter.convert()\n",
|
|
"\n",
|
|
"# Print the signatures from the converted model\n",
|
|
"interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
|
|
"signatures = interpreter.get_signature_list()\n",
|
|
"print(signatures)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5baa9f17",
|
|
"metadata": {
|
|
"id": "5baa9f17"
|
|
},
|
|
"source": [
|
|
"### From Keras Model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "71f29229",
|
|
"metadata": {
|
|
"id": "71f29229"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Generate a Keras model.\n",
|
|
"keras_model = tf.keras.Sequential(\n",
|
|
" [\n",
|
|
" tf.keras.layers.Dense(2, input_dim=4, activation='relu', name='x'),\n",
|
|
" tf.keras.layers.Dense(1, activation='relu', name='output'),\n",
|
|
" ]\n",
|
|
")\n",
|
|
"\n",
|
|
"# Convert the keras model using TFLiteConverter.\n",
|
|
"# Keras model converter API uses the default signature automatically.\n",
|
|
"converter = tf.lite.TFLiteConverter.from_keras_model(keras_model)\n",
|
|
"tflite_model = converter.convert()\n",
|
|
"\n",
|
|
"# Print the signatures from the converted model\n",
|
|
"interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
|
|
"\n",
|
|
"signatures = interpreter.get_signature_list()\n",
|
|
"print(signatures)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e4d30f85",
|
|
"metadata": {
|
|
"id": "e4d30f85"
|
|
},
|
|
"source": [
|
|
"### From Concrete Functions"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "c9e8a742",
|
|
"metadata": {
|
|
"id": "c9e8a742"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"model = Model()\n",
|
|
"\n",
|
|
"# Convert the concrete functions using TFLiteConverter\n",
|
|
"converter = tf.lite.TFLiteConverter.from_concrete_functions(\n",
|
|
" [model.encode.get_concrete_function(),\n",
|
|
" model.decode.get_concrete_function()], model)\n",
|
|
"converter.target_spec.supported_ops = [\n",
|
|
" tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n",
|
|
" tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n",
|
|
"]\n",
|
|
"tflite_model = converter.convert()\n",
|
|
"\n",
|
|
"# Print the signatures from the converted model\n",
|
|
"interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
|
|
"signatures = interpreter.get_signature_list()\n",
|
|
"print(signatures)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b5e85934",
|
|
"metadata": {
|
|
"id": "b5e85934"
|
|
},
|
|
"source": [
|
|
"## Run Signatures\n",
|
|
"\n",
|
|
"TensorFlow inference APIs support the signature-based executions:\n",
|
|
"\n",
|
|
"* Accessing the input/output tensors through the names of the inputs and\n",
|
|
" outputs, specified by the signature.\n",
|
|
"* Running each entry point of the graph separately, identified by the\n",
|
|
" signature key.\n",
|
|
"* Support for the SavedModel's initialization procedure.\n",
|
|
"\n",
|
|
"Java, C++ and Python language bindings are currently available. See example the\n",
|
|
"below sections.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ZRBMFciMQmiB",
|
|
"metadata": {
|
|
"id": "ZRBMFciMQmiB"
|
|
},
|
|
"source": [
|
|
"\n",
|
|
"### Java"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "04c5a4fc",
|
|
"metadata": {
|
|
"id": "04c5a4fc"
|
|
},
|
|
"source": [
|
|
"```\n",
|
|
"try (Interpreter interpreter = new Interpreter(file_of_tensorflowlite_model)) {\n",
|
|
" // Run encoding signature.\n",
|
|
" Map<String, Object> inputs = new HashMap<>();\n",
|
|
" inputs.put(\"x\", input);\n",
|
|
" Map<String, Object> outputs = new HashMap<>();\n",
|
|
" outputs.put(\"encoded_result\", encoded_result);\n",
|
|
" interpreter.runSignature(inputs, outputs, \"encode\");\n",
|
|
"\n",
|
|
" // Run decoding signature.\n",
|
|
" Map<String, Object> inputs = new HashMap<>();\n",
|
|
" inputs.put(\"x\", encoded_result);\n",
|
|
" Map<String, Object> outputs = new HashMap<>();\n",
|
|
" outputs.put(\"decoded_result\", decoded_result);\n",
|
|
" interpreter.runSignature(inputs, outputs, \"decode\");\n",
|
|
"}\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5ba86c64",
|
|
"metadata": {
|
|
"id": "5ba86c64"
|
|
},
|
|
"source": [
|
|
"### C++"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "397ad6fd",
|
|
"metadata": {
|
|
"id": "397ad6fd"
|
|
},
|
|
"source": [
|
|
"```\n",
|
|
"SignatureRunner* encode_runner =\n",
|
|
" interpreter->GetSignatureRunner(\"encode\");\n",
|
|
"encode_runner->ResizeInputTensor(\"x\", {100});\n",
|
|
"encode_runner->AllocateTensors();\n",
|
|
"\n",
|
|
"TfLiteTensor* input_tensor = encode_runner->input_tensor(\"x\");\n",
|
|
"float* input = GetTensorData<float>(input_tensor);\n",
|
|
"// Fill `input`.\n",
|
|
"\n",
|
|
"encode_runner->Invoke();\n",
|
|
"\n",
|
|
"const TfLiteTensor* output_tensor = encode_runner->output_tensor(\n",
|
|
" \"encoded_result\");\n",
|
|
"float* output = GetTensorData<float>(output_tensor);\n",
|
|
"// Access `output`.\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0f4c6ad4",
|
|
"metadata": {
|
|
"id": "0f4c6ad4"
|
|
},
|
|
"source": [
|
|
"### Python"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ab7b1963",
|
|
"metadata": {
|
|
"id": "ab7b1963"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Load the TFLite model in TFLite Interpreter\n",
|
|
"interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
|
|
"\n",
|
|
"# Print the signatures from the converted model\n",
|
|
"signatures = interpreter.get_signature_list()\n",
|
|
"print('Signature:', signatures)\n",
|
|
"\n",
|
|
"# encode and decode are callable with input as arguments.\n",
|
|
"encode = interpreter.get_signature_runner('encode')\n",
|
|
"decode = interpreter.get_signature_runner('decode')\n",
|
|
"\n",
|
|
"# 'encoded' and 'decoded' are dictionaries with all outputs from the inference.\n",
|
|
"input = tf.constant([1, 2, 3], dtype=tf.float32)\n",
|
|
"print('Input:', input)\n",
|
|
"encoded = encode(x=input)\n",
|
|
"print('Encoded result:', encoded)\n",
|
|
"decoded = decode(x=encoded['encoded_result'])\n",
|
|
"print('Decoded result:', decoded)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "81b42e5b",
|
|
"metadata": {
|
|
"id": "81b42e5b"
|
|
},
|
|
"source": [
|
|
"## Known limitations\n",
|
|
"\n",
|
|
"* As TFLite interpreter does not gurantee thread safety, the signature runners\n",
|
|
" from the same interpreter won't be executed concurrently.\n",
|
|
"* Support for iOS/Swift is not available yet.\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3032Iof6QqmJ",
|
|
"metadata": {
|
|
"id": "3032Iof6QqmJ"
|
|
},
|
|
"source": [
|
|
"## Updates\n",
|
|
"\n",
|
|
"* Version 2.7\n",
|
|
" - The multiple signature feature is implemented.\n",
|
|
" - All the converter APIs from version two generate signature-enabled\n",
|
|
" TensorFlow Lite models.\n",
|
|
"* Version 2.5\n",
|
|
" - Signature feature is available through the `from_saved_model` converter\n",
|
|
" API."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"collapsed_sections": [],
|
|
"name": "Signatures in TensorFlow Lite",
|
|
"provenance": [],
|
|
"toc_visible": true
|
|
},
|
|
"id": "a1b42e5b",
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"name": "python"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|