wehub/tensorflow--tensorflow
1
0
Fork 0
Code Issues Pull Requests Actions 3 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
cffconvert / validate (push) Has been skipped
Details
License Check / license-check (push) Failing after 2s
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:14:16 +08:00
commit 8a852e4b4e
36502 changed files with 9277225 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tensorflow/lite/examples/android/app/README.md
+9
View File
@@ -0,0 +1,9 @@
# TF Lite Android Example (Deprecated)
This example has been moved to the new
[TensorFlow examples repo](https://github.com/tensorflow/examples), and split
into several distinct examples:
* [Image Classification](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android)
* [Object Detection](https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android)
* [Speech Commands](https://github.com/tensorflow/examples/tree/master/lite/examples/speech_commands/android)
tensorflow/lite/examples/experimental_new_converter/BUILD
+18
View File
@@ -0,0 +1,18 @@
load("@xla//third_party/rules_python/python:py_binary.bzl", "py_binary")
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:LICENSE"],
default_visibility = ["//visibility:public"],
licenses = ["notice"],
)
py_binary(
name = "stack_trace_example",
srcs = ["stack_trace_example.py"],
strict_deps = True,
deps = [
"//tensorflow:tensorflow_py",
"//tensorflow/lite/python:lite",
"@absl_py//absl:app",
],
)
tensorflow/lite/examples/experimental_new_converter/Keras_LSTM_fusion_Codelab.ipynb
+224
View File
@@ -0,0 +1,224 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "Z7gEg4DRBwbO"
},
"source": [
"\n",
"# Overview\n",
"This CodeLab demonstrates how to build a fused TFLite LSTM model for MNIST recognition using Keras, and how to convert it to TensorFlow Lite.\n",
"\n",
"The CodeLab is very similar to the Keras LSTM [CodeLab](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/examples/experimental_new_converter/keras_lstm.ipynb). However, we're creating fused LSTM ops rather than the unfused versoin.\n",
"\n",
"Also note: We're not trying to build the model to be a real world application, but only demonstrate how to use TensorFlow Lite. You can a build a much better model using CNN models. For a more canonical lstm codelab, please see [here](https://github.com/keras-team/keras/blob/master/examples/imdb_lstm.py)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1v9muouWCrLA"
},
"source": [
"\n",
"# Step 0: Prerequisites\n",
"It's recommended to try this feature with the newest TensorFlow nightly pip build."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "P0bSI6A5AWaT"
},
"outputs": [],
"source": [
"!pip install tf-nightly"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zeo4IA1xC4O8"
},
"source": [
"# Step 1: Build the MNIST LSTM model."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "yMtp56hRBvHe"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import tensorflow as tf"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "IQtjKcMYC_nD"
},
"outputs": [],
"source": [
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Input(shape=(28, 28), name='input'),\n",
" tf.keras.layers.LSTM(20, time_major=False, return_sequences=True),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax, name='output')\n",
"])\n",
"model.compile(optimizer='adam',\n",
" loss='sparse_categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qS_79wHVDcri"
},
"source": [
"# Step 2: Train & Evaluate the model.\n",
"We will train the model using MNIST data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-yEAraXGDlcQ"
},
"outputs": [],
"source": [
"# Load MNIST dataset.\n",
"(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
"x_train, x_test = x_train / 255.0, x_test / 255.0\n",
"x_train = x_train.astype(np.float32)\n",
"x_test = x_test.astype(np.float32)\n",
"\n",
"# Change this to True if you want to test the flow rapidly.\n",
"# Train with a small dataset and only 1 epoch. The model will work poorly\n",
"# but this provides a fast way to test if the conversion works end to end.\n",
"_FAST_TRAINING = False\n",
"_EPOCHS = 5\n",
"if _FAST_TRAINING:\n",
" _EPOCHS = 1\n",
" _TRAINING_DATA_COUNT = 1000\n",
" x_train = x_train[:_TRAINING_DATA_COUNT]\n",
" y_train = y_train[:_TRAINING_DATA_COUNT]\n",
"\n",
"model.fit(x_train, y_train, epochs=_EPOCHS)\n",
"model.evaluate(x_test, y_test, verbose=0)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5pGyWlkJDpMQ"
},
"source": [
"# Step 3: Convert the Keras model to TensorFlow Lite model."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tB1NZBUHDogR"
},
"outputs": [],
"source": [
"run_model = tf.function(lambda x: model(x))\n",
"# This is important, let's fix the input size.\n",
"BATCH_SIZE = 1\n",
"STEPS = 28\n",
"INPUT_SIZE = 28\n",
"concrete_func = run_model.get_concrete_function(\n",
" tf.TensorSpec([BATCH_SIZE, STEPS, INPUT_SIZE], model.inputs[0].dtype))\n",
"\n",
"# model directory.\n",
"MODEL_DIR = \"keras_lstm\"\n",
"model.save(MODEL_DIR, save_format=\"tf\", signatures=concrete_func)\n",
"\n",
"converter = tf.lite.TFLiteConverter.from_saved_model(MODEL_DIR)\n",
"tflite_model = converter.convert()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "INFyl-J3FAOY"
},
"source": [
"# Step 4: Check the converted TensorFlow Lite model.\n",
"Now load the TensorFlow Lite model and use the TensorFlow Lite python interpreter to verify the results."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0-b0IKK2FGuO"
},
"outputs": [],
"source": [
"# Run the model with TensorFlow to get expected results.\n",
"TEST_CASES = 10\n",
"\n",
"# Run the model with TensorFlow Lite\n",
"interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
"interpreter.allocate_tensors()\n",
"input_details = interpreter.get_input_details()\n",
"output_details = interpreter.get_output_details()\n",
"\n",
"for i in range(TEST_CASES):\n",
" expected = model.predict(x_test[i:i+1])\n",
" interpreter.set_tensor(input_details[0][\"index\"], x_test[i:i+1, :, :])\n",
" interpreter.invoke()\n",
" result = interpreter.get_tensor(output_details[0][\"index\"])\n",
"\n",
" # Assert if the result of TFLite model is consistent with the TF model.\n",
" np.testing.assert_almost_equal(expected, result, decimal=5)\n",
" print(\"Done. The result of TensorFlow matches the result of TensorFlow Lite.\")\n",
"\n",
" # Please note: TfLite fused Lstm kernel is stateful, so we need to reset\n",
" # the states.\n",
" # Clean up internal states.\n",
" interpreter.reset_all_variables()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Cf6KC9fbFY5f"
},
"source": [
"# Step 5: Let's inspect the converted TFLite model.\n",
"\n",
"Let's check the model, you can see the LSTM will be in it's fused format.\n",
"\n",
"![Fused LSTM](https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/examples/experimental_new_converter/keras_lstm.png)\n"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "Keras LSTM fusion Codelab.ipynb",
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
tensorflow/lite/examples/experimental_new_converter/keras_lstm.ipynb
+234
View File
@@ -0,0 +1,234 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "hRWOI1nxutyx"
},
"source": [
"# Overview\n",
"\n",
"This CodeLab demonstrates how to build a LSTM model for MNIST recognition using Keras, and how to convert it to TensorFlow Lite.\n",
"\n",
"The CodeLab is very similar to the `tf.lite.experimental.nn.TFLiteLSTMCell`\n",
"[CodeLab](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/experimental/examples/lstm/TensorFlowLite_LSTM_Keras_Tutorial.ipynb). However, with the control flow support in the experimental new converter, we can define the model with control flow directly without refactoring the code.\n",
"\n",
"Also note: We're not trying to build the model to be a real world application, but only demonstrate how to use TensorFlow Lite. You can a build a much better model using CNN models. For a more canonical lstm codelab, please see [here](https://github.com/keras-team/keras/blob/master/examples/imdb_lstm.py)."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "wZCbNdY7MNSP"
},
"source": [
"# Step 0: Prerequisites\n",
"\n",
"It's recommended to try this feature with the newest TensorFlow nightly pip build."
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "6Zk2sUHUm5td"
},
"outputs": [],
"source": [
"!pip install tf-nightly --upgrade"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "R3Ku1Lx9vvfX"
},
"source": [
"\n",
"## Step 1: Build the MNIST LSTM model.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "yQpmCIqJPetJ"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import tensorflow as tf"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "wiYZoDlC5SEJ"
},
"outputs": [],
"source": [
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Input(shape=(28, 28), name='input'),\n",
" tf.keras.layers.LSTM(20),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax, name='output')\n",
"])\n",
"model.compile(optimizer='adam',\n",
" loss='sparse_categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Ff6X9gg_wk7K"
},
"source": [
"## Step 2: Train & Evaluate the model.\n",
"We will train the model using MNIST data."
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "23W41fiRPOmh"
},
"outputs": [],
"source": [
"# Load MNIST dataset.\n",
"(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
"x_train, x_test = x_train / 255.0, x_test / 255.0\n",
"x_train = x_train.astype(np.float32)\n",
"x_test = x_test.astype(np.float32)\n",
"\n",
"# Change this to True if you want to test the flow rapidly.\n",
"# Train with a small dataset and only 1 epoch. The model will work poorly\n",
"# but this provides a fast way to test if the conversion works end to end.\n",
"_FAST_TRAINING = False\n",
"_EPOCHS = 5\n",
"if _FAST_TRAINING:\n",
" _EPOCHS = 1\n",
" _TRAINING_DATA_COUNT = 1000\n",
" x_train = x_train[:_TRAINING_DATA_COUNT]\n",
" y_train = y_train[:_TRAINING_DATA_COUNT]\n",
"\n",
"model.fit(x_train, y_train, epochs=_EPOCHS)\n",
"model.evaluate(x_test, y_test, verbose=0)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "NtPJGiIQw0nM"
},
"source": [
"## Step 3: Convert the Keras model to TensorFlow Lite model.\n",
"\n",
"Note here: we just convert to TensorFlow Lite model as usual."
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "Tbuu_8PFz-x_"
},
"outputs": [],
"source": [
"converter = tf.lite.TFLiteConverter.from_keras_model(model)\n",
"tflite_model = converter.convert()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "5rHrZkIuxxar"
},
"source": [
"## Step 4: Check the converted TensorFlow Lite model.\n",
"\n",
"Now load the TensorFlow Lite model and use the TensorFlow Lite python interpreter to verify the results."
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "8lao097MnFf2"
},
"outputs": [],
"source": [
"# Run the model with TensorFlow to get expected results.\n",
"expected = model.predict(x_test[0:1])\n",
"\n",
"# Run the model with TensorFlow Lite\n",
"interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
"interpreter.allocate_tensors()\n",
"input_details = interpreter.get_input_details()\n",
"output_details = interpreter.get_output_details()\n",
"interpreter.set_tensor(input_details[0][\"index\"], x_test[0:1, :, :])\n",
"interpreter.invoke()\n",
"result = interpreter.get_tensor(output_details[0][\"index\"])\n",
"\n",
"# Assert if the result of TFLite model is consistent with the TF model.\n",
"np.testing.assert_almost_equal(expected, result)\n",
"print(\"Done. The result of TensorFlow matches the result of TensorFlow Lite.\")"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "DWhGUkIs71Qu"
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.14+"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
tensorflow/lite/examples/experimental_new_converter/keras_lstm.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 70 KiB

tensorflow/lite/examples/experimental_new_converter/stack_trace_example.py
+83
View File
@@ -0,0 +1,83 @@
# Copyright 2019 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""CodeLab for displaying error stack trace w/ MLIR-based converter."""
import sys
from absl import app
import tensorflow as tf
from tensorflow.lite.python import lite
def suppress_exception(f):
def wrapped():
try:
f()
except: # pylint: disable=bare-except
pass
return wrapped
class TestModule(tf.Module):
"""The test model has unsupported op."""
@tf.function(input_signature=[tf.TensorSpec(shape=[3, 3], dtype=tf.float32)])
def model(self, x):
y = tf.math.reciprocal(x) # Not supported
return y + y
# comment out the `@suppress_exception` to display the stack trace
@suppress_exception
def test_from_saved_model():
"""displaying stack trace when converting saved model."""
test_model = TestModule()
saved_model_path = '/tmp/test.saved_model'
save_options = tf.saved_model.SaveOptions(save_debug_info=True)
tf.saved_model.save(test_model, saved_model_path, options=save_options)
# load the model and convert
converter = lite.TFLiteConverterV2.from_saved_model(saved_model_path)
converter.convert()
# comment out the `@suppress_exception` to display the stack trace
# @suppress_exception
def test_from_concrete_function():
"""displaying stack trace when converting concrete function."""
@tf.function(input_signature=[tf.TensorSpec(shape=[3, 3], dtype=tf.float32)])
def model(x):
y = tf.math.reciprocal(x) # not supported
return y + y
func = model.get_concrete_function()
converter = lite.TFLiteConverterV2.from_concrete_functions([func], model)
converter.convert()
def main(argv):
if len(argv) > 1:
raise app.UsageError('Too many command-line arguments.')
sys.stdout.write('==== Testing from_concrete_functions ====\n')
test_from_concrete_function()
sys.stdout.write('==== Testing from_saved_model ====\n')
test_from_saved_model()
if __name__ == '__main__':
app.run(main)
tensorflow/lite/examples/ios/download_models.sh
Executable
+47
View File
@@ -0,0 +1,47 @@
#!/bin/bash
# Copyright 2017 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
set -ex
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
FLOAT_MODEL_URL="https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_02_22/mobilenet_v1_1.0_224.tgz"
QUANTIZED_MODEL_URL="https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz"
DOWNLOADS_DIR=$(mktemp -d)
cd "$SCRIPT_DIR"
download_and_extract() {
local url="$1"
local dir="$2"
echo "downloading ${url}" >&2
mkdir -p "${dir}"
tempdir=$(mktemp -d)
curl -L ${url} > ${tempdir}/archive.tgz
cd ${dir}
tar zxvf ${tempdir}/archive.tgz
rm -rf ${tempdir}
}
download_and_extract "${FLOAT_MODEL_URL}" "${DOWNLOADS_DIR}/float_model"
download_and_extract "${QUANTIZED_MODEL_URL}" "${DOWNLOADS_DIR}/quantized_model"
cd "$SCRIPT_DIR"
cp "${DOWNLOADS_DIR}/float_model/mobilenet_v1_1.0_224.tflite" "simple/data/mobilenet_v1_1.0_224.tflite"
cp "${DOWNLOADS_DIR}/float_model/mobilenet_v1_1.0_224.tflite" "camera/data/mobilenet_v1_1.0_224.tflite"
cp "${DOWNLOADS_DIR}/quantized_model/mobilenet_v1_1.0_224_quant.tflite" \
'camera/data/mobilenet_quant_v1_224.tflite'
echo "Done"
tensorflow/lite/examples/ios/simple/AppDelegate.h
+21
View File
@@ -0,0 +1,21 @@
// Copyright 2015 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import <UIKit/UIKit.h>
@interface AppDelegate : UIResponder<UIApplicationDelegate>
@property(strong, nonatomic) UIWindow *window;
@end
tensorflow/lite/examples/ios/simple/AppDelegate.mm
+48
View File
@@ -0,0 +1,48 @@
// Copyright 2015 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import "AppDelegate.h"
#import "RunModelViewController.h"
@implementation AppDelegate
- (BOOL)application:(UIApplication *)application
didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
UITabBarController *bar = [[UITabBarController alloc] init];
[bar setViewControllers:@[ [[RunModelViewController alloc] init] ]];
bar.selectedIndex = 0;
self.window = [[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]];
self.window.rootViewController = bar;
[self.window makeKeyAndVisible];
return YES;
}
- (void)applicationWillResignActive:(UIApplication *)application {
}
- (void)applicationDidEnterBackground:(UIApplication *)application {
}
- (void)applicationWillEnterForeground:(UIApplication *)application {
}
- (void)applicationDidBecomeActive:(UIApplication *)application {
}
- (void)applicationWillTerminate:(UIApplication *)application {
}
@end
tensorflow/lite/examples/ios/simple/Podfile
+5
View File
@@ -0,0 +1,5 @@
platform :ios, '8.0'
inhibit_all_warnings!
target 'tflite_simple_example'
pod 'TensorFlowLite', '1.13.1'
tensorflow/lite/examples/ios/simple/RunModel-Info.plist
+47
View File
@@ -0,0 +1,47 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>en</string>
<key>CFBundleDisplayName</key>
<string>tflite-simple-example</string>
<key>CFBundleExecutable</key>
<string>tflite_simple_example</string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>ios-app</string>
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>1.0</string>
<key>LSRequiresIPhoneOS</key>
<true/>
<key>UILaunchStoryboardName</key>
<string>RunModelViewController</string>
<key>UIRequiredDeviceCapabilities</key>
<array>
<string>armv7</string>
</array>
<key>UISupportedInterfaceOrientations</key>
<array>
<string>UIInterfaceOrientationPortrait</string>
<string>UIInterfaceOrientationLandscapeLeft</string>
<string>UIInterfaceOrientationLandscapeRight</string>
</array>
<key>UISupportedInterfaceOrientations~ipad</key>
<array>
<string>UIInterfaceOrientationPortrait</string>
<string>UIInterfaceOrientationPortraitUpsideDown</string>
<string>UIInterfaceOrientationLandscapeLeft</string>
<string>UIInterfaceOrientationLandscapeRight</string>
</array>
</dict>
</plist>
tensorflow/lite/examples/ios/simple/RunModelViewController.h
+24
View File
@@ -0,0 +1,24 @@
// Copyright 2015 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import <UIKit/UIKit.h>
@interface RunModelViewController : UIViewController
- (IBAction)getUrl:(id)sender;
@property(weak, nonatomic) IBOutlet UITextView *urlContentTextView;
@property(weak, nonatomic) IBOutlet UITextField *urlTextField;
@end
tensorflow/lite/examples/ios/simple/RunModelViewController.mm
+212
View File
@@ -0,0 +1,212 @@
// Copyright 2015 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import "RunModelViewController.h"
#include <pthread.h>
#include <unistd.h>
#include <fstream>
#include <iostream>
#include <queue>
#include <sstream>
#include <string>
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/op_resolver.h"
#include "tensorflow/lite/string_util.h"
#include "ios_image_load.h"
NSString* RunInferenceOnImage();
@interface RunModelViewController ()
@end
@implementation RunModelViewController {
}
- (IBAction)getUrl:(id)sender {
NSString* inference_result = RunInferenceOnImage();
self.urlContentTextView.text = inference_result;
}
@end
// Returns the top N confidence values over threshold in the provided vector,
// sorted by confidence in descending order.
static void GetTopN(const float* prediction, const int prediction_size, const int num_results,
const float threshold, std::vector<std::pair<float, int> >* top_results) {
// Will contain top N results in ascending order.
std::priority_queue<std::pair<float, int>, std::vector<std::pair<float, int> >,
std::greater<std::pair<float, int> > >
top_result_pq;
const long count = prediction_size;
for (int i = 0; i < count; ++i) {
const float value = prediction[i];
// Only add it if it beats the threshold and has a chance at being in
// the top N.
if (value < threshold) {
continue;
}
top_result_pq.push(std::pair<float, int>(value, i));
// If at capacity, kick the smallest value out.
if (top_result_pq.size() > num_results) {
top_result_pq.pop();
}
}
// Copy to output vector and reverse into descending order.
while (!top_result_pq.empty()) {
top_results->push_back(top_result_pq.top());
top_result_pq.pop();
}
std::reverse(top_results->begin(), top_results->end());
}
NSString* FilePathForResourceName(NSString* name, NSString* extension) {
NSString* file_path = [[NSBundle mainBundle] pathForResource:name ofType:extension];
if (file_path == NULL) {
NSLog(@"Couldn't find '%@.%@' in bundle.", name, extension);
exit(-1);
}
return file_path;
}
NSString* RunInferenceOnImage() {
NSString* graph = @"mobilenet_v1_1.0_224";
const int num_threads = 1;
std::string input_layer_type = "float";
std::vector<int> sizes = {1, 224, 224, 3};
const NSString* graph_path = FilePathForResourceName(graph, @"tflite");
std::unique_ptr<tflite::FlatBufferModel> model(
tflite::FlatBufferModel::BuildFromFile([graph_path UTF8String]));
if (!model) {
NSLog(@"Failed to mmap model %@.", graph);
exit(-1);
}
NSLog(@"Loaded model %@.", graph);
model->error_reporter();
NSLog(@"Resolved reporter.");
tflite::ops::builtin::BuiltinOpResolver resolver;
RegisterSelectedOps(&resolver);
std::unique_ptr<tflite::Interpreter> interpreter;
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
if (!interpreter) {
NSLog(@"Failed to construct interpreter.");
exit(-1);
}
if (num_threads != -1) {
interpreter->SetNumThreads(num_threads);
}
int input = interpreter->inputs()[0];
if (input_layer_type != "string") {
interpreter->ResizeInputTensor(input, sizes);
}
if (interpreter->AllocateTensors() != kTfLiteOk) {
NSLog(@"Failed to allocate tensors.");
exit(-1);
}
// Read the label list
NSString* labels_path = FilePathForResourceName(@"labels", @"txt");
std::vector<std::string> label_strings;
std::ifstream t;
t.open([labels_path UTF8String]);
std::string line;
while (t) {
std::getline(t, line);
label_strings.push_back(line);
}
t.close();
// Read the Grace Hopper image.
NSString* image_path = FilePathForResourceName(@"grace_hopper", @"jpg");
int image_width;
int image_height;
int image_channels;
std::vector<uint8_t> image_data =
LoadImageFromFile([image_path UTF8String], &image_width, &image_height, &image_channels);
const int wanted_width = 224;
const int wanted_height = 224;
const int wanted_channels = 3;
const float input_mean = 127.5f;
const float input_std = 127.5f;
assert(image_channels >= wanted_channels);
uint8_t* in = image_data.data();
float* out = interpreter->typed_tensor<float>(input);
for (int y = 0; y < wanted_height; ++y) {
const int in_y = (y * image_height) / wanted_height;
uint8_t* in_row = in + (in_y * image_width * image_channels);
float* out_row = out + (y * wanted_width * wanted_channels);
for (int x = 0; x < wanted_width; ++x) {
const int in_x = (x * image_width) / wanted_width;
uint8_t* in_pixel = in_row + (in_x * image_channels);
float* out_pixel = out_row + (x * wanted_channels);
for (int c = 0; c < wanted_channels; ++c) {
out_pixel[c] = (in_pixel[c] - input_mean) / input_std;
}
}
}
if (interpreter->Invoke() != kTfLiteOk) {
NSLog(@"Failed to invoke!");
exit(-1);
}
float* output = interpreter->typed_output_tensor<float>(0);
const int output_size = 1000;
const int kNumResults = 5;
const float kThreshold = 0.1f;
std::vector<std::pair<float, int> > top_results;
GetTopN(output, output_size, kNumResults, kThreshold, &top_results);
std::stringstream ss;
ss.precision(3);
for (const auto& result : top_results) {
const float confidence = result.first;
const int index = result.second;
ss << index << " " << confidence << " ";
// Write out the result as a string
if (index < label_strings.size()) {
// just for safety: theoretically, the output is under 1000 unless there
// is some numerical issues leading to a wrong prediction.
ss << label_strings[index];
} else {
ss << "Prediction: " << index;
}
ss << "\n";
}
std::string predictions = ss.str();
NSString* result = @"";
result = [NSString stringWithFormat:@"%@ - %s", result, predictions.c_str()];
NSLog(@"Predictions: %@", result);
return result;
}
tensorflow/lite/examples/ios/simple/RunModelViewController.xib
+46
View File
@@ -0,0 +1,46 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<document type="com.apple.InterfaceBuilder3.CocoaTouch.XIB" version="3.0" toolsVersion="9531" systemVersion="15D21" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES">
<dependencies>
<plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="9529"/>
</dependencies>
<objects>
<placeholder placeholderIdentifier="IBFilesOwner" id="-1" userLabel="File's Owner" customClass="RunModelViewController">
<connections>
<outlet property="urlContentTextView" destination="quY-AK-ZCn" id="YjW-BO-1Ta"/>
<outlet property="urlTextField" destination="hPw-q5-vh5" id="wmc-b6-2CV"/>
<outlet property="view" destination="1" id="iHm-Rr-4wj"/>
</connections>
</placeholder>
<placeholder placeholderIdentifier="IBFirstResponder" id="-2" customClass="UIResponder"/>
<view contentMode="scaleToFill" id="1">
<rect key="frame" x="0.0" y="0.0" width="320" height="568"/>
<autoresizingMask key="autoresizingMask" widthSizable="YES" heightSizable="YES"/>
<subviews>
<textView clipsSubviews="YES" contentMode="scaleToFill" fixedFrame="YES" editable="NO" text="The results of running the model will appear here." selectable="NO" translatesAutoresizingMaskIntoConstraints="NO" id="quY-AK-ZCn">
<rect key="frame" x="40" y="99" width="240" height="168"/>
<color key="backgroundColor" white="1" alpha="1" colorSpace="calibratedWhite"/>
<fontDescription key="fontDescription" type="system" pointSize="14"/>
<textInputTraits key="textInputTraits" autocapitalizationType="sentences"/>
</textView>
<button opaque="NO" contentMode="scaleToFill" fixedFrame="YES" contentHorizontalAlignment="center" contentVerticalAlignment="center" buttonType="roundedRect" lineBreakMode="middleTruncation" translatesAutoresizingMaskIntoConstraints="NO" id="AAC-Bk-PCC">
<rect key="frame" x="76" y="37" width="168" height="30"/>
<color key="backgroundColor" white="0.33333333333333331" alpha="1" colorSpace="calibratedWhite"/>
<state key="normal" title="Run Model">
<color key="titleShadowColor" white="0.5" alpha="1" colorSpace="calibratedWhite"/>
</state>
<connections>
<action selector="getUrl:" destination="-1" eventType="touchUpInside" id="mdP-nK-k9T"/>
</connections>
</button>
</subviews>
<color key="backgroundColor" red="0.78314738357315861" green="0.79869981749999996" blue="0.56305065858222869" alpha="1" colorSpace="calibratedRGB"/>
</view>
<textField opaque="NO" clipsSubviews="YES" contentMode="scaleToFill" contentHorizontalAlignment="left" contentVerticalAlignment="center" text="http://localhost:8080" borderStyle="roundedRect" placeholder="Enter URL" minimumFontSize="17" id="hPw-q5-vh5">
<rect key="frame" x="0.0" y="0.0" width="280" height="30"/>
<autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMaxY="YES"/>
<fontDescription key="fontDescription" type="system" pointSize="14"/>
<textInputTraits key="textInputTraits"/>
<point key="canvasLocation" x="795" y="44"/>
</textField>
</objects>
</document>
tensorflow/lite/examples/ios/simple/data/grace_hopper.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 72 KiB

tensorflow/lite/examples/ios/simple/ios_image_load.h
+23
View File
@@ -0,0 +1,23 @@
// Copyright 2015 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef TENSORFLOW_LITE_EXAMPLES_IOS_SIMPLE_IOS_IMAGE_LOAD_H_
#define TENSORFLOW_LITE_EXAMPLES_IOS_SIMPLE_IOS_IMAGE_LOAD_H_
#include <vector>
std::vector<uint8_t> LoadImageFromFile(const char* file_name, int* out_width,
int* out_height, int* out_channels);
#endif // TENSORFLOW_LITE_EXAMPLES_IOS_SIMPLE_IOS_IMAGE_LOAD_H_
tensorflow/lite/examples/ios/simple/ios_image_load.mm
+82
View File
@@ -0,0 +1,82 @@
// Copyright 2015 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "ios_image_load.h"
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#import <CoreImage/CoreImage.h>
#import <ImageIO/ImageIO.h>
std::vector<uint8_t> LoadImageFromFile(const char* file_name, int* out_width, int* out_height,
int* out_channels) {
FILE* file_handle = fopen(file_name, "rb");
fseek(file_handle, 0, SEEK_END);
const size_t bytes_in_file = ftell(file_handle);
fseek(file_handle, 0, SEEK_SET);
std::vector<uint8_t> file_data(bytes_in_file);
fread(file_data.data(), 1, bytes_in_file, file_handle);
fclose(file_handle);
CFDataRef file_data_ref =
CFDataCreateWithBytesNoCopy(NULL, file_data.data(), bytes_in_file, kCFAllocatorNull);
CGDataProviderRef image_provider = CGDataProviderCreateWithCFData(file_data_ref);
const char* suffix = strrchr(file_name, '.');
if (!suffix || suffix == file_name) {
suffix = "";
}
CGImageRef image;
if (strcasecmp(suffix, ".png") == 0) {
image = CGImageCreateWithPNGDataProvider(image_provider, NULL, true, kCGRenderingIntentDefault);
} else if ((strcasecmp(suffix, ".jpg") == 0) || (strcasecmp(suffix, ".jpeg") == 0)) {
image =
CGImageCreateWithJPEGDataProvider(image_provider, NULL, true, kCGRenderingIntentDefault);
} else {
CFRelease(image_provider);
CFRelease(file_data_ref);
fprintf(stderr, "Unknown suffix for file '%s'\n", file_name);
*out_width = 0;
*out_height = 0;
*out_channels = 0;
return std::vector<uint8_t>();
}
const int width = (int)CGImageGetWidth(image);
const int height = (int)CGImageGetHeight(image);
const int channels = 4;
CGColorSpaceRef color_space = CGColorSpaceCreateDeviceRGB();
const int bytes_per_row = (width * channels);
const int bytes_in_image = (bytes_per_row * height);
std::vector<uint8_t> result(bytes_in_image);
const int bits_per_component = 8;
CGContextRef context =
CGBitmapContextCreate(result.data(), width, height, bits_per_component, bytes_per_row,
color_space, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(color_space);
CGContextDrawImage(context, CGRectMake(0, 0, width, height), image);
CGContextRelease(context);
CFRelease(image);
CFRelease(image_provider);
CFRelease(file_data_ref);
*out_width = width;
*out_height = height;
*out_channels = channels;
return result;
}
tensorflow/lite/examples/ios/simple/main.mm
+22
View File
@@ -0,0 +1,22 @@
// Copyright 2015 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import <UIKit/UIKit.h>
int main(int argc, char *argv[]) {
@autoreleasepool {
NSString *delegateClassName = @"AppDelegate";
return UIApplicationMain(argc, argv, nil, delegateClassName);
}
}
tensorflow/lite/examples/label_image/BUILD
+100
View File
@@ -0,0 +1,100 @@
# Description:
# TensorFlow Lite Example Label Image.
load("@rules_cc//cc:cc_binary.bzl", "cc_binary")
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//tensorflow/lite:build_def.bzl", "tflite_linkopts")
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:LICENSE"],
default_visibility = ["//visibility:public"],
licenses = ["notice"],
)
exports_files(glob([
"testdata/*.bmp",
]))
cc_binary(
name = "label_image",
srcs = [
"get_top_n.h",
"get_top_n_impl.h",
"label_image.cc",
],
linkopts = tflite_linkopts() + select({
"//tensorflow:android": [
"-pie", # Android 5.0 and later supports only PIE
"-lm", # some builtin ops, e.g., tanh, need -lm
"-Wl,-rpath=/data/local/tmp", # for hexagon delegate
],
"//conditions:default": [],
}),
deps = [
":bitmap_helpers",
"//tensorflow/lite:framework",
"//tensorflow/lite:string",
"//tensorflow/lite/c:c_api_types",
"//tensorflow/lite/c:common",
"//tensorflow/lite/core:cc_api_stable",
"//tensorflow/lite/kernels:builtin_ops",
"//tensorflow/lite/profiling:profile_buffer",
"//tensorflow/lite/profiling:profiler",
"//tensorflow/lite/schema:schema_fbs",
"//tensorflow/lite/tools:command_line_flags",
"//tensorflow/lite/tools:tool_params",
"//tensorflow/lite/tools/delegates:delegate_provider_hdr",
"//tensorflow/lite/tools/delegates:tflite_execution_providers",
],
)
cc_library(
name = "bitmap_helpers",
srcs = ["bitmap_helpers.cc"],
hdrs = [
"bitmap_helpers.h",
"bitmap_helpers_impl.h",
"label_image.h",
"log.h",
],
deps = [
"//tensorflow/core/platform:tstring",
"//tensorflow/lite:builtin_op_data",
"//tensorflow/lite:framework",
"//tensorflow/lite:string",
"//tensorflow/lite:string_util",
"//tensorflow/lite/c:c_api_types",
"//tensorflow/lite/kernels:builtin_ops",
"//tensorflow/lite/schema:schema_fbs",
"@tsl//tsl/platform:tstring",
] + select({
"//tensorflow:android": [
"//tensorflow/lite/delegates/gpu:delegate",
"//tensorflow/lite/delegates/hexagon:hexagon_delegate",
],
"//tensorflow:android_arm64": [
"//tensorflow/lite/delegates/gpu:delegate",
"//tensorflow/lite/delegates/hexagon:hexagon_delegate",
],
"//conditions:default": [],
}),
)
cc_test(
name = "label_image_test",
srcs = [
"get_top_n.h",
"get_top_n_impl.h",
"label_image_test.cc",
],
data = [
"testdata/grace_hopper.bmp",
],
deps = [
":bitmap_helpers",
"//tensorflow/lite/c:c_api_types",
"//tensorflow/lite/c:common",
"@com_google_googletest//:gtest_main",
],
)
tensorflow/lite/examples/label_image/CMakeLists.txt
+89
View File
@@ -0,0 +1,89 @@
#
# Copyright 2021 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The label_image example for Tensorflow Lite.
populate_source_vars("${TFLITE_SOURCE_DIR}/examples/label_image"
TFLITE_LABEL_IMAGE_SRCS
FILTER "_test\\.cc$"
)
list(APPEND TFLITE_LABEL_IMAGE_SRCS
${XLA_SOURCE_DIR}/xla/tsl/util/stats_calculator.cc
${TFLITE_SOURCE_DIR}/profiling/memory_info.cc
${TFLITE_SOURCE_DIR}/profiling/profile_buffer.cc
${TFLITE_SOURCE_DIR}/profiling/profile_summarizer.cc
${TFLITE_SOURCE_DIR}/profiling/profile_summary_formatter.cc
${TFLITE_SOURCE_DIR}/profiling/time.cc
${TFLITE_SOURCE_DIR}/tools/command_line_flags.cc
${TFLITE_SOURCE_DIR}/tools/delegates/default_execution_provider.cc
${TFLITE_SOURCE_DIR}/tools/delegates/delegate_provider.cc
${TFLITE_SOURCE_DIR}/tools/evaluation/utils.cc
${TFLITE_SOURCE_DIR}/tools/tool_params.cc
)
if(TFLITE_ENABLE_XNNPACK)
list(APPEND TFLITE_LABEL_IMAGE_SRCS
${TFLITE_SOURCE_DIR}/tools/delegates/xnnpack_delegate_provider.cc
${TFLITE_SOURCE_DIR}/core/acceleration/configuration/c/xnnpack_plugin.cc
)
else()
set(TFLITE_LABEL_IMAGE_CC_OPTIONS "-DTFLITE_WITHOUT_XNNPACK")
endif() # TFLITE_ENABLE_XNNPACK
if(CMAKE_SYSTEM_NAME MATCHES "Android")
if(_TFLITE_ENABLE_NNAPI)
list(APPEND TFLITE_LABEL_IMAGE_SRCS
${TFLITE_SOURCE_DIR}/tools/delegates/nnapi_delegate_provider.cc
)
endif() # _TFLITE_ENABLE_NNAPI
endif() # Android
if(TFLITE_ENABLE_GPU)
list(APPEND TFLITE_LABEL_IMAGE_SRCS
${TFLITE_SOURCE_DIR}/tools/delegates/gpu_delegate_provider.cc
)
endif() # TFLITE_ENABLE_GPU
if(TFLITE_ENABLE_EXTERNAL_DELEGATE)
list(APPEND TFLITE_LABEL_IMAGE_SRCS
${TFLITE_SOURCE_DIR}/tools/delegates/external_delegate_provider.cc)
endif()
include_directories(label_image
PUBLIC
${CMAKE_BINARY_DIR}
)
add_executable(label_image
${TFLITE_LABEL_IMAGE_SRCS}
)
if(TFLITE_ENABLE_LABEL_IMAGE)
set_target_properties(label_image PROPERTIES EXCLUDE_FROM_ALL FALSE)
if(TFLITE_ENABLE_INSTALL)
install(TARGETS label_image)
endif() # TFLITE_ENABLE_INSTALL
else()
set_target_properties(label_image PROPERTIES EXCLUDE_FROM_ALL TRUE)
endif() # TFLITE_ENABLE_LABEL_IMAGE
target_compile_options(label_image
PRIVATE
${TFLITE_LABEL_IMAGE_CC_OPTIONS}
)
target_link_libraries(label_image
tensorflow-lite
profiling_info_proto
libprotobuf
)
tensorflow/lite/examples/label_image/README.md
+241
View File
@@ -0,0 +1,241 @@
# TensorFlow Lite C++ image classification demo
This example shows how you can load a pre-trained and converted
TensorFlow Lite model and use it to recognize objects in images.
Before you begin,
make sure you [have TensorFlow installed](https://www.tensorflow.org/install).
You also need to
[install Bazel](https://docs.bazel.build/versions/master/install.html) in order
to build this example code. And be sure you have the Python `future` module
installed:
```
pip install future --user
```
## Build the example
First run `$TENSORFLOW_ROOT/configure`. To build for Android, set
Android NDK or configure NDK setting in
`$TENSORFLOW_ROOT/WORKSPACE` first.
Build it for desktop machines (tested on Ubuntu and OS X):
```
bazel build -c opt //tensorflow/lite/examples/label_image:label_image
```
Build it for Android ARMv8:
```
bazel build -c opt --config=android_arm64 \
//tensorflow/lite/examples/label_image:label_image
```
Build it for Android arm-v7a:
```
bazel build -c opt --config=android_arm \
//tensorflow/lite/examples/label_image:label_image
```
## Download sample model and image
You can use any compatible model, but the following MobileNet v1 model offers
a good demonstration of a model trained to recognize 1,000 different objects.
```
# Get model
curl https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_02_22/mobilenet_v1_1.0_224.tgz | tar xzv -C /tmp
# Get labels
curl https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz | tar xzv -C /tmp mobilenet_v1_1.0_224/labels.txt
mv /tmp/mobilenet_v1_1.0_224/labels.txt /tmp/
```
## Run the sample on a desktop
```
bazel-bin/tensorflow/lite/examples/label_image/label_image \
--tflite_model /tmp/mobilenet_v1_1.0_224.tflite \
--labels /tmp/labels.txt \
--image tensorflow/lite/examples/label_image/testdata/grace_hopper.bmp
```
You should see results like this:
```
Loaded model /tmp/mobilenet_v1_1.0_224.tflite
resolved reporter
invoked
average time: 68.12 ms
0.860174: 653 653:military uniform
0.0481017: 907 907:Windsor tie
0.00786704: 466 466:bulletproof vest
0.00644932: 514 514:cornet, horn, trumpet, trump
0.00608029: 543 543:drumstick
```
## Run the sample on an Android device
Prepare data on devices, e.g.,
```
adb push bazel-bin/tensorflow/lite/examples/label_image/label_image /data/local/tmp
adb push /tmp/mobilenet_v1_1.0_224.tflite /data/local/tmp
adb push tensorflow/lite/examples/label_image/testdata/grace_hopper.bmp /data/local/tmp
adb push /tmp/labels.txt /data/local/tmp
```
Run it,
```
adb shell "/data/local/tmp/label_image \
-m /data/local/tmp/mobilenet_v1_1.0_224.tflite \
-i /data/local/tmp/grace_hopper.bmp \
-l /data/local/tmp/labels.txt"
```
then you should see something like the following:
```
Loaded model /data/local/tmp/mobilenet_v1_1.0_224.tflite
resolved reporter
INFO: Initialized TensorFlow Lite runtime.
invoked
average time: 25.03 ms
0.907071: 653 military uniform
0.0372416: 907 Windsor tie
0.00733753: 466 bulletproof vest
0.00592852: 458 bow tie
0.00414091: 514 cornet
```
Run the model with NNAPI delegate (`-a 1`),
```
adb shell "/data/local/tmp/label_image \
-m /data/local/tmp/mobilenet_v1_1.0_224.tflite \
-i /data/local/tmp/grace_hopper.bmp \
-l /data/local/tmp/labels.txt -a 1 -f 1"
```
then you should see something like the following:
```
Loaded model /data/local/tmp/mobilenet_v1_1.0_224.tflite
resolved reporter
INFO: Initialized TensorFlow Lite runtime.
INFO: Created TensorFlow Lite delegate for NNAPI. Applied NNAPI delegate.
invoked
average time:10.348 ms
0.905401: 653 military uniform
0.0379589: 907 Windsor tie
0.00735866: 466 bulletproof vest
0.00605307: 458 bow tie
0.00422573: 514 cornet
```
To run a model with the Hexagon Delegate, assuming we have followed the
[Hexagon Delegate Guide](https://www.tensorflow.org/lite/android/delegates/hexagon)
and installed Hexagon libraries in `/data/local/tmp`. Run it with (`-j 1`)
```
adb shell \
"/data/local/tmp/label_image \
-m /data/local/tmp/mobilenet_v1_1.0_224_quant.tflite \
-i /data/local/tmp/grace_hopper.bmp \
-l /data/local/tmp/labels.txt -j 1"
```
then you should see something like the followings:
```
Loaded model /data/local/tmp/mobilenet_v1_1.0_224_quant.tflite
resolved reporter
INFO: Initialized TensorFlow Lite runtime.
loaded libcdsprpc.so
INFO: Created TensorFlow Lite delegate for Hexagon.
INFO: Hexagon delegate: 31 nodes delegated out of 31 nodes with 1 partitions.
Applied Hexagon delegate.
invoked
average time: 4.231 ms
0.639216: 458 bow tie
0.329412: 653 military uniform
0.00784314: 835 suit
0.00784314: 611 jersey
0.00392157: 514 cornet
```
Run the model with the XNNPACK delegate (`-x 1`),
```shell
adb shell \
"/data/local/tmp/label_image \
-m /data/local/tmp/mobilenet_v1_1.0_224.tflite \
-i /data/local/tmp/grace_hopper.bmp \
-l /data/local/tmp/labels.txt -x 1"
```
then you should see something like the following:
```
Loaded model /data/local/tmp/mobilenet_v1_1.0_224.tflite
resolved reporter
INFO: Initialized TensorFlow Lite runtime. Applied XNNPACK delegate.
invoked
average time: 17.33 ms
0.90707: 653 military uniform
0.0372418: 907 Windsor tie
0.0073376: 466 bulletproof vest
0.00592857: 458 bow tie
0.00414093: 514 cornet
```
With `-h` or any other unsupported flags, `label_image` will list supported
options:
```shell
sargo:/data/local/tmp $ ./label_image -h
./label_image: invalid option -- h
label_image
--accelerated, -a: [0|1], use Android NNAPI or not
--old_accelerated, -d: [0|1], use old Android NNAPI delegate or not
--allow_fp16, -f: [0|1], allow running fp32 models with fp16 or not
--count, -c: loop interpreter->Invoke() for certain times
--gl_backend, -g: [0|1]: use GPU Delegate on Android
--hexagon_delegate, -j: [0|1]: use Hexagon Delegate on Android
--input_mean, -b: input mean
--input_std, -s: input standard deviation
--image, -i: image_name.bmp
--labels, -l: labels for the model
--tflite_model, -m: model_name.tflite
--profiling, -p: [0|1], profiling or not
--num_results, -r: number of results to show
--threads, -t: number of threads
--verbose, -v: [0|1] print more information
--warmup_runs, -w: number of warmup runs
--xnnpack_delegate, -x [0:1]: xnnpack delegate`
```
See the `label_image.cc` source code for other command line options.
Note that this binary also supports more runtime/delegate arguments introduced
by the [delegate registrar](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/delegates).
If there is any conflict, the arguments mentioned earlier are given precedence.
For example, you can run the binary with additional command line options
such as `--use_nnapi=true --nnapi_accelerator_name=google-edgetpu` to utilize
the EdgeTPU in a 4th-gen Pixel phone. Please be aware that the "=" in the option
should not be omitted.
```
adb shell \
"/data/local/tmp/label_image \
-m /data/local/tmp/mobilenet_v1_1.0_224_quant.tflite \
-i /data/local/tmp/grace_hopper.bmp \
-l /data/local/tmp/labels.txt -j 1 \
--use_nnapi=true --nnapi_accelerator_name=google-edgetpu"
```
tensorflow/lite/examples/label_image/bitmap_helpers.cc
+126
View File
@@ -0,0 +1,126 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/lite/examples/label_image/bitmap_helpers.h"
#include <unistd.h> // NOLINT(build/include_order)
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include "tensorflow/lite/examples/label_image/label_image.h"
#include "tensorflow/lite/examples/label_image/log.h"
#include "tsl/platform/ctstring_internal.h"
namespace tflite {
namespace label_image {
std::vector<uint8_t> decode_bmp(const uint8_t* input, int row_size, int width,
int height, int channels, bool top_down) {
std::vector<uint8_t> output(height * width * channels);
for (int i = 0; i < height; i++) {
int src_pos;
int dst_pos;
for (int j = 0; j < width; j++) {
if (!top_down) {
src_pos = ((height - 1 - i) * row_size) + j * channels;
} else {
src_pos = i * row_size + j * channels;
}
dst_pos = (i * width + j) * channels;
switch (channels) {
case 1:
output[dst_pos] = input[src_pos];
break;
case 3:
// BGR -> RGB
output[dst_pos] = input[src_pos + 2];
output[dst_pos + 1] = input[src_pos + 1];
output[dst_pos + 2] = input[src_pos];
break;
case 4:
// BGRA -> RGBA
output[dst_pos] = input[src_pos + 2];
output[dst_pos + 1] = input[src_pos + 1];
output[dst_pos + 2] = input[src_pos];
output[dst_pos + 3] = input[src_pos + 3];
break;
default:
LOG(FATAL) << "Unexpected number of channels: " << channels;
break;
}
}
}
return output;
}
std::vector<uint8_t> read_bmp(const std::string& input_bmp_name, int* width,
int* height, int* channels, Settings* s) {
int begin, end;
std::ifstream file(input_bmp_name, std::ios::in | std::ios::binary);
if (!file) {
LOG(FATAL) << "input file " << input_bmp_name << " not found";
exit(-1);
}
begin = file.tellg();
file.seekg(0, std::ios::end);
end = file.tellg();
size_t len = end - begin;
if (s->verbose) LOG(INFO) << "len: " << len;
std::vector<uint8_t> img_bytes(len);
file.seekg(0, std::ios::beg);
file.read(reinterpret_cast<char*>(img_bytes.data()), len);
const int32_t header_size =
TF_le32toh(*(reinterpret_cast<const int32_t*>(img_bytes.data() + 10)));
*width =
TF_le32toh(*(reinterpret_cast<const int32_t*>(img_bytes.data() + 18)));
*height =
TF_le32toh(*(reinterpret_cast<const int32_t*>(img_bytes.data() + 22)));
const int32_t bpp =
TF_le32toh(*(reinterpret_cast<const int32_t*>(img_bytes.data() + 28)));
*channels = bpp / 8;
if (s->verbose)
LOG(INFO) << "width, height, channels: " << *width << ", " << *height
<< ", " << *channels;
// there may be padding bytes when the width is not a multiple of 4 bytes
// 8 * channels == bits per pixel
const int row_size = (8 * *channels * *width + 31) / 32 * 4;
// if height is negative, data layout is top down
// otherwise, it's bottom up
bool top_down = (*height < 0);
// Decode image, allocating tensor once the image size is known
const uint8_t* bmp_pixels = &img_bytes[header_size];
return decode_bmp(bmp_pixels, row_size, *width, abs(*height), *channels,
top_down);
}
} // namespace label_image
} // namespace tflite
tensorflow/lite/examples/label_image/bitmap_helpers.h
+46
View File
@@ -0,0 +1,46 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_BITMAP_HELPERS_H_
#define TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_BITMAP_HELPERS_H_
#include <string>
#include "tensorflow/lite/examples/label_image/bitmap_helpers_impl.h" // IWYU pragma: export
#include "tensorflow/lite/examples/label_image/label_image.h"
namespace tflite {
namespace label_image {
std::vector<uint8_t> read_bmp(const std::string& input_bmp_name, int* width,
int* height, int* channels, Settings* s);
template <class T>
void resize(T* out, uint8_t* in, int image_height, int image_width,
int image_channels, int wanted_height, int wanted_width,
int wanted_channels, Settings* s);
// explicit instantiation
template void resize<float>(float*, unsigned char*, int, int, int, int, int,
int, Settings*);
template void resize<int8_t>(int8_t*, unsigned char*, int, int, int, int, int,
int, Settings*);
template void resize<uint8_t>(uint8_t*, unsigned char*, int, int, int, int, int,
int, Settings*);
} // namespace label_image
} // namespace tflite
#endif // TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_BITMAP_HELPERS_H_
tensorflow/lite/examples/label_image/bitmap_helpers_impl.h
+105
View File
@@ -0,0 +1,105 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_BITMAP_HELPERS_IMPL_H_
#define TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_BITMAP_HELPERS_IMPL_H_
#include "tensorflow/lite/examples/label_image/label_image.h"
#include "tensorflow/lite/builtin_op_data.h"
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/string_util.h"
namespace tflite {
namespace label_image {
template <class T>
void resize(T* out, uint8_t* in, int image_height, int image_width,
int image_channels, int wanted_height, int wanted_width,
int wanted_channels, Settings* s) {
int number_of_pixels = image_height * image_width * image_channels;
std::unique_ptr<Interpreter> interpreter(new Interpreter);
int base_index = 0;
// two inputs: input and new_sizes
interpreter->AddTensors(2, &base_index);
// one output
interpreter->AddTensors(1, &base_index);
// set input and output tensors
interpreter->SetInputs({0, 1});
interpreter->SetOutputs({2});
// set parameters of tensors
TfLiteQuantizationParams quant;
interpreter->SetTensorParametersReadWrite(
0, kTfLiteFloat32, "input",
{1, image_height, image_width, image_channels}, quant);
interpreter->SetTensorParametersReadWrite(1, kTfLiteInt32, "new_size", {2},
quant);
interpreter->SetTensorParametersReadWrite(
2, kTfLiteFloat32, "output",
{1, wanted_height, wanted_width, wanted_channels}, quant);
ops::builtin::BuiltinOpResolver resolver;
const TfLiteRegistration* resize_op =
resolver.FindOp(BuiltinOperator_RESIZE_BILINEAR, 1);
auto* params = reinterpret_cast<TfLiteResizeBilinearParams*>(
malloc(sizeof(TfLiteResizeBilinearParams)));
params->align_corners = false;
params->half_pixel_centers = false;
interpreter->AddNodeWithParameters({0, 1}, {2}, nullptr, 0, params, resize_op,
nullptr);
interpreter->AllocateTensors();
// fill input image
// in[] are integers, cannot do memcpy() directly
auto input = interpreter->typed_tensor<float>(0);
for (int i = 0; i < number_of_pixels; i++) {
input[i] = in[i];
}
// fill new_sizes
interpreter->typed_tensor<int>(1)[0] = wanted_height;
interpreter->typed_tensor<int>(1)[1] = wanted_width;
interpreter->Invoke();
auto output = interpreter->typed_tensor<float>(2);
auto output_number_of_pixels = wanted_height * wanted_width * wanted_channels;
for (int i = 0; i < output_number_of_pixels; i++) {
switch (s->input_type) {
case kTfLiteFloat32:
out[i] = (output[i] - s->input_mean) / s->input_std;
break;
case kTfLiteInt8:
out[i] = static_cast<int8_t>(output[i] - 128);
break;
case kTfLiteUInt8:
out[i] = static_cast<uint8_t>(output[i]);
break;
default:
break;
}
}
}
} // namespace label_image
} // namespace tflite
#endif // TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_BITMAP_HELPERS_IMPL_H_
tensorflow/lite/examples/label_image/get_top_n.h
+47
View File
@@ -0,0 +1,47 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_GET_TOP_N_H_
#define TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_GET_TOP_N_H_
#include <cstddef>
#include <utility>
#include <vector>
#include "tensorflow/lite/c/c_api_types.h"
#include "tensorflow/lite/examples/label_image/get_top_n_impl.h" // IWYU pragma: export
namespace tflite {
namespace label_image {
template <class T>
void get_top_n(T* prediction, int prediction_size, size_t num_results,
float threshold, std::vector<std::pair<float, int>>* top_results,
TfLiteType input_type);
// explicit instantiation so that we can use them otherwhere
template void get_top_n<float>(float*, int, size_t, float,
std::vector<std::pair<float, int>>*, TfLiteType);
template void get_top_n<int8_t>(int8_t*, int, size_t, float,
std::vector<std::pair<float, int>>*,
TfLiteType);
template void get_top_n<uint8_t>(uint8_t*, int, size_t, float,
std::vector<std::pair<float, int>>*,
TfLiteType);
} // namespace label_image
} // namespace tflite
#endif // TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_GET_TOP_N_H_
tensorflow/lite/examples/label_image/get_top_n_impl.h
+83
View File
@@ -0,0 +1,83 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_GET_TOP_N_IMPL_H_
#define TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_GET_TOP_N_IMPL_H_
#include <algorithm>
#include <functional>
#include <queue>
#include "tensorflow/lite/c/common.h"
namespace tflite {
namespace label_image {
extern bool input_floating;
// Returns the top N confidence values over threshold in the provided vector,
// sorted by confidence in descending order.
template <class T>
void get_top_n(T* prediction, int prediction_size, size_t num_results,
float threshold, std::vector<std::pair<float, int>>* top_results,
TfLiteType input_type) {
// Will contain top N results in ascending order.
std::priority_queue<std::pair<float, int>, std::vector<std::pair<float, int>>,
std::greater<std::pair<float, int>>>
top_result_pq;
const long count = prediction_size; // NOLINT(runtime/int)
float value = 0.0;
for (int i = 0; i < count; ++i) {
switch (input_type) {
case kTfLiteFloat32:
value = prediction[i];
break;
case kTfLiteInt8:
value = (prediction[i] + 128) / 256.0;
break;
case kTfLiteUInt8:
value = prediction[i] / 255.0;
break;
default:
break;
}
// Only add it if it beats the threshold and has a chance at being in
// the top N.
if (value < threshold) {
continue;
}
top_result_pq.push(std::pair<float, int>(value, i));
// If at capacity, kick the smallest value out.
if (top_result_pq.size() > num_results) {
top_result_pq.pop();
}
}
// Copy to output vector and reverse into descending order.
while (!top_result_pq.empty()) {
top_results->push_back(top_result_pq.top());
top_result_pq.pop();
}
std::reverse(top_results->begin(), top_results->end());
}
} // namespace label_image
} // namespace tflite
#endif // TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_GET_TOP_N_IMPL_H_
tensorflow/lite/examples/label_image/label_image.cc
+557
View File
@@ -0,0 +1,557 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/lite/examples/label_image/label_image.h"
#include <fcntl.h> // NOLINT(build/include_order)
#include <getopt.h> // NOLINT(build/include_order)
#include <sys/time.h> // NOLINT(build/include_order)
#include <sys/types.h> // NOLINT(build/include_order)
#include <sys/uio.h> // NOLINT(build/include_order)
#include <unistd.h> // NOLINT(build/include_order)
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <fstream>
#include <iomanip>
#include <iostream>
#include <memory>
#include <string>
#include <utility>
#include <vector>
#include "tensorflow/lite/c/c_api_types.h"
#include "tensorflow/lite/c/common.h"
#include "tensorflow/lite/core/interpreter_builder.h"
#include "tensorflow/lite/examples/label_image/bitmap_helpers.h"
#include "tensorflow/lite/examples/label_image/get_top_n.h"
#include "tensorflow/lite/examples/label_image/log.h"
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model_builder.h"
#include "tensorflow/lite/optional_debug_tools.h"
#include "tensorflow/lite/profiling/profile_buffer.h"
#include "tensorflow/lite/profiling/profiler.h"
#include "tensorflow/lite/schema/schema_generated.h"
#include "tensorflow/lite/string_type.h"
#include "tensorflow/lite/tools/command_line_flags.h"
#include "tensorflow/lite/tools/delegates/delegate_provider.h"
#include "tensorflow/lite/tools/tool_params.h"
namespace tflite {
namespace label_image {
double get_us(struct timeval t) { return (t.tv_sec * 1000000 + t.tv_usec); }
using TfLiteDelegatePtr = tflite::Interpreter::TfLiteDelegatePtr;
using ProvidedDelegateList = tflite::tools::ProvidedDelegateList;
class DelegateProviders {
public:
DelegateProviders() : delegate_list_util_(&params_) {
delegate_list_util_.AddAllDelegateParams();
delegate_list_util_.AppendCmdlineFlags(flags_);
// Remove the "help" flag to avoid printing "--help=false"
params_.RemoveParam("help");
delegate_list_util_.RemoveCmdlineFlag(flags_, "help");
}
// Initialize delegate-related parameters from parsing command line arguments,
// and remove the matching arguments from (*argc, argv). Returns true if all
// recognized arg values are parsed correctly.
bool InitFromCmdlineArgs(int* argc, const char** argv) {
// Note if '--help' is in argv, the Flags::Parse return false,
// see the return expression in Flags::Parse.
return Flags::Parse(argc, argv, flags_);
}
// According to passed-in settings `s`, this function sets corresponding
// parameters that are defined by various delegate execution providers. See
// lite/tools/delegates/README.md for the full list of parameters defined.
void MergeSettingsIntoParams(const Settings& s) {
// Parse settings related to GPU delegate.
// Note that GPU delegate does support OpenCL. 'gl_backend' was introduced
// when the GPU delegate only supports OpenGL. Therefore, we consider
// setting 'gl_backend' to true means using the GPU delegate.
if (s.gl_backend) {
if (!params_.HasParam("use_gpu")) {
LOG(WARN) << "GPU delegate execution provider isn't linked or GPU "
"delegate isn't supported on the platform!";
} else {
params_.Set<bool>("use_gpu", true);
// The parameter "gpu_inference_for_sustained_speed" isn't available for
// iOS devices.
if (params_.HasParam("gpu_inference_for_sustained_speed")) {
params_.Set<bool>("gpu_inference_for_sustained_speed", true);
}
params_.Set<bool>("gpu_precision_loss_allowed", s.allow_fp16);
}
}
// Parse settings related to NNAPI delegate.
if (s.accel) {
if (!params_.HasParam("use_nnapi")) {
LOG(WARN) << "NNAPI delegate execution provider isn't linked or NNAPI "
"delegate isn't supported on the platform!";
} else {
params_.Set<bool>("use_nnapi", true);
params_.Set<bool>("nnapi_allow_fp16", s.allow_fp16);
}
}
// Parse settings related to Hexagon delegate.
if (s.hexagon_delegate) {
if (!params_.HasParam("use_hexagon")) {
LOG(WARN) << "Hexagon delegate execution provider isn't linked or "
"Hexagon delegate isn't supported on the platform!";
} else {
params_.Set<bool>("use_hexagon", true);
params_.Set<bool>("hexagon_profiling", s.profiling);
}
}
// Parse settings related to XNNPACK delegate.
if (s.xnnpack_delegate) {
if (!params_.HasParam("use_xnnpack")) {
LOG(WARN) << "XNNPACK delegate execution provider isn't linked or "
"XNNPACK delegate isn't supported on the platform!";
} else {
params_.Set<bool>("use_xnnpack", true);
params_.Set<int32_t>("num_threads", s.number_of_threads);
}
}
}
// Create a list of TfLite delegates based on what have been initialized (i.e.
// 'params_').
std::vector<ProvidedDelegateList::ProvidedDelegate> CreateAllDelegates()
const {
return delegate_list_util_.CreateAllRankedDelegates();
}
std::string GetHelpMessage(const std::string& cmdline) const {
return Flags::Usage(cmdline, flags_);
}
private:
// Contain delegate-related parameters that are initialized from command-line
// flags.
tflite::tools::ToolParams params_;
// A helper to create TfLite delegates.
ProvidedDelegateList delegate_list_util_;
// Contains valid flags
std::vector<tflite::Flag> flags_;
};
// Takes a file name, and loads a list of labels from it, one per line, and
// returns a vector of the strings. It pads with empty strings so the length
// of the result is a multiple of 16, because our model expects that.
TfLiteStatus ReadLabelsFile(const string& file_name,
std::vector<string>* result,
size_t* found_label_count) {
std::ifstream file(file_name);
if (!file) {
LOG(ERROR) << "Labels file " << file_name << " not found";
return kTfLiteError;
}
result->clear();
string line;
while (std::getline(file, line)) {
result->push_back(line);
}
*found_label_count = result->size();
const int padding = 16;
while (result->size() % padding) {
result->emplace_back();
}
return kTfLiteOk;
}
void PrintProfilingInfo(const profiling::ProfileEvent* e,
uint32_t subgraph_index, uint32_t op_index,
TfLiteRegistration registration) {
// output something like
// time (ms) , Node xxx, OpCode xxx, symbolic name
// 5.352, Node 5, OpCode 4, DEPTHWISE_CONV_2D
LOG(INFO) << std::fixed << std::setw(10) << std::setprecision(3)
<< (e->elapsed_time) / 1000.0 << ", Subgraph " << std::setw(3)
<< std::setprecision(3) << subgraph_index << ", Node "
<< std::setw(3) << std::setprecision(3) << op_index << ", OpCode "
<< std::setw(3) << std::setprecision(3) << registration.builtin_code
<< ", "
<< EnumNameBuiltinOperator(
static_cast<BuiltinOperator>(registration.builtin_code));
}
void RunInference(Settings* settings,
const DelegateProviders& delegate_providers) {
if (!settings->model_name.c_str()) {
LOG(ERROR) << "no model file name";
exit(-1);
}
std::unique_ptr<tflite::FlatBufferModel> model;
std::unique_ptr<tflite::Interpreter> interpreter;
model = tflite::FlatBufferModel::BuildFromFile(settings->model_name.c_str());
if (!model) {
LOG(ERROR) << "Failed to mmap model " << settings->model_name;
exit(-1);
}
settings->model = model.get();
LOG(INFO) << "Loaded model " << settings->model_name;
model->error_reporter();
LOG(INFO) << "resolved reporter";
tflite::ops::builtin::BuiltinOpResolver resolver;
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
if (!interpreter) {
LOG(ERROR) << "Failed to construct interpreter";
exit(-1);
}
interpreter->SetAllowFp16PrecisionForFp32(settings->allow_fp16);
if (settings->verbose) {
LOG(INFO) << "tensors size: " << interpreter->tensors_size();
LOG(INFO) << "nodes size: " << interpreter->nodes_size();
LOG(INFO) << "inputs: " << interpreter->inputs().size();
LOG(INFO) << "input(0) name: " << interpreter->GetInputName(0);
int t_size = interpreter->tensors_size();
for (int i = 0; i < t_size; i++) {
if (interpreter->tensor(i)->name)
LOG(INFO) << i << ": " << interpreter->tensor(i)->name << ", "
<< interpreter->tensor(i)->bytes << ", "
<< interpreter->tensor(i)->type << ", "
<< interpreter->tensor(i)->params.scale << ", "
<< interpreter->tensor(i)->params.zero_point;
}
}
if (settings->number_of_threads != -1) {
interpreter->SetNumThreads(settings->number_of_threads);
}
int image_width = 224;
int image_height = 224;
int image_channels = 3;
std::vector<uint8_t> in = read_bmp(settings->input_bmp_name, &image_width,
&image_height, &image_channels, settings);
int input = interpreter->inputs()[0];
if (settings->verbose) LOG(INFO) << "input: " << input;
const std::vector<int> inputs = interpreter->inputs();
const std::vector<int> outputs = interpreter->outputs();
if (settings->verbose) {
LOG(INFO) << "number of inputs: " << inputs.size();
LOG(INFO) << "number of outputs: " << outputs.size();
}
auto profiler = std::make_unique<profiling::Profiler>(
settings->max_profiling_buffer_entries);
interpreter->SetProfiler(profiler.get());
auto delegates = delegate_providers.CreateAllDelegates();
for (auto& delegate : delegates) {
const auto delegate_name = delegate.provider->GetName();
if (interpreter->ModifyGraphWithDelegate(std::move(delegate.delegate)) !=
kTfLiteOk) {
LOG(ERROR) << "Failed to apply " << delegate_name << " delegate.";
exit(-1);
} else {
LOG(INFO) << "Applied " << delegate_name << " delegate.";
}
}
if (interpreter->AllocateTensors() != kTfLiteOk) {
LOG(ERROR) << "Failed to allocate tensors!";
exit(-1);
}
if (settings->verbose) PrintInterpreterState(interpreter.get());
// get input dimension from the input tensor metadata
// assuming one input only
TfLiteIntArray* dims = interpreter->tensor(input)->dims;
int wanted_height = dims->data[1];
int wanted_width = dims->data[2];
int wanted_channels = dims->data[3];
settings->input_type = interpreter->tensor(input)->type;
switch (settings->input_type) {
case kTfLiteFloat32:
resize<float>(interpreter->typed_tensor<float>(input), in.data(),
image_height, image_width, image_channels, wanted_height,
wanted_width, wanted_channels, settings);
break;
case kTfLiteInt8:
resize<int8_t>(interpreter->typed_tensor<int8_t>(input), in.data(),
image_height, image_width, image_channels, wanted_height,
wanted_width, wanted_channels, settings);
break;
case kTfLiteUInt8:
resize<uint8_t>(interpreter->typed_tensor<uint8_t>(input), in.data(),
image_height, image_width, image_channels, wanted_height,
wanted_width, wanted_channels, settings);
break;
default:
LOG(ERROR) << "cannot handle input type "
<< interpreter->tensor(input)->type << " yet";
exit(-1);
}
if (settings->profiling) profiler->StartProfiling();
for (int i = 0; i < settings->number_of_warmup_runs; i++) {
if (interpreter->Invoke() != kTfLiteOk) {
LOG(ERROR) << "Failed to invoke tflite!";
exit(-1);
}
}
struct timeval start_time, stop_time;
gettimeofday(&start_time, nullptr);
for (int i = 0; i < settings->loop_count; i++) {
if (interpreter->Invoke() != kTfLiteOk) {
LOG(ERROR) << "Failed to invoke tflite!";
exit(-1);
}
}
gettimeofday(&stop_time, nullptr);
LOG(INFO) << "invoked";
LOG(INFO) << "average time: "
<< (get_us(stop_time) - get_us(start_time)) /
(settings->loop_count * 1000)
<< " ms";
if (settings->profiling) {
profiler->StopProfiling();
auto profile_events = profiler->GetProfileEvents();
for (int i = 0; i < profile_events.size(); i++) {
auto subgraph_index = profile_events[i]->extra_event_metadata;
auto op_index = profile_events[i]->event_metadata;
const auto subgraph = interpreter->subgraph(subgraph_index);
const auto node_and_registration =
subgraph->node_and_registration(op_index);
const TfLiteRegistration registration = node_and_registration->second;
PrintProfilingInfo(profile_events[i], subgraph_index, op_index,
registration);
}
}
const float threshold = 0.001f;
std::vector<std::pair<float, int>> top_results;
int output = interpreter->outputs()[0];
TfLiteIntArray* output_dims = interpreter->tensor(output)->dims;
// assume output dims to be something like (1, 1, ... ,size)
auto output_size = output_dims->data[output_dims->size - 1];
switch (interpreter->tensor(output)->type) {
case kTfLiteFloat32:
get_top_n<float>(interpreter->typed_output_tensor<float>(0), output_size,
settings->number_of_results, threshold, &top_results,
settings->input_type);
break;
case kTfLiteInt8:
get_top_n<int8_t>(interpreter->typed_output_tensor<int8_t>(0),
output_size, settings->number_of_results, threshold,
&top_results, settings->input_type);
break;
case kTfLiteUInt8:
get_top_n<uint8_t>(interpreter->typed_output_tensor<uint8_t>(0),
output_size, settings->number_of_results, threshold,
&top_results, settings->input_type);
break;
default:
LOG(ERROR) << "cannot handle output type "
<< interpreter->tensor(output)->type << " yet";
exit(-1);
}
std::vector<string> labels;
size_t label_count;
if (ReadLabelsFile(settings->labels_file_name, &labels, &label_count) !=
kTfLiteOk)
exit(-1);
for (const auto& result : top_results) {
const float confidence = result.first;
const int index = result.second;
LOG(INFO) << confidence << ": " << index << " " << labels[index];
}
// Destory the interpreter earlier than delegates objects.
interpreter.reset();
}
void display_usage(const DelegateProviders& delegate_providers) {
LOG(INFO)
<< "\n"
<< delegate_providers.GetHelpMessage("label_image")
<< "\t--accelerated, -a: [0|1] use Android NNAPI or not\n"
<< "\t--allow_fp16, -f: [0|1], allow running fp32 models with fp16 or "
"not\n"
<< "\t--count, -c: loop interpreter->Invoke() for certain times\n"
<< "\t--gl_backend, -g: [0|1]: use GL GPU Delegate on Android\n"
<< "\t--hexagon_delegate, -j: [0|1]: use Hexagon Delegate on Android\n"
<< "\t--input_mean, -b: input mean\n"
<< "\t--input_std, -s: input standard deviation\n"
<< "\t--image, -i: image_name.bmp\n"
<< "\t--labels, -l: labels for the model\n"
<< "\t--tflite_model, -m: model_name.tflite\n"
<< "\t--profiling, -p: [0|1], profiling or not\n"
<< "\t--num_results, -r: number of results to show\n"
<< "\t--threads, -t: number of threads\n"
<< "\t--verbose, -v: [0|1] print more information\n"
<< "\t--warmup_runs, -w: number of warmup runs\n"
<< "\t--xnnpack_delegate, -x [0:1]: xnnpack delegate\n"
<< "\t--help, -h: Print this help message\n";
}
int Main(int argc, char** argv) {
DelegateProviders delegate_providers;
bool parse_result = delegate_providers.InitFromCmdlineArgs(
&argc, const_cast<const char**>(argv));
if (!parse_result) {
display_usage(delegate_providers);
return EXIT_FAILURE;
}
Settings s;
int c;
while (true) {
static struct option long_options[] = {
{"accelerated", required_argument, nullptr, 'a'},
{"allow_fp16", required_argument, nullptr, 'f'},
{"count", required_argument, nullptr, 'c'},
{"verbose", required_argument, nullptr, 'v'},
{"image", required_argument, nullptr, 'i'},
{"labels", required_argument, nullptr, 'l'},
{"tflite_model", required_argument, nullptr, 'm'},
{"profiling", required_argument, nullptr, 'p'},
{"threads", required_argument, nullptr, 't'},
{"input_mean", required_argument, nullptr, 'b'},
{"input_std", required_argument, nullptr, 's'},
{"num_results", required_argument, nullptr, 'r'},
{"max_profiling_buffer_entries", required_argument, nullptr, 'e'},
{"warmup_runs", required_argument, nullptr, 'w'},
{"gl_backend", required_argument, nullptr, 'g'},
{"hexagon_delegate", required_argument, nullptr, 'j'},
{"xnnpack_delegate", required_argument, nullptr, 'x'},
{"help", no_argument, nullptr, 'h'},
{nullptr, 0, nullptr, 0}};
/* getopt_long stores the option index here. */
int option_index = 0;
c = getopt_long(argc, argv, "a:b:c:d:e:f:g:i:j:l:m:p:r:s:t:v:w:x:h",
long_options, &option_index);
/* Detect the end of the options. */
if (c == -1) break;
switch (c) {
case 'a':
s.accel = strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'b':
s.input_mean = strtod(optarg, nullptr);
break;
case 'c':
s.loop_count =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'e':
s.max_profiling_buffer_entries =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'f':
s.allow_fp16 =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'g':
s.gl_backend =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'i':
s.input_bmp_name = optarg;
break;
case 'j':
s.hexagon_delegate = optarg;
break;
case 'l':
s.labels_file_name = optarg;
break;
case 'm':
s.model_name = optarg;
break;
case 'p':
s.profiling =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'r':
s.number_of_results =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 's':
s.input_std = strtod(optarg, nullptr);
break;
case 't':
s.number_of_threads = strtol( // NOLINT(runtime/deprecated_fn)
optarg, nullptr, 10);
break;
case 'v':
s.verbose =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'w':
s.number_of_warmup_runs =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'x':
s.xnnpack_delegate =
strtol(optarg, nullptr, 10); // NOLINT(runtime/deprecated_fn)
break;
case 'h':
case '?':
/* getopt_long already printed an error message. */
display_usage(delegate_providers);
exit(-1);
default:
exit(-1);
}
}
delegate_providers.MergeSettingsIntoParams(s);
RunInference(&s, delegate_providers);
return 0;
}
} // namespace label_image
} // namespace tflite
int main(int argc, char** argv) {
return tflite::label_image::Main(argc, argv);
}
tensorflow/lite/examples/label_image/label_image.h
+52
View File
@@ -0,0 +1,52 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_LABEL_IMAGE_H_
#define TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_LABEL_IMAGE_H_
#include "tensorflow/lite/c/c_api_types.h"
#include "tensorflow/lite/model.h"
#include "tensorflow/lite/model_builder.h"
#include "tensorflow/lite/string_type.h"
namespace tflite {
namespace label_image {
struct Settings {
bool verbose = false;
bool accel = false;
TfLiteType input_type = kTfLiteFloat32;
bool profiling = false;
bool allow_fp16 = false;
bool gl_backend = false;
bool hexagon_delegate = false;
bool xnnpack_delegate = false;
int loop_count = 1;
float input_mean = 127.5f;
float input_std = 127.5f;
string model_name = "./mobilenet_quant_v1_224.tflite";
tflite::FlatBufferModel* model;
string input_bmp_name = "./grace_hopper.bmp";
string labels_file_name = "./labels.txt";
int number_of_threads = 4;
int number_of_results = 5;
int max_profiling_buffer_entries = 1024;
int number_of_warmup_runs = 2;
};
} // namespace label_image
} // namespace tflite
#endif // TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_LABEL_IMAGE_H_
tensorflow/lite/examples/label_image/label_image_test.cc
+60
View File
@@ -0,0 +1,60 @@
/* Copyright 2016 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/lite/examples/label_image/label_image.h"
#include <cstdint>
#include <string>
#include <utility>
#include <vector>
#include <gtest/gtest.h>
#include "tensorflow/lite/c/c_api_types.h"
#include "tensorflow/lite/examples/label_image/bitmap_helpers.h"
#include "tensorflow/lite/examples/label_image/get_top_n.h"
namespace tflite {
namespace label_image {
TEST(LabelImageTest, GraceHopper) {
std::string lena_file =
"tensorflow/lite/examples/label_image/testdata/"
"grace_hopper.bmp";
int height, width, channels;
Settings s;
s.input_type = kTfLiteUInt8;
std::vector<uint8_t> input =
read_bmp(lena_file, &width, &height, &channels, &s);
ASSERT_EQ(height, 606);
ASSERT_EQ(width, 517);
ASSERT_EQ(channels, 3);
std::vector<uint8_t> output(606 * 517 * 3);
resize<uint8_t>(output.data(), input.data(), 606, 517, 3, 214, 214, 3, &s);
ASSERT_EQ(output[0], 0x15);
ASSERT_EQ(output[214 * 214 * 3 - 1], 0x11);
}
TEST(LabelImageTest, GetTopN) {
uint8_t in[] = {1, 1, 2, 2, 4, 4, 16, 32, 128, 64};
std::vector<std::pair<float, int>> top_results;
get_top_n<uint8_t>(in, 10, 5, 0.025, &top_results, kTfLiteUInt8);
ASSERT_EQ(top_results.size(), 4);
ASSERT_EQ(top_results[0].second, 8);
}
} // namespace label_image
} // namespace tflite
tensorflow/lite/examples/label_image/log.h
+39
View File
@@ -0,0 +1,39 @@
/* Copyright 2017 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_LOG_H_
#define TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_LOG_H_
#include <iostream>
#include <sstream>
namespace tflite {
namespace label_image {
class Log {
std::stringstream stream_;
public:
explicit Log(const char* severity) { stream_ << severity << ": "; }
std::stringstream& Stream() { return stream_; }
~Log() { std::cerr << stream_.str() << std::endl; }
};
#define LOG(severity) tflite::label_image::Log(#severity).Stream()
} // namespace label_image
} // namespace tflite
#endif // TENSORFLOW_LITE_EXAMPLES_LABEL_IMAGE_LOG_H_
tensorflow/lite/examples/label_image/testdata/grace_hopper.bmp
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 919 KiB

tensorflow/lite/examples/minimal/BUILD
+30
View File
@@ -0,0 +1,30 @@
# Description:
# TensorFlow Lite minimal example.
load("@rules_cc//cc:cc_binary.bzl", "cc_binary")
load("//tensorflow/lite:build_def.bzl", "tflite_linkopts")
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:LICENSE"],
default_visibility = ["//visibility:public"],
licenses = ["notice"],
)
cc_binary(
name = "minimal",
srcs = [
"minimal.cc",
],
linkopts = tflite_linkopts() + select({
"//tensorflow:android": [
"-pie", # Android 5.0 and later supports only PIE
"-lm", # some builtin ops, e.g., tanh, need -lm
],
"//conditions:default": [],
}),
deps = [
"//tensorflow/lite:framework",
"//tensorflow/lite/core:cc_api_stable",
"//tensorflow/lite/kernels:builtin_ops",
],
)
tensorflow/lite/examples/minimal/CMakeLists.txt
+60
View File
@@ -0,0 +1,60 @@
#
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Builds the minimal Tensorflow Lite example.
cmake_minimum_required(VERSION 3.16)
project(minimal C CXX)
option(LINK_TFLITE_FLEX "Enable tensorflowlite_flex library linkage" OFF)
set(TENSORFLOW_SOURCE_DIR "" CACHE PATH
"Directory that contains the TensorFlow project"
)
if(NOT TENSORFLOW_SOURCE_DIR)
get_filename_component(TENSORFLOW_SOURCE_DIR
"${CMAKE_CURRENT_LIST_DIR}/../../../../"
ABSOLUTE
)
endif()
add_subdirectory(
"${TENSORFLOW_SOURCE_DIR}/tensorflow/lite"
"${CMAKE_CURRENT_BINARY_DIR}/tensorflow-lite"
EXCLUDE_FROM_ALL
)
set(CMAKE_CXX_STANDARD 17)
add_executable(minimal
minimal.cc
)
if(LINK_TFLITE_FLEX)
find_library(TF_LIB_FLEX tensorflowlite_flex HINTS "${TENSORFLOW_SOURCE_DIR}/bazel-bin/tensorflow/lite/delegates/flex/")
if(NOT TF_LIB_FLEX)
message(FATAL_ERROR "tensorflowlite_flex library not found")
endif()
target_link_libraries(minimal
tensorflow-lite
-Wl,--no-as-needed # Add --no-as-needed since for some toolchains (e.g. Ubuntu) --as-needed is added by default.
${TF_LIB_FLEX}
)
else()
target_link_libraries(minimal
tensorflow-lite
)
endif()
tensorflow/lite/examples/minimal/README.md
+74
View File
@@ -0,0 +1,74 @@
# TensorFlow Lite C++ minimal example
This example shows how you can build a simple TensorFlow Lite application.
#### Step 1. Install CMake tool
It requires CMake 3.16 or higher. On Ubuntu, you can simply run the following
command.
```sh
sudo apt-get install cmake
```
Or you can follow
[the official cmake installation guide](https://cmake.org/install/)
#### Step 2. Install libffi7 package(Optional)
It requires libffi7. On Ubuntu 20.10 or later, you can simply run the following
command.
```sh
wget http://es.archive.ubuntu.com/ubuntu/pool/main/libf/libffi/libffi7_3.3-4_amd64.deb
sudo dpkg -i libffi7_3.3-4_amd64.deb
```
#### Step 3. Clone TensorFlow repository
```sh
git clone https://github.com/tensorflow/tensorflow.git tensorflow_src
```
#### Step 4. Create CMake build directory and run CMake tool
```sh
mkdir minimal_build
cd minimal_build
cmake ../tensorflow_src/tensorflow/lite/examples/minimal
```
#### Step 5. Build TensorFlow Lite
In the minimal_build directory,
```sh
cmake --build . -j
```
#### Step 5. Run the executable
In the minimal_build directory,
```sh
./minimal <path/to/tflite/model>
```
#### Optional: Link with tensorflowlite_flex library
You may want to link with tensorflowlite_flex library to use TF select Ops in
your model.
First tensorflowlite_flex needs to be compiled using bazel in tensorflow_src
directory: `sh bazel build -c opt --cxxopt='--std=c++17' --config=monolithic
tensorflow/lite/delegates/flex:tensorflowlite_flex`
Then when configuring cmake build
([Step 4](#step-4-create-cmake-build-directory-and-run-cmake-tool)), add the
following option:
```sh
cmake ../tensorflow_src/tensorflow/lite/examples/minimal -DLINK_TFLITE_FLEX="ON"
```
And build: `sh cmake --build . -j`
tensorflow/lite/examples/minimal/minimal.cc
+84
View File
@@ -0,0 +1,84 @@
/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include <cstdio>
#include <cstdlib>
#include <memory>
#include "tensorflow/lite/core/interpreter_builder.h"
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model_builder.h"
#include "tensorflow/lite/optional_debug_tools.h"
// This is an example that is minimal to read a model
// from disk and perform inference. There is no data being loaded
// that is up to you to add as a user.
//
// NOTE: Do not add any dependencies to this that cannot be built with
// the minimal makefile. This example must remain trivial to build with
// the minimal build tool.
//
// Usage: minimal <tflite model>
#define TFLITE_MINIMAL_CHECK(x) \
if (!(x)) { \
fprintf(stderr, "Error at %s:%d\n", __FILE__, __LINE__); \
exit(1); \
}
int main(int argc, char* argv[]) {
if (argc != 2) {
fprintf(stderr, "minimal <tflite model>\n");
return 1;
}
const char* filename = argv[1];
// Load model
std::unique_ptr<tflite::FlatBufferModel> model =
tflite::FlatBufferModel::BuildFromFile(filename);
TFLITE_MINIMAL_CHECK(model != nullptr);
// Build the interpreter with the InterpreterBuilder.
// Note: all Interpreters should be built with the InterpreterBuilder,
// which allocates memory for the Interpreter and does various set up
// tasks so that the Interpreter can read the provided model.
tflite::ops::builtin::BuiltinOpResolver resolver;
tflite::InterpreterBuilder builder(*model, resolver);
std::unique_ptr<tflite::Interpreter> interpreter;
builder(&interpreter);
TFLITE_MINIMAL_CHECK(interpreter != nullptr);
// Allocate tensor buffers.
TFLITE_MINIMAL_CHECK(interpreter->AllocateTensors() == kTfLiteOk);
printf("=== Pre-invoke Interpreter State ===\n");
tflite::PrintInterpreterState(interpreter.get());
// Fill input buffers
// TODO(user): Insert code to fill input tensors.
// Note: The buffer of the input tensor with index `i` of type T can
// be accessed with `T* input = interpreter->typed_input_tensor<T>(i);`
// Run inference
TFLITE_MINIMAL_CHECK(interpreter->Invoke() == kTfLiteOk);
printf("\n\n=== Post-invoke Interpreter State ===\n");
tflite::PrintInterpreterState(interpreter.get());
// Read output buffers
// TODO(user): Insert getting data out code.
// Note: The buffer of the output tensor with index `i` of type T can
// be accessed with `T* output = interpreter->typed_output_tensor<T>(i);`
return 0;
}
tensorflow/lite/examples/python/BUILD
+22
View File
@@ -0,0 +1,22 @@
load("@xla//third_party/rules_python/python:py_binary.bzl", "py_binary")
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:LICENSE"],
licenses = ["notice"],
)
# Commented out under the (b/279852433) because caused an error in the OSS
# TODO(zhurakovskyi): Uncomment when fixed.
# copybara:uncomment_begin
# py_binary(
# name = "label_image",
# srcs = ["label_image.py"],
# main = "label_image.py",
# strict_deps = True,
# deps = [
# "//third_party/py/PIL:pil",
# "//third_party/py/numpy",
# "//tensorflow/lite/python:lite",
# ],
# )
# copybara:uncomment_end
tensorflow/lite/examples/python/README.md
+47
View File
@@ -0,0 +1,47 @@
# TensorFlow Lite Python image classification demo
This `label_image.py` script shows how you can load a pre-trained and converted
TensorFlow Lite model and use it to recognize objects in images. The Python
script accepts arguments specifying the model to use, the corresponding labels
file, and the image to process.
**Tip:** If you're using a Raspberry Pi, instead try the
[classify_picamera.py example](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/raspberry_pi).
Before you begin, make sure you
[have TensorFlow installed](https://www.tensorflow.org/install).
## Download sample model and image
You can use any compatible model, but the following MobileNet v1 model offers a
good demonstration of a model trained to recognize 1,000 different objects.
```sh
# Get photo
curl https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/examples/label_image/testdata/grace_hopper.bmp > /tmp/grace_hopper.bmp
# Get model
curl https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_02_22/mobilenet_v1_1.0_224.tgz | tar xzv -C /tmp
# Get labels
curl https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz | tar xzv -C /tmp mobilenet_v1_1.0_224/labels.txt
mv /tmp/mobilenet_v1_1.0_224/labels.txt /tmp/
```
## Run the sample
```sh
python3 label_image.py \
--model_file /tmp/mobilenet_v1_1.0_224.tflite \
--label_file /tmp/labels.txt \
--image /tmp/grace_hopper.bmp
```
You should see results like this:
```
0.728693: military uniform
0.116163: Windsor tie
0.035517: bow tie
0.014874: mortarboard
0.011758: bolo tie
```
tensorflow/lite/examples/python/label_image.py
+128
View File
@@ -0,0 +1,128 @@
# Copyright 2018 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""label_image for tflite."""
import argparse
import time
import numpy as np
from PIL import Image
from tensorflow.lite.python import lite
def load_labels(filename):
with open(filename, 'r') as f:
return [line.strip() for line in f.readlines()]
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument(
'-i',
'--image',
default='/tmp/grace_hopper.bmp',
help='image to be classified')
parser.add_argument(
'-m',
'--model_file',
default='/tmp/mobilenet_v1_1.0_224_quant.tflite',
help='.tflite model to be executed')
parser.add_argument(
'-l',
'--label_file',
default='/tmp/labels.txt',
help='name of file containing labels')
parser.add_argument(
'--input_mean',
default=127.5, type=float,
help='input_mean')
parser.add_argument(
'--input_std',
default=127.5, type=float,
help='input standard deviation')
parser.add_argument(
'--num_threads', default=None, type=int, help='number of threads')
parser.add_argument(
'-e', '--ext_delegate', help='external_delegate_library path')
parser.add_argument(
'-o',
'--ext_delegate_options',
help='external delegate options, \
format: "option1: value1; option2: value2"')
args = parser.parse_args()
ext_delegate = None
ext_delegate_options = {}
# parse extenal delegate options
if args.ext_delegate_options is not None:
options = args.ext_delegate_options.split(';')
for o in options:
kv = o.split(':')
if (len(kv) == 2):
ext_delegate_options[kv[0].strip()] = kv[1].strip()
else:
raise RuntimeError('Error parsing delegate option: ' + o)
# load external delegate
if args.ext_delegate is not None:
print('Loading external delegate from {} with args: {}'.format(
args.ext_delegate, ext_delegate_options))
ext_delegate = [
tflite.load_delegate(args.ext_delegate, ext_delegate_options)
]
interpreter = lite.Interpreter(
model_path=args.model_file,
experimental_delegates=ext_delegate,
num_threads=args.num_threads)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# check the type of the input tensor
floating_model = input_details[0]['dtype'] == np.float32
# NxHxWxC, H:1, W:2
height = input_details[0]['shape'][1]
width = input_details[0]['shape'][2]
img = Image.open(args.image).resize((width, height))
# add N dim
input_data = np.expand_dims(img, axis=0)
if floating_model:
input_data = (np.float32(input_data) - args.input_mean) / args.input_std
interpreter.set_tensor(input_details[0]['index'], input_data)
start_time = time.time()
interpreter.invoke()
stop_time = time.time()
output_data = interpreter.get_tensor(output_details[0]['index'])
results = np.squeeze(output_data)
top_k = results.argsort()[-5:][::-1]
labels = load_labels(args.label_file)
for i in top_k:
if floating_model:
print('{:08.6f}: {}'.format(float(results[i]), labels[i]))
else:
print('{:08.6f}: {}'.format(float(results[i] / 255.0), labels[i]))
print('time: {:.3f}ms'.format((stop_time - start_time) * 1000))