wehub/tensorflow--tensorflow
1
0
Fork 0
Code Issues Pull Requests Actions 3 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
cffconvert / validate (push) Has been skipped
Details
License Check / license-check (push) Failing after 2s
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:14:16 +08:00
commit 8a852e4b4e
36502 changed files with 9277225 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tensorflow/lite/g3doc/android/delegates/hexagon.md
+312
View File
@@ -0,0 +1,312 @@
# TensorFlow Lite Hexagon delegate
<aside class="warning">
<p><b>Warning:</b> The
<a href="https://www.tensorflow.org/lite/android/delegates/nnapi">
NNAPI</a> and <a href="https://www.tensorflow.org/lite/android/delegates/hexagon">
Hexagon</a> delegates are deprecated and no longer supported by TensorFlow
Lite. For more information, see the
<a href="https://developer.android.com/ndk/guides/neuralnetworks/migration-guide">
NNAPI Migration Guide</a> and
<a href="https://www.tensorflow.org/lite/performance/delegates">TF Lite
delegates documentation</a>.</p>
</aside>
This document explains how to use the TensorFlow Lite Hexagon Delegate in your
application using the Java and/or C API. The delegate leverages the Qualcomm
Hexagon library to execute quantized kernels on the DSP. Note that the delegate
is intended to *complement* NNAPI functionality, particularly for devices where
NNAPI DSP acceleration is unavailable (e.g., on older devices, or devices that
don’t yet have a DSP NNAPI driver).
**Supported devices:**
Currently the following Hexagon architecture are supported, including but not
limited to:
* Hexagon 680
* SoC examples: Snapdragon 821, 820, 660
* Hexagon 682
* SoC examples: Snapdragon 835
* Hexagon 685
* SoC examples: Snapdragon 845, Snapdragon 710, QCS410, QCS610, QCS605,
QCS603
* Hexagon 690
* SoC examples: Snapdragon 855, RB5
**Supported models:**
The Hexagon delegate supports all models that conform to our
[8-bit symmetric quantization spec](https://www.tensorflow.org/lite/performance/quantization_spec),
including those generated using
[post-training integer quantization](https://www.tensorflow.org/lite/performance/post_training_integer_quant).
UInt8 models trained with the legacy
[quantization-aware training](https://github.com/tensorflow/tensorflow/tree/r1.13/tensorflow/contrib/quantize)
path are also supported, for example,
[these quantized versions](https://www.tensorflow.org/lite/guide/hosted_models#quantized_models)
on our Hosted Models page.
## Hexagon delegate Java API
```java
public class HexagonDelegate implements Delegate, Closeable {
/*
* Creates a new HexagonDelegate object given the current 'context'.
* Throws UnsupportedOperationException if Hexagon DSP delegation is not
* available on this device.
*/
public HexagonDelegate(Context context) throws UnsupportedOperationException
/**
* Frees TFLite resources in C runtime.
*
* User is expected to call this method explicitly.
*/
@Override
public void close();
}
```
### Example usage
#### Step 1. Edit app/build.gradle to use the nightly Hexagon delegate AAR
```
dependencies {
...
implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT'
implementation 'org.tensorflow:tensorflow-lite-hexagon:0.0.0-nightly-SNAPSHOT'
}
```
#### Step 2. Add Hexagon libraries to your Android app {:#hexagon_versions}
* Download and run hexagon_nn_skel.run. It should provide 3 different shared
libraries “libhexagon_nn_skel.so”, “libhexagon_nn_skel_v65.so”,
“libhexagon_nn_skel_v66.so”
* [v1.10.3](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_1_10_3_1.run)
* [v1.14](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.14.run)
* [v1.17](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.17.0.0.run)
* [v1.20](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.20.0.0.run)
* [v1.20.0.1](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.20.0.1.run)
Caution: The currently released versions of the Hexagon delegate, up to version
1.20.0.1, are no longer supported. An updated version of this delegate is
expected soon.
Note: You must accept the license agreement before using the delegate.
Note: You must use the hexagon_nn libraries with the compatible version of
interface library. Interface library is part of the AAR and fetched by bazel
through the
[config](https://github.com/tensorflow/tensorflow/blob/master/third_party/hexagon/workspace.bzl)
The version in the bazel config is the version you should use.
* Include all 3 in your app with other shared libraries. See
[How to add shared library to your app](#how-to-add-shared-library-to-your-app).
The delegate will automatically pick the one with best performance depending
on the device.
Note: If your app will be built for both 32 and 64-bit ARM devices, then
add the Hexagon shared libs to both 32 and 64-bit lib folders.
#### Step 3. Create a delegate and initialize a TensorFlow Lite Interpreter
```java
import org.tensorflow.lite.HexagonDelegate;
// Create the Delegate instance.
try {
hexagonDelegate = new HexagonDelegate(activity);
tfliteOptions.addDelegate(hexagonDelegate);
} catch (UnsupportedOperationException e) {
// Hexagon delegate is not supported on this device.
}
tfliteInterpreter = new Interpreter(tfliteModel, tfliteOptions);
// Dispose after finished with inference.
tfliteInterpreter.close();
if (hexagonDelegate != null) {
hexagonDelegate.close();
}
```
## Hexagon delegate C API
```c
struct TfLiteHexagonDelegateOptions {
// This corresponds to the debug level in the Hexagon SDK. 0 (default)
// means no debug.
int debug_level;
// This corresponds to powersave_level in the Hexagon SDK.
// where 0 (default) means high performance which means more power
// consumption.
int powersave_level;
// If set to true, performance information about the graph will be dumped
// to Standard output, this includes cpu cycles.
// WARNING: Experimental and subject to change anytime.
bool print_graph_profile;
// If set to true, graph structure will be dumped to Standard output.
// This is usually beneficial to see what actual nodes executed on
// the DSP. Combining with 'debug_level' more information will be printed.
// WARNING: Experimental and subject to change anytime.
bool print_graph_debug;
};
// Return a delegate that uses Hexagon SDK for ops execution.
// Must outlive the interpreter.
TfLiteDelegate*
TfLiteHexagonDelegateCreate(const TfLiteHexagonDelegateOptions* options);
// Do any needed cleanup and delete 'delegate'.
void TfLiteHexagonDelegateDelete(TfLiteDelegate* delegate);
// Initializes the DSP connection.
// This should be called before doing any usage of the delegate.
// "lib_directory_path": Path to the directory which holds the
// shared libraries for the Hexagon NN libraries on the device.
void TfLiteHexagonInitWithPath(const char* lib_directory_path);
// Same as above method but doesn't accept the path params.
// Assumes the environment setup is already done. Only initialize Hexagon.
Void TfLiteHexagonInit();
// Clean up and switch off the DSP connection.
// This should be called after all processing is done and delegate is deleted.
Void TfLiteHexagonTearDown();
```
### Example usage
#### Step 1. Edit app/build.gradle to use the nightly Hexagon delegate AAR
```
dependencies {
...
implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT'
implementation 'org.tensorflow:tensorflow-lite-hexagon:0.0.0-nightly-SNAPSHOT'
}
```
#### Step 2. Add Hexagon libraries to your Android app
* Download and run hexagon_nn_skel.run. It should provide 3 different shared
libraries “libhexagon_nn_skel.so”, “libhexagon_nn_skel_v65.so”,
“libhexagon_nn_skel_v66.so”
* [v1.10.3](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_1_10_3_1.run)
* [v1.14](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.14.run)
* [v1.17](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.17.0.0.run)
* [v1.20](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.20.0.0.run)
* [v1.20.0.1](https://storage.cloud.google.com/download.tensorflow.org/tflite/hexagon_nn_skel_v1.20.0.1.run)
Caution: The currently released versions of the Hexagon delegate, up to version
1.20.0.1, are no longer supported. An updated version of this delegate is
expected soon.
Note: You must accept the license agreement before using the delegate.
Note: You must use the hexagon_nn libraries with the compatible version of
interface library. Interface library is part of the AAR and fetched by bazel
through the
[config](https://github.com/tensorflow/tensorflow/blob/master/third_party/hexagon/workspace.bzl).
The version in the bazel config is the version you should use.
* Include all 3 in your app with other shared libraries. See
[How to add shared library to your app](#how-to-add-shared-library-to-your-app).
The delegate will automatically pick the one with best performance depending
on the device.
Note: If your app will be built for both 32 and 64-bit ARM devices, then
add the Hexagon shared libs to both 32 and 64-bit lib folders.
#### Step 3. Include the C header
* The header file "hexagon_delegate.h" can be downloaded from
[GitHub](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/hexagon/hexagon_delegate.h)
or extracted from the Hexagon delegate AAR.
#### Step 4. Create a delegate and initialize a TensorFlow Lite Interpreter
* In your code, ensure the native Hexagon library is loaded. This can be done
by calling `System.loadLibrary("tensorflowlite_hexagon_jni");` \
in your Activity or Java entry-point.
* Create a delegate, example:
```c
#include "tensorflow/lite/delegates/hexagon/hexagon_delegate.h"
// Assuming shared libraries are under "/data/local/tmp/"
// If files are packaged with native lib in android App then it
// will typically be equivalent to the path provided by
// "getContext().getApplicationInfo().nativeLibraryDir"
const char[] library_directory_path = "/data/local/tmp/";
TfLiteHexagonInitWithPath(library_directory_path); // Needed once at startup.
::tflite::TfLiteHexagonDelegateOptions params = {0};
// 'delegate_ptr' Need to outlive the interpreter. For example,
// If your use case requires resizing the input or anything that can trigger
// re-applying delegates then 'delegate_ptr' must outlive the interpreter.
auto* delegate_ptr = ::tflite::TfLiteHexagonDelegateCreate(&params);
Interpreter::TfLiteDelegatePtr delegate(delegate_ptr,
[](TfLiteDelegate* delegate) {
::tflite::TfLiteHexagonDelegateDelete(delegate);
});
interpreter->ModifyGraphWithDelegate(delegate.get());
// After usage of delegate.
TfLiteHexagonTearDown(); // Needed once at end of app/DSP usage.
```
## Add the shared library to your app
* Create folder “app/src/main/jniLibs”, and create a directory for each target
architecture. For example,
* ARM 64-bit: `app/src/main/jniLibs/arm64-v8a`
* ARM 32-bit: `app/src/main/jniLibs/armeabi-v7a`
* Put your .so in the directory that match the architecture.
Note: If you're using App Bundle for publishing your Application, you might want
to set android.bundle.enableUncompressedNativeLibs=false in the
gradle.properties file.
## Feedback
For issues, please create a
[GitHub](https://github.com/tensorflow/tensorflow/issues/new?template=50-other-issues.md)
issue with all the necessary repro details, including the phone model and board
used (`adb shell getprop ro.product.device` and `adb shell getprop
ro.board.platform`).
## FAQ
* Which ops are supported by the delegate?
* See the current list of
[supported ops and constraints](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/hexagon/README.md)
* How can I tell that the model is using the DSP when I enable the delegate?
* Two log messages will be printed when you enable the delegate - one to
indicate if the delegate was created and another to indicate how many
nodes are running using the delegate. \
`Created TensorFlow Lite delegate for Hexagon.` \
`Hexagon delegate: X nodes delegated out of Y nodes.`
* Do I need all Ops in the model to be supported to run the delegate?
* No, the Model will be partitioned into subgraphs based on the supported
ops. Any unsupported ops will run on the CPU.
* How can I build the Hexagon delegate AAR from source?
* Use `bazel build -c opt --config=android_arm64
tensorflow/lite/delegates/hexagon/java:tensorflow-lite-hexagon`.
* Why does Hexagon delegate fail to initialize although my Android device has
a supported SoC?
* Verify if your device indeed has a supported SoC. Run `adb shell cat
/proc/cpuinfo | grep Hardware` and see if it returns something like
"Hardware : Qualcomm Technologies, Inc MSMXXXX".
* Some phone manufacturers use different SoCs for the same phone model.
Therefore, Hexagon delegate may only work on some but not all devices of
the same phone model.
* Some phone manufactures intentionally restrict the use of Hexagon DSP
from non-system Android apps, making the Hexagon delegate unable to
work.
* My phone has locked DSP access. I rooted the phone and still can't run the
delegate, what to do ?
* Make sure to disable SELinux enforce by running `adb shell setenforce 0`