chore: import upstream snapshot with attribution

tensorflow/lite/g3doc/inference_with_metadata/codegen.md

@@ -0,0 +1,262 @@
+# Generate model interfaces using metadata
+
+Using [TensorFlow Lite Metadata](../models/convert/metadata), developers can generate
+wrapper code to enable integration on Android. For most developers, the
+graphical interface of [Android Studio ML Model Binding](#mlbinding) is the
+easiest to use. If you require more customisation or are using command line
+tooling, the [TensorFlow Lite Codegen](#codegen) is also available.
+
+## Use Android Studio ML Model Binding {:#mlbinding}
+
+For TensorFlow Lite models enhanced with [metadata](../models/convert/metadata.md),
+developers can use Android Studio ML Model Binding to automatically configure
+settings for the project and generate wrapper classes based on the model
+metadata. The wrapper code removes the need to interact directly with
+`ByteBuffer`. Instead, developers can interact with the TensorFlow Lite model
+with typed objects such as `Bitmap` and `Rect`.
+
+Note: Required [Android Studio 4.1](https://developer.android.com/studio) or
+above
+
+### Import a TensorFlow Lite model in Android Studio
+
+1.  Right-click on the module you would like to use the TFLite model or click on
+    `File`, then `New` > `Other` > `TensorFlow Lite Model`
+
+1.  Select the location of your TFLite file. Note that the tooling will
+    configure the module's dependency on your behalf with ML Model binding and
+    all dependencies automatically inserted into your Android module's
+    `build.gradle` file.
+
+    Optional: Select the second checkbox for importing TensorFlow GPU if you
+    want to use GPU acceleration.
+
+1.  Click `Finish`.
+
+1.  The following screen will appear after the import is successful. To start
+    using the model, select Kotlin or Java, copy and paste the code under the
+    `Sample Code` section. You can get back to this screen by double clicking
+    the TFLite model under the `ml` directory in Android Studio.
+
+### Accelerating model inference {:#acceleration}
+
+ML Model Binding provides a way for developers to accelerate their code through
+the use of delegates and the number of threads.
+
+Note: The TensorFlow Lite Interpreter must be created on the same thread as when
+is run. Otherwise, TfLiteGpuDelegate Invoke: GpuDelegate must run on the same
+thread where it was initialized. may occur.
+
+Step 1. Check the module `build.gradle` file that it contains the following
+dependency:
+
+```java
+    dependencies {
+        ...
+        // TFLite GPU delegate 2.3.0 or above is required.
+        implementation 'org.tensorflow:tensorflow-lite-gpu:2.3.0'
+    }
+```
+
+Step 2. Detect if GPU running on the device is compatible with TensorFlow GPU
+delegate, if not run the model using multiple CPU threads:
+
+<div>
+    <devsite-selector>
+    <section>
+      <h3>Kotlin</h3>
+      <p><pre class="prettyprint lang-kotlin">
+    import org.tensorflow.lite.gpu.CompatibilityList
+    import org.tensorflow.lite.gpu.GpuDelegate
+
+    val compatList = CompatibilityList()
+
+    val options = if(compatList.isDelegateSupportedOnThisDevice) {
+        // if the device has a supported GPU, add the GPU delegate
+        Model.Options.Builder().setDevice(Model.Device.GPU).build()
+    } else {
+        // if the GPU is not supported, run on 4 threads
+        Model.Options.Builder().setNumThreads(4).build()
+    }
+
+    // Initialize the model as usual feeding in the options object
+    val myModel = MyModel.newInstance(context, options)
+
+    // Run inference per sample code
+      </pre></p>
+    </section>
+    <section>
+      <h3>Java</h3>
+      <p><pre class="prettyprint lang-java">
+    import org.tensorflow.lite.support.model.Model
+    import org.tensorflow.lite.gpu.CompatibilityList;
+    import org.tensorflow.lite.gpu.GpuDelegate;
+
+    // Initialize interpreter with GPU delegate
+    Model.Options options;
+    CompatibilityList compatList = CompatibilityList();
+
+    if(compatList.isDelegateSupportedOnThisDevice()){
+        // if the device has a supported GPU, add the GPU delegate
+        options = Model.Options.Builder().setDevice(Model.Device.GPU).build();
+    } else {
+        // if the GPU is not supported, run on 4 threads
+        options = Model.Options.Builder().setNumThreads(4).build();
+    }
+
+    MyModel myModel = new MyModel.newInstance(context, options);
+
+    // Run inference per sample code
+      </pre></p>
+    </section>
+    </devsite-selector>
+</div>
+
+## Generate model interfaces with TensorFlow Lite code generator {:#codegen}
+
+Note: TensorFlow Lite wrapper code generator currently only supports Android.
+
+For TensorFlow Lite model enhanced with [metadata](../models/convert/metadata.md),
+developers can use the TensorFlow Lite Android wrapper code generator to create
+platform specific wrapper code. The wrapper code removes the need to interact
+directly with `ByteBuffer`. Instead, developers can interact with the TensorFlow
+Lite model with typed objects such as `Bitmap` and `Rect`.
+
+The usefulness of the code generator depend on the completeness of the
+TensorFlow Lite model's metadata entry. Refer to the `<Codegen usage>` section
+under relevant fields in
+[metadata_schema.fbs](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/metadata/metadata_schema.fbs),
+to see how the codegen tool parses each field.
+
+### Generate wrapper Code
+
+You will need to install the following tooling in your terminal:
+
+```sh
+pip install tflite-support
+```
+
+Once completed, the code generator can be used using the following syntax:
+
+```sh
+tflite_codegen --model=./model_with_metadata/mobilenet_v1_0.75_160_quantized.tflite \
+    --package_name=org.tensorflow.lite.classify \
+    --model_class_name=MyClassifierModel \
+    --destination=./classify_wrapper
+```
+
+The resulting code will be located in the destination directory. If you are
+using [Google Colab](https://colab.research.google.com/) or other remote
+environment, it maybe easier to zip up the result in a zip archive and download
+it to your Android Studio project:
+
+```python
+# Zip up the generated code
+!zip -r classify_wrapper.zip classify_wrapper/
+
+# Download the archive
+from google.colab import files
+files.download('classify_wrapper.zip')
+```
+
+### Using the generated code
+
+#### Step 1: Import the generated code
+
+Unzip the generated code if necessary into a directory structure. The root of
+the generated code is assumed to be `SRC_ROOT`.
+
+Open the Android Studio project where you would like to use the TensorFlow lite
+model and import the generated module by: And File -> New -> Import Module ->
+select `SRC_ROOT`
+
+Using the above example, the directory and the module imported would be called
+`classify_wrapper`.
+
+#### Step 2: Update the app's `build.gradle` file
+
+In the app module that will be consuming the generated library module:
+
+Under the android section, add the following:
+
+```build
+aaptOptions {
+   noCompress "tflite"
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.
+
+Under the dependencies section, add the following:
+
+```build
+implementation project(":classify_wrapper")
+```
+
+#### Step 3: Using the model
+
+```java
+// 1. Initialize the model
+MyClassifierModel myImageClassifier = null;
+
+try {
+    myImageClassifier = new MyClassifierModel(this);
+} catch (IOException io){
+    // Error reading the model
+}
+
+if(null != myImageClassifier) {
+
+    // 2. Set the input with a Bitmap called inputBitmap
+    MyClassifierModel.Inputs inputs = myImageClassifier.createInputs();
+    inputs.loadImage(inputBitmap));
+
+    // 3. Run the model
+    MyClassifierModel.Outputs outputs = myImageClassifier.run(inputs);
+
+    // 4. Retrieve the result
+    Map<String, Float> labeledProbability = outputs.getProbability();
+}
+```
+
+### Accelerating model inference
+
+The generated code provides a way for developers to accelerate their code
+through the use of [delegates](../performance/delegates.md) and the number of
+threads. These can be set when initializing the model object as it takes three
+parameters:
+
+*   **`Context`**: Context from the Android Activity or Service
+*   (Optional) **`Device`**: TFLite acceleration delegate for example
+    GPUDelegate or NNAPIDelegate
+*   (Optional) **`numThreads`**: Number of threads used to run the model -
+    default is one.
+
+For example, to use a NNAPI delegate and up to three threads, you can initialize
+the model like this:
+
+```java
+try {
+    myImageClassifier = new MyClassifierModel(this, Model.Device.NNAPI, 3);
+} catch (IOException io){
+    // Error reading the model
+}
+```
+
+### Troubleshooting
+
+If you get a 'java.io.FileNotFoundException: This file can not be opened as a
+file descriptor; it is probably compressed' error, insert the following lines
+under the android section of the app module that will uses the library module:
+
+```build
+aaptOptions {
+   noCompress "tflite"
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.

tensorflow/lite/g3doc/inference_with_metadata/lite_support.md

@@ -0,0 +1,284 @@
+# Process input and output data with the TensorFlow Lite Support Library
+
+Note: TensorFlow Lite Support Library currently only supports Android.
+
+Mobile application developers typically interact with typed objects such as
+bitmaps or primitives such as integers. However, the TensorFlow Lite interpreter
+API that runs the on-device machine learning model uses tensors in the form of
+ByteBuffer, which can be difficult to debug and manipulate. The
+[TensorFlow Lite Android Support Library](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/java)
+is designed to help process the input and output of TensorFlow Lite models, and
+make the TensorFlow Lite interpreter easier to use.
+
+## Getting Started
+
+### Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import tflite dependencies
+    implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT'
+    // The GPU delegate library is optional. Depend on it as needed.
+    implementation 'org.tensorflow:tensorflow-lite-gpu:0.0.0-nightly-SNAPSHOT'
+    implementation 'org.tensorflow:tensorflow-lite-support:0.0.0-nightly-SNAPSHOT'
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.
+
+Explore the
+[TensorFlow Lite Support Library AAR hosted at MavenCentral](https://search.maven.org/artifact/org.tensorflow/tensorflow-lite-support)
+for different versions of the Support Library.
+
+### Basic image manipulation and conversion
+
+The TensorFlow Lite Support Library has a suite of basic image manipulation
+methods such as crop and resize. To use it, create an `ImagePreprocessor` and
+add the required operations. To convert the image into the tensor format
+required by the TensorFlow Lite interpreter, create a `TensorImage` to be used
+as input:
+
+```java
+import org.tensorflow.lite.DataType;
+import org.tensorflow.lite.support.image.ImageProcessor;
+import org.tensorflow.lite.support.image.TensorImage;
+import org.tensorflow.lite.support.image.ops.ResizeOp;
+
+// Initialization code
+// Create an ImageProcessor with all ops required. For more ops, please
+// refer to the ImageProcessor Architecture section in this README.
+ImageProcessor imageProcessor =
+    new ImageProcessor.Builder()
+        .add(new ResizeOp(224, 224, ResizeOp.ResizeMethod.BILINEAR))
+        .build();
+
+// Create a TensorImage object. This creates the tensor of the corresponding
+// tensor type (uint8 in this case) that the TensorFlow Lite interpreter needs.
+TensorImage tensorImage = new TensorImage(DataType.UINT8);
+
+// Analysis code for every frame
+// Preprocess the image
+tensorImage.load(bitmap);
+tensorImage = imageProcessor.process(tensorImage);
+```
+
+`DataType` of a tensor can be read through the
+[metadata extractor library](../models/convert/metadata.md#read-the-metadata-from-models)
+as well as other model information.
+
+### Basic audio data processing
+
+The TensorFlow Lite Support Library also defines a `TensorAudio` class wrapping
+some basic audio data processing methods. It's mostly used together with
+[AudioRecord](https://developer.android.com/reference/android/media/AudioRecord)
+and captures audio samples in a ring buffer.
+
+```java
+import android.media.AudioRecord;
+import org.tensorflow.lite.support.audio.TensorAudio;
+
+// Create an `AudioRecord` instance.
+AudioRecord record = AudioRecord(...)
+
+// Create a `TensorAudio` object from Android AudioFormat.
+TensorAudio tensorAudio = new TensorAudio(record.getFormat(), size)
+
+// Load all audio samples available in the AudioRecord without blocking.
+tensorAudio.load(record)
+
+// Get the `TensorBuffer` for inference.
+TensorBuffer buffer = tensorAudio.getTensorBuffer()
+```
+
+### Create output objects and run the model
+
+Before running the model, we need to create the container objects that will
+store the result:
+
+```java
+import org.tensorflow.lite.DataType;
+import org.tensorflow.lite.support.tensorbuffer.TensorBuffer;
+
+// Create a container for the result and specify that this is a quantized model.
+// Hence, the 'DataType' is defined as UINT8 (8-bit unsigned integer)
+TensorBuffer probabilityBuffer =
+    TensorBuffer.createFixedSize(new int[]{1, 1001}, DataType.UINT8);
+```
+
+Loading the model and running inference:
+
+```java
+import java.nio.MappedByteBuffer;
+import org.tensorflow.lite.InterpreterFactory;
+import org.tensorflow.lite.InterpreterApi;
+
+// Initialise the model
+try{
+    MappedByteBuffer tfliteModel
+        = FileUtil.loadMappedFile(activity,
+            "mobilenet_v1_1.0_224_quant.tflite");
+    InterpreterApi tflite = new InterpreterFactory().create(
+        tfliteModel, new InterpreterApi.Options());
+} catch (IOException e){
+    Log.e("tfliteSupport", "Error reading model", e);
+}
+
+// Running inference
+if(null != tflite) {
+    tflite.run(tImage.getBuffer(), probabilityBuffer.getBuffer());
+}
+```
+
+### Accessing the result
+
+Developers can access the output directly through
+`probabilityBuffer.getFloatArray()`. If the model produces a quantized output,
+remember to convert the result. For the MobileNet quantized model, the developer
+needs to divide each output value by 255 to obtain the probability ranging from
+0 (least likely) to 1 (most likely) for each category.
+
+### Optional: Mapping results to labels
+
+Developers can also optionally map the results to labels. First, copy the text
+file containing labels into the module’s assets directory. Next, load the label
+file using the following code:
+
+```java
+import org.tensorflow.lite.support.common.FileUtil;
+
+final String ASSOCIATED_AXIS_LABELS = "labels.txt";
+List<String> associatedAxisLabels = null;
+
+try {
+    associatedAxisLabels = FileUtil.loadLabels(this, ASSOCIATED_AXIS_LABELS);
+} catch (IOException e) {
+    Log.e("tfliteSupport", "Error reading label file", e);
+}
+```
+
+The following snippet demonstrates how to associate the probabilities with
+category labels:
+
+```java
+import java.util.Map;
+import org.tensorflow.lite.support.common.TensorProcessor;
+import org.tensorflow.lite.support.common.ops.NormalizeOp;
+import org.tensorflow.lite.support.label.TensorLabel;
+
+// Post-processor which dequantize the result
+TensorProcessor probabilityProcessor =
+    new TensorProcessor.Builder().add(new NormalizeOp(0, 255)).build();
+
+if (null != associatedAxisLabels) {
+    // Map of labels and their corresponding probability
+    TensorLabel labels = new TensorLabel(associatedAxisLabels,
+        probabilityProcessor.process(probabilityBuffer));
+
+    // Create a map to access the result based on label
+    Map<String, Float> floatMap = labels.getMapWithFloatValue();
+}
+```
+
+## Current use-case coverage
+
+The current version of the TensorFlow Lite Support Library covers:
+
+*   common data types (float, uint8, images, audio and array of these objects)
+    as inputs and outputs of tflite models.
+*   basic image operations (crop image, resize and rotate).
+*   normalization and quantization
+*   file utils
+
+Future versions will improve support for text-related applications.
+
+## ImageProcessor Architecture
+
+The design of the `ImageProcessor` allowed the image manipulation operations to
+be defined up front and optimised during the build process. The `ImageProcessor`
+currently supports three basic preprocessing operations, as described in the
+three comments in the code snippet below:
+
+```java
+import org.tensorflow.lite.support.common.ops.NormalizeOp;
+import org.tensorflow.lite.support.common.ops.QuantizeOp;
+import org.tensorflow.lite.support.image.ops.ResizeOp;
+import org.tensorflow.lite.support.image.ops.ResizeWithCropOrPadOp;
+import org.tensorflow.lite.support.image.ops.Rot90Op;
+
+int width = bitmap.getWidth();
+int height = bitmap.getHeight();
+
+int size = height > width ? width : height;
+
+ImageProcessor imageProcessor =
+    new ImageProcessor.Builder()
+        // Center crop the image to the largest square possible
+        .add(new ResizeWithCropOrPadOp(size, size))
+        // Resize using Bilinear or Nearest neighbour
+        .add(new ResizeOp(224, 224, ResizeOp.ResizeMethod.BILINEAR));
+        // Rotation counter-clockwise in 90 degree increments
+        .add(new Rot90Op(rotateDegrees / 90))
+        .add(new NormalizeOp(127.5, 127.5))
+        .add(new QuantizeOp(128.0, 1/128.0))
+        .build();
+```
+
+See more details
+[here](../models/convert/metadata.md#normalization-and-quantization-parameters) about
+normalization and quantization.
+
+The eventual goal of the support library is to support all
+[`tf.image`](https://www.tensorflow.org/api_docs/python/tf/image)
+transformations. This means the transformation will be the same as TensorFlow
+and the implementation will be independent of the operating system.
+
+Developers are also welcome to create custom processors. It is important in
+these cases to be aligned with the training process - i.e. the same
+preprocessing should apply to both training and inference to increase
+reproducibility.
+
+## Quantization
+
+When initiating input or output objects such as `TensorImage` or `TensorBuffer`
+you need to specify their types to be `DataType.UINT8` or `DataType.FLOAT32`.
+
+```java
+TensorImage tensorImage = new TensorImage(DataType.UINT8);
+TensorBuffer probabilityBuffer =
+    TensorBuffer.createFixedSize(new int[]{1, 1001}, DataType.UINT8);
+```
+
+The `TensorProcessor` can be used to quantize input tensors or dequantize output
+tensors. For example, when processing a quantized output `TensorBuffer`, the
+developer can use `DequantizeOp` to dequantize the result to a floating point
+probability between 0 and 1:
+
+```java
+import org.tensorflow.lite.support.common.TensorProcessor;
+
+// Post-processor which dequantize the result
+TensorProcessor probabilityProcessor =
+    new TensorProcessor.Builder().add(new DequantizeOp(0, 1/255.0)).build();
+TensorBuffer dequantizedBuffer = probabilityProcessor.process(probabilityBuffer);
+```
+
+The quantization parameters of a tensor can be read through the
+[metadata extractor library](../models/convert/metadata.md#read-the-metadata-from-models).

tensorflow/lite/g3doc/inference_with_metadata/overview.md

@@ -0,0 +1,65 @@
+# TensorFlow Lite inference with metadata
+
+Inferencing [models with metadata](../models/convert/metadata.md) can be as easy as
+just a few lines of code. TensorFlow Lite metadata contains a rich description
+of what the model does and how to use the model. It can empower code generators
+to automatically generate the inference code for you, such as using the
+[Android Studio ML Binding feature](codegen.md#mlbinding) or
+[TensorFlow Lite Android code generator](codegen.md#codegen). It can also be
+used to configure your custom inference pipeline.
+
+## Tools and libraries
+
+TensorFlow Lite provides varieties of tools and libraries to serve different
+tiers of deployment requirements as follows:
+
+### Generate model interface with Android code generators
+
+There are two ways to automatically generate the necessary Android wrapper code
+for TensorFlow Lite model with metadata:
+
+1.  [Android Studio ML Model Binding](codegen.md#mlbinding) is tooling available
+    within Android Studio to import TensorFlow Lite model through a graphical
+    interface. Android Studio will automatically configure settings for the
+    project and generate wrapper classes based on the model metadata.
+
+2.  [TensorFlow Lite Code Generator](codegen.md#codegen) is an executable that
+    generates model interface automatically based on the metadata. It currently
+    supports Android with Java. The wrapper code removes the need to interact
+    directly with `ByteBuffer`. Instead, developers can interact with the
+    TensorFlow Lite model with typed objects such as `Bitmap` and `Rect`.
+    Android Studio users can also get access to the codegen feature through
+    [Android Studio ML Binding](codegen.md#mlbinding).
+
+### Leverage out-of-box APIs with the TensorFlow Lite Task Library
+
+[TensorFlow Lite Task Library](task_library/overview.md) provides optimized
+ready-to-use model interfaces for popular machine learning tasks, such as image
+classification, question and answer, etc. The model interfaces are specifically
+designed for each task to achieve the best performance and usability. Task
+Library works cross-platform and is supported on Java, C++, and Swift.
+
+### Build custom inference pipelines with the TensorFlow Lite Support Library
+
+[TensorFlow Lite Support Library](lite_support.md) is a cross-platform library
+that helps to customize model interface and build inference pipelines. It
+contains varieties of util methods and data structures to perform pre/post
+processing and data conversion. It is also designed to match the behavior of
+TensorFlow modules, such as TF.Image and TF.Text, ensuring consistency from
+training to inferencing.
+
+## Explore pretrained models with metadata
+
+Browse
+[TensorFlow Lite hosted models](https://www.tensorflow.org/lite/guide/hosted_models)
+and [TensorFlow Hub](https://tfhub.dev/s?deployment-format=lite) to download
+pretrained models with metadata for both vision and text tasks. Also see
+different options of
+[visualizing the metadata](../models/convert/metadata.md#visualize-the-metadata).
+
+## TensorFlow Lite Support GitHub repo
+
+Visit the
+[TensorFlow Lite Support GitHub repo](https://github.com/tensorflow/tflite-support)
+for more examples and source code. Let us know your feedback by creating a
+[new GitHub issue](https://github.com/tensorflow/tflite-support/issues/new).

tensorflow/lite/g3doc/inference_with_metadata/task_library/audio_classifier.md

@@ -0,0 +1,323 @@
+# Integrate audio classifiers
+
+Audio classification is a common use case of Machine Learning to classify the
+sound types. For example, it can identify the bird species by their songs.
+
+The Task Library `AudioClassifier` API can be used to deploy your custom audio
+classifiers or pretrained ones into your mobile app.
+
+## Key features of the AudioClassifier API
+
+*   Input audio processing, e.g. converting PCM 16 bit encoding to PCM
+    Float encoding and the manipulation of the audio ring buffer.
+
+*   Label map locale.
+
+*   Supporting Multi-head classification model.
+
+*   Supporting both single-label and multi-label classification.
+
+*   Score threshold to filter results.
+
+*   Top-k classification results.
+
+*   Label allowlist and denylist.
+
+## Supported audio classifier models
+
+The following models are guaranteed to be compatible with the `AudioClassifier`
+API.
+
+*   Models created by
+    [TensorFlow Lite Model Maker for Audio Classification](https://ai.google.dev/edge/litert/libraries/modify/audio_classification).
+
+*   The
+    [pretrained audio event classification models on TensorFlow Hub](https://tfhub.dev/google/lite-model/yamnet/classification/tflite/1).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in Java
+
+See the
+[Audio Classification reference app](https://github.com/tensorflow/examples/tree/master/lite/examples/audio_classification/android)
+for an example using `AudioClassifier` in an Android app.
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify that the tflite file should not be compressed when building the APK package.
+    aaptOptions {
+        noCompress "tflite"
+    }
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Audio Task Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-audio:0.4.4'
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin:0.4.4'
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the above aaptOptions is not needed
+anymore.
+
+### Step 2: Using the model
+
+```java
+// Initialization
+AudioClassifierOptions options =
+    AudioClassifierOptions.builder()
+        .setBaseOptions(BaseOptions.builder().useGpu().build())
+        .setMaxResults(1)
+        .build();
+AudioClassifier classifier =
+    AudioClassifier.createFromFileAndOptions(context, modelFile, options);
+
+// Start recording
+AudioRecord record = classifier.createAudioRecord();
+record.startRecording();
+
+// Load latest audio samples
+TensorAudio audioTensor = classifier.createInputTensorAudio();
+audioTensor.load(record);
+
+// Run inference
+List<Classifications> results = audioClassifier.classify(audioTensor);
+```
+
+See the
+[source code and javadoc](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/audio/classifier/AudioClassifier.java)
+for more options to configure `AudioClassifier`.
+
+## Run inference in iOS
+
+### Step 1: Install the dependencies
+
+The Task Library supports installation using CocoaPods. Make sure that CocoaPods
+is installed on your system. Please see the
+[CocoaPods installation guide](https://guides.cocoapods.org/using/getting-started.html#getting-started)
+for instructions.
+
+Please see the
+[CocoaPods guide](https://guides.cocoapods.org/using/using-cocoapods.html) for
+details on adding pods to an Xcode project.
+
+Add the `TensorFlowLiteTaskAudio` pod in the Podfile.
+
+```
+target 'MyAppWithTaskAPI' do
+  use_frameworks!
+  pod 'TensorFlowLiteTaskAudio'
+end
+```
+
+Make sure that the `.tflite` model you will be using for inference is present in
+your app bundle.
+
+### Step 2: Using the model
+
+#### Swift
+
+```swift
+// Imports
+import TensorFlowLiteTaskAudio
+import AVFoundation
+
+// Initialization
+guard let modelPath = Bundle.main.path(forResource: "sound_classification",
+                                            ofType: "tflite") else { return }
+
+let options = AudioClassifierOptions(modelPath: modelPath)
+
+// Configure any additional options:
+// options.classificationOptions.maxResults = 3
+
+let classifier = try AudioClassifier.classifier(options: options)
+
+// Create Audio Tensor to hold the input audio samples which are to be classified.
+// Created Audio Tensor has audio format matching the requirements of the audio classifier.
+// For more details, please see:
+// https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/audio/core/audio_tensor/sources/TFLAudioTensor.h
+let audioTensor = classifier.createInputAudioTensor()
+
+// Create Audio Record to record the incoming audio samples from the on-device microphone.
+// Created Audio Record has audio format matching the requirements of the audio classifier.
+// For more details, please see:
+https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/audio/core/audio_record/sources/TFLAudioRecord.h
+let audioRecord = try classifier.createAudioRecord()
+
+// Request record permissions from AVAudioSession before invoking audioRecord.startRecording().
+AVAudioSession.sharedInstance().requestRecordPermission { granted in
+    if granted {
+        DispatchQueue.main.async {
+            // Start recording the incoming audio samples from the on-device microphone.
+            try audioRecord.startRecording()
+
+            // Load the samples currently held by the audio record buffer into the audio tensor.
+            try audioTensor.load(audioRecord: audioRecord)
+
+            // Run inference
+            let classificationResult = try classifier.classify(audioTensor: audioTensor)
+        }
+    }
+}
+```
+
+#### Objective C
+
+```objc
+// Imports
+#import <TensorFlowLiteTaskAudio/TensorFlowLiteTaskAudio.h>
+#import <AVFoundation/AVFoundation.h>
+
+// Initialization
+NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"sound_classification" ofType:@"tflite"];
+
+TFLAudioClassifierOptions *options =
+    [[TFLAudioClassifierOptions alloc] initWithModelPath:modelPath];
+
+// Configure any additional options:
+// options.classificationOptions.maxResults = 3;
+
+TFLAudioClassifier *classifier = [TFLAudioClassifier audioClassifierWithOptions:options
+                                                                          error:nil];
+
+// Create Audio Tensor to hold the input audio samples which are to be classified.
+// Created Audio Tensor has audio format matching the requirements of the audio classifier.
+// For more details, please see:
+// https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/audio/core/audio_tensor/sources/TFLAudioTensor.h
+TFLAudioTensor *audioTensor = [classifier createInputAudioTensor];
+
+// Create Audio Record to record the incoming audio samples from the on-device microphone.
+// Created Audio Record has audio format matching the requirements of the audio classifier.
+// For more details, please see:
+https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/audio/core/audio_record/sources/TFLAudioRecord.h
+TFLAudioRecord *audioRecord = [classifier createAudioRecordWithError:nil];
+
+// Request record permissions from AVAudioSession before invoking -[TFLAudioRecord startRecordingWithError:].
+[[AVAudioSession sharedInstance] requestRecordPermission:^(BOOL granted) {
+    if (granted) {
+        dispatch_async(dispatch_get_main_queue(), ^{
+            // Start recording the incoming audio samples from the on-device microphone.
+            [audioRecord startRecordingWithError:nil];
+
+            // Load the samples currently held by the audio record buffer into the audio tensor.
+            [audioTensor loadAudioRecord:audioRecord withError:nil];
+
+            // Run inference
+            TFLClassificationResult *classificationResult =
+                [classifier classifyWithAudioTensor:audioTensor error:nil];
+
+        });
+    }
+}];
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/audio/sources/TFLAudioClassifier.h)
+for more options to configure `TFLAudioClassifier`.
+
+## Run inference in Python
+
+### Step 1: Install the pip package
+
+```
+pip install tflite-support
+```
+
+Note: Task Library's Audio APIs rely on
+[PortAudio](http://www.portaudio.com/docs/v19-doxydocs/index.html) to record
+audio from the device's microphone. If you intend to use Task Library's
+[AudioRecord](https://ai.google.dev/edge/api/tflite/python/tflite_support/task/audio/AudioRecord)
+for audio recording, you need to install PortAudio on your system.
+
+* Linux: Run `sudo apt-get update && apt-get install libportaudio2`
+* Mac and Windows: PortAudio is installed automatically when installing the
+`tflite-support` pip package.
+
+### Step 2: Using the model
+
+```python
+# Imports
+from tflite_support.task import audio
+from tflite_support.task import core
+from tflite_support.task import processor
+
+# Initialization
+base_options = core.BaseOptions(file_name=model_path)
+classification_options = processor.ClassificationOptions(max_results=2)
+options = audio.AudioClassifierOptions(base_options=base_options, classification_options=classification_options)
+classifier = audio.AudioClassifier.create_from_options(options)
+
+# Alternatively, you can create an audio classifier in the following manner:
+# classifier = audio.AudioClassifier.create_from_file(model_path)
+
+# Run inference
+audio_file = audio.TensorAudio.create_from_wav_file(audio_path, classifier.required_input_buffer_size)
+audio_result = classifier.classify(audio_file)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/audio/audio_classifier.py)
+for more options to configure `AudioClassifier`.
+
+## Run inference in C++
+
+```c++
+// Initialization
+AudioClassifierOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<AudioClassifier> audio_classifier = AudioClassifier::CreateFromOptions(options).value();
+
+// Create input audio buffer from your `audio_data` and `audio_format`.
+// See more information here: tensorflow_lite_support/cc/task/audio/core/audio_buffer.h
+int input_size = audio_classifier->GetRequiredInputBufferSize();
+const std::unique_ptr<AudioBuffer> audio_buffer =
+    AudioBuffer::Create(audio_data, input_size, audio_format).value();
+
+// Run inference
+const ClassificationResult result = audio_classifier->Classify(*audio_buffer).value();
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/audio/audio_classifier.h)
+for more options to configure `AudioClassifier`.
+
+## Model compatibility requirements
+
+The `AudioClassifier` API expects a TFLite model with mandatory
+[TFLite Model Metadata](../../models/convert/metadata.md). See examples of
+creating metadata for audio classifiers using the
+[TensorFlow Lite Metadata Writer API](../../models/convert/metadata_writer_tutorial.ipynb#audio_classifiers).
+
+The compatible audio classifier models should meet the following requirements:
+
+*   Input audio tensor (kTfLiteFloat32)
+
+    -   audio clip of size `[batch x samples]`.
+    -   batch inference is not supported (`batch` is required to be 1).
+    -   for multi-channel models, the channels need to be interleaved.
+
+*   Output score tensor (kTfLiteFloat32)
+
+    -   `[1 x N]` array with `N` represents the class number.
+    -   optional (but recommended) label map(s) as AssociatedFile-s with type
+        TENSOR_AXIS_LABELS, containing one label per line. The first such
+        AssociatedFile (if any) is used to fill the `label` field (named as
+        `class_name` in C++) of the results. The `display_name` field is filled
+        from the AssociatedFile (if any) whose locale matches the
+        `display_names_locale` field of the `AudioClassifierOptions` used at
+        creation time ("en" by default, i.e. English). If none of these are
+        available, only the `index` field of the results will be filled.

tensorflow/lite/g3doc/inference_with_metadata/task_library/bert_nl_classifier.md

@@ -0,0 +1,182 @@
+# Integrate BERT natural language classifier
+
+The Task Library `BertNLClassifier` API is very similar to the `NLClassifier`
+that classifies input text into different categories, except that this API is
+specially tailored for Bert related models that require Wordpiece and
+Sentencepiece tokenizations outside the TFLite model.
+
+## Key features of the BertNLClassifier API
+
+*   Takes a single string as input, performs classification with the string and
+    outputs <Label, Score> pairs as classification results.
+
+*   Performs out-of-graph
+    [Wordpiece](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/text/tokenizers/bert_tokenizer.h)
+    or
+    [Sentencepiece](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/text/tokenizers/sentencepiece_tokenizer.h)
+    tokenizations on input text.
+
+## Supported BertNLClassifier models
+
+The following models are compatible with the `BertNLClassifier` API.
+
+*   Bert Models created by
+    [TensorFlow Lite Model Maker for text Classfication](https://ai.google.dev/edge/litert/libraries/modify/text_classification).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in Java
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Text Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-text:0.4.4'
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.
+
+### Step 2: Run inference using the API
+
+```java
+// Initialization
+BertNLClassifierOptions options =
+    BertNLClassifierOptions.builder()
+        .setBaseOptions(BaseOptions.builder().setNumThreads(4).build())
+        .build();
+BertNLClassifier classifier =
+    BertNLClassifier.createFromFileAndOptions(context, modelFile, options);
+
+// Run inference
+List<Category> results = classifier.classify(input);
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/text/nlclassifier/BertNLClassifier.java)
+for more details.
+
+## Run inference in Swift
+
+### Step 1: Import CocoaPods
+
+Add the TensorFlowLiteTaskText pod in Podfile
+
+```
+target 'MySwiftAppWithTaskAPI' do
+  use_frameworks!
+  pod 'TensorFlowLiteTaskText', '~> 0.4.4'
+end
+```
+
+### Step 2: Run inference using the API
+
+```swift
+// Initialization
+let bertNLClassifier = TFLBertNLClassifier.bertNLClassifier(
+      modelPath: bertModelPath)
+
+// Run inference
+let categories = bertNLClassifier.classify(text: input)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/text/nlclassifier/Sources/TFLBertNLClassifier.h)
+for more details.
+
+## Run inference in C++
+
+```c++
+// Initialization
+BertNLClassifierOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<BertNLClassifier> classifier = BertNLClassifier::CreateFromOptions(options).value();
+
+// Run inference with your input, `input_text`.
+std::vector<core::Category> categories = classifier->Classify(input_text);
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/bert_nl_classifier.h)
+for more details.
+
+## Run inference in Python
+
+### Step 1: Install the pip package
+
+```
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+# Imports
+from tflite_support.task import text
+
+# Initialization
+classifier = text.BertNLClassifier.create_from_file(model_path)
+
+# Run inference
+text_classification_result = classifier.classify(text)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/text/bert_nl_classifier.py)
+for more options to configure `BertNLClassifier`.
+
+## Example results
+
+Here is an example of the classification results of movie reviews using the
+[MobileBert](https://ai.google.dev/edge/litert/libraries/modify/text_classification)
+model from Model Maker.
+
+Input: "it's a charming and often affecting journey"
+
+Output:
+
+```
+category[0]: 'negative' : '0.00006'
+category[1]: 'positive' : '0.99994'
+```
+
+Try out the simple
+[CLI demo tool for BertNLClassifier](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/examples/task/text/desktop/README.md#bertnlclassifier)
+with your own model and test data.
+
+## Model compatibility requirements
+
+The `BetNLClassifier` API expects a TFLite model with mandatory
+[TFLite Model Metadata](../../models/convert/metadata.md).
+
+The Metadata should meet the following requirements:
+
+*   input_process_units for Wordpiece/Sentencepiece Tokenizer
+
+*   3 input tensors with names "ids", "mask" and "segment_ids" for the output of
+    the tokenizer
+
+*   1 output tensor of type float32, with a optionally attached label file. If a
+    label file is attached, the file should be a plain text file with one label
+    per line and the number of labels should match the number of categories as
+    the model outputs.

tensorflow/lite/g3doc/inference_with_metadata/task_library/bert_question_answerer.md

@@ -0,0 +1,197 @@
+# Integrate BERT question answerer
+
+The Task Library `BertQuestionAnswerer` API loads a Bert model and answers
+questions based on the content of a given passage. For more information, see the
+documentation for the Question-Answer model
+<a href="../../examples/bert_qa/overview">here</a>.
+
+## Key features of the BertQuestionAnswerer API
+
+*   Takes two text inputs as question and context and outputs a list of possible
+    answers.
+
+*   Performs out-of-graph Wordpiece or Sentencepiece tokenizations on input
+    text.
+
+## Supported BertQuestionAnswerer models
+
+The following models are compatible with the `BertNLClassifier` API.
+
+*   Models created by
+    [TensorFlow Lite Model Maker for BERT Question Answer](https://www.tensorflow.org/lite/models/modify/model_maker/question_answer).
+
+*   The
+    [pretrained BERT models on TensorFlow Hub](https://tfhub.dev/tensorflow/collections/lite/task-library/bert-question-answerer/1).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in Java
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Text Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-text:0.4.4'
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.
+
+### Step 2: Run inference using the API
+
+```java
+// Initialization
+BertQuestionAnswererOptions options =
+    BertQuestionAnswererOptions.builder()
+        .setBaseOptions(BaseOptions.builder().setNumThreads(4).build())
+        .build();
+BertQuestionAnswerer answerer =
+    BertQuestionAnswerer.createFromFileAndOptions(
+        androidContext, modelFile, options);
+
+// Run inference
+List<QaAnswer> answers = answerer.answer(contextOfTheQuestion, questionToAsk);
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/text/qa/BertQuestionAnswerer.java)
+for more details.
+
+## Run inference in Swift
+
+### Step 1: Import CocoaPods
+
+Add the TensorFlowLiteTaskText pod in Podfile
+
+```
+target 'MySwiftAppWithTaskAPI' do
+  use_frameworks!
+  pod 'TensorFlowLiteTaskText', '~> 0.4.4'
+end
+```
+
+### Step 2: Run inference using the API
+
+```swift
+// Initialization
+let mobileBertAnswerer = TFLBertQuestionAnswerer.questionAnswerer(
+      modelPath: mobileBertModelPath)
+
+// Run inference
+let answers = mobileBertAnswerer.answer(
+      context: context, question: question)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/text/qa/Sources/TFLBertQuestionAnswerer.h)
+for more details.
+
+## Run inference in C++
+
+```c++
+// Initialization
+BertQuestionAnswererOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<BertQuestionAnswerer> answerer = BertQuestionAnswerer::CreateFromOptions(options).value();
+
+// Run inference with your inputs, `context_of_question` and `question_to_ask`.
+std::vector<QaAnswer> positive_results = answerer->Answer(context_of_question, question_to_ask);
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/bert_question_answerer.h)
+for more details.
+
+## Run inference in Python
+
+### Step 1: Install the pip package
+
+```
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+# Imports
+from tflite_support.task import text
+
+# Initialization
+answerer = text.BertQuestionAnswerer.create_from_file(model_path)
+
+# Run inference
+bert_qa_result = answerer.answer(context, question)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/text/bert_question_answerer.py)
+for more options to configure `BertQuestionAnswerer`.
+
+## Example results
+
+Here is an example of the answer results of
+[ALBERT model](https://tfhub.dev/tensorflow/lite-model/albert_lite_base/squadv1/1).
+
+Context: "The Amazon rainforest, alternatively, the Amazon Jungle, also known in
+English as Amazonia, is a moist broadleaf tropical rainforest in the Amazon
+biome that covers most of the Amazon basin of South America. This basin
+encompasses 7,000,000 km2 (2,700,000 sq mi), of which
+5,500,000 km2 (2,100,000 sq mi) are covered by the rainforest. This region
+includes territory belonging to nine nations."
+
+Question: "Where is Amazon rainforest?"
+
+Answers:
+
+```
+answer[0]:  'South America.'
+logit: 1.84847, start_index: 39, end_index: 40
+answer[1]:  'most of the Amazon basin of South America.'
+logit: 1.2921, start_index: 34, end_index: 40
+answer[2]:  'the Amazon basin of South America.'
+logit: -0.0959535, start_index: 36, end_index: 40
+answer[3]:  'the Amazon biome that covers most of the Amazon basin of South America.'
+logit: -0.498558, start_index: 28, end_index: 40
+answer[4]:  'Amazon basin of South America.'
+logit: -0.774266, start_index: 37, end_index: 40
+
+```
+
+Try out the simple
+[CLI demo tool for BertQuestionAnswerer](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/examples/task/text/desktop/README.md#bert-question-answerer)
+with your own model and test data.
+
+## Model compatibility requirements
+
+The `BertQuestionAnswerer` API expects a TFLite model with mandatory
+[TFLite Model Metadata](../../models/convert/metadata).
+
+The Metadata should meet the following requirements:
+
+*   `input_process_units` for Wordpiece/Sentencepiece Tokenizer
+
+*   3 input tensors with names "ids", "mask" and "segment_ids" for the output of
+    the tokenizer
+
+*   2 output tensors with names "end_logits" and "start_logits" to indicate the
+    answer's relative position in the context

tensorflow/lite/g3doc/inference_with_metadata/task_library/customized_task_api.md

@@ -0,0 +1,448 @@
+# Build you own Task API
+
+<a href="overview.md">TensorFlow Lite Task Library</a> provides prebuilt
+native/Android/iOS APIs on top of the same infrastructure that abstracts
+TensorFlow. You can extend the Task API infrastructure to build customized APIs
+if your model is not supported by existing Task libraries.
+
+## Overview
+
+Task API infrastructure has a two-layer structure: the bottom C++ layer
+encapsulating the native TFLite runtime and the top Java/ObjC layer that
+communicates with the C++ layer through JNI or native wrapper.
+
+Implementing all the TensorFlow logic in only C++ minimizes cost, maximizes
+inference performance and simplifies the overall workflow across platforms.
+
+To create a Task class, extend the
+[BaseTaskApi](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/core/base_task_api.h)
+to provide conversion logic between TFLite model interface and Task API
+interface, then use the Java/ObjC utilities to create corresponding APIs. With
+all TensorFlow details hidden, you can deploy the TFLite model in your apps
+without any machine learning knowledge.
+
+TensorFlow Lite provides some prebuilt APIs for most popular
+<a href="overview.md#supported_tasks">Vision and NLP tasks</a>. You can build
+your own APIs for other tasks using the Task API infrastructure.
+
+<div align="center">![prebuilt_task_apis](images/prebuilt_task_apis.svg)
+<div align="center">Figure 1. prebuilt Task APIs
+<div align="left">
+
+## Build your own API with Task API infra
+
+### C++ API
+
+All TFLite details are implemented in the native API. Create an API object by
+using one of the factory functions and get model results by calling functions
+defined in the interface.
+
+#### Sample usage
+
+Here is an example using the C++
+[`BertQuestionAnswerer`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/bert_question_answerer.h)
+for
+[MobileBert](https://tfhub.dev/tensorflow/lite-model/mobilebert/1/default/1).
+
+```cpp
+  char kBertModelPath[] = "path/to/model.tflite";
+  // Create the API from a model file
+  std::unique_ptr<BertQuestionAnswerer> question_answerer =
+      BertQuestionAnswerer::CreateFromFile(kBertModelPath);
+
+  char kContext[] = ...; // context of a question to be answered
+  char kQuestion[] = ...; // question to be answered
+  // ask a question
+  std::vector<QaAnswer> answers = question_answerer.Answer(kContext, kQuestion);
+  // answers[0].text is the best answer
+```
+
+#### Building the API
+
+<div align="center">![native_task_api](images/native_task_api.svg)
+<div align="center">Figure 2. Native Task API
+<div align="left">
+
+To build an API object,you must provide the following information by extending
+[`BaseTaskApi`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/core/base_task_api.h)
+
+*   **Determine the API I/O** - Your API should expose similar input/output
+    across different platforms. e.g. `BertQuestionAnswerer` takes two strings
+    `(std::string& context, std::string& question)` as input and outputs a
+    vector of possible answer and probabilities as `std::vector<QaAnswer>`. This
+    is done by specifying the corresponding types in `BaseTaskApi`'s
+    [template parameter](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/core/base_task_api.h?q="template <class OutputType, class... InputTypes>").
+    With the template parameters specified, the
+    [`BaseTaskApi::Infer`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/core/base_task_api.h?q="Infer\(InputTypes... args\)")
+    function will have the correct input/output types. This function can be
+    directly called by API clients, but it is a good practice to wrap it inside
+    a model-specific function, in this case, `BertQuestionAnswerer::Answer`.
+
+    ```cpp
+    class BertQuestionAnswerer : public BaseTaskApi<
+                                  std::vector<QaAnswer>, // OutputType
+                                  const std::string&, const std::string& // InputTypes
+                                  > {
+      // Model specific function delegating calls to BaseTaskApi::Infer
+      std::vector<QaAnswer> Answer(const std::string& context, const std::string& question) {
+        return Infer(context, question).value();
+      }
+    }
+    ```
+
+*   **Provide conversion logic between API I/O and input/output tensor of the
+    model** - With input and output types specified, the subclasses also need to
+    implement the typed functions
+    [`BaseTaskApi::Preprocess`](https://github.com/tensorflow/tflite-support/blob/5cea306040c40b06d6e0ed4e5baf6c307db7bd00/tensorflow_lite_support/cc/task/core/base_task_api.h#L74)
+    and
+    [`BaseTaskApi::Postprocess`](https://github.com/tensorflow/tflite-support/blob/5cea306040c40b06d6e0ed4e5baf6c307db7bd00/tensorflow_lite_support/cc/task/core/base_task_api.h#L80).
+    The two functions provide
+    [inputs](https://github.com/tensorflow/tensorflow/blob/1b84e5af78f85b8d3c4687b7dee65b78113f81cc/tensorflow/lite/schema/schema.fbs#L1007)
+    and
+    [outputs](https://github.com/tensorflow/tensorflow/blob/1b84e5af78f85b8d3c4687b7dee65b78113f81cc/tensorflow/lite/schema/schema.fbs#L1008)
+    from the TFLite `FlatBuffer`. The subclass is responsible for assigning
+    values from the API I/O to I/O tensors. See the complete implementation
+    example in
+    [`BertQuestionAnswerer`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/bert_question_answerer.cc).
+
+    ```cpp
+    class BertQuestionAnswerer : public BaseTaskApi<
+                                  std::vector<QaAnswer>, // OutputType
+                                  const std::string&, const std::string& // InputTypes
+                                  > {
+      // Convert API input into tensors
+      absl::Status BertQuestionAnswerer::Preprocess(
+        const std::vector<TfLiteTensor*>& input_tensors, // input tensors of the model
+        const std::string& context, const std::string& query // InputType of the API
+      ) {
+        // Perform tokenization on input strings
+        ...
+        // Populate IDs, Masks and SegmentIDs to corresponding input tensors
+        PopulateTensor(input_ids, input_tensors[0]);
+        PopulateTensor(input_mask, input_tensors[1]);
+        PopulateTensor(segment_ids, input_tensors[2]);
+        return absl::OkStatus();
+      }
+
+      // Convert output tensors into API output
+      StatusOr<std::vector<QaAnswer>> // OutputType
+      BertQuestionAnswerer::Postprocess(
+        const std::vector<const TfLiteTensor*>& output_tensors, // output tensors of the model
+      ) {
+        // Get start/end logits of prediction result from output tensors
+        std::vector<float> end_logits;
+        std::vector<float> start_logits;
+        // output_tensors[0]: end_logits FLOAT[1, 384]
+        PopulateVector(output_tensors[0], &end_logits);
+        // output_tensors[1]: start_logits FLOAT[1, 384]
+        PopulateVector(output_tensors[1], &start_logits);
+        ...
+        std::vector<QaAnswer::Pos> orig_results;
+        // Look up the indices from vocabulary file and build results
+        ...
+        return orig_results;
+      }
+    }
+    ```
+
+*   **Create factory functions of the API** - A model file and a
+    [`OpResolver`](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/core/api/op_resolver.h)
+    are needed to initialize the
+    [`tflite::Interpreter`](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/interpreter.h).
+    [`TaskAPIFactory`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/core/task_api_factory.h)
+    provides utility functions to create BaseTaskApi instances.
+
+    Note: By default
+    [`TaskAPIFactory`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/core/task_api_factory.h)
+    provides a
+    [`BuiltInOpResolver`](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/register.h).
+    If your model needs customized ops or a subset of built-in ops, you can
+    register them by creating a
+    [`MutableOpResolver`](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/mutable_op_resolver.h).
+
+    You must also provide any files associated with the model. e.g,
+    `BertQuestionAnswerer` can also have an additional file for its tokenizer's
+    vocabulary.
+
+    ```cpp
+    class BertQuestionAnswerer : public BaseTaskApi<
+                                  std::vector<QaAnswer>, // OutputType
+                                  const std::string&, const std::string& // InputTypes
+                                  > {
+      // Factory function to create the API instance
+      StatusOr<std::unique_ptr<QuestionAnswerer>>
+      BertQuestionAnswerer::CreateBertQuestionAnswerer(
+          const std::string& path_to_model, // model to passed to TaskApiFactory
+          const std::string& path_to_vocab  // additional model specific files
+      ) {
+        // Creates an API object by calling one of the utils from TaskAPIFactory
+        std::unique_ptr<BertQuestionAnswerer> api_to_init;
+        ASSIGN_OR_RETURN(
+            api_to_init,
+            core::TaskAPIFactory::CreateFromFile<BertQuestionAnswerer>(
+                path_to_model,
+                absl::make_unique<tflite::ops::builtin::BuiltinOpResolver>(),
+                kNumLiteThreads));
+
+        // Perform additional model specific initializations
+        // In this case building a vocabulary vector from the vocab file.
+        api_to_init->InitializeVocab(path_to_vocab);
+        return api_to_init;
+      }
+    }
+    ```
+
+### Android API
+
+Create Android APIs by defining Java/Kotlin interface and delegating the logic
+to the C++ layer through JNI. Android API requires native API to be built first.
+
+#### Sample usage
+
+Here is an example using Java
+[`BertQuestionAnswerer`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/text/qa/BertQuestionAnswerer.java)
+for
+[MobileBert](https://tfhub.dev/tensorflow/lite-model/mobilebert/1/default/1).
+
+```java
+  String BERT_MODEL_FILE = "path/to/model.tflite";
+  String VOCAB_FILE = "path/to/vocab.txt";
+  // Create the API from a model file and vocabulary file
+    BertQuestionAnswerer bertQuestionAnswerer =
+        BertQuestionAnswerer.createBertQuestionAnswerer(
+            ApplicationProvider.getApplicationContext(), BERT_MODEL_FILE, VOCAB_FILE);
+
+  String CONTEXT = ...; // context of a question to be answered
+  String QUESTION = ...; // question to be answered
+  // ask a question
+  List<QaAnswer> answers = bertQuestionAnswerer.answer(CONTEXT, QUESTION);
+  // answers.get(0).text is the best answer
+```
+
+#### Building the API
+
+<div align="center">![android_task_api](images/android_task_api.svg)
+<div align="center">Figure 3. Android Task API
+<div align="left">
+
+Similar to Native APIs, to build an API object, the client needs to provide the
+following information by extending
+[`BaseTaskApi`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/core/BaseTaskApi.java),
+which provides JNI handlings for all Java Task APIs.
+
+*   __Determine the API I/O__ - This usually mirrors the native interfaces. e.g
+    `BertQuestionAnswerer` takes `(String context, String question)` as input
+    and outputs `List<QaAnswer>`. The implementation calls a private native
+    function with similar signature, except it has an additional parameter `long
+    nativeHandle`, which is the pointer returned from C++.
+
+    ```java
+    class BertQuestionAnswerer extends BaseTaskApi {
+      public List<QaAnswer> answer(String context, String question) {
+        return answerNative(getNativeHandle(), context, question);
+      }
+
+      private static native List<QaAnswer> answerNative(
+                                            long nativeHandle, // C++ pointer
+                                            String context, String question // API I/O
+                                           );
+
+    }
+    ```
+
+*   __Create factory functions of the API__ - This also mirrors native factory
+    functions, except Android factory functions also need to take
+    [`Context`](https://developer.android.com/reference/android/content/Context)
+    for file access. The implementation calls one of the utilities in
+    [`TaskJniUtils`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/core/TaskJniUtils.java)
+    to build the corresponding C++ API object and pass its pointer to the
+    `BaseTaskApi` constructor.
+
+    ```java
+      class BertQuestionAnswerer extends BaseTaskApi {
+        private static final String BERT_QUESTION_ANSWERER_NATIVE_LIBNAME =
+                                                  "bert_question_answerer_jni";
+
+        // Extending super constructor by providing the
+        // native handle(pointer of corresponding C++ API object)
+        private BertQuestionAnswerer(long nativeHandle) {
+          super(nativeHandle);
+        }
+
+        public static BertQuestionAnswerer createBertQuestionAnswerer(
+                                            Context context, // Accessing Android files
+                                            String pathToModel, String pathToVocab) {
+          return new BertQuestionAnswerer(
+              // The util first try loads the JNI module with name
+              // BERT_QUESTION_ANSWERER_NATIVE_LIBNAME, then opens two files,
+              // converts them into ByteBuffer, finally ::initJniWithBertByteBuffers
+              // is called with the buffer for a C++ API object pointer
+              TaskJniUtils.createHandleWithMultipleAssetFilesFromLibrary(
+                  context,
+                  BertQuestionAnswerer::initJniWithBertByteBuffers,
+                  BERT_QUESTION_ANSWERER_NATIVE_LIBNAME,
+                  pathToModel,
+                  pathToVocab));
+        }
+
+        // modelBuffers[0] is tflite model file buffer, and modelBuffers[1] is vocab file buffer.
+        // returns C++ API object pointer casted to long
+        private static native long initJniWithBertByteBuffers(ByteBuffer... modelBuffers);
+
+      }
+    ```
+
+*   __Implement the JNI module for native functions__ - All Java native methods
+    are implemented by calling a corresponding native function from the JNI
+    module. The factory functions would create a native API object and return
+    its pointer as a long type to Java. In later calls to Java API, the long
+    type pointer is passed back to JNI and cast back to the native API object.
+    The native API results are then converted back to Java results.
+
+    For example, this is how
+    [bert_question_answerer_jni](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/native/task/text/qa/bert_question_answerer_jni.cc)
+    is implemented.
+
+    ```cpp
+      // Implements BertQuestionAnswerer::initJniWithBertByteBuffers
+      extern "C" JNIEXPORT jlong JNICALL
+      Java_org_tensorflow_lite_task_text_qa_BertQuestionAnswerer_initJniWithBertByteBuffers(
+          JNIEnv* env, jclass thiz, jobjectArray model_buffers) {
+        // Convert Java ByteBuffer object into a buffer that can be read by native factory functions
+        absl::string_view model =
+            GetMappedFileBuffer(env, env->GetObjectArrayElement(model_buffers, 0));
+
+        // Creates the native API object
+        absl::StatusOr<std::unique_ptr<QuestionAnswerer>> status =
+            BertQuestionAnswerer::CreateFromBuffer(
+                model.data(), model.size());
+        if (status.ok()) {
+          // converts the object pointer to jlong and return to Java.
+          return reinterpret_cast<jlong>(status->release());
+        } else {
+          return kInvalidPointer;
+        }
+      }
+
+      // Implements BertQuestionAnswerer::answerNative
+      extern "C" JNIEXPORT jobject JNICALL
+      Java_org_tensorflow_lite_task_text_qa_BertQuestionAnswerer_answerNative(
+      JNIEnv* env, jclass thiz, jlong native_handle, jstring context, jstring question) {
+      // Convert long to native API object pointer
+      QuestionAnswerer* question_answerer = reinterpret_cast<QuestionAnswerer*>(native_handle);
+
+      // Calls the native API
+      std::vector<QaAnswer> results = question_answerer->Answer(JStringToString(env, context),
+                                             JStringToString(env, question));
+
+      // Converts native result(std::vector<QaAnswer>) to Java result(List<QaAnswerer>)
+      jclass qa_answer_class =
+        env->FindClass("org/tensorflow/lite/task/text/qa/QaAnswer");
+      jmethodID qa_answer_ctor =
+        env->GetMethodID(qa_answer_class, "<init>", "(Ljava/lang/String;IIF)V");
+      return ConvertVectorToArrayList<QaAnswer>(
+        env, results,
+        [env, qa_answer_class, qa_answer_ctor](const QaAnswer& ans) {
+          jstring text = env->NewStringUTF(ans.text.data());
+          jobject qa_answer =
+              env->NewObject(qa_answer_class, qa_answer_ctor, text, ans.pos.start,
+                             ans.pos.end, ans.pos.logit);
+          env->DeleteLocalRef(text);
+          return qa_answer;
+        });
+      }
+
+      // Implements BaseTaskApi::deinitJni by delete the native object
+      extern "C" JNIEXPORT void JNICALL Java_task_core_BaseTaskApi_deinitJni(
+          JNIEnv* env, jobject thiz, jlong native_handle) {
+        delete reinterpret_cast<QuestionAnswerer*>(native_handle);
+      }
+    ```
+
+### iOS API
+
+Create iOS APIs by wrapping a native API object into a ObjC API object. The
+created API object can be used in either ObjC or Swift. iOS API requires the
+native API to be built first.
+
+#### Sample usage
+
+Here is an example using ObjC
+[`TFLBertQuestionAnswerer`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/text/qa/Sources/TFLBertQuestionAnswerer.h)
+for [MobileBert](https://tfhub.dev/tensorflow/lite-model/mobilebert/1/default/1)
+in Swift.
+
+```swift
+  static let mobileBertModelPath = "path/to/model.tflite";
+  // Create the API from a model file and vocabulary file
+  let mobileBertAnswerer = TFLBertQuestionAnswerer.mobilebertQuestionAnswerer(
+      modelPath: mobileBertModelPath)
+
+  static let context = ...; // context of a question to be answered
+  static let question = ...; // question to be answered
+  // ask a question
+  let answers = mobileBertAnswerer.answer(
+      context: TFLBertQuestionAnswererTest.context, question: TFLBertQuestionAnswererTest.question)
+  // answers.[0].text is the best answer
+```
+
+#### Building the API
+
+<div align="center">![ios_task_api](images/ios_task_api.svg)
+<div align="center">Figure 4. iOS Task API
+<div align="left">
+
+iOS API is a simple ObjC wrapper on top of native API. Build the API by
+following the steps below:
+
+*   **Define the ObjC wrapper** - Define an ObjC class and delegate the
+    implementations to the corresponding native API object. Note the native
+    dependencies can only appear in a .mm file due to Swift's inability to
+    interop with C++.
+
+    *   .h file
+
+    ```objc
+      @interface TFLBertQuestionAnswerer : NSObject
+
+      // Delegate calls to the native BertQuestionAnswerer::CreateBertQuestionAnswerer
+      + (instancetype)mobilebertQuestionAnswererWithModelPath:(NSString*)modelPath
+                                                    vocabPath:(NSString*)vocabPath
+          NS_SWIFT_NAME(mobilebertQuestionAnswerer(modelPath:vocabPath:));
+
+      // Delegate calls to the native BertQuestionAnswerer::Answer
+      - (NSArray<TFLQAAnswer*>*)answerWithContext:(NSString*)context
+                                         question:(NSString*)question
+          NS_SWIFT_NAME(answer(context:question:));
+    }
+    ```
+
+    *   .mm file
+
+    ```objc
+      using BertQuestionAnswererCPP = ::tflite::task::text::BertQuestionAnswerer;
+
+      @implementation TFLBertQuestionAnswerer {
+        // define an iVar for the native API object
+        std::unique_ptr<QuestionAnswererCPP> _bertQuestionAnswerwer;
+      }
+
+      // Initialize the native API object
+      + (instancetype)mobilebertQuestionAnswererWithModelPath:(NSString *)modelPath
+                                              vocabPath:(NSString *)vocabPath {
+        absl::StatusOr<std::unique_ptr<QuestionAnswererCPP>> cQuestionAnswerer =
+            BertQuestionAnswererCPP::CreateBertQuestionAnswerer(MakeString(modelPath),
+                                                                MakeString(vocabPath));
+        _GTMDevAssert(cQuestionAnswerer.ok(), @"Failed to create BertQuestionAnswerer");
+        return [[TFLBertQuestionAnswerer alloc]
+            initWithQuestionAnswerer:std::move(cQuestionAnswerer.value())];
+      }
+
+      // Calls the native API and converts C++ results into ObjC results
+      - (NSArray<TFLQAAnswer *> *)answerWithContext:(NSString *)context question:(NSString *)question {
+        std::vector<QaAnswerCPP> results =
+          _bertQuestionAnswerwer->Answer(MakeString(context), MakeString(question));
+        return [self arrayFromVector:results];
+      }
+    }
+    ```

tensorflow/lite/g3doc/inference_with_metadata/task_library/image_classifier.md

@@ -0,0 +1,303 @@
+# Integrate image classifiers
+
+Image classification is a common use of machine learning to identify what an
+image represents. For example, we might want to know what type of animal appears
+in a given picture. The task of predicting what an image represents is called
+*image classification*. An image classifier is trained to recognize various
+classes of images. For example, a model might be trained to recognize photos
+representing three different types of animals: rabbits, hamsters, and dogs. See
+the
+[image classification overview](https://www.tensorflow.org/lite/examples/image_classification/overview)
+for more information about image classifiers.
+
+Use the Task Library `ImageClassifier` API to deploy your custom image
+classifiers or pretrained ones into your mobile apps.
+
+## Key features of the ImageClassifier API
+
+*   Input image processing, including rotation, resizing, and color space
+    conversion.
+
+*   Region of interest of the input image.
+
+*   Label map locale.
+
+*   Score threshold to filter results.
+
+*   Top-k classification results.
+
+*   Label allowlist and denylist.
+
+## Supported image classifier models
+
+The following models are guaranteed to be compatible with the `ImageClassifier`
+API.
+
+*   Models created by
+    [TensorFlow Lite Model Maker for Image Classification](https://www.tensorflow.org/lite/models/modify/model_maker/image_classification).
+
+*   The
+    [pretrained image classification models on TensorFlow Hub](https://tfhub.dev/tensorflow/collections/lite/task-library/image-classifier/1).
+
+*   Models created by
+    [AutoML Vision Edge Image Classification](https://cloud.google.com/vision/automl/docs/edge-quickstart).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in Java
+
+See the
+[Image Classification reference app](https://github.com/tensorflow/examples/blob/master/lite/examples/image_classification/android/README.md)
+for an example of how to use `ImageClassifier` in an Android app.
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Vision Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-vision'
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin'
+}
+```
+
+### Step 2: Using the model
+
+```java
+// Initialization
+ImageClassifierOptions options =
+    ImageClassifierOptions.builder()
+        .setBaseOptions(BaseOptions.builder().useGpu().build())
+        .setMaxResults(1)
+        .build();
+ImageClassifier imageClassifier =
+    ImageClassifier.createFromFileAndOptions(
+        context, modelFile, options);
+
+// Run inference
+List<Classifications> results = imageClassifier.classify(image);
+```
+
+See the
+[source code and javadoc](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/vision/classifier/ImageClassifier.java)
+for more options to configure `ImageClassifier`.
+
+## Run inference in iOS
+
+### Step 1: Install the dependencies
+
+The Task Library supports installation using CocoaPods. Make sure that CocoaPods
+is installed on your system. Please see the
+[CocoaPods installation guide](https://guides.cocoapods.org/using/getting-started.html#getting-started)
+for instructions.
+
+Please see the
+[CocoaPods guide](https://guides.cocoapods.org/using/using-cocoapods.html) for
+details on adding pods to an Xcode project.
+
+Add the `TensorFlowLiteTaskVision` pod in the Podfile.
+
+```
+target 'MyAppWithTaskAPI' do
+  use_frameworks!
+  pod 'TensorFlowLiteTaskVision'
+end
+```
+
+Make sure that the `.tflite` model you will be using for inference is present in
+your app bundle.
+
+### Step 2: Using the model
+
+#### Swift
+
+```swift
+// Imports
+import TensorFlowLiteTaskVision
+
+// Initialization
+guard let modelPath = Bundle.main.path(forResource: "birds_V1",
+                                            ofType: "tflite") else { return }
+
+let options = ImageClassifierOptions(modelPath: modelPath)
+
+// Configure any additional options:
+// options.classificationOptions.maxResults = 3
+
+let classifier = try ImageClassifier.classifier(options: options)
+
+// Convert the input image to MLImage.
+// There are other sources for MLImage. For more details, please see:
+// https://developers.google.com/ml-kit/reference/ios/mlimage/api/reference/Classes/GMLImage
+guard let image = UIImage (named: "sparrow.jpg"), let mlImage = MLImage(image: image) else { return }
+
+// Run inference
+let classificationResults = try classifier.classify(mlImage: mlImage)
+```
+
+#### Objective C
+
+```objc
+// Imports
+#import <TensorFlowLiteTaskVision/TensorFlowLiteTaskVision.h>
+
+// Initialization
+NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"birds_V1" ofType:@"tflite"];
+
+TFLImageClassifierOptions *options =
+    [[TFLImageClassifierOptions alloc] initWithModelPath:modelPath];
+
+// Configure any additional options:
+// options.classificationOptions.maxResults = 3;
+
+TFLImageClassifier *classifier = [TFLImageClassifier imageClassifierWithOptions:options
+                                                                          error:nil];
+
+// Convert the input image to MLImage.
+UIImage *image = [UIImage imageNamed:@"sparrow.jpg"];
+
+// There are other sources for GMLImage. For more details, please see:
+// https://developers.google.com/ml-kit/reference/ios/mlimage/api/reference/Classes/GMLImage
+GMLImage *gmlImage = [[GMLImage alloc] initWithImage:image];
+
+// Run inference
+TFLClassificationResult *classificationResult =
+    [classifier classifyWithGMLImage:gmlImage error:nil];
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/vision/sources/TFLImageClassifier.h)
+for more options to configure `TFLImageClassifier`.
+
+## Run inference in Python
+
+### Step 1: Install the pip package
+
+```
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+# Imports
+from tflite_support.task import vision
+from tflite_support.task import core
+from tflite_support.task import processor
+
+# Initialization
+base_options = core.BaseOptions(file_name=model_path)
+classification_options = processor.ClassificationOptions(max_results=2)
+options = vision.ImageClassifierOptions(base_options=base_options, classification_options=classification_options)
+classifier = vision.ImageClassifier.create_from_options(options)
+
+# Alternatively, you can create an image classifier in the following manner:
+# classifier = vision.ImageClassifier.create_from_file(model_path)
+
+# Run inference
+image = vision.TensorImage.create_from_file(image_path)
+classification_result = classifier.classify(image)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/vision/image_classifier.py)
+for more options to configure `ImageClassifier`.
+
+## Run inference in C++
+
+```c++
+// Initialization
+ImageClassifierOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<ImageClassifier> image_classifier = ImageClassifier::CreateFromOptions(options).value();
+
+// Create input frame_buffer from your inputs, `image_data` and `image_dimension`.
+// See more information here: tensorflow_lite_support/cc/task/vision/utils/frame_buffer_common_utils.h
+
+std::unique_ptr<FrameBuffer> frame_buffer = CreateFromRgbRawBuffer(
+      image_data, image_dimension);
+
+// Run inference
+const ClassificationResult result = image_classifier->Classify(*frame_buffer).value();
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/vision/image_classifier.h)
+for more options to configure `ImageClassifier`.
+
+## Example results
+
+Here is an example of the classification results of a
+[bird classifier](https://tfhub.dev/google/lite-model/aiy/vision/classifier/birds_V1/3).
+
+<img src="images/sparrow.jpg" alt="sparrow" width="50%">
+
+```
+Results:
+  Rank #0:
+   index       : 671
+   score       : 0.91406
+   class name  : /m/01bwb9
+   display name: Passer domesticus
+  Rank #1:
+   index       : 670
+   score       : 0.00391
+   class name  : /m/01bwbt
+   display name: Passer montanus
+  Rank #2:
+   index       : 495
+   score       : 0.00391
+   class name  : /m/0bwm6m
+   display name: Passer italiae
+```
+
+Try out the simple
+[CLI demo tool for ImageClassifier](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/vision/desktop#image-classifier)
+with your own model and test data.
+
+## Model compatibility requirements
+
+The `ImageClassifier` API expects a TFLite model with mandatory
+[TFLite Model Metadata](https://www.tensorflow.org/lite/models/convert/metadata).
+See examples of creating metadata for image classifiers using the
+[TensorFlow Lite Metadata Writer API](../../models/convert/metadata_writer_tutorial.ipynb#image_classifiers).
+
+The compatible image classifier models should meet the following requirements:
+
+*   Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
+
+    -   image input of size `[batch x height x width x channels]`.
+    -   batch inference is not supported (`batch` is required to be 1).
+    -   only RGB inputs are supported (`channels` is required to be 3).
+    -   if type is kTfLiteFloat32, NormalizationOptions are required to be
+        attached to the metadata for input normalization.
+
+*   Output score tensor (kTfLiteUInt8/kTfLiteFloat32)
+
+    -   with `N` classes and either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1
+        x 1 x N]`
+    -   optional (but recommended) label map(s) as AssociatedFile-s with type
+        TENSOR_AXIS_LABELS, containing one label per line. See the
+        [example label file](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/metadata/python/tests/testdata/image_classifier/labels.txt).
+        The first such AssociatedFile (if any) is used to fill the `label` field
+        (named as `class_name` in C++) of the results. The `display_name` field
+        is filled from the AssociatedFile (if any) whose locale matches the
+        `display_names_locale` field of the `ImageClassifierOptions` used at
+        creation time ("en" by default, i.e. English). If none of these are
+        available, only the `index` field of the results will be filled.

tensorflow/lite/g3doc/inference_with_metadata/task_library/image_embedder.md

@@ -0,0 +1,137 @@
+# Integrate image embedders
+
+Image embedders allow embedding images into a high-dimensional feature vector
+representing the semantic meaning of an image, which can then be compared with
+the feature vector of other images to evaluate their semantic similarity.
+
+As opposed to
+[image search](https://www.tensorflow.org/lite/inference_with_metadata/task_library/image_searcher),
+the image embedder allows computing the similarity between images on-the-fly
+instead of searching through a predefined index built from a corpus of images.
+
+Use the Task Library `ImageEmbedder` API to deploy your custom image embedder
+into your mobile apps.
+
+## Key features of the ImageEmbedder API
+
+*   Input image processing, including rotation, resizing, and color space
+    conversion.
+
+*   Region of interest of the input image.
+
+*   Built-in utility function to compute the
+    [cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) between
+    feature vectors.
+
+## Supported image embedder models
+
+The following models are guaranteed to be compatible with the `ImageEmbedder`
+API.
+
+*   Feature vector models from the
+    [Google Image Modules collection on TensorFlow Hub](https://tfhub.dev/google/collections/image/1).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in C++
+
+```c++
+// Initialization
+ImageEmbedderOptions options:
+options.mutable_model_file_with_metadata()->set_file_name(model_path);
+options.set_l2_normalize(true);
+std::unique_ptr<ImageEmbedder> image_embedder = ImageEmbedder::CreateFromOptions(options).value();
+
+// Create input frame_buffer_1 and frame_buffer_2 from your inputs `image_data1`, `image_data2`, `image_dimension1` and `image_dimension2`.
+// See more information here: tensorflow_lite_support/cc/task/vision/utils/frame_buffer_common_utils.h
+std::unique_ptr<FrameBuffer> frame_buffer_1 = CreateFromRgbRawBuffer(
+      image_data1, image_dimension1);
+std::unique_ptr<FrameBuffer> frame_buffer_2 = CreateFromRgbRawBuffer(
+      image_data2, image_dimension2);
+
+// Run inference on two images.
+const EmbeddingResult result_1 = image_embedder->Embed(*frame_buffer_1);
+const EmbeddingResult result_2 = image_embedder->Embed(*frame_buffer_2);
+
+// Compute cosine similarity.
+double similarity = ImageEmbedder::CosineSimilarity(
+    result_1.embeddings[0].feature_vector(),
+    result_2.embeddings[0].feature_vector());
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/vision/image_embedder.h)
+for more options to configure `ImageEmbedder`.
+
+## Run inference in Python
+
+### Step 1: Install TensorFlow Lite Support Pypi package.
+
+You can install the TensorFlow Lite Support Pypi package using the following
+command:
+
+```sh
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+from tflite_support.task import vision
+
+# Initialization.
+image_embedder = vision.ImageEmbedder.create_from_file(model_path)
+
+# Run inference on two images.
+image_1 = vision.TensorImage.create_from_file('/path/to/image1.jpg')
+result_1 = image_embedder.embed(image_1)
+image_2 = vision.TensorImage.create_from_file('/path/to/image2.jpg')
+result_2 = image_embedder.embed(image_2)
+
+# Compute cosine similarity.
+feature_vector_1 = result_1.embeddings[0].feature_vector
+feature_vector_2 = result_2.embeddings[0].feature_vector
+similarity = image_embedder.cosine_similarity(
+    result_1.embeddings[0].feature_vector, result_2.embeddings[0].feature_vector)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/vision/image_embedder.py)
+for more options to configure `ImageEmbedder`.
+
+## Example results
+
+Cosine similarity between normalized feature vectors return a score between -1
+and 1. Higher is better, i.e. a cosine similarity of 1 means the two vectors are
+identical.
+
+```
+Cosine similarity: 0.954312
+```
+
+Try out the simple
+[CLI demo tool for ImageEmbedder](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/vision/desktop#imageembedder)
+with your own model and test data.
+
+## Model compatibility requirements
+
+The `ImageEmbedder` API expects a TFLite model with optional, but strongly
+recommended
+[TFLite Model Metadata](https://www.tensorflow.org/lite/models/convert/metadata).
+
+The compatible image embedder models should meet the following requirements:
+
+*   An input image tensor (kTfLiteUInt8/kTfLiteFloat32)
+
+    -   image input of size `[batch x height x width x channels]`.
+    -   batch inference is not supported (`batch` is required to be 1).
+    -   only RGB inputs are supported (`channels` is required to be 3).
+    -   if type is kTfLiteFloat32, NormalizationOptions are required to be
+        attached to the metadata for input normalization.
+
+*   At least one output tensor (kTfLiteUInt8/kTfLiteFloat32)
+
+    -   with `N` components corresponding to the `N` dimensions of the returned
+        feature vector for this output layer.
+    -   Either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x N]`.

tensorflow/lite/g3doc/inference_with_metadata/task_library/image_searcher.md

@@ -0,0 +1,174 @@
+# Integrate image searchers
+
+Image search allows searching for similar images in a database of images. It
+works by embedding the search query into a high-dimensional vector representing
+the semantic meaning of the query, followed by similarity search in a
+predefined, custom index using
+[ScaNN](https://github.com/google-research/google-research/tree/master/scann)
+(Scalable Nearest Neighbors).
+
+As opposed to
+[image classification](https://www.tensorflow.org/lite/inference_with_metadata/task_library/image_classifier),
+expanding the number of items that can be recognized doesn't require re-training
+the entire model. New items can be added simply re-building the index. This also
+enables working with larger (100k+ items) databases of images.
+
+Use the Task Library `ImageSearcher` API to deploy your custom image searcher
+into your mobile apps.
+
+## Key features of the ImageSearcher API
+
+*   Takes a single image as input, performs embedding extraction and
+    nearest-neighbor search in the index.
+
+*   Input image processing, including rotation, resizing, and color space
+    conversion.
+
+*   Region of interest of the input image.
+
+## Prerequisites
+
+Before using the `ImageSearcher` API, an index needs to be built based on the
+custom corpus of images to search into. This can be achieved using
+[Model Maker Searcher API](https://www.tensorflow.org/lite/api_docs/python/tflite_model_maker/searcher)
+by following and adapting the
+[tutorial](https://www.tensorflow.org/lite/models/modify/model_maker/text_searcher).
+
+For this you will need:
+
+*   a TFLite image embedder model such as
+    [mobilenet v3](https://tfhub.dev/google/lite-model/imagenet/mobilenet_v3_small_100_224/feature_vector/5/metadata/1).
+    See more pretrained embedder models (a.k.a feature vector models) from the
+    [Google Image Modules collection on TensorFlow Hub](https://tfhub.dev/google/collections/image/1).
+*   your corpus of images.
+
+After this step, you should have a standalone TFLite searcher model (e.g.
+`mobilenet_v3_searcher.tflite`), which is the original image embedder model with
+the index attached into the
+[TFLite Model Metadata](https://www.tensorflow.org/lite/models/convert/metadata).
+
+## Run inference in Java
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` searcher model file to the assets directory of the Android
+module where the model will be run. Specify that the file should not be
+compressed, and add the TensorFlow Lite library to the module’s `build.gradle`
+file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite index file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Vision Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-vision:0.4.4'
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin:0.4.4'
+}
+```
+
+### Step 2: Using the model
+
+```java
+// Initialization
+ImageSearcherOptions options =
+    ImageSearcherOptions.builder()
+        .setBaseOptions(BaseOptions.builder().useGpu().build())
+        .setSearcherOptions(
+            SearcherOptions.builder().setL2Normalize(true).build())
+        .build();
+ImageSearcher imageSearcher =
+    ImageSearcher.createFromFileAndOptions(context, modelFile, options);
+
+// Run inference
+List<NearestNeighbor> results = imageSearcher.search(image);
+```
+
+See the
+[source code and javadoc](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/vision/searcher/ImageSearcher.java)
+for more options to configure the `ImageSearcher`.
+
+## Run inference in C++
+
+```c++
+// Initialization
+ImageSearcherOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+options.mutable_embedding_options()->set_l2_normalize(true);
+std::unique_ptr<ImageSearcher> image_searcher = ImageSearcher::CreateFromOptions(options).value();
+
+// Create input frame_buffer from your inputs, `image_data` and `image_dimension`.
+// See more information here: tensorflow_lite_support/cc/task/vision/utils/frame_buffer_common_utils.h
+std::unique_ptr<FrameBuffer> frame_buffer = CreateFromRgbRawBuffer(
+      image_data, image_dimension);
+
+// Run inference
+const SearchResult result = image_searcher->Search(*frame_buffer).value();
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/vision/image_searcher.h)
+for more options to configure `ImageSearcher`.
+
+## Run inference in Python
+
+### Step 1: Install TensorFlow Lite Support Pypi package.
+
+You can install the TensorFlow Lite Support Pypi package using the following
+command:
+
+```sh
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+from tflite_support.task import vision
+
+# Initialization
+image_searcher = vision.ImageSearcher.create_from_file(model_path)
+
+# Run inference
+image = vision.TensorImage.create_from_file(image_file)
+result = image_searcher.search(image)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/vision/image_searcher.py)
+for more options to configure `ImageSearcher`.
+
+## Example results
+
+```
+Results:
+ Rank#0:
+  metadata: burger
+  distance: 0.13452
+ Rank#1:
+  metadata: car
+  distance: 1.81935
+ Rank#2:
+  metadata: bird
+  distance: 1.96617
+ Rank#3:
+  metadata: dog
+  distance: 2.05610
+ Rank#4:
+  metadata: cat
+  distance: 2.06347
+```
+
+Try out the simple
+[CLI demo tool for ImageSearcher](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/vision/desktop#imagesearcher)
+with your own model and test data.

tensorflow/lite/g3doc/inference_with_metadata/task_library/image_segmenter.md

@@ -0,0 +1,301 @@
+# Integrate image segmenters
+
+Image segmenters predict whether each pixel of an image is associated with a
+certain class. This is in contrast to
+<a href="../../examples/object_detection/overview">object detection</a>,
+which detects objects in rectangular regions, and
+<a href="../../examples/image_classification/overview">image
+classification</a>, which classifies the overall image. See the
+[image segmentation overview](../../examples/segmentation/overview) for more
+information about image segmenters.
+
+Use the Task Library `ImageSegmenter` API to deploy your custom image segmenters
+or pretrained ones into your mobile apps.
+
+## Key features of the ImageSegmenter API
+
+*   Input image processing, including rotation, resizing, and color space
+    conversion.
+
+*   Label map locale.
+
+*   Two output types, category mask and confidence masks.
+
+*   Colored label for display purpose.
+
+## Supported image segmenter models
+
+The following models are guaranteed to be compatible with the `ImageSegmenter`
+API.
+
+*   The
+    [pretrained image segmentation models on TensorFlow Hub](https://tfhub.dev/tensorflow/collections/lite/task-library/image-segmenter/1).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in Java
+
+See the
+[Image Segmentation reference app](https://github.com/tensorflow/examples/tree/master/lite/examples/image_segmentation/android/)
+for an example of how to use `ImageSegmenter` in an Android app.
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Vision Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-vision'
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin'
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.
+
+### Step 2: Using the model
+
+```java
+// Initialization
+ImageSegmenterOptions options =
+    ImageSegmenterOptions.builder()
+        .setBaseOptions(BaseOptions.builder().useGpu().build())
+        .setOutputType(OutputType.CONFIDENCE_MASK)
+        .build();
+ImageSegmenter imageSegmenter =
+    ImageSegmenter.createFromFileAndOptions(context, modelFile, options);
+
+// Run inference
+List<Segmentation> results = imageSegmenter.segment(image);
+```
+
+See the
+[source code and javadoc](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/vision/segmenter/ImageSegmenter.java)
+for more options to configure `ImageSegmenter`.
+
+## Run inference in iOS
+
+### Step 1: Install the dependencies
+
+The Task Library supports installation using CocoaPods. Make sure that CocoaPods
+is installed on your system. Please see the
+[CocoaPods installation guide](https://guides.cocoapods.org/using/getting-started.html#getting-started)
+for instructions.
+
+Please see the
+[CocoaPods guide](https://guides.cocoapods.org/using/using-cocoapods.html) for
+details on adding pods to an Xcode project.
+
+Add the `TensorFlowLiteTaskVision` pod in the Podfile.
+
+```
+target 'MyAppWithTaskAPI' do
+  use_frameworks!
+  pod 'TensorFlowLiteTaskVision'
+end
+```
+
+Make sure that the `.tflite` model you will be using for inference is present in
+your app bundle.
+
+### Step 2: Using the model
+
+#### Swift
+
+```swift
+// Imports
+import TensorFlowLiteTaskVision
+
+// Initialization
+guard let modelPath = Bundle.main.path(forResource: "deeplabv3",
+                                            ofType: "tflite") else { return }
+
+let options = ImageSegmenterOptions(modelPath: modelPath)
+
+// Configure any additional options:
+// options.outputType = OutputType.confidenceMasks
+
+let segmenter = try ImageSegmenter.segmenter(options: options)
+
+// Convert the input image to MLImage.
+// There are other sources for MLImage. For more details, please see:
+// https://developers.google.com/ml-kit/reference/ios/mlimage/api/reference/Classes/GMLImage
+guard let image = UIImage (named: "plane.jpg"), let mlImage = MLImage(image: image) else { return }
+
+// Run inference
+let segmentationResult = try segmenter.segment(mlImage: mlImage)
+```
+
+#### Objective C
+
+```objc
+// Imports
+#import <TensorFlowLiteTaskVision/TensorFlowLiteTaskVision.h>
+
+// Initialization
+NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"deeplabv3" ofType:@"tflite"];
+
+TFLImageSegmenterOptions *options =
+    [[TFLImageSegmenterOptions alloc] initWithModelPath:modelPath];
+
+// Configure any additional options:
+// options.outputType = TFLOutputTypeConfidenceMasks;
+
+TFLImageSegmenter *segmenter = [TFLImageSegmenter imageSegmenterWithOptions:options
+                                                                      error:nil];
+
+// Convert the input image to MLImage.
+UIImage *image = [UIImage imageNamed:@"plane.jpg"];
+
+// There are other sources for GMLImage. For more details, please see:
+// https://developers.google.com/ml-kit/reference/ios/mlimage/api/reference/Classes/GMLImage
+GMLImage *gmlImage = [[GMLImage alloc] initWithImage:image];
+
+// Run inference
+TFLSegmentationResult *segmentationResult =
+    [segmenter segmentWithGMLImage:gmlImage error:nil];
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/vision/sources/TFLImageSegmenter.h)
+for more options to configure `TFLImageSegmenter`.
+
+## Run inference in Python
+
+### Step 1: Install the pip package
+
+```
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+# Imports
+from tflite_support.task import vision
+from tflite_support.task import core
+from tflite_support.task import processor
+
+# Initialization
+base_options = core.BaseOptions(file_name=model_path)
+segmentation_options = processor.SegmentationOptions(
+    output_type=processor.SegmentationOptions.output_type.CATEGORY_MASK)
+options = vision.ImageSegmenterOptions(base_options=base_options, segmentation_options=segmentation_options)
+segmenter = vision.ImageSegmenter.create_from_options(options)
+
+# Alternatively, you can create an image segmenter in the following manner:
+# segmenter = vision.ImageSegmenter.create_from_file(model_path)
+
+# Run inference
+image_file = vision.TensorImage.create_from_file(image_path)
+segmentation_result = segmenter.segment(image_file)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/vision/image_segmenter.py)
+for more options to configure `ImageSegmenter`.
+
+## Run inference in C++
+
+```c++
+// Initialization
+ImageSegmenterOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<ImageSegmenter> image_segmenter = ImageSegmenter::CreateFromOptions(options).value();
+
+// Create input frame_buffer from your inputs, `image_data` and `image_dimension`.
+// See more information here: tensorflow_lite_support/cc/task/vision/utils/frame_buffer_common_utils.h
+std::unique_ptr<FrameBuffer> frame_buffer = CreateFromRgbRawBuffer(
+      image_data, image_dimension);
+
+// Run inference
+const SegmentationResult result = image_segmenter->Segment(*frame_buffer).value();
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/vision/image_segmenter.h)
+for more options to configure `ImageSegmenter`.
+
+## Example results
+
+Here is an example of the segmentation results of
+[deeplab_v3](https://tfhub.dev/tensorflow/lite-model/deeplabv3/1/metadata/1), a
+generic segmentation model available on TensorFlow Hub.
+
+<img src="images/plane.jpg" alt="plane" width="50%">
+
+```
+Color Legend:
+ (r: 000, g: 000, b: 000):
+  index       : 0
+  class name  : background
+ (r: 128, g: 000, b: 000):
+  index       : 1
+  class name  : aeroplane
+
+# (omitting multiple lines for conciseness) ...
+
+ (r: 128, g: 192, b: 000):
+  index       : 19
+  class name  : train
+ (r: 000, g: 064, b: 128):
+  index       : 20
+  class name  : tv
+Tip: use a color picker on the output PNG file to inspect the output mask with
+this legend.
+```
+
+The segmentation category mask should looks like:
+
+<img src="images/segmentation-output.png" alt="segmentation-output" width="30%">
+
+Try out the simple
+[CLI demo tool for ImageSegmenter](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/vision/desktop#image-segmenter)
+with your own model and test data.
+
+## Model compatibility requirements
+
+The `ImageSegmenter` API expects a TFLite model with mandatory
+[TFLite Model Metadata](../../models/convert/metadata). See examples of creating
+metadata for image segmenters using the
+[TensorFlow Lite Metadata Writer API](../../models/convert/metadata_writer_tutorial.ipynb#image_segmenters).
+
+*   Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
+
+    -   image input of size `[batch x height x width x channels]`.
+    -   batch inference is not supported (`batch` is required to be 1).
+    -   only RGB inputs are supported (`channels` is required to be 3).
+    -   if type is kTfLiteFloat32, NormalizationOptions are required to be
+        attached to the metadata for input normalization.
+
+*   Output masks tensor: (kTfLiteUInt8/kTfLiteFloat32)
+
+    -   tensor of size `[batch x mask_height x mask_width x num_classes]`, where
+        `batch` is required to be 1, `mask_width` and `mask_height` are the
+        dimensions of the segmentation masks produced by the model, and
+        `num_classes` is the number of classes supported by the model.
+    -   optional (but recommended) label map(s) can be attached as
+        AssociatedFile-s with type TENSOR_AXIS_LABELS, containing one label per
+        line. The first such AssociatedFile (if any) is used to fill the `label`
+        field (named as `class_name` in C++) of the results. The `display_name`
+        field is filled from the AssociatedFile (if any) whose locale matches
+        the `display_names_locale` field of the `ImageSegmenterOptions` used at
+        creation time ("en" by default, i.e. English). If none of these are
+        available, only the `index` field of the results will be filled.

tensorflow/lite/g3doc/inference_with_metadata/task_library/nl_classifier.md

@@ -0,0 +1,221 @@
+# Integrate Natural language classifier
+
+The Task Library's `NLClassifier` API classifies input text into different
+categories, and is a versatile and configurable API that can handle most text
+classification models.
+
+## Key features of the NLClassifier API
+
+*   Takes a single string as input, performs classification with the string and
+    outputs <Label, Score> pairs as classification results.
+
+*   Optional Regex Tokenization available for input text.
+
+*   Configurable to adapt different classification models.
+
+## Supported NLClassifier models
+
+The following models are guaranteed to be compatible with the `NLClassifier`
+API.
+
+*   The <a href="../../examples/text_classification/overview">movie review
+    sentiment classification</a> model.
+
+*   Models with `average_word_vec` spec created by
+    [TensorFlow Lite Model Maker for text Classification](https://www.tensorflow.org/lite/models/modify/model_maker/text_classification).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in Java
+
+See the
+[Text Classification reference app](https://github.com/tensorflow/examples/blob/master/lite/examples/text_classification/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/textclassification/client/TextClassificationClient.java)
+for an example of how to use `NLClassifier` in an Android app.
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Vision Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-text:0.4.4'
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin:0.4.4'
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.
+
+### Step 2: Run inference using the API
+
+```java
+// Initialization, use NLClassifierOptions to configure input and output tensors
+NLClassifierOptions options =
+    NLClassifierOptions.builder()
+        .setBaseOptions(BaseOptions.builder().useGpu().build())
+        .setInputTensorName(INPUT_TENSOR_NAME)
+        .setOutputScoreTensorName(OUTPUT_SCORE_TENSOR_NAME)
+        .build();
+NLClassifier classifier =
+    NLClassifier.createFromFileAndOptions(context, modelFile, options);
+
+// Run inference
+List<Category> results = classifier.classify(input);
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/text/nlclassifier/NLClassifier.java)
+for more options to configure `NLClassifier`.
+
+## Run inference in Swift
+
+### Step 1: Import CocoaPods
+
+Add the TensorFlowLiteTaskText pod in Podfile
+
+```
+target 'MySwiftAppWithTaskAPI' do
+  use_frameworks!
+  pod 'TensorFlowLiteTaskText', '~> 0.4.4'
+end
+```
+
+### Step 2: Run inference using the API
+
+```swift
+// Initialization
+var modelOptions:TFLNLClassifierOptions = TFLNLClassifierOptions()
+modelOptions.inputTensorName = inputTensorName
+modelOptions.outputScoreTensorName = outputScoreTensorName
+let nlClassifier = TFLNLClassifier.nlClassifier(
+      modelPath: modelPath,
+      options: modelOptions)
+
+// Run inference
+let categories = nlClassifier.classify(text: input)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/text/nlclassifier/Sources/TFLNLClassifier.h)
+for more details.
+
+## Run inference in C++
+
+```c++
+// Initialization
+NLClassifierOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<NLClassifier> classifier = NLClassifier::CreateFromOptions(options).value();
+
+// Run inference with your input, `input_text`.
+std::vector<core::Category> categories = classifier->Classify(input_text);
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/nlclassifier/nl_classifier.h)
+for more details.
+
+## Run inference in Python
+
+### Step 1: Install the pip package
+
+```
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+# Imports
+from tflite_support.task import text
+
+# Initialization
+classifier = text.NLClassifier.create_from_file(model_path)
+
+# Run inference
+text_classification_result = classifier.classify(text)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/text/nl_classifier.py)
+for more options to configure `NLClassifier`.
+
+## Example results
+
+Here is an example of the classification results of the
+[movie review model](https://www.tensorflow.org/lite/examples/text_classification/overview).
+
+Input: "What a waste of my time."
+
+Output:
+
+```
+category[0]: 'Negative' : '0.81313'
+category[1]: 'Positive' : '0.18687'
+```
+
+Try out the simple
+[CLI demo tool for NLClassifier](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/examples/task/text/desktop/README.md#nlclassifier)
+with your own model and test data.
+
+## Model compatibility requirements
+
+Depending on the use case, the `NLClassifier` API can load a TFLite model with
+or without [TFLite Model Metadata](../../models/convert/metadata). See examples
+of creating metadata for natural language classifiers using the
+[TensorFlow Lite Metadata Writer API](../../models/convert/metadata_writer_tutorial.ipynb#nl_classifiers).
+
+The compatible models should meet the following requirements:
+
+*   Input tensor: (kTfLiteString/kTfLiteInt32)
+
+    -   Input of the model should be either a kTfLiteString tensor raw input
+        string or a kTfLiteInt32 tensor for regex tokenized indices of raw input
+        string.
+    -   If input type is kTfLiteString, no
+        [Metadata](../../models/convert/metadata) is required for the model.
+    -   If input type is kTfLiteInt32, a `RegexTokenizer` needs to be set up in
+        the input tensor's
+        [Metadata](https://www.tensorflow.org/lite/models/convert/metadata_writer_tutorial#natural_language_classifiers).
+
+*   Output score tensor:
+    (kTfLiteUInt8/kTfLiteInt8/kTfLiteInt16/kTfLiteFloat32/kTfLiteFloat64)
+
+    -   Mandatory output tensor for the score of each category classified.
+
+    -   If type is one of the Int types, dequantize it to double/float to
+        corresponding platforms
+
+    -   Can have an optional associated file in the output tensor's
+        corresponding [Metadata](../../models/convert/metadata) for category
+        labels, the file should be a plain text file with one label per line,
+        and the number of labels should match the number of categories as the
+        model outputs. See the
+        [example label file](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/metadata/python/tests/testdata/nl_classifier/labels.txt).
+
+*   Output label tensor: (kTfLiteString/kTfLiteInt32)
+
+    -   Optional output tensor for the label for each category, should be of the
+        same length as the output score tensor. If this tensor is not present,
+        the API uses score indices as classnames.
+
+    -   Will be ignored if the associated label file is present in output score
+        tensor's Metadata.

tensorflow/lite/g3doc/inference_with_metadata/task_library/object_detector.md

@@ -0,0 +1,321 @@
+# Integrate object detectors
+
+Object detectors can identify which of a known set of objects might be present
+and provide information about their positions within the given image or a video
+stream. An object detector is trained to detect the presence and location of
+multiple classes of objects. For example, a model might be trained with images
+that contain various pieces of fruit, along with a *label* that specifies the
+class of fruit they represent (e.g. an apple, a banana, or a strawberry), and
+data specifying where each object appears in the image. See the
+[introduction of object detection](../../examples/object_detection/overview)
+for more information about object detectors.
+
+Use the Task Library `ObjectDetector` API to deploy your custom object detectors
+or pretrained ones into your mobile apps.
+
+## Key features of the ObjectDetector API
+
+*   Input image processing, including rotation, resizing, and color space
+    conversion.
+
+*   Label map locale.
+
+*   Score threshold to filter results.
+
+*   Top-k detection results.
+
+*   Label allowlist and denylist.
+
+## Supported object detector models
+
+The following models are guaranteed to be compatible with the `ObjectDetector`
+API.
+
+*   The
+    [pretrained object detection models on TensorFlow Hub](https://tfhub.dev/tensorflow/collections/lite/task-library/object-detector/1).
+
+*   Models created by
+    [AutoML Vision Edge Object Detection](https://cloud.google.com/vision/automl/object-detection/docs).
+
+*   Models created by
+    [TensorFlow Lite Model Maker for object detector](https://www.tensorflow.org/lite/guide/model_maker).
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in Java
+
+See the
+[Object Detection reference app](https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android/)
+for an example of how to use `ObjectDetector` in an Android app.
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` model file to the assets directory of the Android module
+where the model will be run. Specify that the file should not be compressed, and
+add the TensorFlow Lite library to the module’s `build.gradle` file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Vision Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-vision'
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin'
+}
+```
+
+Note: starting from version 4.1 of the Android Gradle plugin, .tflite will be
+added to the noCompress list by default and the aaptOptions above is not needed
+anymore.
+
+### Step 2: Using the model
+
+```java
+// Initialization
+ObjectDetectorOptions options =
+    ObjectDetectorOptions.builder()
+        .setBaseOptions(BaseOptions.builder().useGpu().build())
+        .setMaxResults(1)
+        .build();
+ObjectDetector objectDetector =
+    ObjectDetector.createFromFileAndOptions(
+        context, modelFile, options);
+
+// Run inference
+List<Detection> results = objectDetector.detect(image);
+```
+
+See the
+[source code and javadoc](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/vision/detector/ObjectDetector.java)
+for more options to configure `ObjectDetector`.
+
+## Run inference in iOS
+
+### Step 1: Install the dependencies
+
+The Task Library supports installation using CocoaPods. Make sure that CocoaPods
+is installed on your system. Please see the
+[CocoaPods installation guide](https://guides.cocoapods.org/using/getting-started.html#getting-started)
+for instructions.
+
+Please see the
+[CocoaPods guide](https://guides.cocoapods.org/using/using-cocoapods.html) for
+details on adding pods to an Xcode project.
+
+Add the `TensorFlowLiteTaskVision` pod in the Podfile.
+
+```
+target 'MyAppWithTaskAPI' do
+  use_frameworks!
+  pod 'TensorFlowLiteTaskVision'
+end
+```
+
+Make sure that the `.tflite` model you will be using for inference is present in
+your app bundle.
+
+### Step 2: Using the model
+
+#### Swift
+
+```swift
+// Imports
+import TensorFlowLiteTaskVision
+
+// Initialization
+guard let modelPath = Bundle.main.path(forResource: "ssd_mobilenet_v1",
+                                            ofType: "tflite") else { return }
+
+let options = ObjectDetectorOptions(modelPath: modelPath)
+
+// Configure any additional options:
+// options.classificationOptions.maxResults = 3
+
+let detector = try ObjectDetector.detector(options: options)
+
+// Convert the input image to MLImage.
+// There are other sources for MLImage. For more details, please see:
+// https://developers.google.com/ml-kit/reference/ios/mlimage/api/reference/Classes/GMLImage
+guard let image = UIImage (named: "cats_and_dogs.jpg"), let mlImage = MLImage(image: image) else { return }
+
+// Run inference
+let detectionResult = try detector.detect(mlImage: mlImage)
+```
+
+#### Objective C
+
+```objc
+// Imports
+#import <TensorFlowLiteTaskVision/TensorFlowLiteTaskVision.h>
+
+// Initialization
+NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"ssd_mobilenet_v1" ofType:@"tflite"];
+
+TFLObjectDetectorOptions *options = [[TFLObjectDetectorOptions alloc] initWithModelPath:modelPath];
+
+// Configure any additional options:
+// options.classificationOptions.maxResults = 3;
+
+TFLObjectDetector *detector = [TFLObjectDetector objectDetectorWithOptions:options
+                                                                     error:nil];
+
+// Convert the input image to MLImage.
+UIImage *image = [UIImage imageNamed:@"dogs.jpg"];
+
+// There are other sources for GMLImage. For more details, please see:
+// https://developers.google.com/ml-kit/reference/ios/mlimage/api/reference/Classes/GMLImage
+GMLImage *gmlImage = [[GMLImage alloc] initWithImage:image];
+
+// Run inference
+TFLDetectionResult *detectionResult = [detector detectWithGMLImage:gmlImage error:nil];
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/task/vision/sources/TFLObjectDetector.h)
+for more options to configure `TFLObjectDetector`.
+
+## Run inference in Python
+
+### Step 1: Install the pip package
+
+```
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+# Imports
+from tflite_support.task import vision
+from tflite_support.task import core
+from tflite_support.task import processor
+
+# Initialization
+base_options = core.BaseOptions(file_name=model_path)
+detection_options = processor.DetectionOptions(max_results=2)
+options = vision.ObjectDetectorOptions(base_options=base_options, detection_options=detection_options)
+detector = vision.ObjectDetector.create_from_options(options)
+
+# Alternatively, you can create an object detector in the following manner:
+# detector = vision.ObjectDetector.create_from_file(model_path)
+
+# Run inference
+image = vision.TensorImage.create_from_file(image_path)
+detection_result = detector.detect(image)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/vision/object_detector.py)
+for more options to configure `ObjectDetector`.
+
+## Run inference in C++
+
+```c++
+// Initialization
+ObjectDetectorOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<ObjectDetector> object_detector = ObjectDetector::CreateFromOptions(options).value();
+
+// Create input frame_buffer from your inputs, `image_data` and `image_dimension`.
+// See more information here: tensorflow_lite_support/cc/task/vision/utils/frame_buffer_common_utils.h
+std::unique_ptr<FrameBuffer> frame_buffer = CreateFromRgbRawBuffer(
+      image_data, image_dimension);
+
+// Run inference
+const DetectionResult result = object_detector->Detect(*frame_buffer).value();
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/vision/object_detector.h)
+for more options to configure `ObjectDetector`.
+
+## Example results
+
+Here is an example of the detection results of
+[ssd mobilenet v1](https://tfhub.dev/tensorflow/lite-model/ssd_mobilenet_v1/1/metadata/1)
+from TensorFlow Hub.
+
+<img src="images/dogs.jpg" alt="dogs" width="50%">
+
+```
+Results:
+ Detection #0 (red):
+  Box: (x: 355, y: 133, w: 190, h: 206)
+  Top-1 class:
+   index       : 17
+   score       : 0.73828
+   class name  : dog
+ Detection #1 (green):
+  Box: (x: 103, y: 15, w: 138, h: 369)
+  Top-1 class:
+   index       : 17
+   score       : 0.73047
+   class name  : dog
+```
+
+Render the bounding boxes onto the input image:
+
+<img src="images/detection-output.png" alt="detection output" width="50%">
+
+Try out the simple
+[CLI demo tool for ObjectDetector](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/vision/desktop#object-detector)
+with your own model and test data.
+
+## Model compatibility requirements
+
+The `ObjectDetector` API expects a TFLite model with mandatory
+[TFLite Model Metadata](../../models/convert/metadata). See examples of creating
+metadata for object detectors using the
+[TensorFlow Lite Metadata Writer API](../../models/convert/metadata_writer_tutorial.ipynb#object_detectors).
+
+The compatible object detector models should meet the following requirements:
+
+*   Input image tensor: (kTfLiteUInt8/kTfLiteFloat32)
+
+    -   image input of size `[batch x height x width x channels]`.
+    -   batch inference is not supported (`batch` is required to be 1).
+    -   only RGB inputs are supported (`channels` is required to be 3).
+    -   if type is kTfLiteFloat32, NormalizationOptions are required to be
+        attached to the metadata for input normalization.
+
+*   Output tensors must be the 4 outputs of a `DetectionPostProcess` op, i.e:
+
+    -   Locations tensor (kTfLiteFloat32)
+        -   tensor of size `[1 x num_results x 4]`, the inner array representing
+            bounding boxes in the form [top, left, right, bottom].
+        -   BoundingBoxProperties are required to be attached to the metadata
+            and must specify `type=BOUNDARIES` and `coordinate_type=RATIO.
+    -   Classes tensor (kTfLiteFloat32)
+
+        -   tensor of size `[1 x num_results]`, each value representing the
+            integer index of a class.
+        -   optional (but recommended) label map(s) can be attached as
+            AssociatedFile-s with type TENSOR_VALUE_LABELS, containing one label
+            per line. See the
+            [example label file](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/metadata/python/tests/testdata/object_detector/labelmap.txt).
+            The first such AssociatedFile (if any) is used to fill the
+            `class_name` field of the results. The `display_name` field is
+            filled from the AssociatedFile (if any) whose locale matches the
+            `display_names_locale` field of the `ObjectDetectorOptions` used at
+            creation time ("en" by default, i.e. English). If none of these are
+            available, only the `index` field of the results will be filled.
+
+    -   Scores tensor (kTfLiteFloat32)
+
+        -   tensor of size `[1 x num_results]`, each value representing the
+            score of the detected object.
+
+    -   Number of detection tensor (kTfLiteFloat32)
+
+        -   integer num_results as a tensor of size `[1]`.

tensorflow/lite/g3doc/inference_with_metadata/task_library/overview.md

@@ -0,0 +1,292 @@
+# TensorFlow Lite Task Library
+
+TensorFlow Lite Task Library contains a set of powerful and easy-to-use
+task-specific libraries for app developers to create ML experiences with TFLite.
+It provides optimized out-of-box model interfaces for popular machine learning
+tasks, such as image classification, question and answer, etc. The model
+interfaces are specifically designed for each task to achieve the best
+performance and usability. Task Library works cross-platform and is supported on
+Java, C++, and Swift.
+
+## What to expect from the Task Library
+
+*   **Clean and well-defined APIs usable by non-ML-experts** \
+    Inference can be done within just 5 lines of code. Use the powerful and
+    easy-to-use APIs in the Task library as building blocks to help you easily
+    develop ML with TFLite on mobile devices.
+
+*   **Complex but common data processing** \
+    Supports common vision and natural language processing logic to convert
+    between your data and the data format required by the model. Provides the
+    same, shareable processing logic for training and inference.
+
+*   **High performance gain** \
+    Data processing would take no more than a few milliseconds, ensuring the
+    fast inference experience using TensorFlow Lite.
+
+*   **Extensibility and customization** \
+    You can leverage all benefits the Task Library infrastructure provides and
+    easily build your own Android/iOS inference APIs.
+
+## Supported tasks
+
+Below is the list of the supported task types. The list is expected to grow as
+we continue enabling more and more use cases.
+
+*   **Vision APIs**
+
+    *   [ImageClassifier](image_classifier.md)
+    *   [ObjectDetector](object_detector.md)
+    *   [ImageSegmenter](image_segmenter.md)
+    *   [ImageSearcher](image_searcher.md)
+    *   [ImageEmbedder](image_embedder.md)
+
+*   **Natural Language (NL) APIs**
+
+    *   [NLClassifier](nl_classifier.md)
+    *   [BertNLClassifier](bert_nl_classifier.md)
+    *   [BertQuestionAnswerer](bert_question_answerer.md)
+    *   [TextSearcher](text_searcher.md)
+    *   [TextEmbedder](text_embedder.md)
+
+*   **Audio APIs**
+
+    *   [AudioClassifier](audio_classifier.md)
+
+*   **Custom APIs**
+
+    *   Extend Task API infrastructure and build
+        [customized API](customized_task_api.md).
+
+## Run Task Library with Delegates
+
+[Delegates](https://www.tensorflow.org/lite/performance/delegates) enable
+hardware acceleration of TensorFlow Lite models by leveraging on-device
+accelerators such as the [GPU](https://www.tensorflow.org/lite/performance/gpu)
+and [Coral Edge TPU](https://coral.ai/). Utilizing them for neural network
+operations provides huge benefits in terms of latency and power efficiency. For
+example, GPUs can provide up to a
+[5x speedup](https://blog.tensorflow.org/2020/08/faster-mobile-gpu-inference-with-opencl.html)
+in latency on mobile devices, and Coral Edge TPUs inference
+[10x faster](https://coral.ai/docs/edgetpu/benchmarks/) than desktop CPUs.
+
+Task Library provides easy configuration and fall back options for you to set up
+and use delegates. The following accelerators are now supported in the Task API:
+
+*   Android
+    *   [GPU](https://www.tensorflow.org/lite/performance/gpu): Java / C++
+    *   [NNAPI](https://www.tensorflow.org/lite/android/delegates/nnapi):
+        Java / C++
+    *   [Hexagon](https://www.tensorflow.org/lite/android/delegates/hexagon):
+        C++
+*   Linux / Mac
+    *   [Coral Edge TPU](https://coral.ai/): C++
+*   iOS
+    *   [Core ML delegate](https://www.tensorflow.org/lite/performance/coreml_delegate):
+        C++
+
+Acceleration support in Task Swift / Web API are coming soon.
+
+### Example usage of GPU on Android in Java
+
+Step 1. Add the GPU delegate plugin library to your module's `build.gradle`
+file:
+
+```java
+dependencies {
+    // Import Task Library dependency for vision, text, or audio.
+
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin'
+}
+```
+
+Note: NNAPI comes with the Task Library targets for vision, text, and audio by
+default.
+
+Step 2. Configure GPU delegate in the task options through
+[BaseOptions](https://www.tensorflow.org/lite/api_docs/java/org/tensorflow/lite/task/core/BaseOptions.Builder).
+For example, you can set up GPU in `ObjectDetector` as follows:
+
+```java
+// Turn on GPU delegation.
+BaseOptions baseOptions = BaseOptions.builder().useGpu().build();
+// Configure other options in ObjectDetector
+ObjectDetectorOptions options =
+    ObjectDetectorOptions.builder()
+        .setBaseOptions(baseOptions)
+        .setMaxResults(1)
+        .build();
+
+// Create ObjectDetector from options.
+ObjectDetector objectDetector =
+    ObjectDetector.createFromFileAndOptions(context, modelFile, options);
+
+// Run inference
+List<Detection> results = objectDetector.detect(image);
+```
+
+### Example usage of GPU on Android in C++
+
+Step 1. Depend on the GPU delegate plugin in your bazel build target, such as:
+
+```
+deps = [
+  "//tensorflow_lite_support/acceleration/configuration:gpu_plugin", # for GPU
+]
+```
+
+Note: the `gpu_plugin` target is a separate one from the
+[GPU delegate target](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/delegates/gpu).
+`gpu_plugin` wraps the GPU delegate target, and can provide safety guard, i.e.
+fallback to TFLite CPU path on delegation errors.
+
+Other delegate options include:
+
+```
+"//tensorflow_lite_support/acceleration/configuration:nnapi_plugin", # for NNAPI
+"//tensorflow_lite_support/acceleration/configuration:hexagon_plugin", # for Hexagon
+```
+
+Step 2. Configure GPU delegate in the task options. For example, you can set up
+GPU in `BertQuestionAnswerer` as follows:
+
+```c++
+// Initialization
+BertQuestionAnswererOptions options;
+// Load the TFLite model.
+auto base_options = options.mutable_base_options();
+base_options->mutable_model_file()->set_file_name(model_file);
+// Turn on GPU delegation.
+auto tflite_settings = base_options->mutable_compute_settings()->mutable_tflite_settings();
+tflite_settings->set_delegate(Delegate::GPU);
+// (optional) Turn on automatical fallback to TFLite CPU path on delegation errors.
+tflite_settings->mutable_fallback_settings()->set_allow_automatic_fallback_on_execution_error(true);
+
+// Create QuestionAnswerer from options.
+std::unique_ptr<QuestionAnswerer> answerer = BertQuestionAnswerer::CreateFromOptions(options).value();
+
+// Run inference on GPU.
+std::vector<QaAnswer> results = answerer->Answer(context_of_question, question_to_ask);
+```
+
+Explore more advanced accelerator settings
+[here](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/acceleration/configuration/configuration.proto).
+
+### Example usage of Coral Edge TPU in Python
+
+Configure Coral Edge TPU in the base options of the task. For example, you can
+set up Coral Edge TPU in `ImageClassifier` as follows:
+
+```python
+# Imports
+from tflite_support.task import vision
+from tflite_support.task import core
+
+# Initialize options and turn on Coral Edge TPU delegation.
+base_options = core.BaseOptions(file_name=model_path, use_coral=True)
+options = vision.ImageClassifierOptions(base_options=base_options)
+
+# Create ImageClassifier from options.
+classifier = vision.ImageClassifier.create_from_options(options)
+
+# Run inference on Coral Edge TPU.
+image = vision.TensorImage.create_from_file(image_path)
+classification_result = classifier.classify(image)
+```
+
+### Example usage of Coral Edge TPU in C++
+
+Step 1. Depend on the Coral Edge TPU delegate plugin in your bazel build target,
+such as:
+
+```
+deps = [
+  "//tensorflow_lite_support/acceleration/configuration:edgetpu_coral_plugin", # for Coral Edge TPU
+]
+```
+
+Step 2. Configure Coral Edge TPU in the task options. For example, you can set
+up Coral Edge TPU in `ImageClassifier` as follows:
+
+```c++
+// Initialization
+ImageClassifierOptions options;
+// Load the TFLite model.
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_file);
+// Turn on Coral Edge TPU delegation.
+options.mutable_base_options()->mutable_compute_settings()->mutable_tflite_settings()->set_delegate(Delegate::EDGETPU_CORAL);
+// Create ImageClassifier from options.
+std::unique_ptr<ImageClassifier> image_classifier = ImageClassifier::CreateFromOptions(options).value();
+
+// Run inference on Coral Edge TPU.
+const ClassificationResult result = image_classifier->Classify(*frame_buffer).value();
+```
+
+Step 3. Install the `libusb-1.0-0-dev` package as below. If it is already
+installed, skip to the next step.
+
+```bash
+# On the Linux
+sudo apt-get install libusb-1.0-0-dev
+
+# On the macOS
+port install libusb
+# or
+brew install libusb
+```
+
+Step 4. Compile with the following configurations in your bazel command:
+
+```bash
+# On the Linux
+--define darwinn_portable=1 --linkopt=-lusb-1.0
+
+# On the macOS, add '--linkopt=-lusb-1.0 --linkopt=-L/opt/local/lib/' if you are
+# using MacPorts or '--linkopt=-lusb-1.0 --linkopt=-L/opt/homebrew/lib' if you
+# are using Homebrew.
+--define darwinn_portable=1 --linkopt=-L/opt/local/lib/ --linkopt=-lusb-1.0
+
+# Windows is not supported yet.
+```
+
+Try out the
+[Task Library CLI demo tool](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/vision/desktop)
+with your Coral Edge TPU devices. Explore more on the
+[pretrained Edge TPU models](https://coral.ai/models/) and
+[advanced Edge TPU settings](https://github.com/tensorflow/tensorflow/blob/4d999fda8d68adfdfacd4d0098124f1b2ea57927/tensorflow/lite/acceleration/configuration/configuration.proto#L594).
+
+### Example usage of Core ML Delegate in C++
+
+A complete example can be found at
+[Image Classifier Core ML Delegate Test](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/ios/test/task/vision/image_classifier/TFLImageClassifierCoreMLDelegateTest.mm).
+
+Step 1. Depend on the Core ML delegate plugin in your bazel build target, such
+as:
+
+```
+deps = [
+  "//tensorflow_lite_support/acceleration/configuration:coreml_plugin", # for Core ML Delegate
+]
+```
+
+Step 2. Configure Core ML Delegate in the task options. For example, you can set
+up Core ML Delegate in `ImageClassifier` as follows:
+
+```c++
+// Initialization
+ImageClassifierOptions options;
+// Load the TFLite model.
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_file);
+// Turn on Core ML delegation.
+options.mutable_base_options()->mutable_compute_settings()->mutable_tflite_settings()->set_delegate(::tflite::proto::Delegate::CORE_ML);
+// Set DEVICES_ALL to enable Core ML delegation on any device (in contrast to
+// DEVICES_WITH_NEURAL_ENGINE which creates Core ML delegate only on devices
+// with Apple Neural Engine).
+options.mutable_base_options()->mutable_compute_settings()->mutable_tflite_settings()->mutable_coreml_settings()->set_enabled_devices(::tflite::proto::CoreMLSettings::DEVICES_ALL);
+// Create ImageClassifier from options.
+std::unique_ptr<ImageClassifier> image_classifier = ImageClassifier::CreateFromOptions(options).value();
+
+// Run inference on Core ML.
+const ClassificationResult result = image_classifier->Classify(*frame_buffer).value();
+```

tensorflow/lite/g3doc/inference_with_metadata/task_library/text_embedder.md

@@ -0,0 +1,159 @@
+# Integrate text embedders.
+
+Text embedders allow embedding text into a high-dimensional feature vector
+representing its semantic meaning, which can then be compared with the feature
+vector of other texts to evaluate their semantic similarity.
+
+As opposed to
+[text search](https://www.tensorflow.org/lite/inference_with_metadata/task_library/text_searcher),
+the text embedder allows computing the similarity between texts on-the-fly
+instead of searching through a predefined index built from a corpus.
+
+Use the Task Library `TextEmbedder` API to deploy your custom text embedder into
+your mobile apps.
+
+## Key features of the TextEmbedder API
+
+*   Input text processing, including in-graph or out-of-graph
+    [Wordpiece](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/text/tokenizers/bert_tokenizer.h)
+    or
+    [Sentencepiece](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/text/tokenizers/sentencepiece_tokenizer.h)
+    tokenizations on input text.
+
+*   Built-in utility function to compute the
+    [cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) between
+    feature vectors.
+
+## Supported text embedder models
+
+The following models are guaranteed to be compatible with the `TextEmbedder`
+API.
+
+*   The
+    [Universal Sentence Encoder TFLite model from TensorFlow Hub](https://tfhub.dev/google/lite-model/universal-sentence-encoder-qa-ondevice/1)
+
+*   Custom models that meet the
+    [model compatibility requirements](#model-compatibility-requirements).
+
+## Run inference in C++
+
+```c++
+// Initialization.
+TextEmbedderOptions options:
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+std::unique_ptr<TextEmbedder> text_embedder = TextEmbedder::CreateFromOptions(options).value();
+
+// Run inference with your two inputs, `input_text1` and `input_text2`.
+const EmbeddingResult result_1 = text_embedder->Embed(input_text1);
+const EmbeddingResult result_2 = text_embedder->Embed(input_text2);
+
+// Compute cosine similarity.
+double similarity = TextEmbedder::CosineSimilarity(
+    result_1.embeddings[0].feature_vector()
+    result_2.embeddings[0].feature_vector());
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/text_embedder.h)
+for more options to configure `TextEmbedder`.
+
+## Run inference in Python
+
+### Step 1: Install TensorFlow Lite Support Pypi package.
+
+You can install the TensorFlow Lite Support Pypi package using the following
+command:
+
+```sh
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+from tflite_support.task import text
+
+# Initialization.
+text_embedder = text.TextEmbedder.create_from_file(model_path)
+
+# Run inference on two texts.
+result_1 = text_embedder.embed(text_1)
+result_2 = text_embedder.embed(text_2)
+
+# Compute cosine similarity.
+feature_vector_1 = result_1.embeddings[0].feature_vector
+feature_vector_2 = result_2.embeddings[0].feature_vector
+similarity = text_embedder.cosine_similarity(
+    result_1.embeddings[0].feature_vector, result_2.embeddings[0].feature_vector)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/text/text_embedder.py)
+for more options to configure `TextEmbedder`.
+
+## Example results
+
+Cosine similarity between normalized feature vectors return a score between -1
+and 1. Higher is better, i.e. a cosine similarity of 1 means the two vectors are
+identical.
+
+```
+Cosine similarity: 0.954312
+```
+
+Try out the simple
+[CLI demo tool for TextEmbedder](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/text/desktop#textembedder)
+with your own model and test data.
+
+## Model compatibility requirements
+
+The `TextEmbedder` API expects a TFLite model with mandatory
+[TFLite Model Metadata](https://www.tensorflow.org/lite/models/convert/metadata).
+
+Three main types of models are supported:
+
+*   BERT-based models (see
+    [source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/utils/bert_utils.h)
+    for more details):
+
+    -   Exactly 3 input tensors (kTfLiteString)
+
+        -   IDs tensor, with metadata name "ids",
+        -   Mask tensor, with metadata name "mask".
+        -   Segment IDs tensor, with metadata name "segment_ids"
+
+    -   Exactly one output tensor (kTfLiteUInt8/kTfLiteFloat32)
+
+        -   with `N` components corresponding to the `N` dimensions of the
+            returned feature vector for this output layer.
+        -   Either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x N]`.
+
+    -   An input_process_units for Wordpiece/Sentencepiece Tokenizer
+
+*   Universal Sentence Encoder-based models (see
+    [source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/utils/universal_sentence_encoder_utils.h)
+    for more details):
+
+    -   Exactly 3 input tensors (kTfLiteString)
+
+        -   Query text tensor, with metadata name "inp_text".
+        -   Response context tensor, with metadata name "res_context".
+        -   Response text tensor, with metadata name "res_text".
+
+    -   Exactly 2 output tensors (kTfLiteUInt8/kTfLiteFloat32)
+
+        -   Query encoding tensor, with metadata name "query_encoding".
+        -   Response encoding tensor, with metadata name "response_encoding".
+        -   Both with `N` components corresponding to the `N` dimensions of the
+            returned feature vector for this output layer.
+        -   Both with either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x
+            N]`.
+
+*   Any text embedder model with:
+
+    -   An input text tensor (kTfLiteString)
+    -   At least one output embedding tensor (kTfLiteUInt8/kTfLiteFloat32)
+
+        -   with `N` components corresponding to the `N` dimensions of the
+            returned feature vector for this output layer.
+        -   Either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x N]`.

tensorflow/lite/g3doc/inference_with_metadata/task_library/text_searcher.md

@@ -0,0 +1,176 @@
+# Integrate text searchers
+
+Text search allows searching for semantically similar text in a corpus. It works
+by embedding the search query into a high-dimensional vector representing the
+semantic meaning of the query, followed by similarity search in a predefined,
+custom index using
+[ScaNN](https://github.com/google-research/google-research/tree/master/scann)
+(Scalable Nearest Neighbors).
+
+As opposed to text classification (e.g.
+[Bert natural language classifier](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_nl_classifier)),
+expanding the number of items that can be recognized doesn't require re-training
+the entire model. New items can be added simply re-building the index. This also
+enables working with larger (100k+ items) corpuses.
+
+Use the Task Library `TextSearcher` API to deploy your custom text searcher into
+your mobile apps.
+
+## Key features of the TextSearcher API
+
+*   Takes a single string as input, performs embedding extraction and
+    nearest-neighbor search in the index.
+
+*   Input text processing, including in-graph or out-of-graph
+    [Wordpiece](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/text/tokenizers/bert_tokenizer.h)
+    or
+    [Sentencepiece](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/text/tokenizers/sentencepiece_tokenizer.h)
+    tokenizations on input text.
+
+## Prerequisites
+
+Before using the `TextSearcher` API, an index needs to be built based on the
+custom corpus of text to search into. This can be achieved using
+[Model Maker Searcher API](https://www.tensorflow.org/lite/api_docs/python/tflite_model_maker/searcher)
+by following and adapting the
+[tutorial](https://www.tensorflow.org/lite/models/modify/model_maker/text_searcher).
+
+For this you will need:
+
+*   a TFLite text embedder model, such as the Universal Sentence Encoder. For
+    example,
+    *   the
+        [one](https://storage.googleapis.com/download.tensorflow.org/models/tflite_support/searcher/text_to_image_blogpost/text_embedder.tflite)
+        retrained in this
+        [Colab](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/examples/colab/on_device_text_to_image_search_tflite.ipynb),
+        which is optimized for on-device inference. It takes only 6ms to query a
+        text string on Pixel 6.
+    *   the
+        [quantized](https://tfhub.dev/google/lite-model/universal-sentence-encoder-qa-ondevice/1)
+        one, which is smaller than the above but takes 38ms for each embedding.
+*   your corpus of text.
+
+After this step, you should have a standalone TFLite searcher model (e.g.
+`mobilenet_v3_searcher.tflite`), which is the original text embedder model with
+the index attached into the
+[TFLite Model Metadata](https://www.tensorflow.org/lite/models/convert/metadata).
+
+## Run inference in Java
+
+### Step 1: Import Gradle dependency and other settings
+
+Copy the `.tflite` searcher model file to the assets directory of the Android
+module where the model will be run. Specify that the file should not be
+compressed, and add the TensorFlow Lite library to the module’s `build.gradle`
+file:
+
+```java
+android {
+    // Other settings
+
+    // Specify tflite index file should not be compressed for the app apk
+    aaptOptions {
+        noCompress "tflite"
+    }
+
+}
+
+dependencies {
+    // Other dependencies
+
+    // Import the Task Vision Library dependency (NNAPI is included)
+    implementation 'org.tensorflow:tensorflow-lite-task-vision:0.4.4'
+    // Import the GPU delegate plugin Library for GPU inference
+    implementation 'org.tensorflow:tensorflow-lite-gpu-delegate-plugin:0.4.4'
+}
+```
+
+### Step 2: Using the model
+
+```java
+// Initialization
+TextSearcherOptions options =
+    TextSearcherOptions.builder()
+        .setBaseOptions(BaseOptions.builder().useGpu().build())
+        .setSearcherOptions(
+            SearcherOptions.builder().setL2Normalize(true).build())
+        .build();
+TextSearcher textSearcher =
+    textSearcher.createFromFileAndOptions(context, modelFile, options);
+
+// Run inference
+List<NearestNeighbor> results = textSearcher.search(text);
+```
+
+See the
+[source code and javadoc](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/text/searcher/TextSearcher.java)
+for more options to configure the `TextSearcher`.
+
+## Run inference in C++
+
+```c++
+// Initialization
+TextSearcherOptions options;
+options.mutable_base_options()->mutable_model_file()->set_file_name(model_path);
+options.mutable_embedding_options()->set_l2_normalize(true);
+std::unique_ptr<TextSearcher> text_searcher = TextSearcher::CreateFromOptions(options).value();
+
+// Run inference with your input, `input_text`.
+const SearchResult result = text_searcher->Search(input_text).value();
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/cc/task/text/text_searcher.h)
+for more options to configure `TextSearcher`.
+
+## Run inference in Python
+
+### Step 1: Install TensorFlow Lite Support Pypi package.
+
+You can install the TensorFlow Lite Support Pypi package using the following
+command:
+
+```sh
+pip install tflite-support
+```
+
+### Step 2: Using the model
+
+```python
+from tflite_support.task import text
+
+# Initialization
+text_searcher = text.TextSearcher.create_from_file(model_path)
+
+# Run inference
+result = text_searcher.search(text)
+```
+
+See the
+[source code](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/python/task/text/text_searcher.py)
+for more options to configure `TextSearcher`.
+
+## Example results
+
+```
+Results:
+ Rank#0:
+  metadata: The sun was shining on that day.
+  distance: 0.04618
+ Rank#1:
+  metadata: It was a sunny day.
+  distance: 0.10856
+ Rank#2:
+  metadata: The weather was excellent.
+  distance: 0.15223
+ Rank#3:
+  metadata: The cat is chasing after the mouse.
+  distance: 0.34271
+ Rank#4:
+  metadata: He was very happy with his newly bought car.
+  distance: 0.37703
+```
+
+Try out the simple
+[CLI demo tool for TextSearcher](https://github.com/tensorflow/tflite-support/tree/master/tensorflow_lite_support/examples/task/text/desktop#textsearcher)
+with your own model and test data.