QNN hardware acceleration on Android

This tutorial adds the Qualcomm AI Engine Direct (QNN) TFLite delegate to an Edge Impulse object detection app, so inference runs on the Hexagon NPU of Snapdragon devices. It builds on Camera inference on Android. The reference code is in qnn-hardware-acceleration.

Qualcomm AI Engine Direct stack showing the TensorFlow Lite delegate layer between the app and QNN backends — Where the QNN TFLite delegate fits in the Qualcomm AI Engine Direct stack

What you’ll build

An Android app that:

Runs an Edge Impulse object detection model on frames from the Camera2 API
Runs inference on the Hexagon NPU (HTP) through the QNN delegate
Draws bounding boxes over the camera preview
Logs timing for each stage to Logcat

Performance

The app logs the DSP and inference time for each frame to Logcat, so you can compare runs with and without the delegate on your own device and model:

Before (CPU only)
After (QNN delegate)

Logcat output with CPU inference timing before enabling the QNN delegate — Logcat timing without QNN acceleration

HTP acceleration requires an INT8 quantized model. Performance depends on the device SoC, the model architecture, and how many of the model’s operations QNN supports.

Prerequisites

A trained Edge Impulse object detection model
Android Studio Ladybug 2024.2.2 or later, with Android API 35, NDK 27.0.12077973, and CMake 3.22.1
The Qualcomm AI Engine Direct SDK
A Snapdragon device with a Hexagon NPU, or access to one through Qualcomm Device Cloud

Supported devices

QNN acceleration needs a Qualcomm Hexagon NPU Gen 2 or later:

Category	Devices
Mobile	Snapdragon 8 Gen 1, 2, and 3; Snapdragon 7+ Gen 2 and 3; Snapdragon 6 Gen 1
Embedded	QRB6490 (Rubik Pi 3), QRB5165 (RB5), Dragonwing platforms

Snapdragon reference phone displayed above its device specifications — Example Snapdragon reference device used for testing

Google Tensor processors don’t include a Hexagon NPU, so QNN acceleration doesn’t work on Pixel devices.

1. Clone the repository

git clone https://github.com/edgeimpulse/qnn-hardware-acceleration.git
cd qnn-hardware-acceleration

Open the project in Android Studio and let the Gradle and NDK sync finish.

2. Find the QNN delegate in the SDK

Install the Qualcomm AI Engine Direct SDK. It’s usually installed in one of these locations:

# macOS/Linux
/opt/qairt/<version>/
~/qairt/<version>/

# Windows
C:\qairt\<version>\

Find the Android arm64 copy of libQnnTFLiteDelegate.so. On macOS or Linux:

find /opt/qairt/<version> -type f -name libQnnTFLiteDelegate.so

On Windows, open C:\qairt\<version>\ in File Explorer and search for libQnnTFLiteDelegate.so. The folder that contains it also holds the other libQnn*.so runtime libraries. You copy from this folder in the next step.

3. Copy the QNN libraries

Create the destination folder:

mkdir -p app/src/main/jniLibs/arm64-v8a/

Copy these libraries from the delegate folder into app/src/main/jniLibs/arm64-v8a/:

libQnnTFLiteDelegate.so
libQnnHtp.so
libQnnHtpV**.so                # match your device: V68, V69, V75, or V79
libQnnHtpV**Skel.so            # skeleton library for the same version
libQnnSystem.so
libQnnIr.so
libQnnSaver.so                 # if your SDK includes it
libPlatformValidatorShared.so  # if your SDK includes it
libcdsprpc.so                  # optional; some devices provide it in /vendor

Automated script (experimental)

The repository includes a script that copies the libraries. Set your QAIRT SDK path in the script first, then run it:

sh ./fetchqnnlibs.sh

4. Deploy your model

In Edge Impulse Studio, go to Deployment.
Select Android (C++ library).
Select the Quantized (int8) model.
Click Build and download the .zip file.

Extract it into the project:

unzip ~/Downloads/your-model.zip -d app/src/main/cpp/

5. Configure the Android manifest

In app/src/main/AndroidManifest.xml, set extractNativeLibs and declare libcdsprpc.so so the app can load the QNN libraries:

<application
    android:extractNativeLibs="true"
    ...>

    <uses-native-library
        android:name="libcdsprpc.so"
        android:required="false"/>

    <!-- Existing permissions and activities -->
</application>

6. Build and run

Connect your Snapdragon device over USB and enable USB debugging in Developer options.
In Android Studio, click Run and select your device.

Filter Logcat by MainActivity to see the timing for each stage:

adb logcat -s MainActivity

Timing: DSP 3716 us, inference 527 us, anomaly 0 us

To confirm that the app loaded the QNN libraries, check its memory maps:

adb shell 'pid=$(pidof -s com.yourpackage.name); cat /proc/$pid/maps | grep -i qnn'

If the output lists QNN library paths, acceleration is active.

How it works

Load the QNN delegate

native-lib.cpp sets the library search paths, then loads the QNN TFLite delegate with the HTP backend:

// native-lib.cpp
#include <tensorflow/lite/delegates/external/external_delegate.h>

TfLiteDelegate* LoadQnnDelegate() {
    // Set environment for QNN
    setenv("ADSP_LIBRARY_PATH", GetNativeLibDir() + ":/dsp", 1);
    setenv("LD_LIBRARY_PATH", GetNativeLibDir() + ":" +
           getenv("LD_LIBRARY_PATH"), 1);

    // Configure QNN options
    const char* qnn_options =
        R"({"backend":"htp",
            "htp_performance_mode":"burst",
            "enable_intermediate_outputs":false})";

    // Load delegate
    TfLiteExternalDelegateOptions options =
        TfLiteExternalDelegateOptionsDefault("/path/to/libQnnTFLiteDelegate.so");
    options.insert(&options, "qnn_options", qnn_options);

    return TfLiteExternalDelegateCreate(&options);
}

Set the environment

MainActivity.kt points the library paths at the app’s native library folder on startup:

// MainActivity.kt
private fun setupQnnEnvironment() {
    val nativeLibDir = applicationInfo.nativeLibraryDir

    // Set library paths for QNN
    System.setProperty("ADSP_LIBRARY_PATH", "$nativeLibDir:/dsp")
    System.setProperty("LD_LIBRARY_PATH", "$nativeLibDir:${System.getenv("LD_LIBRARY_PATH")}")

    Log.d(TAG, "QNN environment configured: $nativeLibDir")
}

Project structure

qnn-hardware-acceleration/
├── app/
│   └── src/
│       ├── main/
│       │   ├── java/com/example/test_camera/
│       │   │   ├── MainActivity.kt       # Camera + overlay UI
│       │   │   ├── CameraManager.kt      # Camera2 handling
│       │   │   └── OverlayView.kt        # Bounding box drawing
│       │   ├── cpp/
│       │   │   ├── native-lib.cpp        # JNI + QNN integration
│       │   │   ├── edge-impulse-sdk/     # Your model
│       │   │   ├── model-parameters/
│       │   │   └── tflite-model/
│       │   ├── jniLibs/
│       │   │   └── arm64-v8a/            # QNN shared libraries
│       │   └── AndroidManifest.xml
│       └── CMakeLists.txt
└── build.gradle

Customization

Change the HTP performance mode

Set htp_performance_mode in the QNN options in native-lib.cpp:

const char* qnn_options =
    R"({"backend":"htp",
        "htp_performance_mode":"high_performance",
        "enable_intermediate_outputs":false})";

Mode	Behavior
`burst`	Maximum speed and higher power use (the default)
`high_performance`	Sustained high performance
`balanced`	Balances speed and power
`low_power`	Minimizes power use

Enable profiling

Set profiling_level to write a profile to /sdcard/qnn_profile.json:

const char* qnn_options =
    R"({"backend":"htp",
        "htp_performance_mode":"burst",
        "profiling_level":"detailed",
        "enable_intermediate_outputs":true})";

adb shell ls -l /sdcard/qnn_profile.json

Change the detection threshold

A higher threshold gives fewer false positives. A lower one gives more detections:

// MainActivity.kt
private val DETECTION_THRESHOLD = 0.5f  // try 0.7f for fewer false positives, 0.3f for more detections

Performance tuning

Model

Use INT8 quantization. HTP acceleration requires it.
Reduce the input resolution. A 320×320 input has a quarter of the pixels of a 640×640 input.
Use fewer layers, and check which operations run on the HTP and which fall back to the CPU. See the QNN operator support list.

Runtime

// Use more threads for operations that fall back to the CPU
interpreter->SetNumThreads(4);

// Allocate tensors once
interpreter->AllocateTensors();

// Reuse input/output buffers
TfLiteTensor* input = interpreter->input_tensor(0);
TfLiteTensor* output = interpreter->output_tensor(0);

Frame rate

If processing can’t keep up with the camera, lower the camera frame rate:

val fpsRange = Range(15, 30)  // 15-30 FPS instead of 30-60
captureRequestBuilder.set(
    CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE,
    fpsRange
)

Next steps

Run your model on Qualcomm Device Cloud without your own hardware
Build a speech-to-image app with QNN
Android series overview

OVERVIEW

END-TO-END

TOPICS

TOOLS

INTEGRATIONS

What you’ll build

Performance

Prerequisites

Supported devices

1. Clone the repository

2. Find the QNN delegate in the SDK

3. Copy the QNN libraries

4. Deploy your model

5. Configure the Android manifest

6. Build and run

How it works

Load the QNN delegate

Set the environment

Project structure

Customization

Change the HTP performance mode

Enable profiling

Change the detection threshold

Performance tuning

Model

Runtime

Frame rate

Next steps

Resources

​What you’ll build

​Performance

​Prerequisites

​Supported devices

​1. Clone the repository

​2. Find the QNN delegate in the SDK

​3. Copy the QNN libraries

​4. Deploy your model

​5. Configure the Android manifest

​6. Build and run

​How it works

​Load the QNN delegate

​Set the environment

​Project structure

​Customization

​Change the HTP performance mode

​Enable profiling

​Change the detection threshold

​Performance tuning

​Model

​Runtime

​Frame rate

​Next steps

​Resources

What you’ll build

Performance

Prerequisites

Supported devices

1. Clone the repository

2. Find the QNN delegate in the SDK

3. Copy the QNN libraries

4. Deploy your model

5. Configure the Android manifest

6. Build and run

How it works

Load the QNN delegate

Set the environment

Project structure

Customization

Change the HTP performance mode

Enable profiling

Change the detection threshold

Performance tuning

Model

Runtime

Frame rate

Next steps

Resources