diff --git a/backends/qualcomm/debugger/README.md b/backends/qualcomm/debugger/README.md
index afe2336c1d8..f5fce379095 100644
--- a/backends/qualcomm/debugger/README.md
+++ b/backends/qualcomm/debugger/README.md
@@ -1,93 +1,213 @@
-# QAIRT Visualizer
+# ExecuTorch QNN Debugger & Profiler
-[QAIRT Visualizer](https://pypi.org/project/qairt-visualizer/) is a Python package designed to help you visualize and analyze data from Qualcomm AI Engine Direct (QNN) models. It provides tools to generate and interpret op traces (`optrace`) and QNN HTP Analysis Summary (`QHAS`), enabling detailed insights into your model's performance and behavior.
+This directory bundles three independent debugging and profiling flows for the ExecuTorch QNN backend. They are independent and address different failure modes — jump directly to the section you need.
-## Installation
+**Table of contents**
-You can install the QAIRT Visualizer package directly from [QAIRT Visualizer](https://pypi.org/project/qairt-visualizer/):
+- [Executorch QNN HTP Profiling](#executorch-qnn-htp-profiling)
+- [ExecuTorch QNN Intermediate Output Debugger](#executorch-qnn-intermediate-output-debugger)
+- [ExecuTorch QNN HTP Heap Profiling](#executorch-qnn-htp-heap-profiling)
+
+---
+# Executorch QNN HTP Profiling
+
+This section shows how to produce HTP profiling results and inspect the output reports.
+
+- The most accurate profiling mode is **on device profiling** with `generate_htp_profile_result()`, this requires a connected android device through ADB, see details in section 2.1.
+
+- Another profiling mode is **on host estimiation** with `estimate_htp_profile_result()` this doesn't require a device and can be run on host machine. However, the accuracy and soc support might be limited, see details in section 2.2.
+
+
+Different HTP profiling features support different prepare modes, prepare mode is set in AOT config, and decides which format of QNN internal graph is packed in `.pte` file,
+ * On device profiling support both `online_prepare` and `offline_prepare`
+ * On host profiling reqruires `online_prepare`
+
+**Host Estimation vs On-Device Generation**
+| Mode | Source of measurements | Requires device? | Requires `QnnConfig.online_prepare=True` for `.pte` generation ? |
+|:--------------|:---------------------------------------|:-----------------|:-----------------|
+| `generate_htp_profile_result` (`"optrace"`) | On-device hardware counters (real run) | Yes | No — support both mode (`.dlc` or `.bin`) |
+| `estimate_htp_profile_result` (`"hextimate"`) | Compile-time performance-model estimate | No (host only) | Yes |
+
+
+**Profile level**
+
+| `profile_level` | QNN configuration | Use |
+|:----------------|:------------------|:----|
+| `0` | Profiling disabled | Default inference. |
+| `2` | `QNN_PROFILE_LEVEL_DETAILED` | Collect QNN graph and per-node timing through the ExecuTorch profiler. |
+| `3` | Detailed + `QNN_PROFILE_CONFIG_OPTION_ENABLE_OPTRACE` | Enable HTP Optrace hardware-trace artifacts for `qnn-profile-viewer`; required at export for offline-prepare Optrace. |
+
+The rest of the guide is arranged here:
+1. **`.pte` generation prepare mode:**
+We introduce how to trigger each preparation mode in section 1.1 and 1.2:
+ - 1.1 `online_prepare`: controlled by `QnnConfig.online_prepare=True`.
+ - 1.2 `offline_prepare`: controlled by `QnnConfig.online_prepare=False`.
+
+3. **Public Functions for Generating Profiling Results**
+ + 2.1 `generate_htp_profile_result()`: generates device-based profiling results (Optrace in QNN SDK).
+ + 2.2 `estimate_htp_profile_result()`: estimates host-based profiling results (Hextimate in QNN SDK).
+
+4. **HTP Profile Output Format:** `QnnHtpProfileArtifacts` contains genrated HTML, JSON and chrometrace files.
+
+5. **Qairt-Visualizer:** QAIRT Visualizer can open the QHAS result from `qhas_json` and `chrometrace_json`. Use `QnnHtpProfileArtifacts.visualizer_reports()` to pass related reports.
+## 1. Select AOT Prepare modes for `.pte` Generation
+Users choose one prepare mode before exporting the `.pte`:
+
+| Prepare mode | QNN config | User-facing behavior |
+|:-------------|:-----------|:---------------------|
+| **online_prepare** | `QnnConfig.online_prepare=True` | Export stores a graph description; profiling tools finish preparation later. |
+| **offline_prepare** | `QnnConfig.online_prepare=False` | Export stores the finalized executable form; on-device profiling requires `profile_level=3`. |
+
+Internal detail: online prepare carries a `.dlc`, and offline prepare carries a finalized QNN context binary (`.bin`). Offline prepare must set `profile_level=3` because OpTrace instrumentation has to be baked into that context binary during export. The schematic file used by `qnn-profile-viewer` is generated or unpacked beside the dumped profiling artifacts.
+
+The example demos keep the profiling route explicit:
+
+- `examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_online.py`: on-device OpTrace with `online_prepare`.
+- `examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_offline.py`: on-device OpTrace with `offline_prepare`.
+- `examples/qualcomm/util_scripts/htp_profiling_on_host_hextimate.py`: host-only Hextimate with `online_prepare`.
+
+### 1.1 Online Prepare Mode `.pte`
+
+**Demo script**:
```bash
-pip install qairt-visualizer
+python -m examples.qualcomm.util_scripts.htp_profiling_on_device_op_trace_online \
+ --host ${host} --device ${device} --soc_model ${SOC_MODEL} --build_folder build-android \
+ -a ${path_to_output_folder} --online_prepare
```
-## Quick start
-This command launches an interactive GUI interface to visualize the `optrace` and `QHAS` results.
+**Export call**:
+The export step does not need to set `profile_level`:
+```python
+build_executorch_binary(
+ model=model,
+ qnn_config=qnn_config, # online_prepare=True; profile_level not required
+ file_name=f"{args.artifact}/{pte_filename}",
+ dataset=[example_input],
+ quant_dtype=QuantDtype.use_8a8w,
+)
```
-python -m examples.qualcomm.util_scripts.qairt_visualizer_demo --host ${host} --device {device} --soc_model ${SOC_MODEL} --build_folder build-android -a ${path_to_output_folder} --online_prepare
+
+### 1.2 Offline Prepare Mode `.pte`
+**Demo script**:
+```bash
+python -m examples.qualcomm.util_scripts.htp_profiling_on_device_op_trace_offline \
+ --host ${host} --device ${device} --soc_model ${SOC_MODEL} --build_folder build-android \
+ -a ${path_to_output_folder} --profile_level 3
```
-- If online prepare mode is `enabled`, the following artifacts will be generated:
- - `model`.dlc
- - `optrace`.json
- - `QHAS`.json
-- If online prepare mode is `disabled`, the following artifacts will be generated:
- - `model`.bin
- - `optrace`.json
- - `QHAS`.json
-
-Note: Model visualization is supported only in online prepare mode.
-The `.bin` format is not compatible with the QAIRT visualizer.
-To enable model visualization, please add the `--online_prepare` flag.
-
-## Details
-### 1. Lower to QNN backend
-Generate an ExecuTorch binary for Qualcomm platforms.
-Ensure that qnn_config.profile_level is set to 3, which will generate op_trace.
+
+**Export call**:
+The export step **must** set `profile_level=3`:
```python
qnn_config.profile_level = 3
build_executorch_binary(
model=model,
- qnn_config=qnn_config,
+ qnn_config=qnn_config, # online_prepare=False (default)
file_name=f"{args.artifact}/{pte_filename}",
dataset=[example_input],
quant_dtype=QuantDtype.use_8a8w,
- online_prepare=args.online_prepare,
- optrace=True,
)
```
-### 2. Generate optrace and QHAS
-Generate optrace and QHAS files using QNN tools under $QNN_SDK_ROOT. After finishing, you will get a `binaries_trace` dictionary.
-``` python
-adb = SimpleADB(
- qnn_config=qnn_config,
+
+## 2.1 Generate HTP Profile Result (OpTrace) `generate_htp_profile_result()`
+
+Use `generate_htp_profile_result()` when you want real on-device hardware-counter profiling. This path requires sample inputs and `adb`.
+```python
+from executorch.backends.qualcomm.debugger.utils import generate_htp_profile_result
+
+artifacts = generate_htp_profile_result(
+ artifact_dir=args.artifact,
+ soc_id=get_soc_to_chipset_map()[args.soc_model],
pte_path=f"{args.artifact}/{pte_filename}.pte",
- workspace=f"/data/local/tmp/executorch/{pte_filename}",
-)
-binaries_trace = generate_optrace(
- args, adb, f"{args.artifact}/{pte_filename}.pte", example_input
+ inputs=example_inputs,
+ adb=adb,
)
```
-- **`binaries_trace`**: A dictionary where keys are the dumped file paths and values are tuples containing the paths to the generated optrace and QHAS JSON files.
-- Example 1: {"forward_0.dlc": (optrace.json, optrace_qnn_htp_analysis_summary.json)}
-- Example 2: {"forward_0.bin": (optrace.json, optrace_qnn_htp_analysis_summary.json)}
+## 2.2 Estimation of HTP profiling on Host (Hextimate) `estimate_htp_profile_result()`
+
+Use `estimate_htp_profile_result()` when you want host-only compile-time performance estimation. This path does not use sample inputs or `adb`.
-### 3. Visualizing and Analyzing optrace and QHAS
+> [!WARNING]
+> `estimate_htp_profile_result()` hard-errors on SDKs below 2.41 and on unsupported SoCs.
-Once you have the optrace and QHAS files, you can leverage the QAIRT Visualizer to visualize the model graph, optrace and QHAS data. Here's how you can do it:
+**Demo script**:
+```bash
+python -m examples.qualcomm.util_scripts.htp_profiling_on_host_hextimate \
+ --soc_model QCS9100 -a ${path_to_output_folder} --online_prepare
+```
```python
-import qairt_visualizer
-qairt_visualizer.view(f"{args.artifact}/forward_0.dlc", reports=[optrace, qhas])
+from executorch.backends.qualcomm.debugger.utils import estimate_htp_profile_result
+
+estimates = estimate_htp_profile_result(
+ artifact_dir=args.artifact,
+ soc_id=get_soc_to_chipset_map()[args.soc_model],
+ pte_path=f"{args.artifact}/{pte_filename}.pte",
+)
+```
+
+Limitations:
+- Requires `QnnConfig.online_prepare=True`.
+- Requires QNN SDK >= 2.41.
+- Currently supports only the following soc_model:
+ - SA8540
+ - SA8255
+ - QCS9100
+ - SA8797
+
+## 3. HTP Profiling Output `QnnHtpProfileArtifacts`
+Both public functions `estimate_htp_profile_result()` and `generate_htp_profile_result()` return one `QnnHtpProfileArtifacts` per compiled binary in the `.pte` (partitioned graphs yield multiple entries).
+
+Important fields:
+
+- `qhas_html`: QHAS HTML report. This is the most convenient artifact for viewing the QNN HTP Analysis Summary.
+- `qhas_json`: QHAS JSON report.
+- `chrometrace_json`: Chrome trace JSON. Open it with `chrome://tracing` or Perfetto.
+- `htp_graph_json`: HTP graph JSON after optimization.
+- `htp_graph_before_json`: HTP graph JSON before optimization.
+- `runtrace_json`: runtrace JSON for device profiling, or `None` when not emitted.
+
+Other context fields:
+
+- `binary_path`: dumped internal `.dlc` or `.bin` used by `qnn-profile-viewer`.
+- `mode`: `"optrace"` or `"hextimate"`.
+- `prepare_mode`: `"online"` or `"offline"`.
+
+
+## 4. Viewing HTP Analysis Summary (QHAS) with QAIRT Visualizer
+
+**Install**
+
+```bash
+pip install qairt-visualizer
```
-or
+
+**Usage**
+
+QAIRT Visualizer can open the QHAS result from `qhas_json` and `chrometrace_json`. Use `QnnHtpProfileArtifacts.visualizer_reports()` to pass related reports.
+
```python
import qairt_visualizer
-qairt_visualizer.view(reports=[optrace, qhas])
-```
-- `model`: Path to your QNN model file (e.g., `path_to_your_model.dlc`).
-- **`reports`**: List of report file paths, including the optrace (`optrace.json`) and QHAS (`optrace_qnn_htp_analysis_summary.json`).
+for artifact in artifacts:
+ qairt_visualizer.view(reports=artifact.visualizer_reports())
+ print(f"QHAS HTML: {artifact.qhas_html}")
+```
+**Example**
+The example scripts already call QAIRT Visualizer after producing artifacts when `qairt-visualizer` is installed. If it is not installed, they still print the generated `qhas_html` path:
-Note: Files ending with `.bin` do not support graph visualization in qairt_visualizer.
+- `examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_online.py`
+- `examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_offline.py`
+- `examples/qualcomm/util_scripts/htp_profiling_on_host_hextimate.py`
-## Demo
- 
+
-For more details, visit the [QAIRT Visualizer](https://pypi.org/project/qairt-visualizer/).
+For the viewer package, see [QAIRT Visualizer](https://pypi.org/project/qairt-visualizer/).
# ExecuTorch QNN Intermediate Output Debugger
diff --git a/backends/qualcomm/debugger/utils.py b/backends/qualcomm/debugger/utils.py
index 01cb267e917..84a13744090 100644
--- a/backends/qualcomm/debugger/utils.py
+++ b/backends/qualcomm/debugger/utils.py
@@ -4,12 +4,22 @@
import shutil
import subprocess
import tempfile
-from typing import Sequence, Tuple
+from dataclasses import dataclass
+from typing import List, Literal, Optional, Sequence, Tuple
import executorch.backends.qualcomm.python.PyQnnManagerAdaptor as PyQnnManager
import pandas as pd
import torch
-from executorch.backends.qualcomm.serialization.qc_schema import QcomChipset
+from executorch.backends.qualcomm.serialization.qc_schema import (
+ QcomChipset,
+ QnnExecuTorchProfileLevel,
+)
+from executorch.backends.qualcomm.serialization.qc_schema_serialize import (
+ flatbuffer_to_option,
+)
+from executorch.backends.qualcomm.utils.check_qnn_version import (
+ is_qnn_sdk_version_less_than,
+)
from executorch.backends.qualcomm.utils.utils import dump_context_from_pte
from graphviz import Digraph
@@ -190,14 +200,78 @@ def draw(self):
shutil.move(dot_file, dot_dest_file)
+@dataclass(frozen=True)
+class QnnHtpProfileArtifacts:
+ """Files emitted for one HTP optrace/hextimate run.
+
+ Data fields:
+ - binary_path: dumped `.dlc` or `.bin` input.
+ - mode: `"optrace"` for on-device hardware counters, or `"hextimate"`
+ for host-only compile-time estimation.
+ - prepare_mode: `"online"` for `.dlc` from `QnnConfig.online_prepare=True`,
+ or `"offline"` for `.bin` from `online_prepare=False`.
+ - chrometrace_json: Chrome trace JSON, viewable with `chrome://tracing` or
+ Perfetto.
+ - qhas_json: QHAS JSON
+ - qhas_html: QHAS HTML report.
+ - htp_graph_json: HTP graph JSON after optimization.
+ - htp_graph_before_json: HTP graph JSON before optimization.
+ - runtrace_json: runtrace JSON for optrace, or None when not emitted.
+
+ Use visualizer_reports() for the subset safe to pass to
+ qairt_visualizer.view(reports=...).
+ """
+
+ binary_path: str
+ mode: Literal["optrace", "hextimate"]
+ prepare_mode: Literal["online", "offline"]
+ chrometrace_json: str
+ qhas_json: Optional[str]
+ qhas_html: str
+ htp_graph_json: str
+ htp_graph_before_json: str
+ runtrace_json: Optional[str]
+
+ def visualizer_reports(self) -> List[str]:
+ """Reports safe to pass to qairt_visualizer.view(reports=...)."""
+ reports = [self.chrometrace_json]
+ if self.qhas_json is not None:
+ reports.append(self.qhas_json)
+ return reports
+
+
+# Hextimate (compile-time perf estimation) requires SDK >= 2.41. Below that,
+# qnn-context-binary-generator silently drops the hextimate parameters
+_MIN_SDK_FOR_HEXTIMATE = "2.41"
+_HEXTIMATE_SUPPORTED_SOCS = (
+ QcomChipset.SA8540,
+ QcomChipset.SA8255,
+ QcomChipset.QCS9100,
+ QcomChipset.SA8797,
+)
+
+
+# - QNN: libQnnHtpNetRunExtensions.so
+# - QAIRT: libQairtHtpBackendExtensions.so (QAIRT 2.49+ sdk)
+_BACKEND_EXTENSIONS_LIB = "libQnnHtpNetRunExtensions.so"
+
+
class QnnTool:
+ """Host-side wrapper around the QNN profiling CLI toolchain.
+
+ Compatibility (see README.md §QAIRT Profiling for the user-facing table):
+ - Optrace: SDK 2.37+
+ - Hextimate: SDK 2.41+ (gate and raise error)
+
+ """
+
def __init__(
self,
- tmp_dir,
- sample_input,
+ artifact_dir,
soc_id,
adb,
- build_folder,
+ sample_input=None,
+ build_folder=None,
workspace="/data/local/tmp/qnn_executorch_test",
):
self.qnn_sdk = os.environ.get("QNN_SDK_ROOT", None)
@@ -205,17 +279,18 @@ def __init__(
assert self.qnn_sdk, "QNN_SDK_ROOT was not found in environment variable"
assert self.ndk, "ANDROID_NDK_ROOT was not found in environment variable"
- self.tmp_dir = tmp_dir
+ self.artifact_dir = artifact_dir
self.workspace = workspace
self.adb = adb
self.sample_input = sample_input
self.build_folder = build_folder
- self.root = os.getcwd()
- self.config = {
- "backend_extension_config": {
- "backend_extensions": {
- "config_file_path": "config.json",
- },
+ self.soc_id = soc_id
+
+ def _get_base_config(self):
+ # Generate base device profile — every subprocess call clones from this.
+ return {
+ "backend_extensions": {
+ "config_file_path": os.path.join(self.artifact_dir, "config.json"),
},
"config": {
"devices": [
@@ -224,59 +299,94 @@ def __init__(
"cores": [
{"perf_profile": "burst", "rpc_control_latency": 100}
],
- "soc_id": int(soc_id),
+ "soc_id": int(self.soc_id),
}
]
},
}
- def qnn_context_binary_generator(
- self,
- qnn_binary_file="forward_0.dlc",
- binary_name="forward.serialized",
- ):
- for file_name, data in self.config.items():
- with open(f"{self.tmp_dir}/{file_name}.json", "w") as json_file:
- json.dump(data, json_file, indent=4)
+ def _write_config_files(
+ self, backend_extensions: Optional[dict] = None
+ ) -> Tuple[str, str]:
+ """Write backend_extension_config.json and config.json in artifact_dir.
+
+ Returns (backend_ext_path, config_path).
+ """
+ if backend_extensions is None:
+ backend_extensions = self._get_base_config()["backend_extensions"]
+
+ backend_ext_path = os.path.join(
+ self.artifact_dir, "backend_extension_config.json"
+ )
+ config_path = os.path.join(self.artifact_dir, "config.json")
+ with open(backend_ext_path, "w") as f:
+ json.dump({"backend_extensions": backend_extensions}, f, indent=4)
+ with open(config_path, "w") as f:
+ json.dump(self._get_base_config()["config"], f, indent=4)
+ return backend_ext_path, config_path
+
+ def _run(self, cmd: List[str], step: str) -> None:
+ """Run a subprocess with argv list; assert on non-zero exit."""
+ result = subprocess.run(cmd, capture_output=True)
+ assert result.returncode == 0, (
+ f"{step} failed (exit {result.returncode}): "
+ f"{result.stderr.decode('utf-8', errors='replace')}"
+ )
+
+ def _qnn_context_binary_generator(
+ self,
+ qnn_binary_file: str,
+ binary_name: str,
+ enable_hextimate: bool,
+ ) -> None:
target = "x86_64-linux-clang"
- cmds = [
+ backend_ext = self._get_base_config()["backend_extensions"]
+ if enable_hextimate:
+ backend_ext["shared_library_path"] = (
+ f"{self.qnn_sdk}/lib/{target}/{_BACKEND_EXTENSIONS_LIB}"
+ )
+ backend_ext_path, _ = self._write_config_files(backend_ext)
+
+ cmd = [
f"{self.qnn_sdk}/bin/{target}/qnn-context-binary-generator",
"--backend",
f"{self.qnn_sdk}/lib/{target}/libQnnHtp.so",
"--model",
f"{self.qnn_sdk}/lib/{target}/libQnnModelDlc.so",
"--dlc_path",
- f"{self.tmp_dir}/{qnn_binary_file}",
- f"--config_file {self.tmp_dir}/backend_extension_config.json",
- f"--binary_file {binary_name}",
- f"--output_dir {self.tmp_dir}",
- "--profiling_level detailed",
- "--profiling_option optrace",
+ os.path.join(self.artifact_dir, qnn_binary_file),
+ "--config_file",
+ backend_ext_path,
+ "--binary_file",
+ binary_name,
+ "--output_dir",
+ self.artifact_dir,
+ "--profiling_level",
+ "detailed",
+ "--profiling_option",
+ "optrace",
]
- result = subprocess.run(
- " ".join(cmds),
- shell=True,
- executable="/bin/bash",
- capture_output=True,
- )
- assert os.path.isfile(f"{self.tmp_dir}/{binary_name}.bin"), result.stderr
-
- def qnn_net_run(self, graph_name="forward.serialized"):
-
- self.config["backend_extension_config"]["backend_extensions"][
- "shared_library_path"
- ] = "./libQnnHtpNetRunExtensions.so"
- for file_name, data in self.config.items():
- with open(f"{self.tmp_dir}/{file_name}.json", "w") as json_file:
- json.dump(data, json_file, indent=4)
+ self._run(cmd, "qnn-context-binary-generator")
+ expected = os.path.join(self.artifact_dir, f"{binary_name}.bin")
+ assert os.path.isfile(
+ expected
+ ), f"qnn-context-binary-generator ran but did not produce {expected}"
+
+ def _qnn_net_run(self, graph_name: str) -> None:
+ # backend-extensions library path is device-relative when running via adb
+ backend_ext = {
+ "shared_library_path": f"./{_BACKEND_EXTENSIONS_LIB}",
+ "config_file_path": "config.json",
+ }
+ backend_ext_path, config_path = self._write_config_files(backend_ext)
target = "aarch64-android"
files = [
- f"{self.qnn_sdk}/lib/{target}/libQnnHtpNetRunExtensions.so",
- f"{self.tmp_dir}/backend_extension_config.json",
- f"{self.tmp_dir}/config.json",
- f"{self.tmp_dir}/{graph_name}.bin",
+ f"{self.qnn_sdk}/lib/{target}/{_BACKEND_EXTENSIONS_LIB}",
+ backend_ext_path,
+ config_path,
+ os.path.join(self.artifact_dir, f"{graph_name}.bin"),
f"{self.qnn_sdk}/bin/{target}/qnn-net-run",
]
cmds = [
@@ -303,132 +413,289 @@ def qnn_net_run(self, graph_name="forward.serialized"):
"pull",
"-a",
f"{self.workspace}/output/qnn-profiling-data_0.log",
- self.tmp_dir,
+ self.artifact_dir,
]
)
assert os.path.isfile(
- f"{self.tmp_dir}/qnn-profiling-data_0.log"
- ), f"Error: qnn-profiling-data_0.log not found in {self.tmp_dir}"
-
- def qnn_profile_viewer(self, graph_name="forward_schematic", graph_idx=0):
- self.config["backend_extension_config"] = {"features": {"qhas_json": True}}
- for file_name, data in self.config.items():
- with open(f"{self.tmp_dir}/{file_name}.json", "w") as json_file:
- json.dump(data, json_file, indent=4)
+ f"{self.artifact_dir}/qnn-profiling-data_0.log"
+ ), f"Error: qnn-profiling-data_0.log not found in {self.artifact_dir}"
+
+ def _qnn_profile_viewer(self, schematic_stem: str, graph_idx: int) -> None:
+ # profile-viewer takes its own config schema (`features`), not the
+ # device profile schema. Written to the SAME file name because that's
+ # the flag qnn-profile-viewer expects — a per-step fresh config avoids
+ # any leak from earlier CBG/net-run configs.
+ backend_ext_path = os.path.join(
+ self.artifact_dir, "backend_extension_config.json"
+ )
+ with open(backend_ext_path, "w") as f:
+ json.dump({"features": {"qhas_json": True}}, f, indent=4)
target = "x86_64-linux-clang"
- cmds = [
+ # TODO: remove assumption that AOT dumpped schematic file exists in same cwd
+ # we need to make .pte self-contained.
+ schematic = os.path.join(os.getcwd(), f"{schematic_stem}.bin")
+ assert os.path.isfile(schematic), (
+ f"qnn-profile-viewer expected schematic at {schematic}; "
+ "in case of online_prepare, the context-binary-generator step should have produced it in artifact_dir. "
+ "in case of offline_prepare, the schematic should be dumpped from pte, make sure profiling_level=3 when generating pte. "
+ )
+
+ cmd = [
f"{self.qnn_sdk}/bin/{target}/qnn-profile-viewer",
- f"--config {self.tmp_dir}/backend_extension_config.json",
- f"--schematic {self.root}/{graph_name}.bin",
- f"--reader {self.qnn_sdk}/lib/{target}/libQnnHtpOptraceProfilingReader.so",
- f"--input_log {self.tmp_dir}/qnn-profiling-data_0.log",
- f"--output {self.tmp_dir}/optrace_{graph_idx}.json",
+ "--config",
+ backend_ext_path,
+ "--schematic",
+ schematic,
+ "--reader",
+ f"{self.qnn_sdk}/lib/{target}/libQnnHtpOptraceProfilingReader.so",
+ "--input_log",
+ os.path.join(self.artifact_dir, "qnn-profiling-data_0.log"),
+ "--output",
+ os.path.join(self.artifact_dir, f"optrace_{graph_idx}.json"),
]
- result = subprocess.run(
- " ".join(cmds),
- shell=True,
- executable="/bin/bash",
- capture_output=True,
- )
- assert (
- result.returncode == 0
- ), f"Process failed with error: {result.stderr.decode('utf-8')}"
+ self._run(cmd, "qnn-profile-viewer")
- def generate_optrace(
- self,
- qnn_binary_file="forward_0.dlc",
- ):
+ def _validated_qhas_json(self, qhas_path: str) -> Optional[str]:
+ """Return path if the QHAS JSON parses; None if truncated (hextimate SDK bug).
+
+ Note: QHAS JSON is truncated for hextimate mode (SDK bug, still open as of
+ 2.50 nightly). When time_us == 0, the SDK divides 1e6 / 0 = +Infinity,
+ rapidjson rejects the write, and the JSON stream is cut at ~3900 bytes
+ with "inf_per_s": . We detect this and return qhas_json=None
+ rather than repair — the HTML report and chrometrace remain valid.
"""
- Generate Qnn HTP Optrace Profiling https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/htp_backend.html#qnn-htp-optrace-profiling
- and QNN HTP Analysis Summary (QHAS) https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/htp_backend.html#qnn-htp-analysis-summary-qhas
- . You can utilize the QAIRT Visualizer (https://pypi.org/project/qairt-visualizer/) to visualize the results from the files above.
+ if not os.path.isfile(qhas_path):
+ return None
+ try:
+ with open(qhas_path, "r") as f:
+ json.load(f)
+ return qhas_path
+ except json.JSONDecodeError:
+ return None
+
+ def run(
+ self,
+ mode: Literal["optrace", "hextimate"],
+ binary_file: str,
+ ) -> QnnHtpProfileArtifacts:
+ """Run qnn-profile-viewer for one dumped .dlc/.bin.
+
+ Docs:
+ - Optrace: https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/htp_backend.html#qnn-htp-optrace-profiling
+ - Hextimate: https://docs.qualcomm.com/doc/80-63442-10/topic/htp_backend.html#qnn-htp-hextimate-profiling
+ - QHAS: https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-10/htp_backend.html#qnn-htp-analysis-summary-qhas
"""
- graph_name, file_extension = os.path.splitext(qnn_binary_file)
- assert file_extension in [
- ".dlc",
- ".bin",
- ], f"Invalid file extension '{file_extension}'. Supported extensions are 'dlc' and 'bin'."
+ assert mode in ("optrace", "hextimate"), f"unknown mode {mode!r}"
- # Attempt to extract a numeric index from the end of the graph name (e.g., "forward_123")
- match = re.match(r"^(.*)_(\d+)$", graph_name)
- graph_base_name = graph_name
- graph_idx = 0
+ graph_name, ext = os.path.splitext(binary_file)
+ if mode == "optrace":
+ assert ext in (".dlc", ".bin"), (
+ f"optrace supports .dlc (online prepare) and .bin (offline prepare); "
+ f"got {ext!r}"
+ )
+ else: # hextimate
+ assert ext == ".dlc", (
+ f"hextimate requires .dlc (online prepare); got {ext!r}. "
+ "For offline-prepare context binaries, use mode='optrace' instead."
+ )
+ if is_qnn_sdk_version_less_than(_MIN_SDK_FOR_HEXTIMATE):
+ # SDK < 2.41 silently drops hextimate config and produces a
+ # standard context binary — fail loudly before that trap.
+ raise AssertionError(
+ f"hextimate requires QNN SDK >= {_MIN_SDK_FOR_HEXTIMATE}; "
+ f"the current SDK at $QNN_SDK_ROOT={self.qnn_sdk} is older. "
+ "Older SDKs silently ignore hextimate parameters and emit "
+ "an ordinary profiling log with no hextimate events."
+ )
+
+ prepare_mode: Literal["online", "offline"] = (
+ "online" if ext == ".dlc" else "offline"
+ )
+ # Extract graph index if the file follows the "_" convention.
+ match = re.match(r"^(.*)_(\d+)$", graph_name)
if match:
graph_base_name = match.group(1)
graph_idx = int(match.group(2))
-
- # Handle .dlc file extension by generating a serialized version of the graph
- if file_extension == ".dlc":
- self.qnn_context_binary_generator(
- qnn_binary_file, f"{graph_base_name}.serialized"
+ else:
+ graph_base_name = graph_name
+ graph_idx = 0
+
+ # Step 1: for online-prepare (.dlc), materialize the context binary +
+ # schematic on the host. Offline-prepare (.bin) already has both.
+ if ext == ".dlc":
+ self._qnn_context_binary_generator(
+ qnn_binary_file=binary_file,
+ binary_name=f"{graph_base_name}.serialized",
+ enable_hextimate=(mode == "hextimate"),
)
graph_name = f"{graph_base_name}.serialized"
- # Run the QNN graph and generate the schematic
- self.qnn_net_run(graph_name=graph_name)
- self.qnn_profile_viewer(
- graph_name=f"{graph_base_name}_schematic", graph_idx=graph_idx
+ # Step 2: for optrace, run on device. Hextimate is host-only.
+ if mode == "optrace":
+ self._qnn_net_run(graph_name=graph_name)
+
+ # Step 3: post-process into optrace.json + QHAS side-artifacts.
+ self._qnn_profile_viewer(
+ schematic_stem=f"{graph_base_name}_schematic",
+ graph_idx=graph_idx,
)
- # Clean up the schematic binary file if it exists
- schematic_bin_path = os.path.join(self.root, f"{graph_base_name}_schematic.bin")
- if os.path.isfile(schematic_bin_path):
- os.remove(schematic_bin_path)
+ # Collect the six output files. qnn-profile-viewer names them by
+ # stripping `.json` off --output and appending suffixes.
+ base = os.path.join(self.artifact_dir, f"optrace_{graph_idx}")
+ chrometrace_json = f"{base}.json"
+ qhas_json_candidate = f"{base}_qnn_htp_analysis_summary.json"
+ qhas_html = f"{base}_qnn_htp_analysis_summary.html"
+ htp_graph_json = f"{base}_htp.json"
+ htp_graph_before_json = f"{base}_htp_graph_before.json"
+ runtrace_json = f"{base}_runtrace.json"
- optrace_path = os.path.join(self.tmp_dir, f"optrace_{graph_idx}.json")
- qhas_path = os.path.join(
- self.tmp_dir, f"optrace_{graph_idx}_qnn_htp_analysis_summary.json"
+ assert os.path.isfile(
+ chrometrace_json
+ ), f"qnn-profile-viewer did not produce {chrometrace_json}"
+
+ return QnnHtpProfileArtifacts(
+ binary_path=os.path.join(self.artifact_dir, binary_file),
+ mode=mode,
+ prepare_mode=prepare_mode,
+ chrometrace_json=chrometrace_json,
+ qhas_json=self._validated_qhas_json(qhas_json_candidate),
+ qhas_html=qhas_html,
+ htp_graph_json=htp_graph_json,
+ htp_graph_before_json=htp_graph_before_json,
+ runtrace_json=(runtrace_json if os.path.isfile(runtrace_json) else None),
)
- assert os.path.isfile(optrace_path) and os.path.isfile(qhas_path), (
- "Error: Required files not found - either "
- f"{os.path.basename(optrace_path)} or {os.path.basename(qhas_path)} is missing."
+
+
+def _validate_pte_profile_level(pte_path: str) -> None:
+ """Assert that any offline-prepare .pte was built with profile_level=3."""
+ from executorch.exir._serialize._program import deserialize_pte_binary
+
+ with open(pte_path, "rb") as f:
+ program = deserialize_pte_binary(f.read()).program
+
+ for execution_plan in program.execution_plan:
+ for delegate in execution_plan.delegates:
+ if delegate.id != "QnnBackend":
+ continue
+ spec = delegate.compile_specs[0]
+ options = flatbuffer_to_option(bytes(spec.value))
+ if options.online_prepare:
+ continue # online-prepare: profile_level is set on-host later
+ assert options.profile_level == QnnExecuTorchProfileLevel.kProfileOptrace, (
+ f"{pte_path} was compiled with online_prepare=False and "
+ f"profile_level={options.profile_level.name}."
+ "HTP Profling (Optrace) feature requires profile_level=3 (kProfileOptrace) "
+ "at build_executorch_binary() time — \n"
+ "Please re-export with qnn_config.profile_level=3, or use online_prepare=True"
+ )
+
+
+def _validate_hextimate_soc(soc_id: QcomChipset) -> None:
+ if soc_id not in _HEXTIMATE_SUPPORTED_SOCS:
+ supported = ", ".join(soc.name for soc in _HEXTIMATE_SUPPORTED_SOCS)
+ raise AssertionError(
+ f"hextimate currently supports only {supported}; got {soc_id.name}."
)
- return optrace_path, qhas_path
+
+def _generate_htp_analysis_result(
+ artifact_dir: str,
+ soc_id: QcomChipset,
+ pte_path: str,
+ mode: Literal["optrace", "hextimate"],
+ inputs: Optional[Sequence[Tuple[torch.Tensor]]] = None,
+ adb=None,
+) -> List[QnnHtpProfileArtifacts]:
+ assert mode in ("optrace", "hextimate"), f"unknown mode {mode!r}"
+ if mode == "optrace":
+ assert adb is not None, "optrace requires adb for on-device execution"
+ _validate_pte_profile_level(pte_path)
+
+ dumpfiles = dump_context_from_pte(pte_path, output_dir=artifact_dir)
+
+ qnn_tool = QnnTool(
+ artifact_dir=artifact_dir,
+ sample_input=inputs,
+ soc_id=soc_id,
+ adb=adb,
+ build_folder=(adb.build_path if adb is not None else None),
+ workspace=(adb.workspace if adb is not None else None),
+ )
+
+ return [qnn_tool.run(mode=mode, binary_file=os.path.basename(f)) for f in dumpfiles]
+# backward compatibility shim
def generate_optrace(
- artifact,
+ artifact: str,
soc_id: QcomChipset,
adb,
pte_path: str,
inputs: Sequence[Tuple[torch.Tensor]],
-):
- """
- Generate optrace and QHAS (QNN HTP Analysis Summary) JSON files.
+) -> List[QnnHtpProfileArtifacts]:
+ """Legacy positional wrapper for generate_htp_profile_result()."""
+ return generate_htp_profile_result(artifact, soc_id, pte_path, inputs, adb)
- Args:
- artifact (str): Path to the artifact folder.
- adb (SimpleADB): An object for communicating with Android device
- pte_path (str): The path to the generated PTE file, including the file extension (e.g., model.pte).
- inputs Sequence((Tuple[torch.Tensor])): The input tensors for the model.
+def generate_htp_profile_result(
+ artifact_dir: str,
+ soc_id: QcomChipset,
+ pte_path: str,
+ inputs: Sequence[Tuple[torch.Tensor]],
+ adb,
+) -> List[QnnHtpProfileArtifacts]:
+ """Generate HTP optrace artifacts from a .pte by running on device.
+
+ Arguments:
+ - artifact_dir: host directory for dumped `.dlc`/`.bin`, schematics, QNN
+ configs, profiling logs, and qnn-profile-viewer outputs.
+ - soc_id: target SoC used by the compiled `.pte`; must match the device.
+ - pte_path: `.pte` produced by build_executorch_binary().
+ - inputs: sample input tensors used by qnn-net-run for optrace collection.
+ - adb: SimpleADB helper for pushing files, running qnn-net-run, and
+ pulling `qnn-profiling-data_0.log` from the device.
+
+ Supported prepare modes for the input pte file:
+ - `QnnConfig.online_prepare=True`: `.pte` contains `.dlc`;
+ We will create the context binary and schematic before device execution.
+ - `QnnConfig.online_prepare=False`: `.pte` contains finalized `.bin`, user must
+ also set `QnnConfig.profile_level=3` so optrace instrumentation is already baked in;
+ We will extract schematic and context binary from pte and continue device execution.
- Returns:
- dict: A dictionary where keys are the dumped file paths and values are tuples containing the paths
- to the generated optrace and QHAS JSON files.
"""
- filename, _ = os.path.splitext(pte_path.split(os.sep)[-1])
+ return _generate_htp_analysis_result(
+ artifact_dir=artifact_dir,
+ soc_id=soc_id,
+ pte_path=pte_path,
+ inputs=inputs,
+ mode="optrace",
+ adb=adb,
+ )
- # Dump compiled binaries
- dumpfiles = dump_context_from_pte(pte_path)
- # Generate optrace and QHAS
- qnn_tool = QnnTool(
- artifact,
- inputs,
- soc_id,
- adb,
- build_folder=adb.build_path,
- workspace=adb.workspace,
+def estimate_htp_profile_result(
+ artifact_dir: str,
+ soc_id: QcomChipset,
+ pte_path: str,
+) -> List[QnnHtpProfileArtifacts]:
+ """Estimate HTP performance with host-only hextimate artifacts.
+
+ Arguments:
+ - artifact_dir: host directory for dumped `.dlc`, QNN configs, and
+ qnn-profile-viewer outputs.
+ - soc_id: target SoC used by the compiled `.pte`; currently limited to:
+ SA8540, SA8255, QCS9100, and SA8797.
+ - pte_path: `.pte` produced by build_executorch_binary(), requiring
+ `QnnConfig.online_prepare=True` for `.pte` generation and QNN SDK >= 2.41.
+ """
+ _validate_hextimate_soc(soc_id)
+ return _generate_htp_analysis_result(
+ artifact_dir=artifact_dir,
+ soc_id=soc_id,
+ pte_path=pte_path,
+ mode="hextimate",
)
-
- binaries_trace = {}
- for file in dumpfiles:
- filename = file.split(os.sep)[-1]
- optrace, qhas = qnn_tool.generate_optrace(filename)
- binaries_trace[file] = (optrace, qhas)
- return binaries_trace
diff --git a/backends/qualcomm/serialization/qc_compiler_spec.fbs b/backends/qualcomm/serialization/qc_compiler_spec.fbs
index 57708c959e9..c5fbabb90c3 100644
--- a/backends/qualcomm/serialization/qc_compiler_spec.fbs
+++ b/backends/qualcomm/serialization/qc_compiler_spec.fbs
@@ -65,6 +65,7 @@ enum QcomChipset: int {
SW6100 = 96,
QCM6490 = 93,
SM8845 = 97,
+ SA8540 = 62,
}
/// Indicate the information of the specified SoC.
diff --git a/backends/qualcomm/serialization/qc_schema.py b/backends/qualcomm/serialization/qc_schema.py
index aeffbc069b6..1ad7363fb16 100644
--- a/backends/qualcomm/serialization/qc_schema.py
+++ b/backends/qualcomm/serialization/qc_schema.py
@@ -72,6 +72,7 @@ class QcomChipset(IntEnum):
SW6100 = 96 # v81
QCM6490 = 93 # v68
SM8845 = 97 # v81
+ SA8540 = 62 # v68
@dataclass
@@ -106,6 +107,7 @@ class SocInfo:
QcomChipset.SW6100: SocInfo(QcomChipset.SW6100, HtpInfo(HtpArch.V81, 4)),
QcomChipset.QCM6490: SocInfo(QcomChipset.QCM6490, HtpInfo(HtpArch.V68, 2)),
QcomChipset.SM8845: SocInfo(QcomChipset.SM8845, HtpInfo(HtpArch.V81, 8)),
+ QcomChipset.SA8540: SocInfo(QcomChipset.SA8540, HtpInfo(HtpArch.V68, 8)),
}
diff --git a/backends/qualcomm/tests/test_qnn_delegate.py b/backends/qualcomm/tests/test_qnn_delegate.py
index 1ebda343c8c..2e9571be419 100644
--- a/backends/qualcomm/tests/test_qnn_delegate.py
+++ b/backends/qualcomm/tests/test_qnn_delegate.py
@@ -22,7 +22,10 @@
from executorch.backends.qualcomm._passes.qnn_pass_manager import (
get_qnn_pass_manager_cls,
)
-from executorch.backends.qualcomm.debugger.utils import generate_optrace
+from executorch.backends.qualcomm.debugger.utils import (
+ estimate_htp_profile_result,
+ generate_htp_profile_result,
+)
from executorch.backends.qualcomm.export_utils import (
get_backend_type,
@@ -31,6 +34,7 @@
)
from executorch.backends.qualcomm.quantizer.rules import Q_ANNOTATION_KEY
from executorch.backends.qualcomm.serialization.qc_schema import (
+ QcomChipset,
QnnExecuTorchBackendType,
QnnExecuTorchHtpPerformanceMode,
)
@@ -99,6 +103,19 @@
from torchao.quantization.pt2e.quantizer import SharedQuantizationSpec
+class TestQNNDebuggerProfilePublicApis(unittest.TestCase):
+ def test_estimate_htp_profile_result_rejects_unsupported_soc_before_pte(self):
+ with self.assertRaisesRegex(
+ AssertionError,
+ "hextimate currently supports only.*SA8540.*SA8255.*QCS9100.*SA8797",
+ ):
+ estimate_htp_profile_result(
+ artifact_dir="/path/that/must/not/be/read",
+ soc_id=QcomChipset.SM8650,
+ pte_path="/path/that/must/not/be/read/model.pte",
+ )
+
+
class TestQNNFloatingPointOperator(TestQNN):
def setUp(self):
match get_backend_type(self.backend):
@@ -4443,19 +4460,20 @@ def test_qnn_backend_hadamard_transform_linear(self):
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
adb = self.get_adb_tool(pte_path)
- binaries_trace = generate_optrace(
- tmp_dir,
- self.chipset_table[TestQNN.soc_model],
- adb,
- pte_path,
- [sample_input],
+ artifacts = generate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[TestQNN.soc_model],
+ pte_path=pte_path,
+ inputs=[sample_input],
+ adb=adb,
)
htp_ops = []
- for _, (_, qhas) in binaries_trace.items():
- with open(qhas, "r") as qhas_file:
+ for artifact in artifacts:
+ self.assertIsNotNone(artifact.qhas_json)
+ with open(artifact.qhas_json, "r") as qhas_file:
qhas_data = json.load(qhas_file)
- for row in qhas_data["data"]["qnn_op_types"]["data"]:
- htp_ops.append(row["op"])
+ for row in qhas_data["data"]["qnn_op_types"]["data"]:
+ htp_ops.append(row["op"])
self.assertTrue(
any("HadamardTransform" in op for op in htp_ops),
"Expected linear to be lowered to HadamardTransform "
@@ -4503,19 +4521,20 @@ def test_qnn_backend_hadamard_transform_matmul(self):
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
adb = self.get_adb_tool(pte_path)
- binaries_trace = generate_optrace(
- tmp_dir,
- self.chipset_table[TestQNN.soc_model],
- adb,
- pte_path,
- [sample_input],
+ artifacts = generate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[TestQNN.soc_model],
+ pte_path=pte_path,
+ inputs=[sample_input],
+ adb=adb,
)
htp_ops = []
- for _, (_, qhas) in binaries_trace.items():
- with open(qhas, "r") as qhas_file:
+ for artifact in artifacts:
+ self.assertIsNotNone(artifact.qhas_json)
+ with open(artifact.qhas_json, "r") as qhas_file:
qhas_data = json.load(qhas_file)
- for row in qhas_data["data"]["qnn_op_types"]["data"]:
- htp_ops.append(row["op"])
+ for row in qhas_data["data"]["qnn_op_types"]["data"]:
+ htp_ops.append(row["op"])
self.assertTrue(
any("HadamardTransform" in op for op in htp_ops),
"Expected matmul to be lowered to HadamardTransform "
@@ -4557,19 +4576,20 @@ def test_qnn_backend_hadamard_transform_conv(self):
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
adb = self.get_adb_tool(pte_path)
- binaries_trace = generate_optrace(
- tmp_dir,
- self.chipset_table[TestQNN.soc_model],
- adb,
- pte_path,
- [sample_input],
+ artifacts = generate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[TestQNN.soc_model],
+ pte_path=pte_path,
+ inputs=[sample_input],
+ adb=adb,
)
htp_ops = []
- for _, (_, qhas) in binaries_trace.items():
- with open(qhas, "r") as qhas_file:
+ for artifact in artifacts:
+ self.assertIsNotNone(artifact.qhas_json)
+ with open(artifact.qhas_json, "r") as qhas_file:
qhas_data = json.load(qhas_file)
- for row in qhas_data["data"]["qnn_op_types"]["data"]:
- htp_ops.append(row["op"])
+ for row in qhas_data["data"]["qnn_op_types"]["data"]:
+ htp_ops.append(row["op"])
self.assertTrue(
any("HadamardTransform" in op for op in htp_ops),
"Expected conv to be lowered to HadamardTransform "
@@ -6523,19 +6543,20 @@ def test_qnn_backend_activation_fusion(self):
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
adb = self.get_adb_tool(pte_path)
- binaries_trace = generate_optrace(
- tmp_dir,
- self.chipset_table[TestQNN.soc_model],
- adb,
- pte_path,
- [tc[QCOM_SAMPLE_INPUTS]],
+ artifacts = generate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[TestQNN.soc_model],
+ pte_path=pte_path,
+ inputs=[tc[QCOM_SAMPLE_INPUTS]],
+ adb=adb,
)
htp_ops = []
- for _, (_, qhas) in binaries_trace.items():
- with open(qhas, "r") as qhas_file:
- qhas_data = json.load(qhas_file)
- for row in qhas_data["data"]["htp_op_types"]["data"]:
- htp_ops.append(row["op"])
+ for a in artifacts:
+ self.assertIsNotNone(a.qhas_json)
+ with open(a.qhas_json, "r") as f:
+ qhas_data = json.load(f)
+ for row in qhas_data["data"]["htp_op_types"]["data"]:
+ htp_ops.append(row["op"])
has_conv = any("ConvLayer" in op for op in htp_ops)
self.assertTrue(
has_conv, f"Expected Conv op in HTP ops, got: {htp_ops}"
@@ -6652,20 +6673,21 @@ def test_qnn_backend_masked_softmax(self):
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
adb = self.get_adb_tool(pte_path)
- binaries_trace = generate_optrace(
- tmp_dir,
- self.chipset_table[self.soc_model],
- adb,
- pte_path,
- [sample_input],
+ artifacts = generate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[self.soc_model],
+ pte_path=pte_path,
+ inputs=[sample_input],
+ adb=adb,
)
has_masked_softmax = False
- for _, (_, qhas) in binaries_trace.items():
- with open(qhas, "r") as qhas_file:
- qhas_data = json.load(qhas_file)
- for row in qhas_data["data"]["htp_op_types"]["data"]:
- if "MaskedSoftmax" in row["op"]:
- has_masked_softmax = True
+ for a in artifacts:
+ self.assertIsNotNone(a.qhas_json)
+ with open(a.qhas_json, "r") as f:
+ qhas_data = json.load(f)
+ for row in qhas_data["data"]["htp_op_types"]["data"]:
+ if "MaskedSoftmax" in row["op"]:
+ has_masked_softmax = True
self.assertTrue(has_masked_softmax)
@unittest.skip("UT pass before QNN 2.26, segfault during partitioner")
@@ -7356,13 +7378,16 @@ def test_qnn_backend_dump_context_from_pte(self):
module, sample_input, compiler_spec
).to_executorch()
- with tempfile.TemporaryDirectory() as tmp_dir:
+ with (
+ tempfile.TemporaryDirectory() as tmp_dir,
+ tempfile.TemporaryDirectory() as dump_dir,
+ ):
pte_path = f"{tmp_dir}/model.pte"
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
- dump_context_from_pte(pte_path)
- binary_name = f"{tmp_dir}/forward_0.bin"
+ dump_context_from_pte(pte_path, output_dir=dump_dir)
+ binary_name = f"{dump_dir}/forward_0.bin"
self.assertTrue(os.path.isfile(binary_name))
with open(binary_name, "rb") as f:
stripped_binary = f.read()
@@ -7512,12 +7537,17 @@ def test_qnn_backend_draw_graph(self):
def test_qnn_backend_generate_optrace(self):
if self.enable_x86_64:
self.skipTest(
- "At the moment, testing is only being conducted on the device."
+ "Optrace requires on-device execution; not supported on x86_64 host."
)
module = SimpleModel() # noqa: F405
sample_input = (torch.ones(1, 32, 28, 28), torch.ones(1, 32, 28, 28))
backend_options = generate_htp_compiler_spec(use_fp16=True)
+ # Two compiler specs exercise both prepare modes:
+ # - online-prepare: profile_level is set by the QNN CLI at host-side
+ # context-binary-generation time, so no profile_level here.
+ # - offline-prepare: profile_level=3 is REQUIRED at AoT so the .pte's
+ # embedded context binary carries optrace instrumentation and schematic.bin.
compiler_specs = [
generate_qnn_executorch_compiler_spec(
soc_model=self.chipset_table[TestQNN.soc_model],
@@ -7532,40 +7562,33 @@ def test_qnn_backend_generate_optrace(self):
]
for compiler_spec in compiler_specs:
+ edge_prog_mgr = to_edge_transform_and_lower_to_qnn(
+ module, sample_input, compiler_spec
+ ).to_executorch()
with tempfile.TemporaryDirectory() as tmp_dir:
- edge_prog_mgr = to_edge_transform_and_lower_to_qnn(
- module, sample_input, compiler_spec
- ).to_executorch()
pte_path = f"{tmp_dir}/model.pte"
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
adb = self.get_adb_tool(pte_path)
- binaries_trace = generate_optrace(
- tmp_dir,
- self.chipset_table[self.soc_model],
- adb,
- pte_path,
- [sample_input],
+ artifacts = generate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[self.soc_model],
+ pte_path=pte_path,
+ inputs=[sample_input],
+ adb=adb,
)
- for _, (optrace, qhas) in binaries_trace.items():
- with open(optrace, "r") as optrace_file:
- optrace_data = json.load(optrace_file)
- # {
- # header:
- # {
- # 'header_version': {'major': x, 'minor': y, 'patch': z},
- # 'version': {'major': x, 'minor': y, 'patch': z},
- # 'artifact_type': 'OP_TRACE'
- # }
- # traceEvents:
- # {...}
- # }
- for row in optrace_data["traceEvents"]:
- self.assertIn("pid", row)
- with open(qhas, "r") as qhas_file:
- qhas_data = json.load(qhas_file)
- self.assertIn("data", qhas_data)
+ for a in artifacts:
+ with open(a.chrometrace_json, "r") as f:
+ chrometrace = json.load(f)
+ for row in chrometrace["traceEvents"]:
+ self.assertIn("pid", row)
+ self.assertIsNotNone(
+ a.qhas_json,
+ "optrace mode should produce a valid QHAS JSON.",
+ )
+ with open(a.qhas_json, "r") as f:
+ self.assertIn("data", json.load(f))
class TestQNNQuantizedUtils(TestQNN):
@@ -8603,7 +8626,7 @@ def test_qnn_backend_draw_graph(self):
def test_qnn_backend_generate_optrace(self):
if self.enable_x86_64:
self.skipTest(
- "At the moment, testing is only being conducted on the device."
+ "Optrace requires on-device execution; not supported on x86_64 host."
)
if get_backend_type(self.backend) == QnnExecuTorchBackendType.kLpaiBackend:
self.skipTest("LPAI does not support optrace generation.")
@@ -8612,6 +8635,11 @@ def test_qnn_backend_generate_optrace(self):
module = self.get_qdq_module(module, sample_input)
backend_options = generate_htp_compiler_spec(use_fp16=True)
+ # Two compiler specs exercise both prepare modes:
+ # - online-prepare: profile_level is set by the QNN CLI at host-side
+ # context-binary-generation time, so no profile_level here.
+ # - offline-prepare: profile_level=3 is REQUIRED at AoT so the .pte's
+ # embedded context binary carries optrace instrumentation and schematic.bin.
compiler_specs = [
generate_qnn_executorch_compiler_spec(
soc_model=self.chipset_table[TestQNN.soc_model],
@@ -8626,40 +8654,83 @@ def test_qnn_backend_generate_optrace(self):
]
for compiler_spec in compiler_specs:
+ edge_prog_mgr = to_edge_transform_and_lower_to_qnn(
+ module, sample_input, compiler_spec
+ ).to_executorch()
with tempfile.TemporaryDirectory() as tmp_dir:
- edge_prog_mgr = to_edge_transform_and_lower_to_qnn(
- module, sample_input, compiler_spec
- ).to_executorch()
pte_path = f"{tmp_dir}/model.pte"
with open(pte_path, "wb") as f:
edge_prog_mgr.write_to_file(f)
adb = self.get_adb_tool(pte_path)
- binaries_trace = generate_optrace(
- tmp_dir,
- self.chipset_table[self.soc_model],
- adb,
- pte_path,
- [sample_input],
+ artifacts = generate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[self.soc_model],
+ pte_path=pte_path,
+ inputs=[sample_input],
+ adb=adb,
+ )
+ for a in artifacts:
+ with open(a.chrometrace_json, "r") as f:
+ chrometrace = json.load(f)
+ for row in chrometrace["traceEvents"]:
+ self.assertIn("pid", row)
+ self.assertIsNotNone(
+ a.qhas_json,
+ "optrace mode should produce a valid QHAS JSON ",
+ )
+ with open(a.qhas_json, "r") as f:
+ self.assertIn("data", json.load(f))
+
+ def test_qnn_backend_generate_hextimate(self):
+ if not self.enable_x86_64:
+ self.skipTest(
+ "Hextimate is host-side (compile-time); requires --enable_x86_64."
+ )
+ if get_backend_type(self.backend) == QnnExecuTorchBackendType.kLpaiBackend:
+ self.skipTest("LPAI does not support hextimate generation.")
+ module = SimpleModel() # noqa: F405
+ sample_input = (torch.ones(1, 32, 28, 28), torch.ones(1, 32, 28, 28))
+ module = self.get_qdq_module(module, sample_input)
+ backend_options = generate_htp_compiler_spec(use_fp16=True)
+
+ # Hextimate hard-requires online prepare (.dlc). No profile_level
+ # required — hextimate profiling is attached by the QNN CLI at
+ # context-binary-generation time.
+ compiler_spec = generate_qnn_executorch_compiler_spec(
+ soc_model=self.chipset_table[TestQNN.soc_model],
+ backend_options=backend_options,
+ online_prepare=True,
+ )
+
+ with tempfile.TemporaryDirectory() as tmp_dir:
+ edge_prog_mgr = to_edge_transform_and_lower_to_qnn(
+ module, sample_input, compiler_spec
+ ).to_executorch()
+ pte_path = f"{tmp_dir}/model.pte"
+ with open(pte_path, "wb") as f:
+ edge_prog_mgr.write_to_file(f)
+
+ artifacts = estimate_htp_profile_result(
+ artifact_dir=tmp_dir,
+ soc_id=self.chipset_table[self.soc_model],
+ pte_path=pte_path,
+ )
+ for a in artifacts:
+ with open(a.chrometrace_json, "r") as f:
+ chrometrace = json.load(f)
+ for row in chrometrace["traceEvents"]:
+ self.assertIn("pid", row)
+ # QHAS JSON is truncated by an upstream SDK bug (division by
+ # zero on time_us=0). We surface this by setting qhas_json to
+ # None; the HTML report and chrometrace remain usable.
+ self.assertIsNone(
+ a.qhas_json,
+ "hextimate QHAS JSON is expected to be truncated by the "
+ "SDK bug; QnnTool should have detected it and returned "
+ "qhas_json=None.",
)
- for _, (optrace, qhas) in binaries_trace.items():
- with open(optrace, "r") as optrace_file:
- optrace_data = json.load(optrace_file)
- # {
- # header:
- # {
- # 'header_version': {'major': x, 'minor': y, 'patch': z},
- # 'version': {'major': x, 'minor': y, 'patch': z},
- # 'artifact_type': 'OP_TRACE'
- # }
- # traceEvents:
- # {...}
- # }
- for row in optrace_data["traceEvents"]:
- self.assertIn("pid", row)
- with open(qhas, "r") as qhas_file:
- qhas_data = json.load(qhas_file)
- self.assertIn("data", qhas_data)
+ self.assertTrue(os.path.isfile(a.qhas_html))
def test_qnn_backend_seq_mse(self):
from executorch.backends.qualcomm._passes.seq_mse import SeqMSE
@@ -11439,9 +11510,12 @@ def test_custom_op_2(self):
self.assertTrue(msg["is_close"])
def test_debugger_generate_optrace(self):
+ # This test drives the offline-prepare demo (profile_level=3, no
+ # --online_prepare). See qairt_visualizer_demo_online.py for the
+ # online path.
cmds = [
"python",
- f"{self.executorch_root}/examples/qualcomm/util_scripts/qairt_visualizer_demo.py",
+ f"{self.executorch_root}/examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_offline.py",
"--artifact",
self.artifact_dir,
"--build_folder",
@@ -11468,25 +11542,17 @@ def test_debugger_generate_optrace(self):
msg = json.loads(conn.recv())
if "Error" in msg:
self.fail(msg["Error"])
- else:
- for _, (optrace, qhas) in msg["binaries_trace"].items():
- with open(optrace, "r") as optrace_file:
- optrace_data = json.load(optrace_file)
- # {
- # header:
- # {
- # 'header_version': {'major': x, 'minor': y, 'patch': z},
- # 'version': {'major': x, 'minor': y, 'patch': z},
- # 'artifact_type': 'OP_TRACE'
- # }
- # traceEvents:
- # {...}
- # }
- for row in optrace_data["traceEvents"]:
- self.assertIn("pid", row)
- with open(qhas, "r") as qhas_file:
- qhas_data = json.load(qhas_file)
- self.assertIn("data", qhas_data)
+ for a in msg["artifacts"]:
+ with open(a["chrometrace_json"], "r") as f:
+ chrometrace = json.load(f)
+ for row in chrometrace["traceEvents"]:
+ self.assertIn("pid", row)
+ self.assertIsNotNone(
+ a["qhas_json"],
+ "optrace mode should produce a valid QHAS JSON.",
+ )
+ with open(a["qhas_json"], "r") as f:
+ self.assertIn("data", json.load(f))
def test_intermediate_debugger(self):
cmds = [
diff --git a/backends/qualcomm/utils/utils.py b/backends/qualcomm/utils/utils.py
index 64328b520f1..aa8a665f62a 100644
--- a/backends/qualcomm/utils/utils.py
+++ b/backends/qualcomm/utils/utils.py
@@ -201,10 +201,10 @@ def replace_linear(module: torch.nn.Module):
return replace_linear(module)
-def dump_context_from_pte(pte_path) -> List[str]:
+def dump_context_from_pte(pte_path, output_dir=None) -> List[str]:
"""
- Dump compiled binaries under the same directory of pte_path.
- For partitioned graph, there will be multiple files with names f"{method_name}_{index}".
+ Dump compiled binaries under output_dir, or the same directory as pte_path
+ when output_dir is not set.
'method_name' refers to the name of a method in the nn.Module that was traced to
generate this program, while 'index' indicates the order of execution.
@@ -220,7 +220,8 @@ def dump_context_from_pte(pte_path) -> List[str]:
program = deserialize_pte_binary(program_data).program
- ctx_path = os.path.dirname(pte_path)
+ ctx_path = output_dir or os.path.dirname(pte_path)
+ os.makedirs(ctx_path, exist_ok=True)
dumpfiles = []
for execution_plan in program.execution_plan:
for i, delegate in enumerate(execution_plan.delegates):
@@ -1339,6 +1340,7 @@ def get_soc_to_chipset_map():
"SW6100": QcomChipset.SW6100,
"QCM6490": QcomChipset.QCM6490,
"SM8845": QcomChipset.SM8845,
+ "SA8540": QcomChipset.SA8540,
}
diff --git a/examples/qualcomm/util_scripts/README.md b/examples/qualcomm/util_scripts/README.md
index 0fc45c2ced3..00d7f87ab7f 100644
--- a/examples/qualcomm/util_scripts/README.md
+++ b/examples/qualcomm/util_scripts/README.md
@@ -78,6 +78,14 @@ This tool aims for users who want to deploy models with ExecuTorch runtime. It's
- `cli_example/execute_output/output_{data_index}_{output_index}.pt`.
`data_index` represents the sequence of dataset, `output_index` stands for the order of graph output.
+## HTP Profiling Examples
+
+These examples demonstrate Qualcomm HTP profiling flows:
+
+* `htp_profiling_on_device_op_trace_online.py`: on-device OpTrace with `online_prepare=True`.
+* `htp_profiling_on_device_op_trace_offline.py`: on-device OpTrace with `online_prepare=False` and `profile_level=3`.
+* `htp_profiling_on_host_hextimate.py`: host-side Hextimate.
+
# Generate ET Record
This section describes how to generate an ET record for a .pte program using the provided script.
* Generate ET record for .pte using the provided script:
diff --git a/examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_offline.py b/examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_offline.py
new file mode 100644
index 00000000000..4ecb703f1d0
--- /dev/null
+++ b/examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_offline.py
@@ -0,0 +1,132 @@
+# Copyright (c) Qualcomm Innovation Center, Inc.
+# All rights reserved
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""HTP profiling demo — on-device OpTrace with offline_prepare.
+
+This example exports a `.pte` with `QnnConfig.online_prepare=False`, generates
+on-device HTP OpTrace results with `generate_htp_profile_result()`, and opens
+the QHAS report with QAIRT Visualizer.
+
+Internal detail: offline_prepare embeds a finalized QNN context binary in the
+`.pte`, so `qnn_config.profile_level=3` is required during export.
+
+For the online_prepare OpTrace route, see
+htp_profiling_on_device_op_trace_online.py. For host-only Hextimate, see
+htp_profiling_on_host_hextimate.py.
+"""
+
+import json
+import os
+from multiprocessing.connection import Client
+
+import torch
+from executorch.backends.qualcomm.debugger.utils import generate_htp_profile_result
+from executorch.backends.qualcomm.export_utils import (
+ build_executorch_binary,
+ QnnConfig,
+ setup_common_args_and_variables,
+ SimpleADB,
+)
+from executorch.backends.qualcomm.quantizer.quantizer import QuantDtype
+from executorch.backends.qualcomm.tests.models import SimpleModel
+from executorch.backends.qualcomm.utils.utils import get_soc_to_chipset_map
+
+
+def main(args) -> None:
+ qnn_config = QnnConfig.load_config(args.config_file if args.config_file else args)
+ assert not qnn_config.online_prepare, (
+ "This demo uses on-device OpTrace with offline_prepare; remove "
+ "--online_prepare (or set online_prepare=False in your config file). "
+ "For online prepare, use htp_profiling_on_device_op_trace_online.py."
+ )
+ assert qnn_config.profile_level == 3, (
+ "Offline-prepare requires qnn_config.profile_level=3 so that the AoT "
+ "HtpContext bakes optrace instrumentation into the context binary "
+ "before qnn_context_get_binary() dumps it. Pass --profile_level 3 "
+ "(or set profile_level=3 in your config file)."
+ )
+
+ model = SimpleModel()
+ example_inputs = [(torch.ones(1, 32, 28, 28), torch.ones(1, 32, 28, 28))]
+
+ pte_filename = "qnn_simple_model"
+ os.makedirs(args.artifact, exist_ok=True)
+
+ build_executorch_binary(
+ model=model,
+ qnn_config=qnn_config,
+ file_name=f"{args.artifact}/{pte_filename}",
+ dataset=example_inputs,
+ quant_dtype=QuantDtype.use_8a8w,
+ )
+
+ adb = SimpleADB(
+ qnn_config=qnn_config,
+ pte_path=f"{args.artifact}/{pte_filename}.pte",
+ workspace=f"/data/local/tmp/executorch/{pte_filename}",
+ )
+ artifacts = generate_htp_profile_result(
+ artifact_dir=args.artifact,
+ soc_id=get_soc_to_chipset_map()[args.soc_model],
+ pte_path=f"{args.artifact}/{pte_filename}.pte",
+ inputs=example_inputs,
+ adb=adb,
+ )
+
+ if args.ip and args.port != -1:
+ with Client((args.ip, args.port)) as conn:
+ conn.send(
+ json.dumps(
+ {
+ "artifacts": [
+ {
+ "binary_path": a.binary_path,
+ "mode": a.mode,
+ "prepare_mode": a.prepare_mode,
+ "chrometrace_json": a.chrometrace_json,
+ "qhas_json": a.qhas_json,
+ "qhas_html": a.qhas_html,
+ }
+ for a in artifacts
+ ],
+ }
+ )
+ )
+ return
+
+ try:
+ import qairt_visualizer
+ except ImportError:
+ for a in artifacts:
+ print(f"QHAS HTML: {a.qhas_html}")
+ return
+
+ # Offline-prepare emits .bin binaries, which do not support the graph
+ # view in qairt-visualizer — only the reports are shown.
+ for a in artifacts:
+ qairt_visualizer.view(reports=a.visualizer_reports())
+
+
+if __name__ == "__main__":
+ parser = setup_common_args_and_variables()
+ parser.add_argument(
+ "-a",
+ "--artifact",
+ type=str,
+ default="",
+ help="The folder to store the exported program",
+ )
+
+ args = parser.parse_args()
+
+ try:
+ main(args)
+ except Exception as e:
+ if args.ip and args.port != -1:
+ with Client((args.ip, args.port)) as conn:
+ conn.send(json.dumps({"Error": str(e)}))
+ else:
+ raise Exception(e)
diff --git a/examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_online.py b/examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_online.py
new file mode 100644
index 00000000000..5a8d216883e
--- /dev/null
+++ b/examples/qualcomm/util_scripts/htp_profiling_on_device_op_trace_online.py
@@ -0,0 +1,130 @@
+# Copyright (c) Qualcomm Innovation Center, Inc.
+# All rights reserved
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""HTP profiling demo — on-device OpTrace with online_prepare.
+
+This example exports a `.pte` with `QnnConfig.online_prepare=True`, generates
+on-device HTP OpTrace results with `generate_htp_profile_result()`, and opens
+the QHAS report with QAIRT Visualizer.
+
+Internal detail: online_prepare stores a graph description in the `.pte`; QNN
+tools create the profiled context and schematic during profiling.
+
+For the offline_prepare OpTrace route, see
+htp_profiling_on_device_op_trace_offline.py. For host-only Hextimate, see
+htp_profiling_on_host_hextimate.py.
+"""
+
+import json
+import os
+from multiprocessing.connection import Client
+
+import torch
+from executorch.backends.qualcomm.debugger.utils import generate_htp_profile_result
+from executorch.backends.qualcomm.export_utils import (
+ build_executorch_binary,
+ QnnConfig,
+ setup_common_args_and_variables,
+ SimpleADB,
+)
+from executorch.backends.qualcomm.quantizer.quantizer import QuantDtype
+from executorch.backends.qualcomm.tests.models import SimpleModel
+from executorch.backends.qualcomm.utils.utils import get_soc_to_chipset_map
+
+
+def main(args) -> None:
+ qnn_config = QnnConfig.load_config(args.config_file if args.config_file else args)
+ assert qnn_config.online_prepare, (
+ "This demo uses on-device OpTrace with online_prepare; pass "
+ "--online_prepare (or set online_prepare=True in your config file). "
+ "For offline prepare, use htp_profiling_on_device_op_trace_offline.py."
+ )
+
+ model = SimpleModel()
+ example_inputs = [(torch.ones(1, 32, 28, 28), torch.ones(1, 32, 28, 28))]
+
+ pte_filename = "qnn_simple_model"
+ os.makedirs(args.artifact, exist_ok=True)
+
+ # lower to QNN
+ build_executorch_binary(
+ model=model,
+ qnn_config=qnn_config,
+ file_name=f"{args.artifact}/{pte_filename}",
+ dataset=example_inputs,
+ quant_dtype=QuantDtype.use_8a8w,
+ )
+
+ adb = SimpleADB(
+ qnn_config=qnn_config,
+ pte_path=f"{args.artifact}/{pte_filename}.pte",
+ workspace=f"/data/local/tmp/executorch/{pte_filename}",
+ )
+ artifacts = generate_htp_profile_result(
+ artifact_dir=args.artifact,
+ soc_id=get_soc_to_chipset_map()[args.soc_model],
+ pte_path=f"{args.artifact}/{pte_filename}.pte",
+ inputs=example_inputs,
+ adb=adb,
+ )
+
+ if args.ip and args.port != -1:
+ with Client((args.ip, args.port)) as conn:
+ conn.send(
+ json.dumps(
+ {
+ "artifacts": [
+ {
+ "binary_path": a.binary_path,
+ "mode": a.mode,
+ "prepare_mode": a.prepare_mode,
+ "chrometrace_json": a.chrometrace_json,
+ "qhas_json": a.qhas_json,
+ "qhas_html": a.qhas_html,
+ }
+ for a in artifacts
+ ],
+ }
+ )
+ )
+ return
+
+ try:
+ import qairt_visualizer
+ except ImportError:
+ for a in artifacts:
+ print(f"QHAS HTML: {a.qhas_html}")
+ return
+
+ # Visualize each binary. Online-prepare emits .dlc, which supports the
+ # graph view; offline .bin binaries do not.
+ for a in artifacts:
+ if a.binary_path.endswith(".dlc"):
+ qairt_visualizer.view(a.binary_path, reports=a.visualizer_reports())
+ else:
+ qairt_visualizer.view(reports=a.visualizer_reports())
+
+
+if __name__ == "__main__":
+ parser = setup_common_args_and_variables()
+ parser.add_argument(
+ "-a",
+ "--artifact",
+ type=str,
+ default="",
+ help="The folder to store the exported program",
+ )
+
+ args = parser.parse_args()
+
+ try:
+ main(args)
+ except Exception as e:
+ if args.ip and args.port != -1:
+ with Client((args.ip, args.port)) as conn:
+ conn.send(json.dumps({"Error": str(e)}))
+ else:
+ raise Exception(e)
diff --git a/examples/qualcomm/util_scripts/qairt_visualizer_demo.py b/examples/qualcomm/util_scripts/htp_profiling_on_host_hextimate.py
similarity index 51%
rename from examples/qualcomm/util_scripts/qairt_visualizer_demo.py
rename to examples/qualcomm/util_scripts/htp_profiling_on_host_hextimate.py
index 9d2ce7a8806..b3e9f16dcce 100644
--- a/examples/qualcomm/util_scripts/qairt_visualizer_demo.py
+++ b/examples/qualcomm/util_scripts/htp_profiling_on_host_hextimate.py
@@ -4,18 +4,26 @@
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.
+"""HTP profiling demo — Hextimate on host.
+
+This example exports a `.pte` with `QnnConfig.online_prepare=True`, estimates
+HTP performance on host with `estimate_htp_profile_result()`, and opens the
+QHAS report with QAIRT Visualizer.
+
+Hextimate is compile-time estimation, not an on-device OpTrace run. It does not
+use sample inputs or adb during profiling.
+"""
+
import json
import os
from multiprocessing.connection import Client
-import qairt_visualizer
import torch
-from executorch.backends.qualcomm.debugger.utils import generate_optrace
+from executorch.backends.qualcomm.debugger.utils import estimate_htp_profile_result
from executorch.backends.qualcomm.export_utils import (
build_executorch_binary,
QnnConfig,
setup_common_args_and_variables,
- SimpleADB,
)
from executorch.backends.qualcomm.quantizer.quantizer import QuantDtype
from executorch.backends.qualcomm.tests.models import SimpleModel
@@ -24,17 +32,17 @@
def main(args) -> None:
qnn_config = QnnConfig.load_config(args.config_file if args.config_file else args)
+ assert qnn_config.online_prepare, (
+ "This demo uses host-only Hextimate; pass --online_prepare "
+ "(or set online_prepare=True in your config file)."
+ )
+
model = SimpleModel()
example_inputs = [(torch.ones(1, 32, 28, 28), torch.ones(1, 32, 28, 28))]
pte_filename = "qnn_simple_model"
os.makedirs(args.artifact, exist_ok=True)
- assert (
- qnn_config.profile_level == 3
- ), "Please turn profile_level to 3 for the purpose of this tutorial."
-
- # lower to QNN
build_executorch_binary(
model=model,
qnn_config=qnn_config,
@@ -43,32 +51,43 @@ def main(args) -> None:
quant_dtype=QuantDtype.use_8a8w,
)
- # generate optrace and QHAS
- adb = SimpleADB(
- qnn_config=qnn_config,
+ artifacts = estimate_htp_profile_result(
+ artifact_dir=args.artifact,
+ soc_id=get_soc_to_chipset_map()[args.soc_model],
pte_path=f"{args.artifact}/{pte_filename}.pte",
- workspace=f"/data/local/tmp/executorch/{pte_filename}",
- )
- binaries_trace = generate_optrace(
- args.artifact,
- get_soc_to_chipset_map()[args.soc_model],
- adb,
- f"{args.artifact}/{pte_filename}.pte",
- example_inputs,
)
if args.ip and args.port != -1:
with Client((args.ip, args.port)) as conn:
- conn.send(json.dumps({"binaries_trace": binaries_trace}))
- else:
- # Visualize the model and reports
- for binary, (optrace, qhas) in binaries_trace.items():
- file_extension = os.path.splitext(binary)[-1]
- if file_extension == ".bin":
- qairt_visualizer.view(reports=[optrace, qhas])
- elif file_extension == ".dlc":
- # We only show graph for dlc binary
- qairt_visualizer.view(binary, reports=[optrace, qhas])
+ conn.send(
+ json.dumps(
+ {
+ "artifacts": [
+ {
+ "binary_path": a.binary_path,
+ "mode": a.mode,
+ "prepare_mode": a.prepare_mode,
+ "chrometrace_json": a.chrometrace_json,
+ "qhas_json": a.qhas_json,
+ "qhas_html": a.qhas_html,
+ }
+ for a in artifacts
+ ],
+ }
+ )
+ )
+ return
+
+ try:
+ import qairt_visualizer
+ except ImportError:
+ for a in artifacts:
+ print(f"QHAS HTML: {a.qhas_html}")
+ return
+
+ for a in artifacts:
+ qairt_visualizer.view(a.binary_path, reports=a.visualizer_reports())
+ print(f"QHAS HTML: {a.qhas_html}")
if __name__ == "__main__":