Skip to content

feat(xpu): add dynamic W8A8 INT8 inference for MiniMax-H3 - #1420

Merged
helloyongyang merged 10 commits into
ModelTC:mainfrom
qiuxin2012:gemm_int8
Aug 24, 2026
Merged

feat(xpu): add dynamic W8A8 INT8 inference for MiniMax-H3#1420
helloyongyang merged 10 commits into
ModelTC:mainfrom
qiuxin2012:gemm_int8

Conversation

@qiuxin2012

@qiuxin2012 qiuxin2012 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds dynamic W8A8 INT8 inference support for MiniMax-H3 on Intel XPU.

Kernel

  • Add Intel XPU dynamic W8A8 INT8 GEMM kernel.
  • Add int8-intel-xpu support for MiniMax-H3.
  • Add INT8 checkpoint conversion and inference scripts.
  • Exposes the following API:
sycl_kernels.onednn_w8a8_int8(
    input,
    qweight,
    weight_scales,
    bias=None,
)

oneDNN verbose confirms that the INT8 GEMM path is used:

  matmul,jit:gemm:any
  src:s8 wei:s8 dst:bf16
  attr-scales:src...+wei...

INT8 checkpoint conversion

Run from the LightX2V repository root:

  SOURCE_DIR=/llm/models/MiniMax-H3/transformer \
  OUTPUT_DIR=/llm/models/MiniMax-H3/quantized/int8 \
  bash tools/convert/examples/convert_minimax_h3_int8_xpu.sh

Equivalent converter command:

  python tools/convert/converter.py \
    --source /llm/models/MiniMax-H3/transformer \
    --output /llm/models/MiniMax-H3/quantized/int8 \
    --output_name minimax_h3_int8_xpu \
    --model_type h3 \
    --quantized \
    --linear_type int8 \
    --device cpu \
    --single_file

The generated checkpoint is:

  /llm/models/MiniMax-H3/quantized/int8/minimax_h3_int8_xpu.safetensors

The converted checkpoint contains 350 INT8 weights and 350 per-channel weight scales. Its size is approximately 32 GB.

Inference

  bash scripts/platforms/intel_xpu/run_minimax_h3_t2av_int8.sh

The default inference configuration uses:

  DTYPE=BF16
  SENSITIVE_LAYER_DTYPE=BF16
  dit_quant_scheme=int8-intel-xpu

Validation

  • Kernel tests: 21 passed
  • Standalone LightX2V linear relative RMS error: 0.00173
  • Full MiniMax-H3 T2AV inference completed successfully on B65 and 4xB60
  • Generated video was visually checked and showed no obvious quantization corruption.

@helloyongyang
helloyongyang merged commit d6d21d2 into ModelTC:main Aug 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants