Speech intelligence / XPENG AI
X-AuT.
Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation
arXiv 
GitHub 
Hugging Face 
01 / Overview
Compress the encoder. Preserve the speech behavior.
Removing complete audio-encoder blocks gives speech language models a regular, deployment-friendly architecture, but it also perturbs the embeddings consumed by the decoder. The resulting mismatch can trigger deletion errors and premature end-of-sequence predictions.
X-AuT treats layer selection and recovery as a coupled problem. Starting from Qwen3-ASR-0.6B, it uses matched short-budget behavioral probes to select recoverable layer combinations, then restores the pruned model with representation alignment, cross-scale distillation, scheduled student-policy supervision, and LoRA finetuning.
02 / Method
A progressive path from data selection to recovery.
-
01
Transcript-consistency filtering
A source pool exceeding 280k hours is ranked into nine agreement tiers using the source transcript and two offline ASR hypotheses. Reported training uses the highest-agreement class, with source reweighting during final finetuning.
-
02
Behavior-driven layer selection
Matched recovery probes compare candidate removals on a fixed development suite. X-AuT first removes original layers {1, 18}, then selects {5, 6} for the second pruning hop; pair interactions are measured rather than inferred from single-layer rankings.
-
03
Three-stage recovery
Stage 0 aligns intermediate, bridge, and logit representations. Stage 1 combines teacher-forced distillation with scheduled student-policy contexts. Stage 2 performs low-rate gold-transcript LoRA finetuning.
-
04
Cross-scale supervision
A frozen Qwen3-ASR-1.7B teacher guides the smaller student through learned 2048-to-1024 projections and is discarded at inference. The pretrained language-model backbone remains frozen while attention LoRA adapters recover the decoder interface.
03 / Results
Two practical accuracy-efficiency operating points.
Evaluation covers ten public Chinese-English benchmarks, using CER for Chinese and WER for English. The macro mean weights each benchmark equally.
| Model | Audio-tower params | Param. change | Macro error | Relative change |
|---|---|---|---|---|
| Full-18 baseline | 186.376M | - | 5.61% | - |
| X-AuT-16, Stage 2 | 167.085M | −10.35% | 5.27% | −6.1% |
| X-AuT-14, Stage 2 | 147.794M | −20.70% | 5.75% | +2.5% |
14-layer inference efficiency
| Metric vs. Full-18 | In-vehicle PPU | NVIDIA H800 |
|---|---|---|
| Encoder latency | −21.4% | −11.4% |
| End-to-end latency | −4.7% | −2.6% |
| Peak memory | −4.4% | −2.8% |
Results are descriptive single-run measurements from one model family. The study does not include repeated seeds, utterance-level confidence intervals, or run-to-run latency variance.
04 / Open release
Use the model and adapt it.
The public release includes the complete 14-layer checkpoint, standalone inference, and a compact LoRA adaptation example. The example is a practical Stage-2-style recipe, not a full reproduction of the private three-stage training and data pipeline.
05 / Reference
Cite X-AuT.
@misc{zhang2026xautprogressiveaudioencodercompression,
title={X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation},
author={Haojun Zhang and Yi Zou and Min Chen and Qize Yu and Lianrui Fan and Xini Ding and Hao Li and Shuchang Zhou and Xianming Liu and Shiyu Huang},
year={2026},
eprint={2609.11412},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2609.11412},
}