Tencent has unveiled AngelSpec, an innovative open-source, PyTorch-native framework designed to revolutionize the training of speculative decoding draft models. Moving beyond the conventional approach of seeking a single, averaged drafter, AngelSpec prioritizes real-world workload heterogeneity. It offers a unified framework supporting both autoregressive multi-token prediction (MTP) and the block-parallel DFlash family, allowing developers to tailor draft models precisely to their specific application needs.
Speculative decoding accelerates inference by having a lightweight drafter propose future tokens, which the larger target model then verifies in a single pass. AngelSpec recognizes that the efficiency of this process—how many tokens are accepted and how long the round takes—varies significantly across domains. For high-entropy tasks like open-ended conversation, where many continuations are valid, MTP's shorter candidate sequences are more effective. Conversely, for low-entropy tasks such as code generation or mathematical reasoning, where longer predictable spans exist, block-parallel drafting excels. To address this, AngelSpec provides two specialized drafters: an MTP model trained on diverse conversational data and a block-diffusion model strengthened with code and mathematics samples.
Crucially, AngelSpec tackles the common train-inference mismatch seen in prior models, where errors accumulate during recurrent unrolling. It implements a shared-parameter, multi-depth scheme where a single MTP block is autoregressively unrolled for multiple steps during training, with predictions from one depth feeding the next. This "Training-Time Test" principle, inspired by EAGLE-3, ensures that the model is robustly prepared for long self-generated chains. By sharing parameters while maintaining depth-specific supervision, AngelSpec significantly improves acceptance rates at deeper draft positions, offering researchers and developers a powerful tool to train more efficient, reliable, and specialized speculative decoding models for a wide array of AI applications.