Model Architecture and Specifications
Muse Glimmer, a 30-billion-parameter multimodal model, has been released by Meta AI. The model is a dense causal transformer with a dedicated perception encoder, utilizing grouped-query attention with 32 query heads and 2 KV heads. The attention pattern repeats in a [Local, Local, Local, Global] sequence with a 2,048 sliding window. Key specifications include:
- Total parameters: approximately 30B, including the vision tower
- Context length: 131,072+, vocabulary: 202,048 tokens
- Input: text and image, output: text
- Weight precision: approximately 4-bit, reducing memory requirements to under 20 GB
Licensing and Deployment
Muse Glimmer is licensed under Apache 2.0, with open weights available. The model can run on a single consumer GPU or a Mac, with no network call required. Two quantized builds are available: K-Quant-Dynamic (32 GB VRAM, 0.2% average degradation) and K-Quant-17GB (24 GB VRAM, 1.0% average degradation).
Performance and Comparison
Muse Glimmer demonstrates improved generation speed using DFlash, a block-diffusion drafter. The model outperforms Gemma4-31B and Qwen3.6-27B in thinking mode on several benchmarks, including MCP Atlas and DeepSearch QA. However, it trails behind Qwen3.6-27B on certain tasks, such as OSWorld-Verified and TerminalBench 2.1.