Jalapeño’s first results show industry-leading speed and efficiency in AI inference

OpenAI’s first custom inference silicon, Jalapeño, was built using AI‑driven design tools and then tuned so that its own models could program and optimize the chip. The single‑die architecture targets both throughput and latency, eliminating the typical trade‑off seen in existing accelerators. By measuring useful AI work completed per watt while meeting interactive latency requirements, the chip shows higher token output and lower response times across a range of workloads.

On the public InferenceX benchmark, Jalapeño outperformed leading commercial systems on GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. It delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end‑to‑end latency than the comparison set. For highly interactive scenarios the advantage rose to 2.1 to 4.1 times higher performance. The chip is rated at 700 W but sustained draw stayed at or below 550 W during testing, placing it on the Pareto frontier for performance‑per‑watt versus latency across all three model families.

• Chip power rating: 700 W (measured ≤550 W sustained)
• Performance gains: 1.5‑1.9× AI work/W, 1.7‑3.6× lower latency, 2.1‑4.1× interactive performance
• Models validated: GPT‑OSS 120B, DeepSeek R1 670B, Kimi K2.5 1T
• Benchmark: InferenceX (SemiAnalysis) with normalized power‑based comparison

Why this matters

The results indicate that co‑designing models, software, and silicon within a single organization can yield measurable efficiency gains without sacrificing speed, suggesting a shift toward vertically integrated AI infrastructure. While the source data shows Jalapeño’s current advantages on specific benchmarks, the inference that this approach will scale to future, larger workloads remains speculative pending further generational releases and broader ecosystem adoption.