OpenAI's First In-House Chip Shows Its Work: Jalapeño Runs 700W Against Nvidia's 1,400W, and the Fine Print Is the Interesting Part

At Hot Chips on August 25, OpenAI and Broadcom detailed Jalapeño, the inference-only ASIC they built together, and released benchmark numbers for the first time. The part is a 700W TDP design with HBM4 at 15.4TB/s, a TSMC N3P compute die paired with an N3E I/O chiplet, packaged on CoWoS. On SemiAnalysis's open-source InferenceX suite, the published figures show 1.5x to 1.9x more throughput per kilowatt and 1.7x to 3.6x lower end-to-end latency than Nvidia GB200 and GB300 racks drawing 1,200W to 1,400W. SemiAnalysis spelled out the limits itself: the numbers came from OpenAI, only an 8k-input/1k-output shape was tested, the Nvidia configurations used multi-token prediction while Jalapeño used single-token prediction, the more production-like AgentX suite wasn't run, and these are engineering samples with volume ramping through 2027.

It Isn't the "Tuned for OpenAI's Own Models" Kind of Chip

OpenAI describes Jalapeño as a pure inference ASIC — not a training accelerator repurposed, not a general AI processor — designed around three bottlenecks that bite when inference runs at scale: the cost of moving data, the balance between compute and memory, and networking efficiency. Richard Ho, who runs hardware at OpenAI, framed the result as serving more AI work per unit of power while returning responses faster. Interestingly, SemiAnalysis — which ran the benchmark suite — disagrees with the "specialized for OpenAI's models" framing, arguing it is a generalized inference chip that runs anything. The models used in testing back that up: GPT-OSS 120B, DeepSeek R1 and Kimi K2.5, none of them OpenAI's.

Read the Numbers Together With Their Conditions

The published comparison is a 700W Jalapeño against 1,200W and 1,400W Nvidia racks, at 1.5–1.9x throughput per kilowatt and 1.7–3.6x lower end-to-end latency. The numbers look good, and SemiAnalysis is blunt about where they come from: all of them were provided by OpenAI, who invited the analysts into their lab to confirm the silicon is real, and testing covered a single shape — 8k tokens in, 1k out. The load-bearing caveat is that the competing configurations used multi-token prediction while Jalapeño ran single-token prediction with no speculative decoding, so the two aren't strictly comparable. AgentX, the suite SemiAnalysis actually prefers because it stresses routers and prefix caches the way production does, wasn't run at all. They also propose a fairer reference point: Jalapeño should be measured against Vera Rubin, which likewise uses HBM4, rather than the previous-generation Blackwell. And they offer a counterintuitive read of the lead — it may reflect how quickly the software stack came up rather than superior hardware.

Two Timelines, and Nothing Lands Before 2027

OpenAI and Broadcom say the chip went from initial design to tape-out in nine months, which they present as among the fastest cycles ever for a high-performance part, helped by OpenAI's own models accelerating pieces of the design and optimization work. SemiAnalysis tells it differently: design started in mid-2024, roughly 16 months from assembling the team to tape-out, with the CoWoS tape-out in November 2025. The gap is about where you start counting, but the implication for readers is the same — this didn't appear overnight. The deployment schedule matters more. Small volumes go into OpenAI's own data centers by the end of the year, with a gradual ramp across 2027; what exists today are engineering samples, with a B0 stepping in fab said to add roughly 25% in performance per watt. None of this changes anyone's inference bill in 2026. The significance is elsewhere: a model lab building its own inference silicon has, for the first time, put out numbers that a third-party benchmark can be pointed at, rather than a bar chart from a launch event.

via: SemiAnalysis, "OpenAI Jalapeño: Better Than Nvidia Blackwell", OpenAI's announcement, TechCrunch