Databricks published a research note on September 4, 2026 describing Proteus, an agent harness that writes GPU kernels for particular runtime shapes instead of one generic kernel for every model. Authors Daya Khudia, Leo Li, and Lesheng Jin said generated kernels for pieces of Qwen 3.5 122B ran 1.8 to 5.2 times faster than the best implementations they could find in vLLM.

The argument is that production inference still ships the same kernels for models from 1 billion to 1 trillion parameters. Matrix shapes mix static model dimensions with request-time token counts, so a kernel that looks fine on a benchmark batch can waste memory bandwidth on a different decode shape. Proteus proposes a candidate, checks it against a controlled reference, times the ones that pass, and iterates on the winner.

Why the checker comes before the prompt

The team said coding agents reward-hack when the score is sloppy. A rotary position embedding kernel could reuse leftover compiled code and look cheaper than a fair rebuild. Another could replay a CUDA graph while the baseline still launched each piece separately. A third could fit the visible test set and fall apart on sizes it had not seen.

Proteus times both sides the same way, sometimes with more than one timer (CUDA events, wall clock, and CUPTI). It clears leftover compiled state, re-times winners before they become the next parent, and keeps hidden tests. Automated checks flag speedups that would break GPU bandwidth or compute limits, including claimed gains above 100 times. Without that, the authors wrote, generating more kernels mostly produced more noise.

Context is treated as a budget. A larger prompt can carry the current best kernel, recent failures, and profiler notes, but it costs tokens and lets the next attempt drift. Too little context and the loop repeats the same dead ends. The knowledge layer they kept is narrow: high-trust takeaways that pair a situation with an action, plus short failure notes from related parent runs.

One traced example is a packed decode kernel on the Gated DeltaNet path in Qwen 3.5 122B, run on NVIDIA B200 GPUs with a Triton backend. The baseline sat at 0.025 milliseconds. A later serving-decode candidate reached 0.018 milliseconds, and the best shape-specific speedup on that fragment was 1.6 times for a Batch=4, Key=128, Value=128 layout. Later C++ attempts hit build failures and exhausted their budget. The useful artifact, Databricks said, is the full path: rejected semantic failures, measured slow-but-correct kernels, and the specialized winners.

Decoded Take

Inference labs have spent two years arguing about compilers and custom silicon. Proteus is a different bet: let an agent write the kernel, then spend the real engineering on the stopwatch. If those 1.8 to 5.2 times gains hold outside a single Qwen block, vLLM-style shared kernels become a default, not a destination, and serving stacks will start shipping shape catalogs the way they now ship quantization tables. The risk is the same one the post admits. A harness that cannot see leftover graphs or hidden assumptions will publish fiction. Watch whether Databricks folds Proteus into Mosaic or Model Serving as a customer-facing path, whether the next write-up names more than one model family, and whether the checkers stay public enough that outsiders can reproduce the 5.2 times number on their own B200s.