88Z AI, 22 November — An AMD engineer used Google's Gemini 4 Neon as a coding agent with the Loom toolkit to optimize kernels for Thinking Machines' open-weight Inkling 2 Mini on AMD Strix Halo hardware. In approximately 4.3 hours, the agent produced a package that passed correctness checks. Benchmarks showed decode speeds of 44.8 tok/s, a 1.88x improvement over llama.cpp, with prefill speeds up to 5.9x faster. This demonstrates how AI agents can rapidly narrow AMD's software gap for new model architectures.
At a glance
- Gemini 4 Neon agent completed AMD kernel tuning for Inkling 2 Mini in 4.3 hours
- Decode speed increased by 1.88x, with prefill speeds up to 5.9x faster
- Demonstrates AI agents can rapidly narrow AMD's software gap for new architectures
AI Agent Accelerates Kernel Optimization
An AMD engineer utilized Google's Gemini 4 Neon as a coding agent alongside the AMD Loom toolkit to tune kernels for Thinking Machines' new open-weight model, Inkling 2 Mini. The testing was conducted on a Strix Halo machine featuring the Radeon 8060S GPU. The agent processed the unmodified GGUF file, making approximately 490 tool calls over 4.3 hours to produce a package that passed correctness checks against llama.cpp.
Significant Performance Gains and Industry Implications
Compared to the best llama.cpp build, the optimized version achieved a decode speed of 44.8 tok/s versus 23.8 tok/s (a 1.88x increase). Prefill speeds reached up to 5.9x faster at an 18k-token prompt (1,465 vs 249 tok/s), with a 1.86x aggregate throughput gain across four streams. The decode performance hit approximately 80% of the memory-bandwidth roofline. While llama.cpp's day-one support is still maturing, this result highlights the ability of AI agents to close part of AMD's software gap for new model architectures within hours.
Sources & editorial note
This article is an AI-assisted synthesis by 88Z AI based on the public sources below. Product claims and availability are attributed to their publishers. It does not represent independent testing or first-hand reporting.
- @G2AIStackSource published
Source dates above refer to the original announcements; the date under the headline is this article’s publication on 88Z AI.