The July launch of Android Bench — its LLM benchmark leaderboard for Android growth duties — is now based mostly upon the Harbor framework for evaluating fashions, Google has introduced.
In its weblog publish saying the updates, the corporate famous that when Android Bench was launched in March, the purpose was to supply insights into mannequin capabilities, provide AI choices and to induce enhancing the fashions. On this launch, eight fashions have been added to the leaderboard — Claude Fable 5, Sonnet 5 and Opus 4.8; GLM 5.2; Kimi K2.7; MiniMax M3; and Qwen 3.7 Plus and three.7 Max.
“From the start, we’ve valued an open and clear method, which is why we made our authentic methodology and check harness publicly accessible on GitHub,” the corporate wrote within the weblog. “Now we’re taking collaboration a step additional by providing you with, the Android developer group, an opportunity to form Android Bench.”
Builders can now design and submit Android growth duties and see how the fashions deal with the eventualities essential to them, and now can really run benchmark evaluations to check the fashions they use towards Android’s dataset of duties. These duties could be discovered on this GitHub repository and builders can submit evaluations or discover Android’s information at Harbor Hub.
“With an increasing number of choices for agentic growth,” the weblog concluded, “sustaining a cutting-edge benchmark ensures that the AI help you depend on retains getting smarter, extra useful, and more practical.”
