“Meta simply made brokers a capital expense as an alternative of an working one,” Kenney mentioned. “For 2 years, enterprises have been educated to lease intelligence by the token from another person’s knowledge heart. Muse Glimmer runs the agent on a GPU you personal, on the desk, with the meter switched off. That may be a direct shot on the enterprise mannequin that cloud AI distributors are constructed on, and it comes from the one participant with no cloud API income to guard.”
In its put up saying the brand new mannequin, Meta identified that it has aggressively slimmed it right down to attempt to make it environment friendly and cost-effective.
“At full precision, a 30-billion parameter mannequin would require over 55 GB of reminiscence — way over any client GPU gives,” Meta mentioned. “We use quantization strategies to compress the mannequin’s weights to roughly 4-bit precision, shrinking the language mannequin to beneath 20 GB. This leaves sufficient headroom for the mannequin’s working reminiscence, its KV cache, the notion encoder for picture understanding, and the speculative decoding drafter to run concurrently inside a 24 GB or 32 GB envelope. We validated that this compression introduces minimal to no degradation on agentic duties.”
