French startup ZML launches free product to accelerate inference on many AI chips | TechCrunch

The days of Nvidia’s unprecedented market dominance are not over, but challengers and choices are popping up on all sides.

ZMLa French AI start-up approved by Yann LeCun, Turing Prize winner, published inference-efficient software which allows a variety of large open source language models run on a variety of chips – including Nvidia, AMD, Google TPU, Apple Metal and Intel Arc.

With ZML/LLMDthe recently launched LLM inference server, the company’s ambition is to break down existing silos and make different chips available for AI use cases at their maximum available speed, and sometimes faster, ZML founder Steeve Morin told TechCrunch.

As AI becomes more integrated into our work and daily lives, inference optimization – that is, prompt processing – has outpaced the pace model training in importance, but it often looks uneven behind the scenes, with software and architectural barriers leading to vendor lock-in, Morin said.

The promise of achieving peak performance across a variety of chips is a technological feat, but it could also disrupt the market, amid growing fears about AI costs.

ZML hopes to offer enterprises and clouds the ability to use a combination of chips, some of which could be less expensive or use less power. “The idea is to give people back the power to create their own system and make real efficiencies that allow [AI] to broadcast,” Morin said.

Such software assistance could help new AI chipmakers, many of which come from Europe, Morin observed, citing to Axel, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloudAnd VSORA. But more than their region of origin, what matters to him is that ZML can work with them on “things that have never been done anywhere else in the world”.

This isn’t to say Morin is bearish on Nvidia. He is notin part because of its existing offering. He told TechCrunch that ZML has a good relationship with the AI ​​chip giant, which has been prepare for the rise of inference.

Inference has been such an intense area of ​​investment that this trend has been hailed as “gold rush inference.” ZML therefore has competitors such as The basset houndrecently valued at $13 billion; Inferactfrom the creators of the open source project vLLM; as well as Radix ArchTHE commercial company behind SGLang.

vLLM and SGLang partially compete with LLMD, but Morin’s ambitions for ZML cover a broader spectrum. “We’ve reached the point where we’re co-designing silicon,” he said. He further credited ZML’s lean team of 20 as the reason the Paris-based startup was able to scale quickly, with more releases in the plans.

It also helped that this small team was well funded for its size. With his experience as vice president of engineering for Zenlywhich Snapchat acquired nine figures in 2017Morin raised $20 million from venture capital firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe and Puzzle Ventures.

Unlike ZML’s first public project, the inference-focused project ML framework released in 2024 and updated in MarchZML/LLMD is not open source. But it is launched as a free product with the aim of learning more about its use. “I prefer to measure and [then generate revenue] where it’s most effective without stupidly hindering my growth because I was too greedy from the start,” Morin said.

It is too early to say when ZML/LLMD might become a paid product and what adoption will look like. But the startup’s cap table confirms that other founders are paying attention, including Solomon Hykes, founder of Dagger and Docker, Clément Delangue and Julien Chaumond of Hugging Face, as well as LeCun, now with AMI Laboratories. This also demonstrates that European AI startups we can now build from home. “I couldn’t do ZML anywhere other than Paris,” Morin said.

View biography