November 19, 2024.
Peter Clarke of EE News Europe writes:
d-Matrix Inc. (Santa Clara, Calif.), a Microsoft-backed startup, has launched Corsair, its first AI processor, designed to speed through inferencing tasks.
Corsair offers performance of 60,000 tokens/second at 1 ms/token for Llama3 8B in a single server and 30,000 tokens/second at 2 ms/token for Llama3 70B in a single rack, d-Matrix said. As a result Corsair provides performance, energy efficiency, and cost savings as compared to GPUs and other alternatives, the company asserted.
“We saw transformers and generative AI coming, and founded d-Matrix to address inference challenges around the largest computing opportunity of our time,” said Sid Sheth, cofounder and CEO of d-Matrix. “The first-of-its-kind Corsair compute platform brings blazing fast token generation for high interactivity applications with multiple users, making Gen AI commercially viable.
Read the full article on EE News Europe