GPU goliath claims tech can boost throughput by 2x for Hopper, up to 30x for Blackwell GTC Nvidia's Blackwell Ultra and ...
The open-source inferencing software increases throughput and reduces the cost of LLM token generation, the chipmaker said.