DeepSeek-R1, a 671-billion-parameter model known for its exceptional reasoning capabilities, is now available as an NVIDIA NIM microservice. This collaboration promises to deliver high-speed, efficient AI solutions for complex tasks.

DeepSeek-R1 stands out with its massive 671 billion parameters, significantly surpassing many existing open-source large language models. Its architecture features a mixture-of-experts design, with each layer comprising 256 experts. For every input token, the model engages eight experts in parallel, enabling advanced logical inference, reasoning, mathematics, coding, and language understanding. This design allows DeepSeek-R1 to handle an extensive input context of up to 128,000 tokens, making it adept at processing and generating lengthy and complex content.

Integrating DeepSeek-R1 into NVIDIA's NIM microservice framework simplifies the deployment process for developers and enterprises. The NIM microservice supports industry-standard APIs, ensuring seamless integration into existing systems. By running the microservice on preferred accelerated computing infrastructure, organizations can maintain high standards of security and data privacy. Additionally, with NVIDIA AI Foundry and NeMo software, there's potential for creating customized DeepSeek-R1 microservices tailored to specialized AI agents.

When deployed on a single NVIDIA HGX H200 system equipped with eight H200 GPUs connected via NVLink and NVLink Switch, the DeepSeek-R1 NIM microservice achieves an impressive throughput of up to 3,872 tokens per second. This high performance is attributed to the NVIDIA Hopper architecture's FP8 Transformer Engine and the substantial NVLink bandwidth facilitating efficient communication among the model's experts. Looking ahead, the upcoming NVIDIA Blackwell architecture is expected to further enhance performance with its fifth-generation Tensor Cores, delivering up to 20 petaflops of peak FP4 compute performance and a 72-GPU NVLink domain optimized for inference.

Developers eager to explore the capabilities of the DeepSeek-R1 NIM microservice can access a preview on NVIDIA's platform. The API is anticipated to be available soon as a downloadable NIM microservice, forming part of the NVIDIA AI Enterprise software suite. This development opens avenues for building specialized AI agents that leverage DeepSeek-R1's advanced reasoning capabilities, all within a secure and efficient deployment framework.

The collaboration between DeepSeek-R1 and NVIDIA's NIM microservice framework marks a significant advancement in AI deployment. By combining a state-of-the-art reasoning model with a robust deployment infrastructure, this partnership offers developers and enterprises a powerful tool for implementing sophisticated AI solutions across various applications. Wanna find out more?! head to the official announcement.