Nvidia Releases Nemotron 3.5 Lightning Open AI Model

Nvidia has introduced Nemotron 3.5 Lightning, a new open AI model designed less as an all-purpose answer engine than as a fast workhorse inside long-running AI agent systems.

Released Aug. 11, Nemotron 3.5 Lightning is a 30 billion-parameter mixture-of-experts model that activates 3 billion parameters per token. Nvidia says it supports context windows of up to 1 million tokens and can deliver up to four times the output speed of similar-sized models. The company is offering the model with open weights, training data, and recipes under its OpenMDW-1.1 license.

The more consequential part of the Nemotron 3.5 Lightning release is the job Nvidia expects models like it to perform. Rather than sending every step of an AI agent’s work to a large frontier model, Nvidia envisions systems in which different models handle different classes of tasks.

A larger reasoning model might plan a workflow or handle a difficult decision, while a faster specialized model performs the repeated tool calls, validations, formatting tasks, and other routine operations generated as an agent carries out that plan. That distinction becomes more significant as AI applications move beyond single prompts and responses toward agents that can generate dozens or potentially many more model calls while completing a task.

From One Model to a System of Models

Nvidia describes this approach as a system of models. Its enterprise announcement says modern agents increasingly operate as model ensembles, with different models specializing in different work.

That changes the equation around choosing an AI model. Raw capability remains one factor, but latency, inference cost, deployment location and task specialization become part of the same decision. A model that is not the strongest choice for difficult reasoning may still make sense for a large volume of narrower tasks if it can complete them more quickly and with sufficient accuracy.

Nvidia is reinforcing that idea by releasing NeMo Switchyard alongside Lightning. The open source routing library is designed to direct individual requests to different models based on requirements such as quality, latency and cost. Nvidia says organizations can route work across their own mix of open, proprietary and Nvidia models without rewriting the applications making the requests.

The combination points toward an AI architecture in which model selection becomes a runtime decision rather than a one-time platform choice. Teams building agents may increasingly evaluate the performance of the entire workflow — including which model handles which task — rather than choosing one model to serve every request.

Speed Over Maximum Intelligence

Lightning’s design reflects that tradeoff. Its mixture-of-experts architecture gives the model more total capacity than the number of parameters actually used for each token, while Nvidia has also added multi-token prediction, speculative decoding, and an NVFP4 quantized checkpoint to increase inference efficiency.