On October 15, NVIDIA did something it rarely does: it gave away the good stuff.
Nemotron 3.5 — a family of three open-weight AI models — landed on Hugging Face the same day it was announced. The headline act is Super VL, a 120-billion-parameter multimodal model built for AI agents that read PDFs, look at dashboards, watch short videos, and call tools. The sleeper hit, at least for startups, is Bolt 74B: 74 billion parameters squeezed — through native 2-bit quantization — into 24GB of VRAM. That’s a serious local agent model running on a single high-end RTX card sitting on your desk.
And here’s the part that matters for you: the weights are free to download, the OpenMDW-1.1 license permits commercial use, and NVIDIA published the training recipes and datasets alongside them. This isn’t a demo or a gated beta. It’s infrastructure you can ship.
So the question isn’t “is it good?” The question is “what does a founder actually do with this, and where does the math work?” Let’s get into it. Real talk: I’ve read the model cards, the technical blog, and the benchmarks — every number below is sourced, and every vendor claim is flagged as such. No hype laundering.
What actually launched on October 15
NVIDIA framed the release as coverage “from data center to desktop,” and for once that framing is accurate. Three models, three tiers:
- Super VL (120B-A12B) — the flagship. 120 billion parameters total, 12 billion active per token (it’s a mixture-of-experts model, so only a slice of the network fires on each token). Text, images, documents, and video in; text out. Up to 1 million tokens of context. Available now.
- Bolt 74B-A9B 24GB — the local workhorse. 74 billion parameters, 9 billion active, natively quantized to ~2 bits so the whole thing fits in 24GB of VRAM. 256K-token context. Runs on GeForce RTX PCs (NVIDIA tested on an RTX 5090), DGX Spark, and RTX Spark laptops. Available now, with GGUF builds that drop straight into Ollama, LM Studio, and llama.cpp.
- Bolt E4B 4GB — the edge model. 4GB footprint for conversational agents, voice assistants, and embedded interfaces on GeForce RTX, Jetson, and Spark systems. Landing October 23.
Super VL and Bolt join Lightning (high-throughput, released in August) and Ultra (the heavy reasoning model) in the Nemotron family. So the family now has an answer for every workload shape: throughput, reasoning, multimodal agents, and edge. That completeness is the point — NVIDIA wants one open model family to cover whatever you’re building.
One more detail that matters: Super VL shipped with open datasets and training recipes, not just weights. That’s the difference between “you can use the model” and “you can own the model’s future behavior on your data.” The second one is much more valuable to a startup.
Under the hood: why this model is shaped this way
I’ll keep the architecture section short — you’re a founder, not a researcher — but the design choices explain the economics, so they matter.
Super VL’s language backbone is a hybrid: interleaved Mamba-2 state-space layers, mixture-of-experts blocks, and a handful of full-attention layers, topped with a C-RADIO vision encoder. Why hybrid? Pure attention gets expensive as context grows; Mamba layers handle long sequences far more cheaply. That’s how you get a 1-million-token context window without needing a data center per query. The model was distilled from Nemotron 3 Ultra — NVIDIA’s largest reasoning model — which is how a 120B model claims to match Ultra-class performance at roughly one-fifth the parameter count. (That’s NVIDIA’s framing from its own announcement, so read it as marketing-with-math.)
Then there’s NVFP4 — NVIDIA’s 4-bit floating-point quantization format, trained in with quantization-aware distillation so the compressed model barely loses accuracy. Check the model card’s own numbers: GPQA-Diamond 89.52 (BF16) vs 89.90 (NVFP4), SWE-bench Verified 71.10 vs 71.07, Terminal-Bench 2.1 60.67 vs 62.78. The quantized version is essentially at parity with the full-precision one. That’s genuinely impressive compression engineering, and it’s the whole reason Bolt 74B can exist at 2-bit precision in 24GB.
Practically, Super VL deploys on 1× DGX Spark, 1× H200, 2× H100, or 1× GB200. And multi-token prediction is wired in for native speculative decoding — which is where NVIDIA’s “up to 5x higher throughput for agentic tasks vs other open models in its class” claim comes from (measured on the Artificial Analysis Intelligence Index leaderboard, and yes, that’s a vendor claim — NVIDIA doesn’t name the comparison class).
The founder’s angle: why open weights change the math
Okay, here’s the thing. Every frontier lab will sell you tokens. NVIDIA just handed you the factory.
The license is the story. OpenMDW-1.1 — the Linux Foundation license these models ship under — permits commercial use, fine-tuning, and redistribution. The Hugging Face card says it plainly: “This model is ready for commercial or non-commercial use.” That means you can embed Super VL in a product you sell, fine-tune it on your customers’ data, and never pay NVIDIA a per-token fee. Compare that to the API treadmill, where your gross margin is at the mercy of someone else’s pricing page. We’ve written about the AI price war between the big API labs — prices move, sometimes in your favor, sometimes not. Open weights take you off that ride entirely.
Privacy is a sales feature. If you sell to law firms, clinics, finance teams, or anyone with compliance people, “your documents never leave your infrastructure” is a sentence that closes deals. With Super VL running in your VPC — or Bolt 74B running on a box under your desk — client contracts, medical-adjacent data, and internal docs are never shipped to a third-party API. You can’t buy that guarantee from a hosted API at any price.
Fine-tuning is the moat. Anyone can call the same API you call. Not anyone can fine-tune a 120B multimodal model on five years of their industry’s documents with open recipes. The companies NVIDIA name-drops — Scale AI post-training Super VL for vendor credit processing, Ramp turning receipt images into structured JSON, AI21 on threat intelligence — are all doing exactly this: taking the open base and distilling their own edge into it. That playbook is now available to a ten-person startup, not just Scale AI.
Getting it running: the hands-on path
Theory is cheap. Here’s the actual ladder, from “just curious” to “this is in production”:
Step 1: Try it free on NVIDIA Build
Before you buy a single GPU, prototype against NVIDIA’s hosted endpoint at build.nvidia.com. It’s free for evaluation and development — you get an API key, an OpenAI-compatible endpoint, and you can test whether Super VL actually handles your documents and tools before committing hardware. Think of it as the test drive. Fair warning: it’s for prototyping, not production — NVIDIA’s terms restrict the free tier to internal testing and evaluation.
Step 2: Serve Super VL yourself with vLLM
The model card ships a ready-to-run vLLM configuration — one command serves the NVFP4 build with an OpenAI-compatible API on port 8000. If you have access to a DGX Spark, an H200, or a couple of H100s (cloud rental works fine for the pilot), you can be serving your own endpoint the same afternoon you download the weights. For production, NVIDIA packages it as a NIM — a containerized microservice — and supports it through NVIDIA Dynamo, its distributed inference framework. NVIDIA claims Dynamo setups serve up to 8x more users at 85% lower latency. That’s the vendor’s number, from their own materials — measure it on your workload before you believe it.
Step 3: Run Bolt 74B on your desk
This is the fun one. The GGUF build of Bolt 74B-A9B drops into Ollama or LM Studio like any local model — except it’s a 74B-parameter agent model with tool-calling chops and a 256K context window, running on your 24GB RTX card. We covered Microsoft’s local coding model MAI-Code-1.1-Flash recently, and the pattern is the same: the local-model tier is getting genuinely capable, and the hardware you already own (or can buy once) replaces a monthly API line item.
One honest note: if this stack — vLLM configs, NIM containers, quantization formats — looks like more plumbing than you want to touch, that’s a solved problem too. An AI integration studio like AISquadX can wire the whole thing into your product so your team stays focused on the product itself.
The startup playbook: where it beats paid APIs
Open weights don’t win everywhere. They win in three specific places, and you should know exactly which ones:
1. Always-on agents. The killer use case. A support-triage agent, a document-review loop, a research agent that runs overnight — these burn tokens continuously, which means they burn money continuously on a paid API. On your own hardware, the marginal cost of the ten-thousandth query is basically electricity. If your agent runs 24/7, do the math; the break-even point arrives fast.
2. Privacy-constrained customers. As above — some customers literally cannot use a product that sends their data to a third-party model API. Local or VPC-hosted open models turn “we can’t use AI tools” prospects into customers. That’s not a cost play, that’s a revenue play.
3. Differentiated fine-tunes. The base model is a commodity; everyone downloads the same weights. Your fine-tune on your industry’s documents, your support tickets, your workflows — that’s proprietary. Open weights plus open recipes mean the moat is buildable by a small team, not just a lab.
Where do paid APIs still win? Bursty, unpredictable workloads where you’d be paying for idle GPUs; the absolute frontier of reasoning where closed models still lead; and anywhere you need zero operational overhead. Be honest about which camp you’re in — we’ve mapped out how founders cut their AI API bills without switching stacks at all, and sometimes that’s the right first move.
“The most interesting thing about Nemotron 3.5 isn’t any single benchmark. It’s that a startup can now download a 120B multimodal agent model, fine-tune it on private data, and serve it — legally, commercially — without asking anyone’s permission. The permissionless part is the product.”
The cost math, honestly
Let’s talk money, because “free model” hides real costs:
- Weights: $0. Download free, run free, commercial license included. This part is genuinely free.
- NVIDIA Build (hosted): free to try, not for production. Great for prototyping and eval. Production self-hosting under NVIDIA AI Enterprise is licensed at roughly $4,500 per GPU per year (about $1/GPU/hour in cloud terms) — that’s the number third-party reporting attributes to NVIDIA’s terms, and it’s the real price of the “free” model at production scale.
- Third-party hosts: bring your own per-token bill. Super VL is already on OpenRouter and ModelScope, and hosts like DeepInfra price the same open weights per token at their own rates. Convenient, but you’re back on the API treadmill — just with more vendors to choose from.
- Your own hardware: the break-even play. A cloud H200 rents for a few dollars an hour; a DGX Spark or a 24GB RTX card is a one-time purchase (RTX Spark laptops went on sale October 16 starting at $2,599). If your agents burn more than a few hundred dollars a month in API fees, owned hardware pays for itself in months, not years.
The honest summary: the model is free, the compute isn’t, and the license is what makes the whole thing worth doing. You’re trading a variable per-token cost you can’t control for a fixed infrastructure cost you can.
Honest limitations
Every one of these is real, and I’d rather you hear them from me than discover them at 2 AM:
- It’s an NVIDIA-hardware story. The serving stack — NVFP4, NIM, Dynamo, the vLLM recipes — is tuned for RTX, DGX, and Blackwell GPUs. If your infra is AMD or Apple Silicon, temper your expectations.
- The benchmarks are NVIDIA’s. GPQA-Diamond 89.52, SWE-bench Verified 71.10, Terminal-Bench 2.1 in the 60s — all from NVIDIA’s own evaluation harness. Independent third-party validation hasn’t landed yet. Treat them as directional, not gospel.
- The “5x throughput” claim has an asterisk the size of a house. It’s measured against “other open models in its class” — a class NVIDIA doesn’t name — on NVIDIA’s own leaderboard submission. Your throughput on your workload is the only number that matters.
- “Local” still means serious hardware for Super VL. 1× H200 or a DGX Spark minimum. Bolt 74B is the actually-desk-friendly one; don’t confuse the two.
- 2-bit quantization is aggressive. Bolt 74B’s native 2-bit compression is bold engineering, but quality trade-offs on long, multi-step agentic runs are unproven. Test it on your actual tasks — especially tool-calling chains — before you bet the product on it.
- OpenMDW is open, not open-source. It’s a permissive commercial license from the Linux Foundation, not an OSI-approved open-source license. For 99% of startups that’s a distinction without a difference, but your lawyer should read it before you ship.
Frequently asked questions
Can I really use Nemotron 3.5 commercially for free?
Yes — the weights are free to download and the OpenMDW-1.1 license permits commercial use, fine-tuning, and redistribution, which the Hugging Face model card confirms. The catch is compute: you pay for the GPUs (yours, rented, or via a per-token host), and production NIM deployments need an NVIDIA AI Enterprise license. The model is free; running it isn’t.
Which Nemotron 3.5 model should a startup start with?
Prototype with Super VL on NVIDIA Build’s free hosted endpoint to test whether it handles your documents and tools. If the workload is always-on or privacy-sensitive, evaluate self-hosting Super VL — or drop to Bolt 74B on a 24GB RTX card if your tasks fit a 256K context and local hardware. Bolt E4B (October 23) is for edge and conversational use, not heavy agent work.
How does Super VL compare to GPT, Claude, or Gemini for agents?
On NVIDIA’s own benchmarks it posts strong agentic numbers (Terminal-Bench 2.1 ~61-63, BrowseComp 78, SWE-bench Verified ~71), but independent comparisons against closed frontier models haven’t been published yet, and one analyst noted it trails the leading closed models in overall rankings. The honest answer: it competes on efficiency and openness, not necessarily on raw capability. Its edge is that you can run and fine-tune it yourself.
What hardware do I need to run these models?
Super VL (NVFP4 build): 1× DGX Spark, 1× H200, 2× H100, or 1× GB200. Bolt 74B-A9B: any 24GB VRAM setup — NVIDIA tested on an RTX 5090, and the GGUF build runs in Ollama or LM Studio. Bolt E4B: 4GB footprint, targeting GeForce RTX, Jetson, and RTX/DGX Spark systems. RTX Spark laptops (from $2,599) went on sale October 16.
Is Nemotron 3.5 actually open source?
Not in the OSI sense. It’s open-weight under the OpenMDW-1.1 license: you can download, use commercially, fine-tune, and redistribute the models, and NVIDIA published datasets and training recipes too. That’s more open than most “open” releases, but the license has terms an OSI license wouldn’t — have counsel glance at it before you build a business on it.
The Bottom Line
NVIDIA just put a 120B multimodal agent model and a desk-friendly 74B local model into every founder’s hands, free, with a commercial license, on the same day they were announced. That’s not a normal release cadence — that’s a land grab for the open-model ecosystem, and startups are the beneficiaries.
Here’s my take: don’t treat this as “a new model to try.” Treat it as infrastructure to evaluate. Prototype on NVIDIA Build this week. Run the cost math on your heaviest always-on agent workload. And if the numbers work, you’ve just found the rarest thing in the AI business — a way to cut your two biggest costs, API fees and privacy risk, with the same move.
The weights are free. The moat is what you fine-tune into them.
Sources: NVIDIA’s Nemotron 3.5 announcement, the technical deep-dive on Super VL, the Super VL model card and Bolt 74B model card on Hugging Face, plus independent coverage from Maccus and HPCwire. Benchmark and throughput figures are NVIDIA’s own unless noted.
