Andy Garcia · freelance dev
FRENES
Back to the blog
Open Source

NVIDIA Put '24GB' in the Name of Its New Local Agent Model. The Smallest Recipe on Its Model Card Is Called '28GB', for Graphics Cards of 32 GB and Up.

October 19, 2026 8 min read

On Thursday, October 15, NVIDIA released Nemotron 3.5 Super VL, an open vision-language model for agents with 120 billion parameters, 12 billion of them active per token, and a context window of up to 1 million tokens. Alongside it came Nemotron 3.5 Bolt 74B-A9B 24GB, a 74-billion-parameter model compressed from Super VL to run agents locally. NVIDIA said a third model, Bolt E4B 4GB, would follow on October 23. Super VL is on Hugging Face, OpenRouter, ModelScope and as an NVIDIA NIM microservice. Bolt 74B-A9B is on Hugging Face.

TechCrunch's write-up on October 19 framed the launch as NVIDIA pushing companies to run their own open models in their own data centers, and pitching local models to consumers who are wary of handing an assistant their lives. Kari Briski, who signed NVIDIA's launch post, put the enterprise case in one line: "organizations can post-train Super VL on their own data, deploy it on their own infrastructure and oversee what they build."

That sentence is aimed at people like me. I am CTO of a LegalTech built on Claude, and our users upload documents they would not show a competitor. An open model I could run where the data lives is the obvious second path. So I skipped the adjectives and read the model cards and the license.

The license is the generous part

Both models ship under the OpenMDW License Agreement, version 1.1, published on openmdw.ai, a Linux Foundation site. It is short. It grants permission "to deal in the Model Materials without restriction," including under copyright, patent, database and trade secret rights. If you redistribute, you keep a copy of the agreement and the notices. If you sue anyone claiming the materials infringe a patent or copyright, your rights end. And the agreement "does not impose any restrictions or obligations" on the outputs.

That is unusually clean for this month. On the same October 15, Meta's license for Muse Spark Open required a "Built with Muse Spark Open" credit and sent model-as-a-service businesses above $20 million in revenue over twelve months to Meta for a separate agreement. Mistral's new Timestral weights came with a "Modified MIT" license that withdraws every right once your company's, or your employer's, monthly revenue exceeds $20 million. NVIDIA drew no such line.

The trade-off sits in one sentence, in capitals in the original: you are "solely responsible for (1) clearing rights of other persons." Read next to the model card, that line has content. Super VL's card lists synthetic training data generated with Qwen, DeepSeek, Kimi, GLM and gpt-oss models, filtering with GPT-5.6 Sol and Gemini 3 Flash Preview, and a section titled "Private Non-publicly Accessible Datasets of Third Parties." The launch post says the model "ships with open weights, open datasets and training recipes." Part of the data is open. Not all of it.

For most products that will never matter. For a vendor questionnaire from a law firm, it is a paragraph to keep in the file.

Comparable with Ultra, on some rows

The launch post says Super VL is "comparable with NVIDIA's largest reasoning model at roughly one-fifth the parameter count," and the fastest open agentic model in its class, "5x faster than the next best." NVIDIA's own benchmark table, on the Hugging Face card, is more mixed.

Against Nemotron 3 Ultra, Super VL leads on Terminal-Bench 2.1 (60.67 to 54.80), BrowseComp (78 to 67.85) and SWE-bench Pro (51.78 to 49.66). It trails on SWE-bench Verified (71.10 to 73.9), IFBench (70.86 to 82.9) and tau2-bench telecom (73.19 to 83.0).

One caution on the first of those rows. At launch, the card listed Super VL at 66.70 on Terminal-Bench 2.1. Card updates later on October 15, one of them titled "Update numbers based on Tech Report," changed it to 60.67. Super VL still leads Ultra on that row, but a model card can change after launch, so a number quoted from one needs a date.

Against all six other models in the table, it has the lowest SWE-bench Verified score and the lowest tau2-bench telecom score. On Terminal-Bench 2.1, Qwen3.8-Flash-Next scores 85.80 and GLM-5.3-Flash 82.70, against Super VL's 60.67.

The row that interests me is elsewhere. Artificial Analysis's AA-Omniscience benchmark includes a hallucination rate: how often a model answers wrong when it should have admitted it did not know. Lower is better. In NVIDIA's table, Super VL scores 30.0. Nemotron 3 Ultra scores 22.1, and the other five models range from 30.3 to 95.9. On documents, Super VL sits level with Qwen3.5-122B-A10B: 62.82 to 63.7 on MMLongBench-Doc, 68.12 to 68.3 on OCRBench v2 in English. The card notes that both document benchmarks were run with their official implementation or an internal scaffolding NVIDIA plans to open source.

The launch post opens on the "highest accuracy for specialized always-on agents." The card's table does not show the most accurate open model. It shows speed, a model that guesses less than most of the models it was compared with, and a base built to be post-trained. The launch post's own last chart says as much. In five customer tasks, post-trained versions of Super VL beat Claude Opus 5, GPT-6.1 Sol, GPT-6 Luna or GPT-6 Astra on accuracy, and in all five the base model starts below the closed model it is compared with. The lead comes from post-training on each customer's task. TechCrunch summed up that comparison as Super VL completing tasks faster than Anthropic and OpenAI models, and noted that such claims are standard at launch; the chart is titled "Post-Training Delivers Leading Accuracy for Specialized Agents" and plots accuracy.

NVIDIA's technical blog adds customer numbers, mostly in the future tense. Scale AI's work for Skydance "will decrease" inference cost by 40x and latency by 5x. Ramp "expects" up to 50 percent higher throughput and 10x lower latency. NVIDIA says AI21 and Alice's version improved accuracy on threat intelligence and vulnerability analysis by 41 percent; on the launch chart, those 41 are percentage points, from about 35 to about 76 percent.

The "24GB" model is a single-user model for a big machine

Bolt 74B-A9B keeps 9 billion of its 74 billion parameters active per token. NVIDIA compressed it from Super VL and quantized its experts to 2 bits, 2.0625 bits per parameter, to "fit in 24GB of memory and run alongside other applications." The card says the compression search targeted a footprint below 24 GB at batch size 1 and 128K context.

The llama.cpp card is more specific. It ships two recipes. The "28GB recipe" runs a 65,536-token context and is listed for an RTX Spark machine with 64 GB or a discrete RTX GPU with 32 GB or more. The "32GB recipe" runs the full 262,144-token context and is listed for RTX Spark with 128 GB or a discrete GPU with 48 GB or more. Both serve one concurrent sequence. A 24 GB graphics card is not on the list, and the vLLM quick start also serves one sequence at a time.

None of that is a trick. "24GB" names the model, not your GPU. But the hardware list decides the purchase, and NVIDIA's own speed figures for Bolt, on its research page, were measured with llama.cpp on an RTX Spark laptop with 128 GB. Bolt is a personal agent on a large-memory machine, built for the RTX Spark laptops NVIDIA said would arrive on October 16. It is not a shared server for a team of ten.

Its table carries the same nuance as Super VL's. Against Qwen3.6-35B-A3B in full BF16 precision, Bolt leads on Terminal-Bench 2.1 (50.98 to 44.38) and BrowseComp (66.75 to 48.74). It trails on PinchBench (87.82 to 88.07) and on AA-Omniscience accuracy (16.90 to 19.47). The launch post calls Bolt "the #1 open model in its class on PinchBench." NVIDIA's research page defines the class: models "with a similar memory footprint." The BF16 Qwen in the table is not one of them: 35 billion parameters at two bytes each is about 70 GB of weights.

One footnote I liked. Qwen reported 62.8 on MCP-Atlas for its model; NVIDIA measured 51.2 on the 500 public tasks; Bolt scores 52.20. Bolt's lead on that row depends on who ran the test.

The card lists English and code as the primary languages, with Spanish, French, German, Italian and Japanese "also being supported." I built BidForge, an AI that reads French public tender files for small businesses. "Also supported" is something I would test on real files, not something I would assume.

What I would check before moving a workload

Five things, none of them expensive.

The license file in the exact repository you pull, kept with its notices if you ship the weights. Then hand the "clearing rights" sentence and the card's third-party data section to whoever answers your clients' vendor questionnaires.

A hallucination test on your own documents. Write questions whose answer is not in the file and count how often the model says so. AA-Omniscience is the public version of that test. Yours is the one your clients will care about.

The hardware per concurrent user. The BF16 card lists four H100s or two GB200s for Super VL. The NVFP4 version runs on one H200, two H100s, one GB200 or one DGX Spark, and the DGX Spark vLLM configuration on the card serves one sequence. Before comparing a GPU invoice with an API bill, count simultaneous users, not parameters.

The language. Run your real files, in your language, before trusting an English benchmark.

And the exit. Keep the model behind one thin wrapper, as configuration. I would not swap a production model on a launch post. I would route one well-measured task to the open model, compare cost and errors per completed task, and decide on that.

NVIDIA gave away the cleanest license of the week. The model cards are where it tells you what the models need and where they trail. Read those before the launch post.

Sources

A project like this one?

I design and deploy products like this. Let's talk.

Let's talk