NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB
A downloadable vision-language model for local agents, combining quantized weights, configurable reasoning, and tool calling.
NVIDIA Nemotron Labs Dory1.0-4B
huggingface.co
A downloadable English reasoning model with adjustable inference loops, long context, and local tool-calling APIs.
Visit website
Pricing
Free
Available on
Linux, macOS on Apple Silicon
Category
Llm Models
What is NVIDIA Nemotron Labs Dory1.0-4B?
NVIDIA Nemotron Labs Dory1.0-4B is a downloadable, four-billion-parameter model for English text reasoning and generation. It serves developers and researchers building text applications, evaluating coding assistance, and integrating applications that request external tools.
Its distinguishing control is a looped architecture that repeatedly applies shared layers. Developers can change the computation spent on each token using the same checkpoint, making it useful for comparing reasoning settings before choosing a deployment configuration.
Key features
Adjustable reasoning computation
Changing the recurrent loop count adjusts inference computation without retraining or replacing the model. The architecture reuses a shared block of layers.
Long text context
The model supports up to 256K tokens of context. Deployment settings may use a smaller window to accommodate available memory and concurrency.
Streaming and tool calls
The deployment examples expose an OpenAI-compatible API for chat, streaming responses, and tool-call requests, with runtime-specific parser or chat-template settings.
Multiple checkpoint formats
NVIDIA provides BF16, MXFP8, NVFP4, and Q5_K_M GGUF versions for compatible inference environments.
Configurable cache memory
The llama.cpp integration offers separate caches per loop or a shared last-loop cache. Last-loop caching reduces cache memory but changes predictions and is sensitive to prefill chunk size.
Who is it for?
Researcher comparing reasoning budgets
Compare answer quality and response time across loop settings on a consistent evaluation set.
Developer evaluating coding assistance
Assess the model on code explanation and repository questions before integrating it into a development workflow.
Application team prototyping tool use
Connect a local text application to selected functions and evaluate its tool-call requests.
How to use NVIDIA Nemotron Labs Dory1.0-4B
- For Apple Silicon, follow the GGUF guide and install its listed build prerequisites.
- Build the pinned Dory-enabled llama.cpp revision and download the Q5_K_M checkpoint.
- Start the local server with two loops and the guide's initial context settings.
- Open the local chat interface or connect an API client, then evaluate loop and cache settings on your tasks.
NVIDIA Nemotron Labs Dory1.0-4B pricing
Free
Model-material permissions are free under OpenMDW 1.1. Hardware and hosting are separate costs; redistribution requires retaining the agreement and applicable notices.
Check current pricing ↗What to know before you use it
- Deployment requires specialized builds: the vLLM integration remains an open pull request, and the llama.cpp integration is a draft. Follow the Dory-specific installation instructions.
- The llama.cpp draft reports incomplete long-context qualification, untested Metal support, and unsupported context shifting. NVIDIA's published Apple Silicon recipe therefore still requires workload-specific validation.
- Documented human-language support is English. The BF16 GPU integration lists Linux with NVIDIA Hopper or Blackwell hardware.
Frequently asked questions
Which loop setting should I try first?
NVIDIA recommends two loops initially. Compare end-to-end latency on your workload before increasing the count, because additional computation can also change output length.
Can I run it on a Mac?
NVIDIA provides a native ARM64 Apple Silicon recipe using GGUF and a pinned Dory-enabled llama.cpp build. That recipe excludes Intel Macs and Rosetta.
Can it be used commercially?
Yes. NVIDIA permits commercial and noncommercial use under OpenMDW 1.1, subject to the license terms.