Run DeepSeek locally
Not every job needs the cloud flagship. V4 Flash (37B) is the usual local / edge pick: smaller, faster, cheaper to keep on your own GPU.
Commands drift — follow the current Ollama / vLLM / SGLang docs and the Hugging Face card.
Why Flash for local
V4 Pro is a 671B MoE flagship (~1M context, full vision, ~45 tok/s); most laptops and single workstation GPUs should not start there. Flash’s 37B class (~128K, basic vision, ~112 tok/s on the hosted snapshot), quantized, is what the edge-deploy write-ups on this site actually run.
Local deploy is not a new DeepSeek product. It is “you brought weights + a runtime.” Download checkpoints from Hugging Face deepseek-ai, not from this site. Ollama is a third-party runtime. DSpark is on by default on https://api.deepseek.com/v1; when you self-host you may need an explicit enable in that stack.
When to self-host
Stay local when:
- Data cannot leave the building.
- You need milliseconds on a LAN, not a round trip to the public API.
- You already have a GPU and want to stop paying per token for high QPS.
- You are trying Flash quantized on a laptop before you rent a cluster for Pro.
What local deploy is not
Do not confuse your GPU box with the official API:
- Not a DeepSeek SKU. There is no deepseek-local model ID — you serve Flash or Pro yourself.
- Not official DeepSeek software. Ollama is third-party; vLLM and SGLang are community stacks.
- Not a weight host. Checkpoints come from Hugging Face, not this site.
- Not the official API SLA. Hosted DSpark is on by default; your box may be slower until you enable it.
At a glance
- Usual local pick
- V4 Flash (37B, quantized)
- Pro locally
- 671B MoE — serious hardware only
- Runtimes
- Ollama · vLLM · SGLang
- Weights
- Hugging Face deepseek-ai
- DSpark
- API default on; self-host may need enable
- Ollama
- Third-party runtime
Three common runtimes
Pick the tool that matches the job:
- Ollama — fastest laptop trial: ollama pull / ollama run a V4 Flash tag from the card.
- vLLM — production throughput: vllm serve with the Hugging Face repo id.
- SGLang — another high-throughput serving stack used in the same edge notes.
- After serve: if the box is slower than platform.deepseek.com, check that runtime’s DSpark / speculative-decoding flag.
ollama pull deepseek-v4-flash
ollama run deepseek-v4-flash "Explain speculative decoding in one sentence"
# production-style (check the current vLLM CLI):
vllm serve deepseek-ai/deepseek-v4-flash --max-model-len 128000Local vs API vs tools
Same Flash model, different place it runs:
| Layer | Role | When to open this page |
|---|---|---|
| Local Flash | 37B on your GPU; usual laptop / single-card start | Data stays on the box; you already have a GPU |
| Local Pro | 671B MoE; only if the model card’s hardware fits | You have a serious cluster, not a laptop |
| DeepSeek API | No GPU; DSpark on by default; OpenAI-compatible | You are shipping and data can leave the building |
| DSpark | Accelerator; self-host may need an enable flag | Local tok/s looks worse than the official API |
| Harness / Deep Code / Claude Code | Point at localhost if you expose an OpenAI-style server | The agent should talk to your box, not the cloud |
Boundaries and traps
- Starting with Pro on a laptop. Prefer Flash (quantized) on a single GPU; Pro needs serious hardware.
- Assuming Ollama is official DeepSeek software. It is third-party. Commands and tags follow its docs and the Hugging Face card.
- Expecting hosted DSpark speed from a bare vllm serve. Self-host setups often need an explicit enable.
- Downloading weights from this site. They are not here — use Hugging Face deepseek-ai.
- Copying the sample CLI into production without checking the current vLLM / SGLang flags. They drift.
Related guides
Open weights
Download V4 checkpoints on Hugging Face
DeepSeek V4 Flash
Faster and cheaper for everyday use
DSpark
Speculative decoding that cuts latency
V4 Flash Edge Deployment: Millisecond Responses on Limited Compute
Longer Ollama / vLLM practice notes.
DeepSeek API
OpenAI-compatible API for your apps
FAQ
Can I run V4 Pro locally?
Only if you have the hardware the model card assumes. Pro is a 671B MoE flagship. Most “local DeepSeek” guides on this site mean Flash (37B, often quantized) on a laptop or single GPU. Pull weights from Hugging Face, not from this page.
Is Ollama official DeepSeek software?
No. Ollama is a third-party runtime you can use to serve DeepSeek weights. Tags, pull commands and GPU flags follow Ollama’s docs and the Hugging Face model card — both can change.
Does local include DSpark?
On the official API (https://api.deepseek.com/v1) DSpark is on by default. Self-host setups follow that runtime’s docs; community notes mention a dspark enable step. A bare generate loop is not the same stack as the platform.
Where do the weights come from?
Hugging Face, under the deepseek-ai org — not this site. After you download Flash or Pro checkpoints, serve them with Ollama, vLLM or SGLang. License and file layout live on the model card.
Flash or Pro on a single GPU?
Prefer Flash. 37B quantized is the usual laptop / workstation start. Pro’s 671B MoE needs a serious box; do not treat a failed Pro load as “local DeepSeek is broken.” Try Flash first, then scale hardware if you truly need ~1M context or full vision.
Can Harness or Claude Code talk to localhost?
Yes, if you expose an OpenAI-compatible server from vLLM / SGLang / Ollama. Point the tool at that base URL and set deepseek-v4-flash (or Pro if the box can hold it). Keys for the public API still come from the official platform.
Why is my local vLLM slower than the official API?
The platform stack includes DSpark. A bare vllm serve of the same weights often does not. Enable that runtime’s speculative-decoding or DSpark option, then compare first-token latency on the same prompt. Hardware and batch size still dominate.
Will these commands stay valid?
Not as a contract. ollama pull tags, vllm serve flags and max-model-len names drift. Treat the snippet on this page as a starting shape. The current Ollama / vLLM / SGLang docs and the Hugging Face card win when they disagree.