Self-hosting DeepSeek V4 —
on hardware you can actually own.
V4 Pro needs a datacenter. V4 Flash — the 284B, 13B-active sibling — is the one worth self-hosting: MIT-licensed, within 5% of Pro's benchmarks, and runnable on a Mac Studio or a used dual-3090 rig for well under $2,000.
DeepSeek V4 Flash (284B total, 13B active, MIT) is the only V4 variant worth self-hosting. At Q4_K_M it needs ~142–174GB of combined VRAM+RAM. A Mac Studio with 192GB unified memory runs it at 5–23 tok/s and is the practical sweet spot. A dual RTX 3090 rig with 128GB RAM manages ~2–5 tok/s with CPU offloading. A $500 single-GPU build technically runs it at 0.2–0.5 tok/s — fine for batch jobs, painful for chat. The DeepSeek API at $0.14/M input beats self-hosting economically under ~500M tokens/month. Self-host for privacy, not savings.
Why Self-Host V4 — And Why You Probably Shouldn't
DeepSeek released the V4 family April 24, 2026 under the MIT license — both V4 Pro and V4 Flash sit on Hugging Face with full open weights, no usage restrictions, no royalties. Download, modify, fine-tune, deploy — no permission needed. That's unprecedented for a frontier-class model, and it's the whole reason self-hosting is worth discussing.
But the API is absurdly cheap: V4 Flash runs $0.14/M input, $0.28/M output. At those rates, a $2,500 dual-3090 rig needs roughly 500 million output tokens before it breaks even against the API — before electricity, cooling, or your time. The honest case for self-hosting is privacy, sovereignty, and offline capability, not cost. If those matter, keep reading. If you just want cheap frontier AI, use the API.
V4 Pro vs V4 Flash: Pick the Right One
DeepSeek shipped two models. Only one is self-hostable without a datacenter.
| Spec | V4 Flash | V4 Pro |
|---|---|---|
| Total parameters | 284B (MoE) | 1.6T (MoE) |
| Active per token | 13B | 49B |
| Official weight size | ~160GB | ~865GB |
| Q4_K_M footprint | ~142–174GB | ~800+GB |
| Minimum hardware | Mac Studio 192GB / 2×3090+128GB RAM | 8×H200 or 16×H100 cluster |
| SWE-bench Verified | ~78% | 80.6% |
| Self-host verdict | Practical for enthusiasts | Datacenter only |
The rest of this guide is exclusively about V4 Flash. If you need Pro's extra 2–3%, use the API — self-hosting it needs a cluster starting around $200,000. Flash delivers roughly 90–95% of Pro's quality on coding and reasoning at a fraction of the hardware cost.
Hardware Tiers: What You Actually Need
The number that matters: V4 Flash at Q4_K_M needs roughly 142–174GB of combined memory. No consumer GPU comes close — an RTX 5090 tops out at 32GB — so every consumer setup involves some CPU offloading.
- 0.2–0.5 tok/s — batch only
- Q4_K_M GGUF, full CPU inference
- 128GB RAM hard minimum, 192GB recommended
- Works: overnight review, batch summarization
- Doesn't work: interactive chat
- 2–5 tok/s — usable, patient chat
- Q4_K_M or Q2_K, GPU layers + CPU offload
- 48GB VRAM holds ~30% of model; rest in RAM
- Works: coding assist, RAG, document Q&A
- Bottleneck: DDR4/5 bandwidth
- 5–23 tok/s — genuine daily driver
- Q4_K_M, full model in unified memory
- 192GB holds model + 32K context comfortably
- Works: interactive coding, agents, long reasoning
- Best price-to-performance for Flash
Unified memory means CPU and GPU share one pool of fast RAM — no PCIe bottleneck. For MoE models like V4 Flash, where only 13B of 284B parameters activate per token, that's a perfect match: the model sits in unified memory, active experts stream through the GPU at 400–800GB/s. Community tests show 23 tok/s decode on an M3 Ultra 512GB at Q4_K_M — faster than most multi-GPU x86 rigs, at a fraction of the complexity.
Quantization Guide: Every Format Explained
Quantization is the difference between needing a datacenter (FP16, ~568GB) and running on a Mac Studio (Q4_K_M, ~142GB).
| Format | Weight size | Total VRAM* | Quality loss | Fits on |
|---|---|---|---|---|
| FP16 (full precision) | ~568GB | ~570GB | none (baseline) | 8×H100 / 4×H200 |
| FP4+FP8 (official) | ~160GB | ~170–175GB | negligible | 2×H200 / 4×A100 |
| Q8_0 (GGUF) | ~282GB | ~303GB | near-lossless | 4×A100 / 2×H200 |
| Q5_K_M (GGUF) | ~202GB | ~204GB | ~1–2% vs FP16 | Mac Studio 256GB+ |
| Q4_K_M (GGUF) ★ | ~142–163GB | ~142–174GB | ~2% vs FP16 | Mac Studio 192GB / 6×4090 |
| Q2_K (GGUF) | ~100GB | ~100–120GB | ~5–8% vs FP16 | 2×3090 + 128GB RAM |
| IQ2_XXS (GGUF) | ~73–81GB | ~80–96GB | ~10%+ vs FP16 | 1×5090 + 96GB RAM |
*Includes weights + KV cache at 8K context + ~1.2GB overhead. Longer context scales the cache linearly.
You'll see claims that V4 Flash "only needs 26GB of VRAM because it has 13B active parameters." That's dangerously wrong. MoE models keep all expert weights resident in memory — the router needs to reach any expert at any time. Active parameters cut compute and latency, not memory footprint. The full 284B must be loaded. Trust the weight-size numbers, not the active-parameter math.
Ollama vs llama.cpp vs vLLM
Three options, easiest to most powerful. As of July 2026, upstream llama.cpp has merged V4 support, and community GGUF quants are widely available from teamblobfish, Unsloth, antirez, and mradermacher.
Easiest. One-command pull and run, built on llama.cpp. Under 10 minutes to first token.
Most flexible. Full control over GPU layers, context, batch size, tensor split.
-tsProduction. Continuous batching, tensor/expert parallelism — overkill for single-user setups.
Step-by-Step: Run V4 Flash on Your Hardware
Exact commands across all three tiers, from a clean Linux or macOS environment. Recommended builds: teamblobfish Q4_K_M-XL (~163GB) or Unsloth Q4_K_M (~142GB) — both well-tested with the correct chat template baked in.
🟡 Tier 1 — Budget CPU-only (Ryzen 9 + 128GB DDR5)
# Step 1: build llama.cpp with CPU backend
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make -j$(nproc)
# Step 2: download Q4_K_M GGUF from Hugging Face
pip install huggingface_hub
huggingface-cli download teamblobfish/DeepSeek-V4-Flash-GGUF \
--include "Q4_K_M-XL/*" \
--local-dir ~/models/deepseek-v4-flash
# Step 3: run CPU-only inference (32K context)
./llama-cli \
-m ~/models/deepseek-v4-flash/Q4_K_M-XL/DeepSeek-V4-Flash-Q4_K_M-XL-00001-of-00005.gguf \
-p "Explain the difference between TCP and UDP." \
-n 512 -c 32768 -t 16 --temp 0.7
# Expected: 0.2–0.5 tok/s. Grab a coffee.🔵 Tier 2 — Dual RTX 3090 + 128GB RAM
# Step 1: build llama.cpp with CUDA
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES="86"
cmake --build build -j --target llama-server llama-cli
# Step 2: download Q4_K_M GGUF
huggingface-cli download teamblobfish/DeepSeek-V4-Flash-GGUF \
--include "Q4_K_M-XL/*" --local-dir ~/models/deepseek-v4-flash
# Step 3: launch server, -ngl 20 = 20 layers on GPU (~48GB VRAM), rest in RAM (~95GB)
./build/bin/llama-server \
-m ~/models/deepseek-v4-flash/Q4_K_M-XL/DeepSeek-V4-Flash-Q4_K_M-XL-00001-of-00005.gguf \
--alias "DeepSeek-V4-Flash" --host 0.0.0.0 --port 8080 --jinja \
--ctx-size 32768 --batch-size 2048 --ubatch-size 512 --flash-attn on \
--temp 0.7 --top-p 0.95 -ngl 20 -ts 12,12 --no-repack
# Expected: 2–5 tok/s decode. Usable for patient interactive chat.🟢 Tier 3 — Mac Studio M2/M3 Ultra (192GB unified memory)
# Step 1: install Ollama (easiest path on macOS)
curl -fsSL https://ollama.com/install.sh | sh
# Step 2: pull the community GGUF via Ollama
ollama pull hf.co/unsloth/DeepSeek-V4-Flash-GGUF:Q4_K_M
# Step 3: run interactively
ollama run hf.co/unsloth/DeepSeek-V4-Flash-GGUF:Q4_K_M
# Expected: 5–10 tok/s on M2 Ultra, 15–23 tok/s on M3 Ultra.
# The model loads entirely into unified memory — no CPU offload bottleneck.Real Performance Numbers
From community reports on r/LocalLLaMA, the llama.cpp issue tracker, and hands-on testing — single-user decode speed at 8K–32K context, Q4_K_M unless noted.
| Hardware | Quant | Decode tok/s | Context |
|---|---|---|---|
| Ryzen 9 7950X + 128GB DDR5 | Q4_K_M | 0.2–0.5 | 8K |
| 2× RTX 3090 + 128GB DDR4 | Q4_K_M | 2–5 | 32K |
| RTX 5090 32GB + 96GB DDR5 | IQ2_XXS | ~14 | 256K |
| Mac Studio M2 Ultra 192GB | Q4_K_M | 5–10 | 32K |
| Mac Studio M3 Ultra 512GB | Q4_K_M | 15–23 | 32K |
| 3× RTX PRO 4000 Blackwell | Q4_K_M | ~15 | 32K |
| 4× A100 80GB (vLLM) | FP4+FP8 | 60–90 | 128K |
Connect V4 Flash to Your Dev Tools
Once V4 Flash is running, it exposes an OpenAI-compatible API — point almost any AI coding tool at it by changing the base URL.
Add to config.json: "apiBase": "http://localhost:8080/v1"
Settings → Models → Add Model → OpenAI-compatible. Base URL http://localhost:8080/v1.
Set OPENAI_API_BASE=http://localhost:8080/v1 and OPENAI_API_KEY=not-needed.
Docker: point OPENAI_API_BASE at your local server for a clean chat UI.
Where to Get the Weights
deepseek-ai/DeepSeek-V4-Flash — ~160GB, not GGUF. Use with vLLM or SGLang. Needs 2×H200 or 4×A100.
teamblobfish/DeepSeek-V4-Flash-GGUF — ~163GB. Best for tool-calling agents. Well-tested across CUDA and Metal.
unsloth/DeepSeek-V4-Flash-GGUF — ~142GB, pullable directly via Ollama.
antirez/deepseek-v4-gguf — ~90GB, aggressive quant for 128GB Macs. Less extensively validated.
What Self-Hosted V4 Flash Can't Do — and What You Gain
Known limitations
- Text-only — no vision or audio
- No fine-tuning on consumer hardware — inference only
- Slower and slightly less accurate than the unquantized API
- Training a 284B model needs orders of magnitude more compute
What you gain
- Complete privacy — no data leaves your machine
- No external moderation layer or content filtering
- No API limits, quotas, or surprise bills
- Offline capability — air-gapped and field deployments
Frequently Asked Questions
The Bottom Line
Self-hosting DeepSeek V4 Flash is the most exciting thing to happen to local AI since Llama 3 — and the most humbling. It's genuinely frontier-class, the MIT license is a gift to the open-source community, and running it on a Mac Studio at 23 tok/s is remarkable. But the hardware requirements are real, setup is finicky, and the economics rarely favor self-hosting over DeepSeek's own API. Self-host for privacy, not for savings. If you have a Mac Studio 192GB+ or a multi-GPU rig gathering dust, download the Q4_K_M GGUF tonight. If you're on a $500 budget, use the API instead — better performance, zero setup, and enough credits to last years before matching a budget rig's cost.
You check your GPU temperatures one last time — the model hums quietly in the background.
🔬 Testing Methodology
- Test period
- 7+ days of hands-on use
- Pricing
- Manually verified at time of writing
- Affiliate
- Some links may earn a commission — full disclosure