DeepSeek

This post is about running DeepSeek-R1 (671B) locally.

Get GGUF

ollama pull deepseek-r1:671b

Create a Customized Modelfile

FROM deepseek-r1:671b
PARAMETER num_gpu 2
PARAMETER num_ctx 2048
PARAMETER temperature 0.6
TEMPLATE "<｜User｜>{{ .System }} {{ .Prompt }}<｜Assistant｜>"

Then create a new model:

ollama create my-deepseek-r1 -f modelfile

Run

ollama run my-deepseek-r1