• 0 Posts
  • 5 Comments
Joined 7 months ago
cake
Cake day: March 24th, 2026

help-circle


  • I started off with Ollama, but liked llama.cpp better because you can download one GGUF file the day it’s released and add it to a source controlled config.ini with parameters to make it work best on your hardware.

    I didn’t get serious about local LLMs until I realized I could run Qwen 3.6 35B-A3B on my 3070 with q4 for KV cache and get over 100k context. I found Ollama’s Modelfile approach and model handling in general to be clunky and discouraging important tweaks, but llama.cpp keeps everything simple.