So… I’m with you/them about the vendor lock in but I think they’re slightly underestimating what ollama brings to the table. For example, ollama bundles CUDA libraries so I don’t need to worry about that as I wrangle with my GPU. It’s also well supported by other ecosystems like home assistant in a way that raw openai compatible API server isn’t.

Overall I don’t know if I find these arguments compelling at this time.

Man you can absolutely run something on ollama yourself. Right now I have ollama running qwen 3:4b-instruct on 6vCPU and no GPU and for very simple conversations it’s quick enough. TTFT is slow when prompt is uncached but otherwise TTFT is like 2s and I get maybe 5 words/s out.

If I gave it my 1660 I could probably get ~100tps but my card won’t pass through to my VM properly…

Most of this continent was built by racists who owned car companies. Turns out racists that own car companies really wanna make sure that everyone has to drive and black kids stay away from white kids, and then they build cities to make those things happen.