OmniRoute is also pretty nice for that.

Any PC that can play video games is fully sufficient to generate text with LLMs.

I am all for local inference, but that is a super ignorant and uninformed take. I hope you are not running around complaining about the hallucinations and bad quality of LLM generated stuff, if your baseline are models that fit in 8 GB of VRAM or something. And no, 16 or even 24 GB wouldn’t bring you close to what the big models can do.

Someone linked an interesting article recently which I unfortunately didn’t save. The gist was that Anthropic and the others set out to sell cheaper workforce (I.e. replace programmers), so they can sell the service to all the big software giants out there.

Problem is, that these tools also enabled people to just create own solutions instead of buying existing solutions anymore. Or to rebuild an equivalent for something otherwise proprietary.

So instead of making workers of big tech obsolete, they made big tech obsolete (well, not fully yet, but you know what I mean).

They likely try to counter this a little bit now. Won’t work though, since open weight models have been and still are catching up.

I once called my landlord a leech. But I have no idea if it ever showed up or if he even wanted it. /j

Wait till you stumble on their hats-off trilogy.

Last year our CISO personally reached out to me and asked me to do the course that’s overdue. I was a little nervous for a second and decided to obey.

I have over 8000 unread mails in my inbox. Whoever contacts me used Slack. I don’t give a fuck about emails unless DMs me “hey can you maybe answer that customer mail I forwarded you?”

So I would have a fabulous time with such a stupid policy.

Their own harnesses breaking out of their test environments and hacking third party companies is “oopsi” but their customers using an agent in a controlled manner to reverse engineer firmware of a device they own is bad. I see. Fuckers.