Recommend for Small Model for Home Assistant

[u/MacroNexus]

Hi, are there any recommendations for small models (cos my pc ain't that good, just a rtx 3080 with 64gb ddr4 ram) to host at home as a home assistant (i.e. asking it to switch off the lights, purifiers, or asking it to send me daily briefs on today's news and weather) through OpenClaw?

I am currently using Gemini 3.5 flash lite which is great and the pipeline is working (OpenClaw set up and configured) but am hitting the api limit far too often, so am thinking of transiting to a local model for this.

I asked chatGPT and it gave me a list of models that are quite dated and not sure if there are better ones these days. Thanks in advance!!!

[u/EvolvingDior]

Ling 3.0 Tiny can easily handle all of that. And you should switch to Hermes. OpenClaw is a bloated nightmare.

[u/conifer_v11]

on a 3080 stick to 3-4b instruct at q4, qwen3 4b or llama 3.2 3b both handle intent routing fine.

home assistant work is classify plus fill a json slot, you do not need reasoning for that.

keep gemini for the news brief and let local do the on/off commands. splitting by task beats one model doing both, and it kills the api limit problem for the calls you make all day.

[u/IroesStrongarm]

Not through OpenClaw, but for home assistant assist directly and voice commands in quite happy with Qwen3.5:9b_q4. At 12k context it takes about 6.5Gb Vram.

[u/Healthy-Zebra-9856]

I am not sure what ChatGPT you are using, but it does check the HF & modelscope etc. for current info. There is Lfm 2.5, Ling, Nanbeige4.2 3B. This last one is a 3B but its more capable than most as its a Compressed Loop Transformer.

[u/_TheWolfOfWalmart_]

I use Gemma4 26B-A4B (running on CPU!) with Home Assistant. It's absolutely phenomenal at this, and FAST! Yes even on CPU. It works so well I didn't see a point in hogging up VRAM for this 24/7 always-on model. There are better models overall today for sure, but this one's personality is just perfect for an HA assistant and even if it's not the greatest tool-caller, it's never once messed that up with HA for me.

I asked chatGPT and it gave me a list of models that are quite dated and not sure if there are better ones these days.

Yup. NEVER ask an AI for model recommendations lol. Even if they have web access, they're horrible at this and tend to stick to their very outdated training data unless you really push them to research current models.

[u/Formal_Spirit_5]

You want to run it directly from homeassistant, not from openclaw or other agent. You can run gemma-4 E4B, which will be really fast and responsive to your commands, but a bit stupid, or if you need a little more brains but a bit slower, then gemma-4 12B should be also good, but not sure about speed. For HA you usually dont need much context, 32k is more than enough, or even 16k. MoE models spilled to system RAM will be IMO too slow to make it respond fast enough so it feels interactive.

For it to feel responsive you want to target >1000pp/s and >70t/s . I currently got that with a RX6800 16GB and qwen3.6-35B UD-IQ3_XXS with MTP and its pretty usable - it responds most of the time within 5-6seconds. For some time I also run the gemma-4 E4B, and it was super fast and resposive, however wasnt very good with more complicated requests.

[u/kalimatamijai]

4B models are suitable

[u/AB172234]

Use prism ml’s Bonsai, 1bit version of Qwen 3.6 27b. As per them it preserves 90% of intelligence at 3-4GB of VRAM.