r/homeassistant • u/catversary • 1d ago
š¼ļø Show & Tell Visual Cat Tracker
Enable HLS to view with audio, or disable this notification
My cat loves to hide and pounce on me in the garden, so I asked Claude to build a tracker using a PoE camera mainly to get live/last known positions and escape zone alerts.
- Using the Reolink OMVI Wide Camera (Frigate Re-stream)
- Stream is fed into OpenCV running YOLO26 looking only for cats
- OpenCV is running on a 5080 GPU around 60% utilisation during tracking (I turn this off whenever I bring him inside)
- The X/Y coordinates from the tracker are sent via MQTT to Home Assistant
- Home Assistant presents the same frigate garden stream but overlays a crosshair for the X/Y coordinates.
I've setup ignore zones, escape zones and ptz zones with polygons - which Claude built a HTML editor and JSON exporter for - I just drew the polygons and said what should happen with which zone - so whenever my cat is detected in one of these zones - either an pre/escape automation will start (lights and sounds in the lounge) or PTZ camera on the OMVI will appear in a picture in picture overlay showing my cat if he's outside the wide angle view.
It is honestly unbelievable that this is possible with just natural language and without a single line of YAML.
13
29
u/lostaccountby2fa 1d ago
a RTX 5080!? a $1500+ card?
15
10
u/OrangeBagOffNuts 21h ago
I need to do something similar but to track where my dog poops, save that location somewhere, have a massive garden and it's hard to tell where he drops, sometimes only finding it days later when stepping on it
4
2
u/catversary 16h ago
Itās certainly possible, you can request a grid of your cameras frame, then each cell becomes a zone where the tracker can see how long your pet has dwelled/lingered - you can likely then generate a heat map of how long they have spent at a particular position on the grid and feed back visual confirmation of a defecation to build up a poop time pattern.
At that point you might even be able to predict when, where and how long your dog will poop.
1
16
u/Clusternate 20h ago
all you need is:
- a 1500 GPU
- a working Home assistent hardware setup
- a video Kamera with PTZ
- a PoE Switch
- power to run the GPU at 60%
- AI
- time to understand the code
- time to set this up
- a garden
- a cat
easy
7
u/Westerdutch 17h ago
- AI
- time to understand the code
Those two cancel each other out perfectly, you only need one of them!
4
u/CyberMage256 17h ago
I'd call this "AI-lite" not full AI LLM. This is YOLO26. I basically use the same thing to see if my garbage cans are at the curb or not. I don't check a live feed, but grab a single frame every 5 minutes and process on the CPU. There's zero reason to run an entire LLM for something like this.
2
u/catversary 16h ago
You are correct, I also think there would be far too much latency for any modern llm to sufficiently return tracking positions fast enough to display in real time
1
u/catversary 16h ago
Youāve nailed it, I only have a high level understanding of how everything is wired up, I did not write any of the code/scripts/mqtt connection/home assistant dashboard at all. Everything was done via HA MCP and ssh access.
And when things havenāt worked or I wanted to enhance things, I just asked in natural language for it.
1
u/CyberMage256 13h ago
I'm reminded of William Gibson's quote. "The future is here, it's just not evenly distributed"
0
u/Westerdutch 16h ago
Hey, if you feel fine handing ssh access (or any access really) to AI then you are a 'braver' man than me! To me everything that has to do with security will be done by me and only in ways i understand and can check and maintain.
2
u/catversary 16h ago
Thatās fair enough, you can always run in the approve each request mode (not auto mode) to individually verify every command the LLM runs.
Iāve been tempted to just let it go in auto mode because at this point Iām the bottleneck, and the damage it could do in that mode can be limited with good backup hygiene.
Though I suspect youāre worried about the LLM installing some hidden, malicious package or service somewhere on your LAN to steal your data or worse.
0
u/Westerdutch 15h ago
you can always run in the approve each request mode (not auto mode) to individually verify every command the LLM runs.
Ask yourself how much you trust your AI to self moderate without mistakes.
and the damage it could do in that mode can be limited with good backup hygiene.
If you have nothing of importance (access/passwords) connected to any of your systems then that could be a choice but most of the time there is enough on just about any system to where this is not a smart move. Something as simple as a hacker kid finding a door some ai accidentally opened that gives him access to your cameras and thus your wife sunbathing in the garden can cause you a whole lot of headaches.
Though I suspect youāre worried about the LLM installing some hidden, malicious package or service somewhere on your LAN to steal your data or worse.
Nothing that involved. Just basic fckups many people make in their network all on their own. I know that im fairly on the overly secure side of the scale and i understand that not everyone will have a separate disposable wifi network with no internet access for any iffy iot device in their house but you really do not want to be on the polar opposite side of that scale. My sister had their wifi compromised and they discovered this when the police came knocking on their door because a whole shtstorm of kiddy porn got uploaded from their internet connection.... and gaining a wifi password can be as easy as booting a cheap wifi lightbulb that doesnt follow proper procedure off a network and seeing how it reconnects. When you throw an AI in the mix that does god knows what then you are opening the door to a whole lot of random especially if you do not understand any of what it does, the AI itself sure doesnt know what its doing other than copying patterns till it get something resembling an outcome that it calculate you might want. Keep it on the tightest leash, do not 'auto' any of that.
1
u/CyberMage256 13h ago
I once had Claude tell me to blow away my Firefox profile on my kiosk to fix an issue. It didn't even occur to me I didn't have console access to reconfigure with a mouse / keyboard / screen. It's ssh only. When I commented that Claude responded "Oops. Haha. Sorry."
2
u/dice1111 17h ago
Well, if you already have those things, like the OP probably did, may as well...
2
u/catversary 16h ago
Yes, this exactly - I use the 5080 to game primarily. And already had the PoE camera setup for recording, so this use case was a fun experiment with stuff I already had.
2
u/Westerdutch 17h ago
OMG this is so nerdy. I kind of like it. Its also absolute hardware overkill for the purpose though. But hey, there are two things AI dont care about with burning passion; power use and hardware optimization so you get what you get.
My home assistant hardware is so light weight that frigate isnt even capable of telling a car from a bird (even if that car has been standing there for multiple days) so i will not be attempting this even though i do have ptz reolink cameras in my garden that should be able to do this AND a cat thats both the worst and also the most stubborn escape artist in the world.
2
u/CyberMage256 17h ago
"OpenCV is running on a 5080 GPU around 60% utilisation during tracking (I turn this off whenever I bring him inside)"
Why not put some logic behind "cat not in garden, put live track to sleep" and switch to analyzing a single frame every 15 to 30 seconds until the cat is seen again, then wake up the live tracking? Now no reason to manually flip the switch.
1
u/catversary 16h ago
Interesting optimisation, itād be simple enough - if he isnāt seen for 5 mins, the reduce fps and or processing per second until heās seen again where weād ramp back up to default. I might give that a go, thanks for the idea.
I havenāt gone too far into making it efficient because I always power off the pc when Iām away or heās inside anyway - so this is definitely not meant to be running 24x7
1
u/CyberMage256 13h ago
I'll trade you a near pristine GTX 1070 for that 5080 if you want to save on your power bill. You can tell it's pristine because I haven't blown the dust out of my PC in 6 months so it's probably coated completely with crap. :-)
1
u/GreekHubris 17h ago
Is it possible to do something like that to detect 3D printing spaghetti? Using OpenCV and YOLO26 perhaps?
1
u/catversary 16h ago
I doubt that classification exists in the YOLO model? But potentially something you could train it for with enough reference images.
45
u/xivory93 1d ago
Thatās some expensive tracking.. Google TPU should be enough for what you are doing.. I donāt know how much it will hit you CPU for the tracking but worth a try..