Is there an open source project that does UI regression testing or are we all just wiring agents?

[u/larabyeol]

I've been looking for an open source answer to desktop UI testing for about 4 months and i keep ending up in the same place, which is a pile of general purpose agents and no actual test framework. The agent side is kinda good now with models like Openclaw, Goose where they drive a desktop app, screenshot it, work out what's on screen and click the right thing. That part is solved. However, what none of them have is the boring stuff a suite needs (no runner, assertion model, stable pass or fail…), so you end up writing that layer yourself and then it's yours to maintain forever.

The closest things i've found that are open source are SikuliX, which still runs but is basically frozen and matches raw pixels so it breaks on a DPI change, and the commercial vision based ones like Askui, eggplant get around it by pinning the model to a written script, so the perception stays fuzzy while the execution is deterministic.

Has anyone built that deterministic layer on top of an open agent and had it survive more than 3 months? Happy to be pointed at a project I've missed, thanks in advance!