r/ManjaroLinux 3d ago

Tech Support I kept finding AUR packages that had been silently broken for weeks after Manjaro caught up to Arch, so I wrote guards for it

Every few months I'd find something on my system that had been quietly broken — not crashed, just wrong — and it always traced back to an AUR package built against a library version that no longer existed.

It isn't random. It's two specific failures at opposite ends of the same window, and both are detectable.

While Manjaro is behind Arch. Third-party binary repos (BlackArch, chaotic-aur) build against current Arch. So they ship packages whose dependencies our repos won't have for another week or two. You get a refused transaction, or you force it and now you have a partial upgrade.

When Manjaro catches up. libfoo goes 1.0 → 2.0. Every AUR package built against 1.0 still shows as installed, still has all its files, checksums fine — and can no longer resolve its DT_NEEDED. pacman will never tell you, because from pacman's view nothing is wrong. This is the one that kept getting me.

Separately: an AUR PKGBUILD can point source= at a git repo with no commit, so what you reviewed last week isn't necessarily what builds today.


First, the obvious objection: "this automates pacman --ignore"

Yes. And --ignore is listed on the Arch wiki as a way to cause partial upgrades, so this deserves a real answer rather than hand-waving.

**--ignore does not disable dependency resolution.** I checked rather than assumed — built a throwaway pacman root, installed consumer 1.0 depending on libX=1.0, offered upgrades to both, then ignored libX so that consumer 2.0 would need a libX 2.0 that isn't there:

$ pacman -Syu --noconfirm --ignore libX
warning: ignoring package libX-2.0-1
warning: cannot resolve "libX=2.0", a dependency of "consumer"
error: failed to prepare transaction (could not satisfy dependencies)
:: unable to satisfy dependency 'libX=2.0' required by consumer
exit 1

$ pacman -Q            # nothing moved
consumer 1.0-1
libX 1.0-1

$ pacman -Dk           # pacman's own consistency audit
No database errors have been found!

pacman resolved the whole transaction, saw the hole, refused, and changed nothing. It is not possible to --ignore your way into an unsatisfied dependency; pacman is the backstop, not the script.

That's section 19 of the test suite and runs in CI on every push, against real pacman transactions. It's deliberately a test of pacman's behaviour rather than of my code — the safety argument for the whole thing rests on that property, nothing in my repo would notice if a future pacman changed it, and the failure would be silent.

Beyond that guarantee, what gets held matters:

  • It only holds a package whose version-constrained dependency is already unsatisfiable — something that cannot install correctly at that moment regardless of what I do.
  • It holds the new version back, so the package keeps the dependencies it already had and stays in the state it was already working in. The classic dangerous pattern is pinning a library while upgrading its dependents; this is the opposite direction.
  • The realistic alternative isn't a clean full upgrade. It's a failed transaction, or someone reaching for --ignore by hand with worse aim.
  • Every hold is printed and written to /var/log/safeup.log, so it's a deferred upgrade you can chase — not a package that silently stops updating forever, which is the actual long-term hazard.

The limit, stated plainly: it does not pre-check whether a package it holds is itself required by something else in the transaction. It relies on pacman refusing, then isolating the break. That's the behaviour above, and it's why that's acceptable — but you should know it works that way rather than by its own cleverness.


What the three checks do

  • **safeup** wraps pacman -Syu. Works out which pending third-party upgrades have dependencies that can't be satisfied yet, --ignores only those, and logs every hold. Also recovers from installing X breaks dependency … required by Y by isolating X so one broken package doesn't block your whole upgrade.
  • **aur-rebuild-check** runs ldd over every file owned by every foreign package and reports unresolved sonames. This is the one nothing else does. It filters the false positives that make -bin packages flag forever (libs bundled privately and loaded through an $ORIGIN RPATH).
  • **aur-pin-check** refuses PKGBUILDs whose git source isn't pinned to a commit. #tag= is refused too — a tag can be moved. There's an allowlist, but it re-resolves the tag every run and refuses the package if the SHA changed.

aurinstall / aurupdate run the pin check before yay. If a check refuses something and you disagree, plain yay -S still works and the message says so — a gate you can't bypass just gets uninstalled the first time it's annoying.

aurinstall --chroot (gated default) hands the actual build to chrootbuild, from Manjaro's own manjaro-chrootbuild package. This is a gated check building in a isolated chroot (not for security reasons). If unable to use chroot, it automatically passes to yay.


Testing

65 assertions, in CI. The main suite builds a file-backed loop device, formats it, and constructs a whole throwaway pacman root inside — its own database, its own file:// repos built with the real repo-add, real packages — then drives actual pacman through the real scenarios and checks installed versions afterwards. Nothing on your system is touched. The ldd tests compile real .so files with a real unresolvable DT_NEEDED, so no output is faked. The allowlist test makes a real git repo, vets a tag, moves the tag, and asserts the next run refuses it.

Writing those tests found three real bugs in code I'd been running for months — including one where a verification step exited non-zero after a successful kernel removal, on every machine whose ESP isn't laid out like mine. CI caught that one; my own machine couldn't. That's the argument for the tests.


What it doesn't do: it doesn't make the AUR safe, doesn't tell you a PKGBUILD isn't malicious (pinning proves you build what you reviewed, that's all), doesn't sandbox makepkg, and can't fix a package that genuinely needs a newer lib — holding it is the right answer there. Not a substitute for snapshots.

MIT, pure bash, no runtime deps beyond pacman and coreutils, nothing phones home.

https://github.com/doug445/manjaro-safeaur-updater

Genuinely interested if the dependency logic has a case I haven't hit.

1 Upvotes

7 comments sorted by

2

u/NixNicks 3d ago

Instead of doing all this just

sudo pacman-mirrors --api --set-branch unstable

and

sudo pacman-mirrors --fasttrack 5 && sudo pacman -Syu

You're not lagging behind anymore

1

u/doug445 3d ago

I've actually run this experiment — I switched to unstable in the past for exactly this reason, and broken dependencies still showed up. That's not bad luck; it's structural. "You're not lagging behind anymore" misdiagnoses the problem, because lag is only one of three ways this breaks, and unstable makes one of the others fire more often, not less.

First, unstable doesn't eliminate the lag — Manjaro's own wiki says it "usually runs within 3 days behind Arch package releases." Foreign repos and the AUR target Arch. A 0–3 day window is smaller than stable's ~2 weeks, but it's still a window, and a soname transition only needs a day to strand you.

Second — and this is the one people miss — unstable makes AUR breakage arrive faster. AUR packages are source recipes; the binary you built has the sonames from build day burned into its DT_NEEDED entries. When libfoo.so.8 → libfoo.so.9 lands in your repos, that binary dies with error while loading shared libraries until it's rebuilt — and pacman -Syu will never warn you, because pacman resolves the transaction against its sync databases and declared depends=, which for most AUR packages are unversioned strings. depends=(ffmpeg) does not encode libavcodec.so.62. On unstable you receive every Arch soname bump within days of it happening — before most AUR maintainers have pushed a pkgrel bump, and while orphaned packages will never get one. You've put yourself at the front of the exact race that makes Arch users run checkrebuild (that's why rebuild-detector exists in Arch's extra repo). On stable, by the time the bump reaches you weeks later, the actively-maintained AUR has usually caught up. So unstable trades a predictable, batched breakage moment for a continuous drip of it — which matches what I saw when I ran it.

Third, unstable is Manjaro's staging branch, and staging branches have internal skew of their own. It's an Arch-stable snap plus Manjaro's overlay — kernels, mhwd, the NVIDIA stack, pamac — and the overlay gets rebuilt against the new snap after the snap lands. In that gap, version mismatches between overlay and snap are normal; catching them is literally the branch's job. The wiki is explicit that unstable "may break your system and should therefore only be used by more experienced users and Manjaro testers." Regressions caught on unstable are caught by you.

Also, the commands as posted are subtly wrong: Manjaro's documented procedure after a branch switch is pacman -Syyu, not -Syu — the branch is a path component of the mirror URL (.../manjaro/stable/$repo/$arch), so the sync databases must be force-refreshed across the jump. -Syu only works when unstable's db timestamps happen to be newer, which is luck, not correctness. And --fasttrack is orthogonal — mirror ranking has nothing to do with dependency coherence. Going back down later is a -Syyuu mass-downgrade against repos that may not even carry downgrade candidates for the library versions you're holding.

And step back from the mechanics for a second: the proposed fix for "packages sometimes break" is to run the branch literally named unstable. That's not a workaround, it's a category error — you don't cure a dependency-coherence problem by removing the stability guarantee from the entire base system. The failure domain goes from "some AUR binaries need a rebuild" to "anything, including the kernel you boot from."

So the actual scorecard for "just switch to unstable": cross-repo lag — shrunk, not closed; locally-built binary staleness — untouched, and now firing on Arch's schedule instead of Manjaro's; supply-chain drift (rewritten upstream tags, dead source= URLs in orphaned PKGBUILDs) — untouched, branch has zero bearing on it; and in exchange your kernel, display driver, and boot chain ride the least-tested point of the distribution's pipeline. Nothing in that trade checks a single dependency that pacman -Syu wasn't already checking.

That's the gap the tool covers: it holds foreign packages whose deps have outrun the repos (with pacman still doing full resolution underneath — --ignore never bypasses dependency checking, which is CI-tested against a real pacman root), scans installed foreign packages' DT_NEEDED entries after every catch-up so stale binaries are detected instead of discovered at runtime, verifies pinned AUR tags against git ls-remote before anything builds, and builds in a clean chroot via Manjaro's own manjaro-chrootbuild with dependency chains resolved from .SRCINFO. None of that is replaced by updating faster — I know, because I tried updating faster first.

2

u/kaptnblackbeard 2d ago

Interesting idea, but still trying to get my head around it all and if there isn't a better approach built into AUR or pacman. But your concept does raise a real issue.

First, unstable doesn't eliminate the lag — Manjaro's own wiki says it "usually runs within 3 days behind Arch package releases."

Do you have a link for that? I can't find any reference to this in the wiki and it isn't my experience either. Unstable packages come directly from the Arch repos unless they're Manjaro specific and I can get multiple updates in a day if I run an update more than once.

4

u/nikgnomic 1d ago edited 1d ago

Manjaro Wiki - Switching Branches
Unstable Branch: Unstable is synced several times a day with Arch package releases

Manjaro Wiki - Manjaro: A Different Kind of Beast
Unstable branch is synced several times daily from Arch stable

1

u/kaptnblackbeard 1d ago

I've been thinking about this project, and one big question for me is why Manjaro and not all distros that use the AUR?

1

u/lyidaValkris 12h ago

Watching this with interest. I haven't noted any packages break by this method (and I use the AUR sparingly, as one is supposed to, but I guess that's not an option for some).

While this looks great - have you thought of contacting the pamac maintainers and suggesting these sort of improvements, or contributing this to it? It would be neat if this functionality was baked into Manjaro's meta-package manager.