r/learnmachinelearning • u/Upset-Day9099 • 15h ago

Is it a problem that AI reviews our AI generated code?

Most of our backend is agent written at this point, maybe 70%, and the only consistent review it gets before a human skims it is coderabbit, which catches real things but is still a model reading a model. Our pentest is 5 months out and I keep thinking about the fact that nothing with actual understanding has read most of this codebase.

The part I can't reason my way out of: if the generator and the reviewer share the same blind spots, the review confirms the code instead of checking it. A human reviewer disagrees with you in ways a model trained on the same corpus might not.

For people running security sensitive stuff, do you treat AI review as a real control or just noise reduction before the human? And has anyone actually caught the same-blind-spot problem in the wild?

0 Upvotes
  • permalink
  • reddit

50% Upvoted

2 comments sorted by

1

u/Bomaruto 15h ago

Perhaps you should read the code instead of trying to reason your way out of it.

Until you do then you cannot say anything for certain about the quality of your codebase.

1

u/Kihino 13h ago

AI reviews everything. Humans review critical parts. Security is critical.

Architect what you build in a way so very few daily changes impact security, to stay safe while move fast.