r/technology • u/ControlCAD • 14h ago

Security Linux kernel nears record 2,000 vulnerabilities per release as AI bug hunters scour 40 million lines of code — maintainers say they are "completely overwhelmed" by CVE finds

https://www.tomshardware.com/software/linux/linux-kernel-nears-2-000-cves-per-release-as-ai-bug-hunters-scour-40-million-lines-of-code-maintainers-say-they-are-completely-overwhelmed
1.7k Upvotes

96% Upvoted

271 comments sorted by

View all comments

1.8k

u/Humpaaa 14h ago

I don't care about the millions of AI found CVEs. The important line is: Maintainers are overwhelmed.

We need to realize how much of our everyday infrastructure is not adequately supported, and find ways to divert ressources to these positions.

336

u/mydogeatspoops 14h ago

I’m sure there is an oligarch or commercial interest that would love to support the project for just a few extra lines of code here or there.

123

u/WeekendCautious3377 11h ago

Used to work in software security. First team to get re-orged / layoffed during boom time. Execs never invest in it until something goes horribly wrong

55

u/SnooSnooper 11h ago

And even then they only really care for a couple months/quarters, after which their goldfish-like attention span demands they shift resources to whatever new shiny all the other execs are talking about

13

u/userhwon 10h ago

Execs are the horribly wrong.

4

u/MammothUnique4147 9h ago

This is the way 

3

u/BriefSpecial420 5h ago

If that was different in critical infrastructure. Few lives mean nothing when stock enters the game as long as no extra regulations protect them.

115

u/jethroguardian 13h ago

Melinda Gates could do the funniest thing...

56

u/mayorofdumb 13h ago

MacKenzie would do it to spite Bezos.... shes the reason why they are trying to get rid of marriage equality.

10

u/SeeMonkeyDoMonkey 11h ago

Why do you think Bezos would be unhappy at improvements to Linux?

11

u/atxweirdo 10h ago

It’s not that he would be unhappy but it would be taking power from him and the opportunity to make a lasting change for good which seems to be antithetical to his motivations of late

22

u/fredy31 6h ago

Isnt it exactly how we almost had a linux backdoor a year ago?

Dude maintaining a library thanklessly for 20 years suddenly had a great volunteer to help with the load.

That volunteer then tried to sneak in a backdoor that would let him break openssh and then any linux machine.

3

u/ehalepagneaux 3h ago

I always said if I end up extremely wealthy I'm spending a lot on stuff like this.

445

u/krileon 13h ago

"We found 1000 CVE!" "Ok, and how are they exploited?" "Well the system needs to have already been exploited" ... this has also been the recent "AI FOUND VULNERABILITY" trend.. they're 99% of the time nothingburgers.

239

u/TOGFIAVDF 13h ago edited 13h ago

Yeah, I'm in cybersec and this is a pretty apt description of what these CVE notices are really saying.

Most of these vulnerabilities are only vulnerabilities when the system is already compromised. They still need to be addressed, but it isn't Swiss cheese.

Though, it should be noted that vulnerabilities are still vulnerabilities. Just because a vuln needs an already compromised system to be exploited doesn't mean said vuln can't be used to push an attack further.

edit: wording

38

u/silversurger 11h ago

That's probably part of why they're so overwhelmed. They have to classify, assess the amount of work and prioritize thousands of CVE notices. And until it has been classified, everything is urgent. This all happens before even any code has been written.

9

u/TOGFIAVDF 11h ago

Yep, it has to be brutal.

1

u/drunkbusdriver 26m ago

Just ask the same AI to classify it

64

u/krileon 13h ago

You're not wrong, but a vulnerability that already requires a compromised system goes from "WE NEED TO PATCH NOW" to "maybe sometime next year?" on priority. Regardless still requires a human to deal with and wastes a bunch of time having to look over these.

14

u/CocodaMonkey 11h ago

Already exploited system is doing some heavy lifting here. In many cases already exploited means they need user level access. On a server that is generally the same thing but on a user device it can mean they can bypass locks that have been imposed on them specifically.

21

u/TOGFIAVDF 13h ago

That's what I'm saying.

Hopefully they're using an effective framework to sort their tasks.

12

u/Cow_says_moo 11h ago

I'm not sure if we can still think that way. AI isn't just good at identifying individual vulnerabilities, but also at chaining relatively minor vulnerabilities together into exploits which could be more threatening than initially expected.

10

u/sickofthisshit 11h ago

There's an architectural aspect, though. "Getting root" at one place should be fixed at the one place.

If a stupid bot then tries to exhaustively search all the ways you are fucked once you have root, well, that's the whole kernel, right?

Unless the bots deeply understand this kind of thing (as opposed to understanding the shape of a vulnerability and the shape of an exploit), it creates an enormous amount of noise.

3

u/Sislar 9h ago

A vulnerability can also be a permissions escalation. So a valid user can get admin privileges so not on a compromised system but on a system someone already had legitimate access.

2

u/TOGFIAVDF 8h ago edited 8h ago

Fair distinction, though that falls in its own incident category. Ultimately, anyone with meaningful access to a system is a weak point.

Without giving too much info, a company I've worked for had everything on a single network share. The only restricted folders were individual user desktops, and those were accessible by a handful of authorized users.

Engineering drawings, vendor and order information, sales orders, source code to software, etc. Literally everything accessible by anyone on the network. To make it worse, each users' workstation was a Windows OS with an admin account and access to PowerShell.

It's amazing they didn't deal with more cyber incidents. I guess being a smaller company helped in that regard.

3

u/Bubbagump210 7h ago

Issue: when root can run rm -fr /

1

u/bullwinkle8088 3h ago

Modern rm binaries annoyingly stop you from doing that even as root. i think you can override that but it makes wanton destruction less fun.

Yes, sometimes I just want to watch a system burn.

2

u/Due-Consequence9579 6h ago

Also since the kernel operates at such high trust they use a very liberal definition of cve. Anything out of spec is considered a CVE.

11

u/digiorno 12h ago

That said , this sort of layered attack is exactly how some of the most impressive cyberattacks in history have succeeded. Most places don’t have anything to worry about. But you can bet places like the NSA are trying to harden their systems asap.

30

u/PaulCoddington 13h ago

Unfortunately, devs end up wasting time triaging them.

22

u/QuackQuackViech 9h ago

Yeah I hate this shit too. "Oh no a new CVE has been found, in my report it says 9.9 and its colored red. Red I tell you!! Fix immediately."

Then you read the fine print and it says "Attacker must be omniscient being with direct console access and all admin passwords, then he can inject commands to change the clock."

12

u/Valuable-Worth-1760 12h ago

Critical and high vulns are also exploding everywhere. Mythos' submissions were not inherently lower severity than human submissions.

23

u/krileon 12h ago

Those can get drowned out by 1000+ pointless CVE for vulnerabilities that require a system to have already been heavily compromised. The noise will cause real threats to be missed.

5

u/Valuable-Worth-1760 12h ago

Yeah sure, I just get the impression that folks are distracting from any discussion about wtf do we do about capable AI (risk of competence) with what do we do with slop AI (risk of incompetence). Both real, but if there are capable models than those can also be used to triage slop. Vs if all models are just slop, then we need to focus on getting models to be better/more competent- but that "solution" would make the risks of competence much... worse. Which is why we need to take them seriously, instead of downplaying/distracting from model competence. The general public does not think AI is competent (at anything).

3

u/Humpaaa 12h ago

Still, all those nothingburgers take time to search througha s a maintainer.

1

u/technobrendo 9h ago

Don't they also have a severity score? That makes a huge difference

1

u/kawag 5h ago

That doesn’t matter. Vulnerabilities still need to be investigated and fixed - even if the exploitable code path is not reachable today, who’s to say it will never be reachable? Never get copied and pasted or refactored for use in any other projects?

1

u/Expert_Garlic_2258 2h ago

Or the classic, " if you have physical access to the box..." vulnerabilities

1

u/[deleted] 11h ago

[deleted]

3

u/WealthyMarmot 10h ago

this is how a lot of the most devastating cyberattacks work nowadays. Rarely is there a single vulnerability that will open the door to a hardened system, so attackers chain together multiple exploits to work their way into a privileged position.

57

u/Stilgar314 14h ago

Relevant xkcd

34

u/captain150 13h ago

Man and that comic was years before the xz utils state sponsored supply chain attack. It's amazing the maintainer stood strong against it as long as he did.

14

u/Stilgar314 13h ago

Yeah, from all the relevant xkcd, this happened to be the most relevant.

34

u/romario77 13h ago

this has to be a one time thing - these CVEs were created over many years of development and only now are noticed because of AI.

Once fixed I don't expect as many to appear again. There is no easy solution to this, but I expect better and more robust software after this is sorted out.

23

u/otherwiseguy 13h ago

The only way out is through.

11

u/araujoms 13h ago

I don't think so. The Linux kernel is extremely large. I think if you keep digging you'll keep finding vulnerabilities. In some obscure hardware used by few people, but still. If the maintainers have to look at them they get bogged down all the same.

In the past nobody found vulnerabilities in the less-used parts of the kernel because nobody bothered to look. But now with AI people can spend a few minutes and say "Look, ma, I have a CVE!"

I think what will happen is that people will stop caring, and only serious vulnerabilities affecting the core parts of the kernel will be addressed.

7

u/kenlubin 12h ago

The alternative seems to be that the Linux kernel is cutting support for outdated networking technologies. Maybe of the bugs are in the drivers for outdated technology that no one uses anymore, so there aren't very many eyes on the code.

1

u/DKLancer 5h ago

Well that's basically all OT SCADA devices running power panels and air handlers. Those things are on 30 year refresh cycles.

I worked in a place that air gapped then all onto a different network because there was no possible was to secure it all

1

u/Mr_ToDo 11h ago

I'd be a great day to be a kernel that doesn't need to take all that hardware support internally

1

u/Ok_Turnover_1235 9h ago

That's the beauty of running gentoo

1

u/daHaus 13h ago

That's very optimistic

9

u/Atlatica 11h ago edited 11h ago

I mean there is the wider problem too, which is that there isn't a constant stream of junior-mid level developers using open source to grow their portfolio any more.  There are just pre-AI mid-senior devs, and there are grads with Claude subscriptions and entry level retail 9-5s who vibe code video games in their spare time.

3

u/ilep 11h ago

Cutting from education is a big problem the way I see it. Things are only getting more complicated to learn and there is not enough time to learn basics properly any more. And if you are programming on kernel you do need to understand hardware as well since that is what kernel deals with.

18

u/Senior-Albatross 13h ago edited 13h ago

We'll do what we always do: ignore the obvious problems until something catastrophic happens and then address it afterwards at much greater cost. 

1

u/PlanPlus8582 6h ago

How else are we going to sell solutions without problems?  /s

4

u/bwrca 13h ago

There's trillion dollar companies that built and continue to build their products on top of this platform, but the platform itself lacks adequate engineering support. Not surprised.

4

u/TeutonJon78 9h ago

Also goes back to the "easier to destroy than create" adage.

Setting AI loose on a code base takes relatively no human effort.

Sorting through the output, determining actual severity, and then creating the fix/redesign and getting it responsibly deployed is a huge amount of actual human effort.

16

u/ignatzami 14h ago

That's the Achilles heel of open source. And I don't have a solution for it.

82

u/mad_marble_madness 13h ago edited 13h ago

If you think most commercial software is better staffed, then you have never worked for a commercial software company…

6

u/ignatzami 12h ago

Decade at Microsoft. I know.

9

u/ilep 11h ago

And if you think commercial software has poor security you have never seen industrial automation..

14

u/BastetFurry 13h ago

I do, state funded kernel devs.

21

u/dangerbird2 13h ago

The kernel itself is pretty well funded with pretty much every major tech company contributing money and developer talent. The real issue is smaller projects that are integral parts of the linux ecosystem, but not part of the kernel project itself. See the XZ backdoor incident

8

u/captainstormy 13h ago

I was about to say this myself. I'm a software engineer and Linux System Admin. Been working in the industry over 20 years now. I've been paid by a lot of companies (some you know, many you don't) to submit code to the kernel. Sometimes other parts of a Linux system but mostly the Kernel.

3

u/ignatzami 12h ago

This. The left pad problem writ large. The big visible pieces get funded. The solo devs supporting a package with millions of weekly downloads on NPM don’t get much, if anything.

1

u/nox66 1h ago

It's ironic. Major corporations often benefited from open source, including NPM, PyPI, curl, etc., even without contributing back, and somehow this was relatively sustainable until the corporations themselves created the means for it to not be so. Pretty telling of the situation overall.

3

u/happyscrappy 11h ago

Which state?

Maybe Larry Ellison could call up Trump and come up with a list of 100 trustworthy system devs to take over the linux kernel.

Big Balls can be the director.

-5

u/LegitimateCopy7 13h ago

do you want to uhm... think it through? like actually?

2

u/Nemesis_Ghost 13h ago

It's not just the maintainers, it's also the downstream dependents. My company, like many, are dependent on a lot of OSS software packages in our builds. We have scanning software checking our packages for vulnerabilities & then blocking builds if one is found. It has gotten to the point that my teams can't go a single day w/out having spend an hour or more per app updating & retesting due to vulnerabilities.

2

u/baylonedward 12h ago

Isn't parts of EU shifting to Linux? Maybe they can make money grants related to Linux.

2

u/JamesLahey08 9h ago

How often do you donate to Linux developers?

2

u/brakeb 11h ago

the problem is that maintainers are getting fixes for ancient fucking code that is stuck in the kernel that no one uses anymore or hasn't seen a code update in 5+ years and are expected to fix it.

Rip out old drivers, old services, cruft that no one has touched in years. Feature freeze the kernel until Linus and team has a chance to scrub the kernel and remove old shit. if you're still the gray hair that 'needs' your PCMCIA eth card, you figure out how to make your shitty laptop work with it.

1

u/BriefSpecial420 5h ago

And that attackers don't care about the fact how overwhelmed the devs are. Actually, attackers may even embrace that.

1

u/ComradeMatis 4h ago

I also thing it is important to keep in mind this quote from the article:

> However, many of the findings have been low-priority vulnerabilities — often within obscure driver code — questionable patches, and outright hallucinations, leaving human maintainers to separate useful work from noise.

It reminds of the hysterical headline a while ago regarding Firefox only for an article to come out later on, that many ignored, how many of the alleged bugs were either hallucinations or just flat out got it wrong.

1

u/brakeb 8h ago

before you throw 'moar peoplezzzz' at the problem, perhaps removing 30ish years worth of cruft, unsupported code, maintainerless tech, break out the kernel to allow for more modular driver management, etc...

removing esoteric shit for people to find CVEs against will probably go a long way to stop being 'overwhelmed'. THey seem to be embracing AI to assist with this.

-10

u/TFenrir 14h ago

This will necessitate the automation of software delivery. These PRs will soon have to be handled by AI, autonomously.

I think people will get mad at me for saying this, because they don't want it to be true, but I'm not saying it because I think it's fun and cool and sexy, it's just literally what has to happen if you want to have secure software.

We are months away from autonomous AI swarms, frontier and open weight/source, running rampant through the Internet.

We will very soon see official info released about the next waves of models (likely Astra and the 5.1 line of a claude) and I really want to encourage people to pay attention to these and consider the ramifications of their capabilities.

13

u/neppo95 13h ago

How so when the AI will also merge things that create bugs. AI is great but until it doesn’t regularly make mistakes, an autonomous process would do a lot of harm.

7

u/Legionof1 13h ago

Yep, it has to be better than a human when it comes to this stuff and it just isn’t yet, faster, but not more accurate.

-6

u/TFenrir 13h ago

There are already pipelines in world class software organizations that are currently doing this. Using the very best models, sure, but yes this is already happening.

5

u/neppo95 13h ago

Okay? Doesn’t make it any better.

1

u/TFenrir 13h ago

I mean, I'm not sure what you are saying - AI will definitely merge in bugs. But we already have processes for this because we also merge in bugs. The speed and capability of these models are currently at the point where the net benefit is so great that some of the best developers in the world do not look at the code anymore.

What I'm saying is - your concern is already no longer a concern. It in some ways (we always have had to deal with bugs) has never been a concern.

1

u/neppo95 13h ago

I mean, the evidence directly counters your statement. Microsoft has one of those autonomous processes. Guess what? Windows hasn’t been buggier in its existence and is pretty much falling apart. No, bottomline it is net negative still.

4

u/TFenrir 13h ago

... This isn't evidence. This is just you saying things that you want to be true. Where is the evidence for any assertion in this statement of yours?

1

u/neppo95 10h ago

So pure logic is out of the window I see. Company changes way of working drastically. Products suddenly are broken as fuck. Hmm, would there be a link between the two, I wonder. It's allowed to think you know.

2

u/TFenrir 10h ago

LLMs have only been capable enough that anyone would even try to do what I am describing, in the last 6 months dude.

→ More replies (0)

2

u/Expensive_Finger_973 13h ago

So what happens when the AI is finding bugs and exploits in the code base it wrote and it just ends up being a snake eating its own tail?

2

u/TFenrir 13h ago

Software development is already all about finding the bugs you already wrote. Everyone writes bugs and then fixes them.

1

u/Expensive_Finger_973 11h ago

I know, but what happens when the AI is writing all of the code? Perfect code, or endless churn of AI code with bugs and other AI code to patch it and be found to have bugs later on?

1

u/TFenrir 11h ago

I think by the end of the year, the default assumption will be that AI written code will be better than human code, in almost every context. They will autonomously find and fix bugs faster than we will notice, as they work on swarm where each agent works 10x faster than a human.

It's just... Hard for people to wrap their heads around this.

1

u/bensquirrel 10h ago

There are cost barriers to this at the moment. Frontier labs can do this because their token cost is cheap. Open weight models still need to run somewhere so they are not zero cost either.

0

u/DeterminedThrowaway 13h ago

They downvoted him because they hated the truth. I don't see a single viable alternative offered

0

u/Acrobatic-Ice-5877 12h ago

  I don't care about the millions of AI found CVEs. The important line is: Maintainers are overwhelmed.

You should care. Most of the world runs off Linux.

0

u/psioniclizard 11h ago

One of the points of AI is to replace the need for maintainers.

If AI was as amazing as tech bros say, the first thing they will do is remove Linus or just make  tech bro folk of linux and not push anything up stream.

Seeing as everyone other developer apparently can be replaced, linux maintainers will be no different.

0

u/pbrutsche 11h ago

You're assuming that these "millions" of CVE are legit. The maintainers are overwhelmed with just verifying the CVEs.

The companies that Anthropic granted access to Mythos says it has a 30% true positive rate.

3

u/Humpaaa 11h ago

You're assuming that these "millions" of CVE are legit.

I'm not. That's why i say i don't care about them. Most of them will be low quality. But the maintainers still need to verify every single one. That alone binds a lot of ressources.

0

u/LlorchDurden 12h ago

Boss, I'm tired 🫠

0

u/No-Marionberry-772 10h ago

tbh, I'm just happy to have the ability to tell linx elitists to piss off when they argue that Linux is more secure.

it has always been the case that It is less popular, not more secure.

-3

u/vessel_for_the_soul 14h ago

But it worked as intended so far.

9

u/Humpaaa 14h ago

Do you honestly believe so?
A lot of projects are severely underfunded and understaffed.

0

u/SnakeGD09 13h ago

How else are companies supposed to make enough money to exit with stock buybacks?

3

u/dangerbird2 13h ago

It worked so far because before this past year, it took a little more than some rando with free time and a couple hundred bucks for LLM inference to find and exploit these zero days.