curl is facing a new AI problem
Bug reports written by LLMs are coming in big waves, and they can't just be dismissed as slop anymore.
Daniel Stenberg, founder and lead developer of the hyper-critical open source web tool curl, made a blog post today explaining that the project is facing unprecedented numbers of bug reports that look relatively reliable. He attributes them to LLMs, a source he previously attributed to a mass of low quality reports.
Happening now
In short: The increase in bug reports the curl team has assessed as valid, which include security vulnerabilities, is creating a potentially overwhelming amount of work. Stenberg calls it "high-quality chaos," and he's concerned for the future of curl and projects like it that rely on volunteer contributors who already face taxing workloads.
The background: Earlier this year, Stenberg and the curl team decided to close the project's bug bounty program because low-quality "slop reports" from generative AI tools were being spammed at the program. Now Stenberg says, "The slop situation is not a problem anymore."
High quality reports can't simply be ignored. In the blog post, Stenberg wrote:
This avalanche is going to make maintainer overload even worse. Some projects will have a hard time to handle this kind of backlog expansion without any added maintainers to help.
Zooming out
Why this announcement matters: Open source projects have seen their operations upturned thanks to LLMs in several ways, and this way is a new one. The fact it's affecting one of the most infrastructure-critical projects out there raises the stakes.
Lingering questions: Do the extant bug reports actually cover all the important security issues? As Stenberg points out, bad actors can also find those same and possibly other bugs and take advantage of them rather than report them.
- Also, will this initial wave of reports taper off as all the major issues are quickly found and patched?
- As Stenberg says, "Someone has suggested it might work as with fuzzing, that we will see a plateau within a few years. I suppose we just have to see how it goes."
My take: I'm not a developer, and I can't independently verify that these reports as high-quality Sternberg's team claims. Regardless, they're creating a problem that affects me as a person on the internet.
