A vulnerability report written by an AI tool can be useful, detailed and wrong in a way that is not obvious at first glance. It may include a proposed patch, a severity label and steps that look reproducible. Those are inputs for a maintainer to check, not proof that a security boundary can be crossed.
That distinction matters as more scanning tools send findings directly to open-source projects. Anthropic says its opt-in OSS Scanner sends model-generated reports without human review; it may include a reproducer and proposed patch, but the project explicitly warns that reports can be inaccurate. A recent VulnCheck analysis of Anthropic's Project Glasswing results put this maintainer-triage question in focus. Its assessment concerns one evolving set of reports, though, and should not be treated as a universal accuracy rate for AI security tools.
Source: VulnCheck: November 9 analysis of Project Glasswing results
Keep a possible vulnerability private first
If a report could expose users before a fix exists, start with the repository's security policy. Check `SECURITY.md` and the repository's security page for the preferred channel. On GitHub, a public repository can enable private vulnerability reporting; if it has, anyone can send a report through a private advisory. If that channel is unavailable, follow the maintainer's security policy or ask for a contact without putting exploit details in a public issue.
This is project-specific. For example, curl's current policy keeps high- and critical-severity reports private while it prepares a fix, and handles low- and medium-severity reports publicly. Its rules are an example of one maintainer's disclosure process, not a template every repository should copy. Do not make a potentially exploitable finding public just because another project accepts some reports in public.
Source: GitHub Docs: privately reporting a security vulnerability
Source: curl's vulnerability disclosure policy
Reproduce the report against the code it names
Record the repository, exact commit or release, build options, dependencies and configuration in the report. Then check whether the alleged behavior occurs on that version and whether it still occurs on the current supported release. Many apparent findings are tied to a test build, an obsolete path, a disabled feature or an assumption that the project does not make.
If you run a supplied reproducer or proposed patch, use a disposable environment with no production credentials or sensitive data. Treat it as untrusted code: inspect what it does, limit access to your machine and network, and avoid running it against real users or services. A scanner's own sandbox does not make a copy of its reproducer safe to execute on a maintainer's workstation.
Ask for a small, repeatable case that shows the behavior, the expected result and the actual result. For a security claim, the demonstration should identify what an attacker controls and what protected resource or action becomes reachable. If the report cannot show that path, ask a focused question rather than accepting its severity label.
Source: Anthropic OSS Scanner repository: report and build behavior
Separate a software defect from a security impact
A crash, incorrect response or awkward edge case can be a real bug without being an exploitable vulnerability. Write down the trust boundary: who can trigger the behavior, what permissions they have, what configuration is required and what could be exposed, changed or made unavailable. Compare that with the project's threat model and security policy.
Rate the impact using the project's own criteria, not only a scanner's number or adjective. Curl says its assessment considers the attack vector and complexity, required privileges, build configuration, affected protocols and likely effects on confidentiality, integrity or availability. Your project may weigh those factors differently; the useful habit is to explain the conditions behind the rating.
Check for duplicates, fixes and affected versions
Search the issue tracker, pull requests, release notes and existing advisories for the same code path. A finding may describe an issue already reported privately, an older bug already fixed, or a known limitation rather than a new weakness. Record whether it is a duplicate, a non-security bug, not reproducible or still uncertain; those outcomes should not be collapsed into a single false-positive count.
Confirm the affected and fixed versions before applying a suggested patch. Inspect the change yourself, keep it separate from unrelated cleanup, and add a regression test that fails on the vulnerable behavior and passes after the fix. Review the patch for new paths around the same boundary instead of assuming that a plausible model-written diff closes the problem.
Source: GitHub Docs: managing privately reported vulnerabilities
Close the loop with a clear decision
GitHub's private-report workflow lets maintainers accept a report into a draft advisory, ask the reporter for more information, or close it with an explanation when it is not a security risk. Use a similar clear outcome even when a project relies on another channel. Tell the reporter what you verified, what remains unknown, and whether you need a version, configuration or reproduction detail. Keep the private discussion private until disclosure is coordinated.
Source: GitHub Docs: reviewing and managing private reports
Source: Anthropic: opt-in vulnerability finding for open-source software
A short triage record can save the next maintainer from repeating the work: source and date, affected commit, reproduction result, security impact, duplicate check, decision and disclosure status. It also gives a project evidence to improve its threat model or scanner settings. The goal is neither to accept every AI report nor dismiss them all; it is to make each security decision from reproducible behavior and the project's real users and boundaries.
This guide covers incoming findings for people maintaining software. For the separate risk of an agent changing your own project files, see our guide to protecting an app while using an AI coding agent.