✓ProveIt
For open-source maintainers

Convincing report.
Does the code agree?

ProveIt checks security bug reports against the source code, so you can see which claims hold up and where to investigate.

See a report checked Check your own

Built with NVIDIA Nemotron on Nebius.
Try the saved examples instantly. No sign-in needed.

REPORT #2887487Saved analysis
The report claims

A buffer overflow.
No size check before the copy.

The actual curl code lib/inet_ntop.c
75  len = strlen(tmp);
76  if(len == 0 || len >= size) {
77    errno = ENOSPC;
78    return (NULL);
79  }
80  strcpy(dst, tmp);
81  return dst;
The check is right there.The function rejects a buffer that is too small.

Check the claim. Read the code behind it.

Follow the evidence. Jump to the cited lines.

Make the call. Review a draft reply.

See it in action

Three reports. Three different answers.

Real reports · saved runs
01 The false alarm

The report missed a size check.

The real function checks the size and returns before the copy if the buffer is too small.

Explore the evidence 02 The real bug

This one really crashes curl.

ProveIt found the recursive cleanup path. A separate offline run in a Nebius sandbox caught a stack overflow inside curl.

Explore the evidence 03 The AI's own mistake

The AI believed a made-up quote.

The model accepted a documentation quote that the fetched source did not contain. The citation check withdrew its judgment and asked for human review.

Explore the evidence
How it works

From a report to
something you can check.

Currently built and evaluated for curl.

  1. Find the relevant code

    Check the report's references against the curl version it targets.

  2. Test each claim against it

    Nemotron reads the source and explains what supports or contradicts the report.

  3. Check the AI's evidence, too

    Verify its citations. If the evidence falls short, ask for human review. Selected proof-of-concept reports can also run offline in a Nebius sandbox.

An honest first test

Useful evidence. Still a human decision.

Results & limitations →

77 reports held back during development: 18 labeled AI slop by curl's maintainer, 59 linked to real vulnerabilities.

5 / 18slop reports flagged for lower priority
22 / 59real bugs supported by cited code
2 / 59real bugs wrongly flagged for lower priority

Most other reports need more investigation or a policy decision. ProveIt never closes a report.

Browse all 197 reports Search the full curl collection
Explore the record

Find a report

Compare ProveIt's saved analysis with curl's public outcome. Those outcomes were hidden from the pipeline.

All reports Labeled AI slop Known vulnerabilities
Filter by ProveIt's finding
Any finding References do not check outCode contradicts the claimNeeds a policy reviewMore detail neededNeeds human reviewCode supports the claim