# CVE Vulnerability Research Pipeline
PoC-confirmed CVEs are automatically collected daily, and then **split into two paths**.
```
Collection(auto) ─ Filtering(auto) ─┬─ CTF problem creation material data/study/, data/high-severity.md
└─ Patch analysis → Bypass discovery → Vendor disclosure
data/analysis.json, reports/
```
Both are first-class objectives. Collection is the input stage common to both paths, and in fact, 3 out of 5 reports sent so far have come from this archive.
- Data sources: [NVD](https://nvd.nist.gov/), [CISA KEV](https://www.cisa.gov/known-exploited-vulnerabilities-catalog), [GitHub Advisory Database](https://github.com/advisories), [GitHub repository search](https://github.com/search)
- Collection cycle: Once daily, `0 3 * * *` UTC (GitHub Actions `.github/workflows/collect.yml`)
- Filter: Only save CVEs with CVSS ≥ 6.0 and confirmed PoC (regardless of publication date — old CVEs are collected if PoC emerges today)
- Format: Markdown file per CVE (`data/<year>/<CVE-ID>.md`)
- Study full list: Keep all archive entries with CVSS ≥ 8.0 always up-to-date in `data/high-severity.md`
- Study daily distribution: 5 CVEs per day without duplication to `data/study/<CVE-ID>.md`. Today's new promotions → existing archive → scan entire NVD database in descending CVSS order for those with PoC
> This file is auto-generated/overwritten when `scripts/collect.py` runs. Do not edit directly.
## How it works
1. Add newly published CVEs on NVD (CVSS ≥ 6.0) to candidate pool (`data/.pool.json`)
1b. Also add previously published CVEs in high CVSS order (CRITICAL→HIGH→MEDIUM buckets) 5 per day to pool (backfill regardless of publication date, cursor saved in state file)
2. Cheaply detect candidates with newly attached Exploit reference tags via NVD lastModified bulk query
3. For candidates missed above, recheck at scheduled times (3/7/14/30/60/90/180/365 days after publication) using GitHub repository search
4. If PoC is confirmed, enrich with CISA KEV/GHSA information and save to `data/<year>/<CVE-ID>.md`, remove from pool
5. If schedule completes without PoC confirmation, give up and remove from pool (does not remain in archive)
6. Regenerate `data/high-severity.md` by filtering only CVSS ≥ 8.0 from entire archive each time
6b. Incrementally query GHSA with updatedSince, add **only those with patch commits** to analysis queue (`data/analysis.json`) (not added to candidate pool — GHSA lacks PoC information)
7. Daily distribution for study → `data/study/<CVE-ID>.md`. Priority 1: today's new promotions, Priority 2: existing archive,
Priority 3: scan entire NVD database in descending CVSS order for those with PoC (regardless of publication date, continue via cursor)
## Local execution
```bash
pip install -r requirements.txt
cd scripts
python collect.py # Incremental collection (based on state file)
python collect.py --days 3 # Force recent 3 days of new CVEs into pool (backfill)
python collect.py --verify # Archive PoC verification
```