# CVE Vulnerability Research Pipeline
PoC-confirmed CVEs are collected automatically every day, and the pipeline **branches into two paths**.
```
Collection(automatic) ─ Filtering(automatic) ─┬─ CTF Problem Creation Material data/study/, data/high-severity.md
└─ Patch Analysis → Bypass Discovery → Vendor Disclosure
data/analysis.json, reports/
```
Both are primary objectives. Collection is the common input stage for both paths, and in fact, 3 out of 5 reports sent so far have come from this archive.
- Data sources: [NVD](https://nvd.nist.gov/), [CISA KEV](https://www.cisa.gov/known-exploited-vulnerabilities-catalog), [GitHub Advisory Database](https://github.com/advisories), [GitHub Repository Search](https://github.com/search)
- Collection frequency: Once daily, `0 3 * * *` UTC (GitHub Actions `.github/workflows/collect.yml`)
- Filter: Only save CVSS ≥ 6.0 with confirmed PoC (publication date irrelevant — older CVEs are collected if PoC appears today)
- Format: Markdown file per CVE (`data/<year>/<CVE-ID>.md`)
- Study full list: Always maintain CVSS ≥ 8.0 archive items in `data/high-severity.md`
- Daily study distribution: 5 items per day without duplication in `data/study/<CVE-ID>.md`. Today's newly promoted items → existing archive → scan entire NVD DB in descending CVSS order with PoC, fill in order
> This file is auto-generated/overwritten when `scripts/collect.py` runs. Do not edit directly.
## How It Works
1. Add newly published CVEs (CVSS ≥ 6.0) from NVD to candidate pool (`data/.pool.json`)
1b. Also add previously published CVEs 5 per day by highest CVSS (CRITICAL→HIGH→MEDIUM bucket order) to pool (backfill regardless of publication date; cursor saved in state file)
2. Cheaply detect candidates with newly added Exploit reference tags via NVD lastModified bulk query
3. For candidates not caught above, re-check at scheduled intervals (3/7/14/30/60/90/180/365 days after publication) using GitHub repository search
4. When PoC is confirmed, save to `data/<year>/<CVE-ID>.md` with enriched CISA KEV/GHSA info and remove from pool
5. After finishing the schedule, if PoC is not found, give up and remove from pool (not retained in archive)
6. Regenerate `data/high-severity.md` by filtering CVSS ≥ 8.0 from entire archive each time
6b. Query GHSA incrementally with updatedSince to ingest **only those with patch commits** into analysis queue (`data/analysis.json`) (not added to candidate pool — GHSA lacks PoC info)
7. Daily 5-item study distribution → `data/study/<CVE-ID>.md`. Priority 1: today's newly promoted items, Priority 2: existing archive,
Priority 3: scan entire NVD DB in descending CVSS order for items with PoC (publication date irrelevant; continue with cursor)
## Local Execution
```bash
pip install -r requirements.txt
cd scripts
python collect.py # Incremental collection (based on state file)
python collect.py --days 3 # Force ingest newly published CVEs from last 3 days (backfill)
python collect.py --verify # Archive PoC verification
```