# CVE Vulnerability Research Pipeline
PoCs confirmed in real CVEs are automatically collected daily, and then **split into two paths**.
```
Collection(auto) ─ Filtering(auto) ─┬─ CTF problem creation materials data/study/, data/high-severity.md
└─ Patch analysis → Bypass discovery → Vendor disclosure
data/analysis.json, reports/
```
Both are primary objectives. Collection is the common input stage for both paths, and in fact, 3 out of 5 disclosures sent so far have come from this archive.
- Data sources: [NVD](https://nvd.nist.gov/), [CISA KEV](https://www.cisa.gov/known-exploited-vulnerabilities-catalog), [GitHub Advisory Database](https://github.com/advisories), [GitHub repository search](https://github.com/search)
- Collection cycle: Once daily, `0 3 * * *` UTC (GitHub Actions `.github/workflows/collect.yml`)
- Filter: Only CVSS ≥ 6.0 with confirmed PoC (publication date irrelevant — older CVEs collected if PoC appears today)
- Format: Markdown file per CVE (`data/<year>/<CVE-ID>.md`)
- Study full list: Always maintain CVSS ≥ 8.0 archive items in `data/high-severity.md`
- Study daily distribution: 5 items per day without duplication to `data/study/<CVE-ID>.md`. Today's new promotions → existing archive → NVD full DB sorted by descending CVSS, filled with those having PoC
> This file is auto-generated/overwritten when `scripts/collect.py` runs. Do not edit directly.
## How It Works
1. Add newly published CVEs (CVSS ≥ 6.0) from NVD to candidate pool (`data/.pool.json`)
1b. Also add previously published CVEs by descending CVSS (CRITICAL→HIGH→MEDIUM buckets) at 5 per day to pool (backfill regardless of publication date; cursor saved in state file)
2. Detect candidates with newly attached Exploit reference tags via NVD lastModified bulk query (cost-efficient)
3. For candidates missed above, recheck those reaching review timpoints (3/7/14/30/60/90/180/365 days after publication) via GitHub repository search
4. If PoC confirmed, save to `data/<year>/<CVE-ID>.md` with CISA KEV/GHSA enrichment and remove from pool
5. If PoC not found after full schedule, abandon and remove from pool (not retained in archive)
6. Regenerate `data/high-severity.md` by selecting only CVSS ≥ 8.0 from full archive each time
6b. Incrementally query GHSA via updatedSince, ingest **only those with patch commits** to analysis queue (`data/analysis.json`) (do not add to candidate pool — GHSA lacks PoC info)
7. Study daily distribution → `data/study/<CVE-ID>.md`. Priority 1: today's new promotions, Priority 2: existing archive, Priority 3: search NVD full DB by descending CVSS for those with PoC (publication date irrelevant, continue via cursor)
## Local Execution
```bash
pip install -r requirements.txt
cd scripts
python collect.py # Incremental collection (based on state file)
python collect.py --days 3 # Force recent 3 days of new CVEs (backfill)
python collect.py --verify # Verify archive PoC
```