The problem
The client wanted a defensible dataset on how the largest companies adopt AI, but the signal was scattered across filings, press, and job posts, in formats nothing could ingest.
What we built, and why
- Built the pipeline in stages (collect, normalize, classify, document), each independently re-runnable, so output traces back to its source.
- Treated provenance as a first-class field: every data point carries where it came from, so the dataset stands up to scrutiny.
- Wrote the documentation alongside the data, not after, so the deliverable is usable by someone who wasn't in the room.
The result
A structured, documented dataset covering the Fortune Global 500, delivered with the pipeline that produces it, so it can be refreshed rather than handed over once. (Final figures confirmed with the client.)
Stack
Python · Playwright · pandas · structured outputs