Skip to content
City AI Impact Profile

How this is measured

Methodology v0.1 draft.

Status: draft for the Dec 7 demo. Every choice here is meant to be inspectable by a skeptical economist and explainable to an organizer in one sentence.

Unit of analysis

Occupations (SOC 2018, 6-digit) are where exposure is measured. Places get scores by weighting occupation exposure by local employment.

  • Metro level: BLS OEWS metropolitan estimates (occupation employment and wages by MSA). National coverage.
  • City level and equity: Census ACS 5-year, civilian employed population by occupation group, with race and ethnicity iterations (tables C24010B, D, H, I). This section appears only when the pipeline runs with a Census API key; otherwise profiles are metro-level only.
  • Industry context: BLS QCEW county data (Phase 2 for industry exposure via OEWS staffing patterns).

Two exposure dimensions

Cognitive (generative AI) exposure. Two published, task-based occupation measures. Each is standardized (employment-weighted z-score across US occupations), the available scores are averaged, and the average is converted to a national percentile:

  1. Eloundou, Manning, Mishkin and Rock (2023), "GPTs are GPTs": share of an occupation's tasks where an LLM, alone or with tools, could cut completion time by half or more (β measure).
  2. Felten, Raj and Seamans (2021), AI Occupational Exposure (AIOE): links AI application progress to O*NET abilities.

Context only, not in any score: the Anthropic Economic Index, which reports how people in each occupation use Claude.ai (global conversations, not local), including whether conversations are collaborative or delegated. It describes how a tool is used today. It says nothing about local adoption or about jobs.

Physical (robotics and autonomy) exposure. Weighted blend of two standardized measures, converted to a national percentile:

  1. Webb (2020), "The Impact of Artificial Intelligence on the Labor Market": overlap between occupation tasks and the text of robot patents (one third of the weight).
  2. A routine-manual index built from O*NET Work Activities and Work Context (two thirds): handling and moving objects, controlling machines, operating vehicles and equipment, and equipment-paced work, discounted when the job has above-average caring, repair, public-facing or self-directed work, which robots handle worst.

The weights and the form of the discount were chosen on face validity against a published list of check occupations; the details, the checks that still fail, and the known weaknesses are in pipeline/src/hf_pipeline/exposure/PHYSICAL_INDEX.md. Keeping it separate matters: agriculture, logistics and manufacturing regions can look quiet on generative AI measures and loud on robotics and autonomy.

Percentiles and "high exposure"

Percentiles are employment-weighted across the US: a cognitive percentile of 80 means the occupation is more exposed than the jobs held by 80% of US workers. An occupation is high exposure on a dimension at percentile ≥ 75.

Place indexes:

  • cognitive_exposure_index, physical_exposure_index: employment-weighted mean percentile of the local workforce (US average is 50 by construction).
  • high_cognitive_share, high_physical_share: share of local workers in high-exposure occupations (US average is about 25%).
  • wages_in_high_exposure_usd: annual wage bill of local workers in occupations high on either dimension.
  • Where the metro falls among all metros on each index, reported in fifths. Exact ranks are in the data but not the headline, because most metros sit within a few points of each other.

What this does not say

Exposure is the share of the work AI or automation could touch. It does not predict layoffs. Exposed work can be augmented (people do more, better) or automated (fewer people needed), and which one happens depends on employers, policy, bargaining and community choices. That gap is the reason organizing matters, and the brief says so plainly.

Missing scores

When an occupation lacks a measure, it takes the average of its SOC broad group, then its minor group. OEWS also reports some occupations only as combined codes (for example 31-1120, Home Health and Personal Care Aides), which take the average of their component occupations. Together these cover about 8% of US employment, including two of the largest occupations, and every imputed value is flagged in the data.

Known limits

  • OEWS leaves out self-employed workers, private household workers and most farms that hire directly. Farm-heavy metros are undercounted, which matters for agricultural regions like the Central Valley. Profiles flag this where farming is a notable share of covered jobs.
  • OEWS suppresses small cells; suppressed employment is excluded and the excluded share is reported.
  • Metro scores sit close together: most metros fall between 45 and 55, and the two measures move in opposite directions across metros because they reflect white-collar versus blue-collar job mixes. Profiles use fifths of metros rather than exact ranks, so small gaps are not read as large ones.
  • Exposure measures are built on 2023-era model capabilities and O*NET task descriptions.
  • Metro boundaries mix urban and rural economies. City-level ACS uses broad occupation groups, so it is coarser.
  • Local signals are curated from public reporting and are only as complete as that reporting.