Healthcare AI Safety Gap · Don't break patients
    Peer-reviewed / official25 Oct 2019·US

    Bias in an algorithm covering 200M+ people

    Science (Obermeyer): a widely used tool systematically under-referred Black patients.

    200M+
    people affected by the algorithm

    What the research found

    A landmark study found a risk-prediction algorithm applied to more than 200 million people a year systematically under-referred Black patients, because it used healthcare cost as a proxy for need. Correcting the bias would have raised the share of Black patients flagged for extra care from 17.7% to 46.5%.

    Why it matters for providers

    A 'neutral' metric baked in racial bias at national scale — the canonical warning that every deployed model needs auditing for who it leaves behind.

    Original source
    Science (Obermeyer et al.)
    Read the full report ↗
    Peer-reviewed or official source — the most reliable tier.