While working late in a lab at the Massachusetts Institute of Technology, graduate student Joy Buolamwini encountered a bizarre and disturbing technical obstacle.
She was developing an art installation called the “Aspire Mirror,” an interactive project designed to track a viewer face using a camera and superimpose digital imagery onto their reflection. But no matter how she adjusted the lighting or repositioned the camera, the computer vision software repeatedly failed to detect her face.
Then, Buolamwini tried a bizarre experiment: she slipped a stark white plastic Halloween mask over her face.
Instantly, the software detected the mask and began tracking her movements perfectly. When she took the mask off, the system once again registered nothing.
That single moment—coding in “whiteface” at MIT just to be recognized by a machine—shattered the widespread myth that computer algorithms are inherently neutral and objective. It propelled Buolamwini, a computer scientist, Rhodes Scholar, and self-described “Poet of Code,” on a relentless mission to unmask the systematic racial and gender biases embedded in global artificial intelligence systems.
The discovery of the “Coded Gaze”
Buolamwini realized that the computer vision system failure was not a temporary glitch or an isolated hardware error. It was a direct reflection of a fundamental flaw in how machine learning systems are designed and trained.
Artificial intelligence models learn to recognize human features by analyzing massive training datasets containing thousands of facial images. If those datasets are overwhelmingly composed of lighter skinned, male individuals, the algorithm learns that a “standard human face” fits those specific visual parameters.
Buolamwini coined a term for this technological blind spot: The Coded Gaze.
Much like the concepts of the “male gaze” or “white gaze” in film and literature, the coded gaze describes how the conscious and unconscious biases, priorities, and demographics of software developers are encoded directly into the architecture of automated systems.
When technology companies train algorithms on narrow, unrepresentative samples of humanity, those algorithms inherit and amplify societal exclusions under the guise of computational mathematical objectivity.
How the Coded Gaze Operates in Machine Learning
------------------------------------------------
1. Homogeneous Training Data --> Datasets dominated by lighter-skinned male faces
2. Skewed Model Learning --> System defines "human face" based on narrow training samples
3. Algorithmic Misclassification --> High error rates on darker-skinned and female subjects
4. Real-World Harm --> Unfair surveillance, biased hiring, and wrongful arrests
Landmark research: The Gender Shades project
Determined to measure the full scope of the problem scientifically, Buolamwini joined forces with researcher Timnit Gebru to conduct a rigorous audit of commercial facial analysis algorithms.
Published in twenty eighteen, their groundbreaking study—titled Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification—evaluated commercial facial analysis tools produced by major tech companies, including IBM, Microsoft, and Face++.
To test these systems fairly, Buolamwini created the Pilot Parliament Benchmark, a meticulously curated dataset containing over twelve hundred faces of parliamentarians from African and European nations, structured using the Fitzpatrick Skin Type scale to ensure balanced representation across gender and skin tones.
The empirical results of the study were staggering:
- Lighter-Skinned Males: The commercial algorithms performed almost flawlessly, achieving gender classification error rates under one percent.
- Lighter-Skinned Females: Systems performed relatively well, with error rates remaining under seven percent.
- Darker-Skinned Males: Experienced error rates rising to roughly twelve percent.
- Darker-Skinned Females: The algorithms failed catastrophically, exhibiting error rates as high as thirty-four to thirty-seven percent in aggregate, with error rates spiking up to forty-seven percent for women with the darkest skin tones.
The systems were barely performing better than a random coin toss when trying to classify the gender of darker skinned women. When Buolamwini tested the systems on iconic figures like Michelle Obama, Serena Williams, and Oprah Winfrey, the software routinely misgendered them or failed to detect them entirely.
The Algorithmic Justice League: Art meets activism
Recognizing that scientific papers alone are rarely enough to compel corporate tech monopolies to change, Buolamwini founded the Algorithmic Justice League (AJL).
The AJL was established as an organization situated at the intersection of computer science, public policy, and creative expression, dedicated to exposing algorithmic harms, advocating for public accountability, and protecting civil rights in the age of artificial intelligence.
Buolamwini brought her artistic voice into the fight, using spoken word poetry, visual media, and documentary film to communicate complex technical concepts to the public:
- AI, Ain’t I A Woman?: A striking spoken-word visual audit set to poetry that highlighted commercial AI failures on dark-skinned iconic women, drawing a historical line from Sojourner Truth famous speech to modern algorithmic discrimination.
- Coded Bias: A acclaimed feature documentary following Buolamwini research journey, bringing the dangers of invasive facial recognition surveillance to mainstream global audiences.
- Congressional Testimony: Buolamwini took her findings directly to lawmakers, testifying before the United States Congress regarding the civil rights risks of law enforcement agencies using untested, biased facial recognition systems in public surveillance.
Corporate reckoning and real-world policy impact
The impact of Buolamwini research was swift and transformative. By publicly naming companies and demonstrating systematic racial and gender disparities in commercial software, her work created a wave of corporate accountability:
- Immediate System Upgrades: Within months of the Gender Shades publication, both IBM and Microsoft overhauled their training datasets and released updated facial analysis software that significantly reduced error rates across darker skin tones.
- Moratoriums on Police Surveillance: Following the public outcry surrounding algorithmic bias and police accountability in twenty twenty, major technology companies—including Amazon, IBM, and Microsoft—announced moratoria or outright halts on selling their facial recognition software to law enforcement agencies.
- Legislative Bans: Cities across the United States, including San Francisco, Boston, and Cambridge, enacted municipal bans on government and police use of facial recognition technology, directly citing Buolamwini research on accuracy disparities and civil rights risks.
Through the Safe Face Pledge, an initiative launched by the AJL in partnership with the Georgetown Law Center on Privacy & Technology, Buolamwini established clear ethical boundaries, calling on tech companies to commit never to supply facial recognition technology for lethal autonomous weapons or mass law enforcement surveillance without oversight.
Conclusion and legacy
Dr. Joy Buolamwini work fundamentally reframed the global conversation surrounding artificial intelligence. She proved that computer code is not an impartial arbiter of truth, but a reflection of the human choices, social structures, and historical biases of its creators.
As the “Poet of Code,” she demonstrated that technical mastery and artistic activism can work together to protect human rights. By forcing the world’s largest tech corporations to confront the coded gaze, Buolamwini ensured that as artificial intelligence shapes the future of society, it must be held accountable to compassion, fairness, and full spectrum human inclusion.
