The Data That Doesn’t Exist: Algorithmic Bias and Disability Exclusion
Algorithmic systems are shaping decisions in hiring, healthcare, education and public services, yet they often rely on data that fails to reflect the full diversity of human experience. This gap creates space for Algorithmic Bias and Disability Exclusion, where people with disabilities are overlooked because the systems meant to serve them were never designed with them in mind. The article highlights how missing data becomes a silent barrier, reinforcing inequity and urging a shift toward inclusive digital design.
(ChatGPT generated image)
Artificial intelligence is often described as efficient, neutral, and innovative. Yet its recurring failures around disability point to a deeper structural problem. The information required to treat persons with disabilities fairly has rarely been collected, and when it exists, it is usually incomplete or disconnected from lived experience. This is not simply a technical mistake. It reflects a long history in which the perspectives of persons with disabilities were excluded from the design of systems that now influence every part of life.
Much of the data currently available comes from medical or administrative categories. These categories may document diagnoses, but they do not capture environmental barriers, communication preferences, evolving conditions, or the social and cultural contexts that shape daily life. Many persons live with experiences that fluctuate, are undocumented, or cannot be easily classified. As a result, their realities are virtually absent from the datasets that determine how modern AI behaves.
Why Algorithms Struggle With What They Cannot See
AI systems depend on patterns within the data they are given. When persons with disabilities are missing or misrepresented in that data, the system treats their experiences as unusual or incorrect. The consequences unfold in ways that may be subtle but deeply impactful. Hiring platforms may misread communication styles. Healthcare algorithms may underestimate risks or misinterpret symptoms. Recognition systems may fail to understand diverse expressions, movements, or interaction styles.
Such failures are not random. They arise because the systems were never designed to account for experiences that the data did not contain. The problem is not the presence of disability but rather the absence of meaningful representation in the information that trains these systems.
Why “More Data” Is Not Enough
A common suggestion for fixing bias is to collect more data. Within the context of disability, this approach is not sufficient and may even be harmful. Many experiences cannot be neatly captured through standard data collection. Disclosure is deeply personal and often shaped by stigma, privacy concerns, or inaccessible systems. Forcing additional surveillance or more exhaustive categorization risks repeating the same patterns of exclusion that contributed to the problem.
Real progress requires methods grounded in dignity and participation. The Convention on the Rights of Persons with Disabilities provides guidance on how to move forward responsibly. Article 9 emphasizes the right to access digital and physical environments, including technology. Article 21 affirms the right to communication and access to information in formats that respect individual needs. Together they outline a rights-based path toward inclusive technology that does not simply seek more data but instead seeks meaningful and equitable participation.
Embedding Rights Into Inclusive Technology
The commitments outlined in Articles 9 and 21 offer a clear foundation for creating AI systems that respect the experiences of persons with disabilities. Accessibility becomes a core requirement rather than an afterthought. Interfaces, platforms, and outputs need to be usable with assistive technologies and available in formats that reflect the wide variety of ways people communicate and process information. These requirements help ensure that persons with disabilities can genuinely participate in digital spaces and influence the systems that affect them.
Meaningful participation also requires transparency. Persons must be able to understand what is collected about them, how that information is interpreted, and how it shapes algorithmic outcomes. When this level of clarity is provided, individuals gain the power to make informed decisions about their involvement. This approach aligns with the CRPD’s emphasis on autonomy and full access to information.
Capacity Building as a Rights-Based Approach
Building digital literacy and confidence among persons with disabilities is essential for the development of fair and representative data. Capacity building supports individuals in understanding how algorithms function, how their information flows through digital systems, and what rights they hold within these systems. With greater knowledge, persons can make informed choices about consent, identify inaccessible or exclusionary practices, and influence how AI tools represent their experiences.
When persons with disabilities are equipped to participate actively in digital ecosystems, data becomes richer and more aligned with real-world diversity. This shift transforms the relationship between technology and its users from one of passive data extraction to one of collaboration and agency.
Open-Source Systems Strengthen Autonomy and Trust
Open-source technologies offer a pathway toward more equitable AI. Unlike proprietary systems, open-source platforms can be inspected, modified, and adapted by communities. Persons with disabilities, alongside developers and advocates, can examine how systems function, identify accessibility barriers, and create solutions that meet the required needs.
This openness strengthens trust because individuals can verify how their information is collected and processed. Trust encourages safe and voluntary participation. Over time, systems begin to learn from data that is genuine, contextual, and provided freely rather than under pressure. This approach aligns closely with the rights to access and information outlined in Articles 9 and 21, and it supports innovation that is more responsive and inclusive.
AI That Learns with People Instead of Learning About Them
Capacity building and open-source development create conditions in which data emerges naturally through ethical, voluntary, and informed participation. Algorithms gain insights that reflect lived experience instead of imposing narrow definitions of normality. Persons with disabilities become co-creators of digital environments that respect autonomy, protect rights, and embrace diversity.
Achieving inclusive AI requires more than technical adjustments. It calls for a commitment to rights, agency, and social equity. Grounding AI design in the principles of the CRPD ensures that the data that once did not exist begins to appear in ways that honour privacy, participation, and dignity. This evolving landscape opens the door to technologies that reflect the full richness of human experience and foster fairer and inclusive futures.
Article by: Brian Ndiritu