AI Regulation and Disability Discrimination | DISABLED ENTREPRENEUR

Written by Andrew Jones

Disclaimer: This article is for general information and educational purposes only and does not constitute legal advice. Artificial intelligence regulation is developing rapidly, and the application of the EU Artificial Intelligence Act is being introduced in stages. The Act was also amended in July 2026 by the Digital Omnibus on AI. Individuals and organisations should obtain advice from a suitably qualified legal, equality or data protection professional regarding their particular circumstances.

When Automated Systems Become a Digital Barrier

The EU AI Act represents an important attempt to control discriminatory and high-risk artificial intelligence, but genuine disability equality will require accessible systems, representative data, meaningful human oversight and effective rights of challenge.

Artificial intelligence is increasingly being used to screen job applicants, assess employee performance, determine eligibility for services, calculate insurance risk, evaluate students, prioritise medical care and detect suspected fraud.

These systems are often promoted as efficient, objective and impartial. However, an algorithm is not automatically fair simply because it is operated by a computer.

Artificial intelligence is designed, trained and deployed by people. It learns from data produced by societies and institutions in which discrimination, exclusion and inequality may already exist. When those patterns are reproduced by an automated system, discrimination can become faster, less visible and more difficult to challenge.

For disabled people, the consequences can be particularly serious. An inaccessible application portal may prevent someone from applying for employment. A recruitment algorithm may interpret an employment gap caused by illness as evidence of unreliability. Facial analysis software may misinterpret involuntary movements, facial differences or lack of eye contact. Automated benefit systems may fail to understand fluctuating conditions, while speech-recognition technology may disadvantage people with speech impairments.

The danger is not limited to an obviously discriminatory computer program. Discrimination can also arise when a system has been designed around a narrow idea of what a “normal” person looks, sounds or behaves like.

What Is Algorithmic Disability Discrimination?

Algorithmic discrimination occurs when an automated system produces, contributes to or reinforces an unfair outcome for a person or group.

This may involve direct discrimination, where disability is expressly used to disadvantage someone. More commonly, it may involve indirect or proxy discrimination.

Proxy discrimination occurs when the system uses information that appears neutral but is closely connected to disability.

Examples may include:

Periods of unemployment or interrupted education.
The number of medical appointments a person attends.
Typing speed or the time taken to complete a test.
Voice patterns, eye movement or facial expression.
Patterns of absence from work.
Use of assistive technology.
The ability to respond quickly to online questions.
Credit history affected by disability-related poverty.
Postcode, housing status or reliance on social security.
Previous insurance or healthcare information.

A system may not contain a field labelled “disability”, but it may still infer disability or penalise characteristics associated with it.

Discrimination can also arise from incomplete or unrepresentative data. Where disabled people were excluded from historical employment, education, clinical research or financial services, the data used to train an AI system may treat their circumstances as unusual, undesirable or statistically risky.

The EU Artificial Intelligence Act

The EU Artificial Intelligence Act, Regulation (EU) 2024/1689, establishes a risk-based regulatory framework for the development and use of artificial intelligence.

The Act applies not only to organisations established within the European Union. It can also apply to providers or deployers established in another country where an AI system is placed on the EU market or where its output is used within the European Union. This means that some UK businesses may fall within its territorial scope when supplying AI products or services to EU customers.

The Act divides AI uses into different levels of risk. Some practices are prohibited, some are classified as high-risk and subject to extensive controls, while lower-risk systems may principally be governed by transparency requirements and existing legislation.

The Implementation Position on 2 August 2026

The AI Act entered into force on 1 August 2024, with its provisions scheduled to apply in stages. Certain prohibited AI practices and initial provisions began applying on 2 February 2025.

From 2 August 2026, the European Commission’s AI Office and national authorities begin enforcing further provisions of the Act. Transparency requirements also begin applying to certain interactive and generative systems. For example, people should be informed when they are interacting directly with an AI system rather than a human, while deepfakes and certain AI-generated or manipulated content must be labelled or made detectable.

However, it would be incorrect to suggest that all high-risk AI obligations are now fully operational.

Regulation (EU) 2026/1744, known as the Digital Omnibus on AI, entered into force on 27 July 2026 and extended parts of the timetable. The principal rules for high-risk systems listed in Annex III, including systems used in employment, education, access to essential services, biometrics, migration and law enforcement, are now due to apply from 2 December 2027. High-risk AI embedded in regulated physical products is generally scheduled for 2 August 2028.

Existing equality, employment, consumer and data protection laws continue to apply during the transitional period. Organisations should not assume that postponement of an AI-specific obligation gives them permission to discriminate.

Prohibited Exploitation of Disability

Article 5 of the AI Act prohibits certain practices considered incompatible with fundamental rights.

Of particular relevance is the prohibition on AI systems that exploit the vulnerability of a person or group because of age, disability, or a specific social or economic situation, where the objective or effect is to materially distort behaviour in a way that causes, or is reasonably likely to cause, significant harm.

This could potentially apply where an AI system deliberately targets someone because their disability makes them more susceptible to manipulation.

Examples could include:

Predatory financial advertising directed at people experiencing cognitive impairment.
Manipulative gambling or purchasing prompts targeted at vulnerable users.
Systems that pressure people with learning disabilities into unsuitable contracts.
Deceptive health products targeted at people living with incurable conditions.
AI companions designed to create emotional dependency and encourage harmful spending.

The threshold is significant: the system must exploit vulnerability and materially distort behaviour in a way connected to significant harm. Nevertheless, the express inclusion of disability is an important recognition that technology can exploit disability-related circumstances rather than merely misunderstand them.

Which AI Systems Are Considered High-Risk?

Annex III of the AI Act identifies areas in which automated systems may significantly affect a person’s rights or opportunities.

Employment and Recruitment

AI used to place targeted job advertisements, analyse applications, filter CVs, evaluate candidates, allocate work, monitor employees, assess performance or influence promotion and dismissal may be classified as high-risk.

The European Commission’s AI Act Service Desk specifically recognises that candidate-ranking systems may produce low rankings or exclusion for candidates with disabilities.

Discrimination may occur where a system:

Rejects candidates with employment gaps caused by treatment or illness.
Penalises slower speech, atypical facial expressions or limited eye contact.
Cannot communicate with screen readers or voice-control software.
Assumes that reduced working hours indicate lower commitment.
Treats reasonable adjustments as evidence that a person cannot perform the role.
Uses historical hiring data from a workforce in which disabled people were underrepresented.

An employer should not be able to avoid responsibility by claiming that “the computer made the decision”. The organisation choosing to use the system remains accountable for the employment decision and for compliance with applicable equality legislation.

Education and Examinations

High-risk uses include AI that determines admission, evaluates learning outcomes, assigns educational levels or monitors students for suspected prohibited behaviour during examinations.

Automated proctoring may disadvantage students who:

Need to look away from the screen because of visual fatigue.
Make involuntary movements.
Require rest or toilet breaks.
Use a reader, assistant or communication aid.
Speak aloud as part of a cognitive coping strategy.
Have facial differences that interfere with identity verification.
Cannot remain seated in one position for long periods.

An algorithmic accusation of cheating should never be treated as conclusive without accessible evidence, contextual consideration and meaningful human review.

Benefits, Healthcare and Public Services

The Act identifies AI used to assess eligibility for essential public assistance, healthcare services and benefits, or to grant, reduce, revoke or recover those services, as potentially high-risk.

It also covers certain credit-scoring, life and health insurance, emergency-call classification and emergency healthcare triage systems.

Disabled people may be placed at risk when a system:

Cannot understand a fluctuating or episodic condition.
Treats the absence of recent medical evidence as proof of recovery.
Prioritises easily measurable physical conditions over invisible disabilities.
Assumes that online activity demonstrates an ability to work.
Interprets inconsistent daily functioning as dishonesty.
Uses diagnosis alone without considering individual circumstances.
Calculates a person’s value or treatment priority by reference to assumptions about quality of life.
Flags disability-related expenditure as financial irresponsibility.

Decisions affecting healthcare, income or essential support require individualised consideration. Statistical probability cannot replace a fair assessment of the person.

Data Quality and the Risk of Bias

For high-risk AI systems trained on data, Article 10 requires appropriate data-governance practices.

This includes examining datasets for biases that may affect health and safety, negatively affect fundamental rights or lead to discrimination prohibited under EU law. Appropriate measures must then be taken to detect, prevent and mitigate identified bias.

Training, validation and testing data must also be sufficiently representative for the system’s intended purpose.

Representation is not achieved merely by including a small number of disabled participants. Testing should consider the diversity of disability, including:

Physical and sensory impairments.
Learning disabilities.
Neurodivergence.
Mental health conditions.
Speech and communication impairments.
Facial differences.
Chronic illnesses.
Fluctuating and episodic conditions.
Multiple and intersecting disabilities.
People using assistive technologies.

A system may perform well on average while consistently failing a smaller protected group. Overall accuracy figures can therefore conceal disability discrimination.

Meaningful Human Oversight

Article 14 requires high-risk systems to be capable of effective human oversight.

The person overseeing the system should understand its capabilities and limitations, recognise the risk of automatically trusting its output and be able to disregard, override or reverse the recommendation where appropriate.

Human oversight must be genuine.

It is not meaningful where an employee merely clicks “approve”, lacks authority to change the decision, cannot understand the model, or is expected to process so many cases that individual review becomes impossible.

This is sometimes described as automation bias: the tendency to assume that a computer-generated score must be more reliable than human evidence.

A disabled person may explain why an output is wrong, but that opportunity has little value if the reviewer treats the algorithm as infallible.

Accessibility by Design

Article 9 of the CRPD requires States Parties to identify and eliminate barriers and promote equal access to information, communications, digital technologies, electronic services and services available to the public. It also supports accessible design at an early stage so that accessibility can be achieved at minimum cost.

The AI Act also requires providers of high-risk systems to comply with applicable accessibility requirements under relevant EU accessibility legislation.

Accessibility must cover more than the visual appearance of a website. An AI service should be compatible with:

Screen readers.
Keyboard-only navigation.
Voice-control software.
Braille displays.
Captioning and transcripts.
Sign-language communication where appropriate.
Alternative and augmentative communication.
Plain-language and Easy Read information.
Adjustable time limits.
Accessible identity-verification alternatives.
Human contact routes.

A person should not lose access to a job, service or appeal because the only available process requires speech, facial recognition, rapid typing, fine motor control or use of an inaccessible mobile application.

CRPD Article 5: Equality and Non-Discrimination

Article 5 of the CRPD requires equality before and under the law, effective protection against discrimination and appropriate steps to ensure reasonable accommodation.

The Convention’s definition of disability discrimination includes distinctions, exclusions or restrictions that impair equal enjoyment of human rights. Reasonable accommodation means necessary and appropriate modifications required in an individual case, unless they would impose a disproportionate or undue burden.

In an AI context, reasonable accommodation may include:

Allowing an alternative to a video interview.
Providing additional time for an online assessment.
Accepting an application in another accessible format.
Arranging a human assessment instead of automated facial or voice analysis.
Allowing evidence to be submitted by email or post.
Ensuring a decision is reviewed by someone trained in disability equality.
Providing the explanation and appeal process in an accessible format.

Treating everyone through an identical automated process does not necessarily create equality. Equal treatment may require a different route where the standard system places a disabled person at a substantial disadvantage.

CRPD Article 9: Accessibility

Article 9 extends accessibility to information and communications technologies, electronic services and emergency services, as well as physical environments.

This means an inaccessible algorithmic process can itself become a human-rights barrier.

A benefits system that can only be accessed through biometric identity verification may exclude someone whose disability prevents the technology from recognising them. A chatbot without a reliable route to a human adviser may exclude someone who cannot communicate using its expected language pattern. An emergency-call system trained mainly on standard speech may fail to prioritise a caller with a speech impairment.

Accessibility should be built into procurement, design, testing, deployment and complaint procedures. It should not be treated as a patch added after harm has already occurred.

CRPD Article 22: Respect for Privacy

Article 22 protects disabled people against arbitrary or unlawful interference with their privacy and requires equal protection for personal, health and rehabilitation information.

AI systems can create new privacy risks because they may collect, combine or infer highly sensitive information.

A person may never expressly disclose a disability, but an algorithm may infer one from:

Search history.
Purchases of medical products.
Mobility data.
Attendance at clinics.
Social media activity.
Typing patterns.
Voice recordings.
Use of accessibility settings.
Employment history.
Insurance claims.
Communication with disability organisations.

An inferred health condition can be as sensitive and damaging as information taken directly from a medical record.

Organisations should therefore ask whether disability-related data are genuinely necessary, whether individuals understand how their information will be used, how long it will be retained and whether it may be shared or used for unrelated purposes.

Data collected to provide an adjustment should not quietly be repurposed to evaluate employability, insurance risk or suspected fraud.

Automated Decision-Making and the GDPR

The AI Act does not replace data protection law.

Under Article 22 of the EU General Data Protection Regulation, a person generally has the right not to be subject to a decision based solely on automated processing, including profiling, where it produces legal effects or similarly significantly affects them.

There are exceptions, but safeguards may include human intervention, an opportunity to express a point of view and the ability to contest the decision.

The protection does not apply to every computer-assisted decision. A key issue is whether the decision is based solely on automated processing and whether the human involvement is real rather than symbolic.

Where applicable, data protection rights may also enable a person to ask for information about the existence of automated decision-making, meaningful information about the logic involved and the likely significance and consequences of the processing.

The Right to Know and the Right to an Explanation

Under the AI Act, deployers of Annex III high-risk systems that make or assist with decisions about individuals are required, once the relevant provisions apply, to inform people that they are subject to the use of a high-risk AI system.

Article 86 also provides a right to a clear and meaningful explanation in certain circumstances where the output of an Annex III high-risk system is used to make a decision that produces legal effects or similarly significantly affects a person and adversely affects their health, safety or fundamental rights.

This should help prevent people from receiving unexplained statements such as:

“Your application did not meet our automated criteria.”

A meaningful explanation should identify the role played by the system and the principal factors that affected the outcome. It should be sufficiently clear to allow the person to understand, question and, where appropriate, challenge the decision.

What Can a Disabled Person Do?

A person who suspects that an automated decision has discriminated against them should consider taking the following steps:

Ask whether AI or automated profiling was used.
Request the decision, reasons and evidence in writing.
Ask what information was used and where it came from.
Request an accessible explanation of the role played by the system.
Identify any inaccurate, outdated or misleading information.
Explain how the system failed to account for disability or reasonable accommodation.
Request meaningful human reconsideration by someone authorised to reverse the outcome.
Ask for an accessible alternative process or reasonable adjustment.
Keep copies of applications, screenshots, scores, emails and appeal decisions.
Use the organisation’s complaint, appeal or grievance procedure.
Consider contacting the relevant equality, employment, consumer, sectoral or data protection authority.

Article 85 of the AI Act permits a person or organisation with grounds to believe that the Act has been infringed to submit a complaint to the relevant market-surveillance authority, without removing other administrative or judicial remedies.

The exact route and remedy will depend on the country, the type of decision and which provisions were in force at the relevant time.

What Should Organisations Do?

Responsible organisations should not wait until every high-risk deadline arrives.

They should:

Conduct equality and fundamental-rights impact assessments.
Involve disabled people and disability-led organisations in system design.
Test outcomes separately across different disability groups.
Examine proxy variables and indirect discrimination.
Provide accessible non-digital and human alternatives.
Document how reasonable accommodation will be provided.
Train staff to recognise automation bias.
Give human reviewers sufficient authority and time.
Record when algorithmic outputs are overridden.
Publish clear information about automated decision-making.
Ensure complaints do not simply return to the same algorithm.
Monitor systems after deployment for unexpected discriminatory outcomes.
Suspend a system where there is evidence of serious or repeated harm.

For certain public bodies, private providers of public services and specified high-risk deployments, Article 27 requires a fundamental-rights impact assessment before the system is used. The assessment must consider the context, affected groups and risks to fundamental rights.

Disability inclusion should be part of that assessment from the beginning, not added as a generic sentence at the end.

What Does the EU AI Act Mean for the United Kingdom?

The EU AI Act does not automatically form part of UK domestic law.

Nevertheless, it may apply to a UK provider placing an AI system on the EU market or where the output of a system operated from the UK is used within the European Union.

UK organisations must also consider existing domestic laws and regulatory duties. Depending on the circumstances, these may include the Equality Act 2010, UK GDPR, the Data Protection Act 2018, employment law, consumer protection law, public-sector equality duties and sector-specific regulation.

The Information Commissioner’s Office has warned that profiling and automated decision-making can give rise to discrimination and that organisations should use appropriate technical and organisational measures to prevent it.

A system does not have to breach the EU AI Act before its use becomes unlawful under another legal framework.

Conclusion

Artificial intelligence can support independence, improve communication, assist medical diagnosis and make services more responsive. However, it can also reproduce discrimination at a scale that would be impossible through individual human decision-making.

The EU AI Act is an important regulatory development because it recognises that certain AI practices can threaten health, safety and fundamental rights. Its prohibitions, high-risk classifications, data-quality requirements, accessibility obligations, human-oversight safeguards and rights of explanation have the potential to provide meaningful protection.

However, regulation alone will not eliminate algorithmic ableism.

Disabled people must be involved in designing, testing, procuring and monitoring the systems that affect their lives. Organisations must provide accessible alternatives and meaningful routes to human decision-makers. Regulators must examine outcomes rather than accepting claims of technical neutrality.

Under CRPD Articles 5, 9 and 22, equality, accessibility and privacy are not optional features. They are fundamental rights.

Technology should remove barriers rather than automate them.