Accessibility in e-commerce 2023

Date: January 25th 2024

Author: Caitlin de Rooij

Researchers: Amber Qualm, Caitlin de Rooij, Florian Schroiff

On June 28, the European Accessibility Act comes into effect. This makes digital accessibility for e-commerce services (webshops) a legal obligation. This is not without reason: in Europe, 27% of the population has a disability. In the Netherlands, it is even 32% (source: Consilium Europa).

Online shopping provides a solution for many people with disabilities for whom physical shopping is not always possible. Unfortunately, it appears that even online shopping is not always an option. In many webshops, insufficient or no consideration has been given to accessibility.

For the third consecutive year, Level Level is conducting research on the accessibility status of the 15 largest webshops in the Netherlands, from the Twinkle Top 100. The main finding: at 8 out of the 15 webshops, it is impossible for people with motor or visual impairments to place an order without using a mouse. For the third year in a row, more than half of the largest webshops are inaccessible to people with disabilities. Many of the surveyed webshops have even declined in score compared to the initial research.

Controlling a website without a mouse?

Not everyone can control a website with a mouse. People with, for example, motor or visual impairments may not always be able to use a mouse. They can use assistive technologies to control their computer or phone.

Assistive technology is a broad term and can mean different things. For instance, using your keyboard for control, or a screen reader that reads everything from your screen, or voice control: using your voice to control the computer.

There are many forms of assistive technologies, and they often have one thing in common. That is, keyboard control is the foundation. If your website can be used with just the keyboard, you have already made a significant portion of your website accessible.

In this study, we assess each webshop in 18 steps based on three criteria:

1. Can you place an order using only the keyboard?
2. Can you place an order using a screen reader?
3. What is the code quality of the webshop?

The study was conducted in November 2023. Any changes made to the websites after this period have not been considered in the study. Similar to previous years, the research is divided into several steps.

Results: the top 15

The final scores of the examined webshops are displayed in the table below. Click on the links in the table to view the individual results for each webshop (only available in Dutch).

Rank	Website	Score
1	Apple	88
2	Jumbo	78
3	Bol	73
4	HelloFresh	64
4	GAMMA	64
5	Coolblue	63
6	MediaMarkt	61
6	IKEA	61
7	Albert Heijn	60
8	Amazon	58
9	Zalando	56
9	H&M	56
10	Belsimpel	42
10	Wehkamp	42
11	HEMA	33

Significant differences have been measured, both positive and negative. It is noteworthy that only three online stores have improved in accessibility compared to the baseline measurement (the initial recorded score in 2021 or 2022).

Webshop	Percentage Difference: Baseline Measurement vs. 2023
Apple	0
Jumbo	23
Bol	7
HelloFresh	-6
GAMMA	-19
Coolblue	-9
MediaMarkt	30
IKEA	-6
Albert Heijn	-3
Amazon	-9
Zalando	-8
H&M	-27
Belsimpel	-11
Wehkamp	-1
HEMA	-13

Is it possible to place an order on this website using only the keyboard or a screen reader?*

Website tested with keyboard	2021	2022	2023	Website tested with screen reader	2021	2022	2023
Apple	yes	yes	yes	Apple	yes	yes	yes
Jumbo	no	yes	yes	Jumbo	no	yes	yes
Bol	yes	yes	yes	Bol	yes	yes	yes
HelloFresh	yes	yes	yes	HelloFresh	yes	yes	yes
GAMMA		yes	yes	GAMMA		yes	yes
Coolblue	yes	yes	no	Coolblue	yes	yes	no
MediaMarkt	no	no	yes	MediaMarkt	no	no	yes
IKEA	no	no	no	IKEA	no	no	no
Albert Heijn	yes	yes	yes	Albert Heijn	yes	yes	yes
Amazon	yes	yes	no	Amazon	yes	yes	no
Zalando	no	no	no	Zalando	no	no	no
H&M	yes		no	H&M	yes		no
Belsimpel		no	no	Belsimpel		no	no
Wehkamp	no	no	no	Wehkamp	no	yes	no
HEMA	no	no	no	HEMA	no	no	no

Where can I place an order if I have a disability?

The top 15 is declining

Unfortunately, we observe that many webshops score lower than in the baseline measurement. At Coolblue, Amazon, and H&M, it is even no longer possible to place an order without a mouse this year, whereas it was possible in previous years.

Strong decline in previously accessible webshops

H&M and GAMMA have experienced a significant decline in scores. In 2021, H&M held the second position with a score of 83%. This year, it has decreased by 27%. GAMMA, first included in the study in 2022, initially performed well with a score of 83%, but it has now dropped by 19%.

Both H&M and GAMMA made some changes to their websites, seemingly without consideration for accessibility. As a result, the ordering process has become much more challenging for individuals with disabilities.

At H&M, it is no longer possible to choose a size for a clothing item using the keyboard or screen reader. Without selecting a size, it is not possible to add the product to the shopping cart. This means the ordering process for visitors with disabilities ends at this point.

At GAMMA, it is still possible to place an order; however, the ordering process has become more challenging. Using the main menu and indicating options (such as selecting colors) has become more difficult. It is now impossible to use keyboard-only navigation for filter options in search results. Choosing a delivery time has also become more challenging.

Another significant decline in webshops that were previously already inaccessible

There are also two webshops that performed poorly in previous studies: HEMA and Belsimpel. These websites have critical accessibility issues that make it impossible to place orders. Unfortunately, these problems have not been resolved. On the contrary, more issues have emerged.

Both webshops make it very difficult to navigate the website with the keyboard. With the keyboard, you cannot aim and click as you can with a mouse. Instead, you have to jump from one clickable element to another using the Tab key on the keyboard. A focus indicator indicates which clickable element you are on, allowing you to “click” on it with the Enter key.

At HEMA, the focus indicator has been completely disabled, making it practically impossible for keyboard users to navigate the website. At Belsimpel, the focus indicator is not always visible, making it easy to lose track of your location. Both webshops are highly inaccessible from the first step of the ordering process.

Moreover, in both webshops, it is not possible to choose a payment method with the keyboard or screen reader and then press the order button. It is very frustrating to realize during the final step of the already challenging ordering process that placing an order is not possible at all.

CAPTCHA: a mechanism designed to exclude both bots and people with disabilities

The most notable change this year is observed in webshops that were previously quite accessible. Coolblue and Amazon have made changes to their websites, making it no longer possible to place an order without using a mouse. This is solely due to the implementation of a CAPTCHA, a type of puzzle designed to test whether the user is not a robot. A common type of CAPTCHA involves selecting all images related to a specific subject.

In both cases, these CAPTCHAs cannot be navigated with the keyboard. Coolblue provides an “audio challenge” as an alternative. This challenge involves playing an audio clip, and the user is required to type the letters and numbers heard. However, the audio clips are so unclear that the researchers were unable to complete this task. Successfully completing a CAPTCHA is a prerequisite for creating an account at Coolblue. Since the CAPTCHA is not accessible, creating an account is not possible. Without an account, an order cannot be placed. Consequently, the ordering process at Coolblue cannot be completed. This was possible in 2022.

Amazon has implemented a CAPTCHA before the website can be used. The CAPTCHA does not always appear. When the CAPTCHA is displayed, the entire website becomes unusable for visitors using a screen reader. If the CAPTCHA is not displayed, the user can proceed with the ordering process, but it is still impossible to complete the order in one of the final steps. This is because a payment method cannot be selected using the keyboard and screen reader.

Without these CAPTCHAs and changes to the payment module, it was possible to place orders in previous years at Amazon and Coolblue. Due to these small changes, visitors with motor or visual impairments can no longer place orders at these stores.

No mouse? No discount!

Another noteworthy result is that it is not possible to enter a discount code without using a mouse on two webshops. At Albert Heijn and IKEA, the discount code field is not accessible with the keyboard due to a code error. This means that certain users are excluded from being able to obtain discounts. This situation emphasizes the importance of web accessibility and inclusive design. The inability to enter discount codes with the keyboard creates unequal opportunities for online shoppers with disabilities.

Four companies are doing well

Four companies stand out positively.

Apple

Apple has not only remained consistent, but it has been evident since the first research moment that Apple is committed to accessibility. Apple distinguishes itself with its accessible solutions. With a few small adjustments, they could even achieve a perfect score of 100%.

MediaMarkt

MediaMarkt has scored 30% higher this year than in 2021, marking a significant improvement. In previous studies, it was found that users without a mouse couldn’t select the color or version of a product. It was also impossible to press the order button in the shopping cart, and a payment method could not be chosen. However, in the 2023 study, it became clear that these issues have been resolved, and it is now possible to place an order with just the keyboard or a screen reader.

Currently, one step in the study cannot be performed with the screen reader: using the main menu. If this is addressed, the most significant issues on the website will be resolved. After that, the focus should shift to improving steps that can be performed “with difficulty.” Given the pace at which MediaMarkt has made accessibility improvements over the past two years, it promises positive developments.

Jumbo

A second webshop that stands out due to its high percentage increase is Jumbo, which has seen improvements in both the 2022 and this year’s studies. In 2022, the major obstacles in the ordering process were resolved, making it possible to order without using a mouse from that point onward. This year, clear attention has been given to further optimizing the ordering process, and all steps in the study could be completed. Jumbo has seen a total increase of 23%, now ranking second in the webshop accessibility study.
Jumbo has evidently been proactive in addressing accessibility, as reflected in the study results and supported by Rowan Zajkowski, Design Lead at Jumbo. Rowan wrote about the initial steps and the approach within Jumbo to become more accessible: Organizing Accessibility: Jumbo’s journey towards an accessible webshop.

bol

Bol is the largest webshop in the Netherlands. In previous studies, there was only one step in the research that couldn’t be performed with just the keyboard or a screen reader, which was already a relatively positive result. In the 2023 study, we observed that several optimizations had been implemented, resulting in a 7% increase in their score.

Bol is steadily improving its score. There are no major obstacles that make ordering from Bol impossible for users who cannot use a mouse.

However, there are a few steps within the ordering process that are challenging to execute. For example, the main menu is not expandable with the keyboard. Filters in search results are also difficult to use. Creating an account and choosing a payment method is challenging for screen reader users as well.

Complex interactive elements

Bol.com has worked on optimizing complex interactive elements, such as modals: screens that open over an existing web page.

More complex elements are not always standard in HTML, the markup language of a webpage. As a result, developers often have to build them entirely themselves. It is common to see that accessibility is not taken into account for these elements, and they only work well with a mouse.

In the case of a modal, someone using a mouse can click on the buttons in this overlaying element. However, someone using the keyboard gets stuck on the button behind this window. They have to navigate through every button and link on the underlying page until they reach the end of the page before they can access the buttons in the modal. This is very frustrating.

Bol.com has ensured that keyboard users jump directly to the overlaying screen. This makes a modal much more user-friendly for keyboard and screen reader users.

The danger of “non-critical accessibility issues”

We answer all research questions with “yes,” “with difficulty,” or “no.” Many webshops score “with difficulty” on different aspects, meaning the step can be executed but not always easily—sometimes even with a lot of effort.

If the ordering process is challenging, a visitor may decide to abandon the process. Faced with numerous obstacles, this visitor may perceive the webshop as inaccessible, even if it is technically possible to place an order after significant effort. This phenomenon has been demonstrated in the British study “The Clickaway Pound,” which investigates the amount of money businesses in England lose due to visitors with disabilities clicking away after encountering accessibility issues. In 2019, this amounted to £17 billion.

Kim Denie (25) is an expert in this field. She has had a visual impairment for the past six months, making her eyes highly sensitive to light. She uses special lenses and software to, for example, apply a red filter and enlarge web pages and text.

If I really need something, I rarely ask for help from others if I can’t manage it myself. For things I don’t really need and where the ordering process is so cumbersome, I just click away. Shopping at inaccessible webshops takes too much effort and too much exertion for me. For example, I can no longer shop at my favorite webshop.

Kim Denie

How do I make my website accessible?

How do you make your website accessible? And how do you ensure that you are ready for the European Accessibility Act in 2025? There is no one-click solution for accessibility. While there are tools claiming to achieve this, they often worsen accessibility issues. Read more here about the dangers of such tools.

Accessibility can be measured based on the Web Content Accessibility Guidelines (WCAG). Not all criteria within WCAG are automatically testable. Approximately 30% of the criteria are automatically testable. This also means that making a website accessible is not a one-click process, simply because the remaining 70% of errors cannot be found without manual testing.

This implies that time needs to be invested.

The first step is awareness

The first step towards accessibility is always “awareness.” Simply put, not everyone is aware of accessibility. Becoming aware initiates the process. It is then crucial to spread that awareness so that action is taken.

Learn what the topic entails and what you can do to become accessible. Additionally, it’s crucial to assess the accessibility of your website:

• There are scans available that can test your website for automatically detectable errors. This provides an initial glimpse into the accessibility of your website, although it never gives a complete overview.
• By having your website tested for accessibility, you can obtain a comprehensive overview of identified errors and the overall state of accessibility. Level Level can assist you with this.

Subsequently, it is essential to bring about change.

Designers, developers and content editors

For this group, it is important to understand what accessibility entails and how they can apply it in their field. By making accessibility a part of your expertise and incorporating it from the beginning of the design process, you can ensure that everyone can use your website.

• The A11y Collective offers customized training for designers, developers, and content editors. Learn about the fundamentals or delve into more advanced topics.
• Level Level also provides customized training and workshops in the field of accessibility. These sessions always include explanations about accessibility and its improvement, tailored to your own product.

Webshop owners

Here are some important steps for webshop owners:

1. Look for accessible suppliers: Choose suppliers that offer tools and technologies that comply with accessibility standards.
2. Demand WCAG accessibility in proposals: Ensure that suppliers indicate in their proposals that their products and services adhere to the guidelines of the Web Content Accessibility Guidelines (WCAG).
3. Request proof of expertise: Verify the expertise of suppliers by asking for examples of previous projects and certifications in the field of accessibility.
4. Gain insight into the current accessibility status of your webshop: For example, conduct an audit. Identify areas that need improvement and create a plan to meet the requirements.
5. Acquire a basic understanding of accessibility to convey within the organization.

Making accessibility a priority within a large company is challenging but not impossible.