It is well known that blind or visually impaired users have a poor user experience, as products are usually designed by and for sighted people. Yet, just like sighted people, visually impaired ones have goals to accomplish and tasks to perform when they use websites and apps. Users with disabilities need usability just like users without. 

When we talk about digital accessibility, it is important to think beyond technical accessibility scores and standards and care about people's actual experiences. While technical standards are vital, they do not guarantee a smooth experience for people using screen readers. 

For the readers who can see: Close your eyes and imagine how you would consume this text, navigate through this website or find contact information without help. Any ideas?

Users who are blind or have low vision rely on assistive technology like screen readers to use the web. Applications must therefore be accessible through such devices. Without it, the world wide web becomes unusable. However, that a design is technically speaking accessible doesn’t yet mean that it’s easy to use or even enjoyable. That is why we conduct usability tests with diverse user groups.

Sharing from first-hand experience

In the quest for digital inclusivity, dealing with accessibility challenges can be daunting for stakeholders and teams who lack know-how or budget. That’s why I’d like to share some firsthand experiences inspired by the tests we conducted to assess the accessibility of our company website.

Finding participants to conduct tests with

Companies often feel that it is difficult to find participants to conduct tests with. However, if you look at the numbers, there is no shortage of potential participants: 

• More than 2.5 million people live with a disability or chronic illness. (Source)

• In 2021, 28.4% of the Dutch population has been restricted in activities that people/children normally do for 6 months or longer due to health problems. (Source)

Nevertheless, these users are not commonly represented in recruiting databases, and it can be difficult to find them. Reaching out to the communities themselves goes a long way. Also, working with a recruitment agency specialising in accessibility can help. 

It's worth noting that numerous individuals with disabilities embrace opportunities to enhance the accessibility of technology, recognising the constant challenges they face in their daily lives. This inherent motivation makes recruiting surprisingly simple and affordable. However, whether recruiting through word of mouth or other means, it is crucial to provide generous compensation for each participant. Their valuable expertise is offered as a service to aid in the improvement of a (potentially profitable) product, warranting fair acknowledgement of their contributions.

When testing our website, we collaborated with Ingrid Pfrommer, the founder of OnderAnderen and accessibility advisor for the Dutch government. Ingrid facilitated connections with individuals proficient in using screen readers. This seamless collaboration enabled us to swiftly transition from recruitment to gaining valuable insights. Within a week's time, we were able to identify actionable items and seamlessly incorporate them into our development sprints, ensuring tangible progress towards improved accessibility.

Do you consider conducting regular accessibility tests? Then we recommend you build your own panel of users who can periodically participate in tests. This will help you streamline processes and methods to be efficient and inexpensive at scale. However, it is essential to remain mindful of potential biases that can arise when repeatedly working with the same set of participants. Over time, participants can acquire prior knowledge and familiarity with the solution, potentially introducing unintended influences into the testing process. By acknowledging and managing these limitations, you can maintain the integrity and objectivity of your accessibility testing results.

Recording

Recording research sessions is a valuable practice, and it becomes particularly crucial during accessibility testing. Like in other testing scenarios, listening to participants' verbalisations and observing their actions provides invaluable insights. In addition, in the context of accessibility testing, it is equally important to record the output of the screen reader and track its focus as it navigates across the screen.

Seeing a screen reader in action for the first time makes it clear quickly: Following all these elements simultaneously in real-time without prior experience can be challenging. By recording the sessions, you gain the ability to review and analyse the movements of the screen reader's focus, its output, the participant's actions, and their spoken feedback. This comprehensive approach allows you to capture a holistic view of the testing process and uncover valuable insights that may go unnoticed.

As we embarked on our quest for a more inclusive digital experience for our website, we knew we needed to gain a firsthand perspective. We defined “Jobs to be done”, and conducted accessibility tests with a group of individuals who navigated the website landscape using screen readers.

We invite you to delve into a snippet of a testing recording, for which the participants were asked to “Learn more about the services provided by our team in Melbourne.” Witness firsthand the dynamics between the participant, the screen reader's output, and the navigation of focus across the screen. By experiencing a real-life testing scenario, you can gain a deeper appreciation for the rich information that can be gleaned from recording sessions. So, without further ado, let's explore a fascinating segment of the testing journey together:

Video: Accessibility testing: example 1

As you watch this video, you can get a glimpse into the world of screen readers and their remarkable ability to navigate through a web page. Notice the speed at which the voice traverses the content, weaving its way through the page. 

Did you catch the participant's slight hesitation, the momentary dance of exploration as they landed on the Melbourne page? This behaviour is an indication that they experienced a fleeting sense of uncertainty, not entirely certain if they had arrived at the intended destination. 

Let’s look at a second video, where the screen reader is set to a slower speed and the participant thinks aloud about the root cause of the problem: 

Video: Accessibility testing: example 2

You were able to witness how effortlessly the participant navigates their way to the desired page, guided by our user interface. However, as their curiosity seeks confirmation, they search for the presence of an h1 heading proclaiming "Melbourne." It becomes evident that our headers, in terms of HTML structure, fell short of providing the clarity needed, momentarily unsettling the participant's sense of orientation. In contrast, the service section showcases flawlessly defined headers that empower the screen reader to unfold the structured content. 

The example illustrates a fundamental point: while a sighted person may grasp information through font sizes and visual cues, the screen reader relies on the underlying HTML structure to convey the same meaning. It serves as a poignant reminder of the vital role that clear HTML logic plays in enabling the screen reader to interpret and deliver an inclusive digital experience.

Synthesising information to insights

Through these accessibility tests, we ventured into the world of the visually impaired, observing and listening to their experiences. It was eye-opening to witness the specific obstacles they encountered while interacting with our digital interface. We saw the importance of considering their unique needs and ensuring our website was accessible.

But the significance of these tests extended far beyond individual challenges. As we analysed the observations, patterns began to emerge, weaving a narrative that transcended visual abilities. We realised that the goals of individuals with disabilities were often strikingly similar to those without disabilities. 

Let’s take a look at another video, where we asked participants to find the address of our office in Frankfurt. Observe closely as the participant's mind springs into action, exploring various avenues in their search, only to be met with repeated disappointment. Each attempt falls short, leaving them grappling with a sense of frustration, ultimately reaching the conclusion that the address must not reside on this particular page. – An insight that matches the comments of sighted people who also expect the contact information to be on the side of the cell.

Video: Accessibility testing: example 3

These revelations guided our team towards a profound understanding. By addressing the barriers faced by visually impaired individuals, we were simultaneously enhancing the user experience for all users. We recognised that the improvements we made had a far-reaching impact, benefiting individuals with diverse abilities. This effect, by the way, is called the curb cut effect.

“The challenges faced by visually impaired individuals are not isolated struggles but shared experiences that touch us all. By addressing their needs, we are enhancing the digital journey for every user, regardless of their abilities."

Backed with these insights, we set out to refine our website. We introduced changes that eliminated accessibility obstacles and facilitated smoother user flows. We are tackling this step by step and are aware that some problems have not yet been solved. Like other usability issues, we see accessibility issues as ongoing challenges that we continue to address. 

Summary

In this article, I explored the power of usability testing in uncovering accessibility barriers and broader user flow issues.  I shared firsthand experiences and lessons learned from testing the website of foryouandyourcustomers. I hope you feel inspired to undertake your own quest for making your digital products and services accessible. Feel free to drop me a line, I would be happy to help and accompany you on this journey, for example, with:

• Audits 

• Trainings and workshops

• Creating accessible websites

• Leading your team in adopting an accessibility practice

• Testing with diverse user groups