pendragon.online

Web app that helps the visually impaired read

Web app that can scan and read printed text | HTML, CSS, JavaScript | Responsive | Made for the visually impaired

Define

Software that helps the visually impaired improve their daily life by recognising text, faces and objects is often very expensive, only compatible with specific devices or if the software is free, it doesn’t work very well or is difficult to use. There needs to be a simple, affordable app or website that the visually impaired can use on their existing smartphone to assist them with their daily lives in an independent manner.

This was my final university project and used research from the FFA prototype to ensure that it was compatible with screen readers and other assistive software but expanded its ability to work better for the partially sighted.

Discover

Formative research involved researching existing market solutions, researching the science behind interface design choices and text formatting choices and finding out exactly what visually impaired users require from an app that helps improve their daily life.

Existing solutions proved to be expensive, had to run on specific hardware, often had limited functionality or was only suitable for use with specific impairments, such as dyslexia.

A survey completed by 14 visually impaired individuals concluded that:

  • All respondents needed assistance reading documents.
  • All respondents read all types of document (printed, handwritten and digital) regularly.
  • Most respondents used software to help them read documents.
  • Microsoft Seeing AI and other text-to-speech software are the most common pieces of software for assisting reading amongst the participants.
  • Having the text read to the user was the most commonly used method of assisting with reading. Altering text size and allowing the user to zoom in into documents came second.
  • Allowing the user to be independent and compatibility with screen reader software and OS accessibility features were praised in current solutions.
  • Robotic voices, poor system performance and high costs were criticisms of current solutions.
  • Screen readers and text-to-speech engines were commonly used alongside reading software.

This shows that there is a definite need to produce affordable software, compatible with smartphone devices and assistive software to help the visually impaired read. This is what was created.

A group of people graphic.

Formative usability testing

Formative testing with Microsoft Immersive Reader was conducted. Microsoft Immersive Reader is a Microsoft Office extension which allows the user to more easily increase text size, change fonts and change the font and document colours to make reading easier. A mix of sighted, myopic, dyslexic and partially sighted users were tasked with completing a small user journey on Microsoft Immersive Reader to learn how well its interface worked and if the text formatting settings presented are easy to use and useful to the target audience. Two comprehension questions were also asked.

The test concluded that Immersive Reader was moderately difficult for a visually impaired user to use, but relatively easy for somebody with dyslexic or myopia to use. Those with more severe impairments frequently complained about controls not being in expected locations, controls being too small (many users stating that they'd use magnifying software to complete tasks) and generally poor colour contrast on the interface. Those with dyslexia and myopia generally found the interface easy to navigate and were able to quickly adjust the document formatting to their preferred viewing, whereas it generally took those with more severe impairments longer to complete these tasks. The reading speed of all participants was 94.23 words per minute, which is 94.2% below that of the average adult oral reading speed as calculated by Brysbaert (2019) and 62.36% below that of the average speed calculated in this study.

Data showed that not all usability tasks set were easy to complete with Immersive Reader.

Adjusting the font size and changing the document colour were the only two tasks that visually impaired users could complete instinctively on Microsoft Immersive Reader.

The user's ability to read the text and answer questions based on the text correctly wasn't great.

No question was answered 100% successfully in the comprehension task, with 16% of users not being able to answer either question.

The test with Microsot Immersive Reader also began to also show patterns in the text formatting that visually impaired users required. These observations would be used to help design prototypes of Eyes Without Sight.

Data showed that Sitka was the preferred font.

Sitka (Serif) was generally the preferred font among testers with partial-sight, tunnel-vision and myopia, however Calibri and Comic Sans were also popular options.

Data showed that adjusting the font size and weight was desirable.

Visually impaired testers generally felt that it would be important to be able to adjust the font size and weight, but more felt that adjusting the font size was more important than the weight.

Data showed that changing the font colour was desirable.

Being able to adjust the font colour was deemed a desirable feature by 83% of visually impaired users.

Data showed that black was the preferred document colour.

.Black was the preferred document colour for 70% of users and lavendar the other 30%. White and other colours were not the preference.

After the survey and testing had been completed, it was easier to understand what visually impaired users wanted in an app that would help assist their reading.

The empathy map below consolidates the finding during formative research.

Empathy map showing what visually impaired users want. Mainly independence, ease of use and device compatibility.

Develop and test

The web was chosen as a platform because it meant that the app would work on any device that supported the web technologies used. A variety of libraries and technology was used, such as the Tesseract JavaScript library for optical character recognition and JavaScript’s own camera access, speech recognition and text-to-speech (speech synthesis).

The first iteration

The first iteration featured a similar interface to Microsoft Immersive Reader, but focused on improving it with larger text, higher contrast colours and simplified menus and options. Sliders and buttons were used to configure individual text formatting features and the default document colour was black with white text, proven to be the most common combination in the Microsoft Immersive Reader formative testing. Scientific research was also used to determine default text formatting, such as text size, text colours and font options. The formatting options proven to be the best for visually impaired users in scientific research were all options used in this software.

Wireframe design of the interface, showing the options panel with formatting options on the right.

Wireframe design of the interface, showing the options panel with formatting options on the right, similar to the design of Microsoft Immersive Reader.

Diagram showing the screen reader flow of the Eyes Without Sight first prototype.

Diagram showing the screen reader flow of the proposed interface. Screen reader compatiility was an important factor.

Diagram showing how the options panel looks with the different colour modes enabled.

Diagram showing how the options panel looks with the different colour modes enabled.

Default view of Eyes Without Siight Milestone 1.

Default view of the first prototype that was tested.

Options panel open in Eyes Without Siight Milestone 1.

Options panel open, showing the text formatting options the user had.

Default view of Eyes Without Siight Milestone 1 with options panel open and black text on yellow background selected.

Black text on a yellow background with the options panel open shown.

Default view of Eyes Without Siight Milestone 1 with options panel open and black text on lavendar background selected.

Black text on a lavendar background with the options panel open shown.

Testing the first iteration showed that compared to a standard Microsoft Word document, reading speed and accuracy were similar, but 87% of users preferred the first iteration prototype due to it being easier to use and the options provided being applicable to their needs. One user’s reading speed increased from 28 WPM with Microsoft Word to 97 WPM with this prototype, an increase of over 200%.

Diagram showing that the reading speed was slightly faster in Word Online than Eyes Without Sight.

Generally, reading speed was slightly faster in Microsoft Word Online than Eyes Without Sight.

Diagram showing that the acccuracy was slightly higher in Eyes Without Sight than Word Online.

However, the reading accuracy was higher in Eyes Without Sight than Microsoft Word Online.

Again, during testing more observations about the settings that visually impaired users were using to format the document and text were taken which were used to learn more about the ideal text formatting options for the visually impaired.

Diagram showing that the 75 percent of users used Source Sans Pro font.

Source Sans Pro was by far the most commonly-used font, with 75% of users using this font.

Diagram showing that generally font size is 12vh, letter spacing 3vh and line height 22vh among users with 8 different visual impairments.

The vast majority of users used a font size of 12vh, a line height of 22vh and letter spacing of 3vh.

Diagram showing that white was the most commonly-used font colour.

White was the most commonly-used font colour, with 62% of users using this colour. Black was used by 25% of users and yellow by 13%.

Diagram showing that black was used by 75 percent of users for the document colour.

75% of users used a black background and 25% a lavendar background. Yellow and white was available, but were never used.

The second iteration

The second iteration featured a redesigned interface (inspired by the Optelec Compact 10 HD device), designed to work better on mobile devices. The buttons were larger, the contrast was higher and this prototype also featured working optical character recognition. It also limited the number of settings that a user could change, for example line and letter spacing was automatically increased when the font size was increased to make using the app faster. Speech recognition was also included as an interaction method to help those with impairments that limit field-of-view which make finding buttons difficult.

Design for the capture interface.

Design for the capture screen where the user uses the device camera to scan a printed document.

Design for the settings interface.

Design for the settings interface where the user can change app settings such as default text size.

Design for the text output interface.

Design for the output screen. This interface design features much larger buttons than the last and can be better used with large mobile devices or smartphones in landscape orientation due to havin the buttons on the edges. Text is also now not covered when adjusting the options.

Testing was completed by 109 users remotely and analytics data collected showed that it appeared to be easy to use and many of the defaults set were ideal for the users. Over 100 pieces of text appeared to have been scanned during testing with just a few comments made

Diagram showing the most common events.

Speech recognition and text-to-speech events were the most common, showing that thse features were desired. Formatting events were the third most common, suggesting that altering the formatting wasn't required for all users.

Diagram showing the most common font and document colour combinations used.

The default colour combination of white text on a black background was the most commonly-used, however yellow text on a black background was also a commonly-used colour combination.

Diagram showing the font sizes users preferred.

The default font size in this prototype was 10vh which appeared to be ideal given that 79% of users used a font size between 10 and 30vh. Font sizes over 30vh were uncommon and possibly too large.

Diagram showing the most popular fonts used.

Again, Source Sans Pro was the most commonly used font with 44% of the users preferring this font. This font was also the default. Its popularlity comes from having distinguishable tails on letters which helps improve recognition.

The large buttons and fonts in the design of Eyes Without Sight make it easier to use for users with impairments that limit field-of-view or make focusing difficult than some existing market solutions, such as Microsoft Immersive Reader. The text and interface of Eyes Without Sight is noticeably easier to see with visual impairments than that of Microsoft Immersive Reader.

Diagram showing how Eyes Without Sight and Microsoft Immersive Reader look to sufferers of Glaucoma.
Diagram showing how Eyes Without Sight and Microsoft Immersive Reader look to sufferers of Strabismus.

Outcome

The third iteration included small improvements, such as automatically scrolling text and a larger default font size to save more users needing to change it.

Demonstration of Eyes Without Sight scanning a printed document and allowing the user to adjust the resultant text.

The Eyes Without Sight Milestone 3 prototype running on various devices.

Eyes Without Sight shown on various devices, demonstrating its ability to run on a variety of mobile devices and features such as overlays for the capture view to aid capturing documents for those requiring colour overlays to make reading easier.

Evolve

This web app has been proven to be a success. It can scan a document with a device camera and the user can reformat the text in a way that makes it easier for them to read. Therefore, it is possible that it can be converted into a Progressive Web App (PWA) and released to the app stores to potentially help millions of visually impaired users read text with a device that they already own.

Related

Enjoyed this case study? Have a look at some similar work!

Accessibile Mobile e-Commerce Site

A shocking 98% of the top one million home pages on the internet fail to meet accessibility standards. Mobile use by the blind is also very low. How can this be improved?

Stellardrive Car Infotainment System

Infotainment systems are standard even on entry-level cars these days, so why are they still frustrating to use, slow to operate and poorly rated? What would be better?

Cura-T Facebook Chatbot

How do you make finding clinical trials easier? Interning at Earthware, I worked closely with Cancer Research UK to develop a Facebook Messenger chatbot to do that.