Personalized web for the assistive technology

Sometimes the web authors provide a different content for screen readers than they do that for sighted users. That could be an additional content like "skip to content" navigational links or set of landmarks to create a more reliable document structure. In some cases the web author might want to remove a content, for instance, duping links, or make extra tricks to keep the content accessible if, for example, the author gets out of the standard HTML. In ideal case of course the content is supposed to be quite the same but because of design issues and HTML imperfection it doesn't really happen. The web repletes with examples of special content for the assistive technology.

The need of alternative content

Usually authors don't need to put significant changes into a web page to make it accessible. Keeping in mind usability aspects and following best practices is often enough for good results. In other words this is the kind of changes (except some ARIA) that is supposed to be useful for everybody. Sure it doesn't count the web pages having large pieces of ARIA but that's rather an area of large web apps and custom widgets.

That's how it is but nowadays tendency is getting changed and certain web apps want to provide whole portions of alternative content for the assistive technology.

A few examples are good for demo proposes.

Couple examples

Shared video example. If blind and sighted users want to share the device to watch the movie then it might be good idea to have audio descriptions shown (announced) for the blind user only.

Camera apps. It may be another use case of separate content for blind and sighted users. A camera app shows what's on the camera and may have graphic-only interface like green rectangle showing the thing that is currently in focus. A screen reader user may benefit if the face detection software beeped when face gets in focus since it gives a good chance of nice picture.

Another example can be QR code reader software which helps to find QR code label on the product. In general all camera apps may benefit from giving special instructions for assistive technology users.

Integration vs personalization

So alternative content for assistive technology can be a part of the web app design. A next question is how the alternative content can be added into the web page. Would it be one integral app explicitly and implicitly containing special content or will it be personalized version of the app designed for the assistive technology.

Both approaches have own benefits and disadvantages. So that personalization approach wins in performance since potentially the app doesn't need to combine two different versions into one (and actually it is a big concern of web apps vendors from what I hear). Benefit of integration is people get all-in-one solution what mean users share the app and usually have same experience.

Privacy concerns

A big issue of personalization the people talks about is privacy. If you want to have a personalized version of the web site then you have to tell the web site you use the assistive technology.

The idea of sharing personal information is not comfortable in general for many people and it's quite understandable. But you need to keep in mind that those who wants to know whether the user uses the assistive technology quite likely have a way to detect this. These are solutions like screen reader sniffing flash plugin or JS script based on the difference in content navigation. For example, it didn't take much time for me to write a simple script (can be found attached to Mozilla bug) to detect the NVDA screen reader. These solutions are not perfect and may give false positives but I'm pretty sure they can be improved if somebody wanted.

On the another hand if the user says he wants a specialized version for assistive technology then it doesn't necessary mean the user has the assistive technology running and of course it doesn't necessary mean the user have to share what kind of assistive technology he uses.

So of course there's a privacy concern but it's not bigger than, say, privacy concern of Geolocation API. The difference in sharing and not sharing is rather seeming. After all the user decides.

Tech party

Not sure which approach will get dominant, perhaps that will be some reasonable mix of them. Anyway it would be a good idea to provide web authors the different techniques to implement either approach.

 

JS API

Just a method to detect the presence of assistive technology such as a screen reader allow the web app to load specialized scripts and build personalized version of the app. The idea in implementation can be quite similar to Geolocation API. If the web app wants to know whether an assistive technology is running then the user gets asked if he's ok to share this info. If the user agrees then personalized app is loaded.

aria-hidden

It can be used to hide certain parts of the web page from the assistive technology and (quite a recent change) it can be used to create web page portions for assistive technology only (not visible/operable/etc for sighted users).

Actually at this point ARIA spec doesn't allow aria-hidden to create web content for AT. However w3c pushed this option into the law by resolving HTML5 bug. I admit that next ARIA spec should get in sync with the change. It's worth to notice however that WHATWG spec hasn't been changed yet and probably it won't be.

Also ARIA recommends very limited usage of aria-hidden:

Authors MAY, with caution, use aria-hidden to hide visibly rendered content from assistive technologies only if the act of hiding this content is intended to improve the experience for users of assistive technologies by removing redundant or extraneous content. Authors using aria-hidden to hide visible content from screen readers MUST ensure that identical or equivalent meaning and functionality is exposed to assistive technologies.

I suppose it may be considered as temporary advice and will be neutralized as soon as the web gets more apps having special versions for the assistive technology.

So here's a demo of the approach

<body>
  <div aria-hidden="true">An ordinal version</div>
  <div hidden aria-hidden="false">An assistive technology version</div>
</body>

The minus of the aria-hidden="true" is the user navigable content is hidden from the assistive technology. You may want to read the post why I didn't fell in love with aria-hidden="true" as it's stated. My opinion haven't really changed over years.

The minus of aria-hidden="false" is AT visible only content is not keyboard navigable until AT has special support of it, it's not focusable and has neither dimensions nor position what is a certain restriction for AT. Also it's good to note, screen readers like Orca would ignore aria-hidden="false" content in general because it requires a virtual buffer feature for content navigation but Orca doesn't have it in question.

So aria-hidden has bunch of disadvantages but I admit it has one good benefit which is its simplicity for the web author.

CSS media

 

CSS media features are designed to provide styling depending on features the device has. So if screen reader is detected then the web app can show or hide portions of the page when it makes sense for screen reader users, for example,

@media (screenreader) {

  .sr {

    display: none;

  }

}

<div class="sr">hidden from AT</div>

As you see it's very easy to change the web page and personalize it for the assistive technology user. This technique doesn't have disadvantages of aria-hidden because all content is rendered on the screen. So that the content has position and size, it's focusable and navigable by standard ways.

Nevertheless CSS media is also disputable approach (check out Mozilla bug for details). Note, afaik it's not supported by web browsers yet.

Two screens option

Same screen approach looks quite appealing but it doesn't answer to all designing needs the web authors want. For example, in the case of shared screen to watch the movie it might be a nice option if audio descriptions were shown only for those who needs them.

This idea leads to the option to render CSS media styles virtually. Technically speaking that means the presence of two screens, i.e. one version is rendered on the display (that's a movie), the second version is rendered in memory, for assistive technology only (subtitles). Nowadays a similar thing is implemented for HTML 5 canvas shadow content: it's not visible on the display but the assistive technology literally sees it, i.e. it have an access to layout, position and dimensions, and it can navigate it.

It's not clear however how to share input devices like mouse and keyboard, but probably it could be nicely resolved this or that way. This approach is quite close to aria-hidden technique since it allows same design techniques but it doesn't have disadvantages of it because the content is still rendered.

The web is going to change the way it handles the assistive technology. Quite soon I think.

Windows 8 Narrator on the web

I should notice I'm not a screen reader user and therefore my front-end observations might be not complete or not accurate. Please correct me if I say something wrong.

Windows 8 beta is out. I saw a bunch of posts saying that Microsoft improved accessibility support, in particular that Narrator was greatly refined over previous versions including outstanding performance in IE. So I wanted to play with Narrator and see how it works in Firefox.

The background of the interest is sort of pragmatic. I'm curious whether the market share of screen readers can be shifted after Windows 8 release. The rationally is simple. If some OS provides a set of built-in applications then the user gets all-in-one solution. And if those applications are good enough then many users don't need anything else. So I wasn't going to do any truth worth research but I wanted to see myself how the things look.

On Windows Mozilla bets on IAccessible2, an open standard developed by number of companies including both AT client vendors (screen readers, screen magnifiers etc) and AT server vendors. I don't think this is going to change but if Firefox wasn't nice on Narrator and if Narrator takes a certain percent of the market share then we at Mozilla can't leave this unaddressed.

This time I didn't pay an attention to touch-based accessibility features despite this is definitely a crucial part for mobile web. But this is certainly big area deserving own topic. I focused on how Narrator is doing on the web in classical desktop case.

Narrator

The first noticeable thing is Narrator highlights traversed objects similar to what Voice Over does. This should be a neat feature for teaching and training. I liked the help dialog to see command shortcuts which is good for newbies. Another interesting feature the Narrator allows to bring a dialog containing the list of available actions on the current object and the user can pick up the one to invoke.

Narrator is fast, at least I didn't noticed any lags when I navigated the web. Moreover I've got an impression that sometimes it's faster than NVDA (both on IE and Firefox).

It provides hierarchical virtual cursor: arrow keys allows you to move into container, navigate through siblings within a container and move outside a container. It seems that the implementation is quite plain, it just moves the virtual cursor through accessible tree. You may consider this as a benefit since often this allows you to skip easily a large portions of the page. However taking into account that containers aren't provided by any good description then this is rather useless feature, at least until the user learns the hierarchy for each web site he visits. To Narrator credit I must say it announces the amount of items within certain containers.

On the other hand the web pages and especially web applications have quite complicated trees and that makes navigation not trivial. I'm not experienced user but I feel like NVDA approach is nicer. Narrator approach sounded for me like it's more suitable for tree inspection for debugging proposes rather than for document navigation in the wild.

I looked how Narrator goes with some applications like GMail. I was able to navigate the whole GMail, however sometimes the virtual cursor reached a certain point and didn't move further. I was forced to use tab to skip this place. Also I wasn't able to locate some menus of menu buttons. I think this is because these menus weren't in hierarchy of the button. So you might want to blame Gmail.

Narrator allows to navigate the text, provides commands to traverse the page by paragraphs and supports table navigation but I didn't make it to do more complicated structural navigation like navigation by headings, links etc. It doesn't sound like Narrator has it.

MSAA / UIA / IAccessible2

Windows 8 makes some work to map MSAA to UIA and that allows Narrator to work with MSAA applications. That makes Firefox accessible in principle. For example, you can navigate the web pages and do basic interaction with control elements like open a link, click a button etc. But MSAA accessibility is quite restricted.

Narrator allows the user to navigate by paragraphs, lines, words and characters. It provides some support of text attributes. None of these features work in Firefox. When the document is loaded then Narrator starts reading it. Again not in Firefox.

In summary Firefox doesn't look cool in Narrator. It's sort of expected because Firefox doesn't implement UIA.

I didn't really believe that Microsoft will bridge IAccessible2 to UIA but I hoped. That would allow a number of applications like Eclipse, Open Office or Firefox to work great for Narrator users. Sometimes dreams may not come true.