Of the 30 compliant sites that were user tested, some were very poorly rated by testers. This is because many features of a website that contribute to a good user experience cannot be objectively measured by standards.

An example is the standard governing alternative text (‘alt text’) for graphical elements of a web page. Automated testing against the standard (refer Web Guidelines 6.4.6 Images) indicates whether there is alt text present for an image. However, only testing by a user can perceive whether the alt text is meaningful and accurately describes the information presented in the image.

Similarly, automated testing for the presence of access keys is straightforward. However, how well the access keys work, and how the availability and use of access key on the site is communicated to the user, can only be determined by user testing.

Particular shortcomings were evident in sites featuring forms or tables, where informative captions and labels for table rows and columns were often scarce.

Satisfaction ratings

Based on their experiences in completing set tasks, user testers were asked to rate sites on a 1 to 5 scale:
1 – terrible, 2 – poor, 3 – satisfied, 4 – very satisfied, 5 – fantastic.

Of the 30 sites that were user-tested:

• twenty sites had an average rating of satisfied
• ten sites had an average rating of less than satisfied
• no sites had an average rating of very satisfied or fantastic.

These ratings indicate that the user experience is still less than satisfactory for many users, despite the sites complying with Web Guidelines.