Going Beyond Compliance reports on a survey of 127 selected government websites to determine the level of accessibility for disabled people. Websites included were agencies’ corporate site, key service delivery sites, and some educational resource sites. Appendix 1 lists these websites.

AccEase Ltd was contracted by the Office for Disability Issues in 2006 to conduct this audit survey, which was undertaken over October to December 2006. This follows a similar audit survey conducted by AccEase in 2005 for the Office for Disability Issues (1). The findings of this 2006 survey are comparable with the findings of the 2005 survey, as a similar methodology was used. Appendix 2 describes the methodology.

The Office for Disability Issues has responsibility for monitoring implementation of the New Zealand Disability Strategy (2). The New Zealand Disability Strategy’s vision is of a society that “highly values the lives and continually enhances full participation of disabled people.” It provides a framework to ensure that government departments consider disabled people before making decisions that impact upon them.

All government departments are required o implement the New Zealand Disability Strategy, with plans for action that are reported against annually. The New Zealand Disability Strategy includes a number of objectives that directly relate to accessibility of public information.

The Office for Disability Issues has signalled a whole-of-government priority for government agencies to make themselves accessible to disabled people. Key areas for action are: public information; services; buildings; and employment practices.

The State Services Commission is a stakeholder in this project, and supports the overall outcome of accessible government for all citizens through the E-government Strategy. It administers and monitors the Web Guidelines, which are a set of standards and recommendations with a focus on accessibility.

Adherence to the Web Guidelines will enable websites to be equally usable by all users, regardless of physical or technological issues. Compliance with the Web Guidelines is mandatory (unless an exception is applied for by an agency and granted by the State Services Commission).

Objectives

The objectives of the 2006 survey were to:

• test websites for compliance with all levels of the New Zealand e-Government Web Guidelines (3)
• design a test to check websites’ accessibility, including both automatic and user testing
• assemble and use a test panel of disabled people as a key component of the accessibility testing. The panel has people with a range of impairments, including: low vision; blind; reading/low literacy; Deaf; mobility
• ensure the test methodology will produce results consistent and comparable with the 2005 website accessibility survey
• conduct testing
• report on progress and findings to the Office of Disability Issues.

Methodology

The testing methodology comprised two phases, and was similar to the methodology used in the 2005 survey. The methodology is outlined in Appendix 2.

The first phase involved automated testing of sites against Web Guidelines.

The second phase included manual inspection of sites and underlying code to check compliance with specific accessibility checkpoints in the Web Guidelines.

The second phase also included testing of the top 30 compliant websites by testers with a range of impairments. Comments were collected from testers on features of websites that helped or hindered them when they completed set tasks. Users were asked to give satisfaction ratings of sites based on their experience with the tasks.

These comments and ratings provide a direct insight into a user’s experience, and gives a reality check on how accessible a website is when used in an ordinary way.

Variation in methodology

There were two significant changes in methodology compared with the 2005 survey:

• Deaf people were included in the user testers
• a limit of 30 sites progressed to the user-testing phase, due to budget constraints.

Survey limitations

The survey is a snapshot in time. Some of the websites audited may have been in the process of being rebuilt or were changed shortly after the survey was completed. Therefore, the results for those websites would likely differ if the survey was redone.

There were fewer websites selected for this survey (127 compared with 150 in 2005).

A limit was placed on the number of websites that progressed to user testing in this survey. User testing was done only on the top 30 complying websites, due to budget constraints. This means more than half the 68 websites that sufficiently complied with the Web Guidelines were not eligible for user testing. In 2005, all websites that sufficiently complied with the Web Guidelines (56 in total) were user tested.

Government standards

In January 2006, version 2.1 of the Web Guidelines became mandatory for government departments. This version of the Web Guidelines is the benchmark for this survey, as it was for the 2005 survey.

Since then, the State Services Commission has completed development of revised Web Guidelines, which aim to further improve the consistency and standard of government websites.

The new Web Standards 1.0 was published on 21 March 2007. This new version does not differ radically from the previous Web Guidelines. Instead, the emphasis has been on removing duplications and dated material, and making the standards more useable.