"The site is compact. Logical and easy to follow. No unnecessary information on pages."

There are a number of high-quality websites in the government sector, and many agencies have made a considerable commitment to ensuring that their sites are accessible.

Best sites

"This website could be used as a model for all the government websites. It has everything. It’s eye-catching; has clearly visible labelled links; has clear informative, easily readable text; and gets its message across in as few words as possible."

The best sites, achieving between 75 and 82 points out of a total possible 100 points, are listed below.

4 million careful owners	http://www.4million.org.nz
Bioethics Council	http://www.bioethics.org.nz/
E-government	http://www.e.govt.nz
Families Commission	http://www.familiescommission.govt.nz
Ministry for the Environment	http://www.mfe.govt.nz
Ministry of Transport	http://www.transport.govt.nz
New Zealand Embassies	http://www.nzembassy.com
Office for Disability Issues	http://www.odi.govt.nz
Office for Senior Citizens	http://www.osc.govt.nz
Reduce your rubbish	http://www.reducerubbish.govt.nz
Skill New Zealand	http://www.skillnz.org.nz
StudyLink	http://www.studylink.govt.nz

General comments

"The site map is orange, which makes it hard to see."
"Lots of orange – even orange on orange."
"It is hard to see the orange links or the white links on an orange background."
"The page tops are hard to see because they are in orange."
"Once you get past the home page it’s okay. Lots of orange."
"I found the combination of burnt orange, navy and white, with its cramped text, very hard on my eyes."
"Lots of orange and some poor contrast"
"Orange spots already!"

AccEase testers giving their view of the most popular colour scheme in the test sample

Our survey indicates that the following points require emphasis:

• When pdfs are used as the sole means of providing mission-critical reports and other such information, many users are excluded by such a publications strategy. These include users under time constraints, those connecting via modems, and vision and mobility impaired users. For this reason, when information is provided in pdf, it is a Priority 1 requirements of WCAG, and a ‘must’ requirement of the egovernment guidelines, that it is also available in html.
• All images – even decorative images – must have a text alternative that provides equivalent information to the information provided by the image. It follows that, when the image provides no information, a null alt (alt="") meets the requirement. Screen readers will ignore a null alt, but will speak the word "image" when no alt is provided.
• There are three specific issues to be aware of when writing text alternatives. First, images provided for visual spacing purposes should have null alts – screen reader users find it irritating when sites read "shim, shim, shim" or "spacer, spacer, spacer". Second, the alt for a graphic should be written to provide the user not receiving images with the same information available to a user that is using images. Simply repeating the caption or information available from the surrounding text is redundant and seldom meets the information requirement. Third, complex images such as charts and graphs generally require a paragraph or more of text – either on the same page or linked to the image – that fully describes the content of the image. For example, an image of a flow chart describing the organisational structure of an organisation requires the same information to be provided in html, perhaps as a structured list on a linked page.

Again, this is a Priority 1 and "must" requirement.

• Low vision users require good contrast in order to use a site. Even if the contrast is sufficient to access information with difficulty, low vision users may find it extremely tiring to use a site. Orange was found to be a particular problem – our low vision testers all complained of headaches. Dark text on a light background of the same colour also causes difficulties. The problems experienced by users with a medium or severe vision impairment are also experienced to a lesser extent by most users over the age of 45. This is a large audience segment for most sites. In addition, designers need to allow for common forms of colour blindness. Contrast of text with background is a priority 3 WCAG requirement and a "must" e-government requirement.
• The ability to enlarge text is also a necessity for users with low vision. Key text (such as menu and other navigation information) is particularly important – and particularly problematic when such information is provided in images and therefore does not enlarge. It is also important to ensure that the initial text size is large enough so that the "Largest" setting in Internet Explorer makes text easy for a low vision user to read. This is a Priority 2 and "must" requirement.
• It is an e-government requirement that a standard list of access keys is used for all government sites. This allows users to become familiar with a single set of pan-government keyboard shortcuts. Use of non-standard keys, or no access keys at all, reduces the usefulness of the keys on all other sites by lowering user confidence.
• Focal points for links to other site areas or for returning to the top of the page need to be large enough to be easy to use for people with a mobility impairment such as a tremor or poor co-ordination.
• Rather than focusing on compliance, a good accessibility statement attempts to consider and meet the needs of the widest possible audience
• It is important that home pages should avoid being too busy; that is, presenting too much information. Remember that users with low vision need to retain context when the screen is enlarged, and those with reading impairments are more likely to find the information they needed if the page is simple and clearly laid out.

Issues that make it impossible to access some information

"Found this to be a totally bewildering site. Nothing seemed to be placed logically, and the one thing I found was by accident."

Alts, page size, scripting

"Most of the things I found were done by guesswork or accident. It was giving me a lot of what sounded like html code which was visible to JAWS."

The automated testing disclosed a need for some site managers to address the following:

• All images require text alternatives (alts).
• Programmes for automated testing are readily available, and would pick up many faults that might cause accessibility issues.
• Site managers need to ensure that scripting is accessible, or that text alternatives are available.
• The e-government guidelines mandate against the use of frames, because they isolate some sectors of the community from the information contained in them.
• Files need to be small enough to download quickly on a slow connection (as noted in Appendix 2: Our methodology, for the purpose of this evaluation we considered a file oversized at 200Kb – this is double the e-government standard of 100Kb).
• Automated testing would pick up errors in a template that therefore replicated throughout the site and also missing text alternatives for images that had no information content (such as one pixel spacers – as noted above, these require an empty alt so that screen readers don’t repeat "image" "image" "image" over and over again).
• The WCAG requirement to use a text equivalent to identify the input field to adaptive technology applies to search boxes, which are often found on every page of a site. (Methods for doing this include using an alt for the INPUT element, or coding to associate the input field with a text label.)

Such matters are usually easy to test for and fix, and many sites could be dramatically improved with a few minor changes1.

"The site was, in fact, fairly user-friendly – but if I can’t find what is required, it suggests that there is still something lacking."

Data tables

Good mark-up of data tables is absolutely crucial to making the information they contain available to blind users. For simple data tables, this merely requires column and row headers to be marked using the appropriate html tag (<th>). . The markup of complex data tables is considerably more difficult – but also more important. Poor markup of complex or large data tables excludes blind website users from the information they contain.

For example, on the following chart, reader technology for a blind person will probably read across the columns so that at one stage it will read the figures; 654, 516, 547, 468, 383, 383, 383 with no cognitive link to the category of information. Depending on the html coding and the settings on the screen reader, it may read vertically one column at a time. In either case, the blind person needs to retain a mental image of the headings and on a large chart this is at best difficult. With a correctly coded table, a modern screen reader will read:

"Retail stock and other, 2004 Actual, 654, 2005 Prev. Budget, 516, 2005 Forecast, 547, 2006 Forecast, 468, 2007 Forecast, 383 , 2008 Forecast, 383 , 2009 Forecast, 383 … "

Forecast Statement of Borrowings for the years ending 30 June

($ million)	2004 Actual	2005 Prev. Budget	2005 Forecast	2006 Forecast	2007 Forecast	2008 Forecast	2009 Forecast
Sovereign Guaranteed Debt
New Zealand-Dollar Debt
Government stock	17,351	16,283	15,992	15,313	14,467	15,415	15,385
Treasury bills	5,525	5,393	5,390	5,273	5,327	5,332	5,338
Loans and foreign-exchange contracts	(1,098)	(500)	1,050	275	(625)	(1,618)	(1,615)
Retail stock and other	654	516	547	468	383	383	383
Total New Zealand-Dollar Debt	22,432	21,692	22,979	21,329	19,552	19,512	19,491
Foregin-Currency Debt
United States dollars	3,079	1,998	1,879	2,379	2,880	3,280	3,280
Japanese yen	1,015	729	557	557	557	557	557
European and other currencies	3,432	2,201	2,285	2,285	2,285	2,244	2,244
Total Foreign-Currency Debt	7,526	4,928	4,721	5,221	5,722	6,081	6,081
Total Sovereign Guaranteed Debt	29,958	26,620	27,700	26,550	25,274	25,593	25,572

Use of JavaScript for core functions

"Plenty of useful information and you can get at it. OnMouseOver doesn’t work correctly, but well defined headings are useful."

JavaScript is often turned off by organisational firewalls and does not work with some adaptive technology. Therefore, sites must always supply text based alternatives to any JavaScript functions, particularly navigation or search facilities. This applies particularly to mission-critical information, though the use of JavaScript or other programming to operate peripheral functions (such as games) can also be annoying to those who are therefore excluded from the information when no text-based alternative is provided.

Use of pdfs alone for mission-critical information

"I couldn't answer this question, because the information was in a pdf document."

This document has already commented that pdfs must not be the exclusive mechanism for providing mission-critical information. Pdf documents are generally highly inaccessible to those with vision impairments or slow bandwidths, and somewhat inaccessible to those with mobility impairments.

Conversion software doesn’t work well on documents with coloured backgrounds, multiple columns, and other common design features – and screen-reading software also chokes on such visual effects. For example, dropped capitals or text on a coloured background may be treated as graphics and ignored, so a dropped capital "When" will read as "image hen". Tables may be read one complete column at a time, and multi-column pages may be read across consecutive lines of each column – across each first line of all columns, then each second line and so on.

In addition, images in pdfs are invisible to blind users unless the creator has tagged the file with text alternatives (which is an expensive undertaking, requiring more effort than conversion to html). People with low vision are likely to have problems with colour contrast in the pdf, as well as with finding their place in the document once it is blown up to a size that makes the text legible.

Pdf documents tend to be much larger than equivalent html documents, which reduces the speed of download and increases the amount of storage required. For example, one file we converted from pdf was 2,733 Kb. It became a series of interlinked html files of around 33Kb each, with a total document size (including graphics) of 449 Kb. Users with slow access are therefore prevented from accessing the original file, but can easily browse through the html version.

Pdfs created as a by-product of the print process and produced as a final press quality file should never be published on a website. High, press-quality resolution makes very large files that are slow to download and many retained all the colour images and background design work behind the text. These are difficult if not impossible for readers with low vision. In addition, they often reproduce poorly on a home printer and require extensive use of coloured inks. The e-government guidelines require that documents print well in monochrome on a home printer.

Text alternatives that do not provide true text equivalence

"Overall, this site was poor. While there is plenty of information on this site, labelling seems particularly bad."

When sites provide information in the form of images of graphs, organisational charts and tables, these images require text alternatives that provide the full information available to those that can see the image. A text alternatives that simply repeats the image caption, or that describes the nature of the image such as "Organisational chart" or "Consultation process", but provides little or no information about the image content is of no use to blind readers and excludes them from the information provided by the image. This is particularly annoying where the image is referred to in the text but the content of the image is not disclosed, as in: "Graph 7, below, shows the number of New Zealanders, by age group and location, with Internet access." The blind user knows that useful information is in the graph, but has no access to it.

"It was a real pain that the search box can only be accessed from the home page because you have to keep going back all the time."

Issues that make it difficult to access information

Technologies not used according to specification

"I had to double check whether I had the right information. It was that easy to find."

• Six sites achieved a perfect score on hypertext mark-up language (html) and cascading style sheet (css) validation. While many of the errors on other sites were trivial, non-compliant pages are likely to behave in unexpected ways in non-standard browsers or user agents, causing problems for users of alternative technologies such as screen-readers, hand-held personal organisers and mobile phones, voice-operated computers and so on.

Structure and design not separated

"Good to very good. Site not too cluttered, easy to navigate, big print size, good information."

• There were 15 sites with a perfect score for separation of structure and design. We commend those that use html structural mark-up correctly, and that keep all of their style instructions in separate style sheets. Note that css styles and html mark-up should not be used to give ordinary html text the visual impact of a heading or other structural element, as those who do not have visual information available to them are dependent on the structural markup for clues about the information hierarchy of the page. In-line styles are less useful than separate style sheets because the use of in-line styles can prevent users from applying their own styles to documents in order to improve legibility. They also markedly increase the cost of maintaining the site as each line of code must be changed when style changes are called for.

Insufficient contrast between text and background

"Yuk, yuk, yuk. Are they trying to destroy what vision I have left?"

Colour contrast makes a huge difference to the accessibility of a site – not only for people with vision impairments, but for many others who have less than 20:20 vision. Three sites in the survey achieved a perfect score. Note that good enlargement can often ameliorate the effects of poor contrast, particularly if the colours come from the colder end of the spectrum. We particularly commend site designers who have ensured good contrast of links to pages describing accessibility features, and to pages of Maori text.

While we did not, in this survey, evaluate sites for their use of "web-safe" colours we do recommend that non web-safe colours should be used for background underneath text. There are 216 colours in the web-safe palette compared to approximately 16 million in print. Understandably some designers prefer to use the corporate colours of an agency or not limit themselves to a small palette. However, the use of non-standard colours may result in colours breaking up into multiple small squares on devices such as mobile phones and personal organisers, and this may affect readability.

Information presented exclusively in proprietary non-html formats

"The site was not terrible and I might have marked it as satisfied if not for the poor contrast and navigation and unmarked pdfs with no alternative."

As we had already filtered out the sites that depended heavily on pdf, it is hardly surprising that 23 of the 56 sites achieved a perfect score for providing documents in html (or with accessibility features). Four sites had only html documents. As noted elsewhere in this report, pdfs cause problems for users with vision and mobility impairments, and may be totally inaccessible to those who are blind and/or on slow internet connections. We believe that the WCAG and e-government requirements to present all information in html, even if it is also presented in pdf, to be absolutely essential for user accessibility; html format is superior to pdf format – and, for that matter, to Word and other proprietary formats – for reasons of accessibility and file size.

Missing text alternatives or text alternatives that do not comply with guidelines

"Everything appears to be well labelled and the information is very useable."

Six sites achieved a perfect score for valid text alternatives, complying with all WCAG and e-government guidelines. Others were close. Text alternatives for all non-text information are essential for those who do not have access to the images perhaps because they are blind or are viewing sites with images turned off in order to conserve bandwidth. (Non-text information may include images, graphical representations of text (including symbols), image map regions, animations, applets and programmatic objects, ascii art, frames, scripts, images used as list bullets, spacers, graphical buttons, sounds, stand-alone audio files, audio tracks of video, and video itself.) A good example of the use of text alternatives to video can be seen on the New Plymouth District Council site. A video of a television advertisement offers an accessible transcript of the song on the video so deaf people can appreciate the presentation. It also has a text file describing the video images for those who are blind.

Fixed-sized screens and text

"Text enlarges well, but very poor contrast. One of the few sites I had to scroll back and forth to read across the screen."

Seventeen sites had excellent resizable text and screen width (fluid design) in both Internet Explorer and Firefox. Fluid design makes a huge difference to people with low vision and also to users of non-standard screens such as hand-held devices.

As noted above, menu text should not be provided in images, as images don’t change size even when all other text does so.

We also think it logical to ensure that links to pages describing accessibility features are not the smallest and the hardest to find on sites

Loss of information or functionality when scripting is disabled

"We have a touch of the orange and grey disease with the drop down menus. They are hard to use, particularly if I am tired and my hand-eye co-ordination gets worse."

Most government sites continue to work well, losing neither information nor functionality, when scripting is disabled.

Lack of access keys and ways to bypass the navigation menus

"While this site does include access keys, incorporating punctuation symbols into them is not useful."

We tested for either or both of access keys or a way to skip navigation to reach the main content (or, if the site was built the other way around, to skip the main content in order to reach the navigation). Implementing a standard set of access keys greatly improves accessibility for all users who prefer keyboard access. In particular, keyboard access is vital for those with impaired mobility or coordination. It is also very important to blind users. Even when access keys are implemented, skip navigation features still need to be provided, as users who don’t know the shortcut keys will have the ‘skip navigation’ link read to them and will be able to use it. Blind users and users of mobile devices particularly benefit from skip navigation features.

Language level requiring too many years of schooling

"The language level and added visual information could be looked at to increase the accessibility of this site."

In testing the language level, we took the approach that language pitched at readers with eight years of schooling would achieve a perfect score. Four sites met or exceeded this standard. The International Adult Literacy Survey (1996) 2 found that nearly 50% of adult New Zealanders and 72% of Maori and Pacific adults functioned at the two lowest levels of literacy measured by the survey (level 1 is "only very poor skills"; level 2 is "able to deal only with simple material").

Poorly marked-up forms

"Messy form that seemed to need a whole lot of unnecessary information."

Forms were found on 25 of the 56 sites. Two of these used all necessary mark-up to increase accessibility. See Appendix 2: Our methodology for a description of the necessary mark-up. Forms without mark-up are not accessible to blind users. Correct mark-up can also improve accessibility for cognitively and reading impaired users.

Beyond compliance to user testing

"VERY SATISFIED. Everything seemed to work well, well organised and easy to navigate."

At step 4, we tested the same 56 sites that were tested in step 3. Our user test panel was asked to answer five questions, rate the ease with which they were able to find the answers, and give an overall rating and some general comments on each site 3. The questions required users to find and use:

• a public accountability, consultation or other information document
• a feedback or other type of form
• contact details (some inter-agency sites did not have specific contact details; in this case we sent our testers to find equivalent information about the project the site represented).

For the first three questions, we tailored the question to the site, sending users to find information that we knew was available on the site. For the next two questions, we sent users to find information that is mandatory for government websites, whether or not it was present on the site in question.

• a purpose statement (this might be a specific statement or one gleaned from the information on the home page; the e-government guidelines require that such information is provided for all single-agency or business unit sites)
• an accessibility statement (or, if there is not specific accessibility statement, information about access keys or alternative ways to access the site. That such information is mandatory for government websites is indicated by the inclusion in the e-government guidelines of a required access key for accessing a list of access keys).

There was a strong relationship between the average result for the first three steps and the average result given by users in step 4. For 44 sites, there was small (around one or two point variance between these two averages. Of the remaining 12 cases, only one site was substantially more preferred by users than the compliance-related tests suggested, and this was because one tester gave it a much higher than average rating. The other 11 sites were substantially less preferred by testers than the compliance-related result suggested: in each case, one or more testers gave a zero score for one or more questions because information was impossible to find. In our view, this indicates that compliance testing is a good, but not perfect, proxy for accessibility and an important first step, but nothing can replace user testing.

Issues arising from user testing

"Yuk flashing stuff. Slightly stripy stuff in banner was not comfortable. The information is easy to find and written in good, easy to understand English. I think I could use this site."

Of the 56 sites tested, 48 presented at least one insurmountable barrier to at least one tester (that is, information was either impossible to access or could not be found at all). Of the remaining eight, only three provided all users with an average or better user experience for all five questions. These were the sites of the Bioethics Council, the E-government Unit and Studylink. The Electoral Commission site was the only site in the survey to receive a rating of Excellent (from one tester for one question).

1. See Appendix 6 - for those sites that can be "crawled" by the automated verifier, AccEase is able to produce detailed reports identifying the url of pages which have errors, the nature of the error and the line of code in which they will find the error.
2. The data was collected in 20 countries between 1994 and 1998 using nationally representative samples of the adult population aged 16 to 65.
3. Because of the volume of work required and to ensure consistency across sites, the questions given to our user test panel in step 4 were simple and straightforward. AccEase testers are usually given much more complex tests, requiring them to explore a site in much greater depth.