There were five general problems that testers’ experienced in using sites:

• documents only in PDF format
• accessibility statements are inadequate and inconsistent
• search facilities give unhelpful results
• home page is complex, and has no ‘skip links’
• inconsistent feedback mechanisms.

Documents only in PDF format

The Web Guidelines (6.3.3) says “The primary format for all content available on government websites must be HTML.”
Documents that are only available in PDF format create a number of accessibility barriers:

• they are often large and take considerable time to download via slow connections
• the whole document has to be downloaded, even if the user only wants information on a single page
• there can be difficulties if there are versioning differences between the published document and the user’s Acrobat reader software
• screen reader software has difficulty with early versions of PDF documents
• low vision users who magnify (zoom) the text to high magnification find the text in PDF documents unreadable, as the text does not wrap within available screen width and requires horizontal scrolling
• there can be poor colour contrast between text and backgrounds that make reading difficult, as PDFs are often the end result of a print publication and not designed specifically for online viewing
• if not properly tagged and bookmarked, the documents are not easily navigable
• some users may have difficulty properly installing and using the Acrobat Reader plug-in software and, if so, would be unable to access PDF documents at all.

While there has been an increase in the number of sites with documents in alternative formats, this survey found that there were sites with many public documents available only in PDF format.

On one website, an email address that should be publicly available could only be accessed through a large and inaccessible (untagged) PDF document.

Some tester quotes on experiences with PDFs:

• Blind tester - “Documents were all easy to view and it was good to see PDF not being the only way to view these.”
• Blind tester - “It was good to see the option of Word or PDF documents for downloading!”
• Blind tester - “There was no mention that the report would be a PDF. I'm glad I only opened the summary report and not the full one as this would have taken much much longer to open using Adobe.”

Accessibility statements are inadequate and inconsistent

Web Guidelines state that an accessibility statement and its implementation should be a given. However, some sites place the accessibility statement in what were, to the testers, inaccessible places. In one case, the terminology was changed and not immediately recognised by the tester.

The purpose of an accessibility statement is to assist users who need to know accessibility features of a site, such as access keys. It is not sufficient to be ‘compliant’ by having one somewhere. The accessibility statement needs to be readily accessible and useful.

Some tester quotes on experiences with accessibility statements:

• Blind tester - “Found on the ‘about’ page. Fairly easy to find, though access info not under a heading but just under a link, so couldn't cycle through headings to quickly find it. I just realised that the contact info from previous task is buried one layer deeper with its own contact us link. Might be better to have access and contact info all on the same page, and take away most of those 50 links, as currently involves a bit too much chasing around.”
• Deaf tester - “'Site accessibility' on homepage is in small fonts at the bottom... my eyes are good but [they] could make the font bigger.”
• Blind tester – “Access info found in the ‘about this site’ page under help. This would not be obvious to many people, certainly not placing the access info under ‘help’. A heading such as ‘accessibility’ would be clearer. I wonder why this accessibility link is not available on the front page. Jaws reads the alt 0 access help link as: ‘common slash help alt plus 0’. The word common makes no sense to me. None of the alt combination short cut keys worked consistently, though they worked sometimes. I cannot work out a pattern.”
• Blind tester – “It was nice to see this where it should be.”

Search facilities give unhelpful results

Search facilities used by testers were often found to be inadequate. Some testers relied on using the search facility to find information when the site navigation was difficult.

A good search engine enhances the value of a site for testers. However, testers’ experience showed that a simple word or phrase search, without any other contextual information to filter out irrelevant results, typically produces large result sets. The desired page may or may not have been near the beginning of the result list. Often, the result list did not give enough information for the user to confidently decide which page to follow.

The main issues with inadequate search were:

• search results fail to include pages that exist on the site that match the search argument
• ‘relevance’ algorithms that score each search result produce meaningless information
• search arguments are interpreted strictly, but not how the user intended. For example, ‘statement of intent’ (entered without quote marks) is treated as find any page that contains all of the words rather than how the user intended, which was to find that exact phrase. It would be preferable for search engines to firstly assume that a phrase is being searched for and rank the exact phrase results higher than results that contain all the words
• some advanced search functions, which should assist the user, are too complex to follow. Some use unexplained jargon in the classifications or search areas that make it difficult for users to know what they are really selecting
• some results displays do not show again what the user was searching for. This is especially unhelpful when the result is ‘no matches’.

Some tester quotes on experiences with search facilities:

• Deaf tester - “The report was easy to find because there was a search engine on homepage, so I just typed in the (title of the report). I then realised there were sub sections in the menu bar and clicked on recommendations. If there was no search engine, then I would be absolutely clueless where to find this - thank god for the search engine :)”
• Blind tester - “I could not find this information. I did a search [on the information I was trying to find] and got 10 pages of results most of which were PDF documents. Nothing on the front page seemed relevant.”
• Blind tester - “A search for the keywords produced 237 results, and I could not find any links or information on the front page which seemed to be relevant. Looked in help and frequently asked questions, then followed the site map link. Couldn't find it on the ‘about’ page either.”
• Mobility impaired tester - “Again, no search engine on the site which makes it difficult to find any selection. A search of the entire site took some time, much more than the time allotted.”

Home page is complex, and has no ‘skip links’

Some home pages were very busy and confusing, with too many links. This was difficult for blind people and people with reading difficulties.

Busy homepages were a particular problem for blind testers if the ‘skip links’ function was not correctly placed and there were many links to go through before getting to the page content.

A tester quote on experiences with website navigation: Blind tester - “Skip navigation link is very useful. Most of the access keys don't work at all.”

Inconsistent feedback mechanisms

One of the generic tasks that user testers had to complete on every site was to find the feedback page, and give feedback using the function available.

This task was included for two reasons:

• it is a task that would be expected to be used on any site
• it allows for an accessibility test of an HTML form.

Testers found inconsistencies in feedback mechanisms available. The options that were presented on how to give feedback were forms, of varying accessibility, and email addresses. Sites also varied in where and how feedback mechanisms were located on a site.

Some tester quotes on experiences with giving feedback:

• Low vision tester - “The feedback response is a telephone number, email or fax. I did not do this task with it being in an email format. If the feedback had have been in a form format, I would’ve done this and used Accease's email. It was not hard to find this link.”
• Blind tester – “Very surprising since everything else about this site seems spot on. The contact link simply had phone, postal and email address but no invitation to comment on the site.”