OPTIMAL-EM Framework

This framework was the backbone of my PhD, and is designed to optimise the web accessibility evaluation process. Web accessibility evaluation is often performed through a combination of user evaluation and automated tools, and is supported by methodologies such as the Web Content Accessibility Guidelines (WCAG) and the Website Accessibility Conformance Evaluation Methodology (WCAG-EM). Whilst there is prior art for automated and manual accessibility evaluation, page selection is based on heuristics and understanding, meaning that pages are generally selected on an ad-hoc basis. This framework aims to address this whilst optimising the web accessibility evaluation process by providing auditors with pages that have been statistically analysed and systematically selected, and we assert that this will reduce the risk of discrepancies in the results and lower the human cost of the evaluation.

The framework is split into six stages, and consists of three quantitative metrics:

1. Coverage
2. Complexity
3. Accessibility

And three qualitative metrics:

1. Representativeness
2. Popularity
3. Freshness.

These metrics are adapted from census studies; research areas such as crawling; web accessibility metrics; the Unified Web Evaluation Methodology (UWEM), and the WCAG-EM. Several of these factors relate to heterogeneity, the property of having a diverse set of pages in our population.

For more information, this W4A short paper addresses why the framework was created, and introduces the framework.

This WWW ‘23 paper expands on the framework, applying it to web page representations.

The framework is also outlined in this W4A paper