How we assess the quality of information

We used a checklist of questions to assess the quality of the web pages we link to from this site.

Usability Questions:

  1. Is the site accessible without a login?
  2. Does the site conform to web Accessibility standards?
  3. Is the site design clear and transparent?
  4. Is the site design consistent from one page to another?
  5. Can users find what they need on the site?
  6. Is the format of information clear and appropriate for the audience?

Reliability Questions:

  1. Is it clear who has developed the web site and what their objectives are?
  2. Does the site report a robust quality control procedure?
  3. Is the page content checked by an expert?
  4. Is the page updated regularly?
  5. Does the page cite relevant sources where appropriate?

This scheme was developed after an extensive literature review on web site and health information quality.

How are the scores worked out?

We scored each page for each question on a scale from zero to 3 where:
0 = Never
1 = Sometimes
2 = Mostly
3 = Always

We double the score for Reliability and add this to Usability, giving a score of 0 to 48. This is converted to a percentage and displayed on the Search Results table.

Is the scale valid?

We validated the scale by doing a blind, independent comparison of two different assessors’ scores of the same set of sites. This showed excellent inter-rater reliability (P < 0.00001).It suggests that our quality scoring system has internal validity.

To assess external validity, i.e. the extent to which the scores correspond to actually is better quality information, we have conducted a series of evaluations since 2005, involving patients, carers and health professionals.

These studies have consistently found that the search results produced by UK Prostate Link are superior (in terms both of quality and of relevance) to the Google search engine.

  • Get in touch if you would like to find out more about these evaluations.

Who does the assessments?

All links are assessed by a Minervation staff member. As noted below, we tested the scheme to see if different assessors got the same results. You can read some background information about the assessors here.

How can I improve my website's score?

An ancillary purpose of the project is to help information providers improve the quality of their information. Our two main suggestions for improving scores are:

  • Transparency
    Our scheme places considerable emphasis on the transparency of methods used to create the information. Make sure your site has a clear statement of how the information was put together, what sources you used to research it, and how it is checked for accuracy.
  • Clarity
    Reading from a computer is different from reading from paper. Help your users to scan information by using subheadings; space the information out, don't use long, dense paragraphs; minimise jargon and avoid complex sentences.

We can provide more details of individual assessments on request.