Difference between revisions of "Probation Board of Northern Ireland"
(→Discussion) |
|||
| (3 intermediate revisions by the same user not shown) | |||
| Line 29: | Line 29: | ||
===[[Designing a usability test]]=== | ===[[Designing a usability test]]=== | ||
This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper | This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper | ||
| − | questionnaire produced by the [http://www.pbni.org.uk/ PBNI]. | + | questionnaire, [http://www.pbni.org.uk/annex-a-questionnaire.pdf viewable online], produced by the [http://www.pbni.org.uk/ PBNI]. |
In partnership with the [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)], we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison. | In partnership with the [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)], we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison. | ||
| Line 52: | Line 52: | ||
We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening. | We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening. | ||
| − | For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the consultation questionnaire . Otherwise, there were few difficulties encountered. | + | For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the [http://www.pbni.org.uk/annex-a-questionnaire.pdf consultation questionnaire]. Otherwise, there were few difficulties encountered. |
===[[Usability test results]]=== | ===[[Usability test results]]=== | ||
| − | Using the test protocol, we compared the innovative map-based e-consultation technique with the conventional paper questionnaire. | + | Using the test protocol, we compared the innovative map-based e-consultation technique with the [http://www.pbni.org.uk/annex-a-questionnaire.pdf conventional paper questionnaire]. |
| − | The web map was better than the paper questionnaire on all but one criterion. The exception was the organization of information on the page. | + | The web map was better than the [http://www.pbni.org.uk/annex-a-questionnaire.pdf paper questionnaire] on all but one criterion. The exception was the organization of information on the page. |
| − | During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question. However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer. | + | During the tests, there were frequent complaints about the [http://www.pbni.org.uk/annex-a-questionnaire.pdf paper questionnaire]. Q5 was particularly hard to grasp. It reads like a university or A-level examination question. However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer. |
In contrast, there were few problems when using the on-line map. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use. | In contrast, there were few problems when using the on-line map. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use. | ||
===[[Discussion]]=== | ===[[Discussion]]=== | ||
| + | This case has shown it is possible to design interfaces that reduce the cognitive burden on consultees, compared to traditional questionnaires. This is consistent with a commonly stated principle of human-computer interface design: ''making the interface consistent with the ways the user thinks about the problem''. | ||
| + | |||
| + | Nevertheless, even our map still requires some literacy. sers need to be able to recognise place names and then type in short comments. For a less literate consultee the interface needs to provide information through more photographs and sound. | ||
| + | |||
| + | Although computer maps have been used in consultations on planning issues for over a decade, few consulting bodies have used them in small consultations, because of the cost of preparing the Geographic Information Systems (GIS). But now that lowcost GIS is easily available, from Google maps to the open source GRASS toolkit, such techniques are becoming more feasible. There have even been computer games (See demgames) designed to support e-democracy. | ||
Latest revision as of 19:37, 13 January 2008
Contents
Overview
E-consultation is often criticized on the grounds that different groups of people do not have equal access to electronic communications technologies. However, in our modern times, this criticism is a weak one.
E-consultation techniques are complementary to traditional approaches, and seek to improve participation.
In this case example we discuss what can be done to improve accessibility and usability for groups, who may be otherwise excluded from e-consultation.
Probation Board of Northern Ireland Consultation
In 2005, the PBNI were faced with consdierable structural change. They wished to consult widely on the changes to local probation office locations, and reporting centres, across Northern Ireland.
The PBNI sought opinions from several partner groups, political parties, councillors, community groups, and Individual offenders.
The PBNI drew learning from previous consultations. The most signficant challenge was engaging with offenders who had a low literacy, numeracy and other learning difficulties. However, it was recognised that, regardless, such stakeholders were very capable of texting using their mobile phones.
The project team suggested a number of technologies that could be used in their consultations with organisations.
PBNI consultation managers then spent some time over the autumn of 2005 on designing and planning the consultation, with the help of the Consultation Institute (represented by Stratagem in Ireland).
The Consultation Institute pointed out the disadvantages of consulting on only one out of nine options (i.e. ex-offenders). In addition, the PBNI needed approval from the its corporate managers, board, and the Northern Ireland Office, which created a long delay.
In the end, the PBNI ran a conventional consultation, without any e-consultation component, between 10 March and 2 June 2006. Rather than introducing a new technology for consulting with offenders during this major consultation, they agreed to work with us to do a usability test of an e-consultation technology with ex-offenders.
Designing an e-consultation interface for ex-offenders
Several technologies were considered usable by people with reading and writing difficulties. These used voice, mobile text, or graphical interfaces over the Internet. We chose a high-graphics, low text interface to explore the extent of use by ex-offenders. Also, since the PBNI consultation was about the location of probation offices, we chose to use a geographical, map-based interface.
Google Maps' (maps.google.co.uk) Applicaiton Programme Interface was used, which lets website developers to use their maps in their own applications.
Designing a usability test
This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper questionnaire, viewable online, produced by the PBNI.
In partnership with the Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO), we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison.
We designed a sequence of tasks that started with simple familiarization with the map-based site, then got progressively more involved as the testers gained confidence, until they were entering comments on the consultation topic.
We compared this with how much of the PBNI questionnaire could be filled in within 15 minutes.
We set up Camtasia Studio, a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.
The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. In addition, any comments entered via the keyboard were automatically stored on the server.
After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire, and in a focus group, to discuss their experiences.
Running the usability test
A structured programme guided each e-consultation session. However, there was a very low turnout to scheduled sessions.
A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the PBNI.
We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.
For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the consultation questionnaire. Otherwise, there were few difficulties encountered.
Usability test results
Using the test protocol, we compared the innovative map-based e-consultation technique with the conventional paper questionnaire.
The web map was better than the paper questionnaire on all but one criterion. The exception was the organization of information on the page.
During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question. However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.
In contrast, there were few problems when using the on-line map. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
Discussion
This case has shown it is possible to design interfaces that reduce the cognitive burden on consultees, compared to traditional questionnaires. This is consistent with a commonly stated principle of human-computer interface design: making the interface consistent with the ways the user thinks about the problem.
Nevertheless, even our map still requires some literacy. sers need to be able to recognise place names and then type in short comments. For a less literate consultee the interface needs to provide information through more photographs and sound.
Although computer maps have been used in consultations on planning issues for over a decade, few consulting bodies have used them in small consultations, because of the cost of preparing the Geographic Information Systems (GIS). But now that lowcost GIS is easily available, from Google maps to the open source GRASS toolkit, such techniques are becoming more feasible. There have even been computer games (See demgames) designed to support e-democracy.
