Skip to main content

Museums and the Web

An annual conference exploring the social, cultural, design, technological, economic, and organizational issues of culture, science and heritage on-line.

Levelling up: Towards best practice in evaluating museum games

Abstract

Games, and game-like experiences, are becoming increasingly important to museums seeking to engage new audiences and to provide deeper engagement with existing audiences. Demonstrating the value of commissioning and producing games as a core activity of museums is imperative. At the same time, the sector is encountering a new, cold wave of realism in evaluating its online presences. Old canards about new audiences are being thrown out the window, and a more rigorous focus is being applied to the business of gaining and measuring audience attention.

There is a risk, however, that these engagement metrics, oriented towards page-based content and measurable proxies for interaction in social media are poorly suited for the kind of immersive, deep engagement that good games can offer. This paper will identify and discuss best practice in the field of museum games evaluation in order that such evaluation might also move towards standardisation and wide acceptance, with the end result of improving games in our field.

Most museum games are web-based, and have an explicit educational intent. Examples from our own experience include High Tea, Memory (Wellcome Collection), Launchball, Rizk and Thingdom (Science Museum). These types of games will be the focus of this paper, though we expect the methodology to have applications across other gaming activities.

Drawing from our own institutional practices, across the cultural heritage and informal learning sectors, and looking at the area of ‘persuasive’ games in general, we will focus on initial objective setting and three areas of evaluation:

1. Quantitative evaluation: What are the best tools and methods to measure overall interactions, in-game interactions, and profile game achievement against audience segments?

2. Qualitative evaluation: How can we use interviews, surveys, and focus groups to gain a deeper understanding of player motivation and experience and are they sufficient? Can we find more useful and rich ‘in the wild’ where people freely discuss games?  If we choose that route, what are the best approaches to gathering and analyzing that data?

3. Pre vs Post-release evaluation: How reliable or valid are expert reviews and user testing of games prior to release? How do these results compare with those from summative evaluations?

In each of these areas, we will also consider aligning evaluation approaches for different types of games: autonomous games, and gamified or gameful experiences. Technical appendices will describe both quantitative and qualitative methods.

Type: 

Paper - in formal session

Authors

dannybirchall's picture
Danny Birchall edits the Wellcome Collection website, www.wellcomecollection.org. He has been managing cultural heritage websites for 13 years, previously at the British Film Institute and the Institute of Contemporary Arts. He is currently studying for a Masters degree in Museum Cultures at...
Kate Haley Goldman's picture
Kate Haley Goldman recently joined the Senior Staff of the Center for Interactive Learning, a non-profit founded by the Space Science Institute. Prior to taking this position as Director of Learning Research and Evaluation, she spent ten years as a Senior Research Associate at the Institute for...
Daniel Evans's picture
I have been Head of Web at the Science Museum since 2007. During that time we have developed a wide range of learning-based online games including Launchball, Thingdom and Rizk. Before joining the Science Museum I worked in e-commerce and the book trade for Waterstone's Online.
marthasadie's picture
I am a multimedia producer for the Wellcome Trust, creating films and interactive projects around our research, events, exhibitions and collections. My main interest is in gaming and play and how this can be used to make a subject engaging and, above all, fun. I blog at http://marthasadie....