Museums and the Web

An annual conference exploring the social, cultural, design, technological, economic, and organizational issues of culture, science and heritage on-line.

You are hereevaluation

evaluation


silviaf's picture

Exploring the Relationship between Visitor Motivation and Engagement in Online Museum Audiences

Silvia Filippini Fantoni, Rob Stein, Gray Bowman, Indianapolis Museum of Art, USA

http://www.imamuseum.org/ 

Abstract

In this paper, the authors will describe the rationale, methodology, and results of a series of studies that have been conducted with visitors to the Indianapolis Museum of Art website. The objective of the studies is to better understand people’s motivation for visiting the site and whether this motivation has an impact on the way they engage online. The hope is that these results will provide a reference dataset, and a replicable model for other museums that are interested in better understanding their online audience and in conducting similar studies for their own web efforts.

Natasha Waterson's picture

Delightfully Lost: A New Kind of Wayfinding at Kew

Natasha Waterson and Mike Saunders, Royal Botanic Gardens, Kew, United Kingdom

www.kew.org

Abstract

In October 2010, Kew Gardens commissioned an in-depth study of visitors’ motivations and information needs around its 300-acre site, with the express aim that it should guide the development of new mobile apps. The work involved over 1,500 visitor-tracking observations, 350 mini-interviews, 200 detailed exit interviews, and 85 fulfilment maps; and gave Kew an incredibly useful insight into its visitors’ wants, needs, and resulting behaviours.

It turns out that most Kew visitors have social, emotional, and spiritual, rather than intellectual, motivations during their time here. They do not come hoping to find out more, and they don’t want or need to know precisely where they are all the time. In fact, they love the sense of unguided exploration and the serendipitous discoveries they make at Kew—they want to become “delightfully lost.”

dannybirchall's picture

Levelling Up: Towards Best Practice in Evaluating Museum Games

Danny Birchall and Martha Henson Wellcome Trust; Alexandra Burch and Daniel Evans, Science Museum, UK; Kate Haley Goldman, National Center for Interactive Learning at the Space Science Center, USA

Abstract

Museums make games because games can provide compelling educational engagement with museum themes and content, and the market for games is enormous. Truly understanding whether games are achieving your goals requires evaluation. In this paper, we identify the kind of games that museums make and use case studies of our own casual games to look at the benefits and means of evaluation. Beginning by identifying different kinds of evaluation within the broad framework of formative and summative practices, we suggest ways to plan an evaluation strategy and set objectives for your game. We then look in detail at evaluation methods: paper and wireframe testing, play-testing, soft launching, Google Analytics, surveys, and analysing responses “in the wild.” While we draw on our own experience for examples of best practice, we recognize that this is an area in which everyone has a lot to learn, and we conclude by suggesting some tactics for sharing knowledge across the museums’ sector.

What do we really know about social media in museums?

Keywords: 
social media
Keywords: 
twitter
Keywords: 
Facebook
Keywords: 
metrics
Keywords: 
best practices
Keywords: 
evaluation
Abstract: 

With years of collective experience in social media engagement, many museums are stopping to ask themselves: what have we really learned about the audiences we are connecting with? Have we formed more meaningful relationships with our community and in the process, are we better serving the missions of our institutions?

Type: 
Professional Forum - discuss an issue

Delivering mobile from the ground-up: exploring the approach

Keywords: 
mobile
Keywords: 
digital
Keywords: 
framework
Keywords: 
evaluation
Keywords: 
interpretation
Keywords: 
access
Abstract: 

The challenges of delivering a meaningful ‘mobile’ experience for an institution are not always technology-related. Securing cross-departmental stakeholder investment and ensuring alignment with the institution’s core mission have been key for National Museums Scotland and National Museum Wales in developing a framework for their mobile programme.

Type: 
Paper - in formal session

Levelling up: Towards best practice in evaluating museum games

Keywords: 
games
Keywords: 
evaluation
Keywords: 
best practice
Keywords: 
interaction
Keywords: 
audiences
Keywords: 
gamification
Abstract: 

Games, and game-like experiences, are becoming increasingly important to museums seeking to engage new audiences and to provide deeper engagement with existing audiences. Demonstrating the value of commissioning and producing games as a core activity of museums is imperative.

Type: 
Paper - in formal session

Delightfully lost: a new kind of wayfinding at Kew

Keywords: 
mobile
Keywords: 
audience research
Keywords: 
apps
Keywords: 
evaluation
Keywords: 
science
Keywords: 
wayfinding
Keywords: 
botanic garden
Abstract: 

In October 2010, Kew Gardens commissioned an in-depth study of visitors’ motivations and information needs around its 300 acre site, with the express aim that it should guide the development of new mobile apps.

Type: 
Paper - in formal session

Museums in perspective: share and discuss daily data about museums and their audiences.

Keywords: 
evaluation
Keywords: 
social media
Keywords: 
analytics
Keywords: 
participation
Keywords: 
engagement
Keywords: 
community
Keywords: 
strategy
Abstract: 

It is common practice among museums to publish quarterly or annual reports that include the museum attendance figures, online visits and more recently information about their social network activities. These figures are generally accepted as indicators of the effectiveness of museums activities both online and offline.

Type: 
Paper - in formal session
Authors: 

Using QR codes, mobile apps and wifi tracking data to understand visitor behaviour in exhibitions

Keywords: 
QR codes
Keywords: 
mobile
Keywords: 
evaluation
Keywords: 
r&d
Abstract: 

In the last twelve months QR codes have reached a new level of hype within the museum community - increasing as their proliferation in street advertising in the USA and Europe has increased.

Type: 
Paper - in formal session

Let's Get Real

Keywords: 
evaluation
Keywords: 
success measures
Keywords: 
Failure
Keywords: 
analysis
Keywords: 
action research
Abstract: 

‘Let’s Get Real: How to evaluate online success?’ is a report published by Culture24 in September 2011 at their UK conference of the same name.

Type: 
Professional Forum - discuss an issue
jdalton's picture

Rethinking Evaluation Metrics in Light of Flickr Commons

Paula Bray, and Sebastian Chan, Powerhouse Museum, Australi; Joseph Dalton, New York Public Library, USA; Dianne Dietrich, Cornell University Library, USA; Effie Kapsalis, Smithsonian Institution Archives, USA; Michelle Springer and Helena Zinkham, Library of Congress, USA

Abstract

In the past several years, cultural heritage institutions, including archives, libraries, and museums, have been placing their collections in Web spaces designed for collaboration and communication. Flickr Commons is one example of a highly visible space where cultural heritage institutions have partnered with a popular social networking site to provide greater discovery to, access of, and opportunities to interact with image collections on a large scale. It is important to understand how to measure the impact of these kinds of projects. Traditional metrics, including visit counts, tell only part of the story: much more nuanced information is often found in comments, notes, tags, and other information contributed by the user community. This paper will examine how several institutions on Flickr Commons - the Library of Congress, the Powerhouse Museum, the Smithsonian, New York Public Library, and Cornell University Library - are navigating the concept of evaluation in an emerging arena where compelling statistics are often qualitative, difficult to gather, and ever-changing.

Keywords: Flickr, metrics, evaluation, crowdsourcing, statistics, images

JaneFinnis's picture

How to Evaluate Online Success? A New Piece of Action Research

Jane Finnis, Culture24, United Kingdom; Seb Chan, Powerhouse Museum, Australia; and Rachel Clements, Central Saint Martins, United Kingdom

Abstract

How to measure and define the success of cultural websites and online services is a problem common to all parts of the cultural sector – museums, archives, galleries, arts organizations, libraries and publishers. Worse, funding agencies and government departments lack the expertise to offer guidelines or set standards. The result is a confusing mixture of statistics and reporting that reveals little about genuine user satisfaction.

In mid-2010, a group of 24 organizations and agencies from across the cultural heritage and arts sector in the UK began collaborating on a piece of action research addressing the issues around 'evaluating online success'. Coordinated by Culture24 and working with Seb Chan, the groups of institutions have been developing a framework and best practice guide for measuring online projects both on organizational websites and those presented through social media channels.

Keywords: evaluation, metrics, evaluating cultural websites