When is the next
Museums and the Web?


Archives & Museum Informatics
158 Lee Avenue
Toronto, Ontario
M4E 2P3 Canada
info@ archimuse.com
www.archimuse.com

Search A&MI

Join our Mailing List.
Privacy.

Copyright
Archives & Museum Informatics
2000

Meta-centers: do they work and what might the future hold ñ A case study of Australian Museums On-line.

Kevin Sumption, Powerhouse Museum, Australia

Abstract

The Australian Museums On-Line (AMOL) project is six years old. During this time it has established a reputation as both an innovative and highly popular, Internet-based gateway to Australian museums and galleries. A recent study showed that from February 1998 to July 1999, AMOL received over two and a half million hits from 138,380 users, averaging out at more than 270 unique users a day. As well as individual users AMOL has also attracted a growing number of significant museum and gallery collections. But what have been the tangible and quantifiable benefits of this use and representation? Has the AMOL meta-center approach increased access to cultural resources and communications tools, in ways that have meaningfully benefit both museum professionals and the general public? The AMOL project team was mindful that it was time such questions were answered. Particularly when the number of Australian museums developing their own expensive OPACs grows and they begin to question the usefulness of a meta-center approach. In an effort to examine these issues and chart a way forward, AMOL commissioned the first of two summative evaluations in 1999. In this paper I intend looking at the results of the first quantitative evaluation and development of the second qualitative evaluation, as well discuss a range of marketing, promotional, educational and technological strategies the process has spurred.

Cyberspace: a consensual hallucination experienced daily by billions of legitimate operators in every nation, by children being taught mathematical concepts ... A graphic representation of data abstracted from the banks of every computer in the human system.
William Gibson, Neuromancer, 1984. p67

The 'consensual hallucination,' imagined by William Gibson is now a reality. Over 150 million people last year used the internet. Far from being a mere teaching medium, the Internet has evolved into a ubiquitous publishing, communication, design and research tool beyond the bounds of any single government to control. Over and above the 60,000 interacting networks that constitute the Internet, this epiphenomenon cannot physically be perceived or meaningfully located in space or time. In cultural terms the Internet is truly a place where disembodied souls find freedom in a myriad of chat rooms, multi-user dimensions and news groups (Wertheim, 1999). Last year over three million Australians partook of this consensual hallucination and a further 3.9 million (ABS, 1999) visited a museum. Although to date no single study has established a correlation between museum visitation and Internet use, the Australian heritage sector has reacted with nearly 25 percent of all private and public museums now operating a website. In an effort to provide a gateway to these, as well as information on Australian museums and their collections, Australian Museums On Line was formed in 1995, the same year private access to the Internet became widespread in Australia.

Australian Museums On Line (AMOL) is a joint Commonwealth and State funded website that has established a reputation, both nationally and internationally, as a highly innovative gateway to Australian museums and galleries. Its mix of information products, specially created for both museum workers and the public, gives the site a wonderful eclecticism. Collection descriptions, contact details and images of nearly 1100 Australian museums, sit alongside half a million item level collection records, discussion forums and a variety of on-line journals. Initially proposed by the Heritage Collection Council as an electronic register of moveable cultural heritage material, the AMOL project has attempted to keep pace with changing public and professional needs, through a series of redesigns. In six years AMOL has undergone three major redesigns, each of which has progressively thrown off the shackles of a centralised information repository, and embraced the potential of the meta-center. This metamorphosis has been gradual and not surprisingly parallels that experienced by many museums, as they made the sometimes perilous journey from collection to information centered institutions in the 1990s.

The AMOL project emerged as a direct response to the information explosion experienced by many Australian museums in the nineties, as first collection records and interactives were digitised, then finally the Internet was embraced. This rapid adoption of information technology tools by some museums was inadvertently ëpushedí in Australia by government moves towards greater fiscal self-sufficiency (Sumption, 1999). The introduction of admission charges by many Australian museums in the early 1990s, required museums to more closely align themselves with market forces. At the same time financial pressures to be more cost effective ushered in a period of stringent appraisal of some of the museumsí more costly functions, like collection care and management. So itís not surprising that the 1990s witnessed a rapid up-take of computerised collection management systems. The unanticipated effects of this technological assimilation was the creation of a myriad of digital, text, graphic, photographic and video databases. And whilst technological determinism and fiscal expediency ëpushedí the process of digitisation in museums, the popular explosion of the Internet after 1995 in Australia, provided the ëpullí to make these resources publicly available.

The Internetís popularity in museums is no accident and is just as much a consequence of museumsí new visitor orientation, as it is technological appetite. Superficially its rapid adoption suggests museums were from the outset caught up in the techno-evangelism of the Infopreneurials and Digerati (Robotham,1996) This small select group have for some years espoused a simple but persuasive rhetoric: the Internet will change the way we live, work, play and even think; it will deliver us from the routine of office work via telecommuting and unleash us from the one dimensional interface of traditional educational mediums. However on closer examination it is clear many first generation museum websites, including AMOL, adopted a less ambitious brief, deciding instead to use the medium to market the real museum and its real collections.

Three emerging meta-center typologies

Consequently the vast majority of first generation museum websites were conservative in both the presentation and quantity of information they were willing to make public. What little they did was invariably centralised, highly abridged and only browsable. However in the last two years the technological and political landscape has changed markedly, particularly here in Australia. A voter backlash against the marginalisation of regional Australia prompted many governments and museums to reinvigorate outreach programs. Consequently this decentralisation impetus, together with the emergence of new metadata standards like Dublin Core, spurred the development of more user-centred website typologies. With its ability to create a more comprehensive information environment via interoperable databases and known item or explanatory searching, the meta-center model is now gradually being adopted in Australia, North American and Europe. But because of cost and technical considerations, heritage sector meta-centers are still rare, however even in these early stages I have observed a number of typologies emerging.

Most common are what I term institutional meta-centers. These are the websites of organisations like the Natural History Museum (London), National Library of Australia and Australian War Memorial. Characteristically each of these has extensive collection holdings, numbering in the hundreds of thousands, or even millions. Importantly each has over a number of years developed a close working relationship with a vast research fraternity, for whom most provide in-house research facilities. Consequently most of these institutional meta-centers are in the process of adopting metadata solutions to create database driven websites that will ultimately act as surrogates of corporeal research centres.

Less common are public meta-centres. These are the websites created for, or by, heritage sector advocacy, administrative, coordinating or governance bodies. Institutions like the Institute of Museum and Library Services, the American Association of Museums and the Museum Documentation Association (UK). These agencies fund and develop public meta-centers like The 24 HourMuseum (www.24hourmuseum.org.uk) and American Strategies (www.americanstrategy.org), to increase public awareness of real museums, their exhibitions and to a lesser extent collections. However these initiatives are still extensively experimental and are primarily a response to the perceived increased use of the web by the museum going public. Again as there are few examples of this typology it is difficult to discern a definitive set of characteristics. However one of the most advanced is the 24HourMuseum. Designed as a public gateway to information on British museums, the sites developers have used a sophisticated combination of e-zine format, heritage trails, collection level and institutional contact databases, as well as childrenís pages. The result is a fun, entertaining and informative site with broad appeal.

The third emerging meta-center typology I have identified is that of the Professional meta-center. Here the term ëprofessionalí describes the primary audience for whom these sites were designed. In the case of AMOL and CHIN, those working directly in paid or unpaid heritage positions. This is not to suggest that professional meta-centers are not public-friendly, however there is little doubt that the information architecture, search format and content is intended primarily for users from the heritage sector.

The establishment of organisations like CIMI (the Consortium for the Computer Interchange of Museum Information) in the early 1990s is testimony to the heritage sectorís awareness of the inherit benefits of meta-centers and allied enabling metadata standards. In the proceeding years projects like CIMIís Dublin Core metadata test bed project, and more recently EMIN (European Museumsí Information Institute), have all actively sought to develop standards for museums and galleries. Along with organisations like CHIN (Canadian Heritage Information Network), the MDA (Museum Documentation Association) and AHDS (Arts and Humanities Data Service), AMOL has been an active participant in many of these projects. More importantly AMOL, like CHIN, has also been an early adopter of recommendations and in turn pioneered this, the most ambitious of meta-center typologies.

Meta-centers: Artefactual versus Interactive Multimedia Simulacrum

Whilst I acknowledge that each of the three typologies deploys a wide range of products to service a diverse clientele, they also have a remarkably similar way of describing and presenting the activities of museums. In essence each typology is dominated by what I refer to as an artifactual simulacrum. That is they characteristically discuss museums, their collections and exhibitions, via images of artefacts and item level descriptions. Even public meta-centers have adopted this approach as a very ëpragmaticí way of developing online exhibitions. Typical is the American Strategies, Northern Great Plains online story. (http://www.americanstrategy.org/america/home1.html). Rather than a story with a complex narrative structure similar to that of a book, exhibition or interactive multimedia (IMM), the exhibition is made up of a discrete collection of 900 individual photographs and descriptors. Its authors have neither sought to broadly contextualise the set of images and descriptions, or even related one to another. Instead the story has simply usurped the presentational structure of a database. In so doing the story has avoided using contextual information systems like interactive multimedia (IMM), audiovisuals, sub-theme labels etc.

I believe this artifactual simulacrum is a natural reaction to the political and museological concerns of the last decade when for many simply making the contents of one's often unseen storehouse visible was a priority. After all there is little doubt that the web can release individual museums from the economic and spatial constraints that limit both collecting and display in the real world. Even in my own museum, the Powerhouse Museum, it is estimated that as little as 3% of the total of 380,000 objects in the collection are on display.

However I believe that if the three meta-center typologies are to keep pace with user expectations we need to know that this artifactual simulacrum is indeed what both the public and professional sector desire. This is a critical question because we know that in the real museum, visitors are not content with such a fractured and decontextualised approach. After all if visitors and museum professionals were merely concerned with passively viewing artefacts, then there would have been little need for the complex interpretive and presentational techniques that have come to dominate today's modern museum. It goes without saying then that if the heritage sector was committed to using the web as an extension of contemporary practice as expressed in current exhibition media, then meta-centers would equally be populated by a plethora of dynamic interactive experiences, just like real museums. In essence they would equally embrace a model based around the dynamic and two-way potential of interactive multimedia (interactive simulacrum) as they would the artefact.

However this argument presupposes that all meta-center users have the same needs as visitors to the real museum. Whatís more it presupposes users are motivated to visit a meta-center because it provides an experience analogous to the real museum. However even if this motivation were true, we know that the social and cognitive complexity involved with a visit to a real museum, cannot in its entirety, be currently replicated on-line. As Falk and Dierking observed when defining the parameters of an effective educational visit, each is determined by the visitor's unique personal, physical and social attributes and contexts. In turn each one of these attributes is mediated by senses activated by the material reality of the visitor's social and physical surroundings. Thus the direct correlation of real with virtual experiences is questionable. But where does this leave us? Without definitive feedback on user experiences it is hard to gauge the effectiveness of current artefactual meta-center simulacrum. Thus what we decided to do at AMOL, was conduct an evaluation that could help us determine the effectiveness of our current approach.

The need for a summative evaluation

The adoption of a meta-center approach by AMOL has been a slow process. After all as Neimanis and Geber (1999) point out the journey from centralised information repository to meta-center, involves constantly re-building the network of connections among site components as well as communication relationships between these resources and users. Since its launch in 1995 AMOL has not shirked this responsibility, and has regularly rebuilt part or all of its architecture in-line with changing technological, museological as well as user expectations. An important milestone in this journey was the redesign and re-launch of AMOL in late 1998. The redesign had four principle objectives:

• To streamline information into three sections;
• To improve navigation and interface functionality;
• To increase contextual information on the site via a series of stories;
• To begin the implementation of Dublin Core metadata standards.

This redesign took two months and eventually the AMOL site was re-launched in October 1998. Exactly nine months later the On Line Working Party (OLWP), responsible for overseeing AMOL, decided to initiate a summative evaluation. Critically the evaluation was seen as a strategic means of exploring new directions for the coming eighteen months, a critical time for the project for a variety of reasons. Itís a period when the efficacy of the Heritage Collection Council (HCC) and its projects, including AMOL, will be closely examined. To facilitate informed discussion it was decided thereafter that we needed to access performance data. Ideally this data needed to be of both a qualitative and quantitative nature and as such a two-stage summative evaluation, focusing on three areas was initiated:

1. Participation rates

To ascertain whether or not any of the new features had encouraged new or repeat visitation we needed to measure and compare the siteís usage prior to and after the re-design. In particular we wished to examine the effectiveness of the more ëgraphicí interface and new information architecture and whether or not they reduced user frustration and fatigue.

2. User profiles

As a professional meta-center AMOL had been principally conceived to service the needs of those working in the Australian museum sector. So it was important to establish that this group was actually using the site. Just as importantly we also needed to know who else might. If as we suspected inbound international and interstate tourists were regular users, or indeed students and educators, a summative evaluation would also help us identify which AMOL resources they preferred. This data would then allow us to examine the feasibility of supplementing the site with additional education or tourist specific resources.

3. User experience

Detailed, qualitative information on program satisfaction was also critical for the purpose of improving certain AMOL services. We needed to be able to gauge the current level of satisfaction with new programs like the Open Museum Journal and costly resources like the contextual stories. There was also a need for information on program quality; an issue which from time to time sparked heated debate on AMOLís Listserve AMF (Australian Museums Forum).

Decoding the visitor

Whilst it has been relatively easy to evaluate the success, or otherwise, of some of AMOLs non-web programs via feedback from AMF subscribers and workshop and grant program assessments, evaluation of the usage and effectiveness of the website was more daunting. This is because of a lack of an appropriate, and methodologically rigorous, model around which to formulate an evaluation. So we turned to the real museum. Traditionally a summative evaluation of a museum program involves the cross-referencing of visitor tracking studies with focus group discussions, or visitor questionnaires. Likewise we decided upon a two-stage model that used both positivist and postpositivist approaches to collect and analyze data. The first, a positivist approach, analysed statistical data gleaned over a seventeen-month period from the log file reporter WebTrends. These log file tools allow for the compilation of data sets that, when analysed, can aid in the development of participation trend models built around country of origin, referring site and active organisation statistics. Similarly these tools can also provide important indicators of user experience, through their logs of average time per session, entry points, top path and most accessed directory data.

These log file analysis tools are positivist as they focus on the quantification of outcomes and thus on measurable performance indicators, effectiveness and efficiency. As Darrell Caulley (1992) reminds us, this kind of reporting of human behaviour has an important function as it responds to the accountability demands of people who expect quantitative and ëvalue-freeí information. In this regard a positivist evaluation was deemed appropriate for reporting back to AMOLís senior political managers, who require this kind of quantitative information to make budgetary decisions or confirm overall use/worthiness. This positivist study was carried out by the Powerhouse Museumís in-house evaluation specialist Carol Scott and was completed in October 1999.

The second stage of the evaluation process is about to get underway. Instead of statistical data this postpositivist evaluation will utilise an on-line questionnaire and voluntary tracking study to elicit qualitative data. Unlike a positivist approach, which tends to yield data of little worth to project managers like myself, we hope this approach would glean important demographic details about individual users such as gender, age, country of residence, type of work etc. This kind of information is particularly useful in helping determine future directions based on current project successes or failures.

Findings from stage I ñ participation rates

Not unexpectedly the statistical analysis of WebTrends data demonstrated distinct fluctuations in AMOLís use. Diminished use over the Christmas and other holiday periods pointed to the seasonal nature of AMOL use. These findings also underscored the fact that a significant proportion of AMOL users were in all probability Australian museum professionals, many of whom took holidays over this period. This summation is indeed backed up by additional WebTrends data on top referring sites. Among the top 20 referring sites over the seventeen-month period, cultural organisations made up over half, indicating that the site is being used extensively by the heritage sector.

There was irrefutable evidence that overall there has been a pronounced increase in the number of hits on the website since the introduction of the new design. From February 1998 to July 1999 the site received over two and a half million hits from 138,380 users. This averages out at more than 270 users a day, browsing/researching for an average of around twelve and a half minutes. This compares to fewer than 200 users per day spending around 9 minutes online before the redesign. Further evidence of AMOLs growing popularity is the number of participants in AMOLís discussion groups. When the current AMOL coordination unit took over in 1998, only 115 people subscribed to the AMOL discussion group AMF. AMF participation has now increased to 465 and a further two discussion groups have been added: electricmuse and Craft Curator.

Most interestingly AMF membership is now drawn from eleven countries including Australia, the United States, Canada, the United Kingdom, Japan and Germany. When the logs of user sessions were examined it was found that these countries consistently account for the top 5 or 6 user sessions. Most significantly the evaluation showed a strong correlation between countries accessing AMOL, and those countries that account for the majority of inbound tourists to Australia. With some estimates of over 5,000,000 inbound tourists heading to Australia annually by 2000, this correlation is hard to ignore. Especially when you consider that despite some cultural aberrations, approximately 30% of all inbound tourists visit a museum or gallery (Scott, 1995). This finding suggests that there is a strong probability that AMOL is being used by tourists to plan their itinerary.

The analysis of WebTrends data also highlighted the success of the new intuitive hierarchical navigation scheme and accompanying information architecture. One of the significant innovations of the redesign was the restructuring of the site and its information into three main streams: Museum Craft, Museum Guide and Open Collections. Prior to this new architecture, the top 10 entry pages accounted for only 37.88% of the total user sessions. These figures suggested that users were spending considerable time finding their way into and around the site before arriving at their destination. However, since the re-design the top 10 entry pages have gone up to an average of 71% of user sessions. Thus we concluded that the division of information in three major categories, was well received and an improvement on the previous architecture.

User experience

As quickly as AMOL usage can fall between 5% and 10% during holidays, it climbs between 15% and 20% whenever we feature a high-profile event like the Pandora Expedition. The Pandora Expedition (amol.org.au/pandora/pandora_index.asp) was one of the new contextual stories designed to add value to collections and individual objects and so far has been the most popular story with 35.64% of total story top entry pages. Unlike the other 14 stories, The Pandora Expedition was not only well publicised, but also exhibited two critical attributes that captivated and sustained public interest: a subject matter that had broad appeal, namely a maritime archaeological dive on an eighteenth century wreck; and its deployment of interactive multimedia. Although the story ultimately didnít deploy the full panoply of streaming media, it did allow divers working on a boat 30 metres above the wreck to post photographs of artefacts recovered to the website, as well as answer emails from AMOL users. As such the Pandora Expedition tried as far as is currently possible, to adhere to Andy Lippmanís criteria for truly interactive multimedia (Ryder & Wilson, 1996).

In recent times, critics like Andy Lippman have often lamented the inherent lack of genuine interactivity in contemporary IMM. As is the case with many closed, museum-based IMM, the ability for museum visitors to create and customise their own content is very limited. To avoid this and provide a genuinely engaging and appropriate use of media like the web, Lippman has suggested we should attempt to develop experiences that can accommodate:

Although the Pandora Expedition is but a single story amongst hundreds of AMOL resources, it is one of the few to attempt to embrace Lippmanís criteria and so successfully exchange the artifactual for the interactive simulacrum. Superficially this exchange may seem of little consequence for the vast majority of AMOLís current patronage, who at least in quantitative terms, seem largely satisfied with the site. However I believe the lessons learnt from Pandora are critical if the AMOL project is to not only attract new educational and tourist users, but also maintain heritage users in future. By definition the meta-center concept has evolved as a response to users desires for more responsive, easy to find and authoritative information repositories. But as the publicís appetite for multimedia experiences like streaming audio and video grows, the types of information, as well as presentation mechanisms employed by meta-centerís will likewise need to expand. As such in the not too distant future meta-center developers will need to consider ways and means of incorporating the vast as yet untapped museum audiovisual and interactive holdings. Reassuringly this seems to already be underway with CIMI currently investigating the feasibility of an XML test bed project. In addition if tomorrowís meta-centers are to truly embrace the interactive simulacrum they will also need to negotiate the difficult shift from information provider to information broker. This again is easier said than done. Where the current open architecture of some affordance technologies like IRC, CuSeeMe and artificially intelligent MUDs (multi user dimensions), makes this possible, large parts of the heritage sector are still very reluctant to ëgive-upí or ëcompromise" authorial control!

The major challenge will be to evolve meta-center typologies to a point wherein imagery, objects text and speech are more widely available. In addition we will need to look to encourage the development of information composition, creation, editing and curatorial tools that can genuinely customise information to meet the unique needs of users. Admittedly we are still some way off achieving this utopian meta-center, but perhaps in the not too distant future a visit to AMOL might be a little more engaging.

Imagine you have just clicked on AMOLís new 'Living Museum search facility.' Before beginning you enter a few personal details onto a specially designed page: age, education, interests, how many times you have visited the site previously etc. An avatar then asks you what kind of resources you are seeking. You respond by indicating an interest in indigenous music. Drawing upon AMOLís array of textual, graphic and video databases you are then presented with a number of potential stories. You select the ngaramang bayumi story, which examines contemporary indigenous music and dance. Seconds later a streaming video complete with embedded images, text and sound appears. At any stage you can tunnel into the video, acquire individual information sources, and then later on retrieve these as a newly constituted and uniquely personal composite record. Importantly you are not alone in this microworld, simultaneously a school student studying Aboriginal land rites and a German tourist planning a holiday to Arnhem Land, are present as animated avatars. Using IRC or speech synthesis software you can converse with either. However you keep to yourself and instead are drawn to a showcase wherein Tiwi Islander music and a spinning artefact have been activated by your presence. As you move in close you notice a headdress rotating and the Tiwi islander descant fades to a single voice explaining the process of making the headdresses. The system has recalled that on a previous visit you spent time exploring the construction of African masks. Thus it determined that similarly you may be interested in this specific sub-story. Moving along you notice some images of Tapa cloths, which as you scroll over activate an avatar that recounts their provenance. The avatarís voice and indeed actual Tapa cloth imagery have all been supplied by a collector based in Phoenix, Arizona.

References

Australian Bureau of Statistics. (1999). Australia Now - A Statistical Profile: Culture and Recreation Museums and art museums. http://www.abs.gov.au/

Caulley, D. (1992). Overview of Approaches to program evaluation: the five generations. APS Pathways to Change in the 1990s ñ Program Evaluation in the Public Sector Conference, 6 & 7 April, Sydney.

Neimanis, K., & Geber, E. (1999). Come and Get It" Seek and You Shall Find: Transition from a Central Resource to an Information Meta-Center. Museums and the Web Conference, New Orleans.

Robotham, J. (Feb 6, 1996). The Digerati, Sydney Morning Herald, p1. The term Infopreneurials, or digerati, refers to a prominent businessperson working in information technology or the information industry and includes the likes of Bill Gates and Steve Jobs etc and domestically Sam Mostyn, Steve Vamos and Rupert Murdoch.

Ryder, M., & Wilson, B. (1996). Affordances and Constraints of the Internet for Learning and Instruction. Association for Educational Communications Technology Conference. Indianapolis, Feb. 14-18.

Scott, C. (1995). To market, To market: the inbound tourist and Museums. Paper presented at the Museumís Australia annual conference, Darling Harbour, Sydney.

Sumption, K. (1999) Virtual Museums - primitive rebirth or synthetic recovery? An examination of the role and value of real museum experience in the development of on-line education programs at the Powerhouse Museum. Master of Arts thesis, University of Sydney.

Wertheim, M. (1999). Pearly Gates of Cyberspace: A History of Space from Dante to the Internet. Los Angeles, W.W. Norton & Company.