Mubil, a Digital Laboratory: Creating an Interactive Visitor Experience in a Library-Museum Environment
Alexandra Angeletaki, Norwegian University of Science and Technology (NTNU) University library, Gunnerus branch, Trondheim, Norway
Marcello Carrozzino, Percro laboratory, Scuola Santa Anna, Pisa, Italy.
Chiara Evangelista, Percro laboratory, Scuola Santa Anna, Pisa, Italy.
Letizia Jaccheri Department of Computer and Information Science, NTNU, Norway
Mubil digital laboratory is an international, interdisciplinary cooperation between the historical archives of the NTNU University library, Gunnerus branch; the Department of Computer and Information Science, both under the Norwegian University of Science and Technology of Trondheim, Norway; and Percro, a robotics laboratory of the Scuola Superiore of Santa Anna in Pisa, Italy. The project will create a hybrid exhibition space allowing the visitors of the library to interact with archive documents, using a 3D tool, without touching the authentic physical objects. The project aims at finding ways to design active participation in an immersive environment that stimulates knowledge-seeking through a perception-action interaction procedure. A working station will be created in a hybrid environment (physical and immersive) and the experiment has the educational objective to test the learning outcome of the users. The application will also be accessible through the web. At the Museum and the Web conference 2012, we present our work in progress, so that we can exchange views and get feedback by other professionals that are working with similar projects.
Key words: Augmented books, immersive systems, 3D, web, archives, libraries, VXR systems.
1. Virtual realities and memory Institutions.
Museums and libraries as public or private memory institutions have always been drawing people’s attention to their collections, creating a space of public knowledge. “Material things present themselves in relation to human beings,” (Hooper-Greenhill 1992) and throughout history the role of the “expert” professionals, involved in collecting and exhibiting material objects of the past, was very important for the choices that they were making.
Memory institutions are seen by many scholars (Falk 2009, Christidou 2009) as social arenas where visitors interact with the collections, the information presented and with each other. Goffman (1959) pointed out early on that identity is a sense of self-realisation in a social context that is built through interaction with others, as in the case of a visit to a museum. But as Silverstone points out (Miles, 1994) the communication between an object and the visitor does not stop with the display or the label information given at an exhibition; it continues in the mind and the imagination of the visitor that has seen it and experienced it. It becomes then part of the visitor’s agenda. When Falk introduced a visitor’s museum-experience model in 1992, he visualized the process through which an individual proceeds in perceiving information while visiting a museum. He did not include the virtual and immersive perspective of such an experience. In the 1980s, virtual museums and digital archives were created as extensions of the physical memory institutions (Styliani 2009:520). Libraries and museums in the 21st century have thus become virtual destinations because of the increased use of emerging technology (Tonta, 2008: 1).
When Falk later in 2009 introduced the factor of the visitor’s motivation that creates a museum-visitor- experience, he created a long debate about the definition of the scientific methodology in collecting evidence to support such an argument. In our research Falk’s visitor experience model is used to define the relationship between historical archives of libraries and their users. The increased use of Information and Communication Technologies in libraries and historical archives provides us with a framework for meeting new challenges in dissemination practice. In our view, interaction through the use of technology, between content and visitors adds the possibility for the visitors to use their innate knowledge during their visit and “create new knowledge as learners and social agents” ( Hooper-Greenhill 2000).
In such a context the main question to be asked is: How can we best apply new technology in presenting and allowing visitors to interact with heritage collections in order to enhance learning?
Contemporary libraries often engage software engineers and computer scientists to create technical solutions for the ongoing digitization of the records and archives and for the dissemination of the stories they carry. The digitization of collections is used as a preservation method (Styliani 2009, Styliadis 2008) even though the validity, the resistance duration and the archiving is discussed as a risk component. They engage emerging technologies in order to make information content more attractive and accessible to a broader public. The questions are many to keep in mind and will even multiply as new technologies emerge.
The use of digital technology today raises questions that are situated at the convergence of the arts, technologies, and socio-cultural memory preserving institutions (museums and libraries) challenging traditional notions of how cultural heritage can and should be represented, interpreted and disseminated. Kalay, 2007
The aim in our view is to reinforce public interest by creating dynamic linkages between old collections and new technology, communication platforms, exhibition layouts and educational programs. That is to use technological applications to reinterpret cultural collections and reinforce knowledge production.
The use of Web 2.0 and the ever-increasing development of interactive software and new technological platforms together with a decrease in costs have provided libraries and historical collections with a new space of contact to their users. In our view they become museums for written documents of the past. Research shows (Arvanitis, 2005) that the use of mobile phone applications, videos, and digital pictures appeal to the users that are invited to submit their own products to a new space, the web, and thus create a new communication path between themselves and memory institutions. In our project we intend to create a virtual space for the visitor with 3D applications, virtual reconstructions and augmented reality tools, where s/he will be able to interact, make choices and submit stories about the objects s/he interacts both at the actual institution but also through the web. Our research will then focus on the actual performance of the visitor in such environments. We intend to examine whether the visitor’s experience is modified through the use of interactive 3D technology and in what ways.
2. The Current stage:
The Gunnerus library, today a branch of the NTNU University library, is the oldest scientific library of Norway, established in 1768 by the Norwegian Royal Society of Science in Trondheim (DKNVS). The Society was particularly active in collecting books, manuscripts and archaeological and natural history material from around the world and established an institutional organization that allowed all its partners – that is the museum, the library and the society – to have a decisive impact on the institutionalization of cultural memory in Norway.
The Gunnerus branch (GL) of the NTNU University Library has recently agreed with the NTNU Museum of Natural History and Archaeology in Trondheim (VM) and the Science Centre (Vitensenteret) to cooperate in establishing a new center called the “Science Centre” in the area of Trondheim, known as Kalvskinnet, where the library and the museums are situated today. The actual merging of the institutions is far more than a joint cooperation for the purpose of an exhibition, a known practice otherwise. Though partnership and cooperation between such institutions is not new, the concept of a Museum-Library entity (with technology, science, archaeology and history as its main profile pillars) as a Center to disseminate knowledge and support research indicates a shift of focus and creates a need for farsighted planning.
The historic archives of the Gunnerus library are thus, in the context of this particular research, seen as heritage collections, while the institution that is known as a library is seen as a memory institution facing the challenge of re-negotiating its future role in the society. Memory institutions owning collections of common heritage should become, as Hein states it, “more democratic in structure, more responsive to the interests of a broader public” (2000) and raise issues of social and political importance for the societies they serve. In order to do that, libraries need to revise their aims, their professional strategies and their methods of knowledge dissemination and evaluation. Our project would like to contribute to this quest by introducing a 3D digital laboratory described as follows.
3. The Project: Mubil, A digital Laboratory
Project Manager: Alexandra Angeletaki, Mag. Art. in Classical Archaeology, research Librarian at Gunnerus Library (GL), Norwegian University of Science and Technology(NTNU), Trondheim, Norway
● Chiara Evangelista, Research Fellow,Percro laboratory, Scuola Santa Anna, Pisa, Italy
● Marcello Carrozzino, Ph.D., Assistant Professor, Percro laboratory, Scuola Santa Anna, Pisa, Italy
● Letizia Jaccheri, Professor at the Department of Computer and Information Science, NTNU, Norway.
● Stein Johansen Phd in Biology, Director of the Gunnerus library, NTNU, Norway.
Budget: The project has a budget of 2.5 million NOK and is financed mainly by NTNU, GL, IDI and Percro. 460,000 NOK are contributed by the National Library of Norway through the programme of promoting new dissemination strategies for the libraries sector.
The MUBIL project is working with the creation of a hybrid exhibition space where the content of the historical archives of the Gunnerus library will be presented to a wider public. It will disseminate the digitized content of the historical collections of Gunnerus library in a 3D environment where visitors will be able to interact with the collections in a new way through an enriched virtual laboratory.
The exhibition includes fragments, historical manuscripts, old books, pictures and other objects connected to a certain narrative of the past held in our records. In a joint cooperation with its partners the Mubil project will develop a virtual learning space within the physical space of the library using a three dimensional software tool. The project is an interdisciplinary cooperation and the main aim is to allow the library users to experiment with the “objects” of the library in a three dimensional environment and submit their own products and narratives about the “objects.”
The partner elected for the software development is the PERCRO Laboratory of the Scuola Superiore S. Anna in Pisa. This laboratory has had experience in working on "Information Landscape projects,” a computer application that allows the user to join in a free exploration of a three dimensional environment. The tool developed by the laboratory transcribes digital representations of objects or texts in a 3D environment and allows the interaction of the user within it. The 3D tool called here the “augmented-board” supports different models of experiments in learning and is new, so far, as it allows textual information to be experienced in a virtual context. The third partner, the IDI will provide the project with the expertise needed to test and evaluate the augmented board and its software application under the supervision of Prof. Letizia Jaccheri. The project has a duration of two years from 2011 until 2013 and is divided in two periods. The first year (2012) is dedicated to the development of the tool and the testing period. The second year is dedicated to the enriched use of the tool by visitors at the library.
From the technological point of view, the Mubil project will be articulated in three different steps, each corresponding to a different level of interaction and immersion with the digital content. In the first step (touch books), the books’ content will be presented as simple digital copies of the original manuscript. Users will then be allowed to interact with the text via a series of hotspots placed on the pages which trigger additional overlaid information related to the context with the insertion of explanatory images, videos and sounds. Although the project aims to set up a consistent methodology in order to extend this modality of fruition to any type of digital text complying with a defined format, the first prototypes will refer to two books selected based on their peculiar content and their suitability to be presented in the described form. The first one is a treatise on alchemy, written by Adam Lonicer (1528-1586) a doctor from Frankfurt-Main (figures 2 and 3). In this case the information provided, besides being linked to the text itself and the author, will be related also to alchemy and science in general at that particular historical period. The second book is a travel itinerary, written by an aristocrat from Bergen, Hans H. Lilienskiold (1605-1703) see figure 1. The text describes his travel to Europe in 1650, and his text about Pisa is presented here. He has a detailed account about the history of the city and the monuments he visited there in 1650.
figure 1: A possible Virtual Environment for Touch Books
The interaction with the digital application will consist of a simple point-and-click interface either with a standard desktop mouse or via a touch screen. Books will be presented in a simple 3D environment (figure 1) providing an adequate context and it will be possible to browse their content using natural gestures.
The second step (augmented books), will make use of Interactive Digital Storytelling methodologies in order to provide a structured and dedicated narrative able to arouse more interest about the specific subject and to convey more effectively the knowledge which we want to transmit. In this phase, a set of images of these ancient books will be identified and selected. These images will become the context of a set of augmented information, i.e. a selection of strategic elements highlighted on the images, using drawings, masks, popup windows etc., providing a linear narrative describing the scene, the situation and additional information on relevant elements. As an example, figure 2, an image from Lonicer's Alchemy book, tells a story about alchemists which can be examined in sequence: the garden of the herbs, the picking of the herbs, the scientists in conversation, the alchemist’s workshop and the doctor who visits the patient with the remedy.
This step has therefore a higher level of immersion, whilst the interaction is limited to the possibility of selecting element elements of the augmented book and inserting additional content (such as text, images) to customize the interpretation of the narrative. This content will be then available to subsequent users in order to allow them to exchange and share their opinions and experiences.
figure 2: an illustration of Adam Lonicer’s book
figure3: the augmented book
The third step (interaction lab) aims to achieve a higher level of immersion and interaction and is meant to give users the opportunity to gather the information read and learnt through text (1D) and image (2D) exploration in the first two steps, and experiment with the knowledge they have gained inside a purposely created 3D virtual environment. In the Mubil framework, this step will be mainly related to Lonicer's book, with the virtual environment being a reconstruction of an alchemic laboratory (figure 5) based on illustrations extracted from the book. The virtual environment will be experienced on several possible levels of growing immersion, starting from a simple desktop environment up to CAVE-like displays. Users will be able to perform experiments in this lab, following indications and information gathered in the previous steps, by interactively using and combining different tools, instruments, and ingredients in order to realize an alchemical recipe. They will then be evaluated and, depending on the correctness of the results and/or of the followed methodology, obtain a score and an alchemical certificate. In order to expand their knowledge and to learn more, users will be allowed to come back to the interaction lab, either in the immersive system located in the library premises (examples of museum immersive systems are extensively provided in Carrozzino et al., 2010) or in a web-based version of the virtual environment (as in Carrozzino et al., 2005), in order to perform more experiments and gather a better score.
figure 4: Interaction Lab
Figure 5: Drawing by Irene Domínguez Márquez, An alchemist workshop from 1600.
In the context of the collaboration of the Mubil project with the ArTe project (Trifonova 2008), the 3D laboratory will be tested and evaluated. Arte project has performed several in-depth studies about how people interact with artistic software, and how their user experiences can be assessed and evaluated (Trifonova 2008). This study will examine existing theory and empirical result methodology found in the field of software engineering relating to interactive installations (Trifonova 2009) in order to collect empirical data of the encounter between the installations and the users.
Results of this work will be used to suggest guidelines for both memory institution professionals and software engineers for evaluating interactive installations and methods of assessing the success of such installations in terms of communication with the public.
Importance of the research: A dynamic learning experience
Virtual heritage environments (VHE), where heritage sites, monuments or artifacts are virtually reconstructed and user interactivity is introduced for learning, constitute “a fashionable synergy between research inquiry, technology, art, and everyday life” (Kalay, 2007). The field is relatively new and the need of improving both the quality of the virtual worlds created and the relation with the target audience is still under constant development (Tonta, 2008). The call of improvement is complex, as it contains technological and performativity issues as well as issues of meaningful content, standards of delivery, problems of preservation and many others not to be discussed in detail here. We will therefore concentrate on the object biographies as perceived by the visitors during their actual visits in contemporary archives/libraries and on the new aspects of the experience if any, that are constituted when they are presented to 3D virtual images of physical objects, as old manuscripts and books. We will try to examine whether transferring physical objects to a 3D dimension is different than the 2D dimension product, which lacks many qualities of the physical object and might therefore limit the user’s engagement and understanding.
Our primary target group is young school students. The fact that they have usually had a vast exposure to updated technology will be a challenge for the project..The interdisciplinary character of the project is one of great importance but also of great challenge. The cooperation between the GL, IDI and Percro will constitute an intervention of mixed character. The use of computer technology as a tool of representation of archives will allow us to guide students through a learning process where they define for themselves their own learning goals and interests. They will then subsequently interact with the virtual application, through an “augmented board” of books and interactive experiments, to collect information and knowledge about a certain period, object, or story.
We as facilitators will involve the users in the development of the board and its preliminary testing. Thus the content developers should try to maintain a dynamic character of creating a virtual heritage environment and an up-to-date picture of technology. Young adults will be performing certain tasks seeking knowledge about the objects, their use and the period they were created in through tactile/optic/audio sensing of the virtual objects of an “augmented board.” They will try to discover the qualities of texture, volume, size and use of the virtual objects and will also be able to contribute with their own interpretation on the qualities lost in the past.
This particular research project will not apply new questions on the value of using emerging technology in representing memory through heritage collections neither will dispute on the positive and negative consequences of digital reconstruction, preservation and interpretation but will rather concentrate on the performance and the outcome during a certain interaction between the visitor and the "augmented board". The visitor’s experience and behavior as s/he interacts with the augmented books or the 3D experiments will be the observation field of the research. The project will try to observe how visitors function in a hybrid setting, where physical and virtual reality connects to their action in the quest of knowledge as a conscious cognitive process. By introducing mixed reality installations and allowing the visitors to think and act alone, one might be able to register the diversity of the actions and the learning results: that is, to try and analyze the learning experience, by registering the learning process of the participants with their help. By correlating traditional views and technological hand-on tasks performances, one might be able to collect new narratives that are visitor-produced. Using the interaction between the participants and the 3d environment they operate in, a realisation of enactment is created, which then exists and evolves through the thoughts and actions of the participants. The process and results of such types of activities are seen in the operational frame of the so called "Enactive Knowledge" and are defined as "gained through a perception-action interaction" cognitive process (Bergamasco, 2006). This kind of knowledge is supposed to be direct and personal based on the experience one gains through multimodal tasks connected to the particular object.
The library can then become an enriched-experience space by introducing its visitors to books, and other relevant objects through multiple uses. The hypothesis to be tested is that users, when entering such an environment, tend to engage actively in performing knowledge quest tasks and can thus modify their experience towards their own interest. Hooper-Greenhill’s (2000) theoretical approach to visitor experience provides us with the argument that we cannot simply impose pre-selected narratives on the visitors and expect a consistent response. Each individual will have his own experiential perception according to his own individualized level of pre-understanding. The visitor’s background, nationality and identity may influence and vary the outcome of the experience to be expected. Thus the visit becomes “a complex process of interpretation” ( Hooper-Greenhill, 2000) and our inquiry might add a new dimension to the debate of creating a dialogue between memory institutions and the individual visitor, which can then manifest itself in experience diversity. In addition, the project will allow us to gather data on how people interact with such software tools and virtual objects in an interactive environment. In collecting and analyzing the data of the visitors’ experience, the project will be adapting and enriching the “Interactive experience model" introduced by Falk (1992), to which we will add a fourth dimension, that of the virtual immersive experience. The overall aim is to communicate with new groups of library users, create engagement and spread knowledge about the collections of the library.
We would like to express our gratitude to the National Library of Norway for their financial support and also we would like to thank the staff of the Gunnerus library, Percro laboratory and IDI and all our collaborators in the Mubil project for the effort and the support they have genuinely given to the project. Finally we thank Roberta Proto for coordinating the work of the students who are working in this project during Spring 2012.
Arvanitis, K. (2005). “Museums outside walls: Mobile phones and the museum in the Everyday.” IADIS International Conference: Mobile Learning 2005, 251.
Bergamasco, M. (2006). Enactive Network of Excellence, Digest 2006. Lucca: Tipografia Francesconi.
Carrozzino M, Bergamasco M (2010). “Beyond virtual museums: Experiencing immersive virtual reality in real museums.” Journal of Cultural Heritage, 11: 452-458.
Carrozzino M., Brogi A., Tecchia F. and Bergamasco M.( 2005). “The 3D interactive visit to Piazza dei Miracoli, Italy.” In ACE ’05: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. New York: ACM Press,192–195.
Carbonell, B. M. (2004). Museum studies: an anthology of contexts. Malden, MA: Blackwell Pub.
Christidou, D. (2010). "Re-Introducing Visitors: Thoughts and Discussion on John Falk’s Notion of Visitors’ Identity-Related Visit Motivations." Papers from the Institute of Archaeology 20.
Constantopoulos, P. (2010). "Digital Curation and Digital Cultural Memory." S. Konstantopoulos et al. (Eds): Lecture Notes in Computer Science, 2010, Volume 6040/2010,1.
Cook, J. M. S. a. T. (2002). "Archives, records, and power: The making of modern memory." Archival Science 2(1): 1-19.
Crane, S. A. (1997). "Writing the individual back into collective memory." The American Historical Review 102(5): 1372.
De Laurentis, C. C. L. (2006). "Digital Knowledge Exploitation: ICT, Memory Institutions and Innovation from Cultural Assets." Journal of Technology Transfer, 31: 77ff.
Derrida, J. (1996). Archive fever: a Freudian impression. Chicago: University of Chicago Press.
Dilevko, J. and L. Gottlieb (2004). The evolution of library and museum partnerships. Westport: Libraries Unlimited.
Falk, J. H. (2009). Identity and the museum visitor experience. Walnut Creek, CA: Left Coast Press.
Falk, J. H. and L. D. Dierking (1992). The museum experience. Washington, D.C.: Whalesback Books.
Foucault, M. and M. Foucault (1972). The archaeology of knowledge and the discourse on language. New York: Pantheon Books.
Goffman, E. (1959). The Presentation of Self in Everyday Life. New York: Doubleday.
Hedstrom, M. (2002). "Archives, memory, and interfaces with the past." Archival Science 2(1-2): 21.
Henderson, W. A., W. J. Hubbard, et al. (1993). "Collection assessment in academic libraries: Institutional effectiveness in microcosm." Library Acquisitions: Practice & Theory 17(2): 197-201.
Hein, S.,H. (2000), The Museum in Transition Washington, D.C.: Smithsonian Books.
Hooper-Greenhill, E. (1992). Museums and the shaping of knowledge. London: Routledge.
Hooper-Greenhill, E. (2000). Museums and the interpretation of visual culture. London, Routledge.
Kalay, Y., T. Kvan, et al. (2007). "New Heritage New Media and Cultural Heritage." from http://public.eblib.com/EBLPublic/PublicView.do?ptiID=325247. Consulted 29 March 2012.
Ketelaar, E. (2001). "Tacit narratives: The meanings of archives." Archival Science 1(2): 131.
Lee, C. A. (2007). "Taking Context Seriously: A Framework for Contextual
Information in Digital Collections." UNC SILS Technical Report 2007-04.
Manoff, M. M. (2004). "Theories of the Archive from Across the Disciplines." Portal: Libraries and the Academy 4(1): 9-25.
Miles, R. and L. Zavala (1994). Towards the museum of the future: new European perspectives. London, Routledge.
Nissley, N. (2002). "The politics of the exhibition: Viewing corporate museums through the paradigmatic lens of organizational memory." British journal of management 13 (Supplement 2): S35.
Nora, P. (1989). "Between Memory and History: Les Lieux de Mémoire." Representations (26): 7-24.
Olick, J. K. and J. Robbins (1998). "Social Memory Studies: From "Collective Memory" to the Historical Sociology of Mnemonic Practices." Annual Review of Sociology 24: 105-140.
Sadler, D. R. (1989). "Formative assessment and the design of instructional systems." Instructional science 18(2): 119-144.
Soysal, Y. N. (2002). “Locating Europe.” European Societies Routledge, part of the Taylor & Francis Group Volume 4, Number 3, 1 September 2002: pp. 265-284 (220).
Styliadis, A. D., I. I. Akbaylar, et al. (2009). "Metadata-based heritage sites modeling with e-learning functionality." Journal of Cultural Heritage 10(2): 296-312.
Styliani, S., L. Fotis, et al. (2009). "Virtual museums, a survey and some issues for consideration." Journal of Cultural Heritage 10(4): 520-528.
Sylaiou, S., M. Economou, et al. (2008). "The evaluation of ARCO: a lesson in curatorial competence and intuition with new technology." Computers in Entertainment (CIE) 6(2): 23.
Sylaiou, S., K. Mania, et al. (2010). "Exploring the relationship between presence and enjoyment in a virtual museum." International Journal of Human-Computer Studies 68(5): 243-253.
Tonta, Y. (2008). "Libraries and Museums in the Flat World: Are They Becoming Virtual Destinations?" from http://eprints.rclis.org/14035/. Consulted 29 March 2012.
Trifonova, A., Jaccheri, L. and Bergaust, K. (2008). Software engineering issues in interactive installation art. International Journal of Arts and Technology (IJART), vol 1(1), [ISSN (Online): 1754-8861 - ISSN (Print): 1754-8853].
Trifonova, A., S. U. Ahmed, et al. (2009). “SArt: Towards Innovation at the Intersection of Software Engineering and Art, Information Systems Development.” Proceedings of The 16th International Conference on Information Systems Development, C. Barry, M. Lang, W. Wojtkowski, K. Conboy and G. Wojtkowski (Eds.) Springer US: 809-827.
Velody, I. (1998). "The archive and the human sciences: notes towards a theory of the archive." History of the human sciences. 11(4): 1.
Zorloni, A. (2010). "Managing performance indicators in visual art museums." Museum Manage. Curatorship Museum Management and Curatorship 25(2): 167-180.