Making the Punishment Fit the Crime: Content-driven Multimedia Development 1
As you may have noticed in your own experience, ideas are dangerous things. They start small and then have a way of exponentially expanding to fill years of time, and ballooning to engage dozens of people in their execution. Thatís if you are lucky.
Over the past few years at SFMOMA, we have been very, very lucky 2. We have not only had an idea, we have seen it through several design iterations, tested it, gotten it funded, staffed up, and are now working on it full-tilt boogie. In this paper we shall give a structural presentation of the development process that has guided our work. We shall discuss how we think of audience in designing our tools, and how we then envision ourselves as future users to build a meta-toolset for continued content authoring in the future.
1. The Idea
The program is called Making Sense of Modern Art. It is an educational multimedia program publishable not only to the Web, but to gallery-based kiosks/flat-panel stations, CD-ROMs, wearable computers and smart tables as well. The goal is, in the final analysis, to have a sea of content, and of critical frameworks, from which we can draw interactive essays on many different subjects, suited to a variety of audiences and delivery platforms, and publish them at will.
With this goal in mind, initial content research began. What was the scope of our inquiry? How would we make sense of modern art? On the highest level we divided our approaches into examinations across the century (Timeline and Themes) and those that would be artwork specific (The Domain of the Work). This paper concerns the question and answer exploration screens in the third, artwork-specific section, to which we have been devoting much of our attention over the past year. After creating an initial list of artworks from our permanent collection to explore, we dove in to look at the issues that would surround each work. Our interest was to use the content itself to drive the process.
Our initial content meetings were conducted a bit like seminars with the lead researcher presenting salient issues about a work. Each issue was then framed as a question and together we sketched what we saw as a mental map of its exploration. For example when looking at Dali and Spain as a Surrealist context we sketched a map which layered the influences of García-Lorca, Civil War, Catalonia, etcÖ
Or when talking about Robert Rauschenberg in his early years at Black Mountain College we laid out a web of collaborative relationships.
Figure 1. Schematic sketch of Rauschenbergís collaborations by Jeannette Redensek. This sketch, evoking Rauschenbergís creative rapports with artists as varied as John Cage, Cy Twombly, Susan Weill, and Jasper Johns, dancers Trisha Brown, Merce Cunningham, and Carolyn Brown, as well as Fluxus artists Paik and Beuys, was a brainstorming document on the way to designing the "Collaboration Web" screen depicted in Figure 4.
With Louise Bourgeoisís large-scale sculpture The Nest we imagined that an image of the work itself might embody the multiple analyses relevant to her working process and influences (Figures 1-4).
3. Generalizing to Build Screen Types
By working with researchers who were self-conscious and articulate about the historical methodologies they were employing, we were able to think explicitly about the user experience. If we were in fact presenting a range of art historical methodologies, how could we put the user in a position to live or re-enact those discovery processes? Even as we confined ourselves to paper storyboard prototypes, we were already thinking not just about what to say, but how the userís doing something would help them understand the matters at hand. What tools might enable us to develop interactivity based in these approaches?
We noted that as we considered more artworks, each spawned a variety of issues and approaches. Certain approaches became generalizable, and we were able to combine salient features from different sketches to form screen types that we felt could serve us well throughout the program. Often, these mapped to specific art historical methods, such as:
All the screens shared a common trait: they served as multipliers of content within a single screen. What the viewer saw on first arriving was not what they saw as they explored the screen. The screen types were inherently polyvalent: each held within it a mechanism for revealing a cascading succession of related media content, as a result of the viewerís engagement.
4. Screen Design Implementation
Armed with our sketches, the time had come to build screen templates. Particularly important at this juncture was working again with the researchers to make sure that the tools would in fact work for the bulk of the content they were beginning to script. We also had to test the screens to make sure that they could be made into templates for our authoring database.
The Slider Gallery (Fig. 5) is a good example of a screen type that came out of the process. Initially created for use as a gallery of monochromatic paintings over time, we knew that it could be used for other types of galleries: works throughout an artistís life, works arranged according to a particular theme, etc.
Similarly the Collaboration Web, which grew out of a sketch (Fig. 1) of Robert Rauschenbergís circle of collaborators and friends, would be applicable to other situations where multiple links could be made between a group of artists, works, or ideas. (Fig. 6) It could also serve, with a slight modification in functionality, as a layered gallery with the works on the screen revealing other works when clicked.
Finally, for the basic garden variety coffee table book/magazine-like web page-multimedia screen, we developed what we called the Variety of Media screen, which could launch close-ups or videos of the various elements it contained.
Somewhat confident that the screens were working for our content, we engaged in two initial waves of usability testing with a professional facilitator in a corporate testing lab. (In this we were inspired by Scott Sayreís presentation at last yearís Museums & the Web conference. (Sayre 1999)) Testing is always both frightening and enlightening; it becomes especially important when new interface conventions are being defined and a broad range of users is envisioned. In the case of Making Sense, the first group of users is museum visitors and web users. We also plan to develop a curriculum guide for use in secondary schools. We tested early with these groups so that we could identify which tools would be most useful for them. Even from static screen print-outs, our teacher-advisors have given us important insights into the areas of the program and the particular functionalities that they will be able to use in the classroom and for which we should consider curriculum development.
6. Authoring Tool Development
The next task was to design a Flash Generator database capable of creating the interactive Flash functionality for each of the programís screens and sequences. Our developers were challenged in this by the loose, non-hierarchical nature of our design process. What was perhaps just an inconvenience became a bit more maddening as they tried to accomplish our next mission: to create an automated "wizard" tool that would walk curators or art historians through the process of putting together a content screen, and that would factor in all possible screen-type combinations. What screen type would lead to what screen? they asked, noting our desire to embed alternative questions within initial answer exploration screens. Our response was invariably: "We canít predict"óa database designerís nightmare. While the end userís entry into the program would in fact be hierarchical, with the user able to move from one of three main interfaces down to question screens and from there into answer exploration screens, once within the exploration screens, there is no inherently hierarchical relationship among them. A user might branch via a supplementary, embedded question from a Collaboration Web to a Slider Gallery or from a Formal Analysis to a Variety of Media screen (Figure 9). This pushed the developers to rethink their design of the authoring tool.
The authoring tool is designed to build not only individual screens, but to assemble them into Flash presentations as well. Presentations are collections of screens that can be destined for the web, kiosks, fixed disk or another delivery platform. The important thing is to have the content available in the form of media assets ready to assemble; then the author-designer can choose the appropriate media to publish to their chosen delivery platform (Figure 10).
Addition of Non-Database Driven Components
While we were aware of the benefits of using database-driven templates to create content, we always knew we wouldnít be satisfied if that was all we provided for our users. That knowledge and our desire to create special content for the web led to the creation of theme screens. As we have written elsewhere (Samis, 1997):
The theme screensí design is still emerging and will continue to develop because each theme will be hand crafted in Flash by teams of author/designers. Like the exploration screens the process will be content driven. While the question and exploration screens focus on individual artworks, the theme screens traverse the century and connect multiple works in their threads. Free from any database constraints we will use typography, animation, and sound to enliven these discussions of issues like "art and the everyday object," "artists and the body," and "Who gave artists the right to make their own rules?" We envision these screens as an opportunity to custom design interactivity for a range of users from children to art historians.
8. " Díoù venons-nous? Que sommes-nous? Où allons-nous?"
Gauguinís famous questions, "Where do we come from, what are we, where are we going?" put us in mind of the different levels of interactivity that this program, and others, have charted. A rough typology follows:
First degree interactivity might be exemplified by a book. It is primarily linear, the type is fixed and immutable, but you can stop whenever you want and dream, you make up in your own mind what each of the characters looks and sounds like. Whatís more, if there is an index or itís alphabetically organized, you can access its contents in a non-linear way.
Second degree interactivity is exemplified by point and click multimedia, i.e., the standard web page with a layout reminiscent of a book or magazine. You can still stop and reflect, pace yourself, etc., and the capacity for video, animation, and audio is addedómuch like an animated illustration on a static page.
Our development strategy for the answer exploration screens in Making Sense of Modern Art is what we will call third degree interactivity. Here we are using the screen templates to model ways in which an educated viewer may come to his or her understanding of the meanings of a work, and allowing the user to emulate those processes, making discoveries and connections for themselves. The screen might undergo fundamental changes in state between the userís arrival and departure, depending on how they use the template at hand. A more dramatic example of third degree interactivity is made available in our Timeline and Concepts & Relations screens, not treated in this paper. There, object-oriented programming permits viewers to compare artworks across time using drag-and-drop functionality.
At the fourth stage, we turn those tools over to the user. For example, the formal analysis tool that highlights sections of an artwork for description could be used by users to add their own annotations or to reference related works of their own choosing (or making?). The same tool could be used in a group discussion forum as a site for people to raise issues about a work. Instead of our supplying the answer "Whatís going on in this picture?" both the questions and answers could be raised by users.
The future of the program is not limited to the web and kiosks in the galleries. Working with Flavia Sparacino of the MIT Media Lab, we are porting content developed in the authoring toolís "sea" of exploration screens to a wearable computer that will enable the work of art to trigger content as you walk through the gallery. Other features of the Making Sense program structure not treated in this paper will bear fruit in physical "smart tables" that will provide social interaction in the galleries along with an exploration of the interconnectedness of works in our collection.
Our ultimate goal for Making Sense of Modern Art is to enable users to co-author program content with usóand with each other. If we can further personalize the programís tools to correspond with their learning styles, then we may reach a fifth degree of interactivity. Just as we are giving ourselves the option to selectively publish content to different delivery platforms, at a later date, we might give users the options to select the kinds of media and interactivity they prefer to experience.
The authors wish to thank Jeannette Redensek and Karen Fiss, successively directors of research on the project, our collaborators in the development of these screen types. Thanks are also due to our patient and long-suffering outside collaborators at Red Eye Digital Media (for engineering the web functionality and authoring tool) and Perimetre Design (for the current generation of screen designs). Making Sense of Modern Art has received generous support from The Getty Grant Program and Compaq Computer Corporation. Finally, none of this work could be done without the support of John Weber, Lori Fogarty, and David Ross at SFMOMA.
1. An alternate and perhaps even more appropriate title would be "Making the Egg Fit the Chicken." [!]
2. By now the reader should be hearing faint echoes of the Chinese curse, "May you live in interesting times."
Sayre, S. (1999). Use and Usability: Lost in the Translation? Presentation at Museums and the Web 1999. Url: http://www.artsmia.org/presentations/mw99/
Samis, P. (1997). The quest for a multimedia approach suited to the complexities of modern and contemporary art: a view from America. In H. Kraütler (Ed.), New Strategies for communication in museums: Proceedings of ICOM/CECA í96 (pp. 26-29). Vienna: ICOM/Austrian National Committee, 1997.