Archives & Museum Informatics
158 Lee Avenue
Toronto, Ontario
M4E 2P3 Canada
info@ archimuse.com
www.archimuse.com

Search A&MI

Join our Mailing List.
Privacy.

Published: March 15, 2001.

The Effect of Surrogation on Viewer Response to Expressional Qualities in Works of Art: Preliminary Findings from the Toledo Picture Study

Bradley L. Taylor, University of Michigan, USA

Abstract

An empirical study conducted at the Toledo (Ohio) Museum of Art examined the responses of 86 non-expert museum visitors to pictures presented in five formats--oil on canvas paintings, printed pages from books, black-and-white glossy photos, color slides, and digital images. Test subjects responded to the same 20 pictures presented in different formats and different showing orders, identifying the expressional content depicted in the images and answering questions designed to test perceptions of the ability of surrogates to reproduce the expressional power of an original oil painting. This paper reports preliminary findings from the Toledo Picture Study.

The study offers findings of significant interest to those seeking to expand access to museum and visual resource collections to non-specialist users. The process of testing on multiple surrogate types is expected to suggest which formats best convey the expressional essence of a work to those individuals searching without the benefit of expert language or a knowledge of artists, art historical periods, or artistic styles. The ability to extend access to the vast audience that exists beyond art history scholars and museum professionals not only serves to advance greater educational, cultural, and societal goals, but serves to restore this body of documentation to the greater public from which it originated.

Keywords: Surrogation, Surrogate, Perception, Original, Art, Expression, Viewer, Interpretation, Image Retrieval

Problem Statement

While important research in the past has facilitated the ability to create, retrieve, and store digital images, the absence of a recognized standard for cataloging and classifying museum collections has kept access to museum collections information far beyond the reach of most interested users. Research centering on those people seeking access to image-based materials 3/4 specifically, studies of their queries, their perception of images, and the interrelationships they see within collections of images 3/4 has lagged far behind the work that has centered on building systems to deliver visual materials (Romer, 1996). The dearth of research on users is exacerbated by the fact that the vast majority of user studies that have been conducted have focused on the behaviors, needs, and perceptions of subject specialists (i.e., Corkill and Mann, 1978; Stone, 1982; Stam, 1984 a, b; Watson-Boone, 1994; Cobbledick, 1996; and, Rasmussen, 1998). A lack of attention to the great numbers of non-expert users has left those without specialized knowledge of art history and museums even further disenfranchised. This study seeks to take an important step forward by clarifying our understanding of (1) how non-specialists perceive the expressional content of images, and (2) how non-specialists respond to different types of surrogates.

One of the primary concerns voiced by writers is how users perceive the surrogate forms that are most often used to represent original works of art. While progress continues to be made on replicating the physical features of works of art, many argue that a work of art's expressional qualities 3/4 those emotional, affective, or spiritual aspects of a piece that define its essence 3/4 become irretrievably lost in the process of surrogation. As the world's museums move toward the large-scale digitization of their collections, the time has clearly come to empirically examine prevailing assumptions about surrogation and to see whether expressional qualities do, in fact, survive in reproductive formats. To that end, the purpose of this study shall be to investigate the effect of surrogation on viewer response to expressional qualities in works of art.

Theoretical Foundations and Research Hypotheses

This new study will use as its theoretical basis Panofsky's (1955) proposed "three levels of meaning" in art and his assertion that the most basic of these levels, the pre-iconographic level, presumes ready comprehension by non-expert test subjects of "primary or natural subject matter," defined as being "of an elementary and easily understandable nature" (Panofsky, 1955, p. 26). Panofsky divides primary or natural subject matter further into the factual and the expressional. Should the expressional content of art truly exist at an "elementary and easily understandable" level, non-expert viewers should be able to identify its presence in both original works of art and representations of those same works of art.

At the same time, this study also acknowledges the impact of the writings of art historian Walter Benjamin and others who warn of the differences that exist between original works of art and reproductions of the same. In spite of what would appear to be conflicting theoretical bases, we offer the following research hypotheses:

H1. We believe that test subjects will be able to identify emotions and feelings across all formats when tested in part I of the study.

H2. We believe that the emotions and feelings identified by test subjects will correlate at statistically significant levels.

H3. We believe, nonetheless, that the results of survey questions will reveal differences between the experience of looking at original works of art and looking at surrogates in various reproductive formats.

H4. We believe that some differences in response will be linked both to familiarity with computers and frequent viewing of original art.

H5. We believe that the differences subjects identify between original works of art and surrogates will point to new topics that will provide the basis for further study, i.e. the historical nature of original documentation, issues of personal response, and the effect of the museum setting on the way subjects experience original works of art.

The Toledo Picture Study

To test these research hypotheses, an experiment was conducted to test non-expert subjects on their perceptions of the similarities and differences that exist between original works of art and four different surrogate formats: printed book pages, digital images, black-and-white glossy photographs, and color slides. The experiment was conducted during the summer of 2000 at the Toledo (Ohio) Museum of Art. This site was selected as the museum showed signs of being receptive to hosting the study, had collections sufficiently large to facilitate the careful selection of paintings for inclusion in the study, and offered a highly motivated roster of volunteer support that proved crucial in the subject recruitment phase of the study. The Toledo community, likewise, was thought to be able to provide a more representative sample of non-expert users for the study than the smaller, university-focused community in which the researcher is based. The Toledo Picture Study was conducted to address the following research question and its attendant components:

Research Question:

What is the effect of surrogation on viewer response to expressional qualities in works of art?

1. Do viewers respond to the expressional content of original works of art and their surrogates in the same way?
2. Do viewers identify the same expressional qualities in both original works of art and their surrogates?
3. What is the nature and magnitude of differences perceived between original works of art and their surrogates?
4. Do demographic factors such as gender, computer use, or museum attendance affect the way viewers respond to the pictures examined in this research?
5. What are the broader implications of these findings?

Participants in the Toledo Picture Study were asked to identify the emotions and feelings they saw in twenty pictures, each drawn from the Toledo Museum of Art's collection of nineteenth- and twentieth-century European and American oil paintings. This focus was selected in order to control for possible differences in the type of art being shown to test subjects and to provide test subjects with a body of work that was likely to seem familiar to them in terms of period and style. In order to control for the influence of the type of art being shown, subjects saw pictures distributed across five different artistic types--action or activity pictures, where the focus of the work is centered largely on physical movement; genre scenes, where the focus of the work is on domestic activities or interior scenes; landscapes; modern art, defined, generally, here as artistic style that begins to push the boundaries of representationalism; and, portraits. Choices for these artistic types or "genres" were based largely on the findings of Kettlewell (1988), Kettlewell, Lipscomb, Evans, and Rosston (1990), and Winston and Cupchik (1992), who suggested that non-expert viewers respond differently to different types of artistic genres.

In order to test for the effect of surrogation, test subjects were shown the pictures distributed across an equal number of the formats being studied--oil on canvas paintings, digital images, black-and-white glossy photographs, color slides projected on a screen, and printed book pages. The four surrogate formats were selected not because they best capture the essence of original art but because they have been among the most common ways that original art has been reproduced for general use--in printed books and catalogs, on computer screens, as the format of choice for documentary purposes in museums, and in countless classroom settings. While general specifications for these surrogate types will be provided in a section to follow, it should be stated that the primary purpose of this research was to examine whether differences in emotional response exist between original works of art and surrogate representations of those same works, not to investigate the subtleties in physical likeness that may have been elicited from gradient differences between types of surrogates (i.e., Panofsky's expressional meaning as opposed to Panofsky's factual meaning). Once the existence of these essential differences have been established empirically, details of the differences that exist within categories of surrogates might, in fact, be of considerable interest.

Experimental Design

A repeated Graeco-Latin squares research design was used to assure both the randomization of which pictures test subjects saw in which reproductive format but also which picture the test subjects saw in which order. The basic 5 ´ 5 grid design that forms the basis of the Latin square allowed the researcher to map out a sequence of "viewings" of five pictures for test subjects that controlled for the overall effect of varying genres and formats. To help control for the possible effect of showing order, another 5 ´ 5 grid was superimposed on the Latin square, creating a Graeco-Latin square; and because each test subject needed to be shepherded through a total of twenty pictures, the pattern was repeated four times, making the basic design for the experiment a repeated Graeco-Latin square design. With the three intervening variables--genre, format, and showing order--thus built into, and counterbalanced by, the research design, the researcher was able to focus directly on the responses given by test subjects without having to be concerned about the fact that, for example, portraits preceded landscapes, digital images followed original works of art, or the fact that, for example, the Sargent immediately preceded the Monet.

Findings from the Pretest

Statistical findings from a pretest conducted in March 2000 at the University of Michigan Museum of Art provide consistent and persuasive support for the research hypotheses advanced earlier. Even with a pretest population of only twelve individuals, the differences in means on all of the questions in part II of the pretest clearly point to differences that became only more apparent with the larger sample size in Toledo. Moreover, qualitative results from the open-ended questions supported these findings and added considerable depth to the numerical results the statistical analysis revealed. What we anticipate in the Toledo Picture Study, then, is confirmation of these initial results, further explication of these findings through more open-ended questioning, and a deeper understanding of the statistical findings by using tests of association to compare our major variables with demographic data gathered on gender, computer use, and museum attendance.

Reliability and Validity

The choice of a repeated Graeco-Latin square research design offset the possibility of noise coming from such intervening factors as genre and subject. The use of a controlled vocabulary as the basis for the attributions made in part I of the study greatly controlled for problems with synonymy that might arise: while it was not possible to control differences in the way subjects perceived different emotions, the language they used in identifying those emotions was, at least, controlled. The pretest helped in identifying several unclear questions and the need to add more questions in order to test further those differences pretest subjects identified. In addition, the pretest process allowed the researcher to address issues in the administration of the pretest that was used to further strengthen the administration of the Toledo Picture Study.

Recruitment and Participation of Test Subjects

Arriving at a successful subject recruitment strategy proved an unanticipated obstacle in executing the study. Museum visitors approached on the gallery floor almost always declined an invitation to participate in the study. An informal survey of museum visitors taken several weeks into the study revealed that most visitors were hesitant to commit to more than an a half hour of time to the study. Reasons given most often were not enough time to give, not wanting to abandon other members of their party in order to participate in the study, and the fact that they were visiting the museum on their way to another destination: lack of financial incentive did not appear to be a factor. Since the Toledo Picture Study took most participants an hour and a half to complete, a different recruitment strategy was deemed essential.

In the end, invitations to participate were extended by e-mail and telephone to a number of people who were thought to be highly motivated to participate. Successful referrals came from museum staff, residents of the Toledo community, personal acquaintances, and from the test subjects themselves. The final subject pool comprised a high number of museum aides (volunteer fund raisers for the museum) and their acquaintances, professional members of the Toledo business community, personal acquaintances, and the visiting public. Their common bond was a lack of formal subject expertise and a willingness to support the museum through their participation in the study. Advance familiarity with the collections was measured and was found to be balanced across the subject pool; that said, only two of the test participants had received any docent training. and were familiar enough with the collections to speak knowledgeably about specific items in the collection. In all, 86 individuals participated in the Toledo Picture Study.

Test subjects were asked specifically to identify "feelings and emotions" they saw depicted in the pictures they were shown. The task in part I of the study was one of identification (i.e., "what feelings and emotions do you see depicted?") as opposed to one of interpretation (i.e., "what feelings or emotions does this create in you?"). Such distinctions need not cause confusion for test participants. Using the example of a Giotto painting, Tilghman suggests that human ability to read grief in a painting does not necessarily evoke grieving in the viewer (objectified feeling). Similarly, those works that may evoke the strongest feeling (i.e., the print that was hanging on the kitchen wall the night old Rover died) may have nothing in their subject matter that relates directly to the response evoked (embodied feeling) (Tilghman, 1970, pp. 2, 3). Possible confusion over such distinctions was addressed as part of the participant instruction process. In part II of the study, subjects responded to a series of questions asking about the process of looking at pictures in different formats and provided demographic information about gender, computer use and museum attendance.

Test subjects selected their responses for part I of the study from a controlled list of terms first published by Shaver et al. (1987) in the Journal of Personality and Social Psychology. This list was deemed suitable for purposes of this experiment since it was developed by scholars within the field of social psychology, was the result of years of study on the terminology of emotions, and yielded an "interrelated set of emotion categories...organized within an abstract to concrete hierarchy" (Shaver et al., 1987, p. 1061). The Shaver et al. vocabulary, while not claiming to be exhaustive, nonetheless offered test subjects a wide variety of terms (136 terms, divided into 25 clusters), and could be used to code the resulting data for statistical analysis. In addition, one of the significant findings from the pretest was both the need for viewers to use a controlled vocabulary for this part of the test and posttest findings revealing that the Shaver et al. (1987) vocabulary offered the broad range of attributes pretest takers were seeking.

Since the nature and ordering of the material provided in part I of the study varied from test subject to test subject, the researcher accompanied test subjects throughout the first half of the study. The researcher identified the next picture to be viewed and provided the picture to the test subject in the appropriate format for viewing. By the time part I of the study was completed, test subjects had been shown four original works of art, four printed pages from books, four digital images, four black-and-white glossy photographs, and four color slides projected on a screen. Test subjects completed part II of the study largely on their own 3/4 they were provided with a quiet space in which to work and enough time to complete the testing at their own pace. The Toledo Picture Study took most test subjects between one and one and one-half hours to complete.

Preparing the Experiment

In addition to the preparation of the testing instrument, much advance effort was expended on the logistics of preparing for the study at the Toledo (Ohio) Museum of Art. With the cooperation of the museum registrar, twenty nineteenth- and twentieth-century Western European and American oil paintings were selected from the museum's permanent collection to provide the subject matter for the testing. Since the test involved showing subjects this artwork in a variety of formats (oil on canvas, printed pages from a book, digital images, black-and-white glossy photographs, and color slides), the registrar and the museum's visual resources staff helped facilitate access to the twenty oil paintings selected for the test in all five formats intended for inclusion. The oil paintings were all on exhibit in the museum's galleries for the duration of the study 3/4 pieces scheduled to travel or pieces in galleries scheduled for reinstallation, thus, were not selected for inclusion in the study. The three largest pieces selected for inclusion were works by Chuck Close (100'' x 84"), John Singer Sargent (65" x 38"), and Paul-Gustave Dore (42" x 72"); the three smallest pieces included were by Eugene Delacroix (23" x 28"), John Lewis Krimmel (16 x 22"), and Henry Walton (25"x 30").

The printed book page surrogates were taken from published catalogs of the museum's European and American collections or from one of several annual reports that were provided by the Toledo registrar. Sizes for the book pictures ranged from 4" x 3" to 6" x 8". Since test subjects saw only four examples in any given format, it is unlikely that any individual subject saw the extremes in size that are provided here: the measurements are intended to provide only a general overview of what individual test subjects might have seen. Black-and-white glossy photographs were taken from the registrar's files 3/4 all photographs measured 8 ´ 10 inches. Two-inch square glass slide surrogates of the oil paintings were provided by the museum's visual resources staff and were shown using a Kodak Ektagraphic III AMT slide projector. The top of the Bretford brand projection screen was raised 96 inches from the floor, with the screen bottom 6 inches off the floor. Both the viewer and the projector were set about 11 1/2 feet from the screen in a interior windowless room lighted only by a small tensor lamp which provided enough light for test subjects to read their controlled vocabularies and mark their test sheets. Image sizes for the projected slides ranged from 4' x 3' to 3' x 4'.

Digital image surrogates were prepared from the same color slides as served as the slide surrogates in the study. The initial scans were made using an Epson Expression 836 XL scanner set at 300 d.p.i. These new scans were cropped and color balanced using Adobe Photoshop 5.0 software viewed on a View Sonic 20 inch color monitor. The resulting TIFF files were then moved onto the museum's central server and were ready for viewing. Test subjects viewed the digital images on a Power Mac 7100/66 AV Power PC, made available by the museum for the duration of the study. The computer's 13-inch monitor was capable of showing only 72 d.p.i., so the initial scans ended up being made at a higher degree of resolution than could be shown to the test participants. Similarly, the computer's modest RAM capabilities resulted in loading times of up to 15 seconds for images to appear on the computer screen. An earlier set of digital scans made from color transparencies was not used because their larger file sizes took even longer to load. Perceived limitations of the hardware notwithstanding, responses to questions about digital images in the Toledo Picture Study closely approximate those in the pretest, which had used more sophisticated hardware and software. In addition, the viewing conditions used for the digital images in the Toledo Picture Study met the same technical specifications as the public access terminals available in the museum's Community Learning Resource Center, the primary center for viewing digital images within the museum.

During the testing period, the catalogues and annual reports, which provided the source material for the printed book pages, remained closed when not being used. When the time came for pretest subjects to look at pictures in the books, they were handed a book which had been opened to the individual page they were to view. The black-and-white glossy photos had been ordered in advance and placed in a manila file folder. Subjects saw only those photographs that were part of their test, the photographs being brought out individually for their viewing. For both the book and the glossy photo examples, caption information was either masked off or was simply not available. When viewing the color slides, subjects waited for the lights to be turned off and were shown only those pictures that were part of their test. For the digital image examples, the screen of the Power Mac remained set to a default screen and individual items were called up one at a time by the researcher; no caption information was available.

Administering the Experiment

Upon arrival at the museum, test subjects were greeted, escorted to the registrar's office, assisted in the storage of coats, purses, etc., and introduced to any museum staff who might be in the immediate vicinity of the testing area, in order to make them feel welcome. The starting point for each test session was a three-sided cubicle used as the digital image viewing area since it allowed for the most privacy. After some initial pleasantries, pretest subjects were handed several stapled sheets of paper that comprised the Participant Instructions and both parts I and II of the study and were asked to read through the Participant Instructions. Upon completion, they signed an informed-consent statement giving their assent to participate in the study and were reminded that their participation in the pretest was completely voluntary. Test subjects were then asked if they had any questions before the testing began and were reminded verbally that what was being sought in part I of the study was their perception of the feelings and emotions displayed in the pictures they were about to see. They selected from among a half-dozen sharpened No. 2 pencils provided for their use in the study and were asked to turn the page and begin part I of the study.

While each test subject saw each of the twenty pictures during the test, they were each shown the pictures only once, in different formats and in a different showing order. Test subjects completed part I of the study with no restrictions on their time nor on the number of responses they were allowed to give. When test subjects showed obvious signs that they were through with a particular picture (using verbal cues such as "OK," "all right," etc., or cues through gesture such as looking up and making eye contact, etc.), they were either shown the next in the format they were viewing or were moved to the next set of pictures. Once test subjects had completed their work on part I of the study, they were brought back to the registrar's office and left on their own to complete part II. Most subjects took approximately 75-90 minutes to complete the entire test. At the end of each test session, an informal conversation took place about the test, the testing instrument, and any questions test subjects wished to ask about the nature of the research being conducted. Most subjects found the test process stimulating and stayed an additional five to ten minutes to share their reflections on the process. At the end of the entire test session, subjects were thanked, given their choice of a souvenir notecard (one of the pictures from the study), and escorted back to the entrance of the museum. A final formal thank-you brought the test session to an end.

The testing was conducted in several different locations within the museum, each selected as being the best location available for viewing the various surrogate formats. Headquarters for the testing was the registrar's office. Inside, two large wooden desks were used for viewing the book page and glossy photo surrogates. The desks provided a generous surface area that allowed not only room for the surrogates themselves but also for the testing instrument, the controlled vocabulary, the personal belongings of the test subject, and for the researcher and test subject to sit at right angles to one another. Lighting in this area came from a series of filtered overhead fluorescent lights and only minimal noise came from the surrounding office activities. None of the test subjects ever mentioned office noise as a distraction. The digital images were viewed within a three-sided cubicle located within another section of the registrar's office. The lighting and relative noise conditions were the same as for viewing the book and glossy photo surrogates. Since much of the surface space of the table in this cubicle was taken up by the computer and the researcher's testing materials, test subjects usually viewed the digital images while holding a clipboard with the testing instrument and controlled vocabulary in their laps. Viewing of the color slides took place in a little-used classroom down the hall from the registrar's office. This allowed the projector and screen to remain up and ready for use throughout the three months of testing without interfering with any daily office activities. Positioning of the projector, the viewer, and the screen has been discussed previously. Finally, for the original works of art, test subjects were led to individual paintings by the researcher and were asked, as much as was possible, to ignore any caption information, concentrating exclusively on the picture itself. Most subjects seemed happy to comply.

Data Analysis

Quantitative results will be analyzed differently for data gathered in part I and part II of the Toledo Picture Study. In both cases, however, quantitative results will be subjected to univariate, bivariate, and multivariate statistical analyses using SPSS 9.0 software. Descriptive statistics will be provided for all quantitative response categories and the appropriate statistical tests, including correlations, analyses of variance (ANOVAs) and multiple analyses of variance (MANOVAs), will be conducted and the results reported, with levels of significance provided. Analyses of variance were selected as the statistical operation of choice in many cases since it allows for the direct comparison and analysis of means across response categories within a given question. The use of nominal level data in the Toledo Picture Study proscribes deeper analysis of the data; while it will be possible to detect differences that exist in the mean responses to specific questions, it will not be possible 3/4 nor is it the point of this research 3/4 to explore the exact nature of these differences.

Responses provided in part I of the study will entail varying numbers of emotional attributes test subjects will ascribe to the twenty pictures they are shown. Test subjects will ascribe a "top term" designation to one of the terms chosen for each picture and those terms will be coded according to their location in the emotional clusters proposed by Shaver et al. (1987). These numerical results will be examined through correlations to see the extent to which "top terms" in the original oil on canvas format correlate with "top terms" given for the same picture in the other four reproductive formats. In addition, comparisons will be made as to the number of terms generated per subject, the number of terms generated per genre, and the number of terms generated per format. This will be achieved by calculating initial means by subject, by genre, and by format, and by comparing those means using a repeated measure Generalized Linear Model (GLM), a form of analysis of variance. Since demographic information on gender, computer use, and museum attendance is being gathered in part II of the study, tests of association will be made between these demographics and the number of terms generated by individuals in part I of the study to see whether any additional insights may be gained.

In addition to the demographic information mentioned, the data from part II of the study also includes responses to a number of survey questions designed to test viewers' perceptions of the process of looking at the artwork in part I of the Toledo Picture Study. Among the information included here are answers to questions about the viewer's confidence in the attributions provided in part I, how easy/difficult the process of making attributions was, and, how familiar the viewer was with the pictures seen in advance of the test. Test subjects are then asked about the relative ease/difficulty of identifying feelings and emotions in each of the formats viewed, the intensity of those feelings, and how successful the various surrogate formats were in their ability to re-create the feeling of looking at an oil on canvas painting. Finally, viewers are asked to rank order all five formats in terms of their ability to convey feeling or emotion. Because many of these questions asked for viewers to respond to the same question across multiple formats, multiple analyses of variance (MANOVAs) were deemed to be the statistical operation most appropriate for analyzing much of this data.

The data collected from the Toledo Picture Study yielded both qualitative and quantitative findings. Qualitative responses to the open-ended questions asked in part II of the study were collated and examined in aggregate and will be used to offer insight into results from the quantitative analysis. Qualitative responses are also likely to generate ideas for new questions that might be incorporated into subsequent research. As this is a study in progress, complete results from the data analysis are not yet available. The results to be reported in the next section arise from the univariate analysis of all data and from the multivariate analyses completed on part II of the study.

Preliminary Findings from the Toledo Picture Study

Eighty-six subjects participated in the two-part Toledo Picture Study during the summer and early fall of 2000. With a foundation based on 20 Western European and American oil paintings drawn from the collections of the Toledo Museum of Art, the study asked subjects both to identify feelings and emotions found in the pictures (each shown to test subjects in one of five different formats) and to offer responses to questions asked about the process of looking at the pictures. In part I of the study 3/4 the naming of emotions and feelings identified in the twenty paintings 3/4 the number of terms generated was calculated by subject, by genre, and by format viewed. The mean value for the number of terms assigned by all test subjects per picture was 4.9, with a standard deviation of 1.7.

The mean values for the number of terms assigned within a given genre are shown in table 1. Recast, test subjects provided mean numbers of emotional attributes, by genre, as follows: activity (6.7), genre scene (5.5), modern (4.6), landscape (4.2), and portrait (4.0). These statistics are significant at the .00 level. While no hypotheses were offered based on response by genre, the findings of Kettlewell (1988), Kettlewell, Lipscomb, Evans, and Rosston (1990), and Winston and Cupchik (1992), suggest that non-expert viewers would indeed respond differently to variations in genre viewed. The overall research design anticipated this result and controlled for any influence by genre by balancing the types of works test subjects viewed.

Table 1. Number of Emotion Terms Assigned, by Genre

Mean values for the number of terms assigned within a given format are shown in table 2. Recast, test subjects provided mean numbers of emotional attributes, by format, as follows: original (5.1), color slide (5.0), book page (4.9), digital image (4.8), and black-and-white photograph (4.7). These statistics are significant at the .39 level, which indicates no meaningful differences between these means. Given the clear preference for works in oil on canvas that emerged in part II of the study, it is interesting to note that this preference did not manifest itself in part I of the study with a higher number of emotion terms being assigned for oil-on-canvas paintings than for any of the reproductive formats. In any event, the overall research design anticipated this possible effect of format and controlled for any influence by balancing the numbers of formats test subjects viewed.

Table 2. Number of Emotion Terms Assigned, by Format

Note: Book Page and Photograph formats were shown in black and white. Original, Digital Image, and Slide formats were shown in color.

Because further data analysis has yet to be completed for part I of the study, our discussion will now turn to the results from part II of the study, the survey component of the study. In response to the question "How confident are you in the identification of feelings and emotions you did in part I of this study?," test subjects provided answers on a five-point Likert scale (with 1 indicating "not very confident" and 5 indicating "very confident"). The mean response was 3.9, with a standard deviation of .74, suggesting that test subjects held a healthy confidence in their emotional attributions 3/4 even the lower end of the range rests near the scale's midpoint.

The responses to this question almost mirror responses obtained for question 2 in part II of the test, "How easy or difficult was it for you to identify the feelings and emotions expressed in the pictures you looked at in part I of this study?" Test subjects provided answers on a five-point Likert scale (with 1 indicating "very difficult" and 5 indicating "very easy"). The mean response was 3.5, with a standard deviation of .84, indicating that while most of the test subjects had confidence in their responses, they didn't necessarily find the attribution process an easy one. Concerns about the attribution process 3/4 borne out by qualitative data gathered 3/4 center on perceived limitations of the controlled vocabulary used to make the attributions, the difficulty in selecting a single "top term" out of multiple attributions, the confounding influence of multiple points of view that test subjects perceived in several of the pictures, and worries about one's ability to distinguish the artist's intent:

I did not find appropriate words on the list to describe the vastness and complicated feelings of the respect for the powerful scenes of nature. There were an overabundance of negative words that did not allow for the wonder and delight for humans amid the wide expanse, grandeur, and dominance of natural forces.

.....

Sometimes the words that could describe my immediate reaction were not on the list 3/4 so I would find myself returning to the picture to see if I could get a different read on it 3/4 usually my first reaction took charge.

.....

The difficulty was choosing one emotion when I felt the artist was expressing several emotions simultaneously.

.....

There were two pictures that had multiple or competing emotions displayed so I had difficulty choosing an emotion for those.

.....

[The problem I had was with] [i]mages that demanded a point of view when it was clear to me that there were (at least) two widely divergent emotions going on in the image (Napoleon, the tavern scene, the desert raid).

.....

It was a bit difficult to separate my emotional reactions [from] the emotion shown on the faces of the subjects.

.....

It is difficult to separate my response from what the artist might be trying to convey.

Question 3 in part II of the study asked test subjects "How familiar were you in advance with the pictures you looked at today?" Test subjects provided answers on a five-point Likert scale (with 1 indicating "not very familiar" and 5 indicating "very familiar"). The mean response was 2.6, with a standard deviation of 1.3, indicating that test subjects were almost evenly divided across the scale. This finding thus suggests that the study's findings are not likely to be skewed by either the subject pool's advance familiarity, or lack thereof, with the works of art included in the study. The various responses to each of these first three survey questions (i.e., confidence, ease/difficulty in identifying emotion, and advance familiarity with the collections) were investigated further through multivariate analysis to examine any possible relationship that may have existed with the responses test subjects gave to the remaining survey questions. No statistically significant findings emerged.

Question 4 in part II of the study offered test subjects an opportunity to evaluate the ease or difficulty of identifying feelings and emotions as expressed in the five different formats viewed. When responding to the question, test subjects were presented with five different five-point Likert scales with response categories ranging from "very difficult" (1) to "very easy" (5).

Table 3. Ease or Difficulty of Identifying Emotion, by Format

Note: Book Page and Photograph formats were shown in black and white. Original, Digital Image, and Slide formats were shown in color.

Table 3 presents the statistics obtained in response to question 4. Recast, test subjects rated the formats on the relative ease or difficulty of identifying emotions in specific formats as follows: original (4.5), color slide (3.7), black-and-white photo (3.0), digital image (2.9), and book page (2.8). The statistics suggest clear differences with regard to the relative ease of identifying emotion in certain formats, with a real convergence of opinion (manifested as a low standard deviation) only with regard to original works of art. The findings of this analysis are significant at the .00 level.

The qualitative comments that accompanied these responses included much discussion about the shortcomings of responding to the emotional content in black-and-white images, a strong preference for large vs. small images, and a preponderance of comments about digital images:

Digital images on a computer screen 3/4 The contrast to color (or light and dark) was missing, which plays a large role in the representation of emotion. The human element or connection was missing.

.....

Computerized images 3/4 liked the color but felt too distanced from the art.

.....

I think black and white formats have an inherent glumness or "negative" feeling; digital images seem the most distant, though, because of their size and my inability to really "get into" them.

.....

Computer images 3/4 clarity and color not as vivid 3/4 the screen, keyboard, apparatus is distracting.

Ultimately, test subjects found the task of identifying feelings and emotions easiest when interacting with an original work of art. As one subject observed,

I think the only way to experience a work is to see it firsthand 3/4 when one looks at a reproduction, one can only respond to the forms and situations (within the work); I believe the real emotion is only given off by experiencing a work in its own totality.

Question 5 of part II of the study asked subjects to evaluate how intensely they experienced feelings and emotions when viewing the five test formats. When responding to the question, test subjects were presented with five five-point Likert scales with response categories ranging from "not very intensely" (1) to "very intensely" (5). Table 4 presents the statistics obtained in response to question 5. Recast, test subjects rated the formats on the relative intensity of emotion experienced as follows: original art (4.5), color slide (3.5), digital image and black-and-white photograph (2.7), and book page (2.6). The findings are significant at the .00 level. The power of the formats that used color to evoke an emotional response are clear here; it is likely that digital images might have placed higher had a larger monitor been available for use in the study. Table 4 presents the statistics obtained in response to question 5.

Table 4. Intensity of Emotions Experienced, by Format

Note: Book Page and Photograph formats were shown in black and white. Original, Digital Image, and Slide formats were shown in color.

The qualitative responses that accompanied question 5 suggest a host of influences 3/4 some physical, some emotional or spiritual in nature 3/4 that contribute to intensifying viewer responses to pictures. Among the most notable physical factors are scale, size of the image being viewed, and both richness and fidelity of color. Those positively responding to the oil on canvas format suggest other, intriguing reasons for their heightened response:

I could easily look closely or move back and see the whole image from a distance or in conjunction with its surroundings.

.....

There's nothing that can replace, to me, an original. The colors and their variation, the light, and its reflection 3/4 none of that "value" is as engaging in a format other than the real one. Slides are nice, digitals are good, but there's an interaction that comes only with the original work 3/4 a presence.

.....

I think seeing the created object first hand reminded me of its creator. This is not to say biography, etc., just that it has a maker, and that [the] maker presumably had intent, intensified the viewing.

.....

When viewing the oil on canvas pictures I felt a personal connection between the artist and the subjects and myself.

.....

Oil on canvas gave me a feeling of closeness to the artist.

Question 6 of part II of the study asked subjects to assess the four surrogate formats in their ability to re-create the feeling of looking at an oil on canvas painting. When responding to the question, test subjects were presented with four five-point Likert scales with response categories ranging from "not very successful" (1) to "very successful" (5). Table 5 presents the statistics obtained in response to question 6. Recast, test subjects rated the formats on the relative ability to re-create the feeling of looking at an oil on canvas painting as follows: color slide (3.6), digital image (2.7), book page (2.5), and black-and-white photograph (2.2). The findings are significant at the .00 level. The responses here almost exactly mirror the responses gathered in the previous question (with the single exception of black-and-white photographs) and serve to reconfirm both sets of findings, i.e. that color slides are perceived to be the single format "most like" oil on canvas paintings, with the other three surrogate formats trailing at by a considerable distance.

Table 5. Ability to Re-Create Feeling of an Oil on Canvas Painting, by Format

Note: Book Page and Photograph formats were shown in black and white. Original, Digital Image, and Slide formats were shown in color.

Finally, test subjects were asked to provide a rank ordering of the formats observed in response to the instruction, "Please rank order the formats you looked at today in terms of their ability to convey feelings and emotions." For the rank ordering, test subjects assigned the values 1 to 5 once each across all five formats; the assignment of 1 indicated the format best able to convey feelings and emotions, a 5 indicated the format worst at accomplishing this. The findings presented are significant at the .00 level. The results of this rank ordering are provided in table 6.

Table 6. Ranked Ability to Convey Feelings and Emotions, by Format

Note: Book Page and Photograph formats were shown in black and white. Original, Digital Image, and Slide formats were shown in color.

This was the first question in which test subjects were asked to make direct comparisons across all five formats. This forced ranking supports findings that were suggested by previous questions in part II of the test: oil on canvas originals are clearly perceived to be best able to impart feeling and emotion. Color slides, once again, place strongly, but book pages and black-and-white photographs trail significantly behind all other formats.

As has been mentioned, demographic data with regard to gender, computer use, and museum attendance was gathered near the end of the part II survey. Eighty-six individuals participated in the Toledo Picture Study: 75% of test subjects were female, 25 % were male. In response to the question "How often do you use a computer to search the World Wide Web?," test subjects provided answers on a five-point scale (with 1 indicating "never" and 5 indicating "daily"). The mean response was 3.7 (a value falling within the "monthly" category), with a standard deviation of 1.4, suggesting not much regular web viewing but a certain polarization of tendencies within the test group. In response to the final demographic question, "How often do you visit museums?" test subjects provided answers on a five-point scale (with 1 indicating "this is my first visit to a museum" and 5 indicating "5 or more times a year"). The mean response was 3.9 (a value falling within the "1-2 times a year" category), with a standard deviation of 1.0, suggesting that a least half the study might be characterized as frequent museum visitors. Attempts to correlate gender, computer use, and museum attendance with responses given to the survey questions failed to identify any relationships that might exist at statistically significant levels. This was not anticipated in this study's hypotheses nor is it what has been suggested by the findings of other studies which have suggested that at least frequent museum attendance has an effect on one's ability to interpret art (Eversmann, Krill, Michael, Twiss-Garrity, and Beck, 1997). While this doesn't compromise the results observed in the Toledo Picture Study, it does leave certain observed phenomena needing explanation.

In all, however, statistical findings from the test provide consistent and persuasive support for the research hypotheses advanced earlier. Responses to the final question of the survey, which asked test subjects to offer any final observations about the five presentation formats, leaves little doubt as to where they stand:

I am here from Cincinnati to see fine art originals. Could have just sent for a museum collection catalogue if pictures were enough. But, they never are. The soul of the artist is only seen by standing in front of the work.

.....

After viewing the reproductions, which were of good quality, the impact of the originals is overwhelming!

.....

I noticed immediately how quickly and how much more excited I was about generating emotions while viewing the oil on canvas works in the gallery. I was disappointed about how quickly that portion went because it was so much more exhilarating.

.....

The real thing is the real thing. The rest is second rate.

Interpretation of Findings

Given the fact that data analysis is still underway on part I of the study data, it is, of course, premature to offer any conclusive statements. Part II of the study data does suggest that 3/4 in any number of ways 3/4 Toledo Picture Study test subjects do, in fact, make distinctions in the nature of their response to original works of art and surrogate representations. Test data show that statistically significant differences exist with regard to the intensity of the emotions they feel, the ease with which they identify feelings and emotions, and in terms of the ability of an original work of art to convey feeling or emotion. The qualitative data suggests that such distinctions are most often discussed in terms of the easily observed physical differences that exist between surrogates and original works of art, i.e. relative size, scale, and richness and fidelity of color, with black and white reproductions suffering considerably in comparison with both originals works of art and other surrogates shown in color.

The most interesting qualitative data directly addresses differences between original works of art and surrogates in terms of the emotional experience or at least in terms of their personal response. These responses, in particular those that mentioned the effect of the gallery's parquet floors and deeply-hued walls, the presence of others in the gallery, or those individuals who shared stories of past experiences at the museum as children, with their families, or as volunteers, all point to exciting new directions for future investigation. This research has focused almost entirely on differences that exist within the physical object; future studies might be well served to look at the gallery itself and it's effect on the way individuals respond to the physical object. Perhaps the differences that have shown themselves in this study may be explained more easily by focusing on people, not artifacts.

Significance of the Study

This study should offer findings of significant interest to those seeking to expand access to museum and visual resource collections to non-specialist users. The process of testing on multiple surrogate types is expected to suggest which formats best convey the essence of a work to those individuals searching without the benefit of expert language or a knowledge of artists, art historical periods, or artistic styles. This should allow those developing systems or sites intending to serve general audiences to gain special insight on how to develop their resources in ways that will be most responsive to their viewers. Resulting innovations might take the form of special user interfaces, the inclusion of additional search fields, or additional guides or search support designed for non-specialists. Similarly, findings about expressional qualities might provide specific direction on how to enhance existing systems for more effective use not only by non-specialists but by others for whom such access might be highly desirable, e.g. those looking for images for use in advertising or design. Finally, the ability to extend access to the vast audience that exists beyond art history scholars and museum professionals not only serves to advance greater educational, cultural, and societal goals, but serves to restore this body of documentation to the greater public from which it originated.

References

Armitage, Linda H. and Peter G.B. Enser. "Analysis of User Need in Image Archives." Journal of Information Science 23 (1997), no. 4, 287-99

Arnheim, Rudolf. "Expression." In Art and Visual Perception: A Psychology of the Creative Eye (Berkeley: Univ. California Press, 1974), pp. 444-61

Bakewell, Elizabeth, William O. Beeman, and Carol McMichael Reese. Object, Image, Inquiry: The Art Historian at Work. Santa Monica, Calif.: The Getty Art History Information Program, 1988

Barry, Ann Marie Seward. Visual Intelligence: Perception, Image, and Manipulation in Visual Communication. Albany, N.Y.: State University of New York, 1997

Batley, Sue. "Visual Information Retrieval--Browsing Strategies in Pictorial Databases." In Online Information 88: Proceedings of the 12th International Online Information Meeting (Medford, N.J.: Learned Information, 1988), pp. 373-81

-----. "Visual Information Retrieval: Browsing Strategies in Pictorial Databases." Ph. D. dissertation. Aberdeen, U.K.: University of Aberdeen, 1988

Bearman, David and Jennifer Trant. "Authenticity of Digital Resources: Towards a Statement of Requirements in the Research Process." D-Lib Magazine (June 1998), p.n.p.

Bednarek, Martina. "Intellectual Access to Pictorial Information." Australian Library Journal 42 (1993), no. 1, 33-46

Bell, Lesley Anne. "Gaining Access to Visual Information: Theory, Analysis, and Practice of Determining Subjects: A Review of the Literature with Descriptive Abstracts." Art Documentation 13 (1994), no. 2, 89-94

Benjamin, Walter. "The Work of Art in the Age of Mechanical Reproduction." In Illuminations (New York: Schocken Books, 1986; rpt.), pp. 217-51

Berlyne, D.E., ed. Studies in the New Experimental Aesthetics: Steps toward an Objective Psychology of Aesthetic Appreciation. Washington, D.C.: Hemisphere Publishing, 1974

Bezruczko, Nikolaus and David H. Schroeder. "Differences in Visual Preferences and Cognitive Aptitudes of Professional Artists and Nonartists." Empirical Studies of the Arts 12 (1994), no. 2, 19-39

Broadbent, Elaine. "A Study of Humanities Faculty Library Information Seeking Behavior." Cataloging & Classification Quarterly 6 (1986), no. 3, 23-37

Bronner, Simon J. "Chain Carvers in Southern Indiana: A Behavioristic Study in Material Culture." Ph.D. dissertation. Bloomington, Ind.: University of Indiana, 1981

Bruce, Roger. "Altering the Culture and the Identity of the Museum: The Risks and Benefits of Providing Networked Information." In Patricia McClung and Christie Stephenson, eds., Images Online: Perspectives on the Museum Educational Site Licensing Project (Los Angeles: The Getty Information Institute, 1998), pp. 57-62

Buswell, Guy Thomas. How People Look at Pictures: A Study of the Psychology of Perception in Art. Chicago: University of Chicago, 1935

Cameron, Duncan F. et al. "Public Attitudes toward Modern Art." Museum 22 (1969), nos. 3/4 , 129-47

Case, Donald Owen. "The Collection and Use of Information by Some American Historians: A Study of Motives and Methods." Library Quarterly 61 (1991), no. 1, 61-82

Cawkell, A.E. "Picture Queries and Picture Databases." Journal of Information Science 19 (1993), no. 6, 409-23

Cities of the United States. Second edition. Detroit: Gale Research, Inc., 1994

City Profiles USA. Detroit: Omnigraphics, Inc., 1999

Cobbledick, Susie. "The Information-Seeking Behavior of Artists: Exploratory Interviews." Library Quarterly 66 (1996), no. 4, 343-72

Collins, Karen. "Providing Subject Access to Images: A Study of User Queries." Master's thesis. Chapel Hill, N.C.: University of North Carolina, 1997

Commission on Preservation and Access. Scholarly Resources in Art History: Issues in Preservation. Washington, D.C.: Commission on Preservation and Access, 1989

Corkill, Cynthia and Margaret Mann. Information Needs in the Humanities: Two Postal Surveys. Sheffield, U.K.: Centre for Research on User Studies, University of Sheffield, 1978

Council on Library and Information Resources. Scholarship, Instruction, and Libraries at the Turn of the Century. Washington, D.C.: The Council, 1999

Cupchik, Gerald C. and Robert J. Gebotys. "Interest and Pleasure as Dimensions of Aesthetic Response." Empirical Studies of the Arts 8 (1990), no. 1, 1-14

Cupchik, Gerald C., Andrew S. Winston, and Rachel S. Herz. "Judgments of Similarity and Difference between Paintings." Visual Arts Research 18 (1992), no. 2, 37-50

Encyclopedia of World Art. 17 vols. London: McGraw-Hill, 1966

Enser, P.G.B. "An Indexing-Free Approach to the Retrieval of Still Images." In British Computer Society 13th Information Retrieval Colloquium (Lancaster, U.K.: T. McEnery, 1991), pp. 41-55

-----. "Query Analysis in a Visual Information Retrieval Context." Journal of Document & Text Management 1 (1993), no. 1, 25-51

-----. "Pictorial Information Retrieval." Journal of Documentation 51 (1995), no. 2, 126-70

Enser, P.G.B. and C.G. McGregor. Analysis of Visual Information Retrieval Queries. British Library R & D Report 6104. London: The British Library Board, 1993

Eversmann, Pauline K., Rosemary T. Krill, Edwina Michael, Beth A. Twiss-Garrity, and Tracy Rae Beck. "Material Culture as Text: Review and Reform of the Literacy Model for Interpretation." In Ann Smart Martin and J. Ritchie Garrison, eds., American Material Culture: The Shape of the Field (Winterthur, Del.: Henry Francis du Pont Winterthur Museum, 1997), pp. 135-67

Fawcett, Trevor. "Subject Indexing in the Visual Arts." Art Libraries Journal 4 (1979), no. 1, 5-17

Firshein, O. and M.A. Fischler. "Describing and Abstracting Pictorial Structures." Pattern Recognition 3 (1971), no. 4, 421-43

Franklin, Margery B., Robert C. Becklen, and Charlotte L. Doyle. "The Influence of Titles on How Paintings Are Seen." Leonardo 26 (1993), no. 2, 103-8

Frost, Carolyn O. "The University of Michigan School of Information Art Image Browser: Designing and Testing a Model for Information Retrieval." In Rebecca Green, ed., Knowledge Organization and Change: Proceedings of the Fourth International ISKO Conference (Frankfurt/Main: Indeks Verlag, 1996), pp. 182-88

Frost, C. Olivia, Karen M. Drabenstott, and Joseph Janes. "Integrating an Image Database into Gopher (School of Information Art Image Browser)." Ann Arbor, Mich.: University of Michigan School of Information, 1997

Frost, C. Olivia and Anna Noakes. "Browsing Patterns Using Broad Classification Categories." In E.K. Jacob, ed., Proceedings of the 9^th ASIS SIG/CR Classification Research Workshop (Medford, N.J.: American Society for Information Science, 1998), pp. 71-89

Frost. C. Olivia, Bradley Taylor, Anna Noakes, Stephen Markel, Deborah Torres, and Karen M. Drabenstott. "Browse and Search Patterns in a Digital Image Database." Information Retrieval 1 (2000), 287-313

Furnas, G.W., T.K. Landauer, L.M. Gomez, and S.T. Dumais. "The Vocabulary Problem in Human-System Communication." Communications of the ACM 30 (1987), no. 11, 964-71

Hardiman, George W. "Identification and Evaluation of Trained and Untrained Observers' Affective Responses to Art Objects: Final Report." Urbana, Ill.: Univ. of Illinois, Dept. of Art. March 1971 ED 052 219

Hastings, Samantha. "An Exploratory Study of Intellectual Access to Digitized Art Images." Ph.D. dissertation. Tallahassee, Fla.: Florida State University, 1994

Hewins, Elizabeth, "Information Need and Use Studies." Annual Review of Information Science and Technology 25 (1990), 145-72

Hindle, Brooke. "How Much Is a Piece of the True Cross Worth?" In Ian M.G. Quimby, ed., Material Culture and the Study of American Life (New York: W.W. Norton, 1978), pp. 5-20

Hooper-Greenhill, Eilean. Museums and Their Visitors. London: Routledge, 1994

Hourihane, Colum. "A Selective Survey of Systems of Subject Classification." In Anthony Hamber, Jean Miles, and William Vaughan, eds., Computers and the History of Art (London: Mansell, 1989), pp. 117-29

Ingwersen, Peter. "Search Procedures in the Library--Analysed from the Cognitive Point of View." Journal of Documentation 38 (1982), no. 3, 165-91

Jenks, Chris, ed. Visual Culture. London: Routledge, 1995

Jones, Lois Swan. Art Information: Research Methods and Resources. Third edition. Dubuque, Iowa: Kendall/Hunt Publishing Co., 1990

Jorgensen, Corinne. "Image Attributes: An Investigation." Ph.D. dissertation. Syracuse, N.Y.: Syracuse University, 1995

-----. "The Applicability of Selected Classification Systems to Image Attributes." Advances in Knowledge Organization 5 (1996), 189-97

-----. "Attributes of Images in Describing Tasks." Information Processing & Management 34 (1998), nos. 2/3, 161-74

-----. "Retrieving the Unretrievable in Electronic Imaging Systems: Emotions, Themes, and Stories." In Bernice E. Rogowitz and Jan P. Allebach, eds., Retrieving the Unretrievable: Art, Aesthetics, and Emotion in Image Retrieval Systems (San Jose, Calif.: International Society for Optical Engineering, 1999), pp. 348-55

-----. "Access to Pictorial Material: A Review of Current Research and Future Prospects." Computers and the Humanities 33 (1999), 293-318

Jussim, Estelle. "The Research Uses of Visual Information." Library Trends 25 (1977), no. 4, 763-78

Keister, Lucinda. "User Types and Queries: Impact on Image Access Systems." In Raya Fidel, Trudi Bellardo Hahn, Edie M. Rasmussen, and Philip J. Smith, eds., Challenges in Indexing Electronic Text and Images (Medford, N.J.: Learned Information, 1994), pp. 7-19

Kettlewell, Neil. "An Examination of Preferences for Subject Matter in Painting." Empirical Studies of the Arts 6 (1988), no. 1, 59-65

Kettlewell, Neil, Sara Lipscomb, Liz Evans, and Karl Rosston. "The Effect of Subject Matter and Degree of Realism on Aesthetic Preferences for Paintings." Empirical Studies of the Arts 8 (1990), no. 1, 85-93

Korf Vidal, Noni. "Experimental Image Taxonomy: An Inquiry into Spontaneous Image Organization." Master's thesis. Ithaca, N.Y.: Cornell University, 1995

Krause, Michael G. "Intellectual Problems of Indexing Picture Collections." Audiovisual Librarian 14 (1988), no. 2, 73-81

Kuhlthau, Carol Collier. "Inside the Search Process: Information Seeking from the User's Perspective." Journal of the American Society for Information Science 42 (1991), no. 5, 361-71

-----. "A Principle of Uncertainty for Information Seeking." Journal of Documentation 49 (1993), no. 4, 339-55

Layne, Sara Shatford. "Artists, Art Historians, and Visual Art Information."

Reference Librarian 1994, no. 47, 23-36

-----. "Some Issues in the Indexing of Images." Journal of the American Society for Information Science 45 (1994), no. 8, 583-88

Leung, C.H.C., D. Hibler, and N. Mwara. "Picture Retrieval by Content Description." Journal of Information Science 18 (1992), no. 2, 111-19

Macdonald, Sharon and Gordon Fyfe, eds. Theorizing Museums: Representing Identity and Diversity in a Changing World. Cambridge, Mass.: Blackwell, 1996

Markey, Karen. "Computer-Assisted Construction of a Guide to Themes and Concepts in Medieval Art and Their Essential Attributes." Ph.D. dissertation. Syracuse, N.Y.: Syracuse University, 1981

-----. "Computer-Assisted Construction of a Thematic Catalog of Primary and Secondary Subject Matter." Visual Resources 3 (1983), 16-49

-----. "Visual Arts Resources and Computers." In Martha E. Williams, ed., Annual Review of Information Science and Technology (Medford, N.J.: Learned Information, 1984), pp. 271-309

-----. Subject Access to Visual Resource Collections: A Model for Computer Construction of Thematic Catalogs. New York: Greenwood Press, 1986

------. "Access to Iconographical Research Collections." Library Trends 37 (1988), no. 2, 154-74

Martin, Ann Smart and J. Ritchie Garrison, eds. American Material Culture:

The Shape of the Field. Winterthur, Del.: Henry Francis du Pont Winterthur Museum, 1997

McClung, Patricia, ed. RLG Digital Image Access Project. Mountain View, Calif.: The Research Libraries Group, 1995

McClung, Patricia and Christie Stephenson, eds. Images Online: Perspectives on the Museum Educational Site Licensing Project. Los Angeles: The Getty Information Institute, 1998

McCormick, Douglas Jess. "Perception and Appreciation of Famous Paintings." Ph.D. dissertation. University of Southern California, 1983

McLaughlin, John P. and Kimberly E. Murphy. "Preference for Profile Orientation in Portraits." Empirical Studies of the Arts 12 (1994), no. 1, 1-7

McMurtrie, Douglas C. The Book: The Story of Printing & Bookmaking. New York: Covici Friede, 1937

McRanor, Shauna. "A Critical Analysis of Intrinsic Value." American Archivist 59 (1996), no. 4, 400-411

Montgomery, Douglas C. Design and Analysis of Experiments. Third edition. New York: John Wiley & Sons, 1991

Mostafa, Javed. "Digital Image Representation and Access." In Martha E. Williams, ed., Annual Review of Information Science and Technology (Medford, N.J.: Learned Information, 1994), pp. 91-135

Niblack, W. and M. Flickner. "Find Me Pictures That Look Like This: IBM's Image Query Project." Advanced Imaging 8 (1993), no. 4, 132-35

O'Connor, Brian C., Mary K. O'Connor, and June M. Abbas. "User Reactions as Access Mechanism: An Exploration Based on Captions for Images." Journal of the American Society for Information Science 50 (1999), no. 8, 681-97

Official Museum Directory. New Providence, N.J.: National Register Publishing Co., 1999

Ohlgren, Thomas H. "Subject Access to Iconographic Data Bases: Theory and Practice." In Joseph Raben and Gregory Marks, eds., Data Bases in the Humanities and Social Sciences (Amsterdam: North-Holland Publishing Company, 1980), pp. 245-50

-----. "Subject Indexing of Visual Resources: A Survey." Visual Resources 1 (1980), no. 1, 67-73

Olmert, Michael. The Smithsonian Book of Books. Washington, D.C.: Smithsonian Books, 1992

Ornager, Susanne. "Theoretical and Empirical User Studies on Accessing Information in Images." In Candy Schwartz and Mark Rorvig, eds., Digital Collections: Implications for Users, Funders, Developers, and Maintainers (Medford, N.J.: American Society for Information Science, 1997), pp. 202-11

Osborne, Harold, ed. The Oxford Companion to Art. Oxford: The Clarendon Press, 1970

Pacey, Philip. "How Art Students Use Libraries--If They Do." Art Libraries Journal 7 (1982), no. 4, 117-19

Palmer, Arlene. "Through the Glass Case: The Curator and the Object." In Ian M.G. Quimby, ed., Material Culture and the Study of American Life (New York: W.W. Norton, 1978), pp. 219-44

Panofsky, Erwin. Meaning in the Visual Arts. Garden City, N.Y.: Doubleday & Co., Inc., 1955

-----. "Original et Reproduction en Fac-simile¢." Cahiers du Musee National d'art Moderne 53 (1995), 45-73

Pommerehne, Werner W. and J. Martin Garcia. "Perfect Reproductions of Works of Art: Substitutes or Heresy?" Journal of Cultural Economics 19 (1995), 237-49

Profiles of America. Milipitas, Calif.: Toucan Valley Publications, Inc., 1995

Prown, Jules David. "Style as Evidence." Winterthur Portfolio 15 (1980), no. 3, 197-210

-----. "Mind in Matter: An Introduction to Material Culture Theory and Method." Winterthur Portfolio 17 (1982), no. 1, 1-18

-----. "The Truth of Material Culture: History or Fiction?" In Steven Lubar and W. David Kingery, eds., History from Things: Essays on Material Culture (Washington, D.C.: Smithsonian Institution Press, 1993), pp. 1-19

Quimby, Ian M.G., ed. Material Culture and the Study of American Life. New York: W.W. Norton & Co., 1978

Rasmussen, Edie M. "Indexing Images." In Martha E. Williams, ed., Annual Review of Information Science and Technology (Medford, N.J.: American Society for Information Science, 1998), pp. 169-96

Roberts, Helene. "Second Hand Images: The Role of Surrogates in Artistic and Cultural Exchange." Visual Resources 9 (1994), no. 4, 335-46

Roddy, Kevin. "Subject Access to Visual Resources: What the 90s Might Portend." Library Hi Tech 9 (1991), no. 1, 45-49

Romer, Donna M. "Image and Multimedia Retrieval." In Research Agenda for Networked Cultural Heritage (Los Angeles: The Getty Art History Information Program, 1996), pp. 49-56

Sanders, Elma. "Originals and Copies: The Meaning in the Object." Visual Resources 6 (1989), no. 2, 181-86

Sarasan, Lenore. "Visual Content Access: An Approach to the Automatic Retrieval of Visual Information." In Laura Corti, ed., Automatic Processing of Art History Data and Documents (Pisa: Scuola Normale Superiore, 1984), pp. 388-405

Sarasan, Lenore and A. Michael Neuner, comps. Museum Collections and Computers: Report of an ASC Survey. Lawrence, Kansas: Association of Systematics Collections, 1983

Schlebecker, John T. "The Use of Objects in Historical Research." In Thomas J. Schlereth, comp. and ed., Material Culture Studies in America (Nashville, AASLH Press, 1982), pp. 106-13

Schlereth, Thomas J. Artifacts and the American Past. Nashville, Tenn.: American Association for State and Local History, 1980

Schwartz, Hillel. The Culture of the Copy: Striking Likenesses, Unreasonable Facsimiles. Cambridge, Mass.: MIT Press, 1996

Shatford, Sara. "Describing a Picture: A Thousand Words Are Seldom Cost Effective." Cataloging & Classification Quarterly 4 (1984), no. 4, 13-30

-----. "Analyzing the Subject of a Picture: A Theoretical Approach." Cataloging & Classification Quarterly 6 (1986), no. 3, 39-62

Shaver, Phillip, Judith Schwartz, Donald Kirson, and Cary O'Connor. "Emotion Knowledge: Further Exploration of a Prototype Approach." Journal of Personality and Social Psychology 52 (1987), no. 6, 1061-86

Sklar, Hinda F. "Why Make Images Available Online: User Perspectives." In Patricia A. McClung, ed., RLG Digital Image Access Project (Mountain View, Calif.: Research Libraries Group, 1995), pp. 11-18

Stam, Deirdre Corcoran. "How Art Historians Look for Information." Art Documentation 3 (1984), no. 4, 117-19

-----. "The Information-Seeking Practices of Art Historians in Museums and Colleges in the United States, 1982-1983." Ph. D. dissertation. New York: Columbia University, 1984

-----. "Tracking Art Historians: On Information Needs and Information- Seeking Behaviour." Art Libraries Journal 14 (1989), no. 3, 13-16

-----. "Artists and Art Libraries." Art Libraries Journal 20 (1995), no. 2, 21-24

Steinberg, S.H. Five Hundred Years of Printing. Third edition. Middlesex, U.K.: Penguin Books, 1977

Stephenson, Christie and Patricia McClung, eds. Delivering Digital Images: Cultural Heritage Resources for Education. Los Angeles: The Getty Information Institute, 1998

Stoan, Stephen K. "Research and Information Retrieval among Academic Researchers: Implications for Library Instruction." Library Trends 39 (1991), no. 3, 238-58

Stone, Sue. "Humanities Scholars: Information Needs and Uses." Journal of Documentation 38 (1982), no. 4, 292-313

Sutherland, John. "Image Collections: Librarians, Users, and Their Needs." Art Libraries Journal 7 (1982), no. 2, 41-49

Svenonius, Elaine. "Access to Nonbook Materials: The Limits of Subject Indexing for Visual and Aural Languages." Journal of the American Society for Information Science 45 (1994), no. 8, 600-606

Taylor, Robert S. "Question-Negotiation and Information Seeking in Libraries." College and Research Libraries 29 (1968), 178-94

Thompson, Colin. "Why Do You Need to See the Original Painting Anyway?" Visual Resources 2 (1981/82), nos. 1/2/3, 21-36

Tibbo, Helen. "Indexing for the Humanities." Journal of the American Society for Information Science 45 (1994), no. 8, 607-19

Tilghman, B.R. The Expression of Emotion in the Visual Arts: A Philosophical Inquiry. The Hague: Martinus Nijhoff, 1970

Toledo Museum of Art. European Paintings. Toledo, Ohio: The Toledo Museum of Art, 1976

-----. American Paintings. Toledo, Ohio: The Toledo Museum of Art, 1979

Turner, Jane, ed. The Dictionary of Art. 34 vols. London: Macmillan, 1996

Turner, James M. "Subject Access to Pictures: Considerations in the Surrogation and Indexing of Visual Documents for Storage and Retrieval." Visual Resources 9 (1993), no. 3, 241-71

-----. "Determining the Subject Content of Still and Moving Image Documents for Storage and Retrieval: An Experimental Investigation." Ph.D. dissertation. Toronto: University of Toronto, 1994

-----. "Indexing 'Ordinary' Pictures for Storage and Retrieval." Visual Resources 10 (1994), no. 3, 265-73

Walker, Alvin, Jr., ed. Thesaurus of Psychological Index Terms. Eighth edition. Washington, D.C.: American Psychological Association, 1997

Watson-Boone, Rebecca. "The Information Needs and Habits of Humanities Scholars." RQ 34 (1994), no. 2, 203-16

Weintraub, Karl J. "The Humanistic Scholar and the Library." Library Quarterly 50 (1980), no. 1, 22-39

Wiberly, Stephen, Jr. and William G. Jones. "Patterns of Information Seeking in the Humanities." College and Research Libraries 50 (1989), no. 6, 638-45

Wilson, Thomas D. "On User Studies and Information Needs." Journal of Documentation 37 (1981), no. 1, 3-15

-----. "The Cognitive Approach to Information-Seeking Behavior and Information Use." Social Science Information Studies 4 (1984), nos. 2/3, 197-204

Winston, Andrew S. and Gerald C. Cupchik. "The Evaluation of High Art and Popular Art by Naïve and Experienced Viewers." Visual Arts Research 18 (1992), no. 1, 1-14

Acknowledgments

Special thanks are extended to Patricia Whitesides, Registrar of the Toledo Museum of Art, the staff of the Toledo Museum of Art, and the kind people of the Toledo community, all of whom contributed generously to the success of this research.