How did I get here?

Political cartoons are an interesting combination of image and text that provide learning and research materials for areas of study including art, graphic design, history, political science and journalism. For my final collection project paper, I sought to draw upon and synthesize what I have learned so far about the visual digital landscape and to apply it to a proposal for a family collection of political cartoons and drawings produced by a great uncle, Guido Janes, from about 1920 to 1950.

Most of the objects for this collection are currently in storage so I was somewhat limited in the ability to get down to the specific items in the collection. I wanted to provide the background information that could be used to present the concept to family members as well as to an academic library that might be interested in actually building the collection. Since this was a class project, I also wanted to include some of the general concepts that apply to digital collections.

In the future, I would like to start adding more of the detail that will apply to this specific collection. There are a two links below that can illustrate how a collection of this type might be utilized by a university library.  

The final link shows a collection that features an aunt of my great uncle, Cora Benneson. One of his pen and ink drawings is included in the collection but unfortunately no cartoon examples. The Cora Benneson site could also serve as another type of option for this collection that would have a regional historical application.

Example of an academic library political cartoon collection:

George Washington University Guide to the Clifford K. Berryman Collection: Collection Organization, Subject Terms, Detailed Descriptions of the Records

http://www.aladin.wrlc.org/gsdl/collect/faids/import/MS2024.shtml

Clifford K. Berryman Collection: About This Collection

http://www.aladin.wrlc.org/gsdl/collect/berryman/berryman.shtml

Cora Benneson Site:

Alliance Library System: Early Illinois Women, Cora Agnes Benneson , Childhood home – (example of pen and ink drawing by Guido D. Janes, nephew)

http://www.alliancelibrarysystem.com/IllinoisWomen/files/qp/htm1/qp000007.html

Some final thoughts…

As we have been examining the use of images in the digital landscape, we have hopefully addressed some of the issues that are faced not only in using images but the manner in which profound changes are taking place in the evolution of the meaning we find in visual information as it continues to evolve. The ability to bring together so many vast resources is both exhilarating and overwhelming. Those of us in libraries and in the role of educators have great opportunities to make some of those meaningful connections for our patrons and students. We at least can be a part of providing the environment in which individuals can seek out that meaning on their own.

For learning, resources must be contextualized to determine situational relevance and meaning. Resources also need to be recontextualized to enable the use of information gleaned from various resources (Hill and Hannafin 2001, 38).

Images can bring such richness to our world as well as create controversy. So much visual information comes and goes so quickly and much of it is not analyzed as to the information content that is being communicated, exchanged or manipulated. The power inherent in the visual and the speed with which it is increasing makes the use of visual information in the learning environment more vital than ever.

Our ability to use images for instruction now goes far beyond what has been possible in the past. The following is no less true for images than for any other type of information.

Multiple resources may be aggregated within a single application, but can be reused in theoretically unlimited combinations…Meaning is influenced more by the diversity than the singularity of the perspective taken. Multiple resources are accessed and interpreted for meaning, evaluated for veracity and utility, compared with competing perspectives… (40).

The dynamic nature of  the digital environment challenges all of us with technical and legal considerations that become an everyday part of our efforts on behalf of our patrons and/or students. For this among other reasons, it is important for us to retain an understanding and appreciation of the pre-digital world of images in order to better serve their use in the digital landscape.

As we have traveled over some small portion of the visual digital landscape, I might have started with the words of Nicholas Burbules when he states that:

I would like to begin this essay with the experience of getting lost on the World Wide Web…Movement along this path represents a decision by the user to leave the known content of the page on the screen to an unknown page whose content may edify, surprise, or confuse.

I am still  wandering at best and not nearly content with where I have gotten. In my next addition to this journey, I will present an explanation of a final paper for my Digital Collections class. The exercise poked holes in the coherence of my journey and I may need some time to regroup before I am ready to start out again. I find myself in that very painful state of aporia but as Burbules also goes on to say, “There is no way to remain open to the possibility of happening upon (new things) if we are not prepared to accept getting lost.”

Burbules, Nicholas C. 1997. Aporia: Webs, Passages, Getting Lost, and Learning to Go On. (Accessed 12/5/2008 http://www.ed.uiuc.edu/EPS/PES-yearbook/97_docs/burbules.html)

Hill, Janette R. and Hannafin, Michael. 2001. Teaching and learning in digital environments. Educational Technology Research & Development 49(3): 37-52

Image Use & Intellectual Property Rights

Last time we touched a bit on the issue of copyright and the use of images in the academic environment. In the VIUS study that we looked at, student users were found to be utilizing primarily Internet sources for their images. I recently came across a 1998 study, predating the VIUS study, that examined student use of images taken from the Web.  The authors used web pages created by students and examined the homepage of the site for any intellectual property right infringements.

As we know (or maybe not) from copyright law, it is difficult to determine if the use of materials is an infringement or not. Students may be aware that Copyright Law protects use for “scholarship and research” but not be aware that the protection is not absolute. The law is kept purposely vague so that each use can be examined on a case by case basis. As stated by the research authors,

Given the nature of copyright law, it is not possible to say definitively that any particular use of protected material constitutes an actionable infringement. Only after the property owner claims that an infringement has occurred is it necessary for the court to determine whether a particular use is fair or whether it is a violation of the Copyright Law. Recognizing this fact, it is not possible for an objective observer to say with confidence that a particular use of intellectual property is an infringement.

In the research study, the authors developed a 3 tier system to evaluate the use of images by students. The first level was the most exact and included the use of images that specifically showed a “copyright, reserved, or trademark seal” (Herbeck and Hunter 1998, 59). The next 2 levels were left more to the interpretation of the researchers. Level 2 included those images that the researchers felt exhibited “probable use of intellectual property (such as) photographs of professional sports teams and celebrities…syndicated cartoons and television characters” (59). Level 3 included such things as “school specific graphics, including images of institutional photos, seals, logos, shields, mascots, and fraternity and sorority seals” (59).

Taken collectively, the researchers found “that 42.5 percent of pages contained some form of likely violation” (59). Since this study was done 10 years ago, I would suspect that the problem is even greater today. The authors rightly felt that students did not fully understand the implications of their use of images. It seems that students are not alone in this. On the other hand, we see the “chilling effect” on the use of images that comes from the growing fear of litigation by other groups such as faculty.

Another point brought up by the researchers was that of the institutional responsibility that the university might have in allowing students to improperly use protected images. One of the important aspects of information literacy instruction is teaching students to use materials ethically and responsibly. It is no less important with images than with text based materials, but the confusion over the line between “fair use” for scholarship and research and the violation of intellectual property rights leaves us all in a state of uncertainty. This leaves me with more concern about the nature of the electronic environment and our ability to manage it in such a way that does not impede the potential for intellectual inquiry that is it’s great promise.

Herbeck, Dale A., and Christopher D. Hunter. 1998. Intellectual property in cyberspace: The use of protected images on the World Wide Web. Communication Research Research Reports 15(1): 57-63. Communication and Mass Media Complete, EBSCOhost (last accessed November 23, 2008)

Academic Image Use

Well, I am not sure what happened but I just lost an hours’ worth of writing! This will be a bit shorter than I had intended but I am continuing on with my thoughts on the use of images in the academic environment. I had mentioned a user study in an earlier post, the Visual Image User Study (VIUS), that had some interesting findings. Although the study was started in 2001 and completed in 2003, there were many issues raised that are just as applicable today. The study found a high demand for the use of images across many academic areas. There was, as expected, higher image use in the arts for example but also some unexpected results such as higher digital, versus analog, image use in environmental studies as compared to the arts.

While faculty and students were interested in the idea of a campus integrated image delivery system, both user groups were concerned about the content of such a system in contrast to the image sources they were primarily using. While art history faculty might have been relying more on collections of analog images in the form of 35 mm slides, student users were utilizing Web resources. For this reason, library image database use was only moderate even though demand for images was high.

On of the other issues of concern to faculty was copyright. I would suspect that this is even more of a concern 5 years after the completion of the VIUS study. In our current economic and regulatory climate, I am a bit disheartened by an atmosphere that is increasingly restrictive in a time when it is even more important to study the use of images as we are experiencing such an explosion of visual information.

In the writing that I lost, I delved a bit more into copyright issues which I will have to return to at a later point. Suffice it to say, that an integrated campus image delivery system might be a way to address copyright concerns for users and copyright holders. Users concern for content however, is not a matter of copyright issues alone. There must be sufficient quantity and quality of images as well. Just knowing that images are available for use is not enough if the images are so limited as to discourage utilizing the database.

An article by Henry Pisciotta, written before the results of the VIUS study came out, addresses this problem. Pisciotta states that, “…critical mass is defined as a problem of matching sufficient quantity of database images to a sufficient quantity of users.” Studies such as the VIUS study can provide the support for the development of image databases that will provide the type of content that users want. The challenge will be to compete with the Web. As noted by Pisciotta in his 2003 article,

The Web apparently contains more than ten billion pictures…However, even if incorrect by a factor of ten these numbers soar far beyond the numbers of digital imaging projects undertaken specifically by libraries, archives, and museums. These types of cultural institutions have lost the race if we consider ‘critical mass’ to be a quantity of images and users (127).

Pisciotta goes on to discuss the nature of user image searching on the Web and a number of image use models that have been applied in the academic environment. One aspect of these models mentioned by the author that we have seen evolving in sites such as Flickr, is “Users’ direct contribution of metadata to a centralized database…as a means of describing the multiple readings of a picture” (132).

 

Pisciotta, Henry. 2003. Image delivery and the critical masses. Journal of Library Administration 39(2/3): 123-138.

Visual Image User Study (VIUS). 2003. Summary Report: Penn State University Libraries. (last accessed 11/15/2008 http://www.libraries.psu.edu/vius/SummaryReport.pdf)

Visual Culture & Visual Ecology

I love all of the concepts and controversies surrounding images, their meaning and all of the ways that this involves the way we see the world. It now seems that my academic background running as it has through education, fine art, mass communications and library science has brought me to an interesting juncture in our current electronic evolution. According to James Marcum in his 2002 article Beyond visual culture: The challenge of visual ecology,

Today we live in…a visual culture in which print and graphics, television and telecommunications, videos and movies, and computer displays…convey information of such intensity that it diminishes the dominance of speech and print media. (However)The library profession remains grounded in textual, print media, creating vulnerability amidst a culture increasingly characterized as visual (189).

I know…just what we need, another reason to predict the demise or diminishing role of the library in our society. My love for images, visual information, signs and symbols etc. does not alter my love for the printed word but I do think that Marcum is right in the sense that visual materials are often placed in a second class status within some environments including libraries. “Historically, modern society denigrates the role of the image” (191). It is not that images or visual information are not represented within the conventional library, but the perception at times seems to be that the use visual elements, on the OPAC for example, is seen as dumbing down the system. Particularly in an academic environment, users are expected to be beyond needing “pictures” to get around and therefore less in need of graphic elements to navigate. We desperately need to draw students to our systems but we are often still insisting on raising their levels of perception to print versus images in the way we choose to design our systems. It seems to me, at times, like the internal struggle in dealing with student use of Google. We denigrate it in favor of “real” resources in the library instead of working with students to integrate all sources of information into search strategies. “For the visual ecology to prevail in the world of academe requires providing intellectual access to images” (199).

I remember somewhere in my class text the example given of research that shows that the human eye is able to process thumbnail images more quickly than text-based information. This is simply the way we are wired. It only stands to reason that information retrieval systems that are more visually oriented will be more user friendly. Hence the GUI (graphic user interface). This also makes me think of Steve Krug’s book Don”t make me think: A common sense approach to web usability. It does not improve our standing within our culture if we, as librarians, let the bias against visual navigation and representation within our systems affect how we design access to our digital or electronic resources. “The idolization of language and the reduction of cognition to a processing of codes and symbols – mathematical and textual – have marginalized the study of the image” (192).

Edward Tufte, The Visual Display of Quantitative Information, is another author who comes to mind who has done much in elevating the ability of visual design or images used to represent quantitative data and to represent information in a way that is more immediately grasped by the human mind. As Marcum goes on to state,

Information must be simplified and managed into presentations that have meaning in order to fully utilize and comprehend the vast reservoirs of scientific, business, and public data that are accumulating, and a growing number of researchers are using visualization to analyze and present their data better (196)…It requires but little reflection to determine that librarians cannot leave these skills and techniques to others without jeopardizing the future of the profession (198). The rise of the digital library is an important step in the development of the library as a cultural resource. The effort remains focused overmuch, however, on text and the digitization of text for greater access. The greater challenge is to visualize the total culture, the visual ecology, in order to select wisely what should be preserved for posterity (201).

Marcum, James W. 2002. Beyond visual culture: The challenge of visual ecology. portal: Libraries and the Academy 2(2): 189-206.

“Collections for All Time”

In mylast blog entry, I ended with the intent of looking a bit more in depth at the VIUS user study that examined the image use behaviors and needs of students, faculty and researchers in several academic areas of study. I was also interested in taking a bit of a detour into the concept of visual culture in order to explore the implications for librarians and information professionals due to the increased use of visual information in the online environment. Ihave found since the last post, however, that my reading and class discussions have veered into some additional related areas that I would like to discuss in this blog and perhaps revisit the VIUS study and visual culture in a later post.

This all started, I suppose, with a discussion in my class archives group about archival “provenance” or “original order.” Both terms relate to the manner in which archival materials are organized. Organization can be seen as a structure within which the meaning content of the archival object is described and most importantly the means by which the object is placed within the context in which the object was first created. Most of my thinking is currently based, of course, on how this applies to visual objects such a photographs. I had come across an ALA featured digital collection of the week, ( http://www.shorpy.com/shorpy ), which I posted as an example for my archives collection discussion group. I saw this collection as an example of the manner in which digital images could be reorganized into new and meaningful collections.

This does, however, bring up some questions pointed out by one of my classmates about the organization and significance of digital image collections. When an image is taken from or lifted out of  the original order of the archive in which it has previously been collected, do we not lose some or all of the meaning content within the image? There is also the question of the manner in which a digital collection may merely replicate what is already being done in a conventional archive. What is the justification for going to all of the lengths and costs of creating a digital collection, or reconfiguring a collection, if these things are already being done?

My own thoughts are that, while it is no less and perhaps even more important to preserve provenance and original order, the digital environment offers new ways of considering/configuring and “seeing” visual information. One of the things that stuck in my mind from some of my readings is the reality that much of the visual information now in archives has no individual description. As stated in our class text,

An important issue with special collections is that they may not be cataloged down to the individual item level. Collections of photographs, for example, may simply be placed in folders at the level of perhaps 100 photos per folder, and the entire folder given a catalog record…As a result, one cannot pick a photograph at random from this file and assume… (what the picture is “about”) Collection-level cataloging makes it difficult for the user to decide whether there is an answer to a question in a particular folder. In traditional libraries this was dealt with by delivering the entire folder to the user and having the user flip through it” ( Lesk 2005, 241- 242).

While our text also goes on to say that , “Digitally, browsing may be more tedious and complex and not serve the purpose” (242), does not a digital collection perhaps offer the opportunity to lift out images from their folders and present them in a way that users can decide if there is an image that does provide an answer to a question or an information need or also simply make them aware of unexplored connections between the vast amount of visual information that we currently have in store?

While I am optimistic about the great promise of digital collections, I do agree that,

…individual evaluation is necessary to decide when…the digital image may be more useful that the original, and when the digital image is likely to be only a finding device…(since) Special collections may sometimes be valued as artifacts, which poses additional problems in the digital realm…they often contain unique items…the digitization must not destroy the original, and users are more likely to insist that a digitized version serve merely as a finding aid, rather than a substitute for the original”  (2005, 244-245).

There have been several other articles in our class readings this week that addressed some of these issues involved in determining what materials should be digitized or that have the type of significance that justifies the time, expense and on-going commitment needed to preserve digital collections. Also, when we are considering the organization of digital materials and how we provide access to users, we get into the question of how traditional library standards of organization apply in the digital environment. While the advent of digitization has brought great challenges, it also brings the opportunity for us to reconsider how we organize information and how this will serve users now and into the future.

As stated in an article by Bob Pymm, “…the question of acquiring, selecting and preserving our documentary heritage, in whatever form, has reawakened the debate into how libraries, as one of the institutions charged with the responsibilities for preserving the cultural memory of their society, select material for permanent retention” ( 2006, 61-62). Pymm goes on to discuss examples of how we might apply criteria for determining significance in order to better decide on which materials to place into digital collections and assure preservation of our cultural memory.

In addressing the manner in which we organize information or digital collections, an article co-authored by Kimmo Tuominen states that,

A difference in perspective leads to a different articulation (combination of terms). As information professionals, we have to ask how particular ways of linking terms and representing phenomena affect the ways in which the world is presented to us. Ultimately, this is the question that makes librarianship such an important profession…traditional classification languages have in practice become deeply embedded in…the physical organization of library collections…in digital libraries, it is much easier to introduce alternative orders enabling users also to challenge existing perspectives, classifications, and vocabularies (2003, 563-564).

Tuominen’s article considers several conceptual approaches to information organization and the subsequent application to digital collections. It is very exciting to consider the various conceptual ideas and how the very nature of knowledge production can be impacted by the organizational structures that we are now developing. Also for further reading along these lines is an article by Andrew Bullen. Bullen proposes a model and tools for image digital collections based on concepts from Library 2.0. He sums up a lot of my impressions when he states that,

As professionals, we have embraced the concept of preserving materials as digital objects, defined a wide-ranging series of increasingly sophisticated cataloging (metadata) systems, and developed quality assurance and trusted repository programs. in my very humble opinion, however, we have yet to embrace the next logical step, which is to provide context for the images in our collections. Our image repositories are grouped together with as much relevance as posters in display at a neighborhood head shop; except for an overarching collection theme, our digital objects exist in splendid isolation, beautifully cataloged but bereft of supporting and defining context (2008, 31).

Bullen presents the concept of a “nodal image” as an one that has connections or relationships to many others. Through these connections, he proposes a “…great chain of linkages the ‘long tale,’ a design concept that allows digital repositories to explore all of the connections between narratives and images. The long tale takes a discrete object and expands out its connections” (34). In contrast to the idea that we would be taking images out of context by suggesting other relationships, Bullen proposes the opposite. It is by exposing these other relationships or connections that we further put the images in context.

The model Bullen sees, is one that would depend on audience participation. This is due to the fact that most institutions would not have the necessary resources. By including audience participation, the digital collection site expands its “author and editor base…Each population brings some part of the narrative to share” (35). Bullen ends his article with these words, “We have any number of resources that we, as a profession, have digitized and made available online. I believe that our next step is to tie them together” (35).

 

Bullen, Andrew. 2008. The ‘long tale’: Using Web 2.0 concepts to enhance digital collections. Computers in Libraries. 28(9): 3-35.

Lesk, Michael and Michael Lesk. 2005. Understanding digital libraries. Boston: Elsevier.

Pymm, Bob. 2006. Building collections for all time: The issue of significance. Australian Academic and Research Libraries: AARL. 37(1): 61-73.

Tuominen, Kimmo, Sanna Talja, Reijo Savolainen. 2003. multiperspective digital libraries: The implications of constructionism for the development of digital libraries. Journal of the American Society for Information Science and Technology. 54(6): 561-569.

User Studies

The question this week is, how do we provide for user needs when we develop digital image collections and services. First, of course, we must know what those needs are. In my class chapter reading for this week, author Michael Lesk states that, “Talking to users about what they want is useful…But talking to users about how searching should be done is not likely to be effective…It is hard to guess what people might want, and more effective to ask them” (2005, 227). He later goes on to say, however, that, “Users, unfortunately, aren’t very good at saying what they want, and we haven’t done very well at building systems to help extract this from them” (231). In other words, how might we build information/image retrieval systems that in effect replicate reference interviews in order to engage with the user while negotiating the information/image need.

We have addressed to some degree the issues involved with assigning metadata or descriptions to images by which users access and retrieve image content.  Even if we had many of these issues solved, such as a system that allowed us to effectively categorize and describe image content, the user is likely to come to our system with a need that is not fully developed or a mental model of the system that does not reflect the actual system organizational model.

Training users to use information retrieval systems falls under the scope of information literacy instruction. Within information literacy, images and visual literacy will increasingly take on greater importance. Visual information is increasingly taking on a prominent role in the online environment. In a 2005 article, “Visual Images and Information Literacy”, author Loanne Snavely states, “The time may come, or possibly it has already arrived, when images overtake the word as the dominant medium for communication” (27). She emphasizes the need for librarians to consider the increase in visual information and work to incorporate this into library user sources, services and training.

…as various disciplines continue to rely more and more heavily on visual materials, librarians are and will be teaching more about information literacy issues in the visual realm, about effective searching strategies, appropriate sources, and evaluative techniques for assessing images (2005, 32).

We still need to provide an intermediary between the system and the user. It does not seem likely that users are uniformly going to develop the type of search skills that will consistently interface with image retrieval systems. The ideal information retrieval system, as one that is user centered, is not yet a reality. Many digital image collections were built more with the content in mind and are not easily accessible by users. As Snavely states, however, “Finding the right content to digitize, and providing reader and instructional services for them have become much higher priorities” (2005, 28).

User studies can be used to gain additional insight and provide direction in determining user needs for digital images. The Snavely article illustrates this with a review of the Visual Image User Study (VIUS) that was performed at Pennsylvania State University. This study examined and compared the use of images by students and faculty in a variety of academic disciplines. If you would like to review some of the findings, the link is provided below. For next time, I would like to explore this study in more depth and later perhaps look a little more at the evolving nature of visual information in our culture (visual culture).

 

Visual Image User Study  (VIUS)

http://www.libraries.psu.edu/vius/reports.html

Lesk, Michael, and Michael Lesk. 2004. Understanding digital libraries. Boston: Elsevier.

Snavely, Loanne. 2005. Visual images and information literacy. Reference & User Services Quarterly 45(1): 27-32.

The Medium is the Message

 

The medium is the message – Marshall McLuhan, Understanding Media: The Extensions of Man 1964

Each image begins in some way as a representation but then can take on its own characteristics as an object. A photograph of a building may have been originally taken merely as an historical record. The photographic representation or image may then become an image/object in an archival collection. The image is removed once more when it is put into digital form.

What happens to the message or meaning with each transference? If the medium is the message, what happens to an image/representation/object when it is translated or produced in the digital medium? Since we reduce the image information to digital bits, do we loose or gain emphasis on the elements that make images/objects distinct from one another?

For an example of this, we might consider the comparison between an oil painting and a watercolor. These are examples of two different art mediums. In the art world, watercolors have at best a secondary status to oil paintings. Within the digital realm, will this persist? If over time, the primary form of consumption of images becomes digital, will the original medium matter? Will there be an equal status for images born digital or will other technological preferences become or have already become the new status determinants?

It is important to consider how the meaning or content of an image may be altered when translated from one medium to another. This has implications for those who bear the responsibilities for collecting and preserving image collections. In one of our suggested readings for my class in Digital Collections, the author states that, “it remains crucial for librarians to concern themselves with the package, container, or format and not just the content of collections…” (Manoff 2006, 322). The article discusses “how the properties of electronic objects alter our ways of creating and consuming information…(and) how knowledge is shaped by the technologies used to produce and distribute it” (2006, 311).

These considerations apply no less to visual information as, “A critical focus on electronic objects has also meant a rethinking of the importance of the visual and a new concern with the graphical elements of textuality…Critics are thus exploring how images are read, studied, and understood” (2006, 312). Although the digital medium allows both great potential and great harm due to new ways of combining and manipulating images, “Any content, whether print or digital, is subject to the physical limitations of the technology used to produce and distribute it… (and) Since medium shapes content, theorists are asking what it means to translate content from one medium to another” (2006, 313).

One class article reading led to another in which Author N. Katherine Hayles states, “I propose regarding the transformation of a print document into an electronic text as a form of translation, which is inevitably also an act of interpretation” (2003, 263). Although the author was referring to print content, I no less consider this the case, and perhaps even more so, with visual or image content. Hayles also addresses this statement in light of The William Blake Archive which I discussed earlier in this blog. Of the archive Hayles states that,

The site is informed throughout by an enlightened editorial policy, for the editors state that they take the ‘work’ to be the book considered as a unique physical object…The editors…are meticulous in insisting that even small differences in materiality potentially affect meaning…the editors recognize that what they are doing is simulating, not reproducing, print texts…but there is little or no theoretical exploration of what it means to read an electronic text produced in this fashion rather than the print original. Great attention is paid to the relation of meaning to linguistic and bibliographic codes and almost none to the relation of meaning to digital codes (2006, 264-265).

http://www.blakearchive.org/blake/

Earlier I questioned what happens to images in a variety of formats, when they are similarly translated into digital form. As Hayles states, however, “In this respect and many others, electronic texts (and it could be said of electronic images) are indeed not self-identical. As a processes they exhibit sensitive dependence on temporal and spatial contexts, to say nothing of their absolute dependence on specific hardware and software configurations” (276).

Remember also the use of a Van Gogh painting in an earlier post when we were considering an image placed within the format of a PDF page versus the image displayed on its own without accompanying text. If we search for examples of Van Gogh’s bedroom paintings in Google Images, we are presented with many options. This is a telling example of some of the “translations” that can occur in the digital environment, along with the fact that Van Gogh produced many versions himself. The following site is also interesting in light of work being done in the digital environment based on earlier art/image forms. http://www.btinternet.com/~peter.clements/vangogh/art.htm

Hayles, N. Katherine. 2003. Translating media: Why we should rethink textuality. The Yale Journal of Criticism 16(2): 263-290.

Manoff, Marlene. 2006. The materiality of digital collections: Theoretical and historical perspectives. Libraries and the Academy 6(3): 311-325.

Semantic Modeling and the Reduction of the Semantic Gap

What interests me most is the meaning content of images but in order to retrieve images we need to find a common ground or model that allows each image to be successfully retrieved by users. Perhaps it does not ultimately matter that the image content is not fully ‘interpreted’ for the user. Would we want to do this anyway even if we could? If it is enough to describe what is in the image, what the image is ‘of’, then that might be enough to allow the user to find the image. We leave some of the determination of what the image is ‘about’ to the user. User access is the goal, not exhaustive image interpretation.

Striving to reduce the semantic gap between what the user ’sees’ and what he/she is able to ‘describe’, or put into words, offers some conceptual clues for model development. Vogle and Schiele found that early efforts often merely distinguished image content between such concepts as indoor vs. outdoor, or waterfalls vs. mountains (2007, 133). With the increasing number of images being produced in this electronic age, the option to search only by such general terms is less than adequate.

Vogle and Schiele present a model, based on images classified as natural scenes, in which the image is separated into grids. Through background study in other fields including image perception, some basic image ‘concepts’ or terms were selected. These included; sky, water, grass, trunks, foliage, field, rocks, flowers and sand. Each image of a natural scene is divided into a 10×10 grid.

Each section is assigned to or equals one of the concepts. Nine sections or 9% of the image might ‘equal’ sand. In this way, it can be determined in a quantitative manner what the image is ‘of’. The image might contain a lower percentage of sand and a higher percentage of water. It might be seen that  having only nine concepts and having each section assigned to only one category might present a problem but it was found that, “On average only half an image region per image can not be assigned to one of the nine concept classes…image regions that contain two semantic concepts in equal amounts (are) annotated with both concepts” (135). As a result, “Each image is described by the frequency of occurrence of a semantic concept” (141).

 You might ask, why just the nine concepts for classifying each natural scene image. Would it not be better to have more concepts in order to improve search results? Interestingly, it was found that.

…classification confusions show that members of smaller classes are often confused with the semantically most similar larger class. That is, grass and flowers are almost exclusively confused with foliage but not vice versa. Similarly, field and sand are frequently confused with rocks, but also not vice versa. The fact that it is more difficult to classify small concept classes is also the main argument against using more semantic concepts…subsequent scene categorizations based in ten or eleven instead of nine semantic concepts achieved lower accuracy. On the other hand, a reduction of the number of semantic concepts by not using small classes such as sand or trunks also resulted in a degraded categorization. These smaller classes provide the necessary detail information for discriminating between semantically similar categories (139).

There is a lot more to the Vogle Schiele article. Some of it gets a bit technical but the illustrations given help explain the concepts. One such concept involves the use of image category prototypes. This would involve the manner in which an image would be classified as a “natural scene, for example. A category prototype can be determined by certain “weighted attributes through which the category membership can be determined” (140). A prototype of a natural scene then, would have certain attributes and each “image is assigned to the category it has the smallest distance to” (141). This method of categorization, however, was found not to be as effective as the SVM approach that essentially determines image categories by voting on each image based on a system of 15 classifiers.

A variety of experiments were conducted utilizing various methodologies to determine accuracy and the degree to which semantic modeling improves results. One of the findings gives us an interesting insight into the complexity of image classification. It was found that, “If too much water is misclassified, rivers/lakes images are confused with forests or mountains depending on the amount of foliage and rocks in the image. If too much water has incorrectly been detected in rivers/lakes images, they are confused with coasts” (146). The words in bold represent image categories and those in italics represent image concepts.

Also in their experiments with classification accuracy or consistency of categorization, the authors conclude that, “In fact the images show that there is sometimes no ‘correct’ or ‘incorrect’ answer when categorizing images. How much foliage makes a rivers/lakes-image to a forests-image and vice versa? How far (away) must a mountain be so that the image moves from mountains-category to the plains-category. The conclusion we draw from these observations is that pure hard-decision categorization should not be the goal” (146).Experiments also included a comparison of the ability of the “semantic image retrieval system to rank natural scenes similarly to humans” (149).  

Vogel, Julia and Bernt Schiele. 2007. Semantic modeling of natural scenes for content-based image  retrieval. International Journal of Computer Vision 72(2): 133-157.

“I can’t tell you what I want, but I’ll know it when I see it”

Well, this week my reading has been focused on metadata. Metadata is the information about or description of an image that allows us to organize and provide for access to the image. Without metadata, such as the title of a painting or the artist, we would not be able to find digital images. This folows from my last post in that the subject of metadata gets to the core of the problems with describing visual or image content. In order for image content to be organized and consistently accessible to users, we have to develop a system by which we can consistently classify and describe visual content. Unfortunately, the consistency of even professional indexers when it comes to image content is quite low. Consistency is a problem as well in text-based documents.

As with a search for a book, we can quickly access information by title or author but what if these are unknown? Subject matter is another means of access but this is where we get into the problem of interpretation with images. From the example on my last post, we might want something on the subject of ‘mother and child’ but these images can vary widely in the meaning content. Simple subject searches can bring up so many results which can prove frustrating to users. For librarians working with patrons looking for images, it often takes a very skillful understanding of types of images and the manner in which a particular image collection is organized in order to mediate between the user and the image search.

I recently came across an article entitled “I can’t tell you what I want, but I’ll know it when I see it” (Goodrum2005). This pretty much sums up the problem. The promise of having so many images seemingly at our finger tips only exacerbates the inherent barriers to successful image search and retrieval. User expectations coupled with the unpredictability of personal interpretation can pose significant barriers to retrieval even with the assistance of a librarian. An interesting side note is that images themselves are often used to mediate text and image retrieval. The human eye is able to scan image content more quickly in comparison to text which allows the user to more efficiently select something that relates to their information need (Goodrum 2005, 52 and Lesk 2004, 115).

Even with a consistent retrieval system, there is still the problem of the ’semantic gap’ between the image content and the ability of the user to describe that content or the desired retrieval. As described in the results of a study in the Goodrum article,

When users request an image, they tend to describe the image itself rather that the meaning or emotive content of the image; their indexing vocabulary largely reflects what they expect to see in the image. This is in contrast with image indexing vocabulary derived by a system where indexing terms are extracted from text surrounding the image…the text does not describe the image. Rather, the image is used to support the text. System vocabulary may describe higher-level concepts such as industrial pollution rather than smoke or smoke stacks. The semantic gap between users and system may be bridged through human reference techniques that help translate concepts and notions in user queries into appropriate search terminology (2005, 53).

Some efforts at developing systems to more closely reflect user search terminology has involved allowing users to create their own tags for images in order to increase the key terms by which users can find access to images. This allows for those users who may have different interpretations of an image than would, for example, a specialist in a particular field of study or an cataloger who specializes in a particular type of collection. In our class text, however, author Michael Lesk states, “…asking people to label concepts does not produce unique and reliable answers. Even professional indexers do not produce entirely reproducible results” (2004, 119). In an earlier post, I had included an example about the William Blake Archive in which users are encouraged to comment on the terms that are listed describing the collection contents (http://blakearchive.org/blake/).  I think this is a worthwhile attempt to increase the accessibility of a collection but it will be interesting to see if long-term results are consistent with Lesks’ comment.

Another attempt at increasing accessibility mentioned in our text is the IBM QBIC project which “lets people draw sketches and use them to retrieve pictures” (Lesk 2004, 101). An example of a museum that is using this technology is the Hermitage Museum (http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicSearch.mac/qbic?se1Lang=English) which allows users to “search for images by drawing and positioning shapes and colors within a drawing frame” (Goodrum 2005, 47). While I personally find this very appealing, in practice, I am not sure of the widespread application or adoption of this method. See what you think?

This is barely the tip of the iceberg on the topic of image matadata. There are many initiatives addressing the problems of image retrieval. Some of the standards under development address various image formats. In the recently released Guidelines for Handling Image Metadata, www.metadataworkinggroup.org, it is stated that “…in the area of digital photography…it is not currently practical to organize and query images based solely on the millions of image pixels. Instead it is best to use metadata properties that describe what the photo represents and where, when and how the image was taken” (2008, 6, accessed 9/27/2008).

For next time we might delve into the concept of semantic modeling for image retrieval.       

Goodrum, Abby A. 2005. I can’t tell you what I want, but I’ll know it when I see it. Reference and User Services Quarterly 45(1): 46-53.

Lesk, Michael. 2004. Understanding digital libraries. Boston: Elsevier.

Vogel, Julia and Bernt Schiele. 2007. Semantic modeling of natural scenes for content-based image retrieval. International Journal of Computer Vision 72(2): 133-157.

Zeng, Marcia Lei and Qin, Jian. 2008. Introduction: Fundamentals of metadata. Metadata. New York: Neal-Schuman Publishers: 3-13.