Information Recall of Internet News: Does Design Make a Difference? A pilot Study


Download 121.84 Kb.
Date conversion13.01.2017
Size121.84 Kb.

Information Recall of Internet News:

Does Design Make a Difference? A Pilot Study

Val Pipps, University of Akron
Heather Walter, University of Akron

Kathleen Endres, University of Akron

Patrick Tabatcher, University of Akron


This study looks at the effect of the design of online news presentation on content recall. One of four versions of a story was read online. Results of a post-test showed that recall was significantly higher after students read the text only or abbreviated text with photos and captions options over photos with captions and videos or animated graphics. Participants over age 21 had significantly better content recall after viewing the photos with captions and video prototype.


Dr. Val Pipps is an assistant professor in the School of Communication at The University of Akron. He was a working journalist in both print and for the Web for more than 30 years before teaching full time.

Dr. Heather L. Walter is an associate professor in the School of Communication at The University of Akron. Her main area of research is focused on organizational communication and experimental research design.

Dr. Kathleen Endres is a distinguished professor in the School of Communication at The University of Akron. She is best known for her work in magazine history.

Patrick Tabatcher is a senior multimedia producer in Instructional Services at The University of Akron.


The current economic environment has ravaged the newspaper industry. Unable to deal with continuing advertising cutbacks, reduced circulation and ever increasing production costs, publishing companies are looking for ways to control their mounting financial losses. Earlier this year, E.W. Scripps Co. opted to shut down the Rocky Mountain News in Denver after 150 years.1

Detroit News owner MediaNews Group and Detroit Free Press owner Gannett turned to a new business model, emphasizing digital news and providing home delivery for the print product only on Thursday, Friday, and Sunday.2 Hearst Corp. closed down its 146-year-old Seattle Post-Intelligencer in favor of an entirely digital news product.3 The Newhouse family’s Advance Publications announced plans to close the 174-year-old Ann Arbor News in favor of a “web-focused news operation built from the ground up” (

Newspapers have long fostered a connection with the digital world. More than 25 years ago, the Columbus Dispatch became the first newspaper to go online with a CompuServe dial-up service.5 But it wasn’t until the 1990s that daily newspapers got serious about introducing digital versions of their paper products. As Riley, Keough, Christiansen, Meilich, and Pierson explained, 1996 marked the beginning of a two-year growth spurt, when the number of newspapers having an online version went from 154 to 3,600.6

Yet what was happening in 2009 was different. As the Seattle PI and Ann Arbor News illustrate, newspapers were looking to the online world for their future. But what forms should those products take? What is the most effective way of communicating to the public—and have the readers remember what they read?

This pilot study was designed to begin to try to answer those questions in an experiment testing four versions of an online news story for content recall.

Literature Review

Content recall has long been the focus of scholarly research. For six decades, researchers from a variety of disciplines have analyzed content recall of news, information, and messages provided by the media. As each new medium was introduced—and widely adopted by the American public—researchers revisited the issue of content recall.

In 1949 the focus was on radio, news listening, and memory. In a study by Harrell, Brown, and Schramm,7 the researchers became one of the first—although certainly not the last—to point out the shortcomings of radio news with regard to content recall. The three reported the low rate of information recall, regardless of news format or repetition. According to the study, participants remembered items dealing with human interest or “spectacular stories of crime and disaster” as opposed to more substantive public affairs news.8 The researchers equated this to sensational newspaper coverage, although newspapers were not included in their study.

Williams, Paul, and Ogilvie9 saw more promise in broadcast media in their study in 1957. In this experiment comparing television, radio, print, and “live” delivery of information, the researchers found participants exposed to television and radio scored the highest content recall—both short term and long term —compared to those exposed to print and/or “live” delivery of information. This experiment, however, did not deal with news reporting, nor did it include newspaper and/or magazine coverage. Nonetheless, the study seemed to keep the hope for broadcasting alive.

Tichenor, Donohue, and Olien10 held out that hope for television, too. In their ground-breaking work on the knowledge-gap theory and the mass media, the three found that the better educated retained more information from newspaper coverage of complex scientific topics than the less educated. The researchers were not sure if the findings could be extended to television. They speculated, “Since television use tends to be less correlated with education, there is a possibility that television may be a ‘knowledge leveler’ in some areas.”11

Subsequent researchers12 have shown that Tichenor, Donohue, and Olien were overly optimistic. Katz, Adoni, and Parness’ work13 is especially on point. In their examination of recall of radio and television news, the researchers found television more effective in information recall than radio—or “seeing the news”—was more effective than merely hearing it.14 However, neither television nor radio scored impressive content recall results. Moreover, television was not found to be the panacea that Tichenor, Donogue, and Olien hoped. As Katz, Adoni, and Parness concluded, their findings were consistent “with the knowledge-gap hypothesis, and does little for our expectation that television news would contribute most to the least well-off, informationally speaking.”15

Over the decades, researchers searched for an explanation for broadcasting’s information-recall shortcomings. Some suggested "contextual interference," or viewers doing other things while they watched television,16 the trivialization of news,17 and the characteristics of television news itself.18

After the introduction of Gannett’s colorful, graphically intensive USA Today on September 15, 1982, dailies shifted their designs, using more color and graphics. The Society of Newspaper Design found that from 1984 to 1988 the percentage of dailies with graphics capability grew from 40 to 90 percent.19 The greater use of maps, charts, and three-dimensional graphs caused some researchers to study the link between graphics and information recall. A number of researchers found that tables, graphs, and maps enhanced reader performance and recall.20

The widespread use of the Internet brought researchers back to familiar recall territory, examining the new medium—the computer—and its effect. These studies again took a comparative approach, this time looking at newspapers, radio, television, and the computer to ascertain content recall. These studies have tended to come to similar conclusions. Newspapers scored the highest level in content recall, followed by computer, television, and radio.21 Cultural media tradition may influence content recall results, however. Facorro and DeFleur22 found that a nation’s tradition of media use can influence content recall results. In Spain computer penetration is not as great as in America, and radio plays a greater role in informing the public. Thus, newspaper content recall was the highest for both American and Spanish groups. However, radio recall scored a close second for the Spanish group; television followed, and the computer scored a distant fourth.23

As websites evolved, adding more multimedia elements, researchers returned to the issue of design and its influence on content recall. Researchers from the education discipline especially saw great benefits in interactive media and learning, emphasizing the deep comprehension for students working with interactive media24 and the benefits of student control of the speed of learning through computer-based instruction.25

Researchers from the communication field have not been as enthusiastic in their findings the interactive media. In 2000 Berry26 reported on an experiment on Web-page design and content recall. Her study offered participants two options—text only and the other multimedia based. While the comprehension scores were low for both sites, Berry reported that the multimedia site had slightly better recall scores than the text-only site, but not statistically significant.

The Poynter Institute did the most comprehensive study dealing with the issue of design and recall to date. The “Eyetracking the News” report27 looked at how different print (newspaper) and online designs influenced information recall. The researchers offered the participants three different versions of the same story. These options ranged from traditional, text-heavy stories to a very visual, graphics-laden presentation. Researchers reported that newspaper/broadsheet readers across the board had better information recall than online participants. However, the best information recall for both print and online was the graphics-laden presentation.28

Although eyetracking research has been used as a tool to help gauge usability of website designs, the research method has been largely ignored by the newspaper industry. The Poynter Institute has been the forerunner in using the eyetracking devices in researching how newspapers and Web pages are read. Technological advances have produced eyetracking paraphernalia that is less cumbersome, less obtrusive, and more accurate, but measurement problems exist. Salvucci and Golberg noted that although there was general agreement that visual and cognitive processing do occur during eye fixations, “it is less clear exactly when fixations start and when they end.” 29

There is also the inherent problem of interpreting eyetracking data. Jacob and Karn note that there is a “lack of standard terminology and definitions for even the most fundamental concepts used in eyetracking data interpretation.”30 Jacob and Karn also point out, in discussing measurement, that using the eye as a computer input device can be a problem. “Moving one’s eyes is often an almost subconscious act…. it is relatively difficult to control eye position consciously and precisely at all times. The eyes continually dart from spot to spot even when its owners thinks he or she is looking steadily at a single object.”31


H1: Recall will be greater for respondents reading text-plus graphics version (Appendix B—a page with the edited story plus photos and captions and video) than text alone (Appendix A—the longest version of the story with no photos or video or graphics).

H2: Recall will be greater for respondents reading graphics-rich version with no text (Appendix C—no text, only photos and captions and video) than text-plus graphics (Appendix B) or text alone (Appendix A).

H3: Recall will be least effective for respondents reading text-only version (Appendix A) and viewing the animated graphics32 version (Appendix D—static graphic and animated graphic only, no text or photos or video).

In order to test these hypotheses, the researchers developed four different versions of a story for online presentation that encompassed a variety of presentation alternatives. The four alternatives were:

  1. Text only (see Appendix A).

  2. Abbreviated text with photos and captions and video (see Appendix B).

  3. Photos and captions and videos (no text) (see Appendix C).

  4. Animated graphics (see Appendix D).

The materials provided in each of these versions were based on the series “Altered Oceans,” which appeared in the Los Angeles Times from July 30 to August 3, 2006.33 The researchers specifically used content from part one of that series, “A Primeval Tide of Toxins.”

The researchers developed a pre-test of five questions, testing the respondent’s prior knowledge of the subject matter (see Appendix E). Those same five questions were asked after exposure to one of the versions. Demographic questions were also asked after the post-test (see Appendix F). The four versions, pre-test, post-test, and demographic material were stored on a central server. Access to the central server was password protected so participants were affiliated with The University of Akron and no student could participate in the experiment more than once. 34

Each participant was presented with a screen explaining the study and allowing him or her to opt out. By opting in, participants were presented with a pre-test. After the pre-test, each participant was randomly assigned one of the four versions (text only; abbreviated text with photos and captions; photos and captions and videos—no text; and animated graphics). One communication class of 16 students participated while in the classroom. The remainder of the participants viewed the stories outside of a classroom setting—wherever they were most comfortable or they had access to a computer.

Participation was voluntary in all instances. There was no time limit placed on how long a participant could spend reading, viewing or interacting with the content. Videos and animations could be replayed multiple times. However, the participants could not return to the version they had viewed once they began the post-test.

Participants in this study, students at The University of Akron, were enrolled in a range of classes across campus, including communication, women’s studies, and environmental studies, during the spring 2009 semester. In all, 146 college students—77 men and 69 women—participated in the study. Undergraduates and post-secondary students represented the largest number of participants (N=137); only eight graduate students participated, and one fit into neither category. Respondents were also asked their age; 66 were under 21, and 66 were over 21; four did not report an age. The largest number of the participants reported that they were arts majors, which included communication (N=87); followed by business majors (N=24), and those from the science technology, engineering, and mathematics disciplines (N=16).35


H1, H2, and H3 all hypothesized the relationship between the content versions and information recall. An ANOVA and Bonferroni post-hoc test statistics were obtained. Full results can be see in Tables 1, 2, and 3.

H1 argued recall would be greater for respondents reading text-plus graphics version than text alone. Results indicated that there is no significant difference in post-test scores for those who saw the text with photos and captions (M = 3.70, SD = 1.311) versus those who saw the text-only version (M = 3.62, SD = 1.197). These two versions, however, had the greatest rate of content recall of the four tested.

H2 hypothesized that recall would be greater for respondents reading the graphics-rich version with no text than text-plus graphics or text-alone versions. Surprisingly, the ANOVA results indicated that with the graphics-rich version—no text ( M = 2.58, SD = 1.232), there was statistically significantly less content recall, F(3, 121) = 7.362, p < .001, than in the text alone (M = 3.62, SD = 1.197) or the text with photos and captions (M = 3.70, SD = 1.311) versions. There was no significant difference between the graphics-rich version with no text and the animated graphics ( M = 2.70, SD = 1.104).

H3 hypothesized that recall would be least effective for respondents reading the text-only version and viewing the animated graphics version. Table 2 indicates that the most highly recalled mean occurred when the participants were given the text with photos and caption version (M = 3.70, SD = 1.311), followed by the all text version (M = 3.62, SD = 1.197). The less effective versions were the no text animated graphics prototype (M = 2.70, SD = 1.311) and the graphics rich with no text version ( M = 2.58, SD = 1.252)

Table 1

ANOVA results for content recall based on content version


Post-Test Score

Sum of Squares


Mean Square



Between groups





Within groups







Table 2

Descriptive statistics for content recall based on the version viewed


Post-Test Score



Standard Deviation

All text no video or photo




Text + three photos w/captions




8 photos w/captions + 3 videos No text




Animated graphics (3) No text








Table 3

Post-hoc Bonferroni examination of the various methods of content delivery

with content recall

Multiple Comparisons

Dependent Variable: Post-Test Score


(I) Method

(J) Method

Mean Difference (I-J)

Std. Error


95% Confidence Interval

Lower Bound

Upper Bound

Lower Bound

Upper Bound

Lower Bound

All text no video or photo

Text + three photos w/captions






8 photos w/captions + 3 videos No text






Animated graphics (3) No text






Text + 3 photos w/captions

All text no video or photo






8 photos w/captions + 3 videos No text






Animated graphics (3) No text






8 photos w/captions + 3 videos No text

All text no video or photo






Text + three photos w/captions






Animated graphics (3) No text






Animated graphics (3) No text

All text no video or photo






Text + three photos w/captions






8 photos w/captions + 3 videos No text






* The mean difference is significant at the .05 level.


The year 2009 may be remembered as the beginning of the shift of emphasis of the news industry from print to online. With the Seattle Post-Intelligencer leading the way as the largest circulation newspaper to become an online-only news product and the announcement of other newspapers planning to follow suit, questions about presentation of information arise. Where should resources go? Should news online veer more toward graphic presentation and away from the text heavy stories?

No one project can answer those important questions. Our research studied different formats as it relates specifically to information recall. It did not address the issue of preference—allowing the respondent to select the format—which might influence the amount of information that was recalled. There was the attempt to allow the participant to view the story in his or her natural setting, with all the distractions that are the norm. With this approach we were not able to keep all conditions constant as we would in an artificial setting and, therefore, cannot as confidently attribute the results to the differences in the versions of the story.

Although research is not abundant concerning the format of news online and recall, studies on the subject are mixed. Berry gave participants only two options—text and multimedia—and her results showed no significant difference between the two options. The Poynter Institute’s Eyetrack studies have been the most comprehensive. The 2004 Eyetrack study found that recall was better for text information than for multimedia—audio with photo and caption36. However, in the 2007 Eyetrack study, which looked at both print and online using three prototypes of news presentation, two of which relied on “text blocks and visual storytelling” and a third that “displayed no narrative, with the story told through an explanatory map, a Q&A, a numbers chart, and more” found that the graphic-intense information in both print and online provided the greatest recall.

The results of this study indicate, contrary to the latest Poynter study, that text alone or text with photos is best in presenting news online based on recall. The recall of news after reading the text version or reading the text version with photos and caption and video was significantly higher than the version with only photos and captions and video and the version with animated graphics only. Both recall means of the text and text with photos options were high (3.62 and 3.70 respectively out of 5). However, all the options we presented showed good recall of the information, highlighting the fact that people are acclimated to reading news online and remembering what they read. Obtaining the information through the multimedia version—photos with captions and video—scored worst on our recall (2.58 mean score) and was not significantly lower than the animated graphics version (2.70 mean score).

The difference between our results and Poynter’s may be related to the fact that our format differed markedly from any of the studies conducted thus far. We looked only at news online. Also, we presented four versions rather than two or three, with each of our prototypes distinctly different from the others. No other study has presented a graphics option without text. Both of Poynter’s print and online graphic presentations in the 2007 study did not include narrative, but the prototypes were text intensive. Their online versions provided no video or graphic-only scenario.

Further Research

There are many factors facing editors of news organizations in their decisions concerning how news should be delivered online. Providing information that enhances the understanding of a subject or issue is one of the objectives. This pilot study is the beginning of an approach we believe will ultimately help news organizations better judge how to deliver news. More study is planned that will not only expand the number of students responding and the disciplines of those students but will also include the general adult population. We need to look at the different educational and socioeconomic backgrounds of news readers in different parts of the country as a means of revisiting knowledge-gap research. Will the explosion of information online help narrow that gap or can presentation have an effect? We will also increase the number of questions in both the pre- and post-test. Related research should be initiated that looks at the way information is presented in graphics. Is animation better than graphics that utilize more text-heavy approach in the presentation?

Also, we have not addressed a reader’s preference in how news is presented. Although people might say they want to get their information through video or graphics, when the time comes to choose the format, will they return to a text-rich presentation?

With these approaches we can come closer to identifying where resources should be used and understand the realities of online news readership.


These results could have an impact for news organizations writing stories for the Web. If, as the data suggest, text alone or text with photos or graphic elements combined are the best methods for presenting information, news organizations—which are looking at ways to trim resources yet provide information—should focus on what they do best—write news stories and provide photos and/or graphics that illustrate them. That has been the substance of the news industry for 200 years, and although more research is needed, this study suggests organizations should continue that course as they move resources to presenting news online.

Appendix A

Appendix C

Appendix D

Appendix D

Appendix E


Appendix F



 Lynn DeBrin, “Rocky Mountain News to Close, Publish Final Edition Friday,” Rocky Mountain News, February 26, 2009, p. 1.

2 Paul Anger, “Paper Shifts Focus in Digital World,”, Dec. 17, 2008. Retrieved March 15, 2009 from (

3 Dan Richman and Andrea James, “Seattle P-I to Publish Last Edition Tuesday,, March 17, 2009. Retrieved March 18, 2009, from (

4 “Ann Arbor Newspaper Will Close in July,”, March 24, 2009. Retrieved March 27, 2009, from (

5 The Columbus Dispatch first went online on July 1, 1980. CompuServe worked with 10 other AP member papers in this initiative. The other newspapers were The Washington Post, New York Times, The Minneapolis Star Tribune, The San Francisco Chronicle, The San Francisco Examiner, the Los Angeles Times, The Virginian-Pilot/Ledger Star, The Middlesex News, The St. Louis Post-Dispatch, and the Atlanta Journal-Constitution. The CompuServe/AP collaboration ended in 1982. PoynterOnline, March 1, 2008. Retrieved March 17, 2009, from (

6 Patricia Riley, Colleen M. Keough, Thora Christiansen, Ofer Meilich, and Jillian Pierson, “Community or Colony: The Case of Online Newspapers and the Web,” Journal of Computer-Mediated Communication 4(1998). Retrieved February 8, 2009, from

7 Thomas W. Harrell, Donald E. Brown, and Wilbur Schramm, “Memory in Radio News Listening,” Journal of Applied Psychology 33 (1949), 265-274.

8 Harrell, Brown, and Schramm, “Memory in Radio News Listening,” 274.

9 D.C. Williams, J. Paul, and J.C. Ogilvie, “Mass Media, Learning and Retention,” Canadian Journal of Psychology 11 (1957), 157-163

10 Philip J. Tichenor, George A. Donohue, and Clarice N. Olien, “Mass Media Flow and Differential Growth in Knowledge,” The Public Opinion Quarterly 34 (1970), 159-170.

11Tichenor, Donohue, and Olien, “Mass Media Flow,” 170

12 See, for example, Jan Kleinnijenhuis, “Newspaper Complexity of the Knowledge Gap,” European Journal of Communication 6 (1991), 499-522, and Sei-Hill Kim, “Testing the Knowledge Gap Hypothesis in South Korea: Traditional News Media, the Internet and Political Learning,” International Journal of Public Opinion Research 20 (2008), 193-210.

13 Elihu Katz, Hanna Adonis, and Pnina Parness, “Remembering the News: What the Picture Adds to Recall,” Journalism Quarterly 54 (1977), 231-239

14 Katz, Adonis, and Parness, “Remembering the News,” 239.

15 Ibid.

16 Daniel R. Anderson and John J. Burns, “Paying Attention to Television,” in J. Bryant and D. Zillmann (Eds.), Responding to the Screen: Reception and Reaction Processes, Hillsdale, N.J.: Erlbaum, 3-25, and W. G. Woodall, D.K. Davis, and H. Sahin, “From the Boob Tube to the Black Box: Television News Comprehension from an Information Processing Perspective,” Journal of Broadcasting 27 (1983), 1-23.

17 Neil Postman, Amusing Ourselves to Death: Public Discourse in the Age of Show Business, New York: Penguin Books, 1985.

18 Because news is brief and entertaining, researchers suggest that viewers cognitively process television news at a lower level than print. Annie Lang, “Effects of Chronological Presentation of Information on Processing and Memory for Broadcast News,” Journal of Broadcast and Electronic Media 40 (1996), 460-477, and Robert H. Wicks, “Improvement Over Time in Recall of Media Information: An Exploratory Study,” Journal of Broadcasting and Electronic Media 36 (1992), 287-302.

19 Steve Pasternack and Sandra Utt, “Reader Use and Understanding of Newspaper Infographics,” Newspaper Research Journal 11 (Spring 1990), 28.

20 See, for example, Becky K. Peterson, “Tables and Graphs Improve Reader Performance and Reader Reaction,” Journal of Business Communication 20 (Spring 1983), 57-55; Jyotika Ramaprasad, “Informational Recall in Newspapers,” Newspaper Research Journal 12 (Summer 1991), 92-103; and Jeane Amlund, Janet Gaffney, and Raymond Kulhavy, “Map Feature Content and Text Recall in Good and Poor Readers,” Journal of Reading Behavior 17 (1985), 317-330.

21 See, for example, Melvin L DeFleur, Lucinda Davenport, Mary Cronin, and Margaret DeFleur, “Audience Recall of News Stories Presented by Newspaper, Computer, Television and Radio,” Journalism Quarterly 69 (Winter 1992), 1010-1022

22 Luis Buceta Facorro and Melvin L. DeFleur, “A Cross-Cultural Experiment on How Well Audiences Remember News Stories from Newspaper, Computer, Television, and Radio Sources,” Journalism Quarterly 70 (Autumn 1993), 585-601.

23 Ibid.

24 For one of the best overviews of research on interactive media and learning, see David J. Ayersman, “Reviewing the Research on Hypermedia-Based Learning,” Journal of Research in Computing in Education 28 (Summer 1996), 500-525.

25 See, for example, Lawrence Najjar, “Multimedia Information and Learning,” Journal of Educational Multimedia and Hypermedia 5 (1996), 129-150.

26 Leigh Talbert Berry, “Comprehension and Recall of Internet News: A Quantitative Study of Web Page Design,” Journal of Magazine and New Media Research 3 (2001) online full reference retrieved March 14, 2009.

27 Pegie Stark Adam, Sara Quinn, and Rick Edmonds, Eyetracking the News: A Study of Print and Online Reading, St. Petersburg, FL, 2007.

28 Adam, Quinn, and Edmonds, Eyetracking the News, 81

29 Dario D. Salvucci and Joseph H. Goldberg, “Identifying Fixations and Saccades in Eye-Tracking Protocols,” in Proceedings of the 2000 Symposium on Eye Tracking Research and Applications (2000), 71-78. New York: ACM Press. Retrieved September 25, 2009, from ( ).


30 Robert J.K. Jacob and Keith S. Karn, “Eye Tracking in Human-Computer Interaction and Usability Research: Ready to deliver the promises,” in J. Hyona, R. Radach, and H. Deubel (Eds.), The Mind’s Eye: Cognitive and applied aspects of eye movement research (2003), 573-605. Amsterdam: Elsevier. Retrieved Sept. 25, 2009, from

31 Jacob and Karn, “Eye Tracking,” 584.

32 This option offered animation with machine interaction (mouse rollovers) but no text.

33 This series, written by Kenneth R. Weiss and Usha Lee McFarling with photos and video by Rick Loomis, went on to win the George Polk Award and the Pulitzer Prize for explanatory journalism. The series was used with permission from the Los Angeles Times. The full series appears online at,0,7842752.special.

34 Results were stored in a secure online database that kept the participants’ responses anonymous.

35 Other respondents included the following fields: education (N=10); humanities (N=1); social sciences (N=1), and other (N=5).

36 Steve Outing and Laura Ruel, Project Managers, Eyetrack III: Online News Consumer Behavior in the Age of Multimedia. Retrieved March 11, 2009, from

Journal of Magazine & New Media Research

Vol. 11, No. 1 Fall 2009


The database is protected by copyright © 2017
send message

    Main page