Enriched and reductive interpretations in the digital narration of picture books

Introduction

Although Gerard Genette, in his study of 1982 regarding transtextual relations, included illustrations among the possible paratexts within a work, in picture books, the illustration cannot be considered to be an accessory, but part of one whole multimodal text (Lewis, 2001; Nikolajeva & Scott, 2001; Nodelman, 1988). If picture books are to be considered as a whole in which word and image are not the only pieces that form them (Van-der-Linden, 2015), in this work we will consider the relationships between word and image according to different explicative models (Mourão, 2011; 2012; Nikolajeva & Scott, 2001; Nodelman, 1988), from interrelation (for which some terms such as collaboration or interdependency, among others, are proposed), to the substitution of the literary text by images in silent books, and the relations of contradiction or counterpoint (Agosto, 1999; Nikolajeva & Scott, 2001; Rhedin, 1992).

This last type of relationship obliges the reader to rebuild the direction of the text when perceiving the existence of two different narrators that disagree with each other and transmit different versions of the same event: the literary narrator that “pronounces” the words and the visual narrator that shows what is happening (Díaz-Armas, 2010; Dueñas-Lorente, 2022; Kümmerling-Meibauer, 1999; Nières-Chevrel, 2003).

Genette indicated in 1982 that the elements located on the threshold of the text are responsible for the reading contract as bearers, at least, of architextual information (Genette, 1982). Throughout that passage of Palimpsestes he is referring to the peritext, although in that moment he is only using the term paratext, which would some years later be divided in epitext and peritext (1987).

As picture books are, to begin with, a book whose recipient is a child reader, their peritext present singularities such as the frequent absence of elements that can be found in other books (index, introductions, prologues, epilogues, names of chapters, footnotes or endnotes, pagination), as well as the importance acquired by images and texts included in the spaces that form the peritext: cover and back cover, endpapers, title page or copyright page.

The spaces of the peritext in picture books have received meticulous attention (Consejo-Pano, 2014; Díaz-Armas, 2006; Harris, 2005; Lluch-Crespo, 2009; Martínez et al., 2016; Nikolajeva & Scott, 2001; Ramos & Mattos, 2018; Sipe, 2001; Sipe & McGuire, 2020; Uluğ & Bayraktar, 2014), and there are studies that elaborate on the typology and the possibilities offered by covers and back covers (Sipe & Brightman, 2005; Mattos et al., 2016), dust jackets and belly bands (Mattos, 2016; Ramos & Mattos, 2018), endpapers (Consejo-Pano, 2011; Duran & Bosch, 2011; Sipe & McGuire, 2006; 2020), title pages (Sotto-Mayor, 2016), barcodes (Ramos, 2017), the picture book design (Ramos, 2016; 2019) or the typography (Phinney & Colabucci, 2010; Serafini & Clausen, 2012; Timpany et al., 2014).

Some studies have highlighted a functionality in the elements of the peritext that goes beyond the anticipation, the activation of the reading process, or the formation of the reading contract. A characteristic that is singular regarding picture books -present also in other artistic texts characterised by narrativity, as in film, where it is increasingly more popular to find post-credit scenes- is the use of spaces of the peritext to initiate or continue the narration or conclude it. Regarding this characteristic, that is a growing tendency in picture books (Kunde, 2021) and that can also be found in the illustrated book, where it can be used for adding a framing narrative (Díaz-Armas, 2003). In Spanish, there have been several terms proposed by different academics: “desbordamiento narrativo” (Díaz-Armas, 2003; 2006) −which is the one that we will use in this article− “textualización de los paratextos” (Consejo-Pano, 2014) or “hermeneutic function” (in its inclusive variant) of peritexts, added by Kunde (2021) to the function typology proposed by Gross and Latham (2017) to analyse the peritexts in the nonfiction book (Kunde, 2021; Kunde et al., 2022).

This characteristic can lead to the narration starting before the first sentence and continuing after the last one, adding a preface or an epilogue (Bosch y Duran, 2011; Díaz-Armas, 2003; 2006); it can introduce the space and time in which the story is set in (Díaz-Armas, 2006); or incorporate “un juego o una información independiente de la historia” (Bosch y Duran, 2011, p. 16), whereas the back endpapers can get to be a narrative zone where the story is concluded or, at times, where the reader is guided back to the beginning of it (Consejo-Pano, 2014; Lambert, 2010), or introducing a new open ending, contradicting the previous narrative (Díaz-Armas, 2006; Kunde, 2021).

Mediation and digital narration

Among the numerous studies on children readers’ response toward picture books (Arizpe & Styles, 2002; Harris, 2005; Mourão, 2011; 2012; Pantaleo, 2009; 2010; Sipe, 2001; Sipe & Brightman, 2005), there are many observations on how the initial peritext helps the reader to sign a reading contract (Sipe & McGuire, 2006; Sipe & Brightman, 2009).

Studies on typology and teaching practices insist on the importance of the performance of the teacher regarding the construction of meaning by children readers (Arizpe, 2022; Díaz-Armas, 2008; Dueñas et al., 2014; Ghiso & McGuire, 2007; González-Vázquez & Gutiérrez-Sebastián, 2024; Mourão, 2012; Munita, 2021; Muñoz et al., 2018). To do so, it would be necessary to have an open attitude towards the students’ possible interpretations of the text (Harris, 2005), a process that requires reexamining the elements in the peritext after reading (Harris, 2005; Martínez et al., 2016), as in many cases, the narration starts before the first sentence appears -and it is important that the reader notices that (Mourão, 2011)- or continues after the last one.

The narrator who reads a picture book in front of an audience has the possibility of using their voice to clarify aspects during the reading, amplifying part of the verbal text, or commenting on it, which would include, among other possibilities, describing what happens in the illustrations (Ellis & Mourão, 2021; Ellis & Brewster, 2014). Picture books provide resources for the oral narration, as the possibility of waiting for the suitable moment to turn the page; of referring to parts of the books, including the elements that are part of the peritext; or of commenting -or at least, showing- the illustrations that add information that is not provided by the text, because “A mere reading out aloud of the verbal text would deny the children the opportunity to fully experience the read-aloud ‘event’ and to connect with the whole picturebook” (Ellis & Mourão, 2021, p. 23).

While these teaching strategies are commonly developed in educational contexts (or libraries or other places and events related to reading mediation), they are also frequently broadcast on radio or TV programmes. With the arrival and rise of internet, they have persisted, transformed into digital literary texts characterized by their multimodal communication -if the picture book is already multimodal in itself, the web has multiplied its expressive possibilities (Cassany, 2012) – asynchronous, transferred (and not autonomous), and, depending on their relationship with the original text, either reproduced or adapted (Cassany, 2012).

This type of digital narration consists of the reproduction, shared through a platform or social network, of a digital file in which the source literary text is preserved. It is accompanied by a performance that is more or less faithful to the original text, which may or may not show the printed text -in this case of a book- to the camera and reproduces it either in a total or fragmentary way, all for a restricted or general audience.

The reproduction of picture books through the web poses interesting challenges for mediators that use this form of reproduction, because, in the same way we have described an insufficient mediation in the synchronous oral narration -with the presence of speaker and receiver in the same place- when the mediator does not allow the audience to contemplate the illustrations or the free interpretation of a text (Ellis & Mourão, 2021), these circumstances could also happen in the digital asynchronous reproduction of a picture book.

Although there are studies on animated digital narrations (Allué & Cassany, 2023; Cassany, 2012; Lluch-Crespo, 2008; Manresa & Margallo, 2016), the non-animated digital narrations of picture books do not seem to have received the same attention. An exception is Bajour's (2022) publication, which examines whether mediators respect or explain the silences in picturebooks when narrating them in videos published on YouTube.

Method

Then, it seems relevant to analyse the reproductions of picture books that are present on the web, and analyse the strategies used by mediators to show (or not) the illustrations, especially when a) these are in the peritext, a zone that is not frequently recognized by its readers as a carrier of narrative information, b)when the illustrations question the reading if only the literary narration was taken into consideration, or c) when there are details in the illustrations that are necessary for the construction of meaning. The omissions of these aspects would cancel the ironic meaning of the story or its ambiguity and polysemy.

To do so, we decided to study the strategies used in digital narrations of one picture book that would gather all these characteristics. Then, we needed the concurrence of two factors: a picture book that would gather all these aspects and that said picture book would have enough reproductions to make it possible to draw significative conclusions. Both requirements are found in The bear under the stairs, by Helen Cooper (1993).

We searched the web (on the platforms YouTube, Facebook, and Vimeo) in July 2021 (the dates coincide with the COVID-19 pandemic, a reason why maybe the sample became more abundant), through the words that identify the title of Helen Cooper’s picture book in the languages known by the members in the research group, writing the title as it was translated into said languages.

The search generated a total of 53 digital narrations in English, Spanish, and Catalan, which, for space and legibility reasons, we separate in the document that can be consulted in the Zenodo archive (Quesada-Padrón et al., 2024), where we provide the fundamental data for its localization and description: the code assigned to each video, the title, channel, platform -YouTube, Vimeo or Facebook- duration and link.

As we intended to identify if the mediators permitted or not their audience to access the whole polysemic richness of the work, showing or hiding what the illustration contains, and considering that that could be done with words pronounced but also in the pages reproduced or omitted, we studied both the visual and the verbal narration that are shown in the video, which we call, respectively, resulting verbal text and resulting visual text. We do so to distinguish them from the original literary and visual text, which are part of this multimodal text that we find in Cooper’s work, but that might have been played fragmentarily or entirely in the digital narration, or even by amplifying (adding linguistic material in the resulting verbal text) or by highlighting (through some way to catch the audience’s attention through the resulting visual text, on some details that are found in the illustration).

To do so, we have collected data in a table that made it possible for us to observe the oral text, but also the position of the camera: if the camera showed the book or the narrator or both things; if it showed all the pages or only some of them; if it showed the whole surface of the page, without ignoring any details or if, on the contrary, this was made in a fragmentary way and if, during the digital narration, the mediator interacted or not with the audience of the video.

In the resulting visual text, it was important to verify if the camera showed or hid:

In the resulting verbal text, it was important to verify:

We predicted that there would be, at least, two ways of not ignoring that narrative information: the first one, by adding words to describe what is shown in the illustrations (or referring to them with deictics: “But here…” and/or rhetoric questions: “¿Who is this? ¿What is happening here?”); the second, making the audience notice the existence of that information in the illustrations, though other paralinguistic signs (the voice tone) or non-verbal: pointing them with the finger or through some other gesture.

We also predicted that the paratexts generated in these digital documents would generate other significative data (Tabernero-Sala, 2016), so we analysed the introductions, the credit titles, and even the comments when these were activated, to collect the opinion of receivers and speakers, with their answers to these, in these asynchronous narrations.

We took into consideration parameters and characteristics that can be observed in the non-digital narrations pointed out in the previously commented works of Ellis and Brewster (2014); Ellis and Mourão (2021), Häggström (2020) or Mourão (2012). However, since this was not the objective of our research, we categorized the performance as dramatization, narration or reading, but without exploring deeply the use of gesture, voice tones, etcetera.

The group of parameters created for analysing the videos were the following:

After collecting the data using these parameters, we categorized it, and this categorization allowed us to obtain the results that we will describe in the following section, following the same order as the parameters mentioned above.

Results

17 out of the 53 videos were found on Facebook (32.7%), 34 on YouTube (64.15%), and 2 on Vimeo (3.7%). Nearly half were published in the United States (45%), followed by the United Kingdom (32%), Latin America (17%), and Spain (6%). Most of the videos (40 out of 53) were in English, 12 in Spanish, and one in Catalan. Most of these (48 out of 53) belonged to users with professional accounts managed by educational institutions, libraries, associations, or professionals in those fields, such as teachers or librarians. A total of 44 videos were published in 2020, and at least four featured teachers narrating the storybook so that their students could enjoy it during the COVID-19 lockdown. These teachers wore face masks, used encouraging phrases, recommended staying home, and, in two cases, the videos were recorded as Zoom video calls. More than half of the videos (28), whether from personal or professional accounts, were published by users with channels almost exclusively dedicated to storytelling.

In July 2021, 40 of these videos had reached between 1 and 800 views, and only 8 of them had between 1.000 and 9.000 views at that time. The number of “likes” also varied considerably: 35 videos had fewer than 100 positive reactions, while others reached up to 1.000.

There wasn’t much information to gather from the comments section since most of the videos, hosted on YouTube, were marked as children’s content and were therefore protected by blocking comments. In six of the videos, viewers mentioned that their children enjoyed the story. Only once (in video 49) did someone disagree with the narrator’s interpretation of the text, directly engaging with aspects related to the objectives of our study. Specifically, in a comment on a video in Spanish, a viewer expressed disagreement with the mediator’s interpretation of the story (claiming that the bear was a figment of William’s imagination), and, later on, the narrator also responded to that comment.

In more than half of the videos (57%), both the book and the narrator were shown at the same time. Some videos (17%) used a top-down shot showing only the book’s pages and the narrator’s hands turning them, accompanied by voice-over narration. Others opted for slideshow formats (25%), either with (8%) or without (17%) a superimposed image of the narrator telling the story.

When both the book and the narrator were shown at the same time, the narrator was typically seated on a sofa, bed, or chair (38.9%), sitting on the floor (2%), or standing (6.1%). In these cases, viewers could hear the narration but could not see the book in detail due to the distance from the camera or low resolution. The book was easier to observe when the camera focused directly on it, though the narrator’s gestures were less visible—something that, on the other hand, we did not analyze in this study.

Two types of productions were identified. The first, more elaborate and less common (5.6%), used more than one camera and included post-production editing, allowing for simultaneous shots of the picture book and the narrator. These videos often featured credits or background music. The second, the more prevalent type of production (94.4%), could be described as a homemade video, typically recorded with a phone or a static camera in a single shot, or downloaded from a video call application.

Figure 1Visual narration technique and image quality 

The videos published on YouTube were primarily aimed at viewers who were already subscribed to the channel. As a result, narrators with a larger number of subscribers did not begin directly with the story but instead included an introduction such as a greeting, a welcome to their channel, or a brief overview of the topic they were about to cover. This was the case in 30 of the videos. In these instances, there was a noticeable tendency to pose questions about the story, and, given the asynchronous nature of the narration, the mediators themselves would suggest possible initial hypotheses. At the end of the narration, they often included comments and follow-up questions for the viewers.

A significant part of the narrators (46%) read the verbal text from the picture book without ever elaborating on it. They simply turned the pages, showing them either fully or partially.

Figure 2Reading strategy 

The literary text, as it appears in the original work, is reproduced entirety in all the videos analyzed, without omitting a single word. However, only a few videos (19.2%) display all the illustrations entirely, and in 5.7% of the videos, the book itself is not shown at all; instead, only the narrator is visible while reading or dramatizing the story. Therefore, while the literary text is always presented in its complete form, the visual text is often only partially reproduced, as we will explore further.

Figure 3Strategies of omission or amplification 

Regarding the peritext, the resulting verbal text often includes elements present on the cover: the title was mentioned by all the narrators, and the author was mentioned by 35 of them. However, this is not the case for the resulting visual text: in 7 of the videos, the cover is not shown.

Figure 4The peritext (the cover) 

The peritextual elements inside the book do not seem to garner the same attention. Only 23 narrators showed the pages that appear before the first sentence. The other 30 narrators skipped showing the cover (which depicts the bear stepping off the boat), the back cover (where the bear is seen waiting for the bus, above the dedication), and the following page (where the bear enters a house). Instead, they opened the book directly to the page containing the first sentence.

The verbal text is not usually amplified during these moments of the narration by adding descriptive words about the actions depicted in the illustrations, nor do the narrators draw attention to the visual elements—at least not with deictic expressions or rhetorical questions prompting viewers to focus on illustrations that lack text. None of the 30 narrators who did not show the peritextual pages described the actions depicted there. Among the 23 narrators who did show the illustrations, only 7 made any observations about what was seen in the pictures or began the narration from the cover, while 16 started their narration by reading the first sentence.

As for the final peritext, the last page of the book, where the bear is seen parachuting onto another house, was not shown in 20.8% of the cases. However, of the 42 remaining videos that did show this page, only 12 expanded on the verbal text to describe what was happening in the illustration.

Figure 5Visual reproduction of the initial and final peritext 

This lack of attention does not extend to pages that contain both words and illustrations. The words are never not read, and thus, pages with text are almost always shown, except in the 5.7% of cases where the book is not shown at all.

Regarding whether the entire surface of the page is displayed—that is, if the viewer is allowed to see all the details of the illustration—most videos (39) showed a fragmented reproduction: The narrators did not display the entire image but rather zoomed in on specific details, or the illustrations were not clearly visible due to the camera’s distance from the book.

It is significant to observe what is hidden. 40 of the mediators show the full page where the bear is hiding behind the screen, but in 13 videos (9 of which were narrated in Spanish), this detail is not visible, either because the entire page is not shown or because only details are presented, deliberately excluding the bear.

When amplifications occur in the resulting verbal text, they can happen on pages with or without text and illustrations. For example, when the text says “and at night, while William dreamed...” (Cooper, 1993a, p. 11-12), 15 narrators describe the actions of the bear. As for the initial peritext, the narration was only amplified in 5 cases. Some of these amplifications were used to interpret the story, offering opinions about the existence or nonexistence of the bear that scares William. In the final peritext, only 12 of the mediators expanded upon the image of the bear parachuting.

In interpreting Helen Cooper's picture book, we find notable differences. Among the 53 narrators, 13 conclude that the bear exists, 13 believe the bear is a product of William’s imagination, and the remaining 27 do not take a clear position either way.

In some cases (videos 17 and 39), the explanations become contradictory. Certain scenes are said to represent William's imagination, but when it comes to the images at the end, it is interpreted that the bear leaves the house and heads toward another child's house.

The 13 mediators who interpret that the bear exists amplified the narration both in the peritext and in the threshold of the illustration. However, the other 13 who amplified the narration to claim that the bear did not exist had hidden details from the illustration in the resulting visual text, such as the image of the bear hiding behind the screen or the images in the final peritext. The remaining 27 did not add any interpretation in either direction.

The fundamental attitude, then, is to conceal details that complicate the reading. But in a book where those details are found in the illustrations, the only way to convey these contradictions to the audience is by pointing them out or amplifying the text. This leads to two types of amplification, which we categorize as rich or reductive.

Rich amplifications emerge to make sense of actions that are only shown in the illustration. These amplifications attend to all the clues aimed at an attentive reader and challenge the version given by the literary narrator, regardless of the conclusion drawn from them.

This type of amplification appears in a small number of mediators (15), but when amplifications occurred, they were more common than reductive ones (7), which will be discussed next. An example of this is the continuation of the narration suggesting that the bear will reach another child’s house, who, in turn, will also be afraid—explanations that point to an open-ended conclusion. This contrasts with narrations that chose to ignore these last pages in both the visual and verbal text, implying that the story ends when William falls asleep and the bear disappears—forever, from his imagination.

Reducing amplifications refer to those interpretations in which details that challenge the literary narrator’s version have been omitted. An example of this is in video number 14, where the narrator emphasizes with words, tone of voice, and gestures while reading the first page, that William was afraid of bears because he thought he had seen one: “It was all because one day he thought—remember what I said—he thought [the emphasis is added to these words in italics] that he’d seen a bear there under the stairs,” and she even underscores it with a comment, “Poor William, who thought there was a bear under the stairs.” Furthermore, she anticipates from the beginning that what William saw was not a bear, but rather “some furry material.”

Another mediator, driven by the need to discuss the irrational nature of the story, directly addresses the viewer with a rhetorical question about the evident fictional nature of the tale, emphasizing the absurdity of some actions, such as when the bear draws a portrait of William.

Mediators who perform reductive amplifications seem to have intentionally concealed the clues that could lead to alternative interpretations, showing only select images and only in part (especially the detail of the bear hiding behind the screen in the dark room or the illustrations where the bear is inside the house while the humans are sleeping), so that these details align with their interpretation.

Others take advantage of the deliberate ambiguity of the phrase “And at night, while William dreamed” (Cooper, 1993a, p. 11-12) to explain that everything the bear was doing—and thus, the bear's very existence—is part of William's dream.

Conclusions

This recourse to "sueñismo de ocasión" (Montes, 2001), visible in amplifications that seek to finish the story with a rational interpretation, is employed by several mediators who base their entire interpretation of the story on this phrase and episode. For them, the actions depicting the bear flying to another house serve to justify their own interpretation: the bear never existed; it was merely a product of the child’s imagination and nightmares.

In some cases, the concealment of the initial and final peritexts or clues in the illustrations removes any elements that contradict the interpretation the mediator has constructed during their appropriation of the text.

In other cases, reductive amplification has been carried out despite the mediator being attentive to details in the illustration that others had overlooked. These mediators may be guided by their conceptions of what is acceptable or understandable for a child reader, believing the child may be either unprepared for a complex story or too impressionable for a narrative that could potentially frighten them.

This research suggests that some mediators who present picture books to an audience through narration (whether digital or in-person, as the conclusions are clearly applicable to live narrations) may be giving insufficient attention to the illustrations. This lack of attention seems to stem from a limited familiarity with picture books, leading them to apply knowledge based primarily on their experience with non-multimodal literary texts. This could explain why some mediators jump straight from the cover to the page with the first sentence, fail to pause at the final pages if they are located after the last sentence, and do not recognize the peritext as part of the narrative. They also fail to notice contradictions between the words and images, missing out on crucial clues needed to construct meaning. Based on these findings, the particularities of picture books appear to be insurmountable obstacles for some adult readers.

A revealing example of how adult and child readers interact differently with picture books was verbalized by one narrator (number 33), who admitted that in a first reading, she had believed the bear was not real, and it was her child who showed her otherwise.

These issues affect not only comprehension but also mediation practices. The materiality of the book as an object offers the reader the possibility of becoming an interactive reader of a text that resists linear reading (Tabernero-Salas, 2018). This characteristic of the picture book can be reproduced or at least facilitated in shared and joint reading practices between an adult mediator and a child reader. However, the analyzed digital narrations are one-way and present not only a linear reading trajectory but, in many cases, also an incomplete and fragmented one.

In digital narrations, it would be important to consider some of the recommendations made for oral storytelling of picture books in real-life situations: not settling for simply reading the verbal text of a picture book (Ellis & Brewster, 2014; Ellis & Mourão, 2021), revisiting the peritext after reading (Harris, 2005; Martínez et al., 2016), and even interrupting a linear reading to go back and examine details already mentioned, if the reading requires it.

On the other hand, the recommendation to maintain an open attitude toward the possible interpretations that readers (especially young ones) may have (Harris, 2005) demands, at the very least, that in an asynchronous reproduction, significant parts of the text should not be hidden but should be amplified or at least highlighted or pointed out to encourage the reading process.

The research provides relevant insights into the significant role that adults play in transmitting and, therefore, in reading and interpreting multimodal literary texts. The outcome of mediation that is not attentive to the signals in the text may nullify the work’s rich polysemy, denying young readers the opportunity to construct their own meanings.

AUTHOR'S CONTRIBUTION

Cristina Quesada-Padrón: Project management; Formal analysis; Conceptualization; Data Curation; Writing - original draft; Writing - proofreading and editing; Research; Methodology; Resources; Supervision; Validation; Visualization.

Jesús Díaz-Armas: Project management; Formal analysis; Conceptualization; Data Curation; Writing - original draft; Writing - proofreading and editing; Research; Methodology; Resources; Supervision; Validation; Visualization.

Carmen A. Perdomo-López: Conceptualization; Data Curation; Research.

Patricia Mauclair: Conceptualization; Data Curation; Research.