Multimedia Learning

Documenting our experiences for various purposes has been a human constant since creation. The use of imagery in various cultures to accomplish this could be ascribed to the lack of a written language, such as the petroglyphs of the American Southwest, or to the prevalence of illiteracy, such as the religious icons of Russian peasantry. We might equally acknowledge that imagery is an instinctively powerful medium, and has been used throughout history for that reason. Why is this so? We can certainly point to the speed of visual perception, or its permanence. But imagery, which Stephen Kosslyn terms “a basic form of cognition (that) plays a central role in many human activities”, can be used to tap into long term memory. Images are encoded into the brain via the ventral and dorsal systems, which are inputs to associative memory. Working memory processes visual and verbal information through two separate channels, and learners can use both effectively to select and organize new information that can be integrated and passed into long term memory as a schema. It is important to note that textual information alone, such as this page, may be encoded and processed via the visual channel. However, studies have shown that inclusion of images with text increases the likelihood that the learner will employ both channels, and therefore create the potential for deeper learning. This approach is not foolproof however, and instructional designers must create artifacts that complement cognitive architecture, or risk overloading working memory.

Coding information into memory through using both visual and verbal channels is known as dual-coding theory. An image viewed by a learner is encoded by the visuospatial sketchpad component of working memory as an imagen. Descriptive text in proximity to the image is generally encoded by the phonological loop component of working memory as a logogen, but may also be encoded via the visuospatial sketchpad. The internal representation between the image and the text must also be encoded in working memory…this referential connection is key for successful integration with long term memory. Memory for this integrated information may be improved by the two systems acting as load-balancers for working memory, and perhaps also through the enhanced information maintenance ability of working memory when text is bound to an image through proximity. Presenting the learner with visual and verbal explanations that are positioned next to each other, rather than simply found on the same page, is termed the contiguity effect, and has been shown to enable deep learning of this dual-coded information through effective construction of referential connections that are stored and retrieved from long-term memory in the form of schemata.

However, dual-coding can actually incur cognitive load if not employed carefully. If the visual and verbal explanations are not closely integrated on the page, working memory must be directed to locating and comparing the referents. Known as the split-attention effect, this reduces working memory’s capacity to support learning. Dual-coding based instructional design can also incur working memory load if textual descriptions merely redescribe an image or diagram. This redundancy effect requires working memory to differentiate between unnecessary and essential information, a non value-added process with deleterious results. These cognitive load effects highlight the potential risk of combining images with text inappropriately for both domain experts and novices, which can be further understood when seen in context with some functions served by placing images in text. Decorational images are just that, decoration with no educational component. An image of a seagull placed next to a passage about the ocean would add no value, and would likely elicit the redundancy effect. Representational images depict the actual textual descriptions, and depending on their complexity may incur the split-attention affect. Organizational images complement the structure of the text, and would likely provide the desired benefits of dual-coding theory as they support the text rather than competing with it. With this background, the use of imagery to drive knowledge transfer can produce deep learning and practical benefits.