Barry R. Hill
Lebanon Valley College
With the dramatic shift from analog to digital technology in music production, a major issue arises in how audio education programs provide instruction for students desiring to become recording engineers. The challenge is helping novices develop a solid foundation of how recording systems in general operate so they can more easily transfer that understanding to any new and different type of technology they may face in the future. Analog systems have always been relatively straightforward and consistent in interface design and operations; thus universal operational patterns are more easily understood and applied. Software and digital hardware systems, however, are anything but consistent and do not generally follow traditional paradigms of operation and layout. This makes it nearly impossible for inexperienced engineers to do anything but learn each system by rote, severely inhibiting transfer.
For a variety of reasons, many educational institutions are installing Digidesign’s® Pro Tools® and similar software systems in their labs. Though students can easily be taught specific operations in programs such as Pro Tools, they have a difficult time understanding conceptually how such systems work. This causes major issues when confronted with a novel situation or different type of system. Novices do not have the background and experience professionals have developed over the years that allow them to visualize underlying structures and operations of recording systems. Professional engineers possess a knowledge structure that has been refined over time, resulting in a generic mental model of how analog and software-based recording systems generally operate. These conceptual models provide a template that can be quickly applied to different systems. Thus an individual does not have to learn a new system or software package from scratch, but can instead transfer the knowledge of how it should work to figure out more easily how to get it done on the new system. In other words, instead of just teaching students how to use Pro Tools, audio instruction should help students learn the underlying system operations and concepts that apply to any system type, both analog and digital.
By examining how professionals approach recording systems, and eliciting their mental models, we can draw comparisons to the developing models possessed by students in recording education programs. Patterns and major concepts from the experts can then be applied to improving audio education, an academic area that has suffered from a notable lack of targeted research and attention in instructional matters.
The purpose of this particular study was to elicit and compare mental models of recording systems, and Pro Tools specifically, between professional recording engineers and students who were about to graduate from a college degree program in recording technology. This comparison was presumed to reveal structural and procedural knowledge gaps and inaccuracies that students possess, providing direction as to how the curriculum could be modified to diminish this gap.
Background to the Problem
Fundamental recording procedures and techniques using analog equipment have not changed much over the years—analog recording systems primarily differ in terms of scale and features, not operational concepts. A Mackie twelve-channel mixer operates essentially the same as a large-for-mat eighty-channel SSL. It has therefore been fairly straightforward for novices to gradually develop an appropriate concept of how such systems operate. Traditional analog systems are physical devices, providing tactile and visual representations of operational procedures; novice engineers can thus more easily “see” the underlying schemes of operations. Once they learn one particular recording console they can generally transfer that knowledge to another system. The basic operational controls are always present, differing only in location, complexity, and sometimes terminology. Fundamental signal routing options, though they may be accomplished in slightly different ways, are always available in order to perform standard procedures. An engineer with an appropriate knowledge of how recording systems operate would know what to look for when confronted with a new system, providing efficient transfer of knowledge and a much-reduced learning curve.
The difficulty in recent years stems from the proliferation of digital and software-based recording systems. Such systems essentially take all of the various processing and routing equipment in the recording studio and duplicate their functions within software. This “studio-in-a-box” concept has tremendous appeal and offers powerful production capabilities, but the result is a very complex product design based on a virtual interface where core functions are accessed via numerous menu options, windows, and “soft” buttons. There is no way to “see” exactly what is happening to the audio signals as is possible on a hardware device where every function is physically controlled by knobs, switches, faders, and cables organized in a logical, ergonomic layout. Experienced engineers can apply their mental knowledge and understanding of how recording systems work and eventually figure out newer digital devices. Novices, however, have no such background to draw upon and must learn procedures by rote for each system they use. Development of recurring patterns comes much slower, if at all. The wide variation in product design does not foster and support user awareness of such core operational patterns, making it even more difficult for novices to develop standardized structural concepts in the field of audio recording.
However, in spite of these design differences all recording systems do in fact provide core operational patterns and concepts for routing and processing audio signals during the recording process. Student engineers need to learn these fundamental operating patterns of signal flow in order to transfer their skills to the wide variety of recording systems and devices on the market. The goal is for young engineers who find themselves in front of a different system to avoid having to start from scratch. Instead, they should be able to draw on prior experience and knowledge about how such systems work underneath all the various controls and options. In other words, they need to develop an accurate mental model of recording systems.
A mental model represents an individual’s conceptualization of how a system operates, based on experience and/or instruction. Borgman (1986) explains that mental models “describe a cognitive mechanism for representing and making inferences about a system or problem which the user builds as he or she interacts with and learns about the system” (p. 48). A model of a device is “formed largely by interpreting its perceived actions and its visible structure” (Norman 2002, p. 17). For a simple example, take the volume control on a home stereo system. Most often it is a rotary knob, and users readily expect that turning it to the right (clockwise) will increase the volume. Why? Because this action has become “standard” in our culture, and people have learned this through experience with similar knobs.
Next, consider the automobile. Most of the basic controls function the same regardless of manufacturer or model. When experienced drivers sit in a new vehicle they have never seen before, they know what to do in order to go faster, slower, and turn right. There may be features that are not offered on other cars, or are presented somewhat differently, but the basic operational controls and patterns are the same. Now, imagine an automobile where the turning mechanism is a horizontal slider mounted on the dash. This violates the driver’s mental model of steering, but perhaps if the control were labeled appropriately accommodation might not be too difficult. However, if the slider were mounted vertically, the driver’s conceptual model of steering would be violated more severely for two reasons: people are used to turning a wheel to steer a car, and traditional design dictates that a vertical control adjusts something up and down, not left and right. The way in which a system’s design presents itself to the user is crucial for facilitating correct operation; violations in interface and system design hinder the user in accomplishing tasks.
In the recording industry, long-established standards of analog equipment and procedures have helped develop relatively consistent mental models among professional recording engineers. Analog recording consoles generally use the same (or very similar) physical layouts and controls. Every engineer knows what an insert point is on the patchbay and what it is used for. The same goes for faders, auxiliary pots, bussing, and so forth. Much of this is rapidly coming “unglued” for two reasons: 1) digital and software-based systems by their very nature violate, or at least obscure, the analog paradigm to some degree, and 2) manufacturers seem determined to develop products purposefully different from their competitors, preventing consistency among systems that would otherwise help foster some semblance of a common conceptual model.
An unspoken issue in audio education today is the increasingly widespread use of Pro Tools. This software-based recording system has become the de facto standard in today’s recording and film industries, and so educational programs are selecting this particular product so students will be familiar with the system once they enter the industry. It is also much less expensive than traditional analog hardware and therefore more accessible for institutions with limited budgets. This makes logical sense on the surface, but the limitations of such an approach are crucial to understand.
First, students have difficulty developing accurate mental models of such a system since functions and operations are virtual and are rarely transparent to the user. Second, despite the fact that Pro Tools is a standard, there are numerous other software- and hardware-based recording systems in use today. They represent a wide array of design concepts and operational patterns, and so the issue is again the development of accurate system models that allow users to transfer effectively and efficiently their generic models to any new system. Lastly, students learning without appropriate system models do two things: they learn functions by rote (which are therefore not generally transferable to novel situations), and they develop their own conceptual models, which are much more likely to be inaccurate and inappropriate to help them in future situations (Norman 2002, Parush 2004).
From personal experience at our institution, we have found that students who have learned recording techniques on analog equipment first before moving on to digital systems sometimes do not seem to do well in transferring the knowledge gained from the analog experience to the soft-ware-based Pro Tools system. Pro Tools uses the typical analog recording console and operational patterns as its design metaphor (though rather loosely), but as it is a virtual representation many of those functions and procedures are not immediately visible, nor are they straightforward, often lost in a maze of menus, windows, and new terminology. The result is that students seem to have difficulty applying standard analog procedures and struggle to find ways to accomplish recording tasks. The conceptual models we have attempted to help them develop do not always seem to transfer very well, either because they do not see the connection and/or because the software obscures the relationships. Students who have performed adequately on analog consoles, which are fairly straightforward to understand, have difficulty functioning effectively within the virtual software environment. This should not be confused with merely learning how to record, mix, and use plug-ins in Pro Tools; when confronted with a unique signal-routing situation to accomplish a certain task, the students’ fragile mental models often fail. By identifying exactly what is missing and inaccurate in their models, we can change instructional methods to compensate.
In order to accomplish this, it is necessary to find out what mental models professional recording engineers possess about recording systems and how they apply them to various tasks when using Pro Tools systems. Using this as a benchmark, we can compare with students who have largely completed a degree program in recording engineering and determine what models they possess for such a system. The differences can provide clues that could inform instructional strategies and practices throughout the curriculum, including decisions regarding selection of system types (hardware and software) for installation in school recording studio laboratories. Though the lessons learned from this study can be useful for other educational institutions, the objective was to investigate the situation at the researcher’s school specifically, which in turn dictated decisions on methodology and overall study design.
Review of the Literature
How do human beings organize and structure the knowledge they accumulate over time? How do we approach a novel situation and attempt to predict potential solutions and outcomes? Why do individuals carry so many varying, and often conflicting, notions about even simple concepts and principles? Research into mental models has helped psychologists, scientists, design engineers, teachers, and many others understand and work with how humans structure knowledge. We want to know how people understand a specific content domain, whether that be the law of gravity, the major concepts in organic chemistry, or the operational workings of a nuclear power plant.
Educational psychology research has shown that individuals undergoing the same instructional experience leave with widely varying understandings of the covered concepts (Perkins and Unger 1999, Perkins 1992). For instance, studies have revealed that a majority of high school graduates do not possess accurate understandings of even the most basic principles in science and math, such as the distinction between heat and temperature (Perkins 1991). These are not the low achievers, but the “bright” academically motivated students who are college-bound to major in these areas. Engineers in a nuclear power plant can suddenly find themselves faced with flashing and buzzing warning indicators, necessitating immediate diagnosis as to the probable cause. Unfortunately this presents a highly complicated challenge for training due to the multitude of differing possibilities, requiring the operators to have an absolutely accurate mental model of how the entire system works under various normal as well as problematic conditions (Norman 2002). A more common example of simple mental model usage is how people believe a home heating thermostat operates. Upon entering their chilly home, people will typically turn the temperature up much higher than they really desire under the impression that it will warm the house faster. This is incorrect, but it stems from an inaccurate conceptual image of how the system works (Norman 2002). Identifying how people perceive systems and content domains is tremendously useful, and mental models provide a theoretical framework in which to accomplish this.
One of the most common applications for applying mental model research is the comparison between experts and novices in a given content domain (Jonassen, Beissner, and Yacci 1993; Gentner and Stevens 1983; Carley and Palmquist 1992; Villachica, et al. 2001). Novices are limited by their lack of real experience—the terminology and theories they are learning are not yet meaningful to them. Experts, on the other hand, approach a task or problem with not only a vast knowledge base from which to draw upon, but also the ability to utilize that information in ways that come only from long experience (Johnson 1988, Jonassen and Henning 1999). It is this wide array of experiences that provide authentic meaning and connections to the content, representing a much higher cognitive level of awareness and application.
It is much easier to package knowledge base content for instructional purposes than it is to help beginners think and operate in ways that are as efficient and effective as that exhibited by professionals; in other words, it is one thing to deliver content during instruction, but quite another to help the learner structure that information in meaningful ways that can then be applied in real situations. The experience gained by experts over the years of performing tasks and confronting new problems develops specific, purposeful relationships between the items stored in their memories. This allows them to approach a novel situation, identify key components relevant to the immediate task, and develop a solution in a process that is much more efficient and streamlined than what a beginner must follow (Gentner and Stevens 1983, Johnson 1988). Experienced medical doctors can provide a nearly immediate diagnosis to presented symptoms simply because they have a rich database of similar situations and results. Interns or residents must sift through pages of memorized information in the attempt to narrow down possible options. By examining and comparing the mental knowledge structures of both groups, performance gaps can be identified and results employed to improve instructional strategies and practice (Jonassen, et al. 1993, Jonassen 1995).
For this study, the mental models of professional and novice recording engineers, pertaining to generic recording systems and application to Pro Tools software, were elicited and analyzed so as to answer the following research questions:
· What mental models do professional, expert recording engineers possess about recording systems? · How do these engineers apply this structural knowledge specifically to Pro Tools recording systems? · What mental models do graduating seniors possess about recording systems? · How do these novices apply this structural knowledge to Pro Tools recording systems?
Participants included three professional recording engineers and ten seniors in a four-year college music recording degree program. The number of participants reflects typical population sizes for qualitative studies, primarily due to the tremendous complexity and amount of data generated through such methods. It also provides an appropriate snapshot of the overall student population in the degree program at the college. Criteria for selection of experts required that they have several years of experience in studio recording on some variety of recording system types. Mastery of Pro Tools was not considered essential for all experts as the objective was to determine how experienced engineers approach task performance on such a system, not how quickly they can complete the task. A mixture of expertise between the professionals was considered desirable so as to provide a diverse comparison of mental model approaches. Student selection involved identifying individuals who had generally performed well academically throughout the program while also representing a range of competency in their understanding and performance with recording systems. All students had experience with increasingly complex analog recording systems during the sophomore year, software-based systems the junior year, and finally a senior-level capstone course utilizing a combination of analog and digital systems.
The review of the literature of mental model studies clearly suggests the need for a combination of methods for the purpose of eliciting and comparing mental models. The objective is to learn not only what individuals know about a subject (content knowledge), but to determine the particular relationships and connections they happen to perceive among that content (structural knowledge). Further, it is critical to observe how individuals actually apply this knowledge in real situations and problems (procedural knowledge). While content and structural knowledge is often extracted and represented through quantitative data collection and analysis, eliciting complementary procedural knowledge, along with the cognitive thought processes involved, requires actual performance by participants (Jonassen, Beissner, and Yacci 1993; Jonassen 1995). This type of procedure is best served through the think-aloud protocol and rich description of the activity, a qualitative method commonly found throughout the literature in psychology and educational research (Ericsson and Simon 1993). Additionally, a qualitative examination of the structural knowledge data provides a more in-depth, potentially more meaningful insight into what the data might represent. Thus a mixed-method research design was completed, combining some quantitative, but mostly qualitative, methodology. Bransford, Brown, and Cocking (1999) discuss the development of an interdisciplinary approach to studying the mind and learning and note that “The introduction of rigorous qualitative research methodologies have provided perspectives on learning that complement and enrich the experimental research traditions” (chap. 1, Development of the Science of Learning). So while certain quantitative data collection and processing was employed, nearly all evaluation of the data was processed through qualitative methodology. Additionally, the investigator’s personal knowledge of the participants provided highly beneficial insight into each individual’s background, academic performance, and personality that helped reveal clues about thinking patterns and performance abilities.
This study can be characterized as an exploratory, descriptive investigation that sought to reveal existing knowledge structures of experts and novices in order to inform and improve instructional practice at the investigator’s institution. Further information on methodology and results can be found in the source monograph Eliciting, analyzing, and comparing mental models of complex audio recording systems between professional and novice recording engineers (Hill 2006).
Data Collection and Analysis
Structural knowledge of generic recording systems was elicited through the card sort procedure, a long-established method for cognitive psychology research (Jonassen, et al. 1993; Harper, Jentsch, Berry, Lau, Bowers, and Salas 2003; Fiore, Cuevas, and Oser 2003; Cheatham and Lane 2002; Chi, Hutchinson, and Robin 1989; Miller 1969). The study participants were presented with twenty recording system terms and graphics such as bus, assignment matrix, aux send, and insert point. They were asked to sort these into groups in any arrangement that seemed to make sense to them. Thus items that seemed to be similar in some way were grouped together. Figure 1 is a screenshot of a sample card sort completed by the investigator.
For example, an experienced engineer would most likely group mic pre, assignment matrix, and bus in the same group as they all represent the signal path for recording a microphone signal to the multitrack recorder. The resulting groups tend to show how individuals “see” the various concepts and how they relate to one another. This proximity data was processed through a statistical processing algorithm called multidimensional scaling (MDS), found in standard software packages such as SPSS, which interprets distances among the grouped data in order to produce cognitive maps. These maps provide a visual representation of how closely or distantly related the twenty terms were perceived to be by the participants.
Figure 2 is a group MDS map representing how two of the experts organized their sorts. The shading and arrows were added during analysis to help indicate significant clusters of terms that emerged. An accurate map should show tight clusters of terms that relate to each other either conceptually or operationally in ways that make sense for that particular content; for example, in music history all classical composers would be grouped into one pile separate from baroque composers. A lone term located between clusters, such as AssignMatrix in the middle of figure 2, usually means that item was placed into different piles (groups) by the participants included in the map plot. Most of the terms were grouped identically by both individuals, showing a high degree of consistency between their mental models. Several cognitive maps were produced from the participant sort data to represent experts and novices in various combinations for comparison. Additionally, the original card sorts were each examined qualitatively to determine what individuals might have been thinking as they formed their particular sort. This is accomplished by searching for
Figure 2. Cognitive map representing how Experts 2 and 3 organized their card sort terms. Shading and arrows were manually added during analysis to highlight significant groupings common to both participants.
patterns in the results and is a common analysis procedure for the card sort method. The purpose for this is primarily to provide useful clues and insight that help the investigator better understand and decipher the cognitive map results, thereby informing the overall study analysis and conclusions.
Procedural knowledge of how participants applied their structural knowledge of recording systems to Pro Tools was elicited through the think-aloud protocol (Ericsson and Simon 1993). Each participant completed five individual tasks in Pro Tools; these consisted of various session situations where the engineer must set up the software to accomplish a specific task. Difficulty ranged from simple processing on a single audio track to more complex parallel bussing and processing, requiring a comprehension of traditional signal routing and how this is achieved in the Pro Tools software. Think-aloud protocol requires the participants to verbalize everything they are thinking as they go through the exercises; this provides an insight into how they are approaching the problem. All user verbalizations and on-screen software actions were recorded by the computer as QuickTime files, facilitating later transcription, coding, and analysis.
Analysis of the protocol produced a variety of data displays that facilitated comparisons among students, experts, and between the two groups. The objective was to identify whether or not participants seemed to have solid mental models to solve the tasks, whether they were able to successfully complete each task, and whether the software seemed to get in their way.
As predicted, the expert cognitive maps were more distinctly clustered and organized than the student maps. General fundamentals of operational concepts and patterns were consistent among the professional engineers. Analysis of the map in figure 1 produced five distinct clusters of terms categorized as follows:
· Cluster 1: Main channel routing/bussing
· Cluster 2: Auxiliary send/return routing
· Cluster 3: Monitoring
· Cluster 4: Inserts for signal processing
· Cluster 5: Channel properties/routing
This organization of the card sort terms accurately reflects how these concepts apply to recording engineering. They also represent an operational grouping, based on application of how these terms and functions are used in recording. Student sorts were sometimes more conceptual in organization, which reflects how students first learn the concepts and then, over time, begin to see how application changes their perceived relationship. For example, a few students grouped all terms related to the word “aux.” Therefore, they might include aux send, aux return, pre-fader, and post-fader in one card pile. Operationally these terms represent two different functions: aux send and pre-fader apply to creating a headphone cue mix while aux send, aux return, and post-fader are used for adding effects to a mix. Thus, to provide an operational sort such as that provided by the experts, these would be divided into two distinct card piles.
Although the experts demonstrated consistent core models, there were differences among them that originated from their differing backgrounds and experiences. One expert, with limited analog background, had spent several years working only in Pro Tools with little diversity in the types of work performed. Thus certain concepts in the card sort exercise were not part of her working mental model and were simply grouped into a fairly meaningless “catch-all” pile. Her overall sort organization was basic and functional with little in the way of a broader perspective. The most experienced engineer demonstrated a higher-level abstraction of the terms, creating a more hierarchical grouping that reflected a solid awareness of both hardware and software systems and how they relate to the basic terminology used in the exercise. This reflected his long experience using both analog and software technology in his everyday work. Overall, the fundamentals were present and common among all three engineers, but clearly experts generate their own specific models that are a product of their particular backgrounds, type of work performed, and how they happen to perceive concepts of recording engineering.
Students generally performed well, but there were scattered plots and less-defined clusters indicating unclear organization of generic recording system terms and operation (see figure 3).
Some of this is typical for novices who simply have not had enough time and experience to solidify their structural knowledge of the field. Upon matching individual results with specific students, however, it also became clear that the weakest performers were those who had not spent nearly the same amount of time in the school labs working on recording projects. The
Figure 3. Cognitive map of all ten students showing a more scattered organization of card sort terms.
best performers, whose card sorts indicated clear understanding of operational and conceptual relationships among the recording terms presented, were individuals who seemed highly motivated during their time in college to learn engineering and had spent a great deal of additional time practicing in the studios. So, accurate mental models not only develop as a product of instructional experience (all students completed the same coursework which emphasized fundamental system models), but are modified through individual interest and types and amounts of experiences. This is consistent with the literature of mental model studies and applications.
The Pro Tools procedural exercises were also quite revealing. As predicted, a few students had simply not spent enough time with the software to remember where to find certain functions. Sometimes they had relatively accurate mental models that enabled them to target likely methods for completing the tasks; other times their models were simply insufficient to get them through the complex, confusing software interface. For example, a student who demonstrated a relatively accurate model did so by verbalizing accurately the signal flow structure and procedure that was required, but could not figure out how to accomplish the task in the software. A student with a weak model would demonstrate confused, unclear, and even random concepts of signal flow and task completion as he talked through his software procedures. Overall, however, most students seemed to exhibit fairly structured, accurate mental models of engineering concepts relevant to the tasks at hand. The graph in Figure 4, as extracted from the coded verbalization transcripts, represents whether participants seemed to have accurate mental models (structured model concepts) to help them through each task.
Most students were fairly comfortable as they approached the Pro Tools tasks—this does not mean they were able to complete them all successfully, however. Even the most proficient students misinterpreted various software terms and functionalities in Pro Tools—not an unexpected outcome. Pro Tools is quite arcane in many ways; even though it generally attempts to follow traditional analog system paradigms, much of the interface is cryptic and confusing. For example, consider a sample screenshot from the software showing two boxes labeled A1 and A1-2 respectively (Figure 5). What is the function for each of these? One relates to input source selection for that track, the other to output options (which include both hardware and internal bus options), but the software does not indicate this, making it difficult for a new user to grasp exactly what all these options represent.
Common functions are often not easily found in software such as Pro Tools; many do not resemble the physical controls typically found on analog consoles and are usually buried in software menus and obscure terminology. Nowhere can an insert point be found by name until clicking on a tiny button labeled only with a nondescriptive graphic. There is no assignment matrix in Pro Tools, which is used for bussing on an analog console;
Figure 5. Pro Tools® track output selector, which routes signals to various processing buses and hardware outputs. © 2004 Avid Technology, Inc. All rights reserved.
instead track outputs as well as aux sends can be routed to the same virtual, internal buses—a more flexible, complex array that is difficult for novices to grasp. Students in the study often employed an audio track for reverb returns instead of an aux track. These are two distinct track options in Pro Tools, but the operational difference is lost on the novice. One student finally summed up his frustration with the wry statement, “something simply labeled ‘aux send’ would be handy.” Experienced engineers eventually figure this out, but novices who do not fundamentally understand bussing, aux send and return loops, and do not possess an overall image of how recording systems are structured have little chance of comprehending what this all actually means. So, they resort to consulting the manual and learning by rote, which does not transfer to other systems and software. Figure 6 indicates the striking balance between students exhibiting structured, accurate models (correct concept) whose performance broke down (incorrect result) due to the poor interface design in Pro Tools.
The experts varied in their success with the procedural protocol based on their experience with the software. Two engineers were experienced with Pro Tools and successfully completed nearly every task. Interestingly, each of them approached various tasks somewhat differently based on individual experience and knowledge. The most experienced engineer was able to think on a much deeper level whereas the other expert, while proficient at using the fundamentals of the software, possessed a much narrower and limited mental model of how the system worked and how to solve new problems—a direct relationship to her less-varied and shorter-term work experience. This is consistent with mental model research, which has shown that a wide variety of rich experiences helps build powerful models over time.
The third expert was not familiar with Pro Tools at all, and he struggled mightily to solve the first easy procedural task. What sets him apart from the weaker students who also had difficulties is that his long experience with recording systems allowed him to target specific concepts and operations he knew were relevant to the task—he only needed to find where they were in the software. Thus, after taking considerable time to learn his way around Pro Tools during the first task, the remaining exercises were completed fairly quickly with a high rate of success. This is testament to the power of a strong, accurate mental model and should be the goal for any instructional program.
Conclusions and Discussion
Answering the research questions posed for this study involve a great variety of issues as described below. However, it all essentially boils down to a fundamental difference between experts and novices. The expert engineers’ mental models of recording systems are hierarchical, rich, complex, and highly connected through extensive experience. While core recording concepts are fairly consistent among them, each expert’s model is unique depending upon his or her particular experience and type of work. Student mental models, while quite respectable, demonstrate a lower degree of connectedness. Concepts are not as concrete, nor are operational relationships and patterns. In some cases, gaps and inaccuracies exist. All of this matches the literature of expert/novice comparisons; however, there are a variety of complex factors involved in the development of accurate mental models, requiring a more extensive explanation.
Importance of Experience for Model Development
The experts in this study benefited from years of experience in a variety of situations working with different types of systems. The better performing novices also were distinguished by having spent more time than the others in the recording studios throughout their degree program. Not only does this promote automation of common procedures, but each novel situation forces consulting and expansion of the model as new information, processing, and results are assimilated. The level of accuracy of the model as it develops is based on two factors: a grounded awareness of the fundamentals that underlie more complex situations (Norman 2002, Jonassen et al. 1993, Parush 2004, Bransford et al. 1999), and a large variety of similar, but not identical, experiences for reinforcement and novelty (Salomon and Perkins 1989, Johnson-Baird et al. 1998, Bransford et al. 1999).
Highly structured mental models are fundamental for expert development (Anderson, Spiro, and Anderson 1978; Kuipers and Kassirer 1984; Kieras and Bovair 1984). This structure comes from accumulation and assimilation of new knowledge and, more importantly, meaningful connections. Experience is the key, and the result is that experts can apply relevant information to a new problem with a higher level perspective that comes from their extensive experience in a variety of situations. Novices have no such streamlined, targeted processing abilities and must sift through a great deal of information that may, or may not, be applicable or even present in their knowledge base (Johnson 1988, Jonassen and Henning 1999, Gentner and Stevens 1983).
Accurate Models Depend on Appropriate System Models During Instruction
Though some of the students possessed fairly structured models of recording systems, there were numerous uncertainties and inaccuracies in spite of the instructional emphasis on fundamental principles. The challenge for teaching novices is to help them develop accurate, though relatively simple, models that are fundamentally sound (Mayer 1989; Coll and Treagust 2003; Kieras and Bovair 1984; Fiore, Cuevas, and Oser 2003). The key is to help them learn correct fundamentals in order to reduce misunderstanding while providing extended experiences that force them to see when their inadequate models will not work (Clement and Steinberg 2002; Norman 2002). These correct fundamentals come from presentation and reinforcement of appropriate system models during instruction. This increases the likelihood that students will base continued experiences and instruction on a grounded foundation. If left on their own, individual perceptions will dominate model development, almost ensuring inadequate and inaccurate system models that will hamper performance (Norman 2002).
Instructional models must be simple enough for the novice to understand, yet accurate and faithful to the larger system or content area so that learners can fill in the details as their structural knowledge develops. Solid foundations can better facilitate transfer to more complex and novel problems, but the experiences must be authentic and extendable to real applications. Many instructional design theories support this concept: microworlds, elaboration theory, constructivist learning environments, and goal-based scenarios to name a few (Reiber 1992, Reigeluth 1999, Jonassen 1999, Schank et al. 1999). Mental model research supports the notion that indi-viduals’ models “seem to be simpler than formal mechanics” (Gentner and Stevens 1983, p. 53). In other words, the models people tend to develop usually do not match complex system designs, thereby creating the potential for inaccurate understanding and use of the system (Norman 2002); Pro Tools is an excellent example of this. However, it should be cautioned that oversimplifying the instructional model has the potential to cause confusion when applied to the more complex, complete system (Seel 1999), again reinforcing the issue of not sacrificing accuracy and applicability in the interest of accessibility for novices. For audio education, learning a simple analog console greatly facilitates adaptation to a more complex analog console and eventually digital systems. Exposing students to digital and software systems first is overwhelming, confusing, and is unable to help facilitate clear mental models.
Accessibility of system models is often enhanced through use of analogies (Clement and Steinberg 2002, Brandt 1999). By presenting a new concept as similar in some way to something already understood, learners can more easily identify with, and begin to understand, the new material (Bransford 1999, Perkins and Unger 1999). A common example is that of using water pressure and flow to explain how electrical voltage and current works (Gentner and Gentner 1983). When teaching audio recording, software systems can be more easily understood by comparing them to analog operational patterns as long as the learners have actually used analog systems.
Significant concepts of the system or subject need to be clearly presented. Instructional materials and methods should make meaningful connections of the system explicit to the learners so as to help encode these into their models (Mayer 1989). Audio educators must not merely explain and show how systems operate; they should also explicitly show the various signal flow routing, connections, and so forth. This should be done visually, with graphics, charts, and other materials. Instructors cannot assume learners “get it” during class and lab instruction.
Mental Models are Unstable
Models require use. Briefly learning a concept or procedure and then not practicing it for some time will cause erosion of whatever connections might have existed. The students who performed most poorly on the card sort and software protocol had not spent much time in the studios, nor had they used Pro Tools very much in the preceding year. They were then forced to recall information from long-term memory that had not been utilized. These chunks of information are fragmented, often inaccurate, and largely devoid of meaningful relationships (Norman 1983).
Accurate Mental Models Require Active Thought and Attention
Deep thinking seems to encourage development of more highly detailed and structured models. Active processing has been shown to make a significant difference in learning environments—passive reception of inert knowledge fails to “stick” and enable transfer to other situations (Salomon and Perkins 1989, Bransford et al. 1999, Perkins 1992). The students who during their coursework had demonstrated the ability and interest in thinking through various concepts about recording seemed to perform better overall. Those who did not seem to “dig in” as much tended to show more inaccuracies and less structure in their models. The most experienced expert in this study is a deep thinker who labored over his tasks, considering different options and recalling prior experiences. One of the most successful students took seriously the challenge of learning all he could about the recording process throughout his degree program, and his deep awareness of the field is evident both through his card sort and the procedural protocol.
Personal Interest Fosters Active Mental Model Development
Active thought processing most often occurs when the subject area is personally relevant and meaningful to an individual (Perkins 1992, Bransford et al. 1999, Schank 1999, 2000). Personal interest in the process is a crucial factor—students who want to learn recording will apply their attention to the tasks. Some students are not as intrinsically motivated, resulting in higher instances of incomplete and incorrect understanding of material from class. A comparison between two of the experts also supports this; while one professional is very capable at what she does with Pro Tools at work, she is not as driven to learn more, to apply her knowledge in different situations. The other engineer is very interested in the process, carefully and deliberately looking for connections, new information, etc. As David Perkins asserts in Smart Schools (1992), “People learn much of what they have a reasonable opportunity and motivation to learn.” ( p. 45). Thus personal interest and hands-on application are key for students who want to develop adequate mental models of recording systems and pro-cesses. The students who performed better than others demonstrated a stronger personal interest for recording systems; not only did they apply themselves in classes, but they also spent additional time in the studios and, in some cases, on their own Pro Tools systems they had purchased.
Thus there are numerous factors that contribute to mental model development. Each individual possesses a unique combination of these issues, resulting in a wide variation in how novices and even experts perceive any content area. This causes tremendous implications for educators as they attempt to develop adequate instructional environments.
Implications for Instructional Design
Several conclusions can be offered as suggestions for improving instructional design of audio recording educational programs. Though the design and scope of this study was limited to that of the investigator’s institution, it is presumed that these ideas are broad enough to be generalized to many audio recording educational programs. Fundamentally, these can be applied to most any instructional situation regardless of discipline.
Recommendations for the Study Site Institution
The students’ mental models generally reflected the system models emphasized in their instructional program. Many of the fundamentals learned throughout their coursework were evidenced in both the card sort and procedural protocol exercises. The major differences seemed to reflect how each individual approached and applied himself or herself to learning; the students who were highly motivated to learn engineering and practice in the labs performed better overall. Aside from the various general recommendations detailed below, not much can be done from an instructional perspective to address these issues as they are based on personal, individual choices.
To help close the perceived gap between analog and digital systems, a stronger connection should be made specifically showing students how traditional system models apply in Pro Tools, helping them transition more effectively to their digital-based coursework. This has already been introduced, demonstrating how specific routing and operational patterns from analog consoles are implemented in Pro Tools software.
Design Effective System Models for Instruction
Appropriate system models must be presented during instruction. These must clearly describe and visualize the operating patterns contained within general concepts. It is not sufficient to simply show students how to accomplish certain procedures on a recording system and assume they see the patterns. It is also quite easy for an experienced instructor to forget that students do not share the same level and amount of experience and knowledge about the field. Therefore concepts that may seem obvious and common are not nearly so for the novice. The most experienced engineer’s card sort revealed a conceptual model that makes sense operationally, but would be very difficult for a novice to grasp. His model was hierarchical and transcended analog and digital system awareness, a status not yet within reach of novices.
Learning Tasks Must be Authentic and Meaningful
There has been much research and discussion about making learning meaningful for the learner and authentic to the real world. Decontextualizing instruction does not equip students with cognitive methodologies to handle new situations and solve real problems. Numerous theories and concepts have been developed with this in mind such as constructivism, constructivist learning environments (Jonassen 1999), goal-based scenarios (Schank 1999), open learning environments (Hannafin, Land, and Oliver 1999), cognitive flexibility (Spiro et al. 1992), problem-based learning (Savery and Duffy 1996), microworlds (Rieber 1992), anchored instruction (Bransford et al. 1990) and situated learning (Anderson, Reder, and Simon 1996).
Johnson (1988) compared troubleshooting performance between experts and novices. One recommendation for instruction was that instead of introducing electronics by studying standard electrical theory equations, students should first begin to understand what a circuit is and does. This qualitative perspective provides an abstraction that then facilitates application of quantitative reasoning, i.e., solving equations.
“Rote learning causes problems” (Norman 2002, p. 68). Rote learning is decontextualized, so there are no meaningful purposes for the learner. Memorized procedures do not provide information that can help solve problems or figure out different situations.
All of these point to the need for developing learning environments that have meaning, purpose, and are of interest to the learner. In the field of recording this is easily implemented as students can complete recording projects in the lab throughout their coursework. These projects should be grounded in the instructional system models, reflect practices in the industry, and provide opportunities for students to explore, create, and come away with products they can enjoy and use.
Instruction Must Provide Extensive, Guided Practice
Students must have opportunity for extensive, guided practice with the systems (Bransford et al. 1999, Salomon and Perkins 1989). Merely spending lots of time in the lab is insufficient—in order to develop accurate models this experience and exploration must be guided and supervised by a knowledgeable instructor, otherwise misperceptions and bad habits will become embedded into the students’ models. These practices are very difficult to correct, and any misunderstanding of the basics will carry through unless discovered and resolved early (Norman 2002, Parush 2004).
Instructional Experience Must Provide Variety and Novelty
This experience must take place on a variety of system types, preferably both analog and digital. In the investigator’s own institution, students are initially exposed to an analog console that, while relatively complex, is accessible for the novice with proper model presentation and description. They then transfer this knowledge and experience to a more complex analog console before finally working with software-based systems such as Pro Tools. Judging from years of observation, fine-tuning, and the results of this study, this approach appears to be working satisfactorily for most students who are personally interested in developing as recording engineers.
Instructors must be cautious in their choices of projects and assessment design. When users know exactly how to perform a task, there is little overt thought processes involved—in other words, they do not have to run their models to figure it out. One cannot assume in such a case that the learners know the concepts behind the learned procedures. In order to draw this framework into the open, learners must be taken beyond familiar tasks to force transfer to occur.
The Use of the Pro Tools Recording System for Instructional Applications
Finally, it is crucial that educational institutions do not employ Pro Tools for early instruction. There is great pressure and practical reasons to do so—the entire professional industry uses Pro Tools, and so recording programs are expected to train students on what they will be using after graduation. However, this represents a near-sighted perspective. What the recording industry needs long-term are engineers who understand fundamental operating models on which all recording systems are based. These individuals are then able to transfer their knowledge to other systems that they happen to work with. If engineers do not possess adequate mental models of such systems, they will resort to a step-by-step learning process for each different system, learning by rote, and less able to apply concepts between systems. This also severely restricts their ability to figure out complex situations they have not encountered before, something that occurs nearly every day in music production. Current professionals have transitioned successfully to digital technology probably due in large part to their rich analog experiences; the concern is that new engineers with only digital, software-specific training will not be able to easily learn newer technologies in the future, greatly inhibiting their performance and adaptability.
Anderson, J. R., L. M. Reder, and H. A. Simon. “Situated learning and education.” Educational Researcher 25, no. 4 (1996): 5–11.
Anderson, R. C., R. J. Spiro, and M. C. Anderson. “Schemata as scaffolding for the representation of information in connected discourse.” American Educational Research Journal 15, no. 3 (1978): 433–440.
Borgman, C. L. “The user’s mental model of an information retrieval system: An experiment on a prototype online catalog.” International Journal of Man-Machine Studies 24 (1986): 47–64.
Brandt, D. S. “Analogies and the face of technology.” Computers in Libraries (November/December 1999): 42–45.
Bransford, J. D. et al. “Anchored instruction: Why we need it and how technology can help.” In D. Nix and R. Spiro (Eds.), Cognition, education and multimedia. Hillsdale, New Jersey: Erlbaum Associates, 1990.
Bransford, J. D., A. L. Brown, and R. R. Cocking, ed. “How people learn: Brain, mind, experience, and school,” 1999.
Carley, K. and M. Palmquist. “Extracting, representing, and analyzing mental models.” Social Forces 70, no. 3 (1992): 601.
Cheatham, D. B., and S. Lane. “Differential access hypothesis: The effects of task and information type on the validity of knowledge acquisition methods.” In Proceedings of the Forty-Sixth Meeting of the Human Factors and Ergonomic Society. Baltimore, Maryland: Human Factors and Ergonomic Society, pp. 487–491, 2002.
Chi, M. T., J. E. Hutchinson, and A. F. Robin. “How inferences about novel domain-related concepts can be constrained by structural knowledge.” Merrill-Palmer Quarterly 35 (1989): 27–62.
Clement, J. J. and M. S. Steinberg. “Step-wise evolution of mental models of electric circuits: A ‘learning-aloud’ case study.” The Journal of the Learning Sciences 11, no. 4 (2002): 389–452.
Coll, R. K. and D. F. Treagust. “Investigation of secondary school, undergraduate, and graduate learners’ mental models of ionic bonding.” Journal of Research in Science Teaching 40, no. 5 (2003): 464–486.
Ericsson, K. A. and H. A. Simon. Protocol analysis: Verbal reports as data. Cambridge, Mass.: MIT Press, 1993.
Fiore, S. M., H. M. Cuevas, and R. L. Oser. “A picture is worth a thousand connections: The facilitate effects of diagrams on mental model development and task performance.” Computer in Human Behavior 19 (2003): 185–199.
Gentner, D. and D. Gentner. “Flowing waters or teeming crowds: Mental models of electricity.” In. A. Stevens and D. Gentner (Eds.), Mental Models (pp. 99–130). Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1983.
Gentner, D. A. and A. Stevens. Mental Models. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1983.
Hannafin, M., S. Land, and K. Oliver. “Open learning environments: Foundations, methods, and models.” In C. Reigeluth (Ed.) Instruc-tional-design theories and models: A new paradigm of instructional theory, Vol II (pp. 115–140). Mahwah, New Jersey: Erlbaum, 1999.
Harper, M. E., F. G. Jentsch, D. Berry, H. C. Lau, C. Bowers, and E. Salas. “TPL-KATS-card sort: A tool for assessing structural knowledge.” Behavior Research Methods, Instruments, & Computers 35, no. 4 (2003): 577–584.
Hill, B. R. “Eliciting, analyzing, and comparing mental models of complex audio recording systems between professional and novice recording engineers.” Unpublished doctoral dissertation. The Pennsylvania State University, University Park, 2006.
Johnson-Laird, P. N., V. Girotto, and P. Legrenzi. Mental models: A gentle guide for outsiders. (1998). University of Michigan, School of Information Website
Johnson, S. D. “Cognitive analysis of expert and novice troubleshooting performance.” Performance Improvement Quarterly 1, no. 3 (1988): 38–54.
Jonassen, D. H. “Operationalizing mental models: Strategies for assessing mental models to support meaningful learning and design-supportive learning environments.” Proceedings of the Computer Support for Collaborative Learning (CSCL) Conference. Mahwah, New Jersey: Erlbaum Associates, p. 26, 1995.
Jonassen, D. H. “Designing constructivist learning environments.” In C. Reigeluth (Ed.) Instructional-design theories and models: A new
paradigm of instructional theory, Vol II (pp. 215–240). Mahwah:
New Jersey: Erlbaum, 1999.
Jonassen, D. H., and P. Henning. “Mental models: Knowledge in the head and knowledge in the world.” Educational Technology 39, no. 3 (1999): 39–42.
Jonassen, D. H., K. Beissner, and M. Yacci. Structural Knowledge: Techniques for representing, conveying, and acquiring structural knowledge. Hillsdale, New Jersey: Erlbaum, 1993.
Kieras, D. and S. Bovair, S. “The role of mental models in learning to operate a device.” Cognitive Science 8 (1984): 255–273.
Kuipers, B. and J. P. Kassirer. “Causal reasoning in medicine: Analysis of a protocol.” Cognitive Science 8 (1984): 363–385.
Mayer, R. E. “Models for understanding.” Review of Educational Research 59, no. 1 (1989): 43–64.
Miller, G. A. “A psychological method to investigate verbal concepts.” Journal of Mathematical Psychology 6 (1969): 169–191.
Norman, D. “Some observations on mental models.” In Gentner and Stevens (Ed.) Mental Models (pp. 7–14). Hillsdale, New Jersey: Erlbaum, 1983.
Norman. D. The design of everyday things. New York: Basic Books, 2002.
Parush, A. Interview with Donald Norman on mental models. (2004). Carleton University, Human Oriented Technology Lab Website:
Perkins, D. N. and C. Unger. “Teaching and learning for understanding.” In C. Reigeluth (Ed.) Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp. 91–114). Mahwah, New Jersey: Erlbaum, 1999.
Perkins, D. N. “Technology Meets Constructivism: Do They Make a Marriage?” Educational Technology 18 (1991).
Perkins, D. N. Smart schools: Better thinking and learning for every child. New York: Free Press, 1992.
Reigeluth, C., ed. Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp. 161–182). Mahwah, New Jersey: Erlbaum, 1999.
Rieber, Lloyd P. “Computer-Based Microworlds: A Bridge Between Constructivism and Direct Instruction.” Educational Technology Research and Development 40, no. 1 (1992): 93–103.
Salomon, G. and D. Perkins. “Rocky roads to transfer: rethinking mechanisms of a neglected phenomenon.” Educational Psychologist 24, no. 2 (1989): 113-142.
Savery, J. R., and T. M. Duffy. “Problem-based learning: An instructional model.” Constructivist learning environments. Educational Technology Publications (1996): 135–148.
Schank, R. Coloring outside the lines. New York: HarperCollins, 2000.
Schank, R., T. Berman, and K. Macpherson. “Learning by doing.” In C. Reigeluth (Ed.) Instructional-design theories and models: A new paradigm of instructional theory, Vol II (pp. 161-182). Mahwah, New Jersey: Erlbaum, 1999.
Seel, N. M. “Educational diagnosis of mental models: Assessment problems and technology-based solutions.” [Electronic version]. Journal of Structural Learning & Intelligent Systems 14, no. 2 (1999): 153–185.
Spiro, R. J., P. J. Feltovich, J. M. Jacobson, and R. L. Coulson. “Cognitive flexibility.” In T. Duffy and D. Jonassen (Eds.) Constructivism, and the technology of instruction: a conversation (pp. 57-75). Hillsdale, New Jersey: Erlbaum, 1992.
Villachica, S. W., L. L. Lohr, L. Summers, N. Lowell, S. Roberts, M. Javeri, E. Hunt, C. Mahoney, and C. Conn. “A cognitive map of human performance technology: A study of domain expertise.” 24th Annual Proceedings of Selected Research and Development, National Convention of the Association for Educational Communications and Technology (2001): 437–444.
Digidesign® and Pro Tools® are registered trademarks of Avid Tech
nology, Inc. in the United States and/or other countries. All other trade
marks contained herein are the property of their respective owners.
BARRY R. HILL is an Associate Professor of Music and Director of the Music Recording Technology Program at Lebanon Valley College of Pennsylvania. He is a member of NARAS, AES, and MEIEA, and has extensive experience as a recording engineer. Dr. Hill is also Director for the college’s Center for Excellence in Teaching and Learning, reflecting his research interest in higher education instructional design. Along with providing information and support for college faculty in their teaching, he works with K-12 music teachers to help them rethink their approaches to music teaching and learning. His book Learning Recording Engineering: Fundamental Concepts and Operations is specifically designed based on his research into learning theory. Earned degrees include a doctorate in Instructional Design from The Pennsylvania State University, Masters in Music Technology and Interactive Media from New York University, and a Bachelor of Science in Music with Recording Arts from the University of North Carolina-Asheville.