Volume 33 Number 5 Summer 2001 Applying the Concerns-Based Adoption Model to Research on Computers in Classrooms C. Paul Newhouse Abstract Computers in Classrooms: the Adoption of an Innovation In the 1990s, research efforts in various countries began to develop and apply models for investigating the implementation of computers in classrooms (e.g., Cicchelli & Baecher, 1990; Collis, 1994; Marcinkiewicz, 1995; Rieber & Welliver, 1989; Sandholtz, Ringstaff, & Dwyer, 1992). Many of these were based on teacher concerns about innovations involving the use of computers in the classroom, often referred to as concerns-based models. Most concerns-based models have evolved from the work of Fuller with regard to the concerns of teachers as they developed their pedagogical skills. The concerns-based adoption model, or CBAM , was developed from Fuller’s model in the early 1970s and has since been widely applied to the implementation of educational innovations in general. The model associated with the Apple Classrooms of Tomorrow (ACOT) project , the instructional transformation model, and the Project Information Technology (PIT) models developed in The Netherlands are specific to the implementation of computers in schools and were ultimately based on Fuller’s model. However, the CBAM model has been more fully developed and applied and, thus, is more often referred to by other models. The research in educational computing that has applied these models has typically grown out of interest in issues concerned with the implementation of computers into classrooms. It has become increasingly clear that to address issues such as the effectiveness of using computers to support learning and why computers have had such little effect on schooling, research needs to address how computer support is implemented and, particularly, take into account the specific concerns of the teacher. Marcinkiewicz (1994) argues for the use of concerns-based models in educational computing research because to “understand how to achieve integration, we need to study teachers and what makes them use computers, and we need to study computers and what makes teachers want to—or need to—use them” (p. 234). The Concerns-Based Adoption Model (CBAM) Many models currently being developed for use in research with computers in classrooms have their roots in the CBAM Project from the Southwest Educational Development Laboratory at the University of Texas. The CBAM model for implementing and evaluating the adoption of any innovation in education was first published in the mid-1970s and has undergone further validation over the past 20 years. The CBAM model comprises three key dimensions, stages of concern (SoC), levels of use (LoU), and innovation configuration (IC), the first two being explanatory and the third diagnostic in nature and scope. Each dimension represents a facet of the change process, with SoC and LoU focusing on the implementor, while the IC considers the nature of the innovation itself. The SoC and LoU dimensions were developed out of the work of Fuller, but the IC was developed much later. Associated with each dimension is a designated research method and an instrument to collect and present appropriate data. The CBAM requires the researcher to be immersed within the scene of the innovation and to continually refine judgments associated with the diagnostic dimensions. The SoC describe how teachers perceive an innovation and their feelings about it. It uses a standard set of stages to describe teachers’ concerns about the innovation (Table 1). The instrument used is a questionnaire with a set of scales to prepare a numerical and graphical representation of the type and strengths of participants’ concerns.
The LoU dimension identifies what a teacher is doing or not doing relative to the innovation. It is the sequence (perhaps invariant) that users pass through as they gain confidence and skill in using an innovation, resulting in higher levels of use from nonuse to institutionalisation. The sequence of levels is provided in Table 1. The LoU uses a structured “interview and observation” methodology to obtain the data needed to place participants at one of these levels. The IC focuses on describing the innovation and its operational forms. Although the SoC and LoU deal generically with the change process from the social-psychological perspective of those undergoing the change process the IC circumscribes the innovation. It uses existing documentation about the innovation and interviews with participants, including facilitators, to prepare a two-dimensional chart of the innovation. A series of statements, known as components, are constructed to define the intended outcomes of the innovation. These components are usually listed vertically, must be able to be observed, and represent the innovation implemented fully and successfully. For each component, a range of variations representing a less than satisfactory implementation is described. Variations are listed horizontally, thus forming the two-dimensional chart. The Use of Concerns-Based Models in Computer-Supported Learning Research The application of CBAM, or models based on CBAM is gaining interest throughout the world. Most interest typically appears to be with the LoU and SoC dimensions (i.e., user focus). Very little has been reported which includes an IC (i.e., innovation focus). Of the few who applied all three dimensions to a study, Carbines and Hope both considered the use of computers in primary school classes. Only a small number of other smaller studies have also been reported. Researchers in Europe and the United States have modified the SoC and LoU to describe the use of computers in classrooms by teachers. Some have even attempted to construct instruments to measure the LoU of a teacher or class. Researchers associated with the ACOT projects have developed similar concerns-based models. Hope (1995) conducted a study using all three dimensions and instruments of CBAM to investigate the effect of microcomputer technology on 18 classroom teachers at an elementary school in Florida. He was a participant investigator (principal) looking in to the use of desktop computers to support teacher administrative tasks. The study was very limited in scope, with the IC defining a small number of very specific outcomes. Carbines also applied all three CBAM dimensions in an Australian study investigating the relationship between the degree of implementation of computers for learning in primary schools and selected characteristics of those schools. Moersch (1997) reported his development of a “levels of technology implementation” (LoTi) framework, which defines seven levels of implementation of computers in a school: nonuse, awareness, exploration, infusion, integration (mechanical), integration (routine), expansion, and refinement. The levels are based on the original CBAM levels. From this framework Moersch has developed an instrument to calculate what he refers to as the computer efficiency of a school site. Computer efficiency is defined as the “degree to which computers are being used to support concept-based or process-based instruction, consequential learning, and higher-order thinking skills” (Moersch, p. 52). The instrument accumulates the products of the LoTi level, proportion of computer use, proportion of student use, and number of computers to produce an index for comparison between schools. Clearly the originators of the CBAM model would not approve of such an instrument because it uses a questionnaire rather than an ’interview and observation’ methodology and uses numerical calculations to arrive at levels. Larger projects have typically developed their own models and instruments. In many cases, these models have substantially modified the original dimensions and instruments, which is not condoned by the originators of the CBAM model. Hall and Hord (1987) explain that such modification would require further validation in line with the original development and could not rely on the validation of the original CBAM instruments. A number of models are in their early stages of development but appear to have difficulty in containing the breadth of innovation involved in bringing computers into the classroom. Instructional Transformation Model Rieber and Welliver (1989) and later Marcinkiewicz (1994) developed the IT model to help schools use technology to design their restructuring plans. The IT model draws on the CBAM model and proposes a hierarchy for the successful application of technology to education using a LoU type of approach. Marcinkiewicz and Welliver (1993) applied the instructional transformation model by developing the level of computer use (LCU) questionnaire to measure the level of use of computers in classrooms by teachers. This hierarchy involves six levels as shown in Table 2.
Researchers were motivated by the perceived “discrepancy between advocacy for the use of computers in education and their actual use by teachers” (Rieber & Welliver, 1989, p. 1). Originally, they considered the full six-level model. As a result, items were written for all five stages (i.e., movement between the six levels) of the model and followed the progressive nature of the model. However, when this was tested, they found it was difficult to classify teacher responses. Because it appeared that several dimensions overlapped, they finally focused specifically on whether the use of the computer was integral and necessary to the intentions of the teacher. Thus, the final LCU considered only the expendability of computers—that is, the boundary between the levels utilization and integration—and classified teachers into three levels of computer use; with “Nonuse” being the third. Rieber and Welliver define utilization as “teachers make use of the computers for many educational activities but are not committed” (Rieber & Welliver, p. 28). They described integration as involving the “crucial turning point of fully implementing the computer in education,” because at this stage “teachers assign a purposeful role to the computer” and demonstrate a “commitment to using the computer for appropriate activities and processes is involved in this step” (Rieber & Welliver, p. 28). The key criterion is that at this level “the computer technology cannot be taken away without disrupting the educational process” (Rieber & Welliver, p. 28). According to the model, “the Integration stage is further characterized by the dimension of a teacher’s emergent self-awareness of a role change in teaching from teacher-centred to learner-centred” (Rieber & Welliver, p. 4). However, the LCU did not encompass this dimension, because it would be difficult to collect data using a questionnaire. The authors of the CBAM model specifically claim that their LoU dimension cannot be measured by a questionnaire but rather requires interview and observation. The format of the LCU questionnaire eventually used Nunnally’s paired-comparisons technique; a technique that allows statistical tests to be applied to measure the reliability of this instrument in detecting the boundary between utilization and integration. This instrument was then used in a number of studies as a dependent variable. One study (Marcinkiewicz & Welliver, 1993) involved 170 elementary teachers, with the results being shown to be statistically highly reliable, which demonstrated that the model identified “at least two progressive levels.” The study ensured that teachers were confronted with computers in their classrooms over a period of time to overcome the typical situation where teachers ignore computers. The study concluded that the results suggested that, “the adoption of computer use may occur incrementally or hierarchically as described by instructional transformation” (Marcinkiewicz, 1994, p. 232). That is, it supported the concept of a sequential and hierarchical model to describe the adoption of computer use in the classroom by teachers. ACOT Model for Teacher Proficiency in Technology-Based Classrooms The ACOT projects have been conducted for many years in the USA and have been well reported (Dwyer et al., 1991; Sandholtz et al., 1992). Originally, there were five projects (located at different sites) with varying parameters but all involved classroom learning environments referred to as “high-access-to-technology environments.” A developmental model for categorising the progression of teachers toward expertise in technology-based classroom management was developed from these projects. The model defined three stages of teacher proficiency with technology: survival, mastery, and impact. It should be noted that the ACOT research team also developed a model to describe a five-stage pattern of instructional change: entry, adoption, adaptation, appropriation, and invention (Dwyer et al.). They distinguished this model from the former, which they claimed dealt primarily with the concerns of teachers for classroom management associated with having computers in their classrooms (Sandholtz et al.). In discussing the ACOT projects, Mandinach and Cline (1994) added an innovation stage to the ACOT model for their systems thinking and curriculum innovation (STACI) project. They described this stage in terms of the teacher being involved in restructuring the curriculum and learning activities. However, they did not assume that all teachers went through these stages systematically, for they recognised that some teachers would move between stages both in a progressive and regressive sense depending on a variety of factors and pressures. Sandholtz et al. (1992) concluded from the ACOT projects that teachers changed slowly, often regressed temporarily, and that “teachers progress through stages of concern in an idiosyncratic manner” (p. 479). However, Mandinach and Cline proposed a manner in which teachers moved between the STACI model’s four stages and developed three systems models using system thinking concepts and system diagrams. These models were used to discuss computer-based curriculum innovations at three levels: student learning level, classroom processes level, and organizational change level. The model, developed for the second of these levels, the classroom processes level model involves five domains: instruction, curriculum, resources, support, and accountability. Each of these domains has variables that are either stocks (accumulators) or flows (add to or delete from stocks). For example, the instruction domain has four variables: two stocks, interactive learning and learner-directed learning, and two flows, technology and student and teacher role change (Figure 1). Technology feeds into interactive learning, which in turn feeds into learner-directed learning, which is also affected by the change in roles of students and teachers. That is, this model assumes that using technology will support a move to both interactive learning and learner-direct learning, which Mandinach and Cline (1994) claim are “the two hallmarks of computer-based curriculum innovations” (p. 181). This supported the finding of Dwyer et al. (1991) that ACOT teachers changed their pedagogy to be more child centred, involving collaborative environments that had a more active orientation. 
Figure 1. The instruction domain of the classroom processes level model from the STACI project (Mandinach & Cline, 1994). The SoC with Information Technology Model The Netherlands has long been considered a home of technological innovation in education. For more than a decade, educational researchers, policy makers, administrators, and teachers in The Netherlands have been working together to improve their education system, which they perceive should include a significant role for computer technology. The focus has been to “reform school curriculum, integrate information technology into the new curriculum, and implement new approaches to teacher support and in-service, all at the same time” (Collis, 1994, p. 12). Much of this reform has been embodied in two related projects: Project on the Implementation of New Technologies (PRINT) and Project Information Technology (PIT). These projects had a significant evaluative component that was based on using a modified CBAM model to evaluate teachers. The PRINT project proposed a seven-phase SoC model that represents the obstacles teachers must overcome to make use of computers in their classrooms (Vernooy-Gerritsen, 1994). The project led to the PIT project, which refined these stages to a CBAM-based model, referred to as levels of involvement with IT as an innovation in school practice (Collis, 1994). Collis (1994) was involved in using CBAM to evaluate teachers in the PIT project. The aim of PIT was to support teachers in moving to higher levels of involvement, that is, at least an extended impact level of involvement. She also felt that as a result of PIT, even non-PIT teachers should move to at least Level 4, a routine use level of involvement. The project also considered interrelationships between variables and level of involvement. The evaluation gave questionnaires to 725 teachers, asking for their perception of their current level of involvement and the level they expected to reach by the end of PIT. PIT teachers were also asked to assess the level of involvement of non-PIT teachers in their subject area at their school. The Study: Portable Computers in a School A three-year interpretive study (1993–1995) was conducted at a Western Australian private girls’ school, Hillview College (fictional name), to evaluate the implementation of a portable computer programme (referred to as the PCP). Four years later (1999), a short follow-up study was conducted at the same site. This article only reports on one aspect of this large study (reported further in Newhouse, 1998), the application of the CBAM to the innovation. In the secondary section of the school, the programme began with all Year 8 students (13 years old) having Macintosh portable computers in the first year. This was extended to Year 9 (14 years old) and Year 10 (15 years old) students progressively over the next two years. The wider study addressed the impacts of student-owned, portable computers on students, teachers, the curriculum, and the classroom-learning environment. In each year of the main study, data were collected about students, teachers, and a selection of classes using observations of lessons, interviews, questionnaires, and administrative data and documents obtained from the school’s administration. In particular, the three diagnostic dimensions of the CBAM were used: IC, LoU, and SoC. CBAM was used to collect data from the teachers and “map” the programme as an innovation. The SoC was also used in the follow-up study in 1999. CBAM Data Each of the three diagnostic dimensions of CBAM (IC, SoC, and LoU) has a designated method and an instrument to collect and present appropriate data associated with it. Each of them requires the researcher to be immersed within the scene of the innovation and to continually refine judgments associated with the diagnostic dimensions. IC Checklist An IC is used to define the innovation and its satisfactory implementation (Hall & Hord, 1987). A two-dimensional checklist is constructed to represent the IC. In the study, a number of school policy documents associated with the programme were used to develop an eight-component checklist with three or four variations using the guidelines developed by the CBAM project team (Heck, Stiegelbauer, Hall, & Loucks, 1975). Initial attempts at developing the IC checklist were shown to Professor Shirley Hord, a member of the CBAM project team, to Dr. David Carter a local independent expert familiar with the CBAM method, and to a number of senior teachers at Hillview College. Based on their feedback, wording modifications were made to the checklist. For the PCP, “use” of the innovation meant that teachers facilitated the use of the portable computers by students to match the components listed on the IC checklist. All senior staff were asked to indicate on the IC what they thought should be the boundary between satisfactory and unsatisfactory implementation of the PCP. Based on their responses, an innovation configuration of satisfactory implementation was constructed (Table 3). A conservative approach was taken in combining the responses of the senior staff. For each component, the highest numbered variation permitted by any of the senior staff was allocated as the point of satisfactory implementation. This provided a benchmark against which teacher-class combinations may be compared.
SoC Questionnaire The SoC questionnaire includes a set of scales to prepare a numerical and graphical representation of the type and strengths of participant concerns toward the innovation. The SoC questionnaire used in the study was part of the final teacher survey and was only modified from the CBAM original by replacing the word “innovation” with the words “portable computer programme.” The questionnaire contained 35 items, each with an eight-point response rating scale. Use of the SoC questionnaire was not validated because it was a standard instrument that has been used for many years by many researchers and the study followed the procedures recommended by the authors of the instrument. This was supported by Hord (personal communication, May 1996), “The continuous use of [the SOC questionnaire] across nationalities and cultures seems to suggest that concept and items hold-up (are validated) appropriately to this time.” The entire teaching staff of Hillview was surveyed late in the third year of the study. There were 73 staff surveyed, of whom 51 (70%) responded (Table 4). The analysis and interpretation of the data from the SoC is complex and case dependent. The manual provides some “rule of thumb” and “typical profile” approaches such as those described below.
An analysis of the major stage of concern for teachers is summarised in Table 4 and indicates that about 50% of the staff were represented by the awareness stage. For some, this appeared to be a lack of interest in the PCP either because it did not fit their teaching style (they did not have the time nor inclination) or it was not seen as relevant to their curriculum area. It is likely that some of those in the awareness group were simply “not worried” rather than being “not interested.” Ten people in this group indicated that they used the computers weekly with their classes and, therefore, it could be assumed that their apparent lack of concern was due to a high level of satisfaction with the current implementation of the innovation. From the SoC data a staff profile was graphed (Figure 2) indicating that the concerns of staff in 1995 were relatively introductory. The profile for stages 0 to 4 is a typical nonuse profile but clearly a few teachers had concerns at the collaboration and refocussing stages (5 and 6). It was not possible to determine who all the teachers with high Stage 6 values were because many questionnaires were submitted anonymously. For Stage 0, a mean percentile less than the 40th percentile is considered low, while greater than the 75th percentile is regarded as high. Therefore, the awareness score is relatively high, indicating that many staff were just becoming aware of the PCP innovation even though most had been at the school for the three years. The staff SoC profile in 1999 shows a shift away from the earlier stages toward the personal and management stages. 
Figure 2. CBAM Stage of Concern profile of teachers as mean percentile for whole staff for each stage. Level of Use Interview The LoU uses a focussed interview with an interview guide supplemented by observations to place participants in a hierarchical level. It is important to note that outcomes from the LoU interview should be modified or validated by comparisons with other sources of data, such as observations of the teacher. Ideally, the researcher should closely monitor and observe the teacher over an extended period. This did occur for the teachers involved in the case studies. Unlike the SoC questionnaire, CBAM does not provide a standard LoU interview schedule, because this will depend on the nature of the innovation as represented by the IC. The structured LoU interview used by the study was constructed from the format suggested by CBAM and also incorporated questions based on those asked in teacher interviews conducted in the two previous years of the study. The interview consisted of 25 questions and an interview schedule-recording sheet was produced to allow the researcher to make notes on responses to each question. All interviews were audiotaped with the permission of the interviewee. Twenty-three teachers were interviewed throughout the year using the schedule. Many of the interviews were transcribed from audiotape. CBAM provides a LoU interview rating sheet that is a two-dimensional grid with the eight levels forming the rows and seven categories of LoU forming the columns. They provide for each cell a general description of behaviour likely to indicate that the level should be applied to that dimension. Using the interview transcripts, a level was allocated for each dimension and then an overall level allocated. To assist in this process, a table was constructed to link interview questions with the dimensions. To assign a level to an interviewed teacher, the researcher started by using the “decision point” questions built into the interview and then considered modifying this one level up or down if other data collected concerning that teacher did not fully support the initial level allocated. In particular, if a teacher had been observed with classes a number of times, had completed a questionnaire and included his or her name, or provided more information from other questions in the interview, these data were sufficient to change the allocated level if required. This process of allocating a LoU was applied by the researcher and an independent assessor. Related information for other data sources was then collated to allow the researcher to arrive at a LoU for the teacher. The full analysis process recommended by the CBAM model was used to allocate levels only to those teachers who were used as case studies. A summary of LoU estimates of interviewed teachers is shown in Table 5. Clearly the level of implementation of the PCP, and, therefore, use of the computers, was relatively low. Only nine teachers were found to be implementing the programme to at least a routine level. The most consistent finding was that almost none of the teachers facilitated the use of the computers in class for Year 10 students and that only a few indicated that they facilitated some use with Year Eight students.
It was found to be difficult to classify teachers by a LoU because many varied their actions over the year according to factors such as the classes they taught, the nature of the curriculum, and their own level of enthusiasm. It was easier to classify teacher-class combinations. For example, the influence of the tertiary entrance examination was illustrated by the comments of two teachers of Year 10 classes, “Not sure [computers are the] best way to go. Students may not be able to write for exams” and “TEE [tertiary entrance examination] exam writing is a problem. [Therefore, c]omputers are not a priority for me at the moment.” As a result, it is difficult to consider the LoU as describing a sequential path for teachers. Copyright © 2001, ISTE (International Society for Technology in Education). All rights reserved. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
