Home PublicationsL&LIssuesVolume 26 (1998-1999) May (No. 8)

	Assessing Current Technology Use in the Classroom
	A Key to Efficient Staff Development and Technology Planning
	By Christopher Moersch

How exactly can we quantify how teachers are using technology in the classroom and the general academic achievement that results from their instructional technology (IT) practices? As school systems nationwide plan their purchases of additional hardware, software, and related peripherals as well as their related staff development activities, information about each school’s current IT practices is critical. Recent studies have found strong links among technology, academic achievement, staff development, and classroom instructional practices. Using test scores from the 1996 National Assessment of Educational Practices, for example, Wenglinsky (1998) found that:

• Eighth graders whose teachers used computers mostly for “simulations and applications”—generally associated with higher-order thinking—performed better on NAEP than did students whose teachers did not.
• Eighth graders whose teachers used computers mostly for drill and practice—generally associated with lower-order thinking—performed worse.
• Fourth graders whose teachers used computers mainly for math or learning games scored higher than did students whose teachers did not.
• In both grades, students whose teachers had professional development in computers outperformed students whose teachers did not.

Middleton (1998) found a statistically significant difference between student performance on standardized test scores and how teachers were implementing technology in the classroom. When teachers used higher levels of technology to augment instruction, their students had significantly better scores on the Metropolitan Achievement Test than did students whose teachers used little or no technology in class.

Most research studies that have explored connections between IT and academic achievement have concentrated on specific software delivery approaches such as integrated learning systems (Brush, 1997; Clariana, 1996) or software applications and their effects on learners. However, trying to determine the effectiveness on learners of every conceivable application from Excel to Kid Pix is both impractical and of little benefit to educational technology.

To bridge the gap between technology use and instruction and provide a data-driven approach to staff development and technology planning, the Level of Technology Implementation (LoTi) questionnaire was created. This questionnaire is designed to determine the level of a classroom teacher’s technology implementation by generating a profile for the teacher across three specific domains: LoTi, personal computer use (PCU), and current instructional practices (CIP) (Moersch, 1995).

The PCU profile assesses each classroom teacher’s comfort and proficiency (e.g., troubleshooting simple hardware problems, using multimedia applications) with microcomputers. The CIP profile reveals the teacher’s inclination toward instructional practices that are consistent with a learner-based curriculum design. Table 1 shows three developmental levels of instructional practices and the changes that occur as a teacher moves from a subject-matter approach to a learner-based instructional design.

Table 1. Stages of Instructional Practices

The relationship between a teacher’s LoTi profile and CIP is significant. As a classroom teacher progresses from one level to the next in the LoTi framework, a corresponding series of changes to the instructional curriculum can be observed. The instructional focus shifts from a teacher-centered to a learner-centered orientation, while the use of computers shifts from an emphasis on isolated uses (e.g., drill-and-practice applications) to an expanded view of technology as a process, product, and tool to help students find viable solutions to real-world problems.

The LoTi questionnaire can generate information about each teacher’s CIP, PCU, and LoTi and thus help educators target specific follow-up interventions that address each classroom teacher’s current IT needs. In this way, an overall staff-development program can increase its efficiency and its effectiveness. Research has found a statistically significant correlation among students’ academic achievement, the amount of professional development, and a teacher’s LoTi. As mentioned earlier, students whose teachers were using a higher level of technology in their instruction scored much higher on standardized tests than did students whose teachers used little or no technology in the classroom.

Case Study

Teachers from a school cluster in the Los Angeles Unified School District (LAUSD) recently participated in a technology audit using the LoTi questionnaire. More than 120 respondents assigned scores to the LoTi’s 50 statements using the following scale:

For example, if a statement was not true of the respondent’s classroom IT practices now, then the statement would be scored 1 or 2. If a statement accurately described the respondent’s classroom IT practices, then it would be scored 6 or 7. The questionnaire considered neither the complexity of software applications used at the school site nor the frequency of their use. The information reflected only the perceptions of staff members who took the survey.

Figure 1. Figure 1 shows the LoTi ranking for the 122 teachers from the school cluster who participated in the technology audit. Based on their responses, 49% of the teachers’ highest level of instruction achieved a Level 2 classroom use of technology. This means that technology-based tools supplement existing instructional program as tutorials, educational games, and simulations. The electronic technology is used as either extension activities or enrichment exercises.

Approximately 28% of the teachers’ highest level corresponded with Level 4 classroom technology use. This means that technology-based tools are integrated in a way that enriches students’ understanding of pertinent concepts, themes, and processes. Technology— multimedia, telecommunications, databases, spreadsheets, word processing—is perceived as a tool to identify and solve problems related to an overall theme or concept.

Findings

A summary of the findings from this technology audit revealed the following:

1. Approximately 49% of staff members chose Level 2 as their highest level of technology implementation.
2. The intensity of the staff’s response to the Level 2 implementation of technology was at the lower end of the somewhat true scale.
3. Approximately 30% of staff members recorded their highest level of technology implementation at Level 4A or greater.
4. The intensity of the staff’s response to the Level 4A implementation of technology was at the lower end of the somewhat true scale.
5. The intensity of the staff’s response to the Level 4B, 5, or 6 implementation of technology was at the upper end of the not true scale.
6. Approximately 98% of staff members rated their ability to use basic software applications or troubleshoot routine computer problems as either not true or somewhat true.
7. The remaining 2% of staff members perceived their ability to use basic software applications or troubleshoot routine computer problems as very true.
8. Approximately 39% of staff members perceived their instructional practices as aligning with a learner-based design as not true.
9. Approximately 59% of staff members perceived their instructional practices as aligning with a learner-based design as somewhat true.
10. The remaining 2% of staff members rated their instructional practices as aligning with a learner-based design as very true.
11. Approximately 11% of the staff does not have access to computers for instructional purposes.

Implications for Staff Development

The findings from the LoTi questionnaire for the school cluster in the LAUSD clearly describe a staff that is functioning at the lower end of the LoTi and PCU domains and at the midrange of the CIP domain. What types of professional development and technology purchases would be needed to advance the staff to higher levels of technology use? Based on the cluster’s profile, the following recommendations would be offered for the current school year.

1. Ensure that every classroom teacher has at least one functional computer and printer for instruction. The research showed that the surveyed teachers do not have adequate equipment. Staff members who lack basic computer essentials often become frustrated, resentful, and apathetic about using technology in the classroom.
2. Organize a series of interventions for the lower LoTi-level staff members. The interventions should model specific strategies and techniques for integrating higher-order thinking skills with tool-based applications on the available classroom computers. For example, techniques that would be consistent with this recommendation include modeling the integration of data analysis with classroom experiments, field investigations, simulations, and surveys using any available spreadsheet and graphing program (e.g., LabQuest, Excel, or AppleWorks—formerly ClarisWorks). This approach has four advantages:
   1. It reinforces important complex thinking skills, including problem solving, decision making, scientific inquiry, and inductive and deductive reasoning.
   2. It is easily accessible and generally user-friendly (most computers already have AppleWorks or Excel installed with a built-in tutorial).
   3. It lends itself to students asking lots of questions.
   4. It can be seamlessly integrated into the mathematics, science, social studies, and language arts standards.
3. Provide a series of interventions for lower-level LoTi teachers that model techniques for managing existing classroom computers using the concept of the one-computer classroom. Many teachers place too much emphasis on learning an application than on exploring meaningful and consequential outcomes of the technology’s effects on a learner.
4. Provide a series of interventions for higher-level LoTi teachers that model specific strategies and techniques—such as an experiential-based action model—for integrating a system’s approach to thinking and reasoning skills with the computers that are available.
5. Let these higher-level teachers (approximately 30% of our respondents) design model technology-integration units that can be shared with others throughout the school cluster. This may improve these teachers’ perceptions of their ability to integrate and also move them to a higher level of technology implementation.
6. Make whatever staff development interventions are needed to increase staff members’ confidence in using and troubleshooting personal computers. This might, for example, involve a computer lab instructor designing minicourses to increase each teacher’s proficiency with a computer.
7. Perform needed staff interventions to move a greater percentage of teachers to a learner-based curriculum design. Interventions consistent with this recommendation may involve authentic assessment practices and experiential curriculum design. Currently, 2% of staff members perceived that their instructional practices aligned with a learner-based design.

Conclusion

This article shows how the LoTi questionnaire can provide school systems with a data-driven approach to IT decision making and the subsequent effect of the approach on learners. Beyond its utility as an IT “needs assessment,” the LoTi questionnaire is also an accountability mechanism for school systems so that they can justify their expenditures for IT—computers, local area networks, and professional development—in light of mounting public concern that tax dollars are being encumbered exclusively for technology.

References

Brush, T. A. (1997). The effects on student achievement and attitudes when using integrated learning systems with cooperative groups. Educational Technology Research and Development, 45, 51–64.

Clariana, R. B. (1996). Differential achievement gains for mathematics computation, concepts, and applications with an integrated learning system. Journal of Computers in Mathematics and Science Teaching, 15(3), 203–215.

Middleton, B. (1998). The impact of instructional technology on student academic achievement in reading and mathematics (Doctoral dissertation, South Carolina State University).

Moersch, C. (1995). Levels of technology implementation (LoTi): A framework for measuring classroom technology use. Learning & Leading with Technology, 23(3), 40–42. Available: www.iste.org/L&L.

Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics [Online document]. Princeton, NJ: ETS Policy Information Center. Available: www.ets.org/research/pic/technolog.html.

 More information about LoTi can be found at www.iste.org/L&L.

Chris Moersch (chris@learning-quest.com) is cofounder and director of the National Business Education Alliance in Corvallis, Oregon. He has written numerous articles on technology implementation, curriculum design, and performance-based assessment. Dr. Moersch is currently an adjunct professor at Oregon State University and a past school administrator and classroom teacher from southern California.

Copyright © 1999, ISTE (International Society for Technology in Education). All rights reserved.