 |
Edited by Dr. David J. Ayersman, Mary Washington College, and Dr. W. Michael Reed, New York University
|
| formerly Journal of Research on Computing in Education |
Volume 33 Number 5 Summer 2001
Teacher Interaction
Motivating At-Risk Students in Web-Based High School Courses, Part III
Stephen Lehman, Douglas F. Kauffman, Mary Jane White, Christy A. Horn, Roger H. Bruning
University of Nebraska–Lincoln
Results
Because our small sample size (n = 4 per group) limited the power of our analyses and precluded fully meaningful significance testing, we report effect sizes using eta squared ( ) for all variables, using Cohen’s (1977) guidelines for evaluating effect sizes ( = .01 as small, = .06 as moderate, and = .15 as large effect sizes). Cohen has argued, along with others (e.g., Cook & Campbell, 1979; Lipsey, 1990) that in cases of low statistical power such as this, interventions may lack statistical significance but have substantial practical significance. For this reason, we computed effect sizes to gain an estimate of the treatment effect that was less influenced by small sample size than traditional significance testing. h2 estimates both the linear and nonlinear variation in a relationship. (See Kirk, 1982 for further details and a computational formula.) Though reporting only effect sizes may be less than ideal, the results reported represent a multiweek intervention and a large number of observations. They also are highly consistent with qualitative observations made by lab monitors and teachers. The high degree of consistency between the qualitative and quantitative results strengthen our belief that effect sizes accurately reflect the results of our experimental manipulation.
E-mail Effort Ratings
The pattern of means for e-mail effort ratings (Table 2) suggests that students from the conditions enhanced with motivation-building statements and personal-investment/caring statements displayed more effort than students in the baseline professional condition. Although e-mails from the control condition were rated as displaying less effort than those in the enhanced conditions, these differences were not significant F(1, 15) < 1 for motivation building and personal investment, F(1, 15) = 2.48 for motivation and personal-investment interaction.
|
Table 2. Means and Standard Deviations of Dependent Variables by Experimental Condition
|
 |
|
Condition
|
Low Motivation
|
High Motivation
|
|
|
|
| |
Low Personal Investment
|
High Personal Investment
|
Low Personal Investment
|
High Personal Investment
|
|
|
Effort
|
|
|
|
|
|
|
|
|
|
M
|
1.52
|
|
2.07
|
|
2.23
|
|
1.88
|
|
|
SD
|
(.47)
|
|
(.47)
|
|
(.52)
|
|
(.79)
|
|
|
Engagement
|
|
|
|
|
|
|
|
|
|
M
|
2.38
|
|
4.38
|
|
3.50
|
|
3.44
|
|
|
SD
|
(1.35)
|
|
(.43)
|
|
(1.02)
|
|
(.66)
|
|
|
Hours
|
|
|
|
|
|
|
|
|
|
M
|
43.56
|
|
47.81
|
|
45.5
|
|
50.94
|
|
|
SD
|
(19.06)
|
|
(6.42)
|
|
(5.01)
|
|
(13.74)
|
|
|
Grade
|
|
|
|
|
|
|
|
|
|
M
|
52.25
|
|
83.00
|
|
90.75
|
|
90.25
|
|
|
SD
|
(44.31)
|
|
(5.32)
|
|
(9.34)
|
|
(5.44)
|
|
|
This lack of statistical significance is not surprising given the small sample size and accompanying low power (Cook & Campbell, 1979). Analysis of the effect sizes showed that motivation building had a moderate effect (= .06), personal investment/caring had a small effect (= .01), and the motivation-building and personal-investment interaction had a large effect (= .17).
Lab Monitor Engagement Rating
As with the effort on e-mails variable, lab monitor engagement ratings were lower in the baseline professional condition than in the enhanced conditions, but the differences were not significant. Both personal investment and the motivation and personal-investment interaction, however, had large effect sizes (= .26 and = .29 respectively). Effect size for motivation was negligible (< .01).
Time Spent in the Course
Analyzing the number of hours worked by condition once again revealed a pattern of means similar to those found in the engagement measures, with students in the baseline professional condition working fewer hours than those in the enhanced conditions. Those differences, however, were not significant and effect sizes for the hours worked were generally small (= .01, .05, and < .01 for motivation building, personal investment and the motivation-building and personal-investment interaction, respectively).
Final Grades
Students in the baseline professional condition again scored lower on final grade than those in the enhanced conditions. Although the differences were again not significant, effect sizes were all large (= .13 for motivation building, = .25 for personal investment, and = .13 for the personal-investment and motivation-building interaction). In interpreting the pattern of means and effect sizes, the results point to the conclusion that adding personal investment to e-mails already possessing motivating interactions does little to improve students’ final grade over and above the improvements generated by the motivating enhancements. These findings should be interpreted cautiously, however, in that one student in the baseline professional condition failed to complete any assignments (final grade = 0) and, therefore, constituted an extreme outlier. If this student’s grade is eliminated from the analysis, the mean score of the baseline professional group increases (M = 69.67), but the group mean remains highly variable (SD = 33.54) and substantially below the enhanced conditions.
Ability Measures
The consistency of the findings across measures and the effect sizes for the interventions strongly point to the possibility that enhancing the teacher–student interaction will improve at-risk students’ chances of success in Web-based courses. In reflecting on the lab monitors’ observations, however, we noted a pattern that suggested that student engagement is likely influenced by individual differences in students, particularly in level of writing skill. To investigate this possibility, we analyzed the writing quality of two assignments submitted early in the experiment, before substantive interaction had taken place between the teacher and the student. We reasoned that at this point most of the variation in writing quality would be due to individual differences rather than any experimental intervention. Two independent raters, blind to condition of the students, rated the assignments using the Oregon Analytic Model (Wolfe, 1993), which incorporates such factors as ideas, organization, word choice, and sentence fluency into the rating of writing quality. Mean quality ratings were correlated with other outcome measures. As can be seen in Table 3, a significant relationship existed between writing ability and student effort on e-mails (r = .47, p = .03), and the relationship between writing ability and lab monitors’ rating of student engagement was marginally reliable (r = .39, p = .07). These relationships suggest that interventions enhancing teacher–student interaction are subject to the influence of individual differences.
|
Table 3. Interrelations among Engagement Measures and Quality of Writing
|
|
|
Outcome Measures
|
Hours Worked
|
Writing Quality
|
Course Grade
|
Lab Monitor Rating
|
|
|
Effort on e-mails
|
.02
|
|
.47
|
* |
.19
|
|
.32
|
|
|
Hours worked
|
|
|
–.06
|
|
.64
|
** |
.49
|
* |
|
Writing quality
|
|
|
|
|
.17
|
|
.39
|
|
|
Course grade
|
|
|
|
|
|
|
.77
|
** |
|
|
* Correlation is significant at the .05 level (1-tailed). ** Correlation is significant at the .01 level (1-tailed).
|
Discussion
Our findings indicate that enhancing teacher–student interactions holds promise as an intervention aimed at increasing at-risk student success rates in Web-based courses. Our optimism is based on a recurring pattern of means, moderate to large effect sizes, and firsthand observation of students as they engaged in a beginning composition course. At the same time, the study provides evidence for considerable individual variability in course engagement, particularly as it relates to level of writing skill. Although communication from teacher to student is potentially important in shaping student success, much still remains in the hands of the learners. Nonetheless, the evidence suggests that enhancing teacher interaction may be a useful intervention to increase at-risk students’ engagement in Web-based instruction.
Our studies suggest that both personal-investment and motivation-building interactions increased student engagement in a setting facilitated by lab monitors (see also Bruning, Landis, Hoffman, & Grosskopf, 1993). In a “pure” distance education setting where students do not have such structural support, it seems probable that the effects of enhancing teacher–student interactions may be even more important to at-risk student success. Lab monitors’ observations of student behavior support this hypothesis. Lab monitors noted several instances where they observed higher levels of on-task behavior with course materials and evidence of more positive attitudes in connections when the teacher provided motivating or personally invested communication to the student. One explanation for this is that at-risk students’ relationships with the teacher may have caused the students to place more value on the educational tasks they associated with their instructors. Another explanation may be that the students felt a greater sense of responsibility to complete the course when they perceived that the course involved greater human accountability.
In regard to making a choice between implementing either personal-investment or motivation-building interventions with at-risk students, our results suggest that motivation building may be more effective in enhancing engagement. We observed that motivation building often served double duty as personal investment because students commonly construed motivation-building communication as evidence of the teacher’s caring for them. Both motivation building and personal investment were important to at-risk student engagement, but motivation building appeared to be more robust. Motivation-building interactions may also be more effective because of their tendency to contain more content-oriented feedback than personal-investment interactions. This interpretation would be consistent with earlier research that has shown that content-related feedback maintains intrinsic motivation, whereas lack of content-related feedback can decrease intrinsic motivation and self-regulation (Bandura, 1997; Butler & Nisan, 1986; Straub, 1997). We also found no advantage to combining motivation-building and personal-investment enhancements. Rather, the effect sizes for the motivation-building and personal-investment interaction term indicate that increasing the level of personal investment to messages with motivation-building content may have the potential to detract from the effectiveness of a motivation-oriented intervention. Higher levels of content-related feedback in the motivational statements could provide one explanation for the advantage of motivation-building statements, although there is no apparent reason that combining personal investment with motivation building should result in decreases in engagement.
Our study provides preliminary evidence that the nature of teacher communication is important to the success of at-risk students in Web-based courses. The study of teacher–student interaction in Web-based environments is a promising area for future research, we feel, and may be especially important in light of the current lack of research on the effects of teacher–student interaction on student achievement. Future studies should not only explore the effects of teacher interaction enhancements with larger sample sizes but also examine whether different categories of risk factors (e.g., school-related, family-related, and individual factors) differentially affect student response to enhancements of the quality of teacher–student interaction. In our experiment, we observed a high level of variation in levels of ability, knowledge, and motivation, no doubt reflecting the diverse set of risk factors that produce different learner profiles. Greater understanding of the effects of particular risk factors as they relate to student success would enable course designers to build in features that could specifically address the particular needs of at-risk students.
For the at-risk students in our study, at least, the promise of technology seemed bright, with many of them expressing strong interest in taking more classes on the Web. At the same time, however, technology—even with all of its multimedia capabilities—was clearly insufficient as a motivator for most of them. Without motivational support, at-risk student engagement in Web-based courses dropped precipitously. Unless teachers work systematically at motivating and engaging their students in Web-based courses, many at-risk students will remain at risk.
In summary, our observations of at-risk students in Web-based courses suggest that maintaining student engagement in this environment is likely to be challenging for online educators, but that motivation-building communications provide at least one important key. With such enhancements, Web-based instruction may provide a means to reach many students who experience difficulty in traditional educational systems. Because of the broad diversity of student characteristics, however, the effectiveness of interventions may depend on the extent to which the intervention addresses the student’s particular needs.
Acknowledgment
This research was supported by CLASS, a Star Schools project of the U.S. Department of Education. The authors thank Gregg Schraw, Marcy Reisetter, and Trish Lehman for their helpful comments on earlier versions of this article.
Contributors
Stephen Lehman is a postdoctoral fellow at the Center for Instructional Innovation, University of Nebraska–Lincoln.
Douglas Kauffman and Mary Jane White are research associates at the Center for Instructional Innovation, University of Nebraska–Lincoln.
Christy Horn and Roger Bruning codirect the Center for Instructional Innovation, University of Nebraska–Lincoln.
Contact
Roger H. Bruning
Center for Instructional Innovation
209 Teacher’s College Hall
University of Nebraska–Lincoln
Lincoln, NE 68588-0384
rbruning@unl.edu
References
Atkinson, H. (1964). An introduction to motivation. Princeton, NJ: Van Nostrand.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman.
Battistich, V., Solomon, D., Watson, M., & Schaps, E. (1997). Caring school communities. Educational Psychologist, 32, 137–151.
Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497–529.
Birch, S. H., & Ladd, G. W. (1996). Interpersonal relationships in the school environment and children’s early school adjustment: The role of teacher and peers. In J. Juvonen & K. R. Wentzel (Eds.), Social motivation: Understanding children’s school adjustment (pp. 199–225). New York: Cambridge.
Bosworth, K. (1995). Caring for others and being cared for: Students talk caring in school. Phi Delta Kappan, 76, 686–693.
Brooks, R. B. (1994). Children at risk: Fostering resilience and hope. American Journal of Orthopsychiatry, 64, 545–553.
Bruning, R., Landis, M., Hoffman, E., & Grosskopf, K. (1993). Perspectives on an interactive satellite-based Japanese language course. K–12 Distance Education, 7, 22–38.
Butler, R., & Nisan, M. (1986). Effects of no feedback, task-related comments, and grades on intrinsic motivation and performance. Journal of Educational Psychology, 78, 210–216.
Carroll, A., Durkin, K, Hattie, J., & Houghton, S. (1997). Goal setting among adolescents: A comparison of delinquent, at-risk, and not-at-risk youth. Journal of Educational Psychology, 89, 441–450.
Cohen, J. (1977). Statistical power analysis for the behavioral sciences (rev. ed.). New York: Academic Press.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimental design and analysis: Issues for field settings. Dallas, TX: Houghton Mifflin.
Gladieux, L. E., & Swail, W. S. (1999). The virtual university and educational opportunity: Issues of equity and access for the next generation [Online document]. Available: http://collegeboard.org/policy/html/April.pdf.
Goodenow, C. (1993). Classroom belonging among early adolescent students: Relationships to motivation and achievement. Journal of Early Adolescence, 13, 21–43.
Kirk, R. E. (1982). Experimental design: Procedures for the behavioral sciences (2nd ed.). Belmont, CA: Brooks, Brooks and Cole.
Kramer-Schlosser, L. (1992). Teacher distance and student disengagement: School lives on the margin. Journal of Teacher Education, 43, 128–140.
Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage Publications.
McWhirter, J. J., McWhirter, B. T., McWhirter, A. M., & McWhirter, E. H. (1993). At risk youth: A comprehensive response. Pacific Grove, CA: Brooks/Cole.
Means, B. (1997). Critical issue: Using technology to enhance engaged learning for at-risk students. [Online document]. Naperville, IL: North Central Regional Educational Laboratory. Available: www.ncrel.org/sdrs/areas/issues/students/atrisk/at400.htm.
Midgeley, C., Feldlaufer, H., & Eccles, J. S. (1989). Student/teacher relations and attitudes toward mathematics before and after the transition to junior high school. Child Development, 60, 981–992.
Moje, E. B. (1996). “I teach students, not subjects”: Teacher-student relationships as contexts for secondary literacy. Reading Research Quarterly, 31, 172–195.
National Telecommunications and Information Administration. (1999). Falling through the Net: Defining the digital divide [Online document]. Washington, DC: Author. Available: www.ntia.doc.gov/ntiahome/fttn99/contents.html.
Noblit, G.W. (1993). Power and caring. American Educational Research Journal, 30, 23–38.
Noddings, N. (1992). The challenge to care in schools: An alternative approach to education. New York: Teachers College Press.
Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66, 543–578.
Rak, C. F., & Patterson, L. E. (1996). Promoting resilience in at-risk children. Journal of Counseling and Development, 74, 368–373.
Schunk, D. H., & Swartz, C. W. (1993). Goals and progress feedback: Effects on self-efficacy and writing achievement. Contemporary Educational Psychology, 18, 337–354.
Snyder, C. R. (1995). Conceptualizing, measuring, and nurturing hope. Journal of Counseling and Development, 73, 355–360.
Snyder, C. R., Harris, C., Anderson, J. R., Holleran, S. A., Irving, L. M., Sigmon, S. T., Yoshinobu, L., Gibb, J., Langelle, C., & Harney, P. (1991). The will and the ways: Development and validation of an individual-differences measure of hope. Journal of Personality and Social Psychology, 60, 570–585.
Straub, R. (1997). Students’ reactions to teacher comments: An exploratory study. Research in Teaching English, 31, 91–119.
Weiner, B. (1986). An attributional theory of motivation and emotion. New York: Sperling-Verlag.
Wentzel, K. R. (1996). Social goals and social relationships as motivators of school adjustment. In J. Juvonen & K. R. Wentzel (Eds.), Social motivation: Understanding children’s school adjustment (pp. 226–247). New York: Cambridge.
Wentzel, K. R. (1997). Student motivation in middle school: The role of perceived pedagogical caring. Journal of Educational Psychology, 89, 411–419.
Wentzel, K. R. (1998). Social relationships and motivation in middle school: The role of parents, teachers and peers. Journal of Educational Psychology, 90, 202–209.
Wentzel, K. R., & Asher, S. R. (1995). The academic lives of neglected, rejected, popular, and controversial children. Child Development, 66, 754–763.
Werner, E. E. (1984). Resilient children. Young Children, 39, 68–72.
Wolfe, B. (1993). Oregon statewide writing assessment: 1991 and 1992 with student writings Grades 8 and 11. Salem: Oregon Department of Education.
Back to Page 1
Back to Page 2
Copyright © 2001, ISTE (International Society for Technology in Education). All rights reserved.
|
