Home PublicationsJRTEPast IssuesVolume 33 Number 5

	Edited by Dr. David J. Ayersman, Mary Washington College, and Dr. W. Michael Reed, New York University
formerly Journal of Research on Computing in Education

Volume 33 Number 5 Summer 2001

Attitudes and Achievements
Comparing Computer-Based and Paper-Based Homework Assignments in Mathematics

Chi Kuen Wong
Hong Kong Baptist University

Abstract
This study investigated the effects of three formats of computer-based homework and the paper-based homework, on the achievement, retention, attitudes, and homework time of lower secondary students in Hong Kong. The three computer homework treatments developed specially for this research are computer drill and practice, computer games, and computer-aided discovery. Although some caution is needed in interpreting the results of the statistical analysis used in this study, computer drill-and-practice homework proved to be the most beneficial and took the least amount of time to complete. The primary reason for the apparent success of computer drill-and-practice seems to be the availability of online help and immediate feedback. In addition, students were engaged by this type of homework activity and, therefore, encouraged to complete all of the problems in a particular assignment.

The computer is one of the important inventions of the 20th century. It has come to influence almost every aspect of our daily lives. As a result, Papert (1988) makes the point that there are urgent needs for educators to contemplate effective ways of using computers in the school curriculum. Tolman and Allred (1984) stated:

The computer is influencing many aspects of learning in both the school and the home. An understanding of its role and potential is therefore of the utmost importance to educators as they look to the future. (p. 5)

Because the computer can be used as an interactive source of information, it can make some changes to the paper-and-pencil homework. Homework software with “help” facilities has the additional bonus of allowing the students to overcome difficulties and motivating them in attempting problems. It can mark completed homework in real time and give the students immediate feedback, something impossible in paper-based homework. Moreover, the interactive nature of computerized homework assignments allows for the possibility of group involvement. Such assignments can take the form of “games” so that not only is the homework made more enjoyable, it has the added benefit of being a shared learning experience in which group socialization skills come into play.

If homework assignments are to be completed using computers, it is important that attempts have to be made to answer the following questions:

1. Does computer-based homework increase student achievement and retention of the subject?
2. Can the use of computer-based homework improve students’ attitude toward the subject?
3. Is there a difference in the amount of time spent on computer-based homework as opposed to paper-based homework assignments?

Because a broad base of experience and materials exists for mathematics instructional computing, this study focused on use of computers in mathematics homework. The topics of rate, ratio, and proportion were selected as the subject areas for the study.

The Research Study

The purpose of this study, involving lower-grade students in secondary schools, was to make a comparison of computer-based and paper-based homework. Three computer homework formats were developed and used specially for this research: computer drill and practice, computer game, and computer-aided discovery. The computer drill-and-practice homework provided immediate feedback and online help facilities. The computer game homework presented the homework problems in the form of games. The computer-aided discovery homework, a preparatory homework, allowed students to discover relationships and/or formulas that would be taught in the subsequent lessons. All three of the computer-based formats, as well as the paper-based homework assignments, were studied with regard to student achievement, retention, and overall attitude. The amount of time necessary to complete the assignments was also a factor of consideration and comparison.

Sample

The research sample used in this study consisted of 187 lower secondary students from five classes in each of five coeducational secondary schools (one class per school) in Hong Kong. The students were assigned to the four groups using a stratified random process. The computer drill-and-practice group consisted of 46 students (25 male, 21 female). The computer game group also consisted of 46 students (25 male, 21 female). The computer-aided discovery group consisted of 48 students (23 male, 25 female). The paper-based homework group consisted of 47 students (23 male, 24 female).

Treatment Materials

In preparing for this study, I found there to be a distinct lack of appropriate computer software. Because the availability of such software was so critical, I was forced to undertake its development myself. As a result, I was able to create software specifically designed to fit the local mathematics curriculum and incorporate special modules to collect the data necessary for my research.

Computer Drill-and-Practice Homework

It is commonly believed that the longer a student works on a computer drill-and-practice exercise, the more bored he or she will get. Therefore, the length of computer sessions should be carefully chosen in order to avoid the boredom that prevents students from coming back for more. Alessi and Trollip (1985) suggest that, in the absence of a strong motivator, the appropriate length of a drill session should be approximately 15 minutes. This suggestion was taken into consideration during the design of the software.

The software was designed to provide hints to help students solve problems. Perrenet and Groen (1993) conducted research on the effectiveness of hints for mathematics problems; they found hints that give only warnings against certain mistakes were ineffective. This negative formation might draw students’ attention to incorrect ways of approaching the problem rather than helping them recognise relevant properties of the problem. Based on these finding, the hints used in the study software were formulated positively and definitively.

Computer Game Homework

Three computer games, SHOPPER, MATCHER, and TREASURE were developed to cover the mathematics topics under study. SHOPPER is a shopping game designed for the students to solve problems on rate. The student is required to buy the necessary items for a barbecue. However, because the student does not have enough cash to buy all the required items, the student has to win some cash/discount coupons and/or other items as prizes in two ways:

1. winning some mathematics games in the Game World and
2. giving correct answers to some mathematics problems in the Quiz World.

MATCHER is a card game dealing with ratio. Three cards are displayed on screen initially. Each card contains a brief description such as “Divide $122 into 2 parts in the ratio 3:4” or “The larger part is $40.” Students can discard and draw cards until they find that the description on each of the three cards is related. The computer supplies the cards randomly.

TREASURE is a treasure adventure game primarily designed for students to solve direct proportion problems. Players have to pass through a giant village and a forest before they can get the treasure. Players determine their route as a series of waypoints. At any time they may return to a previous waypoint and try another route.

Computer-Aided Discovery Homework

Five software programs, each accompanied by a worksheet, were developed to allow students to explore the concepts and/or formulas on rate, ratio, scale factor, and direct proportion.

The students are asked to complete a table on the worksheet with the aid of the software programs and then to explore possible relationships or formulas connecting the variables by investigating the data pattern in the table. The computer also helped students check the correctness of the relationships/formulas they discovered.

Paper-Based Homework

Paper-based homework problems used in this study were derived from existing problems appearing as exercises in the students’ textbook. They were the same as those used in the computer drill-and-practice homework. Unlike the computer-based format, however, answers to paper-based homework problems had to be written out on paper.

Measurement Instruments

Mathematics Achievement Test

This test was adapted from Hart, Brown, Kerslake, and Kuchemann’s (1985) ratio and proportion diagnostic test and Onslow’s (1986) rate diagnostic test. According to Ridgway (1987), the CSMS test is well presented and interesting. Emphasis is placed on understanding of mathematical concepts rather than testing individual technical skills. Some of the questions were altered slightly to ensure that students in Hong Kong would be familiar with the context, but in general, both the figures used in the equations and the style of each question remained unchanged. I translated this test into Chinese. The translation was then verified by a translator for correctness and reviewed by two lower-secondary Chinese language teachers for assessing the appropriateness of the language used for students. The final version is a bilingual test with a format similar to the mathematics attainment test set by the Hong Kong Education Department. This attainment test is administered to all Hong Kong secondary school students every year.

A panel of two experienced lower-secondary mathematics teachers established the content validity of the test. The reliability coefficient of the test was found to be 0.8531.

Mathematics Attitude Scale

The Revised Math Attitude Scale was selected as the instrument for measuring attitude towards mathematics. This test was developed by Aiken (Aiken & Dreger, 1961) and consists of 10 positive and 10 negative statements with student responses recorded on a five-point Likert scale (Shaw & Wright, 1967). The authors report a test–retest reliability coefficient of 0.94 (Aiken & Dreger). Shaw and Wright have stated that this scale has a satisfactory reliability and validity.

I translated the scale into Chinese. The translation was then verified by two translators for correctness and reviewed by two language teachers for language appropriateness. To assess the reliability of the scale, the Chinese version was administered to 91 lower-secondary students on two occasions, 18 days apart. The test–retest reliability coefficient was found to be 0.85.

Procedures

All groups took the mathematics achievement and attitude scale pretests prior to the beginning of the study. All the tests were administered by me and one assistant. Students were told there was no right answer to any statement in the attitude scale and questionnaires, that no grades would be given and that honest opinions were required. They were further assured that their answers were for my personal use and that neither teachers nor parents would see the completed tests.

Because not all students had computers at home, all of them were required to stay at school to complete their homework after class. The computer-aided discovery homework was scheduled before the topic was taught. The other three types of homework were scheduled on the same day the topic was taught. My assistant and I were stationed in the computer room where students were completing computer-based homework, while another assistant was stationed in the classroom where students were writing their paper-based homework. To avoid extra teaching, neither the helpers nor I answered any questions on the homework problems except on the use of homework software and computer operations.

In an attempt to control for parental and other influence as well as interaction among treatment groups, the students were not allowed to take the disks and user’s guides home or use them outside the times set by me. Although this introduced an unnatural element, it was judged necessary to gain tighter experimental control.

Software programs recorded the time spent on the computer-based homework, while an assistant recorded the paper-based homework time manually.

At the end of the study, two posttests were administered: one immediate and one delayed. The delayed posttest was administered in the 12th week after the immediate posttest.

Results

Analysis of covariance (ANCOVA) was used to analyse the effects of different computer-based homework formats on student achievement, retention, and attitudes. The results are summarised in Table 1.

Table 1. Comparison of Student Achievement, Retention, Attitude and Homework Time of the Four Groups
			Adjusted Mean
Homework Group	N		Math Posttest		Math Delayed Posttest		Attitude Posttest		Homework Mean Time (minutes)
Computer drill and practice	46		59.20		61.57		47.39		50.35
Computer game	46		54.83		57.47		48.32		92.56
Computer-aided discovery	48		51.40		53.70		47.65		86.27
Paper-based	47		52.21		54.05		46.01		46.49

Mathematics Achievement

The immediate achievement was measured by the mathematics achievement posttest. The results of the ANCOVA with mathematics achievement pretest score as a covariate show a clear significant difference, F(2,183) = 14.83, p < .001 among the adjusted means of the four groups. With the further application of Bryant-Paulson post-hoc test, it is concluded that there were significant differences between the computer drill-and-practice group and the computer-aided discovery group and between the computer drill-and-practice group and the paper-based group.

Retention

Retention was measured by the delayed posttest, which was administered in the 12th week after the immediate posttest. A covariate analysis with mathematics achievement pretest score as a covariate was conducted to determine if there was a significant difference in retention. The result shows a significant difference in retention among the four groups, F(2,183) = 13.23, p < .001.

The Bryant-Paulson test was applied again to determine the significant pairwise difference. It found significant differences between the computer drill-and-practice group and the computer-aided discovery group, and between the computer drill-and-practice group and the paper-based group.

Attitude toward Mathematics

The results of the analysis of covariance on the attitude posttest scores show no significant difference in attitude toward mathematics among the four groups when pretreatment attitude was a covariate.

Homework Time

Analysis of variance was used to determine whether there was a significant difference in homework time among the four groups. The results indicate a significant difference between one or more of the groups, F(2,183) = 60.55, p < .001. Hence, the Scheffe test was applied. This test showed a significant difference was found in the following pairs of groups:

• computer drill and practice and computer game
• computer drill and practice and computer-aided discovery
• paper-based and computer game
• paper-based and computer-aided discovery

Discussion

Mathematics Achievement

The statistical analysis indicates that the students receiving computer drill-and-practice homework achieved a better result on the posttest than those students receiving computer-aided discovery homework or paper-based homework. One possible explanation could be the availability of immediate feedback and the online help facility in solving mathematics homework problems. Students involved with the computer drill-and-practice homework were able to immediately determine whether or not their answers were correct. This allowed them to reattempt the problem if the answer was incorrect and ultimately increased the number of correctly answered problems. However, neither immediate feedback nor online help facility was provided in paper-based homework. This also explained why the frequency of correctly answered homework problems in the computer drill-and-practice group was 32% higher than that in paper-based homework group.

Although the computer game homework also provided immediate feedback, it did not provide online help. Because approximately half of the self-test problems in the computer-aided discovery homework were multiple choice questions, students could select another choice without thinking carefully after knowing that their previous choice was incorrect. Online help, therefore, was essentially irrelevant.

The results of the questionnaire analysis indicate that 67% of the students had neither a private tutor nor a family member to help them with their mathematics. The online help facility could give them a hand when they did not know how to tackle a mathematics problem, and this may have led to an increase in the number of homework problems being attempted. Moreover, the availability of online help may have been a significant factor in reducing student anxiety toward homework. The computer-aided discovery group scored the lowest adjusted mean in the posttest. One possible interpretation could be that this group did their discovery homework before the topics were taught and that no homework was given to reinforce what they had learnt in the lessons. Another possible explanation could be related to the achievement-test items. It was a conventional test, and most of the test items required students to give numerical answers or select the right statements. However, the focus of the computer-aided discovery homework used in this study was to discover relationships and/or formulas by analysing data. In some questions, students were required to describe their discovery in words. Therefore, this group of students may have been at a disadvantage when attempting a conventional mathematics test. The test items would need to have a discovery orientation for the computer-aided discovery group to score higher than the other treatment groups. Nevertheless, designing an achievement test in this manner would skew the results in favour of the computer-aided discovery group because the other groups would not have access to discovery-oriented homework.

Because a number of researchers found educational games to be effective for learning mathematics (Marty, 1985; Pulos & Sneider, 1994; Randel, Morris, Wetzel, & Witehill, 1992), it was anticipated that the computer game group would perform significantly better than the other groups in respect to the mathematics achievement test score. Study findings did not support this contention, however. Though the computer game group scored the second highest mean in the mathematics posttest, its mean was not significantly different from those of the computer-aided discovery group and the paper-based homework group. The major reason seems to be that students in the computer game group required extra effort to master the user interface of each game. Effort spent on understanding the game itself diverted attention away from solving its mathematics problems. Furthermore, specific tactics were needed to win the game. For example, determining the card to be discarded in the MATCHER game and choosing a prize and a coupon in the SHOPPER game both required some tactics to increase the chance of winning. But, the achievement test in this study did not test any new abilities or insights that the students in the computer game group may have picked up.

Because the drill-and-practice homework required students to attempt all the problems in an exercise before proceeding to the next, it was found that the number of un-attempted questions dropped (approximately 72%) substantially from the mathematics pretest to posttest in comparison with the other groups. (All three of the other groups allowed students to skip problems.) Although it may be difficult for a teacher to monitor students to ensure they attempt all the problems, the computer is an appropriate machine to do this task. In addition, the online help facility can encourage students to attempt problems that seem difficult at first glance.

Retention

The adjusted mean of the delayed posttest of the computer drill-and-practice group was found to be significantly higher than those of the computer-aided discovery group and paper-based homework groups. This finding is consistent with the results of the posttest. Because the same test was used for both posttest and delayed posttest, the previous discussion of the effects of test items on different homework groups may also be applied. Another reason seems to be that the homework treatment had prolonged effects on students at least until the administration of the delayed posttest, approximately three months after the posttest.

It was found that the adjusted mean of the delayed posttest of each treatment group was slightly higher than that of the posttest. This may be because the topic of indirect proportion was taught after the posttest and that learning about that topic helps students have a deeper understanding on the topics of rate, ratio, and proportion (e.g., the inverse relationship in rate). Furthermore, students received the formal assessment in the form of a test or examination conducted by their schools during the 12-week period between posttest and delayed posttest. This may have refreshed students on the topics under study during their revision and, thus, resulted in better retention.

Some critics of drill-and-practice software argue that its benefits are all short-term and that other approaches are needed for long-term retention. No evidence was found to support this assertion over the 12-week period that separated the two posttests given in this study.

Attitude toward Mathematics

No significant difference in student attitude among the four treatment groups was found when the mathematics attitude pretest score was used as the covariate. This finding is consistent with the results reported in other studies (Marty, 1985; Niederhauser, 1994). It indicates the likelihood that a study covering approximately three weeks is insufficient to significantly influence attitudes toward mathematics that may have taken students years to establish.

Another possibility may be that the false null hypothesis was retained (Type II error). If such an error occurs, it may have been that the attitude scale used in this study was not sensitive enough to determine the subtle changes that may have occurred during the study.

Table 2 shows that the means of the mathematics attitude scale in the posttest of all the treatment groups, except the computer game group, are slightly lower than their corresponding means in the pretest. Because most of the students in the research sample had no computer at home, students of all the treatment groups were required to stay in school after regular class time to do their homework. This arrangement may have caused students who resented the imposition on their free time to have negative attitudes toward the subject matter and homework.

Table 2. Comparison of Mean of Mathematics Attitude between Pretest and Posttest
Group	Pretest		Posttest		Gain
Computer drill and practice	48.22		47.63		–0.59
Computer game	48.72		49.02		0.30
Computer-aided discovery	47.19		46.96		–0.23
Paper-based	47.68		45.77		–1.91

Although results show the mean of the mathematics attitude scale in posttest of the computer game group to be slightly higher than the mean in pretest, this study cannot determine whether this difference was due to the entertainment of the game, the usefulness of immediate feedback on answers, or some other factors.

Homework Time

The mean (92.56 minimum) of the homework time of the computer game group was the highest. This finding is consistent with Gordon’s (1972) claim that games require more time than learning most conventional materials. The major reason seems to be that all the homework games were written in English. Very often, students had to refer to the user’s guide for the corresponding Chinese translation of some narratives, messages, or vocabulary displayed on screen. Another reason is that students may have had to spend extra time mastering the different games. (It is believed that students will need less time to master future games because of the hands-on experience gained during this study.) Learning time may be further reduced by using a graphical user interface (GUI) instead of the command menu-type interface used in the games in this study.

The computer-aided discovery group got the second highest mean (86.27 minimum) of the homework time. This result corresponds to claims made by Bittinger (1968) and Cronbach (1966). In some worksheets, students were required to describe their discoveries in words (the description could be written in Chinese). I observed that many students had difficulty writing their discoveries. Some students wanted to leave the question blank even though they had discovered the relationship correctly. In addition to the discovery work, students also had to complete self-tests. This explains why this particular homework group, on average, took a longer time than the computer drill-and-practice group and the paper-based homework group.

The average homework time (50.35 minimum) of the computer drill-and-practice group was very close to that (46.49 minimum) of the paper-based homework group. This may, in part, be because both groups had an identical number of homework problems. The availability of immediate feedback and online help seem to be the main reasons that the computer drill-and-practice group spent slightly more time on their assignment. These aids were not available to students doing paper-based homework. Not only did computer drill-and-practice students have to learn the software before solving any problems, the presence of immediate feedback encouraged students to continue to search for the correct answer.

In fact, it seems unreasonable that the average homework time of the computer drill-and-practice group is just 3.86 minutes higher than that of the paper-based group. Students in the computer drill-and-practice group should have taken much more extra time to access online help, reattempt problems, and become familiar with the user interface. One possible explanation may be that in addition to solving the problems, almost all students receiving paper-based homework wrote their arithmetic expressions on the homework sheet, though this was not formally required. Students in the computer drill-and-practice group were excused from having to “show their work” because the computer program only accepted numerical answers. Because of this, time spent solving problems was raised for the paper-based homework group and reduced for computer users.

Conclusions and Recommendations

The results of this study demonstrate the viability of the three formats of computer-based homework. Students receiving computer drill-and-practice homework performed significantly better in achievement and retention than did the students completing paper-based homework. Also, students in the computer game group and computer-aided discovery group performed as well as those in the paper-based group. Based on these findings, the future of computer-based homework assignments looks promising. However, the availability of appropriate software may be a significant factor in determining the success of implementing computer-based homework. Most of the educational software available in Hong Kong is imported. It is neither tailor-made for local context nor ideally suited for use in homework assignments. Because much of the commercially available software is currently written in English, there is a language barrier that must also be overcome. Therefore, it is hoped that textbook authors and publishers begin to develop software and implementation guides for use with their written material. Because the development of educational software needs expertise from both the education and software engineering fields, education departments and computer science departments in universities should collaborate with one another.

The implementation of this type of homework also involves managerial and staff training issues. Administrators will have to allocate sufficient funding for equipment and teachers will have to be adequately trained. Most importantly, schools may be required to make their computer labs available after regular hours to accommodate students who do not have access to computers at home. All these issues have resource implications and require the support from the Education Department.

Acknowledgement

The author would like to express his sincere thanks to Dr. Richard Phillips of University of Nottingham for his help and advice in this research.

Contributor

Chi Kuen Wong is an assistant professor of computer science at Hong Kong Baptist University. His research interests are information technology in education and intelligent agents.

Contact

Dr. C. K. Wong
Department of Computer Science
Hong Kong Baptist University
Kowloon Tong, Hong Kong
kckwong@comp.hkbu.edu.hk

References

Aiken, L. R., Jr., & Dreger, R. M. (1961). The effect of attitudes on performance in mathematics. Journal of Educational Psychology, 52(1), 19–24.

Alessi, S. M., & Trollip, S. R. (1985). Computer-based instruction: Methods and development. Englewood Cliffs, NJ: Prentice Hall.

Bittinger, M. L. (1968). A review of discovery. The Mathematics Teacher, 61(2), 140–146.

Cronbach, L. J. (1966). The logic of experiments on discovery. In L. S. Shulman & E. H. Keislar (Eds.), Learning by discovery: A critical approach (pp. 76–92). Chicago: Rand McNally & Co.

Gordon, A. K. (1972). Games for growth. Chicago: Science Research Associate, Inc.

Hart, K., Brown, M., Kerslake, D., & Kuchemann, D. (1985). Chelsea Diagnostic Mathematics Tests, Teacher’s Guide. Windsor, Berkshire, United Kingdom: NFER-Nelson.

Marty, J. F. (1985). Selected effects of a computer game on achievement, attitude, and graphing ability in secondary school algebra (Doctoral dissertation, Oregon State University, 1985). Dissertation Abstract International, 47(1), 113A.

Niederhauser, D. S. (1994). The role of computer-assisted instruction in supporting fifth-grade mathematics instructions: Cognitive and attitudinal outcomes (Doctoral dissertation, University of Utah, 1994). Dissertation Abstracts International, 56(1), 168A.

Onslow, B. A. (1986). Overcoming conceptual obstacles concerning rates: Design and implementation of a diagnostic teaching unit. Unpublished doctoral dissertation, University of Nottingham, United Kingdom.

Papert, S. (1988). Computers and computer cultures. In A. Jones & P. Scrimshaw (Eds.), Computers in education (pp. 5–13). Milton Keynes, United Kingdom: Open University Press.

Perrenet, J., & Groen, W. (1993). A hint is not always a help. Educational Studies in Mathematics, 25(4), 307–329.

Pulos, S., & Sneider, C. (1994). Designing and evaluating effective games for teaching science and mathematics: An illustration from coordinate geometry. Focus on Learning Problems in Mathematics, 16(3), 23–42.

Randel, J. M., Morris, B. A., Wetzel, C. D., & Witehill, B. V. (1992). The effectiveness of games for educational purposes: A review of recent research. Simulation and Gaming, 23(3), 261–276.

Ridgway, J. (1987). A review of mathematics tests. Windsor, Berkshire, United Kingdom: NFER-NELSON.

Shaw, M. E., & Wright, J. M. (1967). Scales for the measurement of attitudes. New York: McGraw-Hill.

Tolman, M. N., & Allred, R. A. (1984). The computer and education: What research says to the teachers. Washington, DC: National Education Association Professional Library.

Copyright © 2001, ISTE (International Society for Technology in Education). All rights reserved.