William Gibbs is an associate professor and the
head of the
Department of Media at Eastern Illinois University.
He received
his PhD in instructional systems from Pennsylvania
State University.
His research interests include knowledge
acquisition, technology-based
learning environments, and methods for effective
instructional
software evaluation.
Pat R. Graves is a professor in the School of
Business at
Eastern Illinois University. She teaches business
communications
and computer-related courses. Her research interests
include
the visual display of information and
computer-mediated communication.
Dr. Graves is co-editor of the Business Education
Index
published by Delta Pi Epsilon, a graduate research
association
for business teacher educators.
Ronan S. Bernas is an assistant professor at the
Department
of Psychology of Eastern Illinois University. He
received
his PhD in psychology (Committee on Human
Development) from
the University of Chicago in 1995. His research is
on argumentative
and explanatory discourse. He examines the learning
and conceptual
changes that occur during argumentative and
explanatory discourse.
|
Revised Criteria with Overall Means
3.5–5.0
Arranged by Category
|
|
Mean
|
Revised Criterion
|
|
Instruction planning: Planning of
instruction relating
to courseware use.
|
|
4.64
|
|
(a) Is a definition of the target audience
given in
the courseware?
|
| |
|
(b) Is a definition of prerequisite skills
given in
the courseware?
|
|
4.26
|
|
Is the content and vocabulary level for
intended users
appropriate?
|
|
4.21
|
|
(a) Does the courseware match the curriculum?
|
| |
|
(b) Is it possible to customize courseware
use to make
it match the curriculum?
|
|
3.51
|
|
How much computer experience would a learner
need to
operate the courseware?
|
|
Support issues: How much support is
provided when
you purchase the courseware and for
maintenance during
use.
|
|
4.59
|
|
Are the computer hardware and software
requirements
for the courseware specified?
|
|
4.36
|
|
Are instructions for software installation
and operation
explicit, accurate, and complete?
|
|
4.13
|
|
If the courseware is placed in a computer
laboratory,
will learners be able to operate the software
without
intervention or assistance from a lab manager?
|
|
Documentation: The documents, reference
materials,
and other help systems accompanying the
courseware.
|
|
3.85
|
|
(a) Is a user/technical guide or online
documentation
provided with the courseware?
|
| |
|
(b) If online documentation is provided, is
it intelligent?
|
|
3.68
|
|
Does the user/technical guide or online
documentation
provide sufficient detail and complete indexes
of the
information available in the courseware?
|
|
Program adequacy: The extent to which the
courseware
performs as intended.
|
|
4.45
|
|
(a) Does the program run as intended on the
hardware
the learners will use?
|
| |
|
(b) Does the software run seamlessly under
different
operating systems?
|
|
4.38
|
|
Can the courseware install itself?
|
|
Classroom management issues: The impact
the courseware
has on how classroom activities are
managed.
|
|
4.06
|
|
Is the courseware easy to implement if it is
used in
classroom teaching?
|
|
3.90
|
|
Can learner data be collected efficiently?
|
|
3.87
|
|
Is the courseware interesting for learning
purposes?
|
|
3.75
|
|
Are learners’ answers and data recorded
for teacher
record keeping?
|
|
3.64
|
|
(a) Does the courseware allow teachers and
learners
to present the results of their work?
|
| |
|
(b) Does the courseware provide learners
knowledge
of results?
|
|
Instructional adequacy: The extent to
which the
courseware relates to and supports class
instruction.
|
|
4.72
|
|
Are practice activities provided in the
courseware
to actively involve the learner?
|
|
4.60
|
|
Are instructional objectives clearly stated?
|
|
4.55
|
|
Is the purpose of the courseware and the
instructional
activities, tasks, exercises, etc. needed to
complete
the learning task made explicit?
|
|
4.51
|
|
Does the courseware provide adequate support
to help
learners accomplish the learning objectives?
|
|
4.39
|
|
Are varying levels of difficulty available to
provide
learners with challenges according to their
skill level?
|
|
4.21
|
|
Are there means available to reconcile
learners’
present knowledge to information presented in
the courseware?
|
|
Information content: The extent to which
information
is current, correct, and appropriate.
|
|
4.91
|
|
Does the courseware provide information that
accurately
represents the topic?
|
|
4.56
|
|
Are examples, practice exercises, and
feedback meaningful
and relevant?
|
|
4.53
|
|
Is the information current?
|
|
4.38
|
|
Does the content fit the stated scope and
purpose?
|
|
3.79
|
|
(a) Is the underlying structure of the
courseware logical?
|
| |
|
(b) Are learners free to study the content in
a sequence
logical to them?
|
| |
|
(c) Is information segmented or chunked in
meaningful
ways for learners?
|
|
Information reliability: The basis to
judge truth
and validity of information.
|
|
4.74
|
|
Does the courseware provide accurate
information?
|
|
3.94
|
|
Is the content presented trustworthy (e.g.,
are sources
cited, or is evidence provided that shows the
courseware
has been subjected to a peer-reviewed
process)?
|
|
Clear, concise, and unbiased language:
Understandable
language used in the courseware is concisely
written.
|
|
4.66
|
|
Is courseware content presented clearly:
|
| |
|
(a) Is text information presented clearly?
|
| |
|
(b) Is pictorial information presented
clearly?
|
| |
|
(c) Is graphical information presented
clearly?
|
| |
|
(d) Is auditory information presented
clearly?
|
| |
|
(e) Is video information presented clearly?
|
|
4.28
|
|
Is the content free from bias, misleading
information,
and unsupported claims?
|
|
4.28
|
|
Is courseware content appropriate for the
subject:
|
| |
|
(a) Is text information suitable to the
subject?
|
| |
|
(b) Is pictorial information suitable to the
subject?
|
| |
|
(c) Is graphical information suitable to the
subject?
|
| |
|
(d) Is auditory information suitable to the
subject?
|
| |
|
(e) Is video information suitable to the
subject?
|
|
4.09
|
|
Is the content free from extraneous cultural,
gender,
and other stereotypical bias?
|
|
3.96
|
|
Is the content presented in segments that are
understandable?
|
|
3.86
|
|
(a) Is text free from spelling errors?
|
| |
|
(b) Is the text free from jargon?
|
|
Interface design and navigation: The
interface design
(menus, buttons, etc.) of the courseware.
|
|
4.55
|
|
Is the courseware screen layout (screen
elements—titles,
text areas, navigation buttons, etc.) easy to
understand?
|
|
4.53
|
|
Do learners understand directions for using
the courseware?
|
|
4.49
|
|
Is it easy to find sections of the courseware
and to
move between sections?
|
|
4.41
|
|
Can learners freely explore the courseware?
|
|
4.38
|
|
Do learners have a sense of position within
the courseware?
|
|
4.34
|
|
Do learners have control over the pace of the
instruction
presented in the courseware?
|
|
4.30
|
|
Do learners have control over navigation
through the
courseware?
|
|
4.28
|
|
Are special effects, necessary, meaningful,
and purposeful?
|
|
4.13
|
|
Is the courseware screen layout consistent?
|
|
4.02
|
|
Are the hyperlinks in the courseware clearly
marked
or annotated?
|
|
3.92
|
|
Do screen designs provide orientation
information (such
as section name, lesson/topic name, page name,
or page
number) to maintain "context"?
|
|
3.91
|
|
Will the courseware accept alternative input
devices
to accommodate individuals with disabilities?
|
|
Feedback and interactivity: What feedback
is given
to students and teachers who use the
courseware.
|
|
4.66
|
|
If a test is used, are test questions
relevant to courseware
objectives?
|
|
4.62
|
|
Is feedback appropriate to the courseware
context,
learning tasks, learner response, and learning
environment?
|
|
4.34
|
|
If feedback is auditory, can it be turned
off? If feedback
is turned off, is an alternative feedback
mechanism
available?
|
|
4.32
|
|
Is the frequency of feedback appropriate to
the courseware
context, learning tasks, learner response, and
learning
environment?
|
|
4.19
|
|
Does the courseware provide learners with
opportunities
to check their progress?
|
|
3.64
|
|
Is the timing of feedback appropriate to the
courseware
context, learning tasks, learner response, and
learning
environment?
|
|
Visual adequacy: The extent to which the
courseware
is visually appealing and easy to
interpret.
|
|
4.49
|
|
Do pictures, graphic images, and other
mediated representations
facilitate understanding?
|
|
4.42
|
|
Is text free from typographical errors?
|
|
4.30
|
|
Are readability and legibility guidelines
(e.g., sufficient
color contrast between text and background,
hierarchy
of font sizes, limited use of all caps, etc.)
appropriate
for the courseware, learning task, and
learners.
|
|
4.02
|
|
Are screen designs appropriate for the
courseware,
learning task, and learners.
|
|
3.77
|
|
If the courseware illustrates concepts using
analogies
or metaphors are they appropriate for the
learning tasks
and learners?
|
|
Motivation and attitude: How much the
courseware
motivates students.
|
|
4.45
|
|
Does the content seem relevant to learners so
they
will want to learn more?
|
|
4.41
|
|
Does the courseware arouse attention and
maintain learner
interest?
|
|
4.07
|
|
Do learners seem satisfied with the learning
experiences
provided in the courseware?
|
|
3.95
|
|
Do learners feel positive about learning the
content
of the courseware?
|
|
3.81
|
|
Do teachers/trainers feel positive about the
courseware?
|
|
Evidence of effectiveness: How effective
the courseware
has been.
|
|
4.64
|
|
(a) If evaluation is used in the courseware,
does it
measure whether objectives have been met?
|
| |
|
(b) Were there any educational gains for
students?
|
| |
|
(c) Does the courseware evaluate
learning/performance?
|
| |
|
(d) Does the courseware provide information
to teachers
on how it measures student learning?
|
| |
|
(e) Does the courseware provide information
to students
on how it measures student learning?
|
