Tools:
Fayette County Public Schools
Note. The Web sites listed
on this
page were valid when this issue of L&L went to press.
We have
no control over these sites, and the Web is very volatile.
Please
let us know if you find a broken link, and we’ll do
our best
to fix it.
Student
Impact
Rubric
Data
Collection Instruments: www.sun-associates.com/eval/sample
Fayette
County Public Schools Update
On
February 14, 2000, Sun Associates presented the final
report on
Fayette County’s formative evaluation to the district
school
committee. Though the district is only just beginning to
digest
the report and its implications, the project has already
generated
considerable discussion around a variety of technology
integration
issues. Thus far, the discussion has centered around two
areas:
(1) the link between technology and pedagogy and (2) the
link between
our evaluation work and existing, traditional assessments.
Technology and Pedagogy
Our
evaluation found that although technology was widely used
by Fayette
County teachers as a productivity tool, considerable room
for growth
existed in terms of how teachers used technology to enrich
and reform
their teaching practice. In other words, the majority of
the district’s
teachers were using technology to support existing
instructional
models and practices. Readers familiar with the research
base relating
to technology and educational change will quickly
recognize that
Fayette County’s teachers are for the most part at
the adoption
and appropriation levels of the Apple Classrooms of
Tomorrow
(ACOT) framework. At these stages, teachers are using
basic technology
tools to augment, reinforce, and support existing
practices.
In
Fayette County’s indicator rubrics, teacher behaviors
associated
with adoption or appropriation are indicative of
performance at
the midrange on the district’s four-level scale. To
attain
higher levels of achievement, teachers would need to
exhibit behaviors
more in line with that ACOT research terms
appropriation
or invention. Furthermore, the indicators
themselves provide
a range of examples of just what sorts of practices and
outcomes
one would expect to find/observe among teachers performing
at these
higher levels.
Fayette
County did not set out to confirm the ACOT research
through its
formative evaluation process. Rather, the district
evaluation committee
simply identified what it wanted to find in terms
of teacher
use of technology and the effects that different degrees
of technology
integration could have on student learning. As the various
degrees
of achievement were determined, it soon became obvious
that these
levels mapped to and confirmed those documented by ACOT
and other
research related to change. A review of this research is
suggested
for any district considering an evaluation of where its
teachers
are in terms of technology integration. Still, it is
extremely important
that the district create its own performance indicators
which interpret
levels of teacher adoption as actually observable and
realistic
behaviors within the context of the evaluating district.
As we point
out in the main article, all districts have unique
expectations
for teachers and students. These need to be reflected in
the developed
indicators if the evaluation is to be taken seriously, or
considered
valid, by the district’s teachers, administrators,
and community.
Validity is required for the evaluation to reach its
potential as
a formative assessment tool.
The Link
Between Formative Technology Evaluation and Traditional
Student
Assessments
After
considering the indicators, rubrics, and findings
developed for
Fayette County’s educational technology evaluation,
some teachers
and administrators in the district asked if there
wasn’t another
“bottom line” that we had missed; that is, what
is the
connection—if any—between student and teacher
technology
use and traditional student assessments such as
standardized test
scores. Given the amount of attention provided to these
traditional
assessments, it is certainly expected that one should
attempt to
tie all important educational initiatives, such as
technology, to
performance gains or losses on these scores.
We
at Sun Associates believe that a link exists between
technology
and student performance as measured by traditional
assessments,
but this link is not direct. In other words, technology
alone is
not responsible for increases in student achievement.
Rather, the
integration of technology is one of a number of changes
and improvements
made to the teaching and learning environment. When these
changes
are made systemically, then student performance
increases.
Perhaps
the strongest point we made in the district’s full
evaluation
report is that technology alone has little positive
effect
on student achievement or teacher behavior. In fact, we
found that
as a whole, Fayette County schools are reasonably rich in
technology
tools and devices. For the most part, teachers actively
use this
technology for personal productivity. But, we found that
the simple
existence of the devices and the fact that teachers know
how to
use them does not ensure that technology will be used in
ways that
affect student achievement.
Educational
research is absolute in its findings that student
achievement increases
when learning activities are engaging and
student-centered. Learning
needs to be standards based, relevant, attuned to the
individual
student’s style of learning, and holistic—that
is, tied
to a student’s prior knowledge, experience, and
interests.
Another way of saying all of this is to shorthand these
descriptions
and state that high performing student learning is engaged
and constructivist
in nature and guided by strong and meaningful curriculum
frameworks.
Students who are able to work in environments that
encourage this
type of learning will achieve their maximum
potential. Although
existing standardized tests do at best only an adequate
job of truly
measuring student achievement and knowledge, students who
are the
products of supportive learning environments and reformed
teaching
practices will generally score higher than students who
are not.
When
we examine the various indicators developed by the Fayette
County
Technology Evaluation Committee, we see that the
indicators have
been written to “reward” those uses of
technology that
are products of and supports to high-performance learning
environments.
The committee’s work and the indicators they
developed were
informed by a knowledge of what the research says about
technology
and learning. High-performing schools use technology in
ways that
score high on the committee’s indicators. What the
evaluation
shows is that most schools could do better in terms of how
technology
is being used to support student achievement. In other
words, we
are certain that as the district scores higher on its
technology
rubrics, it would also as a whole have higher standardized
test
scores.
It
all comes back to the same basic point: technology is only
useful—and,
in this case, we can say impactful—when used by a
skilled teacher.
The issue is pedagogy, not computer skills.
Subsequently,
schools where teachers are employing reformed
instructional practices,
where administrators support and expect such practices,
where students
are actively and excitedly involved in the process of
learning—these
schools will be “high performing schools.” And
when we
visited those schools in Fayette County, we found
technology being
used by students and teachers in ways that would score
very highly
on our technology rubrics.
Lexington
Public Schools: Assessing Technology Staff
Development
Since
1997, we have worked with teachers and administrators in
the Lexington
(Massachusetts) Public Schools (http://lps.lexingtonma.org)
to construct and implement a variety of assessments
relating to
Lexington’s technology plan. Lexington’s
five-year technology
plan calls for each year to be devoted to a particular
strategic
theme. Year One (1996–97) emphasized completing the
district’s
network infrastructure. Year Two’s theme,
“teacher immersion,”
emphasized developing teacher technology skills. That
year, we helped
the district assess their technology staff development
efforts.
This afforded a more limited and tighter focus than an
overall evaluation
of technology’s effect on student achievement. The
findings
from Year Two provided valuable background for subsequent
years
in which the emphasis would be placed on developing and
implementing
technology-supported curricula.
In
fall 1997, Lexington convened a districtwide technology
evaluation
committee with a very similar stakeholder composition to
that of
Fayette County. Over the course of a day-long meeting, the
committee
developed evaluation questions rooted in that year’s
specific
technology plan goals. This method of developing questions
rooted
in planning goals addresses the district’s basic need
to know
whether they are meeting their goals for IT and the
various implementation
process steps.
After
setting their basic evaluation questions, the district
created a
mechanism for assessing their achievement in meeting the
Year Two
goals. This was accomplished using a multilevel rubric
that organized
indicators of success toward meeting an overall indicator
of what
it “looked like” to achieve the performance goal
associated
with each evaluation question. These rubrics can be viewed
online
at www.sun-associates.com/eval/sample.
As in Fayette County, Lexington’s evaluation
committee worked
to develop indicators relevant to actual behaviors and
attributes
appropriate for Lexington Public School teachers to when
meeting
the Year Two goal of “teacher immersion.”
Just
as in Fayette County, Lexington’s evaluation
committee took
time to debate exactly what teacher behaviors were
exemplary of
meeting the district’s technology professional
development
goals. The time spent doing this represented a critical
period of
reflection. In my experience, reflection is one of the
major benefits
of conducting a technology evaluation effort. Reflection
on anticipated
and desired outcomes helps deepen a district’s
collective understanding
of technology integration. This deeper understanding among
stakeholders
will translate into greater support for technology and the
benefits
it brings to the classroom.
Resources
Apple
Computer Corporation. (1996). Changing the conversation
about teaching,
learning, and technology: A report on 10 years of ACOT
research.
Cupertino, CA: Author. Available: www.apple.com/education/k12/leadership/acot/library.html.
Fayette
County Public Schools: www.fayette.k12.ky.us
Loucks-Horsley,
S., Hewson, P.W., Love, N., & Stiles, K.E. (1998).
Designing
professional development for teachers of science and
mathematics.
Thousand Oaks, CA: Corwin Press.
Milken Exchange’s Seven Dimensions for Gauging Progress of Technology.
Back
to “How Do We Know It’s Working”
Copyright © 1991, ISTE (International Society for
Technology in Education).
All rights reserved.
|
