There is one message totalling 153 lines in this issue.

Topics of the day:

  1. Student-centered evaluation of distance learning at Virginia Tech

----------------------------------------------------------------------

Date:    Mon, 19 Sep 1994 10:31:39 -0400
From:    Barbara Lockee <Barbara.Lockee@VT.EDU>
Subject: Student-centered evaluation of distance learning at Virginia Tech

The following is a paper which was presented at the Tenth Annual Conference
on Distance Teaching and Learning in Madison, Wisconsin August 10-12, 1994.

FORMING THE FUTURE FROM THE PAST:
STUDENT-CENTERED EVALUATION OF DISTANCE LEARNING
AT VIRGINIA TECH

Tom Head
Barbara Lockee

Virginia Tech


        The Commonwealth Graduate Engineering Program is a collaborative
effort between the major institutions of higher education in Virginia to
meet the needs of the growing number of non-traditional students in
engineering.  The program offers 38 graduate courses each year via
satellite serving approximately 3000 students at both on-campus and
off-campus sites.  The originating sites, or "host institutions" for the
broadcast of engineering courses are Old Dominion University, the
University of Virginia, Virginia Commonwealth University, and Virginia
Tech.  Over thirty receive sites are used throughout the country, including
other universities and corporate destinations.  Virginia Tech is
responsible for the implementation of program evaluation, including data
collection and analysis.
        Ongoing evaluation is an integral part of program evolution in
keeping with good instructional design practice.  Learner assessment of
teaching strategies, delivery mechanisms, and other relevant instructional
tools, provides program coordinators with invaluable information as to how
courses can be changed to better meet student needs.  Over the past 12
years, the Commonwealth Graduate Engineering Program has been continually
refined and developed with the assistance of its students.  The
learner-based evaluation information is useful in the continuing effort to
improve the quality of the courses, meet the needs of the students, and
provide expanded educational opportunities to the Commonwealth.
        Student evaluation of the graduate engineering program is conducted
for all courses taught in the program.  Each semester an evaluation week is
designated near the end of the term.  Concurrent evaluation times between
on-campus and remote sites help ensure timeliness and an opportunity for
all students to respond.
        The same evaluation instrument, an op-scan questionnaire, is used
for on- and off-campus students.  However, off-campus students are asked to
respond to additional questions regarding issues unique to their learning
environment.  The first portion of the instrument is dedicated to analysis
of student perceptions of instruction and deals with topics such as
accomplishment of course objectives, relevance of outside assignments, and
instructor performance in general.  The latter part of the questionnaire
asks for the evaluation of specific remote site factors such as audio and
video quality, timely distribution of course handouts and homework,
textbook and course material availability and problem resolution and
assistance by the site coordinator.
        Along with Likert scale data, both portions of the questionnaire
ask for written feedback.  This provides the students with an opportunity
to expound on desired topics such as specific logistical or technical
problems.  While on-site and off-site assistance is available for immediate
problem solving, such written disclosure helps document problems in order
to provide improvements in subsequent courses.  Also, students can provide
written critiques of instruction.  For example, if a particular teaching
method is not effectively communicated through a televised medium, learner
evaluations can provide the instruction with such feedback which results in
course improvements or the instructor choosing to withdraw from the
program.
        Upon completion, the forms are sent to the Office of Distance
Learning at Virginia Tech where written comments are recorded and the
op-scans are analyzed.
        Each year a formal report is forwarded to the State Council of
Higher Education in Virginia, to the participating institutions and the
site coordinators.  The following is a summary of the results of the
student evaluations for the 1993-94 academic year.

*       The overall quality of instruction was rated as good to excellent
by 81% of the students.

*       Most students (89%) feel that course objectives are met and the
level of difficulty (91%) is as expected for graduate level courses.

*       Visual and audio reception are considered adequate to excellent by
99% of the students.

*       Textbook and course material availability is considered adequate by
91% of the students.

*       The distribution of course material, with respect to timeliness, is
considered at least adequate by 93% of the students.

*       Course registration procedures are considered adequate by 97% of
the students.

*       Problem resolution and assistance by site coordinators was
considered at least adequate by 97% of the students.

*       The value of televised courses is rated as average to much above
average by 70% of students when compared with other graduate courses taken
in a traditional classroom setting.

*       Students gave an overall rating of 3.1 on a scale of 4.0 for the
quality of instruction in the program.

A comparison of course grades for the Fall Semester, 1993 showed an average
of 3.3 for students at remote sites and 3.5 for on-campus students on a 4.0
scale.

        The student evaluation instrument is refined each year in order to
ensure its validity.  An evaluation by students will be continued for the
coming academic year.  Evaluation information is useful in the continuing
effort to improve the quality of the courses, maintain high technical
standards, meet the needs of the students, and provide expanded educational
opportunities to the Commonwealth.








Dr. Head is currently Director of Media Services at Virginia Tech.  Media
Services is a comprehensive service organization which provides video and
distance learning production services as well as photo, graphic design,
printing, mail services.  Media Services also houses a faculty development
initiative which will involve all faculty over a four-year cycle in
implementation of instructional technology in their teaching.  He teaches a
graduate seminar on instructional graphics, and his research interests
include the effects of graphic imagery on learning and new paradigms for
distance education.  Prior to holding this position, he served as head of
audiovisual services, and as an instructional developer for the university.
He previously was Director of Instructional Services at Hamline University
and was a research chemist for ADM Chemicals in St. Paul, Minnesota.
E-mail:  Tom.Head@vt.edu
Fax   :  (703)231-5922


Barbara Lockee is a doctoral student in Instructional Technology at
Virginia Tech.  Her assistantship in the Office of Distance Learning
involves the bi-annual evaluation of the Commonwealth Graduate Engineering
Program.  Her research interests include instructional systems development
and cultural influences on learning.  Before attending Virginia Tech, she
taught instructional media production courses at Appalachian State
University in Boone, North Carolina.
E-mail:  Barbara.Lockee@vt.edu
Fax   :  (703)231-5922

Barbara Lockee                                  Voice:  (703)231-6079
Evaluation Coordinator                          Fax:    (703)231-5922
Office of Distance Learning at Virginia Tech    Internet:Barbara.Lockee@vt.edu
Blacksburg, VA  24061-0232

------------------------------

End of ASAT-EVA Digest - 18 Sep 1994 to 20 Sep 1994
***************************************************