Preliminary Evaluation of CTER Online

Gregory L. Waddoups
Sandra R. Levin

University of Illinois
College of Education
 

Introduction

The Curriculum, Technology, and Education Reform (CTER) masters degree specialization is designed to provide practicing teachers with the opportunity to earn a coherent high quality masters degree at home or at their local school through using personal computers and Internet connections. Over the past several years, the Illinois State Board of Education has revised the state’s pre-college (K-12) curriculum standards. One focus of these standards is the use of information technology to facilitate a project- oriented approach to learning and instruction. The College of Education has received increasingly frequent requests from practicing teachers interested in a masters degree program to upgrade their knowledge and skills in these areas, but who are unable to do so because their schedules prevent them from attending on-campus classes at the University. Since the admitting our first groups of twenty six students, the CTER program has received over 100 requests for admission into the program. Students requesting information are particularly interested in the unique opportunity to learn from nationally known faculty through using Internet connections.

CTER Student Profile

Students in the CTER program represent eight school districts and 15 schools in the state of Illinois. CTER students also teach at all levels in primary and secondary schools and teach subjects ranging from mathematics, language arts, music, special education, and science. In addition, five district technology coordinators are involved as students in the CTER program. Given the diverse background of the CTER student population, it is not surprising that there is a great deal of diversity in technical skill level among the students. During the summer orientation, we conducted a survey to measure students technical skill level, their experience with the World Wide Web, and their learning preferences.

  1. Computer applications

  2. It is interesting to note that the CTER students came to the class with a wide variety of skills in using the computer. The distribution of student’s in skills using the World Wide Web (WWW) and in using synchronous communication programs is skewed toward less experienced.
     
    Skills
    No Experience
    Beginning
    Intermediate
    Advanced
    Constructing Web Pages
    8
    11
    5
    2
    Using Spread Sheet Software
    0
    14
    10
    2
    Group Discussion (Electronic Bulletin Board)
    10
    9
    5
    2
    Using Presentation software
    5
    6
    11
    4
    CD-ROM referencing software
    0
    5
    11
    10
    Word Processing software
    0
    1
    15
    10
    Use of Multimedia applications
    0
    7
    13
    5

     

    Experience with WWW

    Student’s experience with the WWW was also widely distributed with most students reporting that they had no experience or were beginners in downloading programs from the web.
     
    Questions No Experience Beginning Intermediate Advanced
    Access Audio from WWW using Plugins such as Real Player 15 6 2 3
    Navigate in WWW 0 9 10 6
    Download programs from the internet 15 11 6 4
    Use Acrobat Reader to open pdf files 16 2 5 3
    Bookmark sites 3 3 11 9

  3. Learner profile
It is relatively clear, but not surprising that students are motivated to use computer technologies and claim to enjoy them, however, they are more ambiguous about the effectiveness of working in groups or working alone.
 
Questions
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
In enjoy using the computer
0
0
0
7
19
I do not enjoy using the Internet
16
7
1
1
1
I am not motivated to use new technologies
20
5
0
1
0
When I am asked to use new technologies, I will try them even if I feel a bit
1
0
0
10
15
I am confident I will do well with network-based communication
0
0
3
13
10

 
 

Question

Strongly disagree Disagree Neutral Agree Strongly 

Agree

Working on my own is the most effective way to learn
4
4
12
5
1
Working in groups is the most effective way to learn
1
2
10
11
2
I learn best when the teacher uses a lecture format
5
11
8
2
0
I learn best when working on projects
0
1
4
10
11
I enjoy learning by reading
1
2
12
9
2
I enjoy working independently and seldom need reassurance that I am doing things correctly
0
7
9
10
0
It is very important that the trainee and trainer are in the same room
3
15
7
1
0
If this masters degree program were to use a traditional format it would be better
10
14
2
0
0
It is not necessary to be in a traditional classroom
1
0
1
11
13

(D) Open ended Questions

The students in the CTER program were attracted to the flexibility of the program and the opportunity it provided them to learn how to successful integrate technology into their classrooms. Because technology is both the medium and the content in this program, students believe the Ed Online program will immerse them in many of the technologies they will use in their classrooms. As a result, they believe this program will help improve the quality of their teaching. The concerns the students have about the program are related to time constraints to successfully participate in the program and whether their technology skills are sufficient.
 
Questions Responses
(1) What attracted you to the CTER on-line program? Seven themes emerged and are listed in the order of most frequent to less frequent occurances: Flexibility of an on-line program (15), Skills learned in the program can be easily integrated into classroom teaching (12), An opportunity to increase my knowledge of using technology in the K-12 classroom (4), Sharing experiences with other educators using computers in the classroom (2), Integrating technology into the K-12 classroom curriculum is motivating to students (2), For technology coordinators, this program is an opportunity to focus on an overall strategy for staff development, evaluation, etc. (1), Best on-line program currently available (1)
(2) What do you see as program benefits? Three major themes emerged: to improve teaching (16), to gain more knowledge of technology (12), and Enhance my students’ learning experiences (5). Other responses included: Develop instructional materials (2), Model skills learned for colleagues (1), Learn more about assessing the use of technology in schools (1), Networking with other technology coordinators (1)
(3) Program concerns Lack of technology experience and time constraints were respondents main concerns: not knowing enough in the beginning (8), time this summer (7), no time when teaching (3), not very creative person (3), 1st time program offered (2), different teaching field than others (2), not in the classroom (2)
(4) Other Assistance Respondents sited non-financial assistance the most: equipment support (18), administrative support (12), assistance on the job (10), tuition reimbursement (5), release time (5), not financially supported (3)

 

CTER Summer Courses

  1. Course Descriptions:
While this is an on-line course, all CTER students were required to participate in a weekend workshop at the Champaign campus at the beginning of the summer. During the workshop, students meet CTER instructors and their fellow class members and formed project groups. In addition, a series of topics were presented to ensure that participants possess the essential skills and abilities to fully participate in the course activities. These topics included the basics of operating systems, downloading and installation of software, uploading files to a server, etc. Attendance at this workshop was required of all students regardless of their technical proficiency. (1) C&I 335 The purpose of this course is to assist practicing teachers (and those desiring to assist in the training of teachers) in developing essential computer technology skills and a corpus of technology-enhanced instructional material related to their teaching assignments. A large portion of the course focused on using the Internet and World Wide Web as instructional resources, however students were encouraged to explore instructional software such as drills, tutorials and simulations. In addition, students were introduced to productivity tools such as word processors, databases and spreadsheets; and also new technologies that may not yet be available for classroom use. It is important that educators develop awareness and some experience with all of these instructional applications for computers, so they will be able to select appropriate technologies for the types of learning they wish to promote. All of the modules of this course are activity based, and allow the students in this course to determine the particular tools that best suit their needs as they develop resources that they will be able to use with their students.

(2) Edpsy 387

The focus of this course is on the exploration of innovative Computer Uses in Education using technology. This is a project-oriented course, and class members will conduct a variety of miniprojects exploring a range  of new uses for computer technology in the classroom.  In addition, class members will continue to develop the Major Project begun in C&I 335, incorporating ideas and techniques gleaned in this second summer course.  This latter project may serve as the semester project to be implemented in a classroom setting and evaluated in Chip Bruce's C&I 490 IET course this autumn.

(B) CTER Summer Course Evaluations

    1. Quantitative Responses:
The students in the CTER program evaluated C&I335 and EdPsy using an evaluation format identical to the one completed by on campus UIUC students. The CTER staff developed a survey, created a survey web template, and connected the template to a File Maker Pro database.
 
ICES Question
C&I 335 Edpsy387
  1. Rate the instructor's overall teaching effectiveness.
3.8571 4.4345
2. Rate the overall quality of this course. 3.9098 4.3919
  1. The instructor stated clearly what was expected of 
students.
3.6667 4.0870
4. How well did the instructor coordinate different activities of this 

course?

3.8571 4.4348
  1. Should more/less time be provided to review and 
synthesize the 

course material?

3.8095 3.6957
6. How would you rate the instructional materials used in

this course?

3.7619 4.3043
7. The grading procedures for the course were: 4.0000 4.2174
8. How much work did this course require? 4.2857 3.9130
9. Did this course improve your understanding of concepts and 

principles in this field?

4.2057 4.6522
10. Did you develop a clearer sense of professional 

responsibility from this course?

3.8095 4.3478
11. How would you characterize the instructor's command of the subject matter? 4.3810 4.7391
12. How effective was the instructor in presenting material in lecture/discussion? 3.8824 4.4000
13. The instructor was conscientious about his/her responsibilities. 4.5000 4.8696
14. The instructor encouraged development of new viewpoints and appreciations. 4.2381 4.7826
15. Evaluations of my work were made in a constructive manner. 4.1905 4.6522
16. The instructor promoted an atmosphere conducive to work and

learning

4.1429 4.7391
17. The workload for this class was: 3.9048 3.4348
    1. Qualitative Responses
From the qualitative responses, it is clear that students are finding value in the project orientation of the class and in the developing competence with new resources such as TAPPED In and web page creation. Given that students entered this program with the expectation of improving their teaching, it is not surprising that the assignments they judged most useful were those that related to ways in which they could improve their teaching. Related to this point, these students seem particularly sensitive to "busy work" or assignments that have little connection to their teaching.

The students are satisfied with the technical support offered by the CTER program. Almost all students reported that if they used technical support it was excellent. Students used office hours more in Ed Psy 387 than in C&I 335, the reason for this difference is the greater variety of office hours held. The only technical concern students had concerning the summer classes were the instability of the server. Students in both classes were concerned about the instability of the course server.
 
 
 
 

Questions

C&I 335
Ed Psy 387
Most useful assignment accessibility (7), explore new tech (7), webpage & ePortfolio creation (6), major project (6), all assignments (6), creating units - lesson plans (4), identifying resources (4), legal & ethical issues (2), readings (1), variety of activities (1), listservs (1), dream classroom (1) final project (9), Tapped In (9), readings (5), learning something new from all assignments (3), Javascript- exploring (3), web evaluations (3), exploring new technologies (2), PowerPoint with audio (1), outside expert in WebBoard (1)
Least useful assignment identifying local resources and technology plans - busy work (5), none (4), professor’s comments on webpage design (2), lack of guidance on assignments (2), creating detailed lesson plans - because they will need adjustments (1), feedback on assignments not meaningful (1), more hands-on demonstrations of new technologies needed in face-to-face meetings (1) none (5), PowerPoint audio narration (4), completing the Technology Competency Database - too much redundancy (4), summarizing articles (3), Javascript assignment was frustrating (2), evaluating online resources - was a repeat of an assignment in C&I 335 (1), assignment 2C (1), TAPPED IN (1)
Technical problems Learning Resource Server down (9), uploading files to LRS (8), not enough written instructions to take home from orientation (5), too much new technology to learn immediately (2), program catered to people using Macintosh computers (1), incompatibility between AOL and WebBoard (1), using WebBoard (1), did not have Claris Home Page available at home or school (1), fear of using technology (1), lack of experience (1), hardware at home and school inadequate for technologies used and incompatibility of different versions of software (1), slow modem (1), would like more instruction on using images (1) WebBoard not dependable for large groups (14), TAPPED IN frustrating (12), LRS server down (6), home computer inadequate for new technologies (2), Javascript assignment (2), none (1), problems using a PC with assignments (1)
Technical support excellent (9), not used (4), when using WebBoard don’t use AOL (1) excellent (15), add PP slide show/web in orientation & possibly Tapped In (15), more handouts (1), cleared directions for project (1), need more instructions for presenting on the web with WebBoard or Tapped In (1), Paul (TA) as 2nd instructor (1)
On-line office hours 1-3 (13), 8-12 (5), none (3), 4-7 (1) 1-3 (8), 4-7 (7), 8-12 (5), none (1)
Email  4-7 (9), 1-3 (8), 8-12 (2), >12 (2), none (1) 1-3 (10), 4-7 (6), 8-12 (2), >12 (2), none 

Students were pleased with the use of the web board 75% felt it was either very effective or effective in keeping contact with the teacher. There is a clear discrepancy in how students felt about response groups in C&I335 versus Ed Psy 387, probably due to the high use of these groups in C&I335 as compared to Ed Psy 387. From conversations with case study participants, it was clear that they constructed in Ed Psy 387 groups were self organized, whereas in C&I 335 students utilized the response group system provided in the CTER orientation.
 
 
 

Question

C&I 335
Edpsy 387
New office hours provide a variety of day & evening hours (14), fine as is (4), doesn’t matter (4) fine as it (varied hrs.) (16), doesn’t matter (2), prefer sun afternoon in fall (1), prefer evening hrs. (1)
WebBoard contact very effective (6), chat effective (2), OK (9), chat OK (1), discussion OK (1), not very effective (2), both AOL users) Very effective (9), OK (8), OK chat only (2), not very effective (2), not very effective chat only (2)
Response groups effectiveness very effective (8), effective (7), not very effective (5) not effective (8), very effective (6), effective (5)
Response groups recommended changes no changes (14), need common interest or topic in groups formed (2), immediate feedback important (1), no response (1), instructor should decide groups (1), make response groups optional no changes (11), eliminate groups (2), continue people in the same groups (by subject & grade level) throughout course or program (2), too many response groups (1), email notification when someone 
Course strengths immersion into technology (9), online format (5), expertise of instructor (5), variety of activities (5), gained more confidence in using technology (2), limited enrollment (1), orientation to help forma group (1), sharing of ideas (1), project that can be used in class (1), legal issues using technology (1), heightened awareness of tech in school (1), meeting needs of diverse populations (1) used a variety of technologies (9), excellent instructor & TA (6), reading & information was relevant & useful (2), tech support (2), WebBoard (2), Tapped In (2), final project/sharing (1), active learning - hands-on (1), online format (1), increased confidence with technology (1), responsive, available & friendly (1) 
Course limitations unclear expectations from teacher (8), too much work in 4 week course and uneven workload (6), hardware & server problems (5), time (1), getting used to new format of instruction (1), some communication problems (1), too much busywork (1), learned a lot, but not sure how/if it will help me teach students better (1), too much of course grade based on final project (1) problems with current technology (6), acceptable/exemplary what constitutes an A (4), syllabus not always ready to work ahead (3), not enough personal direction (1), time- weeks 3 & 4 were intense (1), better tutorial for Javascript (1), summer course too short to explore so many technologies (1), not seeing final projects (1), no limitations (1)
Web templates very useful (7), OK (6), No (4), no response (3), didn’t matter (2)  
WebBoard feelings N/A very satisfied (10), satisfied (9), unsatisfied (5)
TAPPED IN  N/A very satisfied (11), satisfied (10), unsatisfied (2), very unsatisfied (1)
CTERbase  N/A very satisfied (20), satisfied (4)
Reflector  N/A very satisfied (17), satisfied (7)

(B) Classroom Communication

Communication between the professor and student was facilitated through synchronous office hours using Webboard, and asynchronous email. In addition, students maintained contact with technical support people through the telephone, Webboard, and email. Students maintained contact with each other via an email reflector group, posting on Webboard, and telephone contact. In addition, students reported that they met with classmates in their homes or schools when possible. In both classes, there was a great deal of communication between and among the students and professor.

(1) Webboard

Webboard was a primary means for students to communicate with one another and provided a virtual context for responding to the work of their classmates. Nineteen out of twenty six students reported being either satisfied or very satisfied with Webboard. According to one faculty member, students liked the threaded conferences (as evidenced by the large number of postings and accesses). This format provided a place for students to give thoughtful suggestions to assist classmates to improve their work. In addition, the chat mechanism of Webboard provided a space for virtual office hours. The Javascript Chat worked well up to about 23 simultaneous users. Because the CTER program has 26 students, during students’ final project in presentations EdPsy 387, the Javascript Chat "ground to a snail's pace when the last couple students entered the chatroom". Consideration needs to be made about developing technologies that can maintain chat for more than twenty five students.

(2) Tapped In

During the second week of the course, the students were required to use

TAPPED IN in two ways: 1) for professional collaboration among their own

subgroups in the class, and 2) to draw upon some of the resources available

in TAPPED IN. So, subgroups of students met in the CTER room, but mostly

ended up talking about TAPPED IN and the course (according to the

transcripts they submitted). Students continued to meet informally to discuss more substantive matters. About 2/3's of them attended the Thursday (July 16th)

session on "Encouraging Girls to Pursue Math and Science". Several of

them were very positive about resources (both human and web-based)

available in the Math Forum room and in the ED's OASIS room.

During the last week of the course, TAPPED IN was used as a forum

for final project presentations. Students selected whether they wanted to

present within TAPPED IN or within Webboard (another, simpler, web-based

conferencing system we also used). Seven decided to use TAPPED IN, and

those presentations on Thursday, July 30, went very well

According to faculty report, CTER students started out somewhat dubious about TAPPED IN, because of the complexities of the system, but became much more positive

about it as the class proceeded. Twenty one out of twenty three respondents were either very satisfied or satisfied with the use of TAPPED IN

Campus, College, and Department resources utilized

(A) Hardware and Software Resources

CTER has utilized a variety of resources internal and external to the College of Education. Internal resources include: COE Web Server for some course webpages, Rooms 10-16 and 37 for Orientation and on-campus meetings. The Learning Resource Server (LRS) and Learning Resources Server Database (LRSDB), currently housed in Room 372, are used for student work folders and databases. The Graduate Programs secretary currently maintains the CTER student enrollment and information request databases in the C&I department. Technical support is housed in Room 372 and has been utilized extensively throughout the summer and Fall of 1998. Course development has required the use of specialized hardware and software, some of which is available in the College.

The CTER program has also drawn upon resources external to the College of Education. The Academic Outreach and Public Service office handles course registrations, provides CTER students with access to on-campus libraries, and audio bridge capability for synchronous conference calls for small group discussions. The Web development group in this office has provided set-up and maintenance of WebBoards for course discussions and on-line chat, a secure server for course listservs, consulting on the purchase of hardware, and some graphic design expertise. Some course development has required the use of specialized hardware and software for multimedia production not available in the College. Although inconvenient for faculty and research assistants, the Academic Outreach and Public Service office, NCSA, and the visualization lab at Beckman have been able to temporarily provide space for this development work.

In addition to resources on the UIUC campus, SRI has provided space on their TAPPED IN MOO to facilitate on-line discussions between CTER students and between CTER students and other educators in a variety of disciplines across the country. TAPPED IN has also be used for CTER student presentations.

(B) Human Resources

The CTER program supports one half-time project coordinator, two technical support and program development RAs housed in Room 372, and additional RAs to assist faculty in course development. The CTER program has also drawn up the expertise of other RAs in the College for modifying the Technology Competency Database (TCD) to track course assignments (CTERbase).

The College of Education has provided summer salaries for two faculty members to teach C&I 335 and Ed Psy 387 this summer. The C&I department has released one faculty member from teaching one on-campus course in order to accommodate the needs of the CTER program. This Spring (1999), two faculty members (one from C&I and one from EPS) will co-teach EPS 304. In addition, Ed-Online has provided an RA for evaluation of the CTER program.

Education and evaluation experts external to UIUC has also participated in the CTER courses conducted to date.

(C) CTER Tuition Revenues

Twenty-six students are enrolled in the CTER program. Two students have tuition and fee waivers, therefore, 24 CTER students are generating tuition revenue. Each course brings in $12,000 - $13,000 in revenue. With eight courses in the program, the departments in the College of Education are expected to receive $96,000 - $104,000 less course expenses incurred by the Academic Outreach and Public Service office. At this time, the CTER program is incurring course expenses in the use of the audio bridge. These expenses are being monitored

Summary

The CTER Online summer courses were successful in providing a coherent and high quality experience for students. From the relatively high ICES evaluation scores, it is clear that students were satisfied with the content and delivery of the courses. Indeed, one case study student reported that she learned more this summer than she learned in her entire four years as an undergraduate at a regional University. Although the content was challenging, the CTER students developed a core set of skills necessary for integrating technology in the classroom.
 
 












CTER Online

Department of Curriculum and Instruction

University of Illinois
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Summer Course Evaluation