EVALUATING INSTRUCTIONAL TECHNOLOGIES

By Gloria Henke

With President Clinton mandating that all schools be wired and every classroom have Internet access, schools have been rushing to meet the hardware standards that the government and local school boards have demanded. Taxpayers, however, are casting a suspicious eye toward the expenditures on technology. They are beginning to ask for tangible evidence that their tax dollars are not being wasted on items that give the appearance of improving learning, but may not be producing measurable results at all.

Many of the articles in this assignment grapple with the difficulty of determining whether using technology results in educational gains for students. An article in the May 1998 issue of Technology & Learning magazine discusses some of the problems with creating authentic assessment tools in this area. One problem is that some improvements may not be measurable using traditional evaluation tools. For example, studies have shown that drill and practice mathematics software programs can raise students’ scores on standardized math tests. But standardized tests have come under fire lately because they tend to measure rote learning and not higher level thinking processes. Designing a test to determine improvements in analyzing and synthesizing material is much more difficult.

A second problem lies in determining whether the faculty has improved instruction by incorporating technology into their lessons. In the March, 1999 Reading Online article " Challenges for the Evaluation of New Information and Communication Technologies", Bertram Bruce mentions the teachers who "merely rewrite their current methods in a new medium without any substantive effect on students' learning." In other words, some teachers are using new technology to teach the same old lessons. A teacher recently revealed that his students hate it when he uses Power Point to teach a history lesson because it is very impersonal and boring, in spite of his graphics and orderly presentation. A math teacher uses Power Point to demonstrate how to solve an algebraic equation. Although he inserts different colors to highlight the variable and feels that he is being very creative in his approach to solving equations, in reality, he is merely using an electronic textbook. Worse, in generating the solution ahead of time, he has not allowed for students' creative approaches to solving the problem. Other teachers feel they are using technology in their lessons when they bring their students into the computer lab to type a research paper.

Although these teachers have satisfied the administration’s request for incorporating technology into the curriculum, their students are not actively engaged in the learning process.  They are probably reaping few benefits from the teacher's use of technology in the lesson.

It may be difficult to create adequate tools to assess the benefits of using technology, but schools should have some method of measuring whether students are improving as a result. Even though many forms exist to evaluate web pages and web quests, few exist which evaluate lessons incorporating other types of technology.  An Educator's Guide to Evaluating The Use of Technology in Schools and Classrooms gives some guidelines for assessment, but the sample questions seem too numerous and lengthy.  If the evaluation checklist is too lengthy, teachers will not take the time to do it. The form should be concise, yet informative. To expect a teacher to answer a long questionnaire in addition to designing the lesson seems overly burdensome at this stage. In time, adequate assessments will probably be developed. (This probably is a project for another course!)

In The White Papers from The Secretary's Conference on Educational Technology, there are many links addressing these concerns.