Week 4:
Validity & Reliability
TOPIC: What is the difference between validity and reliability in your
assessments? Why are
they important?
We all agreed that the validity and reliabilty are very difficult subjects
for eductors to consider. It
is difficult to make sure that both of these are done, when we simply want
to know if our
students are learning and how we can reteach if the students need help.
It is also difficult it is to
create assessments that are both valid and reliable. It seems that reliability
is the easier of the
two to consistently measure. There aren't as many weight issue involving
reliability. It is easy to
see that a student is reliably doing well on tests assessing math skills.
What is much harder to
see is if those tests are really assessing those specific math skill, if
other factors are influencing
the score (i.e. reading skills) and whether their are any other consequences
of the assessment.
Zan thought that reliability was one of the factors that individual teachers
do have a control over!
Whether the test is too long/short, whether there are short stretching
breaks in long tests,
whether a student got into a fight at recess before the test. Whether his
mom got beat up by her
boyfriend last night. These are factors that the teacher should be aware
of and take into
consideration when reviewing the results of the assessment.
We agreed that validity is what the standardize test makers should be concerned
with, especially
the interpretation of the results of the tests. This occurs when results
of ISAT and the IOWA
Basic test are sent back to us. We take the results, restructure our teaching
and try for
different/better results next year.
These issues reminded Melissa of a recent news item. It seems that the
winners of the
Scripps-Howard Spelling Bee in recent years have all been home-schooled.
The news reports
made a big deal about this and it seemed to suggest that home-schooled
students were
smarter/better educated then other students. The reporters are looking
at reliability issues- these
student consistently win the spelling bee. But issues of validity, are
ignored- are the students only
studying spelling in order to succeed at this contest and, as a consequence
are lacking in other
skills and knowledge? Obviously, the reporters are not seriously studying
all of the issues
involved with the spelling bee, but it does relate to the issues in this
chapter.
Why are these issues so important? The most important reason is because
we want to know
what our students are learning! We want to be able to say they can do this
and why we can
prove it. If they can't do something, we want to be able to adjust so they
can learn. Tammy
does not penalize students for misspelling words, grammar, etc. on questions
on science tests.
The purpose of the test is to assess science content, not English skills.
She tells her students not
to worry unless she can't tell what they are talking about. Zan thinks
it is also important for
communication between teachers. We can help each other with this interpretation.
We are
sharing right now!
Questions: how does guessing affect the validity of multiple choice tests?
I might be a good
guesser, but still not know the material tested. Some tests penalize wrong
answers, others don't.
How do these different ways of scoring change the validity of the test?
And what about those
who guess the pattern in multiple choice test? Tammy remembers a student
being disqualified
from winning a JETS competition in high school. She was always impressed
that he had been
"smart enough" to figure out the pattern. Sure he was smart, but he wasn't
being assessed on
finding the pattern.
The Cognitive Ability Tests that were recently given at Tammy school only
had
one test form. Now that we am aware of reliability and validity, we wonder
how this test would
rate.