Should Standards and Assessments be Piloted?

A call for piloting standards and assessments has been raised by educators around the world who are faced with the problems inherent in moving to outcomes-based learning.  This from David B. Cohen in the US is representative of the sorts of things I’ve heard:

Tweet Let's pilot new standards and assessments

That verb to pilot has a couple of common meanings.  Its most common meaning is to lead or guide, typically in difficult conditions.  That definition doesn’t fit the context of the Tweet. I suspect many of  Cohen’s Twitter followers would say the standards and assessments are the difficult conditions.

The second meaning is probably closer to what Cohen has in mind, but even it is not a good match for the context. To pilot can mean to set a course and see that the vessel arrives at its destination.  That meaning does not suggest that there’s flaw in the vessel, only that it requires a skilled operator.  I doubt that the folks who are opposed to new standards and new assessments would be caught dead suggesting that better quality teachers would have no problem using them.

It seems to me that although Cohen uses pilot as a verb, he wants the word to be understood in its adjectival meaning of testing or experimental, as in the phrase “a pilot program to train monkeys to run cash registers.”

Even assuming Cohen wants a  limited pilot program to test standards and assessments, I still see a problem.

Standards just are

By themselves, the number of people who meet a particular standard doesn’t tell anything about whether a standard is good or bad.

If the carnival ride has a requirement, “people must be 48-inches high to ride the Cyclone,”  having a random sample of 1,000 people line up against the standard won’t tell whether they standard is good or bad.  The standard might be set too high for the ride to be profitable for the operator or too low to allow people to ride in relative safety, but those determinations cannot be made just on the basis of the percentage of people who meet the standard.

Educational standards are supposedly the sort everyone can meet, while standards for joining the Rockettes or the Navy Seals are intended to be those which only a few can meet.  Both types of standards can be inappropriate for a multitude of reasons.

One of the criticisms of the Common Core State Standards is that some of the grade-level standards are not appropriate to students’ developmental level at that grade. If true (and I think it is), that’s a serious problem.

It is not, however, a problem that’s like to be solved through small-scale experiments. The folks responsible for the overall standards will have to be convinced by seeing lots of data over a few years—with some assistance from experts in child and adolescent development—that the objective needs to be moved to a different grade level.

Assessments are testable

Unlike standards, assessments can and should be tested.  Assessments, however,  are evaluated in terms of how well they measure achievement of the standards.

To a considerable extent, assessments can be tested by small groups of the intended users to get rid of the least valid, least reliable assessments. Of course, if the standards were inappropriate to begin with, the assessments are going to be out of whack, too.

I have some sympathy for teachers who feel they are being forced to work with new standards and assessments without adequate preparation. I’m also willing to grant that first couple years of new standards and new assessments are going to be a pretty tough slog.

However, I believe teachers can work with (and around) new standards and assessments if they put their minds to it.

A workable approach

A District of Columbia ELA teacher who spoke at a webinar I attended recently told about how she implemented the Common Core in her classroom. She chose a few grade-specific standards that she thoroughly agreed with and worked all year teaching in-depth to achieve that standard.

When the standardized test showed her students didn’t do well on those standards, she said that test was not a valid assessment of students’ understanding of those topics.  She knew her students knew that material well.

She did the same things the next year that she’d done the first year.

The second year her students did very well: Between the first and second years, the standardized test was changed so the test items aligned better with the standards.

I suspect that  teachers will find that if they work consistently through the year toward a few of the standards they feel comfortable working with their students to achieve,  do their own assessments to show students’ learning, and not change their teaching to align with a poor test, they’ll be successful, too.

Advertisements

Leave a comment

Filed under Assessment of students

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s