September 28, 2006

Are Leland STAR and API tests scores fair?

Parents, administrators cry foul over science testing methods

By Lorraine Gabbert
Staff Writer

When Leland High School’s STAR and API scores seemed low, many parents asked why. It turns out that the way the students are tested—or not tested—in science and math may have something to do with it.

Leland High School experienced the least growth of Almaden Valley schools on its API score in 2006, dropping from 862 in 2005 to 859 in 2006. “I don’t think the API is a good representation of what happened at Leland,” says Dr. Aaron Buchanan, director of testing and educational accountability. “When I look at the percentage of kids scoring proficient or advanced, Leland increased quite nicely.”

At issue is not only the limited types of tests offered by the state, but also which tests are given to the students.
The state of California offers science tests in biology, life sciences, chemistry, earth science, physics and integrated sciences. However, Leland High School, like other area schools, offers courses in addition to these, including zoology, physiology and conceptual physics.

“I mentioned to my son that Leland scores didn’t seem as high as I expected,” said one mom. “He told me it was because they didn’t have enough science tests. There was definitely some confusion…which may have compromised the accuracy of the scoring.”

According to her son, some students went home at lunch because they didn’t have any tests to take and others took tests for subjects they didn’t have.

“The problem is that our freshman are either in chemistry or conceptual physics, but there is no conceptual physics test for them to take,” says Nancy Dauenhauer, Leland High School assistant principal of guidance, “so there was a mix-up of what test freshmen in conceptual physics were going to take. I thought the person at the district office was going to order us the integrated science test for them, but he thought he wasn’t supposed to.” Eventually, these tests were ordered and given to students at a later date.

“We’re trying to get clarification as to whether or not we were correct in giving the test to our conceptual physics students,” says Dauenhauer. “The integrated science test does have some physics material on it, but it also has biology and chemistry and more, so it’s not really the best test to give. However, the physics test given by the state is really geared towards the junior or senior level. Conceptual physics is more like an introduction to physics.”

Bottom line, she adds, “Science testing is becoming a big problem.”

In addition, in 10th grade, students are required by the state to take a life sciences test although not all the students take biology in their sophomore year.

According to State Testing Director Deb Sigman, schools and school districts are to determine if standard course test blueprints sufficiently match the courses for which there are no specific tests. “However, we don’t encourage any district to have a student sit for an exam for which they have not studied the material,” says Sigman.

“We expect students to take a test for which they have studied the material. If there is not a test that matches the course, we would not expect students to sit for that test. The assessment office has always directed districts not to test students for courses that didn’t match the blueprints. I think there may have been some misunderstanding and that the district may have inappropriately had students sit for tests which weren’t appropriate for them to be taking.”

Leland High School Principal Bob Setterlund is frustrated by this situation. “We’re told that students should take the tests for the classes they’re enrolled in,” he says, “but we offer a lot more classes than these.”

According to Buchanan, this issue affects not only science, but math as well, and any students who aren’t taking standards-based courses. In all these cases, if there are no tests specifically designed for these subjects and if a student is not tested, he receives an arbitrary score of 200 points. “There are penalties that go with not taking a test,” says Buchanan. “It doesn’t affect a student’s individual scores, but it does when the API is calculated. It doesn’t affect the NCLB (No Child Left Behind) tests or adequate yearly progress, but it does affect the Academic Performance Index.” There are additional repercussions, as well. For example, if a ninth grader takes General Math instead of Algebra or Geometry, his score gets bumped down two levels on the API.

Sigman does not consider the 200-point allocation a penalty. “If a student received a score of 200,” she says, “would that bring down the school’s score? I think not. The API is a growth model, so if a student goes into a base with a score of 200 and ends up with a score of 200 the following year, there is no penalty for the district at all, since it’s status quo, so this issue of penalty is not a true penalty from the accountability side.”

However, schools like Leland contend that because a score of 200 is far below basic it reflects poorly on the school.

“In the past, I made our zoology and physiology students take the biology test,” says Dauenhauer, “but then there were questions coming from the district, so last year I sided on the side of caution trying to do the right thing. But, I think we paid a price for it and it doesn’t really reflect the science level of our students.”

“I keep hoping the state will change this, especially in regards to science,” says Buchanan. “There are classes that don’t match up with the standards that are good classes and it’s hard for the schools to know what to do. Not everything fits neatly into earth science, biology, chemistry and physics.”

Setterlund doesn’t believe that students or a school should be punished for the state’s lack of tests in these subjects. “Students shouldn’t be counted at all if there aren’t appropriate tests available for them to take,” he says. “Every course offered is a caliber lab-science class. The state should offer tests in the whole area of the sciences and they don’t.”

Unfortunately, it is not considered cost effective for the state to create additional tests for subjects taken by a relatively small student population. “It’s always great to have tests for kids who are taking a course for incentive,” says Pat McCabe, director of policy and evaluation for the California Department of Education, “but the problem is the cost which has to be weighed. Designing and giving a test costs millions of dollars. If you only have 5,000 kids statewide taking the test, is that worth it? If 1,000 kids take a test that costs $1 million, it would cost $1,000 per kid to take that test.”

“There’s been quite an uproar in the field since we added science to the API,” says Jan Chladek, STAR office manager with the California Department of Education. “Schools feel that if they are given a score of 200 for those students who aren’t being tested, they are being penalized because students could score much higher if they were taking a test. Statistically, people tell me that it evens out and doesn’t make any difference, but schools keep putting their students, like those taking a zoology class, into tests that they weren’t prepared for.”

To address this issue, McCabe has written an item, which will go before the state board in November recommending that students who are not taking a standards-based course should not be given a score at all. “If a student doesn’t take a test in science, he is assigned 200 points,” says McCabe. “We are bringing forth a recommendation to change that procedure.”

According to McCabe, there was a good reason that the 200 points were allocated in 2003. “We were afraid the [schools and] districts would manipulate who takes the tests,” he says, “and only let their very best students take the test. Those scores would then be given 20 percent of the weight on the API. So, we instituted this assignment of 200 so more districts would encourage their students to take more demanding courses and subsequently take the tests.”

However, the methodology of how the API is calculated has changed. Rather than a content-weight basis, the state now calculates how many students are tested. “But,” agrees McCabe, “in the case of having students take the test that they’re not in the courses for, that is something that is not acceptable. If the course title or content is relatively close, then they may take the test, but to have a physics student taking a biology test is not acceptable.”

If the state board passes McCabe’s resolution, it would exempt a student who didn’t take a course from taking another test or having their score counted. “The downside would be if it encouraged districts to have students shy away from these courses,” he says. “We’re afraid that with not having tests in certain areas, school districts will encourage their poorer students to take a course like zoology, where their score won’t be counted, and their better students to take biology and thus take the test. We’re going to be watching that very carefully statewide over the next few years to be sure it doesn’t occur, but we’re worried about it, just as this school is potentially manipulating their test scores by having students take the test without taking the course.”

At this point, questions remain. “There is a general problem with the testing program in the sciences,” says Setterlund, “and we don’t have a complete answer for it.” To address this issue, the staff at Leland High School has been in communication with the California Department of Education and is sending an administrator to Sacramento to work toward a solution.

“It only makes sense,” said Setterlund, “that if you don’t have a test for a subject, and the students are not supposed to take a test, that they don’t get penalized for it.” They are also trying to arrange a meeting with assistant principals of instruction from the 10 highest scoring schools in Santa Clara County. “Five of these schools have scores that went down, and like us don’t understand why,” he says, “so we’re going to get together and compare notes and try to learn from each other.”

Buchanan, as well as Setterlund and Dauenhauer hope that the state will accept McCabe’s proposal to exempt students in other classes from a score of 200. “It would really be terrific if the students taking these other science classes didn’t have their scores counted,” says Dauenhauer, “because it would really help me understand the data and help teachers with direction for school improvement. We’re trying to do the right thing and want to make sure we have a good assessment, but we also want our kids to be tested fairly.”