Thursday, September 2, 2010

What Does the SAT Actually Measure? (Or 'In Cautious Defense of the SAT')

There's a lot of controversy over what the SAT actually measures -- and, in fact, whether it actually measures anything at all. Although the test was originally conceived as a form of IQ test, the notion that it is actually capable of measuring innate scholastic ability has essentially been debunked, as has the notion that it can successfully predict a student's ultimate success or failure in college (the only thing it has been shown to correlate with is freshman college grades). So the question remains then: if the SAT does not actually measure intelligence or academic potential, what on earth does it actually measure?

My response, thoroughly non-empirical and based strictly on personal experience, would be as follows: the SAT (Critical Reading) measures students' ability to understand, summarize, make simple inferences, and compare arguments from relatively sophisticated texts -- in other words, skills that students at competitive colleges must at minimum possess in order to be successful. After all, if you can't truly understand someone else's argument based on a close reading of specific textual elements, how can you possibly formulate a coherent response to it?

Hence, at the most competitive schools, a high score on the SAT is viewed as a baseline criterion for admission rather than as a remarkable achievement. The test favors the kind of rigorous, precise, linear thinking that the academic world is based on. Academics, like everyone in the world, tend to gravitate toward people who think like they do. And that is precisely the point: the SAT is not intended to measure overall intelligence, at least not anymore. On the contrary, it is designed to measure a particular kind of academic intelligence. Even if university professors must often encourage their students to abandon the kind of formulaic thinking they learned in high school, they nevertheless prize the kind of interpretation and logic skills that tend to lead to a high SAT score.

In no way do I mean to suggest that an SAT score indicates a student's ultimate capacity to acquire these skills (I don't think anyone would disagree that many extremely bright students fail to acquire them because of limited education opportunities or socioeconomic factors.) The SAT only serves to measure these particular skills at a certain moment in time.

It is of course possible to be absolutely brilliant out-of-the-box thinker and do horrendously on the SAT. Einstein, so the line of thinking goes, was a terrible student who probably would have flunked the math portion, and certainly a writer like Joyce would have gotten at best a 6 on the essay because of his unconventional style. But thinking like that misses the point: 99.9% of high school juniors and seniors are neither potential Einsteins nor Joyces. In fact, for the vast majority of them, writing a coherent -- if formulaic -- four or five paragraph essay with perfect grammar, clear transitions, examples that clearly support the thesis, varied but correct sentence structure, and an acknowledgment of potential counter-arguments is in and of itself a major achievement. Academic writing prizes clarity and sophistication of ideas and expression -- most professional academic writing is extraordinarily formulaic; it's the ideas that count, not necessarily the framework in which they are presented -- and an essay that includes those two elements will generally score well regardless of any other factors.

Critics of the SAT often invoke comparisons to the kind of high-school leaving tests required in other countries (British A-Levels; the French Baccalauréat; the German Abitur, etc.), tests that require a high level of composition and understanding of complex ideas, and on this point I agree completely -- ideally (idealistically?), the US would have a similar required exam, one more suited to gauging skills far more profoundly and holistically than the SAT can even begin to claim to do. The US, however, is unlikely to implement such an exam anytime soon.

That leaves us with the SAT. And based on my personal experiences, in the majority of cases (note that I say the majority, not all), SAT scores are in fact a generally accurate reflection of a student's mastery of the underlying skill sets being tested. In other words, students who repeatedly score in the low-to-mid 500's on Critical Reading generally do so not only because they fall into the traps set by the test-makers, but also because often they cannot identify the main point of the majority of passages or even figure out how to determine it, never mind sum it up in a few words. Tone questions are difficult because they do not know how to determine whether an author is agreeing or disagreeing with a particular concept, not just because option (B) was half right and half wrong. Asked to write a summary of the argument presented in a passage or formulate a cogent response to it, most would be equally unable to do so. And a low essay score does not result just because a student fails to plug in the right formula, but rather because the argument is incoherent and the writing ungrammatical and sloppy. In other words, the SAT isn't responsible for creating weaknesses -- it simply reflects them where they already exist. Unfortunately, it is often easier -- not to mention more fashionable -- for students to blame the test than to take a hard look at their actual skills and truly attempt to remedy any glaring weaknesses.

All this can lead to a very dangerous mindset that I've observed in a number of my students: The SAT is stupid and pointless and only tests how well you can take it. Therefore, the skills tested on the SAT are stupid and pointless and I shouldn't bother to learn them. And furthermore, since I don't naturally think the way the SAT wants me to think, learning to think that way would be an insult to my individuality.

Nothing could be further from the truth. While it may not be immediately obvious, the SAT tests fundamental skills of analysis and interpretation that are extraordinarily important. The ability to read a text closely and determine with the greatest precision possible what an author is actually attempting to say, as well as the rhetorical strategies (s)he uses to say it, is an extraordinarily powerful tool. This is not merely a question of analyzing "pointless" literary works, but rather one of viewing any text (advertisement, magazine article, etc.) as a deliberate construction designed to make a particular point by triggering a particular response in the reader or consumer.

So, to sum up: Is the SAT a perfect test? Absolutely not. Is it a great test? Not really. As a crude diagnostic of certain extremely important skills, however, it is remarkably accurate, and for that reason and in the absence of a more comprehensive alternative, it deserves to be taken seriously.

2 comments:

  1. I agree with you. However, what is your thinking about the internatioanl students whose native languages are not English? As an Asian, I struggle with Critical Reading -especially, passage based reading- section. I have difficulty in understanding answer choices; really, it takes a lot of time.

    One more thing, I think critical reading passage is difficult, sometimes, not because of English ability but because of cultural thing. How do you think?

    ReplyDelete
  2. This is a bold opinion, but you make a good point. The SAT is a tricky test though, and is not students who need a test that is more straightforward structurally.

    I actually think that the ACT is a bit more straightforward of a test than the SAT. Here are my reasons:
    http://preppedandpolished.com/the-act-a-candid-test/

    Alexis Avila, Founder of Prepped & Polished

    ReplyDelete