Game Brain: Profiles: GQ »

This is what prompted Malcolm Gladwell’s response/article on concussions and the NFL.

Vince Young : The Frontal Cortex »

Posted on: November 30, 2009 1:31 PM, by Jonah Lehrer

There was no sentence in How We Decide that I regretted more than this one, which was first written in the fall of 2007, when Vince Young was the starting quarterback for the Tennessee Titans:

Vince Young ended up excelling in the pros.

I was discussing the statistical disconnect between a QB’s score on the Wonderlic intelligence test - an abbreviated version of the IQ test - and their performance in the pros. The league requires that every player in the draft take the Wonderlic. The test is twelve minutes long and consists of 50 questions, which get progressively harder as the test goes along. Here’s an example of a easy Wonderlic question:

“Paper sells for 21 cents per pad. What will four pads cost?”

And here’s a hard Wonderlic question:

“Three individuals form a partnership and agree to divide the profits equally. X invests $9,000, Y invests $7,000, Z invests $4,000. If the profits are $4,800, how much less does X receive than if the profits were divided in proportion to the amount invested?”

The underlying assumption of the Wonderlic test is that players who are better at math and logic problems will make better decisions in the pocket. At first glance, this seems like a reasonable conjecture. No other position in sports requires such extreme cognitive talents. A successful quarterback will need to memorize hundreds of offensive plays and dozens of different defensive formations. They’ll need to spend hours studying game tape of their opponents so that, when they’re on the field, they can put that knowledge to use. In many instances, quarterbacks are even responsible for changing the play at the line of scrimmage. They are like a coach with shoulder pads.

As a result, NFL teams start to get nervous whenever a quarterback scores too far below the average for the position, which is 25. (In comparison, the average score for computer programmers is 28. Janitors, on average, score 15, as do running backs.) But here’s the funny thing: the Wonderlic test, at least for QBs, is famously useless; the NFL is full of quarterbacks who achieved success despite bombing the test. Dan Marino scored 14. Brett Favre’s Wonderlic score was 22, while Randall Cunningham and Terry Bradshaw both scored 15. All of these quarterbacks have been, or will be, inducted into the Hall of Fame. (In recent years, Favre has surpassed many of the passing records once held by Marino, such as most passing yards and touchdowns in a career.) Furthermore, several quarterbacks with unusually high Wonderlic scores⎯players like Alex Smith and Matt Leinart, who both scored above 35 on the test and were top ten picks in the 2005 NFL draft⎯have struggled in the NFL, largely because they make poor decisions on the field.

All of which leads me back to Vince Young, who reportedly scored a 6 on the test. (Subsequent reports indicated that he scored a 16 on his second test.) After How We Decidewent to press, Young behaved somewhat erratically and was benched. I got numerous emails mocking my NFL analysis. But now I’m delighted to say that Mr. Young is back.Yesterday, the QB with the low Wonderlic dismantled the QB with a score of 35:

Almost four years after Vince Young led the Texas Longhorns to a national-championship victory over the Matt Leinart-led USC Trojans, those two college stars — the first two quarterbacks selected in the 2006 draft — finally faced each other again.

Last time, Young scored the winning touchdown on fourth down at the end of the game.

This time?

You guessed it.

Young threw a fourth-down touchdown pass on the game’s final play to lift the Tennessee Titans to a 20-17 victory over the Arizona Cardinals, quarterbacked by Leinart.

The point is absolutely not that the IQ test doesn’t matter. Measures of intelligence remain extremely relevant for predicting academic success, and remain one of the most predictive measures of real world achievement. (Although it’s worth pointing out IQ scores are still remarkably unimportant, and consistently account for less than 5 percent of individual variation in non-academic achievement.) Instead, I think the disassociation between QB performance and IQ performance is a reminder that many tasks are driven by non-cognitive factors, which rely on a very different set of mental skills. Here’s how I describe it in the book:

The reason there is virtually no correlation between the results of the Wonderlic and the success of quarterbacks in the NFL is that finding the open man involves a very different set of decision-making skills than solving an algebra problem. While quarterbacks need to grapple with complexity⎯the typical offensive playbook is several inches thick⎯they don’t make sense of the football field the way they make sense of questions on a multiple-choice exam. The Wonderlic measures a specific kind of thought process, but the best quarterbacks don’t think in the pocket. There isn’t time.

Instead, I’d suggest that QB decision-making requires a surfeit of emotional intelligence - the athletes must develop their unconscious pattern-detecting abilities - which isn’t measured on the Wonderlic. For too long, we’ve pretended that making decisions in the pocket is like doing a math problem, when it’s actually much closer to swinging a bat in a baseball game: the velocity of the game makes deliberate thought impossible. Until we come up with a test that reliably measures the sort of EQ required by QBs, I’d suggest that NFL teams stop taking the Wonderlic into account. Sean Jones, a member of the Oakland Raiders’ personnel department, said it best when asked if Young’s low Wonderlic score might hurt his chances in the NFL draft:

“All I need to know about Vince Young is that he came up with one of the greatest performances ever in the Rose Bowl. In the fourth quarter, I saw (USC coach) Pete Carroll throw every kind of blitz at Vince. I saw Vince read the blitz and beat the blitz. I don’t care what his Wonderlic score is. The only score I care about is 41-38.”

Fourth Down : The Frontal Cortex »

Posted on: November 17, 2009 5:09 PM, by Jonah Lehrer

Bill Belichick has never been the most popular coach in the NFL, but his Sunday night decision to go for it on 4th and 2 on his own 28 with two minutes remaining in the fourth quarter has even his fans crying foul. I bring up this football decision not because I’m interested in a debate - as a Pats fan, the last five minutes of that game were excruciating - but because I think it illustrates the difficulty of making rational decisions, even when the evidence supports the call.

I’ve blogged about the research of UC Berkeley economist David Romer before, but his basic thesis, based on an exhaustive statistical analysis of 4th down scenarios, is that NFL coaches are irrationally risk-averse. They punt the ball way too frequently and kick far too many field goals.

Belichick was an econ major, and has expressed a familiarity with Romer’s research. Nicholas Beaudrot has persuasively shown how, from this econometric perspective, Belichick’s bizarre decision actually makes perfect sense:

On 4th down, with 2 yards or fewer to go, New England has gained a first down on approximately 66% of its attempts with Tom Brady as quarterback. The Colts had one timeout. If the Patriots gain a first down, the game ends; they can slowly walk to burn a few seconds, then take a knee on each down to end the game. If they don’t gain a first down, the Colts would still need to score a touchdown to win the game. Let’s give the Colts a probability P of getting the six if the ball starts at the 28 yard line. So if the Patriots try for the first, their chance of losing is

(Probability of 4th down failure) x 
(Probability of Colts scoring a TD from the 28 Yard line) = 0.33P

The average New England punt nets about 40 yards. Let’s give the Colts a probability Q of scoring a TD on a driving starting at the Indianapolis 32. Then, the chance of the Patriots losing is simply Q. For Belichick’s decision to make sense, we just have to believe that he gave his team a lower chance of losing. In math terms, that would mean 0.33P < Q. Doing some algebra leaves you with P < 3Q. In other words, for the Patriots to have made the right decision, we only have to believe the Colts odds of scoring a TD on a drive starting 28 yards from the end zone are less than three times the odds of the same outcome starting from 68 yards out. The win probability graph for the game suggests that, given 1st-and-10 from New England’s 29, the Colts had roughly a 51% chance of winning in the actual situation. We have to believe that their chances under the punt scenario were above 17% for Belichick to have made a bad good decision. Considering the Colts’ have scored touchdowns on 30% of their offensive possessions, my guess is that this was a good one.

The reason I bring up this analysis is to demonstrate that even defensible decisions can have wrenching emotional consequences. Belichick’s call might have been statistically correct, but it felt horribly wrong.

And this kind of contradiction isn’t just relevant for football coaches. Just consider health care: the only way we’re ever going to reduce medical costs is to restrict procedures that haven’t passed evidence-based efficacy tests. Maybe that means 40 year old women don’t get mammograms, or that we treat prostrate cancer less aggressively, or that we stop performing spinal fusion surgeries. Although there’s solid evidence to question all of these medical options, such changes provoke intense debate. Why? Because our emotions don’t understand statistics. Because when we have back pain we want an MRI. Because when it’s our father with prostate cancer we want the most aggressive possible treatments. And so on.

The point is that there’s often an indefatigable gap between the rigors of cost-benefit analyses and the emotional hunches that drive our decisions. We say we want to follow the evidence, but then the evidence rubs against a bias like loss aversion, and so we make an exception. We’ll follow the evidence next time.

So here’s my cheeky proposal for lowering the cost of health care: Put Belichick in charge of Medicare. Nobody likes him anyways, and he’s clearly able to follow the math even when it feels like a mistake.

PS. Razib addresses a similar issue from a slightly different angle.

Update: Here’s more evidence that Belichick’s decision was eminently rational, and made them 9 percent more likely to win than punting the ball.