Influence not as simple as Gladwell would have you believe! - Data Mining: Text Mining, Visualization and Social Media »

The invited talks at ICWSM were especially good this year. I want to highlight a few points from Duncan Watt’s talk and Jon Kleinberg’s talks.

1. Social influence makes the selection for success less predictable. In other words, judged against independent measures of quality, if an audience is influenced by knowledge of community behaviour, it will select or promote with less correlation to quality than you would think. You may think ‘so much for the wisdom of the crowds’ but, of course, WOC is all about aggregating over independent judgments, not socially influenced ones – see Experimental Studies of Inequality and Unpredictability in an Artificial Cultural Market.
2. We know less about our friends than we think we do. In the Friend Sense experiment, it was demonstrated that we project our opinions onto friends about whom we make assumptions regarding political beliefs. Watt’s concerns about the misrepresentation of polarization might be contrasted with the experiments reported in Nick Carr’s book The Big Switch in which a) small preferences lead to deep segregation and b) homophilly leads to extremism.
3. Diffusion of information may ‘long circuit’ the small worlds of social networks. In Kleinberg’s presentation regarding the study of the largest internet chain mail (a petition) he described the role of the threshold model of diffusion in which we require multiple receipts of a stimulus (e.g. a chain mail letter) to pass it on, we are more sensitive to our immediate community – our strong links – than to small-world building weak links. This seems to have some relationship with Watt’s work on Challenging the Influentials Hypothesis and both his criticism of the disease analogy and his focus on the importance of the network structure, not some magical power of the ‘influential’.

Bob Sutton: Testosterone Levels, Top Dogs, and Collective Group Confidence »

My favorite behavioral science website, BPS Research Digest, posted a summary of an amazingly weird and rather troubling psychological experiment.  The upshot is that people —- both men and women —- vary in testosterone levels and (no surprise), when people with high testosterone levels aren’t in leadership positions, “they can find it stressful and uncomfortable when denied the status that they crave.”  A bit more surprising is that the reverse is true as well, that “people low in testosterone find it uncomfortable to be placed in positions of authority.” The main finding from the research is that when groups suffer from “mismatch” between status and testosterone levels (where those with high testosterone levels are placed at the bottom of the pecking order, and those with low levels are placed at the top), the group has less confidence in its abilities get things done.  I quote from the BPS summary:

Michael Zyphur and colleagues assigned 92 groups of between 4 and 7 undergrads to an on-going task that involved meeting twice a week for 12 weeks, and included creating a professional management-training video. Six weeks into the project the researches measured the participants’ testosterone levels via saliva samples. They also asked all members in each group to vote on each others’ status. Then six weeks after that, at the end of the project, the researchers measured each group’s collective efficacy by summing members’ confidence in their group’s ability to succeed.

The key finding was that groups made up of members whose status was out of synch with their testosterone level tended to have the lowest collective efficacy. The researchers think that testosterone-status mismatch within a group probably has a detrimental effect on that group’s collective confidence. However, another possibility, which they acknowledge, is that a lack of group confidence leads to a mismatch between testosterone levels and status among group members.

The implication is fairly horrifying —- perhaps companies will start using testosterone levels to make decisions about whether or not to put people in leadership positions.  Even if it is “evidence-based” (although these results are preliminary), the thought makes me a bit sick.

Here is the reference:

Zyphur, M., Narayanan, J., Koh, G., & Koh, D. (2009). Testosterone–status mismatch lowers collective efficacy in groups: Evidence from a slope-as-predictor multilevel structural equation model. Organizational Behavior and Human Decision Processes, 110 (2), 70-79.

I actually posted about this article before, but liked what Sutton says and how he wrote about it, so voila!

Fourth Down : The Frontal Cortex »

Posted on: November 17, 2009 5:09 PM, by Jonah Lehrer

Bill Belichick has never been the most popular coach in the NFL, but his Sunday night decision to go for it on 4th and 2 on his own 28 with two minutes remaining in the fourth quarter has even his fans crying foul. I bring up this football decision not because I’m interested in a debate - as a Pats fan, the last five minutes of that game were excruciating - but because I think it illustrates the difficulty of making rational decisions, even when the evidence supports the call.

I’ve blogged about the research of UC Berkeley economist David Romer before, but his basic thesis, based on an exhaustive statistical analysis of 4th down scenarios, is that NFL coaches are irrationally risk-averse. They punt the ball way too frequently and kick far too many field goals.

Belichick was an econ major, and has expressed a familiarity with Romer’s research. Nicholas Beaudrot has persuasively shown how, from this econometric perspective, Belichick’s bizarre decision actually makes perfect sense:

On 4th down, with 2 yards or fewer to go, New England has gained a first down on approximately 66% of its attempts with Tom Brady as quarterback. The Colts had one timeout. If the Patriots gain a first down, the game ends; they can slowly walk to burn a few seconds, then take a knee on each down to end the game. If they don’t gain a first down, the Colts would still need to score a touchdown to win the game. Let’s give the Colts a probability P of getting the six if the ball starts at the 28 yard line. So if the Patriots try for the first, their chance of losing is

(Probability of 4th down failure) x 
(Probability of Colts scoring a TD from the 28 Yard line) = 0.33P

The average New England punt nets about 40 yards. Let’s give the Colts a probability Q of scoring a TD on a driving starting at the Indianapolis 32. Then, the chance of the Patriots losing is simply Q. For Belichick’s decision to make sense, we just have to believe that he gave his team a lower chance of losing. In math terms, that would mean 0.33P < Q. Doing some algebra leaves you with P < 3Q. In other words, for the Patriots to have made the right decision, we only have to believe the Colts odds of scoring a TD on a drive starting 28 yards from the end zone are less than three times the odds of the same outcome starting from 68 yards out. The win probability graph for the game suggests that, given 1st-and-10 from New England’s 29, the Colts had roughly a 51% chance of winning in the actual situation. We have to believe that their chances under the punt scenario were above 17% for Belichick to have made a bad good decision. Considering the Colts’ have scored touchdowns on 30% of their offensive possessions, my guess is that this was a good one.

The reason I bring up this analysis is to demonstrate that even defensible decisions can have wrenching emotional consequences. Belichick’s call might have been statistically correct, but it felt horribly wrong.

And this kind of contradiction isn’t just relevant for football coaches. Just consider health care: the only way we’re ever going to reduce medical costs is to restrict procedures that haven’t passed evidence-based efficacy tests. Maybe that means 40 year old women don’t get mammograms, or that we treat prostrate cancer less aggressively, or that we stop performing spinal fusion surgeries. Although there’s solid evidence to question all of these medical options, such changes provoke intense debate. Why? Because our emotions don’t understand statistics. Because when we have back pain we want an MRI. Because when it’s our father with prostate cancer we want the most aggressive possible treatments. And so on.

The point is that there’s often an indefatigable gap between the rigors of cost-benefit analyses and the emotional hunches that drive our decisions. We say we want to follow the evidence, but then the evidence rubs against a bias like loss aversion, and so we make an exception. We’ll follow the evidence next time.

So here’s my cheeky proposal for lowering the cost of health care: Put Belichick in charge of Medicare. Nobody likes him anyways, and he’s clearly able to follow the math even when it feels like a mistake.

PS. Razib addresses a similar issue from a slightly different angle.

Update: Here’s more evidence that Belichick’s decision was eminently rational, and made them 9 percent more likely to win than punting the ball.