Clutch Shooting Isn't Easy (unless it's a free throw)

Last Thursday night, the Bulls trailed the Thunder at home by 1 point with 4.9 seconds left on the clock. The Bulls' inbounds play was designed to get a shot for Pau Gasol, but Gasol improvised instead with a touch pass to E'Twaun Moore on the perimeter. Moore's three point shot was good and gave the Bulls a 2 point lead with just two seconds to go (cue Benny the Bull).

You don't need any fancy analytics to know that Moore's shot was important. But we can use win probability to quantify just how important that shot was. Moore's three pointer gave the Bulls a win probability of 92.9%. If he had missed, their win probability would have been just 6.4%, a swing of 86.5% between the two outcomes (note: you can find these numbers for any shot by mousing over the leverage bar charts at the bottom of each win probability graph).

The amount of win probability hanging in the balance (i.e. "swing") on any particular shot makes for a simple and intuitive measure of shot importance. What makes Moore's shot all the more remarkable is that three point attempts in those situations tend to miss even more than a "normal" three pointer. Over the past four seasons, there have been 168 three point shots attempted in which at least an 80% win probability swing hinged upon the shot's outcome. Of those 168 shots, only 30 were successful. That is a success rate of just 18%, or just half the 35% success rate for all three point shots.

It turns out this pattern holds pretty broadly for all field goals. The chart below summarizes two point field goal percentage as a function of win probability swing (for the past four NBA seasons). I have bucketed win probability swing into increments of 1%. For example, there have been 26 shot attempts with a potential win probability swing of 40%. 38.5% of those shots were successful. For shots with a swing of 0% (aka "garbage time"), the average field goal percentage is a much higher 49.3%.

There is a clear downward trend in field goal success as a function of importance. I fit a simple logistic regression model to the data, using swing as the sole independent variable. The modeled result is shown as the solid line on the chart.

We see a similar pattern for three point shots as well.

There are several potential explanations for this observed pattern:

Important shots are more likely to be well defended
Somewhat related to #1, defenses may force teams into shots from longer distances when the game is on the line
Psychological. Perhaps the charts above are evidence of "choking" in clutch situations.

With respect to #2, it turns out that shot distance is somewhat correlated with shot importance. So, I ran another regression for two point shot success, this time using both swing and shot distance as independent variables. Both variables were statistically significant, meaning that even after controlling for shot distance, important shots still have a lower success rate.

What about the psychological component? Is there any way to disentangle this from the effects of stronger defense? If only there were a way to remove defense from the equation. Perhaps a situation in which players are allowed to take uncontested shots from a consistent distance.....

The chart below shows how free throw percentage correlates with importance. If "choking" was a real psychological effect and the underlying cause of low percentage shooting in clutch situations, then it stands to reason one would see the same result for free throws.

In contrast to the field goal results, there does not appear to be a decline in free throw success for clutch situations. In fact, if I fit a logistic regression model to the data, it indicates an ever so slight improvement in free throw percentage when the game is on the line, but this effect was not statistically significant. This result is consistent with a similar study of NFL field goal kickers. In that study, Brian Burke found that there appears to be no difference in field goal accuracy between "clutch" and non-"clutch" kicks.

To be fair, there could be selection bias in the free throw results. In close games, teams may be more likely to keep their good free throw shooters on the floor, especially if they hold the lead. As a follow up post, I may try to control for that bias by comparing against each shooter's season free throw average.

One caveat to the free throw results: if you focus on just the extreme situations in which the win probability swing is at least 30%, free throw percentage drops to 64%, compared to a league average of 75%. However, this is not a huge drop and the sample size is only 70. As I add additional seasons to my win probability database, I can rerun this study with additional data to see if this is real or just statistical noise.