A WPA-Based Reliever Rating System

Win Probability Added (WPA) is a fun stat. It is also very useful, especially when one wants to classify things that are otherwise difficult to quantify in a single number, like a reliever’s effectiveness in a given game.

(If you are new to WPA, please read this primer

on FanGraphs. The premise is really pretty simple.)

We all know that Saves are silly. Consequently, many folks have attempted to create other stats that better rate relievers’ performance. Many of these stats are useful–first batters retired, inherited runners stranded, and the like–but the ones that do the best job, in my opinion, are those that utilize WPA. For instance, FanGraphs has introduced “Shutdowns” and “Meltdowns” in an attempt to replace saves/holds and blown saves with something a bit more useful. Shutdowns are defined as relief outings with a WPA > .06 and Meltdowns are those with a a WPA < -.06.

I like Shutdowns and Meltdowns, and you should definitely use them (see a leaderboard here

; Shutdowns and Meltdowns are “SD” and “MD,” respectively). However, I also like a bit more granularity in my stats, so I’ve added some extra WPA-based categories:

• A “Good” game is any with a WPA of at least .01 (i.e., increasing the team’s win chances by at least 1%)
• A “Bad” game has a WPA of -.01 or less
• A “Neutral” game is anything in between “Good” and “Bad” (a win probability change of less than 1% in either direction)
• An “Excellent” game has a WPA of at least .1 (10%)
• A “Terrible” game has a WPA of -.1 or less

All relief appearances fall into one of the first 3 categories. The last two are like super-sized versions of “Shutdowns” and “Meltdowns”. Sorry for the uncreative names; perhaps I’ll rename the categories later.

To rate a reliever, we can look at a variety of things: the % of Good games (or of bad games), a ratio of Good games to Bad games, or a differential between either Good and Bad games or Excellent and Terrible games. Let’s start with the percentages. Here are the 10 relievers with the highest percentage of “Good” games (out of all games, minimum of 10 appearances):

For reference, the league average Good% is 53.7%, so all of these guys have exceeded the league average by quite a bit. Three of these guys aren’t even closers (Venters, Adams, and Clippard), and 2 of the closers didn’t have that job at the start of the season (Santos and Madson). The Padres, with both Bell and Adams, have perhaps the best 1-2 punch in baseball.

Next up, here are the top 10 players in the ratio of Good games to Bad games (“Good/Bad Ratio”):

The league-average Good/Bad Ratio is 2.36.

Neutral games don’t count in this ratio, so the standings are a bit different. Probably the biggest surprise is Jason Isringhausen; coming into this year, many people thought his career was over. He made the Mets, though, and he’s done very well so far, with 9 Good outings and 3 Neutral ones in his 12 appearances. Including Isringhausen, Matt Lindstrom, and Grant Balfour, 5 of the 12 players are non-closers.

Now, let’s raise the bar a little and only consider “Excellent” games. Who has the highest percentage of games that meet that standard (WPA of at least .1)?

The league average Excellent% is 17.1%.

Now this is an interesting mix. New additions include the flamethrowing young closer Jordan Walden and two veteran on-again-off-again closers, Matt Capps and J.J. Putz. The only two pitchers to appear on all 3 leaderboards are Heath Bell and Sergio Santos.

I’ll have more with these classifications throughout the season.