ESPN’s QB special undermines QB stat


ESPN unveiled its new Total Quarterback Rating statistic with a TV special on Friday night, and the odd thing about it is that the people featured on the special didn’t seem to buy into the stat.

Although ESPN wants fans to believe that Total QBR is a big step forward in statistical analysis of the quarterback position, the special started with Monday Night Football analysts Jon Gruden and Ron Jaworski both expressing skepticism.

“Statistics don’t measure courage, they don’t measure leadership, they don’t measure poise,” Gruden said. “Football is the ultimate team game. On every play, there’s 11 guys involved, a coaching staff, and I think the quarterback gets way too much credit when things go good and way too much credit when things go bad. So I’m real skeptical, as I always have been, of statistics.

Added Jaworski, “I believe there’s only one statistic that is important, and that’s winning.”

Once ESPN put those caveats aside and explained the stat itself, it did sound interesting: Total QBR incorporates aspects of passing that aren’t part of the traditional passer rating, like yards after catch and dropped passes, and it also includes things like running, sacks and fumbles, none of which are part of the NFL’s official passer rating. If you enjoy the kind of work that Bill James has done in baseball, you’ll probably want to check Total QBR out.

But you’ll also probably find that baseball, with its one-on-one, pitcher vs. batter nature, lends itself to analyzing individual players in a way that football doesn’t. I think Total QBR does a good job of telling us how successful a quarterback has been, but I question whether it can really tell us how much of that quarterback’s success is his own, and how much of it is the result of his offensive line and his receivers.

I have my own quibbles with Total QBR, and they’re not the same as Gruden’s and Jaworski’s: I think the “clutch index” aspect of Total QBR seems ill-defined, and I think it was a mistake for Total QBR to treat all opposing defenses the same, as if completing a pass with Darrelle Revis in coverage is the same as completing a pass against some third-string scrub cornerback.

Total QBR is probably a better measurement of a quarterback’s play than the standard passer rating, and that makes it a step forward. But it has some real flaws, which even ESPN’s own special to promote the stat couldn’t hide.

36 responses to “ESPN’s QB special undermines QB stat

  1. The best quarterback rating is wins. Throughout the history of football the quarterback is responsible for wins more than anyone else including the head coach. Trent Dilfer wasn’t the greatest quarterback but if Chad Henne or Alex Smith or Tavaris Jackson was the quarterback for the Ravens that year they wouldn’t have won the SuperBowl.

  2. Fundamentally, football is not a precise game anyhow. What does every play end with? Spotting the ball, which is nothing more than a point estimate.

  3. What Jaws says about winning I hear a lot of other people, including QBs say also. The thing is that to me, Ws and Ls are like SB rings…how many you “have” isn’t as important as what YOU PERSONALLY did to help or hurt your team in obtaining them.

    It’s sort of a basebore pitcher mentality to assign W-L to QBs anyway which is another reason I loathe that media predilection. As is mentioned, basebore is far more of an individual game than football. In football, the ONLY individual who should sport a W-L record next to their name is a head coach. Otherwise, it’s a franchise statistic, not an individual one.

    Stats can be interesting and relevant but I think they always need the context of someone having watched the games to try to match them up with the eyeball test.

  4. I read what goes into the rating . I seen who the top rated QB’s from last season was . Its not that big of a deal to me . Just more jocking of Brady Manning Rodgers ETC . The one thing i didnt like about the rating is why are sacks being counted as a stat for throwing ? A qb is already effected by bad oline play . He gets sacked alot . But this new system penalizes him for something he cant control and thats his protection .

  5. QBR is an incremental improvement in evaluating the play of QB’s. Passer Rating was always flawed for many reasons but it served it’s purpose until a better alternative emerged as it has now.

    The one real problem with QBR is that it’s not readily accessible to the average fan because so much of the rating is subject to changing conditions in the game state.

    Still it’s an enormous improvement in measuring something that is probably impossible to completely tack down.

  6. There really isn’t a way to effectively measure QBs against each other without putting them all in the exact same situations, don’t care how many thousands of lines of code you have.

  7. I don’t think any stat line fully captures athletic performance but especially one with so many decisions. From what play to even run or audible to, to throw or not throw, to which receiver, what teammates you have and advantages and disadvates they give you … they’ll never fully “get it”. I even think baseball stats are flawed ….. for years Bobby Abreu hits 3rd and around .300 …. but in certain situations where you’ve got a guy on 2nd or 3rd and you’re being “edge-pitched” … you make the decision to swing and not to walk … and yeah, you give up “stats” but you cover the better matchup, take a chance on your skills and leadership vs. leaving the control up to the opposition. Combination of skills and decision making and I’m not sure they’ll ever do the job necessary to get the decision making part! Let alone the variable of what teammates you have and what conditions they present!

  8. More of the same old crap, big media trying to cram b***sh** at us and calling it roses. If you believe that ESPN doesn’t have an agenda behind this move, I got some ocean front property in arizona I sell you cheap!

  9. I think overall, like the article said, it is an improvement, but not perfect. My biggest issue is that poor OL play affects QB perfromance negatively…if anything they should get credit for performing in spite of a weak OL.

  10. As far as I’m concerned, you can evaluate a QB by three easily observable criteria:

    Third Down, Red Zone and Fourth Quarter of close, winnable games.

  11. I saw where QBR won’t penalize quarterbacks for hail-mary interceptions. Then does the quarterback get credit for one that is complete? If it bounces off three players hands and is miraculously caught for a touchdown, is that a measure of the quarterback’s skill? I don’t think so. They also want to discount tipped passes that result in interception. What about tipped passes that result in a completion? Sacks? Is it a lineman’s fault? A coverage sack? Now you are talking about the subjective opinion someone sitting at a computer looking at tape. I am sure there is a thick manuscript that explains it all, like the tax code. I think they should have just incorporated rushing into the passer rating and left it at that.

  12. New rating = total nonsense. The individual plays are all evaluated, but they are weighted on how close the game is (score differential)….so if a QB has a lousy defense, his rating would be higher for doing the exact same thing as opposed to if he had a shutdown defense which did not allow the other team to score. Total crap.

  13. My biggest problem with the ESPN special is that I still have no idea what the formula actually is. Sure I know most of the parts and the types of stats that go into calculating it, but I still don’t know the actual formula. Passer rating may be complicated, but at least I could figure it out with a pad and pencil (or an excel spreadsheet or an online calculator on Google.) As far as I know, ESPN could just be making up numbers.

    I would have liked to see at least one example, even if it was just a single game and not a season, going step by step and explaining how the final number is achieved.

  14. QBR is an absolute load of nonsense

    It is all subjective, they said themselves that they’ll add/deduct points based on how THEY felt the QB performed in a situaton. They’re just making up numbers to put into their rating

    While the tradition rating is flawed, it’d be far better if it simply included rushing yards/TDs

  15. The worst part of this thing is that we will now be force-fed it by ESPN and will be sick of it by November.

  16. Michael Smith, your argument is the quality of the opponent? Ridiculous. Even a baseball hitters batting average does not care about the quality of the pitcher, idiot. If a guy is batting .300, no one asks for is BA against aces, 2nd or 3rd rotation guys. It’s not readily available in common stats. This is a stupid argument that we need to know whether the QB’s rating was against the no. 1 defense or no. 32. No one cares about that in the end.

  17. How about the obvious consideration, do we really need another stat number to tell us who is better than the next guy? Of course not! Fans can figure out who stinks and who doesn’t without some abstract number. The football talking heads are slaves to stats because they need something, anything to talk about when they are on the air and stroking players.

    The QB rating systems are all useless compared to looking at the basic composite numbers like completion %, yds/attempt, and third down completion%.

    Yards per game? That’s a terrible number to judge by. Td’s/ints on the other hand is really a good indicator imo.

  18. Has anyone applied the new QBR system side by side with the old passer rating to the QB stats for say…2010, Mike?

    Be interesting to see what they look like.

  19. @woodg8

    Sorry, woody…even that won’t make Mike Vick #1.

    Instead, maybe they need to add in games lost due to injury. :laughing:

  20. The QBR is not perfect, but it is certainly better than the antiquated rating system in place. I argue that the QB position is evolving and that new elements to successful quarterbacking must be measured to properly evaluate. It’s almost as if the 158.3 perfect rating system, was designed, just so everyday fans, could not possibly decipher the calculations.

    Over time, with tweaks, I think the QBR will be OK. I welcome the attempt.

  21. There are too many variables for any QB stat to be truly valid. O-line, defensive scheme, play called, weather, field conditions, receivers, routes, DB ability, down/distance, situation within the game, etc. The criteria is subjective and can be interpreted differently and that will skew the data. If the data can be skewed, it can be manipulated to an intended result.

  22. I didn’t like the old QB rating and I don’t like this one. The first one didn’t account for certain aspects, this new one seems to penalize QBs for things they can not control, also saw that it penalizes for YAC, well how the QB places the ball is important to the WR gaining YAC. That’s just one example. You know how I rate how good a QB did? I do this weird thing called “watch” the game

  23. BS! There is no way Manning has a better passer rating than Brady.

    Manning threw a pic in the SB. Why wouldn’t they deduct points from him for that? Not only that, he didn’t play well in the SB that they did win.

    Might have known that ESPN would come up with a system that would put Manning ahead of Brady.

    More Patriot hatred.

  24. @zidanevalor

    Great comment. Maybe someone could make a QB rating that anyone could calculate like you said, in a similar fashion. Take the passer rating and add the important components that are missing like 3rd Down, Red Zone, 4th Quarter, etc. and somehow add in rushing the ball.

  25. iminyrhead says:Aug 6, 2011 11:34 AM ESPN has become EXTREMELY anoying! The only stats that matter are WINS and Losses … PERIOD!!………………………… you can say that again…espn out of pocket …. they create drama than say it wasent me…meaning in bizz of upseting fans. with bold nonsense ..they promote players that they like or dic eat…

  26. lets get serious and focus on what we are discussing. Wind and losses are the only thing that matters?
    Is that how you will draft your QB? Then Tebow should have been the #1 QB in the draft.

    Courage and leadership are measured ij the performance of your teammates and therefore is measured in this new ESPN stat. Is it perfect? Hell no.

    Does it measure a. completion against Revis? Are you kidding me? Does ERA measure against pitching to Fielder? Come on now.

    PFT is becoming a bitching session..of which I am bitching about. 😉

  27. No Gruden fan here, but what he said is very true. Example Danny White may have great stats, but he has no SB rings, while Staubach did. He was more poised and a better leader.

  28. Total QBR is a joke…if you have read how it is configured then you can instantly see some of the ridiculousness that goes into it. How far the ball travels in the air? Is that important? No. But at least Gruden didn’t sound like a moron when expressing his opinion like Jaws did. In Jaws’ eyes Trent Dilfer must be better than Dan Marino. Or Vince Young must be better than Kyle Orton. I mean they are “winners” in Jaws’ words, while the other two fell/fall short. He’s a completely ignorant df that doesn’t need to talk football ever again. How someone can be sooo bad at their job yet still keep it is beyond me, he is the equivalent of Freddie Mitchell. Winning is a TEAM game, a great defense, great run game, great OL (any combination of those) will go a long way in making a bad QB look good. And vis-a-versa.

  29. “But you’ll also probably find that baseball, with its one-on-one, pitcher vs. batter nature, lends itself to analyzing individual players in a way that football doesn’t.”

    BINGO. What I’ve been saying ever since ESPN got crazy with football stats about 20 years ago.

    The fixation with stats is a baseball thing, and does not translate to football as well, as much as a nation of bean-counter sports fans wants to use stats to justify their views. Stats in football are a guideline, then you have to look at the context that produced those stats. Stats are much less subjective in baseball.

  30. They also have not changed the rules in baseball to hand the advantage to the offense, as the NFL did some years ago, or increase the length of the season by 15%.

  31. ok, correction. They did expand the mlb season from 154-162, but that is not 15%. The main argument is the defensive rules changes in football absolutely changed the game of football.

Leave a Reply

You must be logged in to leave a comment. Not a member? Register now!

This site uses Akismet to reduce spam. Learn how your comment data is processed.