Replacement ref “audit” misses the point


The Wall Street Journal recently rattled off a variety of statistics regarding the performance of the replacement officials.  The metrics chosen, however, are grossly superficial, inherently misleading, and in one instance factually incorrect.

(Please, Florio, tell us what you really think.)

First, the “audit” reviews the issue of replay review.  Coaches have thrown the red flag 29 times, an 11 percent increase over last year.  But only 31 percent of the calls have been overturned — down from 52 percent in 2011 and 42 percent in 2010.  The article then claims that a “challenge call sends the play to an upstairs booth, where it’s reviewed by an official who isn’t a replacement.”

With that statement alone, the entire article should be ignored.  Anyone who watches any amount of NFL football knows that, when the red challenge flag is thrown, the decision is made on the field by a referee.  And since the referee is now a replacement, the “audit” ignores the reality that the percentage of calls overturned could be skewed by the fact that the replacement referee who reviews the call may be getting it wrong, too.  (The “audit” also offers no stats at all regarding replays initiated by the booth, a procedure that applies in the final two minutes of each half, after any scoring play, and after any turnover.)

Second, the “audit” says that, on average, the games are six minutes slower.  While explaining that the difference is not “apocalyptic” (what a relief), it ignores the reality that one or two games that last a ridiculously long time make the entire system look bad.  That’s happening this year, all too often.

Third, the “audit” looks at the number of flags thrown.  Through two weeks, the yellow flag has flown 470 times.  It’s a difference of 11 from last year.  (The article doesn’t say whether it’s 11 more or 11 less.)  Throwing around terms like “consistency,” this portion of the “audit” creates the false impression that, if the total number of flags are the same, the replacements must be doing an equivalent job.

Anyone with a functioning brain should be offended by that one.  Comparing raw data on the number of penalties sheds no light on whether the right decisions are being made.  The “audit” tells us nothing about flags that were mistakenly thrown — or about penalties that mistakenly weren’t called.

Fourth, the “audit” claims that the replacements “punish,” with an increase in pass interference and holding calls, along with a spike in personal fouls from four to 21.  (It’s a little hard to believe that through two weeks of the 2011 season there were only four personal fouls called.)  This glosses over the fact that the replacements aren’t calling illegal contact with receivers (a rule that doesn’t exist at lower levels of the sport), and that defensive backs apparently are continuing the forbidden pushing, pulling, and/or shoving after the ball is in the air — which is when pass interference can be called.

Also, the increase in holding penalties could be a product of the fact that holding can be called on every play.  Astute officials call holding only when it occurs in the vicinity of the ball carrier; stuff away from the play doesn’t matter, and rarely is called by the regular officials.  Replacements may be calling it beyond the area of the man with the football.

Fifth, the “audit” claims that the replacements are ignoring certain types of penalties, like illegal shift and illegal man downfield.  Again, this kind of basic “analysis” tells us nothing.  Maybe teams are engaged in fewer illegal shifts.  Or maybe they’re sending fewer illegal linemen down the field.

All told, it was a useless exercise that makes the replacements look better than they really are.  And it’s hard not to wonder whether the Wall Street Journal came up with the various categories on their own, or whether the NFL instigated the exercise.  (Indeed, unless someone at the Journal did all of the counting of penalties by hand, the raw data could have come from only one place.)

Here are the two primary truths that this trumped-up “audit” ignores.  First, the only way to know how the replacements are performing is to review the play-by-play grades that are generated by the league based on a review of coaching tape, and to compare those grade to the grades from the first two weeks of 2011.  After the first week of games, during which the replacements looked the part, acted the part, and sounded the part, the aftermath resulted in multiple sources telling PFT that the average officiating errors per game exceeded 30.  In contrast, the regular officials had average mistakes in the single digits.

Second, the replacement officials in Week Two failed the eyeball test.  So regardless of meaningless stats like those in the Journal article or relevant numbers like the actual grades generated by the league office, when the product coming through the TV looks and sounds different, people notice.

Then again, people still don’t care.  And until the NFL feels a pinch to the bottom line, the NFL won’t care, either.

38 responses to “Replacement ref “audit” misses the point

  1. “Second, the replacement officials in Week Two failed the eyeball test.”

    This is probably the most important point. As my first boss once told me: “It’s better to look right than be right.”

  2. I never understood why in the NFL the replay official in the booth doesn’t decide the play. Wouldn’t we rather have some guy inside staring at a 60 inch screen like I am instead of someone under a hood on the playing field staring at a 4 inch one? They do it in college, the NFL should switch over.

  3. Would anyone honestly expect The Wall Street Journal to be critical of management?

    Any labor can easily be replaced, right? Except the CEO, that is.

  4. Fans need to have a voice, and it should have a financial impact on the NFL in order for their voice to be heard. While it would be ridiculous to think that fans would not attend games in protest over the terrible officiating conducted by the replacement refs, perhaps it might be a good idea for fans to refuse to purchase any NFL products until the replacement refs are dismissed.

  5. Fact is I have football!! Replacement ref’s or not I am watching football. I accept the fact that they will blow calls that may cost my team a win but in the long run I will take a loss for my team as long as I am able to enjoy the sport I love watching! Pile on all you want, personally I do not think they are doing that bad a job considering the pressure and everything!

  6. It’s not the number of flags that’s the issue.

    Nobody watching these games are going, “Well, they’re throwing about the right number of flags.”

    It’s the quality of the calls and non-calls that are an issue.

    If it’s true that the league’s own referee grading system has found the replacements making over 3 times as many mistakes as the regular refs (30 versus 8 or 9), it doesn’t make me feel better knowing that at least the same number of flags are being thrown.

  7. I like that one Redskin, better to ‘look right than be right’, seems many a folks have had a good career living by that creed.

    Nothing will be done until one of the Ref’s is caught taking a bribe to throw the game, IF that happens, we’ll see the real Ref’s quickly, otherwise it’s status quo.

  8. As far as the length of the games go, keep in mind you have to “normalize” that number for commercials. Are we to assume the NFL has not added commercials? NEVER ASSUME.

    It happens.

  9. Enough with the media piling on the replacement officials.
    I could understand if the regular officials were consistant every week, but we all know they weren’t…they were horrible for the most part.
    Not one game has been decided by a bad call. Can we make the same claim with the regular officials?
    I was getting sick watching Tirico on Monday night whining like a baby about the officials. The only humerous thing was he didn’t know the rules himself. He didn’t realize all scoring plays are automatically reviewed. How about we all pile on him for being an unprepared idiot!

  10. It’s gotta be tough realizing not everyone reporting on the situation has an agenda… That said, I’m still waiting for the post-game “the replacement officials were horrible” post, since I know a “the replacement officials were pretty good” post will never happen…

  11. To me they havent been that bad, this is just more of a mob mentality by the media towards something (and of course the masses follow). There have been some on the job training type things like spotting balls, taking too long on replays & not being as authoritative as they should be but overall I dont notice much of a difference. The regulars had too much of an impact in games & made huge costly errors as well….it doesnt seem all that much different to me….

  12. It’s funny how when the announcers don’t constantly bring up the refereeing, it seems to go unnoticed. Last night’s game had good flow and was accurately called. It helped that Nessler and Mayock (both of who I am impartial to) didn’t draw attention to the subject.

  13. “Enough with the media piling on the replacement officials.”

    The media is piling on Goodell and the NFL for the most part. Unfortunately to make their point they have to show the egregious difference between the replacements and the real refs. Many in the media do a fair job of pointing out that the replacements are doing their job to the best of their ability, but that just isn’t good enough. Now there are statistics being provided that show what many people already could see, but not put a number on: The replacements are over 200% worse than the regular refs. Anyone who is saying it isn’t that bad, either isn’t watching a large number of games, doesn’t know much about the game, doesn’t care about the quality of the game they are watching or works for the NFL front office. It IS happening and it will get worse. Blaming the media for blowing it up in this day and age of 24/7 coverage and feedback is like blaming your dog for eating a steak that you left on the kitchen floor. Blaming the players for taking advantage of the refs is a complete cop-out for addressing the real issue. If you are going to blame anyone, blame yourself for continuing to support the NFL and Goodell.

  14. bkostela says:
    Sep 21, 2012 9:35 AM
    It’s funny how when the announcers don’t constantly bring up the refereeing, it seems to go unnoticed. Last night’s game had good flow and was accurately called. It helped that Nessler and Mayock (both of who I am impartial to) didn’t draw attention to the subject.


    Bingo. The announcers have definately been part of the problem so far. Guys would never criticize a player or coach or league exec or even the old refs when they make a bad call feel free to go ballistic on the new refs at every opportunity.

    Take Gruden. The guy has been Johnny Sunshine ever since he entered the broadcast booth. Never say’s a bad word about any player or coach or old referee. He was left with 3 choices during the Bronco’s game :

    1. Criticize Peyton Manning for throwing 3 INTS and talk about how his passes seem to float on the deeper routes.

    2. Criticize John Fox for being unprofessional. Besides yelling at the refs, Fox was pushing players and encouraging his own players to disregard the refs.

    3. Criticize the replacement refs because you assume that the old referees will come back and you’ll never have to see them again.

    He never said a bad word about Fox or Manning the entire game. Fox was a HUGE part as to why that game got out of control. The thing is, if FOX had treated the old officials the same way, the same thing would have happened but he would never do that because he’d have to deal with them game after game, year after year.

    The system only works when everyone acts like adults.

  15. It stands to reason that the replacement refs’ performance will only improve as a function of time. Let’s not swallow hook, line, and sinker this fiction that the locked out refs—and ONLY the locked out refs—can do the job well. I don’t think you don’t need an IQ of 180 to be an NFL officiating crew member. The job requires a thorough familiarity with the “Official Rules of the NFL,” attentiveness, good vision, and good judgment, which is born out of experience. Experience comes with time. And if the locked-out refs are unwilling to accept a 401(k)-style defined-contribution pension plan, then their replacements may end up having all the time they need.

  16. Sadly, I think the only way the NFL and the NFLRA come to an accord is if the NFLPA waves their middle finger at the situation and refuses to play on Sunday. They would have the real officials on the field that week if 104 players refused to come out of the locker room.

  17. Also: I think some of the things coming out of the lockout are good. Like the fact that the offensive and defensive line are playign 1970’s style football, and while I believe that pass interference is still something that ought to be called, the fighting beyond the 5 yards being let go is really making the game interesting and fun to watch. Let the WR’s block, and be blocked, and fought with.

  18. This is the Wall Street Journal who caters to corporate business. Of course they are going to have a bias against a group of workers exercising their right to strike. If Sports Illustrated wrote a piece about the ins and outs of insider trading on wall street I would ignore it.

  19. rhodeislandpatriotsfan says:
    Sep 21, 2012 9:49 AM
    It stands to reason that the replacement refs’ performance will only improve as a function of time. Let’s not swallow hook, line, and sinker this fiction that the locked out refs—and ONLY the locked out refs—can do the job well.


    I’m also not sure why the NCAA is getting a pass on this either. The best choice would’ve been to use the DIV I football refs for the duration, same as they did with the last labor dispute with the refs. This time the division I refs were banned from working NFL games by the NCAA.

    As in most situations, you work with what you have, not what you wish you had. The NFL is working with the best refs available during this lockout.

  20. The replacements are pretty awful. But then again, I think what the regular refs are asking for for a part-time job is pretty ridiculous. The longer this goes on, the better the replacements will get, and the regular refs really will eventually become expendable.

    It’s not ideal, and I’d prefer they reach a deal, but the bad calls, longer games, general confustion, etc don’t exactly ruin the entertainment value. They’re just annoying.

  21. I’ve had little to no problem with the penalty calls. Of course I have a very limited view of what is going on all over the field.

    My problem is the procedural stuff- not knowing when to do clock run-offs, not always knowing from what point a penalty should be marked from, not knowing when a coach can and can’t throw a challenge flag and what to do when they throw one they shouldn’t have, etc.

    Overall, I just think it reflects incredibly poorly on the League’s respect for the game. The issues they are holding out on would cost them little to resolve. This is not a pro-labor stance, just a common sense one.

    Like the lawyer said to Mark Zuckerberg in The Social Network- just settle with them. In the long run, it’s a parking ticket.

  22. Good thing we football fans know not to go to the WSJ for ANYTHING related to football. We still trust FORBES to give us our financial information related to the league.

  23. The only thing that is going to cause the NFL to cave in is an egregious, game-changing referee decision that occurs during a nationally televised game. Absent that, the NFL will be able to continue to argue that doing without the best referees is small beans.

  24. I’m also not sure why the NCAA is getting a pass on this either.


    Eagleswin IIRC the guy who runs the NCAA officials is a locked out NFL official (remember, all those officials have other jobs). So when it came time to find the “best” replacements, he obviously prevented that from happening.

  25. They are definitely not quite as good as the regulars, and over the course of the season will probably make a few more errors (note the regulars make PLENTY of errors themselves), but I won’t find myself screaming at the screen any more about them than I ever have. I have no problem with them at all.

    And yes, I love the notion of another union being put in their place for thinking a group of employees can extort compensation from a company by threatening to screw the business and ultimately me the consumer. Unions have no place in America any longer. There are so many labor laws in existence now that the unions only contribution is creating discourse between the employee and management.

  26. prospero63 says:
    Sep 21, 2012 10:12 AM

    Eagleswin IIRC the guy who runs the NCAA officials is a locked out NFL official (remember, all those officials have other jobs). So when it came time to find the “best” replacements, he obviously prevented that from happening.


    I did not know that. It seems like that would be a conflict of interest. He’s acting in his personal best interest rather than in the best interest of his NCAA officials.

    We get huge articles on here about the new refs other jobs but we never hear about the old refs other jobs. It would’ve been nice to see a huge article on the front page talking about this old ref’s conflict of interest.

  27. I’m also not sure why the NCAA is getting a pass on this either.


    Eagleswin IIRC the guy who runs the NCAA officials is a locked out NFL official (remember, all those officials have other jobs). So when it came time to find the “best” replacements, he obviously prevented that from happening.


    Exactly. And on top of that, the NCAA doesn’t want to have to deal with all the media/coach/player/fan backlash created by their own replacements. It’s the NFL’s problem (which I think is less a problem than the media would like it to be), why would the NCAA want to effectively take the bullet for them?

  28. To hell with the regular officials. When the NFL finds out which of the replacement crews are the best they’ll be in the national games and the majority of people won’t see a difference. These replacement guys are getting better. It hasn’t stopped me from watching hours and hours of football so lets just tell the regular refs don’t bother coming back and move on!!

  29. One has to remember that the Wall Street Journal is owned by the Murdoch Empire – thus, the Fox News of journalism:)

  30. “The replacements are over 200% worse than the regular refs”.

    That’s a nice number to throw out there, but can you back it up?
    Didn’t think so.
    You’d have to go back and review every call made by every regular official and compare it to every call made by the replacements. Then take into account whether it was an obvious call or not.
    Good luck.
    I’ll go by what I see and what I clearly remember about the regular officials. They were terrible.

  31. I think that we are all forgetting how bad the regular refs are. When I see a bad call or non-call against my team I wonder would the “real” refs have got it right? And the answer is generally “I doubt it.”

Leave a Reply

You must be logged in to leave a comment. Not a member? Register now!