Evidenced Based Medicine (EBM) is the idea that for too long doctors have relied on their keen intuition and experience-informed judgment to diagnose illnesses and prescribe treatments. In many ways, the medical professions see themselves as high priests of the art of medicine, a lot like how some football coaches see themselves as shamans of the mystical craft of football. It turns out that study after study finds that diagnoses are often more accurate when the doctors play the percentages, and not the percentages of conventional wisdom, but the real percentages dictated by statistics and probability. Here's a post by a doctor who sees the similarities between what we do here at Advanced NFL Stats and his profession. I could really re-title this site Evidenced Based Football. It's really the same idea. Of course, we're not saving lives around here. We're doing something far more important--analyzing grown men crashing into each other while wearing crash helmets and silly costumes.Can a run defense be too good? Scaring your opponent into choosing a more optimal strategy mix is self-defeating.

This WSJ article lists which 2010 teams were 'most exciting.' It reminded me of a quick run-down that Peter Keating from ESPN asked me to do a couple months ago. Using the Excitement Index for each game's WP graphs, we can actually calculate which teams had the most exciting and least exciting seasons. The WSJ article isn't far off from the real answers. I'll do a full post on the topic soon.

Sara Holladay's annual ranking of the accuracy of fantasy football projections. As you can tell, there is no one system/analyst that's any better than the others over multiple years or in multiple positions. This should tell you that it's overwhelmingly luck. Whatever you do, do not pay for fantasy football projections. You might as well visit a psychic. The free projections are just as good, and your own projections are probably not far off. We all have the same information available to us. Sara's annual rankings are indispensable.

37 Responses to “Roundup 1/29/11”

To quote the Kobe Bryant article:"And let's be clear: The numbers that doom Bryant's campaign as the king of crunch time are not really statistics. They're not formulas, or algorithms. They're really just counting -- both makes and misses for the player and the team."

Thats a great quote, one that should be brought up more often when talking with statistical skeptics. Most of the things done in the sports stat world are really really simple, like counting. Lets stop calling it "statistics" for a while and just start calling it "counting" and see how people react.

Not related to any of the stories, but I wanted to point something out. I noticed today that armchairanalysis.com has a play-by-play download available for seasons 2000-2009 for free. This seems to be a fully parsed and coded dataset. I presume it's of a high quality because they charge $695 for the 2010 data.

It's 430,000 rows for the play-by-play so you need to be able to use databases if you want to analyse it, but it looks a good download for any budding statisticians out there.

The key phrase in the Steelers' postseason success article is, "If you assume seasons are independent from one another..."

That's a pretty big if. In fact without it, the numbers are pretty meaningless. Unfortunately, it's not as if rosters and coaches are removed and randomly distributed each season. We've also seen enough dynasties and persistent success among elite teams over the years to intuitively know this isn't a good assumption. How unlikely was the 49ers success in the 1980s? The Patriots from 2001 to 2004? Pretty soon it doesn't appear to be as unlikely as they claim. Just because you can make the calculations doesn't always mean you should. It's like in the stock market when 8 standard deviation events seem to happen much more often than they should. Are they really that unlikely, or were the modeling assumptions incorrect?

Regarding the Steelers postseason post, what I liked about it is the author was explicit about the assumptions. He also pointed out that the odds of any single team being so lucky is different than one specific team being so lukcy.

Ben playing the worst game of any QB ever to be bailed out by the rest of his team in a SB doesn't matter, because "he" won. But there were just as many saying "Brady's won three while Manning's choked time after time, and that's all that matters." Seems like football is as much a team sport as bowling.

Jim - people will believe whatever they want. Peyton throws zero INTs and goes for 8.4 NY/A versus the Jets - the 4th best pass defense in the league on NY/A - and takes the team down for a 50 yard FG to take the lead with 53 seconds left. None of that matters to the narrative though. The Colts lost, so it must be because Peyton choked.

It's a bit like how Aaron Rodgers is getting crowned lately. Sure, he started out on fire versus Chicago, but he threw two interceptions and led an offense that, for the final 40 minutes of the game, didn't score a single point. But no, that doesn't fit the narrative, so the Packers won because Rodgers is awesome.

I'm not saying Rodgers isn't good. He's scarily good most of the time, but the NFC Championship game wasn't one of those times. A bit of reality from the press would be nice sometimes.

Granted, Fat Ben's stats take a hit (pun intended) when accounting for sacks. Granted, this is sort of a multiple endpoint argument--going back 3 years or further, Fat Ben's stats are a bit shakier. But a good argument could be made that Fat Ben is just now hitting his prime. To say that he will never stack up to Brady/Manning stats-wise? False, unless you are looking for loser stats like total yards or TD passes. He's got the meaningful stats right now.

The story on "Rodgers' ridiculous rating" says the problem with NFL passer rating is that it's obsolete because the game has changed so much since it was created in 1970 -- but it was even *worse* then. The big problem with it is that it so over-weights completions that a completed pass that loses yards (in any amount!) gets a rating of 79.2. Yet it was designed to set the average rating at 50. In 1971 the league average rating was 59.3, and all but four QBs in the league could have inflated their ratings (most by *a lot*) by dumping off a lot of passes for losses. Since then the average completion rate has risen so much that today only about a quarter of starting QBs could do that. (Only.)

Poor Voros his dips work is the greatest single achievement in sabrmetrics IMHOIt is really sad he hasn't benefitted from it and is also suffering from mental health issues I really hope he stays healthy makes a massive breakthrough on soccer and makes a fortuneHe is an inspiration to allus amateurs who analyse sportsJames

'Evidence-based medicine'? Now, I like the occasional stat in my football (shaken, not stirred), but I'm not sure that current 'evidence-based' medicine (though sometimes an improvement over expert intuition) reveals anything but how unsure one ought to be about football stats.

For instance, guys: want to know if you should ever do ANYTHING about your results on a PSA test? Want to know if you should even HAVE a PSA test, EVER? Go to 'Statistician to the Stars!' Matt Briggs's Decision Calculator and find out.

I am looking forward to the article on the most exicting games. I've always loved the EI and CBF. Although, I have always wanted to see a metric on how close the game was, maybe something like average distance from 50% or something

"The year-to-year instability of batting average on balls in play (BABIP) remains the most difficult concept to fathom. Groundball-to-flyball ratio correlates for pitchers over their careers. BABIP correlates with groundball-to-flyball ratio. Thus, the averages on balls in play should be consistent. Only they're not. Not close."

Jim - people will believe whatever they want... Ian, I hear you. But the NFL and its announcers encourage them. Watching the Pro Bowl today when Matt Ryan came in the announcers said, "When you discuss QBs you can talk about statistics and more statistics forever, but the only number that really matters is wins and Matt Ryan has won 33 games, more than any other QB who..." etc.

So what if the NFL announcers encourage people to believe silly things like "Matt Ryan has won 33 games..." or "the Chicago bears have never come bask from a 7 point deficit in the playoffs" etc etc etc.

I want to understand the game better and so enjoy it more and have an advantage (fantasy, gambling)over those who believe whatever the announcers say.

There seems to be two schools of thought in this type of analysis (and in sabrmetrics) those who want to convert the non-believers and rage against the blasphemies of the annoncers and those who realise that they have discovered the true path and prefer to leave the ignorant masses to burn in the hellfires of ignorance.

Although such religous imagery is probably not appropriate to a pasttime which relies on scepticism and not simply believing everything you are told (including this).

"Jim, So what if the NFL announcers encourage people to believe silly things..."

James, to the extent pro sports is just entertainment it doesn't matter any more for the NFL than it does for the WWE. Fair enough. But IMHO the pro sports community including media and fans is a model for the real world, and one can see the exact same illogic and false beliefs (and I mean in many cases the *exact* same) at work in the real-world communities of investing, business, medicine (as Brian noted above) and worst of all politics -- where they can be really seriously costly -- for the exact same reasons why they exist in the sports world. That's why I'm interested in sabermetrics and football sites such as this. I don't really care about the outcome of any game (I'm not betting) but in understanding how things work and being able to apply the lessons learned here elsewhere.

So when expert analysts -- who after all are paid a lot of money to better inform the amateur public -- push on the public false ideas that they *know* are false (does Bradshaw really believe that all the games the Steelers lost during his career *he* lost?) because the public loves it and it's lucrative for them to do it, it rubs me the wrong way. In medicine this can kill you. In politics, what do we call it? But yeah, maybe I'm too sensitive about it all.

I agree that when a doctor or politician bends or cherry picks the facts to suit their original position that can have devastating effects and I also agree that the public is too uncritical in accepting evidence from the media.

But I think most of them believe they are doing the right thing, e.g. doctors who support homeopathy, right-wing politicians who see climate change as a vast conspiracy. I see these as people with closed minds as they have decided their stance on the issue and no amount of evidence or reasoning will change it as their position probably wasn’t based on evidence in the first place.

I work in a job dealing with science and politics so I see this at first hand on controversial issues with both sides accepting research that supports their position and ridiculing that which does not when in reality all science has some flaws (there is always one more experiment you could do).

But I disagree that Sports commentators DELIBERATELY mislead the audience be saying things they know to be false I think a much more likely scenario is that they haven’t thought about things the way we do and lack the numeracy to do so. I see these people as having open but uncritical minds as they don’t have the ability/experience/need to question the info spouted by authority figures.

One exception might be Ross Tucker on ESPN podcasts who regularly ridicules one of his co-hosts who comes out with ridiculous sounding streaks (e.g. “this is first win by more than 14 points on the road against AFC West oppositon since 2002”) when describing every game.

I think the answer may fall somewhere inbetween...some sports commentators probably know better, most probably don't. I do think that most could be convinced at least half of what they say doesn't make sense.

Just last night, I heard Mark Schlereth talk about how the Steelers have an advantage because of their experienced team which has already won a couple of Super Bowls and how that will help them for preparation this week. But when the game starts and the ball is kicked off, all those advantages go out the window.

Re your hunch about HFA, 2 questions, the first of which is admittedly rhetorical:

1) How do you propose we measure "instinct" and "aggressive" and "unfamiliar?"

2) If you're dubious about officiating being a cause of HFA, to what do you attribute the research findings supporting its impact? Or is your doubt more along the lines of "I think it's a factor, but not the most important factor" rather than "I don't think it's a factor at all?"

1. That's a very tough question. I don't know how to measure instinct, yet certainly instincts exist. Aggressiveness can be measured in many ways. There was a study that measured testosterone levels in home vs visiting players. Familiarity can be measured by how often players played in certain venues.

The "biased officiating" explanation for HFA doesn't seem to square well with the observation that HFA is strongest at the start of the game and diminishes thereafter, across sports. Nor with evidence that the visitors' degree of familiarity with a field affects the size of HFA -- such as the several posts at PFR.com reporting that in the NFL HFA is highest when new stadiums open, in interconference games, etc., and smallest at the shared Giants/Jet stadium because twice as many teams run through there.

Saw your comments in the Birnbaum post. I definitely agree with your implicit point that we shouldn't infer officiating bias from penalty differential. Visitors can have more penalties for a myriad of reasons besides "those refs are in the tank for the home team." I also agree with the basic idea that penalty differential would be caused to some extent by players/teams adopting a higher-risk strategy while losing. However, I have 2 questions about this:

1) It seems circular to offer "the losing team increases their rule-breaking" as an answer to the question, "why do visiting teams lose more often?" Basically you're extrapolating "visiting" from "losing", and then saying, "visiting teams LOSE more often because they're LOSING more often." Which comes first, the losing or the visiting? What explains HFA when visiting teams aren't losing teams?

2) Relatedly, doesn't this assume that teams/players always adopt the "correct" strategy? Isn't that an unrealistic assumption, as evidenced by the constant strategic failures you astutely detail on the site? The high-risk strategy argument only works as an explanation of HFA if teams/players actually adopt that high-rish strategy when losing, no? You know about a million times more than me about game theory, so what are your thoughts about this? Straighten me out if need be.

p.s. I've got a masters in sports psych, so I've got some comments about your discussion of anxiety in the hawks vs. doves post, but I've my corporate time-stealing limit at the moment, so I'll come back with those comments later.

"the losing team increases their rule-breaking" is not offered as an answer to the home-field advantage question. It's meant as a possible explanation for what may be interpreted as "referee bias."

We know visiting teams lose more often. The theory is that losing teams tend to committ more penalties (increase good ol' variance). Thus, it's a proposition that referees call more penalties on visiting teams because they are...committing more penalties (because they are losing). That's how the theory goes, anyway.

Like many other unanswered issues...the answer isn't necessarily polarized - in other words, we could very well be seeing both referee bias as well as more (true) penalties from a losing team. Things are rarely black and white.

1-Yes, the argument is circular. So it's the authors' burden to show that the referee-bias effect exceeds that which we would expect when losing players are penalized more often.

2-A team doesn't have to be losing the game for players to commit more penalties. On any single play in any single game, a player who is 'beat' by his counterpart often has no choice but to commit a foul (for which he is not always penalized). For example, a CB who is beat downfield by a WR or an OL beat by a DE will sometimes (smartly) commit pass interference or holding. Players who are winning individual battles do not need to hold or interfere.

I don't think it's an unfair assumption to say that losing teams try to "bend the rules" or however you want to say it. It's different from playing a higher-risk strategy of going for it on 4th down or something.

Think back to that Dallas-Washington game early this year. The Cowboys were down 6 with one last play. One of the offensive linemen got beat, but rather than let Romo get sacked, he very intentionally held his opponent and a game-tying TD was taken back. Once he was beat, he did the right thing...it's a guaranteed loss if Romo get's sacked. By holding, at least there's the chance the hold isn't called. Extreme example, but extrapolate what you will from that.

just for clarification, when "losing team increases their rule-breaking" is used as an explanation for perceived "perceived referee bias" in the context of a discussion about "perceived referee bias" being an explanation of HFA, then, by definition, "losing team increases their rule-breaking" is being offered as an indirect explanation for HFA. that was my point. i was talking about the indirect effect that Brian's proposing. sorry i wasn't clearer about that.

brian-

when you lay it out that way, it makes much more sense insofar as we can reduce games down to plays. i'm in favor of that, obviously, and think it'd be awfully neat for someone to do an HFA study at the play level (SR anyone)? Given that we were talking about HFA in the context of wins at the game level, I was interpreting your risk-taking idea in a more generalized "take more risks when you're losing the game" kind of way rather than a "take a risk when you get beat on the play, regardless of whether winning or losing at the time" kind of way. thanks for clearing things up.

I don't think I quite follow the logic of that Danny. "Losing team increases their rule breaking" is not being offered as an indirect explanation for HFA. HFA is causing the rule-breaking, because HFA results in visiting teams losing more often which results in increased rule-breaking on their part. What exactly is causing the HFA is still up in the air.

Rule-breaking is an indirect effect of HFA, not explanation of HFA. Maybe we're all just getting caught up in semantics.

my logic is pretty straightforward i think. HFA is the phenomenon whereby home teams win more often than road teams. the research interest is in what causes home teams to win more than road teams (i.e., HFA). that's why people offer explanations for HFA like, "the refs are in the tank for NE," or "SEA's got a 12th man," or "SF had to travel 3000 miles east to play CAR." they're answering the question, "what causes HFA?" not the question, "what does HFA cause?" indeed, the entire reason this discussion thread began was because brian linked to a post about a book that offers a cause of HFA; namely, penalty differential.

so the next question is, "why is there a penalty differential?" stated differently, "what causes the thing that allegedly causes HFA?" brian's theory as i understand it is that visiting players -- by virtue of anxiety -- are more likely to be playing suboptimally, and have to resort to rule-breaking to match the home opponents' optimal play. in turn, this increased propensity for rule-breaking on the part of the visiting team is explaining why we see a penalty differential in favor of the home team. therefore, being a visitor causes more rule-breaking, which causes the penalty differential that helps home teams to win more often.

from a scientific method perspective, it seems like you're putting the cart before the horse. in a causal system, the independent variable has to preceed the dependendent variable in time, obviously. therefore, HFA can't cause rule-breaking as you propose because the rule-breaking occurs prior to the home team winning, not vice versa. except for perhaps the hurricanes of the 80s, teams don't commit penalties after the game's already over.

at bottom, though, i think that you're right about this being a bit semantical, at least in one respect. i think that we're defining HFA in 2 different ways. if we accept the empirical definition of HFA (i.e., mine), then the logic works in the HFA-as-outcome direction. if we define HFA more along the lines of, "well, they're at home, so they have an advantage by default, (i.e., yours)" then the logic can work in the rule-breaking-as-outcome direction. i just don't see, though, how HFA can be exogenous to a causal system representing the HFA phenomenon. it'd be like trying to explain winning by treating the final score as external to the system. successful plays, turnovers, etc. cause the final score, not vice versa. your definition and logic seeem to be implying the vice versa.

But how can you then say HFA does not cause rule-breaking based on the theory? Being the visitor (which happens before anything else) causes sub-optimal play which causes rule-breaking. It's an indirect effect based on this theory.

The independent variable does precede the dependent variable - HFA is established before a game even begins. I think the problem is that you are only granting HFA when the home team wins - "HFA can't cause rule-breaking as you propose because the rule-breaking occurs prior to the home team winning, not vice versa." HFA does occur prior to the home-team winning. Empirical evidence does tell us that being at home grants you an advantage by default.

ok, yeah, this is just semantics. the empirical definition of HFA is "the observation that home teams win more often." you say, "HFA does occur prior to the home team winning." no. being at home is what occurs prior to the home team winning. or, as you say, being the visitor occurs prior to the home team winning. HFA, in the context of empiricial research, is not simply being at home or being away. being at home isn't some magical thing that, in and of itself, confers winning onto a team. after all, assuming that it's not magic is the whole point of studying why it occurs.

But is there really a difference between "being at home" and HFA? Isn't that what HFA is - being at home, regardless of why exactly that gives you an advantage? Obviously HFA is a construct that probably includes many things that factor in to the advantage (whether it's referee bias or familiarity/unfamiliarity with surroundings - whatever it may be), but what we do know -based on empirical observation - is that it confers an a priori advantage. Whatever the advantages are, and wherever they come from, they have increased the probability of a win by the home team. The means by which they increase the probability might be anything (again, referee bias, familiarity, etc.).

HFA--->any number of factors that are caused by being home--->increased probability of winning

and again, this may just entirely be semantics. Maybe you want to replace "HFA" with "being at home" and then just call the whole chain HFA? Might this be a frequentist/Bayesian disagreement? I think we have enough (empirical) knowledge to assume an advantage is conferred upon the home team before kickoff, and every predictive model and vegas line would agree. It doesn't matter if being home is magical or not...the advantage conclusively exists, regardless of an individual outcome.

@BBurkeESPN

ANS COMMUNITY

Interested in publishing your own football research, analysis, or stat-based commentary? Advanced NFL Stats Community is the site to share your thoughts and ideas. There's plenty of data available to get started. All submissions will be accepted and published. Check it out!

Support Military Families

If you enjoy Advanced Football Analytics, please consider a small donation to The Fisher House, a place where families of injured servicemen can stay while visiting their hospitalized heroes.