Monday, January 11, 2016

Where do Good NFL Coaches Come From?

My daughter is still just an infant, but while we're still at least several years away from having to answer the big mysterious questions she'll have, I wanted to make sure I started to get some answers ready. Little kids are naturally curious about the world and so it's only a matter of time before my baby tugs on my sleeve to ask me a fundamental question about our place on this Earth and how we got here.

Of course, I'm talking about where NFL coaches come from. (If she wants to talk babies, she can ask her mother)

The Eagles Coaching Carnival of 2016 is in full swing with fun and excitement for all ages (yes -- apparently even Tom Coughlin). Thrill rides of hearing potential candidates come out of nowhere only to disappear. Delicious snacks in the form of endless blog hot takes we can mindlessly consume as we go. The horrifying sideshow of people seriously suggesting Pat Shurmur as the answer.

It's definitely a carnival. And what every carnival needs is a cold splash of narrative-destroying reality.

So I wanted to put the question of Where NFL Coaches Come From up for objective analysis. Because as the Jeff Lurie and the Eagles brain-trust debate and the rest of us are left with nothing but tea leaves and private jet flight tracking, it sure seems worth our time.

There were a couple big questions that I was curious to dig into:

1 - Coaches can come from a variety of backgrounds...but do certain backgrounds out-perform others?

2 - When a team replaces its coach, do they over-correct and hire someone with the opposite background of their former coach?

Let's quickly explain both questions before getting into the analysis.

My first question is focused on coach background. Coaches, based on their prior roles, typically come in with expectations attached to their respective resumes. If you were most recently an offensive coordinator, you're expected to focus more attention on that side of the ball. The same goes for defensive coordinators. Then there are former head coaches at both the NFL and college level. All of these candidates are treated as roughly equally legitimate by the media -- but should they be treated that way?

Are there performance patterns in which types of coaches do well and which don't? I haven't seen anyone write about it, so it seemed worthy of investigation.

The second question is a bit more abstract, and it's based on conventional narratives that I've heard. It's the idea that when a coach is fired, NFL owners want to go out and find someone who directly addresses the shortcomings of the prior coach and as a result will get much better results.

That means if you had a defensive head coach and just couldn't win, maybe it's time to try an offensive guy. If you had a college head coach and it didn't work, maybe get an experienced NFL head coach. You get the idea. For anyone following the Eagles, you'll see it attached to Tom Coughlin following as a candidate behind Chip Kelly (lack of NFL experience replaced by TONS of NFL experience). Now technically, this narrative is often used with coaches' specific styles (e.g., players coach vs. stern disciplinarian), but until someone creates an objective measure of style, we're left with background/experience. So that's what I'm focused on here.


I've used coaching records from Pro Football Reference to track coaching changes and performance since 2000. Using this data, we can identify coaching tenure, performance (measured by playoff appearances and super bowls).

But to define a coaches' background, we needed to establish a framework on how to bucket all these guys, most of whom have spent decades in all kinds of weird coaching roles.

Here's how I've broken them down, into four major buckets:

College - Coaches who immediately before being hired in the NFL, were head coaches or coordinators at NCAA programs

Offensive - Coaches who immediately before being hired as NFL head coaches, were in NFL offensive assistant roles (e.g., offensive coordinator, QBs coach, etc.), and have no prior NFL head coaching experience

Defensive - Coaches who immediately before being hired as NFL head coaches, were in NFL DEFENSIVE assistant roles (e.g., defensive coordinator) and have no prior NFL head coaching experience

Head Coaches - Coaches who at any point before being hired, had previously served as an NFL head coach (excluding interim)

The Head Coaching bucket is a bit tricky in the case of some coaches who were Head Coaches at one point, got fired, and then got new head coaching jobs (e.g., Jack Del Rio, who coached the Jaguars, then was a defensive coach after getting fired, and now coaches the Raiders...or Bill Belichick, who coached the Browns, got fired, and was an assistant before taking the Patriots head coaching spot). Because previously running a team as Head Coach is hugely significant and constantly highlighted as a source of valuable experience, I defined anyone who had previously served as an NFL Head Coach in this bucket.

Outside of that, the data seemed fairly clear cut (except Marc Trestman who coached in the CFL...I put him in the College bucket only because his hiring seemed so doesn't move the data much in any case)

All told, the data included 137 different coaching tenures since 2000, each defined by their coaches' background. Coaching tenures are independent from each other, so Belichick's Cleveland tenure is a separate instance from his performance in New England (when we joined the Browns he was a former defensive assistant and when he joined the Patriots he was a former head coach). It's also important to note that seasons before 2000 were included for performance evaluation when a coach stretched from the 1990's into the 2000's.

Results - NFL Coaching Performance by Background

Before we get into how these types of coaches perform, it's interesting to consider the distribution of these types of coaches and how those have changed over time. Below is a graph illustrating the four types of coaches and how their relative frequency has flowed since 2000.

What I think is interesting is how you can spot some slight themes in the shifts. A few more college coaches in the early 2000's, but then a switch back to experienced head coaches. 2008-2011 was a great time to be an offensive coordinator, as college coaches disappeared from the landscape and prior head coaches shrank their share. More recently, college coaches expanded in 2013-2014 but now appear on a downswing again along with the once high-flying offensive coordinators.

This doesn't tell us anything about performance -- but it does suggest that as a league, there are noticeable trends that can be either progressive (college/offense) or more conventional (experienced head coaches).

So with that as a preamble, how did these types of coaches actually do? The table below lays out the coaches by background and compares them on three metrics:

- Average tenure (which is in my view a major indicator of good performance)
- Average number of playoff appearances
- Total number of Super Bowls won

Things don't look that great for the college boys.

That's the thing that jumps out when you look at the table -- the relative strength of both defensive and experienced head coaches relative to college (and a lesser extent offensive NFL assistants). Coaches that jump to the NFL from college have an average tenure 1-1.5 years shorter than coaches who have been either head coaches or defensive assistants before and fewer playoff appearances. Offensive assistants don't fair all that much better, with slightly longer tenure than college but shorter than defensive assistants or experienced coaches and with fewer playoff appearances.

If you consider Super Bowls your only relevant metric, then you really should be in the market for an experienced Head Coach. Consider that list of 11 championships...

Belichick (4) - coached the Browns before the Patriots
Coughlin (2) - coached the Jaguars before the Giants
Shanahan (2) - coached the Raiders before the Broncos
Dungy (1) - coached the Buccaneers before the Colts
Gruden (1) - coached the Raiders before the Buccaneers
Carroll (1) - coached the Patriots before the Seahawks

By the way, NONE of those coaches won Super Bowls at their first head coaching stop (although six other coaches did, Payton, Billick and McCarthy were offensive and Cowher, Tomlin and John Harbaugh were defensive [I remembered Harbuagh as a special teams coach but he also coached the secondary while in Philadelphia, which thankfully saved us from another bucket]).

The college coaches, by comparison, have had far less success than their experienced peers. In fact, only four former college coaches since 2000 have made the playoffs more than once (try to guess, it's a fun trivia question):

Dennis Green - 8 playoff appearances with the Vikings (btw, I had no idea Green was a college coach before he took the Minnesota job in 1992, but he coached at Stanford.)
Tom Coughlin - 4 playoff appearances in his FIRST NFL head coaching gig with the Jaguars. Coughlin coached at Boston College before joining Jacksonville
Steve Mariucci - 4 playoff appearances with the 49ers after coaching at Cal
Jim Harbaugh - 3 playoff appearances with the 49ers after coaching at Stanford

But apart from those four guys, the college veterans have not had a ton of NFL success since 2000. It's a decent list of NFL flameouts, including Nick Saban, Bobby Petrino, Greg Schiano, Steve Spurrier, Lane Kiffin, and now Chip Kelly.

The lesson appears to be, if you're going to hire a college coach, you might want to wait until after they get fired from their first NFL head coach job (unfortunately for Eagles fans).

Results - NFL Coaching Choices

The second question I had was whether team owners/GMs significantly flip their prior strategy when they hire a new coach. Specifically, will they hire someone different than the old regime?

In this case, I looked at the 100+ instances of coaches getting fired and replaced since 2000, and compared the old coaches' background to the newly hired coach. Were they frequently the same? Or were they significantly different?

The table below has the comparisons. It's organized by column, each column contains the distribution of new coaches hired and adds to 100%. As one example, if you look at the column on the far right, in instances where a team fired a coach who had previously been an NFL head coach, they replaced them with a college coach 13% of the time, but replaced them with an offensive assistant 34% of the time.

There are two major themes I can see in this table. The first is that with college vs. pro coaches, there does seem to be a tendency to shift away from more/less NFL experience. You'll notice immediately that no team has ever fired a former college coach and hired ANOTHER former college coach. When a college coach gets fired, teams most frequently look for an experienced NFL head coach (of note for Eagles prognosticators). When an experienced NFL head coach gets fired, that's when teams are most interesting in college coaches (although still low in absolute terms).

That evidence fits a bit with our narrative that NFL owners will go the opposite way from their prior coaches' background, but the other theme I see doesn't fit at all. When NFL teams fire coaches who were formerly offensive or defensive assistants, they are most likely to hire those same types of backgrounds. That means if an NFL team fires it's defense-first head coach, it's actually more likely it'll hire another defensive-coach than an offensive-coordinator to change things up (example: The NY Jets hired Todd Bowles to replace Rex Ryan this past year).

That surprises me a bit, but I'm super excited to feel free to ignore narratives about how firing an offensive coach means a team is now looking for someone with a defense-first approach.

Closing Thoughts

The data certainly implies that college coaches have a rough transition to the NFL, and that once-bitten, NFL owners return to the comfort of prior pro football experience. I can't say I blame them looking at the historical data. But what I also think is interesting is the significant success from second-time head coaches. While there are twice as many instances of defensive/offensive coordinators becoming head coaches, the experienced NFL coaches generally outperform them in tenure, playoff appearances, and super bowls.

Now, a large part of that is likely survivorship bias -- crappy head coaches really don't get second chances, so the ones that do are highly likely to be more talented. But even if that's still suggests that hiring former head coaches that have succeeded in the past might be the best strategy.

If you think I'm coming to that conclusion because I'm trying to talk myself into Tom Coughlin as the Eagles next head coach (because he just interviewed), you might not be wrong.

No comments: