Sunday, April 24, 2016

Should Rookie QBs Start Right Away?

Anyone remember an Eagles off-season where they didn't make headlines?

Howie Roseman and company just set off a massive trade bomb, exchanging a bounty of current and future draft picks with the Cleveland Browns in exchange for the second overall pick in the draft -- which the team will use on a quarterback.

Now let's ignore which of the top quarterback prospects the Eagles are going to take -- and let's focus on the choice the Eagles will have to make after they draft in the highest slot since they took that guy from Syracuse.

Should they start the rookie?

Now we know what Roseman has indicated they'll do. It appears the Eagles will keep Sam Bradford and Chase Daniel and let them be the primary quarterbacks this year (Sam Bradford's rumored trade demand notwithstanding).

But that's not our question -- our question is what SHOULD the Eagles do with their newly drafted first round QB?

I was having this debate over email with friends -- is it better to start a rookie QB right away or to sit him for a year. We were talking pros and cons of the two approaches, but kept reaching for standard QB narrative tropes -- that sitting would let him learn, avoid bad habits, etc.

We've heard all these things before. There are people who would say that rookies really benefit from time to 'learn the speed of the NFL game.' There are others who would argue the best way to learn the game is to actually play it. Both the sit and wait camp (look what happened to David Carr) and the start them right away (Andrew Luck is good!) have tons on anecdotes to throw around.

But no one really looked at the data -- and some brief google searches didn't scratch my itch -- so I thought we should fix that.

I pulled together data from all first round quarterbacks drafted since 2000, and looked across each of them to identify the first year they truly became 'starting quarterback.' That distinction is based on playing time (if you started most of your team's games) and media reports on when quarterbacks were named starter going into the season.

We have have two groups -- guys who started immediately and guys who had to wait. With that distinction, we can compare how they performed and see if either group is distinctly outperforming the other. The implications could suggest how the Eagles SHOULD handle their new QB of the future.


We have 42 quarterbacks in our sample -- and the split is roughly 60/40 -- 24 of those quarterbacks started immediately and the remaining 18 took over the job some year(s) after their rookie season. Below is a table of our QBs and how they were defined:

I pulled in a whole slew of performance metrics compared the two groups. And what's key in comparing the two groups of QBs to each other is lining up their first starting years with each other. It makes no sense to compare the rookie years of both groups when one didn't play. So in our analysis, you'll see I aligned all the quarterbacks around their respective First Starting Year - the first season where they were their team's starting QB.

If you'd argue that sitting a rookie quarterback for a year or more to acclimate to the NFL would be beneficial, then we should see some outperformance either in that First Starting Year or in their career performance trajectory.

Below are illustrations of average performance of those QBs who started immediately after being drafted (orange) and those who had to wait (blue). There are three charts, the first is using QB rating, the second is using Passing Yards per Attempt, and the third is using Pro-Football-Reference's Approximate Value metric. See if you notice a pattern.

In looking across all these metrics -- I wasn't able to identify any clear outperformance by quarterbacks who waited to get their job. Across all the variables, the guys who wait extra year(s) perform pretty much just as well as the guys who start right away.

Also - you don't see any significant trajectory changes over time - by that I mean the guys who wait don't get any kind of long-term performance advantage in doing so.

The only difference I could really see was in their ages (obviously). QBs starting immediately are on average, 22.8 years old. QBs who wait, they're 24.1 when they get the job. It just doesn't appear as though there's any difference beyond a slight age gap.


This data would suggest that all that time riding the pine isn't really worth it for the QB's ultimate performance. They sit, they go to meetings and practices, but once they actually start they do just about as well as the guys who are thrown right into the fire.

Now, if we were in college and debating the merit of red-shirting freshmen quarterbacks, there's really not much downside to having the prospect wait. College players have a defined window of eligibility and cannot play beyond it. But NFL rookie QBs have much longer potential tenures, and they start their careers on fixed contract terms with clocks that tick even when they don't play.

If you're a college coach, redshirting a quarterback means you'll still get four years of performance, you just need to wait one year.

If you're an NFL GM/coach and you have a first round rookie quarterback, that quarterback is on a fixed four-year contract (five with a team option). If you bench that quarterback for a year, that contact becomes a three-year deal with an option. Benching him does come at a cost - because that's one less year on his rookie deal.

If you keep a quarterback on the bench for a year and he turns out to be a superstar, well then you've wasted a year of his term. If he's a bust, then it's kind of a moot point because you're probably going to lose your job regardless.

What's particularly interesting is that the NFL may have figured this out.

Below is an illustration of all the QBs in our sample by draft year. They're split across our two buckets, those who waited to start and those who started immediately. See if anything jumps out at you.

The practice of benching a rookie first round quarterback for a year or more to learn the ropes has declined noticeably since 2000. Before 2008, only one-third of first round QBs started right away. Since 2008? About 80% of them have started immediately.

It's now much more rare to see a QB drafted in the first round and benched for a year. The only one drafted in the last few years who wasn't made starter on Day 1?

Johnny Manziel.

While it's hard to identify a specific driver for the shift in team behavior -- it may be that teams have looked at the data and come to the same conclusion -- that you don't gain very much by waiting.

So what does that mean for the Eagles?

Well, data suggests there's likely no performance benefit to sitting the new quarterback on the bench a year. We could debate that point if that quarterback is Carson Wentz, who lacks playing experience -- but I doubt he's a special exception to the overall trend.

If there's no real benefit to benching him, then what you SHOULD do is look to move Sam Bradford. While there are some sunk financial costs from the new deal you just signed him to -- moving him might be a way to recoup some of the draft capital you burnt to move up in the draft. That's probably what I'd do.

Some might argue that Bradford gives you the best chance to win now and that in a weakened NFC East you should definitely take that chance. My counter to that would be as follows:

- You've just spent a ton of capital to acquire a QB, and the data suggests keeping him on the bench won't make him that much better
- If the new quarterback can't be ready to play right away -- why is he worth giving up so much in draft assets
- Sam Bradford is not a Super Bowl caliber quarterback - demonstrated by his entire career to date - so what's the point in continuing to run him out there if the ceiling is limited?

It doesn't appear that the Eagles are going to go this route -- all indications are Bradford will be the starter and the Eagles new quarterback will sit and wait. Just know that the evidence suggests an extra year of waiting won't make much of a difference.

Wednesday, April 20, 2016

Quick Takes on a Massive Eagles Trade

Last night all of Eagles Twitter was, well, atwitter, with rumors of the team trading up for the Browns pick in the NFL draft. The Eagles would move up from #8 overall, which they acquired from Miami for Kiko Alonso and Byron Maxwell, for #2 overall. Such a move would put them in a spot to guarantee the ability to draft one of the two top quarterback prospects -- Jared Goff or Carson Wentz.

I was really hoping the rumors were false, and then, sure enough, news broke just a few hours ago that the Eagles and Browns made the trade.

The Eagles acquire the 2nd overall pick and an additional conditional fourth round pick next year in exchange for pick #8, 3rd and 4th round picks this year, next year's first round pick, 2018's second round pick.

That's a mouthful. But basically, the Eagles gave up a ton of picks to move to #2. So, here are a few quick thoughts:

- I hate this trade: I don't believe trading up is a successful strategy in NFL drafting, and the evidence seems to back me up. There's just too much uncertainty around the success of any individual football player to put all your eggs in one basket. Just go back and look at the data, rare is the example that works out for the team moving up. I'm not going to re-litigate the issue, but Bill Barnwell recently posted an article on ESPN making exactly the same point analysts have made for years. Draft picks are lottery tickets, you should always want more.

- Just because I hate it, doesn't mean it will work out: Look, given all that, I'm still an Eagles fan, and I'd much rather this trade become the exception that teams will cite for years and years as the time moving up for a QB worked out. The one counterargument I can think of to my 'never trade up' philosophy is when it comes to the game's most valuable position.

My last analysis looked at where QB value is generated, and a majority comes from first round draft picks (one of my recent posts, link on the right). Also, that share of value has been progressively shifting towards first round draft picks over time.

Basically, the place you really need to look for a stud quarterback is early in the draft. And if you believe that, then moving up to take the best prospect seems a little less insane

- The odds say this won't work: I don't want to be a Debbie Downer...but drafting QBs is a low-odds game even when it's at the high end of the first round. Here's a list of QBs drafted in the top 10 since 2000:

Jameis Winston
Marcus Mariota
Blake Bortles
Andrew Luck
Robert Griffin
Ryan Tannehill
Cam Newton
Jake Locker
Blaine Gabbert
Sam Bradford
Matthew Stafford
Mark Sanchez
Matt Ryan
JaMarcus Russell
Vince Young
Matt Leinart
Alex Smith
Eli Manning
Philip Rivers
Carson Palmer
Byron Leftwich
David Carr
Joey Harrington
Michael Vick

Not exactly a murderer's row. Go through that list and count the number of teams that would do the same thing all over again if they had the choice. If you hold out the newest guys (Winston, Mariota, Bortles) I think you come up with 7 of 21. I'd guess that Luck, Newton, Stafford, Ryan, Manning, Rivers, and Palmer would all be repeated. And if Wentz/Goff turns out to be in that category then great for the Eagles...but the data would suggest there's a 66% chance at being wrong. Now, you could certainly argue that Bradford hasn't worked out and there was no other way to get a QB - but let's not pretend like this is a lock to succeed

- I worry that recency bias played a huge role in this: My concern as the Eagles approached this draft was that Roseman has been overly concerned with getting his targets sniped by another team. My take is that he saw what happened in 2014, where he had several targets and all of them went just before he was forced to take Marcus Smith, and he's vowed never to let that happen again. That means fewer trade downs and way more aggressive trade ups to get 'his guy' because it's so fresh in his mind. I worried that it could lead Roseman to overpay to move up, and we may be in exactly that scenario.

- This determines Howie and Doug's future in Philadelphia: Some of my other research into coaching tenure made one thing abundantly clear. Coaches get one shot at investing draft resources in a QB, they almost never get more than one. So Pederson's job security is now entirely dependent on whether the new Eagles QB can play or not. Roseman, even more so than Pederson, is now in the same boat. No one is going to remember he traded Kiko Alonso or DeMarco Murray, or even that he drafted Marcus Smith (w/ Chip). The singular defining event of Roseman's entire tenure is this draft pick -- he'll either have a long career here or be gone in three years, but there's really no in between.

- Howie thinks this team doesn't have lots of holes: Make no mistake, the Eagles gave up a boatload of draft assets to move up and get this quarterback. Not as much as the Redskins in going after RGIII, but a LOT of draft assets. To me, that signals both a clear need for a quarterback and that Roseman is fairly confident the rest of the team is pretty solid. If the team had a TON of holes, giving up so much draft capital would be hard to reconcile, but I think Roseman thinks the Eagles aren't that bad. On this point I think he could very well be right.

The Eagles have a bunch of good defensive assets -- and now Jim Schwartz to run them. They also have offensive assets that I don't think are as bad as people believe. I think Kelce and Johnson are good, and supplemented with new FA additions. I think Ertz and Matthews are good. I think Ryan Mathews is good. Do they need more talent at receiver? Probably. Do they need another lineman and a starting cornerback? Probably. But I don't think this team is in bad shape - maybe that's just wishful thinking.

But the trade up for the #2 pick is a massive decision that will define the Eagles success/failure for the next five years. While I still think the odds are against us, I must remind everyone of one simple question:

When's the last time the Cleveland Browns came out ahead in any trade???

Wednesday, April 6, 2016

Should NFL teams draft developmental QBs?

Matt Barkley

Mike Kafka

Andy Hall

All of these guys were at one time the Eagles 'developmental QB of the future.' It's a standard NFL draft narrative for teams to use a mid/late round draft pick on a quarterback when they already have confirmed starters on the roster.

When that happens, pundits love to talk about how the pick is a plan for the future, how the young quarterback will come in and learn at the foot of the experienced starter and be groomed to eventually take over.

The problem I've always had with this narrative is that I don't think it really ever plays out, and that those picks are generally a poor use of draft capital.

But I've never seen the data - does it make sense to use late round draft picks on quarterbacks? Do they succeed more often than I think they do?

So in the interest in destroying cliched narratives - I decided to take a look.


I started by pulling data on all quarterback seasons since 2000 (because Y2K is a nice round time bound). For each quarterback-season, I included the way in which that quarterback was acquired, and broke it down across the following buckets:

- First round pick
- 2nd/3rd round pick (Now Day 2 draft picks)
- 4th/5th/6th/7th round picks/Undrafted FA (Now Day 3 picks)
- Free Agency (excluding undrafted guys)
- Trade (yes, there are a few of these guys too...)

A quarterback's acquisition definition held as long as they didn't change teams, but once they move teams, their next season starts a reclassification. For example, Peyton Manning is defined as a First Rounder for his tenure with the Colts, but defined as a Free Agent for his tenure with the Broncos.

We also needed some measures of performance - so I'm leveraging Pro-Football-Reference's 'Approximate Value' metric. That's a metric created by the PFR guys to try and generalize performance over the course of a season - and I think it's better to use this type of measure rather than something like passing yards, QB wins, or something else that has some very tricky flaws.


The first look I put together was a view of Approximate Value distribution across our acquisition methods for each season. This was to see how much 'value' was generated by each group of quarterbacks, and see how that shifts over time. The year over year mix is illustrated below:

The first thing that you'll notice is that first round quarterbacks have typically generated 40%+ of all quarterback value over the last decade. If you include Free Agents, that jumps to about 60% of all value.

Late Round picks? Less than 20%.

A final observation from this data is that the mix has definitely shifted over time. The share of value created by first round picks has increased, while the share of value from picks made in the second round or after has declined from well over 40% in 2000-2001 to 25% in 2016.

Interesting, and we'll come back to that.

But you could look at these distributions and raise the question of playing time as a driver of that mix. First round quarterbacks are likely to get more starts than quarterbacks acquired in later rounds, so what happens when we adjust for that?

Great question.

I went back to the data on each quarterback season and adjusted the Approximate Value to a per game basis. This way, we can control a bit for playing time and the natural tendency to start first rounders won't skew our data.

Once we created our per game approximate values, I graphed the distribution of results by acquisition method.

We can see that, on a per game basis, the middle 50% of first round quarterbacks is significantly higher than all other acquisition methods. The remaining methods are relatively close to each other, but the group with the lowest middle 50% is our set of late round/undrafted quarterbacks. When you adjust for playing time, late round guys do significantly worse (except for the maximum value which is quite high -- we can call that the Tom Brady factor).

One last quick measure I looked at, almost as a check, was Pro Bowls. I know Pro Bowls aren't the best metric to judge true performance, but it's correlated with performance and I wanted to check what we've already seen here. So, below, is a quick list of the number of quarterbacks in each acquisition method bucket, the number of Pro Bowlers, and the share of quarterbacks in that bucket who have been to at least one Pro Bowl. (Note: players are unique by acquisition method, so Peyton Manning is in here once for the Colts as a First rounder and once in here for the Broncos as a FA)

- First Round: 58 quarterbacks, 19 pro bowlers, 33%
- 2nd/3rd Round: 55 quarterbacks, 8 pro bowlers, 15%
- Late Round/Undrafted: 168 quarterbacks, 8 pro bowlers, 5%
- Trade: 47 quarterbacks, 8 pro bowlers, 17%
- Free Agent: 125 quarterbacks, 12 pro bowlers, 10%

Again, this shouldn't come as a surprise. Late round picks at quarterback rarely do much of anything. They're extremely unlikely to make the pro bowl -- whereas a third of first round quarterbacks since 2000 have made it.

But let's bring back a point our earlier analysis showed -- that the value created by late round quarterbacks has declined dramatically over the last 16 years. I think that's extremely interesting, particularly given the fact that NFL offenses have shifted more towards passing and placed more of a premium on quarterback play. Few would argue that quarterbacks are more valuable than ever.

But quarterback value is being created more and more by those taken at the top of the draft. The Pro Bowl data confirms this a bit -- the most recent late round quarterback draft pick to make at least one Pro Bowl was in 2006, ten years ago (and it was Derek Anderson!). Maybe Kirk Cousins changes that someday, but less and less value is being generated by quarterbacks taken in the later rounds.

I thought part of it might be, that with increased emphasis on quarterback play, teams are reaching for QBs earlier and earlier in the draft and that's had a natural inflation effect on where they're drafted.

But I checked the data and it's just not true.

Are these newer first round quarterbacks just better than they used to be?

I don't think that's the case either.

At this point, my theory as to what's driving the shift is QB longevity. Quarterbacks drafted in the first round are more likely to be good, but with both improved medical technology and tweaked rules to keep them safe, quarterback tenure should be increasing. So I'd hypothesize that good quarterbacks are keeping their jobs longer because they aren't getting hurt, and the longer they play the more the mix shifts to more first rounders as teams without them bring them in.

We've seen the data that shows just how much more value first rounders create than other acquisition methods -- the implication there is clear -- the best way to get a good quarterback is to take one in the first round. And as those teams find good quarterbacks in the first round (unless you're in Cleveland), they hang on to them.

But the other implication is that taking 'developmental quarterbacks' in the later rounds is generally a waste of time. I'd have to check with other analysis on other positions' success in the later rounds, but I'm fairly confident the opportunity cost of taking a quarterback vs. another position is non-trivial. I'd suggest that teams looking for a developmental QB are better served signing a free agent that's been in the league, and use their picks on other positions with a higher likelihood of success.

And if you find yourself in need of a quarterback -- remember the best place to find them is in the first round of the draft. The idea that you'll get a Super Bowl QB in the later rounds is pretty much a fantasy.