I've been re-watching The Wire pretty much all week...HBO decided to run marathons and when you have a new infant, there's nothing you'd rather do than sit down on the couch and explain to her the intricacies of the Baltimore drug trade.
I spent a bunch of time re-watching parts of season 1, surprisingly enjoyed some time with season 2 the next day, and made sure I caught a decent chunk of season 3 (everyone's favorite)....and it got me thinking of a couple things.
The first was a much longer project that I thought of a couple years ago and never got around to. I want to do a Morality Power Ranking for the entire Wire cast...but jeez that's going to take forever even just to get everyone's list of awful sins straight. (Still a good idea though, and a much better subject for debate than Grantland's Wire bracket in my view).
But as I said, I'm not doing that. The other thing I was wondering about...is who is a more natural fan of the Wire? Hardcore liberals...or hardcore conservatives???
I don't think the answer is that straightforward.
I'm sure most people would argue that it's overtly liberal...in large part because I think the show's fan base tends to BE liberal. And sure, there are lots of things that would suggest it's aligned to a liberal point of view:
- The War of Drugs is a complete failure (pretty much the whole freaking series)
- Inner city youth have a woefully inadequate support system and/or safety net (season 4)
- Old, rich white people are frequently shown to have no clue (e.g., the guy from Johns Hopkins hiring Bunny Colvin, the newspaper publisher in season 5)
- And of course, the whole drug legalization experiment section of Season 3 is basically liberal policy porn
However, I think if a conservative wanted to make the argument, he/she could find a whole bunch of themes that fit with their worldview:
- Labor unions are ineffectual and corrupt (Season 2)
- Government officials are corrupt and driven largely by a desire for either money or power (season 4, any scene featuring Clay Davis)
- The lack of stable, nuclear family structure is what's lacking for inner city youth (season 4)
While the show itself does seem clear on a number of issues (e.g., end the war on drugs, community-based policing), I do think it's open enough to allow people on either side to claim it fits their philosophy. The conservatives would watch it and say, 'there's an example of how government solutions simply DO NOT work!', whereas the liberals would argue, 'there's an example of where we need different policies that WILL WORK!'
I'd be interested to hear more about how typical liberals or conservatives feel about the show. And I definitely think they should start the series back up for a sixth season...can you imagine The Wire doing a season on the healthcare system???
Come on HBO - that needs to happen.
Tuesday, December 30, 2014
Thursday, December 25, 2014
Why Airlines Compete on Price and Not Comfort
I'm finding that with a new baby in the house - my blog posts have to be EXTREMELY efficient! Not that I don't have free time anymore, but those brief windows of free time give me like 8 minutes every couple of hours, rather than a huge block of time. So, basically, I have to have any thoughts pretty well articulated before I sit at the laptop to type stuff up (but oh well, if nothing else it will force me to be concise).
I was reading an article the other day about air travel in the U.S. - and about how we really get the crappy experience we deserve.
The rationale is, while we as American travelers have the option of purchasing more expensive plane tickets that offer better treatment (e.g., better seats, better service, more amenities), we almost always just buy the cheapest one.
As evidence, the authors show that Spirit airlines (the poster child for nickel and dime fees for everything) is the nation's most profitable, and Virgin America, Richard Branson's much more comfortable experience-based airline, makes a lot less money.
And yes, that's an interesting idea. But when pressed for a rationale, the authors just offer the idea that well, a plane ticket is an expensive item and so relative price differentials "seem bigger", in part because there are fees that aren't shown on the sticker price:
One possible explanation for this behavior: When it comes to expensive items like plane tickets, the price differentials just seem bigger. I ran through some of the numbers on the phone with Kaplan. To keep things simple, we decided to compare two airlines at different points on the pricing spectrum: JetBlue and Spirit. In the latest fiscal quarter, the average flight cost $164.80 on JetBlue and $84.50 on Spirit. JetBlue also collected an average $22 in additional fees, or so-called ancillary revenue, from its passengers, and Spirit another $54. Looking at those numbers alone, the average fare on Spirit is about $50 cheaper than on JetBlue. But to truly compare the services, you have to make a few adjustments. For starters, flights on Spirit tend to be about 20 percent shorter than those on JetBlue. Spirit also crams 178 seats onto the Airbus A320, while JetBlue’s current layout accommodates 150. When you factor these things in, it turns out that JetBlue would have to take in about $196 on average just to make the same revenue per passenger that Spirit does. By those standards, JetBlue’s flights are actually a better deal. But based on sticker price alone, they don’t necessarily feel like it.
I'm a little confused as to why the author is making a point about revenue per passenger - because we're talking about price and customer's willingness to pay, but whatever. Regardless, this explanation fell a little short to me. Yes an airline ticket is expensive, and therefore people may be more sensitive to price because the absolute dollars are large...but there are entire massive segments of industries built on charging price premiums for 'better experiences'.
Hell, anything that could potentially market itself as luxury. Alcohol, fashion, hotels - there are plenty of industries that have major segments of 'luxury' - whereas for domestic airline travel, there really isn't a similar offer (technically there are private jets and NetJets etc., but my assumption is that's pretty small relative to things like luxury hotels).
So...what else might be driving that result in the market???
There's the idea that customers don't value a 'good' air travel experience - but that just seems overly simplistic to me. Let's think about some things that actual might impact a person's purchase decision:
1 - Purchase Channel: I think this plays a role. I don't know how everyone else buys airline tickets, but I buy mine online. Usually searching via Kayak (others use Orbitz, Expedia, Priceline, or airlines direct websites), and while you can filter for a couple dimensions like airports, airlines, number of stops, and takeoff/landing times - the overwhelming factor and automatic sort variable is price. It's the biggest number and as a default sort variable, it's effectively telling you that price is by far the most important variable. An interesting comparison is the world of hotels, when you search for a hotel you actually get a quality rating - this is true even on Kayak - where you can filter by the number of stars in a rating. Airlines have no such thing, even though you could argue Spirit is a low-star experience and Virgin America is a higher one.
2 - Delayed Consumption: I think another major issue that guides people to purchase based on price alone is the nature of air travel purchases and delayed consumption/gratification. Most people buy plane tickets pretty far in advance (I assume), as most trips are planned and last-minute tickets cost a fortune - exclusively for business folks and people looking to make grand romantic gestures. Now, if you're looking to have a nice evening out on a Saturday, you might be willing to make a reservation at a fancy restaurant - but you're going to have a nice meal later that night. If you're buying a plane ticket for 10 weeks out, that experience may feel much further away. Because it's so far off, maybe the internal discount rate you place on that experience makes your willingness to pay lower. If I told you that you could pay an extra $50 for a first class seat on a plane flight TONIGHT, you might be more likely than if I told you that you could pay an extra $50 for a first class upgrade on some flight in three months.
3 - Lack of Airline Control Over Experience: OK, let's say an airline really wants to make sure customers have the best possible experience. They want the best flight attendants, the biggest chairs, the deepest reclines...you know what would happen when you got home after you're flight? You'd still complain about the crappy TSA security lines, really slow baggage handling waits, long taxi stand lines, and if you're flying out at LGA - the fact that they don't have any decent food beyond security. Those are just a couple examples - but reflective of the fact that providing a great experience when you're on the plane is only part of the overall trip, there's a whole bunch of absolutely crappy travel you're going to have to deal with anyway. Those levels of service are just good enough to keep people from skipping flights altogether (i.e., really awful), and in the case of TSA - has no connection towards making your experience pleasant. If those parts of the experience are going to be terrible, how can you expect any airline to try and make their experience plush and wonderful?
Those were the three things that initially occurred to me - but they all seem legitimate. And unless someone really presents a totally different class of experience that can outweigh all these other factors - I'd assume airlines will still compete to offer the lowest prices and therefore the most cost effective service possible.
I was reading an article the other day about air travel in the U.S. - and about how we really get the crappy experience we deserve.
The rationale is, while we as American travelers have the option of purchasing more expensive plane tickets that offer better treatment (e.g., better seats, better service, more amenities), we almost always just buy the cheapest one.
As evidence, the authors show that Spirit airlines (the poster child for nickel and dime fees for everything) is the nation's most profitable, and Virgin America, Richard Branson's much more comfortable experience-based airline, makes a lot less money.
And yes, that's an interesting idea. But when pressed for a rationale, the authors just offer the idea that well, a plane ticket is an expensive item and so relative price differentials "seem bigger", in part because there are fees that aren't shown on the sticker price:
One possible explanation for this behavior: When it comes to expensive items like plane tickets, the price differentials just seem bigger. I ran through some of the numbers on the phone with Kaplan. To keep things simple, we decided to compare two airlines at different points on the pricing spectrum: JetBlue and Spirit. In the latest fiscal quarter, the average flight cost $164.80 on JetBlue and $84.50 on Spirit. JetBlue also collected an average $22 in additional fees, or so-called ancillary revenue, from its passengers, and Spirit another $54. Looking at those numbers alone, the average fare on Spirit is about $50 cheaper than on JetBlue. But to truly compare the services, you have to make a few adjustments. For starters, flights on Spirit tend to be about 20 percent shorter than those on JetBlue. Spirit also crams 178 seats onto the Airbus A320, while JetBlue’s current layout accommodates 150. When you factor these things in, it turns out that JetBlue would have to take in about $196 on average just to make the same revenue per passenger that Spirit does. By those standards, JetBlue’s flights are actually a better deal. But based on sticker price alone, they don’t necessarily feel like it.
I'm a little confused as to why the author is making a point about revenue per passenger - because we're talking about price and customer's willingness to pay, but whatever. Regardless, this explanation fell a little short to me. Yes an airline ticket is expensive, and therefore people may be more sensitive to price because the absolute dollars are large...but there are entire massive segments of industries built on charging price premiums for 'better experiences'.
Hell, anything that could potentially market itself as luxury. Alcohol, fashion, hotels - there are plenty of industries that have major segments of 'luxury' - whereas for domestic airline travel, there really isn't a similar offer (technically there are private jets and NetJets etc., but my assumption is that's pretty small relative to things like luxury hotels).
So...what else might be driving that result in the market???
There's the idea that customers don't value a 'good' air travel experience - but that just seems overly simplistic to me. Let's think about some things that actual might impact a person's purchase decision:
1 - Purchase Channel: I think this plays a role. I don't know how everyone else buys airline tickets, but I buy mine online. Usually searching via Kayak (others use Orbitz, Expedia, Priceline, or airlines direct websites), and while you can filter for a couple dimensions like airports, airlines, number of stops, and takeoff/landing times - the overwhelming factor and automatic sort variable is price. It's the biggest number and as a default sort variable, it's effectively telling you that price is by far the most important variable. An interesting comparison is the world of hotels, when you search for a hotel you actually get a quality rating - this is true even on Kayak - where you can filter by the number of stars in a rating. Airlines have no such thing, even though you could argue Spirit is a low-star experience and Virgin America is a higher one.
2 - Delayed Consumption: I think another major issue that guides people to purchase based on price alone is the nature of air travel purchases and delayed consumption/gratification. Most people buy plane tickets pretty far in advance (I assume), as most trips are planned and last-minute tickets cost a fortune - exclusively for business folks and people looking to make grand romantic gestures. Now, if you're looking to have a nice evening out on a Saturday, you might be willing to make a reservation at a fancy restaurant - but you're going to have a nice meal later that night. If you're buying a plane ticket for 10 weeks out, that experience may feel much further away. Because it's so far off, maybe the internal discount rate you place on that experience makes your willingness to pay lower. If I told you that you could pay an extra $50 for a first class seat on a plane flight TONIGHT, you might be more likely than if I told you that you could pay an extra $50 for a first class upgrade on some flight in three months.
3 - Lack of Airline Control Over Experience: OK, let's say an airline really wants to make sure customers have the best possible experience. They want the best flight attendants, the biggest chairs, the deepest reclines...you know what would happen when you got home after you're flight? You'd still complain about the crappy TSA security lines, really slow baggage handling waits, long taxi stand lines, and if you're flying out at LGA - the fact that they don't have any decent food beyond security. Those are just a couple examples - but reflective of the fact that providing a great experience when you're on the plane is only part of the overall trip, there's a whole bunch of absolutely crappy travel you're going to have to deal with anyway. Those levels of service are just good enough to keep people from skipping flights altogether (i.e., really awful), and in the case of TSA - has no connection towards making your experience pleasant. If those parts of the experience are going to be terrible, how can you expect any airline to try and make their experience plush and wonderful?
Those were the three things that initially occurred to me - but they all seem legitimate. And unless someone really presents a totally different class of experience that can outweigh all these other factors - I'd assume airlines will still compete to offer the lowest prices and therefore the most cost effective service possible.
Wednesday, December 17, 2014
Movie Theater Subscriptions
The big news coming out of the movie industry today is the pulling of 'The Interview' due to pressure from (potentially) North Korean hackers. While that's a pretty interesting story, I noticed a different article today about the movie industry that got me thinking.
AMC, one of the major theater chains, is testing a subscription plan for effectively, all-you-can-eat moviegoing.
Now a major movie theater chain is trying to step onto the subscription gravy train as it seeks to reverse attendance declines, especially among young moviegoers. Leawood-based AMC Theaters, the No. 2 chain in North America behind Regal Entertainment, has agreed to a pilot partnership with MoviePass, a three-year-old company focused on letting people attend a movie a day for one monthly fee.
“It frankly wouldn’t be smart to ignore the success of subscription in other areas of media,” said Christina Sternberg, senior vice president for corporate strategy at AMC, which operates 4,959 movie screens.
In January, AMC theaters in Boston and Denver will begin working in concert with MoviePass to offer monthly subscription packages for $45 and $35. More cities will be added.
So if you live in one of those select cities, you'll have the opportunity to buy a movie theater subscription and go to as many movies as you want...well, technically you could go to a max of 31 (one per day). But that should still be plenty.
However - I'm going to go ahead and guess that this model really isn't going to work, at least in the way it's structured right now.
I think as currently constructed, the price is way too high and only creates an adverse selection problem.
A monthly bill for $35 isn't exactly small, it's significantly higher than Netflix at <$10 (which to me is the threshold at which point you might forget you have it).
As a result, I think the only people you'll get flocking to it are super-hardcore moviegoers, who likely are already showing up frequently. As an indicator, I love going to the movies (I even saw that ridiculous Denzel movie where he kills all the Russian gangsters in a hardware store), and I would never think $35/month is worth it.
Clearly what AMC is going for is what so many gym chains are able to do - sign people up, and then have them NOT show up to the theater. But I think it's a stretch to be able to persuade someone to sign up for a movie subscription rather than a gym subscription. At least going to the gym has an aspiration attached to it - people think that by signing up for a gym they'll change their behavior and get in better shape. But I don't think anyone aspires to see 5 movies a month outside of maybe an aspiring film critic.
So unless they've got some great promotional pricing or other kinds of offers, I think it'll be hard to suck people in.
Just do the math, even at $35 a month, for you to breakeven on that investment, you'd need to go to at least 3 movies a month (assuming a $12 ticket price). So in order to get some value for your money, you really need to be going to the movies each week (4 movies would bring your ticket price to ~$9 a ticket, which is at least a little bit of a discount)
But do you know anyone who can really go to a movie every week? I'm not sure there are even enough movies to fill that schedule, again, unless you're a critic.
I just don't see someone willingly signing up for such a hefty subscription, particularly when you consider the fact that most movie trips aren't individual (though these subscriptions are). So if you have a subscription, and you're dating someone, you're going to suggest a night at the movies just about every date night. But unless you're dating another movie subscriber, you might not be able to get them on board, as they'll be paying full price (maybe you could buy them popcorn to even things out).
I'm assuming a large part of this pricing must be tied to how the theater chains kick ticket sales back to the movie companies. I'm not sure how they would figure out the internal bookkeeping of these subscription plans (If I'm charged for $35 and see three movies, does my $35 get divided by three across them? If I only see 1, does that movie get a full $35 cut?) Keeping the price per ticket high would ensure that the theater can still kick a lot back to the movie companies while driving more trips and concession purchases -- which is what I'm guessing the rationale is -- I just don't think any casual moviegoer is going to go for it.
If it were me, I'd explore a couple other alternatives. Why wouldn't there be opportunity for a family subscription (or a companion pass type deal like Southwest)? A family subscription that would allow different family configurations (mom+dad, mom+dad+kids) some discounted rate could potentially drive more trips. I'd also give some consideration to a limited seasonal pass. People go to the movies a lot during the summer - so maybe you create some kind of all-you-can-watch package and aim it squarely at the post-holiday/award season but pre-memorial day window.
These are just a couple ideas, but I don't think charging people $35 a month is going to appreciably get more butts in the seats. I just don't think people will pay that much.
AMC, one of the major theater chains, is testing a subscription plan for effectively, all-you-can-eat moviegoing.
Now a major movie theater chain is trying to step onto the subscription gravy train as it seeks to reverse attendance declines, especially among young moviegoers. Leawood-based AMC Theaters, the No. 2 chain in North America behind Regal Entertainment, has agreed to a pilot partnership with MoviePass, a three-year-old company focused on letting people attend a movie a day for one monthly fee.
“It frankly wouldn’t be smart to ignore the success of subscription in other areas of media,” said Christina Sternberg, senior vice president for corporate strategy at AMC, which operates 4,959 movie screens.
In January, AMC theaters in Boston and Denver will begin working in concert with MoviePass to offer monthly subscription packages for $45 and $35. More cities will be added.
So if you live in one of those select cities, you'll have the opportunity to buy a movie theater subscription and go to as many movies as you want...well, technically you could go to a max of 31 (one per day). But that should still be plenty.
However - I'm going to go ahead and guess that this model really isn't going to work, at least in the way it's structured right now.
I think as currently constructed, the price is way too high and only creates an adverse selection problem.
A monthly bill for $35 isn't exactly small, it's significantly higher than Netflix at <$10 (which to me is the threshold at which point you might forget you have it).
As a result, I think the only people you'll get flocking to it are super-hardcore moviegoers, who likely are already showing up frequently. As an indicator, I love going to the movies (I even saw that ridiculous Denzel movie where he kills all the Russian gangsters in a hardware store), and I would never think $35/month is worth it.
Clearly what AMC is going for is what so many gym chains are able to do - sign people up, and then have them NOT show up to the theater. But I think it's a stretch to be able to persuade someone to sign up for a movie subscription rather than a gym subscription. At least going to the gym has an aspiration attached to it - people think that by signing up for a gym they'll change their behavior and get in better shape. But I don't think anyone aspires to see 5 movies a month outside of maybe an aspiring film critic.
So unless they've got some great promotional pricing or other kinds of offers, I think it'll be hard to suck people in.
Just do the math, even at $35 a month, for you to breakeven on that investment, you'd need to go to at least 3 movies a month (assuming a $12 ticket price). So in order to get some value for your money, you really need to be going to the movies each week (4 movies would bring your ticket price to ~$9 a ticket, which is at least a little bit of a discount)
But do you know anyone who can really go to a movie every week? I'm not sure there are even enough movies to fill that schedule, again, unless you're a critic.
I just don't see someone willingly signing up for such a hefty subscription, particularly when you consider the fact that most movie trips aren't individual (though these subscriptions are). So if you have a subscription, and you're dating someone, you're going to suggest a night at the movies just about every date night. But unless you're dating another movie subscriber, you might not be able to get them on board, as they'll be paying full price (maybe you could buy them popcorn to even things out).
I'm assuming a large part of this pricing must be tied to how the theater chains kick ticket sales back to the movie companies. I'm not sure how they would figure out the internal bookkeeping of these subscription plans (If I'm charged for $35 and see three movies, does my $35 get divided by three across them? If I only see 1, does that movie get a full $35 cut?) Keeping the price per ticket high would ensure that the theater can still kick a lot back to the movie companies while driving more trips and concession purchases -- which is what I'm guessing the rationale is -- I just don't think any casual moviegoer is going to go for it.
If it were me, I'd explore a couple other alternatives. Why wouldn't there be opportunity for a family subscription (or a companion pass type deal like Southwest)? A family subscription that would allow different family configurations (mom+dad, mom+dad+kids) some discounted rate could potentially drive more trips. I'd also give some consideration to a limited seasonal pass. People go to the movies a lot during the summer - so maybe you create some kind of all-you-can-watch package and aim it squarely at the post-holiday/award season but pre-memorial day window.
These are just a couple ideas, but I don't think charging people $35 a month is going to appreciably get more butts in the seats. I just don't think people will pay that much.
Wednesday, December 10, 2014
How Would you Beat Uber?
When I ride in a cab, I'm usually not eager to start a conversation. That's especially true when I'm headed to or from an airport - because those rides are pretty long and can be at some unfortunate hours.
But when the cab driver is interested in some kind of discussion, I'll usually humor them a bit. Of course, in the last couple years, I've noticed a distinct trend in the cab drivers who want to talk.
Many of them want to talk about Uber. And many of them want to tell me why Uber is evil and wrong.
They always start innocuously enough by asking if I ever use the service. And given that I'm trying to avoid a conversation, and I'm REALLY trying to avoid an argument, I'll always feign ignorance and tell them I've never heard of it. Of course, that doesn't stop them from going on and complaining about how the app is taking over the world.
That progress continues, with Uber recently announcing another fundraise at a mega valuation of $40B - more than tons of mature and well known corporations.
But for years, angry cab drivers, cab companies, legislators and other incumbent interests have tried to block Uber's advance.
But in lots of cases, these efforts have fallen short as the company continuously puts forth a technically easy, efficient and cheap car ride.
You have to feel for the cab/limo industry as it gets disrupted like crazy - granted, their disruption results in lower prices for us all - but I was thinking and trying to put myself in their shoes. If my company was a legacy limo provider and I had to compete with Uber, what would I do?
The first realization I'd need to come to grips with is that I absolutely can't provide a better experience than Uber, Lyft, or similar competitors. They have extremely low fixed costs because they don't operate their own fleet of vehicles. They also have a system that ensures most cars are clean and well maintained (their review system), and as long as they have enough people willing to drive their cars for little money, they have a pretty good supply of cars on the market.
If I can't provide a better experience, and I can't provide a better price, then I'm in deep trouble.
So what you've seen these companies do is try and play the regulatory angle, arguing that Uber is skirting the rules that real taxi/limo companies have to abide by. That's not a bad idea - but it's not going to work either.
First, customers aren't going to show you any sympathy. In fact, they'll likely tell whichever politicians you seek to influence to go the other way. Such actions make you look like you're just blocking progress to preserve your business (which of course, you are). That's not a long-term strategy.
I think the answer (to the extent there is one), is a slightly different tack on the regulatory angle. You can't beat Uber by turning regulators against them, but maybe, you can persuade folks to fear for their safety enough that they'll voluntarily move away from lower prices and a better experience.
You'd need some total William Randolph Hearst-style yellow journalism of Fox News fear-mongering, but maybe you could convince a lot of people that Uber drivers are dangerous. You would have to scour the news for any story, even minor ones that show Uber drivers doing bad things. Any incident involving Uber drivers would be another flashpoint to say, 'This company doesn't care about your safety'
Here's a great example that appeared today
"The online car service Uber says it has "removed" one of its drivers as police investigate allegations that he sexually assaulted a female customer in Lincoln Square last month."
Uber drivers sexually assault their customers? I know that would certainly make some folks a little leery of trusting a random guy from giving them a ride.
The same would go for car accidents. I'm still not clear on how Uber's insurance works, but if you can find passengers who were injured while riding in an Uber, you definitely need to get their story reported.
You'd need to find all these stories and blow them up, ideally out of proportion. Then you'd need to contrast that with your own company's operations - which would have more thorough background checks, insurance coverage, etc. Illustrating that difference at a high enough level, and maybe you can convince a good enough section of the population to eschew Uber, or you drive Uber's costs up, which would help you anyway. In a best case scenario, you could get a genuine interest from regulators to tighten their rules - at which point you'd come off less like a sore loser and could officially present yourself as the protector of public safety.
Now, with that said, I don't really think that would work. It's pretty tough to stop the snowball effect and now they have tons of users. But if you can't really compete, it might be worth a try.
But when the cab driver is interested in some kind of discussion, I'll usually humor them a bit. Of course, in the last couple years, I've noticed a distinct trend in the cab drivers who want to talk.
Many of them want to talk about Uber. And many of them want to tell me why Uber is evil and wrong.
They always start innocuously enough by asking if I ever use the service. And given that I'm trying to avoid a conversation, and I'm REALLY trying to avoid an argument, I'll always feign ignorance and tell them I've never heard of it. Of course, that doesn't stop them from going on and complaining about how the app is taking over the world.
That progress continues, with Uber recently announcing another fundraise at a mega valuation of $40B - more than tons of mature and well known corporations.
But for years, angry cab drivers, cab companies, legislators and other incumbent interests have tried to block Uber's advance.
But in lots of cases, these efforts have fallen short as the company continuously puts forth a technically easy, efficient and cheap car ride.
You have to feel for the cab/limo industry as it gets disrupted like crazy - granted, their disruption results in lower prices for us all - but I was thinking and trying to put myself in their shoes. If my company was a legacy limo provider and I had to compete with Uber, what would I do?
The first realization I'd need to come to grips with is that I absolutely can't provide a better experience than Uber, Lyft, or similar competitors. They have extremely low fixed costs because they don't operate their own fleet of vehicles. They also have a system that ensures most cars are clean and well maintained (their review system), and as long as they have enough people willing to drive their cars for little money, they have a pretty good supply of cars on the market.
If I can't provide a better experience, and I can't provide a better price, then I'm in deep trouble.
So what you've seen these companies do is try and play the regulatory angle, arguing that Uber is skirting the rules that real taxi/limo companies have to abide by. That's not a bad idea - but it's not going to work either.
First, customers aren't going to show you any sympathy. In fact, they'll likely tell whichever politicians you seek to influence to go the other way. Such actions make you look like you're just blocking progress to preserve your business (which of course, you are). That's not a long-term strategy.
I think the answer (to the extent there is one), is a slightly different tack on the regulatory angle. You can't beat Uber by turning regulators against them, but maybe, you can persuade folks to fear for their safety enough that they'll voluntarily move away from lower prices and a better experience.
You'd need some total William Randolph Hearst-style yellow journalism of Fox News fear-mongering, but maybe you could convince a lot of people that Uber drivers are dangerous. You would have to scour the news for any story, even minor ones that show Uber drivers doing bad things. Any incident involving Uber drivers would be another flashpoint to say, 'This company doesn't care about your safety'
Here's a great example that appeared today
"The online car service Uber says it has "removed" one of its drivers as police investigate allegations that he sexually assaulted a female customer in Lincoln Square last month."
Uber drivers sexually assault their customers? I know that would certainly make some folks a little leery of trusting a random guy from giving them a ride.
The same would go for car accidents. I'm still not clear on how Uber's insurance works, but if you can find passengers who were injured while riding in an Uber, you definitely need to get their story reported.
You'd need to find all these stories and blow them up, ideally out of proportion. Then you'd need to contrast that with your own company's operations - which would have more thorough background checks, insurance coverage, etc. Illustrating that difference at a high enough level, and maybe you can convince a good enough section of the population to eschew Uber, or you drive Uber's costs up, which would help you anyway. In a best case scenario, you could get a genuine interest from regulators to tighten their rules - at which point you'd come off less like a sore loser and could officially present yourself as the protector of public safety.
Now, with that said, I don't really think that would work. It's pretty tough to stop the snowball effect and now they have tons of users. But if you can't really compete, it might be worth a try.
Monday, December 1, 2014
Theoretical Decision Model for NFL Coaches Challenges
One of the great things about the NFL is that every game features tons of small coach and player decisions -- but to be more precise -- one of the great things about the NFL is the ability for armchair QBs like me to second guess any and all of those decisions!
Which is why I've had such a hard time getting the Eagles week 8 loss to the Arizona Cardinals out of my head. I know the Eagles just finished a great Thanksgiving pasting of their division rival - but that Cardinals game keeps coming back to me. Specifically, the Eagles decision not to challenge a call late in the fourth quarter.
If you'll recall, the score was tied late in the fourth quarter (17-17), and the Eagles had the ball deep in Cardinals territory. On 2nd down with 4 yards to go from the Cardinals five yard line, Eagles running back Chris Polk ran the ball for a three yard gain - but one that looked like it may have been spotted incorrectly by the officials.
In my mind - a challenge was automatic. But the Eagles disagreed - they kicked a field goal, gave up a touchdown on the next Cardinals possession, and lost the game.
Afterwards, what bothered me, besides the fact that the Eagles lost - was the attitude towards challenging the play. Here was Chip Kelly's response when he was asked about it:
"“I don’t think the league rate at overturning spots, in terms of where his knee was down—most of the time the err, I think percentage wise in the league, if you study the percentage of it there you err on the previous call on the field because it’s very difficult to tell on the spot unless it’s kind of a clear-cut open field kind of thing,” Kelly said."
This was what I took issue with. The Eagles didn't challenge because most spot challenges aren't overturned.
Now, that may be true (another article I saw reported that only about one-third of spot challenges have been overturned this year) - but that reasoning has one giant flaw - and that's situational leverage.
What I mean by that is - what is the relative upside to winning the challenge, given the game situation. Winning a spot challenge to get an extra two yards when you're up by 35 points would be meaningless, but winning a challenge to put you in FG range with 2:30 left in the fourth quarter in a tie game could be huge.
Treating those potential challenges, because they're both spots of the football, as the same leaves out a massive part of the equation.
So, I started to think, is there some way we can actually calculate that piece of the puzzle? And could that potentially shed light on when teams should challenge calls???
I think the answer is unequivocally yes, and yes.
Logic on Challenge Win Probability
The reason I started thinking along these lines at all is that I was comparing the Eagles situation, or any NFL team for that matter, to a poker player making the decision to call a final bet. If you're playing a game of poker and deciding whether to call - the decision isn't just about whether you think you have the best hand, it's also about how much you'd stand to win. Calling a $50 bet with a pot of $500 is a LOT more attractive than calling a $500 bet with a pot of $50.
That's what I mean when I talk about leverage. Now in poker, players will use the concept of pot-odds to help them make their decision on whether or not to call. Let's take that first situation I just described. A $50 bet to win a pot of $500. In that situation, winning the hand will pay out at 10:1, so if you thought you had a 50/50 shot at having the best hand, would you call the bet?
Of course you would. If you thought you'd win 50% of the the time a 10:1 payout will, over time, be very profitable.
Now in the other situation - a $50 bet to win $500 - well now you're getting a 1:10 payout. Now if you have a 50/50 shot of having the best hand - would you call this bet? Of course not - because the upside potential is horrible!
Poker players will use these pot odds to help determine the win probability required for a bet to make money. In my first scenario, if a poker player guesses he has a 25% chance of having the best hand - he should still call the bet. That calculation makes the decision to call much clearer.
To me, challenging a call in the NFL is the exact same thing. NFL coaches should not only be considering whether or not they'll be winning the challenge in absolute terms, but they must also consider the upside of winning the challenge itself. In fact, if you work backwards, you can calculate the probability of winning a challenge you would need for the challenge to be worthwhile - just like poker players impute the win probability they need to call a bet. Only then should you be referring to historical overturn rates or anything else to see if it's a smart bet.
So...if you're on board with the logic, let's put it into practice:
Estimating Challenge Win Probability Required
OK - so for any of this to make sense, we need a basic metric to get at odds of winning the football game. As a coach, that's the metric we're trying to maximize - and we need to simplify all the various game situation components we have (time left, down and distance, score differential, etc.)
Fortunately, that already exists in the form of win probability estimates. Brian Burke of Advanced Football Analytics has his model, Pro Football Reference has one as well, there may be others - but I'm going to use Burke's for these examples because he has a calculator that let's you play with different situational variables like the examples I'm going to use.
So if we wanted a simple equation to help us evaluate whether or not to challenge the call on the field, what would that look like?
(Odds of Winning a Challenge)*(Benefit from Winning a Challenge) - (Odds of Losing a Challenge)*(Cost of a Challenge)
In this equation - you would challenge whenever the result is greater than zero, and you would not challenge when the result is less than zero.
To determine what odds of winning we would need to want to throw the challenge flag, we just have to set our equation equal to zero and solve for the missing variable. So let's dig into each component in a bit more detail:
1. The Odds of Winning the Challenge: This is what we want to solve for, the same metric the poker players use. It's going to be in percentage terms from 0% to 100%.
2. The Odds of Losing the Challenge: This is just the companion to our odds of winning the challenge. Together they have to add up to 100%. If you have a 75% chance of winning the challenge, well, you have a 25% chance of losing. Pretty straightforward. If the odds of winning the challenge = X, then this variable is just (1-X)
3. Benefit from Winning a Challenge: OK, this is the key component to the equation. I mentioned earlier we would use win probability estimates, and that's what we're going to use to estimate our 'benefit' from winning a challenge. If you're deciding to challenge the call on the field, you're effectively looking to change the current game situation. In this context, game situation is a combination of field position, down and distance, score differential, possession, and time remaining. As I mentioned earlier, win probability models can determine the probability of winning for any individual game situation - that is, any unique combination of those factors.
For example: if your team is starting the second half with the ball in a first and ten on your own 20 and a 3 touchdown lead - your win probability is estimated at 94%
If your team has the same field position, but now is down by 6 points and with only five minutes left in the 4th quarter - your win probability is 21%
Make sense? OK.
Now in the case of a challenge, a coach is looking to overturn a ruling that will result in a new improved game situation. And in just about every challenge I've ever seen, we know exactly what the old and the new situations would be. This variable in our equation then, is just WP^Overturn - WP^Upheld. That's our positive benefit from winning the challenge.
Here's an extreme example to illustrate a high leverage situation:
Let's say you're the Eagles, only its 1991 and you have the greatest video game QB of all time, QB Eagles - because this is Tecmo Super Bowl.
Now let's say it's 3rd down and ten from your own 20 - because you've thrown two long bombs to Fred Barnett that haven't worked yet. And let's say the score is tied.
So you call the long bomb again, and true to Tecmo form, immediately take a 30 step drop for QB Eagles into the back of the endzone, uncorking a beautiful rainbow that travels 108 yards in the air towards Barnett. He does that thing where the received jumps up and they show the cut-scene of him catching it, and he lands at the two yard line.
But the ref rules him out-of-bounds...says his foot touched down out.
Now let's say Tecmo has added the functionality to challenge the ruling on the field - what would the benefit be??? (I only now realize this example could probably be done with the latest version of Madden - and would make a lot more sense to anyone under 30 - ugh)
So if we use Brian Burke's calculator, here are is the first scenario - the one where if you win the challenge:
Score Differential: 0
Time Left: 2:00 Second Quarter (made up)
Field Position: Opposition 2 yard line
Down: First
Distance: 2 YTG
Challenge Victory Win Probability - 70% (In part because you're expected to get 5.8 points off this drive - even though the computer has NO IDEA how awesome QB Eagles is)
Now - what's the scenario if you LOSE the challenge
Score Differential: 0
Time Left: 2:00 Second Quarter (made up)
Field Position: Own 20 yard line
Down: Fourth (an incompletion on third makes it fourth down - you should totally go for it, FWIW)
Distance: 10 YTG
Challenge Loss Win Probability - 45%
So - if you won that challenge, not only would you get some good yardage to pad QB Eagles stats, but you'd also go from a 45% win probability to a 70% probability.
The benefit then - would be 70-45, a 25% improvement.
This is the element, the payoff, that we need.
4. The Cost of a Challenge: Slightly more tricky, but necessary to understand the cost of our challenge. If a challenge didn't cost anything - there'd be only upside to throwing the flag and we'd have red flags galore. The only people who would be happy would be Mike Carey and the other officiating experts - the rest of us would be super pissed. So there's a cost to being wrong on a challenge in the form of a timeout. Note - there's no cost if you win a challenge - so we don't need to account for that in the first half of our equation.
But the loss of a timeout is a bit harder to quantify. It's obviously valuable - but we'd need some estimate of it in terms of win probability for this math to really hold up. Fortunately, Brian Burke and the Sports Collective have done independent work on exactly that question (I don't have the links anymore but if you're really interested in it, just google it.
Now - their research shows a timeout to be worth between 3% and 5% win probability. So I split the difference and assumed 4%. Now - there are a whole bunch of non-trivial caveats as the value of timeouts would technically be very dynamic. It would vary depending on the game situation as well as how many timeouts remain in general (the final timeout is more valuable than the first). But that's a question for another time. We're going to use 4% - because I don't have the data/time to churn through that question on my own.
Putting it all together
Now we have all the pieces that we need to put our equation together. So let's take our Tecmo example...I used that as an example of a very high payoff....that means you would be willing to challenge even if you thought the odds of it getting overturned were smaller than normal. Instinctively that makes sense, but what does the math say?
(Odds of Winning a Challenge)*(Benefit from Winning a Challenge) - (Odds of Losing a Challenge)*(Cost of a Challenge)
Odds of Winning a Challenge = X% (remember we're solving for this)
Benefit from Winning a Challenge = (.70-.45) = .25 WP
Odds of Losing a Challenge = (1-X)%
Cost of a Challenge = .04 WP
(X)(.25) - (1-X)(.04) = 0
X, or the probability of winning the challenge required for it to be worthwhile = 14%
And that makes a lot of sense! You've got tremendous upside to winning the challenge, so even if you think there's only a 20% chance the ref will overturn it, it's still worth it to try, because the payoff is so great.
Now - what if the payoff wouldn't be great? What does our equation say in that case...let's take another example.
Let's say the score is tied 0-0 in the third quarter with 12 minutes to go, the Eagles have the ball on their own 20, second down and one yard to go. They run a WR screen where the pass appears to skim the ground as Jeremy Maclin grabs it before getting tackled with a one yard gain. It's ruled incomplete, but maybe we could challenge it and get it overturned (maybe the coach has Maclin in a fantasy PPR league)
So - what does the WP calculator say about these two scenarios, the first where the Eagles win the challenge (1st and 10 from their 21), the second where they don't win (3rd and 1 from their 20)
Well - the benefit in this case is 9% win probability (Eagles go from 50% win probability to 59% with the improved field position).
So if we put that in the equation, what does that tell us?
(X)(.09) - (1-X)(.04) = 0
X = 80%
So in this case, the payoff is so much lower, that you need to believe there's at least an 80% likelihood that the ref will overturn the ruling on the field.
Conclusion
The purpose of this thought exercise wasn't to create a bunch of weird examples - but what I wanted to do is illustrate a mental approach to challenging calls in NFL games. Most coaches probably pursue an approach that's intuitively aligned with this thinking if not calculated out to the same degree. Understanding the likelihood of winning the challenge is good - but it needs to be taken within the broader context of how that change in game situation will impact likelihood to win.
If it's a big play, the less confident you need to be in the replay to take a shot with a challenge. This is even more important in cases where there might not be enough time for coaches to see a replay.
Ideally - it would be great to create a simple widget where anyone could input the before and after circumstances of the challenge and determine the probability you would need to be willing to challenge. That would be fun to play with - and you could even create some simple diagram or framework that outlines when a challenge requires a very high confidence and when a challenge doesn't require much at all.
That's something I'd do if I had a win probability model of my own - but I don't, so you'll have to live with the theory. But in my mind, this is the right way to think about potentially challenging the ruling on the field.
Oh - and by the way, that Eagles-Cardinals challenge that got me so steamed in the first place. The model suggests I was wrong (although there's a big caveat).
Because the play happened so late, in a tie game, with the Eagles only a couple yards from the end zone, the win probability for a 1st down vs. a third down doesn't change things all that much (.82 vs. .81). And because the win probability difference is so small, there's not nearly enough upside.
However - I will say that this points out at least one of the shortcomings in this approach. There's no probability model out there that will account for time outs remaining on either team - and I think that would've played a role in the 'true' win probability. The Eagles ability to run the clock with a 1st down would've forced Arizona to use their timeouts or left them with little time left to score. So I think the win probability impact from winning that challenge would actually be a bit more.
But why should we let reality get in the way of some pretty cool theory!
Which is why I've had such a hard time getting the Eagles week 8 loss to the Arizona Cardinals out of my head. I know the Eagles just finished a great Thanksgiving pasting of their division rival - but that Cardinals game keeps coming back to me. Specifically, the Eagles decision not to challenge a call late in the fourth quarter.
If you'll recall, the score was tied late in the fourth quarter (17-17), and the Eagles had the ball deep in Cardinals territory. On 2nd down with 4 yards to go from the Cardinals five yard line, Eagles running back Chris Polk ran the ball for a three yard gain - but one that looked like it may have been spotted incorrectly by the officials.
In my mind - a challenge was automatic. But the Eagles disagreed - they kicked a field goal, gave up a touchdown on the next Cardinals possession, and lost the game.
Afterwards, what bothered me, besides the fact that the Eagles lost - was the attitude towards challenging the play. Here was Chip Kelly's response when he was asked about it:
"“I don’t think the league rate at overturning spots, in terms of where his knee was down—most of the time the err, I think percentage wise in the league, if you study the percentage of it there you err on the previous call on the field because it’s very difficult to tell on the spot unless it’s kind of a clear-cut open field kind of thing,” Kelly said."
This was what I took issue with. The Eagles didn't challenge because most spot challenges aren't overturned.
Now, that may be true (another article I saw reported that only about one-third of spot challenges have been overturned this year) - but that reasoning has one giant flaw - and that's situational leverage.
What I mean by that is - what is the relative upside to winning the challenge, given the game situation. Winning a spot challenge to get an extra two yards when you're up by 35 points would be meaningless, but winning a challenge to put you in FG range with 2:30 left in the fourth quarter in a tie game could be huge.
Treating those potential challenges, because they're both spots of the football, as the same leaves out a massive part of the equation.
So, I started to think, is there some way we can actually calculate that piece of the puzzle? And could that potentially shed light on when teams should challenge calls???
I think the answer is unequivocally yes, and yes.
Logic on Challenge Win Probability
The reason I started thinking along these lines at all is that I was comparing the Eagles situation, or any NFL team for that matter, to a poker player making the decision to call a final bet. If you're playing a game of poker and deciding whether to call - the decision isn't just about whether you think you have the best hand, it's also about how much you'd stand to win. Calling a $50 bet with a pot of $500 is a LOT more attractive than calling a $500 bet with a pot of $50.
That's what I mean when I talk about leverage. Now in poker, players will use the concept of pot-odds to help them make their decision on whether or not to call. Let's take that first situation I just described. A $50 bet to win a pot of $500. In that situation, winning the hand will pay out at 10:1, so if you thought you had a 50/50 shot at having the best hand, would you call the bet?
Of course you would. If you thought you'd win 50% of the the time a 10:1 payout will, over time, be very profitable.
Now in the other situation - a $50 bet to win $500 - well now you're getting a 1:10 payout. Now if you have a 50/50 shot of having the best hand - would you call this bet? Of course not - because the upside potential is horrible!
Poker players will use these pot odds to help determine the win probability required for a bet to make money. In my first scenario, if a poker player guesses he has a 25% chance of having the best hand - he should still call the bet. That calculation makes the decision to call much clearer.
To me, challenging a call in the NFL is the exact same thing. NFL coaches should not only be considering whether or not they'll be winning the challenge in absolute terms, but they must also consider the upside of winning the challenge itself. In fact, if you work backwards, you can calculate the probability of winning a challenge you would need for the challenge to be worthwhile - just like poker players impute the win probability they need to call a bet. Only then should you be referring to historical overturn rates or anything else to see if it's a smart bet.
So...if you're on board with the logic, let's put it into practice:
Estimating Challenge Win Probability Required
OK - so for any of this to make sense, we need a basic metric to get at odds of winning the football game. As a coach, that's the metric we're trying to maximize - and we need to simplify all the various game situation components we have (time left, down and distance, score differential, etc.)
Fortunately, that already exists in the form of win probability estimates. Brian Burke of Advanced Football Analytics has his model, Pro Football Reference has one as well, there may be others - but I'm going to use Burke's for these examples because he has a calculator that let's you play with different situational variables like the examples I'm going to use.
So if we wanted a simple equation to help us evaluate whether or not to challenge the call on the field, what would that look like?
(Odds of Winning a Challenge)*(Benefit from Winning a Challenge) - (Odds of Losing a Challenge)*(Cost of a Challenge)
In this equation - you would challenge whenever the result is greater than zero, and you would not challenge when the result is less than zero.
To determine what odds of winning we would need to want to throw the challenge flag, we just have to set our equation equal to zero and solve for the missing variable. So let's dig into each component in a bit more detail:
1. The Odds of Winning the Challenge: This is what we want to solve for, the same metric the poker players use. It's going to be in percentage terms from 0% to 100%.
2. The Odds of Losing the Challenge: This is just the companion to our odds of winning the challenge. Together they have to add up to 100%. If you have a 75% chance of winning the challenge, well, you have a 25% chance of losing. Pretty straightforward. If the odds of winning the challenge = X, then this variable is just (1-X)
3. Benefit from Winning a Challenge: OK, this is the key component to the equation. I mentioned earlier we would use win probability estimates, and that's what we're going to use to estimate our 'benefit' from winning a challenge. If you're deciding to challenge the call on the field, you're effectively looking to change the current game situation. In this context, game situation is a combination of field position, down and distance, score differential, possession, and time remaining. As I mentioned earlier, win probability models can determine the probability of winning for any individual game situation - that is, any unique combination of those factors.
For example: if your team is starting the second half with the ball in a first and ten on your own 20 and a 3 touchdown lead - your win probability is estimated at 94%
If your team has the same field position, but now is down by 6 points and with only five minutes left in the 4th quarter - your win probability is 21%
Make sense? OK.
Now in the case of a challenge, a coach is looking to overturn a ruling that will result in a new improved game situation. And in just about every challenge I've ever seen, we know exactly what the old and the new situations would be. This variable in our equation then, is just WP^Overturn - WP^Upheld. That's our positive benefit from winning the challenge.
Here's an extreme example to illustrate a high leverage situation:
Let's say you're the Eagles, only its 1991 and you have the greatest video game QB of all time, QB Eagles - because this is Tecmo Super Bowl.
Now let's say it's 3rd down and ten from your own 20 - because you've thrown two long bombs to Fred Barnett that haven't worked yet. And let's say the score is tied.
So you call the long bomb again, and true to Tecmo form, immediately take a 30 step drop for QB Eagles into the back of the endzone, uncorking a beautiful rainbow that travels 108 yards in the air towards Barnett. He does that thing where the received jumps up and they show the cut-scene of him catching it, and he lands at the two yard line.
But the ref rules him out-of-bounds...says his foot touched down out.
Now let's say Tecmo has added the functionality to challenge the ruling on the field - what would the benefit be??? (I only now realize this example could probably be done with the latest version of Madden - and would make a lot more sense to anyone under 30 - ugh)
So if we use Brian Burke's calculator, here are is the first scenario - the one where if you win the challenge:
Score Differential: 0
Time Left: 2:00 Second Quarter (made up)
Field Position: Opposition 2 yard line
Down: First
Distance: 2 YTG
Challenge Victory Win Probability - 70% (In part because you're expected to get 5.8 points off this drive - even though the computer has NO IDEA how awesome QB Eagles is)
Now - what's the scenario if you LOSE the challenge
Score Differential: 0
Time Left: 2:00 Second Quarter (made up)
Field Position: Own 20 yard line
Down: Fourth (an incompletion on third makes it fourth down - you should totally go for it, FWIW)
Distance: 10 YTG
Challenge Loss Win Probability - 45%
So - if you won that challenge, not only would you get some good yardage to pad QB Eagles stats, but you'd also go from a 45% win probability to a 70% probability.
The benefit then - would be 70-45, a 25% improvement.
This is the element, the payoff, that we need.
4. The Cost of a Challenge: Slightly more tricky, but necessary to understand the cost of our challenge. If a challenge didn't cost anything - there'd be only upside to throwing the flag and we'd have red flags galore. The only people who would be happy would be Mike Carey and the other officiating experts - the rest of us would be super pissed. So there's a cost to being wrong on a challenge in the form of a timeout. Note - there's no cost if you win a challenge - so we don't need to account for that in the first half of our equation.
But the loss of a timeout is a bit harder to quantify. It's obviously valuable - but we'd need some estimate of it in terms of win probability for this math to really hold up. Fortunately, Brian Burke and the Sports Collective have done independent work on exactly that question (I don't have the links anymore but if you're really interested in it, just google it.
Now - their research shows a timeout to be worth between 3% and 5% win probability. So I split the difference and assumed 4%. Now - there are a whole bunch of non-trivial caveats as the value of timeouts would technically be very dynamic. It would vary depending on the game situation as well as how many timeouts remain in general (the final timeout is more valuable than the first). But that's a question for another time. We're going to use 4% - because I don't have the data/time to churn through that question on my own.
Putting it all together
Now we have all the pieces that we need to put our equation together. So let's take our Tecmo example...I used that as an example of a very high payoff....that means you would be willing to challenge even if you thought the odds of it getting overturned were smaller than normal. Instinctively that makes sense, but what does the math say?
(Odds of Winning a Challenge)*(Benefit from Winning a Challenge) - (Odds of Losing a Challenge)*(Cost of a Challenge)
Odds of Winning a Challenge = X% (remember we're solving for this)
Benefit from Winning a Challenge = (.70-.45) = .25 WP
Odds of Losing a Challenge = (1-X)%
Cost of a Challenge = .04 WP
(X)(.25) - (1-X)(.04) = 0
X, or the probability of winning the challenge required for it to be worthwhile = 14%
And that makes a lot of sense! You've got tremendous upside to winning the challenge, so even if you think there's only a 20% chance the ref will overturn it, it's still worth it to try, because the payoff is so great.
Now - what if the payoff wouldn't be great? What does our equation say in that case...let's take another example.
Let's say the score is tied 0-0 in the third quarter with 12 minutes to go, the Eagles have the ball on their own 20, second down and one yard to go. They run a WR screen where the pass appears to skim the ground as Jeremy Maclin grabs it before getting tackled with a one yard gain. It's ruled incomplete, but maybe we could challenge it and get it overturned (maybe the coach has Maclin in a fantasy PPR league)
So - what does the WP calculator say about these two scenarios, the first where the Eagles win the challenge (1st and 10 from their 21), the second where they don't win (3rd and 1 from their 20)
Well - the benefit in this case is 9% win probability (Eagles go from 50% win probability to 59% with the improved field position).
So if we put that in the equation, what does that tell us?
(X)(.09) - (1-X)(.04) = 0
X = 80%
So in this case, the payoff is so much lower, that you need to believe there's at least an 80% likelihood that the ref will overturn the ruling on the field.
Conclusion
The purpose of this thought exercise wasn't to create a bunch of weird examples - but what I wanted to do is illustrate a mental approach to challenging calls in NFL games. Most coaches probably pursue an approach that's intuitively aligned with this thinking if not calculated out to the same degree. Understanding the likelihood of winning the challenge is good - but it needs to be taken within the broader context of how that change in game situation will impact likelihood to win.
If it's a big play, the less confident you need to be in the replay to take a shot with a challenge. This is even more important in cases where there might not be enough time for coaches to see a replay.
Ideally - it would be great to create a simple widget where anyone could input the before and after circumstances of the challenge and determine the probability you would need to be willing to challenge. That would be fun to play with - and you could even create some simple diagram or framework that outlines when a challenge requires a very high confidence and when a challenge doesn't require much at all.
That's something I'd do if I had a win probability model of my own - but I don't, so you'll have to live with the theory. But in my mind, this is the right way to think about potentially challenging the ruling on the field.
Oh - and by the way, that Eagles-Cardinals challenge that got me so steamed in the first place. The model suggests I was wrong (although there's a big caveat).
Because the play happened so late, in a tie game, with the Eagles only a couple yards from the end zone, the win probability for a 1st down vs. a third down doesn't change things all that much (.82 vs. .81). And because the win probability difference is so small, there's not nearly enough upside.
However - I will say that this points out at least one of the shortcomings in this approach. There's no probability model out there that will account for time outs remaining on either team - and I think that would've played a role in the 'true' win probability. The Eagles ability to run the clock with a 1st down would've forced Arizona to use their timeouts or left them with little time left to score. So I think the win probability impact from winning that challenge would actually be a bit more.
But why should we let reality get in the way of some pretty cool theory!
Subscribe to:
Posts (Atom)