down the rabbit hole of Leafs analytics

This SAS shuttle bus has become so familiar in my neighbourhood, I failed to notice how hilarious it has become. They’re using the Maple Leafs to promote their brand. It’s funny because the Leafs have negative branding value at the moment, and SAS appears to be advertising their role in the team’s bottomless decline.

Fans are pitching licensed Leaf apparel on the ice.  Police have charged them with something, probably heresy.  The club just fired a highly respected coach.  Under the interim coach, the team has won once and lost seven times.  Players are defiant, dejected, or just confused.  Media experts are calling for wholesale replacement of the organization’s talent and leadership.  No one gives them much chance of making the playoffs in a league where even below average teams can make it into the first round.

leafs-standings1-300x181Keep in mind that the Leafs franchise makes more money than any of its competitors.  It has no excuse for being to uncompetitive. If the Maple Leafs dodge the playoffs this year, it will be a feat of management, coaching, and athletic futility worthy of a Harvard case study.

It is against this backdrop that SAS pastes its logo alongside the banner message: ANALYTICS: THE GAME CHANGER. This gives rise to the question, “How have they changed the game?”

The Leafs don’t seem to be much different. As a group, they remain the unlovable losers they’ve been during the long droughts between quixotic heroes like Matts Sundin, Dougie Gilmour, Wendell Clark, Darryl Sittler, Eddie Shack, and Dave Keon.

They’ve been so bad for so long, you could launch a multi-billion dollar class action suit for productivity lost due to depression. Someone should pay for all those Saturday night come-from-ahead losses, Sunday morning hang overs, and mournful Mondays around the water cooler. Plus ca change….

Although every movement of every player is now tracked and analyzed every second of every game, I can’t find any metrics about the performance of Toronto’s new, SAS supported analytics team, Darryl Metcalf, Rob Pettapiece and Cam Charron. Last summer, Leaf’s GM Brendan Shanahan purged the coaching assistants, replacing them with, among others, the current coach, Peter Horachek. At the same time, he made a highly publicized commitment to incorporate analytics in everyday decisions about the team.

Head coach, Randy Carlyle, grudgingly toed the company line, conceding that analytics were an inescapable fact of life in professional sport. After “Moneyball” glamourized the power of Billy Beane’s sabermetrics to help baseball clubs perform better on the diamond, and on the balance sheet, hockey had followed other sports into the big data revolution. For a desperate organization like the Leafs, a spastic, full length lunge at this apparent panacea was inevitable.

Of course some of these analytic tools were already available, but the former GM, Brian Burke, was skeptical about their value. He took the view that decisions could be validated by analytics, but couldn’t be based on them.  To quote him, in a moment of prime Burkiness, “Statistics are like a lamp post to a drunk . . . useful for support but not for illumination.”

So it’s safe to assume that analytics weren’t central to Burke’s decision to hire Carlyle, nor to assemble the team inherited by the current GM, Dave Nonis, nor to Carlyle’s coaching philosophy and tactics. With the analytics bar set so low in 2013-14, the game changing impact of SAS-powered analystics should be noticeable by the midpoint of the 2014-15 season.

Yet the Leafs are still the Leafs. Phaneuf is still a weak captain and a worse defenceman.  Horachek is still a lame duck coach, and Kessel is still uncoachable. These qualities aren’t apparent in quantitative data. Nor is the delicate ecology of locker room leadership and off-ice inspiration evident in charts that track ice time, attacking zone starts, shots taken, shots given, shots blocked, goals for, goals against, etc.

If, for example, a senior player sets a horrible example for the rookies, it may play out in individual performance data without revealing anything about the complex, off-ice relationships that slowed the rookies’ progress.  Symptoms of a problem may show up in their performance data, but analytics won’t identify a cause or suggest a cure.

In this regard, Burke and Carlyle were right to resist reliance on analytics. The value of data-based hockey decisions is exaggerated in terms of both relevance and reliability. They never said that it is utterly irrelevant and unreliable, but that it should be subordinate to human judgement.

Think for a moment about why sabremetrics made so much sense in baseball. Baseball is much simpler than hockey. It’s a team sport in only the weakest sense. The two most valued skills, pitching and hitting, are done in isolation. The batter is alone in the box. The pitcher is alone on the mound. Their numbers determine their salaries, whether or not their team wins or loses, whether or not they switch employers. The game is static, broken down into very short intervals of highly repetitive and predictable actions, making every aspect of fielding, throwing, and running the base paths, easy to measure.

Indeed, it is this constant counting and calculating that fills the time between moments of excitement in a baseball game. Batting and earned run averages are flashed up on the scoreboard every time a new players steps onto the diamond. Statistics are the lifeblood of the game that’s otherwise sedentary, slow, and frankly, dull.  There’s more continuous action in a curling bonspiel, with a lot less spitting and jock scratching.

To a lesser extent, the same is true of basketball and football. Football provides so little action, there is plenty of time left to ruminate over statistics. The average football play is 10-15 seconds long. It takes three hours and twelve minutes to complete an NFL game in which the players are in motion for a mere 11 minutes. That leaves plenty of time for data wonks to talk stats.

Basketball is more fluid and improvisational, however its economy, like baseball’s, is based on individual statistics. Lebron James is the NBA’s dominant player, and is paid accordingly whether he’s winning championships in Miami or struggling with second-rate talent in Cleveland. Like other NBA stars, the spectacle of his talent eclipses his team’s performance. He doesn’t have to lift them up to have superstar status. He could choose to outshine  them, like Coby Bryant, setting scoring records on a team that’s going nowhere.  The NBA is full of one-man highlight reels.

Hockey is different in ways that limit the relevance and reliability of analytics. Unlike baseball and football, the pace is furious, making data collection more difficult. Unlike baseball and basketball, it is a violent contact sport, demanding unquantifiable character traits such as courage and discipline. Unlike football and basketball, elite players can only succeed by elevating the play of their teammates through inspirational playmaking.

The last point probably adds the most complexity to the application of analytics.. It’s easy to think of football, baseball, or basketball stars who put up astonishing numbers while playing for ordinary teams. They can excel as individual athletes who succeed without relying on their supporting casts to the same degree as hockey’s superstars.

Look at the difference between the careers of Crosby and Ovechkin, for example. Pittsburgh players couldn’t resist the opportunities that Crosby created for then. They play better because he’s there, and he plays better because of them. Washington, in contrast, has taken years to figure out how to exploit the prodigious talents of the sulky, selfish, Ovechkin, whose name may never be engraved on the Stanley Cup.

Wayne Gretzky would have put up great numbers anywhere he played but he optimized his personal abilities, while winning Stanley cups, by making his teammates better. He used his heightened spatial awareness and anticipation to put the puck where he wanted his teammates to go, creating opportunities for them that they couldn’t even see. If he didn’t, opposing defenders could have given him all their attention, diminishing his individual achievements and team and his teams’ records.

When Lebron’s teammates don’t give him the ball enough, he loses interest. Tom Brady throws the ball to where his receivers are expected to be, according to the play book, and if they’re not there in time, the pass is wasted and he’s visibly annoyed. Juan Bautista can’t afford to think about teammates when he swings for the fences; their presence is irrelevant in that concentrated moment.

In hockey, Phil Kessel’s early breakout from the Leafs’ zone helps offensively, but hurts defensively, resulting in a higher scoring total and a lower +/- average.  Praised as a pure scorer, he’s allowed to deviate from basic principles of team play, but at a cost to his team’s support of his individual aspirations. Individual greatness is harder to achieve in a violent, high-paced contest that requires improvisation and leadership, courage and self-discipline, in addition to the strength, speed, and skills required by all sports.

Analytics aren’t sophisticated enough to discern the subtle interplay of individual and team dynamics, or to isolate the effects of relationships between management, coaches, elite players, and the lesser lights who make their achievements possible. Data on positional play and situational tactics are clearly valuable, and cannot be analyzed without sophisticated tools. But the evaluation of character, motivation, experience, and relationships isn’t adequately factored in.

leafs-2-water-demandAs a layman, analyzing the Leafs, I tend to focus on different metrics to explain their poor performance. First, hockey fans are passionate and loyal to the sport, in all its forms, at all times, in all places. As proof, I offer the City of Toronto’s chart of water demand during the last Olympic gold hockey final.

Now these are metrics I can understand. Manic fans were straining their bladders throughout all three periods, then dashing for the bathroom during each pause in the action. Some must have been bursting during the overtime. Obviously the Maple Leafs don’t have to be marketing geniuses to sell hockey in Toronto.

Here is another measure of success that really matters. According to Forbes, the franchise is worth vastly more in 2014 than it was back in the 90’s, despite the fact that the team is no more competitive now than it was then.  It’s on-ice success rate has been 55% or lower for 15 of the past 24 years, and 8 of the past 10 years.

What this suggests is that the Maple LeafMLSE valuations, data-drunk media and fans, have been analyzing the wrong numbers in the wrong way. They’ve been studying the players. They should have been studying the fans and the owners of the franchise. That’s where the real action is.

This is a business that succeeds in spite of itself. It’s worth more than ever to investors even though its product is certifiably mediocre. Because it has a monopoly on professional hockey in a market that’s mad for the sport, demand exceeds supply. They can’t lose.

The only way they can improve profits is to add revenue from a few playoff games, but this is incidental to their real business. If they started winning, they couldn’t market the melodrama of failed draft picks, free agent signings, coaching changes, and media meltdowns. No one would pay attention if the Leafs’ story was just about hockey. Now they’re a national tragedy, a car wreck in the centre lane that no one can resist gawking at.  The gate revenue from a few playoff games may not be worth the reputational damage they’d suffer by winning consistently.

Which brings me back to my question about SAS. What is this company, with its $3 billion+ in annual sales, saying about the quality of its software and consulting services?  If they can’t shed light on the Leaf’s woes, maybe they’ve proven Burke and Carlyle right. Maybe MLSE, with its bizarro business model, is like the drunk, using analytics for support, not illumination.

This is a silly example of the difficulty we all have marshalling vast amounts of data for previously unimagined purposes.  Our reach exceeds our grasp.  Though we have become more sophisticated in the technology and methods of analysis, the logic required to interpret and apply what we “learn” hasn’t caught up yet.  The results can be entertaining, as in this case, or can lead to more serious consequences, as in the misrepresentation of polling data I raged about last summer.

Comments are closed.