We’re often told that something increased or decreased by a certain percentage.  This isn’t very useful unless we’re also given some additional information.  Here’s an example.

In mid November online media and news websites that cover gaming and digital technology widely reported that Yusuf Mehdi, the Head of Xbox Business Strategy and Marketing at Microsoft, announced in a blog post that Xbox One (XBO) sales “have more than tripled”. Sales increasing by more than 300% sounds very good.  Is it as good as it sounds?  Did the gaming and technology press ask this question?

Simply stating that something increased or decreased by X% doesn’t really tell you very much. You need more information to determine if the increase or decrease is large enough to be important.  The first and most basic thing you need to know is what was the original number that increased or decreased.  If the original number is small, the increase or decrease will also be small.  To take an extreme example, if Microsoft had only sold 1 XBO, their sales would triple if they sold 3 more. To take a real example, book sales for my book business, Monkey Books, more than doubled from September to October.  That doesn’t sound as good as XBO sales more than tripling but it still sounds pretty good.  In actuality, October was a little better than average for Monkey Books but September was the worst month we’ve had in almost two years.  October sales were over 200% better because September sales were so low.  If you don’t know the original number, you don’t know very much at all when you are told that something increased or decreased by a certain percent.  This is so basic and fundamental that reporting the original number should be automatic.

The second thing you need to know when you hear one of these “sales increased by X%” statements is what the time frame is. Did XBO sales triple from one day to the next? One week to the next? One month to the next? One quarter to the next?  Sales tripling from one day to the next is not likely to mean very much; sales tripling from one quarter to the next could be important.

Finally, to evaluate a statement like Mehdi’s you need to have a context for comparison. Mehdi’s blog post appeared in mid November when Christmas shopping shifts into high gear.  Are console sales typically higher in November than October? Less than three times higher? More?

The Playstation 4 (PS4) and the XBO both launched in October 2014 and the PS4 had outsold the XBO every month since launch when Mehdi posted his “sales have tripled” announcement. He posted the announcement about two weeks after Microsoft cut the price of the XBO by $50 for the holiday sales season. For the most part the gaming and technology press responded to Mehdi’s post by either dutifully reporting the increase as fact or citing it as evidence that the price cut was successful and the XBO was gaining ground on the PS4.  With only one notable exception, the basic and important questions of what was the base number of sales that tripled, what are the time frames involved, and how much do console sales usually increase in November, were not pursued or even mentioned by many online media sources that cover gaming or digital technology.

Unfortunately it’s impossible to answer questions about how many XBOs have been sold to date or when those sales took place. In April Microsoft announced that 5 million units had been shipped to retailers but did not provide any information about how many of those consoles had been sold through to customers. Since April Microsoft has tended to make announcements about “XBox sales” that combine XBO sales and Xbox 360 sales in a single number.  Sony, meanwhile, has reported that the PS4 is averaging roughly 1 million units sold to customers per month since launch. What evidence is available indicates that this more, and maybe a whole lot more, than the XBO is selling.

Although questions about how many XBOs were sold in October and November cannot be definitively answered, the gaming and digital media should have brought these questions up.  The “sales have tripled” claim is empty if you don’t know how many sales you’re talking about.  If Microsoft matched Sony’s 1 million units sold to customers in October and tripled that number in November, they had a great month.  If they sold 100K units in October and 300K units in November, they’re in serious trouble.

The time frame over which XBO sales tripled was provided by Mehdi in his blog post when he wrote “. . . sales have skyrocketed since the new price took effect on Nov. 2. Compared to the previous week, Xbox One sales in the US have more than tripled . . .” Mehdi and Microsoft deserve praise for giving us this basic and necessary information.  Unfortunately, the fact that sales tripled in the week following a major price cut when compared to the week preceding the price cut makes the sales increase much less impressive.  First, a single week is much too small a time period to provide any important information about the overall performance of a new game console.  Too many factors can skew sales one way or the other during such a small time frame.

More important, there is an obvious factor involved in the two week period that Mehdi reported that leads you to think that sales tripling during this period may have little or no importance.  Microsoft announced the price cut for the XBO a week before the reduction went into effect and the impending discount was widely reported in the gaming and technology press.  This would lead you to suspect that anyone knowledgeable about the consoles who had been considering buying an XBO would wait a week until the $50 price reduction went into effect.  If this happened, Mehdi’s “previous week” may well have seen a markedly reduced number of XBO sales while the following week may have seen an inflated number of sales as people who had been waiting bought the console as soon as the sale went into effect.  If those two things happened, sales tripling over this particular two week period isn’t very impressive.

The gaming and technology press should have pointed this out as it’s critical for evaluating the “sales have tripled” announcement. As far as I know, no one did.

The only online source that covers gaming that I saw that brought up the issue of providing a context for comparison for the “sales have tripled” announcement was an article in Forbes that pulled together Xbox 360 sales data for October and November in the years 2009 through 2012.  2013 is an outlier because the XBO launched that month which almost certainly had an effect on Xbox 360 sales.

As expected if November sales ramp up as people shop for Christmas, the Xbox 360 sold more units in November than in October every year from 2009 through 2012.  The percentage increases in the number of Xbox 360s sold in November compared to October reported in the Forbes article are as follows.

• 2009 – 328%
• 2010 – 422%
• 2011 – 433%
• 2012 – 467%

This is information that is important for understanding Mehdi’s announcement that XBO sales had tripled and Forbes deserves praise for providing it.  The article in Forbes is dismissive of the importance of Mehdi’s announcement based on the argument that sales tripling in November is just business as usual. Unfortunately, Forbes’ argument is beside the point. Mehdi wasn’t comparing November to October when he wrote that XBO sales had tripled, he was comparing the first week in November to the last week in October, and he explicitly made this clear in his blog post. Forbes ignored this.  Given that one can expect consoles to increase throughout November as we get closer and closer to Christmas and that there is the further expectation of a large increase in sales at the end of November when Black Friday and Cyber Monday take place, you can’t legitimately drawn any conclusions about November sales based on sales during the first week of the month.

When all is said and done what can be said about Mehdi’s announcement regarding XBO sales?  XBO sales tripled in the first week of November as compared to the last week in October.  This doesn’t give us any useful information about the success of the XBO console in general because sales in the last week of October were likely to have been artificially depressed as people waited for the announced price cut and sales during the first week in November were likely to have been artificially inflated because of people who had been waiting and bought an XBO as soon as it went on sale.

What can be said about the online gaming and digital media’s coverage of Mehdi’s announcement?  They did not address the basic and fundamental question of what was the original number of units sold in the last week of October that tripled in the first week of November. They didn’t point out that a week-to-week comparison of sales is too small to be meaningful. They didn’t point out that a sales comparison over these two weeks is especially meaningless because of the price reduction that separated one week from the other. One article provided important data about typical Xbox sales in October and November and then misused these data by using them to dismiss a sales announcement that compared one week to the next. The media failed miserably.

It’s not really very surprising that a spokesperson for a major corporation like Microsoft would present information about the company and its products in the best possible light (speaking kindly) or in ways that are knowingly and purposefully  designed to be deceptive and then vetted by corporate lawyers to avoid claims of fraud (speaking unkindly).  It’s disturbing, however, that, for the most part, the online media that covers gaming and digital technology didn’t ask the basic and obvious questions that have to be answered if “sales have tripled” is going to be anything more than empty and meaningless marketing blather. You don’t need advanced degrees in economics or mathematics to either think of or try to answer these questions.  The math involved is taught in grade school. You just need to think about what you’re told and then ask what should be the first questions that come to mind.