Netflix, Amazon ratings explained: What do these numbers actually mean?

The Irishman, the latest from filmmaker Martin Scorsese and his Raging muse Robert De Niro, propelled Netflix into the awards race this year with five Golden Globe nominations, including for best drama film and best director. Outside of awards, the film seems to be receiving just as much attention.

On Tuesday, Netflix tweeted that The Irishman was watched by 26.4 million accounts globally within the first seven days on the platform, a figure repeated by Netflix head Ted Sarandos during a conference in New York. Third-party ratings tracker Nielsen logged a reach of 17.1 million unique views in the U.S. for The Irishman in the first five days.

Which leads to a few questions: Where do these numbers come from; what’s the difference between “watched by accounts” and “unique views”; why are the numbers different; and what do these numbers even mean?

The latter is a query that has dogged the industry and applies to other streaming video on demand (SVOD) platforms besides Netflix. Most of these services notoriously keep their viewer metrics close to the vest; for a time, there was no third party to verify numbers when one of them, like Netflix, released data.

Nielsen is attempting to shed more light on the matter in new efforts to track viewing data for streamers, though with some blanks yet to be filled in. In a phone interview with EW, Brian Fuhrer, Nielsen’s SVP of Product Leadership, explains part of the goal. “What I wanted to do when we put this data out,” he says, “is to have milestones to compare it to.”

How it works

Nielsen tracks the ratings for SVOD programs in a very similar way they track ratings for standard broadcast and cable series: through Nielsen Families. These are housholds across the country chosen by Nielsen to act as their data pool, consisting of varying demographics. The families who are selected and choose to participate allow the company to collect data on specifics, like who’s watching any given program on their television; how many people are watching; when they’re watching; and how long they’re watching for.“We use the same exact technology — we’ve expanded it and enhanced it — that we do for all our traditional television ratings,” Fuhrer says. Through transmission streams from devices connected to TVs, audio encoders within those streams, and additional data provided to Nielsen by these Nielsen families, the company is able to track U.S. viewers. Stats on international audiences, as well as programs watched through mobile, tablet, and computer devices are not included in their data.

For The Irishman, Nielsen broke it down in terms of average audience and reach — the same way it breaks down programs on non-streaming TV networks, Fuhrer says. When The Irishman premiered on Netflix on Nov. 27, Nielsen determined the film drew in an “average minute audience” (an average audience) of 2.6 million viewers in the U.S. with a “reach” (those who watched at least 6 minutes of the program) of 3.9 million unique views. Over the course of the film’s first five days on the service, Nielsen determined the film had an average minute audience of 13.2 million viewers and a reach of 17.1 million unique views.

On The Irishman‘s premiere date on Netflix, 751,000 U.S. viewers watched the film in its entirety; Nielsen states that’s roughly 18 percent of its audience that day. On the Friday following the film’s Netflix debut (Nov. 29), Nielsen tracked 930,000 U.S. viewers watched the film in its entirety, which accounts for rewatches.

Before dropping on Netflix, The Irishman screened in select theaters in five markets, beginning on Nov. 1. To date, the film grossed $825,204.

So, is this a “success” for Netflix? “There are so many different variables that go into defining a success,” Fuhrer says. For instance, though he initially thought the viewing metrics for Netflix’s The Crown were low, he still considers it a success because it hit with an older audience, bringing a new demographic to subscribe to the platform.

However, he did add that The Irishman “definitely did have a strong audience.” That’s based on comparison data taken of Bird Box (the Sandra Bullock-led sci-fi film) and El Camino: A Breaking Bad Movie (starring Aaron Paul). According to Nielsen’s numbers, Bird Box‘s premiere date saw an average minute audience 2.9 million U.S. viewers, with roughly 18 percent watching in its entirety. El Camino, meanwhile, had a 2.6 million average minute audience on its premiere with 11 percent watching in its entirety. Over the course of their first five days, Bird Box‘s average rose to 16.9 million and El Camino‘s came out to 8.2 million.

Netflix records its metrics internally differently. If one Netflix user account watches at least 70 percent of a film, that counts as one view. That account “may include multiple views and viewers but is only counted once,” a rep for the service previously clarified to EW. In other words, if your parents are watching The Irishman from your hometown, while you’re watching the movie from the same overall account in the comfort of your own home across state lines, that view still counts as one. So, when Netflix said The Irishman was watched by 26,404,081 accounts globally within the first seven days on the platform, that means 26.4 million user accounts watched more than 70 percent of the film in that time frame.

After Bird Box‘s premiere, a shareholder letter from the company, released in January 2019, said that film had garnered 45,037,125 account views in December and “over 80 million member households” within “its first four weeks on Netflix.”

For TV, there’s more “nuance,” as a Netflix rep mentioned over previous emails. One view is added if a user account watches at least 70 percent of one episode of a series. (The same rules about multiple users in the same account applies.)

On Jan. 17, Netflix said YOU was on track to be viewed by more than 40 million members in its first four weeks on the service after season 1 dropped on Dec. 26. Translation: Netflix projected 40 million Netflix accounts around the world would have watched at least 70 percent of one episode of YOU within the series’ first four weeks on Netflix. How many of those accounts include those users who binged the entire first season of YOU? How many only watched 70 percent of the first episode and washed their hands of it? It’s unclear.

The other major SVOD platform that Nielsen tracks is Amazon. Jennifer Salke, head of Amazon Studios said in August 2019 that The Boys, which premiered July 26 of this year, became “one of our most watched Amazon Original series by our Prime Video customers.” No specific figures were mentioned at the time, but then Nielsen reported The Boys averaged 4.1 million viewers per episode over the first 10 days. A rep for Amazon declined to speak on this topic.

While it feels like new streaming platforms are popping up every day — Disney’s Disney+, WarnerMedia’s HBO Max, NBC’s Peacock, and the mobile-first Quibi were announced this year — Nielsen currently only offers external data on Netflix and Amazon, though Fuhrer hopes to broaden that platforms in the future.

Why it matters

There are even more discrepancies between how Nielsen and Netflix track their shows. (And Amazon remains quiet on the topic.)

Nielsen tracks in the U.S. only, while Netflix tracks worldwide. Nielsen tracks through devices connected to a television screen, while Netflix includes mobile, tablet, and computer views. Nielsen releases both average viewer metrics for entire seasons and episode-by-episode, while Netflix (externally at least) does not.

Nielsen tracks metrics in this way in the hopes of developing something as close to standard for the television industry. “We have the obligation to the industry and our clients to put it out in exactly the language that they’re used to seeing and have it be comparable to traditional TV,” Fuhrer says. SVOD platforms, however, are not standard television, which makes this process difficult.

Fuhrer agrees that streaming audiences consume shows differently than audiences for traditional networks, often times binging entire seasons at a time or watching series weeks or even months after the programs premiere. That’s why Fuhrer says they’re constantly trying to refine their data.

“The question that everyone asks is, ‘What is my rating?'” Fuhrer mentions of Nielsen’s clientele. “But then the second question asked is, ‘Okay, but how does it do in comparison to everybody else?’ So there’s a little bit of tension in the industry.”

One such tension includes shows like YOU. Season 1 of the Penn Badgley-led series premiered on Lifetime with 824,000 overnight viewers (not accounting for DVR) and a 0.2 rating among adults 18-49. By its season finale, it drew 526,000 overnight views. Netflix had already scooped up the international rights to YOU before then taking the domestic rights. When you consider the discrepencies between how Nielsen and Netflix track their metrics, does Nielsen’s 824,000 premiere figure compare to Netflix’s reported 40 million across four weeks? No.

Another lies in industry entities doing businesswith streaming platforms, whether that’s SVOD looking to make a deal to acquire streaming rights to popular programs (ex: Friends, The Office) or a SVOD platform looking to “save” a broadcast/cable show after cancellation (ex: The Expanse, Lucifer). As one source put it, if you’re trying to do business with a SVOD platform and you’re making decisions based on metrics that are only coming from that platform you’re trying to do business with, can those metrics really be trusted as fair representation?

Fuhrer gives points to a different, more real-world application of this data. One Day at a Time was canceled by Netflix this year in a move that seemed to deeply rile the fans. It was then picked back up by Pop TV. “It was the first time where [a client] took our Netflix data to sell it into the broadcast networks,” he says. “They [Sony TV] took the viewers on Netflix and they showed that they over indexed. These are heavy viewers of comedy on broadcast networks. They said, ‘You put this on there you’re going to be drawing a lot of viewers'”

Any questions?

Yes, we have a few. So does Nielsen.

The main thing is to not blow these numbers out of proportion. Some will be inclined (and have in the past with previous Netflix films) to equate Netflix or Nielsen’s metrics of The Irishman to movie ticket sales. “It’s really hard to take something that premieres, in effect, for free on TV — obviously with the subscription to Netflix — and compare it to how it does against going to a movie,” Fuhrer warns.

“There are always blanks spots and there are always new blank spots,” he adds, but the “No. 1 takeaway is to make sure you see this [data] so you can appropriately evaluate the content.” Fuhrer hopes to bring more speed to Nielsen’s data tracking, reach a wider international data pool, and more firmly understand which viewers are watching on mobile and tablet devices.

Netflix, too, has promised more transparency, though we’re not sure when that time will come. A rep for Netflix was not able to offer more context or insights for this story outside of what Netflix already disclosed previously over social media and in financial statements. “Over the next several months, we’re going to be rolling out more specific and granular data and reporting,” Sarandos said in April. “First to our producers, then our members and, of course, to the press over time and be more fully transparent about what people are watching on Netflix around the world.”

For Nielsen, Fuhrer says, “The biggest thing right now is just continue to generate the data.”

Related content: