HSX Forum

Movies

Reply Topic List All Forums

report abuse

So here is the answer to JDolphin's question about how prediction sites performed with blockbusters:

Posted by: RogerMore on Jan 01, 09:52 in response to RogerMore's post Following on from last night's movie prediction accuracy...

These are the average accuracy for each site for the top 20 openers of the year - AVNGR BATM3 HGAME TWLI5 SPID4 JB23 HOBIT LORAX BRAVE MADG3 MIB3 SWHTM TED TAKE2 PROME RALPH ICEA4 HTRNS THVOW BORN4

RS 62.0%

MTC 78.0%

BOCOM 78.3%

WW 72.5%

BOG 72.6%

BOM 77.6%

FilmGo 77.9%

EW 77.3%

LAT 78.3%

V 78.1%

THR 76.8%

Whisper 77.6%

Average 77.7%

Most of these are between 76% and 78%, so the prediction sites were actually more accurate for blockbusters than they were on average. This makes sense because while it's harder to pinpoint the exactl number, it's a lot easier to get within 10% of the right number. For a $10 million opener, you can get 90+% accuracy by predicting anywhere between 9 and 11 million. With a $100M opener, you can get the same accuracy with a prediction of anywhere between $90M and $110M. Easier.

So instead, let's look at the biggest surprises of 2012 and how the various prediction sites performed and see if anyone stands out. The 20 most profitable adjusts (in H$ terms, if you were holding them the right way at Friday reset) of the year were AVNGR HGAME LORAX THVOW TED DVNSD BATM3 SAFEH JB23 HTRNS SWHTM ICEA4 ROAGE TLKAM HOBIT MGCMK TMBOY MIB3 TAKE2 PNAC4

RS 53.4%

MTC 68.1%

BOCOM 55.8%

WW 56.0%

BOG 63.0%

BOM 61.2%

FilmGo 52.9%

EW 64.2%

LAT 64.4%

V 60.8%

THR 66.7%

Whisper 57.0%

Average 59.8%

Obviously the numbers are a lot lower because all of these openers surprised everyone. Interestingly, the best ones at picking the surprises - in the mid and high 60s - were all the ones with direct access to tracking from customer surveys - MTC, THR, LAT and (presumably) EW. I think this shows that even if you can't swear by tracking, you certainly can't ignore it either. Sites that performed their own analysis did less well, which to some extent supports moviesnob's suggestion of groupthink and reluctance to putting out a number that goes against the perceived outcome.

But at the same time I think that a number is just a number - and the analysis can still be correct even if the number is wrong. We've seen plenty of weekends where one predictor has been very upbeat about a movie, while another is more pessimistic - yet they have still come out with very similar number predictions. So I wouldn't dismiss a site with low accuracy as being one not worth visiting. I'd rather judge them by the quality of their analysis.

Following on from last night's movie prediction accuracy thread, here are the accuracy figures for 2011 and 2010. RogerMore Jan 01, 09:27

Time for RS to update its model. They are consistent losers. {nm} elchan Jan 01, 09:38

Can you believe they are a business? Makes me think I should start my own movie tracking company. {nm} RogerMore Jan 01, 09:54

i dont think they make much money off it. it supplements their main business (they are a theater chain) {nm} Moviesnob Jan 01, 12:35

Then I hope they don't rely on it when deciding how many theatres should show the new releases :) {nm} RogerMore Jan 01, 14:44

It might work for them, at least on a small scale, their data might line up for them, but it's less useful on a national scale {nm} Moviesnob Jan 01, 15:04

Yeah, I'm just being snarky. Thanks for the info. {nm} RogerMore Jan 01, 16:21

So here is the answer to JDolphin's question about how prediction sites performed with blockbusters: RogerMore Jan 01, 09:52

Wow - thanks a lot for all that work!! Happy New Year! {nm} JDolphin Jan 01, 13:20

How did I do overall at BOF for 2011? think somewhere in the 73-75% range? {nm} xiayun Jan 01, 10:05

I'm away from my computer at the moment, but for Jan to May 2012 you were at 77%, second to EW RogerMore Jan 01, 10:54

thx for putting all that together {nm} tealfan Jan 01, 10:24