First-Of-Its-Kind Study Shows Critics Often Way Off From Public On Movies
NEW YORK — Critics’ reviews are often relied upon by moviegoers when it comes to deciding whether or not to take a trip down to the box office or watching movies on demand. As it turns out, those all-important reviews aren’t so important after all — a first-of-its-kind study finds that movie tastes are largely idiosyncratic, and quite often differ from the opinions of the “experts.”
Researchers at New York University (NYU) surveyed a set of over 3,000 individuals on what they thought of 200-plus big studio films, hoping to find similarities between how the general public and critics perceived the quality of a box office blockbuster. The study is believed to be the first to examine how the general public’s opinion matches up with how the “pros” feel.
“Critics may be adept at evaluating films, but that doesn’t mean their assessments will accurately predict how much the public will like what they see,” says co-author Jake Whritner, a graduate of the Cinema Studies Program at NYU and currently part of the Cognitive and Data Science Lab at Rutgers University-Newark, in a NYU news release.
In addition to their films of preference, the participants were also asked to provide demographic information (e.g., their age, gender, etc.), and whether they considered the reviews of professional critics in deciding to see a movie.
For a basis of comparison, the NYU researchers pulled professional reviews from 42 critics or rating sites, such as IMDB, for each film.
While the researchers found that opinions on a given film greatly diverged, a general trend was observed: the more watchers a film had, the higher its ratings.
Meanwhile, the review of a professional critic, such as the late Roger Ebert, didn’t reflect the perception of the general audience better than an individual randomly selected from the sample. The only segment more likely to fall in line with critics, the researchers discovered, were other critics.
Similarly, the segment most likely to agree with the opinions of the non-critics were those of other non-critics on the mass rating sites like IMDB.
“What we find enjoyable in movies is strikingly subjective—so much so that the industry’s targeting of film goers by broad demographic categories seems off the mark,” says Pascal Wallisch, a clinical assistant professor in New York University’s Department of Psychology and the senior author of the study.
Interestingly, gender played a noteworthy role in the results. Males were significantly more likely to agree with one another on a film’s quality than females, although there wasn’t high agreement as a whole. Age wasn’t found to be a reliable determinant of opinion.
The researchers determined that people who disagreed on movies were typically about 1.25 stars apart from one another on a scale of 0 to 4 stars.
“Something about being a critic seems to make the recommendations of critics unsuitable for predicting the movie taste of regular people,” the authors stated in regards to their findings. “This study is the first to quantify this in an adequately powered fashion, and it helps to explain why people often perceive critics to be out of touch.”
Because film preferences vary so greatly, the study’s authors recommend “either consult[ing] sites that aggregate individual judgements, or find[ing] other individuals or critics with similar tastes.”
The study’s findings were published in the journal Projections.