Posted on

An Exercise for Bias Detection

A great exercise to train your bias-detecting skills is to check on a high volume of outlets –say, eight to ten–across the political spectrum in the 6-12 hours right after a big political story breaks. I did this right after the release of the Nunes memo on Friday, Feb 2. This particular story provided an especially good occasion for comparison across sites for several reasons, including:

-It was a big political story, so nearly everyone covered it. It’s easier to compare bias when each source is covering the same story.

-The underlying story is fact-dense, meaning that a lot of stories about it are long:

-As a result, it is easier to tell when an article is omitting facts.

-It is also easier to compare how even highly factual stories (i.e., scores of “1” and “2” on the Veracity and Expression scales) characterize particular facts to create a slight partisan lean.

-There are both long and short stories on the subject. Comparison between longer and shorter stories lets you more easily find facts that are omitted in order to frame the issues one way or another.

-News outlets have had quite a while to prepare for this coming story, so those inclined to spin it one way or the other have had time to develop the spin. Several outlets had multiple fact, analysis, and opinion stories within the 12 hours following the story breaking. You could count the number of stories on each site and rate their bias and get a more complex view of the source’s bias.

I grabbed screenshots of several sources across the spectrum from the evening of Feb. 2 and morning of Feb. 3. These are from the Internet Wayback Machine https://web.archive.org (if you haven’t used it before, it’s a great tool that allows you to see what websites looked like at previous dates and times).  Screenshots from the following shots are below:

FoxNews.com

Breitbart.com

NationalReview.com

RedState.com

WashingtonPost.com

NYTimes.com

Huffpost.com

TheDailyBeast.com

Slate.com

BipartisanReport.com

 

You can get a good sense of bias from taking a look at the headlines, but you can get deeper insight from reading the articles themselves. For some sources, the headlines are a lot more dramatic than the articles themselves; for others, the articles are equally or more biased.

If you want to rank these articles (based on the articles themselves, or just on the headlines and pages below) on a blank version of the chart, I recommend placing the ones that seem most extremely biased first, then placing the ones that seem less biased. It’s easiest to identify the most extreme of a set, and then place the rest in relative positions. There’s not always a source or story that will land in whatever you consider “the middle,” but you can find some that are closer than others.

Going through this exercise is especially beneficial when big stories like this break. I know it is time-consuming to read so many sources and stories, so most people don’t read laterally like this very often, if ever (if you do, nice work!).  Doing so from time to time can help you remember that people are reading very different things from you, and increase your awareness of the range of biases across the spectrum. It can also help you identify how detect more subtle bias in the sources you regularly rely on.

Happy bias-detecting!

FOX NEWS

BREITBART

NATIONAL REVIEW

REDSTATE

 

WASHINGTON POST

NYTIMES

HUFFPOST

THE DAILY BEAST

SLATE

BIPARTISAN REPORT

 

Leave a Reply

10 Comments on "An Exercise for Bias Detection"

Notify of
avatar
Sort by:   newest | oldest | most voted
David G Shaw
Guest
Thank you Vanessa! Wonderful effort! Coincidental timing to say the least. I was trying to explain a chapter in a book I’m writing to a couple of close (politically biased) friends this morning where that explains media bias as being driven by formulated marketing efforts that more or less “flatter” the biased viewer by validating their opinion appealling to their ego, dopamine release, etc. for the sake of higher and deeper market penetration for increased advertising revenues; needless to say after two hours of reason and analogies I hadn’t gotten very far in gaining any acceptance of this as a… Read more »
Joe
Guest

Question, not a comment: during the time you have been a careful, methodical analyst of news media outlets, and focusing on the top middle tier of those outlets (AP, Reuters, NYT, etc.,), have you noticed a shift in the frequency and amount of opinion contained in their news articles?

Birrell Walsh
Guest

Thank you!

I am grateful when someone uses their education to benefit and clarify the public discourse. I also appreciate the hard work involved – so thank you a second time.

Ken Rhines
Guest
Too funny! I picked the same event to dig into, for the same reasons! I’m focusing on looking at omitted facts by horizontal category, and whether there are consistent patterns. This requires work: first reading and ingesting a lot of articles across a large sample of sites to create a complete fact “set”, and then doing all kinds of counts and summaries to see who reports –or doesn’t report– what. And then there’s the overlap of “news,” “analysis” and “opinion” just to make things interesting. But, I’ll let ya know how it goes when I’m done. Your point about how… Read more »
Andrew
Guest

Hi Vanessa,

Any chance you could at The Young Turks to your media biased chart? It would be great to see where you rank them. It’s also a little funny they didn’t make the cut.