Posted on

News Quality

 

news-quality-v4

We are living in a time where we have more information available to each of us than ever before in history. However, we are not all proficient at distinguishing between good information and bad information. This is true for liberal, moderate, and conservative people. I submit that these two circumstances are highly related to why our country is so politically polarized at the moment.

Why is it that I can have such different views on the same subject or topic as someone else who lives in the same country? Take the polarizing example of people’s opinions on Hillary. Why do I think she is qualified and inspiring but others think she is literally evil incarnate? I don’t know her personally. And neither do you. We must both admit that our opinions of her are informed by the news sources we read and believe. And news sources vary widely in what they report.

Which news sources should we believe, when there are so many to choose from, and each one is telling you not to believe another one? I put together this chart of which news sources I think you should use and which ones you should not. If you value my opinion as someone who both is reasonable and well-informed, you may find it helpful. If you don’t really care what I think, it will be useless to you. These are my subjective opinions based on having read many news stories from each of the listed sites. The only credibility or authority I can claim in this regard is that I read and write analytically for a living.

Before you look at the chart, I’d like to address the fact that many people object to media sources on the basis that they are “mainstream.” They say “I don’t believe the mainstream media! They are owned by big corporations and do things for money!” But where did they get that idea? From another media source. Remember that each media source has their own incentives (like monetary ones) to get people to listen to them and not to someone else. You have to evaluate media based on something other than the fact that one source told you not to listen to another source.

Remember that journalism is a professional and academic field with a set of agreed-upon standards. People get degrees in it and people who are really good at it get jobs in it at good organizations. Peer review helps ensure mainstream sources adhere to standards; if a story doesn’t meet those standards, other news outlets report on that. Not believing the mainstream media just because it is mainstream is like not believing a mainstream doctor or a mainstream lawyer. Sure, you should question and rate the quality of what the newspaper, doctor, or lawyer says, but you shouldn’t dismiss them out of hand because the paper is big, the doctor works at a hospital, or the lawyer works at a firm.

The chart is pretty self-explanatory. Here are some caveats and reasons for my rankings:

-I am operating out of the assumption that the less blatantly partisan the source is, the more accurate it is.
-I understand that individual reporters, even at the most reputable news sources, have their own personal biases and opinions. The rankings are an overall ranking of each site.
-“Sensational” means the article have titles like “So and so DESTROYS so and so with THIS response!”
-“Clickbait” means the articles have titles like “She walked into a meeting. What happened next will shock you!”
-“Conspiracy theories” means shit that is just made up. Like National Enquirer type stories.
-I’m sure this will offend some people that typically agree with me politically. Sorry.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

70 thoughts on “News Quality

  1. I curate online sources for a news monitoring services. I’m interested in knowing who created this chart, which I saw posted elsewhere early today. I presume you want privacy, so I won’t use names.

    Which of these is true?

    You are V.
    You borrowed this from V.
    V borrowed this from you.

    Thanks!

    1. Hi there! I’m V. Twitter @vlotero. Thanks!

      1. Hi, I’m interested in using this in my English class. We are reading it, and then I am giving them a blank chart to fill out and justify. I changed “shit” to “stuff”. I want to make sure I have permission to use the article and change the word.

        1. Hi there,

          Yes, of course you may change the bad words in the article. Sorry I didn’t make it very kid-friendly on the first go around. Same permission applies for any of my other blog posts. Thanks for teaching!

          Vanessa

  2. I saw this shared on Facebook and liked the image so much I shared it myself. I might have put the NYT and WaPo a touch more left and WSJ a touch more right but overall it’s dead on. Expect this to get some circulation, thanks for making it!

  3. I don’t see CBS News on the chart. Is there a reason for that? Thanks for the cool work you did with this.

  4. So, did you create this and are we permitted to share with credit or not share? This is great, at the very least a great talking point.

  5. Nice Chart. I’ve seen it a few times this morning on twitter & facebook.

    A small labeling suggestion, you should choose one: “don’t read this” or “just no” and use it the same on both sides.

  6. Vanessa (I will use your name since you provided it via your Twitter handle),

    I like the chart ( and agree with much of it). Please don’t get me wrong, this is not meant as criticism, but what is your expertise when it comes to Journalism? I see that you are a Patent Attorney, and you work in Intellectual Property. I am sure that you are well educated. I just want to know how you came to judge the news sources cited in your chart. I am a high school English teacher. I try to teach my students to verify their sources. I would like to use your chart in a lesson, but I am unsure of the methods you used for your evaluation. Can you give me a better idea of how you came to these conclusions?

    1. Absolutely. I have gotten quite a few inquiries of this nature. I will follow up with a blog post soon, because my answer is long.

    2. I will follow up with full post on this question shortly. I’ve been getting it a lot. It’s a long answer.

  7. I know Canada is basically invisible but if you’re going to include BBC, you might want to consider CBC.
    Otherwise, love the graphic.

    1. Agree. Based on my limited experience with CBC, I would say in the within the middle gray circle, towards the top and just slightly to the left.

  8. This is brilliant.

    Of course people will disagree on some aspects. But that’s life.

    I wanted to comment here, because of the amazing amount of stupid comments on the facebook version.

    An interesting experiment would be to chart the stupid facebook comments into four groups:
    * People who think your slant is way left (e.g. “CNN is communist”)
    * People who think you slant right (e.g. “HuffPo is Fascist.”)
    * People who think it’s funny to tell you you omitted something and then posting a link to some ironic outlet (e.g. a tiny, rural paper; a food blog; a special-interest rag like “Kilts and Swords Weekly”)
    * People who linked to something they think is real (e.g. “Veterans Today” holocaust denial, or “Scholars for 9-11 Truth”.)

    1. Haha, I like how you think. I certainly have received enough material!

  9. this is fantastic. it helps I think for people to see it thanks !!

  10. Just curious. Why is CBS News not on this list at all?

    1. It’s there now. Wasn’t on a previous version. Unintentionally omitted.

      1. This is awesome. But, I must say it’s an oversight to exclude The New Yorker while including The Atlantic!

  11. Thanks for your chart, which I feel is quite accurate.

    One intimately related issue regarding the worth of news sources is whether or not they cover stories that should be covered, or shouldn’t, and how visibly they cover them – lead stories with lots of information (and possibly analysis) or buried and with minimal information.

    For example, major news sources generally neglected or avoided covering the Bernie Sanders campaign until he finally was making such an impact that they had to say something about him. Meanwhile, Trump received coverage ad nauseam for saying and doing stupid things, which resulted in a great deal of free publicity for him – at the expense (to us) of not covering some items of real news.

    Also, the protesters-protectors at Standing Rock received little or no coverage until the Big Oil protectors (militarized police) escalated confrontations to the point where there were fires and protectors were being seriously injured.

    When these sorts of things happen, we are forced to seek out less well-known or unknown sources for information, and these sources often are biased and tend to promote an agenda.

    One thing I have been aware of is that if a news source – public radio comes to mind; National and Minnesota – is willing to present differing sides of stories, many conservatives will immediately label it liberal because they hear a liberal side that they reject. It is somewhat legitimate to label public radio as liberal simply because of the fact that it requires a somewhat liberal approach to things to consider a broad spectrum of views., whereas the conservative point of view is inherently narrow and non-inclusive. The bias of the listener defines the bias of the source.

    1. Very good points, Carl. Your response hearkens to the instinct I have, which is that we, as consumers of information, tend to order what we want to eat. That puts a complex burden on journalists, as well as a potential conflict of interest, particularly for those with The better intentions.

    2. I think that grouping people into liberal and conservative is narrow minded. You also don’t seem to grasp your own bias. But maybe none of us truly can.

      1. I fully agree that I am biased. I tried to account for this. If you are interested, I wrote a new post discussing how I tried to factor that in.

  12. I wrote about something similar today, but my old Pew chart isn’t nearly as interesting as yours. May I have your permission to add yours to my post, with appropriate attribution and link?

    1. Yes, of course!

      1. Just came back to say thank you and apologize for being so slow to do so. Your chart generated a lot of thoughtful discussion.

  13. Good chart, but your post has one weird section:

    “They say ‘I don’t believe the mainstream media! They are owned by big corporations and do things for money!’ But where did they get that idea? From another media source.”

    Ownership of media is public record. You don’t need the media to learn that the media is owned by big corporations.

    For example, Comcast owns the following: NBC, Telemundo, CNBC, and over 20 tv stations. They also hold large stakes in MSNBC and The Weather Channel. National Amusements Incorporated controls almost 200 cable channels and over 100 radio stations. Disney owns ESPN and ABC. The list goes on.

    1. I don’t disagree that it is widely known who owns media companies. You are correct that people can find out how media companies are organized and which outlets they own. I meant that mainstream media bashing is mostly done by other, lesser known media sources. My point to this section was that a large corporate structure does not, in and of itself, mean that a media source is disreputable. People think it does though, and I attribute the source of that sentiment mostly to the smaller, fringe media outlets.

      1. Vanessa,

        Has anyone provided you any evidence of the corporations that own these media outlets controlling what is reported through said outlets?

        1. I also have opinions of the role of ownership and money in news, but that will have to be another post.

    2. It is particularly relevant to point out in this discussion that Fox News–of course–is corporate owned and for-profit as much as any of the networks.

  14. All Generalizations are False and youre chart is a genirlaization. Interesting article, the chart shows a gross generalization of a problematic topic. in the article you develop the subject and presents arguments, but part of the problem is that people do not share your article, but only the chart. They accept the chart without any critical thinking and see it as a fact. The problem is not the sources of the information but in the way read them.

    1. If you look at my About page, you’ll see the first thing I say on there is “All generalizations are false, including this one.” Some generalizations are true.

  15. Vanessa,

    Where on the chart would you place the Drudge Report? I am definitely looking forward to seeing your rating criteria, and thank you in advance for taking the time to evaluate this problem.

    1. “Just no,” bottom right corner. I realize they are mostly an aggregator, but an aggregator of other “just no.”

      1. Thank you, Vanessa!

  16. This chart has generated interest and conversation on my Facebook wall today. The main thing that a few of us have been surprised about was the relative vertical ranking of Fox and MSNBC vs. CNN and USA Today (all of which feel about on par with one another to me). I realize you can’t defend every choice you made, but if there was a short rationale for what caused you to put them relative to one another in this way, we’d be curious.

    1. Long reply coming in a new post, but short reply on just the respective vertical rankings:

      Fox: Higher than you probably think it should be because I a tried to account for my own moderate-left bias and consider the high esteem it holds with approximately 46% of the electorate.
      MSNBC: High analytically primarily because of deep dives into individual political stories and lesser-covered world news.
      USA Today: Low because their primary purpose is that of a news source you go to when you haven’t heard of the story at all before. Just the facts, written simply. Allows you to read the upper bubbles and know the background. Basic is not necessarily a bad thing here.
      CNN: The chart structure resulted in me being very harsh on CNN. They do have good journalists and good stories, but the low ranking was primarily due to sensationalism generated by their 24 hr TV news format itself. Online written articles are much better, but I think of CNN primarily as their TV channel. Reasons include high repetition of the same US political stories rather than exploration of world events, “breaking News” designations for mundane occurrences. These critiques could also apply to Fox and MSNBC, but I think Fox and MSNBC’s audiences know what they are in for.

      Hope that helps

  17. Thank you for this chart! From what I’ve seen on Facebook, “upper bubble” types like me seem to appreciate it. I appreciate a visual guide to help people balance their bias and choose their sources.

    1. Great! Glad it was useful.

  18. Hi,
    Your chart totally appealed to my left brain the moment it was shared with me! The succeeding thoughts however was the source. I wasn’t given the link to your website so it took a couple of days to find you. And like Mr. Greg Jenkins, even though the layout and placements are almost self-evident, I’m still curious as to your thinking and the charting process – so please let me know when you publish your blog post. With your permission, I would like to post an edited version on my FB page – with credit and link, of course. And if you would like to see my edited version, I would be happy to send it but I’ll need an address.

    1. I will let you know when I post it. I’d be happy to see your edited version. I’ll e-mail you.

  19. Thank you so much for doing this! I too have been wrestling with how we navigate in a world where we can’t even agree on what news sources can be relied upon and trusted. Regardless whether one agrees or disagrees with the placement of any particular news source, you’ve created an invaluable tool to frame the discussion. A political Rorschach test of sorts. Looking forward to your next blog post explaining the methodology behind your conclusions.

    1. Thanks! That new post is up now 🙂

  20. An interesting chart, Vanessa. I agree with most of it, but would argue with placing USA Today low on the quality scale. Just because the articles tend to be short does not equate in my mind with low quality. Further, I view it as quite valuable because they do a good deal of in-depth, original reporting on major issues of our times. One example that comes to mind was a comprehensive article on incarceration in America.

    I was a little disappointed that the New Yorker was not on your chart. I think it’s tops for quality but I recognize that its circulation is not on a par with the other sources.

  21. DISCLAIMER: I have not completely read your chart. A highly regarded friend of mine told me to provide my opinion of whether or not it is accurate. I only read the ‘journalistic quality’ axis and ‘partisan bias’ axis. Unable to understand the axes, I said I would not read the chart until I read the source.

    What is your response to critics who say that you are spreading a ‘chart’ that is solely based on subjective opinion, but labeled as if there are reproducible scores associated with the x and y-axis?

    If you were trying to make an accurate, personal opinion piece, you would have written an analytical essay citing examples that exist in reality (whether it cite a specific title or a 2 million word corpus from BBC news articles).

    Instead, you present a ‘chart’ that lacks any indication that the scientific method was applied. Thus, it is just as misleading as the articles and headlines presented in fake news outlets. AND. You tweet out the original and REVISED chart without reference to your subjective disclaimers in this blog post. It is poison.

    Instead of using your ‘insight’ to perform accurate research into why bias exists, you created another example of poor fundamental statistics for professors to mock in there Stats-101 undergraduate classes (and that’s being generous).

    Let me respond to the only information regarding your ‘chart’.

    “-I am operating out of the assumption that the less blatantly partisan the source is, the more accurate it is.”
    ::RESPONSE:: I don’t understand what your definition of ‘blatantly partisan’ or ‘accurate’ is, nor would any other objective party after reading your post/chart.

    “-I understand that individual reporters, even at the most reputable news sources, have their own personal biases and opinions. The rankings are an overall ranking of each site.”
    ::RESPONSE::You do not present a ranking in your post. Neither ‘individual reporters’ or ‘sites’ are ranked in your post (I would love to see the rankings if you forgot to post them).

    “-“Sensational” means the article have titles like “So and so DESTROYS so and so with THIS response!””
    ::RESPONSE:: What article titles are you referring to? What is your golden standard regarding ‘sensational’? What time period did you pull these articles from (last 10 months?, 5 years?, 50 years?)? I can utilize many datasets to reproduce your score for ‘sensational’ if you provide your measure.

    -“Clickbait” means the articles have titles like “She walked into a meeting. What happened next will shock you!”
    ::RESPONSE:: Again, only you know your measure. I am sure you are extremely busy but providing your model would be beneficial for continuing research in this area. The brightest Natural Language Processing minds are currently working to solve these types of problems and your measure of “clickbait” would push cutting edge research years ahead. I’m also confused how this fits into the x and y-axis I read from the chart. I simply don’t understand/am not smart enough to see multiple dimensions in a 2D graph that has no specific measure for the x or y dimension.

    “-“Conspiracy theories” means shit that is just made up. Like National Enquirer type stories.”
    ::RESPONSE:: You have created a dependence on your chart to justify your explanation of it. Apparently ‘National Enquirer’ is your definition of Conspiracy Theories (again I do not know if ‘NE’ is used in your chart because I only looked at the axis, but I am assuming it is since it is your baseline for an apparent cluster of your ‘chart’. Also, it is a 2-D chart so perhaps you applied a form of dimensionality reduction if you didn’t use a clustering approach? Another great insight that many readers would be interested in. I simply don’t understand how “partisan-bias” and “journalism quality” measures have not been defined but there are attempts to define other measures that are built off of them).

    -I’m sure this will offend some people that typically agree with me politically.
    ::RESPONSE:: This is not an explanation of the chart so disregarded.

    After reading the source (this blog post?) I see a fundamental issue with unprofessional, borderline ignorant, research being consumed by people looking for a dose of confirmation bias. Please start performing fundamentally sound research in your area of expertise, or create strong analytical opinion pieces that avoids quantitative representations of qualitative results.

    The irony: one of the goals of this ‘chart’ was to help distinguish between good information and bad information.

    1. I understand why you are upset with the existence of my highly subjective, opinion-based chart. For better or worse, there is almost no barrier for entry to posting opinions to the internet, and no criteria to meet for whether information gets widely shared. No qualifications are necessary for having people listen to you or for having them regard the content you create as an authority. This is upsetting to me as well, but that is the state of information on the internet.

      Consider that few people are as discerning about their sources of information as you. For some, even the most basic insight into how news media can be categorized might be helpful in comparison to having to reference point at all.

      I do share your interest in well-supported, well researched conclusions. You asked me to provide my model in the event that it would be beneficial for continuing research in this area. I have added a new blog post that discusses my reasoning further, and which provides suggestions for how such research could be conducted in the future.

    2. Sully-

      Would you consider providing your Twitter address?

    3. What on earth do you mean by the scientific method being applied here? That term is so misused, widely misunderstood and thrown about in attempts to prop up questionable positions.

      It’s a formidable task trying to get factual news as opposed to filtered facts. Not a bad chart. I don’t agree with it but I haven’t seen a better one.

  22. Thanks for this! looking forward to your explanation re how you developed it all. Is it possible to make something like this using facts vs opinions? I imagine it would be, but only after defining criteria and collecting data – tons of work! Anyway re your comment “Why is it that I can have such different views on the same subject or topic as someone else who lives in the same country?” I just came across a super helpful book intro that you may be interested in. It’s a great and it looks like it’s been around for awhile. http://www.press.uchicago.edu/Misc/Chicago/467716.html

    1. Thanks for the book recommendation! I will check it out. My new post is up about the reasoning and methodology behind it.

    2. Only just saw votero’s News Quality chart shared by a friend today on FB. I, too, am searching for credible sources of information. The chapter from the book you linked was incredibly eye-opening for me. George Lakoff has vividly explained the differing world views and miscommunications between conservatives and liberals. It gave me a point of reference from which I may be able to understand viewpoints that I previously just suspected. Thank you for the link, not sure I can ponder the book in its entirety but maybe.

  23. […] would probably start with this chart that Vaness Otero, an attorney in Denver, developed. It’s a helpful visual that could be a […]

  24. Thank you for this! A terrific starting point for thoughtful and (hopefully) civilized discussions. I look forward to reading your recent post on how you decided on the various placements.

    One question… I can’t seen to make out the source located between David Wolfe and Addicting Info. I can see the word “Report”, but not what precedes it… 🙁

    Thank you!

    1. Thanks! That’s “Bipartisan Report.”

  25. Whoa! This blog looks just like my old one! It’s on a totally different topic but it has pretty much the same layout and
    design. Superb choice of colors!

  26. […] 2)      With all the talk about false news, this chart may be helpful in discerning the more consistently accurate news sources. http://www.allgeneralizationsarefalse.com/?p=53 […]

  27. Hi! Under the creative commons license, would I be allowed to have this made poster size for my classroom? Alternatively, do you have any plans of producing this as a poster for purchase? Because I would love to be able to buy a high-quality version of it for my classroom. I teach government to high school seniors, and this is exactly the kind of information they need.

    1. Yes, feel free to print or otherwise reproduce. I do plan on making some posters available for sale, but anyone can make their own.

  28. […] at the following chart, produced in Nov. of 2016 (see here), which attempts to produce a guide for news […]

  29. Thank you so much for your work on this — both the chart and the explanation of your methodology are so helpful!

    I work in a school district that is wrestling with how to help students with these same issues. We used your chart as a starting place and tweaked it to include other sources our students regularly come across in research and on social media. Would love to a) credit you with the original format and b) share it with you and get feedback on our edits (if you’re willing!).

  30. I do not see Al Jazeera. Am i missing it?

    1. It’s on my most updated chart. See my most recent post. Right there in the middle.

  31. Thank you so much I showed this to one of my very right wing friends and he began to pick it apart. I told him that the point is to make you aware of the wide range of information. it is not important if you feel you must adjust the chart a few degrees to the right or left. but to become more aware of the extreme and distorted . To vet the information you receive is becoming more difficult this is great thanks

  32. A friend called my attention to the fact that Time & Newsweek aren’t there. I realize you must get a TON of these sorts of requests, but if you’ve got the time (and energy – you must be running on coffee & 5 Hour by now!) would you mind adding them in?
    Many thanks again for such a fantastic reference guide!

    1. I’d put Time just under The Guardian and Newsweek just under Vox. I do get lots of requests, but I’m glad it is generating so much discussion. There are many news sources that aren’t on here, but I am working to see what I can do about that 🙂

      1. I was also wondering about Time, Newsweek, USNews, and Bloomberg. I’ve started referring to this any time I don’t recognize a source before I read because I’ve decided I don’t want to give any clicks to those outside of the center while reading about controversial topics now. While I understand magazines are more likely to have an editorial perspective and that’s why we choose one over another, I would expect a newspaper to be central. But in the age of digital news, we don’t have that kind of distinction anymore.

  33. Where do you think Foreign Policy, New Republic, and/or The New Yorker fall?

  34. Hi Vanessa, thanks for this resource! I wanted to share a couple other resources that may be of interest to you:

    http://www.opensources.co/ – Created by communications professor Melissa Zimdars and librarians in light of the fake news discussions. She had started the original Google Doc that went viral post-election: https://docs.google.com/document/d/10eA5-mCZLSS4MQY5QGb5ewC3VAL6pLkT53V_81ZyitM/preview

    http://www.allsides.com/bias/bias-ratings – A database that “exposes bias and provides multiple angles on the same story so you can quickly get the full picture, not just one slant.”

Leave a Reply

Your email address will not be published. Required fields are marked *