Posted on

The Chart, Version 3.0: What, Exactly, Are We Reading?

 

TL;DR: What’s new in this chart is:

  • I edited the categories on the vertical axis to more accurately describe the contents of the news sources ranked therein (long discussion below).
  • I stuffed as many sources (from both version 1.0 and 2.0, plus some new ones) on here as I could, in response to all the “what about ______ source” questions I got. Now the logos are pretty tiny. If you have a request for a ranking of a particular source, let me know in the comments.
  • I changed the subheading under “Hyper-Partisan” from “questionable journalistic value” to “expressly promotes views.” This is because “hyper-partisan” does not always mean that the facts reported in the stories are necessarily “questionable.” Some analysis sources in these columns do good fact-finding in support of their expressly partisan stances. I didn’t want anyone to think those sources were necessarily “bad” just because they hyper-partisan (though they could be “bad” for other reasons.
  • I added a key that indicates what the circles and ellipses mean. They mean that a source within a particular circle or ellipse can often have stories that fall within that circle/ellipse’s range. This is, of course, not true for all sources
  • Green/Yellow/Orange/Red Key. Within each square: Green is news, yellow is fair interpretations of the news, orange is unfair interpretations of the news, and red is nonsense damaging to public discourse.

Just read this one more thing: It’s best to think of the position of a source as a weighted average position of the stories within each source. That is, I rank some sources in a particular spot because most of its stories fall in that spot. However, I weight the ranking downward is if it has a significant number of stories (even if they are a minority) that fall in the orange or red areas. For example, if Daily Kos has 75% of its stories fall under yellow (e.g., “analysis,” and “opinion, fair”), but 25% fall under orange (selective, unfair, hyper-partisan), it is rated overall in the orange. I rank them like this is because, in my view, the orange and red-type content is damaging to the overall media landscape, and if a significant enough number of stories fall in that category, readers should rely on it less. This is a subjective judgment on my part, but I think it is defensible.

OK, you can go now unless you just really love reading about this media analysis stuff. News nerds, proceed for more discussion about ranking the news.

As I discussed in my post entitled “The Chart, Second Edition: What Makes a News Source Good?” the most accurate and helpful way to analyze a news source is to analyze its individual stories, and the most accurate way to analyze an individual story is to analyze its individual sentences. I recently started a blog series where I rank individual stories on this chart and provide a written analysis that scores the article itself on a sentence-by-sentence basis, and separately scores the title, graphics, lede, and other visual elements. See a couple of examples here. Categorizing and ranking the news is hard to do because there are so very many factors. But I’m convinced that the most accurate way to analyze and categorize news is to look as closely at it as possible, and measure everything about it that is measurable. I think we can improve our media landscape by doing this and coming up with novel and accurate ways to rank and score the news, and then teaching others how to do the same. If you like how I analyze articles in my blog series, and have a request for a particular article, let me know in the comments. I’m interested in talking about individual articles, and what makes them good and bad, with you.

As I’ve been analyzing articles on an element-by element, sentence-by-sentence basis, it became apparent to me that individual elements and sentences can be ranked or categorized in several ways, and that my chart needed some revisions for accuracy.

So far I have settled on at least three different dimensions, or metric, upon which an individual sentence can be ranked. These are 1) the Veracity metric, 2) the Expression metric, and 3) the Fairness metric

The primary way statements are currently evaluated in the news are on the basis of truthfulness, which is arguably the most important ranking metric. Several existing fact-checking sites, such as Politifact and Washington Post Fact Checker, use a scale to rate the veracity of statements; Politifact has six levels and Washington Post Fact Checker has four, reflecting that many statements are not entirely either true or false. I score each sentence on a similar “Veracity” metric, as follows:

  • True and Complete
  • Mostly True/ True but Incomplete
  • Mixed True and False
  • Mostly False or Misleading
  • False

Since there are many reputable organizations that do this type of fact-checking work, according to well-established industry standards, (see, e.g., Poynter International Fact Checking Network), I do not replicate this work myself but rather rely on these sources for fact checking.

It is valid and important to rate articles and statements for truthfulness. But it is apparent  that sentences can vary in quality in other ways. One way, which I discussed in my previous post (The Chart, Second Edition: What makes a News Source ‘Good’) is on what I call an “Expression” scale of fact-to-opinion. The Expression scale I use goes like this:

  • (Presented as) Fact
  • (Presented as) Fact/Analysis (or persuasively-worded fact)
  • (Presented as) Analysis (well-supported by fact, reasonable)
  • (Presented as) Analysis/Opinion (somewhat supported by fact)
  • (Presented as) Opinion (unsupported by facts or by highly disputed facts)

In ranking stories and sentences, I believe it is important to distinguish between fact, analysis, and opinion, and to value fact-reporting as more essential to news than either analysis or opinion. Opinion isn’t necessarily bad, but it’s important to distinguish that it is not news, which is why I rank it lower on the chart than analysis or fact reporting.

Note that the ranking here includes whether something is “presented as” fact, analysis, etc. This Expression scale focuses on the syntax and intent of the sentence, but not necessarily the absolute veracity. For example, a sentence could be presented as a fact but may be completely false or completely true. It wouldn’t be accurate to characterize a false statement, presented as fact, as an “opinion.” A sentence presented as opinion is one that provides a strong conclusion, but can’t truly be verified or debunked, because it is a conclusion based on too many individual things. I’ll write more on this metric separately, but for now, I submit that it is an important one because it is a second dimension of ranking that can be applied consistently to any sentence. Also, I submit that a false or misleading statement that is presented as a fact is more damaging to a sentence’s credibility than a false or misleading statement presented as mere opinion.

The need for another metric became apparent when asking the question “what is this sentence for?” of each and every sentence. Sometimes, a sentence that is completely true and presented as fact can strike a reader as biased for some reason. There are several ways in which a sentence can be “biased,” even if true. For example, sentences that are not relevant to the current story, or not timely, or that provide a quote out of context, can strike a reader as unfair because they appear to be inserted merely for the purpose of persuasion. It is true that readers can be persuaded by any kind of fact or opinion, but it seems “fair” to use certain facts and opinions to persuade while unfair to use other kinds.

I submit that the following characteristics of sentences can make them seem unfair:

-Not relevant to present story

-Not timely

-Ad hominem (personal) attacks

-Name-calling

-Other character attacks

-Quotes inserted to prove the truth of what the speaker is saying

-Sentences including persuasive facts but which omit facts that would tend to prove the opposite point

-Emotionally-charged adjectives

-Any fact, analysis, or opinion statement that is based on false, misleading, or highly disputed premises

This is not an exhaustive list of what makes a sentence unfair, and I suspect that the more articles I analyze, the more accurate and comprehensive I can make this list over time. I welcome feedback on what other characteristics make a sentence unfair, and I’ll write more on this metric in the future. Admittedly, many of these factors have a subjective component. Some of the standards I used to make a call on whether a sentence was “fair” or unfair” are the same ones in the Federal Rules of Evidence (i.e., the ones that judges use to rule on objections in court). These rules define complex concepts such as relevance and permissible character evidence, and determine what is fair for a jury to consider in court. I have a sense that a similar set of comprehensive rules for legal evidence could be developed for journalism fairness. For now, these initial identifiers of fairness metric helped me distinguish the presence of unfair sentences in articles. I now use a “Fairness” metric in addition to the Veracity scale and the Expression scale. This metric only has two measures, and therefore requires a call to be made between:

  • Fair
  • Unfair

By identifying a percentage of sentences that were unfair, I was able to gain an additional perspective on what an overall article was doing, which helped me create some more accurate descriptions of types of articles on the vertical quality axis. In my previous chart (second edition), the fact-to-opinion metric was the primary basis for the vertical ranking descriptions, so it looked like this:

In using all three metrics, 1) the Veracity scale, 2), the fact-to-opinion Expression scale, and 3) the Fairness scale, I came up with what I believe are more accurate descriptions of article types, which looks like this:

As shown, the top three categories are the same, but the lower ranked categories are more specifically described than in the previous version. The new categories are “Opinion; Fair Persuasion,” “Selective or Incomplete Story; Unfair Persuasion,” “Propaganda/Contains Misleading Facts,” and “Contains Inaccurate/ Fabricated Info.” If you look at the news sources that fall into these categories, I think you’ll find that these descriptions more accurately describe many of the stories within the sources.

Thanks for reading about my media categorizing endeavors. I believe it is possible (though difficult) to categorize the news, and that doing so accurately is a worthy endeavor. In future posts and chart editions I’ll dive into other metrics I’ve been using and refining, such as those pertaining to partisanship, topic focus (e.g., story selection bias), and news source ownership.

If you would like a blank version for education purposes, here you go:

Third Edition Blank

And here is a lower-resolution version for download on mobile devices:

223 thoughts on “The Chart, Version 3.0: What, Exactly, Are We Reading?

  1. Where is Vice News?

      1. Hi Vanessa, you might want to take a closer look at Natural News. They are far from left/liberal. In fact many of their articles skew borderline alt-right.

        1. Yes. See other comments about Natural News. I think I have some good reasons for placing them there, but I will likely move them in the next edition.

      2. Hi Vanessa! I think you’re really on to something here. I’d have to contemplate several different sections more intently to have any real comment about your process. Having said that its surprising about half the rankings dont intuituvely make sense to me. So question- have you considered streamlining your teview process such that you could pull in a mixed group of about 25 people from both the right and left to provide their analysis and rerank based in that? Good display that as an alternative assessment not replace the current version.

        1. Hi Renee. Yes, I have considered getting more consensus. I think that would be valuable.

  2. Please add Truthdig to the list they have been around as long as the Huffington Post
    and you might look at TYT

    1. Will do.

  3. Thank you for updating, Vanessa!

    Based on your first chart, I started following only the sources in the gray circle and the green oval and dropped everything else. One of the sources I started reading was The Hill, and I would argue that it skews more liberal than is demonstrated in your charts.

    Having said that, I think your analysis is fantastic and I share it with my college students to help them evaluate the quality of media sources.

    1. Thanks for the feedback. I agree that it seems more liberal lately, but I think it is because of their tendency to report on petty, palace-intrigue type of stories, of which there are a lot during this administration. For example, they have a lot of “stories” about what Trump or Trump Jr. just tweeted, which seems liberal because those stories tend to be embarrassing. Glad you use it to teach media literacy!

  4. Where is Fox News? I don’t see it on the chart except for the Daily Wire.

    1. It is right above Daily Wire–those are two separate logos.

  5. Wall Street Journal = minimal partisan bias.

    No, seriously? It’s just more articulate version of Fox. It’s why stuff like this is so dangerously wrong. You’ve defined the center-right/rightist Establishment as “the center” and then located everything relative to the right-wing Establishment.

    1. I think you could reasonably say that about some recent opinion articles, especially over the last year (and one in particular about two weeks ago) but the majority of their content is not as hyper-partisan as that. It’s a lot of factual financial reporting. The overall placement is a weighted average. I’ll rank individual articles in the future; some would be down towards Fox News but most I think would be right where I’ve placed WSJ. I typically hear feedback that sources at the center of this chart skew more liberal, rather than too conservative, but I hear both kinds of feedback. “Dangerously wrong” is, I think, a bit extreme of a way to describe.

      1. If both sides complain equally, then you know you are correct. 😉

        1. It’s what I strive for, I guess 😉

    2. I think you need to separate the WSJ reporting from the WSJ editorials/opinion pieces.

      1. I agree. The most accurate way to rank would be on an article-by-article basis. I’ll be putting out versions of charts with more specific detail like that.

      2. I agree. And though this just may betray bias, business interests need reliable information to make good choices about where to get the most bang for the buck (ie, where the largest profits can be extracted from production, which implies that labor, environmentalists and social movements must be rendered as powerless as possible, whether by death squads or the skillful adoption and application of laws passed by corporate-captured governments and well-bribed politicians). The editorial pages supply the idealogical justifications for the vast systems of exploitation, repression that results.

        It’s a bit like the hard sciences, which require the unrestricted use of reason to draw useful (and therefore valuable) conclusions based on experimentation, observation, repeatability, and peer review. The sciences reveal unequivocally that human activities are destroying ecosystems at a breakneck pace, which is why they are under constant assault. And while that assault is underway, of course, corporate-sector scientists and engineers are working feverishly to discover the most efficient means of capitalizing on previously un-exploitable arctic resources. Making a killing of the extinction of the planet requires both the editorial page and reliable hard news.

    3. The WSJ’s investigative reporting is still top notch. While the editorial content has always leaned conservative, it still had some integrity in my opinion. But they lost a lot of their better opinion writers like Bret Stephens and others once the Republican party became the Trump party. Over the last year their editorials have moved much further to the world of Trump and the far right. It’s sad. I’ve subscribed to the WSJ for over 30 years and even when I didn’t agree, most of the time I could respect their logic and view. But after William McGurn recently said that Rep. Devin Nunes is the only one with any integrity with all the committee investigation, well…sad.

  6. I didn’t read this well but it looks similar to ‘content analyses’ and some older approaches (which did something like paragraph analyses of social science papers to see if the words used in a paragraph were related in a consistant manner , or instead just appeared to be some random collection strung together to appear to make an argument. An automatic computer program ‘postmodern generator’ can write academic papers automatically by combining phrases and words, and arernt too different from ‘real’ ones, and similar programs exist for computer sciences and art.) Lots of studies do content analyses of internet web sites and links —look for memes and how they diffuse.

    I was wondering if one can or does weight news sites not just by content as you seem to do (ie reliability, bias…) but also ‘importance ‘ or readership size. eg a NY Times story measured by some ‘truth’ metric would be weighted more than one in a small newspaper.
    One could have a sort of ‘truth temperature scale’ or map for US media world. Like areas devoid of truth , or areas which are on the ‘west’ side of the truth (eg left coast) versus right—a sort of arrow or direction.
    One would get an ‘average truth temperature’ of USA (though it might have to have 2 orientations–since i could see some articles might both be or same truth value or ‘facturality’; but diagree (eg ‘honest articles ‘ that imply that say Trump was involved with Russia during 2016 election, versus ones that say Hilary was (or about whether Iraq had WMD, or whether war in afghanistand is ‘winning’ or ‘losing’ , or which predictions about elections are more truthful.)

    1. Thanks for the feedback. I’m looking at several of the other ways of measuring that you mention.

      1. Might be as simple as creating a 2nd version of the graph that sizes the area of the logos by their reader-/listener-/viewer-ship. One of the fallacious claims the garbage sites use very often is false equivalency: sure we’re propagandizing but look, the other side has a source just like this!

        If there’s a glut of garbage-followers on one side and a microscopic audience for the equivalently partisan crap, it would be helpful* to have some validation that extreme partisanship is not balanced. If on the other hand there’s a relatively even distribution of audiences toward extremism and fake news, that would be even more helpful because it’s easier to find common ground when both sides are relatively equally at fault.

        * [toward a supposed goal of national reunification]

        1. I believe you are correct. I am working on a number of different versions to convey such ideas, and this is at the top of the list. Thanks!

  7. Much better than the last version but a few fixes still needed. To say “David Wolfe” and “Natural News” are in any way “left” is insulting. Its pure quackery thats neither left or right but rather belongs underneath Enquirer. Daily Kos and Alternet belong where Huffpo is and Occupy Democrats and U.S. Uncut belong where they are, to compare to the bias/quality of the sources across from them on the right. Also, to be fair, putting AlJazeera in right in the middle is equally rediculous, who are we kidding, its editorial slant is clearly quite left, it belongs between WaPo and Democracy now!.

    1. Fair points on all.

      1. I agree with his! Great points. And great chart!

  8. Very interesting. I am always in question with each news source when hearing about a new story. People tend to say one source is pushing one view or the other. It is very positive to see some of the main stream outlets (abc, NBC, cbs, wallstreet, npr) in the fact/mainstream section. Thanks for taking the time to provide this type of information. I don’t have the time to do things like this!

  9. Where is Salon?

    1. I’ll be following up on individual requests like these shortly–I have a lot of them though, so thanks for your patience.

  10. Awesome work! My aunt shared this on fb.
    I get some news from independent.co.uk and would be interested in where that source falls within the metrics. Thank you.

    1. I’ll be following up on individual requests like these shortly–I have a lot of them though, so thanks for your patience.

  11. I would urge you to look at TalkingPointsMemo. I _think_ they’ve got enough of a readership to be considered in a broad study like this. They have many active investigative journalists and have been active for over 15 years. I consider them left leaning but reliable. I’d be interested in seeing your take.

    Thanks for this analysis. It’s really good to have a summarized graphic for reference when someone brings up a “news source”.

    1. Thanks! I’ll be following up on individual requests like these shortly–I have a lot of them though, so thanks for your patience.

    2. That’s the first one I looked for too.

  12. Great job Vanessa. I think you greatly improved the chart’s accuracy.

  13. Where are the Young Turks?

    1. I’ll be following up on individual requests like these shortly–I have a lot of them though, so thanks for your patience.

  14. this is so helpful and interesting. Did I miss GQ and New York magazine in the graphic?

    1. I’ll be following up on individual requests like these shortly–I have a lot of them though, so thanks for your patience.

  15. Have you considered where specific shows might land on the grid? For example, I see where MSNBC is placed, but I’m curious where you might place Rachel Maddow in relation to that… or how her show contributes to the placement you ultimately decided on.

    1. Yes. Thanks for asking. One can (and should) rank individual articles, shows, and stories on this chart. It is more accurate the more you drill down on a particular story. I have done so with some individual articles in previous posts. The overall ranking of a source is a sort of weighted average. For example, I would rank Rachel Maddow’s show as very high “Complex Analysis” and between the skews liberal/hyper-partisan liberal categories. Lawrence O’Donnell’s show, however, I would rank quite a bit lower, on the low end of of “Opinion” and farther to the left. Though throughout the day, the MSNBC reporting has more ordinary news, including straight fact reporting, but viewers mostly rely on it for the left-leaning analysis and opinion it provides during its evening shows, which is why my overall ranking has it squarely in the opinion section. I think the more opinionated shows drag down the overall quality of the network because the opinion shows shades people’s overall impression of the network so much. That’s why I have it lower than Rachel Maddow’s show, or the other daytime news broadcasts, would otherwise be.

  16. TYT news?

    1. I’ll be following up on individual requests like these shortly–I have a lot of them though, so thanks for your patience.

  17. Thank you for your work. Can you tell us what logo is under CNN?

    1. There’s nothing directly under CNN (e.g., behind it–that’s just how the logo came out), but farther down, if that’s what you’re talking about, is Real Farmacy.

  18. I don’t see the Wall Street Journal, which I would say is middle Conservative.

    1. It’s on there. I agree.

  19. Where would you rate ProPublica?

    1. I’ll be following up on individual requests like these shortly–I have a lot of them though, so thanks for your patience.

  20. Is there a reason you did not list the Sinclair Broadcast Group?

    1. Yes, because it is an ownership group and not a particular outlet. I rank the outlets rather than ownership group. I am aware of their political slant and forays into local news.

  21. Great stuff! Congrats and thanks! With just a little tweaking, this could also be a guide to just having a quality discussion intended to understand and explore, rather than dismiss and overcome.

  22. Thanks for providing this well thought out and well presented information. Did I just over look “Salon” or was it not rated?

    1. It’s not on there, but I will follow up on individual requests!

  23. Clearly your opinion is biased. CNN reputable? And your other posts are clearly trying to manipulate Trump supporters. Trump is the best thing this country has seen all century. If you cannot understand why the country is thrilled Hillary lost it is futile to explain. Good luck and stop crying.

    1. I agree that my opinion is biased.

  24. Vice news?

  25. Thanks for all the good work you’ve put into this ongoing project. It’s a valuable service for those who are interested enough to benefit from it, to whatever depth they might use your chart and presentation.

    One thought regarding the amount of detail offered is that you could provide one large chart with all of the news sources you’ve considered, and then maybe two or three less detailed editions in which you’ve eliminated some or many of the sources to simplify the view for those who are less interested in the more minor sources. Or, perhaps you could provide one edition of the chart that includes only the more major news sources, and another edition (or several?) showing only the more minor or lesser known news sources.

    There is an aspect of news reporting that I think significantly impacts the quality of what is available to us. Much of what passes for “news” these days is actually more on the order of infotainment or entertainment. Over the decades, We the People have set ourselves up for getting more infotainment and less good news coverage, investigative journalism and quality analysis. We’ve done this by rewarding the media — with our time, attention and money — to provide us with more entertainment and less to provide real news.

    News is too often delivered in a fast paced, clipped manner that runs everything together in a manner such that everything seems to have the same value – which is very little – and there is not time to consider deeply or evaluate adequately. There is also a layer of cuteness, cleverness and snarkiness that can tend to obscure the actual news content, and it’s all part of the entertainment aspect of delivering the news. It is also part of the manipulative aspect of news delivery that is provided with an agenda.

    There’s a lot of complexity in the delivery of the news, and more so with the delivery of analysis and opinion. Besides the actual factual news content or statements of analysis or opinion, this includes the overall structure of the article: what is presented first; how does it flow; what vocabulary is chosen to present it; whether it is chronological or mixed; whether it provides any background context or not; whether it consists of reasonable paragraphs or nearly every sentence is a paragraph; etc. Then there are the reports that use LOTS OF UPPER CASE or “pull quotes” to emphasize some aspects while diminishing or ignoring others, often with an intent to add emotional impact to push an agenda rather than to simply provide real content. Much of what is published using some of the more dramatic and emotional techniques is provided with an agenda as well as for infotainment purposes, with the entertaining aspect of it making it easier to feel comfortable buying into the agenda.

    My intent (agenda) behind saying all of this is to suggest that your evaluation of news sources should also include an “entertainment factor” or perhaps an “infotainment factor” in which a higher score would mean little if any entertainment value and a lower score would mean more entertainment value. This score would then add to or detract from the source as being a valuable news source.

    Thanks, again. I appreciate the attention and effort you have dedicated to this work. It’s definitely a challenging task you’ve undertaken.

    1. I appreciate your comment about an infotainment scale. I recommend you look at mine about corporatocratic bias in the media. I expect there would be a great overlap in the two categories.

  26. I cannot savec the image or read it on my mobile device. Please makr the image donwloadable.

    1. Ok, there is now a lower-resolution image at the end of the post. See if that helps.

      1. Thank you for your efforts. Not everyone will agree, but it certainly gives them a metric to work with. I wish more people considered the source.

  27. God love you, Vanessa for your efforts!
    While I am a professional librarian, I also serve as a university instructor.
    Might we need a new ‘blank chart’ that includes the Green/Yellow/Orange/Red rectangles?

    I teach a 3.0 credit course in the business school; information literacy & research.
    Evaluating resources is a critical skill addressed in competency standards in higher education. Standard 3: The information literate student evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system.
    http://www.ala.org/Template.cfm?Section=Home&template=/ContentManagement/ContentDisplay.cfm&ContentID=33553

    I use your chart as an in-class activity. I pass out paper prints of news outlet logos to students and instruct them to research their assigned outlet. I project the ‘blank chart’ on a whiteboard. The students decide based on their research where their assigned news outlet logo should be positioned on the chart/board. They use a magnet to hold the logo in the proper position considering both the vertical and horizontal axis.
    We then compare our results with your completed chart.
    Do we need a new ‘blank chart’ that includes the Green/Yellow/Orange/Red rectangles?

    1. Hi Ann! Glad to hear about how you use it. Many other college professors and high school teachers are doing the same. There’s a link to a blank one with the rectangles toward the end of this post.

  28. The only thing wrong is that infowars should have broken off the chart to the bottom and be starting its own chart that calls all other charts fake news. I mean, Louise Mensch is a weird conspiracy theorist, but not “JFK and Hitler are both alive on the secret UN moon base with the lizard people leaders of the Illuminati” level weird conspiracy.

    1. Haha, fair point. Just think of the distances on the chart between sources as not absolutely relative or to scale.

  29. Forbes? I have the impression that it’s generally a bit to the right, but some articles are definitely outliers in that assessment.

    Also interested in TYT, Jimmy Dore, and Independent.co.uk as others have requested.

    1. I agree about Forbes…close to WSJ or Fiscal Times. Will follow up on those.

  30. Do you have a spreadsheet with the actual values given to each outlet or could you zoom in on the grey circle? It’s really messy and some items I don’t know if that is actually where they are or if its just best fit for the image. An interactive chart where you have the dots for locations and mouse over to see the outlet would be nice.

    1. I’m working on a version like that. Thanks!

  31. This is a brilliant chart Vanessa, thanks for the update.

    Can you elaborate on why you chose to place MSNBC higher than Fox News? Maybe because I skew somewhat conservative it seems odd, but every time I turn on MSNBC it’s Rachel Maddow or that guy with the stupid glasses doing some really biased hit job. I’m not fan of Fox News but they seem like both sides of the same coin.

    1. Without going into too much detail, it has to do with the percentage of statements on opinion shows (Hannity, Tucker Carlson, Watters’ World) and by contributors to those shows (Greg Jarrett) that are actually misleading or false. Yes, both networks have lots of opinion and unfair statements, but misleading statements really impact the quality of a source, and Fox News has more. There are some existing studies that get into this, but not exactly in the way I analyze stories. I will be putting out some example comparisons in the future, such as between Rachel Maddow and Sean Hannity which will look like this but longer: http://www.allgeneralizationsarefalse.com/not-fake-news-but-still-awful-for-other-reasons-analysis-of-two-examples-from-the-echo-chambers-this-week/. But those take a long time, and the analyses I’ve done aren’t formally written up yet. I don’t like to ask people to just take my word for it, so instead I’m asking you to be patient, and I promise I will come out with these write-ups.

  32. The up-down axis seems solid, but I’m not sure about the left-right one. Why is Jacobin to the right of Occupy Democrats?

    1. It’s not 100% based on percentages of articles, but it has to do with a larger percentage of content on Occupy Democrats being conclusory and short. Jacobin’s articles are much longer and allow for argument and nuance. I know a lot of people have a problem with Jacobin being anywhere but “as left as possible” because of their self-described “socialist” label. In my view, their focus on what they call democratic socialism appears very focused on economics, whereas American liberalism encompasses more than that. Occupy Dems is relentlessly liberal on all the issues. I’m open to moving Jacobin based on further evaluation and feedback. Thanks for the comment.

  33. Wow! This chart should be in the national news!

  34. Vanessa, it’s good and useful work and I’m impressed. Being an amateur linguist, I can see some areas of congruence between your work and my fiddling around. You may have to break the chart down into segments, just to be as inclusive and detailed as possible. I wonder if or where you would consider putting satire and political humor articles or do they need to be on a different chart ? I mean, MAD magazine has been a major influence on adolescent thought for years.

    1. Thanks. I’ve avoided putting satire on here because I do think it needs its own chart. Any axes ranking comedy would have to account for the format and language of the comedy, and that would be a whole other level of linguistic analysis. Further, I’ve had a hard time finding any good or popular conservative satire. If anyone can point me towards some, I’d love to see.

  35. I love this! Great work! When you have time I’m curious where you’d put the Daily Skimm.

    1. Will add it to my list!

  36. Please encourage C-Span to use this chart whenever they have a guest on.

  37. great map! really interesting! did you consider ranking Business Insider, too?

    1. On my list!

  38. First, thank you so much for your work! I’m sure this must have taken a great deal of time! I saw it on one of my Facebook friends walls and shared it onto my page. In these times it is now more important than ever to know which news agencies we can count on to give us the truth so that we can make informed decisions. Thank you! What a wonderful public service you have done!❤️

  39. Legalize marijuana. Legalize or regulate recreational drugs.
    Legalize all adult consensual sexual activity. End government restrictions on abortion.
    End police violence. End justice systems that prey on the poor, uneducated, and minorities.
    Protect the 1st and 4th amendments. Restore rights to criminal defendants.
    End wars. Reduce military spending–drastically.
    Borders are arbitrary. Everyone should have the right to travel (immigrate) wherever they want.
    Separate church and state. Give LGBT the same marriage rights as heterosexuals.

    Q: Guess which organization this is.
    Huffington Post?
    Democracy Now?

    A: Reason

    1. Reason is a pretty unapologetically libertarian publication. I know libertarians like to think of themselves as centrists because of the whole “socially liberal/fiscally conservative” policy positions, but there’s pretty broad consensus among Democrats and Republicans that Libertarians’ governing policies align with Republicans more than Democrats. This consensus, I believe, has arisen because any Libertarian-friendly politicians in elected positions (there aren’t many, but, for example, Ron and Rand Paul) typically identify s or caucus with Republicans. Also, Libertarians in government tend to not be super interested in implementing new laws (less government!) that would actually protect or advance progressive policies. I believe this is why liberals/progressives/Democrats tend to not vote for Libertarians, while some Republicans do. But notice that I don’t have Reason ranked as the most conservative publication. And if I ranked individual articles on there, there would certainly be some that would rank farther to the left because of the particular political issue being discussed.

    2. Well said, Jeremy. Thank you.

  40. Have you thought about making this a more interactive version? Something database-driven that can update automatically as you enter additional articles etc; that can be filtered by the viewer for specific networks/sources, and perhaps by timespan? I think this has potential to be a powerful tool if it can continue as such.

    1. Yes–I’m already working on it! Stay tuned. But please keep the suggestions coming. I’m sure there are a lot of useful ways to visualize the information people want that I haven’t thought of.

  41. The owner of Natural News is a libertarian.

    1. Partisan ratings here are more a function of the content of the article and who tends to pick them up and share them–not necessarily the ownership of the source. I think the owners of most ridiculous sites, unfortunately, know what they are doing and take advantage of gullible people.

      1. I dunno, the natural health crazy that Natural News caters to seems to know no side. My pretty conservative mother would send me things from them and Mercola. I’ve also seen a ton of anti-Obama/anti-Democrat rants within the articles on NN.

        1. I hear that. Natural News is one that many observers here have disputed, and I agree that they do have a mix of anti-liberal and anti-conservative stuff on there. It was my impression that they had more stories that were latched onto by left-learning conspiracy theorists, but in future versions I will likely move them more toward the center. I think it will be necessary to label the “center” as encompassing not just “centrist” or “mainstream” political views, but also as encompassing sources that have extremely partisan articles, but on both sides, with their averaging resulting in a placement them somewhere closer to the middle.

      2. I would say that Natural News has some articles that might be considered liberal, though in the last election they supported Trump. It would be interesting to see a discussion of terms “liberal” and “conservative.” I think Natural News crosses those lines, though I agree that either way it’s a terrible source for news.

        1. (Repeating an earlier reply RE: Natural News) I hear that. Natural News is one that many observers here have disputed, and I agree that they do have a mix of anti-liberal and anti-conservative stuff on there. It was my impression that they had more stories that were latched onto by left-learning conspiracy theorists, but in future versions I will likely move them more toward the center. I think it will be necessary to label the “center” as encompassing not just “centrist” or “mainstream” political views, but also as encompassing sources that have extremely partisan articles, but on both sides, with their averaging resulting in a placement them somewhere closer to the middle.

  42. Fabulous work, Vanessa. Much appreciated. Keep the updates coming!

    1. Thanks!

  43. I can comprehend why you placed MSNBC there, but not why NBC News isn’t right next to it, because they are the same operation. CNN’s place that low is questionable. And, it seems like the legacy of the Big 3 evening newscasts is influence their respective organizations place on the chart even though CBS News has a decidedly left leaning slant compared to ABC News or PBS Newshour for example. The categories and distinctions are good though.

    1. NBC is such a large organization that I think its subsidiaries (and/or affiliates) are worth distinguishing. By “NBC,” I mean their nightly news, website, and reporting services. I think most people think of MSNBC and CNBC as distinct enough entities with different political slants. As for CNN–everyone has a different opinion about them, and I’ll probably dedicate a whole future chart to them.

      1. On the surface you might think that, but NBC Nightly News utilizes the exact same staff as MSNBC. MSNBC doesn’t have a separate web presence it’s all the one NBC News website. CNBC does slant pro-business, and maybe more conservative politically, I haven’t watched it in years. I just feel like you’re trying to grant legitimacy to NBC by putting them at the top and in the middle but then maybe the current environment has influenced placing MSNBC down and to the left. If MSNBC is down and to the left NBC News would certainly be down and to the left, because as I said previously they are the same organization, there’s no separation between NBC News staff and MSNBC staff.

  44. Thanks so much for your work on this–really good stuff.

    I’m curious about how quantitative the metrics here are. Would you say this is closer to ordinal-scale information here (e.g., we know that CNN is to the left of The Economist, but can’t really tell if CNN is really *twice* as liberal-leaning as USA Today) or more like actual interval-level information? The neat symmetry is what makes me wonder–that seems too convenient to be real.

    Thanks!

    1. Great question. This chart is definitely not to scale. And though I am aiming to thoroughly quantify as many metrics as possible to back these rankings, I am still putting together the massive amounts of data that would be required to put these on a reasonably accurate scale. You can tell that it is not to scale when looking at how close some sources are–most observers would agree that InfoWars is many times worse than many of the sources near it.

      And I agree–I don’t think the symmetry reflects the actual existing media landscape. I mostly wanted to represent sources in each category as examples.

  45. Have you considered doing some kind of rating scale for each news source? Since it would be virtually impossible to include every news source in this chart, maybe you could develop a rating equivalence that represents where they fall on the chart. For example InfoWars could be R4H, where R means it is Right of center, 4 because it is in the fourth column and H because it is in row H.

    1. Yes. Thanks for the idea. I’ll incorporate that into future functionality.

  46. It seems weird to think of Reason as leaning conservative. But I suppose it publishes more stories that biased conservatives would like than those that biased liberals would like. Maybe it’s an average? I suppose that’s what comes of a one dimensional political scale.

    But anyway, thanks for the great work!

    1. Thanks. Here’s a response I wrote previously to another question about Reason:

      Reason is a pretty unapologetically libertarian publication. I know libertarians like to think of themselves as centrists because of the whole “socially liberal/fiscally conservative” policy positions, but there’s pretty broad consensus among Democrats and Republicans that Libertarians’ governing policies align with Republicans more than Democrats. This consensus, I believe, has arisen because any Libertarian-friendly politicians in elected positions (there aren’t many, but, for example, Ron and Rand Paul) typically identify s or caucus with Republicans. Also, Libertarians in government tend to not be super interested in implementing new laws (less government!) that would actually protect or advance progressive policies. I believe this is why liberals/progressives/Democrats tend to not vote for Libertarians, while some Republicans do. But notice that I don’t have Reason ranked as the most conservative publication. And if I ranked individual articles on there, there would certainly be some that would rank farther to the left because of the particular political issue being discussed.

  47. I couldn’t help but laugh at your chart. First of all, to put CNN in any kind of reputable catagory shows your own partisan bias. Some of them were caught on camera admitting to reporting “fake news” for ratings. Also, they have a heavy partisan bias to the left.

    Then you rank Fox News in a box that is harmful to the public? You cannot be serious! They may have a couple opinion shows, but they always give bith sides a chance to have their say. Why is that harmful? I think it’s quite the opposite especially when even their opinion shows end up reporting better news than other “news” stations.

  48. I really think you have enough specifics on the list that you ought to get rid of the generic “local news in a liberal city” and “local news in a conservative city.” I have no idea whether you would consider Denver “liberal” or “conservative,” and the Denver Post serves a broad area not easily characterized. And I would bet you haven’t sampled the paper’s coverage sufficiently — nothing like what you have done with other named sources.

  49. Vanessa– great chart! I’m elated that this exists in the universe. Would you consider replacing the word “liberal” with “progressive” in the bottom of the chart in order to accurately reflect the opposite of “conservative” which you have on the right side of the chart? Other alternatives seems even less fair (liberal — illiberal, liberal — conservative, un-conservative — conservative, etc.) Also, would you consider a version of the chart sans logos and with text only because a lot of noise is introduced by the logos that needn’t obscure the overall content? I will pledge $100 toward any graphic designer who can, on your behalf, present you some alternative visualizations of the same content as well. (Because I love what you’re doing, and I think the chart can be even more persuasive and clear.) Maybe I’ll just save my $100 and take a shot at it myself if you are interested.

    Other thoughts… a school of journalism or philosophy or juris prudence or information theory should absolutely take this in 10 different directions based on those disciplines. I love that you’ve included the Federal Rules of Evidence, and believe there are a lot of other heuristics that could be included/considered such as Wikipedia’s razor for consensus, fairness, completeness, etc.

    Finally, lots of folks unwittingly tailor their news bubbles by their click behavior. So there are a category of news outlets that feed people junk food diets of their own design (Google News?). So potentially we need an X axis on the chart that determines whether one’s news is tailored to him/herself or a community of people who look at news as a cultural/community driven civic duty. In the absence of that perhaps we need a tailored list of FB, and Twitter news outlets to follow.

    Cheers!

    1. Hi Max!

      I appreciate your comment. I’m not convinced that most Americans consider the term “progressive” the opposite of “conservative,” so I would hesitate to make that change. I will definitely put out text-only versions per several requests. I’m working on a number of useful ways to present this information visually (and dynamically/interactively!) It would help if people threw some $$ my way, because I am enlisting professional and technical help for these projects. I’ll have a (hopefully unoffensive) donation field up soon for those so inclined.

      You are absolutely right that many heuristics can be included. I’m trying to come up with the best, most complete, verifiable, and repeatable metrics. It’s a work in progress that will likely be refined over many iterations.

      On your last point–very true. A separate chart of anything that aggregates news for a user could be useful.

      Cheers!

  50. Great project. Thank you!!

    Would love to see a future edition for podcasts. I’m getting more and more of my news analysis through podcasts.

    1. I’ll add it to my list–it may be a while before I get to those. It takes longer to listen to a lot of podcasts!

  51. thank you for this. If you had a page on Facebook, I would love to see your work on my feed, and I would share it.

    1. I do. My page is viewable by followers!

  52. Apparently, NBC and CNN are not as reliable as your chart indicates. See my article https://www.huffingtonpost.com/entry/59209876e4b0e8f558bb2716

    1. Thanks for sharing your article. The overall ranking of reliability is intended to be an average; certainly even the best outlets have instances of where they have been mistaken, fooled, or were just outright wrong. People tend to remember those quite well be unforgiving. But I submit that their overall journalistic practices and ethics are usually high. (NBC more so than CNN).

  53. This chart is well reasoned and carefully thought out. I’m really glad to see it updated with more media outlets. It would be interesting to see a scatter plot of stories from a single media outlet over a 1 year period using your grading scale. Looking forward to 4.0!

    1. I’m already on it! Stay tuned!

  54. Your article lists several “characteristics of sentences can make them seem unfair,” including “Quotes inserted to prove the truth of what the speaker is saying.” While I think I understand the others, I’m not sure about this one. Are you talking about sources quoting *themselves* to demonstrate their own reliability, including a third party quoting a source to establish the same source’s reliability? Or are you talking about something else?

    The list of characteristics is fascinating to me, as a professional writer of text that straddles the boundary between fact and opinion, and that is expected to be “biased,” but within particular limits.

    1. So, I will definitely write a whole blog post on this topic in the future, because there is a lot to measure in the use of quotes.

      But what I mean here is related to the concept of hearsay in court. Generally, hearsay is “an out of court statement, made in court, to prove the truth of the matter asserted.” Hearsay is generally not admissible as evidence to a jury, subject to several exceptions. Of course an article is not court, but I think the principles that inform the hearsay rules are applicable in terms of what’s fair and what’s not. Often, what determines whether a statement is admissible is the concept of what the purpose is of introducing the statement. You generally can’t introduce a statement to prove “the truth of the matter asserted.” For example, if a witness is testifying that a man ran up to her and said “I’m about to pass out,” that statement is not admissible FOR THE PURPOSE of proving that the man was drunk or about to pass out, or whatever. However, it can be admissible for other reasons. One example reason would be if the statement was introduced to show the effect on the listener. So the witness could say what the man said if it gave her a reason to do take a subsequent action, like calling for medical help. There are a lot of other exceptions why hearsay would be admissible, but it’s generally viewed as unfair evidence because the jury can’t judge for themselves the tone, the speaker, and the credibility hearing it secondhand.

      What I mean by “quotes inserted to prove the truth of what the speaker is saying” (which I find unfair) is best explained through examples. Quotes in a headline are usually the most unfair examples of this. A local, conservative Alabama newspaper just ran a headline about Roy Moore’s sexual misconduct allegations along the lines of “Moore calls claims “evil” and “complete garbage.” That’s quite a spin just right there. A more neutral headline would be something like “Report alleges four counts of sexual misconduct by Roy Moore.” By inserting Moore’s quotes about the claims instead, it gives weight to Moore’s claim. In other words, the quotes “evil” and “garbage” are there to prove (or enhance the credibility of) the truth of what the speaker is saying. Fox News does this all the time, especially on twitter. They will use the quote of an opinion guest as the headline of the “news story,” which frames it as “news.” Here’s one from tonight “Rep. Matt Gaetz: ” The American people are tired of this double standard where there’s one set of rules for the Clintons, and another set of rules for everyone else.” I find that to be unfair use of quptes, and it happens all the time.

  55. I love this, and appreciate the hard work you have put into it. It serves as a wake-up call and reminds people that not everything they see reposted breathlessly is actually true. Where you get your news matters. There’s no way any of us can read all of it, so thinking people should ensure that they’re consuming not just pap that makes them feel justified and reinforced, but is actually factual and based in some objective reality. Thanks for making something so compact and understandable. I’m a fan and have been since the first edition of your chart came out!

  56. Hey, I know that you’ve posted that you’ve received a lot of requests lately, but I was curious about a variety of YouTube sources in particular :

    The Young Turks
    TYT Politics
    TYT Nation
    The Jimmy Dore Show
    The Humanist Report
    The Real News
    The David Pakman Show
    The Rational National
    Thom Hartman
    Sam Seder
    Secular Talk

    1. Thanks for the requests. I’ll put them on my list. The YouTube stuff will probably take a little longer for me to get to than the written stuff, but I promise I will follow up!

      1. This is actually quite fascinating to bring up the YouTubers.

        Around August 2016 after the Democratic and Republican Campaigns I was distraught and lost. The WSJ, NYT, NPR and other news sources I trusted were reporting things that made no sense compared to what I was seeing in my own life.

        So I started getting news from certain online sources while using the Facebook feature to block entities like Fox, WSJ, NPR, Vox, NYT, MSNBC, CBS.

        The sources that did make sense to me:
        Tim Pool, Rubin Report, Sam Harris, Scott Adams.
        And then I came across Mike Cernovich. I wouldn’t count him as reputable but his influence is massive. I felt like ignoring him would be at my own peril.

        But why did I start paying attention to Tim Pool and the others?
        They didn’t give me finger-wagging, hand-wringing shouting matches. They struck me as genuinely curious and operated outside their echo chambers. There’s room in their narratives for the Trump supporter who just couldn’t vote for Clinton. Whereas, the corporate media painted polarized worlds of Us vs. Them; The Wise vs. The Wretched.

        Reality is highly nuanced and I needed nuanced reporting, not punditry.

        Pool, Adams, et. al. do have their own biases and openly share them. I’m all good with that. But they also gave me the nuance that I wasn’t getting from the other sources.

    2. they are all fake news far left leaning

      1. Care to elaborate with specifics?

  57. Fascinating stuff, I have often counseled my friends and associates to avoid most news from the partisan extremes as its’ often utter self-serving balderdash. What would be interesting would be to look into what degree Corporate Influence skews an outlet’s editorial stance. But that would require adding a third axis to the matrix. I find the placement of the outlets be accurate, and better than the previous version. One of the things I have to keep in mind, and beware of, that one’s perception of the partisanship and veracity of an outlet is going to be perceived relative to where one places oneself on the political spectrum.

  58. Thank you for your hard work. I love it. A minor design suggestion, next time you publish it, consider making the dashed lines a lighter grey or translucent. There is a lot of detail in your chart and since you added the colored boxes I think the heavy emphasis on the dashed lines isn’t as important as it otherwise would be. Keep up the good work!

  59. I host a website at https://www.teamwolcottcycling.com/act/act where I have a Recent News listing. Your chart is very useful. I would like to post the jpg of your chart so that readers can learn to be more discerning about news sources. May I have permission to post it with the appropriate credit given?

    1. Yes, you may post it at your site. Thanks for asking!

  60. Meh. This is media theory without ever having read critical theory, or, having read it, ignored it. Start with some undergraduate level Chomsky and Hermann, and then spend some time in the Baudrillard Garden. Go watch a video or three from Adam Curtis. Over some coffee read Mark Fisher’s works. The centre, as described here, isn’t. It’s centre-right, and that’s being charitable. But that’s the skew of American Media, and its laughable defenders of the political-economy and status quo.

    1. I’m always looking for ways to improve. Thanks for the suggestions.

  61. Why is there no review of opinion that is supported by facts?

    1. I think that would be more accurately categorized as “analysis.”

  62. This is a pretty good start but unfortunately, most of the sources you have listed as middle of the road or “mainstream” are actually highly corporatist-biased. In effect, what that means is that there is a third dimension here which perhaps sets right wing and left wing approaches against each other without acknowledging the overwhelming power of the corporatist oligarchic state. Perhaps you would be willing to create a second chart dealing with this aspect of news bias.

    1. Also, where is Propublica, an excellent news source.

      1. On my list!

    2. I’ll certainly discuss, and perhaps chart, measures having to do with ownership and topic focus in the future, which are not captured on this chart.

  63. Don’t see the TYT network on here, do you rank them? Thx

  64. I think it would be neat to have a bucket that sits right above the apogee, named “Primary Sources” or similar. While there certainly will always be a place in the world for news reporting, digging up (errr, googling for…), reading through, and analyzing an actual source pieces of information is a refreshing exercise. I highly recommend it!!

    You could include things like Supreme Court transcripts, government budget reports, proposed legislature, and so on.

    This would be a step away from the original purpose of this wonderful chart (I’ll hold my “you need to move X and inch to the Y” comments), but it seems like a natural evolution, especially for folks looking at this to determine how they should become best informed.

    1. Excellent idea. Thanks for it!

  65. I’m surprised that the Daily Mail is so close to the center….pretty much unreadable as a news source.

    1. It’s still right on that orange/red line, which is pretty bad.

  66. Thank-you, Vanessa, for the site, good stuff! 🙂 Similar to your model, I have been working on one that looks at presidential elections and how news sources historically label the declared candidates, in contrast to their rating from the American Conservative Union, American’s for Democratic Action, and in some cases, Barone’s Almanac of American Politics. Specifically, I am looking at the years 1980, 1988, 2000, 2008 & 2016 (open elections/no incumbent). An example would be how in 1988 a ~liberal Republican might be labeled “conservative” or a ~conservative Democrat labeled liberal, or vice-versa. Any advice on how I could make it more focused, perhaps presenting it as an infographic?

    1. Sure–I’ll e-mail you!

  67. This article is an excellent generalization of the general periodicals

    1. Isn’t it?

  68. What about religious news sources like National Catholic Register, Vatican News, etc. I’m more interested in reliability then political designation. Other sources like TBN, the 700 club etc. would also be interesting. Jewish and Islamic news sources also would be interesting.

    1. Thanks for asking. I’ll look into those and put them on my list to evaluate.

  69. This is a great update! Love the increasing rigor/thoughtfulness/clarity of each edition.

    I think you’ve hinted at something like this in the works, but I’d add a vote for a chart that provides more detail on fewer sources. I think visualizing the full range of programming/articles for the 10-15 most trafficked sources would be telling. It might create opportunity to separate out different shows on a network or distinguish between the bias of a paper’s reporting vs. their editorial page.

    And, I know you’ve got a long list, but for the purposes of helping folks spot biased sources, I think including think tank publications (e.g. Think Progress from CAP or the Daily Signal from Heritage) could be really helpful. Then again, a chart of think tanks could be its own beast!

    1. Excellent ideas. Indeed I am working on charting individual stories for popular sources. And I do like the idea of a chart of think tanks, or at least think tank sources!

  70. Vanessa, have you considered a model that also illustrates the audience size of each network? Ie: what is the influence of this distribution?

    1. Yes. Working on it. That’s an important metric but requires a lot of data (which I am currently compiling).

  71. Awesome work!

    A question: How are you producing the x-axis metric? The y-axis is quality, but the end members of your x-axis also include quality. It seems like the last two categories on the x-axis are conflating quality and partisanship.

    1. Thanks for the question. I will have a whole future blog post on the x-axis, but some of the general measures are: promotion of an idea associated with one side or the other; use of terms that sides prefer in a debate (e.g., pro-life, pro-choice, death tax, estate tax, etc.); petty attacks (non-substantive ones about appearance, for example) of a political candidate; use of quotes from a politician to promote the truth of their position; framing of an issue/headline to paint a politician in an unfairly bad light; non-neutral headlines; and omission of facts that would support an opposing viewpoint or conclusion are some of the metrics I use.

      You are correct that there are overlaps in quality and partisanship, but I would argue that sources can be “hyper-partisan” and still high quality, and I have several sources in those categories. There are also trash sources that aren’t necessarily partisan (e.g., National Enquirer, which does have partisan trash but also non-partisan trash). There just seem to be a lot of sources that are on the lowest category for quality because publishers often use terrible content for partisan purposes.

  72. A friend recently shared an article from gatestoneinstitute.org. Where does Gatestone fall on this chart?

    1. I’ll follow up on these individual requests. Thanks for your patience!

  73. Natural News is mostly a do-yourself type site with a lot of Patriot-type connections. They’ve said a few times that they think Left-leaning politicians are seeking to keep them from using natural cures. The owner is notoriously a Tea Partier.

    1. (Repeating an earlier response RE: Natural News) I hear that. Natural News is one that many observers here have disputed, and I agree that they do have a mix of anti-liberal and anti-conservative stuff on there. It was my impression that they had more stories that were latched onto by left-learning conspiracy theorists, but in future versions I will likely move them more toward the center. I think it will be necessary to label the “center” as encompassing not just “centrist” or “mainstream” political views, but also as encompassing sources that have extremely partisan articles, but on both sides, with their averaging resulting in a placement them somewhere closer to the middle. Also, I do consider the ownership, but it is not determinative–the articles are. The reason for this is that someone can own a site for disseminating misinformation that primarily targets people of their opposite political persuasion (e.g., the Patribotics blog owner).

  74. While the infographic is interesting, it doesn’t give an accurate view of how extremist and the “extreme right” is compare to the “extreme left”, and how mainstream. Several of the left wing “Red Rectangle” sites appear to be health sites, not even political or general news sites, so I’m not even sure why they’re there (how is NaturalNews.com on there? You could probably find a NASCAR site that was equally right wing!)

    If you look at the ranking of the Red Rectangle sites:

    * NINE of the “right wing” Red Rectangle sites are in Alexa’s top 10,000
    * FOUR of the “right wing” Red Rectangle sites are top 1,000
    * Not a single one of the “left wing” Red Rectangle sites is in the top 10,000

    If you do a future version of this infographic, weighting the popularity and influence of these sites would be really helpful.

    1. And here’s a link to a table showing those rankings: https://i.imgur.com/CwPdRa6.jpg

    2. Hi Lisa,

      I don’t mean to imply that there is an equal number of sites on the right and left, or that they have equal influence on public discourse or policy. I’m certain they are not, and statistics such as the ones you cite absolutely support that proposition. I also don’t assert that the various sites are equal, quantitatively, in their level of extremism or misinformation. This chart is not to scale on such quantitative measures; Infowars stands out as an example, and is quite a bit worse than its nearest-ranked neighbors. Such are the limitations of a visual graph. I understand people may have an impression that I am trying to imply equality between the sides, but I do not assert that. I just wanted to show that there are examples that fall within each category, and that they can be ranked in some sort of relationship to each other.

      I do plan on putting out versions that rank them by popularity in the future. Influence would be a different thing to measure. There are some studies that have attempted to do that already.

      Best,
      Vanessa

      1. Thanks for responding! I’ll mention the above in the post I wrote. I did get the impression from your other comments that you have no personal bias, which is why the selection seemed curious.

        I do think this is a really valuable area of research and I hope you continue to do more of it. It’s a huge task though, so maybe crowdsourcing some of it might be an idea? Eg it would be good to compare English language media to other languages, if that were ever possible.

  75. Why did you leave Wikileaks out?

    1. Well…Wikileaks is a whole other ball of wax. Since its primary focus is publishing large volumes of leaked documents, I don’t think it is appropriately categorized as a “news source” (even though that term is defined so loosely in this chart).

  76. All generalizations are false. My favorite antinomical statement.

    Anyway I recommend you add Deutsche Welle (English language) and the Straits Times to yor list. I use Al Jazeera, BBC, the Times and them to get an idea what’s going on in the World.

    Here’s an example from the Straits Times on dealing with urban flooding. It’s interesting that Atlanta is ding the same thing for the same reason….

    http://www.straitstimes.com/asia/east-asia/sponge-cities-the-solution-to-chinas-flooding-woes

    p.s., I spent several years in southwest Asia, including the Arabian Peninsula, Pakistan, and Afghanistan. Al Jazeera seemed to be the only news organization that usually got things right.

    JCM
    Colonel, United States Air Force

    1. Thanks so much for the feedback. I will look into those sources. It’s valuable to get perspectives from people who have spent significant time outside the U.S. Thank you for your service!

      1. Great graphic and article.

        There is massive scope for going international if you have the bandwidth – i find it fascinating to read/watch English language news from China (CCTV), Japan (NHK), Russia (RT), France and Germany and realising just how big the zoo of biases is.

        You have Daily Mail, Guardian and BBC already but there is a British-only chart ready to break out. As well as the Independent mentioned by others, there is the Times (the original “paper of record”, but sadly drooping in standards in recent years), Telegraph and loads of tabloid stuff (Mirror, Sun, Express…)

        Finally, as a site driven by statistical analysis it would be fun to see some analysis applied to 538 (do they come out liberal…?)

        Again, thanks for doing this.

        1. Those are great ideas. Probably won’t get to international versions very soon, but I’ll put those all on my list. Thanks!

  77. Thanks very much for doing this. I’m going to print out a copy for reference and stay tuned for updates.

    One of the things that bother me about social media is how often people share information from unreliable sources. I don’t want to be one of these people. And I want a good tool to give me a quick idea of the reliability or bias of a news source that’s new to me. This does the trick nicely. Great work!

    1. Thanks! I appreciate it.

  78. Gateway pundit seems to have gotten a lot of attention these last couple days around the Moore story. Just had someone asking me about them as a source; was surprised not to see them listed.

    1. I’ll follow up on this one. Thanks!

  79. I love this chart–but ever since the first version I’ve been wondering what the red logo in the lower left corner (right above Occupy Democrats) is. I can’t read anything but the orange word “Report.” It’s not any of the “–Reports” I know of (that aren’t identifiable on the chart)–what is that logo? It’s totally unreadable on the chart. (The graphic designer who did that logo should find another line of work)

    1. Hi! It’s Bipartisan Report. I can’t speak ill of any graphic designers, being so limited in such skills myself.

  80. Have issues with a hierarchy that puts “complex analysis” *below* “fact reporting” and even puts “analysis” besides (just *above*) “opinion” (opinions and journalistic analysises are very different things, indeed). Clearly, sources such as The Economist, The Guardian or The New Yorker which do explain contexts, significance and consequences should be considered as *superior* to news agencies (AP, AFP, Reuters, etc.) who mainly produce raw news material (including verbatims of pure PR bullshit) that not only feeds the work honorable news outlets but provides the fodder for fake news.
    Personally, I would have reversed the order of the 3 first lines (3,2,1 instead of 1,2,3) of the graph (also as a matter of respect for hard-working journalists and media outlets).

    1. I totally understand, and my very first version of the chart had it exactly in that order. I initially considered those very sources you mention, which I respect highly, to be the “best” for my own personal information gathering. I explain my reasons for changing the ranking in my second version of the chart. http://www.allgeneralizationsarefalse.com/the-chart-second-edition/. There are many valid arguments for what constitutes the “best.” In reality, I think both raw fact reporting (i.e., through wire services) and complex analysis are the best for different reasons. However, I think the value of the chart lies more in distinguishing what’s not good at all.

  81. I would just note that the order of Complex Analysis > Analysis > Opinion/fair persuasion isn’t quite right. The top category (closest to fact reporting) should be Analysis (or perhaps “Simple Analysis”, then Complex Analysis (which is necessarily tinted with opinion), then Opinion/fair persuasion.

    1. You may be right. There is a lot of complex analysis that is loaded with opinion. However, there is a lot of complex analysis that is loaded with data to support conclusions. Before making changes, I want to refine how to measure the articles in these spaces, and that will require evaluation of a larger set of articles. Thanks for pointing that out though–it is helpful.

  82. I admire the amount of work and traction you’ve gotten with this, and the amount of effort you are putting into responding to comments. (Another IP attorney here, if that helps my credibility; also used to do data visualization as a computer programmer). Here’s my comments:

    (1) Another commenter mentioned WSJ’d editorial page being way right, but its news is accurate. This is true. But the data-display problem goes beyond WSJ. The Guardian, The Intercept, and even Mother Jones, have won awards for investigative hard news reporting, but certainly no one would argue that their editorial positions are liberal, often very much so. To put them under “complex analysis”, where “fact reporting” is a separate category, is misleading, since they do in fact do “fact reporting”. Maybe two graphs, separate for opinion/news interpretation and reputation for straight news-reporting? Or change y axis to be “percentage of content which is hard news reporting”, which I think is more what you mean by it. Maybe then it could become “opinionated interpretations of fact” about halfway down and then

    (2) To classify something as “utter garbage” based on its left/right position seems un-useful. I would just make the X-axis left-right rather than about quality, so with categories like “skews liberal”, “very liberal”, “super-liberal”. I am mostly familiar with the left side, so I can confidently say, for example, that Jacobin is waaay to the left of the Atlantic (e.g., The Atlantic employs David Frum, a former GW Bush speechwriter; Jacobin self-describes on their webpage as “a leading voice of the American left, offering socialist perspectives on politics, economics, and culture”). And while Jacobin does “complex analysis” it does almost no news reporting. But it’s not utter garbage. It’s just really far left. National Review probably would be a right-ride equivalent, which should go further right and a little further down. Coming from the left, I don’t like their ideas, but I approximately trust them not to distort reality. You can see how you’ve wasted white space in the top-left and top-right; you have made an assumption that high-quality and high partisanship are mutually exclusive.

    (3) The use of “but still reputable” implies that if you are outside that range, the publication is no longer “reputable”. Consider deleting. “Skews liberal” or “Skews Conservative” is an accurate assessment by itself.

    (4) Another axis-label issue: “overall quality” is also an unfair overgeneralization (all generalizations are false, but some are falser than others!). What would it mean to say that Reuters, Bloomberg, and USA Today have higher “overall quality” than the NYT? I think maybe you mean how little editorial bias they have, but that’s not necessarily “quality”. The NYT has 122 Pulitzers. USA Today has zero.

    (5) I suspect you’ve put Bloomberg way up top with Reuters because it reports a lot of “pure facts” like financial data, but if you look at Bloomberg View, it’s got a fair amount of editorializing.

    1. Hi Joseph! I appreciate your well-thought out comments. We IP attorneys are a rare breed 🙂 I largely agree with your points, and I’ll respond like any lawyer to a long e-mail: in corresponding bullets.

      1) The data display problem can be largely solved by ranking individual articles. The axes become highly accurate when I do that. The problem with displaying entire sources is that I am displaying them as an average. Generally, the sources I put toward the top of “complex analysis” are there not because they have the most complex of analyses (though some do), but because they do also produce investigative reporting (e.g., the Guardian, Intercept, Mother Jones). That is the same reason I have WaPo and NYT on the edge between “fact reporting” and “complex analysis”, because one can rely on them heavily for both. But they also have stories that fall into mere analysis and opinion. The ranking, right now is meant to be a quasi-quantitative one, with the logos settling on the kinds of stories one can most rely on from that source. You touched on this with the “percentage of content” idea.” I’ll put out more single-article rankings and I think that will lend more credibility to my y-axis. I’m sure it can be further improved, but I’m going to take in more ideas and do more analysis before I make changes.

      2) I agree. The “utter garbage” label is an artifact of my original chart which had a lot of internet snark. Little did I know it would become a valued resource and that people would expect it to be all data-driven and respectable. I agree that partisanship and quality are not mutually exclusive, and I was bothered by the white space; there’s no high-quality garbage.

      3) True. I deleted the label “questionable journalistic value” under “hyper-partisan” for that reason

      4) As we discussed, the y-axis is hard. However, I think the categories are specific enough, and people certainly expect to read this visual as having “good” at the top and “bad” at the bottom. Now there is good reason to argue that the complex analysis sources are “better” than the raw fact reporting sources, including wire services. In my very first version of the chart, I had them inverted. However, as I explained in my blog post about my second version, http://www.allgeneralizationsarefalse.com/the-chart-second-edition/ there are reasons to value fact reporting over any type of analysis. Namely, none of the analysis would exist without the fact reporting. You get one important story, and thousands of takes will flow from it. I think it’s important to value those who are going to the primary sources as the most important part of the media ecosystem. The true reporter is the heartbeat of it all, more so, I would argue, than the smart thinker with the beautiful prose. And I will also defend USA Today for the role they fill. Sometimes people just need the most basic version of the story.

      5) Yes, that’s why they are up there. Individual ranking of Bloomberg articles would solve the problem.

  83. MSNBC is opinion but FOX is propaganda? That right there shows your bias. Both have opion shows and skew the reporting to thier side of the political spectrum so of you put them both in opinion you would be right.

    1. I understand your general sentiment here, but in the samples I have used to create my assessment, Fox is quantitatively worse in their opinion shows (i.e., misleading statements, percentage of opinion statements, overt promotion of partisan ideas). I don’t think one could predict that both networks would be exactly the same in quality and partisanship, but one just to the left and one to the right. That would be a strange result.

  84. Where is “The Atlantic”?

    1. It’s there.

  85. Hi Vanessa,

    Thanks so much for this! It is a wonderful project, something that is important to me personally and something I could use in the classroom. I love too that it is a living project, guided by new information and input.

    I have a worry about where to put the National Enquirer. Maybe you have data that suggests that liberals buy that magazine as much as conservatives but if not, I think they should move to the right.

    There are at least two issues though. One is what kinds of stories are reported? Since the Enquirer has a LOT of politically neutral material, they come out as politically neutral. But in the case of the Enquirer it isn’t the headlines, it is the story. And one could tell a story that is neutral on the outside — where both the left and the right come off as evil — yet be very slanted politically.

    So I ask: Why isn’t the National Enquirer far further to the right?

    1. You may be right that they should be further to the right. I don’t have a hard numerical average of the political leanings of their stories. It’s more an impression I am left with after looking at of a number of headline and articles sampled over several days. In my samples, they seemed to have more right-leaning stories, but not enough to make them an obvious conservative publication. A ranking of individual stories will always be more accurate.

      A common theme I have noticed with these very low-quality publications (Natural News, Real Farmacy)is something you mentioned: that they are slanted politically but not necessarily against one side. They may be more railing against “the man” “them” “the government,” or “big corporations.” I may have to create a slightly different designation for stories in these categories, to indicate something is wrong with them but that to put them in a “middle” of a spectrum is not quite accurate.

  86. The Young Turks is missing from your chart. they track higher than Newsmax.

  87. I don’t see a request for zerohedge.
    Thanks for doing this.

    1. Will do!

  88. This is brilliant Vanessa! Incredibly useful as a reference for truth-seekers with the understanding it is a moving target you are trying to follow!

    One question though…is it possible to get a text version? I find some of the logos hard to read.

    Thank you for your work on this!

    1. Thanks. Good idea to have a text version. I’ll work on it.

  89. Vanessa,

    Awesome work, thank you for keep updating this one!!!

    Check out some of this work, maybe it can help you with yours:

    https://medium.com/linalgo/predict-political-bias-using-python-b8575eedef13

    We have all articles for download and maybe it can enhance your data set.

    Vladimir

    1. Hi Vladimir,

      Thanks so much for sharing. I definitely have been looking for people who have done work on software analysis of large sets of published content. Thanks for the resource. I will look into your work further.

  90. I highly disagree with your placement of DailyKos.

    Kos varies from plain fact-based journalism to fiery op-ed. One extreme is one and the other extreme is the other. Because it’s made of diaries written by independents and not controlled by a unified editorial, it doesn’t deserve a single placement. And if it belongs 75% in the yellow and 25% in the orange, then it doesn’t belong 50% in the orange. It belongs at most 25% into the orange and more like spanning the borderline.

    This is a prime example of “Lying with Graphs.”

    What you get out of Kos depends on what you read.

    1. I agree that a more accurate ranking would be of individual articles, which I will put out. Nonetheless, the reason I weight them (which you may disagree with, of course) is because I want to capture a measure of “what people count on them for most.” It is my impression that the fiery partisan stuff tends to drive their clicks and views, and therefore, is what many readers rely on them for, and is the primary lens through which those on the other side view them through. I could get a more accurate measure of “what people rely on them for” by ranking any sites most shared or most read articles; many sites freely provide that information. In comparison, readers may similarly rely on WaPo, for example, for their analysis and opinion pieces, and those may also drive clicks and views. But because WaPo also produces such a high volume of original reporting (they have consistently had between 600-800 journalists and staffers over any given time within the last several years, and their reporting stories are syndicated in local papers all across the country), I think it’s fair to categorize them as a source that people rely on for fact reporting.

  91. Why is it that everything i hear on infowars comes true but they are considered fake?

    1. Please give an example of a story, or set of stories you are referring to, and I can respond more specifically.

  92. […] created this infographic showing (predominantly US) news and politics media sources and their respective reliability and […]

  93. Is there a plot of this but with circles proportional to the total viewership of each news source? It would be interesting to see what’s it good news and what news are people consuming.

    1. Yes, I am working on that.

  94. Vanessa,

    Great work. I like that you are taking a scientific approach and adding your assumptions. This needed to be done. I wish your wordpress site was a little easier to find and get to. Let me know if you need some help! Keep up the good work!

    1. Thanks! I’ll contact you.

  95. Are you employed by any of these company’s that you put on your chart? Or, do you receive income from any of these company’s that you put on your chart?

    1. I am not employed by any of these companies and I do not receive income from any of them.

      1. Characterizing the Mainstream section with a subtitle in parenthesis “(minimal partisan bias)” is inaccurate to say the least. And, we all, all of us, know that characterization is actually tremendously inaccurate. Enclosing that group with a green rectangle that you state with your key as denoting “news” may be correct technically speaking, but we all, all of us, also know the rampant non-reporting of significant news stories with this Mainstream group is tremendously problematic. “News”? Yeah. OK. I do agree with your most important statement, though, “why should you listen to me about the quality of news sources? You shouldn’t.” That statement is our one point of agreement. You are a product of the tremendously liberal, if not outright fascist/leftist, west coast environment education machine, and you now reside in Colorado whose radical liberal culture we all see speaks for itself. You are a propagandist participating in the attempt to promote the perception of the Mainstream media as being the apex of news.

        1. Colorado is hardly a radical liberal culture. More than 50% of Colorado is inhabited by either conservative Republicans or conservative Libertarians. And you seem to somehow equate fascist with leftist and both with West Coast, none of which is accurate. Fascism is a right-wing, nationalist and authoritarian, often racist, ideology; Hitler and Mussolini were both proponents of fascism. The West Coast is not a monolith of fascism, nor of wild-eyed liberals, either; there are plenty of conservatives and extreme conservatives on the West Coast. Mr. Wasnick, I suspect you are an acolyte of Breitbart and/or Alex Jones and certainly Fox News, judging by your rather rude and ill-informed reply to this woman’s painstaking and accurate charting. You might want to try some of the other listed publications/websites to get a more balanced, less “right-wing talking points” kind of view.

          1. Legalizing marijuana for social use is radical. Your Antifa guys are the epitome of Facism. Evidence Berkely. Mrs. Diane W, you suspect wrong. Her chart is inaccurate. How do you know it was painstaking? That chart could have easily been slapped together in 5 minutes. And, if you also think the mainstream media is characterized accurately as “minimal partisan bias” then you are simply a part of one of the saddest and biggest problems in this country. And, you are participating in causes that are adverse to the USA.

  96. What an excellent resource! Could you include the Globe and Mail out of Toronto?

  97. I’ll give you a clue as to why your chart is not working. It does not address credibility.

    Many of these mainstream media sites have zero credibility in the eyes of many Trump voters/Infowars viewers.

    It will take at least 10 years for these msm outlets to get back any of the credibility that they threw away in 2016.

    That’s it. I’ve said too much already. But that’s okay, because you won’t believe me anyway.

  98. I would rather see this by individual media personality. I have serious doubts CNN is as in the middle as described based on watching their personalities deliver the news. I see way too much personal opinion from Cuomo and others. When I see reporting, the first thing I do is go to google and check their social media and previous bylines, then start scouring for personal opinion so I can determine their bias. Personally, I see almost every story by CNN is anti-Trump as much as almost every Fox story is pro-Trump…two sides of the same coin. On the other hand, someone mentioned Al Jazeera as being left…I think their puff pieces are far left, but their straight news reporting is middle. Just my opinion…worth about as much as everyone else’s.

  99. I don’t see the CBC on there. Is it too specific to Canada? I’d be curious as to where you’d rank it.

    1. Yes. I only have selected international publications that seem to get a fair amount of traffic in the US. It quickly becomes problematic on the political bias scale, because other countries’ left-right spectra inherently differ. My assessments of the international publications on here are mostly based on their coverage of US stories. Those stories tend to be more neutral with respect to US politics than they are in their own countries with respect to their own politics. For example, BBC and Daily Mail stories about the US tend to be less partisan than their own stories about UK politics.

  100. Vice News?

  101. Hi, Vanessa, really cool. One minor edit, I think, where you say first in Key, “Circles and ellipses represent range that stories within a source often fall (not true of all sources though),” I think you mean to say, “(not true for all stories though)” as I might find most NYT-sourced stories pretty fair but then come across one story that flames red against Trump with no holds barred.

    1. Ah, thanks. Good catch. I’ll edit.

  102. Thanks for this! I’m a US govt teacher and appreciate the effort and chart….I might quibble with a few positions (The Nation is much further to the left than The Atlantic!)and think you should add The Financial Times somewhere (it’s the source I encourage students to go to for international news), but otherwise a fine job!

    1. Thanks! I’ll add the Financial Times to my list!

  103. You continue to amaze, Vanessa, with the analysis and energy you put into the Chart–as well as responses to the comments about it. Thank you.
    I recall a discussion a while back abt how to consider what’s missing from a news source–individual facts, stories, or entire topics. That’s a thorny one but very important. What are you thinking about that, these days?

    1. Hi Susan. You must be reading my mind. My next post will be about that!

Leave a Reply

Your email address will not be published. Required fields are marked *