Posted on

The Chart, Second Edition: What Makes A News Source “Good?”

In my original news chart, I wrestled with the questions of what made news sources “good” and came up with some categories that generally resonated with people. I ranked sources on a vertical axis with those at the top ranked as “high quality” and those at the bottom as “low quality.” I characterized the sources, from top to bottom, in this order: Complex, Analytical, Meets High Standards, Basic, and Sensational/ Clickbait. This mostly works, because it results in sources regarded as high-brow or classy (e.g., The Atlantic, The Economist) being ranked high on the axis, and trashy sources (e.g., Addicting Info, Conservative Tribune) being ranked low, and most sophisticated news consumers agree with that. However, the vertical placements ended up causing me and others some consternation, because some of the placements relative to other outlets didn’t make sense. The most common questions I got were along these lines:

“Does FOX News really “meet high standards,” on par with something like the New York Times?” (I think no.)

“Is USA today really that bad?” (I think no.)

“Is Slate really “better” or “higher quality” than, say, AP or Reuters just because it is analytical?” (I think no.)

“Is CNN really that bad?” (I think yes.)

These questions and my instinctive responses to them made me want to reevaluate what makes news sources high or low quality.

I believe that answer to that question lies in what makes an individual article (or show/story/broadcast) high or low quality. Article quality can vary greatly even within the same news source. One should be able to rank an individual article on the chart in the same way one ranks a whole news source. So, what makes an article/story high or low quality? It’s hard to completely eliminate one’s own bias on that issue, but one way to try to do it consistently is to categorize and rate the actual sentences and words that make it up its headline and the article itself. In order to try to rank any article on the chart in a consistent, objective-as-possible manner, I started doing sentence-by-sentence analyses of different types of articles.

In analyzing what kind of sentences make up articles, it became apparent that most sentences fall into (or in-between) the categories of 1) fact, 2) analysis, or 3) opinion. Based on the percentages of these kinds of sentences in an article, articles themselves can be classified in categories of fact, analysis, and opinion as well. Helpfully, some print newspapers actually label articles as “analysis” or “opinion.” However, most news sources, especially on TV or the internet, do not. I set about analyzing stories that were not pre-labeled as “analysis” or “opinion” on a sentence-by-sentence basis. I discovered that my overall impression of the quality of an article was largely a function of the proportion of fact sentences to analysis sentences to opinion sentences. As a result, I classified stories into “fact-reporting,” “analysis,” and “opinion” stories. Ones with high proportions of “fact” sentences (e.g., 90% + fact statements) were what I refer to here as traditional “fact-reporting” news pieces. These are the kinds of stories that have historically been the basis of late 20th century-to-early-21st century journalism, and what people used to refer to exclusively as “news.” They are the “who,” “what,” “when,” and “where” pieces (not necessarily “why”). I classified ones with high proportions of “analysis” sentences (e.g., 30%-50% analytical statements) as “analysis” stories, which are the types of stories commonly found in publications like The Economist or websites such as Vox. I classified stories with high proportions of opinion sentences (e.g., 30%-50% opinion statements) as “opinion,” which are typically the types of stories found on websites such as Breitbart or Occupy Democrats.

(If you’ve made it this far, bless your heart for caring so much about the news you read.)

In the past, national evening news programs, local evening news programs, and the front pages of print newspapers were dominated by fact-reporting stories. Now, however, many sources people consider to be “news sources” are actually dominated by analysis and opinion pieces. This chart ranks media outlets that people consider to be, at some level “news sources,” even though many of them are comprised entirely of analysis and opinion pieces.

In my previous version of the chart, I had regarded analysis pieces as “higher quality” than the fact-reporting pieces because they took the facts and applied them to form well-supported conclusions. I like analytical writing, which is essentially critical thinking. However, analysis has a lot in common with opinion, and writing that is intended to be analytical often strays into opinion territory. (Note—I’m defining “analysis” as conclusions well supported by facts and “opinion” as conclusions poorly supported or unsupported by facts). Fact-reporting articles—true “scoops”—typically have the intent of just reporting the facts and typically have a very high percentage (e.g., 90%+) of fact-statement sentences, whereas both analysis and opinion articles have the intent of persuading an audience and often have a comparatively high percentage of analysis and opinion statement sentences (30%-50%). So, although I initially had the quality axis of “news” laid out top to bottom as:

That ranking is more reflective of the quality of writing rather than the quality of news sources. Good analysis is often written persuasively and well, fact-reporting is often written directly but well, and opinion writing is often (but not always) written poorly or is most easily discredited.  I submit that given the confusion caused by the overwhelming number of organizations proclaiming to be (or which are commonly confused with) “news sources,” it is more important to rank the quality of news sources than the quality of writing. I further submit, for reasons outlined below, that the percentage of fact reporting articles and stories should be used as the most determinative factor by which a news source is ranked in quality on this chart.

Therefore, I believe a more relevant ranking of the quality of news sources would be:

 

 

I assert that one of the biggest problems with our current news media landscape is that there is too much analysis and opinion available in relation to factual reporting.  New technologies have given more people more platforms to contribute analysis and opinion pieces, so many “news sources” have popped up to compete for readers’ attention. Unfortunately news consumers often do not recognize the difference between actual fact-reporting news and the analysis and opinion writing about that news.  This increase in “news sources” has not corresponded with an increase in actual journalists or news reporting, though. Many local and national print news organizations have reduced their numbers of journalists, while many of the biggest ones have merely maintained similar numbers of journalists over the past 10 years or so.
Furthermore, primarily analytical news sources also have several downsides. One downside is that they can alienate news consumers by making what people consider “news sources” so complex or partisan that it is tiring to consume any “news.”  For example, CNN, MSNBC and FOX News, which are primarily analysis and opinion-driven, can make news consumers too weary to pay attention to fact-based reporting from, say, AP or Reuters. Another problem with analysis and opinion-driven news sources is that it can be difficult for casual readers to differentiate between good analysis and pure opinion.
There are several good reasons why we should value fact-reporting sentences, fact-reporting articles, and fact-reporting news sources high on the quality scale of news, at least on this chart. For one, reported facts take a lot of work to obtain. They require journalists on the ground investigating and interviewing. Once a story is reported, dozens, hundreds, or thousands of other writers can chime in with their analysis or opinions of it. This is not to say analysis and opinion writing isn’t important. The critical thinking presented in analytical writing—especially good, complex analysis—is essential to public discourse. Our society’s best ideas are advanced by analytical articles. This piece you are reading now is analytical. But analysis in the news wouldn’t even exist without the underlying factual reporting.

For example, AP and Reuters have maintained around 2,000-2,500 journalists each over that time, while the New York Times and Washington Post have fluctuated in the 500-1000 range over the same period. The value of these organizations with large staffs of journalists, editors, and other newsroom employees is hard to overstate; not only do they provide a majority of the fact-reporting stories everyone else relies on, but they have the capacity to provide high-quality editorial review that stands up to industry scrutiny. In contrast, even some of the most popular analysis and opinion sites can be run with just a few dozen writers and staff; the number of these “news” websites, news aggregator websites, blogs, and podcasts has seemingly grown exponentially.

I believe improvements to our media landscape can be made if two things happen: 1) if news consumers start valuing factual reporting much more and analysis/opinion articles much less and 2) if news consumers become accustomed to differentiating articles in those categories.. Regarding point #2, I think it would be helpful if we narrowed the definition of “news” to only refer to fact reporting, and referred to everything else as “analysis” or “opinion.” It would be helpful if people could recognize the relative contributions of fact-reporting news organizations versus analysis and opinion sources. If people recognize just how much of what they read and watch is intended to persuade them, they may become more conscious and thoughtful about how much they allow themselves to be persuaded. One can hope.

To contribute to those goals, I’ve reordered the chart to value fact-reporting articles as the highest quality and everything else lower, even though there is some really excellent analysis out there. As a baseline, news consumers should understand when something is news (fact-reporting) and when it is not. On the new chart, the sources with the best analysis, but little reporting are at the top, but right under the sources that are comprised of high percentages of reporting articles. The most opinion-driven sources are at the bottom. There’s room for other things at the bottom below pure opinion, which can include sources that are sensationalist, clickbait, frequently factually incorrect, or which otherwise don’t meet recognized journalism standards.

On this version, I’ve included a number of different sources, mostly in the analysis and opinion categories, and left the most popular mainstream sources from the original chart, but have reordered some of them. Now, the rankings are more consistent with my initial answers to the example questions at the beginning of this post. Fox News is now ranked far lower than the New York Times for two main reasons; one, Fox News is dominated by opinion and analysis, and two, it has gotten precipitously worse in other measures (sensational chyrons, loss of experienced journalists, hyperbolic analysis by contributors, etc.) within the last six months. USA today, despite its basic nature, has been elevated because of its high percentage or fact-reporting stories. Slate, though it provides thoughtful, well-written analysis, is ranked lower than AP and Reuters, which better reflects their relative contributions to the news ecosystem. CNN still sucks, but it is clearer why now; CNN has the resources to provide twenty-four hours of news—it could provide Americans with a detailed global-to-local synopsis of the world—but instead it chooses to spend 5% of its time fact reporting a handful of stories, comprising mostly American political drama and maybe one violent leading world news story, and 95% on analysis and opinion ranging from the competent to the inane.

My analysis of news sources in the manner I’ve described herein has revealed that individual stories can and should be ranked on the chart in the same manner, and that individual stories can be placed in different places than the news sources in which they are published. I’ll be putting out individual story rankings and reasoning for those rankings from time to time for those that are interested. I’ll also take requests for rankings of sources and individual stories in the comments and on twitter. Thanks for reading and thinking.

*Update: a high-resolution PDF version is available here: Second Edition News Chart.V2

And a blank version is here: Second Edition Blank

 

Posted on

What is the difference between the statues of George Washington and Robert E. Lee?

The pro-confederate-statue side asks this question, likely in earnest, and it is worth grappling with the distinction. Indeed, since slavery is evil and horrible, as generally agreed by liberals and conservatives alike, and both men owned slaves, why is it preferable to take down the confederate statue and not the Washington statue?

This is not cut and dry, or “obvious” to everyone, and we shouldn’t treat it as such. It is a difficult task to distinguish between two things that are alike in some ways and different in others, so let’s look at the details and facts of these cases in order to distinguish, like courts do.

It is a general rule that we put up statues of good people and not bad ones, but this in itself is a hard rule to follow because no one person is all good or all bad. It’s a bit easier to distinguish with some people than others. MLK=almost all good and Hitler=almost all bad is not hard. I think it is legitimately closer with both George Washington and Robert E. Lee. I think the reason the argument comes down to GW=mostly good (despite slaves!) is because he is most known and respected for 1) fighting in the Revolutionary War for American independence, which modern Americans view as a righteous cause, and 2) being our first President. The argument comes down to Lee=mostly bad (plus slaves!) because he is most known for 1) fighting in the Civil War for the cause of keeping slaves, which most modern Americans view as a morally wrong cause.

The question of what they are most known for is an important one, because that is usually the same reason their statue was put up in the first place. When it comes to the question of whether to take one down, people tend to base their opinion on the questions on 1) what it meant when it was put up in the first place and 2) what it means now, in the context of history. With GW, it was put up because of his role in the Revolution and as President. With Lee, it was put up during an era of brutal reinforcement of white supremacy (see comments for link discussing this history) with a purpose of intimidating recently freed slaves. Today, in the context of history, GW’s statues are widely seen as a reflection of his leadership and role as a founder, not his role as a slave owner. Most people don’t go to a GW monument for the purpose of celebrating his slave ownership. Today, though, in the context of history, Lee’s statues are commonly given two negative meanings: First, they serve as a reminder of white supremacy to black people, and second, they serve as a rallying point for actual white supremacists. Yes, to many people, it may mean a “commemoration of Southern history” too, but if it’s 50% a brutal white supremacist reminder/rallying point and 50% Southern history commemoration, that’s enough to justify it being removed. We have made a moral decision as a society that its (even partial) role as a white supremacy beacon is not acceptable, in response to a particular flash point of a white supremacist resurgence. We have not made a similar decision about the Washington statues, because there has been no recent flash point around those.

However, I can’t actually morally justify Washington owning slaves, and that practice is indeed so reprehensible that it is valid to argue that if slavery is that wrong, then we should take down the statues of any slave holder, no matter how “good” they were otherwise. Joe Paterno’s statue was taken down because his biggest moral failing—protecting a child predator—outweighed the other good he had done. Perhaps the removal of Washington (slave owner) and Jefferson (slave owner and likely slave rapist) is the morally correct thing to do. We would likely remove the statues of contemporary heroes (say, MLK or Wayne Gretzky) if we suddenly found out they were rapists or owned slaves.

But there is a distinguishing factor between how we judge the actions of contemporaries compared to how we judge those of historical figures, and that is the factor of relative morality of a time in history compared to the present. Those who argue “slave owners weren’t all bad people” are inherently taking this factor into account. Yes, we all view slavery as evil now, but when it was a somewhat normalized aspect of society, it is plausible and even likely that many slave owners tried to live what they thought were upstanding moral lives in many ways. They may even have had moral dilemmas about slavery but felt that it was an intractable problem for them to solve, let alone forgo participation in. “Slave owners were not all bad people” ( a typically conservative argument) is a very similar argument to “George Washington’s statue should remain up because he did other good things, even though he owned slaves” (an argument liberals are currently making in relation to the confederate statue issue). “George Washington was not all bad,” essentially.

It seems that the right thing to do is to take down the Confederate statues because the of the bad things they were best known for (explicitly fighting for slavery), plus the reasons they were put up, plus the reasons they cause people pain now. But we must also admit that it would be logically consistent to remove other slave owners, even our founding fathers, if some contemporary flash point were to bring the issue of how bad slavery really is to the forefront. Perhaps it is a moral failing of our current time that we have not come to this realization yet. Perhaps future generations will come to the consensus that the founding fathers’ statues should be removed and hold it against our generations that we did not. Perhaps they will judge us harshly for tolerating other injustices, like unequal  women’s rights and queer rights for so long. Societal morals evolve over time. In the near term, though, it is likely that the “contemporary, widely-held perception of the statues” factor and the “relative morality of the time of the person” factor saves the Washington and Jefferson statues now but not the Confederate statues. So down with the Confederate statues. And shame, at least, on the moral failings of those whose statues we leave in place.