Posted on

Part 2 of 4: Why Measuring Political Bias is So Hard, and How We Can Do It Anyway: The Media Bias Chart Horizontal Axis

Post Two of a Four-Part Series


The Media Bias Chart Horizontal Axis:


How to Define Political Bias in a Meaningful, Useful Way

In part one of this series I laid out some problems with existing ways of measuring bias and outlined a proposed new methodology for rating such bias in news sources within a defined taxonomy (the horizontal axis of the Media Bias Chart).

In this post, I’ll first define what the terms “partisanship” and “political bias” in this taxonomy (“partisanship” and “political bias” are used somewhat interchangeably here, though they are distinguishable in some aspects). More specifically, I’ll define what the concepts of “liberal,” neutral/center,” and “conservative” mean within the scope of this chart, and the reasoning behind these definitions. Then, I’ll discuss what the horizontal categories on the chart represent.

For clarity, let’s go one step further back and specify that what “political/partisan” bias even means. Here, I refer to the preference for policy positions that are available for individual people to have on particular topics that are subject to legislation by government. I am not referring to individual people themselves as left- or right-biased. In other words, the definitions are topic-focused, not people-focused. For example, I will define policy positions, such as “taxes should be higher/the same/lower on wealthy people” as liberal/centrist/conservative, rather than define individual people, like journalists or politicians, as themselves being liberal/centrist/conservative.

Regarding the answer to the questions of what “liberal,” “centrist,” and “conservative” (hereinafter referred to as simply “liberal/conservative” or “left/right”) policy positions are, this is difficult to answer because 1) what is considered liberal or conservative is a moving target over time, 2) there isn’t necessarily a “center” on each topic, and 3) some people will always disagree on the definitions I or anyone else may come up with.

As an initial matter, many people object to trying to confine partisanship to a left-right axis, arguing that there are other dimensions, such as establishment-populist, or freedom-regulation. Those that insist that these dimensions exist and should be accounted for tend to be libertarians and/or people who feel their political positions are too nuanced to be captured by a simple right/left dimension. However, several forces, including our country’s two-party system, tend to flatten those other dimensions into the liberal-conservative dimension that most Americans easily recognize. As Steven Pinker states in his book Blank Slate, “while many things in life are arranged along a continuum, decisions must often be binary.” For more on this concept, see Pinker’s book or Maxwell Stearn’s writing on political dimensionality here. Therefore, I will stick with the liberal/conservative dimension, because it covers most bias issues, and because this is a visual two-dimensional chart. A visual chart cannot show analysis like written words can, so if you find yourself getting upset about a nuanced idea that is not depicted on the chart, try to remember that this is a chart, and one of the reasons it has reached so many people is that it is a picture that necessarily simplifies some concepts. Don’t worry, someone has probably already written an excellent and nuanced article about your point.

Regarding the question of “what locations on the chart correspond to particular liberal and conservative positions,” the answer is tricky because the horizontal dimension actually represents two distinct bias concepts: 1) Political position bias and 2) Linguistic expression bias.  Political position bias refers to the “rightness” or “leftness”—the extremism—of a particular political position itself. For example, if an article portrays an extreme right-wing position, such as white nationalism, favorably, even if the portrayal is only mildly favorable, would be ranked far to the right. Linguistic expression bias represents the degree to which an article or source promotes a political position through linguistic rhetoric, even if the political position itself is not extreme. For example, if an article uses extreme language and hyperbole to promote the concept that climate change is caused by humans, which is not an extreme position in and of itself, the article it would be ranked far to the left.

Although the questions of what constitutes bias are hard, I believe it is worthwhile and possible to come up with definitions for the horizontal categories, thereby creating a taxonomy on which reasonable people of differing political beliefs can find agreement. I think it is also worthwhile to create a methodology for ranking within the taxonomy, so reasonable people of differing political beliefs can rank the same sources and come up with similar results.

Several commentators on this blog have brought up a concept called the Overton Window, which refers to what constitutes acceptable political discourse during a particular time, and which inherently recognizes that the window shifts over time. The left-right dimension of this chart attempts to capture the range of political discourse (not just the acceptable portion) in our media at the present time. This is hard to capture, but I believe we can do this if we account for enough inputs. What I specifically refer to as “inputs” are the communications that exist throughout our political system from three groups of people, namely, elected officials, journalists, and citizens.

The communications that emanate from each of these groups are important and influential in different ways. The communications of elected officials are of course important because they have the actual power to change laws. The communications of journalists are important because their platforms give them influence over how citizens see political events. The communications of citizens are important because they are the most numerous, and because they have collective power over what the elected officials (by voting) and journalists (by reading and watching) say.

It is not always obvious whose political views (of these three groups) influence the politics of our overall society. For example, many wonder, is it the media that influences citizens and elected officials? Or do citizens influence the media and elected officials? Or do elected officials influence citizens and the media?  I submit that each of these groups (which overlap, of course) influence each other in varying degrees at different times, and that push and pull of influence is what causes definitions of “left” and “right” evolve over time. For example, one can argue that the civil rights movement was caused first by citizens who influenced the media and politicians. One can argue that Fox News influenced some citizens and politicians to become more conservative on a range of issues over the last 20 years. One can argue that Obama’s endorsement of same-sex marriage influenced many citizens and media to accept it. One can argue that Trump influenced citizens and media to tolerate lower standards of behavior and experience. Each of these instances is an example of how the influence of various actors results in movement of the political spectrum over time.

The Media Bias Chart takes into account the communications from each of these groups in different ways. As I’ll describe here, the categories themselves are largely defined by the communications of elected officials themselves. The placement of the media sources within the categories reflects the communications of the media about the elected officials and the citizens.

There are seven columns on the chart:

Before defining what these seven columns represent, I’ll start with defining the scope of the chart. The versions I am creating now strictly refer to United States political partisanship. Maybe eventually this project can include international versions, but that’s beyond the scope right now. To the extent sources from other countries are included on this chart (e.g., BBC, The Guardian, The Economist, Daily Mail, etc.), their political bias rating is only with respect to their treatment of US political stories. That is, a BBC story about a British political issue would not be included in the evaluation of BBC on this chart. This is because a large basis for the evaluation of political bias is “comparison between sources.”

In order to compare the bias of sources, one has to look at multiple sources writing about the same or similar topics. Further, one has to know what comprises the political spectrum from the extremes to neutral, and the linguistic rhetoric commonly used in those countries to categorize those topics, in order to then categorize those stories.  It is highly inaccurate, if not impossible, to rate the political bias of, for example, a single story from the BBC about British politics against the US political and media landscape.

To illustrate how our impression of bias is largely dependent on comparison, I invite you to do an exercise: look up an article on Al-Jazeera, RT, BBC, or CBC—one about a topic local to the region or country from which they report. Be sure to select an article about a news topic that you are totally unfamiliar with. Chances are the article will strike you as politically neutral because you are unable to place the political issue within a spectrum of political positions on it. To the extent you are able to detect bias, it is likely due to linguistic indicators, or some reliance on comparison to your knowledge of related US political issues. It is likely your assessment of an article as unbiased would be at odds with a politically astute resident of the country about which you are reading.

Turning back to defining the categories on this chart, precisely defining what comprises the center (of our political system and of the chart) is the hardest, but I think that something approximating a consensus “center” can best be found by defining the easier-to-recognize outer left and right and then working inward towards the middle. I assert that it is easier to define the sides of an issue rather than the center of it for several reasons. One of those reasons is that some issues do not have a center. Another is because politicians tend to advocate for positions of a particular side, not for positions that are in the center. The “center” or a “compromise” is typically a result of a negotiation between two sides, and is not what politicians typically run on.

Similarly, it is easiest to detect bias in a media source the more extreme or egregious the bias is, and harder the more nuanced or unintentional it is. When it comes to articles and sources listed toward the center of the chart, or which just slightly “skew liberal” or “skew conservative,” there is room for reasonable minds to disagree as to exactly how biased these are. Whether a particular observer views these articles as skewing slightly one way or another is largely dependent on the observer’s own political leaning. These articles and sources may contain only nuanced bias, reflected in the choice of one particular term over another, or in the emphasis of certain facts at the beginning of an article and others at the end.

Referring briefly back to the vertical columns of the chart:

Most journalists at reputable sources, when writing fact-based stories (i.e., ranked in the top two vertical rows of the chart, as opposed to purposeful analysis or opinion listed in rows below) attempt to write stories that are as unbiased as possible. But since they are human, they inherently have some political bias, which many manifest subconsciously in their writing.

I submit that the absolute placement of an article or source in the middle three columns of the chart (skews liberal/neutral-balance/skews conservative) is not as important as whether an article or source falls within those middle columns or within the outer columns (hyper-partisan and most extreme). In other words, it is important for all media consumers to recognize when a source is egregiously biased. Though I stated earlier that it is easier to detect egregious bias, the state or our politics indicates that too many people are still unable to so detect it. If a moderately conservative person finds a particular article in Time Magazine skews liberal, and a moderately liberal person thinks the same article skews conservative, that disagreement can generate interesting and healthy debate, but those debates are not the dangerous ones creating extremism and damaging polarization. Those two people are reading something fundamentally reputable in the first place and engaging in productive political discourse. The worst problems with our media environment arise when people follow the hyper-partisan and most extremely biased sources and don’t even recognize that they are biased.

What Comprises the “Politics” that we are discussing, anyway?

Again, we need more baselines and definitions, because the term “politics” is broad. For our purposes here, we can define U.S. politics generally as things our elected officials have the ability to influence. And a good way to determine what our politicians think they can influence is by looking at the topics they solicit feedback on and discuss when they run for office. These often represent topics they take up in legislation when in office.

To generate a useful list of these topics, I took a look at the contact forms for each of my Senators from the state of Colorado, Michael Bennett (D) and Cory Gardner (R). Fortunately (I think), our state happens to have two fairly moderate members of their respective parties representing us in the Senate. On each of their e-mail contact forms, there is a long drop-down list of political issues about which you can contact them. The lists are long, with over 35 issues on each, and negligible differences in how they categorize them.

On some political issues, there is a fairly wide left-right political divide, and on others, there tends to be more consensus. Because this chart measures political bias, it is most helpful to identify the topics on those lists for which a discernible political divide exists. Some topics are so widely agreed upon that their political significance is negligible, and their inclusion in an article would not tell you much about the source’s political bias; for example, an article about a discovery on Jupiter would be fairly non-political. Those tend to go in the middle column absent other factors (e.g., if the article was about how cuts in NASA’s budget are limiting discoveries on Jupiter and that is a bad thing, that would be more political because it includes another political topic: budgets). For the purposes of ranking bias on this chart, it is most useful to identify what political positions have a wide left-to-right spectrum of positions.

From the political topics on my Senators’ lists, I consolidated and selected the ones with the most discernible political spectrums. These are:


Campaign Finance

Civil Rights





Higher Education


K-12 Education



Food stamps/ Welfare

Gun Control

Health Care


Social Security


Foreign Policy

When I refer to parties’ and politicians’ “positions,” I generally mean positions about these topics listed. Some of these are more polarizing than others, meaning the existing “extremes” are further apart than on other issues. For example, I submit that abortion and gun control are more polarizing than higher education.

In order to categorize how left, right, or center a position is, I used the proxies of the positions of current elected officials, as explained in further detail below. I then created a table of positions for each topic that fall into each of the categories. Because this is an evolving project and these positions change over time (sometimes significantly within a short period of time), I currently have only have this table in this paper-and-pencil copy, but I plan on converting it to an electronic format eventually. Here are some hard-to-see pictures of it:

Because the political spectrum changes over time, these positions should be revaluated and updated over fairly frequent time periods: every six months, for example.

Having defined various political positions as falling within particular categories, one can then methodically use the advocacy or favorable treatment of these positions in an article to rank them in those corresponding categories.

I submit it is possible to separate the concepts of political extremism (as measured horizontally on the chart) from quality (as measured vertically) to a certain extent, and for certain political issues. In other words, more extreme political positions do not always have to correlate with quality. The distribution of sources on the chart appears to indicate that the more extreme a position is, the lower quality the article or source is, but that is not necessarily due to the extremism of the position itself. Consider the columns of “hyper-partisan” liberal and conservative. Notice that many sources fall completely or partially within these columns, all the way from “fact reporting” down to “contains inaccurate/fabricated info.”

The reporting of certain facts themselves can create a compelling case for an idea that may be considered politically extreme at the time of its reporting; for example, at a time where adoption of children by gay couples was largely banned by law, an article reporting a study which finds that children raised by gay couples turn out to be just as happy and well-adjusted as those raised by straight couples would appear to take a very liberal policy position. Therefore, it is possible for “fact reporting” articles to fall in the “hyper-partisan” category.

Similarly, even strongly hyper-partisan positions can be supported with analysis and opinion arguments of varying quality. For example, arguments that are strong, compelling, made good faith, based on valid moral concerns, and which do not omit relevant facts from the other side can be made for even somewhat radical economic/social concepts, like libertarianism and socialism. However, worse arguments can be made for these things as well, and those quality-lowering factors are what drag articles or sources down the chart. I submit it is even possible (but rare) to write high-quality stories and arguments about some even more highly-polarized topics, such as abortion. That is, you could have a high-quality complex analysis article that advocates for an extreme position on abortion (e.g., no abortion or birth control on the right, or publicly funded abortion and birth control on the left) that would fall in the top of the “complex analysis” row and right on the right-most or left-most “hyper-partisan” line. However, the nature of very extreme positions is that they tend to be extreme precisely because they ignore some realities and/or valid concerns of the other side. The more extreme the position, the more likely it is to rely on ignored or omitted facts, and the more untenable it is for an elected official to hold. Therefore, those positions that are too extreme for even any politician to hold (in the “most extreme” liberal/conservative) columns are all in the lowest categories for quality due to their misleading and inaccurate natures.


What Comprises Linguistic Expression Bias?

As previously discussed, the horizontal categories also represent levels of linguistic partisanship; that is, I propose that the use of certain words in certain contexts can indicate levels of bias. I refer to these as simply “biased words” herein. They comprise words in the following four categories: 1) words with political connotations connecting them to certain parties or positions, 2) adjectives that don’t necessarily have a political connotation themselves, but when used to describe a political actor, party, or position, indicate political bias, 3) insults and pejoratives commonly used to describe certain political opponents, and 4) bogeymen.

The first category of biased words refers to the preferred terminology about a political position or political topic by one side or the other. These include characterizations of positions like being for/against abortion as “pro-life” or “pro-choice,” or referring to certain immigrants as “illegal aliens” or “undocumented immigrants.”  These kinds of words can correlate with quality as well, because certain ones are used as insults or in a derogatory manner, which necessarily fall into the category of “unfair persuasion” on the quality scale.

The second category of biased words refers to adjectives used for ad hominem (personal) attacks on politicians. For example, if an article applies the words “ugly” or “stupid” to politicians, those words are biased words. Such words also correlate with low quality because they are unnecessarily mean, and therefore fall into the category of “unfair persuasion.”

The third category of words includes specific insults and pejorative terms that have inherent contemporary political connotations. Examples include “deplorables,” “snowflakes,” “leftists,” and “the mainstream media.”

The fourth category of words—bogeymen—refers to people or groups that may or may not exist, but whose names are invoked by politicians or media figures to incite fear, anger, or loathing among their constituents or audience. These may be real people or groups that have committed bad acts, or acts perceived as bad by their political opponents. However, they evolve into “bogeymen” terms when they become used as abstractions of these acts, thereby transforming into a sort of common enemy. Examples include “the Muslim Brotherhood,” “the 1%,” “the Deep State,” and “Big Pharma.”

In addition to the table that I created for mapping political positions to categories on the chart, I made another table that lists biased words from the four categories above. I placed these words and phrases into the horizontal categories, categorizing the biased words themselves based on degree of bias. Again, since this is a work in progress, I only have this in pencil-and-paper format, but I plan to put it in electronic form soon. Here are some more hard-to-see pictures of that table:

I’d be grateful for commentators to help me supplement this list and bring new words that should be included to my attention. I submit that, like the table of political positions, these lists of words should be updated frequently as well, because certain terms gain and fade in popularity fairly frequently. For example, “The Koch Brothers” are much more en vogue as a bogeyman than “Karl Rove” nowadays, though that was different just a few years ago.

In the next post (Part 3), I’ll go through each of the seven columns in more detail and list more examples of what political positions and biased words correspond with each. Finally, in Part 4, I’ll lay out how I take an article or story, and, using the criteria I’ve laid out here, go through the steps of ranking it as discussed in Part 1, which are 1) creating an initial placement of left, right, or neutral based on the topic of the article itself, and 2) measuring certain factors that exist within the article. I’ll also discuss step 3, which is accounting for context by counting and evaluating factors that exist outside of the article.

Leave a Reply

7 Comments on "Part 2 of 4: Why Measuring Political Bias is So Hard, and How We Can Do It Anyway: The Media Bias Chart Horizontal Axis"

newest oldest most voted
Notify of
Carolyn Bell

Your chart isn’t displaying on the page right now. I refer to it often, but just got on this page to look again and the chart is missing. The text of the page is there, but no chart.


hello im student in a government course, and a fellow student brought the media bias chart to me, and i would like to know how, or what variables you used to determine the data on the media bias chart,.

Bill Innis

It’s more than obvious what your leanings are. We give 3 networks (MSNBC, CNN and FOX) equal time and this is representative of their bias:
Yours too, based on seeing your chart!

Dominic Thephasdin

I can’t find the BBC on here.

Bob Balcom

Could you include It is a heavily read bi-weekly that appears to use good sources and is fact based but also seems to have a very left bias. Here is an example appearing on my FB feed today: