When I set out to do sentiment and toxicity analysis of Canada’s Question Period sessions, I held out hope that all the number crunching would reveal some hidden insight into Canadian politics. Perhaps something lurking a bit below the surface that only a bit of Python, a few APIs and TextBlob could reveal.
Sadly that hasn’t happened but the whole exercise has revealed a few things that surprised me.
One of my goals with this project was to try and determine what the most negative political party was, at least as viewed through the lens of Question Period. As part of my job at the CBC I have watched a lot of Question Period speeches and I was convinced I knew what the answer would be.
The Question Period Analyzer uses a Python library called TextBlob to measure the sentiment of Question Period speeches. TextBlob’s sentiment analysis function assigns a polarity value of -1 to 1 for each piece of text. The closer a text’s polarity is to -1, the more negative the text is considered. Similarly the closer a piece of text’s polarity value is to 1 the more positive it should be.
Here are a few samples of text and their corresponding TextBlob assigned sentiment polarity values:
- The flight was awful, I will never fly Air Stuart again. = -0.7
- The flight was amazing. I love Air Duncan. = +0.55
- That flight was really something. = +0.2
- That flight was something. = 0.0
- The pilot on the flight seemed out to lunch. = 0.0
- That was the best flight I have ever been on. Kudos to the crew and pilot. = 1.0
- That was the worst flight I have ever been on. The pilot was awful and the flight crew was mean. = – 0.77
TextBlob does a good job at classifying clearly negative statements like #1 and #7 and really positive ones like #2 and #6, but it (like most sentiment analyzers) struggles with examples that are less clear cut.
For most of us, if we heard someone say #3 and #4, at least in reference to a flight, we would assume it was a negative statement. Although if we heard someone say the same thing about a concert, we might consider it to be positive. Similarly when I read example #5, it is clearly negative. I understand that calling someone “out to lunch” in this context is negative, but TextBlob considers it to be a neutral statement.
Humans have the ability to contextualize speech in a way that computer models cannot and sentiment analysis models often struggle with sarcasm and turns of speech.
Sentiment analysis models are not perfect, but given a large body of data they can reveal patterns, as they are good at determining the clear cut positive and negative statements.
Given all that, how should we measure which party is the most negative?
The table below gives a sense of the overall sentiment picture of Question Period :
|Party||-||+||=||Total||Dif||Avg -||Avg +||Avg =|
Not all parties are given equal opportunities to speak during Question Period, so using the total amount of negative speeches to determine the most negative party doesn’t provide an accurate answer to which party is most negative. The average amount of negative speeches gives a better picture and reveals that all of the opposition parties fall around about 30% negative speeches. The Liberal MPs have the lowest amount of negative speeches at 13%.
Polarity scores are a continuum. So perhaps using an average amount of negative speeches isn’t the proper way to label the most negative party either. If a speech is measured as a polarity of -0.8 (which is really negative) or -0.1 (which might be only a smidge negative) they are both considered negative. Perhaps a better way to determine the most negative party is to look at the average sentiment score of all speeches by each party :
From this table you can see that no one party has an average polarity of below 0, so maybe no party is really negative as a whole. You can see from the table again the Liberals are the most positive party and the Conservatives, Bloc, and NDP are all in the same ballpark polarity wise with an average of around 0.05.
Opposition parties in parliament are working to hold governments to account, highlighting their failures and challenges, so perhaps it is to be expected that the overall sentiment of opposition parties is negative. With more time, I would like to take a look at Question Period sentiment from before the previous election when the Conservatives were in power and see if there is this same difference in sentiment between the governing and opposition parties.
After watching all those Question Period sessions, I honestly didn’t think there would be as much parity between the sentiment of the opposition parties. I was convinced that one party in particular would come out as the most negative.
Analyzing Question Period with TextBlob sentiment analysis has essentially revealed that all the opposition parties share roughly the same level of negativity, the Liberal party is the least negative and taken as a whole none of the parties are on average were negative or significantly positive for that matter.