We find Brexit-related Tweets from Twitter’s list of suspected Russian troll accounts

We have been collecting Twitter data on the UK-EU Brexit referendum since August 2015 and we currently have over 62 million Tweets. We collect data in several ways but the data set we discuss here is gathered using a variety of brexit related hashtags.

Twitter released a list of user names and id numbers of 2,752 Twitter accounts that the company identified as being related to Russian troll activity relating the US 2016 election.

We searched within our data set for signs of activity from these accounts. We found 3468 tweets from 419 users on this list. 400 of these tweets were on the date of the EU-referendum, 23rd June 2016 and were from 38 users. 432 were in the week of the referendum from 58 users.

We also took a look at the user defined location field in these tweets. Some have no location specified (127) but of those that do most (370) say they are from the USA (America, USA, US, state names, cities, towns). Only three of the users we find say they are from the UK.

Many of the tweets actually came after the EU-referendum polls closed — 78%. Only 776 tweets were before 23rd June 2016 10pm BST.

We found that 59% of the tweet text started with ‘rt’ and are therefore likely to be retweets. This leaves 41% that is not a direct retweet and may be original content.

We are confident that our findings are a conservative estimate of the activity of the 2752 accounts identified by Twitter as problematic. We have an automated process in place for deleting archived tweets when Twitter send out a request to do so. Typically when they suspend or delete accounts (as they have with this list) this is what Twitter do. We may, therefore, already have deleted relevant data from our archive. Also we only collect the allowed sample of free data from Twitter for our research. So although we have over 62 million tweets collected, we will not have captured all of the activity by these users.

Caution, however, needs to be exerted when trying to assess the influence of these users in relation to Brexit. First, the numbers, even if conservative, are relatively small. 3468 tweets out of 62 million in our data set came from 419 users on Twitter’s list of 2752 suspect accounts. Second most (78%) of the tweeting using Brexit-related hashtags that we find, took place after the date of the referendum.

On the other, hand, we should not assume that there was no influence either. We know, for example,  that some of these users had high numbers of followers. More nuanced data is needed on what happened to these 3468 tweets. How often were they retweeted for example?

It is important to remember that when searching for the users on the Twitter list, we are examining the behaviour of users identified as having attempted to disrupt the US election. It is perhaps, unsurprising that those tweets that appear in our Brexit-related data set post-date the referendum. Although these users are using Brexit-related hashtags, we cannot say whether they were primarily trying to influence Brexit itself or whether Brexit was simply an issue recognised as disruptive, or as amplifying issues around immigration and free trade blocs that resonated in US election debate.

The significance of these findings is that they provide the first hard evidence that users identified by Twitter as having Russian links and as seeking to influence the US election, were also actively tweeting on Brexit-related issues. To establish the extent to which the Brexit debate, or indeed the UK general election, were influenced by such users we need an equivalent list of users seeking to target these specific events and a complete data set. This would allow more nuanced analysis of the level, type and significance of activity and influence to be assessed. This is an issue of vital public interest but the much needed detail is in danger of being lost in the hype surrounding this issue.

This blog is written by Clare Llewellyn and Laura Cram of the Neuropolitics Research Lab from the University of Edinburgh. The project is part of the UK in a Changing Europe programme and is funded by the ESRC.

General Election 2017: a Twitter Analysis

Introduction

This work is produced by researchers at the Neuropolitics Research Lab, School of Social and Political Science and the School of Informatics at the University of Edinburgh. In this report we provide an analysis of the social media posts on the British general election 2017 over the month running up to the vote. We find that pro-Labour sentiment dominates the Twitter conversation around GE2017 and that there is also a disproportionate presence of the Scottish National Party (SNP), given the UK-wide nature of a Westminster election. Substantive issues have featured much less prominently and in a less sustained manner in the Twitter debate than pro and anti leader and political party posts. However, the issue of Brexit has provided a consistent backdrop to the GE2017 conversation and has rarely dropped out of the top three most popular hashtags in the last month. Brexit has been the issue of the GE2017 campaign, eclipsing even the NHS. We found the conversation in the GE2017 Twitter debate to be heavily influenced both by external events and by the top-down introduction of hashtags by broadcast media outlets, often associated with specific programmes and the mediatised political debates. Hashtags like these have a significant impact on the shape of the data collected from Twitter and might distort studies with short data-collection windows but are usually short-lived with little long term impact on the Twitter conversation. If the current polling is to be believed Jeremy Corbyn is unlikely to do as badly as was anticipated when the election was first called. Traditional media sources were slow to pick up on this change in public opinion whereas this trend could be seen early on in social media and throughout the month of May.

Data Collection

A set of 56 keywords related to the British general election in 2017 (GE2017) was used to collect tweets on the topic. The Twitter streaming API was used to retrieve tweets containing any of these keywords between the April 29, 2017 and June 4, 2017. The keywords consists of hashtags, accounts, and terms representing phrases on the elections (e.g. #GE2017, general elections), politicians involved in the elections (Theresa May, Corbyn, #jc4pm), and related topics (e.g. Brexit, NHS).

During the period of study, over 34 million posts were collected, where 9.6 millions are tweets and 25 million are retweets. Figure 1 illustrates the volume of tweets/retweets collected daily during the period of study.

Figure 1. Distribution of the collected tweets/retweets over the period of study

Analysis

As shown in Figure 1, the number of posts collected per day, there is clear evidence of the event driven nature of the Twittersphere. As well as observing a rise in the overall number of posts related to GE2017 over the month, peaks in the data can readily be associated with events such as the 5 May local elections and the Manchester bombing (with an associated drop in GE2017 posting as the campaigns paused in its aftermath). Interestingly, the official release of the Labour manifesto did not produce a significant spike in overall Twitter posts, perhaps due its earlier leaking. However, as can be seen in Figure 4, it did enjoy two mini boosts (becoming one of the top three most used hashtags on the day of its leak and again on the day of its official launch). On 18 May the Conservative manifesto launch coincided with the ITV leaders’ debate, producing a significant Twitter boost. Analysis of hashtags used, in Figure 4, indicates that the Conservative manifesto launch was indeed the larger contributor to this boost. As we are approaching the June 8 election date, Twitter traffic is evidently increasing. However, the increasingly event driven, and often top-down, shaping of the conversation is striking. Spikes in the Twitter data are closely linked with major media events and TV debates and, as we can see in Figure 4, are strongly influenced by the official hashtags promoted by the media companies.

Figure 2. Relative rate of retweeting to tweeting each day. The red line indicates the linear regression or trend line (positive gradient).

Figure 2 shows the average daily retweet versus tweet rate of the election related Twitter traffic. There appears to be a steadily increasing tendency to resend and reuse existing information as the election draws closer rather than to generate novel content.

Most popular hashtags over the collection period

The dominant role played by broadcast-driven and promoted hashtags is clear in Table 1. Of the Top 20 categories of most employed hashtags, during the month preceding GE2017, the number two slot is occupied by those related to television and radio shows and the number five slot by the hashtags associated with the TV debates. In Figure 4, we see that these hashtags generated significant spikes, but prove ephemeral when contrasted with issues like Brexit which persisted throughout the campaign. There are very few substantive issues discussed in a sustained manner in relation to GE2017. Most of the Twitter traffic is pro or anti-leaders or political parties. However, of the issues discussed, Brexit dominates. It is the forth most common hashtag employed (935,456) in the discussion of GE2017 and is employed more than twice as often as the next issue of significance, the NHS (420,092). The only other issues to feature in the top twenty hashtags over the full collection period are the potential further Scottish referendum (176,382) and the so-called dementia tax (154,007). The appearance of the Scottish independence question as the thirteenth most common hashtag employed in the discussion of GE2017 is particularly striking, given the UK wide nature of our data collection and the likely more localised interest in this issue in Scotland. This provides an indicator of the salience of this issue to those mobilised to tweet from Scotland.

Most striking in this data set is the overwhelming dominance of Labour tweeting. Tweets using hashtags do not necessarily indicate support but do highlight areas of discussion. With over one million tweets using Labour hashtags in our data set, Labour party coverage out-performs Conservative coverage by almost three times. There are no tweets employing anti-Labour hashtags in our top twenty most used hashtags collection. There are twice as many Labour (1,062,908) as Corbyn hashtags in the tweets (503,307). However, Corbyn hashtags, in position number six, still significantly outperform May hashtags (302,494) in position number nine. The pro-Labour momentum is boosted by the widespread use of Labour-promoted hashtags such as #forthemany, #forthemanynotthefew. Despite May’s attempt to focus her campaign on her own leadership, rather than actively campaigning under the Conservative party banner, Conservative hashtags (381,647), at position number eight, marginally outrank May hashtags in the collection. Once again, however, the disproportionate presence of the Scottish National Party (SNP) in slot number eleven (244,481), given that only Scottish voters can elect this party, was striking. Interestingly it is the SNP, not the First Minister Nicola Sturgeon, that appears in the most common hashtag list. The Liberal Democrats, and their leader Tim Farron, do not figure in the top twenty hashtags. UKIP occupies slot nineteen (141,011), although Paul Nuttall does not appear.

Table 1: Top 20 Hashtag Categories Used throughout the Election Debate (the top 100 hashtags were grouped by topic and the top 20 topics selected)

Top Hashtags Grouped as Number of Hashtags
#GE2017, #GE17, #GeneralElection, #GeneralElection2017, #Election2017 General Election 3,624,566
#BBCQT, #marr, #Peston, #r4today, #NewsNight, #BBCSP, #VictoriaLIVE, #BBCDP, #WomansHour, #TheOneShow TV/Radio 1,195,880
#VoteLabour, #Labour, #ImVotingLabour Labour 1,062,908
#Brexit Brexit 935,456
#BBCDebate, #BattleForNumber10, #ITVDebate, #LeadersDebate, #MayvCorbyn Hustings / Debates 844,514
#JC4PM, #Corbyn, #JeremyCorbyn Corbyn 503,307
#NHS, #VoteNHS, #SaveOurNHS NHS 420,092
#Tories, #Tory, #conservatives, #conservatives, #VoteConservative, #conservative Conservatives 381,647
#TheresaMay, #May May 302,494
#ForTheMany, #ForTheManyNotTheFew For the Many 271,251
#voteSNP, #SNP SNP 244,481
#ToryManifesto Tory Manifesto 214,973
#ScotRef, #indyref2, #Scotland Scottish referendum 176,382
#ToriesOut Tories Out 172,380
#RegistertoVote, #Vote, #WhyVote, #Register2Vote Register to vote 165,757
#Manchester, #Londonattacks, #LondonBridge, #London Terrorist Attacks 154,497
#DementiaTax, #Socialcare Social Care 154,007
#LabourManifesto Labour Manifesto 149,731
#UKIP UKIP 141,011
#BBCElection, #BBC BBC 124,755

 

Figure 3: Percentage Share of Top 20 Hashtags Used in the Election Debate

Daily peaks in popular hashtag use

We also looked at frequency peaks of popular hashtags broken down by day over the collection period. This allows us to spot issues that may not have been tweeted about most overall but which also motivated people to comment in large numbers. The hashtags shown here appear in the top three on at least one of the days during the campaign. The event driven nature of Twitter is particularly obvious here. We can see that the largest peaks in the data come from debate-type events covered by traditional media, such as #BBCDebate, #BattleForNumber10 and #BBCQT. Similarly, other TV programmes also cause spikes in the data with #peston and #marr being particularly noticeable. We can also see that the Tory manifesto caused more of an impact that the Labour manifesto, possibly due to the leaking of the Labour manifesto, where we see two peaks of influence rather than one. We see the appearance of certain policy issues such as the so-called Dementia Tax and Fox hunting and in particular we see the sustained appearance of Brexit throughout the campaign. There are some issues which are notable by their absence as peak issues, for example the NHS and the economy. Also apparent are events that have taken place during the election campaign. Here we can see peaks of discussion of both the #NHSCyberAttack and the #LondonAttacks.

Figure 4. Hashtags that peaked on a given day on Twitter during the study period

Most mentioned and retweeted accounts over the collection period

The story of Labour dominance in the Twittersphere continues when we examine the most retweeted accounts and the accounts that get the most mentions by others. Jeremy Corbyn tops both of these lists and, though Theresa May (654,417) is the second most mentioned account, she is mentioned only half as often as Corbyn (1,367,392). The difference between the two main party accounts @uklabour (323,027) and @conservatives (307,550) is much less. Both leaders are mentioned much more often than their respective parties, perhaps confirming the presidential tenor of the campaign. Striking once again is the disproportionate presence of the SNP (145,937) and this time also their leader Nicola Sturgeon (116,360) at positions six and seven in this UK wide debate. The SNP outperforms the Liberal Democrats (93,473) and their leader Tim Farron (69,009). Scottish Conservative and Unionist Party leader, Ruth Davidson (69,334), is also mentioned marginally more often than Tim Farron. UKIP (80,855) is the tenth most mentioned account, but again Paul Nuttal does not feature. The presence of the official media accounts, and also of polling agencies like Yougov, is again a prominent feature of the most mentioned accounts in relation to the GE2017 Twitter conversation. The most retweeted accounts were again heavily pro-Labour. We also saw, as we might expect, a strong presence of the professional media and of campaign bodies here. The generation of popular memes by @laboureoin also proved to be a very effective strategy for encouraging retweets carrying a socialist message.

Table 2: Top Retweeted and Top Mentioned Twitter Accounts

Top retweeted accounts Top mentioned accounts
Account Count Account Count
@jeremycorbyn 821,499 @jeremycorbyn 1,367,392
@laboureoin 426,912 @theresa_may 654,417
@nhsmillion 342,440 @uklabour 323,027
@rachael_swindon 290,587 @conservatives 307,550
@owenjones84 224,605 @bbcnews 154,898
@socialistvoice 189,830 @thesnp 145,937
@jeremycorbyn4pm 186,269 @nicolasturgeon 116,360
@davidjo52951945 178,768 @skynews 97,287
@uklabour 171,577 @libdems 93,473
@davidschneider 164,457 @ukip 80,885
@jamesmelville 162,626 @thecanarysays 80,176
@chunkymark 161,681 @bbclaurak 77,324
@toryfibs 146,873 @ruthdavidsonmsp 69,334
@independent 139,540 @timfarron 69,009
@el4jc 119,623 @lbc 65,571
@britainelects 115,655 @yougov 62,741
@paulmasonnews 108,302 @borisjohnson 61,292
@imajsaclaimant 108,130 @guardian 58,880
@aaronbastani 101,101 @afneil 52,216
@peterstefanovi2 99,875 @johnmcdonnellmp 49,271

Daily peaks in most mentioned accounts

The top mentioned accounts were selected in the same way as the most frequent hashtags. All of these accounts appear in the top three most mentioned on at least one of the days during the campaign. Here we can see that @jeremycorbyn has been mentioned much more frequently than @theresamay throughout the campaign. We see three political parties mentioned; @conservatives, @uklabour and surprisingly again @theSNP. We can see how events shape peaks with the single peak in mentions for the Scottish Conservative and Unionist Party leader @ruthdavidsonmsp reflecting her announcement of a u-turn on prescription charges in Scotland. We can also see the influence of journalists here with @krishgm, @bbclaurak and @afneil.

Figure 5. Mentions that peaked on a given day on Twitter during the study period

Most mentioned topics by and linked to politicians

We checked the most frequent terms used by key politicians themselves and we compare these with the most frequent terms used by others when mentioning these politicians. We can consider the terms used by the politicians as their attempts to influence debate and to set the agenda. This cloud reflects what the politicians want to talk about. The terms in the tweets from others mentioning the politicians can be considered to be the topics that Twitter users are trying to direct towards the politicians, these clouds reflect the agenda that Twitter users are associating with that politician.

In the visualisations you can see the terms side by side for each politician. There are several things that can be observed:

  • Jeremy Corbyn is the only politician that directly challenges another politician – we can see this through the use of @theresamay. The other politicians do mention other leaders but they do not use specific @ mentions to interact with them;
  • Theresa May and Tim Farron do not mention their parties often;
  • Ruth Davidson who is often associated with distancing herself from the Scottish Tory brand actually tweets about her party quite often;
  • Whereas Theresa May often tweets about Brexit this is not echoed in the tweets of those that mention her;
  • Brexit does occur in the tweets mentioning Tim Farron, Nicola Sturgeon and Ruth Davidson;
  • None of the Scottish leaders tweet heavily about independence but those tweeting about Kezia Dugdale and Ruth Davidson associate both of them heavily with this debate;
  • Nicola Sturgeon, heavily associated in the traditional media with Scottish independence, is not associated with this by those mentioning her or seeking to interact with her on Twitter;
  • The debates feature heavily in the tweets mentioning politicians except for Kezia Dugdale;
  • Jeremy Corbyn and Nicola Sturgeon make most sophisticated use of Twitter devices such as hashtags and @ mentions.

 

Account own tweets Tweets mentioning the account
Jeremy Corbyn

Jeremy Corbyn

Theresa May

Theresa May

Tim Farron

Tim Farron

Nicola Sturgeon

Nicola Sturgeon

Ruth Davidson

Ruth Davidson

Kezia Dugdale

Kezia Dugdale

Figure 6: Most frequent terms employed by key politicians v the most frequent terms used by others when mentioning these politicians.

Discussion and Conclusions

In this study, we applied a quantitative analysis to the Twitter posts on the British general elections 2017 during the month period preceding the election. Our analysis included a set of around 35 million tweets on the topic. Here we present a preliminary exploratory analysis of this data set. Twitter analysis has strengths and weaknesses. Twitter users are not representative of the wider public – they are self-selected users not those chosen on the basis of careful sampling by opinion pollsters. Twitter users tend to be highly motivated (with an axe to grind), younger than average (though not exclusively young) [1] and are likely more often men [2] when engaged in political debate. So any insights are partial. That said, Twitter can be a reflection of spontaneous, motivated behaviour. Analysing Twitter narratives helps us to see where those highly motivated individuals position themselves in relation to the debate, what appears to provoke peaks in motivated activity and also what the overall trends are in these vocal and active publics. It also helps us to explore who sets agendas and shapes conversations in the Twittersphere and how effective or ephemeral these narratives are.

Events played a key role is shaping the Twitter conversation. These events can take many forms, election-specific events such as the debates, media events such as television and radio programs and physical events such as the NHS cyber attack and terrorist attacks. Twitter does not exist in a vacuum and the conversation that occurs there is often prompted by external events.

Researchers at the Centre for Research in Communication and Culture (CRCC) at Loughborough University have studied media coverage of the election campaigns over two weeks, 18th – 31st May. They analysed discussion in television news and print media. They observed that in the first week, starting on the 18th May, the Conservatives and Theresa May receive more coverage than Labour and Jeremy Corbyn. In contrast, we find that the social media discussion during this time was focused on Labour and Corbyn. The second week saw a rise in the traditional media coverage of the Labour party which was slightly more than the Conservatives. This rise was for the most part driven by increased traditional media coverage of Jeremy Corbyn bringing this more in line with the social media data. The Loughborough data also shows a high prevalence of Scottish MSP’s with Nicola Sturgeon and Ruth Davidson both appearing in the top 10. This is mirrored in our social media data set.

In the last week of May the top five most prominent issues seen in television and print media were the electoral process, Brexit, health care, taxation and the economy/business. We can see this echoed to some extent in the social media data, with the manifestos, Brexit, and the NHS. We do not see the appearance of taxation, the economy or business issues.

Our data set shows an overwhelming dominance of pro-Labour tweeting. With over 1 million tweets using Labour hashtags, Labour party coverage out-performs pro-Conservative coverage by almost three times. There is a disproportionate presence of the SNP in this social media set and given that only Scottish voters can elect this party this was particularly striking. These three observations were also noted in a week long study (1-7th May) conducted by researchers at the Oxford University’s Internet Institute [4].

It is important not to exaggerate the novelty of new social media. New media to some extent appears to be simply an extension of old media and we see in our data set how broadcast media often generates additional coverage by effectively reporting on itself. Key bursts in Twitter throughout the GE2017 campaign came from people using hashtags associated with TV debates and programmes like Question Time. These hashtags are created by the programme makers and die off almost immediately. This is not surprising as they are instantaneous or throw-away hashtags associated with a specific programme. It would be difficult to claim that these had any significant role in setting a political agenda or in shaping debates, rather they act as a means of tracking who was watching and engaging with the particular media generated events.

Twitter is of course not representative of the voting public as a whole, and therefore not necessarily a clear reflection of “the many, not the few”. However, whilst Twitter cannot be used to predict elections and the overwhelming support we see for Labour and Jeremy Corbyn may not be fully reflected in the ballot boxes, it is a useful tool in giving us the mood of those who are motivated enough to comment in social media. Tweeters are typically highly motivated and perhaps those who initially see themselves as the underdogs in the debate, excluded from mainstream coverage. This has been apparent in a number of recent campaigns. The YES campaign, though ultimately unsuccessful, dominated social media in the Scottish independence referendum. Leave groups did the same in the 2016 Brexit campaign and Trump’s dominance in social media transformed US election coverage, with both these campaigns ultimately triumphing at the polls. As the Loughborough study shows, Corbyn’s campaign did not initially enjoy the access to the traditional media that May was afforded [3]. This may explain the surge in social media activity which subsequently developed a life of its own and has ultimately had to be acknowledged by the mainstream media. This also fits with the high presence of the SNP in our data set, with the Scottish debate marginalised at the UK level. If the current polling is to be believed Jeremy Corbyn is unlikely to do as badly as was anticipated when the election was first called. Traditional media sources were slow to pick up on this change in public opinion whereas this trend could be seen early on in social media and throughout the month of May.

References

[1]http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0115545

[2]http://www.bbc.co.uk/news/blogs-trending-32137886

[3]http://blog.lboro.ac.uk/crcc/general-election/media-coverage-of-the-2017-general-election-campaign-report-3/?utm_source=Twitter&utm_campaign=ge2017&utm_content=coverage#coverage

[4]http://www.ox.ac.uk/news/2017-05-31-labour-dominating-twitter-conversation-uk-election-campaign-says-study

This work was produced by:

Laura Cram, Clare Llewellyn, Robin Hill Neuropolitics Research Lab
School of Political and Social Science University of Edinburgh {laura.cram, c.a.llewellyn, r.l.hill}@ed.ac.uk

Walid Magdy
Institute for Language, Cognition & Computation School of Informatics
University of Edinburgh wmagdy@inf.ed.ac.uk

Everything is Up for Grabs in Brexit Process

Just as with the other existing models – Norway, Switzerland, Canada, Greenland – the new British relationship with the EU will have to be purpose built. If Scotland is to maximise its advantage in that process, says Laura Cram, it will need to be out of the blocks before the Article 50 starting pistol is fired. 

No-one knows precisely when the Article 50 process, formally notifying the intention of the UK to leave the EU, will be triggered. But being prepared is essential not just to respond swiftly but to shape the direction the subsequent negotiations take. We know only that the clock is ticking but not when the alarm is set to go off. The challenge for Michael Russell, as the Scottish Government’s new Brexit Minister, will be to re-shape the boundaries of the possible in Scottish-UK-EU relations before the Article 50 process begins.

As every returning Olympian will testify, responding to the sound of the starter’s gun comes at the end of an intense and lengthy process of physical and psychological preparation, scenario planning, sizing up the opposition and understanding the nature of the terrain. Not everything is within an athlete’s control, but a successful Olympian will have prepared for what they can and will be aware of where the uncontrollable might surface. They will have thought through what action to take and how to both create and to seize opportunities. Carefully constructed teams of experts support their efforts. Control of the race is key and preparation is key to control. To fail to prepare, in high stake games, is to prepare to fail.

Writing in the 1970s Ernst Haas, one of the earliest scholars of European integration, described the scenario in which a very large number of organisational actors find themselves with confused and clashing perceptions of the optimal solutions to the problems they face as a “turbulent policy field”. Actors pursue mutually incompatible objectives and are unsure of the trade-offs between these objectives. At the same time these actors need to cooperate with each other as they are tied into a network of interdependencies. Haas describes this scenario as being like a “giant simultaneous chess match over which the judges have lost control”. Confusion and uncertainty are dominant in a turbulent decision field and this confusion can be sub-national, national, supra-national and global all at the same time.

The Brexit process is just such a turbulent field. Beyond the formalities of the legal process, laid out but as yet untested in Article 50, we have no precedent to draw on. It is even controversial at EU level which EU institution will lead the negotiations. Within the UK, different territorial perspectives prevail. Uncertainty creates a suboptimal decision environment. Uncertainty produces a tendency to cling to the status quo, and when there is no status quo to cling to – a tendency towards delay and indecision. This is a tendency to be resisted at all costs.

Talk of existing models – Norway, Swiss, Canadian, Greenland – as templates for Brexit are examples of such status quo bias. Until these models were developed none of them existed. There is no UK model or rUK model for non-member state relations with the EU only because it has not yet been imagined or constructed. None of the existing templates were created for departing member states or for non-departing territories within departing member states. The flip side of a turbulent policy field, according to Haas, is that “everything is up for grabs”. A turbulent field is also an optimal environment for creative thinking and bold policy entrepreneurship. The trick is to be the one shaping the options and framing the ensuing debate.

Policy studies consistently show that the most effective moment for agenda-setting is when an idea is not yet a spark in policy-makers’ eyes. The persuasive power of solid policy analysis and clear scenario planning has the potential to frame the debates that ensue. The earlier that clear alternative scenarios are developed, the more time there is to normalize these ideas with other decision-makers within the UK and across the EU. Softening up the decision environment by exposing new scenarios to decision partners, until they don’t seem so new, radical or threatening is key. Pro-actively framing the debate and shaping the negotiations, in a time-sensitive environment, is crucial. Clear scenarios for what Scotland wants in relation to EU and UK membership in the case of a hard or soft UK-wide Brexit, or in the case of an independent relationship with the EU, need to be developed, disseminated and normalized in the debate without delay.

It is the UK Government that will fire the starter’s gun at what is considered to be an optimal time for them. That may not coincide with Scotland’s preference or interests. Jumping the gun will be an Olympian effort but the Scottish Government is no stranger to this.

This article was written by Laura Cram of the Neuropolitics Research Lab from the University of Edinburgh. It is re-blogged from the the Centre for Constitutional Change’s blog. It was originally co-published with the Herald

Understanding What the European Union Means to You: Neuropolitics, Behaviour and Identity

euro+jack_straightOn Europe Day, May 9th 2016, the Neuropolitics Research Lab is popping up at the National Library of Scotland. University of Edinburgh researchers introduce a unique approach to understanding what the European Union means to you. Join our interactive drop-in workshops and lightning talks, exploring what our brains, behaviour and even our activities on social media reveal about our attitudes to the EU. View the #myimageoftheEU gallery throughout the day and even add your own image. In the evening, sign up for the expert Q&A session on What the European Means in Different Parts of the UK.

The daytime event is unticketed so feel free to pop in throughout the day.

Timetable:

10.00 – 12.00 Brain, Behaviour and Attitudes to The European Union

A drop in session for demonstrations of what our brains, behaviour and social media activities tell us about our attitudes to the European Union this includes hands-on workshops. View our #myimageoftheEU gallery and watch or join in our interactive sessions to learn more about what eye-tracking, face-emotion coding, social media analysis and brain imaging can tell us about our attitudes to the EU.

12.00 – 14.00 What Does the EU Really Mean To You?

Catch one of our lightning talks on how the public imagines the European Union and what neuropolitics can tell us about this. A series of different talks, each lasting 10 minutes, will start every 20 minutes. You are welcome to drop in for a specific talk or stay for as many as you can. In addition view our #myimageoftheEU gallery, we asked the public to tweet how they see the EU see a gallery of the best and most though provoking images and add your own.

14.00 – 16.00 Closed session for schools visit and workshop

18.00 – 19.30 ‘What Does the EU Mean’ – Panel discussion and a Q&A

The discussion is to be chaired by Kenneth Macdonald, BBC Scotland Science Correspondent, the speakers include Professor Anand Menon, King College London, Director UK in a Changing EU programme; Professor Micheal Keating, University of Aberdeen, Director Centre for Constitutional Change and Senior Fellow, UK in a Changing EU programme; Dr Huw Pritchard member of the Wales Governance Centre and Professor Laura Cram, Director NRlabs Neuropolitics Research, University of Edinburgh, and Senior Fellow, UK in a Changing EU programme.

Tickets for the evening event can be obtained through eventbrite:

https://www.eventbrite.com/e/what-does-the-eu-mean-tickets-24624986999

Your Image of the EU: Launch of #myimageoftheEU

The people of the UK are soon to vote in a referendum on the UK’s membership of the EU. What are your opinions on the EU? What does it make you think of? What do you see in your daily life that evokes thoughts of the EU?

We are inviting you to get involved and be part of the debate! Tweet your images, cartoons, videos and comments that capture your image of the EU. Examples might be EU funding signs, or flags in different spots, but the only limit is your imagination.

We will use your images to create a Twitter gallery that will be accessible to the public, academics, policymakers and political elites. We would particularly welcome images from those who don’t feel a part of the central debate. Get your voices heard. We will be using some of these images in an art installation on Europe Day, 9 May 2016. Prizes will be awarded for the best images.

To participate, tweet images stating where you spotted them, or where you are based, and how the image makes you feel about the EU to @myimageoftheEU, using the hashtag #myimageoftheEU, or email them to us. See the tweets featured here for examples of what to do or visit our Twitter wall.

Our #ImagineEurope project is part of the Economic and Social Research Council’s The UK in a Changing Europe programme. Look out for our regular updates as the project tracks developments in the debate on the UK’s membership of the EU and follow us on Twitter @myimageoftheEU for more information on this and other projects.

This article was originally published on the ImagineEurope Storify.

Exploring Bias: Comparing Approaches for Collecting Twitter Data

In our Imagine Europe project, we are tracking the UK’s EU referendum debate to explore the various ways in which the public imagines the European Union. We are using Twitter to map trends in response to emerging events. This analysis allows us to gain a more nuanced understanding of those who are motivated to comment on UK-EU-related topics. See our Twitter demo for interactive visualisations of the data.

We have collected Twitter data on the referendum debate for the past five months. We are using three methods for collecting data from Twitter: 1) Using hashtags chosen by an expert panel as search queries; 2) Collecting a random sample without specified search terms and extracting referendum-appropriate data automatically; 3) Collecting from the three official campaign groups @vote_leave, @LeaveEUOfficial and @StrongerIn.

Each method of collection influences what data will be collected and therefore each data set has certain biases. The hashtag and random stream sample sets are heavily influenced by the terms used for data collection. Those terms differ greatly when automatically extracted (the random stream set) or chosen by experts (the hashtag set).

The expert method is designed to follow a wider variety of terms that the experts expect will become discussion topics over the longer-term referendum debate, whereas the automatic method extracts data using terms which are commonly associated with known referendum-specific terms. Examining the three different sets allows us to contrast what is being collected and gives us the ability to have a broader understanding of public and elite opinion. In particular, we are examining how topics differ between these data sets and how they influence each other.

The hashtag set is the largest by a considerable amount. During the five-month period, the set collected using hashtags contained 5,556,027 tweets. The set extracted from the random stream has 8,777 tweets and the official campaigns 2,606 tweets. To determine how relevant the data collection is to the debate, we extracted 100 tweets from each set and asked three annotators to consider the relevance of each tweet in two ways: 1) whether it is directly relevant to the UK-EU referendum debate, and 2) whether it is about a topic that would likely influence voter opinion.

We found that the data from the official campaign groups and data automatically extracted from the random stream are more relevant to the topic than the data gathered using hashtags. The hashtag set has a low relevance score for ‘directly relevant to the referendum debate’ but this rises significantly when the topics that will influence the debate are considered.

This was as we expected. The differences can be explained as follows. The official campaign set contains the information from the campaign groups which are publishing tweets in order to influence the debate. This gives us a small, very specific, very opinion-driven set. The random stream set gives a set of data from the wider public, but only tweets that contain terms that are closely related to the debate, therefore providing a very topic-specific set. The hashtag-gathered set is a much larger set, collected using a wider variety of terms. It contains more non-relevant information but also covers the topics likely to influence voters not identified in the other sets.

We did find that much more data was collected by the hashtag method in early September. On further inspection this data relates to refugees and migrants. This shows that the campaign groups are not talking about the refugee crisis or related migration issues. It is not being directly related to the UK-EU referendum debate, but instead it is being widely discussed.

Analysing the frequency of commonly used hashtags gives an indication of topics discussed in each of the datasets. Much of the discussion in the tweets from the official campaign and the random stream data are directly related to the UK-EU referendum. This is echoed by the hashtags #brexit, #leaveeu, #voteleave and #euref being the top four most frequent in both collections.

Hashtags with a pro-Leave sentiment appear more frequently in all three of our data sets. We do not see any pro-Remain hashtags appearing in the hashtag-gathered set, and only #strongerin and #remainineu in the random stream set. We have a very small number of pro-Remain hashtags in the official campaign data.

Since we are collecting from the three campaign groups and only one is pro-Remain, we would expect a lower level of pro-Remain hashtags in the official set – but not as low as we are seeing. This suggests that either pro-Remain supporters don’t use hashtags, use them in unexpected ways or there is a strong pro-Leave sentiment on Twitter.

We also see another phenomenon within the data – where hashtags are used to draw attention to specific themes. Within the official stream, certain hashtags have been heavily used by the two pro-Leave campaigns. For example, @LeaveEUOfficial launched #theknoweu, #justsaying, #fudgeoff and #twibbon and @vote_leave launched #wrongthenwrongnow and #theinvisableman. We can see that #twibbon also appears in the random stream data set and therefore has cross-pollinated and is being discussed by the wider public. The @StrongerIn campaign does not seem to be using hashtags to the same extent and rarely uses any beyond #strongerin. It is possible that the lack of use of hashtags by the @StrongerIn campaign means that their supporters are not using hashtags as well. This is something we will need to investigate further.

In the hashtag-gathered data, many of the top hashtags indicate a focus on the topic of refugees (#refugeeswelcome, #refugee, #refugeecrisis) and in discussing particular countries (#uk, #usa, #syria, #germany). In the random stream data, we also see a discussion of the referendum-specific terms #brexit and #leaveeu, but very little occurrence of the #strongerin hashtag.

Our #ImagineEurope project is part of the Economic and Social Research Council’s The UK in a Changing Europe programme. Look out for our regular updates as the project tracks developments in the debate on the UK’s membership of the EU and follow us on Twitter @myimageoftheEU for more information on this and other projects.

This article was originally published on the ImagineEurope Storify.

UK-EU Twitter Sentiment Analysis: An analysis of the sentiment in the twittersphere towards the UK leaving or remaining in the EU

In this article, we look at the sentiment of hashtags towards the UK’s EU membership in our dataset. We look at sentiment in general and, in particular, the difficulty in measuring it. In a previous article we discussed in detail the hashtags used by groups campaigning for the UK to either remain in or to leave the European Union (@StrongerIn,@vote_leave, @LeaveEUOfficial).

One of the aspects we want to address in our work is the identification of sentiment in our dataset. We want to ask questions such as: Are people in the UK positive or negative about the EU? and Can we quantify this opinion by topic and/or geography?

Identifying the target of sentiment expressed in a tweet or a piece of text in general is a hard task. There is software available that measures the strength and direction of sentiment in a segment of text, but it is harder to identify what that sentiment is expressed towards. This is what we call the target of the sentiment and it is difficult to automatically extract this target.

For example, we have the tweet below. We can clearly see that this tweet is pro-Remain. If we use a general sentiment analysis tool, the text from this tweet would have a positive sentiment score based on the language used. We can see that there are two targets in this sentence which both have positive sentiment – Sir John Major who ‘is clear’ and Britain which ‘is stronger in Europe’.

After the identification of sentiment polarity, strength and target, we would aim to combine this information to see how these relate to more general topics of discussion and to the relationship between the UK and the EU. In the tweet above, even though a pro-Remain sentiment is expressed, it is difficult to automatically relate the positive sentiment towards the UK and a pro-Remain point of view.

With this tweet, we can use further context and qualifying information, such as the hashtag – but not all tweets contain this information. Of course, not all EU-related tweets have a stance on the future of the UK-EU relationship. We are currently looking into ways of addressing this issue in more detail and will report our results at a later date.

To present an initial overview of sentiment in the data collected, we are calculating the percentage of hashtags in the data which express a polarity of either pro-Leave or pro-Remain.

We launched the Neuropolitics Research Labs website on 1 December 2015, where you can find more information on our work.


Neuropolitics Research

The Neuropolitics Research Lab produces transdisciplinary research, utilising developments in the cognitive neurosciences, to shed new light on political attitudes, identities and decision-behaviours. Our aim is to test the utility of methods more typically associated with neuroscience, informatics and cognitive psychology in helping us to understand more about political attitudes and behaviours.


As part of this site, we have launched the interactive #ImagineEurope Twitter Demonstrator. This provides access to visualisations of the Twitter data we have collected since 7 August 2015.


Twitter Analysis

Social media is being used to monitor ongoing shifts in public imaginings of the European Union at this critical time. Here Twitter is used to track current trends using advanced Twitter analytics, hashtag tracking, sentiment scoring (indicating the rising and falling emotional content of tweets) and trend analysis in response to emerging events.


On the #ImagineEurope Twitter demo there is a sentiment dial – a map showing locations extracted from Tweets and a wordle of commonly used hashtags. This is an overview of the entire dataset. Further pages can be linked to that show sentiment, locations and hashtag wordles by day.

The dataset presented in our interactive Twitter demonstrator is gathered using a set of hashtags related to the upcoming referendum on the UK’s EU membership. We discussed how we collect the data in a previous article. Since then, we have added new hashtags to reflect the ongoing discussion and those used by the Leave and Remain campaigns. These are: #migrant, #refugee,#strongerin, #leadnotleave, #voteremain, #britainout, #leaveeu, #voteleave,#beleave,#loveeuropeleaveeu.

We have initially adopted the approach of counting the hashtags in our dataset which clearly have a pro-Leave or pro-Remain bias. The Remain hashtags used in the sentiment calculation are:#yes2eu, #yestoeu, #betteroffin, #votein, #ukineu, #bremain, #strongerin,#leadnotleave,#voteremain. The Leave hashtags are: #brexit, #no2eu, #notoeu, #betteroffout, #voteout,#britainout, #leaveeu, #loveeuropeleaveeu, #voteleave, #beleave.

We can see from the dial that most of the data in our sets contains Leave hashtags. We can break this down to show the counts per hashtag.

This breakdown shows that the #brexit hashtag is the most used. This hashtag is not always used to signal support of #brexit. We have discussed in a previous article how in fact this hashtag is used by both pro-Leave and pro-Remain groups. #Brexit appears to be used as a label of the referendum discussion in general rather than an indicator of sentiment direction.

To move beyond the current solution, we are working with The TAG Research Laboratory at the University of Sussex. The Laboratory is involved in research into social media analysis. We are working with them to adapt their software to our data to find a solution to this tricky sentiment problem.


TAG Research Laboratory

In Summer 2010, Jeremy Reffin, and I co-founded the Text Analytics Group (TAG) Research Laboratory. We are currently a team…


Our #ImagineEurope project is part of the Economic and Social Research Council’s The UK in a Changing Europe programme. Look out for our regular updates as the project tracks developments in the debate on the UK’s membership of the EU and follow us on Twitter @myimageoftheEU for more information on this and other projects.

This article was originally published on the ImagineEurope Storify.

Tweet Beginnings – An analysis of how the EU Leave and Remain camps are using Twitter

Social media is used by campaign groups to get their message across during the lead up to both elections and referendums.
Social media is used by campaign groups to get their message across during the lead up to both elections and referendums. Bond et al. (2012) showed in a study of 61 million users in the 2010 US congressional elections how facebook posts ‘directly influenced political self-expression’.


A 61-Million-Person Experiment in Social Influence and Political Mobilisation

Human behaviour is thought to spread through face-to-face social networks, but it is difficult to identify social influence effects in observational studies, and it is unknown whether online social networks operate in the same way. Here we report results from a randomised controlled trial of political mobilisation messages delivered to 61 million Facebook users during the 2010 US Congressional election.


Twitter in particular has been described as a tool that campaigners and politicians can use to shape media and public perception (Conway et al 2015). Forums such as Twitter allow voters to participate in debate. It allows discussion, idea challenging and information exchange and allows the public to become engaged in the process.


The Rise of Twitter in the Political Campaign: Searching for Intermedia Agenda-Setting Effects in the Presidential Primary

Questions exist over the extent to which social media content may bypass, follow, or attract the attention of traditional media. This study sheds light on such dynamics by examining intermedia agenda-setting effects among the Twitter feeds of the 2012 presidential primary candidates, Twitter feeds of the Republican and Democratic parties, and articles published in the nation’s top newspapers. Daily issue frequencies within media were analyzed using time series analysis. A symbiotic relationship was found between agendas in Twitter posts and traditional news, with varying levels of intensity and differential time lags by issue. While traditional media follow candidates on certain topics, on others they are able to predict the political agenda on Twitter.


In the referendum on Scottish independence several studies were conducted including the one, published in the Guardian, that used Twitter to analysis the level of discussion on different topics associated with the referendum. This highlighted the issues that were important to the (Twitter) public and how the campaign groups interacted with them on those topics. In this study they note how the campaigns were out of touch with the breath of discussion taking place.


Scottish Independence: Which Issues Have Led the Twitter Debate in 2014?

A study by researchers at the University of Glasgow has analysed Tweets that use the #indyref hashtag. Over time, the currency emerges as the most mentioned topic of discussion but oil, the EU and taxation have also been frequent matters of debate. Topical events and policy announcements drive discussion at specific moments, leading to large spikes.


It must be remembered that observations of what happens in social media only represents the discussion that is taking place in social media. This can be a very bias sample – this is discussed further by Quinlan et al (2015). Great care must be taken to not extrapolate this directly to society at large.


Online Discussion and the 2014 Scottish Independence Referendum: Flaming Keyboards or Forums for Deliberation?

Referendums often fail to live up to a deliberative standard, with many characterised by low levels of knowledge, disinterest and misinformation, negativity and a focus on extraneous issues to which voters are voting. Social media offers new avenues for referendums to incorporate a greater deliberative dimension.


In a previous storify we looked at the hashtags from the EU leave and remain camps. This time we have are looking at a more extensive analysis of the tweets from the official Twitter accounts of the campaigns @vote_leave, @StrongerIn and @LeaveEUOfficial. This is intended to be an initial exploration of their use of Twitter and a comparison between the accounts.

We can see from the total number of tweets graph below that there is a large difference in the number of tweets sent by each group. With both leave groups tweeting more than the @StongerIn campaign. @StrongerIn started tweeting on the 18th September 2015, it has 6,073 followers and has tweeted 112 times. @LeaveEUOfficial started on 11th July 2015, has 31.6K followers and has tweeted 990 times, @vote_leave started on 8th October 2015, has 7,402 followers and has tweeted 629 times. These figures include the times they retweet tweets from others but not the times they are retweeted themselves.

We can see that @LeaveEUOfficial is the most established account and has tweeted the most and has the most followers. The tweeting activity from this account is very consistent with tweets most days – usually between 8-20 tweets per day. Very occasionally they will tweet a lot more for example they tweeted 38 times on 18th November. @vote_leave are far less established but have generally tweeted more per day than the other two accounts. They tend to cluster tweet – so they will tweet a lot for a few days then go quiet. Their biggest tweet day was the 12th October with 42 tweets. @StrongerIn tweets consistently but at a lower volume – usually between 1-4 tweets a day. Their highest tweet day was 19th November with 9 tweets.

It is likely that the campaigns aim to reach people through social media. This means as many people as possible reading the tweets. Tweets are either read on a time line by followers of an account, read by the followers of others when they are retweeted or read when users search twitter either using search words or hashtags. To improve your reach therefore the aim is to recruit more followers or to be retweeted as widely as possible.

We know that the @LeaveOfficialEU has many more followers than either of the other accounts so has a larger reach than the other two accounts. With this in mind we look at the approaches and strategies taken by the three accounts. When the groups tweet they differ in how they structure their content and what extra media they include. Tweets tend to get retweeted if they contain images, they are more discoverable (through searching) if they contain hashtags (#) and they can be targeted to certain individuals (generally with the hope of a retweet) if they contain a mention (@).

We can, to a certain extent, measure the success of a tweet by the number of times it is retweeted and the number of times it is favourited. We can see @LeaveEUOfficial on average per tweet gets both more retweets and favourites. They are followed in quantity of both retweets and favourites by @StrongerIn then @vote_leave. There is a smaller difference in number of favourites than retweets between @vote_leave and @StrongerIn. @StrongerIn tend to use more mentions and images per tweet than the other accounts – this may be increasing the success of the tweets and making up, to some extent, for their lower number of followers. As discussed in a previous post @StrongerIn uses less hashtags than the other accounts – it is possible that the later start date, and the lower volume of tweets may account for the smaller number of followers.

We can look at the types of users the accounts are trying to reach by analysing those that they mention in their tweets. In the graphs below we see the most frequent 10 (or less if not available) user accounts mentioned. We can see that the @StrongerIn campaign often mentions members of its own organisation such as Will Straw, Lucy Thomas, and Karren Brady or known remain supporters such as Patrick McFadden – who are presumably more likely to be positive towards their message and retweet. They also mention Megan Dunn President of the National Union of Student – this may be a strategy to reach students and therefore court the youth vote. They also reach out to the vote_leave campaign through mentions of Dominic Cummings (@odysseanproject) and @vote_leave. This could be trying to reach those supportive of the leave campaigns and challenge their beliefs.

You can see in the graph above that @LeaveEUOfficial is reaching out to main stream media by mentioning news outlets – the Telegraph, Daily Express, Guardian and New Statesman. These accounts have a lot of followers so a retweet from one of these accounts would have a large reach. They also mention @TheKnow_EU which was an account for@LeaveEUOfficial – presumably to make sure followers switched over from the old account. They also mention supporters of their view such as Andy Wigmore, Robert Kimbell (@RedHotSquirrel) and Nigel Farage in a manner similar to @StrongerIn.

@vote_leave reach out / challenge the @StrongerIn campaign through mentions of Lucy Thomas, StrongerIn and @euromove – a grass roots group. They show that they are campaigning on a business platform by mentioning the CBI, the BNE and Roland Rudd. They also mention the politicians David Cameron and George Osborne this could be a strategy either to influence those politicians directly or to influence their followers.

 

We then look at the types of topics discussed in the text of the tweets by looking at commonly used hashtags (the graphs above). @LeaveEuOfficial use hashtags to position themselves. They classify their tweets with hashtags in the same way a librarian would classify a book with specific terms. These hashtags provide a level of context about their position in the discussion in a very small number of characters. They use the hashtags such as #euref, #eu and #uk to frame the debate and establish what they are talking about. They use #leaveEU, #theknow and #brexit to establish a pro-leave direction. Interestingly they also use #justsaying and #justsayin with high frequency. We can see how how the urban dictionary defines that hashtag below.


Just sayin’

A term coined to be used at the end of something insulting or offensive to take the heat off you when you say it


@vote_leave uses #voteleave, #brexit and #eu in a similar way to @LeaveEUOfficial, they also use #EUCO in the same way. They use hashtags to interact with and/or criticise the CBI with #cbi2015 and #dodgyCBI this again shows that they are aiming to campaign on a business platform. We can also see that they use specific hashtags to point at a certain issues, for example to criticise David Cameron’s appearances at a European summit they use #theinvisibleman. They often use the catchphrase ‘wrongthenwrongnow’ as a hashtag.

@StrongerIn do not use many hashtags. They use them for positioning in the debate #eu, #euref. They also used #modi and #modiintheuk to highlight the Indian Prime Minister’s support for the the UK remaining in Europe.

As not all groups use hashtags to the same degree we also analyse the text from the tweets. Below we can see word clouds from each of the accounts showing the frequency of hashtags, individual words and two word terms. Commonly used words (like ‘and’ or ‘the’ ) have been removed.

We can see that because @LeaveEUOfficial use many hashtags in their tweets the hashtag cloud is similar to both the frequent word and two word term clouds. The @StrongerIn clouds indicate a focus on the economy and what is provided by the EU (‘EU provides’).

The @vote_leave word and two word clouds again strongly reflect the hashtag cloud. The two word cloud shows how this campaign discussed (and criticised) the BSE (Britain Stronger in Europe) launch. For some reason this isn’t something they used hashtags to provide a context for – possibly there was no hashtag used by others in referring to the launch. You can see in their two word cloud that there is a message in their campaign urging voters to take control – in the use of both ‘voteleave takecontrol’ and ‘takecontrol eu’.

This initial study shows us that all three groups use Twitter in very different ways. Currently @LeaveEUOffical has a stronger reach – it has more followers, retweets and favourites. It uses traditional media through mentions to extend that reach. @StrongerIn tweets consistently but at a lower rate and attempts to increase its reach by the use of images. Both @StrongerIn and @vote_leave challenge each other and each others supporters directly. @vote_leave tweet in a clustered manner and react quickly to current events. We will continue to study these approaches as the campaigns continue and see if any of theses strategies change as the referendum draws nearer.

This is written as part of the #ImagineEurope project. The project is part of the Economic and Social Research Council’s UK in a Changing Europe programme. Look out for our regular updates as the project tracks developments in the debate on the UK’s continued membership of the EU and follow us @myimageoftheEU on twitter.


This article was originally published on the imagineEurope Storify

Getting the message across in the twittersphere: How Remain and Leave camps use #hashtags

Working Towards Sentiment Analysis


In our work investigating how people discuss the EU within Twitter one of our aims is to determine sentiment, both pro and anti the EU, and in relation to the referendum on UK membership of the EU – pro-remain or pro-leave.

The approach we have taken initially is very straightforward. If tweeters use hashtags associated with the leave camp (including; #brexit, #no2eu , #notoeu, #betteroffout, #voteout #eureform, #britainout, #leaveeu, #voteleave, #beleave, #loveeuropeleaveeu, #leaveeu) then we judge their sentiment to be pro-leave. If they use hashtags associated with the remain camp ( #yes2eu, #yestoeu, #betteroffin, #votein, #ukineu, #bremain, #strongerin, #leadnotleave, #voteremain) then their sentiment was pro-remain.

If we do this we get the following results:


And we can see that these results are fairly consistent day by day:


But they are not consistent with polling data – such as the ICM tracker below which shows remain with the higher score. So why is that?


Firstly – we need to remember that research using Twitter data can only ever tell us what people who use Twitter think and does not necessarily reflect the population at large. Interestingly this has been a problem for pollsters trying to include this data in their election prediction polls (Metaxas, Mustafaraj & Gayo-Avello).

Secondly, we also need to remember that, as we discussed in a previous post, people tend to tweet against things rather than for them.

Thirdly, we are currently restricting our analysis to sentiment associated with hashtags. This means that if the tweet doesn’t have a hashtag we are not analysing it’s sentiment.

If we take a look at the data we can see some differences. There is a difference in style between the way that tweets and in particular hashtags are used between both camps. The leave camps tend to use many hashtags – see below for examples from both LEAVE.EU and Vote Leave – especially LEAVE.EU.



LEAVE.EU use hashtags in their twitter bio – this may encourage followers to use them.

The remain camp do not tend to use as many hashtags in their posts – for example:


Tweets that would be considered pro-remain often also include hashtags we have classified as pro-leave. This could be for several factors such as positioning themselves within the debate by using a popular hashtag or trying to talk to those which hold opposite views.


We can see this by looking at the data. When we look at hashtags that are used in conjunction with #strongerin and #brexit (graphs below). We find, overall, a much lower use of #strongerin and we find that it is used with leave hashtags especially #brexit, #leaveEU, #voteLeave. Where as #brexit is used not used with remain hashtags at all but with other leave hashtags.



In the future we aim to do a more sophisticated form of sentiment analysis where we analyse the text with the tweets. This leads on to a final problem that is often discussed in association with sentiment analysis of text, and one that we also see here, which is identifying the target of the sentiment. The target is the item that the sentiment is expressed towards. In the tweet by Richard Corbett MEP above the ‘leave’ sentiment is expressed towards the ‘U think being part of EU holds back trade with rest of world? ‘ and the ‘remain’ sentiment is expressed within ‘Think again’. Automatically identifying the text that the target is associated with is not always easy. This is not an easy issue to tackle and we have’t even begun to discuss jokes and sarcasm yet!

So you might be tempted to think that if this data is not representative of the general public why are we looking at it? What we can look at is how this data changes. If we can identify the differences and track the relationships between Twitter data and a more general public opinion we can start to hypothesise about how changes in the Twitter data equate to public opinion more widely. We’ll talk about this a lot more in the future.


Our project is part of the Economic and Social Research Council’s UK in a Changing Europe programme. Look out for our regular updates as the project tracks developments in the debate on the UK’s continued membership of the EU and follow us @myimageoftheEU.

This article was originally published on the imagineEurope Storify.

Imagining the EU: Words associated with EU and Europe

We are continuing to investigate how people discuss the EU on Twitter. Our aim is to discover common themes which may emerge leading up to the referendum on the UK’s inclusion in the EU. One of the things that we are looking at is hashtags associated with the terms EU and Europe. The following graph shows us hashtags that are used in conjuction with either #EU or #Europe.



We have been collecting data since 9th August 2015 so we are beginning to build up quite a big dataset – as of the 27th October we had 3,109,130 tweets associated with the EU (gathered using a variety of EU based terms). See our previous article for details.

https://storify.com/ImagineEurope/building-a-twitter-dataset-to-find-out-views-on-th

We can see that the majority of the discussion over our collection period has been on refugees and migrants. We discussed these terms in some detail in another post.

https://storify.com/ImagineEurope/what-is-associated-with-europe

Does this suggest that refugees and migrants will be hot topics that people will consider when voting? Or are they just a reflection of what is currently happening? We’ll have to do more research to find out. Keep following us for updates on this question.
Looking at the data in more detail there are also some things in here that we might not expect. For example ‘opkillingbay’ shows up with a high frequency as does ‘yearinspace’. This is because using terms like EU means we have a broad collection strategy. We’ll need to do some post collection clean up before we present a more fine-grained analysis.


#OpKillingBay press release – Anonymous to save dolphins



We can see immediately that the frequency of the hashtags supporting the UK leaving the EU (brexit, no2eu, betteroffout) is much higher than that of those supporting the UK staying in the EU (yes2eu, betteroffin, votein, Bremain). The frequency of the remain-in hashtags is so low that they don’t even appear in this chart. This is not surprising when we look at our basic sentiment analysis over the same period.



2% of the tweets we have collected are in favour of remaining in the EU compared with 98% associated with a desire to leave the EU. Our regular followers will remember however that twitter is not necessarily representative of public opinion. The existence of a term or hashtag does not always indicate support – it can be more likely that people tweet against a subject (Barber and Rivero 2014). We’ll be keeping a careful eye on this and will return to the question of sentiment in twitter in a later post.


Our project is part of the Economic and Social Research Council’s The UK in a Changing Europe programme. Look out for our regular updates as the project tracks developments in the debate on the UK’s continued membership of the EU and follow us on Twitter @myimageoftheEU.


Neuropolitics Research Lab – People – Politics and International Relations (PIR)

Neuropolitics research politics experiments using fMRI brain scanning.
WWW.POL.ED.AC.UK


Laura Cram is Senior Fellow, The UK in a Changing Europe, investigating The European Union in the Public Imagination: Maximising the Impact of Transdisciplinary Insights (ESRC/ES/N003985/1).