Direkt zum Hauptbereich

#cssws15 2nd GESIS Computational Social Science Winter Symposium 2015

Workshop on #R and Twitter

On 1st Dec. 2015 I will give a workshop at the GESIS Computational Social Science Winter Symposium 2015.

Using R to harvest the Twitter STREAM API

R is a free software programming language and software environment for statistical computing and graphics. Because R is a programming language its usage is not limited to the field of statistics. There are already many R-packages to cover the whole spectrum of social media analysis from web scraping to text mining. This is a hands-on-tutorial to analyze data from twitter in R. In four steps, the tutorial demonstrates how to get the desired data, how to “clean” it, how to analyze it, and how to visualize the results. By following these steps the participants will gain knowledge about the general structure of R, its basic grammar, some relevant packages for social media analyses, the twitter streaming API, and about some basic programming concepts. The tutorial addresses readers with little previous knowledge about programming. The aim is to demonstrate that the efforts in learning a programming language instead of using off-the-shelf-solutions are rewarded with greater flexibility for creative social media analysis. The R language has become one of the most prominent tools among statisticians and data miners. Because it is free software and due to its outstanding capacities which are enhanced by a huge community of contributors R has become the first choice at many universities in teaching statistics. 
Example plot from the tutorial.

Data science is more than just statistics. Statisticians survey data in a very careful way to come up with representative samples structured in nice and neat tables. But social media data is not structured this way. It is just out there somewhere on the internet and we have to get it on our computer and transform it in a way that is suitable for analyzing. This step normally involves the use of an API. In a second step, social media data normally has to be cleaned. E.g. there might be duplicates, missing values or wrong specifications of objects. The cleaned data can be used to test hypothesis, to find hidden patterns, to analyze it with descriptive statistics or more advanced machine learning algorithms. Whatever we find out in the end, it is very important to present the results in a way that reduces the complexity of the original data drastically. For this last step, data science has developed visualization tools. All these four steps can be done directly in R. In the tutorial we will connect to twitter and get tweets located in California. This sample will then be analyzed and visualized. The research question for this example is the following: Are trends on Twitter regional localized? On the one hand, twitter is a global social media platform that connects people all over the world. It is therefore reasonable to argue that regional differences are not so important on Twitter. On the other hand, people use twitter to communicate about what is going on in their real lives. Since the real live takes place in a specified space, it is reasonable to argue that information on twitter should show a lot of regional differences.
The tutorial addresses readers with little previous knowledge about programming. The aim is to demonstrate that the efforts in learning a programming language instead of using off-the-shelf-solutions are rewarded with greater flexibility for creative social media analysis.
All participants should bring a laptop with R and R-Studio installed, as well as a valid Twitter account.
Labels:

Kommentare

Kommentar veröffentlichen

Beliebte Posts aus diesem Blog

#RTutorial: Using R to Harvest the Twitter STREAM API

Initializing the Twitter API In this tutorial, the so called STREAMING-API from Twitter is used. This API provides real-time access to Twitter, so the results are dependent from what is actually going on, right now. Before we start, we have to initialize the Twitter-API. To use the Twitter API, a consumer key and consumer secret is required. Therefore, you have to register as a developer who is creating a Twitter app. Create a Twitter account and then sign in at https://apps.twitter.com/. The account has to be verified with a phone number. This can be done on the Twitter webpage in the account settings. Fill in name, description and any valid URL with leading “http://”. It is important NOT to provide any call-back URL, because otherwise the registration from R will not function. After this, you can see a summary of your newly created app with a link to “manage keys and access tokens”. The consumer key and consumer secret that can be found there have to be copied into the following R-...
19 Kommentare
Mehr anzeigen

Deep-Dive Impfeffektivität: Eine kritische Datenanalyse der RKI-Berechnungen / Teil 1: Die Methode

Die Einschätzung, wie effektiv die COVID-Impfung ist, ist eine der politisch relevantesten Kennzahlen derzeit. Insbesondere für die Einschätzungen der Angemessenheit einr Impfpflicht ist diese Zahl extrem wichtig. In der Vergangenheit hat sich immer wieder gezeigt, dass die Berechnungen des RKI nicht in jeder Hinsicht eindeutig sind, sondern auf vielen Annahmen beruhen, die man auch kritisch hinterfragen kann und muss. Für die politische Datenwissenschaft ist es daher essenziell, diese Berechnungen nachvollziehbar zu machen. In diesem Beitrag wird das methodische Vorgehen des RKI zur Berechnung der Impfeffektivität analysiert. Die Informationen dazu entstammen den RKI-Wochenberichten .  In einem zweiten Teil habe ich die konkreten Berechnungen des RKI so weit wie möglich rekonstruiert und kann daher zeigen, wie stark die Ergebnisse schwanken, wenn Annahmen leicht verändert werden. Meine Erkenntnisse aus der folgenden Analyse: Das RKI verwendet zur Berechnung der Impfeffektivität di...
Mehr anzeigen

Zwischen #AfDErfolg und Medienberichten gibt es einen kausalen Effekt - und der lässt sich berechnen

BuzzFeed Deutschland  hat Daten zusammengetragen, mit denen sich der Einfluss der Medien auf die Umfragewerte der AfD berechnen lässt. Wir zeigen, dass es einen statistischen Kausalzusammenhang zwischen der Häufigkeit der Berichte über die AfD und ihren Umfragewerten gibt. Hierfür verwenden wir Granger-Causality-Analyse. Das Ergebnis ist, dass vier bis fünf Wochen, nachdem vermehrt über die AfD berichtet wurde auch die Umfragewerte signifikant steigen. Was ist in den Daten Die Daten enthalten vor allem eine Zeitreihe mit wöchentlichen Aktivitäten zur Suche „AfD“ bei Google News (über Google Trends bezogen) und die Umfrageergebnisse der Afd in den Umfragen der Institute EMNID, Forschungsgruppe Wahlen, GMS, Infratest dimap, Forsa, INSA/YouGov und Allensbach. Aus diesen Werten haben wir einen wöchentlichen Durchschnitt gebildet. AfD-Umfragen/Google News, absolut Die absoluten Werte geben erst einmal keinen Aufschluss über einen Zusammenhang. Die Analyse Zeitreihen h...
3 Kommentare
Mehr anzeigen