index.Rmd

---
title: "From tweets to maps using Python"
author: "Arend Ligtenberg, Jan Verbesselt"
date: "`r format(Sys.time(), '%d %B, %Y')`"
output:
  knitrBootstrap::bootstrap_document:
    title: "Python for Geo-Scripting"
    theme: "simplex"
    highlight: Tomorrow Night Bright
    menu: FALSE
    theme.chooser: TRUE
    highlight.chooser: TRUE
---

# [WUR Geoscripting](https://geoscripting-wur.github.io/) <img src="http://www.wur.nl/upload/b43b7095-e452-482a-8969-fed9a50393a8_WUR_RGB_standard.png" alt="WUR logo" style="height: 35px;"/>

Hi. Good morning! Today we will have fun with Python by playing with social media data. 

# Harvesting tweets with Python

As background knowledge it is assumed that you know what a JSON is. If not, have a look at [JSON Intro](https://www.w3schools.com/js/js_json_intro.asp). Also acquaint yourself yourself with the datastructure of tweets. You can find it on [API Tweets](https://dev.twitter.com/overview/api/tweets). Try to find out for example where spatial references are made in a tweet.

* Morning: Self-study by working on the two notebooks below. 
* 14u-1430u: Feedback and presentation about social media as a source of spatial data
* 14:30-17:00: Assignment (~depends on registration for the twitter api)

# Setup your Conda environment

There are various packages to work with twitter data in Python.
One great Python package for twitter data handling is [Twython](https://twython.readthedocs.io/en/latest/). Twython is an actively maintained, pure Python wrapper for the Twitter API. It supports both normal and streaming Twitter APIs. Today we will work with both APIs. Let's setup the conda environment for today with the Spyder IDE.

```{r, eval=FALSE, engine='bash'}
conda create --name twitterpython python=3 geopandas spyder
conda activate twitterpython
conda install --channel conda-forge twython folium matplotlib psycopg2
```

# Python notebooks for the self-study

Below are the Jupyter Notebooks that contain the tutorials for today.

[Connecting and harvesting real-time tweets](http://nbviewer.jupyter.org/github/GeoScripting-WUR/PythonTwitter/blob/master/ConnectingandharvestingtweetswithPython.ipynb)

[Harvesting real-time tweets](http://nbviewer.jupyter.org/github/GeoScripting-WUR/PythonTwitter/blob/master/HarvestingRealTimeTweets.ipynb)


# Exercise 14: Optional for Geo-scripting students

Create a twitter harvesting application that harvests tweets (either real-time or stored), and create a spatial dataset and map from it. 
The application should be able to: collect tweets based on a thematic and/or spatial query. 

Suggestions:

* Feed a spatial database with information tweeted real time from a certain location and plot it on a map.
* Create a map indicating where traffic jams, disruptions in the public transport or ... took place last week.
* Create a map indicating the home countries of international tourists visiting Amsterdam (or Wageningen if you like)
* Go out with your smartphone (what else should you do during lunch), collect spatial information (for examples POIs on the campus) by tweeting from various locations using a specific hashtag (make sure location is enabled on your twitter account), harvest these tweets with an application, and create a map from it.
* .... think of something else.

Send in your application as a python script as well as a screenshot of the map you produced.

If you want to go beyond todays tutorials, have a look at some cookbooks and examples:

* [Mining-the-Social-Web](https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition/blob/master/ipynb/Chapter%201%20-%20Mining%20Twitter.ipynb)
* [Twitter Cookbook](http://nbviewer.jupyter.org/github/ptwobrussell/Mining-the-Social-Web-2nd-Edition/blob/master/ipynb/Chapter%209%20-%20Twitter%20Cookbook.ipynb)
* [Postgis Quickstart](https://live.osgeo.org/en/quickstart/postgis_quickstart.html)