Project 1
Demonetization in India
A data visualized story of tweets
When the Indian Prime Minister, Narendra Modi, announced the demonetization of ₹500 and ₹1000 banknotes of the Mahatma Gandhi Series overnight, it wiped 86% of the cash circulated in the country and the shook the world economy.
In this data-comic we explore public conversations on Twitter through two individuals who find themselves in the middle of a frenzied discourse.
Context
Demonetization, first announced to fight black-money, resulted in unprecedented events for India - months of long queues outside banks and ATMs to exchange notes, a significant stagnation of small businesses and endless debates over the implications of the controversial decision. All of this could be seen clearly on one platform more than any other, Twitter.
Timeline
April-May 2019
2 weeks
Tools
Excel, Sublime Text -Python, Tableau, Traficata Data Wrangler, Zoho Analytics
Teammates
Suveer Garg, helped with data cleaning using Python
Responsibilities
I cleaned the dataset and wrangled it, created and analyzed visualizations and finally developed a data-story to communicate the insights.
Research Questions
1. What was the volume of the online discourse and how did it change over time?
2. How did Twitter users react (sentiments) and how did it change over time?
3. Who was driving the conversations?
4. Which tweets trended most?
Data story
Jaspreet & Arminder are two friends who use Twitter and are affected by the demonetization step taken by the Indian government on 8 November 2017.
The following data-comic depicts their experiences and perceptions, and that of other Twitter users, on to the issue, in November 2017 and April 2018.
Demonetization declared on November 8
What were people talking about on Twitter in November?
• Support and ownership of the movement: suport, people, Modi (the Prime Minister), impact, good, blackmoney.
• Alternative solutions: Paytm (an e-payment application), cash, bank, atms, notes.
• Pushback: Opposition: protest, queues, effect, time.
Dataset Constraints
1. The dataset contains tweets from only two days in November 2016 (22 and 23), and 10 days in April 2017 (11 to 21).
2. Data scraping done by API is inefficient - quite a few tweet texts are distorted or missing.
3. Most columns in the dataset were not useful for analysis and were therefore removed.
4. Most tweets are retweets - Therefore, the dataset contains a large number of duplicates.
5. After cleaning, the sample size has become much smaller. Initially - 14940 rows and 15 columns. Final - 4903 rows and 7 columns.
Summary
The analysis outlines that the discourse sways in favor of and against the controversial move alternatively while keeping one thing constant, the belief that the move of demonetization was inherently ’good,’ as can be seen by the persistence of the positive sentiment despite numerous hardships in the country.
We observe that in November 2016, the discussion is around the proposed benefits of demonetization, confusion and resistance.
Coming to April 2017, we see people discussing the effects of this move. The discussions were then based on citizen and media analysis of the move.
Who was driving the conversation?
Among a number of popular tweets, and tweeps (users who posted), the most popular tweep was sainath_kits, however, the most popular tweet was posted by smarpitvarty.
What were the sentiments of the conversation drivers?
Most popular tweet creator sainath_kits is seen replying to a number of users in fairly neutral tweets, with a hint of of positive sentiments.
Manually inspecting the content of the popular tweets revealed that a good number of them are in support of demonetization.
In the coming months -
How did the volume of online conversation on Twitter change in April?
This trend line illustrates that tweets, retweets and the twitter discourse occurs in alternate fashion and are dependent on external factors.
Who was driving the conversation then?
1SunnyElias got the higest number of retweets. Each of these creators had only one tweet that made them overly popular.
What were the conversation drivers talking about?
The most popular tweets either shed more light on the topic; ie, provided an external link to elaborate on demonetization, or were positive and supportive of the move.
But did all Tweeps or Twitter users feel the same way?
There were just as much negative sentiment online as there was positive.
An interesting picture is built around the nature of the discourse - The positive and negative sentiments seem to be responding to each other. When one rises, the other grows in the next day or so trying to match it.
Also, April 1, 2017 marks the beginning of the new financial year.
Conversation on Twitter in April
• Twitter was filled with people expressing their opinions on the effect demonetization has had on India: indian, stil, now, big, idea, civil, etc.
• People converse around cash, economic, atms, rbi (Reverse Bank of India - a central regulatory body) and BJP, the ruling party in India.
• There is still looming uncertainity and a divide between opinions so we can see usage of poor vs rich, income & tax, bad, don’t, time, war, issue, hit etc.
Project 2
Government hiring in Baltimore
A data visualized story of employment data
In this data story, we explore State hiring trends in the city of Baltimore. We begin by asking the following research questions -
1. What are some interesting fluctuations in the hiring rates over the years?
2. What do we know about hiring in individual State departments during these fluctuations?
3. How has hiring over the years varied at the most interesting State departments?
Timeline
May 2019
1 week
Tools
Excel, Tableau
Teammates
None, Solo project
Dataset Constraint
The dataset contains 2017 employee’s hiring data, which is assumed here to be representative of a larger, more complete story around State hiring in Baltimore.
2004-06
Department-wise quarterly hirings from 2004 to 2006
Hiring at Police and Fire departments is much larger than the other departments.
There are a few departments who increased hiring in 2005, but almost all of them hired more in 2006.
2007-09
Reduced quaterly hirings during 2008-09 recession
The fall in hiring in 2008 is severe in some departments (Fire Dept, R&P), while it is more severe in others (Police Dept, DPW) in 2009.
Exceptions - the Health department does not see any major fluctuations while the Water, Some departments see a marginal increase in hiring in 2008 and 2009.
2013-15
Boost in hiring during 2013-15
2016-17
Growth in hiring continues in 2016-17
Taking a closer look at hiring in the Fire Department
The hiring numbers fluctuate often over the years and there seems to be no trend which alternate peaks suggest that hiring would randomly pick up once in a few years.
The largest drop in hiring can be seen in 2012 after the sudden increase in 2011.
Taking a closer look at hiring in the Police Department
Although the increase in hiring numbers has been comparatively more stable in this department, we still see ample fluctuations in the latter years after hiring numbers had increased.
Reflection
I learned to deal with large amount of social media and government data and make it useful for analysis. I used a number of data-wrangling, exploration and visualization tools like Tableau for these projects and enjoyed structuring the narratives while allowing users to interact. Due to limitations in interactions and the dataset, the charts do not facilitate any in-depth exploration.
