Creating an API to tap into Twitter with Open Source Software
When I was a kid, there was never enough money to buy full price games. I’d get a magazine that had demos of games, and after I’d played those to death, I’d turn to the back on the magazine where there were code listings. I’d type them out and hack away at the creations. Later on I’d order public domain games from magazines that came with source code, altering the code to learn how they worked and whether I could modify them.
In 2001 I dropped out of the final year of my CS degree (I had my Diploma in S/W Engineering) to take an IT Apprenticeship that lasted 24 months. I spent the first 12 months in various departments, with the remaining 12 months in a software development team for a consultancy, building a master patient index system.
At the end of the apprenticeship I was offered a job, staying there for 5 years before leaving to join a start-up, followed by a consultancy where I moved up the ladder to become a senior consultant and technical lead. I broke out on my own and became an independent contractor for a while, then returned to employment.
In between that, I completed an MSc in Computer Science with a focus on Bayesian Theorem and Text Analytics, and how this can be used to surface insights in social media data. I started to document these activities on my blog, which gained readership and resulted in my code experiments being picked up by Twitter themselves. These would later be used to help build Pluralsight Courses and community work, which resulted in an MVP Award in AI.
Building the Social Opinion API
I’ve been working with the Twitter API for many years and was working on a research project through an MSc I was doing outside of my day job. The project involved Twitter analytics, and I built an API using machine learning-based approaches to help me classify Twitter data using C#.
Shortly after this Twitter ran a developer initiative over 3 years called #Promote. I joined a few dots and submitted minimal viable products to #Promote each year, each of them building on the predecessor and all built using .NET. These products could help surface signals such as keywords, commercial intent and more.
One iteration was centred around audience segmentation and creating audiences for marketing purposes. That is, finding the right person at the right time, with the right message. Through my #Promote submissions, Twitter’s DevRel and Ad Partnership Teams contacted me to show demos of the software. I’ve since built good relationships with the DevRel and Product Teams as a result of these activities.
Why Did I build the API?
Through various conversations I’d had at the time, I knew a major Twitter API release was being planned, and I remembered the pain I experienced as a developer when trying to consume the Twitter API back in the early years!
No one else was working on building a .NET SDK that would target v2 of the Twitter API, so I wanted this to be the first on the market for .NET Developers. I wanted to make it easy for developers to consume these new APIs and build a community around the SDK. This goal makes it pretty easy for me to build and add new features to a social media analytics platform I work on in my spare time. It’s written in .NET Core so it’s cross platform, and an abundance of free tooling and services such as VS Community, GitHub and Azure Credits make it easier than ever to build and ship when you have limited resources.
The biggest challenge for me was finding the time to work on projects while keeping up with a regular job and family life. I got up 2 hours earlier each day and spent time at the weekends on open source activities, as well as engaging directly with the Twitter DevRel team on multiple occasions. It took four months from the start until the first iteration.
How developers can use the API
Whether you’re looking to grab social media analytics, perform topic analysis of Twitter data or build integrations with the Twitter API, there are many possible ways to use the Social Opinion API. You could even identify the most discussed products, services, locations or businesses, or use it for ad-tech and marketing. There are also many reasons why you should consider giving it a go:
- Accelerates development with the Twitter API
- Shields developers from having to write low level HTTP and security code (OAuth, etc.)
- Gives developers a rich set of easy to use objects (e.g. Tweet.Text) in their code, thereby making the development experience simpler
- You can easily surface Twitter data in existing applications with a few lines of code
- It can be added in a few clicks via a NuGet package, or via the code on GitHub
User feedback has also helped shape the project itself – the data surfaced by the SDK/API was used to build an early version of a SaaS tool called Social Opinion. I’ve also had direct feedback saying data in the dashboards are a great way to see what’s forming the public conversation in one place.
Using open source software
Open source can help developers learn about new practices or approaches to development. I think we live in a bit of an API economy; more than ever it’s becoming easier to pull code examples or APIs from places like GitHub. You can use open source to quickly build prototypes and plug gaps in your existing software or understanding.
My advice to those on their open source journey is simple: don’t be intimidated about shipping your work. It can be nerve wracking when sharing your work online – but learn to dance with that! If you have an idea, just code it or write it and get it out there. It could be something that ends up helping you with your daily job, like an internal tool. Your work might not be for everyone, but that’s okay!
More from the author
Jamie Maguire is a Software Architect with nearly 20 years’ experience architecting and building solutions using the .NET stack. For more of Jamie’s articles, be sure to check out his website.
Want to learn more about this project? Be sure to view the project page, as well as Jamie’s overview of the API/SDK on his website. Also check out:
- Jamie’s blog detailing a minimal viable product (Social Opinion) built with the API/SDK
- SocialOpinion on Developer.Twitter
- Pluralsight Course on the Twitter API
- Jamie’s GitHub repo
More from the OSS series
- Bringing open source sentiment analysis assistance to neurodiverse people
- Using open source software to boost your PowerShell skills
- Creating a bionic arm with open source software
- Saving lives from natural disasters with open source software
- Using open source software to connect charities with people in need of social housing
