Increasing access to digital intelligence for entire organizations is our biggest challenge

Interview with Prerna Pant, General Manager at Circus Social, a social listening and analysis company based in Singapore.

Hi Prerna, what is your background and what is included in your current role at Circus Social?

You could describe me as a storyteller; I started telling stories for brands very early in my career – and I have continued to do so till today. I’m a Co-Founder and General Manager at Circus Social. My role includes business development, operations, marketing and client management – and spans across all our products and offerings.

What differs Circus Social from other media intelligence companies in APAC?

To start with – we’re a company that was built ground up by marketers, for marketers. Too often we find that solutions are made for marketers by those who have not walked in those shoes before – and we wanted to change that. Secondly, we’re an Asia-first martech company. We were born and bred in Asia – meaning that we specialize in the APAC region in terms of language, sentiment, source and local coverage throughout the region, including Myanmar, Vietnam, Thailand, Indonesia, Philippines, Japan, Korea and more, and have a local grasp on culture and online behavior.

What are your greatest challenges ahead at Circus Social when it comes to serving your customer analysis and developing your offer?

I think we face many of the same challenges that our competitors face in terms of data access, features and innovation. But personally – I believe the biggest challenge for us lies in increasing access to the world of digital intelligence – and taking it from the marketers’ ecosystem to the entire organization.

If any, what specific needs are there in your region for media intelligence that you think may differ from the rest of the world?

The greatest one is localization; this goes far beyond just language and platforms. The way in which consumers share product reviews on Pantip in Thailand is massively different from how Koreans behave on Naver, and it is the job of digital intelligence companies to identify and capitalize on this for our clients.

Do you find the region diverse in the sense that it is challenging to offer comprehensive products and services throughout the region? If so, in what way?

Each market comes with its own challenges, but I think that’s what makes it more exciting to work in this region. We have to keep learning, keep innovating and continue to challenge the status quo in each market. The initial entry into each market is typically the one we have to get right first, and once the foundation has been laid, it’s essential that we don’t drop the ball and continue to offer solutions that match the needs and nuances of each market.

Can you provide a specific example where one (or more) of your clients has made changes based on the insights or analysis you provided them? 

There are many capabilities that our platform, 20/Twenty, provides, ranging from crisis monitoring and campaign tracking, to consumer behavior and insights. I’ve found that getting that right from the very beginning and being a partner to your clients instead of being a service provider is what makes that difference.

We’ve worked with automobile brands that found that crisis situations were growing differently on social platforms vs. media sources – identified through custom features on 20/Twenty that allow you to track trending content by source type, spikes in conversations and specialized data tagging – and hence, they could easily measure actions and address problems once discovering this.

In a completely different setting, we helped an Asian supermarket understand why ‘mommy shoppers’ were declining rapidly through social listening on parenting forums and review sites. The main reason was that the width of the aisles in their grocery stores was too narrow to fit a pram – and hence aggravated mothers were dissuading others from shopping at their outlets.

The applications are obviously endless – it’s about getting to the insights quickly and more effectively.

Have you recently, or are you about to, release any new technology-based solutions that will add on to or improve services you offer your clients? If so, what solutions, and how will your customers benefit from them?

We have a lot of exciting features from a tech standpoint that will position us to be the leaders in viral content predictability and influencer identification. These are based on client requests and feedback, as is much of our tech roadmap.

When it comes to licensing content for media monitoring in your region, which countries are the most progressive and which are lagging behind?

Although we see this changing rapidly, markets such as Singapore, Hong Kong and Australia are certainly more comfortable and advanced with licensing data properly. This is feeding into other markets, such as Myanmar and Vietnam very quickly, and with early adopters setting the standard that new markets can follow.

Which social platforms are the most important to your clients, and which ones do you see as having the most potential in the future when it comes to gathering relevant information for your customers?

The importance of platforms changes by client, use case and industry; however, I also truly believe that you can’t ignore any of them today as conversations and people are very interlinked. For example, if your use case is reputation management – you must realize that a crisis can break anywhere, anytime. Similarly, I also always tell my clients that campaign periods aren’t the only time that consumers talk about you – it’s important to be ‘always on.’

When it comes to the actual data behind the media intelligence you do, what kind of data or media that you don’t currently use for media intelligence today, can be interesting in the future?

Expanding on image and video recognition and applying that to a variety of platforms is certainly interesting for us. We also see great potential in audience segmentation and predictive technology.

By Renata Ilitsky

 

This article originally appeared on Twingly’s Blog: https://blog.twingly.com/2018/06/20/increasing-access-to-digital-intelligence/

Check out how the 20/Twenty Social Monitoring platform has grown!

Growth of 20/Twenty Social Monitoring & Intelligence Platform – Under the Hood

Growth of 20/Twenty Social Monitoring & Intelligence Platform – Under the Hood

We have often been asked about the challenges we faced in scaling up our technology stack to manage big data. I have attempted to address this in this post which is the first of a series of blog posts on this and similar topics.

20/Twenty was created ground-up as the most intuitive and easy to use cloud based (SaaS) Social Monitoring & Intelligence platform in the world.   Based on our deep understanding of what marketers needed and the awesome designs we created, we signed up our first client even before the product was officially launched. The pressure to quickly deliver the first version of the product was intense 🙂

From an engineering point of view, there’s a huge amount of data that we pull (Think Big Data!), process, augment and then visualize in the platform all on a near real-time basis. Imagine someone tweeting and it appears on our platform within a few seconds along with augmented information including Gender, Sentiment, Engagement, Spam score etc.

The evolution of 20/Twenty has already seen a few stages of growth. The graph below shows how 20/Twenty data has grown over the last 2 years since our product launch. This is a really cool growth for a startup like Circus Social both from a business perspective as well as from an engineering standpoint. We used several tricks from the books as well as a few practical hacks to ensure our ability to fetch, process, augment and visualize high volumes of data continued to become better, though this journey was not without pain!

social-intelligence-20twenty-big-data-growth

 

Stage 1

We created over 200 custom marketing applications in our previous avatar at Circus Social working with some of the biggest brands in the world. We used the same open source technologies (PHP / MySQL) to create the first version of 20/Twenty. This worked well and as our data grew in the first few months, we continued to grow vertically by adding more capacity (CPU/RAM).

Most of the queries from the application were read queries whereas a bulk of “write operations” were being performed by our data crawlers. We therefore created an efficient master-slave architecture where the application would read from the slaves and the crawler scripts would write into the master. This worked well in general but the exponential increase in the volume of data meant that certain queries were running extremely slow and impacting the user experience.

Stage 2

Since our data volume was growing exponentially and the relational aspects of the database were not the core of our application, we realized that sooner or later, we would have to move to a NoSQL database. However, the performance issues that were cropping up had to be sorted quickly and without a downtime. We quickly realized that we needed a dedicated search engine and MySQL was not good enough for this purpose.

We explored several options and Elasticsearch came to our rescue here. Elasticsearch is a distributed, RESTful search and analytics engine that centrally stores your data in a manner which can be retrieved / read really fast by your applications. Our awesome tech team deployed this in a matter of days. The improvement in performance was remarkable. The plan worked and we cheered!

Stage 3

Word spread in Singapore and Asia about how good our platform was (and our sales team did a good job too!) and we continued to sign up new clients. The volume of data continued to grow for existing clients as well as new clients. The tech stack of MySQL and ElasticSearch did not let us down but we wanted to create an architecture that would scale infinitely, if there’s a thing like that.

In Stage 3, we moved the core of our database from MySQL to Cassandra (Elasticsearch was now interacting with Cassandra) and the backend code from PHP to Node.js. We also migrated most of our front end code to Angular.js for better performance. This was a major architectural change on a live application being used by several clients so we created a parallel production like environment and ran it parallelly for several weeks to ensure everything was working as desired before switching over.

While we did the above, we continued to work on cool new features on the product and opened up our data API’s to a few clients who wanted a deeper integration with their own applications. Other tools we used during this and other stages were Postman, Github and JIRA.

As we scale further from here, we will probably have newer and more exciting technology challenges and we will keep posting about them. If you are excited to work on some of these, do write to us at careers@circussocial.com