Beyond Social Listening: Social Media Data + Other Data Sources = More Insightful Reports

If you’ve visited our blog before, you’ll know that there are several ways that social listening can be leveraged to understand consumer habits, set benchmarks, and track conversations. But did you know that it is also possible combine other data sources with social data to come up with more robust insights?

At Circus Social, we have been integrating various types of data in our reports for our clients. Read on to know how we do it!

 

#1: Integrating Social Data with Google Search Data

Social chatter is a good indicator that people are interested about a certain topic – but what about topics they don’t necessarily talk about on social media? In some cases, search data can be a stronger proof of interest than buzz volume.

Let’s say you want to discover the top skin care concerns in Singapore. Since it is an embarrassing topic, people may not necessarily talk about acne or psoriasis on Twitter or Instagram. But they are likely to search for solutions for these problems on more ‘private’ search platforms, like Google.

So in this case, search data can indicate interest, but social data can explain qualitatively why people are interested in that particular topic – for example, reasons why acne is a problem for them, products they use to manage their acne, and many explanations.

Search gives you a trigger, Social gives you the reason for the trigger.

So, where exactly can you access Google search data? Trends in Google search volumes are openly available in the Google Trends website. This site analyzes the popularity of search queries in Google across various locations and languages. You can also get search data from the Keyword Planner tool if you have a Google Adwords account. Compared with Google Trends, this tool provides more accurate search volumes (especially if you have a paying account).

More insightful reports - easily done by combining different data sources & social media!
More insightful reports – easily done by combining different data sources & social media!

Popular Use Cases:

  1. Trend Spotting/Tracking – Search data is especially useful for identifying trends because it allows people to search for topics even before they start talking about them and can help indicate when interest for a topic started. Then, social data can be used to look at specific conversation themes about these trends.
  2. Brand Health Tracking – Especially when looked at in comparison with competitors, search data can tell you where your brand stands, specifically in terms of awareness. Social data is a good reference for brand associations and you can also complement this data with related search terms, which can be found in both Google Trends and Adwords Keyword Planner.
  3. Campaign Tracking – Other than looking at increase in social conversations, it helps to see if your campaign had an impact on how much people have searched about your brand on Google.

 

#2: Using Social Data to Complement Traditional Research Data

We have a number of Market Research Agency partners and they usually ask us to look at social media data to dig deeper into traditional research tracker findings.

Traditional research is often not able to fully explain the ‘whys’ behind findings and numbers a lot of times. Once a study concludes, traditional researchers just need to work with the data that they have. If they have additional questions that need to be answered, they would need to start another study. This is where digital research comes in – because it is possible to pull historical data using social listening tools.

In one of the projects that we did, the Retail Audit and Brand Health Tracker Data showed that a client’s competitor brand had an increase in Market Share and Consumption, despite a decrease in advertising spends. Their Brand Equity score increased including all specific imagery statements (i.e. Love/Taste). What could have caused the increase of this score?

Working together with the Market Research Agency, we were able to identify specific periods when scores increased and looked at social conversations during these periods. After deep-diving into these conversations, we found out that consumers created a new demand occasion for the product which became popular in that country.

More insightful reports - easily done by combining different data sources & social media!
More insightful reports – easily done by combining different data sources & social media!

Popular Use Cases:

  1. Brand Health/Equity Tracking – Market Research can be the basis of qualitative social media data analysis.
  2. Supplementing Qualitative Research – For one of our clients, we found out that online review sites were influential sources of information that affected consumer purchase behavior. Through social listening, we were able to identify which particular websites consumers visited and Qualitative Research explained why people preferred these websites.

 

#3: Combining Social Data with Sales Data

If traditional research has not been done, social data can often explain movements in sales volumes and provides more in-depth insights. Just take note though, there may be a lag between when people talk or search about a product and when they actually purchase it. Hence, it is best to analyze these data points over longer durations to really see the correlations between them.

More insightful reports - easily done by combining different data sources & social media!
More insightful reports – easily done by combining different data sources & social media!

Popular Use Cases:

  1. Campaign Tracking – Ideally, campaigns that you launch should have an impact on sales. Aside from the usual metrics analyzed for post-campaign evaluation reports such as reach, impressions, volume of social conversations, social engagements, etc., you may also look at the correlations between these data points and sales data.
  2. Consumer Purchase Behavior Analysis – For some brands, they may notice that sales volumes peak and drop at certain months of the year. These may not necessarily be because of campaigns especially if they are not implemented on a regular basis. Reasons for seasonality in purchase trends can be uncovered by looking at social data alongside sales data.

 

By no means are these the only data sources that can be integrated with social data (you can also look at website traffic data, digital media performance metrics, among others). The key is to track as much data points as you can and organize your data so you can easily discover correlations whenever you need them.

You’ll never know how much value you can get from your data if you don’t try playing around with it!

Circus Social can help you look at your social, search, sales & many other forms of data to uncover deeper insights – just like how we have helped our clients make sense of theirs!

3 Ways to Leverage on Social Listening for (more) Effective Digital Marketing

As we usher in the Lunar New Year, it’s a good time for us to look back on the 2018’s highs and lows.

Did you achieve the targets and hit the digital KPI’s you set? If not, are you making the most out of the information you gathered about your consumers and achieving high engagement & association scores linked to your objectives?

Leveraging on social listening to understand consumer habits, benchmark and track conversations can do wonders and perfectly complement your marketing strategy.

Here’s how!

Identify Conversation Themes and Brand Associations

Maintaining brand image and driving positive associations requires a structured plan that is built on consumer insights and spotting opportunities from competitive tracking.

 

 

To do this effectively, we encourage our clients to dive into the conversations that netizens are having and focusing on the key category themes that your brand or competitors are mostly mentioned in. Identify common phrases and emotions to get an idea on what netizens associate your brand with. It could be an emotion or a consumption moment; You’ll be surprised at the type of chatter you can find on social media.

 

How you can make use of it

By identifying these common conversation themes, you can streamline and optimize your content calendars for various digital channels. Work with your social media leads or brand ambassadors to map out your communication materials for campaigns that are in line with category conversations.  This allows you to strengthen your brand voice by sharing content linked to existing themes or topics of discussion that your audience is already engaging in.  . Showing your understanding of the key topics that consumers’ discuss and ask about, is important if you want engagement and traction online.

 

Spot popular channels to get targeted/effective reach

Marketing on all social media channels would probably get a lot of reach for your brand, but are you really reaching the audience that matters to you? Are your ad placements and reach numbers converting into leads fast enough? If they’re not, then probably you need to rethink about your strategy.

Why aren’t my reach numbers converting into leads?

Paid media can be a hefty investment so it’s important to target areas that have high traffic and an audience base that talks about relevant topics. For example, if you are marketing electronics to people aged between 21 and 25, then perhaps placing ads and banners on popular E-Commerce sites, might get you a higher click-through-rate as compared to placing it on search engine portals. Determine popular “hang outs” by looking into your consumer online habits (e.g. Do they leave reviews on sites or comment directly on brand ambassadors’ pages? If they are camera-lovers, do they visit Hardwarezone or Nikon Rumours?). You can then focus on these sites for paid media to effectively target your audience. You might also need to consider the type of content, messaging and communication style per channel to create more relevant content. Think about the types of questions your audience is asking on these channels, the pain points they regularly share, and what types of thought-leadership and educational content you can create to drive organic traffic, or boost your paid media as well.

 

Establish Engagement Level Norms during the Year: During Campaigns, and Quiet Periods

“We have to be Always-on!”

That’s something that many of us believe in, but what happens when you are “off ?” Getting traction during periods where your brand does not have a big launch or activation can be a challenge. Instead of aiming to achieve such high traction all the time, establish some engagement norms to identify a healthy traction range for your brand.

What do you mean by a healthy range for engagement?  

Identify the level of engagement that your brand and their biggest competitors typically garner on a weekly or monthly basis. This average can serve as a benchmark as to how much traction your brand should be gathering. Deviating too far from it will be an alert for you to relook your social media engagement strategies and identify opportunities to act. You can even take it one step further by identifying an upper and lower limit of engagement levels to see how much your engagement levels can afford to fluctuate.

 

 

Instead of aiming to achieve high traction all the time, establish some engagement norms.

 

The key takeaway is to understand your audience. Social media intelligence can tell you a lot on your key target market. From brand awareness to brand associations, these are just some of the ways that you can make use of social listening to improve your digital marketing strategies. So the question is, have you been listening?

 

 

Finding Content Opportunities during the Demonetization Crisis

On the day India’s Prime Minister, Narendra Modi, declared the currency demonetization, the country witnessed a mixed reaction from its people. The very next day, the whole country was in turmoil and rushed to exchange these notes from banks and their ATMs, – only to find that banks and ATMs were struggling to meet these suddenly increased demands. Widespread chaos ensued across the nation and hundreds of thousands of people were adversely affected, most evidently the lower strata including farmers, daily wage earners, middle class and SME’s. Even large corporates were forced to shut operations for multiple days and lay off hundreds of employees. Other incidents representative of the impact of demonetization were the endless long queues of harried, starving and deprived men and women in front of ATMs and banks, once flourishing street stalls and shops, now a deserted scene.
Demonetization and Narendra Modi
The decision made by PM Modi to demonetize 86% of the notes in circulation was one that was initiated with various interests in mind. The one thing that it has undoubtedly successfully accomplished, is that of the widescale adoption of digital payments technology by Indians of all strata, nationwide.

In the wake of this massive push towards digital payments technology and moving towards a cashless, economy in India- one of our payments solutions clients saw this as a great opportunity to leverage their presence and services. The global payments company sought to identify more growth opportunities for their business and to increase share and adoption of its digital payments solutions. At such an opportune time, in order for it to reach out to this massive audience, it was extremely crucial to identify and understand the voice of public opinion, thoughts and issues surrounding this new loss of currency. An understanding of what the masses felt, what were their pain points and issues, could help the brand provide a solution in the form of its services which at its core was all about going cashless and embracing digital means of payments.

This is where 20/Twenty, our proprietary social intelligence and listening platform, helped the brand achieve insight and clear view into the opinions, thoughts and trends of the consumers moving to digital payments in India. A continued study of competition has also helped the brand keep a close tab on competitors as well as payment wallet solution providers’ activities of interest.

We at Circus Social, leveraged data from 20/Twenty to deliver daily insight reports which informed the brand of key developments and areas of opportunity with the following assessments –

• Consumer sentiments towards demonetization
• Identification of pain points of consumers as well as SME’s
• Tracking various campaign efforts of the government and competitors to learn what worked and resonated
• Tracking daily updates of other e-wallet companies like Paytm, MobiKwik etc.
• Measuring and categorizing different types of consumer engagement
• Identifying key influencers

The client also leveraged our social listening platform to get daily automated alerts when there were key issues being discussed, changes in consumer sentiment as well as growing trends. Alerts were based on keywords and consumer sentiments, when they reached a particular threshold the client received a notification. These e-mail alerts helped our client to be alerted to new and important updates and changes around India’s demonetization fiasco. With the help of 20/Twenty, the team was able to tap into the pulse of the masses and understand their sentiment on the whole demonetization drive.

Here are some of the interesting trends and insights from this crucial period:

• Over a period of two months’ people have understood that downloading an e-wallet app to pay bills is very convenient. Hence, adoption has been quick and easy with people using their e-wallets for to pay almost everything now including mobile bills/recharge, utility bills, cab bookings, movie tickets, etc.
• Post demonetization, e-wallet companies like Paytm, MobiKwik and FreeCharge have seen a huge surge in their daily transactions like never before. Paytm, MobiKwik, FreeCharge have already been in the overdrive to reach out to small traders to commence transactions on their platform. From running full-page ads in newspapers to making their presence felt on TV and digital, Paytm, FreeCharge, MobiKwik etc. are focusing on educating the consumers, expanding their user base and promoting cashless transactions.
• Moreover, since a lot of merchants don’t have card readers for debit/credit transactions, they have started to accept e-wallets as a mode of payment.
• Despite the positive developments post demonetization there still exists a section of people who are unhappy with PM’s decision of currency ban as they are forced to stand in long ques to withdraw their own cash. Also there are shops which still don’t accept any form of e-payment
• Infrastructure still a problem as its unable to cope with demands for new cash. ATMs are constantly out of cash and long queues till the wee hours of the morning persist resulting in severe consequences for consumers with them unable to pay and support their day-to-day consumption.
Whether or not India will benefit from the demonetization drive is still a big question mark but Mr. Modi’s demonetization initiative has definitely been a boon for India’s e-payment providers and they definitely have right future.
The future of e-payment providers:
• The future seems to be extremely strong. Smaller restaurants, vegetable venders are now open to accepting e-wallet payment, some merchants do not accept credit/debit cards but are now accepting e-wallet transactions.
• There has been a constant increase in the number of electronic transactions since demonetization started. Out of the 700 million debit card users in the country, 450 million earlier used the facility only at ATMs. Now, they are swiping cards at PoS (point-of-sale) terminals. This is encouraging for a move to a cashless economy.
• E-wallet transactions are convenient as these are conducted through mobile phones with internet connection.
• This is the time, for Digital Wallet Companies to grow further and be the next way of growth because people are moving towards digital life by accessing internet facility to their life.
Demonetization Memes

So far so positive, but what could hold back this trend?
Having specified the benefits of electronic payment system, it is necessary to mention its drawbacks:
• Restrictions: Each payment system has certain restrictions regarding the maximum amount in the account, the number of transactions per day and the withdrawal amount.
• The risk of being hacked: As more and more people download and figure out how to use mobile wallets such as MobiKwik, FreeCharge or Paytm, there is also increasing concern over the security of transactions on these apps. If one follows the security rules the threat is lesser. Most reputed wallets adhere to all RBI security specifications and also have extra layers of security. However, most wallet apps only require a simple click for its operation. So, if you lose your phone, whoever finds it has control of your wallet. Only thing which may save you is a PIN or fingerprint protection when you send or transfer money. Though all apps are inherently safe, or as safe as your bank account, there are a few things a user has to do to ensure the financial information or money on their phones is not misused.
• The problem of transferring money between different payment systems. Usually the majority of electronic payment systems do not cooperate with each other. In this case, you have to use the services of e-currency exchange, and it can be time-consuming if you still do not have a trusted service for this purpose.
• The lack of anonymity. The information about all the transactions, including the amount, time and recipient are stored in the database of the payment system. And it means the intelligence agency has an access to this information.
• The necessity of Internet access. If Internet connection fails, you cannot get to your online account.
In general, the advantages of electronic payment system outweigh its disadvantages and they have bigger opportunities comparing with ones of traditional wire transfers.

Check out how the 20/Twenty Social Monitoring platform has grown!

Growth of 20/Twenty Social Monitoring & Intelligence Platform – Under the Hood

Growth of 20/Twenty Social Monitoring & Intelligence Platform – Under the Hood

We have often been asked about the challenges we faced in scaling up our technology stack to manage big data. I have attempted to address this in this post which is the first of a series of blog posts on this and similar topics.

20/Twenty was created ground-up as the most intuitive and easy to use cloud based (SaaS) Social Monitoring & Intelligence platform in the world.   Based on our deep understanding of what marketers needed and the awesome designs we created, we signed up our first client even before the product was officially launched. The pressure to quickly deliver the first version of the product was intense 🙂

From an engineering point of view, there’s a huge amount of data that we pull (Think Big Data!), process, augment and then visualize in the platform all on a near real-time basis. Imagine someone tweeting and it appears on our platform within a few seconds along with augmented information including Gender, Sentiment, Engagement, Spam score etc.

The evolution of 20/Twenty has already seen a few stages of growth. The graph below shows how 20/Twenty data has grown over the last 2 years since our product launch. This is a really cool growth for a startup like Circus Social both from a business perspective as well as from an engineering standpoint. We used several tricks from the books as well as a few practical hacks to ensure our ability to fetch, process, augment and visualize high volumes of data continued to become better, though this journey was not without pain!

social-intelligence-20twenty-big-data-growth

 

Stage 1

We created over 200 custom marketing applications in our previous avatar at Circus Social working with some of the biggest brands in the world. We used the same open source technologies (PHP / MySQL) to create the first version of 20/Twenty. This worked well and as our data grew in the first few months, we continued to grow vertically by adding more capacity (CPU/RAM).

Most of the queries from the application were read queries whereas a bulk of “write operations” were being performed by our data crawlers. We therefore created an efficient master-slave architecture where the application would read from the slaves and the crawler scripts would write into the master. This worked well in general but the exponential increase in the volume of data meant that certain queries were running extremely slow and impacting the user experience.

Stage 2

Since our data volume was growing exponentially and the relational aspects of the database were not the core of our application, we realized that sooner or later, we would have to move to a NoSQL database. However, the performance issues that were cropping up had to be sorted quickly and without a downtime. We quickly realized that we needed a dedicated search engine and MySQL was not good enough for this purpose.

We explored several options and Elasticsearch came to our rescue here. Elasticsearch is a distributed, RESTful search and analytics engine that centrally stores your data in a manner which can be retrieved / read really fast by your applications. Our awesome tech team deployed this in a matter of days. The improvement in performance was remarkable. The plan worked and we cheered!

Stage 3

Word spread in Singapore and Asia about how good our platform was (and our sales team did a good job too!) and we continued to sign up new clients. The volume of data continued to grow for existing clients as well as new clients. The tech stack of MySQL and ElasticSearch did not let us down but we wanted to create an architecture that would scale infinitely, if there’s a thing like that.

In Stage 3, we moved the core of our database from MySQL to Cassandra (Elasticsearch was now interacting with Cassandra) and the backend code from PHP to Node.js. We also migrated most of our front end code to Angular.js for better performance. This was a major architectural change on a live application being used by several clients so we created a parallel production like environment and ran it parallelly for several weeks to ensure everything was working as desired before switching over.

While we did the above, we continued to work on cool new features on the product and opened up our data API’s to a few clients who wanted a deeper integration with their own applications. Other tools we used during this and other stages were Postman, Github and JIRA.

As we scale further from here, we will probably have newer and more exciting technology challenges and we will keep posting about them. If you are excited to work on some of these, do write to us at careers@circussocial.com