Social media has opened up a whole new set of data opportunities for marketers. The world is eager to share their opinions and have their voice felt by the masses. You can make use of this by actively plugging their statements into your strategic workflows.
For instance, if you run an AI startup and you choose to collect comments using the keyword “agentic AI” then you’ll be able to understand what exactly people like about these tools, and what their pain-points are. By harnessing this, you can make precise, user-oriented decisions on how you position your products and services..
Subnet 13, Data-Universe, offers a lightweight method of collecting social media data and incorporating it into your goals. Here’s three tips on how to make the most of it, using X and Reddit as a source.
Data-Universe allows you to collect social media posts by hashtag and keyword on X, and subreddit and keyword on Reddit. But you don’t just want to pick any terms that you think are relevant, as not every phrase is uttered often enough.
Before you launch the Data Universe Data Collection job, check that the searches are meaningful to you. Go on X and enter your hashtags, words, or phrases into the search bar. If they return with regular, relevant information, that means they’ll be a great source for Data-Universe to utilize. You don’t need to read every post, just scroll through the search results to see if it’s appropriately populated.
For Reddit, your best move is to check out the subreddits you’re interested in to ensure they have an active community. After that, use Reddit’s search function to check if your keywords are used within the forum.
With Reddit, you get the option to collect via keyword, Subreddit, or a combination of the two. With just keywords, you can collect data from across the entire platform, not being limited to one specific place. This is good if there are many subreddits that discuss topics you’re interested in, especially if there’s no one hub for relevant discourse. For example, if your keyword is “LLM” (large language model) you wouldn’t want to limit yourself just to one ML forum - you’d want the whole of Reddit.
Choosing to just collect from a subreddit (without any keywords) is ideal if there is one specific place where users congregate to chat. For instance, if you’re launching a Bittensor subnet, you’ll probably get the most use out of collecting from r/bittensor_.
If you’re looking to capture a small portion of a large subreddit, then using a combination of a keyword plus a subreddit is ideal. As an example, if you were launching a blockchain project but were only interested in DeFi, you could collect posts regarding the keyword “DeFi” exclusively from r/cryptocurrency.
Note: Using only one keyword for data collection on X or Reddit is more effective than using multiple. Not only does Data-Universe efficiently pick up these posts, but it’ll be easier to query them using a large-language model.
After you’ve set your Data Collection job, you should see posts getting collected. It can sometimes take a few minutes-to-hours to pick up a sizable amount. After you’ve got a strong number, you can export the data into CSV or parquet formats. This will give you a document containing the raw data. You can also access a real-time updated stream of social media data using our Data Universe API.
If you already use a sentiment analysis service, you can plug this data into your app. However, if you don’t have one to hand (or your sentiment analysis service doesn’t allow the uploading of CSV/parquet files), you can actually run the data through an LLM, and ask it to build insights for you.
We recommend Claude for data analysis service, as it has a larger context window. However, most industry-standard models will be sufficient.
Tell the LLM what you’re looking for, and ask it to create graphs, metrics, and summaries of its findings. For instance, if you’re looking for people’s opinions on #AIArt, you can download your CSV file about the topic, upload it to an LLM, and prompt it by writing:
“Provide me the sentiment analysis overview regarding AI art. Track how positive or negative people feel about the topic, and draw charts that depict this. I’d like one chart that shows the overall sentiment, and one which tracks sentiment changes over the timeframe within the data. Give me a 1,000-word executive summary of your findings, with quotes from the posts.”
This will give you a breakdown of your data. If there’s anything which you’d like to dial down on, you can query the model to gain further insights.
It’s always enticing to look at positive sentiment, as it gives you strong signals of what people are fascinated and elated about. However, there’s much to learn from negative opinions. Oftentimes, people will take to social media to degenerate a service, or to critique a topic - this is extremely useful information as it can show you what pain-points people have, so you can either target their problems and build precise solutions.
Bear in mind that for some topics, you’ll incur a very vocal minority, which can skew your insights. You will need to use your own discretion to determine whether issues are being raised are a source of genuine concern, or whether they are outlier scenarios.
To action these tips, head to Data-Universe, and launch a data collection task either through our UI or via the API.
This is your gateway to low-cost real-time social media data-scraping. Try it out for free - the first $5 of credits are on us.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.