Monitoring keywords that are mentioned in Reddit posts and comments can be very useful to monitor your
brand, your products, your competitors, stay informed about specific topics... Go is a great language for
such social listening applications. In this article, we will see how to write a simple Go program that
monitors keywords on Reddit.
Social Media Listening on Reddit
Social listening on Reddit is an incredibly powerful tool for both individuals and
organizations across
a range of objectives. Here are several reasons why it's particularly important:
-
• Reddit is comprised of thousands of communities (subreddits), each with its unique interests,
language,
and culture. Social listening on Reddit can give you deep insights into what specific audiences are talking
about, their opinions, needs, and desires. This understanding can inform product development, marketing
strategies, and content creation to better match the audience's preferences.
- • By monitoring Reddit for mentions of your brand or products, you can understand the public
perception and address any misinformation or negative sentiment before it escalates. Engaging authentically
with the community can help improve your brand's reputation and turn detractors into supporters.
- • Reddit is often ahead of the curve when it comes to the latest trends, meme cultures, and
discussions.
By engaging in social listening, organizations and individuals can identify emerging trends early on. This
can be particularly beneficial for marketers, content creators, and product developers looking to stay ahead
in their respective fields.
- • Social listening on Reddit can also be used to keep an eye on competitors. Observing
discussions around
competing products or brands can reveal strengths and weaknesses you can leverage in your strategies. It
also provides insight into how competitors are perceived by the community, enabling you to position your
offerings more effectively.
- • Last of all, Reddit users often share their experiences with products or services, ask for
advice, and
seek solutions
to their problems. By actively listening, companies can identify and address these users' needs directly on
the platform. This can significantly enhance customer satisfaction and loyalty. Engaging the community in
authentic discussions can also humanize your brand, building stronger relationships with potential and
current customers.
How to Monitor Specific Keywords on Reddit?
Reddit exposes a couple of free API endpoints that allow you to get every new posts or comments
made on
the platform. These endpoints are not very well documented.
In order to get the last 100 Reddit posts, you should make a GET HTTP request to the following
API
endpoint: https://www.reddit.com/r/all/new/.json?limit=100
In order to get the last 100 Reddit comments, you should make a GET HTTP request to the
following API
endpoint: https://www.reddit.com/r/all/comments/.json?limit=100
The response of these API endpoints is a JSON object that contains a list of posts or
comments.
Here is a (truncated) example of a response I am getting when requesting the posts endpoint:
curl https://www.reddit.com/r/all/new/.json?limit=100
{
"kind": "Listing",
"data": {
"after": "t3_1asad4n",
"dist": 100,
"modhash": "ne8fi0fr55b56b8a75f8075df95fa2f03951cb5812b0f9660d",
"geo_filter": "",
"children": [
{
"kind": "t3",
"data": {
"approved_at_utc": null,
"subreddit": "GunAccessoriesForSale",
"selftext": "Morning gents. I\u2019m looking to snag up your forgotten factory yellow spring for the 509T. I need to source one for a buddy who lost his and I cannot find any available anywhere! \n\nIf one of you have the yellow spring laying around, looking to pay $50 shipped\u2026 \n\nTo my 509t owners, it\u2019s the \u201clight\u201d spring that comes in a plastic bag in the carrying case. \n\nThanks in advance ",
"author_fullname": "t2_2ezh71n6",
"saved": false,
"mod_reason_title": null,
"gilded": 0,
"clicked": false,
"title": "[WTB] 509T yellow spring",
"link_flair_richtext": [],
"subreddit_name_prefixed": "r/GunAccessoriesForSale",
[...]
"contest_mode": false,
"mod_reports": [],
"author_patreon_flair": false,
"author_flair_text_color": "dark",
"permalink": "/r/GunAccessoriesForSale/comments/1asadbj/wtb_509t_yellow_spring/",
"parent_whitelist_status": null,
"stickied": false,
"url": "https://www.reddit.com/r/GunAccessoriesForSale/comments/1asadbj/wtb_509t_yellow_spring/",
"subreddit_subscribers": 182613,
"created_utc": 1708094934.0,
"num_crossposts": 0,
"media": null,
"is_video": false
}
},
[...]
]
}
}
As you can see, Reddit returns an array of objects. Each object is a post or a comment
(depending
on the endpoint you requested) that
contains comprehensive details like the content of the submission, url of the submission, creation date and
time, etc. Most of them
won't be useful, and "selftext" is undoubtably the most important element as it contains the content of the post
or comment.
Results are presented in descending order. Please note that you cannot request more than 100
posts or comments at the same time.
Why Use Go to Monitor Reddit in Real Time?
The Go programming language offers several powerful features that make it an excellent choice
for retrieving Reddit posts and comments in real time. While other programming languages can also be used for
this purpose, there are specific advantages to using Go, especially when dealing with real-time data processing.
Here's why Go stands out:
First of all, Go's lightweight threads, known as goroutines, allow for efficient multitasking
and concurrency. This is particularly useful when you're fetching multiple Reddit posts or comments
simultaneously, as each fetch operation can be run in its own goroutine.
Secondly, Go's standard library includes a comprehensive and efficient HTTP client that
simplifies the process of making web requests. This is essential for interacting with Reddit's API to fetch
posts and comments. Go's HTTP client supports contexts, allowing for request timeouts and cancellations. This is
critical for real-time applications where you want to ensure your application remains responsive and is not hung
up on delayed responses.
Finally, Go is a compiled language, which typically means that applications written in Go are
fast and have a small footprint. This is beneficial for real-time applications that need to process large
volumes of data quickly. Go's garbage collector is designed to be efficient and to keep latency low, which is
crucial for maintaining performance in real-time data fetching scenarios.
A Simple Go Program That Watches a Keyword in Reddit Posts
Here is a step-by-step plan about making a Go program that monitors the keyword "kwatch.io" in
Go posts:
-
1. Define a struct Post to hold the relevant fields: selftext, title, and permalink.
-
2. Define a struct Data to hold the children field which is a slice of Post.
-
3. Define a struct Response to hold the data field which is of type Data.
-
4. Create a function fetchPosts that sends a GET request to the Reddit API endpoint and decodes the
JSON response into a Response object.
-
5. In the fetchPosts function, range over the children field of the data field of the Response object
and print the selftext, title, and permalink fields of each Post if the title or the selftext contain
"kwatch.io".
-
6. In the main function, call the fetchPosts function every second using a ticker.
Here is the Go code:
package main
import (
"encoding/json"
"fmt"
"net/http"
"time"
)
type Post struct {
Selftext string `json:"selftext"`
Title string `json:"title"`
Permalink string `json:"permalink"`
}
type Data struct {
Children []struct {
Data Post `json:"data"`
} `json:"children"`
}
type Response struct {
Data Data `json:"data"`
}
func fetchPosts() {
resp, err := http.Get("https://www.reddit.com/r/all/new/.json?limit=100")
if err != nil {
fmt.Println(err)
return
}
defer resp.Body.Close()
var r Response
err = json.NewDecoder(resp.Body).Decode(&r)
if err != nil {
fmt.Println(err)
return
}
for _, child := range r.Data.Children {
if strings.Contains(child.Data.Title, "kwatch.io") || strings.Contains(child.Data.Selftext, "kwatch.io") {
fmt.Println("Title:", child.Data.Title)
fmt.Println("Selftext:", child.Data.Selftext)
fmt.Println("Permalink:", child.Data.Permalink)
fmt.Println()
}
}
}
func main() {
ticker := time.NewTicker(1 * time.Second)
for range ticker.C {
fetchPosts()
}
}
This program will fetch the last 100 new posts from Reddit every second (asynchronously) and
print the title, selftext, and permalink of each post to the console. You can do the same thing with Reddit
comments by simply changing the API endpoint URL.
Here are a couple of ideas about how you could improve this program:
-
• Retrieve the date and time of creation of the last post or comment and keep it in memory, next time
you fetch the posts or comments, only print the new ones
-
• Use Go contexts in order to cancel a request that is timing out
-
• Catch the returned HTTP code and retry the request in case of error
-
• Use proxies in order to rotate IP addresses and thus avoid being blocked by Reddit
-
• Make keyword search case insensitive
Conclusion
Monitoring specific keywords on Reddit is possible thanks to a fairly simple Go program.
Productionizing such a program can be challenging though. First because Reddit is good at
blocking you if you make too many requests, but also because their API endpoints return a lot of posts and
comments at the same time, which means that your Go program has to handle this high volume smartly.
If you do not want to build and maintain such a system by yourself, we recommend that you use
our platform instead: register on KWatch.io
here.
Arthur
Go Developer KWatch.io
KWatch.io