Does tagging improve discoverability? Results of my experiment
The results were surprising
Hey all,
Hope you’ve had an awesome week.
Six weeks ago, I wrote a smash hit post called How to Use Tags to Grow Your Substack. In it I picked apart the suddenly popular notion that adding tags to Substack posts was the reason some creators were getting massive growth.
I called bullshit. And Hamish McKenzie, co-founder of Substack did too.
In the week after I published that post, tags remained a hot HOT topic. And, I learned three more things about tags:
I saw data that showed increased weekly views for 5 Substack writers who had started doing strategic tagging.
Substack published new data showing that in 2025, the Substack app is showing readers posts from publications they are not subscribed to. Tags might play a role in that process.
I discovered a way to access a list of what I’d been calling pseudo-categories, but are now called ‘interests’.
That was enough information for me to reconsider: maybe tags were contributing to discoverability in Substack; perhaps strategic tagging could increase page views and traffic and maybe get me more subscribers.
It was time to run an experiment to find out.
The experiment
Just because one Substack expert said 5 of their premium subscribers got huge increases in traffic after they started tagging is not proof that tagging works. That same expert offers Notes boosts to premium subscribers too, so the growth could have come from that or from other changes they implemented after they joined the premium group.
I needed to see for myself. And get some of my own data to share with you.
My experimental method was simple. I would add 5 carefully chosen tags to every new post in both my publications, try to keep everything else the same, and see if my 30-day views and traffic improved compared to the prior 30-day period.
The tag strategy
I chose five tags for each publication and added the same five tags to every post.
For my food safety publication, I used two tags that are a perfect fit for my topic but are not Substack interests, and three tags that are Substack interests. Substack interests are topics you can choose from the Content Preferences settings of your personal Substack account.
For Pubstack Success, I chose five tags that are Substack interests: digital media, nonfiction, online learning, marketing, and entrepreneurship.
I used the same tags in all the posts I published from 1st April to 1st May.
During that period, I published the usual number of posts in my food safety publication and one fewer post than usual in Pubstack Success for a total of 15 posts.
The results
30-day views
After one month of strategic tagging, the 30-day post view stats were essentially unchanged in my food safety publication, increasing by just 0.2% to 22.9K. For Pubstack Success, the 30-day post views decreased by 17% to 26.9K.
Yes, that’s fewer views after I started tagging.
Traffic
The stats for traffic are even worse. Both publications appeared to have received significantly less traffic in the month after I started tagging. I say “appeared” to be worse because Substack stats are only presented graphically, not numerically.
External stats
I also reviewed data from Google Analytics and Google Search Console and found contradictory information. For example, Analytics recorded a huge spike in page views on a single day in the month after I implemented the tagging strategy for my food safety publication, and almost double the total page views compared to the month before. On the other hand, Substack’s stats for traffic showed no spike and no increase in post views.
Search Console recorded a 4.5% increase in impressions but fewer clicks compared to the previous month, with none of the ranked search terms being the same as any of the tags I’d chosen for this experiment.
Subscriber growth rate
Finally, I checked the rate of change of subscriber counts to see if either of my lists had grown faster after I implemented the tagging strategy.
In my food safety publication, the number of new subscribers I got in the month after implementing the strategy was 32% lower than the number I got the month before. My growth rate decreased significantly.
For Pubstack Success, I got more subscribers than I’d gained the previous month, 6% more, despite having fewer views and less traffic.
Comment
Before I jump into a deep dive of these results, it’s worth noting that my food safety publication is a ‘for work’ publication and Easter fell squarely in the middle of my experimental period. This could explain why traffic and subscriber growth were lower and slower for that publication over that period.
It’s also worth noting that most of my food safety subscribers do not find me through Substack, with 82% coming from outside the Substack universe. By contrast, only 2% of my Pubstack Success subscribers come from outside Substack. This means that any boost in Substack discoverability from tagging would be much more noticeable in Pubstack Success compared to my food safety publication.
What can we learn from this?
The aim of my tag experiment was to see if I could improve discoverability for my publications within the Substack universe. That is, would Substack users who are not followers or subscribers see my posts more often if I started tagging?
The answer is…. probably not.
Here’s a summary of what happened when I started adding tags to posts in my two publications.
30-day post views decreased significantly
Traffic decreased significantly
Subscriber growth rate slowed significantly for one publication and increased slightly for the other.
With fewer views and less traffic, it seems tags do not improve discoverability, although this might be confounded by the fact that Easter fell within the test period and I posted less often in Pubstack Success during the test period compared to the month prior.
The surprising result for me was the small but noticeable increase in subscriber growth rate for Pubstck Success after I started tagging. This publication gets most of its new subscribers from within the Substack universe.
How did I get more subscribers for Pubstack Success during the experiment when views and traffic were both down compared to the previous month? Was it because of tags, or was it due to other mysterious forces?
My guess is that it wasn’t because of tags, nor anything I did on Notes, nor because of the exposure I may have got from collabs, such as my participation in the 10K Secrets book, because these would all have resulted in more people seeing my pages and posts, which would have caused an increase in traffic and/or post views, not a decrease.
On the other hand, maybe my posts were being seen by more of the right people, and fewer of the wrong people, leading to more sign-ups from fewer views. Who knows!
The only other thing I can think of is that I might have got extra subscribers from recommendations, which create instant subscribers without page views. Unfortunately, the stats for recommendations are not detailed enough for me to explore that theory.
So, what do I think about tags after running this experiment?
My food safety publication, which gets most of its subscribers from outside the Substack universe, performed noticeably worse during the experiment, with less traffic and significantly slower growth in subscriber count. I doubt this has anything to do with tagging, but I think it’s safe to conclude that tagging did not result in growth for that publication.
Pubstack Success, a publication that gets most of its subscribers from within the Substack universe, also performed poorly during the experiment, with a significant drop in page views and less traffic compared to the previous month, likely due to fewer posts than usual. However, despite this, its subscriber growth rate increased by a small amount.
I don’t know how Pubstack Success got more subscribers with fewer views and less traffic after I started tagging. Maybe tags allowed Substack to target my ideal audience better. Or maybe it had nothing to do with tags, with the growth coming from recommendations. There’s not enough data to know for sure.
But those few extra subscribers are something, at least. And because tagging is low effort, I’m going to keep adding tags to my Pubstack Success posts, just in case it’s helping!
Okay that’s it from me for this week, have a good one,
Karen
P.S. If you love what I do, there’s now a way for you to show your appreciation. I’ve activated payments in this publication, but will be keeping all content free and open access (no paywalls). Become a paid subscriber to support my efforts and say thank you for three years of low-hype, sensible Substack wisdom.
Thanks for taking a look at this so empirically, Karen. And somehow you got through that whole post without once saying "I told you so!"
Great info, Karen! Thanks for doing the test and sharing the results. It’s disappointing that Substack doesn’t include tags in long form posts in searches and browsing, but not surprising. I’ve done a little testing myself using search and what stood out was the search got hits if the search keyword was in the title, the pub’s name, or the pub’s short description. My takeaway was forget tags (unless you want to group your own posts, and I do, so I use them for that reason), and instead, be strategic about your pub’s short description. I haven’t tweaked mine yet, but I will once my content broadens.