Clear, practical, and refreshingly honest. You've convinced me this is one shiny button worth pressing. I like how you balance the excitement with realistic caveats, making it feel achievable and useful.
I question if subscribers should be included in the test. I always open your posts because I want to read them - no matter the title. So I skewed the results toward the title I received regardless of its "excellence," didn't I? (I'm assuming I'm in the test group since my email was time-stamped 8:19 ET.)
I see what you mean, Cathy, about skewing results, and I was initially thinking the same thing. But after I put on my statistics cap I decided it's a non-issue.
The dataset is quite big - 320 people per group - so it's safe to assume there were also readers with similar behaviour in the other two groups. You effectively cancelled each other out.
Not only intructive but entertaining too. Thank you.
PS where do we get first-hand news about Substack updates etc. I keep finding out about such things second-hand (no offense). Or maybe I'm just not paying attention?
So I was excited to do a HL test this week, and I scheduled my email with the test. Then before it went out, I made some edits to the email, I re-scheduled it and I thought I checked the two headlines for the test, and by editing the post itself, the system wiped out my two different headlines and just sent the test with the SAME headline twice. Wahhhhh. And of course the other unused headline was the one that I suspected would be better. Sighhhh.
I’m delighted I was in the test group for the Shiny Button! It’s my fave. It definitely got my attention when I saw it in my app feed. I’m thinking this will be the winner! Love being part of the experiment. 😊
Oh no! Are we all going to end up with brilliant newsletters with boring titles? I'd rather stick to the shiny red button, even if it means fewer readers. We have to be true to our authentic selves and our art, surely? Growth isn't everything. Even if capitalism tells us it is.
Motivation to get to 200 subscribers. 😅
Clear, practical, and refreshingly honest. You've convinced me this is one shiny button worth pressing. I like how you balance the excitement with realistic caveats, making it feel achievable and useful.
Press the Shiny Button, and I sure did. I love that title! I would not have opened the email for the alternative suggestions.
Instead I would have yawned and moved on. Next?
That’s what I feared, Nancy. The other headlines were too boring. But tbh I don’t know if I can come up with multiple non-boring headlines every time!
Btw the boring headline won 😟
Seriously? Why? 🤔
🤷♀️
I ended up trying the title test. My alternate title won. I have to concede, it actually was the better title.🤔
Awe…vanilla won🥲
Press the Shiny Button is a beautiful headline - who wouldn’t?
I question if subscribers should be included in the test. I always open your posts because I want to read them - no matter the title. So I skewed the results toward the title I received regardless of its "excellence," didn't I? (I'm assuming I'm in the test group since my email was time-stamped 8:19 ET.)
BTW, I vote Shiny Button, as well. Love that!!
I see what you mean, Cathy, about skewing results, and I was initially thinking the same thing. But after I put on my statistics cap I decided it's a non-issue.
The dataset is quite big - 320 people per group - so it's safe to assume there were also readers with similar behaviour in the other two groups. You effectively cancelled each other out.
And thank you for always opening my posts 🤩🤩
Not only intructive but entertaining too. Thank you.
PS where do we get first-hand news about Substack updates etc. I keep finding out about such things second-hand (no offense). Or maybe I'm just not paying attention?
Hi Mike, for first-hand news, subscribe to the official publication of the Substack Team, called On Substack https://on.substack.com/ . And if you use Notes you can follow the Substack Team https://substack.com/@substackteam and co-founder Hamish McKenzie https://substack.com/@hamish
Thanks, Karen.
I would have EASILY guessed that "Is Headline Testing Worth It?" might be the winner. But I was wrong!!!
I know, right?!
So I was excited to do a HL test this week, and I scheduled my email with the test. Then before it went out, I made some edits to the email, I re-scheduled it and I thought I checked the two headlines for the test, and by editing the post itself, the system wiped out my two different headlines and just sent the test with the SAME headline twice. Wahhhhh. And of course the other unused headline was the one that I suspected would be better. Sighhhh.
Oh no, Larissa, that sucks. Although I quite liked the headline. Good to know that edits can break the system.
I'm looking forward to my test post next week. I'll be publishing results shortly afterwards 😁
I’m delighted I was in the test group for the Shiny Button! It’s my fave. It definitely got my attention when I saw it in my app feed. I’m thinking this will be the winner! Love being part of the experiment. 😊
Ah... looks like my fave, Press the Shiny Button was the winner! 💕 I read about this new feature yesterday, and can't wait to try it! Thanks Karen! 💕
Sadly no, the winner was the boring one about data-driven whatever 😕
Oh no! Are we all going to end up with brilliant newsletters with boring titles? I'd rather stick to the shiny red button, even if it means fewer readers. We have to be true to our authentic selves and our art, surely? Growth isn't everything. Even if capitalism tells us it is.
Agreed, Ali! In future, I'll only be testing non-boring titles. And if I can't think of any other non-boring ones I'll skip the test.
I have every confidence in your ability to think up many exciting titles, and I will definitely do the same! 💕