Rapidly Iterating on Landing Pages with Mechanical Turk

GroupMail enables users to easily create email groups by forwarding existing threads. There is a built in viral component, where people create groups with their friends, and hopefully their friends create more groups with more friends, etc. However converting from my website is my first starting point. Up until now I didn’t have very much traffic, but enough to know that no one is converting and my landing page must be terrible.

If you’ve never used Mechanical Turk before, Mechanical Turk is an Amazon product where you can pay random people on the internet to do something for you for a fixed fee (I was offering 5 cents). It’s an incredibly cheap way to do market research, but the downside is there is some selection bias in who you’re going to get. I setup a survey which asks the following

  1. Go to https://www.groupmail.in. In your own words describe what this business does
  2. Would you use this service
  3. If so, give me an example of how you would use this service
  4. If not, why wouldn’t you use this service.

I’m only offering 5 cents per survey, so I got back some ridiculous answers. For example for question 1 about what this business does — here are some of the nonsense answers I got.

  1. Infrastructure to hold email groups in the cloud
  2. FUN
  3. deal with your mail

The good news is, most people who weren’t giving me garbage did know what this business does. There was also a small sample that confuses GroupMail with online marketing mailing lists like MailChimp.

So the problem is there is no quantitative benchmark for what is a good score here this would be amazing as a conversion rate but since these are paid survey takers it’s hard to anchor these results. Slightly less than half said they would use GroupMail . 24/102 people said they don’t use email groups, and 23/102 people said they didn’t see any value to the service. I don’t think there is much I can do for those people.

I was able to get some interesting qualitative information. Here are the interesting reasons for rejection.

  • I run my own mailserver
  • I would not be comfortable giving out my peers/co-workers email addresses to an external service, and also I can just create groups in outlook or gmail and make my own grouped messages that way.
  • Sending group emails is never difficult or time consuming. My standard email client handles this perfectly well.

I’m surprised that someone who runs their own mail server still wants to fill out my survey for cash, but there were a few key takeaways

  • The website does not inspire trust
  • People don’t understand how this is different than client side email groups
  • People don’t understand how this service is differentiated from google and yahoo groups
  • Marketing a service that provide marginal benefit but not a huge benefit is harder than I thought

Using Mechanical Turk to Benchmark against a competitor

I also wanted to baseline these results, so I created another survey from the perspective of a competitor, gaggle email. I asked the following questions

  1. Please go to https://gaggle.email/. In your own words, what does this company do?
  2. Would you use this service?

If they answer yes

  1. How would you use it
  2. Please go to https://www.groupmail.in. which is a free competitor to gaggle.email, would you use groupmail.in

If they answer no

  1. Why wouldn’t you use it
  2. Please go to https://groupmail.in, which is a free competitor to gaggle.email, would you use groupmail.in

This was pretty interesting to me because it allows me to benchmark the performance of my landing page against the performance of a competitor and find out what sets them apart from me.

Slightly less than half said they would use the service — which is basically the same performance as GroupMail.

Only 36 percent would switch to GroupMail — which is pretty damaging to me considering my service is free. Here are the reasons cited:

  • Seems more established
  • I liked the design of the page better
  • It seemed more legitimate
  • The website seemed simpler and more appealing.
  • gagglemail actually seems like it would be far easier to use which I like.

Wow, so I have a very tiny impact on people who don’t like gaggle.email.

One person got what I was going for!

Groupmail feels soothing to the nominal of all the painful google’s and gaggle just seems to add on to it.

Unfortunately it’s only one person.

Here are my takeaways from this exercise

  • Professionalism and trustworthiness matters, gaggle.email has that, groupmail does not
  • There were many comments about how gaggle and groupmail are both low value propositions compared to free alternatives. This is encouraging because it shows to me that you can build a successful business out of a marginal improvement (gaggle.email is profitable, and I would classify them as a marginal improvement over google groups)
  • In terms of benchmarks, GroupMail got a similar rejection rate to gaggle.email, so it’s not terrible, but there are definitely things that can be improved

Based on this feedback, I’ve made text changes to differentiate GroupMail from common alternatives, both client side email groups, as well as mailing lists from Google and Yahoo. I haven’t made any changes regarding trustworthiness or professionalism, as those require much more work.

read original article here