How Email Testing Paved the Way for 400% ROI in Direct Mail


How do you guarantee solid results in direct mail when you have no idea what the offer, price point, or creative approach should be?


Test first in email. Email test results can correlate to direct mail results when testing audience, subject line, offer, etc. Best of all, email has a 24 hour curve to predictable results—great for getting quick audience feedback.

No Time to Lose

I was a marketing director at a technology publishing company. The order had come down from the stakeholders – we had a 6 month window to take the market before the competition could come out with a matching product. The US Geological Survey had just released their quad topographic map data. This was something surveyors and engineers craved digitally. For mapping software companies, the race was on. We had a near-ready mousetrap. For a company used to mailing in the millions, the urge throughout the company was go big and mail now! Only one problem – we had no historical data to build our strategy.

Until then, we were mostly focused on the consumer market. We had not identified customers in the institutional or business marketplace. We also had no B2B or B2i results to draw from when attempting to land on audience, price point, offer and creative. With millions on the line, mailing millions out of the gate would have been beyond reckless.

However, we did not have the time to ramp-up with a solid predictable mail program. Each mailing took 6 weeks to arrive at predictable results, plus two weeks of production before the next launch. Eight weeks would have swallowed a third of the go-to-market window. The competition would have enough room to catch up. Market conditions required we go big fast.

Out of desperation, I convinced the stakeholders to allow us two weeks of email testing to determine the best price point. That led to a valuable opportunity for further actionable insights.

Staging the Email Test

It dawned on me that the subject line of an email is pretty much the same thing as the messaging on an outer envelope of a direct mail package. Body copy exists in both email and direct mail. In fact, you could closely correlate components of email to direct mail. Therefore, email could very effectively inform our initial control package.

Though we only had about 30,000 “power-user” emails in our database, we fortunately had reliable sources for business and institutional email lists. We lined up a test of about 20 different segments: Civil Engineers, Surveyors, Academics, Fire-Rescue, Land Use, Parks, Public Works, etc. and tested a variety of offers, subject lines, and creative over the course of one week. In setting up the tests we made sure of the following:

  1. Test one variable per cell
  2. Test large enough to get reliable indication


Though some of the segments in email were much smaller than the direct mail lists, we didn’t need statistically significant outcomes to march forward. Indications were plenty good—certainly better than shooting from the hip.

Results Were Predictable!

Tests led us to a $99 price point. It was clear that showing a real USGS map pulled better – even though it was somewhat uglier than our other fancy topographic images. We had a clear winner of a subject line. Also, five segments stood way out ahead of the others.

For our first mailing we were able to comfortably test in the hundreds of thousands. We grew to millions. Though we continued to execute a fairly complex test structure, the original email driven control remained superior wave after wave. Within 6 months we added over 42,000 new customers, yielding a 400% ROI.

Years later I had the benefit of meeting with the competition. Their gain in new customers was about 2,500 (about 16% of what we achieved). We took the market. Without that email testing first, I’m not sure we would have.

Lessons Learned

I repeated this process throughout my career with a number of direct marketing clients. Email continues to be a good indicator for developing direct mail control packages. In fact, most forms pf media correlate with email. Email is great for finding direction.

I also experimented wrapping direct mail deployments with email support, and have seen that greatly enhance results. (Perhaps a topic for a future post!).