Response to a Deliverability Rant

Justin Foster from WhatCounts, an email service provider based in Seattle, wrote a very lengthy posting about email deliverability on the WhatCounts blog yesterday.  There’s some good stuff in it, but there are a couple of things I’d like to clarify from Return Path‘s perspective.

Justin’s main point is spot-on.  Listening to email service providers talk about deliverability is a little bit like eating fruit salad:  there are apples and oranges, and quite frankly pineapples and berries as well.  Everyone speaks in a different language.  We think the most relevant metric to use from a mailer’s perspective is inbox placement rate.  Let’s face it – nothing else matters.  Being in a junk mail folder is as good as being blocked or bounced.

Justin’s secondary point is also a good one.  An email service provider only has a limited amount of influence over a mailer’s inbox placement rate.  Service providers can and must set up an ironclad email sending infrastructure; they can and must support dedicated IP addresses for larger mailers; they can and must support all major authentication protocols — none of these things is in any way a trivial undertaking.  In addition, service providers should (but don’t have to) offer easy or integrated access to third-party deliverability tools and services that are on the market.  But at the end of the day, most of the major levers that impact deliverability (complaint rates, volume spikiness, content, registration/data sources/processes) are pulled by the mailer, not the service provider.  More on that in a minute.

I’d like to clarify a couple of things Justin talks about when it comes to third-party deliverability services.

Ok, so he’s correct that seed lists only work off of a sample of email addresses and therefore can’t tell a mailer with 100% certainty which individual messages reach the inbox or get blocked or filtered.  However, when sampling is done correctly, it’s an incredibly powerful measurement tool.  Email deliverability sampling gives mailers significantly more data than any other source about the inbox placement rate of their campaigns.  Since this kind of data is by nature post-event reporting, the most interesting thing to glean from it is changes in inbox placement from one campaign to another.  As long as the sampling is done consistently, that tells a mailer the most critical need-to-know information about how the levers of deliverability are working.

For example, we released our semi-annual deliverability tracking study for the first half of 2005 yesterday, which (download the whitepaper with tracking study details here or view the press release here).  We don’t publicly release mailer-specific data, but the data that went into this study about specific clients is very telling.  Clients who start working with us and have, say a 75% inbox placement rate — then work hard on the levers of deliverability and raise it to 95% on a sampled basis, can see the improvements as their sales and other key email metrics jump by 20%.  Just because there’s a small margin of error on the sample doesn’t render the process useless.

Second, Justin issues a big buyer beware about Bonded Sender and other “reputation” services (quotes deliberate – more on that in a minute as well).  Back in June, we released a study about Bonded Sender clients which showed that mailers who qualified for Bonded Sender saw an average of a 21% improvement in inbox delivery rates (range of 15%-24%) at ISPs who use Bonded Sender such as MSN, Hotmail, and Roadrunner.  We were pretty careful about the data used to analyze this.  We only looked at mailers who were clients both before and after joining the Bonded Sender program for enough time to be relevant, and we looked at a huge number (100,000+) of campaigns.  Yes, it’s still “early days” for accreditation programs, but we think we’re off to a good start with them given this data, and the program isn’t all that expensive relative to what mailers pay for just about everything else in their email deployment arsenal.

Finally, let me come back to the two “more on that in a minute” points from above.  I’ll start with the second one — Bonded Sender is an accreditation program, or a whitelist, NOT a reputation service.  Accreditation and Reputation services are both critical components in the fight to improve inbox placement of legitimate, permissioned, marketing emails, but they’re very different kinds of programs (a little background on why they’re important and how they fit with authentication here).

Accreditation services like Bonded Sender work because, for the very best mailers, third parties like TRUSTe essentially vouch that a mailer is super high quality — enough so that an ISP can feel comfortable putting mail from that mailer in the inbox without subjecting it to the same level of scrutiny as random inbound mail.

There are no real, time-tested reputation services for mailers in the market today.  We’re in the process of launching one now called Sender Score.  Sender Score (and no doubt the other reputation services which will follow it) is designed to help mailers measure the most critical levers of deliverability so they can work at solving the underlying root cause problems that lead to low inbox placement.  This is really powerful stuff, and it will ultimately prove our (and Justin’s) theory that mailers have much more control over their inbox placement rate/deliverability than service providers.

Where does all this lead?  Two simple messages:  (1) if you outsource your email deployment to an email service provider, pick your provider carefully and make sure they do a good job at the infrastructure-related levers of email deliverability that they do control.  (2) whether you handle email deployment in-house or outsource it to a service provider, your inbox placement rate is largely in your control. Make sure you do everything you can to measure it and look closely at the levers, whether you work with a third-party deliverability service or not.

Apologies for the lengthy posting.

In 2005, I wrote a post called Unfolding the Map in which I looked at these two seemingly opposing philosophies from successful entrepreneurs:

• If you don’t have a map, you can’t get lost
• If you don’t have a map, you can’t get where you’re going

and tried to combine them when thinking about product roadmapping. The same contradiction and combination could be applied to anything, including coaching and development.

That’s why I was excited to read my friend Matt Spielman’s new book, Inflection Points: How to Work and Live with Purpose. Matt worked at Return Path twice over the years — first as employee #3 (more on that in a minute) and then over a decade later as CMO. We live near each other and know each other’s families. I’ve been lucky enough to see his career unfold and develop into what it is today, a flourishing coaching business called Inflection Point Partners that helps clients tremendously…and that also feeds Matt’s soul.

When I first met Matt and he joined me and Jack to launch Return Path in 1999, he was fresh out of business school and focused on sales and marketing from his prior career in investment banking. Our idea was that he would do the same for us as we got our product in market. But as I started focusing more on what kind of company we wanted to build and how to get there, Matt became my leading thought partner on those topics. When we got to about 25 people, he and I created a new role for him — head of Human Capital and Organization Development. While a bit clunky, that title meant that Matt was the principal person helping me create at small scale what we later branded our People First philosophy. That philosophy and the practices we developed out of it led to 20 years of a strong track record of investing in people and helping over 1,300 colleagues grow their careers by being simple, actionable, and broad-based in the way we handled feedback and development planning. This started back in 2000.

Matt’s book puts the ethos that I saw percolating over 20 years ago into a tight framework around his coaching methodology of the GPS (Game Plan System). The book is short and sweet and walks through both the philosophy and the framework in accessible terms. And while it’s true that you have to be open to new ideas, open to serendipity, and go with flow sometimes…it’s also true that if you have specific goals in mind, you are unlikely to achieve them without a focused effort.

I’ve written a lot about coaching lately between The Impact of a Good Coach and another recent post about a strong coaching framework about intentionality in Russell Benaroya’s book. In that second post, I noted that “While I have become less and less of a life planner as I’ve gotten older under the headline of ‘man plans, God laughs,’ I am a huge believer in being intentional about everything. And that pretty much sums up Matt’s book: If you don’t have a map, you can’t get where you’re going.

Firsts, Still

After more than 13 years in the job, I run into “firsts” less and less often these days.  But in the past week, I’ve had three of them. They’re incredibly different, and it’s awkward to write about them in the same post, but the “firsts” theme holds them together.

One was incredibly tragic — one of our colleagues at Return Path died suddenly and unexpectedly.  Even though we’ve lost two other employees in the last 18 months to cancer, there was something different about this one.  While there’s no good way to die, the suddenness of Joel’s passing was a real shock to me and to the organization, and of course more importantly, to his wife.

The second was that I came face to face with a judge in the state of Delaware for the first time around some litigation we’re in the middle of now.  While I can’t comment on this for obvious reasons, you never think when you decide to incorporate in Delaware that a trip to a courthouse in Wilmington is in your future.

The third, which can only be described as bittersweet, is that we had our first long-time employee retire!  Now THAT’S something you never think about when you run a startup.  But Sophie Miller Audette, one of our first 20 employees going back to 2000 and the sixth longest tenured person at the company today, has decided to retire and move on to other adventures in her already rich life.  A quick search on my blog reveals that I’ve blogged about Sophie three times since I started OnlyOnce 9 years ago (as of next week).  The first time was in 2004 when I quoted her memorable line, “In my next life, I want to come back as a client.”  The second and third times were in 2005 and were about the company’s commitment to helping to find a cure for Multiple Sclerosis, which Sophie was diagnosed with almost 10 years ago now.  Sophie has been an inspiration to many of us for a long time, and while we’ll miss her day-to-day, she’ll always be part of the Return Path family.  Picture of her, me, and Anita at her “retirement dinner” earlier this week below.

I always say that one of the best parts about being in this job for this long is that there are always new challenges and new opportunities to learn and grow.  The last couple weeks, full of firsts, proved the point!

• A VC: Scars From The Last Bubble
  Fred has a good posting on some of the downsides of having managed through the bubble bursting. I wrote about this (a little bit) last year in Ratcheting Up is Hard to Do (/2005/01/ratcheting_up_i.html), but Fred’s posti
  (tags: Entrepreneur VC Management)

• Feld Thoughts
  Brad has a good posting today about entrepreneur accountability — along the lines of my “Forecast Early and Often” theme. — /2005/11/notsocounter_cl.html
  (tags: VC Entrepreneur)

In From the Perimeter

I’m at the Direct Marketing Association’s annual massive trade show (DMA*05) in Atlanta.  While there are lots of things to potentially blog about, I think the most interesting one is the simplest.  When I started attending the DMA’s shows six years ago, the only interactive marketeing companies who exhibited were email vendors and the occasional sweepstakes company — and any interactive marketing company who did bother to show up was relegated to a small booth space in a corner of the trade show floor, away from the real action.  A friend of mine once told me it was easy for him to hit all the email guys at DMA — just walk around the perimeter of the room.

It’s 2005, and oh how things have changed.  The DMA put the “Interactive Marketing Pavilion” center stage this year, literally in the middle of the floor.  Besides Return Path, loads of other interactive marketing companies (and not just the email and sweeps guys!) have prime real estate at the show.  Within eyeshot of our booth are fellow email companies SilverPop, StrongMail, WhatCounts, Accucast, and ExactTarget, as well as analytics companies like Omniture, online ad companies like Blue Lithium, Kanoodle, and Advertising.com, lead gen companies like Cool Savings, and even a search firm or two.

The move is more than symbolic and more than just the fact that online marketing vendors have been around long enough to bid on better booth locations (although no doubt both of those things are true).  It’s representative of the way mainstream marketers now conduct business — increasingly online and increasingly multi-channel.  Online is another important part of the mix, not the stepchild.

Online marketing firms are now in from the perimeter, and we are happy to be here!

Counter Cliché:  And Founders, Too

This week, Fred’s chiche is that "the success of a company is in inverse proportion to the number of venture capitalists on the board".

I’d argue that the same statement is true of founders or management.

Boards help govern the company and watch out for shareholder interests.  Boards give outside perspectives and strategic advice to the company’s leadership.  Boards hire and fire the CEO.  And — more and more every day with large public companies — boards keep management honest.  How can these critical functions occur when a Board has too many members of the management team on it?  They can’t.  We’ve had outside directors at Return Path from Day 1.

I’m not advocating that Boards meet 100% apart from senior management.  On the contrary, our most productive Board meetings at Return Path are the ones where we have lots of management participation.  But execs present and discuss — and don’t vote — and they generally leave the last 30-60 minutes of every meeting for just the Board to discuss issues in private.  I’m also not advocating that CEOs don’t sit on boards or that the CEO never hold the Chairman role.  I think both of those items are critical to unify the watchdog function of looking out for all company stakeholders — shareholders, employees, and customers — at the highest level.

But while the success of a company may well be in inverse proportion to the number of venture capitalists on the board, that same success is jeopardized by too many execs, too.

That’s not just the title of a mediocre 1980’s sitcom starring Tony Danza, it’s a question I get periodically, including last week in an interview. A writer I know is working on an article on entrepreneurship and asked me, “Before you started your own business, how did you like working for other people?”

The question made me think a little bit. I know what she was asking — how I liked being the boss instead of working for one — but the way she phrased it is interesting and revealing about what it’s like to be a CEO. One of the biggest differences between being in a company and starting or running one is that you’re not working for a person, you’re working for many people.

As CEO of the company, I work for a Board and shareholders, I work for our customers, and I work for our employees. That’s how I approach the job, anyway.

Return Path’s Board of Directors is my boss, even though I’m one of the people on it. I report to the Board, and the Board is responsible for hiring and (hopefully not) firing the CEO, so technically, that’s my boss. The Board is also made up (for small private companies, anyway) of representatives of our biggest shareholders. As the main owners of the business, they are concerned with the growth, profitability, and overall health of the company, and they want to make sure we are building shareholder value day in, day out. That’s one very important perspective for me to have every day.

But I also work for our customers. I have to see myself as serving them — and more important, I have to steer the organization to believe that our customers are at the top of our food chain. If I do, then things will go well in the business. We will have the right products in the market at the right time to bring in new accounts. We will have a tremendous service delivery organization that wows customers and keeps them coming back for more. We will beat out our competition any day of the week. We will keep people paying our bills!

Most important, though, I work for our employees. This is very simple. An organization thrives because the people who make it up come to work inspired, focused, and productive. When they don’t, it doesn’t. I can’t wave a wand and make everyone happy all the time, but I try to focus a significant part of my time on making sure this is a great work environment; that the managers and executives are religiously focused on developing, managing, and motivating their teams; and that we’re doing a good job of communicating our mission, our values, and why each person’s job is important to the cause. This one’s the hardest of the three to get right, but it’s worth the effort.

Certainly, I don’t respond to each of my “bosses” every day as I would a direct supervisor, but in the long haul, I have to balance out the needs and interests of all three constituencies in order to have the organization be successful.

I’ve posted a lot over the years about feedback in all forms, but in particular how much I benefit from my 360 reviews and any form of “upward” feedback.  I’ve also posted about running a 360 process for/with your Board, modeled on Bill Campbell’s formula from Intuit.

I have a lot of institutional investors in our cap table at Return Path.  I was struck this week by two emails that landed in my inbox literally adjacent to each other.  One was from one of our institutional investors, sharing guidelines and timetables for doing CEO reviews across its portfolio.  The other was from one of our other institutional investors, and it invited me to participate in a feedback process to evaluate how well our investor performs for us as a Board member and strategic advisor.  It even had the Net Promoter Score question of would I recommend this investor to another entrepreneur!

The juxtaposition gave me a minute to reflect on the fact that over the 18 years of Return Path’s life, I’ve been asked to participate in feedback processes for Board members a few times, but not often.  Then I went to the thought that all of my reviews over the years have been self-initiated as well.  Just as it can be easy for a CEO to skip his or her review even when the rest of the company is going through a review cycle, it can be easy for investors to never even think about getting a review unless they get one internally at their firms.  I suspect many CEOs are reviewed by their Board, if not formally, then informally at every quarterly Board meeting.

It’s unfortunately a rare best practice for a venture capitalist or any other institutional investor to ask for CEO feedback.  I bet the ones who ask for it are probably the best ones in the first place, even though they probably still benefit from the feedback.  But regardless, it is good to set the tone for a portfolio that feedback is a gift, in all directions.

Running a Productive Offsite

A couple OnlyOnce readers asked me to do a post on how I run senior team offsites.  It’s a great part of our management meeting routine at Return Path, and one that Patrick Lencioni talks about extensively in Death by Meeting (review, book) – a book worth reading if you care about this topic.

My senior team has four offsites per year.  I love them.  They are, along with my Board meetings, my favorite times of the year at work.  Here’s my formula for these meetings:

–          WHY:  There are a few purposes to our offsites.  One for us is that our senior team is geographically distributed across 4 geographies at the executive level and 6 or 7 at the broader management team level.  So for us, these are the only times of the year that we are actually in the same place.  But even if we were all in one place, we’d still do them.  The main purpose of the offsite is to pull up from the day-to-day and tackle strategic issues or things that just require more uninterrupted time.  The secondary purpose is to continue to build and develop the team, both personal relationships and team dynamics.  It’s critically important to build and sustain deep relationships across the Executive Team.  We need this time in order to be a coordinated, cohesive, high trust, aligned leadership team for the company.  As the company has expanded (particularly to diverse geographies), our senior team development has become increasingly critical

–          WHO:  Every offsite includes what we call our Executive Committee, which is for the most part, my direct reports, though that group also includes a couple C/SVP titled people who don’t report directly to me but who run significant parts of the company (7-8 people total).  Two of the four offsites we also invite the broader leadership team, which is for the most part all of the people reporting into the Executive Committee (another 20 people).  That part is new as we’ve gotten bigger.  In the earlier days, it was just my staff, and maybe one or two other people as needed for specific topics

–          WHERE:  Offsites aren’t always offsite for us.  We vary location to make geography work for people.  And we try to contain costs across all of them.  So every year, probably 2 of them are actually in one of our offices or at an inexpensive nearby hotel.  Then the other 2 are at somewhat nicer places, usually one at a conference-oriented hotel and then one at a more fun resort kind of place.  Even when we are in one of our offices, we really treat it like an offsite – no other meetings, etc., and we make sure we are out together at dinner every night

–          WHEN:  4x/year at roughly equal intervals.  We used to do them right before Board meetings as partial prep for those meetings, but that got too crowded.  Now we basically do them between Board meetings.  The only timing that’s critical is the end of year session which is all about budgeting and planning for the following year.  Our general formula when it’s the smaller group is two days and at least one, maybe two dinners.  When it’s the larger group, it’s three days and at least two dinners.  For longer meetings, we try to do at least a few hours of fun activity built into the schedule so it’s not all work.

–          WHAT:  Our offsites are super rigorous.  We put our heads together to wrestle with (sometimes solve) tough business problems – from how we’re running the company, to what’s happening with our culture, to strategic problems with our products, services and operations.  The agenda for these offsites varies widely, but the format is usually pretty consistent.  I usually open every offsite with some remarks and overall themes – a mini-state-of-the-union.  Then we do some kind of “check-in” exercise either about what people want to get out of the offsite, or something more fun like an envisioning exercise, something on a whiteboard or with post-its, etc.  We always try to spend half a day on team and individual development.  Each of us reads out our key development plan items from our most recent individual 360, does a self-assessment, then the rest of the team piles on with other data and opinions, so we keep each other honest and keep the feedback flowing.  Then we have a team development plan check-in that’s the same, but about how the team is interacting.  We always have one or two major topics to discuss coming in, and each of those has an owner and materials or a discussion paper sent out a few days ahead of time.  Then we usually have a laundry list of smaller items ranging from dumb/tactical to brain-teasing that we work in between topics or over meals (every meal has an agenda!).  There’s also time at breaks for sub-group meetings and ad hoc conversations.  We do try to come up for air, but the together time is so valuable that we squeeze every drop out of it.  Some of our best “meetings” over the years have happened side-by-side on elliptical trainers in the hotel gym at 6 a.m.  We usually have a closing check-out, next steps recap type of exercise as well.

–          HOW:  Lots of our time together is just the team, but we usually have our long-time executive coach Marc Maltz from Triad Consulting  facilitate the development plan section of the meeting.

I’m sure I missed some key things here.  Team, feel free to comment and add.  Others with other experiences, please do the same!

Drawing the LineWe are having a bit of a debate at the moment internally around our Sender Score deliverability business about how to handle clients who are in businesses that are, shall we say, not exactly as pure as the driven snow.  As a company that provides software and services to businesses without a vertical focus, we are often approached by all sorts of companies wanting our services where we don’t love what they do.  Examples include:

Gambling
Tobacco
Neutriceuticals
Guns
Adult content or products

Our challenges are along three dimensions, each of which is a little different.  But common threads run through all three dimensions. 

Dimension 1:  Our deliverability technology platform.   Our basic technology is used by mailers of all shapes and sizes to preview their campaigns, monitor their deliverability, and analyze their reputation metrics.  It doesn’t deploy campaigns.  Do we care who the users are?

Dimension 2:  Our full service deliverability practice that comes with consulting and high-touch account management.  This service offering has an additional layer of complexity in that our employees work closely with accounts and their web sites.  We already allow employees to opt-out of accounts where they find the work objectionable.  But is that enough?

Dimension 3:  Our whitelist, Sender Score Certified. This one is even trickier.  On the one hand, our program has fairly clear, published standards.  We do a thorough qualitative check of the client’s web site and email program to make sure, among other things, that the program is opt-in.  We monitor the client’s quantitative reputation metrics in real-time to make sure its complaint rate is low, signifying that its customers like (or at least don’t mind) receiving its email.  On the other hand, this program is supposed to signify the best of the best for email marketing and newsletters, which is why it’s used by so many ISPs and filters as their standard for defining “good mail.”  And yet on a third hand (perhaps there’s some sort of herbal remedy that can help me with that problem), for many ISPs, our program is their only whitelist, so clients who are above board, even if in a grey industry, may have no other option.

So is it our place to legislate morality, or should we just focus on what’s legal and what’s not legal?  How much accountability do clients bear for content that shows up in their emails from advertisers?  For example, and I’m making this up, what do we do if a men’s health magazine that’s a client has links in its email newsletters that are placed by an affiliate network that click through to a pornography site?  What if the pornography in question is legal in one country but not another?  How much time and energy should we spend vetting clients before we take them on?  Or monitoring them around these issues once they’re a client?  Does it matter which product they’re using?

I’d love feedback from the outside world (or the inside world) on how we should think about and handle these issues.