Data-Analytics-and-Dirty-DataBy Brian Hansford, director and marketing automation practice lead for Heinz Marketing

Is the title of this blog post extreme? Not really.

Data management and data hygiene are probably the most overlooked components to a successful marketing automation initiative, right after content.

I recently learned of a large enterprise with marketing automation that was blacklisted by ISPs.  The blacklist status effectively shuts down all outbound marketing efforts right as they are heading into Q4.  This happened because of poor data management and data hygiene.  It can happen to any organization with marketing automation.  When it happens, heads roll because it impacts customer engagement and revenue.  The audit process for getting off ISP blacklists isn’t easy. Can you imagine being responsible and for this with your organization?

Building a plan with a budget can prevent these nightmare situations.

Here are eye-opening stats on B2B data quality:

According to Sirius Decisions

  • 25% of the average B2B database is inaccurate
  • 60% of companies surveyed had an overall data health scale of “unreliable”
  • 80% of companies have “risky” phone contact records

NetProspex reports in their 2013 State of Marketing Data

  • 64% of companies surveyed have “unreliable” data and 34% have “questionable” data
  • 61% of companies surveyed said 35% of their records were “incomplete”

How do CMO’s and Demand Generation Directors avoid getting their companies blacklisted? How do Modern Marketers ensure over 90% email deliverability month after month? How do companies maintain CAN-SPAM Act compliance? What sources of data are used to build, maintain, and append the database?  How do marketing automation teams keep out the spam traps and keep the good contact records up to date?  These are mission-critical questions that need a plan.  Operating without a plan puts marketing operations, reputation, and revenue at risk!

Here are steps I recommend as the starting point for effective management.

Establish a data management strategy
Do you have a data strategy?  Chances are you don’t.  Build the strategy and business case that a budget will support.  A marketing automation platform needs regular health checks and ongoing maintenance.  Define the standards for complete records and manage how the database grows and where records come from.  Define the minimum standards for deliverability rates.  Establish requirements for data providers.

Marketing Automation and CRM platforms need regular updates, health checks, de-dupes, and data appends. A database will have a natural decay of contact data because people change jobs, companies go out of business, and mergers happen.  All of these events directly impact the effectiveness of a database.  Keeping the database healthy is strategic to maintaining a strong revenue pipeline.

Check your database for a baseline of health.
Get a database health scan before implementing the strategy or randomly buying tools or hiring consultants.  A good health check should identify spam traps and determine record completeness at a minimum.  Don’t forget to check phone records as well.  Find a reputable organization like  Unlock The Inbox,  Reachforce or NetProspex to conduct a health scan for a baseline assessment.

Budget for data hygiene in 2014
Imagine the situation where in July 2014 your email deliverability tanks to 80% and pipeline opportunities decline. Not fun, right?  Data hygiene is an investment that builds marketing automation effectiveness and drives revenue!  Investments are typically needed to for tools and services regularly clean out spam traps, update and append data records, and perform detailed de-duplication.

CRM and marketing automation platforms have some basic tools for de-duplication and there are often free utilities or connectors that can serve as temporary solutions.  Finding service providers and platforms that can enhance the free tools are a prudent step, especially when starting a clean-up process.

Maintain Regular Maintenance and Updates
A database is like a car. Regular maintenance keeps the machine running smoothly and efficiently with the best performance.  Here are just some of the items to monitor and manage:

  • Deliverability – anything below 90% is a red flag.  Why are messages not delivered?
  • Opportunity pipeline growth – Are conversions and opportunities decreasing?  Many factors go into this but even the best content and campaigns will fall flat without a healthy database.
  • Reputable Data Sources and Partners – We all get the spam emails from the offshore list brokers that offer segmented B2B contacts for pennies on the dollar.  Don’t. Do. It.  Chances are cheap lists are loaded with questionable contacts scraped from websites by offshore sweatshops.
  • Monitor the soft email bounces – are they increasing over time? Why are soft bounces happening?
  • Monitor sender scores using services like ReturnPath. This is especially critical for marketing organizations using dedicated IP addresses for outbound marketing.  The higher a sender score, the better the reputation which helps ISP’s monitor and allow emails from trusted sources.
  • Prevent Duplicate Records – Use de-duplication platforms like RingLead whenever importing new data, or when performing regular updates.  This keeps records clean and can merge duplicate records in the CRM or marketing automation platform.
  • Append and Update Records – Keep records up to date with a data provider to ensure the proper taxonomy, addresses, phone numbers, email addresses, and syntax are correct.
  • Segment Old Records – Don’t delete old or outdated records completely.  I recommend placing old records in a quarantine or sandbox that is kept away from marketing and sales operations.  Even after de-duplication and data append, old records may be needed for an audit or opportunity research.
  • Contact cadence and governance – Who can send outbound communications, when, how often, and who receives the messages?  Set a clear policy that the entire organization understands.
  • CAN SPAM – Does your marketing operations team understand CAN SPAM requirements for the US?  What about Canadian and EU requirements? Ignorance is not the same as innocence when trouble arises.  The US Federal Trade Commission offers these guidelines for compliance.

Plan Wisely
Data management and hygiene is strategic and helps keep marketing automation and demand generation running strong.  Putting off the health checks and maintenance will directly impact revenue opportunities, customer engagements, reputations, and even whether or not someone keeps a job.

What is your plan for 2014?

  • Ryan Schwartz

    This is a fantastic topic, and definitely one that I imagine resonates across marketing and sales teams of all sizes.

    There are a number of challenges to data hygiene, and unfortunately some of the biggest challenges occur before you even begin implementing a solution. Data hygiene projects are usually large, cross-team, and ongoing. Solving data hygiene issues have many implications, numerous systems are often already integrated, and cleansing one system implies you have to clean the data across all the related systems, or determine how the integration of those systems will be affected if the cleansing doesn’t occur concurrently. This can lead to a question of ownership. If Sales Ops owns the cleansing of CRM data, Marketing Ops owns the cleansing of MAP data, and IT/Engineering owns the cleansing of related systems, the project quickly becomes massive (and let’s admit it, a bit scary). I think it’s easy to agree that high data quality, a de-duped environment, and good data hygiene are important, but how do we get there?

    I think the ideas in this article are fantastic and essential. But let’s take a step back. The first step is having all teams admit there is a problem. The challenge with data hygiene is that the problem grows over time, managers and employees adjust, and the obviousness of the problem becomes distorted by perceptions that the issue is manageable and not critical. This allows data hygiene projects to be de-prioritized by other critical projects the company is pursuing. So I think the first step, is proving there is a problem; using the SiriusDecisions stats above, interviewing Sales team members who are directly experiencing the inefficiencies, documenting the challenges with marketing segmentation, and highlighting the bloating (or understating) of the KPI metrics, especially those related to revenue forecasting, are a great place to start. With this information you can demonstrate that data hygiene is not a data issue, its a money issue, and good data hygiene directly relates to revenue. You can use this information to get executive sponsorship by the cross-team stakeholders, and get the project not only started, but funded.

    Once you have executive sponsorship, and the project is prioritized, you can address the issues of ownership. In some cases it may make sense to hire outside consultants to own the project, meet with the various stakeholders, and provide expert recommendations on how to address. In other cases, it can probably be handled internally. But I recommend that a project manager (singular) be named, because a project that involves multiple groups, even if cooperative, become difficult and daunting without a dedicated project manager.

    Lastly, data hygiene isn’t just about data, it’s about process and architecture. Achieving data hygiene is the direct result of correcting the processes that allow for poor data hygiene, building an architecture that supports a clean data set, and implementing the necessary tools and technologies to not only de-dupe data, but prevent duplicate creation. All three are pillars of data hygiene, and viewing each in a vacuum will prevent you from accomplishing a healthy database.


    • Brian Hansford

      Ryan, I really appreciate your feedback on the complexity of data health and how the entire organization needs to mobilize to support the effort as a strategic initiative.

      I just read an article this morning where Nate Silver talks about Big Data hype that seems relevant to this topic. Silver states: “None of this big data stuff is going to be a solution for companies who have poor corporate cultures where you have trouble communicating ideas, where you don’t have everyone on the same page. You need buy-in from all levels of the organization, you need C-level people who understand the value of analytics, you need analysts who understand what business problems are trying to be solved and what the organizational goals are.” Boom.

    • vanman12


      I agreed with your comment regarding data hygiene. I’m a DBA who have been going to school to get my MBA and for the first time in my life, I start to see the “business
      requirements” and not just the technical side of my current job.

      Data cleansing is a difficult process and many of the data cleansing, data transformation project I was involved in is often part of the building of a data warehouse. Understanding the data, being able to pick the proper tool for the job and having the support of upper management and having a project manager who understand all phases of the process are keys.

  • amandabatista

    So important! Brian, your title is spot on. Think about it: Any structure needs a sound foundation to build upon- data management is that foundation. Without it, you can’t build a house of process-oriented, engaging marketing that actually means something to your customers and buyers. Great tips here. Thanks, Brian!

    • Brian Hansford

      Thanks Amanda! I think marketers have grown accustomed to overlook the importance of data health. But they do so at great risk now. With a regularly maintained database the revenue engine will sputter and falter! Thanks for your comment!

  • Cherish Edwards

    Brian, this is such a great topic and your solution-oriented approach of planning and budgeting for data hygiene is on target. I believe that all too often there’s a set it and forget mentality when it comes to technology. I mean if something is working, why mess with it right? But we know that this is not the case. Eventually a lack of planning, attention, optimization and discipline will manifest itself in ways that cannot easily be undone. I do believe that many organizations are starting to realize that a maintenance process and the skill set to do it can return measurable results. One of the many benefits of a MAP is the instant and persistent data available that lets you analyze your tuning efforts. By focusing on your data — you are creating an opportunity to optimize. More importantly, you are creating an opportunity on a regular basis to have consensus on definitions, business rules, policies and procedures on your revenue generation efforts.

    What we’ve seen at Net-Results is that many organizations are turning to their agencies to meet this need. Agencies are increasingly providing highly trained specialists to their clients to train their staff and provide embedded specialized resources at their client offices to meet these needs. With the breadth of expertise that agencies have, along with their ability to attract top talent, it’s a synergistic relationship that’s a unique competitive advantage for both agency and client. Thanks for exploring this important topic.

    • Brian Hansford

      Thanks Cherish. I really appreciate your comment. Focus on data creates opportunities! I especially appreciate how you talk about business rules, definitions, and process. I think agencies are a great resource for companies to rely on to help with data health and ongoing optimization. It’s an investment that helps drive revenue and engagement. The sad thing is many companies don’t realize how important it is until a crisis happens. It pays to plan and budget for the effort.

      • Cherish Edwards

        As you know, experience matters Brian. Your insights will certainly help your clients prepare for and manage this important process.

    • Rhonda Wunderlin

      Well said… The challenge of the “set it and forget it” approach with technology with the accompanying result “Eventually a lack of planning, attention, optimization and discipline will manifest itself in ways that cannot easily be undone” is dangerously accurate.

  • Good post, Brian. One additional thought: companies can go ahead and automatically unsubscribe anyone who hasn’t engaged with the company recently, and then send them one last chance email. According to a recent MarketingSherpa study, this is the #1 thing companies can do to increase deliverability.
    PS :Why not tag Marketo in the post as well?

    • Brian Hansford

      Thanks for your comment Jon. Unsubscribing or segmenting out the non-respondents definitely helps focus the effort to improve overall performance.

      I have Marketo tagged in the blog! You can them above 🙂

  • Good post. We also avoid buying lists in most cases because it’s just too big of a “what if”, and that’s where spam trap emails often exist.

  • Michelle Ison

    I wish I could “like” this post more than once.

    • Brian Hansford

      Thanks Michelle! Hope we have a chance to meet at EE13!

  • Rhonda Wunderlin

    Great post Brian! I think many of us monitor what’ been done, i.e. monitoring deliverability, sender score, bounces etc… While this is needed, it is a reactionary approach. I like the proactive points you make here, like monitoring database health regularly, appending & updating, identifying inactive contacts and developing a strategy for re-engaging them, monitoring contact cadence and instituting a governance policy. Once we get better at being proactive with our data we’ll increase deliverability, improve sender score and reduce bounces. In other words, spend time preventing the fire and you’ll have less need to worry about putting the fire out. 😉

    • Brian Hansford

      I’m so honored you like my post – you have a high bar! 🙂 I absolutely agree about being proactive. Sadly people wait for the crisis to develop. I think so much of that is because the database isn’t viewed as a strategic asset. Maybe a few more high profile blacklists will send a message that helps CxO’s focus. Once the initial clean up is done, and as long as the database is set up properly, data health isn’t hard. And it offers fantastic ROI. (Or catastrophic negative-ROI when not done.)

  • Meetul Shah

    Great topic Brian! This topic is bit close to my heart.
    Having suffered the data quality issues at all my companies and its tumultuous impact on our sales and marketing! I think problem is two folded at least forSMB – there is a lack of process and proactiveness on the company side but technology is behind on this as well. For example – if I want to validate all the contact in my MA or CRM database to make sure the completeness regarding email and phone number there is no good way for me to do it on a recurring basis. For email you still can do a soft (SMTP) validation but for phone only way for me to validate this number belongs to person x is using human – which is very expensive and cost prohibitive. How has your customer done this successfully
    and cost effectively?

    • Brian Hansford

      Thanks Meetul. I think the database tools are mostly available as batch updates. There are some tools you can run that are platform specific which can be automated close to “real-time” operation. For example, with Eloqua there is a contact “Washing Machine” program, along with other normalization tools that users have created. Tools like RingLead help eliminate the duplicates and do it very well. But to append the data with incomplete records, we use services from companies like ReachForce or NetProspex. They both provide services to update the phone numbers using their database updates. These updates can’t always be done in real-time, which I think is the data nirvana we all want. But it’s better than letting the database go for years without any health service at all.

  • When you have an internet company you should have seo services at thhe
    same time to avoid future dangers which might mar the progress
    of your business.The end results typically arre not instantaneous alongg thhe lines of Elements howedver successes will hafe to be detectasble once 3 months
    upon a program. Imagine not having to spend hours cleaning thhe sides and bottom of ykur swimming pool.

  • Whats up very nice blog!! Guy .. Beautiful ..
    Superb .. I will bookmark your blog and take the feeds also?
    I am happy to seek out a lot of helpful information right here within the publish, we
    need develop extra strategies on this regard, thanks for sharing.
    . . . . .