But did it work? Measuring outcomes of communication and other initiatives

August 5, 2011 at 1:37 pm Leave a comment

We all want to make a difference; but most of us hope no one will ask us to prove it. Measuring outcomes—or return on investment (ROI)—is key to managing projects or organizations well. No one can afford to throw away money on initiatives that don’t work. But measurement is often an afterthought or inadequate, taking an in-my-heart-I-know-it-worked approach.

Good outcome measurement need not be complicated. It’s easiest if it’s part of the plan from the start.

What’s the goal?

Every initiative and activity has a goal. Marketing communications aim to increase sales. Training aims to improve skill and performance. Other communication initiatives may aim to increase awareness, understanding or trust and commitment. The goal of most projects or initiatives is change. (The exception is crisis communication, where the goal is to prevent attitude change in the face of a negative situation.)

Changes fall into three categories:

  • Knowledge – awareness or understanding of information, skill level
  • Feeling – level of trust, commitment and engagement, organizational culture
  • Action – quality of performance (accuracy, speed, sales, service, safety, productivity), turnover and stress leave costs

Often there is more than one goal and they may be interrelated. For instance, action requires that you know how to improve performance and feel motivated to do so. While the gold standard of goals is a change in action or performance, it can be useful to measure knowledge and feelings as well. If you fail to get the performance boost management typically seeks, you need to know if the problem was in awareness, skill or motivation.

Where are we?

Making a difference assumes that you know where people were to begin with. That requires benchmark measures that establish a need for change. Sometimes it is simply a matter of identifying a relevant factor that the organization already tracks, such as absenteeism, turnover, number and length of stress leaves, safety incidents involving lost time, sales, service calls and complaints. Other measures are trackable but perhaps not currently tracked, such as errors on timesheets and other forms, time to correct errors, or number of contacts required to meet sales objectives. Even goals as fuzzy as employee or community engagement can be measured by, for example, counting comments posted in response to items on the organization’s website. Or, of course, you could do a survey. Of all these benchmark measurements, surveys (and interviews and focus groups) are the most expensive. Response rates can be low and are getting lower as people become “surveyed to death.” The cost is especially high with employee surveys, because management is effectively paying staff to be unproductive for as long as it takes to complete the survey, interview or focus group.

Intrusive vs. invisible measures

Measuring outcomes of initiatives can be invisible to the affected individual or group, or may be obvious to them when it requires effort on their part. Each approach has its advantages.

Intrusive measurement has the potential to be an agent of change in its own right. Surveys, interviews, focus groups and knowledge tests raise people’s awareness of the subject being discussed and get people thinking about it. The impact can even spread beyond those surveyed if they talk to others about the subject or if their changed behaviour influences others. Tom Peters said, “What gets measured gets managed.” I would add that what gets measured in this fashion gets seen as important. In some instances, the extra costs of intrusive measures can be worth it in terms of better outcomes for the organization.

Measurement that is invisible to those not engaged in the data collection process costs less and is generally not subject to such response biases as trying to look good to others by giving a socially acceptable answer.

Web analytics include details of webpage visits and length of stay, or poll responses (e.g., % of visitors who click “Yes” to “Was this article useful?”, or number of visitors who “Like” a post). Check inbound links to identify key stakeholders or audiences for your services. Comments posted can be analyzed for number and content. Information about registrations or subscribing to online sources can indicate deeper interest in a topic or trust in a particular source. In relying on web analytics, consider whether everyone you are interested in has equal access to the internet or organization’s intranet.

Human resources, accounting staff and supervisors may have information relevant to the outcomes you want to measure. For example, employee commitment to the organization can be measured looking at absenteeism and voluntary turnover rates. Quality of performance can be measured by looking at time to goal completion (e.g., sales, work placements, profitability measures) and error rates (e.g., time lost due to accidents, customer service costs).

Invisible outcome measures can be very creative, depending on the goal, the initiative or the situation. A critical, sometimes limiting factor, in their use is whether collection involves an invasion of privacy and breach of trust. “Big brother” perceptions will reduce engagement, which is particularly problematic if your initiative’s goal was to increase engagement. Keep collection and reporting as generic and anonymous as possible.

Final thoughts

Although not all efforts require financial measures of ROI to be calculated, increasingly stakeholders are demanding that organizations be able to demonstrate that their activities are effective and efficient in meeting desired outcomes. In tight times, those who can produce solid evidence of effectiveness on demand are more likely to survive and thrive than those who can’t.

Entry filed under: Management, Research. Tags: , , , , .

Packing 101: How to go anywhere with only what you can carry Management is neither leadership nor supervision

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 12 other followers

Recent Posts

%d bloggers like this: