Web Analytics Rock Stars

Throughout my career in web analytics I have had the opportunity to work on web projects for some of the largest and most well-known websites in the world, and I’ve been able to do this from many different perspectives, including client-side, vendor-side, and advertising agency-side. While I expected these websites to be immature in some of the more advanced web analytics topics such as behavioral targeting, multivariate testing, and tagging, I didn’t expect for most of them to have problems with ongoing performance reporting and representation of data, which they did. These companies all struggled with most of the aspects related to putting together a dynamic and effective monthly report: extracting accurate data from their measurement tools, data organization and visualization, report formatting (including delivery vehicle), and analysis/insights/recommendations. Most of them were decent with analysis; however, because everything else took so much longer than it should they didn’t have the proper amount of time to spend on analysis, so even that suffered. Fortunately, the solution to this is relatively simple in explanation and not much more complex in execution: data quality, automation, and a consistent, accessible report format - what we call an Integrated Digital Marketing Scorecard.

At this point you’re probably making a decision as to whether or not you should read the rest of this article — I don’t blame you but I’d like to give you a few reasons to continue reading. While this topic has been written about and discussed quite a bit, I feel that it still has yet to sink in because I’m not seeing it take hold in the work that I do. And while I don’t profess to be a true EXPERT at anything (I have been practicing Web Analytics for about 6 years now), I am an intelligent, logical, and creative thinker and like to think that I just get it. And as previously noted, I’ve had the opportunity to do this type of work for lots of different websites from a multitude of perspectives — the most helpful being my time at a very large and well-known digital advertising agency. Most of what comes next are lessons I learned while managing Measurement Operations at that agency.

To formally define the problem, analysts are spending a much larger % of their available time extracting and formatting report data and not enough time on the analysis, insights, and recommendations - where the true utility of the data comes into play as it is converted to decision-making information. In my rough estimation analysts are spending about 75-80% of their time on the tactical reporting work and only about 25% of their time on the real analysis work. In a perfect world this would be reversed - spending just 25% of their time on tactical work and 75% on analysis.

As mentioned, the solution that we have come up with to address these problems is what we call an Integrated Digital Marketing Scorecard (or IDMS). “Integrated” because it is a single report that includes data from multiple data sources, “Digital Marketing” because it contains data from all online marketing and website measurement systems, and “Scorecard” because it is a visual display of key performance indicators on a single screen, providing user-driven comparison and progress metrics and used to make management decisions.

Let’s look at each of the problems individually, as well as how they are addressed with the IDMS…

Extracting accurate data from their measurement tools
At one point not too long ago in my Web Analytics career I was managing the monthly reporting for nearly 15 separate websites, all of which wanted their report as close to the beginning of the month as possible. Each of the sites had their own set of complexities and issues: tagging problems, custom report issues, communication breakdowns with clients, etc. There always seemed to be some issue with some data point each month that meant I could not fully automate my data extracts because I had to scrub the data in some way. This is where Data Quality comes in. At the core of any good, automated reporting solution is high quality data. It can sometimes take a good amount of money and/or resources and certainly process changes to achieve the quality of data needed to support this style of reporting, but from experience I can say that it will pay for itself in time saving and decision making support.

In this role I was so incredibly busy during the first few weeks of the month that I chose to only partially automate my data extractions and pull the remainder of the data manually. I justified this by telling myself that it would actually take longer to automate the query for a data point than to pull it manually. I also failed to consistently document how I would manually derive a certain data point and would need to reconstruct it and compare it to the previous month’s number to make sure I did it correctly. This was incredibly time consuming. This speaks to the second component of the solution, automation. The bottom line is that if the calculation of a data point can be done fully automated, always do it. It will save time and reduce the occurrences of errors compared to pulling the data manually. When we build an IDMS for a client our goal is for there to be as few steps as possible each month to update the data and populate all charts and graphs. In many cases we’re able to get this down to 4 steps: select a new month, click button to run queries, validate new data, click button to update tables/charts/graphs.

Data organization and visualization
Analysts struggle to find the best way to organize, visualize, and present the information in their reports. It is very rare that I’m impressed when an analyst shows me a report they authored that they feel is one of their best pieces of work (by the way, don’t bring an example report to an interview unless it looks like something you would see on TV - you know who you are). It amazes me the lack of attention to detail that is pervasive across the industry. Presentation of the report can, in some cases, determine its success or failure. If you have bad data to report, at least make sure it is presented professionally.

There are a few guidelines we follow when organizing and visualizing the data in an IDMS:

• Organize the data in a tabular or navigational format so that the user can page through different sets of data while remaining on the same screen.
• Group similar data points together into the different tabs and have them follow some type of logical progression (i.e. Acquire-Convert-Retain)
• Show a minimal amount of data on a screen, but allow the viewer to interact with the report to show historical data and compare it against more recent data.
• Only show one or two data points on a graph that are related — don’t overcomplicate it with 3 dimensions and multiple metrics, etc.

Report formatting (including delivery vehicle)
It can take a very long time to manually update all of the charts and graphs and screenshots necessary for a given report, we get around this by extending the automation to include updating the charts and graphs in the IDMS, not just run the queries. Most people don’t fully understand the formatting capabilities of Excel but there are a lot of fancy things that can be done with formatting and interactivity - especially in Excel 2007. Have you ever heard of the camera function? If not, Google it and I’m sure you’ll find a great use for it.

Speaking of Excel 2007, that is our preferred vehicle for delivering monthly reports. It supports the tabular navigation previously discussed, all historical data can be included within the file in hidden sheets, the Scorecard can be made to look like a polished web application, and most importantly, it’s in Excel - something that everyone has access to. Alternatively, tools like Excelcius can be used to build this same type of functionality but publish it as an interactive Flash object embedded within a PDF file. I also love the idea of delivering these via the web, but that would require some web development because as of yet there is still not an available tool that I know of that can WYSIWYG a Scorecard.

There are quite a few other report formatting topics that I could discuss, but to keep this relatively readable I’ll add just one more that truly sets the IDMS apart from other solutions — specific real estate is set aside in the Scorecard for the analysts’ insights, observations, and recommendations. This section updates when the viewer pages through the different tabs in the Scorecard so that it corresponds with the data they are viewing. This is where data is turned into information that can drive decisions - without this analysis a report is not incredibly useful for obvious reasons.

The best way to approach writing the commentary is to first understand the business questions that need to be answered every month. The questions shouldn’t simply be worded versions of KPIs like “How many visitors came to the site this month compared to next month?” — they should be business questions like “Which media tactics drove the highest engagement with product content?” These are the questions that will be answered in the analysis section every month, supported by the KPIs and data in the Scorecard.

So what does this look like when it’s all said and done? Something like this:

If you made it through this article, THANKS FOR READING. I sincerely hope that it was worth the time you spent reading it. If you’d like to chat further about this topic or others, please feel free to reach out to me via email at Jason.Widup@WebTrends.com. My business partner and I recently rejoined WebTrends (yes, we used to work there a few years ago) to start a Digital Marketing Optimization consulting practice where one of our core service offerings will be to design and develop Integrated Digital Marketing Scorecards like those described in this article. Happy Analyzing!

Posted in: Online Analytics by Jason Widup