Wednesday, March 24, 2010

Free versus paid tools: A review

I've been mulling this one over for a while now. I've worked with paid and free tools before, but in my current role I use both Google and Yahoo Analytics on my company's sites. However, for all their strengths, there are some things that the free tools can't provide me with. Should I fork out the company's cash to bring in a paid solution? This post aims to provide an overview of the strengths of the various tools, look at the two sides in the paid / free tool discussions going on in the Digital Analytics world, and start me on the long process of making up my mind!

Free Tools

Some free tools are big players in the web analytics market. In the WAA 2009 Outlook Survey 30% of respondents used a free vendor solution, whilst a recent Forrester report stated that 53% of enterprises surveyed used a free solution as their primary tool, and 71% use one in some capacity.

There are currently two main players in the free tools market - Yahoo! Analytics and Google Analytics. The latter is freely available and the most popular free tool, being used to track 28% of all websites. Yahoo! Analytics is not so easy to acquire, and thus isn't so widespread. There are, of course, other free tools out there such as Woopra and GetClicky, which focus more on simple real-time reporting, and as such will be excluded from this post. Whilst the reporting available in Yahoo! Analytics is more advanced than Google Analytics, is it an "Enterprise Class" solution? What makes a tool "Enterprise Class"? I've spoken to a number of people who believe that Yahoo! Analytics is strong enough to compete with the paid tools out there; not many people would say the same of Google, at least for now.

First off, let's compare the two free tools. James Dutton has a fantastic post analysing the two. His analysis breaks the two tools into various categories, and scores them accordingly. Yahoo! comes out better than Google for most, quite often by a large amount. User management, code customisation, report bookmarking and filtering all fair better under Yahoo! than Google. However, Google has a much better support network, both in terms of documented help files, as well as forums and online guidance. Campaign tracking is also better for Yahoo! than Google, as is path analysis, and custom report design.

James' post is so comprehensive there's little, if anything, that needs to be added. In my experience Google is the more user-friendly of the two, although I've had more experience of it. However, Yahoo! Analytics real strength is its adaptability, allowing the user to amend reports on the fly, drill-down into the data and then link to related reports. With its limited help facilities it might take you a while to figure out how to do it, but once you have you're away.

When people talk about "Enterprise Class" solutions, what do they mean? Essentially the ability to tailor your tool to allow you to measure your data how you want to, be it custom segmentation, metrics or dimensions, or the reporting itself. In this post Eric Peterson discussed Google and Yahoo! Analytics, and concluded that "I don't think that Google Analytics is appropriate for free-standing use within the true enterprise". Yahoo! Analytics, however, "is an Enterprise-class solution," according to Eric, designed to support business with custom data collection, reporting and segmentation needs. Nevertheless, whilst Yahoo! may have the edge for the two main free tools, don't discount Google altoghether. Google have clear plans to become "Enterprise Class", and its improvement over the last few years has been astounding, moving from a basic tracking mechanism to a tool which is now pushing the boundaries of the industry, and forcing established companies to justify their fees, and in some cases catch up. Analytics intelligence, custom alerts, segmentation, advanced analysis, Mobile, custom variables - the list is long and distinguished. Whilst many tools have these features, no provider has released so much so quickly. And, true to Google's ethos, the tools are easy to use and interpret.

Paid Tools

So that's the free tools covered. But what distinguishes them from their paid competitors, other than the price tag? And just how important are these extra features?

A recent analysis by Forrester looked at the main players in the paid web analytics tools market, as well as the aforementioned free tools. Firstly they looked at the factors that respondents considered when selecting a vendor. The most important were: importance of data accuracy, and the flexibility of the tool, both in terms of servicing the business needs, and reporting - an enterprise tool!

They then took these factors and others, and ranked the various tools accordingingly. Briefly returning to the free tools, they ranked Yahoo! Analytics behind Google Analytics, although this was due solely to the product and corporate strategy weightings that they used; in terms or the current offering, Yahoo! was comfortably ahead of Google.

Returning to the paid tools, the big four (Omniture, Coremetrics, Unica and WebTrends) were ranked closely, and significantly ahead of the two free tools. They highlighted these tools' data handling, reporting and analysis, in addition to the ancillary marketing applications and plug-ins. Looking at the scores for these tools it is interesting how each have clear perceived strengths and weaknesses; there is no one clear winner from the list - all have at least one drawback. Furthermore, for each section there is either a clear winner, loser or both. If a company were just using this report to make their choice, it would be a simple matter of determining which category(ies) were most important to them and the choice would be clear(er).

Comparing the paid tools to the free ones for their current offerings (leaving out strategy rankings), it's clear that the paid tools are comfortably ahead of the free. However, the gap is smallest for the metrics, dimension & correlations and reporting categories. It's the data handling and service support that distinguish the two groups, where paying the money delivers the most return. Of course there are plenty of consultants out there who would be willing to help out with any technical implementation issues you may have for the free tools, no doubt for significantly less than the cost of an annual licence fee for a paid tool.

Let the fight commence

OK, so now we know the strengths and weaknesses of the different tools. The question is, does it pay to pay? We've looked at the two types of tools, now let's look at the two camps. On the one side of the fence are those who support the use of paid tools, claiming that the free ones just don't cut it. The powerful analysis engines that can be plugged into the main tool enable us to slice and dice the data in new and insightful ways, looking at non-aggregated data. Eric Peterson has recently been talking about the bifurcation of tools: Yahoo Analytics and/or Google Analytics are great for business users, but you need an enterprise tool (e.g Omniture Insights or WebTrends Visitor Intelligence) for the analyst to deliver really detailed insight. It's clear that these tools are more powerful than the free ones, and with the resource can deliver more insight than the free ones ever could. By combining a powerful tool with a free one, or setting up some simple dashboards in the powerful tool, companies can deliver the appropriate level of data to its employees.

On the other side there are those who claim that the free tools, whilst not being as powerful as the paid ones, provide the analyst with enough information. They state that it's as much about the quality of the analyst as the tool. The analyst needs to ask the right questions of the tool and correctly interpret the resulting data - simply exporting a data puke and mailing it out won't do anything, even if it did come out of a very flashy and expensive tool. In a recent post Avinash explains that setting up a powerful paid tool takes time, and you get a hobbled tool - you'll need to upgrade and get plugins to get all the information. This wastes time and money, and still won't present insights on a plate to you. Applying this to the 10/90 rule, if you've not got the resource to bring in a large enough analytical function that can manipulate these tools to the best of their ability, don't waste the money on an expensive tool.

Of course both Eric and Avinash have their reasons for being on their respective sides of the fence, for different reasons both would be out of a job if they didn't. In many ways though they agree with each other, but just choose to focus on different parts of the issue. No-one disputes that paid tools aren't better than the free tools, although, conversely, no-one would dispute that the free tools aren't catching up. The issue boils down to implementation and resource, both financial and human.

On the one hand, if a company has unlimited resource and cash, has the management in place to see through an effective implementation, and is not desperate for results, then the best route to take would be installing an expensive tool. Conversely, if a company has limited cash and human capital, little time or knowledge for installing and needs results straight away, it should be obvious what to do. Of course, most companies lie within these two extremes, so the decision is rarely that straightforward. What is clear though is that implementing a complicated web analytics tool is a drawn-out process, requiring not only the technical knowledge, but a great planning skills, and the full engagement of the various stakeholders involved. Simple it is not. To further complicate matters, you have to contend with the issue of trying to map what you need the tool to do to what it can do, not what you're told it can do. Unless you really know what your goals for the tool are, you risk being sold a pup.

Obligatory UK Naval-Gazing Section

At this point in a brief aside I'd like to focus on the UK, where web analytics has been acknowledged but not engaged yet for most companies. Obviously there are exceptions, and the blue-chips here are ahead of the rest of us. But the fact remains that for most companies, senior management are aware of web analytics and the power it has to drive through change and appreciate the need to investigate it, but have not been convinced of the benefits of fully implementing it. Most companies don't have a full time web analyst, let alone a team, and haven't allocated the money to buy in a tool. This will happen, but we are still at an early stage in the cycle. Effectively we are being asked to prove what can be done with a free tool before we shell out £££s on something better.

So what's the conclusion to all this? As any guru or university student will tell you, the correct answer is "it depends". If you've got the resources (money and people) to justify a paid tool, then you might reap the benefits as part of a medium to long term plan to improve the analytics of your company. But it requires dedication, good management and focus to pull this off. I've seen companies falter with their paid solution as the implementation has let them down, and people drown in the data and data issues. Focus on the human capital first, and apply the most suitable tool to it to derive the insight we all crave.

Friday, March 19, 2010

Web Analytics Wednesday London

Last Wednesday was the second WAW in London that I attended, and this time featured a presentation by Alec Cochrane using real data discussing how he's been working on amalgamating hundreds of Governmental websites (thousands if Bob Mitchell is to be believed), with a focus on page layout and the customer journey. As Miles mentioned in his post on the event, this was a really refreshing presentation as it wasn't a sales pitch, or indeed even cover sales analysis, these sites being informational. Who'd have thought the boiler replacement scheme could be so interesting? A great start to the evening.

The main thing that grabbed me about the evening was the friendliness of the participants (although the free beer might have helped!), and their willingness to not only talk to newcomers, but also to listen and offer advice. Speaking as someone who still thinks of himself as a newcomer to this field, it was fantastic to talk to so many experts and to pick up some ideas from them.

The side-discussion led by Bob on the future for the UK wing of the WAA was invigorating, with a variety of thoughts on what individuals wanted to get out of it. We talked about the education and engagement side of things, whilst issues around entry standards and cost were also raised.

It was a really enjoyable evening, despite having to share a train carriage for 2 hours with a Manchester-bound "character", and to anyone in the industry who's yet to attend a WAW event, I urge you to go. Did I mention free beer?

Friday, March 12, 2010


I've been thinking about the need for a stronger sense of community for the web analytics industry in the UK, and was inspired to write about it after the creation of a WAAUK Twitter account.

Miles Bennett recently wrote about the need for a WAA UK. He made a number of good points which got me thinking. Do we actually need a regional WAA? Does not the sharing of information over the internet through Twitter and blog posts suffice? I believe that whilst these are great, we would actually need a central point of contact where standards could be defined, and events organised so discussions can take place.

In a 2008 post Eric Peterson states that Europe (and thus the UK) are 2-4 years behind the US, and reckons that web analytics adoption is due to take off on this side of the pond.

The creation of a regional WAA would help Europe close the gap. But I think (as a Brit) that a WAA UK would be more helpful. To me there are two key reasons to this. Firstly, geography. One of the key problems of the existing WAA is the sense of isolation that Europeans feel from the US-dominated WAA. Secondly, there are varying levels of development across Europe, as Eric stated in his post. The creation of a European WAA would just be a microcosm of the current set-up, with people in the UK feeling isolated from what's going on in the continent, and differing levels of adoption. There wouldn't be the cohesion that us Europeans are currently craving.

Steve Jackson in his 2008 post essentially backs up Eric, agreeing that it was time for the EUWAA to be formed. He's since posted another article in follow up to Miles about the need for a Finnish WAA, looking at it from a budget perspective as much as anything - how can we have a global WAA if it runs as a not-for-profit organisation? We need the funds from a global organisation to drive the industry forward in less developed countries.

If we can find enough people in the UK to organise a national WAA, then to my mind that's the clear way forward. The WAA should encourage engagement and participation (as it already does well in North America). As someone who works in the web analytics industry, I want to communicate with others, get advice and share experiences with others; not only online, but face-to-face. I read recently that if you're not worried about not doing enough and keeping on top of things, you're not doing it right. I know I'm always trying to keep up and learn how to do things better. So count me in, even if it is only as the teaboy...


Welcome to the site. Have a look around.
Real Time Web Analytics
Feedback Form
Website Feedback