Wednesday, May 12, 2010

Education in web analytics: Experience vs qualifications

This is the first of a couple of posts that loosely fit under the category of education in the web analytics field. This one looks at some of the courses and qualifications out there for web analysts, and the benefits of them versus everyday experience.

There are a number of courses that are available to web analysts looking to improve their knowledge. One of the most well-known is the UBC Award of Achievement in Web Analytics. Whilst being a comprehensive course, it's geared towards people with little prior knowledge of the topic, with the first module named an "Introduction to Web Analytics". There is also an extension course, the Web Intelligence Certificate from the UBC and UC Irvine Extension, which as a prerequisite requires the UBC Award of Achievement to have been completed, and focuses on coursework.

The WAA Base Camp consists of two days of training which allows the student to get a "solid foundation of online marketing analytics knowledge and authoritative course material in a workshop environment". Unfortunately, as with a lot of WAA content, it seems to be very US-centric, with no courses available outside of North America this year, and no previous workshops outside of there since September 2008.

One complaint about why people don't join the WAA is that there's no proof of the member's quality. This has been answered by the WAA with new certification. There's been some discussion about its validity, its availability to non-WAA members and concern around pricing and how to take it - see here for an example. The WAA have responded to these in attempt to make the issue clear. There's general agreement that it's tough, with two to three years of experience required; its aim is to test your analysis skills rather than your knowledge of a particular analytics package, and it has a quasi-MBA slant to it. In a recent post Stéphane Hamel discusses the benefits of taking the test.

The Google Analytics IQ test is proving to be quite popular. So much so that they've recently had to make it tougher, raising the pass rate to 80%. Obviously this is tied to one package, and has an 18 month shelf life.

There seems to be agreement that certification is better for those earlier on in their careers. Those who've been in the industry longer have less need to demonstrate and prove their skills. However, those with experience can easily be siloed - all too often I've seen adverts for web analyst positions which talk more about the need for experience with a particular analytics package, rather than concentrating on the skills required to do the job properly. Having a qualification like the WAA certification would demonstrate your ability to bring the necessary mindset and skills to the role to successfully carry it out. The individal could show their knowledge of the technical, marketing and statistical skill sets required for this job, rather than have to spell it out on a CV. The technical knowledge one would have acquired through gaining the qualification would go a long way to helping out with any package-specific issues that may be faced in a new role.

The downside is that many of these qualifications are expensive (especially if you have to travel to take them as Steve Jackson notes in his afore-referenced blogpost). But labour economics dictates that as long as prospective employers can be assured of the quality of the test, this should only go to highlight the candidate's quality (and self-belief) - that they were willing and confident enough to invest in themselves to set themselves apart from the market. That the qualification has to be re-taken regularly for such a changing market would also demonstrate the candidate's quality.

Finally, the obligatory UK naval-gazing section. Whilst in the US where there is more knowledge about web analytics industry, here in Blighty fewer companies and managers are familiar with it. They may well be aware of what a web analyst does and how they can help provide insight and drive action etc (although possibly not to the extent a similar position in the States would), but I suspect they wouldn't be so familiar with the role of the WAA, or have any knowledge of the varying types of qualifications out there. This then, for now, lessens the impact of obtaining one of these qualifications on this side of the pond, although hopefully this will change soon.


  1. Great post Lawrence! It is something I have been questioning and wanting to do myself. 'Questioning' because I dont think it will actually add any extra value to a future employer over experience and proven success.

    I have also been 'wanting' to do a qualification just because I feel like I have to. But on the flip side, there is a vast amount of free info readily available on the web and now with Twitter/LinkedIn it allows me to ask questions to 'real' experts and get quick responses back.

    Until certification carries any 'actual' weight I will still be 50/50. I just thingk actual experience and real life proven success carries more weight. I could be wrong.

    Someone with a WAA certification with no experience Vs someone with 15 years experience at 5 blue-chip ccompanies with no certification carries more weight does it not? Hmmmmm.....

  2. Thanks! I think that's the difference between some of the qualifications out there though - the UBC course is a "how to do WA", whereas the Certificate does require experience to be passed, testing the candidate on how they would answer this question and that. So I don't think anyone with no experience would be able to pass it. Thus your last choice is one between someone with 15 years experience at blue chip companies, and someone with at least 3 years experience in the industry who can prove they're up to the job. Whilst that doesn't answer the choice, it would certainly make it harder...

  3. This is very old argument, let me explain. Back when Windows 200 server and related products came out, there was a shortage of good skilled IT staff. Anyone with 2-3 years Windows NT4 or 2000 experience could command a very good salary.

    Lots of would be IT people saw this as a way to jump on the more money bandwagon and achieve high salaries.

    This is now happening, in the Web Analytics and Search Engine Optimisation arena. These people were called paper MCSEs, they knew what the books told them. They did know anything about networking and they didn’t do the job on a day to day basis.

    Don’t get me wrong I’m not saying that they cannot do the job I’m saying that practical experience is also required.

    I’ve worked for many blue chip companies in the past and if a consultant turned up on site the client wanted to know what experience they had.

    I myself hold a number of Google qualifications and the Google Analytics one being one of them as well as a number of internet marketing qualifications. The qualifications demonstrate that I know what I’m talking about but my clients still ask for recommendation and examples of my work.

    So experience counts!!!!!!!

  4. Lawrence,

    Great post here. I think that until the certification becomes a standard condition of employment, that it won't really catch on.

    I'd like to see more certifications for the other part of analytics: implementation, data collection and data quality.


  5. Background: I'm tutoring all 4 courses at UBC as well as "fundamentals of business analysis", teaching a full-semester, master-level course on web analytics, 20+ years of experience (15 on the web), member of the Board of the WAA and closely involved with the Certification, speaker at eMetrics and other conferences, and of course, consulting, WASP, maturity model...

    Anyway, all of this to say I think I know about the topic, but at the same time I'm probably biased :)

    You are right about UBC - it is mostly for people doing a career shift toward web analytics. Basecamp is quick, fast overview for entry-level analysts. BTW, there is a Basecamp during eMetrics London (and you can go just to that if you don't want to go to the conference).

    I'm honestly surprised by the reactions to the WAA Certification. As Winston mentioned, we are somewhat in a chicken and egg situation - people wait for the industry reaction but there need to be some people taking it and talking about its value (I did the real test at eMetrics SJ, waiting for my grade!). Even for experienced people I would argue that it makes sense to take the Certification - if you are experienced, the cost of the test is marginal - and experienced people are the best to talk and say "ho yes, it's worth it, and as an employer I will seek employees with this Certification" - as Vicky Brock did! John Lovett also did the test and commented he was surprised by the quality and depth of the test - and all those who took it in SJ, despite most of us are perceived as very experienced in the industry, found it valuable, challenging and worth our money and time.


  6. Winston - Perhaps I should clarify here - I'm not making the case that qualifications trump experience, far from it. Merely that those who have the experience shouldn't rule this particular qualification (WAA Cert) out, and that it would help them to stand out in the crowd.
    Rudi - Thanks. I agree - but as Stéphane says, it's a chicken and egg situation, so that might be a little while, especially in the UK.
    Stéphane - Thanks for the comment - I agree. The skills required to pass the test appear to set it apart from other qualifications out there and would certainly allow an individual to stand outfrom others, regardless of their experience (subject to having a minimum level required to complete it, hence it's a complement rather than a substitute).

  7. I took the test at the London eMetrics. It was very tough, especially the case studies which each felt like doing a whole day's work in 15 minutes!

    Honestly, after the exam I was extremely concerned that I had over exposed myself and was starting to regret taking it at all. The test really makes you push yourself, it worked me much harder than I expected. I challenge anyone to take the test and not be impressed with the breadth and depth of the way it interrogates your skills.

    I have taken vendor exams and they are good tool testers but in terms of real problem solving they do not compare at all. The WAA exam is much much tougher.

    Based on my experience, if you claim to be a web analytics professional then you should, once confident in your abilities, take this test. Be sure though, if you masquerade as an analytics ninja this test will floor you.

    I took it and passed, but it was hard work - good work WAA and thanks for my certification!



Real Time Web Analytics
Feedback Form
Website Feedback