benrobb

May 2009 Update:  I wrote this post over 2 years ago while I was still a Master’s student at BYU.  In the interest of full disclosure, I now work at Omniture.  Because I get the bulk of my traffic from search engines and variations on the Google Analytics vs. Omniture theme are always in my top 20 search terms, I wanted to make sure that my readers knew the current situation.  

Not everything I wrote about GA is still true.  There are lots of neat things that GA does now that it didn’t do before, probably the coolest of which is Advanced Segmentation.  My opinion is that SiteCatalyst is definitely worth the investment if you have the resources in place to take advantage of it.  I write this so you know the source of your information.  So now that this topic is taken care of, read on.

—

I know I’ve written about this before, but as I become more familiar with the different products, I continue to find things that I think are important. There is one main point that I’d like to hit with this post and that is report customization.

Google Analytics: minimal customization allowed
At first glance, Google Analytics appears to offer some very similar reports to SiteCatalyst. With the exception of the Commerce section of SiteCatalyst, Google offers very similar reports and data. And while both products allow you to change the date range, Google does not offer any customization beyond that.

Omniture SiteCatalyst: everything is customizeable
I may be overstating this a bit, but only slightly. Omniture allows you to easily create filters to filter through your data as well as to define your own metrics and have your reports generated in terms of your new metrics. These calculated metrics are based on other standard metrics and applies some math in the analysis. The standard metrics can be on a report-specific basis or a totals basis, so the percentages can be calculated from all metrics across the site or on a line item basis.

This one additional capability that Omniture provides might make it worth the price tag. The word on the street is that Omniture won’t be worth your time unless you’re paying them over $500 a month. And since Omniture charges you by the page view, you’re looking at some serious traffic.

I suppose I’ve probably mentioned this before, but Omniture is sponsoring a web analytics competition at Brigham Young University. I’m happy to announce that after a weekend of stewing my group was just informed that we’ve made it into the finals. Of the 40 or so teams that signed up originally, they’ve selected 4 to go into the finals. We just found out and the finals are only 3 days away.

There isn’t a whole lot I can say about the specifics of the project, but it was interesting to note the things that we did well and the things that we didn’t do so well on.

Observation #1
We began our presentation by identifying a business goal for the client’s web site. What is it that they’re trying to accomplish with the site? We then broke that objective down into a few success events that would demonstrate that the site’s users were progressing towards the end goal we had identified. Then we explained a few key performance indicators that would measure the customer’s progress towards these success events and goals.

For this particular client, their web site was a sales site and the objective was obviously to generate revenue (orders, etc). Our problem was that we started by stating revenue as the objective and spent the rest of the presentation talking about how to increase orders without going back to revenue.

I suppose that picking one and staying consistent would satisfice, but for a site that generates revenue, I think it wisest to phrase everything in those terms.

Observation #2
We weren’t as careful as we should have been in our use of terms. For example we used the phrase “conversion rate” a bit loosely and got called on it in the Q/A section of our presentation. We presented the conversion rate in terms of what percentage of overall orders each search term was responsible for when in reality the conversion rate should have been expressed as Orders/Search. Or perhaps even better, as Revenue/Search.

Our efforts to explain our meaning in the Q/A part following our presentation I think missed the mark, so we need to go back and make sure that what we say is getting the proper meaning across.

I suppose the wrap-up to this little stream of consciousness is that in data-driven analytics you’ve got to be specific, you’ve got to be consistent, and you’ve got to ask the right questions.

It’s been awhile since I wrote anything related to Web Analytics.  I’ve been involved in a competition sponsored by Omniture where we pour over a sites data and figure out what recommendations we can make to the company’s management to improve site and boost online revenue.  I can’t say much since I signed a Non-Disclosure agreement, but it has certainly been interesting.

I ran across this article and thought it was interesting in the context of what I’d been doing in the competition.  Anyone familiar with the way that web statistics are collected knows that there are issues with accuracy.  When spread over millions of visitors, the problems cease to be as significant, but they do exist.  The article highlights a few of the issues with some of the existing metrics and starts a discussion about what could replace some of the common ones that we use today.  I don’t want to rehash everything here, but will point you to What Will Replace the Page Impression? if you’re interested in some more reading.

So I had a momentary lapse into insanity over this last President’s day weekend and tried out a few other distributions of Linux. I began with Gentoo. I believe that compiling software from source is the worst possible software distribution model, but I figure it’s always wise to know the enemy (and I saw a friend running it with some really sweet 3d animations).

Gentoo
I figured I’d give it a try. I will say this for Gentoo. There is some excellent documentation available and my knowledge of Linux and operating systems in general was greatly enriched, but it’s not for me. I booted off their mini install CD, downloaded the stage3 tarball, unpacked it along with portage. I partitioned disks with fdisk, I formatted partitions and put the ext2, ext3, and reiser filesystems on them. I compiled kernels from source, and recompiled them when my ethernet drivers weren’t detected properly. I manually configured IP addresses, I compiled X11 and got the graphical environment working.

I had two problems. Problem 1: Despite excellent documentation, I still didn’t really know what to do when errors happened. The documentation (while very good) gives you many options, but I still ran into errors that just weren’t accounted for. Problem 2: Everything takes so long to compile. X took hours.

Having done this, I can see why Gentoo could be (and probably is) much quicker than the other mainstream distros because every piece of software on the system is compiled for the exact hardware it’s running on. If you hate software bloat, then Gentoo is the system for you, but after a weekend of experimentation, Gentoo is not for me.

Others
I also gave Mandriva a run. I downloaded the Mini iso of the 2007 distribution which promptly errored out in the install (it couldn’t find it’s own built in video drivers for my old nVidia GeForce4). I went back to Ubuntu. Sure it’s got a general kernel that still supports the i386 architecture, which means it doesn’t get top notch performance from my i686 processor, but it works. It installed with no problems, updated itself, and let me configure it to work remarkably like my MacBook Pro.

So when it all came down in the end. I still haven’t found a distribution that works for me as well as Ubuntu. I don’t have the time to spend to do upkeep on a Gentoo system, and all those other mainstream distros are just so full of stuff (I’m not going to burn 4 cds just to test it out). Ubuntu pulls through for me once again.

I’m sorry Ubuntu. It won’t happen again.

This was a strange exception that I struggled with for several hours this evening, but I have finally prevailed.

The Background
For my Information Architecture class, we’ve got to create a web service provider and a web service requester, this provided the initial impetus for my recent push into web services and the last few posts dedicated to the topic. Now our professor has published a web service that basically dumps his database out into an array of complex data types.

His database is a weather reporting database that has zip code, temperature, humidity, etc. The databse is small (currently 3 entries) and populated with completely fabricated data, this is a learning experience not an accurate forecaster.

Our task is to subscribe to his web service, and publish our own that will accept a string parameter as the zip code and return the single weather report for just that zip code. I decided that I’d just subscribe to his web service in my controller, find the appropriate weather report, create my own Struct for my web service, and push it out to my subscribers.

The Problem
Once I had received my professor’s SOAP object back, I stepped into it to retrieve the zip codes from each entry. When I found the matching zip code, I mapped his keys and values to my own with a line of code like this:

return WeatherReport.new(:cityzip => row.cityzip, :temperature => row.temperature, :etc => ad naseum)

There are ten different attributes that my object must return to my subscribers, so it was slightly tedious to put all these things together. I prevailed.

When subscribing to my own web service via WSDL, however, I received the following exception:

Exception: Cannot map WeatherReport to SOAP/OM

WeatherReport is the name of the Struct that I was using as my return type. I began meticulously checking all my code. I had already written code to subscribe directly to my professor’s web service, so I checked the SOAP object coming back from that. I used a copy of the same code to subscribe in my own web service and could not figure out what was going on (hours pass).

I finally ran across this email archive that had a fix, but not an answer.

The Fix
I had to change my code to this:

return WeatherReport.new(:cityzip => "#{row.cityzip}", :temperature => "#{row.temperature}", :etc => "#{ad naseum}")

The “#{row.cityzip}” means that the object inside the {} should be interpreted as a variable and it’s value should appear in the string instead of the variable name. It’s just a shorthand way of concatenating strings with the values stored in string variables (and in case you were wondering, yes, you must use double quotes or it won’t work).

Now the most interesting thing about this error is that row.cityzip == "#{row.cityzip}", meaning that the expression evalutates to true. For all intents and purposes, these two strings are exactly the same.

Peter (the guy with the fix) speculates on what might be the cause, but he was also using the a Struct (via SOAP4R), so I assume that the problem’s source may lie in that class. Given the dearth of readily available information on the topic, I hope you find this useful.