Build a small business empire you can be proud of

Welcome to Microblogger! First time here?
I know there's a lot going on, click here to Get Started.

6 Comments

Check Your Counter & Statistic Tracker’s Privacy Settings

top-secretA few years ago, I was part of a blog network called the Money Blog Network. The network consisted of half a dozen personal finance blogs and we banded together to try to secure group advertising deals. Many of the network’s members make up a list of the oldest personal finance blogs around and we did some fun things together. We sold a few advertising deals, we had forums that were pretty popular, but we eventually disbanded over one issue. (more on that in a second, first some background)

We joined forces so we could better sell ads. This was before the days of bigger brand-name aggregators, like Federated Media, partnering with a lot of smaller bloggers and trying to sell big blocks of ads. We were all relatively small and so creating a network was the quickest way to get bigger. Back then, we would split advertising revenue based on traffic. Our ad deals weren’t necessarily based on traffic but we felt that the split seemed fair since traffic is a good quantitative way to split revenue. A site with 200,000 pageviews a month should get more than a site with 100,000 pageviews, right?

As part of that split, we used a common hitcounter, Sitemeter, and left our statistics public. This help the person managing the advertisement deal keep track of traffic and split the income properly. After about a year, I realized I didn’t want Bargaineering’s traffic statistics public. Leaving my overall traffic statistics public wasn’t the issue, it was the referrer data that concerned me. As I explained my reasoning, which I’ll elaborate below, a few other members joined me and it created a fracture in the group, one that ultimately proved fatal.

It also marked the first time I began using competitive research to improve my blog.

So, why did I want to hide my Sitemeter data?

You Can Scrape It For Juicy Data

The biggest reason is that you can scrape Sitemeter’s pages for valuable keyword data. Knowing a competitor’s keyword data is like knowing their state secrets. It gives you ideas of what keywords you might want to try to target and how well you might do for the term, assuming you and your competitor are similar in popularity. If you see a keyword come up a lot, that’s one sign that the keyword is popular and that it’s driving a lot of traffic.

You don’t need to use third-party tools that guess at popularity, your ability to rank for it, or to provide an analysis – you have real world data right in front of you.

You learn how people are getting to the site. Does your competitor have partnerships that drive a lot of traffic? Are they leveraging social media in a way that you aren’t? Maybe they’re getting a lot of Pinterest traffic and you’ve never even looked at Pinterest. Maybe they’re using Stumbleupon with devastating effectiveness in a way that you haven’t thought about? Maybe Facebook is the way to go or there’s some social news site you’ve never heard of that drives great traffic.

These are all valuable insights made available when your stats tracker’s privacy setting isn’t set high enough.

Is It Ethical?

Yes, I believe it is – there’s no hacking or stealing or anything nefarious involved. This is all public data that the site owner made available.

The next question, and I think this is relevant if you’re in a niche where you have friendships with a lot of other bloggers, is whether this is the right thing to do. I was a little torn. On one hand, I don’t want to necessarily “take” from someone else. On the other hand, the internet is a very big place and the pie is large enough for everyone to get involved. Just because I target a keyword doesn’t mean I’m taking it away from someone else, I’m just joining the party. There are other competitors out there who will also be using this data and they will be far more aggressive in their pursuit.

As for actually scraping, I settled on a happy medium that satisfied me – I told everyone (that I had a relationship with) ahead of time that I’d be doing this. If they didn’t increase their privacy setting, then I assumed they were OK with it. (That’s why I’m suggesting that you increase your privacy level if you want to stop yourself from giving out this data!)

How Hard Is It To Do?

It’s not hard at all. All you need is a simple PHP script that loads up the Sitemeter page (curl) for a site’s referrals and parses all the data. Then, throw the script into a cron job, run every X minutes depending on the popularity of the site, and put the output into a .csv file. it took me less than an hour and the hardest part is figuring out the text to parse, which just takes time and testing. Please don’t ask for the code, I won’t be distributing it.

How Effective Is This Today?

Six or seven years ago, this strategy was devastatingly effective.

It was before the popular keyword tools were available (those tools are now mostly gone because Google threatened to revoke API keys if those tools published keyword data on individual sites) and when blogs could rank relatively high for long tail keywords without much effort.

Nowadays, with encrypted search, this is less and less revealing (and on Monday, it sounds like encrypted search will make this even less revealing). Encrypted search is the reason why you see the keyword “not provided” a lot in your organic search referral data and that’s because the trackers only get gibberish strings of data, not the simple to read strings of old.

Nowadays, it’s still useful but lacks punch and transparency. With Sitemeter, you can still see Entry pages, the first page someone saw when they visited the site. If you see the same ones pop up over and over again, then you know what pages are popular. You can start analyzing the keywords you see and begin guessing. This will get you to a cluster of keywords and that’s good enough. It’s not handed to you on a silver platter but it’s pretty darn close.

Also, you have fewer sites using stats packages that reveal this sort of information (Google Analytics doesn’t reveal anything, it’s the most powerful tracker you can get… and it’s free). Additionally, it’s harder to just write something and have it rank for a long tail keyword. It’s not impossible but it’s not as easy as it used to be.

That said, you can still use it to find useful and actionable intel but the juiciest part has been mostly neutered.

How to Change Your Sitemeter Settings

It’s quite simple really, just log in to your Sitemeter account, click on Manager and then Privacy Level. Anything above Normal will work.
sitemeter-settings

For this, it’s all about playing good defense. Closing this privacy loop won’t make your site better but it will protect it from any aggressive competitors.

If you’re not using Sitemeter, just make sure your hitcounter doesn’t reveal your juicy sensitive bits!

Sharing is caring! Tweet about this on TwitterShare on Facebook1Share on Google+0Buffer this pageShare on LinkedIn0Share on StumbleUpon0Email this to someone
The following two tabs change content below.

Jim

In 2005, I founded a personal finance blog (Bargaineering.com) that became successful enough that I quit my career as a software developer in the defense industry. It is my goal to share everything I learned so that you can do the same - build an online business that let's you pursue your passion.

6 responses to “Check Your Counter & Statistic Tracker’s Privacy Settings”

  1. Michael says:

    Yeah, you’re giving away a whole lot of valuable data if you don’t lock these things down. Even without a script, it’s relatively easy to troll someone’s stats and figure out valuable search phrases. Of course, with Google moving toward 100% encrypted searches, it’s getting harder and harder to figure out your own terms much less someone else’s.

    • Jim says:

      Yeah, often times you can just look with your eyes, especially on Entry Pages. With referrer logs, it was harder because the text is so long and the keywords fall off the table.

      Yes, encrypted search is a pain. What stinks is that Microblogger has a few visits from Google to the homepage… no idea what terms. You can’t even “guess” based on the Entrance Page because it’s the homepage. 🙁

  2. Michael says:

    Yeah, at least you can still see the landing pages for search visitors and make an educated guess about what’s driving your search traffic.

    • Jim says:

      Yeah, it’s just less concrete. I’d been reading about a way to use Webmaster Tools data to extrapolate, but even then it’s a bit of a guess.

  3. Adrienne says:

    Call me dumb here Jim but wouldn’t you have to have a Site Meter account in order to worry about this or not? Tells you what I know.

    Now also can’t you just look at the source code on the page to see what keywords they’re using or am I missing something else. I know it won’t tell you were the traffic is coming from but if you’re looking for the keywords they’re using then I would think that would be the easiest way. That I know how to do.

    Hey, I never claimed to be a genius at this stuff which is why I’m asking these dumb questions.

    Be gentle on me okay! lol…

    ~Adrienne

    • Jim says:

      Yes, you would need a Sitemeter account (I looked and you just use Google Analytics, so nothing to worry about on this front), which was a lot more popular several years ago.

      And you could look at the source and their Meta tags, but that’s just a hint at what they’re targeted, not what they managed to rank for. But you can use that as a start and use other tools (like SEMRush) to narrow it down.

      No one is a genius, the moment you think you are… then you’ll be made to look the fool. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *