The New York Times launched its paywall globally today, and Ive been putting the barrier through its paces. Visits from search engines like Google, for convoluted reasons, are limited to five per day. But as it turns out, visits from any Google site search or not appear blocked.
To test how the new paywall works and interacts with search engines I triggered it for myself in Firefox. At first, I had no luck. After clicking on well past the allowed 20 views per month, I couldnt get the wall to appear.
Restarting my browser (I was in Firefox) finally got the paywall to trigger. As was promised, I was still able to get past this when I came from links on social sites such as Twitter. In addition, I was able to get past the paywall when I came from an ordinary link, such as from River Of Times, which is a collection of New York Times links.
Blocked From Google, Yahoo amp; Bing
Search engines are also supposed to let you past the paywall, if youve hit it, but only for up to five visits per day. Thats a barrier that the New York Times has erected against whatever it considers to be a major search engine.
To discover which search engines got this special attention, I went to the Google.com, Yahoo.com, Bing.com, Ask.com and Blekko.com, found five stories from the New York Times in each of them, and clicked through.
When I tried to view more than five, I was blocked when coming from these:
I wasnt blocked when coming from Ask and Blekko. In addition, I wasnt blocked when I came from any of these country-specific versions of Google that I tested:
With Google UK (google.co.uk), I specifically tested to see if I could view more than five articles, in case the New York Time was perhaps applying a different limit for each version of Google. This doesnt appear to be the case. I was able to view more than five pages, when coming from Google UK.
Blocking Google Reader
Next, I wondered if the New York Times was only metering Google search-related visits (which is what the New York Times said it would do) or instead metering visits from other Google services. As it turns out, any Google service may get blocked.
For example, I subscribe to the RSS feed from the New York Times in Google Reader. Once the paywall was triggered, I could no longer view articles from Google Reader without a paywall block coming up.
This shouldnt happen. For one, the New York Times has said that links from social media sites wouldnt be blocked, and Id say Google Reader qualifies as a social media site. But more important, the New York Times has also previously said that ANY link from another site will let visitors through, unmetered, except in the case of search engines.
Google Reader isnt a search engine, but the New York Times is treating it that way. More crazy, if I subscribed to the RSS feed in my browser, the New York Times consistently let me view whatever I wanted.
Blocking Google Buzz
Few would argue that Google Buzz isnt a social media outlet. It might not be anywhere near as popular as Facebook or Twitter, but it is clearly a social media service. But in my testing, I wasnt able to view an article I shared on Google Buzz, because Id hit the paywall limit.
Again, this shouldnt have been the case.
Blocking Gmail
I tried another test. I emailed myself a link to a New York Times article to my Gmail account, which is run by Google. Again, I got blocked. The same thing happened when I clicked from an emailed link sent to my Yahoo Mail account.
Im pretty sure that if Id clicked on these links from an email account not also run by a major search engine that Id have gotten through but that really shouldnt be the case.
Postscript: HTTPS To Blame?
After I wrote my story above, a reader wrote suggesting that perhaps the secure connection (https) that Google provides for services like Google Reader, Buzz and Gmail might be to blame. The connection might not be passing referrer information that indicates someone is coming from these services, and so the paywall goes up.
I dont think this is the case, but its worth noting. I did some further testing on the morning of March 29, the day after I wrote this. I found:
I also found clicks from Twitter, even from a secure connection, were allowed through the paywall. So, theres more going on. Secure connections can pass referrer information, if they want, to my understanding. Its also possible that the secure connection might redirect a click in a variety of ways to pass on different referrer information.
More Paywall Observations
The paywall is pretty leaky. Its not going to stop anyone who wants to get past and knows even a little bit about the internet. Aside from admitting people via any link, you can also get through by:
There are other tips, too, such as those Mashable has compiled.
Youre Blocking Search Engines Why?
The purpose of me writing about this isnt to encourage people to avoid the paywall. Heck, the New York Times is deliberately going out of its way to keep it from triggering for many people who come to the site.
Instead, its to revisit the absurdity of how search engines ended up getting slapped with one of the few limits out there. It all started with the New York Times trying to keep limits that Google allows publishers as part of its First Click Free program, limits that make little sense when pretty much its Any Click Free now at the New York Times.
After questions were raised about why Google was getting singled out, the New York Times then had to concoct a story about how all major search engines would get blocked, with a confusing message suggesting that the New York Times was using tools that the search engines themselves provide for this.
They dont. The blocking and metering is all done by the New York Times itself and done so poorly, right now, that it is inadvertently blocking non-search services from these companies. Personally, I think the search engine limits should just get dropped.
Further Background
My article below covers more background on the New York Times blocking of search engines and the confusing statements it has made:
Over at my personal blog, I also did a send-up of the pricing plans for digital subscriptions (in short buy a print subscription. You get digital access, should you really think you need it, for a much better price)
