robots.txt - Uncrawlable code.google.com ?

Posted by mypapit on 19 Nov 2007 in Funny Stuff, Internet Stuff, Open Source

 

I found this thing by accident when I tried to search for my project hosted on Google Code Hosting service website.

http://code.google.com/robots.txt

PLAIN TEXT
CODE:
  1. User-agent: *
  2. Disallow:

Does Google intentionally block other web crawler from indexing Google Code site? or was it a glitch?

Bookmark this article These icons link to social bookmarking sites where readers can share and discover new web pages.
  • digg
  • YahooMyWeb
  • NewsVine
  • Netvouz
  • Reddit
  • Spurl
  • Furl
  • del.icio.us
  • StumbleUpon
  • Technorati
  • TwitThis
Tags: , , , , , ,

Keep updated with this website! : Subscribe to your email

Recommended Reading

  • No related posts

3 smashing comments for this post.

  1. kucau Said:
    November 19th, 2007 at 11:13 pm

    google is the next microsoft? btw papit. hows ur CTR after the new adsense clickable area?

  2. mypapit Said:
    November 22nd, 2007 at 7:10 am

    no change in the CTR, same as before sometimes higher, funny. But the eCPM dropped a bit.

  3. riza Said:
    November 25th, 2007 at 10:23 pm

    My name is Riza zaenal H
    I need Linux CD/DVD. May you please assist me to find
    * Fedora Core
    * Gentoo
    * Debian

    My Address
    City :Cimahi
    Street : Jl. Cihanjuang Rt01 Rw 19 Cimahi Utara
    City Cimahi
    Province : Jawa Barat
    post code : 40513
    Phone :(022)6644861
    Indonesia

Leave a Comment

Subscribe by email

Enter your Email