robots.txt – Uncrawlable code.google.com ?
|
|
I found this thing by accident when I tried to search for my project hosted on Google Code Hosting service website.
http://code.google.com/robots.txt
User-agent: * Disallow:
Does Google intentionally block other web crawler from indexing Google Code site? or was it a glitch?
Tags: code, crawler, google, hosting, open source, opensource, search engine
Keep updated with the latest posts, be a part of over 1,000 subscribers! : 
Subscribe to your email
You might also want to read...
- No related posts
November 19th, 2007 at 11:13 pm
google is the next microsoft? btw papit. hows ur CTR after the new adsense clickable area?
November 22nd, 2007 at 7:10 am
no change in the CTR, same as before sometimes higher, funny. But the eCPM dropped a bit.
November 25th, 2007 at 10:23 pm
My name is Riza zaenal H
I need Linux CD/DVD. May you please assist me to find
* Fedora Core
* Gentoo
* Debian
My Address
City :Cimahi
Street : Jl. Cihanjuang Rt01 Rw 19 Cimahi Utara
City Cimahi
Province : Jawa Barat
post code : 40513
Phone :(022)6644861
Indonesia