Wednesday, June 04, 2003

Google appears to index the "main" page of this site, but not the archives.

If you search for things that are still on the main page, it will generally find them (assuming they have been there long enough for it to have crawled over them). If you search for something that has been archived and disappeared from the main page, it will sometimes find the old version of the main page from the cache, and even then it doesn't always actually retrieve the right version from the cache. But more often it finds nothing.

Very rarely it will actually find the archive. This appears to be limited to cases where it followed a permalink directly into that individual archive file, either from the main page or from another web site altogether.

One would expect it to simply follow the "archive" link down from the main page and then crawl into the individual archives from there.

Comments: