The Education Forum > Forum not visible in Google

Sid Walker

Apr 11 2007, 08:39 PM

A search on Google for

"Len Colby" "Sid Walker"

should bring up hundreds of references, as Len and myself have appeared on the same page (webpage, that is) on many occasions in this forum.

In fact, as of today, it brings up no references at all.

Is this deliberate?

I can only assume it is, as by default Google would spider the site.

Is the forum invisible on Google as a matter of forum policy?

If so, why?

Sid Walker

Apr 11 2007, 09:32 PM

Google, of course, accounts for the lion's share of web searches as of April 2007, but I did a more comprehensive search for the same search terms, using the Vivisimo.com facility which search several sources at once.

Two searches, using slightly different parameters, are recorded below.

QUOTE

These sources have been queried:
· Ask - No result retrieved in 0.175s, 37 requested. (1 page requested - 1 OK)
· Gigablast - Top 2 results retrieved out of 2 in 0.459s, 37 requested. (1 page requested - 1 OK)
· MSN - Top 1 results retrieved out of 2 in 0.5s, 37 requested. (1 page requested - 1 OK)
· Open Directory - not queried, error (1 ms) - "Len Colby" and "Sid Walker" : No query was entered and a query is required. (ID: SEARCH_ENGINE_NO_QUERY)
· Sponsored Listings - Top 1 results retrieved out of 1 in 0.276s, 4 requested. (1 page requested - 1 OK)
· Wisenut - not queried, query syntax not supported.
1. The Education Forum -> Political Conspiracies [new window][full window][frame][cache][preview][clusters]
18 Len Colby. 273. 13th April 2006 - 04:51 PM. Last post by: Len Colby ... Last post by: Sid Walker. . A question for Al Carrier and others with ... Len Colby. . Three questions for Jack White. 2 Len Colby. 243. 16th ...
educationforum.ipbhost.com/index.php?act=SF&f=209-Gigablast 1
2. Sid Walker - Viewing Profile [new window][full window][frame][preview][clusters]
Len Colby ... Other users have left no comments for Sid Walker.
educationforum.ipbhost.com/...p?showuser=4537-MSN 1
3. The Education Forum -> Political Conspiracies [new window][full window][frame][cache][preview][clusters]
Yesterday, 01:17 PM. Last post by: Sid Walker. . Internet and the TV ... this forum (2 Guests and 0 Anonymous Users). 2 Members: Len Colby, ... Len Colby. 1316. 8th March 2007 - 02:40 PM. Last post by: J. Raymond ...

QUOTE

These sources have been queried:

· BBC - No result retrieved in 5.009s, 20 requested. (2 pages requested - 2 timed out)
· Gigablast - Top 2 results retrieved out of 2 in 0.304s, 19 requested. (1 page requested - 1 OK)
· LII - No result retrieved in 0.747s, 20 requested. (1 page requested - 1 OK)
· PubMed@NIH - No result retrieved in 0.492s, 19 requested. (2 pages requested - 1 OK - 1 failed)
· YahooBizNews - No result retrieved in 0.166s, 19 requested. (1 page requested - 1 OK)
· YahooSportNews - No result retrieved in 0.165s, 19 requested. (1 page requested - 1 OK)
· YahooTechNews - No result retrieved in 0.153s, 19 requested. (1 page requested - 1 OK)
· eBay - No result retrieved in 0.503s, 100 requested. (1 page requested - 1 OK)
1. The Education Forum -> Political Conspiracies [new window][full window][frame][cache][preview][clusters]
18 Len Colby. 273. 13th April 2006 - 04:51 PM. Last post by: Len Colby ... Last post by: Sid Walker. . A question for Al Carrier and others with ... Len Colby. . Three questions for Jack White. 2 Len Colby. 243. 16th ...
educationforum.ipbhost.com/index.php?act=SF&f=209-Gigablast 1
2. The Education Forum -> Political Conspiracies [new window][full window][frame][cache][preview][clusters]
Yesterday, 01:17 PM. Last post by: Sid Walker. . Internet and the TV ... this forum (2 Guests and 0 Anonymous Users). 2 Members: Len Colby, ... Len Colby. 1316. 8th March 2007 - 02:40 PM. Last post by: J. Raymond ...
educationforum.ipbhost.com/index.php?showforum=209-Gigablast 2

In summary, Gigablast had two pages, MSN one... that's all.

This means forum contents are effectively invisible to anyone outside the forum network.

What's more, given the rather poor internal search facility, even someone at the site may have difficulty finding what they seek quickly.

Mark Stapleton

Apr 12 2007, 09:37 AM

That's very bad news. The Forum's a great resource for many topics.

In relation to political conspiracies, including JFK, I think this Forum has a format which is highly accessible and more easy to follow for beginners than that of competitors.

Not much use if we're operating in a vacuum.

Andy Walker

Apr 12 2007, 10:05 AM

I denied all spider bot access about 4 weeks ago when we were having some issues with bandwidth. This is why search engine searches will bring up older forum threads but not very recent ones.
I have given them access again today and the bots should be visible in the active user list at the bottom of the page when they visit - soon a google search for "sid walker" should bring up a list of threads from the forum of intense wisdom and insight

Mark Stapleton

Apr 12 2007, 02:45 PM

QUOTE (Andy Walker @ Apr 12 2007, 10:05 AM)

I denied all spider bot access about 4 weeks ago when we were having some issues with bandwidth. This is why search engine searches will bring up older forum threads but not very recent ones.
I have given them access again today and the bots should be visible in the active user list at the bottom of the page when they visit - soon a google search for "sid walker" should bring up a list of threads from the forum of intense wisdom and insight

How's England's World Cup tilt progressing?

Those old Aussie pensioners haven't lost a game in.....uh.....how many years?

Andy Walker

Apr 12 2007, 03:13 PM

QUOTE (Mark Stapleton @ Apr 12 2007, 02:45 PM)

QUOTE (Andy Walker @ Apr 12 2007, 10:05 AM)

I denied all spider bot access about 4 weeks ago when we were having some issues with bandwidth. This is why search engine searches will bring up older forum threads but not very recent ones.
I have given them access again today and the bots should be visible in the active user list at the bottom of the page when they visit - soon a google search for "sid walker" should bring up a list of threads from the forum of intense wisdom and insight

How's England's World Cup tilt progressing?

Those old Aussie pensioners haven't lost a game in.....uh.....how many years?

Ah..... the short term memory loss of the elderly Ocker!!!

http://news.bbc.co.uk/sport1/hi/cricket/england/6341721.stm

Sid Walker

Apr 13 2007, 12:06 AM

QUOTE (Andy Walker @ Apr 12 2007, 10:05 AM)

I denied all spider bot access about 4 weeks ago when we were having some issues with bandwidth. This is why search engine searches will bring up older forum threads but not very recent ones.
I have given them access again today and the bots should be visible in the active user list at the bottom of the page when they visit - soon a google search for "sid walker" should bring up a list of threads from the forum of intense wisdom and insight

Well thanks Andy.

To state the obvious, this is not about being able to search my own threads on Google.

I gave the example of searches on "Sid Walker" and "Len Colby" as I was fairly sure these terms do not appear anywhere else.

The serious issue is that, as far as I can tell, Google has not been spidering the forum. If the forum was spidered over a month ago.. well, the pages have already vanished from the cache. I'd have thought they'd linger longer, but I'm not a search engine wonk. Perhaps that's what happens in 2007.

Here are other searches that illustrate the point:

"Gus Hall and the American Communist Party" "jim glover" - no results in Google as of today.

"nosenko, Angleton and Oswald" - no results in Google as of today.

"Olover Stone and Fidel Castro" - no results in Google as of today (this is a very old thread).

One search did yield a result.:

"Assassination, Terrorism and the Arms Trade" gave:

QUOTE

Results 1 - 5 of 5 for "Assassination, Terrorism and the Arms Trade" . (0.18 seconds)

Board Message

Board Message. Sorry, an error occurred. If you are unsure on how to use a feature, or don't know why you got this error message, try looking through the ...
educationforum.ipbhost.com/index.php?showtopic=5799 - 39k - Cached - Similar pages

Spartacus Educational - Home Page
Assassination, Terrorism and the Arms Trade: The Contracting Out of U.S. Foreign Policy: 1940-2006 · HistoryWorld · Tewkesbury · Dealey Plaza UK ...
www.spartacus.schoolnet.co.uk/ - 41k - Cached - Similar pages

Richard V. Secord
Namebase: Richard V. Secord · Assassination, Terrorism and the Arms Trade: The Contracting Out of U.S. Foreign Policy: 1940-1990 ...
www.spartacus.schoolnet.co.uk/JFKsecordR.htm - 64k - Cached - Similar pages
[ More results from www.spartacus.schoolnet.co.uk ]

Optevi News Tracker - World News - Center for Constitutional Rights
Assassination, Terrorism and the Arms Trade - The Educa, educationforum.ipbhost. MediaLens :: View topic - Reports and links on Iraqi to, medialens.org ...

www.optevi.net/newstracker/default.aspx?f=1&e=23018 - 50k - Cached - Similar pages
Optevi News Tracker - World News - South Vietnamese Army

Assassination, Terrorism and the Arms Trade - The Educa, educationforum.ipbhost. Journalism and Terrorism:, freedomforum.org ...
www.optevi.net/newstracker/default.aspx?f=1&e=3570 - 38k - Cached - Similar pages
[ More results from www.optevi.net ]

In this case, the forum comes up first. However, the google entry does not encourage further viewing.

There are two steps, IMHO, that may be needed (assuming the intent is to make the contents of this forum available to the wider world).

The first is to enable spidering. The second (which may be necessary) is to optimize the pages for search engines.

An example of where it seems to work well may help illustrate what I think should be possible here.

I visited Richard Dawkins' Forum.

While there, I selected a thread at random: < Politics & Current Affairs. ~ Trouble in Paradise

Then I selected a block of text on that page, again at random:

QUOTE

Ever been in a fued? Ever had a family fued! They're horrible! Hey, remember that tv show, Family Fued!

Then I typed that same text into Google and clicked Search.

As I expected, first entry up matched that text perfectly. It also made sense as a Google entry:

QUOTE

RichardDawkins.net Forum ~ View topic - Trouble in Paradise
Ya, fued! Ever been in a fued? Ever had a family fued! They're horrible! Hey, remember that tv show, Family Fued! That was funny. There was fueding all over ...
richarddawkins.net/forum/viewtopic.php?t=8120&view=previous&sid=d13afe50c9d718f05ee4c1aa0e6ce8d6 - 32k - Cached - Similar pages

-

Mark Stapleton

Apr 13 2007, 03:23 AM

QUOTE (Andy Walker @ Apr 12 2007, 03:13 PM)

QUOTE (Mark Stapleton @ Apr 12 2007, 02:45 PM)

QUOTE (Andy Walker @ Apr 12 2007, 10:05 AM)

I denied all spider bot access about 4 weeks ago when we were having some issues with bandwidth. This is why search engine searches will bring up older forum threads but not very recent ones.
I have given them access again today and the bots should be visible in the active user list at the bottom of the page when they visit - soon a google search for "sid walker" should bring up a list of threads from the forum of intense wisdom and insight

How's England's World Cup tilt progressing?

Those old Aussie pensioners haven't lost a game in.....uh.....how many years?

Ah..... the short term memory loss of the elderly Ocker!!!

http://news.bbc.co.uk/sport1/hi/cricket/england/6341721.stm

Of course, I was referring to the World Cup matches--the one's where Australia TRIES to win--not those Mickey Mouse sideshows.

Here's a recent article from Chappelli. 24 WC games without loss for Australia. The last time Australia lost a WC game was in the last millenium (1000-2000 AD):

http://content-www.cricinfo.com/talk/conte...dia/289673.html

Shanet Clark

Apr 13 2007, 05:33 AM

A few years ago, topic thread KEYWORDS on this international education
forum correllated with search engine responses and directed interested
students to the site relative to the volume of our correspondence.

This is no longer true and all manner of strange and petty citations
appear before the true ED FORUM links.......

Suspicious, but not suprising

>>>,>>>,>>>

Andy Walker

Apr 13 2007, 10:16 AM

QUOTE (Mark Stapleton @ Apr 13 2007, 03:23 AM)

QUOTE (Andy Walker @ Apr 12 2007, 03:13 PM)

QUOTE (Mark Stapleton @ Apr 12 2007, 02:45 PM)

QUOTE (Andy Walker @ Apr 12 2007, 10:05 AM)

I denied all spider bot access about 4 weeks ago when we were having some issues with bandwidth. This is why search engine searches will bring up older forum threads but not very recent ones.
I have given them access again today and the bots should be visible in the active user list at the bottom of the page when they visit - soon a google search for "sid walker" should bring up a list of threads from the forum of intense wisdom and insight

How's England's World Cup tilt progressing?

Those old Aussie pensioners haven't lost a game in.....uh.....how many years?

Ah..... the short term memory loss of the elderly Ocker!!!

http://news.bbc.co.uk/sport1/hi/cricket/england/6341721.stm

Of course, I was referring to the World Cup matches--the one's where Australia TRIES to win--not those Mickey Mouse sideshows.

Here's a recent article from Chappelli. 24 WC games without loss for Australia. The last time Australia lost a WC game was in the last millenium (1000-2000 AD):

http://content-www.cricinfo.com/talk/conte...dia/289673.html

So anything Australia loses is automatically "Mickey Mouse"