Forum search switched to Google... NOT!

[Artemiy]

Hey all,

I am here to report that the forum search has been switched to Google search.

The downside is you cannot easily search posts by a certain member, and search in a certain forum (yet), but I will look into making this possible later on.

The main benefit, of course, is speed - the search is virtually instantaneous! Also, we had about 500 queries a day and since our server no longer has to load the database server with them, the forum should work a bit faster.

[dbijoux]

500 queries a day? That's about 1 search every 3 minutes.

I wonder if there is a way(index/caching) to make them faster, but I'm not aware of anything off the top of my head using PHP/mySQL.

Do you think Google is very good, comparitively? The few times I've tried to use it, the results were bad like random stuff from 2007.

[EZX12]

...I liked your "home-brewed" SEARCH feature, Artemiy. It always seemed to work GREAT (and, as you mentioned, it offered the ability to search by member or forum).

Side question: How often, and how deeply, will Google "crawl" the Roland Clan forum? Google's own information does not specify, but they do it make it clear that first and foremost, it is based on PageRank:
http://www.google.com/support/webmaster ... swer=34439

Don't give in to the 'Borg!


(Just a joke Artemiy! I completely understand if this makes site maintenance easier or reduces needed resources. I certainly hope that you are NOT paying for the Google site search feature... and, if anything, I hope it can somehow generate revenue for you and all the terrific work you put into this forum).

[Artemiy]

dbijoux:

It's not that 500 searches a day were putting the forum down, but they were taking a lot of time, up to 30-40 seconds each. Maybe I need to optimize our database somehow, as the database search query itself is what it is, just one query.

I kind of like how Google works, I tested it a bit and it will find the exact stuff I am looking for, under a second. So here I just wanted people to find stuff faster.

I'll be bringing back some previous functionality, though, like being abele to search posts made by some user.

[Artemiy]

EZX12:

No I am not paying for this of course, Goodle might indeed give me some ad revenue from the search but it won't be more than $5 per month

[dbijoux]

No doubt, Google could certainly be more powerful with all the search operators and such. I possibly wasn't trying hard enough and didn't dig in too far. I just happened to notice the results seemed a bit random. For example, I could imagine people reviving threads from years ago that have already been re-answered last week.

As for optimization, in SQL proper you can index tables, but this could be pretty intense on posts text. It can definitely help performance to have the Primary Key, Foreign Keys and other important fields indexed. Not knowing how the database tables are designed I can't really say, but the actual queries could possibly be optimized too. That could be syntactically and/or by reducing redundant queries so database hits are minimized.

Of course, the real delay more likely comes back to searching those fields with paragraphs of text and hundreds of characters. Which is where google's farms of algorithms are going to win everytime.

There are also little things like caching queries that might improve performance(e.g. there is no reason to hit the database everytime someone loads a forum topics page).

[Sliverdigger]

Here's a question - I tried the new search to find a "bagpipes" post of about a week ago. I got only 2 replies (and 9 ads) with neither being the post I had read. Is there a delay in the post being available to the search?

[Artemiy]

Sliverdigger, you are correct, the old search returned about 20 topics about bagpipes, while Google shows only two. I guess we have to revert to the old way

[dbijoux]

I forgot about the ads, since I have blinders to such. That's a good example.

Interestingly enough, you get 5 results from a Google site: search.

http://www.google.com/#hl=en&source=hp&q=site%3Arolandc...

[Artemiy]

Damn that looks like rubbish then...

I tried the same search at google.com and it indeed returns 5 results, vs. 2 here on the site and vs. about 20 results from the database search. So looks like the only thing that Google does well, is not so good actually...

[Artemiy]

Okay, the old search is back!

[howardS]

Hey Art, I tried the google search a number of days ago and couldn't find some old threads and then tried it again a day or two later and it seemed to work a little better. I assumed that some optimization was required so perhaps it just needed more fine tuning and time to index it all.

[Artemiy]

I think I will rewrite the script to allow both searches.

[EZX12]

Artemiy...

Yay! The "Artemiy Search Engine" is coming back! ...and possibly TWO search engines. You are the best Artemiy!

[Sliverdigger]

Re-did the "bagpipes" search.
Only took 12 seconds
Got all the posts including even this one
Very neat/clean/readable/usable
No ads
Worth the wait
Thank you very much!!