Database lagging.

**Quid** · 10-25-2004, 13:26

Originally Posted by The Tuffen

Tosa,

I got this error message last nite

I was going to post it yesterday but couldn't as i kept getting the error message when trying to post.

I have had all of the above mentioned also very often. Something is definitely not working as it should. Let us hope it is all sorted now.

Quid

**The Tuffen** · 10-25-2004, 13:32

It seems to be better today, although i have had the usual error message (without the warning bit on) when i clicked on see all active users on the main page

**TosaInu** · 10-25-2004, 14:07

Yes those were the messages. This morning, about 5 hours earlier from now, I also got those with extra bits about problems in tables. The host took the entire database (not on 3.5" floppies I hope) and copied them to a new machine. During or after that some tables damaged. That problem is fixed now.

I saw a few minor seconds lasting glitches too, I'm not sure whether that's because of cached pages (?) or new problems with SQL. I just loaded the memberpage fine The Tuffen. We'll keep our fingers crossed for a while.

Thanks for the feedback.

**The Tuffen** · 10-25-2004, 16:54

its good to hear that its been fixed.

**TosaInu** · 10-25-2004, 17:22

Well.. we had just another downtime. But we'll continue working on it. The forumsearch function puts some stress on the database (size alone). We've added a Google search to the site. I think it works fine and will open some options when we need to optimize. I like some user feedback about this search too.

**Adrian II** · 10-25-2004, 23:48

Originally Posted by TosaInu

I like some user feedback about this search too.

First of all, thanks for the hard work on behalf of all of us, TosaInu. I'm sure you'll work it out. I'm not so sure about the Google search though. I used it to search for all posts under my own avatar on totalwar.org (must be hundreds by now) and got only 7 results. Then I looked for a post I made some time ago about Korea and Google couldn't find it, no matter how accurate and discrete the keywords I entered. Experiences with Google search functions on other websites aren't encouraging either, so I'm told.

Anything else you want checked, you name it.

**therother** · 10-26-2004, 08:49

Originally Posted by TosaInu

Well.. we had just another downtime. But we'll continue working on it. The forumsearch function puts some stress on the database (size alone). We've added a Google search to the site. I think it works fine and will open some options when we need to optimize. I like some user feedback about this search too.

Hi

I want to second those thanks for all your efforts on our behalf over the last few days. One thing though: the board search function seems to be completely disabled at the moment. Is this permanent now we have Google?

I have to admit, though, I'm not overly convinced by the Google search function either. I tried a few searches for my own posts, and it was tricky to find them. And I had an advantage as I wrote them, and so knew key phrases to search for.

The native search function was a great deal better, especially as it was up to date, much easier to use, and more flexible.

But if the search function is going to collapse the forums with the regularity that we've seen recently, then Google is far better than nothing IMO.

**TosaInu** · 10-26-2004, 10:36

Hello therother,

I've seen some board software and search is a 'weird' thing. The perfect search doesn't exist I think, or you must have a very good server or a small board. In short: if a forum has 100 MB of content, than it also has a searchindex of about that size (it can appear to be non existant in FullTextSearch).

There are various systems and optimisations to get an acceptable search function each having cons and pros. Some boards offer different strategies and so does this one. I chose the supported default one and modified it a bit with the given options. It took a couple of days to figure it out, generate, (Pentium 4 2.6 needed about 8 hours to do the grunt), cut in manegeable pieces, upload, insert, complement and activate.

Result: a pretty complete search method, short words like JHC, CD, DVD, MAA and a load of others were recognized. That searchindex is also something that can cause serious problems in our database, even if it's not available to anyone (because it is indirectly: every post made has to crawl through it).

I knew all that on forehand, but the alternative had/has cons too (it doesn't find anything below 4 chars, is not officially supported and some boards perform worse with it) and it takes time to figure those out. I thought we could be happy until 1,000,000 posts.

Google Search is meant to be an addition. I've found some neat results with it using 2 letter words, but it's also lacking. Google obviously doesn't have the whole lot indexed (yet?).

The server move improved performance, but didn't cure all whoes, I've been around this forum all day and saw some SQL seconds burps and one larger downtime of some minutes. Last night, just 10 minutes before calling it a day, it crashed again and a table got damaged. It wasn't that straightforward to fix as the corrupted tables yesterday morning and my uneducated fear was that a next corruption could bring down the whole lot permanently. I live in a different timezone than the host: many supportcalls don't have the 30 seconds response time. So, I should take the board down for public until professional help arrives (solving the problem can still take days then) or delete #1 suspect, the monstrous searchlog which could be restored in the worst case.

So, Google Search is not meant to replace forum search, it was not meant to disable forum search now, it was not meant to disable it permanently and I was and still am looking to get that other forum search method working (got private testboards running it). Unfortunately the crashes forced me to erase the current forum search right now.

The host mailed me today that two other, far bigger sites on their network use this same messageboard. They run without the search, as the search kills servers.

We'll soon try the other method when this board runs stable (search is not the only thing that has the potential to disturb parties), there are modifications for that one as well. If all that fails? Then we have to think about reinstating a prunning policy for the OT forums.

I'm sorry for the inconvenience and you having to read all this, I hope it explains a bit though.