Database lagging.

[frogbeastegg]

The topics are working correctly in firefox now. I only turned the PC off for the night, nothing more.

[TosaInu]

Glad to hear it solved.

[TosaInu]

I used the Google search today and it seems to me that it returns more results (also about old subjects) than a couple of weeks ago. Is this imagination?

[therother]

I used the Google search today and it seems to me that it returns more results (also about old subjects) than a couple of weeks ago. Is this imagination?

I've noticed that it's getting better also, unless we're having a group hallucination or it's wishful thinking.

Funny that you should resurrect this thread, but there seemed to be a few problems with the server/database yesterday (30th) about 7ish - I got the dreaded, "There seems to have been a slight problem with the Guild database" for the first time in a while.

[therother]

Sorry to once again pester you with this, but could we have an update on the status of the board search function?

Also, you mentioned that you were testing various implementations of MySQL fulltext search. Just as a matter of interest, which performed better - the native (but as yet unsupported) feature or hacks such as JohnWoo's.

[ah_dut]

I encountered serious lag and a few posting failures today

[TosaInu]

Therother. I've tested the standard SQL search, it was working pretty fine. Problem is that I can't alter the tables myself with PHP as they are too big. Host has to do that.

Yes, it's lagging again ah-dut.

[frogbeastegg]

More tech trouble? For the last week or so I've found it very hard to get the Guild to load. Everything has 30 second or longer loading times, if it loads at all. Much of the time I'm left waiting for a few minutes before I get "there seems to be a problem with the database, please hit refresh blah blah". About a quarter of my attempted posts fail with that message and it takes several minutes for everything to load so I can try again. Most of my attempts to answer my PM's result in the database error message. One time my post appeared but the topic's last post date/owner didn't update, nor did the last post/topic listing in the guild. It was as if my post did not exist outside of the inside of the topic. One post came out mangled with extra spaces inserted in it, and then it took ages for the several step editing process which then broke with the database error right at the end. The post did update successfully though.

:hits post, prays:

[TosaInu]

Hello frogbeastegg,

On top of the normal problems, there are PHP attacks. Attacks and countermeasurements slow down boardperformance. I just received reply from the host about a couple of things and the board will likely be offline for some time over the next few days. We'll try to keep this at a minimum. My advise is to write posts in your textapplication and copy/paste when done.

I'll post again when the grunt is done.

[Big King Sanctaphrax]

On top of the normal problems, there are PHP attacks. Attacks and countermeasurements slow down boardperformance

Attacks? From whom?

[Adrian II]

Attacks? From whom?

Let me guess. There's a Santy.c worm out (you got it, it was released on 12/24) that attacks not just php boards, but all php scripts and pages.

[TosaInu]

Hello,

There are indeed worms and PHP exploits, and it's affecting many sites. The recent slowdowns are caused by mechanisms to defend against it. As far as I know it didn't come through yet, there seems to be a serious effort though to harm PHP, so don't be surprised when this or another board is gone at some day.

[Uesugi Kenshin]

Right, you can just ignore my e-mail about the boards seeming to be down, between 3 and 10 today. I am obviously able to get back on.

[frogbeastegg]

Now I can’t send PMs at all; I click send and get dropped in a blank page. From that point on the org is broken; I can’t get back to it via any of my bookmarks; they begin to load and instantly stop. About 10 minutes later it starts working again for no apparent reason. This has happened several times now. I’ve also ended up with the forums becoming inaccessible with nothing but broken links after simply browsing for around 5 minutes.

[TosaInu]

Hello,

It could be because of the host trying to do something with the database required for the searchfunction. Working on it at the moment.

I'll do a quick investigation and decide what to do, the board will be down from several minutes up to an hour or more. Depending what/how it will be done.

[TosaInu]

Board will be down starting now.

[TosaInu]

Stage 1 completed.

[TosaInu]

Hello,

A table locked up, it took the tool an hour to repair it.

[Red Harvest]

I've lost several posts today. Even short ones I'm now copying to the clipboard before I hit send.

Hope you can work this out. All the TW boards seem to be having increasing levels of trouble lately.

[therother]

It could be because of the host trying to do something with the database required for the searchfunction. Working on it at the moment.

Excellent news! Does this mean we may have it back in the near future, if they are successful?

[Hosakawa Tito]

That is excellent news. I experienced some serious lag etc.. while "taking care of some business" here about 2 hours ago, but within the last 1/2 hour or so things have returned to normal. Definitely copy any "longwinded" post, my specialty or you may lose it.

[frogbeastegg]

:puts book labelled 'cross species mind control' away: Thank you, Louis.

I only managed to get her one yesterday afternoon and evening, for maybe 2 minutes, if that. I was reading the update about a broken table in this thread when the org broke again; how ironic. Apparently though I was showing up on the active list even though I could not get the site to load, and it was loading as normal for Louis. I was not sat here and trying to get back all that time, but since I did a heck of a lot of writing and played some MP RTW I was able to try to get through for a good few hours.

So my problem with the org only loading for a second before halting as if I never clicked the link is more localised than I thought. I've done the expected checks to firewalls etc and nothing helped. Also nothing at all is changed on my PC those times I do manage to get here. I have not tested my PM capacity yet; yesterday morning I managed a couple through endless retrying, about 4 failures to each success.

EDIT: Whatever the problem is it's still present today; the links broke again shortly after this post.

[TosaInu]

The board will be down today as I'll work on the database. I'll start within a few minutes from now and it may take a while. Hard to say how long, but it could be a few hours.

[TosaInu]

Hello,

Time for some updating.

We had a new searchsystem, it was a pain to get it. The reason is that the computer has to perform a one time operation on a huge amount of data. The host, being a master on this computer, had to pull some tricks to get it done.

The operation took just a few minutes on my Pentium, but it drained all memory and required about a GigaByte in c:\Windows\temp. It was on, so staff could test it out (would be a shame to blow the entire server), but I was forced to delete it again during the last downtime. Server just refused to load it. The database got rid of 200,000 OT posts now.

I guess it's back to the drawing table as far as Search is concerned, queries can take up a few seconds and they are performed on the same tables as used to store your posts in. This can lock up the tables, which will cause a forum downtime. This is a problem for many larger sites and while we are not that large, our database is definately not 8 Gig in size, our hardware is not what those guys use either.

The not so great performance of the Search (20 seconds while 1 person can use it is not that fast) may be related to something else, for now it's gone while we'll monitor performance of the bare system. Still looking around for improvements. For now you'll have to use the Google search, which is surprisingly fast and returns more results than before.

I'm looking into creating a partial forumsearch (say limited to RTW and modding forums). Should be smaller and just work.

Added a tiny tweak that should help narrowband: CSS files are stored in files and no longer served from the database. It's only 6 kb, but it's cached in your temporary internet directory now instead of downloading it again and again.

[therother]

Thanks for the update, Tosa. As always, it's much appreciated.

I'm looking into creating a partial forumsearch (say limited to RTW and modding forums). Should be smaller and just work.

Interesting idea. I think it is certainly true that the lack of a comprehensive search function for these forums is a little constraining for users.

[therother]

Can't speak for anyone else, but in my experience the boards have been lightening fast for the last few days. Lag is almost non-existent, and I've been seeing download speeds in the 30KBytes/s range. Long may it continue!

[Uesugi Kenshin]

It has seemed faster to me, but I am also keeping two windows open instead of one, so I can load a page with one while I read a thread. I am much more efficient and I think the speed is up!