What the @#$@@ is going on with NSBB?!?!

[sandberg23]

Sorry about the downtime lately and the rough edges today as things were getting patched. At least I have found the penultimate problem and have a patch solution in place.

 

NSBB has grown to the point where there are close to 800,000 posts, 480,000 words used and 54,000,000 combinations of words and posts stored in the database. As new posts were added or as people were doing searches, it was completely hosing up the system. Basically, I found the limits of mysql as it is installed on my server. I have temporarily disabled the search function and edited out the portions of code that search the wordlist and matchlist when posts are entered. That should take care of the short term problem, but not the longer term issue as we can't live without a search function forever.

 

I'm working on alternatives and hope to implement something this weekend.

 

Once again, sorry about the inconveniences lately and thanks for your patience.

 

Would deleting old threads that haven't been active since before Opening Day 2004 be effective? I very much doubt people are going to miss those much.

 

Maybe putting in an "auto delete" feature in threads that have been inactive for, say, 18 months? I'm sure it would give you enough room, at least for now.

 

You could have an archive for the more popular or funny threads, like the famous Nomar thread.

 

Also just thinking out loud.

 

Wasn't around then. Could you let me know what thread that is?

 

here you go

[Andy]

Sorry about the downtime lately and the rough edges today as things were getting patched. At least I have found the penultimate problem and have a patch solution in place.

 

NSBB has grown to the point where there are close to 800,000 posts, 480,000 words used and 54,000,000 combinations of words and posts stored in the database. As new posts were added or as people were doing searches, it was completely hosing up the system. Basically, I found the limits of mysql as it is installed on my server. I have temporarily disabled the search function and edited out the portions of code that search the wordlist and matchlist when posts are entered. That should take care of the short term problem, but not the longer term issue as we can't live without a search function forever.

 

I'm working on alternatives and hope to implement something this weekend.

 

Once again, sorry about the inconveniences lately and thanks for your patience.

 

Would deleting old threads that haven't been active since before Opening Day 2004 be effective? I very much doubt people are going to miss those much.

 

Maybe putting in an "auto delete" feature in threads that have been inactive for, say, 18 months? I'm sure it would give you enough room, at least for now.

 

You could have an archive for the more popular or funny threads, like the famous Nomar thread.

 

Also just thinking out loud.

 

Wasn't around then. Could you let me know what thread that is?

 

here you go

Classic. :) It's too bad I wasn't here at the time, I would have been able to enjoy it even more thoroughly.

[Tim]

Were there any performance issues today? Because we had well over 100,000 page loads today alone. While not a record for NSBB, that is certainly well above average.

[Transmogrified Tiger]

Were there any performance issues today? Because we had well over 100,000 page loads today alone. While not a record for NSBB, that is certainly well above average.

 

I had a couple connection problems.

[Andy]

Were there any performance issues today? Because we had well over 100,000 page loads today alone. While not a record for NSBB, that is certainly well above average.

I had a couple of slow pageloads, but other than that, scot-free.

[adamb0719]

Were there any performance issues today? Because we had well over 100,000 page loads today alone. While not a record for NSBB, that is certainly well above average.

 

There were a few times pages would not load right away, but nothing too bad.

[vance_the_cubs_fan]

Were there any performance issues today? Because we had well over 100,000 page loads today alone. While not a record for NSBB, that is certainly well above average.

 

Around 3:30-4:00 CST, I had a few issues, but after that it was very fast again.

[erik316wttn]

Were there any performance issues today? Because we had well over 100,000 page loads today alone. While not a record for NSBB, that is certainly well above average.

 

Everything worked just fine for me except for the search features and the "view my posts" links.

[Careless]

my "view new posts" has broken in the past 12 hours or so.

[erik316wttn]

What about deleting old game threads from 2003 and 2004?

[Laura]

We're going to prune down (= DELETE) a lot of the old game threads, but there's going to be a process for doing so and people will have a chance to defend any threads they want to keep around. We'll set up some standards for preserving threads (playoff games; historical games; player debuts, etc.).

 

Between my schedule and NSBB's schedule, I haven't had the chance to get the ball rolling on that ...but we will soon.

 

Keep in mind that deleting threads = deleting the posts in the threads = individual posts counts taking a hit. I may see 11,000 posts again soon. :)

[J.R.]

Why not just archive the old game threads?

 

I hate to see any resource vanish, even one as trivial as most of the stuff in the Game Threads forum.

[Laura]

Why not just archive the old game threads?

 

I hate to see any resource vanish, even one as trivial as most of the stuff in the Game Threads forum.

 

Archiving would be the first choice. The problem as I understand it is that there isn't an obvious useful archiving system ready to go right now. If we need to start clearing out and/or "pruning" old threads to keep the site operational, then old game threads are one area that makes sense to consider for that.

 

If nothing else, I'd try to get them backed up in hard data format. We'd have to come up with a collective process for choosing which game threads to preserve on site. It's not something that would be done unilaterally -- they'll be discussion about it first.

 

I'd rather run the risk of losing online access to old game threads than continue to go on indefinitely without having any search function. Without a search, we have a TON of resources ... we just don't have ready access to them.

[J.R.]

I'd rather run the risk of losing online access to old game threads than continue to go on indefinitely without having any search function. Without a search, we have a TON of resources ... we just don't have ready access to them.

 

Touché.

[Tim]

Just to let everyone know what is going on...

 

I have taken a backup of the entire database and I'm planning on creating an archive that cannot be updated. Then I was going to prune older threads from here. Only I ran into a problem...there are so many records here that the prune function timed out on my 10 entry test forum (the prune function searches all 750K+ records).

 

In the meantime, it's been pretty stable lately so I'll search for a better pruning mechanism and then try again next weekend when the load is lower. Given that it will be Christmas weekend, though, no promises on how much time I'll have to spend. :D

[CubinNY]

Given that it will be Christmas weekend, though, no promises on how much time I'll have to spend. :D

 

Bah humbug! :D

 

We can get along just fine without you working on Christmas

[CuseCubFan69]

Given that it will be Christmas weekend, though, no promises on how much time I'll have to spend. :D

 

Bah humbug! :D

 

We can get along just fine without you working on Christmas

 

I agree with CiNY.

[NCCubFan]

Just to let everyone know what is going on...

 

I have taken a backup of the entire database and I'm planning on creating an archive that cannot be updated. Then I was going to prune older threads from here. Only I ran into a problem...there are so many records here that the prune function timed out on my 10 entry test forum (the prune function searches all 750K+ records).

 

In the meantime, it's been pretty stable lately so I'll search for a better pruning mechanism and then try again next weekend when the load is lower. Given that it will be Christmas weekend, though, no promises on how much time I'll have to spend. :D

You should just ask Santa for a board that doesn't go down. :D