Recent site performance issues

[Karlos]

As most of you will know (if not, see here: http://www.amiga.org/forums/showthread.php?t=54618), the site has been sluggish of late and with intermittent errors popping up.

We've been monitoring the situation for a while and we've ascertained that the site is occasionally tip toeing over the maximum allowed concurrent connections to the database at peak times.

Until a more permanent solution can be found (read a beefier server/hosting package), I've been given the go-ahead on an evil plan I hatched to try and mitigate the issue to some extent. This plan is to basically cache content (for a short time) that is served to all those visitors that aren't logged in.

Allow me to expand on that:

On any normal day, logged-in users are typically outnumbered by anonymous visitors by anything up to a factor of 10 or more. By caching the generated content of several key pages for a few short minutes, the total number of database queries per second can be dramatically reduced and with it, the danger that the server will go over the limit.

On the downside, if you are a regular lurker, you won't necessarily see the most up-to-the-minute view of the page, so perhaps it's time to log in at long last

Also on the downside, the thread view statistics will be skewed, since for every cached serving, no counters will be updated. Still, better than not being able to see the site at all, right?

When I upload the patch, there could be some momentary shenanigans. Please stand by...

[nicholas]

Sneaky kludges ftw!

[Karlos]

Sneaky kludges ftw!

Think of it as enforcing the Royston Vasey policy :lol:

vBulletin is a beast when it comes to resource utilisation. What I've done is to rename several key php files and add subsitutes for them that add the caching aspect. It's managed in a way that requires no database queries at all when you hit the cache. However, you do hit the disk, so it isn't for free, either.

[Karlos]

One other caveat to note is that if you aren't logged in, the indication as to which threads have been read or not will also not be accurate since you will likely be viewing a cached version of the content that someone else was responsible for generating.

[nicholas]

Think of it as enforcing the Royston Vasey policy :lol:

vBulletin is a beast when it comes to resource utilisation. What I've done is to rename several key php files and add subsitutes for them that add the caching aspect. It's managed in a way that requires no database queries at all when you hit the cache. However, you do hit the disk, so it isn't for free, either.

Memcached not any help?

[Karlos]

Memcached not any help?

Sure, but you need to be running it and that usually means you have your own dedicated server.

[nicholas]

Sure, but you need to be running it and that usually means you have your own dedicated server.

Oh I see. Time perhaps for the owners to put the subscription fees and ad revenue towards a server then?

[Karlos]

Oh I see. Time for the owners to put the subscription fees and ad revenue towards a server then.

Whatever they do longer term, we need to at least mitigate the recent problems. Taking the strain off the database will hopefully accomplish that. Every time you read a page in a thread there are several database queries and typically more writes than there are than reads. For a "cached" page, you are getting several filesystem reads: one to check the file exists and another  to check it's changed time and finally retrieving it's contents. The first two reads actually are only one since checking for the file existing causes an fstat() call and php caches the results of those over the lifetime of a script.

[Lando]

There are some other weird things happening with the site.  When I was reading this thread, I happened to look up at the top of the page and it said "Welcome, Transition, you last logged in today at 5:16 AM".

I am not Transition, I am Lando.  I logged out and logged back in as me, how did I ever get logged in as Transition?

[TheGoose]

Good job Karlos, all those explanations and solutions are right-on in my mind.
Amiga.org is umm, too health of a community at times. A good problem to have my boss would say.

[Pyromania]

There are some other weird things happening with the site.  When I was reading this thread, I happened to look up at the top of the page and it said "Welcome, Transition, you last logged in today at 5:16 AM".

I am not Transition, I am Lando.  I logged out and logged back in as me, how did I ever get logged in as Transition?

It was a momentary flux in the Matrix.

[AmigaHeretic]

There are some other weird things happening with the site.  When I was reading this thread, I happened to look up at the top of the page and it said "Welcome, Transition, you last logged in today at 5:16 AM".

I am not Transition, I am Lando.  I logged out and logged back in as me, how did I ever get logged in as Transition?

Yeah, I'm logged in.  On the front page for this thread is says " Pyromania" is the last person to comment in this thread.  I've checked it a few times before posting this and Lando is the last person to post in the thread.

So did "Pyromania" actually post in the thread and I can't see it even though I am logged in (for a while and several refreshes) or does the system think Lando was Pyromania?

EDIT:  

Ok, now after submitting my post I see the Goose and  Pyromania had both posted.  I couldn't see either post even after opening up another refreshing a couple times to see if it was an error and then trying another browser window and refreshing.  Weird.

[Karlos]

Ok, it looks like this plan requires more work - it seems likely that at the moment you log in, that first page gets cached.

I will revert the changes. Luckily, that's not difficult.

-edit-

*done*

[Karlos]

There are some other weird things happening with the site.  When I was reading this thread, I happened to look up at the top of the page and it said "Welcome, Transition, you last logged in today at 5:16 AM".

I am not Transition, I am Lando.  I logged out and logged back in as me, how did I ever get logged in as Transition?

You didn't. You were served a page that Pyromania managed to get cached after logging in. If you had gone anywhere, you'd not have had access to any of his account.

[Karlos]

Anyway, theory is good, implementation needs more work :lol: