Scaling Your Web Applications

It is pretty much talked about now. Not as much as bout Web 2.0, semantic web and so on, but still a lot.

What is it? Why is it? Who needs it?

If you're into building common blogs or simple “visit card-type” websites, I give 95% you'll never need them to be scalable. You won't even need to think of performance in most cases.

But if you want more, if you want or have to build high-traffic sites or if you're starting new project and aim to get a lot of traffic (and convert that traffic a lot of those small green paper things), you need to have more or less clear understanding.

Ask yourself a question: “How scalable my site is?” If you can answer it, you will doubtfully find something new to you in this articles.

Don't know an answer? Then ask you three questions:

1. How easy will it be to make my site handle 100 times more requests per minute than it handles now?
2. How easily will it be to make my site operate 1000 times more data that it operates now?
3. How easy will it be to make my site run on 10 servers instead of one?

You get it now, right? Scalability is the matter of how easily your application takes more data, more requests and uses more hardware.

You've got two questions (among other) you want to ask me now:

1. How do I scale existing applications?
2. How do I build new applications so they are scalable?

The first question has no clear answer. I'd rather say: “Pay me for that”. Or “Find someone experienced”, if I am busy at the moment you ask me.
There is no panacea, it's highly individual. I only can advise some general things.

Optimise. Find bottlenecks, rewrite code to be faster, denormalize your SQL tables, sacrificing disk space to query speed, analize your queries, add proper indexes and remove unnecessary ones, find someone who can fine-tune your database and web server software,

Cache. Store rarely changed but frequently requested data somewhere, don't caculate it. Use files, free memory (if any), database again (yes! use a table to store user-ready data rather than running a slower query to retireve this data everytime)

Divide et impera. Separate your mind from your body static content from dinamic, clusterize you database tables, separate business logic from view.

You may try to put a lightweight webserver in front of apache to let it quickly serve the static files, proxy other requests to apache and cache whatever is possible to cache without breaking site's functionality. I will post an separate article on that, until then you can use google

The second question is easier to answer, but still there's no panacea and it's all mostly about the same things as above or about how to make your code more easier to scale in future.

Separate your code and view. Use Model-View-Controller architecture. Sometimes, though, it is OK to combine Model and Controller, but it will be much harder to change the backend or add caching when Model and Controller aren't divided. Use any templating system you like or your HTML feel comfortable with.

Separate “useful” data from metadata. For example use different databases for articles and search index, if you use your own (or 3rd party) search instead of MySQL's FULLTEXT indexes. Store stats in separate database. It could cost you some performance, but when you'll grow out of one server, you won't have to rewrite code and split your database across two servers: it was already split. You can even put everything into one database, but do not hard-code it. Use config file like this:

//data $db['data'].server="localhost"; $db['data'].user="dbuser"; $db['data'].pass="dbpass"; $db['data'].dbname="database"; //stats $db['stats'].server="localhost"; $db['stats'].user="dbuser"; $db['stats'].pass="dbpass"; $db['stats'].dbname="database"; //search $db['search'].server="localhost"; $db['search'].user="dbuser"; $db['search'].pass="dbpass"; $db['search'].dbname="database"; And later, move data to different servers and change the config.

Think of performance. Cache rarely-changed data (first you should make it clear to yourself what is rare and what is not in your application workcycle), prefer php (or your language's library) functions to you own: they are written in lower-level language and work faster, use MySQL's ORDER BY rather than sort the query result with php's sort, use LIMIT, if possible, use one JOIN query instread of querying once and then repeating second query in foreach cycle. There are lots of performance tips, and they'll be the topic of another article.

Top

Filed under: work Tags: scalability, scale, mysql, php, optimisation