In creating high performance, heavy load websites network administrators are faced with the challenge of creating a system that can handle a heavy load and deliver pages in a timely manner that people have come to expect. There are 2 schools of thought on the subject, one school believes in buying the fastest possible hardware and hoping this will stand up to the load. While the other feels lots of small cheap boxes working together can achieve the same or more than the single large machine. I tend to be of the opinion cheap is good, and it seemed to work for google so it can’t be all bad. Now lucky for use we live in the day and age when a fast computer can be bought for cheap, and there is some advanced software that is free. So building a cluster is now something that can be done dirt cheap and very simply.

In my setup I chose FreeBSD as my OS. I believe what I have done can be achieved as easily with most modern UNIX systems. However, since I’ve used FreeBSD the most it was my first choice. There are 2 things I would recommend, first use boxes that are identical, this will ease the installation process and make sure you can take the boxes apart as needed.

The first thing to do is assess what your web application will need. In my case I needed 2 things, First a fast database which I happened to already have established on my network, and second a big file store, again just my luck this was on the network ready to go.

Next step is to design a network architecture this will vary widely depending on the capabilities of your hosting facility what equipment you have at your disposal. My decision was to create 2 private networks so as to isolate data and to maximize bandwidth usage for each specific type of traffic. This means on one network there is only HTTP traffic, and on the second network there is only DB and NFS traffic. This means these two will not be interfering with each other when it comes to bandwidth resources.

In the next article I will cover how to setup the firewall and load balancing, and finally I will cover how to setup the actual webservers that will be used in the cluster.

[tags]FreeBSD, lighttpd, NFS, mysql[/tags]