-
http://collectd.org/
Collectd is a simple, small utility to log system load every 10 seconds. It's written in C so it's not a script and offers good performance.
-
http://collectl.sourceforge.net/index.html
Collectl is a system monitoring tool. It logs a wide range of system metrics (disk, cpu, io, etc) and stores them in a variety of formats. It tries to cover all the bases unlike some other tools which focus on one or two methods.
-
http://developer.yahoo.com/performance/
Yahoo!'s Exceptional Performance team evangelizes best practices for improving web performance.
-
http://developer.yahoo.com/yslow/
YSlow is an extension for Firebug which shows ways to speed up page loads. It's published by Yahoo as part of their developer network.
-
http://engineering.gomiso.com/2011/01/04/easy-monitoring-of-varnish-with-munin/
Very useful walkthrough on setting up munin to monitor Varnish. Explains how to install and configure munin, which is very simple and very valuable.
-
http://filereplica.com/
A file is split into n chunks, encoded such that any m chunks (m<n) can be used to rebuild the original file. It can use half of the alternative storage requirements, but requires cpu overhead.
-
http://furquim.org/chironfs/
ChironFS is a fuse based filesystem that supports multiple backends. Writes are written to multiple hosts, reads come from a single host. It removes single points of failure and provides real time duplication / backup.
-
http://haproxy.1wt.eu/
HAProxy is a TCP proxy / load balancer which will work for HTTP and a range of other applications. It advertises a small but expert user base and offers a range of very useful load balancing features. Apparently it competes with pound and ultramonkey.
-
http://kosmosfs.sourceforge.net/
An open source, distributed, failure resilient file system similar to GFS (from google). Supports automatic addition of nodes, redundancy, and so on.
-
http://lucene.apache.org/hadoop/
Hadoop is a distributed computing platform / filesystem. It provides HDFS which is similar to GFS (Google Filesystem) and MapReduce which splits programming tasks into small chunks. It's target is 10'000 node clusters.
-
http://network.com/
Grid computing from Sun.
-
http://nimbusdb.com/how_it_works.html
Cloud based database that tries to achieve a new type of scalable, highly available, fault tolerant persistent data storage layer in "the cloud".
-
http://reductivelabs.com/trac/puppet
Puppet is a system to abstract system administrator tasks such as configuring web servers, deploying database servers, and so on. It provides an abstract language to specify configurations and then handles deployment for you.
-
http://reductivelabs.com/trac/puppet/
Put simply, Puppet is a system for automating system administration tasks. It's a language to define services, nodes, dependencies, and so on. Then it handles the "realisation" or "execution" of those definitions across multiple platforms, etc.
-
http://serverfault.com/questions/130185/store-varnish-cache-in-hard-disk
Useful pointers on how to store varnish cache on disk. Very easy, recommends using ext2 and noatime / nodiratime to improve disk throughput. Smart.
-
http://stackoverflow.com/questions/1316852/alternative-to-memcached-that-can-persist-to-disk
Very interesting question that has lots of useful links on persistent object / value stores that are similar to memcache. Answer seems to be couchbase.
-
http://stackoverflow.com/questions/240333/how-do-you-measure-page-load-speed
More info on measuring page load speed from StackOverflow.
-
http://themomorohoax.com/2010/07/31/ruby-chef-tutorial
Handy article introducing chef and providing a few links. A little out of date now, but provides a useful intro.
-
http://unixbhaskar.wordpress.com/2010/11/12/measure-website-response-time-through-curl/
Handy article explaining how to measure the (speed) time of an http request using curl. The key is -w "%{time_connect}:%{time_starttransfer}:%{time_total}\n" which outputs the times.
-
http://varnish.projects.linpro.no/
Varnish is a state-of-the-art, high-performance HTTP accelerator (reverse proxy). Varnish is designed by a BSD kernel developer to make full use of BSD / Linux kernel's ability to manage memory. It is extremely fast, up to 10 - 20 times faster than squid.
-
http://weoceo.weogeo.com/
WeoCEO is designed to provide a constant front end to Amazon EC2 / S3 services. It provides stable IP, load balancing and DNS management to ensure your app stays online.
-
http://wiki.systemimager.org/index.php/Main_Page
SystemImager is software which automates Linux installs, software distribution, and production deployment. It allows you to install complete system images to many hosts, from one image server. It's quite advanced, would take some learning.
-
http://wpmu.org/scaling-wordpress-wpmu-buddypress-like-edublogs/
Interesting write up by James at Incsub on how they handle scale on edublogs. Some good advice although a little light on specifics, a good starting point for research.
-
http://www.3tera.com/
3tera provide a grid platform to deploy "virtual appliances" on a "virtual datacentre" built on top of commodity hardware, similar to Amazon's EC2/S3.
-
http://www.capify.org/
Capistrano is a system built in Ruby to execute commands on many servers, in parallel via SSH. Config files, jobs, etc are written in Ruby.
