slabtop command (part of the package procps) shows top memory objects used by the kernel.
dstat can help you figure out what is happening. dstat -cdnpmgs --top-bio --top-cpu --top-mem
Also have a look at smem ("smem -kt"), it can show you nicely what is in your swap.
Varnish is a state-of-the-art, high-performance HTTP accelerator. It uses the advanced features in Linux 2.6, FreeBSD 6/7 and Solaris 10 to achieve its high performance.
memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.