slabtop command (part of the package procps) shows top memory objects used by the kernel.
dstat can help you figure out what is happening. dstat -cdnpmgs --top-bio --top-cpu --top-mem
Also have a look at smem ("smem -kt"), it can show you nicely what is in your swap.
M. Martin, J. Unbehauen, and S. Auer. Proceedings of 7th Extended Semantic Web Conference (ESWC 2010), 30 May -- 3 June 2010, Heraklion, Crete, Greece, volume 6089 of Lecture Notes in Computer Science, page 304-318. Berlin / Heidelberg, Springer, (2010)