Freeplane is a powerful and free software for building the mind maps. It is a redesigned version of the well known FreeMind, and is created by one of FreeMind's key developers
Sciplore has produced an interesting hybrid between a mind mapping and reference management tool. Freemind Scholar adds two basic features over the excellent Freemind: you can have references (at this time, only bibTeX) inserted, and you can drag and drop hig hlights on pdfs (the pdf is linked).
This looks like the perfect IDE for sketching notes while reading papers.