The COMPASS Map Labeler is a part of a study which investigates the effect of social tagging and end-user map annotation on search & retrieval in online map portals. The primary outcome of this effort will be a 'ground truth' for the evaluation of map search engines.