This explains several ways to use consistent colors across plots (same way works with scales). However, the easy way explained in one of the alternative answers does not seem to work for some versions of R/ggplot2. Opting for the accepted answer always works though.
S. Lukin, K. Bowden, C. Barackman, and M. Walker. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France, European Language Resources Association (ELRA), (May 2016)
P. Singer, D. Helic, A. Hotho, and M. Strohmaier. Proceedings of the 24th International Conference on World Wide Web, page 1003--1013. International World Wide Web Conferences Steering Committee, (2015)