The civil rights movement wasn't easy for anybody.