Some Brexiteers are passionate defenders of the benefits of immigration. Others just can't wait to slam the doors shut.