We cannot allow Brexit to be driven by narrow and divisive Tory ideology.