Conservative chairman Brendon Lewis has called on Boris Johnson to apologise after he said Muslim women wearing full-face veils looked “like letter boxes”.