Authorize based on the destination IP