It would be faster just to follow him because following CB would only cost about a minute. Waiting for CB would take about 1:15, then you would have to add driving back to the mission and the trip to the port.