So the rules say you have to place 4th in every race. If you placed 5th-8th, retried it and placed 4th or better you would have placed 4th or better in the race. In short my question is can you retry?

Yes, you can keep retrying. Usually just worth resetting at that point, but you can definitely submit a run with multiple attempts on a track if you'd like.