I was beside the Costal Cave entrance and I was able to go over the shortcut bridge without ever activating it, but it was invisible. I tried to cross over but it didnt go all the way, then I went to camp to try forcing it to go down to see if I could cross the area. After leaving camp it worked but it seems there is an invisible wall... If someone found out how to reproduce it, it could maybe work in another area.

I had to load previous save because I got stuck, the invisible bridge does not exist anymore.