As far as I can tell, you need to start the quick swapping before breaching the red circle around the enemies. You get a few seconds to run before needing to swap back and forth again. If you pick up artifacts or open chests, should do a swap right before interacting with the item/chest otherwise enemies might engage in combat.

You can pick up scraps, weapon parts, & items, open chests, and interact with story progressing or area leaving prompts. If there was a way to get past House of Bones and the Sealed Gate without the requirement of a fight, then you could probably beat the whole game without fighting anyone.