There really no issue unless you keep the fish out of water for a long period. If you don't like fizzing fish you can take that lead strap off your marker buoy and make a small hook. I tie it to a baitcaster. Just hook it on the lip and start the fish down. Most often they are nice and just get towed to bottom.
I've fished areas for 2-3 hrs after towing one to bottom and never seen them again.