this is what i would build to stop fish from going back down the hole
buy a pipe style wood clamp put a rod through the pipe with a round plate on one end a bit smaller then the hole and a 90 deg handle at the other end, clamp it to the ice with the round plate against the ice, when you get a fish in the hole you turn the handle which puts the plate blocking the bottom of the hole, sorry if i am not explaining it well enough