That's good ol' yellow grub-harmless when the fish is cooked properly and vey easy to "pop" out of the fillets with the tip of your knife. You missed out on some tasty dinner there; I'd be willing to bet it's tought to find a perch anywhere in central/southern Ont that doesn't have any yellow grub at this time of year.