I think your doing all right, like was said build that trust and good relationship over time and then try again. It never hurts to find out what he drinks from his cousin and stop in around the holiday season with a thank you bottle for access to the fishing.
Cold calling and asking for permission is one of the toughest things to do and even harder on the land owner as he has no idea who or what you are and wanting to carry a gun on his property. I personally would have asked his cousin before I knocked on the door and asked to use his name as it would have given him a reference to talk to about you. I would try to work it in some time in the future if you see him while fishing,it can't hurt.
Good Luck with it Verno.