Techically you only need your passport if you land on US soil. I frequently fish NY waters and seldom worry about having my passport. That being said a buddy once asked an immigration officer we saw in Gander Mountain if he needed a Passport while fishing NY water, the guys answer was,"why not bring it, you never know who may stop you and ask for it"