Maybe the guy lost his phone, wallet, or whatever overboard there? If its not yours, leave it alone. My parents taught me that when I was just a child. Now, if the buoy was floating around, not anchored, then yeah, pick it up as litter.
IMO it was pretty clear someone had set it there for a reason, and you had no business messing with it.