Sounds like your bait was a tad on the small side. Try a big ol' gob, use panty hose or roe bag material to tie it up with, change your bait every 15 minutes. Old cheese works too, I like a live bait personally myself... hugeass minnows or small suckers (6-8") may work better. Adapt, improvise, and overcome!
You didn't say how long you were out for, the fish may just not have been biting while you were there. Don't give up, keep going back and you will eventually catch something.