Just up from the waterfront is The Victory. Nice pub, excellent range and well kept bitters, ciders and largers and good pub food. Really nice people, thoroughly recommended.
The Idle Rocks Hotel at the other end of town (with a great view over the anchorage) I believe thinks itself rather too smart for yachties. We were in shorts and T shirts and were turned away from having lunch on their terrace and were told it was very busy when plainly it was not. So I think yachties in yachty gear are unwelcome there. Take your blazer and grey flannels and you'll be welcomed I imagine. Not for us!