I would keep it. By next summer, i predict you will use it more. Pools are great at wearing kids out and when our oldest was 2 we started to go almost daily--head over right after pickup (snack in the car), let him swim with one parent while the other fed the baby, then eat a small dinner in the car on the way home. He'd be so tired and ready for bed and we enjoyed it too. Now we have 3 who are older and do basically the same thing except that only one parent takes all three kids and the other gets to relax at home and make dinner. We shower them there and they eat then go straight to bed. And all that is just our during-the-week routine. On weekends they can spend HOURS there.
One thing that might help you get your money's worth this summer is a wet suit for your kid. Little ones often get cold so fast that it feels like a waste to drive all the way there for 20 minutes. We liked these:
http://www.warmbelly.com
I also think the pool is a great place to meet other families, but only if you go regularly at the same time--e.g., decide to go every Wednesday night and Sunday afternoon and you will be more likely to see the same folks over and over and can strike up a conversation that way.