fly from Arusha, for sure. If you want to relax AND shop/tour you probably need to split up your stay - stay in Stonetown to tour and shop, check out markets, etc., and head out to a beach resort for the other.
we stayed at what used to be emerson and green, now 236 Hurumzi, in Stonetown. A 10 in character, probably a 7 in comfort (not a western style hotel with great water pressure and fluffy towels for ex.):
http://www.i-escape.com/236-hurumzi/overview. If you're willing to accept holiday in comfort I'd highly recommend.
We had a similar experience at Matemwe Beach - lovely, real "corona commercial" beach, but not exquisite comfort. I think you could find that elsewhere if you wanted but we wanted local charm. Also look carefully about snorkeling - IIRC there wasn't much to see in that area - we went diving but they drove us out to another location to catch the boat.
Jealous, so jealous... we loved Zanz.