I too find voice quality/timbre very important, but it's different from accents. I'm French and remember coming to this country for grad school and finding the accent of one of my classmates very interesting: he came from South Carolina. His voice had a cadence I'd never heard before. The lilt was musical and soothing.
My husband's voice is part of why I find him attractive. Due to work, we had a long-distance relationship for 2 years. We spent long hours over the phone - if his voice wasn't great, we would have split!