It matters a great deal. Especially if you're going to need a mortgage on an unusual property, etc, etc and it also depends on how significant your assets are. If we're talking north of $10MM you should see a private banker.
Don't bother with JP Morgan at that level though, I tried them with an amount just north of that and the gentleman very kindly told me they would only open new accounts for those with $20MM or more to deposit.
Try US TRUST.