I like it. I think you should really do what you want since you will be the one wearing it for decades to come. It doesn't matter what other people think. I love colored gemstones (emeralds are my favorites). If I could go back in time, I would probably get an emerald engagement ring. Although I love my diamond solitaire.
I know for a fact that many people are loving colored gemstone rings. Kate Middleton brought them back into style with Diana's huge sapphire ring and imitations have been selling like hot-cakes around the world. And big name stores have followed suit. I did a search just for fun and saw that place I got my engagement ring, James Allen, has a whole section of engagement rings with colored gemstone sidestones (diamond center stone with colored sidestones, that is)
http://www.jamesallen.com/engagement-rings/settings-with-colored-sidestones/
Colored gemstone are HAWT today!
It's good to get opinions and feedback but you really need to follow your gut.
good luck!