I've used MetroParentRelief for short term/emergency care and they're good.
My impression is that any agency will be more expensive as they have fees, and I also think that the nannies who are registered with agencies tend to be more professionally/career minded. Therefore they negotiate better and command higher rates.
But that's just my very limited experience.
I'd be interested in other responses to this as we may be having to replace our long-term nanny within the next couple of months and I'd consider using an agency.