I'm copying what I wrote on another thread re this very thing, so people who are worrying will see it:
As someone whose 2mo old was hospitalized for more than a week with RSV, more than a decade ago...
If the doctor sends you home and s/he is still worrying you, take the baby to a hospital ASAP. I took my child in the doctor (not our regular ped) who said all is fine. It was not fine. Four hours later I went to the ER because I thought DC was dehydrated, but DC was whisked away due to lack of oxygen and was undergoing a spinal tap 2 hours later.
Always listen to your instincts, more than any doctor. I cannot stress this enough.
***
Glad your kid was negative OP!