Personally, I felt fine right up till egg retrieval; I had to give myself my trigger shot in a friend's bathroom because I was hanging out that night.

If you feel okay, I don't see why there'd be any reason to cut back on your activities.
AFTER your egg retrieval, however, you'll want to take it easy. They're sticking a big needle through your vagina and into your ovaries multiple times, and it'll hurt. The retrieval also releases fluid so you'll be bloated (and, if you're anything like me, you won't want to be too far from a bathroom). I got by fine with Tylenol that one day but I have friends who have needed something stronger than that. I was very happy to have taken the day off.
For transfer, you'll also probably want to take the day off, too. There's no reason to go on bedrest but my doctors gave me Valium to relax my uterus and I wouldn't have been any use at work. So I went home after transfer and took a nice long nap.