I used to work as a secretary for a construction company--and I know we often hired temp workers in general manual labor positions. Some of them were hired long term (months at a time), while others were only hired for a day or two, depending on the specific needs of the project.
They were not doing any skilled work (electricians, plumbers, etc) it was more like --move that pile of boxes from over here, to way over there--type of thing
This is the company we used
http://www.laborready.com/#
It looks like they have some locations in the DC area