I picked the last of the tomatoes, and pulled out the plants, over the weekend. I want to use the long weekend to get the garden ready for winter (need to fix a raised bed, empty pots, etc). If you're not doing stuff like that, you can let them go until first frost.