I’m not a manager but got thrown into a project lead position with little support. I found myself a coach (within the organization) and it’s been so helpful.

Yes, you should be trained.
Talk to your HR department and ask if a program for new managers is available.
I am sharing here an online program you can also complete via Coursera, which can get you started.
https://www.coursera.org/learn/management-skills-for-new-managers

Two other resources that can be helpful, these are books:
1) The First 90 Days
2) The Speed of Trust

I just became a first-time manager 2 weeks ago. I have no idea what I’m doing, I’m just assigning work in an environment where I’m so busy with my own work. Is my company supposed to provide me a specific kind of support? My manager is not providing any support on that front. Do I need to ask for training / coaching from HR? What is reasonable to ask for?