Hi, OP! First let me say that I've been through this too, and I wish you all the best.
I will say that I have to agree with others here that the more qualifiers you have, the lower your chances/the longer your timeline will be. It's so hard, I know, because secondary infertility steals your control over the situation entirely. But you really do have to remind yourself that there's only so much you control with a pregnancy too, and so many variables that can go "wrong" - i.e. you wanted the other gender, genetic abnormalities, etc.
In our case, we worked with a non-profit agency through which we engaged in a home study class that really prepared us well for the entire process. (We did open adoption, by the way - a concept which we really needed that time and learning to wrap our heads around, and which has been great.) In recent years our agency's client base has shrunk and the vast majority of cases involved some level of substance abuse. We too had our limits, but realized that we were not going to ever have a "perfect" situation. Just by dint of birth mom making an adoption plan, you know things are not perfect.
In our case, birth mom is early-mid 20's and didn't know she was pregnant. Like most young women of her age, she drank, she smoked, and she copped to 1-2 instances of cocaine use before realizing she was pregnant. She had literally no pre-natal care. We were trepidatious, but our child is now ~1.5 years, is perfectly healthy/on track with all milestones, is a happy, laid-back kid, and is an absolute joy. Our biological child has some issues we didn't expect, and our adopted child may well, too. You just don't know and the hardest - HARDEST - thing for me to learn and accept as a parent is that I can't control everything and that all we really have is an illusion of control whether our children are biological or adopted.
We put forward very few qualifiers in our materials. The agency informed all waiting families of "special situations" involving substance abuse or other major issues, and we had the ability to pull ourselves from consideration or go forward in each case. We were fortunate to be chosen within a year of our home study completion. It has been one of the best, most rewarding experiences of my life, and I'm so glad every day that I didn't let my fear of the "what ifs" stop me from moving forward. I love that kid more than life, and so does our biological child. Our happiness is immeasurable.
So, I don't know the answer to what is the right way for you to go, but I wanted to share my story and wish you all of the best. I know all of the feelings you are having, and they are scary as hell. But the potential rewards, at least for me, have been so much greater. Good luck!