I'm sorry to hear this has been such a struggle for you. I definitely know what temptation within marriage is like, but marriage is a lifelong commitment, as is raising a child together. I commend you for sticking to your physical morals up to this point and desiring to stay with your family.

Have you ever been completely honest to your wife about this? I feel that if you want to be able to conquer it, it's something the two of you need to fight together, no matter how hard that may be. Also, have you ever seen a counselor to help you sort out these feelings? Talking it out with somebody in depth can greatly increase your understanding of it and get you some great suggestions in dealing with it. Marriage therapy can also greatly benefit you once you are ready for it. Praying this becomes easier and that you will continue to remain loyal to your family.

This sounds like a difficult situation, and one that I can somewhat relate to within my marriage (minus the fertility issues). Having kids is a huge decision and huge responsibility, and so it's very important to understand each others' feelings on the matter and be able to arrive at a consensus over time. Have you considered couples counseling to help you both talk more honestly about your feelings on the matter? You will also need to come to a decision about how committed you are to her; do you care about kids more than her, or does she satisfy you regardless of whether you end up having or adopting kids? Do you prioritize marriage over your future goals, or vice versa? You may need more time to think about questions like that, but they are definitely important to consider. Praying it becomes easier for you.