At the heart of all successful marriages is the commitment of the partners to one another, NOT just a feeling. The 'fact' that someone has 'fallen out of love' with their spouse and now is 'in love' with someone else offers NO justification for abandoning the marriage.

In that context your willingness to make and effort with adults is, I would argue, your fulfilment of God's call to 'love one another'; certainly you don't deserve the label 'selfish' for how you are living.

The other comment I'd throw in is that men tend to do better at relating to each other when they are doing stuff together rather than just sitting around talking. Thus sports, clearing the church lot or watching a game are usually more successful than just being together. So the question is: are there any practical volunteering opportunities in your church or community that might help build relationships?