I don't think you are wrong to think this way at all. It is really frustrating with guests. I think the best thing for you to do would be with your Fi to sit down and talk with these people, or email them, or do something to find out. Like you said the one has only been saying things to other family and friends so maybe you need to hear from him first hand on the situation. And In some ways I supposed if they have divorced or split up and both want to come then it is their issue and they are going to have to deal with one another. It does suck that none of your family is going. Most of all just remember that its your wedding and the only people who have to be there are you and your FI! And its your special day and you need to do what will make you the happiest!! Good luck with everything!