I feel your pain! When we ordered the invites the woman at the store made a comment about calling people who didn't RSVP after the date had passed to find out the response. My FI and I refuse to call anyone. We had some people not rsvp to our engagement party and the stress and frustration from that has made us decide not to extend the courtesy. I have no sympathy for people who do not rsvp. Unless someone specifically calls you and asks for more time I would count them as a NO. When a plane ticket, time off from work, and a hotel reservation are needed I don't think anyone will decide last minute especially with the high prices. Good luck!!
P.S. My rsvp date is May 4th. We should compare notes and see how many people actually don't respond. Haha.