Ugh. That sounds rough. I can see why you would be frustrated. From what I'm hearing it's not even so much about the actual holiday (although I know that is a factor) as much as the fact that you and Billy had a deal (seemingly) that this year 4th of July would be with your family and he not only ignores the deal but choses to pursue splitting up the family even though he knows this is upsetting to you.
We tend to alternate big holidays (Thanksgiving, Christmas Eve, Christmas, Easter - even though Brian and I don't celebrate Easter) every year and stuff like Memorial Day, 4th of July etc, we just spend alone. Since the 4th of July is a big deal for your family I can see why you would want to take the kids to that (hell, even I want to go, sounds like fun!). I'm not quite sure what the solution would be to resolve the 4th of July arrangements but maybe this whole situation is a good opportunity for you guys to at some point discuss a plan for holidays and come to a final agreement on how to handle holidays. Whether it be alternating every year or having set holiday's you spend with with certain families (ie. Christmas with yours, Thanksgiving with his), etc.