Not inviting family friends causing issues

We have chosen not to invite a family FH was close with growing up: while the parents haven’t done anything, I’ve only met the mother once in 4 years and we generally don’t talk to them anymore.


The older sibling purposefully excluded me from their wedding, doesn’t really reach out to my FH, has never spoken to me since the wedding, and has shut down all attempts by FH to meet up with the four of us.
We are inviting the brother who is FH’s age and who he was especially close to growing up. They’ve grown way apart and the brother isn’t that polite to us now, but out of respect for the past relationship we will invite him.
The problem is that this is causing a lot of issues with FH’s family. His family is not especially close to this family anymore either (they all live pretty far) but they’re very angry that the whole family isn’t invited. Keep in mind that we’re paying for this whole event. I really didn’t want to have people there who don’t support our marriage (they didn’t congratulate us when we got engaged) but I don’t want to start off on such a bad foot with his family either. Should I just suck it up and invite them? The thought makes me physically sick but this doesn’t seem to be working either