How Do You Narrow Down A Guest List??!!

Hello lovelies, Please help! We have submitted an initial number to the hotel for 100 people (we had to put the deposit down), however after doing a basic list alone, I'm at 74 (yikes) without his list. He says people need to get cut since per person it aint cheap. We both come from a Caribbean family in which my mother is treating like a second wedding and will say "No, you have to invite so and so!" How did you handle this? Were the people who didnt get invites upset with you?