No Kids Rule

We don’t want to have kids at the wedding but we’re getting pushback from family already. Our guestlist for a wedding next July is currently at 185. My cousins on my dads said are all out of state (Tennessee & Georgia) and my one cousin mentioned how excited her daughter (my 2nd cousin) is to come to the wedding. My FH has a larger family who is all local and majority of them will be attending the wedding . He is not inviting any kids on his side but I’m worried my side won’t come because of the rule and my side is already smaller 😔. Thoughts? If we invite kids, that will easily add 20 more guests 😩 (my side is semi big but his side is huge)!