Is this boutonnière too big?

It's beautiful. I think we would have to see it on a person to judge.

Just remove the thistle from the side and put it behind the rose sort of stacked in the middle.

Or, use a smaller filler. And if you want to make the flower tighter, just put a wire through each side and pull them tight and twist at the bottom. It will close the flower and make it tighter and smaller.

I think removing the greenery could help make it smaller if you're wanting that!

4" long boutonniere

For someone else on a post several months ago, I measured H's boutonniere from daughter's wedding. It was exactly 4" in length and seemed perfectly proportioned. (And, he's 6'3", so a tall man.)

Yes, I think it's too big.

I actually think it's fine as is

I personally don't think so! But I also think it depends on the size & height of the person.