Assigned Seating for 1 long table?

I think you've answered your question. I would suggest that you do assign seats, to prevent exactly what you're describing. I've seen where only the table is assigned, which won't work for your situation but works fine usually. I've also seen no assigned seating, which turned into kind of a free for all with people shifting chairs and rearranging tables to fit more or fewer people.

I agree with Jacks (and OP) for the reasons described. Also, in not assigning seats you'll likely end up with an empty seat here and there, and a family will be forced to break up and all sit separately. Best to begin with assigned seats, and if people feel like switching around, they can.