Are courthouse weddings religious?

Courthouses are not religious places and don't have priests or any other type of clergy.

Correct. You would only be married by the Judge or Justice of the Peace (depending on your town's structure). As they are elected government officials, they wouldn't be able to perform a religious ceremony even if you wanted them to.

It's done by a judge or a Justice of the Peace. When I got married the first time, though, the city's preference was for you to get married by a JOP outside the courthouse. We found a lovely man who had a whole little setup adjacent to his home. However, without discussing it with us first, he *did* include some religious quotes in the service. I'd be really surprised for that to happen with an actual courthouse wedding, but it's still worth mentioning your preferences, just in case.