Are Sunday weddings really that bad?

I'm Jewish and my partner and I are planning to have a mostly-secular Jewish ceremony for our wedding. Since Friday night and Saturday are Shabbat, it's much easier to find a rabbi to officiate if the wedding is on a Sunday, and while we're not very observant I would be slightly more comfortable with a Sunday wedding than getting married on Shabbat.

At least half our guest list will have to be flying in (though we're trying to choose somewhere centrally located so it's as easy as possible), and I thought a Sunday wouldn't be that different from a Saturday-- either way you might need to take time off work for one day or a half day for travel. However, based on a lot of opinions on this sub, it sounds like lots of people think a Sunday wedding is a horrible inconvenience, e.g. that no one will drink and no one will dance and it's extra problematic to take Monday off, and that it might even scare guests off from attending at all.

Is a Sunday wedding really that bad? If we explain in the Save the Date or invitation that it's a Jewish ceremony, will that be a mitigating factor/make people more understanding or more willing to come? Would you be happier about a Sunday wedding if you knew it was for religious/cultural reasons? I really want my friends and loved ones to be able to come and to have a good time, and I'm sort of taken aback at the idea that putting it on a different day of the weekend would make as much of a difference as some people say.

I'd be willing to consider a Saturday wedding and trying to find a Jewish friend to officiate instead of a rabbi, but given that on top of the religious aspect Saturday is also more expensive and harder to book for a lot of vendors/venues, it would have to make a pretty big impact on people's experience to be worth it.

EDIT:
Thank you so much to everyone who's been commenting, I really appreciate all the responses!! I think we'll probably do a Sunday, since it sounds like a lot of people have had good outcomes in practice, but we'll poll our important people first in case anyone has an especially restrictive job/other reservations & make sure to do save the dates way ahead of time for people who need to take time off. I appreciate hearing from so many people who understand why it would be important to me not to get married on a Saturday & who've had good experiences with Sunday weddings, it's very reassuring