I'm a capable man and I have had to do it in the past but it isn't much fun. Even if you know what you are doing the basic un-fun things are; You usually get dirty; Undoing the wheel nuts can be physically exhausting or even impossible; There is stress over whether the spare will be properly inflated when you lower the car back down - this one is ok at your home but by the side of the road it is a real pain to discover the spare is flat too. Plus, other traffic is very unnerving. And in general, it is something most people don't do very often, if at all, so it is natural to be nervous because you don't want your car falling off the jack or whatever.
I think I can understand why some would shrink from it. Maybe it should be part of the driving test? At least as theory. I can also appreciate that some females, and men too, may have difficulty in removing wheel nuts. Especially those that have been tightened by a windy gun and been in place for some time. The answer for drivers that lack confidence in wheel changing is to join the AA or RAC of course. Personally it's not a problem for me as I weigh 20 stones and have had plenty of practice.