Here's the thing with the personal experience card. Sometimes it's valid and sometimes it isn't.
Your average woman would know and notice a lot more of ingrained sexism in society, thus the experiences of a woman are more valid than the general day-to-day observations of a man. Similarly a black person will know more about racism than a white person, a trans person know more about transphobia than a non trans. All perfectly reasonable, all perfectly legitimate.
A woman from London who has never left the UK on the other hand, will likely know next to nothing about the sexism inherent in Vancouver, say, whereas a man from Vancouver may or may not know more. A black man who has lived all his life in Nairobi won't know about racism in New York etc etc.
Equally, you might be a particularly fortunate or unfortunate individual experiencing less or more of the inherent prejudice than another member of the same group. A lucky black man (in this particular sphere of luck), maybe a rich dentist may know less of racial struggles than a black man from a poor neighbourhood because he's experienced less, whereas a black guy in the rural Deep South might know more than a suburbanite because he's experienced more.
That's why critical evaluation is a useful tool. Personal experience, particularly when weighted with membership of a minority or underprivileged class is a useful knowledge base, but even within such classes, there is a lot of variation.
Plus, of course, being a member of a particular group doesn't mean that you're immune to false positives. I remember being told a story by an American friend (and, truth be told, it's incredibly difficult to imagine the same thing happening in the UK, so you're absolutely right about experiences in one country not necessarily translating to experiences in another) at his work. A delivery guy, rather than going in via the delivery entrance at the back of the building, went in through the front. My friend, while signing for the parcel, jokingly said "aren't you people supposed to go round the back?". The delivery man was extremely offended because he was black, and thought that my friend was saying that black people should use the back entrances of buildings. The fact that there were potential racial implications to what he was saying hadn't even occurred to my friend, who was making a generic delivery person joke.
Now, they sorted it out and shook hands but had they not, had the delivery guy not made a big deal out of it (either because he didn't feel like getting into an argument, or because he didn't have the confidence to make an issue of it) that delivery guy would have gone away thinking that he'd just experienced a shocking example of racism. And perhaps there's a case to be made that my friend
should have thought of the potential implications of what he was about to say (although, ironically, that would have involved thinking of the delivery guy's race as his primary characteristic, which I'd say was one of the defining characteristics of racism), but I don't think that you can say that what he said was actually racist. Certainly not in intent.
But how many times do people generally believe that they've experienced some form of prejudice when they actually haven't? This is not to belittle or discount the experiences of those people, or indeed to claim that prejudices don't exist and aren't a huge problem, but just to point out that our experiences of reality aren't necessarily an accurate representation of reality and that they therefore should always be approached with a degree of scepticism, and empirical data should always be preferred, if such a thing is possible.
It would have been legitimate for the delivery guy to feel like he'd just encountered racism. That doesn't mean that he did.