There are indeed other groups who experience this.

I once met a man who had discovered one day in his mid-twenties that he was actually an illegal immigrant. His parents hadn't been able to afford to bring him when they (legally) moved here, so they smuggled him into the country as a toddler and got him counterfeit papers. He stayed in a job where he was treated horribly, because filling out tax paperwork for a new employer might mean deportation to a country he barely remembered living in, where he had no friends or family.

I, myself, have a criminal record. A sex offense to boot, though nothing like the dark-alley, windowless-van offenses you're likely to be imagining. (You'll forgive me for not discussing the details.) I served my time, and it's been expunged in theory, but I was able to find my conviction using a free trial from an online service, along with a picture of my face and my address pointed out on a map. It had been expunged years before I went to college, but I still had to write a letter to a special review board as part of my application process, and I'm not allowed in any of the university's housing buildings. When a job application requires a background check, I no longer bother applying.

Of our three cases, mine is the most justifiable, since I actually did something wrong (genuinely wrong, not just illegal), but I'm afraid it might also be the easiest to explain to others. Everyone has been 18 and stupid, but illegal immigrants and trans people are alien and frightening.