Nearly every X-series game has some manner of fork in its events. But said forks usually tie back into a common ending, X5 and X6 are unusual for the fact that they feature alternate endings.
Then there's the old question of what the hell happened during X4. The game presents it as either X or Zero handling things solo. The two storylines cannot co-exist as presented, and yet sequels insist that's exactly how it should be, leaving the fans to guess at how everything is supposed to tie in (I still think it'd be awesome if one day a sequel showed X being the one to kill Colonel, but that's just me).
and he's not at all relevant to the story any longer
I don't know what the hell game you were playing. Zero vs. High Max was the most ep
ic non-animated cutscene ever.
Story isn't exactly the highlight of the X series, but X6 was among the deeper ones. People just tend to underestimate it because of the lousy localization; they lose the will to even bother trying to follow along.
Also on DeviantArt, Rumble, DLive.tv, and the Fediverse (@freespeechextremist.com and @bae.st)