Goddamned bats. (Anything that attacks you when you try to jump, causing you to fall into a pit or spikes.)
Specifically, those things with the helicopter blades (prominently seen towards the end of the intro stage to X1). They'd get me all the time there and in Chill Penguin's stage (at least in the GBC version of the stage).
That's why you memorize the flight path and appearance spots; don't know how, but I was always better with the X series in this regard than the Classic.
What gets me are level parts that you have to memorize PIXEL-PERFECTLY, or DIE, Quickman's stage being a prime contender with some parts of X6 coming to mind.
Enemy-wise, I can handle just about every normal enemy type, but the bosses tend to cause trouble (at least, again, in the Classic series). Of course, once I pull out their weakness, I make thy [twin slasher] face.