Honestly, every game has issues, most just bandwagon on Sonic '06 because it was rushed to Christmas and technically an unfinished game, resulting in many, many issues; they took that, the load times, and the bad story and accentuated the negatives way out of proportion. If a glitch happened, it'd be comparable to how ProtonJon's LP of Superman 64 would be counted against the game along with everything else. I hated the game myself, but mainly because it didn't feel right to me rather than 'holy [parasitic bomb] glitches', and just seemed off the entire time. Also mach speed sections. (Then again, this is also opinion, so sorry if I seem like i'm lumping in everyone who hates Sonic '06 together; I just see a lot of people pretty much state 'glitches', 'story', or 'loading times' as the problems and often seem like they only watched the infamous Pokecapn LP of the game instead.)
On-topic, X4 is probably mainly gushed over due to nostalgia, but that doesn't mean it's a poor-quality title or anything. Takes the formula of the three games before it, tries to step it up a notch and doesn't quite hit all the targets. Weird boss balancing, mostly seen in Final Weapon, like the General with Zero and the fact that one form of Final Sigma is harmless, but the other form can be a [tornado fang]ing nightmare for either character; a larger emphasis on story that shafts X a bit for Zero, give or take the opinions on the matter; and inconsistent stage design (the Rider level for Stingray was a full two-parter, yet Storm Owl gets one moderate level part and then the second part is less than a minute in length if you can beat the miniboss fast enough; Final Weapon, again, is also a very weird design).
The soundtrack is something I can remember, and I prefer the game, in some aspects, over X5 as it just feels more polished than its sequel, but at the same time the game just feels very different from every other entry, even with X5 & X6 continuing gameplay-wise from X4. Kind of like how X2 and X3 always felt strange compared to X1.