And this exactly why it is so annoying to me, young upstarts are not supposed to win against more experience fighters
Says who? There are a number of factors that may contribute to the defeat of their opponent. The most obvious ones would be the opponent totally underestimating the hero just because they're younger and - presumably - less experienced; and/or the opponent in question being too cocky/arrogant for their own good. If you were challenged by a ten-year-old kid, you'd probably think you have the upper hand, right?
Then that kid pulls out some crazy martial arts/monster/trading card/McGuffin and whups your sorry ass.
Ouch.
What do you do then? Run back to your superior and say, "Hey, boss, I got beaten up by a ten-year-old kid earlier"? Like hell you are. Chances are, you'll either bugger off fast or, for whatever reason (humiliation, respect, impression, etc.), you'll decide to join the kid's party. Or it could be contributed to honour and valour and whatnot; see below.
while they may have the talent, they don't necessarily have the experience to back it up
Experience what? If you've got talent, use it. True, experience does help significantly - especially in real life, which we're obviously not discussing here - if you've got talent and you can exploit it, then exploit the hell out of it. If you're a Combat Pragmatist - one who plays "dirty", so to speak - then all the better (this doesn't seem to happen too much in shonen, though; too much emphasis on "honour and valour" and whatnot).
And just because they're older and more experienced doesn't mean they're better, or that they even know what they're doing.
and they are supposed to be 'friends' with the poor bastard they come across on their journey?
Yes. Honour and valour and whatnot; see above.
Also, bear in mind this is shonen/seinen manga. Since when did it ever need to make any sense?