Like the title says.
As soon as he got the Saber in Zero 1, his memory started coming back to him little by little, and by the end of Z1, although he couldnt remember it exactly, he was still able to tell from their fight that Copy X was far weaker than X was. (physically AND mentally)
In Z2, Phoenix Magnion uses that attack where he summons from Zero's past Colonel, Bit, Agile, and Vile. I mean, I get that there's a difference between what he REMEMBERS, and what is in his brain but locked away in his subconscious, but, did he actually remember them? or was Phoenix merely drawing from Zero's brain data, which regardless of what he remembers, no doubt has data related to them?
By Zero 4, when Ciel mentions Area Zero, he mentions it's a "painful reminder of a war long past", which, always seemed to me like he was implying he remembered the Colony Drop. And of course, his [tornado fang] you Weil speech, where he says he never called himself a hero, etc etc.
So, just HOW much of his memory had Zero regained by the end of Z4? I don't think he ever regained it fully, but surely he got some of it back.