Yes it's a different part of ourselves that reincarnates. It does not remember past incarnations, but it carries with it the collective spiritual evolutionary status of past lives. We are more than we were before, and we are likely to manifest character that is the sum of our past lives.
We may have qualities like being stronger, wiser, more compassionate beings because of previous lives. A spirit guide speaking through a trance medium in a lecture I attended said that Mozart had been a musician in nine previous lives before he was born Mozart the prodigy. So we carry over what we have learned in past lives as latent characteristics of the new life.
Mozart (W.A.) was the son of an avid and successful musician and composer (by the standards of the time; music was a social pursuit with few opportunities for full-time professionals.) Mozart Senior was also a teacher, and the author of a well-received music instruction book, who intensively trained his son in music from his earliest childhood, foregoing further advancement in his own music career to do so.
The "born Mozart the prodigy" narrative seems to be leaving that part out. I wonder why?
Of course, you might claim that naturally the spirits would choose a music-rich learning environment for the next incarnation of a spirit who had already been a musician in nine previous lives. That way, "nature" (genetics), "nurture" (the music-rich learning environment young Mozart grew up in), and "spiritual evolutionary status" would all align toward creating a prodigy. That makes sense, right?
Actually, no, in the context of the rest of your narrative, it doesn't. If your view of reincarnation were situated within a traditionally polytheistic milieu where gods and invisible spirits meddle around with human affairs in order to create prodigies (or more traditionally, great heroes) from time to time, mostly for their own amusement, then sure, that would fit.
But if the purpose of reincarnation is for fledgling spirits to experience and learn lot of different things,
why would a spirit who had already experienced nine lifetimes of being a musician need more of that kind of experience? Do we
all need to experience being one of the greatest musical prodigies in the world in order to move on, and if so, how long is
that going to take? (It's not as though the Mozart spirit had already perfected every other life skill. For instance, he at times lived beyond his means and suffered from debt and penury as a result, a common failing of a great many incarnated spirits then and now.)
I remember hearing that the soul retains perfect memories of all events of past lives, and if there were any doubt about a memory everything that ever happened leaves an energy trace that can be accessed in the higher realms of the spirit world. So we could review everything we have done and see it again. I do not know how memory is stored in the soul. It is not a question I ever though of asking a spirit guide when I used to attend lectures. What I learned comes from the 1970's and I have not been in contact with any church for twenty years. So I can't learn anything new.
Maybe I've had nine previous lifetimes of being people who, when told a tale of marvelous processes doing marvelous things, were inclined to ask, "how does that actually work?"
Perhaps that's why I've reached a state where, though no prodigy, I've gained some confidence concluding that when the answer appears to be "we cannot possibly know, because the process is entirely inherently invisible, intangible, ineffable, and untestable," the actual answer is that the process doesn't exist in the claimed form at all. We can't answer the question of what kind of glue sticks the stars to the firmament, not because the stars are too far away to tell or because the glue is made of abstract matter that only exists in a higher plane, but because there is no firmament.
In the case at hand, there is no evidence for intangible spirits having vast perfect memories, and there seems to be no way to acquire such evidence. There is lots of evidence for vast amounts of all kinds of information being stored in, and transferred from person to person by, material things in the world. One tiny example is the textbook on violin playing written by W. A. Mozart's father, which can still be examined and read today.
Therefore I think it's far more likely that we, like the Mozarts, evolve in our understanding not by storing up individual memories and experiences between one individual incarnation and another, but by adding to and receiving from the store of understanding in the world. (Which you and I are even doing right now with this dialog, so good for us!) It's not the solitary selfish individual process you describe, but a shared endeavor. The Buddha had at least one genuine revelation under the Bodhi Tree: our individual selves are largely illusion. If you accept that and ignore your ego's protests to the contrary, then you'll see that you will be (and already have been) reincarnated not serially but in parallel, as everyone else, and that the karmic accounts, such as they actually are, are kept in plain sight in this world, not in some mysterious invisible elsewhere.