Its not a jump at all. You stated: "I'd go a step further and just get rid of parties". A political party is an association of people who come together to further their collective political aims. The only way to "get rid of them", would be to make them illegal.
OK, hypothetically, Congress passes a bill completely banning political parties, and Trump signs it into law. The SCOTUS, for some reason, upholds the new law. How should the 2028 election be organized in your opinion?
Ohh, and please do note that I've been a registered independent since 2008, and that I do believe the primary system we have has caused increasingly worse candidates in recent years. But, thats because most people do not pay attention to them. Ergo, inattention and apathy by the public are why we keep getting two candidates that don't appeal to the mainstream and we are left with choosing the lesser of two evils in most elections.