I would develop the cheapest possible solid rocket fuel, wrap it up in cardboard composite, and make standardized solid fuel boosters in volume so as to bring down production costs. I would then cluster these together into various launch configurations, put the guidance mechanisms at the top, and kick the whole thing to 3km/s @ 40,000ft (or thereabout) using a reusable jet-engine booster. I would then run like hell.
There's quite a bit more validity to this position than you may realize...
There are several vehicles at or near deployment that push launch costs down into the $10M range for moderately sized payloads -- total cost is often more important that "dollar per kilogram." To wit, I could design an Orion for you that launched 1,000,000 tons at 100 dollars per kilogram, but you couldn't afford it. But I digress.
Most of the new wave of low-cost launch systems are dumb, constructed simply, and use (at least some) solid propulsion. Solids are preferred for low-cost launches primarily because of the ease of handling. You don't need a tank farm, you don't deal with so many cryogenic fluids, and your operations are much simpler. The tradeoff is that solid rockets are generally less powerful in terms of specific impulse.
The Rutan spaceplane uses an interesting "hybrid" fuel, basically a bed of rubber that is solid and relatively inert which reacts with liquid nitrous oxide. Because the combustion components are safe and inert, relatively speaking, the rocket's design is straightforward and hence inexpensive. This approach is not quite ready for prime time, but it's a good example of newer thinking.
Totally reuseable vehicles are not practical at the moment, though. Materials technology simply hasn't caught up. Simpler propulsion will lead to lower demands on materials, however, although there is still the problem of re-entry.
In the extremely long-term, it seems most feasible to me that we will develop power transmission methods of launch -- targeting extremely high-energy lasers or masers on ablative targets covering the underside of our "launch vehicles," perhaps a highly inert ceramic that generates thrust as it is melted away. The advantage here is that the main lift engine remains on the ground, and safety is nearly 100%. Such a technique is also scalable to virtually any size, provided the key challenges can be overcome. (The main challenge is how to get that much power through the plume, once the burn begins, followed by creating beams with that kind of efficiency.) It would, however, require an enormous capital investment, one that would only pay dividends after a fantastic number of launches.
At some point it isn't worth making the launch vehicle cheaper. The cargo itself has to come down in price as well. Rad-hard computing, precision instruments, maneuvering, tracking stations, software, testing and certification costs -- to say nothing of environmental control for crewed birds -- are still comparable to the cost of the launch vehicle. All this has to get cheaper too, or else we're already approaching the point of diminishing returns in launch costs (for uncrewed systems only). Of course, economy of scale will help here as well.
I don't believe space elevators will ever be practical. Orion doesn't strike me as practical unless we're faced with a one-time desperate need, though I know other professionals who disagree with me on this point. Railgun launch is a maybe, but I've never heard a good way to deal with the atmosphere -- you kind of have to launch from a vacuum.
NASA scientist, by the way, though opinions here are mine alone, as always.