No, it's true according to that. The rocket has to undergo acceleration when it leaves the Earth, when it slows down turns around and speed back up, and when it slows down once more on arrival. That's what makes the time going slower on the rocket matter and makes the rocket's reference frame fundamentally different from the Earth's, because acceleration is NOT relative.
You CAN view it as "speeding up the universe" in the same way that running makes the Earth move around you because all the objects are now moving relative to you. Both are equally "crazy."
Again, time dilation has been observed in numerous experiments. It's extremely well confirmed.
I'll find or make some graphs for you demonstrating how this makes sense later (gotta sleep now). I admit it is very counter-intuitive, but it is true and backed up by all evidence.