Aristotle would agree that not hitting back is sometimes the courageous act (that is, hitting back might be rash).
And this is where we get to having-your-cake-and eating-it time.
Aristotle commends two virtues, courage and prudence. While condemning two vices, rashness and cowardice.
Well, we can all agree on that. The question is, where do we draw the line?
Your way of reasoning gives you a perfect way of idealizing everything that you do. When you take a risk, that's courage. When you don't take a risk, that's prudence.
Now let's look at the people whom you
don't like. Well, when
they take a risk, that's rashness, but when they
don't take a risk, that's cowardice.
Hooray, you get to have your cake
and eat it. You are very privileged in this respect. We mere mortals have to figure out morality for ourselves, but fortunately you are always objectively right.
I wish I was you, and was always right. But given the fact that everyone in the world has different views on morality, I guess that this could only happen to one person. How fortunate for you that it should happen to be you. How fortunate I am that I know you and can benefit from your infinite wisdom.