Indeed! Johnson supposedly tried to refute Berkeley by kicking the rock and saying that. It is, however, well-known as a fallacious refutation. If anything, the similarity of the argument makes my point.
According to realism/materialism, the rock that Johnson saw and felt when kicking it was entirely a brain-constructed hallucination. All of Johnson's experiences, which is what he appeals to in his argument, where in his head alone. Still according to realism/materialism, the real world is an abstract, colorless, formless realm of interacting electromagnetic fields that modulate our brain-constructed hallucinations. Therefore, if anything, Johnson's argument favors idealism: if the rock he felt was the real deal, then idealism holds.