I agree, but that is only half of the story. My brain , and therefore qualitative experiences, are all subject to physical laws in the end. It is impossible to have something that is not contingent on physics.
BUT
Things are not that simple. Because there are systems in which the whole can influence the behaviour of the parts.
That is, billions of neurons are firing in my brain , my brain can then behave in such a way, that its behaviour constrains how the neurons will fire in the future.
It is like a loop , the neurons fire, I move my hand, I grab my coffee, I take a sip, this behaviour determines which neurons will fire in the future.
It is not only like neurons determine how the brain will act, it is also that the whole system determines how neurons will act. It is not only a feed forward system, there is a backpropagation .
And if Consciousness (i.e qualia) is also an emergent phenomenon of the collection of many neurons firing at once, this whole system also constrains how the neurons will fire in the future.
So , in a sense, it is possible that consciousness also influences the physical, because it is physical in that it is a process of several networks of neurons firing in harmoney.