It's not ALL a fetish. But a lot of what has cropped up recently certainly is.
Historically, part of the reason for the intensive clinical requirements before a person was allowed to transition was specifically to weed out AGP males, as well as those who are experiencing dysphoria as a transient event, often linked to other trauma or mental disorders.
The handful who were allowed to transition and live as if they were women to the best of their ability, were almost exclusively those who were HSTS (homosexual transsexual), whose homosexuality was extreme to a point of them wanting to be female so as to be more attractive to other males. For those HSTS males, the dysphoria has strong neurological components, including errant activation of the part of the brain that is responsible for our perception of ourselves (same area that activates with BIID and with anorexia nervosa).
The push to remove the clinical requirements and allow legal and social transition on the basis of self-declaration is largely driven by AGP males who were denied the clinical backing required. Clinicians identify those males as ones with a paraphilia - it remains a diagnosis in DSM-5, as a subclass under transvestic disorders. It is a sexual fetish, and a significant element of the fetish is wrapped up in being able to transgress female boundaries, and invade female spaces. That transgression and the discomfort it causes females, is part of what creates the sexual arousal in the first place.
Whenever you run across someone online (or IRL) talking about their "gender euphoria", be aware that this is a person talking about how they get an erection and feel sexual arousal at crossing female boundaries, and forcing females to take part in their sexual fantasies without consent.