This is one reason why organization such as FIRE are valuable, and the fact that both liberals and conservatives support them is an asset, not a liability.
While this is a fair point, it also suggests that you may not be quite understanding the issues that lead to a understandable fundamental distrust of FIRE.
To sum up how "conservative" strategies handle problems far too often -
In the background, they work to remove the safeguards against the problem and shape policy to worsen the factors that actually drive the problems, thus making the problems more likely to happen. The actual goals in play may not be to cause the problems, of course, but collateral damage is still damage.
In the middle ground, they work to shape public opinion about the problems with a bunch of less than candid tactics, often perverting principles in the process as they spin the situation to gain some advantage.
In the foreground, there are big performative actions following up on that, once a critical mass has been reached. Those actions never seem to actually do much to make the problems or the drivers for the problems better, though.
FIRE's reason for actually existing is, in short, to make college faculty and students more "conservative" and to make policy more friendly to "conservative" goals, as part of larger efforts to shape the US. Its mechanism for doing so is to advocate for a right-wing version of "free speech" principles and serve as a beacon to show how much "conservatives" support Free Speech, at the same time as other conservatives, often funded by the same organizations, work to erode Free Speech. We've seen similar in a nutshell with Musk, really brazenly, as he's claimed to advocate for Free Speech while actually seeking to control speech, but much the same seems to be nigh ubiquitous among Republican administrations these days.
To go bit further, at a fundamental level, the value of Free Speech is increased by the positive effects that it produces and decreased by the negative effects produced. The usual right-wing perversion of Free Speech makes Free Speech the end goal, not the positive effects it produces, which then serves as protection against consequences for the lies they push and the harm being caused by them and reduces the value of Free Speech as a whole.
FIRE supporting Harvard and opposing the crap that Gorka spat out is pretty much a no brainer. What Trump's trying to do there (and with pretty much everything else) is horrendous for Free Speech. Both liberals and conservatives should indeed be working together to oppose Trump's assault on Free Speech, because it very much is harmful to all of us. FIRE can indeed serve a valuable role in that. That FIRE and those behind it played a role in bringing us to the point where Trump's even in a position that he can be be doing what he's doing, though, is still an entirely relevant issue to raise and an entirely relevant reason for fundamental distrust.