What if someone started telling everyone you know that you are a pedophile? Would you defend their right to free speech too?
Absolutely.
You obviously dont get it.
Oh I do, Bubba, believe me. When I was young and naïve I too believed that freedom of speech was paramount. No exceptions. Then when I was about 20 (nearly 50 years ago now, blimey) I read about how American civil liberties groups were defending the right of American Nazis to celebrate Hitler's birthday by marching through Jewish areas of several major cities - past the homes of concentration camp survivors. And I thought "Uh, wait a minute ...".
What I belatedly realised, Bubba, was that sometimes rights conflict.
The right of those Nazis to freedom of speech conflicted with the right of concentration camp survivors to live the rest of their lives free from fear and intimidation.
The right of scumbags to spread malicious lies about you conflicts with your right to not have your reputation blackened without cause.
The right of crackpots and scumbags to cherry pick, misinterpret and misrepresent scientific data to make it look like it supports the opposite conclusions to the ones it actually supports during a major health crisis conflicts with the right of the gullible and scientifically illiterate to not be misled into making bad decisions which could seriously damage their health.
So there will inevitably come times when a choice has to be made as to which of the rights in conflict is going to take precedence. There is always a conversation to be had, but in practise most societies use a "greatest good of the greatest number" guideline to make such decisions. Hence such societies do, in fact, limit freedom of speech to prevent defamation, libel, hate speech etc.
When I say I would prefer that the gullible and scientifically illiterate not be exposed to disinformation it is your welfare, and that of others like you, of which I am thinking. I am neither gullible nor scientifically illiterate, so I am not in danger of falling for the BS myself.