Figuring out which is "the assaulting group" is often problematic.
Example: I saw some video of the May 1 confrontation in Portland outside of a bar. An Antifa crowd was there. Some Proud Boys (I think) started verbally harassing them. The Antifa folks, of course, responded. Then one of the Antifa crowd threw a drink at the Proud Boys. One of the Proud Boys responded with pepper spray. One of the Antifa people then pulled out a truly amazing pepper spray aerosol sprayer. This thing made a cloud that was huge. It covered a front that was about six people wide. I don't know where you could buy such a thing, because it sure as heck wasn't a personal defense device. Can you buy riot control gear for personal use? Or was it a homemade contraption? Anyway, at that, most of the Proud Boys dispersed, but on the edges of the camera view, outside of the pepper spray cloud, you could see that a few scuffles had broken out, and news reports indicated there were injuries.
So, who assaulted whom? The first verbal harassment was the Proud Boys. The first physical escalation was Antifa. The first pepper spray was Proud Boys, aimed specifically at the drink-thrower. Then there was the general pepper spray cloud from Antifa aimed at anyone near them. Who would a jury say was guilty in that scenario?
For my part, I would be generally more critical of the Proud Boys, because they had no reason to be there in the first place. They were looking for a fight. On the other hand, Antifa obliged. It's not straightforward to cast blame in that case.
By the same token, the more recent violence in Portland had the police telling people that the Proud Boys should go to one place, and Antifa should go to the other. Antifa defied the orders, and by most accounts, also engaged in physical attacks when they got there. Once again, I'm sure there is plenty of blame to be cast at both sides, but on balance, it seems to me that Antifa was more of an aggressor.
And what about the case where one group has a parade permit, and a different group physically blocks their way, preventing them from using the space for which they have a permit? Are those blocking the way valiantly defending the streets? It seems to me that they are breaking the law. I won't give a pass to people who initiate violence in order to continue through the blocked street, but what ought to happen in that case is that the "parade" ought to stop, and they ought to call on the local police, who ought then to order the people blocking the path to disperse or be arrested. If that doesn't happen, violence is inevitable.
And if that street-blocking only occurred when confronted with the neo-Nazis at Charlottesville, or the Proud Boys in Portland, I could at least have some sympathy. However, I've seen video of the same sorts of folks in Antifa-style clothing do that to pro-Trump marches. I'm not all that fond of Republicans, and even less so of the Trump wing of the party, but if they get a parade permit, people ought to get over it and get out of their way. Jeer at them from the sides of the road if you wish, as is your right, but stay within the law.