I am familiar with the pyrotechnics hobbyist literature. You are no doubt aware that Weingart, Shimizu, and Rev. Lancaster's books, while being "how to" manuals for fireworks, don't cover HE (maybe a little, it's been awhile, but I think Lancaster had some). And, even their salutes wouldn't make practical destructive devices in the way that would interest a terrorist - in contrast with what you'd find in the Anarchist's cookbook or Inspire.
There is certainly overlap. But I do think that recipe books directed specifically at amateurs looking primarily for a "bomb build" are enough of a distinct category to be usefully targeted. Some of the stuff out of Paladin Press would cross the line - the "Black Book" or "Poor Man's James Bond" would be good examples.
I agree that someone could still learn the craft. My parallel would be comparing a trained chemist to someone without skills wanting to make meth. It's about barriers to entry, skin in the game and making impulses harder to act on - even though you can't prevent a dedicated student from pursuing a degree in mayhem.
I've been a member of PGI and other organized pyrotechnics clubs. There is a wealth of knowledge, but even in this group, I found it a rare member who knew how to make/use HE. They had a healthy respect for "the line." And gaining the practical knowledge to make reliable, functioning pyro didn't translate directly into what the Kewl Bomz Kidz were after. If you've dabbled, you know the level of technical expertise is different, even between making a good BP and the most basic HE. It isn't simply a matter of mixing A and B, there's the whole theory bit, the explosive train to consider, actual laboratory chemistry instead of just balancing an oxidizer and fuel.
I also understand the kind of low-power devices used in Boston are fairly simple to construct by re-purposing existing materials. But the Oklahoma bombing level of destructive devices are not.
And again, someone traveling the path of Tim McVeigh, without a clear cookbook to follow, will have to do a certain amount of experimentation - trial and error that can very well reveal their nefarious intent before they are able to do the nasty deed. So I think it's useful to restrict, as much as possible, all the resources we can. Not with the expectation of preventing all we'd like to, but preventing more than if we adopted the laissez-faire stance.