While I hate to make the distinction, there is a difference between a FOREIGN terrorist organization (FTO) and a DOMESTIC terrorist organization.
I believe I understand your concerns, but because there is a difference between the two types, there is a political effort to remove citizenship from someone who pledges allegiance to a FTO on the grounds that by doing so, they have voluntarily renounced their allegiance to the United States. Currently, if I'm understanding what I Googled properly, a natural-born American citizen can only lose their citizenship if they renounce it themselves (although I'm not sure if there's an official procedure for that sort of thing), but naturalized citizens can have their citizenship revoked for any number of reasons, allegiance to a FTO among them (and many individuals who are doing so are naturalized citizens, not natural-born, although there are some natural-born ones who are gravitating toward FTOs for whatever reason). Following September 11th, numerous politicians wanted to amend the law to allow for revoking U.S. citizenship if it could be proven someone pledged allegiance to a FTO on the grounds that by doing so, the person had voluntarily renounced their U.S. citizenship, but I don't know how far that effort got. It does seem I wasn't entirely accurate in saying that is one of the few ways to lose U.S. citizenship, but the effort has been made in the past, so it's still a possibility I think.
Regardless, it's a different story if you're talking about a DOMESTIC terrorist organization, of which there are several in the U.S. But because they are homegrown, the idea of renouncing citizenship becomes much more of a sticky wicket than doing so if someone is pledging to a FTO; the actions of the domestic groups are usually entirely confined to this country, and while they are illegal in the extreme, there is no aid of any kind being given to any foreign entities, recognized as governments or not, so there is no grounds to remove citizenship there. The vast majority of domestic groups I am aware of still view themselves as U.S. citizens, even if they don't like how the government is being run or think violent protest is the right way to effect the changes they are after; very very few, if any, view themselves as other than U.S. citizens.
I guess what I'm trying to say is, if someone pledges allegiance to ISIS or any of the FTOs currently on the State Department's recognized list, they could theoretically be viewed as candidates for loss of citizenship, but if they're doing so with, say, the KKK or the ELF, for example, they wouldn't be because those organizations are, sadly, operating solely in the U.S.