All legitimately cast ballots should be counted quickly and accurately, yes, ideally (with the understanding that "legitimately cast" can open its own can of worms). The real world is not such an ideal world, unfortunately, especially when numerous other concerns get mixed in, rightly or wrongly (and with plenty of both in the US, at last check). With that said, though, I do quite support acting to make things much better than they are, of course, so that legitimately cast ballots will all be able to be counted swiftly and accurately (and, to add something more to that... verifiably!) in all but the most nonsensical situations.
With that said, your line of argument reminds me of a rule of thumb. The moment that one declares some specific thing sacrosanct in morality (for example, innocents should not be harmed, ever), is the moment that they are admitting that they're simply refusing to deal with the complexity of real life and the reality of conflicting priorities.