I share your concerns.
However, there is at least one more hurdle. This has been covered elsewhere, by me, Devil's Advocate, and perhaps others. The law that allows Congress to challenge electoral votes has some strings attached. You said
" Does it matter on what basis they challenge? Not at all, they don't have be honest. They can just claim "I believe there was election fraud" for example,"
By law, it does matter on what basis they challenge. "I believe there was election fraud", isn't good enough to meet the demands of the law under which they are pretending to operate, so the Supreme Court would strike down the challenge.
The reason I share you concerns, though, is that anyone who was willing to ignore the law, the votes, and the basic tenets of our Constitutioal Republic, probably wouldn't let a little thing like a Supreme Court ruling stand in their way. Through their actions this year, they have already shown that they are willing to stand aside and do nothing while someone attempts to subvert democracy and the rule of law. If they are willing to do that, I don't have confidence that in the future their successors won't go farther. Even though this attempt is fizzling out into nothingness, it still went a lot farther than it ever ought to have gone, and with a lot less protest than their ought to have been.