I don't think we can infer all that much from unwillingness to testify. (At least, reluctance to testify. So far, no one has talked about subpoenas, just invitations. We don't know who among the witnesses would lawyer up and refuse to testify if subpoenaed.)
Whoever goes into that room to testify will be put on public display. They will have their private lives discussed by people like us, but also including everyone they know. Every new contact they make in the future will begin by the other party knowing, and having an opinion about, their life as a high school partier and someone who either was sexually assaulted, committed sexual assault, or lied about sexual assault. The questioning won't be limited to just this one alleged incident, and in the hyperpartisan atmosphere that will exist, whoever testifies will be in an environment where several United States senators will be very happy to charge them with perjury at the slightest provocation. Why would anyone want to be dragged into this?