On Vaccine Development and Testing

[Roboramma]

I've been thinking a little about the timeline necessary for vaccine development. From what I've managed to understand so far, the development of a vaccine can be done relatively quickly, but the testing phase takes a considerable fraction of the total time before widespread deployment is possible.

What I'm wondering is:

1. Is that testing mainly looking at safety or efficacy? Or are both equally important concerns? That is, when a vaccine is first developed are we relatively confident that it will be effective, but not confident in its safety? Or vice-versa? Or both?

2. On both of those issues, what is the degree of risk? For instance, would a newly developed but not yet tested vaccine have a 0.1% chance of adverse side-effects? 0.001%? 10%? Similarly for efficacy? What are the chances, at least based on past vaccine development data, that a newly developed vaccine won't be effective?

I think you can probably see where I'm going with this, which is that given the current situation, might it make sense to begin widespread use of a newly developed vaccine before it's been tested to the degree that we would want to test in more normal situations? The usual testing regime is based on a cost-benefit analysis in a different situation, where the benefit of deploying a vaccine early isn't all that great and we want to make sure it's as safe and effective as possible.

But the current situation will lead to a different cost-benefit analysis. I have no idea to what extent that would change policy, if at all, but it seems worthwhile to consider that old policies made in normal times about testing might need to be reassessed for the current situation.

 

[Roboramma]

To give at least some information to help answer my own questions, I just saw this in a New York Times article:

https://www.nytimes.com/2020/03/22/health/coronavirus-restrictions-us.html

In the past, some experimental vaccines have produced serious side effects, like Guillain-Barre syndrome, which can paralyze and kill. A greater danger, experts said, is that some experimental vaccines, paradoxically, cause “immune enhancement,” meaning they make it more likely, not less, that recipients will get a disease. That would be a disaster.

One candidate coronavirus vaccine Dr. Hotez invented 10 years ago in the wake of SARS, he said, had to be abandoned when it appeared to make mice more likely to die from pneumonia when they were experimentally infected with the virus.

In theory, the testing process could be sped up with “challenge trials,” in which healthy volunteers get the vaccine and then are deliberately infected. But that is ethically fraught when there is no cure for Covid-19. Even some healthy young people have died from this virus.

 

[The Great Zaganza]

We are talking about a drug that would potentially be used, possibly again and again for years, by billions of people.
At these scales, even very rare side-effects happen in numbers as to be worrisome.

Add to that that the early recipients would be those most at risk to die from the virus, who also happen to be the ones with already compromised immune systems; i.e. those who might have a weak immuno-response to a vaccine in the first place. It is far from clear that a vaccine tested on young subjects would actually benefit those who need it most.

The decision to use it at scale will depend largely on the availability of an effective treatment.

 

[Puppycow]

What I'm wondering is:

1. Is that testing mainly looking at safety or efficacy? Or are both equally important concerns? That is, when a vaccine is first developed are we relatively confident that it will be effective, but not confident in its safety? Or vice-versa? Or both?

The testing is done in phases. Usually, there's an animal testing phase first. I believe they skipped that phase or decided to do it concurrently rather than sequentially to save time. A Phase 1 trial is the safety phase. It will generally test a small number of healthy subjects, and it's only for testing the safety. Phase 2 and Phase 3 are for testing the efficacy (and also the safety, just in case some safety issues come up that weren't found in Phase 1). Each phase generally tests more and more subjects.

https://en.wikipedia.org/wiki/Phases_of_clinical_research

 

[Roboramma]

We are talking about a drug that would potentially be used, possibly again and again for years, by billions of people.
At these scales, even very rare side-effects happen in numbers as to be worrisome.

Sure, but that has to be compared not with nothing, but with not using the vaccine, or using it later.

You have to compare the number of people killed or made sick by the vaccine to the number of people killed by the virus in the absence of the vaccine, during the period of delay caused by testing.

 

[The Atheist]

You have to compare the number of people killed or made sick by the vaccine to the number of people killed by the virus in the absence of the vaccine, during the period of delay caused by testing.

And if the testing has corners cut and it's 10% fatal? You'd have made matters a lot worse, and I bet it would kill indiscriminately, unlike the virus, which is pretty targeted at Boomers in the death column.

I saw Christine Lagarde say she expected Euros to have viable vaccines this year.

Either she knows something we don't, or she's indulging in a little wishful thinking to calm the people.

 

[Puppycow]

I think they are cutting some corners, but they aren't exactly starting from scratch either. There was work on a SARS vaccine in the past that got abandoned because SARS was contained at an early stage. They could sort of pick up that work where they left off. In addition, they make a new flu vaccine every year. A lot of the ingredients that go into a vaccine are already known, and it's only really the active ingredient that they need to test.

A phase 1 trial began a week ago.

https://www.nih.gov/news-events/new...trial-investigational-vaccine-covid-19-begins

I'm sure similar work is also happening in other countries. I don't know if they've gotten to the phase 1 stage yet, but a candidate vaccine has already been developed. Now it's just a matter of putting it through the trials as fast as they can manage to do that, and also preparing whatever they need to have ready to mass-produce it assuming that the trials are successful.

 

[Roboramma]

And if the testing has corners cut and it's 10% fatal? You'd have made matters a lot worse, and I bet it would kill indiscriminately, unlike the virus, which is pretty targeted at Boomers in the death column.

Absolutely. Which is why I'm not saying that we should be skipping steps, I'm asking how likely that 10% fatal (or whatever) number actually is. There's a reason that my OP is phrased as questions: I don't know what the answers are, and the usual testing might still make perfect sense. If you look at my second post in this thread you'll see some reasons for that testing, for instance.

We've done plenty of vaccine testing in the past. So we should have some idea about how often complications arise in the first stages of testing.

There's also a potential middle way, which is just to expand the number of people in the first stages of testing. That testing stage that has 100,000 people? Expand that number to 10,000,000, for instance (but only if doing so makes sense based on the data, I'm only saying that this isn't an all or nothing thing where we either put an untested vaccine out for billions of people or follow the usual testing procedures).

 

[Puppycow]

Another corner I could imagine them cutting (and it's a calculated risk of course) is after phase 1, go ahead with phase 2 and 3 concurrently. Normally of course, you would never do that, but since time is of the essence here, it might be worth the risk?

Just a thought.

 

[Puppycow]

Absolutely. Which is why I'm not saying that we should be skipping steps, I'm asking how likely that 10% fatal (or whatever) number actually is. There's a reason that my OP is phrased as questions: I don't know what the answers are, and the usual testing might still make perfect sense. If you look at my second post in this thread you'll see some reasons for that testing, for instance.

Presumably if there's a 10% fatality risk, that would show up in the phase 1 trial. There's 45 volunteers currently being given the experimental vaccine. It would be highly unlikely that all 45 of them would be OK under those circumstances. Phase 2 typically involves 100-300 subjects and phase 3, 300-3000. Obviously, you want to minimize the risk, which usually means gradually increasing the number of subjects, but since every day you spend doing testing could mean thousands or tens of thousands or more people dying out there in the world, maybe you start phase 3 pretty soon after you start phase 2 (and before phase 2 is complete)?

There's also a potential middle way, which is just to expand the number of people in the first stages of testing. That testing stage that has 100,000 people? Expand that number to 10,000,000, for instance (but only if doing so makes sense based on the data, I'm only saying that this isn't an all or nothing thing where we either put an untested vaccine out for billions of people or follow the usual testing procedures).

Generally a trial would involve fewer people than that, but there is something called phase 4, which is monitoring after approval for side effects.

 

[The Atheist]

We've done plenty of vaccine testing in the past. So we should have some idea about how often complications arise in the first stages of testing.

Even if we did, it wouldn't be valid, because they're all different and you can't draw inferences from any other trial, or even an average of other trials.

 

[Ivor the Engineer]

Is this vaccine going to be made in volume using eggs like the seasonal flu vaccine?

If so, how many months could be saved to deployment by using a process (if one exists) that doesn't require eggs?

 

[Ivor the Engineer]

Partially answered my own question:

https://www.cdc.gov/flu/prevent/how-fluvaccine-made.htm

There is a third production technology for flu vaccines that was approved for use in the U.S. market in 2013 and that involves using recombinant technologyexternal icon. Recombinant flu vaccines do not require having a candidate vaccine virus (CVV) sample to produce. Instead, recombinant vaccines are created synthetically. To make a recombinant vaccine, flu scientists first obtain DNA, i.e., genetic instructions, for making a surface protein called hemagglutinin (HA) found on influenza viruses. HA is an antigen, which is a feature of a flu virus that triggers the human immune system to create antibodies that specifically target the virus. This DNA for making flu virus HA antigen is then combined with a baculovirus, a virus that infects invertebrates. This results in a “recombinant” virus. The role of the baculovirus is to help transport the DNA instructions for making flu virus HA antigen into a host cell. Once the recombinant virus enters a Food and Drug Administration (FDA) qualified host cell line, it instructs the cells to rapidly produce the HA antigen. This antigen is grown in bulk, collected, purified, and then packaged as recombinant flu vaccine. These vaccines are then quality and potency tested by FDA prior to FDA approving release of the vaccine lots to the public.

This production method does not require an egg-grown vaccine virus and does not use chicken eggs at all in the production process. While there are other vaccines on the U.S. market that use similar recombinant manufacturing processes, there is only one influenza vaccine produced using recombinant technology approved by the FDA for use in the United States at this time. This production process is the fastest because it is not limited by the selection of vaccine viruses that are adapted for growth in eggs or the development of cell-based vaccine viruses.

 

[Roboramma]

Even if we did, it wouldn't be valid, because they're all different and you can't draw inferences from any other trial, or even an average of other trials.

Of course it would be valid. It wouldn't tell us what will necessarily happen with this one, but it would establish a base rate at which such complications tend to arise.

Based on current knowledge, there's some probability of complications from an untested vaccine. What is that probability? You think knowledge about past vaccine development doesn't tell us anything about that probability?

 

[Giordano]

Sure, but that has to be compared not with nothing, but with not using the vaccine, or using it later.

You have to compare the number of people killed or made sick by the vaccine to the number of people killed by the virus in the absence of the vaccine, during the period of delay caused by testing.

Exactly. But the complexity with a vaccine is that you give it to healthy people, so a modest risk that would be acceptable in a cancer treatment drug would not be acceptable in a vaccine. You might think that most people will eventually be infected by Covid-19 so any vaccine with a risk below that of the virus infection itself would be acceptable. But we don’t really know the risk/death rate for the virus in every population and for all possible treatments. Many people will never be infected. And many infected will have mild symptoms.

We will need to understand all that, plus the risks of any vaccine in various populations, to even begin to do the calculations of risk to benefit.

 

[Giordano]

Is this vaccine going to be made in volume using eggs like the seasonal flu vaccine?

If so, how many months could be saved to deployment by using a process (if one exists) that doesn't require eggs?

I suspect it is pretty easy to quickly ramp up fertile egg production. And relatively cheap. There already are in place huge commercial chicken breeding and egg production facilities.

 

[angrysoba]

And if the testing has corners cut and it's 10% fatal? You'd have made matters a lot worse, and I bet it would kill indiscriminately, unlike the virus, which is pretty targeted at Boomers in the death column.

I saw Christine Lagarde say she expected Euros to have viable vaccines this year.

Either she knows something we don't, or she's indulging in a little wishful thinking to calm the people.

******Opinion Warning*******

They could just give it to the Boomers first, in place of animal testing.

 

[The Atheist]

You think knowledge about past vaccine development doesn't tell us anything about that probability?

No, and I don't see how it could, apart from any vaccines relating to coronaviruses. Why would testing an influenza or rubella vaccine give an insight into something completely different?

******Opinion Warning*******

They could just give it to the Boomers first, in place of animal testing.

Haha!

That's not a bad idea. I'd just about give it a go.

 

[Roboramma]

No, and I don't see how it could, apart from any vaccines relating to coronaviruses. Why would testing an influenza or rubella vaccine give an insight into something completely different?

Then I suggest you learn something about bayesian probability.

The actual probability that an untested vaccine shouldn't be used is either 0 or 1. We just don't know. Probabilities are based on our current knowledge, and historical information used to give a base rate is obviously relevant.

Knowledge about the differences between diseases and how that might impact any particular vaccine's likelihood to work is of course also relevant.

 

[Roboramma]

Exactly. But the complexity with a vaccine is that you give it to healthy people, so a modest risk that would be acceptable in a cancer treatment drug would not be acceptable in a vaccine. You might think that most people will eventually be infected by Covid-19 so any vaccine with a risk below that of the virus infection itself would be acceptable. But we don’t really know the risk/death rate for the virus in every population and for all possible treatments. Many people will never be infected. And many infected will have mild symptoms.

We will need to understand all that, plus the risks of any vaccine in various populations, to even begin to do the calculations of risk to benefit.

Yes, that makes sense.

My point has basically been that the risk-benefit analysis here is different from in normal times, but it seems that the granularity in the processes of vaccine development only allows for some minor ways of taking advantage of that. (And thanks to Puppycow for posting some examples). If you try to shift things more than a little, you end up shifting them too much.