Brighter days ahead

As the dark and frustrating 2020 is winding down, I feel incredibly optimistic about 2021 and beyond. Part of it is my entrepreneurial nature that requires it and part of it is a number of recent developments that bring me hope and I love drinking hope for breakfast.

A number of recent readouts from SARS-CoV-2 trials, particularly those from Pfizer and Moderna, which use a novel mRNA platform, look efficacious and safe in the short-term. (Long-term safety can not be evaluated in rapid clinical trials but the FDA guidelines provide for long-term, post-licensure, safety monitoring.) As soon as these vaccines are made available to the general public and assuming no major safety issues are surfaced, I am getting vaccinated and resuming my pre-pandemic travel schedule.

In case you missed it, on May 30th we witnessed the first American manned space flight since 2011. I watched it in real-time that Saturday and then many more times with my son Andrei who just loves watching rockets being launched into space. Since then, watching launches on Saturdays became a Novik family tradition. Seeing Andrei’s eyes light up every time we do it, brings me an unreasonable amount of joy.

Two weeks ago I ordered my first Virtual Reality headset — Oculus Quest 2. I don’t love the Facebook login requirement but as a newcomer to the world of VR, I am completely blown away by how easily my senses are fooled and by the near-perfect rendering of 3D worlds. It is clear to me that VR is a major technological trend with ramifications far beyond gaming. I can’t wait to virtually sit in front of my family and friends all over the world and interact with them as if we are in the same room. The technology is not quite there to make the experience realistic but I have little doubt that it’s coming. (As a side note, the game Room in VR is nothing short of amazing.)

During the summer, we made a lot of progress at Generable with fitting large meta-analytic models for oncology drugs, and our abstract was accepted at SITC 2020, an immuno-oncology conference. This is a big deal for us as it represents the first publication outside of statistics journals. Most of the work was done by Jacqueline Buros and Krzysztof Sakrejda and is a culmination of our year-long research collaboration with AstraZeneca.

And if this is not enough, 2021 promises to be a much more sane US Federal Government with adults finally taking over and mitigating the Fifth Risk. No doubt countless problems remain (I don’t want to list them) but I am feeling lucky and optimistic.

Communists and Dictators, Liberals and Conservatives, Left and Right, Right and Wrong

I grew up in the Soviet Union, more precisely in Soviet Latvia. Thanks in part to my dad, I was able to come to the United States in 1989, just before the collapse of the USSR. My dad immigrated to the US in 1978 during the so-called first wave of Jewish immigration.

I attended a relatively typical Soviet high school in Riga; it was creatively called School #87, and it specialized in Physics and Chemistry. Not all schools had a specialization, but this one did. I remember attending organic Chemistry lectures at the local university. I hated organic chemistry. High school was a 2-year affair in the old Soviet Union, so grades 9 and 10. The unwritten rule of the Soviet school system was that when you entered first grade, you became an “October Kid”, referring to the October Bolshevik revolution, and received a little red star to be worn on the lapel of your navy blue uniform jacket. The star had Lenin’s portrait engraved in the middle.

In middle school, you would be promoted to a “Pioneer” which was considered the second stage of your communist journey. You were not asked to join – you simply aged into it. You were given a red, two-pronged tie which looked quite festive atop of a white button-down shirt. I wore mine just like everyone else.

Then came high school. In high school, you were supposed to join the “Komsomol” — a communist youth organization, the last step before the actual party membership. Unlike the Pioneers, Komsomol wanted the appearance of a volunteer organization. You were supposed to ask to join and to my knowledge everyone did. I did not. Now, before you think that I was some kind of Solzhenitsyn or Sakharov, let me assure you that I was not. In the late 80s, Gorbachev’s perestroika, which means rebuilding, was in full force. The iron grip was starting to loosen and people started feeling that they can express themselves freely without facing the wrath of the state. Had these been the 70s, 60s, or before, this act of noncompliance would have gotten me and my family into serious trouble, and I doubt that I would have been so brave.

Even so, I hated the communist party and all it represented. Being raised in an educated household, I was aware of some of the atrocities committed by Soviet dictators and their henchmen against their own citizens, and their hypocritical rhetoric of equality and brotherhood was not lost on me. It was also obvious that a centrally planned economy was a colossal failure. And so when it was time to join the Komsomol, our “class leader”, Polina Kononovna, an ardent communist, insisted that I start getting ready for membership. “No, thank you,” I said. “I don’t feel like am ready.” Eventually, she left me alone. It was a small but important win for me.

When I landed in JFK in September 1989, I was 17 and I met my father essentially for the first time — my parents divorced when I was 3 years old and even though my mother tells me that I saw him a few times before he left for the US, I had very little recollection of our interactions. In 1989, my dad was already a successful physician in New Jersey. Like most immigrants from communist countries like Cuba and the Soviet Union, he is a staunch conservative and a proud Republican. After experiencing the evils of communism first hand, people like my dad recoil at anything that appears to have socialist overtones. Their analysis is quite shallow, to be sure, but who has the time or desire to do a deep-dive on the fundamental differences between American liberals and Russian or Cuban communists? I certainly did not, and so I blindly inherited my dad’s conservative principles without being too political. I did not vote for a long time and when I finally did, it was for George W Bush, a decision that I later regretted.

Over the years I made it a point to learn more about American history and the American political system. The more I learned, the more conflicted I felt. On one hand, unlike the Soviet Union, the American experiment was an incredibly successful enterprise that managed to establish an independent judiciary, a free press and freedom of speech, and at least on paper, a guarantee of equal protection to all its citizens. The level of average economic prosperity was also unprecedented. On the other hand, it does not take too much research to learn about the less flattering history: atrocious treatment of Native Americans by the colonists (e.g. purposely infecting them with smallpox, an act of biological warfare), the history of slavery and racism so potent that it lasts to this day, the civil war, women’s suffrage movement (women were not allowed to vote until 1920; see Elizabeth Cady Stanton and Susan B. Anthony), the internment of Japanese Americans during the second world war (deemed constitutional by the US Supreme Court! see Korematsu v. United States), Vietnam, and Chicago 7, just to name a few.

It turns out that no country is perfect and neither is my beloved US. While living in Russia I learned, then later forgot, and then again remembered that one needs to question what the government, any government is doing to and for its citizens. It now seems obvious to me what Adam Gopnik so eloquently described in his book “A Thousand Small Sanities”: liberal, progressive movement must be diametrically opposed to both the far-right (e.g. Fascism) and the far-left (e.g. Communism). These two movements devolve into totalitarian, oppressive states, and have more in common with each other than with a liberal, progressive agenda, with its respect for humanity, human rights, and human dignity above all else.

During the cold war, the world was under a Communist threat. I think in general, the US overestimated just how far USSR was willing to go — Khrushchev was fully aware and clearly unwilling to participate in the mutually assured destruction during the Cuban Missile Crisis although Castro seemed to be ready to annihilate the world. Today, by any reasonable account, the world is in danger of succumbing to right-wing nationalism, the likes of which we have not seen since the second world war. Unfortunately, American conservative politicians don’t realize how far to the right they are moving and under the direction of Donald Trump keep falling further into the abyss. As I sit here and write these words, Trump is refusing to recognize the legitimacy of the election results, an election that he so clearly lost. I never thought I would witness a coup attempt accompanied by this kind of authoritarian rhetoric from an American president. To see Trump’s playbook, if you can call it that, compare his speeches to other autocrats throughout the world: Erdoğan in Turkey, Orbán in Hungary, Bolsonaro in Brazil, and the list goes on. Their dangerous message is the same: restore the country’s greatness while dispensing with basic democratic institutions, like free elections, for example.

When 2016 arrived and with the benefit of a more rounded education I promised myself that I will participate in the political process and vote in every election. Hilary Clinton was not my favorite candidate but against Trump, she seemed like a no-brainer. When Trump won, I was concerned, but I fully accepted the will of the people (even though he lost the popular vote), and I wished him well hoping that the weight of the office would induce a sense of responsibility in a seemingly irresponsible showman. Alas, this did not happen. In his book “The Fifth Risk”, Michel Lewis described just how badly he mismanaged what was a fairly well-functioning Federal Government, starting with the transition in which he refused to participate. (By Federal Government I mean federal agencies, not the legislative branch which has been in stalemate for many years.) During the 2020 election, my favorite candidate was Andrew Yang, who had some fresh ideas like UBI and did not rely on the old-school Democrat talking points like tax-the-rich. (I mean, sure, tax them, but that’s not going to solve our fundamental problems, not to mention that the rich have a lot of options when it comes to avoiding paying taxes.)

Now, in case you are trying to classify me, I am not a Democrat, I do not belong to any political party, but today I think it is important for me to go on record. When in 10 or 20 years from now and with the benefit of perspective, this period in American history is rightly recognized as one the darkest, most fragile moments since the beginning of the republic, I want my kids and my grandkids to know where I stood. I unequivocally reject Trump, Trumpism, and all that it represents. Today’s Republican representatives in Congress (but not the people who voted for them) that stand behind Trump to protect their own skin, afraid of the Twitter mobs that he may unleash on them — I do not wish you any harm but there will be a day of atonement, a judgment day when you will be unable to look your kids and your countrymen in the eye. Your names will forever be marred by your cowardice, your inhumanity, your demagoguery, and your lies, willingly told and repeated. While supporting this regime to the very end, you have sold your country and your democracy short. Shame on you! The kind of nonsense propaganda I hear from you today is comparable to what I heard from Soviet apparatchiks in the 80s and 90s.

To the 70 million Americans that voted for Trump (and to the 5 or 6 of you who will read this blog post), I want to be clear — you are not my enemy and I hope I am not yours. As a rule, I usually refrain from voicing my political opinions on social media, nothing good seems to come of it, but I did have the following Twitter conversation the other day:

I can understand my correspondent’s position but to implicate 70 million people in antisemitism because the guy they voted for is a racist is a bit much. For me personally, it is also a little too close to home. You see, my own Jewish father voted for Trump, twice, I think. What am I supposed to do with that now? He is the same father who helped me immigrate from the Soviet Union, the same father who opened his house to me and supported me, the same father who stayed with me in the hospital when I had my shoulder operation. Am I supposed to reject him just because in a momentary lapse of reason that unfortunately lasted for 4 years, he voted for Trump? Absolutely not! I do not reject him or the 70 million citizens whose concerns were ignored for way too long, legitimate concerns, who, I hope, out of desperation voted for Trump. I do not reject you. I believe you will come around, eventually, to empower representatives that put our country’s interests ahead of their own. 

Thanks to Jacki Novik for proofreading and providing helpful comments.

Estimating uncertainty in the Pfizer vaccine effectiveness

On Nov 20, 2020, the New York Times published an article titled: “New Pfizer Results: Coronavirus Vaccine Is Safe and 95% Effective.” Unfortunately, the article does not report the uncertainty in this probability and so we will try to estimate it from data.
 

Assumptions

n <- 4.4e4  # number of volunteers
r_c <- 162  # number of events in control
r_t <- 8    # number of events in vaccine group

NYT reports a 44 thousand person trial with half of the people going to treatment and half to control. They further report that 162 people developed COVID in the control group and 8 were in the vaccine group.

The Pfizer protocol defines vaccine effectiveness as follows:

\[
\text{VE} = 1 – \frac{p_{t}}{p_{c}}
\]
Here \(p_{t}\) is infection rate in vaccinated group and \(p_{c}\) is the rate in the control group.

Model

Also, let’s assume that we have no prior beliefs about the effectiveness rate and so our model is as follows: \[
\begin{align}
p_{c} \sim \textsf{beta}(1, 1) \\
p_{t} \sim \textsf{beta}(1, 1) \\
y_{c} \sim \textsf{binomial}(n_{c},p_{c}) \\
y_{t} \sim \textsf{binomial}(n_{t},p_{t}) \\
\end{align}
\]
The treatment effect and \(VE\) can be computed directly from this model.

\[
\begin{align}
\text{effect} = p_{t} – p_{c} \\
\text{VE} = 1 – \frac{p_{t}}{p_{c}}
\end{align}
\]

The effect will have a distribution and to get the probability of the effect (this is different from VE), we sum up the negative area under the effect distribution. For this problem, we do not need Stan, but I am including it here to show how easy it is to specify this model, once we write down the math above.

data {
  int<lower=1> r_c; // num events, control
  int<lower=1> r_t; // num events, treatment
  int<lower=1> n_c; // num cases, control
  int<lower=1> n_t; // num cases, treatment
  int<lower=1> a;   // prior a for beta(a, b)
  int<lower=1> b;   // prior b for beta(a, b)
}
parameters {
  real<lower=0, upper=1> p_c; // binomial p for control
  real<lower=0, upper=1> p_t; // binomial p for treatment 
}
model {
  p_c ~ beta(a, b); // prior for control
  p_t ~ beta(a, b); // prior for treatment
  r_c ~ binomial(n_c, p_c); // likelihood for control
  r_t ~ binomial(n_t, p_t); // likelihood for treatment
}
generated quantities {
  real effect   = p_t - p_c;      // treatment effect
  real VE       = 1 - p_t / p_c;  // vaccine effectiveness
  real log_odds = log(p_t / (1 - p_t)) -
                  log(p_c / (1 - p_c));
}

Running the model and plotting the results

Let’s run this model from R and make a few plots.

library(cmdstanr)
library(posterior)
library(ggplot2)

# first we get the data ready for Stan
d <- list(r_c = r_c, r_t = r_t, n_c = n/2, 
          n_t = n/2, a = 1, b = 1) # beta(1,1) -> uniform prior

# compile the model
mod <- cmdstan_model("vaccine.stan")
# fit the model with MCMC
fit <- mod$sample(
  data = d,
  seed = 123,
  chains = 4,
  parallel_chains = 4,
  refresh = 0
)
## Running MCMC with 4 parallel chains...
## 
## Chain 1 finished in 0.2 seconds.
## Chain 2 finished in 0.2 seconds.
## Chain 3 finished in 0.2 seconds.
## Chain 4 finished in 0.2 seconds.
## 
## All 4 chains finished successfully.
## Mean chain execution time: 0.2 seconds.
## Total execution time: 0.3 seconds.
# extract the draws
draws <- fit$draws()

# Convert to data frame
draws <- posterior::as_draws_df(draws)
head(draws)
## # A draws_df: 6 iterations, 1 chains, and 6 variables
##    lp__    p_c     p_t  effect   VE log_odds
## 1 -1041 0.0072 0.00045 -0.0068 0.94     -2.8
## 2 -1041 0.0075 0.00034 -0.0071 0.95     -3.1
## 3 -1044 0.0084 0.00072 -0.0077 0.91     -2.5
## 4 -1043 0.0065 0.00027 -0.0063 0.96     -3.2
## 5 -1043 0.0068 0.00026 -0.0065 0.96     -3.2
## 6 -1042 0.0078 0.00029 -0.0075 0.96     -3.3
## # ... hidden meta-columns {'.chain', '.iteration', '.draw'}
# look at the distribution of the effect
ggplot(draws, aes(x = effect*1e4)) +
  geom_density(fill = "blue", alpha = .2) +
  expand_limits(y = 0) + theme_minimal() +
  xlab("Size of the effect") +
    ggtitle("Reduciton in infections on treatment per 10,000
people")

# Probability that there is an effect is the negative mass 
# of the effect distribution; more negative favors treatment
# -- there is no question that there is an effect
round(mean(draws$effect < 0), 2)
## [1] 1
ggplot(draws, aes(x = log_odds)) +
  geom_density(fill = "blue", alpha = .2) +
  expand_limits(y = 0) + theme_minimal() +
  xlab("Log odds") + 
  ggtitle("Log odds of the treatment effect.
More negative, less likely to get infected on treatment")

label_txt <- paste("median =", round(median(draws$VE), 2))
ggplot(draws, aes(x = VE)) +
  geom_density(fill = "blue", alpha = .2) +
  expand_limits(y = 0) + theme_minimal() +
  geom_vline(xintercept = median(draws$VE), size = 0.2) +
  annotate("text", x = 0.958, y = 10, label = label_txt, size = 3) +
  xlab("Vaccine effectiveness") +
  ggtitle("Pfizer study protocol defines VE = 1 - Pt/Pc")

quant <- round(quantile(draws$VE, probs = c(0.05, 0.5, 0.95)), 2)

So if we observe 162 infections in the placebo group and 8 in the vaccinated group, the vaccine could be considered to be about 0.95 effective with the likely effectiveness anywhere between 0.91 and 0.97 which represents the median and the 90% quantile interval. We should insist that reporters produce uncertainty estimates and the reporters, in turn, should insist that companies provide them.