AidsVax stats are under pressure within days of VaxGen's results announcement

The failure of AidsVax to prevent infection with HIV - in clinical trial results published in 2003 - triggered an intense debate about the controversial product and its manufacturer, VaxGen Inc of Brisbane, California. Mail to this website, maintained by Brian Deer, shows that existing material on a VaxGen-AidsVax index is read by significant numbers. This page seeks to inform the discussion

VaxGen Results May Be Statistically 'Weaker' Than Originally Reported

kaisernetwork.org:daily HIV/AIDS report#16273#16273 [Feb 27, 2003]

Officials at Brisbane, Calif.-based VaxGen yesterday said that statistical evidence about the effectiveness of its experimental AIDS vaccine AIDSVAX may be "weaker" than initially thought, the Wall Street Journal reports (Wall Street Journal, 2/27). VaxGen on Monday announced that AIDSVAX reduced the rate of new HIV infections by only 3.8% among people who received the vaccine, compared with clinical trial participants who received a placebo injection, but said that it was effective among African Americans, Asians and other non-white, non-Hispanic volunteers. The study consisted of 5,108 gay or bisexual men and 309 women who were HIV-negative when they began the trial, but at high risk for HIV infection because they had sex partners who injected drugs or had sex with men. About twice as many people were randomly assigned to receive injections of the vaccine than were assigned to the placebo group (Kaiser Daily HIV/AIDS Report, 2/24). VaxGen said that in a subgroup of 498 non-white, non-Hispanic volunteers the vaccine "appeared to provide protection in the range of 30% to 84%," the Journal reports. According to the company, the analysis had less than a 1% chance of being random chance, making it statistically significant. But the analysis has been "criticized" by outside scientists because it is based on only 29 HIV infections among vaccinated participants in that subgroup and those who received a placebo, according to the Journal. Although the company originally said that it took "penalties" to reduce the statistical significance of results obtained from parsing out a large set of data, Lance Ignon, vice president for communications at VaxGen, said yesterday that the company did not take such penalties (Wall Street Journal, 2/27). Steven Self, a professor of biostatistics at the University of Washington who VaxGen consulted, said, "It's probably an honest error"; however, he added that the fact that the lower bound of the confidence interval was above zero provided "some marginal statistical evidence that there is some efficacy in that subgroup" (Pollack, New York Times/San Francisco Chronicle, 2/27). VaxGen CEO Lance Gordon yesterday told the CDC Advisory Committee on Immunization Practices in Atlanta that the results are still "preliminary," USA Today reports. Company scientists had just a week to comb through data, and a "full analysis is still in progress," with more information to be released next month, he said (USA Today, 2/27).

AIDS vaccine numbers off, statistician says Effectiveness for minorities may be overstated
Andrew Pollack, New York Times Thursday, February 27, 2003

VaxGen may have overstated the effectiveness of its AIDS vaccine because it did not make the proper statistical adjustments to its data, an expert consulted by the company said Wednesday.
Steven Self, a professor of biostatistics at the University of Washington, said the company should have lowered the level of confidence with which it said the vaccine appeared to protect blacks, Asians and other non-Hispanic minorities from infection by HIV.
Dr. Donald Francis, the president of VaxGen, said there was some debate among statisticians about the proper adjustments but called it "a tangential issue." Even with the adjustments, he said, the results showing efficacy in minorities would still be statistically significant.
VaxGen reported Monday that its vaccine was ineffective overall in a trial of 5,400 participants. But it said in a subset of 500 non-Hispanic minorities, the vaccine reduced infection by 66.8 percent, a statistically significant result.
The company has since been criticized by outside scientists and AIDS activists for stressing conclusions based on a small number of patients. VaxGen reported that its finding for the minority subgroup had a confidence interval of 30.2 percent to 84.2 percent. That meant there was a 95 percent probability that the actual efficacy of the vaccine in the subgroup was between those points.
But Self said that by his calculation, the low end of the confidence interval should be just above zero. That is because the company should have lowered its confidence to account for the fact that it analyzed multiple subgroups of patients. If the data are cut enough ways, some effectiveness can almost always be found, he said, so such "penalties" to the interval are taken to guard against false positive conclusions. Self said VaxGen asked him for advice after other statisticians began criticizing it for failing to make such adjustments.
"It's probably an honest error," he said. But he said the fact that the lower bound was still above zero still provided "some marginal statistical evidence that there is some efficacy in that subgroup."
VaxGen's Francis said the important thing now is to determine whether there is some biological explanation for why the vaccine appeared to work in some groups. .