--- title: 'A Report Is Not a Rate: The Strengths and Blind Spots of Vaccine Injury Surveillance' track: Vaccine Science and Critical Appraisal source_books: - Vaccines - The Terrain Medicine Handbook references: - https://doi.org/10.1001/jama.2021.24110 - https://doi.org/10.1038/s41591-021-01630-0 --- ## Abstract A patient report of harm after vaccination deserves attention. But the event's timing alone cannot establish the vaccine as its cause, and the absence of a report cannot establish that the event never occurred. This essay separates four tasks frequently confused in vaccine controversies: receiving safety reports, detecting statistical signals, estimating event incidence, and making causal judgments. It argues that public trust requires honest acknowledgment of surveillance limitations without treating those limitations as evidence for every proposed injury. ## The value of an open signal system Passive reporting systems allow clinicians and the public to submit suspected adverse events. Their strength is breadth: unusual patterns may appear before anyone knows to look for them. Their weakness is that the reporting process does not draw an unbiased sample of all vaccinated people. Reports can be stimulated by publicity, differ by severity or clinical attention, include duplicate information or capture background events that would have occurred without vaccination. A raw count therefore does not provide the probability of injury. To estimate incidence, one needs a defensible denominator and reasonably complete case ascertainment. To infer causation, additional comparisons are needed against unexposed periods or groups, while accounting for age, sex, infection patterns, calendar time and baseline medical risk. ## Myocarditis illustrates both sides Postauthorization surveillance detected a myocarditis signal after certain mRNA COVID-19 vaccines, with especially elevated reported rates in adolescent and young adult males after particular doses. The JAMA investigation by Oster and colleagues described clinically reviewed cases and showed how a signal can emerge from a system with known weaknesses. Subsequent studies used other designs to estimate risks in specific populations and compare them with outcomes following SARS-CoV-2 infection. The existence of a genuine vaccine-associated adverse event does not validate every claimed adverse event, but it does disprove the notion that critical surveillance is inherently anti-scientific. Risk is conditional: age, sex, product, dose number, interval and infection history matter. Conversely, the denominator and case definition determine what a study's rate means. A signal reported at one point in a pandemic cannot simply be copied into all subsequent periods without checking changes in products, susceptibility and circulating virus. ## The blind spots that remain Passive systems may miss events that clinicians do not recognize as potentially related to vaccination, especially if onset is delayed or the phenotype is poorly defined. Active surveillance can address some underascertainment through linked health records or targeted follow-up, but those systems bring their own problems: coding inaccuracies, confounding, missing data and decisions about risk windows. A robust monitoring ecology needs multiple methods because no single method dominates in every situation. There is also a symmetrical danger. A surprising rise in diagnoses after a vaccination campaign may reflect changes in testing, healthcare access, seasonality or background incidence. Rejecting such alternatives because they complicate a preferred story turns safety analysis into confirmation bias. The correct question is what pattern would remain if those factors were measured and accounted for. ## The principle of open attribution Suspected harms should be described precisely: onset date, symptoms, objective findings, alternative causes, diagnostic certainty and clinically meaningful outcome. Events with strong temporal and mechanistic support warrant different language from self-reported, nonspecific symptoms without verification. Patients deserve clinical care whether or not causation is established; a causality judgment should not be a prerequisite to investigating a serious presentation. The public objective is not to maximize confidence in vaccines or to maximize suspicion. It is to maximize the probability of recognizing true product-specific harm while avoiding false attribution. That is what good surveillance is for. ## A spontaneous report is a starting point, not a denominator Passive surveillance systems allow clinicians, patients and families to report an event occurring after vaccination. This openness can detect patterns that were too rare to be visible in preauthorization trials, but reports may be incomplete, duplicated, selectively submitted or missing diagnostic confirmation. Events may also coincide with vaccination because both occur frequently within a population. Raw counts cannot establish incidence without knowing how many people were vaccinated, how much time elapsed and how many similar events would have occurred without the exposure. An event reported after immunization is not automatically caused by it; dismissing it for that reason would be equally mistaken. The system's purpose is signal generation followed by more rigorous evaluation. ## How a signal becomes a causal investigation A credible investigation begins by defining a specific outcome and biologically plausible risk interval. Clinicians then confirm cases using consistent diagnostic criteria. Active surveillance or linked administrative data can compare rates across exposure windows and relevant controls, sometimes with a self-controlled case-series approach that reduces stable individual-level confounding. Investigators still must address time-varying infection, healthcare-seeking, calendar season and product differences. For example, a myocarditis signal warrants distinctions among chest pain alone, elevated cardiac biomarkers, imaging-supported diagnoses and severe clinical outcomes. The more precisely an outcome is defined, the less easily coincidence or reporting attention can distort the conclusion. ## Benefit–risk assessment requires subgroup specificity Risks may differ by age, sex, product, dose number, prior infection and prevailing disease incidence. Therefore an average risk ratio is not sufficient to guide every individual discussion. Absolute events per population over a defined period provide a clearer basis for comparison than dramatic ratios without denominators. The benefits of vaccination also change with disease circulation and population immunity, and these should be estimated rather than assumed constant. A rigorous communication framework can acknowledge a genuine but rare adverse event while explaining which groups and circumstances are most affected. Transparency about uncertainty helps the public far more than declaring that every temporal association is causation or that all temporal associations are irrelevant. ## Institutional accountability beyond the signal itself When a safety signal appears, the response should include timely case-definition publication, transparent access to deidentified aggregate findings, independent replication and ongoing reassessment of risk estimates. Researchers should distinguish a confirmed effect from a preliminary statistical alert and update conclusions as new data arrive. Patients with plausible adverse events deserve appropriate clinical assessment irrespective of whether a population-level causal estimate has been finalized. The Institute's proposal is to treat the surveillance pipeline as part of the intervention's total evidence record, not as a public-relations obstacle. The success criterion is faster, more accurate classification of real harms alongside protection against false attribution. ## Reporting intensity changes with attention Publicity can increase the probability that an event is reported, while unfamiliar syndromes may be underreported before their clinical pattern is recognized. Thus, apparent changes in report volume across time may reflect changing awareness as well as changing incidence. Statistical signal detection can help prioritize review but requires confirmatory designs. Useful public dashboards should clearly separate reports received, reports meeting a case definition, estimated background cases and cases for which a causal association is supported. The denominator should be described, particularly when vaccination schedules change. Reports should not be removed from view because causality is uncertain, but neither should uncertainty be hidden behind unqualified totals. The aim of transparency is interpretability. An honest safety system can recognize adverse reactions earlier precisely because it does not demand that every initial report be either accepted as proof or dismissed as noise. ## Evidence boundary Safety reports are signals, not confirmed causal injuries or population incidence rates. This paper endorses thorough investigation and transparent communication, not indiscriminate attribution.