How does an observed versus expected analysis work?

The method used by the Paul-Ehrlich-Institut was published years ago in a peer review journal (by Kries R et al. 2005, https://pubmed.ncbi.nlm.nih.gov/15602672/).

First, the individuals conducting the analysis determine the number of cases of a specific adverse event reported within a plausible time interval after vaccination with a specific vaccine product (observed number: number of suspected cases reported to the Paul-Ehrlich-Institut in which the adverse event was reported as occurring within x days after vaccination with a certain vaccine product). The adverse event analysed could be a medically diagnosed myocarditis, for example.

The number of events based on the background incidence for the adverse event to be analysed (number of cases per 100,000 people per year) within the same time interval of x days in the population immunised with the respective vaccine would then be calculated independently of the given vaccination (expected number). This is the number of vaccination-independent events based on the background incidence of the event, regardless of whether the individuals received the vaccination in question in a comparable population group. The background incidence of an event can be found in publicly valid statistics or peer-reviewed scientific publications.
The observed number is then divided by the expected number (observed versus expected, OE). A 95% confidence interval (Poisson) is calculated for the value thus obtained (point estimator).

A standardized morbidity ratio (SMR) with a lower 95% confidence interval ≥ 1 indicates a safety signal, but this must be further analysed by additional investigations, since the comparison of spontaneous reports with the known incidences from other studies is exploratory due to various methodological limitations.

The OE calculation includes all suspected case reports for a certain event up to and including the day of the evaluation within the selected, plausible time interval between the respective vaccination and the occurrence of the first symptoms of the event (time to onset, TTO).

In addition to the OE analysis, the Paul-Ehrlich-Institut assesses the individual suspected reports according to the World Health Organisation (WHO) recommended evaluation algorithm for causality between vaccination and adverse event (https://www.who.int/publications/i/item/9789241516990). The OE analysis does not take these individual case assessments into account and includes all suspected case reports.