Serial intervals – the time between symptom onset in infector and infectee – are a fundamental quantity in infectious disease control. However, their estimation requires knowledge of individuals’ exposures, typically obtained through resource-intensive contact tracing efforts. We introduce an alternate framework using virus sequences to inform who infected whom and thereby estimate serial intervals. We apply our technique to SARS-CoV-2 sequences from case clusters in the first two COVID-19 waves in Victoria, Australia. We find that our approach offers high resolution, cluster-specific serial interval estimates that are comparable with those obtained from contact data, despite requiring no knowledge of who infected whom and relying on incompletely-sampled data. Compared to a published serial interval, cluster-specific serial intervals can vary estimates of the effective reproduction number by a factor of 2–3. We find that serial interval estimates in settings such as schools and meat processing/packing plants are shorter than those in healthcare facilities.