10d ago
Scaling recipes for single-cell RNA sequencing foundation models: when do scaling laws hold?
Borra · F. · Ciro' · G. · Castellini · A. · Gatti · G. · +4 more
Abstract
Deep learning models exhibit empirical scaling laws whereby performance changes predictably with model size, dataset size, and training compute. Although these relationships are well established in domains such as language and image modelling, their applicability to biological data remains unclear. Here, we investigate scaling behaviour in foundation models trained on large collec tions of single-cell transcriptomes. We show that pre-training loss decreases systematically with model capacity and
Discussion · 0
Sign in to join the discussion.