What this entity is
Canonical name: GroupKFold
Type: method
Known aliases: grouped cross-validation by study; study-held-out cross-validation. 1
Stable external identifier: None reported in the source. 1
GroupKFold is a cross-validation method in which complete studies are held out as groups, preventing runs from the same study from appearing in both training and validation partitions. 1
Use in the source projects
The project used GroupKFold by study to test whether environmental metadata predicted the GOTTCHA2-derived eukaryotic fraction beyond study or batch effects in 2,759 NMDC ReadbasedAnalysis runs from 9 studies. 1
A gradient-boosted model using environment achieved R²=0.35 under random cross-validation but R²=−0.30 under GroupKFold out-of-study validation. 1 Adding sequencing metadata produced R²=−0.39 under GroupKFold, indicating no improvement in out-of-study prediction. 1 The corresponding out-of-study detection AUC was 0.56, approximately chance. 1
These results show that GroupKFold exposed strong cross-study environment associations as largely study- or batch-confounded rather than reliably generalizable across held-out studies. 1
Within the dominant NEON soil study, a model using local environment and geography was evaluated with five-fold cross-validation and achieved R²=+0.17 ± 0.06, contrasting with the cross-study GroupKFold result of R²=−0.30. 1 The within-study analysis controlled approximately for batch but remained potentially affected by sub-batch structure such as sampling campaigns. 1
The soil-frontier analysis refines this interpretation: negative out-of-sample R² under GroupKFold can reflect spatial autocorrelation within folds, train/test distribution shift, or high-leverage outliers, so it does not alone establish genuine global unpredictability. 2 It recommends spatial blocking and explicit decomposition of these effects before assigning a strong biological meaning to a negative R². 2
Interpretation and limitations
GroupKFold is especially relevant to environment-embedding-geography because it distinguishes environmental signal that transfers across studies from signal that may encode study-specific sampling or processing. 1 The method also supports the report’s data-bridging workflow in cross-tenant-data-bridging, where workflow runs were linked to biosamples and studies without treating pooled biosamples as independent observations. 1
The cross-study generalization test was under-powered because it included only approximately 9 studies, and one soil study accounted for approximately 43% of runs. 1 The within-study result was demonstrated for one soil study only and may not extend to aquatic or host-associated collections. 1
The soil-frontier result therefore supports using held-out groups to expose transfer failures but contradicts treating every negative GroupKFold R² as evidence of biological nullity; spatially blocked validation, leverage diagnostics, and distribution-shift checks are needed to distinguish modelling failure from true unpredictability. 2
Related pages
- Metadata Correlates of Eukaryotic Contamination in NMDC Prokaryote-Targeted Metagenomes — source-project summary. 1
- Soil Microbial Dark Matter and the Clay Shield Null Result — soil-scale validation of GroupKFold interpretation. 2
- environment-embedding-geography — environmental and geographic prediction under batch control. 1
- cross-tenant-data-bridging — linking workflow runs with biosamples and studies. 1
- gottcha2 — classifier used to estimate relative eukaryotic abundance. 1