Confirmatory Nulls and Exploratory Signals in Ecological Association Testing

This concept describes how predeclared ecological association tests can remain null while exploratory, coverage-adjusted analyses produce apparently positive signals that are sensitive to mapping, covariates, taxonomic resolution, multiple-testing correction, environmental classification, and pangenome summary metrics. _{1 2 3}

The evidence includes Contamination Gradient vs Functional Potential in ENIGMA Communities, which tested whether contamination in enigma-coral communities was associated with pangenome- and eggnog-derived functional proxies; Ecotype Reanalysis — Environmental vs Human-Associated Species, which reanalyzed environment–gene-content associations across bacterial species; Ecotype Correlation Analysis, which performed a separate phylogeny-versus-environment comparison across 172 species; Lab Fitness Predicts Field Ecology at Oak Ridge, which compared laboratory metal-tolerance measurements with field abundance across 108 Oak Ridge groundwater sites; and Pangenome Openness Analysis, which tested whether species-level pangenome openness predicts the relative influence of environment or phylogeny on gene-content variation. _{1 2 4 5 3}

Evidence Pattern

The ENIGMA analysis used 108 samples with geochemistry and community composition, a geochemistry matrix of shape (108, 49), 41,711 community taxon rows, 212 distinct communities, and 1,392 distinct genera. ₁

Its contamination index combined arsenic, cadmium, chromium, copper, lead, nickel, uranium, and zinc by per-metal log1p z-scoring followed by a row-wise mean; across 108 samples, it ranged from -0.448 to 3.836, with median -0.271 and IQR [-0.363, 0.053]. ₁

The workflow generated 324 site functional-score rows across three mapping modes and 12 model-result rows covering four outcomes across those modes. ₁

The ecotype reanalysis provides a related control for sampling composition. Among 224 species selected with at least 20 genomes containing AlphaEarth embeddings and at least 30% coverage, 106 species (47%) were majority human-associated, 47 (21%) were majority environmental, and 71 (32%) were Mixed/Other. ₂

The new ecotype analysis refines this evidence by analyzing 13,381 genomes across 224 species and obtaining correlation results for 172 species. It used AlphaEarth environmental embeddings, genome metadata and taxonomy, NCBI environmental and isolation-source metadata, pangenome composition, and gene-cluster presence/absence profiles from the kbase_ke_pangenome database. ₄

The pangenome-openness analysis refines the ecotype evidence by testing the pangenome metric directly: among species with both pangenome statistics and ecotype-analysis results, openness had no detectable relationship with either environment or phylogeny effects. The reported Spearman correlations were openness versus environment rho = -0.05, p = 0.54, and openness versus phylogeny rho = 0.03, p = 0.73. These null correlations do not show that pangenome structure is biologically independent of eco-phylogenetic dynamics; they show that this single openness summary did not predict the measured effect sizes in the available overlap. ₃

The Oak Ridge comparison supports the same confirmatory-versus-exploratory distinction in a field-fitness setting: the aggregate laboratory metal-tolerance score was positively but non-significantly associated with the high-uranium/low-uranium abundance ratio (Spearman rho=0.503, p=0.095, n=12 genera), while five of 11 tested genera had FDR-significant bidirectional uranium associations. ₅

Confirmatory Nulls

Predeclared Spearman tests of site_defense_score against contamination were non-significant in both genus-level mapping modes. ₁

In relaxed_all_clades, rho = 0.0587, the 95% bootstrap CI was [-0.128, 0.250], Spearman p = 0.546, and false-discovery-rate (FDR), a multiple-testing correction, q = 0.862. ₁

In strict_single_clade, rho = 0.0682, the 95% bootstrap CI was [-0.111, 0.253], Spearman p = 0.483, and FDR q = 0.849. ₁

The null result was robust to four contamination-index definitions: the composite all-metals index, uranium-only, the top-3-variance-metals index, and the first principal component of metal z-scores. ₁

All eight confirmatory variant tests remained non-significant after FDR, with q = 0.546 across the tests, including uranium-only. ₁

These results support treating the broad genus-level contamination–defense association as unconfirmed rather than as evidence of a monotonic community-level relationship. ₁

The ecotype reanalysis supports this confirmatory-null interpretation at a different ecological scale: environmental species (n=37) had median partial correlation 0.051, compared with 0.084 for human-associated species (n=93) and 0.109 for Mixed/Other species; the one-sided Environmental > Human-associated Mann–Whitney U test gave U=1536 and p=0.83, and the observed difference ran in the opposite direction from the hypothesis. ₂

The continuous analysis likewise found no relationship between the fraction of environmental genomes per species and partial-correlation strength (Spearman rho=-0.085 and p=0.25), or between the fraction of human-associated genomes and correlation strength (rho=0.030 and p=0.69). ₂

The new analysis supports the broad null interpretation while distinguishing phylogenetic structure from ecological association: the median partial correlation was 0.0025 for environment versus 0.0143 for phylogeny; phylogeny dominated the gene-content signal in 60.5% of species, environment in 39.5%, and environmental effects were significant in only 12 species (7.0%) positively and 4 species (2.3%) negatively, with no significant effect in 156 species (90.7%). ₄

The pangenome-openness result supports caution against treating openness as a confirmatory predictor of those effect sizes: openness versus environment had Spearman rho = -0.05 and p-value = 0.54, while openness versus phylogeny had rho = 0.03 and p-value = 0.73. Because the sample was limited to species with both pangenome statistics and ecotype results, and because openness is a single summary metric, this is a null prediction test rather than evidence that open/closed pangenome status has no ecological relevance. ₃

The Oak Ridge result refines this pattern rather than overturning it: the laboratory-to-field tolerance test was classified as not supported but suggestive, despite the positive direction, because only 12 genera were available for the aggregate correlation. ₅

Exploratory Signals

Exploratory coverage-adjusted ordinary least squares (OLS) models included contamination, depth, latitude, longitude, and mapped abundance fraction, and both gave positive defense-score coefficients. In relaxed_all_clades, beta was 0.000751, with 95% bootstrap CI [0.000224, 0.001779], p = 0.000398, and FDR q = 0.0462. The corresponding strict_single_clade estimate was beta = 0.000640, with 95% bootstrap CI [0.000169, 0.001538], p = 0.00354, and FDR q = 0.130. ₁

These results support a coverage-sensitive defense association in the relaxed mode but do not establish a robust association across both mapping modes after global FDR. ₁

Exploratory fraction-aware adjusted models using contamination, mapped abundance fraction, and C(community_fraction_type) also gave positive defense coefficients, but neither was significant after global FDR. For relaxed_all_clades, beta = 0.000607, with 95% bootstrap CI [0.000213, 0.001221], p = 0.00144, and FDR q = 0.0838. The corresponding strict_single_clade estimate was beta = 0.000566, with 95% bootstrap CI [0.000214, 0.001113], p = 0.00548, and FDR q = 0.130. ₁

In the high-coverage subset defined by mapped_abundance_fraction >= 0.25, defense-score Spearman p-values were 0.0207 for the relaxed mode and 0.00980 for the strict mode, while global FDR q-values were 0.301 and 0.189, respectively. ₁

Most non-defense outcomes remained non-significant, although site_stress_score in the strict high-coverage subset had rho = 0.2489 and p = 0.0407. ₁

The ecotype reanalysis refines this pattern by showing that a genuine clinical sampling imbalance need not explain a weak ecological association. Environmental species were not more strongly associated with environment-derived embeddings than human-associated species, despite the AlphaEarth subset’s strong clinical sampling bias. ₂

The pangenome-openness null supports treating such exploratory environmental and phylogenetic effects as not predictable from openness alone: neither openness–environment nor openness–phylogeny was significant in the overlapping species set. The report instead proposes testing auxiliary fraction, Heap’s law alpha, pangenome fluidity, and function-specific categories, indicating that the null may reflect metric resolution rather than absence of adaptation. These are proposed hypotheses, not findings established by the correlation analysis. ₃

The Oak Ridge genus-level results support treating exploratory signals as heterogeneous rather than as a single monotonic effect: five genera passed FDR correction, with Herbaspirillum and Bacteroides increasing and Caulobacter, Sphingomonas, and Pedobacter decreasing with uranium. ₅

Robustness and Sources of Discordance

Within-fraction defense Spearman tests were non-significant for both mapping modes and both fractions: relaxed 0.2_micron_filter p = 0.767, relaxed 10_micron_filter p = 0.898, strict 0.2_micron_filter p = 0.780, and strict 10_micron_filter p = 0.793. ₁

This contradicts interpreting the pooled exploratory defense signal as a robust monotonic relationship that reproduces within individual fraction strata. ₁

The species-proxy mode retained 150 unique-clade genera compared with 530 mapped genera overall, while 380 mapped genera were ambiguous multi-clade genera. ₁

Mean mapped abundance fraction was 0.031 in the species-proxy mode versus 0.343 in the strict and relaxed modes. ₁

The species-proxy defense trend was positive but non-significant, with rho = 0.169 and Spearman p = 0.081, and no high-coverage test was feasible at mapped_abundance_fraction >= 0.25 because of low retained coverage. ₁

The bridge contained 8,242 genus-to-clade bridge rows, with a maximum of 433 clades per genus; the most ambiguous genera were pseudomonas (433), streptomyces (378), prevotella (358), streptococcus (214), and mycobacterium (186). ₁

A total of 862 of 1,392 observed genera were unmapped to the current pangenome bridge. ₁

These coverage, ambiguity, and unmapped-genus patterns refine the interpretation of the exploratory signal: it may reflect functional differences detectable through the retained mapped fraction, while the broader community-level relationship remains unresolved. ₁

The ecotype result supports caution about interpreting apparent ecological signals from heterogeneous sampling, but it also contradicts the specific explanation that clinical composition alone accounts for the null. The environmental group had a higher NaN partial-correlation rate than the human-associated group, 10/47 = 21% versus 7/100 = 7%; the report states that this filtering would, if anything, favor detection of a stronger environmental signal, which was not observed. ₂

The reanalysis also found that its median partial correlation across all 183 species was 0.081, compared with 0.003 in the original ecotype analysis, characterized as a 27x difference. It used all genomes with embeddings, including up to 3,505 genomes per species, rather than diversity-maximizing downsampling with a maximum of 250 genomes. Absolute correlations are therefore not comparable across methods, although the within-method Environmental versus Human-associated comparison remains valid. ₂

The new analysis refines this methodological caution: AlphaEarth embeddings covered only 28.4% of genomes, geographic coordinates were often missing or imprecise, and partial correlations assume linear relationships between distance matrices. It found no significant difference in environmental effects between environmental and host-associated bacteria (p=0.66), and cautioned that host-associated coordinates may represent collection sites rather than actual microenvironments. ₄

The pangenome-openness analysis refines the same limitation: the overlap was restricted to species with both openness statistics and ecotype results, and the report notes that upstream ecotype analyses may have limited power for species with few genomes. Its use of a single openness metric also leaves untested whether auxiliary fraction, Heap’s law alpha, pangenome fluidity, or function-specific gene classes would show associations. ₃

The Oak Ridge study supports the same caution about ecological resolution: 16S amplicon sequencing could not match laboratory organisms at species or strain level, and uranium-associated community restructuring also varied with redox conditions and carbon and energy sources. ₅

Cross-Project Evidence on Confirmatory and Exploratory Tests

ENIGMA contamination: report framing and central digest

The ENIGMA contamination report itself designates its Spearman tests as confirmatory and its adjusted models as exploratory sensitivity analyses. This concept relies on that labeling. ₁

Fraction-aware analyses were added to address potential confounding from collapsing multiple community fractions. They retain community_fraction_type parsed from sdt_community_name. ₁

The report’s overall verdict is that confirmatory evidence remains null and robust to index definition. Exploratory coverage-aware defense associations persist with positive effect estimates but are attenuated under global multiple-testing control. Exploratory significance is concentrated in defense and depends on coverage and covariate specification, so a broader effect across outcomes is not established. ₁

Within this ENIGMA subset, contamination gradients did not translate into a robust community-level shift in inferred stress-related functional potential when features were aggregated at genus resolution from broad COG (Clusters of Orthologous Groups) categories. The report treats two interpretations as compatible with this result but not established by it: contamination-driven turnover that is functionally redundant, and responses at species, strain or pathway scale. ₁

The report cites two outside studies as literature context only. Hemme et al. (2015; PMID: 26583008) reported strong contamination-linked taxonomic restructuring and reduced functional breadth in stressed Oak Ridge groundwater communities. Fan et al. (2025) reported pronounced compositional shifts but only modest functional-diversity decline in a mixed waste-contaminated Oak Ridge aquifer. The report reads its own null genus-aggregated result as consistent with contamination effects that do not collapse into one robust monotonic community-wide proxy. These outside studies are not analyses from this corpus. ₁

The central discoveries digest supports the same reading. In the ENIGMA contamination-functional project (108 overlap samples), primary univariate associations were non-significant across defense, stress, metabolism, and mobilome outcomes in both mapping modes. The contamination–defense association became significant only after mapped-coverage-aware sensitivity models were added (coverage-adjusted OLS contamination p = 3.98e-4 relaxed, 3.54e-3 strict; high-coverage subset Spearman p = 0.0207 relaxed, 0.0098 strict). The digest concludes that the strongest signal depends on mapping coverage and model specification rather than extending across outcomes. ₆

The project later added explicit confirmatory and exploratory labels and applied global Benjamini–Hochberg FDR across all reported p-values. After that step, the digest records the confirmatory genus-level defense Spearman tests as null (p=0.546/0.483; q=0.862/0.849). The strongest surviving signal was exploratory: the relaxed coverage-adjusted defense model retained q=0.046, while strict and other sensitivity signals attenuated above the q<0.1 level. The digest’s q=0.046 corresponds to the report’s FDR q = 0.0462. The digest therefore gives priority to the null confirmatory evidence and frames positive results as exploratory. _{6 1}

The digest also records that four contamination-index constructions (all-metals composite, uranium-only, top-3 variance metals, PCA-PC1) produced no significant confirmatory association after FDR correction (all q=0.546 across 8 confirmatory-variant tests). The null confirmatory result is therefore not an artifact of one index formulation. ₆

The choice of mapping mode itself changes the features being tested. When strict and relaxed modes were computed independently instead of reusing strict outputs, 1,140 of 1,590 genus-feature pairs differed between modes (~71.7%), and site stress scores differed for all 108 samples. Strict-versus-relaxed sensitivity is therefore analytically meaningful and should not be approximated by copied feature tables. This refines taxonomic-resolution-dependent-functional-inference. ₆

Phenotype and pangenome screens

The BacDive phenotype–metal-tolerance analysis shows the same need to separate corrected effects from uncorrected or underpowered ones; here d is Cohen’s d, a standardized mean difference. Seven of ten features passed FDR correction at q < 0.05: Gram stain (d = -0.61), oxidase (d = +0.53), motility (d = +0.35), urease (d = -0.18), enzyme breadth (rho = -0.06), nitrate reduction (d = +0.10), and catalase (d = +0.10). Three did not reach significance. H₂S production gave d = -0.867, n = 880, p = 5.8e-02, q = 7.3e-02; this estimate is unreliable because it rests on only 8 negative controls and is likely inflated by small-sample bias. Metabolite breadth gave rho = -0.013, n = 3,930, p = 4.2e-01, q = 4.7e-01. Acetate utilization gave d = +0.005, n = 422, p = 7.9e-01, q = 7.9e-01. ₇

The report therefore treats the H₂S–chalcophilic-metal association (d = -0.87, underpowered) as a hypothesis, not a finding. It proposes testing that hypothesis experimentally with sulfate-reducing bacteria and zinc/copper challenge. ₇

The hypothesis that metabolically versatile organisms harbor more resistance genes (Martin-Moldes et al. 2015) was not supported: metabolite-utilization breadth showed no correlation with metal score (rho = -0.01). The report explains this by proposing that metabolic versatility operates in different genomic neighborhoods than metal resistance. That explanation is interpretive and was not directly tested. ₇

Statistical significance and effect size also diverged for oxygen tolerance. Across 3,751 species with oxygen-tolerance data, the anaerobe–aerobe difference was negligible (d = -0.016, p = 0.55). Mean scores were 0.221 for facultative anaerobes, 0.216 for aerobes, and 0.215 for anaerobes. The three-group Kruskal–Wallis test gave H = 8.53 and p = 0.014, but the report judges the effect biologically trivial. ₇

When discussing a proposed Gammaproteobacteria comparison, the report notes that the pangenome-scale urease effect is near zero. The FDR-passing phenotype-level urease association (d = -0.18) therefore should not be assumed to carry over to pangenome scale. ₇

The AMR pangenome atlas did not apply formal Bonferroni or FDR correction across its six hypotheses. The report’s reason is that primary-test p-values were extreme (many < 1e-100), so correction would not change any conclusion. It labels its per-gene and per-phylum tests as exploratory and hypothesis-generating. ₈

Visual structure can likewise run ahead of statistical support. UMAP (uniform manifold approximation and projection) case-study plots for K. pneumoniae, S. aureus, and S. enterica showed visible environmental structuring. The environment–ecotype statistical test, however, was underpowered with current metadata. The report therefore neither concludes that ecotypes are unrelated to environment nor treats an association as established. ₉

Subsurface and carbon-census tests

In the clay-confined subsurface project, the H3 pairwise comparison of sulfate-reduction completeness between deep and soil-baseline samples reported OR=33.8 and Fisher p=5×10⁻⁵. This pairwise test is separate from the BH-adjusted H2 test, and the two results should not be conflated. ₁₀

The ENIGMA carbon census did not reject its pre-registered null of uniform coverage across compound classes (χ²=5.07, p=0.53, df=6). The test used the 8 enigma_isolate_call callables by class (Shikimates 5/25, Alkaloids 2/26, Terpenoids 1/17, Fatty acids 0/10, Polyketides 0/2, AA/Peptides 0/2, mixed 0/1). Adding the single measured-fitness lauric acid (Fatty acids 1/10) did not change the verdict. ₁₁

The census also detected no physicochemical separation between groundwater and necromass compounds (all contrasts p>0.14). The report reads this as evidence that a compound’s source does not predict its chemistry. ₁₁

The census’s environmental-enrichment outputs are explicitly exploratory, including the 83-entry soil-versus-freshwater abundance contrast. For its contrasts, the report treats direction and rank as trustworthy and biologically sensible but does not treat the p-values as calibrated significance. It favors the label-free outlier mode (Arm B) as the more defensible discovery signal. Before its enrichment statistics can be treated as defensible, the report says the per-sample rank test should be replaced by a study-aware mixed model or by sample-level permutation that respects study structure. ₁₁

Functional dark matter: pre-registered versus exploratory

In the functional-dark-matter project, 151 accessory dark-gene clusters across 31 species could be tested by comparing carrier with non-carrier genomes. Of these, 10 showed environmental-category enrichment at FDR < 0.05, and 1 showed significant AlphaEarth embedding divergence. ₁₂

The project’s lab–field concordance of 29/47 was marginal by a one-sided binomial test against p = 0.5 (p = 0.072), and the Wilson score 95% CI [0.474, 0.742] includes 0.50. Separately, Fisher’s combined probability across all 47 individual Fisher exact-test p-values gave p = 0.031. That combined test pools the individual association p-values rather than testing the concordance rate against 50%; the report treats it as stronger aggregate evidence that lab–field concordance is non-random. ₁₂

Pre-registered directional predictions provided the clearest confirmatory evidence in this project: all 7 pre-registered NMDC trait predictions had the expected direction (binomial sign test p = 0.0078). ₁₂

The broad exploratory screen, by contrast, is treated as likely confounded. In total, 76 of 105 Spearman correlation tests reached FDR < 0.05 (72.4%). The report warns that this high rate likely reflects confounding: the carrier genera (Pseudomonas, Klebsiella, Bacteroides) are among the most abundant and ubiquitous taxa in NMDC samples, so their abundance may correlate broadly with many abiotic variables regardless of condition class. The report favors the 4/4 pre-registered prediction rate as the more meaningful metric because it tests specific directional hypotheses. ₁₂

The same caution applies to the NMDC trait-condition validation across 76 functional_group columns and 6,365 samples. All 7 pre-registered predictions were reported at FDR < 10⁻²¹: nitrogen-source carriers correlated with nitrogen_fixation (ρ=0.60) and nitrate_denitrification (ρ=0.52); carbon-source carriers with aerobic_chemoheterotrophy (ρ=0.73) and fermentation (ρ=0.59); and anaerobic carriers with fermentation (ρ=0.59) and iron_respiration (ρ=0.45). Another 441/449 exploratory tests reached FDR < 0.05. The report cautions that these correlations likely reflect compositional coupling, because genera abundant in a sample contribute to both carrier abundance and community trait scores, rather than independent gene-phenotype validation. ₁₂

The report’s partial null approximations measure this caution but do not remove it. A sign test gave p = 0.0078 for the 7/7 pre-registered trait directions and p = 0.0625 for the 4/4 pre-registered abiotic directions. The report reads these as non-random directionality, although the abiotic value is above the conventional 0.05 level. Fisher’s combined probability across all 11 pre-registered tests gave p ≈ 0. The report uses an inflation factor of ~20× for exploratory tests (441/449 significant versus 22.5 expected) to quantify the compositional coupling. A full permutation test that shuffles sample labels was not run; the report lists it as a future direction that requires Spark access to raw per-sample matrices. The NMDC trait–condition correlation heatmap (fig21_trait_correlations.png) shows the 7 pre-registered comparisons alongside the exploratory ones. The report cautions that the high pre-registered trait correlations may themselves reflect compositional coupling rather than independent validation. ₁₂

Across these projects, the recurring pattern supports grading evidence by test status rather than by nominal p-values. Predeclared or pre-registered tests were mostly null, including the ENIGMA defense Spearman tests, the carbon-census class-coverage test, and the ecotype environmental-versus-host contrasts. Their own reports label the signals from broad screens, visual embeddings, uncalibrated rank tests, and uncorrected per-gene tests as exploratory, underpowered, or confounded. The main counterexample is the functional-dark-matter pre-registered NMDC directions, where confirmatory tests were positive. Even there, the report warns that the trait-score correlations are likely compositionally coupled. _{1,11,2,4,12,9,8,7}

Pre-registered families and post-hoc revisions in a gene-function atlas

The gene-function ecological agora project defines its primary hypothesis-test family as 4 pre-registered hypotheses, H1–H4 (Bacteroidota PUL; Mycobacteriota mycolic; Cyanobacteria PSII; Alm 2006 r ≈ 0.74). Biome enrichment and residualization replication are supporting test families. Bonferroni correction divides the significance threshold by the number of tests, here giving α = 0.05 / 4 = 0.0125. The report states that its significant findings (NB12 mycolic-acid p<10⁻⁶; NB16 PSII class p<10⁻⁵; P4-D1 enrichments all p<10⁻¹¹) survive this threshold trivially. The pre-registered tests carried specific Cohen’s d thresholds and falsification criteria. The exploratory observations and synthesis findings are descriptive patterns that surfaced during the synthesis pass, and the report says the two should not be conflated. ₁₃

A review rated multiple-testing correction for the 26 methodology revisions a critical-tier suggestion. In response, NB30 listed all formally tested significance claims, grouped them into test families, and applied family-wise Bonferroni and Benjamini–Hochberg false-discovery-rate (FDR) correction. Overall, 14 of 16 formal tests across the 3 families survived Bonferroni at family-specific α (88%). By family, 4 of 4 pre-registered hypotheses survived (100%), as did 5 of 6 P4-D1 biome-enrichment tests (83%) and 5 of 6 P4-D5 residualization-replication tests (83%). The 2 tests that did not survive, Mycobacteriaceae × soil and NB11 regulatory-versus-metabolic, are null results the project had already reported as null. Surviving correction is not the same as supporting a hypothesis. In the pre-registered family, H1 (Bacteroidota PUL) was a qualified pass, and H2 (mycolic) and H3 (PSII) were SUPPORTED. H4, the Alm 2006 r ≈ 0.74 reproduction, survived Bonferroni (p = 1×10⁻⁴³) but was NOT REPRODUCED. The report explains that the p-value was significant because of large n, while r=0.10–0.29 was too small. This refines the corpus-wide pattern: a corrected p-value does not show that a pre-registered effect-size criterion was met. ₁₃

The report rejects treating M1–M26 as 26 independent hypothesis tests on the same data. It describes most M-numbers as pre-registration corrections, methodology revisions documented for transparency. One response calls the M-revisions pre-registration corrections rather than 25 alternative tests, while NB30 refers to 26 methodology revisions; the report does not reconcile the two counts. Three revisions are genuine post-hoc metric changes on already-collected data: M14 (Sankoff replaces parent-rank dispersion), M21 (strict-class housekeeping) and M23 (n_neg ≥ 20 minimum). Each was triggered by a specific failed diagnostic and is reported with that diagnostic, not as a new hypothesis attempt. The report states that its p-values are not corrected for the M14/M21/M23 metric changes specifically. It asks readers to weight findings by effect size, ecological grounding and phenotype-anchor convergence rather than treat individual p-values as corrected. This refines the confirmatory/exploratory distinction by adding a third category, the post-hoc metric revision, which is neither a pre-registered test nor a new hypothesis attempt. ₁₃

Bonferroni was applied per test family (NB11, NB12 and NB16, each 4 tests at α/4). Atlas-wide multiple testing across 13.7M scores is exploratory by design. ₁₃

The Phase 3 architectural deep-dive received the gate verdict PASS_MIXED. Concordance between KO (KEGG Orthology) results and architectures was confirmatory for the consumer/participation signal (r=0.67; threshold r ≥ 0.6) and exploratory for the producer signal (r=0.09; r < 0.6). Phase 4 atlas claims therefore rest on Phase 1B and Phase 2 KO-level results as primary evidence. Phase 3 architectures serve as supporting evidence for consumer-side patterns and only as exploratory commentary for producer-side patterns. This supports grading evidence by test status, because a pre-set concordance gate assigns the confirmatory label to each analysis arm rather than nominal significance. ₁₃

Leaf consistency is the fraction of clade species carrying a KO, computed once per rank × clade × KO tuple. It is a descriptive point estimate, not a tested significance claim. Figure S7 shows distribution shapes (focal clade versus atlas reference). The report presents its medians (mycolic 0.15, PSII 0.88, PUL 0.41) as observations, not p-values, and on that basis holds that no multiple-testing correction applies and rejects the reviewer’s objection. It accepts, as a forward-looking caveat, that any future pipeline using leaf_consistency as a test statistic would need multiple-testing correction. The current synthesis does not test LC against null distributions. ₁₃

Underpowered per-feature and cohort tests

In the Harvard Forest warming project, DNA-organic set enrichment was significant, but no individual RNA KO survived FDR (q<0.10) at n=11+11 across 14K KOs. The specific carbon-cycling RNA-pool signals are nominal p-values that do not survive FDR across 14K KOs, although the report reads the direction of the strongest signals as biologically interpretable. The report attributes the limited per-KO FDR power to sample size in the omics-rich layers (n=28 metagenome, n=39 metatranscriptome) across 12-14K KOs. These per-KO signals are therefore exploratory directional leads rather than confirmed features. ₁₄

Heated mineral samples had significantly fewer detectable metabolites (~7% drop), but no individual metabolite passed BH-FDR at this sample size. The nominal Fisher signals were ChEBI:71028, heated-only (11/11 vs 6/11), p=0.035; ChEBI:30918, control-only (0/11 vs 6/11), p=0.012; and ChEBI:27967, heated-enriched with OR=12 (10/11 vs 5/11), p=0.063. These per-metabolite signals remain nominal leads and do not survive multiple-testing control. ₁₄

In the IBD phage-targeting project, H3e tested whether anti-microbial antibody titers correlate with Tier-A pathobiont abundance within IBD patients. The design used n=67 subjects across 3 sites (CCHMC/Harvard/Emory), with site as a covariate, and an effect threshold of |r|>0.40 calibrated to power 0.80 at n=67 and α=0.05. The plan treated the single-cohort caveat as structural. Inputs were HMP2 MetaPhlAn3 abundance and 2,520 serology measurements from 12 assays, and all 67 serology subjects matched metagenomics. The project ran 48 (assay × species) tests using site-adjusted partial Pearson r. The top pair was ANCA × M. gnavus +0.31 (FDR 0.40), and no pair cleared the prespecified |r|>0.40 + FDR<0.10 threshold. ₁₅

A site-stratified breakdown showed most associations concentrated in 1 of 2 productive sites (Harvard 33 + CCHMC 31 + Emory 3 subjects). Single-site partial r reached |r|≈0.40–0.46 individually, but the associations diluted when the cohort was aggregated. The report labels H3e PARTIAL while stating that it is NOT SUPPORTED at the strict plan threshold. It gives four reasons for the PARTIAL label: the cohort passed sanity checks on canonical IBD-serology patterns, all top pairs had a biologically plausible direction, the site-level magnitudes reached the threshold range, and the plan acknowledged a structural single-cohort caveat. The report also states that n=67 across 3 unevenly distributed sites is structurally low-power for detecting |r|>0.40 at α=0.05 after correction for 48 tests, and that H3e was acknowledged-weak by design. This supports keeping the strict prespecified verdict primary, since site-level effects that reach the threshold do not turn a null pooled test into a confirmed association. ₁₅

Single-study candidate lists show the same need for stricter replication. Provisional E3 Tier-A listed 40 candidates from a single study (HallAB_2017). The leading candidates were H. symbiosa (+4.64), M. gnavus (+4.46), B. coccoides (+4.22), R. faecis (+4.14), C. spiroforme (+4.11), S. salivarius (+4.04) and E. innocuum (+3.68). Under the stricter E3 × HallAB_2017 within-substudy test, Flavonifractor plautii replicated at +2.26 (FDR 0.02). Blautia wexlerae did not replicate (+0.25, FDR 0.80) and was removed from the earlier ‘rock-solid E3 triad’, where its inclusion had relied on cohort-level rather than E3-restricted within-substudy evidence. Bilophila wadsworthia had a pooled centered-log-ratio difference (CLR-Δ) of +0.07, which was not significant, with 3/4 sign concordance. ₁₅

Where the IBD project built explicit permutation nulls, its signals cleared them. In the first case, 52 CD-up and 22 CD-down pathways (CD, Crohn’s disease) passed FDR < 0.10 with |effect| > 0.5 (74 either-direction). The CD-up permutation-null mean was 0.077 ± 0.87, and the empirical p was < 0.001. In the second case, pathway–pathobiont attribution reached a maximum |ρ_meta| of 0.797 against a null of 0.177 ± 0.019 (empirical p < 0.001). There were 137 pathway–pathobiont pairs with |ρ_meta| > 0.4, all 100% sign-concordant across the 3 robust substudies. This contrasts with the functional-dark-matter NMDC screen, where no full sample-label permutation was run and significance inflation was attributed to compositional coupling. _15,12

Phylogeny-controlled metal–niche breadth: a positive confirmatory test with mixed robustness arms

The MicrobeAtlas metal-ecology project links genus-level metal type diversity, inferred from AMRFinderPlus pangenome annotations across 6,789 GTDB species, to global ecological niche breadth derived from a 464,000-sample 16S amplicon atlas. It partitions and controls phylogenetic signal with Pagel’s λ and PGLS (phylogenetic generalized least squares, a regression that accounts for shared ancestry). The report finds that genera with broader metal type resistance repertoires are significantly associated with broader ecological ranges beyond what phylogenetic relatedness predicts. It describes this association as robust to genome size, species richness, prevalence filtering, OTU abundance filtering, and all 13 environment categories. That broad robustness claim must be read alongside the non-significant strict-prevalence and leave-one-metal-out results described below. ₁₆

The report names its primary confirmatory hypothesis as whether metal AMR trait diversity predicts bacterial niche breadth after phylogenetic correction. The 6 simple PGLS models (NB05) with Bonferroni correction constitute that confirmatory analysis, and the report also lists Pagel’s λ for niche and AMR traits (NB04) as confirmatory. Its statement that all other analyses are exploratory or post-hoc robustness checks is therefore broader than its own list. The analyses it explicitly labels exploratory are the multi-predictor PGLS, robustness analyses R1–R4, sensitivity analyses S1–S4, the archaeal PGLS, and all analyses added in response to the post-hoc review process. The analysis was not preregistered, and the confirmatory/exploratory distinction was documented post hoc. A reviewer-response table states that all robustness and sensitivity analyses are labelled exploratory, while the primary confirmatory result (6 PGLS + Bonferroni) stands independently. ₁₆

The report’s multiple-testing table counts ~47 tests, summarized in a list that labels analysis ranges differently from the prose above (R1–R2, R4–R6 and S1–S5); the report does not reconcile the two labellings. ₁₆

• Simple PGLS (confirmatory): 6 tests, Bonferroni (α/6 = 0.0083).
• Pagel’s λ (confirmatory): 6 tests, LRT, no additional correction.
• Multi-predictor PGLS: 3 predictors in 1 model, not corrected beyond model.
• Robustness R1–R2, R4–R6: 9 coefficients, exploratory only.
• Archaeal PGLS R3 (supplementary): 3 coefficients, exploratory only.
• Sensitivity S1–S5: 24 coefficients, exploratory only.
• OTU abundance filter S5 (Pagel’s λ): 1 test, exploratory only.

The primary confirmatory result is metal types → B_std with p = 1.5×10⁻⁴, which the report calls Bonferroni-corrected. The wording does not establish whether 1.5×10⁻⁴ is an adjusted p-value or a p-value assessed against the Bonferroni threshold. The report adds that the primary p also survives a Bonferroni correction for the full 47-test family (threshold = 0.05/47 ≈ 0.0011). It notes that this was not the stated correction method and presents it as an upper bound. A reviewer-response table records the primary result as surviving both Bonferroni (α/47=0.0011) and FDR (q=0.003). The robustness analyses were conducted after the main analysis was complete, in response to anticipated reviewer concerns, and the report asks readers to interpret them as reassurance rather than confirmatory evidence. ₁₆

Under Benjamini–Hochberg FDR across all 47 tests, 23 of 47 were significant at q < 0.05. The report states that all bacterial metal type diversity effects survive FDR correction, but that wording is broader than its own results: the leave-one-metal-out and aquatic-exclusion arms described next did not pass FDR, and the strict-prevalence arm was non-significant. The models it enumerates as surviving are ₁₆:

• simple PGLS B_std (q = 0.0028);
• multi-predictor PGLS (q = 0.0037);
• R1 covariate models (q < 0.004);
• R5 genome size covariate (q = 0.0028);
• 3-covariate PGLS (q = 0.0028);
• S3 within-genus SD (q = 0.007);
• S4 log(n_species) (q = 0.0037).

The S2 leave-one-environment-out models were significant at q < 0.05 for 12 of 13 environments. Excluding aquatic records gave β=+0.0085, SE=0.0039, p=0.031 at n=606 but q=0.061, and the report explicitly marks this exclusion as nominally significant but not FDR-significant. The S1 leave-one-metal-out models did not survive FDR correction; the report says this was expected because excluding one metal reduces predictor variance. The archaeal PGLS remained non-significant (q = 0.55). ₁₆

Each of the 12 other habitat exclusions retained a positive, BH-FDR-significant metal-types coefficient at n=606 (β, SE, p, q) ₁₆:

• excl_agricultural: +0.0107, 0.0040, 0.0073, 0.016
• excl_desert: +0.0120, 0.0042, 0.0040, 0.011
• excl_farm: +0.0115, 0.0042, 0.0061, 0.015
• excl_field: +0.0118, 0.0040, 0.0030, 0.010
• excl_flower: +0.0118, 0.0044, 0.0077, 0.017
• excl_forest: +0.0142, 0.0042, 0.0007, 0.004
• excl_leaf: +0.0130, 0.0043, 0.0025, 0.010
• excl_paddy: +0.0123, 0.0042, 0.0036, 0.010
• excl_peatland: +0.0131, 0.0044, 0.0027, 0.010
• excl_plant: +0.0105, 0.0039, 0.0069, 0.016
• excl_shrub: +0.0126, 0.0043, 0.0034, 0.010
• excl_soil: +0.0113, 0.0039, 0.0035, 0.010

Not every positive estimate in the project clears correction. The core-fraction association with number of environments (n = 606, β = +0.142, p = 0.038) is a nominal result that the report marks as not Bonferroni significant. ₁₆

Under the strict ≥5% prevalence threshold (R4), the metal-types estimate stayed positive but was non-significant (n=379, β=+0.0166, SE=0.0099, p=0.092, no q-value reported). The original estimates were β = +0.0215, SE = 0.0056 and p = 1.5×10⁻⁴. The report attributes the loss of significance primarily to reduced sample size (n drops 37%: 606 → 379) rather than a change in direction or magnitude (β changes from +0.0215 to +0.0166, a 23% reduction). That attribution is the report’s interpretation. The SE rose from 0.0056 to 0.0099 while β also fell, so the strict-filter estimate is both smaller and less precise, and no analysis in the report separates the two contributions. ₁₆

The archaeal PGLS (n = 48 genera) found a positive but non-significant β = +0.0145 (p = 0.467) for metal types. The report describes this as a severely underpowered analysis (11% power at α = 0.05) and says the null should not be interpreted as evidence against the association in archaea. ₁₆

The field-validation Track A used 1,624 groundwater samples. Metal-type diversity had a small positive Spearman association with groundwater prevalence (ρ = +0.112 among 767 genera, p = 0.0019; Mann–Whitney p = 0.007), with a modest effect of 0.62% → 0.81% median prevalence for bottom-Q1 vs top-Q4 genera. A reviewer response describes top-quartile metal-diversity genera as 31% more prevalent in groundwater. Groundwater-specific fold enrichment, however, was not significant (ρ = +0.042 among 767 genera, p = 0.242). Top-quartile and bottom-quartile genera also did not differ in fold enrichment (213 vs 429 genera, p = 0.181). The report calls this a meaningful null: Track A confirms the prevalence association but not groundwater-specific enrichment, so it is consistent with the PGLS finding but does not uniquely support the metal-contamination mechanism. The report proposes two alternatives: the prevalence advantage may be a general breadth effect, or the 1,624-sample groundwater subset may be underpowered to detect a specificity signal. ₁₆

Track B ran a full amplicon pipeline on all 133 PRJNA1084851 samples (24,295 OTUs; 133/133 QC pass) from 8 groundwater monitoring wells. Its Kruskal–Wallis test of community-weighted mean (CWM) metal-type diversity across the 8 wells gave H = 29.10 and p = 0.0001. That is an omnibus test of differences among wells, not a plume-versus-background contrast. The report’s summary and reviewer response nonetheless cite this p-value for higher CWM in contamination-plume wells FW215/FW216. The plume ranking itself is descriptive: median CWM ran FW215 (1.37) > FW216 (1.36) > GP01 (1.23) ≈ GP03 (1.22) > MLSB3 (1.14) > MLSA3 (1.09) ≈ FW202 (1.08) ≈ MLSG4 (1.08). CWM also increased with time after carbon amendment (Spearman ρ = +0.383, p < 0.0001 across all wells; ρ = +0.576, p = 0.0001 in FW216 alone). CWM used only the OTUs joinable to genus metal-diversity data, 16.8% of reads per sample on average. The report warns that if the uncovered genera are systematically more metal-sensitive, CWM would be biased upward, and that robustness to coverage-fraction variation has not been formally tested. It recommends correlating per-sample CWM with per-sample covered-read fraction as a diagnostic. The report states that both tracks independently confirm the PGLS prediction in field data. Given the Track A fold-enrichment null and the Track B coverage limitation, the field tracks are graded here as consistent with the prediction; neither track alone proves the proposed contamination mechanism. ₁₆

This project refines the corpus-wide pattern in which predeclared tests were mostly null. Here the designated confirmatory PGLS is positive and survives family-wise and FDR correction. Its post-hoc robustness arms are mostly concordant, but the strict-prevalence, leave-one-metal-out, archaeal, aquatic-exclusion and groundwater-specificity arms are null or not FDR-significant. Those arms are exploratory reassurance, and the confirmatory label itself was assigned post hoc rather than preregistered. ₁₆

Small-sample anchors, correction tiers and broad screens

The Oak Ridge comparison also records a near-threshold genus. Azospirillum had a nominal positive uranium correlation (rho=+0.20, p=0.042) that was marginal after FDR correction (q=0.077), so it is not among the genera with FDR-significant uranium associations. ₅

The lanthanide methylotrophy atlas grades its H2 hypothesis as partially supported. Soil/sediment was by far the strongest broad-class xoxF enrichment (OR=1.92, p_BH=6×10⁻³⁹ across 13,779 genomes). Marine environments were also significantly enriched (OR=1.31), and host-associated environments were dramatically depleted (OR=0.058, p_BH=0). REE-impacted samples (n=37) showed a 3.5-fold descriptive elevation (10.81 % xoxF rate, odds ratio 3.51; 10.8 % vs 3.7 % baseline). The pre-registered plan-v3 caveat applies to that elevation: n=37 is too small to clear the FDR threshold (p_BH = 0.082). The methylotrophic class contained only 5 genomes and, despite a 20.0 % xoxF rate and odds ratio 8.70, had an adjusted p of 0.20. ₁₇

The report labels its REE-AMD anchor descriptive only because it is small (n=37) and from a single bioproject. It states that inferential claims about REE-impacted environments require a larger and more independent collection. This supports grading small pre-registered strata by sample size: a large descriptive fold change in a small single-source class is not an inferential finding. ₁₇

The plant-microbiome ecotype project shows correction-tier dependence in two separate analyses. In the H7 plant-versus-non-plant subclade scan, the testable species set required ≥3 plant genomes, ≥3 non-plant genomes, and ≥2 subclades after the genome-ID fix. Of the 17 testable species, 5 passed Bonferroni-corrected Fisher’s exact at α = 0.05/17 = 0.0029. Three of those 5 (X. vasicola, P. avellanae, X. campestris) also satisfy Cochran’s rule for chi² and are the most robust; the other 2 pass only via Fisher’s exact, which is not subject to the small-cell constraint. In the same H7 framework, P. amygdali’s borderline chi² p = 0.031 from the 5-species subset did not survive Bonferroni in the larger scan and remained invalid under Cochran. In the separate H6 subclade × host scan, Ralstonia solanacearum passed BH-FDR but not Bonferroni-across (p = 5.9×10⁻³ within / 1.8×10⁻² q-BH). This supports treating the strength of these ecotype associations as dependent on the correction tier and test validity rather than on a single nominal p-value. ₁₈

The same project’s within-genus marker test used n = 200 permutations, which is under-resolved for p ≤ 0.005, so T3SS is marginal. No multiple-testing correction was applied across 15 markers. Under Bonferroni α = 0.0033, T3SS would be demoted, while nitrogen fixation and ACC deaminase would remain significant. ₁₈

In the plant-growth-promotion pangenome project, only 214/11,272 species (1.9%) carry ipdC, which limits statistical power for the H4 stratified analysis. The report says the soil reversal (OR = 0.30, p = 0.02) should be treated as hypothesis-generating rather than conclusive. ₁₉

The pitfalls digest records an independent-evidence gate for Tier-A candidates. Species that pass both within-ecotype differential abundance (possibly leakage-contaminated) and an independent CD-vs-control contrast are treated as the trustworthy Tier-A. In NB04c, this gate shrank the Tier-A list from 33 to 3 rock-solid candidates. This supports the IBD project’s stricter within-substudy replication described above. ₂₀

In the prophage-ecology project, no individual TerL lineage showed significant environment-specific enrichment after FDR correction (0/500 tests). The report reads this as lineage-level environmental adaptation not exceeding what host phylogeny predicts, and turns instead to Shannon entropy analysis of 824 lineages with ≥5 species. The module-level evidence that the report does treat as concordant rests on agreement between two analyses rather than on one screen: B_head_morphogenesis, C_tail, and G_anti_defense were significant in both pangenome enrichment and NMDC correlations. ₂₁

The soil-frontier project did not support the clay shield hypothesis at global scale. The shield-efficiency test gave low-clay CV R² = −0.268 and high-clay CV R² = −0.292, with a difference of 0.024 [95% CI: −0.423, 0.161] that was not significant because the CI includes zero. Clay appeared as a consistent feature (importance ≈ 0.14) but did not improve predictive accuracy in high-clay soils. The report notes that clay content shows correlational associations with microbial functional potential and some interaction with industrial stressors. ₂₂

The report also flags that its negative out-of-sample R² diagnosis is incomplete: all three predictive model families achieved negative CV R², predicting worse than the training mean. It has not distinguished three causes: (a) train/test distributional shift due to spatial autocorrelation within GroupKFold folds, (b) high-leverage outlier samples in test folds, or (c) genuine unpredictability of functional gene counts from the measured predictors at global scale. Only (c) supports a strong null interpretation; (a) and (b) would indicate a modelling failure rather than a biological null. ₂₂

The soil metal functional-genomics screen reports 2,355 significant COG–metal associations (FDR < 0.05) across 9 metals (Cu, Co, Cr, Ni, Zn, Pb, As, Cd, Hg) in 51,748 soil samples. Chromium and Lead drive the strongest signals, and transporters (ABC, RND) and biosynthesis genes dominate the top hits. The report itself flags the discovery rate as high: with 9 metals × 435 COGs = 3,915 tests, a 60% discovery rate. It argues that metals co-contaminate, so tests are not independent and Benjamini–Hochberg FDR will be anti-conservative under positive correlation. It concludes that the true FDR is likely higher than reported and names a partial correlation model as the remedy. This is the report’s methodological assessment, not a recalculated FDR. It supports the functional-dark-matter warning above that very high exploratory discovery rates can reflect shared structure rather than independent associations. _23,12

Effect sizes (Spearman ρ) had not been systematically reported across all 2,355 associations, and the report notes that many may be statistically significant but biologically small. A pending effect-size audit would examine the ρ distribution across the 2,355 associations and flag ρ < 0.05. ₂₃

Together these projects refine the earlier synthesis. A positive confirmatory result can coexist with null exploratory robustness arms, as in the MicrobeAtlas metal–niche study, and pre-registered strata can be too small to infer from, as with the REE-impacted lanthanide class. Significance can also depend on the correction tier, as in the plant ecotype scans, or on dependence among tests, as in the soil metal screen. A negative predictive result can likewise reflect modelling failure rather than a biological null, as the soil-frontier report acknowledges. _{16,17,18,23,22}

Tensions

The principal tension is between null predeclared genus-level Spearman tests and positive exploratory models that adjust for mapped coverage and additional covariates. ₁

The confirmatory tests provide no robust monotonic contamination–defense association, whereas the relaxed coverage-adjusted model has FDR q = 0.0462 and the strict model has FDR q = 0.130. ₁

The discordance is not resolved by the high-coverage subset or fraction-aware analyses, because high-coverage defense tests have global FDR q-values of 0.301 and 0.189, and within-fraction tests are non-significant. ₁

The ecotype reanalysis adds a related methodological tension: the original and reanalysis median partial correlations were 0.003 and 0.081, respectively, but the changed genome sets and downsampling procedures prevent treating the 27x difference as a biological effect. ₂

The new ecotype analysis adds a third, related tension: its environment median partial correlation was 0.0025 and phylogeny median was 0.0143, while the reanalysis median was 0.081. The differing genome coverage, species inclusion, embedding coverage, and sampling procedures prevent treating these values as directly comparable biological estimates. _{4 2}

The pangenome-openness result refines rather than resolves this tension: openness versus environment was rho = -0.05 with p = 0.54, and openness versus phylogeny was rho = 0.03 with p = 0.73, so the species-level openness metric did not explain either set of ecotype effect sizes. This does not contradict the existence of environmental or phylogenetic gene-content effects; it leaves open whether the effects are confined to functional subsets or are missed by the openness summary. ₃

The Oak Ridge evidence supports the confirmatory-null side for aggregate laboratory-to-field prediction but refines the exploratory side by showing statistically significant associations in both directions at genus level after multiple-testing correction. This tension is not resolvable from the field correlations alone because pH, dissolved oxygen, carbon sources, temporal history, community interactions, and genus-to-strain variation were not controlled. ₅

The project evidence therefore supports a cautious hypothesis that contamination-linked functional differentiation may exist at finer taxonomic, pathway, or strain resolution than the current genus-level COG-fraction proxies, but it does not establish that hypothesis as a community-wide finding. ₁

Similarly, the ecotype report proposes—but does not directly establish—that AlphaEarth embeddings may capture regional epidemiological patterns rather than ecological differences, and that unequal genome counts may provide greater statistical power for weak correlations. ₂

The new analysis supports this hypothesis as a limitation rather than resolving it: it suggests that AlphaEarth embeddings may not fully capture ecologically relevant environmental variation, while its result that phylogeny generally dominates whole-genome gene-content similarity does not exclude environmental effects on specific gene subsets. ₄

The functional-dark-matter synthesis summarizes its environmental evidence with figures that must be read alongside their test definitions. It states that 10/137 clusters show significant environmental enrichment, whereas the cluster-level analysis reports enrichment in 10 of 151 testable accessory dark-gene clusters; the quoted passages do not explain the different denominators. The synthesis gives the overall concordance rate as 61.7% against the 50% chance level, and the binomial test of that rate was not significant at 0.05 (p = 0.072). The Fisher’s combined p = 0.031 cited alongside it is a different test that pools individual association p-values; it does not test the concordance rate against 50%. The synthesis also states that NMDC independent validation confirmed all 4 testable pre-registered predictions (nitrogennitrogen, pHpH, anaerobic~dissolved oxygen). These are predictions against abiotic variables, distinct from the 7 pre-registered predictions tested against community functional trait scores, so the two counts describe different analyses rather than conflicting totals. ₁₂

The MicrobeAtlas metal–niche project adds a tension of the opposite shape to the ENIGMA case. Its confirmatory simple PGLS is positive (p = 1.5×10⁻⁴), whereas several exploratory arms are null or not FDR-significant: strict prevalence filtering (p = 0.092), leave-one-metal-out models, archaeal PGLS (q = 0.55), aquatic exclusion (q = 0.061) and groundwater-specific fold enrichment (p = 0.242). The report attributes these mainly to reduced sample size, reduced predictor variance or low power. Those attributions are interpretive, and its abstract’s broad robustness claim is not uniform across arms, so this wiki leaves the tension open. ₁₆

Relation to Other Concepts

This concept supports adaptive-versus-housekeeping-functional-differentiation because nominal p-values and exploratory FDR results differ from the broader evidence needed for a stable ecological association. ₁

It refines taxonomic-resolution-dependent-functional-inference by showing that mapped abundance fraction changes the apparent defense association and that low-coverage modes can lose usable signal. ₁

It supports taxonomic-resolution-dependent-functional-inference because strict, relaxed, and species-proxy mapping modes produce different coverage and association results. ₁

It connects to confirmatory-exploratory-ecological-association-discordance as a general framework for separating predeclared evidence from sensitivity-generated hypotheses. ₁

The ecotype reanalysis refines this framework by showing that genome-level environmental classification, continuous environmental fractions, and a more systematic classification scheme can confirm a null without eliminating sampling and methodological explanations. ₂

The pangenome-openness analysis refines this framework by showing that a species-level openness summary did not predict either environmental or phylogenetic gene-content effect sizes, while leaving function-specific and alternative openness metrics as testable explanations for the null. ₃

The Oak Ridge comparison supports this framework by showing that a suggestive aggregate laboratory–field relationship can coexist with significant, directionally mixed genus-level associations, emphasizing the need to separate broad confirmatory hypotheses from finer exploratory contrasts. ₅

The functional-dark-matter evidence connects this concept to lab-field-fitness-concordance: pre-registered directional predictions passed sign tests, while broad exploratory NMDC screens were judged likely compositionally confounded. ₁₂ The carbon-census call for study-aware enrichment statistics connects it to study-batch-confounding-of-environmental-associations. ₁₁ The non-significant and underpowered BacDive phenotype features connect it to phenotype-database-coverage-bias. ₇

The MicrobeAtlas metal–niche and groundwater-prevalence results connect this concept to environmental-resistome, and the loss of significance under strict prevalence filtering connects it to sampling-depth-and-downsampling-effects. ₁₆ The small, descriptive REE-impacted xoxF stratum connects it to environmental-distribution-of-lanthanide-methylotrophy. ₁₇ The NB04c independent-evidence gate connects it to selection-on-outcome-leakage. ₂₀

Open Directions

• Replace broad COG-fraction proxies with curated metal-stress gene sets and pathway-level summaries, then test whether the confirmatory contamination association remains null at finer functional resolution. ₁
• Add species- or strain-level ENIGMA or metagenomic data and repeat the confirmatory and coverage-adjusted models to test whether genus-level aggregation masks adaptation. ₁
• Fit models including depth, location cluster, sampling date, and compositional controls, using mixed-effects or hierarchical structure, to determine whether the exploratory defense association persists beyond coarse location_prefix adjustment. ₁
• Investigate the 862 unmapped genera and expand the genus-to-clade bridge to test whether missing coverage changes the direction or magnitude of the contamination–functional association. ₁
• Reanalyze the 212 sample-fraction rows with preregistered fraction-stratified and pooled contrasts to determine whether the pooled exploratory signal can be reproduced across the 0.2_micron_filter and 10_micron_filter fractions. ₁
• Compare downsampled and full-genome ecotype extraction under a shared genome set, then add genome count as a covariate, to determine whether the 27x partial-correlation discrepancy is methodological or reflects sampling power. ₂
• Repeat the ecotype association using functional subsets and structured ENVO terms to test whether whole-genome Jaccard distances or coarse environmental categories mask environment-linked gene-content signals. ₂
• Recompute the original, reanalysis, and new ecotype correlations on a shared genome set with matched embedding coverage and nonlinear distance-based methods, then test COG categories including V-Defense and L-Mobile to determine whether environmental effects are localized to functional subsets. ₄
• Compare openness with auxiliary fraction, Heap’s law alpha, and pangenome fluidity on the same species and genome set, then test openness-by-lifestyle interactions and L-Mobile/V-Defense subsets to determine whether the null is specific to the single openness metric or extends to functional pangenome structure. ₃
• Match Fitness Browser metal-tolerance scores to species- or strain-resolved Oak Ridge metagenomic observations and fit multivariate models controlling for pH, redox, carbon sources, and sampling date to test whether the suggestive aggregate relationship and bidirectional genus signals persist. ₅
• Test the underpowered H₂S–chalcophilic-metal hypothesis (d = -0.87) experimentally with sulfate-reducing bacteria under zinc/copper challenge, to determine whether the non-significant BacDive association reflects the small sample of 8 negative controls rather than the absence of an effect. ₇
• Replace the ENIGMA carbon-census per-sample rank test with a study-aware mixed model or a sample permutation that respects study structure, to determine whether the exploratory environmental enrichment ranking yields defensible, calibrated enrichment statistics. ₁₁
• Re-derive the functional-dark-matter denominator of testable clusters (151 versus 137), then rerun the NMDC carrier–trait correlations against compositionally controlled nulls, to separate genuine gene–condition concordance from compositional coupling. ₁₂
• Apply FDR correction to the AMR pangenome atlas per-gene and per-phylum tests, to identify which hypothesis-generating signals survive multiple-testing control. ₈
• Run the full sample-label permutation test on raw NMDC per-sample matrices (requires Spark access) to measure how much of the 441/449 exploratory significance and the 7/7 pre-registered trait correlations survives once compositional coupling is broken. ₁₂
• Test leaf_consistency against null distributions with multiple-testing correction to determine whether the descriptive focal-versus-atlas differences in Figure S7 (mycolic 0.15, PSII 0.88, PUL 0.41) are significant departures. ₁₃
• Add serology-matched cohorts and fit site-as-random-effect models for the 48 assay × species pairs to test whether the single-site |r|≈0.40–0.46 associations reproduce or reflect site-specific structure. ₁₅
• Increase omics replication beyond n=11+11, or test pre-specified carbon-cycling KO and ChEBI sets, to determine whether the nominal Harvard Forest warming signals survive FDR. ₁₄
• Refit the MicrobeAtlas metal-types PGLS on random 379-genus subsamples of the full 606-genus set, without the strict ≥5% prevalence filter, to test whether the strict-filter p = 0.092 reflects reduced n, as the report proposes, or the smaller β under stricter filtering. ₁₆
• Correlate per-sample Track B CWM with per-sample covered-read fraction, as the report recommends, and add a pairwise plume-well versus non-plume-well contrast, to test whether the FW215/FW216 ranking survives the 16.8% read-coverage limitation. ₁₆
• Assemble REE-impacted genomes from additional independent bioprojects beyond the single-bioproject n=37 anchor, then retest xoxF enrichment under the plan-v3 FDR threshold, to determine whether the descriptive 3.5-fold elevation is inferentially supported. ₁₇
• Run the proposed partial correlation model and the pending ρ < 0.05 effect-size audit across the 2,355 COG–metal associations, to estimate how many associations survive dependence-aware correction with non-trivial effect sizes. ₂₃
• Compare spatially blocked and random cross-validation folds, with and without high-leverage test samples, to determine whether the negative CV R² in the soil-frontier models is a modelling failure or a biological null. ₂₂
• Rerun the plant-marker within-genus permutation test with more than 200 permutations and correction across the 15 markers, to resolve whether the marginal T3SS signal survives. ₁₈