Unequal evidence callability can make ecological contrasts untestable

Core idea

Ecological comparison requires comparable evidence on both sides of a contrast. When pathway, organism, or phenotype calls exist for only one source or group, the comparison can become untestable rather than merely statistically underpowered. ₁ “Callability” means that a compound could be linked to an organism-level utilization prediction or measured fitness result through the queried resources; it does not mean that the compound is biologically absent from an uncalled group. ₁

Metabolic-utilization inference is also shaped by whether a compound has a resolvable structure, database representation, reaction linkage, literature support, and genome-level annotation. ₁ “Organism-dark” therefore marks a resource-defined coverage boundary and discovery signal, not evidence that a compound is unknown to science or unusable by microorganisms. ₁ This extends genomic-under-representation from uncharacterized genomic functions to compounds that cannot currently be connected to organisms, pathways, or genetic determinants, and refines metabolic-model-gapfilling by separating missing pathway links from compounds lacking usable database representation. ₁

This distinction connects ENIGMA Carbon Census — Tiered Knowledge Census of 83 Enrichment Compounds to occurrence-versus-catabolic-activity, sampling-depth-and-downsampling-effects, and ecotype-environment-gene-content. ₁ The Ecotype Reanalysis — Environmental vs Human-Associated Species refines this principle beyond compound utilization: an ecological environment-versus-gene-content contrast can remain weak even after clinical sampling bias is explicitly measured, so unequal representation should be tested as a possible confounder rather than assumed to explain the result. ₂

The ENIGMA evidence funnel and its dark boundary

The ENIGMA Carbon Census evaluated 83 enrichment compounds: 59 from SSO groundwater and 24 from necromass. ₁ All 83 were structure-resolved with InChIKeys through PubChem, 54 were linked to KEGG, and 9 were initially callable through either an ENIGMA-isolate utilizer call or a Tier-1 measured RB-TnSeq (random barcode transposon sequencing, a pooled mutant-fitness assay) carbon-source fitness experiment. ₁ The resulting funnel was 83 compounds → 83 structure-resolved → 54 KEGG-linked → 9 callable → 74 organism-dark. ₁

The 74/83 dark compounds, or 89% of the census, were not linkable to organism-level utilization through the queried the KBase Data Lakehouse and curated resources. ₁ The dark fraction was similar between sources: 53/59 groundwater compounds, or 90%, and 21/24 necromass compounds, or 88%, were organism-dark. ₁ This supports the interpretation that the observed gap tracks chemical representation and annotation coverage more strongly than sampling source, but does not establish that source has no biological effect. ₁

The dark set is heterogeneous: 33 compounds were KEGG-linked but lacked a reaction in the queried genomes, 29 were fully orphan compounds with no KEGG link, 6 were biosynthesis-known but catabolism-unknown, and 6 were represented only by generic reactions. ₁ The 29 fully orphan compounds are the hardest discovery targets because they lack both a KEGG link and a genome-level reaction connection; they were disproportionately necromass-derived, comprising 13/24 necromass compounds versus 16/59 groundwater compounds. ₁ This refines organism-darkness by distinguishing missing pathway links from missing chemical identifiers or database entries. ₁

The six biosynthesis-known dark compounds—Tyramine, guanidineacetic acid, cinnamic acid, caffeic acid, palmitic acid, and farnesol—should not be treated as equivalent to fully orphan compounds because external MIBiG and biosynthetic-literature consultation may clarify their biology. ₁ This supports evidence-triangulation-for-functional-annotation: different dark buckets require different evidence and should not be collapsed into one unknown class. ₁

Evidence asymmetry and chemical-space coverage

The 8 ENIGMA-isolate-callable compounds were salicylic acid, 3-hydroxybenzoic acid, 4-hydroxybenzaldehyde, phthalic acid, terephthalic acid, Phenylethylamine, xanthine, and Abscisic acid. ₁ Lauric acid, a Fatty acid, was additionally callable only on a measured-fitness basis. Its Tier-1 measured carbon-source experiment in the Fitness Browser comes from a reference bacterium, not an ENIGMA isolate. The census table therefore lists 0 ENIGMA-isolate strains, 0 high-certainty strains, 0 genera and 0 field genera for it, and it does not appear in the ENIGMA-isolate deliverable. ₁ The reported 9 callables therefore comprise 8 ENIGMA-isolate calls and 1 measured-fitness call. The Abscisic acid isolate call rested on a single 2-strain call. ₁

The isolate-callable compounds were concentrated in Shikimates/Phenylpropanoids: salicylic acid, 3-hydroxybenzoic acid, 4-hydroxybenzaldehyde, phthalic acid, and terephthalic acid were in that class; Phenylethylamine and xanthine were in Alkaloids; and Abscisic acid was in Terpenoids. ₁ The class-count test gave χ²=5.07, p=0.53, df=6, with Shikimates 5/25, Alkaloids 2/26, Terpenoids 1/17, Fatty acids 0/10, Polyketides 0/2, AA/Peptides 0/2, and mixed 0/1. ₁ Adding measured-fitness lauric acid did not change the verdict. ₁

The result suggests the hypothesis that compounds in better-represented chemical classes are more likely to acquire callable utilization evidence, but the census did not formally support that hypothesis. ₁ The null result does not establish equal callability across classes because the callable sample was small and confounded with annotation coverage. ₁ It is directionally useful for enrichment design but underpowered and confounded with annotation coverage. ₁

Callable compounds had median Complexity 133 versus 207 for dark compounds, with p=0.034; median MolecularWeight 152 versus 179, with p=0.066; and median HeavyAtomCount 11 versus 13, with p=0.057. ₁ They also showed directionally higher polarity, with TPSA 57 versus 41 and hydrogen-bond donors 2 versus 1. ₁ These comparisons were descriptive rather than calibrated inference because only n=9 callable compounds were available and the Mann–Whitney p-values were uncorrected. ₁ The report reads the Complexity difference (uncorrected Mann–Whitney p=0.034) as at least partly an annotation-coverage ceiling, because curated catabolism resources mainly cover simple, pollutant-adjacent aromatics. ₁

The pattern supports the hypothesis that simpler, more familiar, or more database-represented compounds are easier to connect to utilization evidence, but it does not distinguish biological bioavailability from an annotation-coverage ceiling. ₁ Simple, common, pollutant-adjacent aromatics may be overrepresented among callable compounds because they are also more likely to appear in KEGG, ModelSEED, and genome-depot annotations. ₁ This supports environmental-resistome and structural-annotation-gap by showing that computational visibility may reflect schema and annotation coverage rather than biological importance. ₁

Source contrasts and evidence tiers

The dark fraction was similar for the two source groups: 53/59 groundwater compounds, or 90%, and 21/24 necromass compounds, or 88%, were organism-dark. ₁ Thus, source labels did not supply balanced compound-level evidence for testing whether groundwater and necromass compounds differed in organismal utilization architecture. ₁

The imbalance was sharper for source-tracking hypothesis H3: only 2 of the 8 ENIGMA-isolate-callable compounds were necromass-sourced, and both were phthalate-class aromatics with Actinomycetota-heavy utilizers. ₁ Lauric acid was necromass-sourced, but its call came only from measured fitness in a reference bacterium, with no ENIGMA-isolate rows and no field data. ₁ Consequently, the project reported an SSO field-occurrence atlas rather than a statistical groundwater-versus-necromass source contrast. ₁

This is an evidence-coverage limitation, not evidence that the two sources have equivalent biology. ₁ The available callable set was too small and compositionally confounded to separate source effects from compound-class and annotation-coverage effects. ₁ A source contrast is testable only when both source groups have sufficient, comparable calls for the same type of biological evidence. ₁ Otherwise, an observed difference can reflect resource visibility, chemical-class composition, or assay availability rather than ecology. ₁

The census therefore supports the conclusion that H3 was untestable and confounded, rather than supporting or rejecting a groundwater-versus-necromass utilization difference. ₁ H4 illustrates the same principle: it was strongly supported for the 8 ENIGMA-isolate-callable compounds but null for the other 75 compounds lacking isolate-level placement. ₁ The null result for those 75 compounds is a missing-evidence result, not a biological finding that they lack phylogenetically structured utilizers. ₁ The report also states that lauric acid, the 9th callable, sits outside both H4 arms. It qualifies only on measured RB-TnSeq fitness in a reference bacterium and contributes no ENIGMA-isolate phylogenetic prediction. This leaves a framing tension in the report’s split between 8 strongly supported compounds and an ‘other 75’ null group. The page records that tension as reported and does not reconcile the counts. ₁

The ecotype reanalysis supports the broader distinction between a measured null contrast and a callability failure, but also refines the role of sampling composition. Among 224 species selected with at least 20 genomes containing AlphaEarth embeddings and at least 30% coverage, 106 species (47%) were majority human-associated, 47 (21%) were majority environmental, and 71 (32%) were Mixed/Other. ₂ Environmental species had a median partial correlation of 0.051, compared with 0.084 for human-associated species and 0.109 for Mixed/Other species; the one-sided Environmental > Human-associated test gave U=1536 and p=0.83. ₂ The continuous environmental-fraction analysis likewise found rho=-0.085 and p=0.25, while the human-associated-fraction analysis gave rho=0.030 and p=0.69. ₂

Thus, the clinical sampling bias was real but did not explain the weak environment–gene-content relationship. ₂ This contrasts with any interpretation that unequal clinical representation alone accounts for the null ecological signal: within one consistent reanalysis methodology, environmental species did not show stronger correlations. ₂ However, 10/47 environmental species (21%) had NaN correlations, compared with 13/66 Mixed/Other species (20%) and 7/100 human-associated species (7%); the environmental group was therefore more heavily filtered, so the result remains a callability-sensitive comparison rather than proof that all ecological contrasts are equally observable. ₂

The report itself flags this differential exclusion. Of the 30 species with NaN partial correlations, the NaN rate was highest for Environmental (10/47 = 21%) and Mixed/Other (13/66 = 20%) and lowest for Human-associated (7/100 = 7%). ₂ The authors argue that losing more environmental species would, if anything, bias toward a stronger environmental signal, which is the opposite of what was observed. On that reasoning, addressing the bias would only strengthen the null result. That bias direction is the authors’ interpretation; it was not directly tested or corrected. ₂ The NaN denominators (47, 66, 100) also differ from the group counts reported for the 224 selected species (47, 71, 106), and the report does not reconcile them. ₂

Operational definitions can change the result

Xanthine exposed a category error in the representation-to-function pipeline. ₁ Allowlisted reaction R02107, xanthine→urate, was treated as carbon-catabolic even though it represents purine nitrogen acquisition rather than carbon catabolism. ₁ Removing R02107 from the carbon allowlist reduced the effective carbon-callable set from 9 to 8, although committed tables still contained xanthine because they were not regenerated. ₁

This supports pathway-versus-reaction-evidence-resolution and composite-functional-annotation: chemical identity and database presence require directional reaction validation before establishing biological utilization. ₁ The dark boundary is filter-dependent: the 8 ENIGMA-isolate calls used a genome-prevalence-<10% signature-reaction filter retaining reactions assigned to KEGG degradation maps or a 3-reaction curated allowlist, whereas lauric acid was independently callable through measured fitness and was not subject to this filter. ₁ A different reaction filter could change which compounds are considered callable or dark. ₁

The ecotype reanalysis further refines this methodological point: its all-genome extraction produced a median partial correlation of 0.081 across 183 species, compared with 0.003 in the original downsampled analysis, described as a 27x difference. ₂ The report states that absolute correlations are not comparable across those methods, while the within-method Environmental-versus-Human-associated comparison remains valid. ₂ This shows that evidence callability and extraction design can alter the apparent strength of an ecological contrast even when the contrast itself is retained. ₂

Why environmental occurrence cannot repair missing catabolic evidence

The environmental atlas covered 86 implicated genera using 3825 taxonomy-bearing NMDC metagenomes and 302 Planet Microbe marine runs. ₁ In NMDC, 83/86 genera were detected in 1719 metagenomes, and 99% of samples were labeled through two independent ontology systems. ₁ The SSO field atlas detected 62 implicated utilizer genera at genus resolution, with top field prevalences approximately 0.7–0.9; 3-hydroxybenzoic-acid utilizers reached 0.90 field prevalence. ₁

These observations provide organismal abundance or occurrence, not evidence that the organisms catabolize the census compounds, because no environmental dataset measured those compounds. ₁ The atlas therefore cannot substitute for missing utilization calls when the question concerns source-specific degradation capacity. ₁ Broad occurrence, including strong periphyton representation of Burkholderiales/Comamonadaceae, cannot establish compound degradation or activity. ₁ This refines ecotype-environment-gene-content and supports environment-embedding-geography by separating environmental presence from evidence for the focal metabolic phenotype. ₁

Periphyton-sited enrichment is nevertheless a practical sampling hypothesis: listed genera reached approximately 96–97% prevalence in periphyton, with mean relative abundance approximately 0.005–0.009. ₁ These data can prioritize candidate source communities, but environmental occurrence alone cannot rescue a missing compound-to-pathway link or validate utilization of a particular dark compound. ₁

The ecotype reanalysis supports this separation of occurrence from functional relevance: environmental embeddings had a 3.4x geographic-signal ratio versus 2.0x for human-associated embeddings, yet the environmental signal did not predict stronger gene-content correlations. ₂ The report offers, as a hypothesis rather than an established mechanism, that embeddings may capture regional epidemiological structure in human-associated species rather than ecological differences. ₂

From resource-defined darkness to experiments

The 74 dark compounds define a ranked experimental target space rather than merely missing annotations. ₁ Wet-lab enrichment was recommended to begin with the 29 fully orphan compounds, especially necromass-heavy alkaloids and terpenoids, while the 6 biosynthesis-known compounds should first receive MIBiG or biosynthetic-literature consultation. ₁

Class-level catabolic literature exists for compounds including monoterpenes and nicotine, while the project’s zero literature rescues resulted from a shallow PubMed-title-only screen. ₁ Targeted PaperBLAST or abstract-level PubMed mining could therefore reclassify part of the dark set without new wet-lab observation, but this remains a hypothesis-generating route rather than evidence that the compounds are already biologically characterized. ₁

The census connects compound identity, reaction annotation, measured fitness, taxonomy, and environmental occurrence across multiple resources in a tiered workflow. ₁ This supports evidence-triangulation-for-functional-annotation and reinforces evidence-triangulation-for-functional-annotation and metabolic-model-gapfilling as routes for converting resource-defined gaps into testable hypotheses. ₁

Callability limits beyond the carbon census

The gap between a measured null and an untestable contrast recurs wherever per-genome or per-sample metadata are sparse. In the AMR (antimicrobial resistance) strain-variation analysis, 52.7% of genomes had no classifiable isolation_source. After strict expected-frequency criteria were applied, only 2 species had enough within-species environmental diversity for chi-squared testing of environment–ecotype associations. ₃ Coverage also depended on the classifier: the NCBI keyword classifier assigned environments to 91% of species (1,190/1,307), whereas the BacDive approximation classified 35% (459/1,307). ₃ This supports the core idea and extends cultivation-collection-bias-in-ecological-genomics: environment labels can be widely available at species level while genome-level environmental contrasts remain largely uncallable. ₃

Missing metadata fields can make a hypothesis untestable regardless of sample size. NMDC does not populate fields for DNA-extraction kit, size fractionation/filtration, host-depletion method or library preparation, and the processing boolean has_filtration was all false. The strongest literature-motivated wet-lab levers (host depletion and extraction kit) therefore remained an unmeasured residual in the eukaryotic-admixture analysis. ₄ Testing those factors would require user-supplied per-study protocols or another collection that records extraction kit, host-depletion method and size fraction. ₄

The Harvard Forest warming analysis shows the same limit within a single designed study. Its 42 biosamples spanned treatment (control vs heated) × horizon (organic vs mineral) × incubation (direct vs lab-incubated). The DNA cohort was n=28 and the RNA cohort n=39 because the underlying NMDC pipeline did not produce metagenomes from organic-direct or some mineral-direct samples, which drives an H1 sensitivity caveat. ₅ An NMDC parsing artifact left abiotic_features all zeros for these samples. No in-lakehouse soil temperature, pH or nitrogen measurements were available, so the +5°C treatment label was the only environmental contrast. ₅ Sampling at a single timepoint, 2017-05-24, also precluded detection of seasonal effects. ₅

Assay-compendium coverage imposes the same limit on fitness-based contrasts. The Caulobacter RB-TnSeq compendium in the Fitness Browser (orgId Caulo, 198 experiments) contained zero pure iron-limitation experiments. The iron-limitation arm of the Fur-release analysis therefore could not be tested, and H2 was descoped to the envelope-stress axis only under the plan’s preflight rule. ₆ This is a missing-condition result, not evidence that Fur-released genes lack iron-limitation phenotypes, and it supports fitness-condition-coverage-prioritization-bias. ₆

Detection asymmetry can make even the direction of a change uncomputable. In the same Caulobacter project, MsbA-like CCNA_00307 was detected only in the rescued strain (abundance 300 in 4672, NaN in the other strains), so its direction relative to wild type could not be computed. ₆ In a metabolomics overlap analysis, three amino-acid pathways (cys, his and lys) remained untestable because the relevant compounds were not detected in the 175-sample overlap cohort. ₇

In the IBD phage-targeting project, Klebsiella oxytoca fell below the prevalence filter and was not tested. ₈ The project reported an iron/heme association of OR=8.1 for E. coli, but metabolomics did not measure this axis because LC-MS undersamples siderophores. ₈ The m/z-only bridge also recovered none of the 9 polyamines behind the HMP2 OR=14.6 theme enrichment. Putrescine (89.11), spermidine (146.17) and spermine (203.22) are low-mass HILIC-positive compounds that the bridge did not match. The polyamine enrichment therefore stands as a single-cohort finding rather than a failed biological replication. ₈

Design-level data availability also bounded the IBD tests. H3c was only partially supported. Iron+genotoxin BGCs (biosynthetic gene clusters) were unique to E. coli within the actionable Tier-A core (1 of 6). However, the strict species × BGC interaction-term test was not run, because species-stratified per-sample BGC abundance was not in the pre-computed mart slice. ₈ Of the six actionable Tier-A core species, only Flavonifractor plautii is in the 59-species Kumbhari panel. The H3b test therefore mainly evaluates strain-adaptation inference methodology on commensals and directly tests one actionable Tier-A species. ₈ The planned NB10b cross-cohort sensitivity test was not executed because raw reads were unavailable. ₈

Annotation-pipeline coverage can create an apparent extra class of isolates. The project profiled 643 annotated PROTECT Pseudomonas aeruginosa genomes for 32 virulence genes. Of these, 304 (47%) were PAO1-like (ExoS+), 48 (7%) were PA14-like (ExoU+), and 291 (45%) were intermediate because they lacked clear T3SS (type III secretion system) annotation in GenomeDepot. Among classifiable isolates, 86% were PAO1-like. ₉ A separate count over 651 PROTECT genomes likewise reported 47% exoS, 7% exoU and 45% unannotated for T3SS effectors, which the report attributes to a GenomeDepot gene-name coverage gap. ₉ Gene names from GenomeDepot and bakta/eggNOG were unreliable across databases, so the cross-database phylogeny used Pfam domain IDs. The two databases shared 2,850 domains, with ~3,200 unique domains per PA genome. ₉ The intermediate group should therefore be read as an annotation-callability class, not a demonstrated biological strain type. ₉

The choice of annotation substrate can make a hypothesis structurally impossible rather than null. In the gene-function ecological agora project, the bakta annotation substrate had zero PSII (photosystem II) Pfam hits, so the Cyanobacteria PSII hypothesis test could not have been run on it. ₁₀ At Cyanobacteriota phylum rank, 21 KOs (KEGG orthologs) gave a producer effect of d=+1.60 with p=7×10⁻⁶. The consumer effect was NaN and its p-value n/a because no consumer null was available at phylum rank. ₁₀ Composition-based donor inference was deferred because the per-CDS sequence data needed for codon-usage and k-mer analyses were not in the KBase Data Lakehouse queryable schemas. ₁₀

Cross-resource joins limit which comparisons exist at all. In the FW300 metabolic-consistency analysis, only 21/58 Web of Microbes metabolites (36%) could be tested against any other database, and only 3 metabolites achieved four-way coverage. The untested 64% may harbor additional discordances. ₁₁ In the CF formulation project, the fact_pairwise_interaction table proved identical to fact_carbon_utilization (correlation = 1.0). Per-substrate co-culture effects therefore could not be assessed from endpoint OD (optical density, a growth readout) data, and the interaction analysis relied solely on the RFU-based (relative fluorescence units) competition assay. ₉

Trait coverage can collapse onto lineage. Direct Fitness Browser–BacDive matching by taxonomy ID yielded 12 organisms representing 6 unique species. All Gram-typed organisms were Gram-negative and all urease-typed organisms were urease-negative. The single anaerobe (Methanococcus maripaludis) had 1 metal tested versus 4–5 for aerobes, which at n = 1 is not interpretable. ₁₂ The 12-organism direct validation was therefore underpowered and could not test the Gram-stain association within the set. ₁₂ More broadly, the data contained no Gram-positive Proteobacteria and no Gram-negative Actinomycetes. In a predictive model, the Gram-negative metal-tolerance advantage was therefore indistinguishable from being a Proteobacterium. ₁₂ This supports phenotype-database-coverage-bias and phylogenetic-confounding-of-pangenome-associations. ₁₂

Sampling design can likewise remove one side of a contrast. The Bacillota_B anchor cohort was borehole/porewater-dominated by construction, so it cannot test whether rock-attached Bacillota_B differ in gene content. ₁₃ In the SSO ASV (amplicon sequence variant) ecology project, groundwater ASV data covered only 5 of 9 wells (L7, L9, M4, M6, U2). The missing wells were M5, the denitrification hotspot, and U3, the plume entry. ₁₄ Across these projects, unequal evidence availability was reported as an untestable, descoped or single-cohort result rather than as a null biological finding. This refines null-results-under-limited-statistical-resolution. _3,6,8,13,14

Non-identifiable designs and stratum-level callability

Some contrasts fail not from sparse labels but from a design in which no stratum carries both arms. curatedMetagenomicData pools samples across ≈ 51 source studies; in the ecotype-assigned slice of 8,489 samples, 45 sub-studies have ≥ 10 HC (healthy control) samples and 5 have ≥ 10 CD (Crohn’s disease) samples, but zero sub-studies contain both. ₈ A pooled CD-versus-HC LME (linear mixed-effects) model with a sub-study random effect is therefore structurally unidentifiable, because the random effect perfectly predicts diagnosis, and the project verified empirically that statsmodels.mixedlm silently fails to converge on every battery species under this design. ₈ This extends the core idea from missing measurements to missing design overlap and supports study-batch-confounding-of-environmental-associations: the obstacle is the confounded design rather than the sample count. ₈

The confound-free alternative the project adopts is a within-IBD-sub-study CD-versus-nonIBD contrast, for which 4 cMD studies carry both groups. ₈ Stratifying that contrast by ecotype before meta-analysis tests whether the canonical CD signature differs by ecotype and produces ecotype-specific target lists that are structurally free of leakage, which supports selection-on-outcome-leakage. ₈ Three (sub-study × ecotype) cells met the ≥ 10 CD and ≥ 10 nonIBD eligibility bar: E1 HallAB_2017 with 67 CD and 41 nonIBD, E1 NielsenHB_2014 with 15 CD and 239 nonIBD, and E3 HallAB_2017 with 22 CD and 31 nonIBD. ₈

Callability then differs sharply by stratum. E0 and E2 are the healthy-cohort ecotypes and no IBD-study nonIBD samples live in them, so E1 is meta-viable across two sub-studies while E3 is single-study-only (HallAB_2017). ₈ For E0 and E2 the confound-free CD-versus-nonIBD contrast is not computable at all, because no IBD sub-study in cMD populates them with nonIBD controls. ₈ The report therefore proposes conditionally that if UC Davis has E0 patients, described as 27 % of the cohort, their Tier-A evidence must be drawn from cross-ecotype cohort-level evidence (the 5 engraftment-confirmed pathobionts) or from an ecotype-agnostic within-sub-study CD-versus-nonIBD contrast; this is a stated fallback, not a tested result. ₈

Single-study callability is reported as provisional rather than replicated. The E3 list should be treated as provisional until a second cMD-IBD sub-study that populates E3 with ≥ 10 CD and ≥ 10 nonIBD samples becomes available. ₈ NB04e’s E3 list of 40 candidates derives from HallAB_2017 only, and the HMP2 projection places only 10 of 130 subjects in E3 versus 106 in E1, so HMP2 cannot provide a second-sub-study E3 Tier-A meta-analysis. ₈ The design is written to fail gracefully when (sub-study × ecotype × diagnosis) cells are too small, reporting explicitly which ecotypes are meta-viable, single-study-only or not viable; this refines null-results-under-limited-statistical-resolution by making the viability class itself part of the reported result. ₈

Power is also unequal across strata within one cohort. E1 has 231 CD samples while E0, E2 and E3 have 12-30, which limits within-ecotype detection in the smaller ecotypes: an E0 Gokushovirus signal with Cliff’s δ (a nonparametric effect size) of -0.55 is called biologically interesting but does not pass strict FDR < 0.10 (false discovery rate, the expected share of false positives among calls) because of power. ₈ The extended classifier has only 3 E2 and 11 E0 disease-versus-HC training samples and is effectively an E1-versus-E3 binary in practice. ₈

Coverage is modality-specific, so one cohort supports different contrasts in different data types. The pathway notebook’s primary contrast was a within-IBD-sub-study CD-versus-nonIBD meta-analysis on fact_pathway_abundance, built from HUMAnN3 MetaCyc functional profiles and covering CMD_IBD only. ₈ Because that table is CMD_IBD only, with no HC pathway coverage, the pathway refit is within-disease only. ₈ Re-verifying sub-study meta-viability for the pathway modality gave 3 robust sub-studies (HallAB_2017, IjazUZ_2017, NielsenHB_2014) plus 1 boundary case (LiJ_2014, nonIBD = 10), with VilaAV_2018 excluded (CD = 216, nonIBD = 0), and 575 unstratified MetaCyc pathways reduced to 409 after a 10 %-prevalence filter; the report notes this is not the “4 meta-viable” framing inherited from the taxonomic-modality counts. ₈

Two IBD tests were dropped rather than reported as nulls. The cMD raw-reads-based strain-resolution test was judged not feasible, and the no-raw-reads substitute is a genome-content survey of pks, yersiniabactin and enterobactin co-occurrence cross-referenced with strain-frequency data from the Kumbhari competition table. ₈ This refines the page’s earlier note on the unexecuted H3c test: the analysis that was run uses a precomputed CD-versus-HC main-effect enrichment table, which is consistent with H3c but cannot formally distinguish a species × BGC interaction from a species main effect plus a BGC main effect, and the strict interaction-term test was dropped as a structural project-scope limitation rather than a deferred follow-up because it would require species-stratified per-sample BGC abundance from raw outputs. ₈

Metabolite-class bridging shows the same partial coverage. The polyamine theme could not bridge by m/z because of a low-mass HILIC-positive (hydrophilic interaction chromatography, positive-ion mode) gap, and secondary bile acids bridged at 40 %, where cohort-aggregate differential abundance is weaker evidence than the paired sample-level stream. ₈ This refines cross-cohort-microbiome-portability: a theme that cannot be matched across platforms is unreplicated, not refuted. ₈

Resource absence versus biological absence in phage targeting

Absence from a resource is not biological absence. For Eggerthella lenta the compilation found PMBT5 siphovirus literature but no PhageFoundry collection entry and no detectable in-vivo correlation; for Hungatella hathewayi it found no known phages, no collection entry and no detectable in-vivo phage correlation. ₈ The report treats agreement among curated precedent, PhageFoundry susceptibility and endogenous Podoviridae/Myoviridae correlation as a three-layer convergence signal, so that where one layer shows phage availability the others tend to show it and where one shows a gap the others confirm the gap. ₈ Read as callability, however, collection absence and correlation non-detection do not establish that no phage exists, which supports phage-therapy-evidence-translation. ₈

Underpowered is not null: the archaeal metal test

A sharper version of this page’s claim is that a severely underpowered test should be reported as a coverage statement rather than a null. Only 48 archaeal genera had both MicrobeAtlas representation (≥ 3 OTUs, operational taxonomic units) and AMR data in the pangenome resource. ₁₅ The archaeal PGLS (phylogenetic generalized least squares, regression that corrects for shared ancestry) was run and yielded non-significant results in the expected positive direction for metal types, β = +0.0145, p = 0.467, with standard errors greater than 1× the coefficient — consistent with severe underpowering rather than a true null effect. ₁₅ The report states that these results are not included as evidence for or against the hypothesis but only to quantify the statistical gap between current data and what would be needed, with current power 11 % at α = 0.05. ₁₅

All three archaeal predictors were non-significant at n = 48: metal types (z) β = +0.0145, SE = 0.0198, t = 0.73, p = 0.467, λ = 0.567; AMR clusters (z) β = +0.0146, SE = 0.0209, t = 0.70, p = 0.488, λ = 0.567; and core fraction (z) β = +0.0002, SE = 0.0211, t = 0.01, p = 0.991, λ = 0.553. ₁₅

The metal-types β of +0.0145 is in the same positive direction as the bacterial +0.0215, but its SE is nearly as large as the estimate (t = 0.73), which the report reads as severe underpowering. ₁₅ The 48 archaeal genera are also biased toward cultured lineages — methanogens, halophiles, thermoacidophiles — and miss environmentally dominant archaea in MicrobeAtlas such as Thaumarchaeota and Woesearchaeota, so the gap is compositional as well as numeric; this supports cultivation-collection-bias-in-ecological-genomics and genomic-under-representation. ₁₅

The project quantified what closing the gap would take, conditional on its assumptions. Using analytical power from a non-central t-distribution with SE scaling as SE_obs × √(n_obs/n), β_true = +0.0145 and λ fixed at 0.567, expected power at α = 0.05 is 11 % at n = 48 (current), 18 % at n = 100, 32 % at n = 200, 45 % at n = 300, 56 % at n = 400, 74 % at n = 600, 85 % at n = 800 and 92 % at n = 1,000, with the corresponding Bonferroni α = 0.0083 series 3 %, 6 %, 12 %, 21 %, 30 %, 48 %, 64 % and 76 %. ₁₅ Reaching 80 % power requires n ≥ 702 at α = 0.05 or n ≥ 1,084 under Bonferroni, and the report states that the archaeal analysis is not a negative result but a severely underpowered one whose non-significance should not be read as evidence against the association. ₁₅ A supplementary sensitivity table records the same archaeal PGLS row — n = 48, β = +0.0145, SE = 0.0198, p = 0.467, BH-FDR q = 0.548 — as showing no significant effect; it is a sensitivity entry rather than a hypothesis verdict, and the report’s caveat stands that the result is not evidence for or against the hypothesis. ₁₅ These results support environmental-resistome only for bacteria; the archaeal arm remains a coverage gap. ₁₅

Niche-breadth metrics carry the same limitation in a different form: Levins’ B_std (a standardized niche-breadth index) corrects for unequal environment sizes but cannot correct for missing environments or for differences in detection probability. ₁₅ A breadth value is therefore conditional on which environments were sampled and detectable, which refines this page’s distinction between an observed null and an uncallable comparison. ₁₅

One-sided measurement coverage in community metabolic tests

Paired-data coverage can silently reduce a multi-ecosystem test to a single ecosystem. The full NMDC cohort of 220 samples comprised 126 Soil, 33 Freshwater and 61 with unrecorded ecosystem type; all 33 Freshwater samples lacked paired metabolomics data in NMDC and were excluded from H1 testing, though they were retained for the H2 principal-component analysis using pathway completeness alone. ₁₆ H1 is therefore effectively a soil-only test, and whether BQH (Black Queen Hypothesis, in which lineages lose costly functions and come to depend on community-supplied metabolites) dynamics operate in freshwater communities at the same scale is untested rather than negative. ₁₆

Per-pathway coverage set the testable set within the soil arm. Glutamine (n = 4) and proline (n = 9) could not be tested due to insufficient metabolomics coverage, despite their biological importance. ₁₆ Of 15 amino-acid pathways, 13 were testable and 2 were skipped on those grounds, with Spearman r, p and BH-FDR (Benjamini-Hochberg false-discovery-rate) q recorded per pathway. ₁₆

Covariate callability limited inference as well: abiotic features including pH, temperature and total organic carbon were absent (all NaN) for the 174-sample analysis matrix, preventing partial-correlation tests that control for environmental gradients, and the report notes these variables could confound both H1 and H2 results. ₁₆ This refines community-metabolic-interdependence as a correlational result whose environmental confounders were not testable in this cohort. ₁₆

Detection floors, prevalence filters, and query artifacts

A taxon can be present in a survey yet not callable for correlation. Desulfovibrio, the primary ENIGMA model organism, was detected at only 34 % of sites and at very low abundance, with a maximum of 0.09 % relative abundance. ₁₇ The report states explicitly that this rarity makes correlation analysis unreliable, so the organism’s field associations are detection-limited rather than measured absences of association; this refines lab-field-fitness-concordance by showing that the field side of a lab-field comparison can be the uncallable one. ₁₇

Prevalence filters remove taxa from the comparison entirely: Shewanella, Dechlorosoma and Marinobacter were excluded from the correlation analysis because they occurred at fewer than 10 sites. ₁₇

Not every uncallable comparison is a resource limit; some are query artifacts that can be repaired. An initial marine-abundance contrast in the carbon census showed 0/68 genera present in Planet Microbe, not because they were absent but because the genus-name filter matched only genus-rank reference rows instead of aggregating their constituent species. ₁₈ This supports taxonomic-resolution-dependent-functional-inference and adds a diagnostic obligation: an apparent total absence should be checked against the join’s taxonomic rank before it is reported as a coverage boundary. ₁₈

Reference-resource coverage bounds within-species tests

Phylogenetic tests are bounded by which species exist in the reference tree. Of 65 plant-associated species with ≥ 20 genomes, only 18 (28 %) have any phylogenetic-tree distance data in the pangenome resource’s tree-distance pairs. ₁₉ The other 47 are absent from the tree, including major plant-associated taxa Bradyrhizobium japonicum, B. diazoefficiens, Mesorhizobium ciceri, Sinorhizobium medicae, Burkholderia glumae, Methylobacterium extorquens, Streptomyces scabiei, Xylella taiwanensis and Clavibacter michiganensis — a database-coverage limitation the project documented in its pitfalls notes. ₁₉ The report states that expanding tree coverage would directly increase the power of any within-species phylogenetic analysis, so results such as those in genome-wide-versus-locus-specific-ecological-adaptation hold only for the covered subset. ₁₉

A planned test can also fail its own prerequisite. C-score analysis for PGP-pathogen exclusion was not feasible because only 0 PGP-dominant and 2 pathogen-dominant genera were represented in NMDC co-occurrence data, the remaining 67 being dual-nature. ₁₉ A Phase 2b re-test on refined cohorts gave 0 PGP-dominant, 3 pathogen-dominant and 66 dual-or-mixed of 69 NMDC co-occurring genera, the same pattern as Phase 1. ₁₉ Because the dual-nature class swamps the others whether the Phase 1 25-marker or the Phase 2 17-marker panel is used, the report concludes the C-score test is structurally underpowered in this dataset rather than a marker-panel artifact, and documents the plan’s verification target of ≥ 5 PGP-only genera as NOT MET with the failure-as-result interpretation made explicit. ₁₉

The same project reports a host × subclade mapping output with 0 rows because the data were insufficient — an explicit empty-output record rather than a negative mapping. ₁₉

Single-gene assay coverage

Gene-level predictions inherit the coverage of the assay tables. NDH-2 has no growth data, so the central prediction that NDH-2 compensates on glucose cannot be directly tested with that dataset; this supports genetic-perturbation-coverage-bias. ₂₀

Organism coverage can confine a phenotype question to a single background. ManXYZ is present only in E. coli K-12 (Keio collection) among the 48 Fitness Browser organisms, full fitness data are presented for 168 conditions there, and no Klebsiella strain in the Fitness Browser carries ManXYZ by gene name. ₂₁ PhageFoundry does carry ManYZ annotations in Klebsiella — 28 ManZ proteins and 4,619 PTS_EIIC Pfam hits — but it is GenomeDepot format with genome annotations only and no fitness or TnSeq tables, so Klebsiella-specific ManXYZ fitness is unmeasured rather than shown to differ from E. coli. ₂₁

Tensions

The census showed a directional concentration of callable compounds in simpler, commonly represented chemical classes, but H1 was not formally supported by the chi-square test. ₁ This tension should remain an annotation-coverage hypothesis rather than being resolved as either a true chemical-class gradient or its absence. ₁

The principal tension is between biological darkness and database darkness. ₁ The 29 fully orphan compounds were not linkable through the queried resources, but “organism-dark” does not mean unknown to science; the literature screen was shallow. ₁ The result therefore supports a resource-limited discovery frontier while leaving open how much of the 74-compound dark set reflects absent biology, absent annotations, absent literature retrieval, or an overly restrictive reaction filter. ₁

A second tension is between environmental presence and functional evidence. ₁ The atlas provides occurrence and abundance data for implicated genera, including the 86-genera, 3825-NMDC-metagenome, and 302-Planet-Microbe-run scope, but not compound-resolved catabolic activity. ₁ Environmental occurrence can prioritize enrichment sources without establishing that observed organisms consume a particular dark compound. ₁

A third tension concerns environmental callability versus the interpretation of null ecological contrasts. The ecotype reanalysis found no stronger environment–gene-content association among environmental species despite confirming clinical sampling bias, with U=1536 and p=0.83 for Environmental > Human-associated. ₂ Yet environmental species also had a higher NaN rate, 21%, than human-associated species, 7%, and the report notes that the extraction methodology produced a 27x difference in overall median correlation relative to the original analysis. ₂ The result supports the within-method null comparison but leaves unresolved how much ecological signal is lost through species-level filtering, genome-count imbalance, and extraction choices. ₂

Open Directions

• Apply targeted PaperBLAST and abstract-level PubMed mining to the 74 organism-dark compounds, especially the 29 fully orphan compounds, asking how many are reclassified beyond the title-only screen. ₁
• Apply MIBiG and biosynthetic-literature searches to Tyramine, guanidineacetic acid, cinnamic acid, caffeic acid, palmitic acid, and farnesol, asking whether biosynthetic evidence distinguishes catabolism-unknown from genuinely unlinked compounds. ₁
• Re-run the census with complementary annotation pipelines and explicit provenance tracking, asking whether the 29 fully orphan compounds remain chemically unrepresented or become linkable through alternative databases. ₁
• Regenerate committed tables after removing R02107 from the carbon allowlist, asking whether xanthine changes callable-set composition or downstream class comparisons. ₁
• Recompute callable status under alternative catabolic-direction filters and compare dark sets, asking how much the 74-compound frontier depends on reaction-selection rules. ₁
• Compare callable and dark sets with calibrated, multiple-testing-corrected physicochemical models using a larger compound collection, asking whether Complexity, MolecularWeight, HeavyAtomCount, TPSA, and hydrogen-bond donors predict annotation status independently of chemical class. ₁
• Test the 74 dark compounds in targeted enrichment and measured-fitness experiments, beginning with the 29 fully orphan and 6 biosynthesis-known/catabolism-unknown compounds, asking which resource-defined gaps correspond to realized utilization. ₁
• Re-run the groundwater-versus-necromass comparison after promoting dark compounds through targeted searches, matching compounds by chemical class and evidence tier to ask whether the source contrast remains. ₁
• Combine periphyton enrichment with compound-resolved growth assays, genome sequencing, pathway reconstruction, and metatranscriptomic or metaproteomic assays, asking whether the Burkholderiales/Comamonadaceae reservoir actually transforms prioritized compounds. ₁
• Apply study-aware mixed models or sample-level permutations to NMDC and SSO metadata, accounting for study structure, compositionality, zero-inflated abundance data, and sample-level environment labels, asking whether source-associated occurrence differences persist. ₁
• Integrate chemical identity, reaction evidence, measured fitness, taxonomy, and environmental metadata through provenance-aware joins to ask which missing evidence layer most often blocks conversion from dark to callable status. ₁
• Recompute the ecotype environment-versus-gene-content comparison with matched genome counts and explicit NaN diagnostics, asking whether unequal species-level callability changes the Environmental-versus-Human-associated contrast. ₂
• Directly compare full-genome extraction with diversity-maximizing downsampling under identical species, distance, and correlation procedures, asking what produces the 27x difference in median partial correlation. ₂
• Repeat the ecotype analysis using functional subsets such as transport and secondary-metabolism genes, structured ENVO terms, and genome count as a covariate, asking whether whole-genome distances mask environment-specific functional signals. ₂
• Directly test the authors’ stated direction of NaN-exclusion bias, for example by recovering NaN species with alternative correlation estimators. The question is whether the Environmental-versus-Human-associated null holds once the 30 excluded species are included. ₂
• Re-run the AMR environment–ecotype tests using NCBI keyword classifier labels at genome level, asking whether more than 2 species become chi-squared-testable. ₃
• Add pure iron-limitation RB-TnSeq experiments for Caulobacter, asking whether the descoped H2 iron arm shows phenotypes for Fur-released genes. ₆
• Repair the NMDC abiotic_features parsing for the Harvard Forest samples, asking whether soil temperature, pH or nitrogen differ beyond the +5°C treatment label. ₅
• Acquire per-study extraction-kit, host-depletion and size-fraction protocols, asking whether these wet-lab levers explain eukaryotic admixture beyond the factors NMDC records. ₄
• Assign T3SS status to the 291 intermediate PROTECT genomes using Pfam-domain profiles, asking whether the PAO1-like majority persists once annotation gaps are closed. ₉
• Obtain species-stratified per-sample BGC abundance from raw functional-profiling outputs as new work outside the completed project’s scope, since the strict species × BGC interaction test was dropped there as a structural scope limitation rather than a deferred follow-up, asking whether such a test supports H3c. ₈
• Add a second cMD-IBD sub-study that populates E3 with ≥ 10 CD and ≥ 10 nonIBD samples, asking whether the 40 HallAB_2017-only E3 candidates replicate outside one study. ₈
• Find or assemble a cohort in which single sub-studies contain both CD and HC samples, asking whether the pooled CD-versus-HC comparison that was structurally unidentifiable in cMD becomes estimable. ₈
• Extend the archaeal MicrobeAtlas-AMR join toward the n ≥ 702 the power analysis requires for 80 % power at α = 0.05, including uncultured lineages such as Thaumarchaeota and Woesearchaeota, asking whether the +0.0145 metal-type direction becomes resolvable. ₁₅
• Pair the 33 Freshwater NMDC samples with metabolomics, asking whether the soil-only H1 correlation extends to freshwater communities. ₁₆
• Recover abiotic features for the 174-sample NMDC matrix, asking whether partial correlations controlling for pH, temperature and total organic carbon preserve the H1 and H2 results. ₁₆
• Populate phylogenetic-tree distance pairs for the 47 plant-associated species currently absent from the tree, asking whether within-species phylogenetic tests change once coverage rises above 18/65. ₁₉
• Re-run genus-level occurrence contrasts with species-aggregating taxonomy joins, asking how many apparent absences like the 0/68 Planet Microbe result are query artifacts rather than coverage limits. ₁₈
• Acquire Klebsiella fitness or TnSeq data for ManYZ-carrying strains, asking whether E. coli K-12 ManXYZ phenotypes transfer across genera. ₂₁
• Measure growth phenotypes for NDH-2 perturbations, asking whether NDH-2 compensates on glucose as the respiratory-wiring prediction states. ₂₀
• Re-test the field uranium associations of low-abundance taxa such as Desulfovibrio with a deeper or targeted assay, asking whether the 34 %-of-sites detection floor, not the ecology, produced the unreliable correlation. ₁₇