Data Infrastructure, Provenance, and Research Practice

This topic covers the layer beneath every biological claim in the corpus: how data in the KBase Data Lakehouse is owned, named, addressed, joined, ingested, executed and reviewed. The corpus speaks to it directly because several projects audit the platform itself. One inventories the lakehouse’s tables and cross-tenant links, where a tenant is a governance group that owns a set of databases ₁. Another resolves what resources named “nmdc” contain ₂. A central digest records operational failures ₃, and a cross-project digest summarizes what adversarial review caught ₄. A shared column or an executable join is only a starting point for validation ₁, and committed artifacts do not by themselves establish provenance ₃.

Literature Context

The candidate literature addresses reproducible pipelines, curation and spurious signal in microbiome data. MetaSanity combines analyses from 11 existing genome evaluation and annotation suites into a single distributable workflow and stores all relevant outputs in an SQL database for querying [PMID 32426808]. The abstract describes a software design and reports no benchmark of reproducibility gains. MicrobeTCM used a standardized retrieval strategy that identified 419 experiment articles, then applied double-entry extraction to reduce the uncertainty of natural language extraction [PMID 38272335]. Decontam classifies contaminant sequences using two patterns: they appear at higher frequencies in low-concentration samples, and they are often found in negative controls. In a dilution series it substantially reduced technical variation [PMID 30558668]. STENSL starts from the point that source tracking across large repositories, such as the Earth Microbiome Project with over 200,000 samples from more than 50 types of environments, needs source selection. Without it, noncontributing sources add considerable error [PMID 36047699].

A second strand documents gaps in reference databases and unstable taxonomy. In the pig gut catalog, 86% (2309) of 2673 species-level genome bins were unknown based on current databases, and 28% of genes were unknown proteins [PMID 33597514]. A global urban atlas of 4,728 samples from 60 cities reported 10,928 viruses, 1,302 bacteria, 2 archaea and 838,532 CRISPR arrays not found in reference databases [PMID 34043940]. Genome-based revision of the Geodermatophilaceae found that Blastococcus, Geodermatophilus and Modestobacter were paraphyletic. It proposed new genera and reclassified species, for example Modestobacter deserti as Goekera deserti comb. nov. [PMID 36439792]. The same study found in silico chemotaxonomy overall consistent with wet-lab results, although discriminatory levels varied by trait [PMID 36439792].

This corpus is broadly consistent with that literature and extends it from sequences to identities and resources.

• Spurious signal. Decontam assumes that artifacts can masquerade as biology [PMID 30558668]. The corpus’s strain-name collision, which matched ENIGMA MT20 to Streptococcus pneumoniae and injected 1,751 spurious clinical genomes, is an analogous failure. Its source is a join key, not contaminating DNA.
• Nuisance sources. STENSL’s concern with noncontributing sources [PMID 36047699] parallels the NMDC scale trap, where 16,640 genuine biosamples sit beside a 51,711,888-sample NCBI mirror.
• Reference gaps. These gaps [PMID 33597514, PMID 34043940] fit the corpus’s organism-dark result, in which 74/83 (89%) of enrichment compounds could not be called.
• Taxonomic renames. Genus-level reclassification [PMID 36439792] shows that taxonomy keeps changing. taxonomic-nomenclature-reconciliation documents one downstream hazard of such change: a naive pivot across legacy and modern names gave log₂FC (base-two logarithm of fold change) ≈ 28 for every renamed species.

The corpus adds an execution-side caveat that MetaSanity’s abstract does not address [PMID 32426808]. In one documented case, a notebook run exited 0 while leaving zero cell outputs. Some committed outputs also had no notebook history. Each incident is documented once, so its frequency is unknown (analysis-provenance-and-reproducible-outputs). Double-entry extraction [PMID 38272335] resembles independent adversarial review. However, the corpus’s contrast of “no critical issues” against 5 critical and 6 important issues rests on a single case. No candidate addresses silent table skips during ingestion or citations fabricated by reviewers.

What the Corpus Shows

Inventory breadth is not realized integration. The data-atlas project counts 1,740 deduplicated tables across 119 databases, 17 tenants, and 10 funding agencies or programs, with 536 unordered cross-tenant bridges through 29 canonical keys ₁. These counts describe potential connectivity; only UC1 of the five proposed bridge use cases was sample-validated ₁. DOE accounts for approximately 78% of tables, and reuse does not follow volume: ENIGMA holds 36% of tables but appears in only 6 audited projects, while 51 of 66 projects (77%) span multiple tenants ₁. data-landscape-ownership-and-coverage-bias argues that apparent topic availability can reflect the location and funding history of data rather than a balanced distribution of observations ₁. Single-organism checks show that inventory breadth does not guarantee coverage of a specific organism _5,6,7. ENIGMA CORAL contains no Desulfovibrio vulgaris Hildenborough gene-level fitness data ₇, and E. coli K-12 had 0 predictions from GapMind, a pathway-completeness predictor ₆.

(Tenant, topic) linkage graph — edges = ≥1 shared canonical join key. From berdl_data_atlas.

Names and addresses do not encode provenance. The 20 database names containing “nmdc” (for NMDC, the DOE-BER National Microbiome Data Collaborative) resolve to 7 real, maintained resources in six provenance classes, including NCBI and Pfam re-hosts and a namesake collision with NEON, the NSF National Ecological Observatory Network ₂. The genuine NMDC biosample set holds 16,640 samples, while the co-hosted NCBI mirror holds 51,711,888, a ~3,000× scale trap ₂. provenance-aware-resource-discovery therefore argues that discovery should expose provenance, authority, scale, tenant placement and currency together _2,3. Addresses are also changing: after the Delta-to-Iceberg migration, what is now kbase.ke_pangenome.genome was previously kbase_ke_pangenome.genome, so queries copied from archived projects can fail, yet a blanket underscore-to-dot rewrite is unsafe because migration remains incomplete ₃.

A shared key is not a shared identity. cross-tenant-data-bridging separates four evidence states: shared schema keys, overlapping values, an executable join, and an interpretable result ₁. Its sharpest case is strain names. Linking by ncbi_strain_identifiers gave 12 of 32 incorrect genus matches; ENIGMA MT20 (Rhodanobacter glycinis) matched Streptococcus pneumoniae and injected 1,751 spurious clinical genomes _3,8. The repair used GTDB-Tk (Genome Taxonomy Database toolkit) assignments plus genus cross-checks, reducing verified linkages from 32 to 20 but eliminating all false matches ₈. taxonomic-nomenclature-reconciliation extends this logic to taxonomic renames: when cohorts used legacy and modern names, a naive pivot gave log₂FC (base-two logarithm of fold change) ≈ 28 for every renamed species ₃. The phage-targeting project built a synonymy layer mapping 2,417 aliases to 1,848 canonical species; its value as a general convention is proposed from one project, not tested across others ₉.

Validated joins yield bounded results, including nulls and explicit gaps. multi-omics-integration shows what careful matching produces. For Pseudomonas FW300-N2E3, a four-resource join found 17/21 (81%) testable metabolites fully concordant, but only 21/58 WoM (Web of Microbes) metabolites were testable ₁₀. In the ENIGMA carbon census, only 9 of 83 enrichment compounds were callable through ENIGMA-isolate predictions or measured fitness, and 74/83 (89%) remained organism-dark, so the result is a mapped knowledge gap ₁₁. In the ecotype reanalysis, environmental species did not have stronger partial correlations than human-associated species (U=1536, p=0.83), a defensible null ₁₂. The genotype-to-phenotype project’s 46,389 genome × condition pairs narrow to 486 strain × condition anchor pairs in which growth curves, RB-TnSeq (random barcode transposon sequencing) fitness and annotations all coexist ₈.

Ingestion and execution can fail silently. foreign-dump-ingestion-fidelity documents MySQL dumps with no header row and NULL encoded as literal \N ₃. In one case parse_sql_schema() read a leading inline comment as a column name, producing an invalid schema, and the ingest pipeline then silently skipped the whole table with no error or quarantine entry ₃. Permissive SQLite decoding completes an export but replaces invalid bytes with U+FFFD ₃. analysis-provenance-and-reproducible-outputs adds two execution-side failures. On this JupyterHub, jupyter nbconvert ... --inplace can exit 0 while leaving zero cell outputs; separately, NB08/NB09/NB10 existed only as committed outputs, with no notebook history ₃. Each failure is documented once, so its frequency across projects is unknown ₃.

Adversarial review is a distinct quality-assurance layer. adversarial-research-quality-assurance reports that routine review of one project’s notebooks found “no critical issues”, while adversarial review of the same files found 5 critical and 6 important issues _4,9. The digest and the project record are two accounts of this single case, not a replication _4,9. The adversarial pass caught feature leakage (samples clustered on taxon abundances, then the same taxa tested within cluster), missing null distributions and hard-coded verdict logic ₄. After repair, a permutation null built by randomizing ecotype labels over 200 permutations gave a mean Jaccard (set-overlap) similarity of 0.785 ± 0.054 against an observed Jaccard of 0.104, empirical p = 0.000; the conclusion survived, but the original statistic had been the wrong one _4,9. Reviewers can themselves fabricate citations ₁₃: the functional-atlas project documented a Mendoza 2020 PMID hijack at REVIEW_5, while REVIEW_8 had 0 fabrications and 7/7 DOIs auto-verified ₁₃. The same class of repair took 7 notebooks when caught at notebook scope, but a single plan-edit pass when caught at plan scope ₄.

Tensions and Caveats

Several reports state figures in their prose that disagree with their own tables. Because this wiki copies numbers exactly, any single figure carried forward takes one side (conflicts/conflict—adversarial-research-quality-assurance—a6ef7613) _3,9,14. A related case is the E. coli × phage matrix. It is stated as a 96-phage × 188-strain × 17,672-pair inventory, but one project uses a 94-phage denominator elsewhere (conflicts/conflict—adversarial-research-quality-assurance—eca086c5) _9,14.

On bridging, three disputes remain open:

• Module aggregation. The co-fitness project reports stronger module-level effects, while the field-versus-lab project finds no significant link between module conservation and field activity (conflicts/conflict—cross-tenant-data-bridging—8325c1f4) _15,7.
• Tryptophan. Tryptophan had 231 significant Fitness Browser genes and a complete GapMind pathway, yet 0/50 P. fluorescens strains used it as a carbon source ₁₀. Elsewhere, binary growth on tryptophan was predicted with AUC (area under the receiver operating characteristic curve, measuring separation of growers from non-growers) 0.933 ₈. The endpoints differ, and whether these findings truly conflict is itself open (conflicts/conflict—cross-tenant-data-bridging—d89b8efa).
• IBD contrasts. One strand reports a strong inflammatory bowel disease association that its own project labels hypothesis-generating rather than clinical validation ₉; another reports that pooled diagnosis in the ecotype analyses was structurally unidentifiable ₃. Whether cross-tenant IBD bridges therefore face a maturity problem or a design problem is open (conflicts/conflict—cross-tenant-data-bridging—34478c5a).

On comparability, literature attention is so unevenly spread that paper count cannot serve as a uniform evidence proxy ₁₆; separately, an effect-size summary moved by a large factor between analyses because sampling differed, while its hypothesis tests stayed null ₁₂. These remain separate sides whose relationship is unstated (conflicts/conflict—cross-tenant-data-bridging—dd5b9877) _16,12.

On provenance, the atlas’s “NMDC integrated” tag is read by the audit as provenance blur (conflicts/conflict—provenance-aware-resource-discovery—0fe66702, conflicts/conflict—data-landscape-ownership-and-coverage-bias—4b4b2ec9) _1,2. If that reading holds, agency-level shares may reflect naming conventions rather than origin, which remains a hypothesis _2,1. Three further caveats apply to discovery:

• Metadata counts succeeded through one access surface but are unreliable over REST, the web-service interface (conflicts/conflict—provenance-aware-resource-discovery—f0c8f1e3) _2,3.
• “UniRef” names both partial kbase_uniref* samples and dated refdata_uniref*_2026_01 collections (conflicts/conflict—provenance-aware-resource-discovery—f19db99c) _3,1.
• The audit’s provenance classes are inferences, because no ingestion manifest is exposed in-catalog ₂.

On taxonomy, two bridges from BacDive (strain-phenotype database) to pangenomes (species-level gene collections pooled across genomes) report 43.4% versus 38.4% strain-to-pangenome coverage, with the cause unsettled (conflicts/conflict—taxonomic-nomenclature-reconciliation—2208cd83) _17,18. Stripping GTDB suffixes may merge clades that GTDB keeps separate (conflicts/conflict—taxonomic-nomenclature-reconciliation—a209158c) _17,19.

Where to Go Deeper

• cross-tenant-data-bridging — the four-state validation framework and field-level join traps _1,3.
• taxonomic-nomenclature-reconciliation — the MT20 collision, GTDB renames and accession-format mismatches that can masquerade as nulls _3,8,20.
• provenance-aware-resource-discovery — what discovery must expose, with per-collection scale counts _2,1.
• data-landscape-ownership-and-coverage-bias — how ownership shapes coverage and reuse ₁.
• adversarial-research-quality-assurance — recurring failure classes and plan-scope review ₄.
• analysis-provenance-and-reproducible-outputs — a reproducibility standard drawn from two incidents ₃.
• foreign-dump-ingestion-fidelity — silent table skips and encoding trade-offs ₃.
• multi-omics-integration — what validated cross-modal joins concluded _10,11.

Key entities: kbase-ke-pangenome, gtdb, kescience-fitnessbrowser, bacdive, gapmind.

Reports: BERDL Data Atlas — Inventory, Topic Map, and Cross-Reference Synergies, NMDC Context Audit, Genotype × Condition to Phenotype Prediction from ENIGMA Growth Curves, Metagenome-Prioritized Phage Cocktails for Crohn’s Disease and IBD, Gene Function Ecological Agora, Metabolic Consistency of Pseudomonas FW300-N2E3 Across Four KBase Data Lakehouse Databases.