Schema-Level Bridges and Validated Cross-Tenant Data Integration

Scope

Cross-tenant integration has four evidence states: shared schema keys, overlapping values, an executable join, and a scientifically interpretable result. Schema-level bridges indicate structural compatibility, not valid identifier overlap, biological equivalence, or documented use. ₁ A shared column or declared join key may differ in meaning, format, provenance, namespace, or value space; a syntactically executable SQL join is therefore only a validation starting point. ₁

The BERDL Data Atlas — Inventory, Topic Map, and Cross-Reference Synergies inventories 1,740 deduplicated tables across 119 databases, 17 tenants, and 10 funding agencies or programs, identifying 536 unordered cross-tenant bridges at tenant-by-topic-cell granularity through 29 canonical keys. ₁ The keys span genome, taxonomy, sample, annotation, pathway, biochemistry, protein, phage, literature, and workspace relationships. sample_id occurs across 10 tenants; genome_id and ncbi_taxon_id across 9; feature_id across 9; and ec_number across 8. ₁ Some bridges share up to 7 keys, including kbase.pathway–kescience.pathway, kescience.pathway–phagefoundry.mobile_phage, and refdata.structural–kescience.structural. ₁ These counts describe potential connectivity, not demonstrated value overlap or realized analytical use.

The pitfalls refines this framework into provenance, identifier integrity, value-space compatibility, and evidence or study-design validity checks. KBase Data Lakehouse’s Delta-to-Iceberg migration means live discovery should prefer catalog.namespace.table addresses such as kbase.ke_pangenome.genome, fall back to underscore-form addresses only when necessary, and inspect live schemas rather than archived SQL. ₂ In a data_lakehouse_ingest config, the tenant field is a governance-group name rather than a database prefix; confusing tenant and dataset names can cause access failures or wrong database paths. ₂

The NMDC Context Audit supports the provenance requirement: 20 NMDC-named entries resolve to 7 maintained resources across three tenants and six provenance classes, including genuine NMDC resources, NCBI and Pfam re-hosts, Arkin and NMDC-derived KBase products, and a NEON namesake collision. ₃ The nmdc tenant does not contain every NMDC-related resource. ₂ A resource name, tenant, shared column, or successful query is therefore only an entry point to validation.

The Web of Microbes Data Explorer supports the same framework through a metabolite-to-fitness bridge across Web of Microbes (WoM), Fitness Browser, ModelSEED, GapMind, and pangenome resources. The 2018 WoM snapshot contains 37 organisms and 589 metabolites, but absent consumption data, ambiguous compound matching, and limited organism coverage constrain interpretation. ₄ Cross-collection reachability does not establish that a metabolite is consumed, that a compound identifier is unique, or that a metabolite-to-gene inference is biologically testable.

The Prophage Gene Modules and Terminase-Defined Lineages Across Bacterial Phylogeny and Environmental Gradients supports extending validation from schema access to biological evidence through a join of eggNOG-annotated gene modules, environmental and phylogenetic metadata, TerL sequences, contig localization, and NMDC metagenomic samples. The analysis covered 27,702 bacterial species, 4,005,537 gene clusters, and 6,365 NMDC samples, but NMDC prophage burden was inferred indirectly from taxonomy and was not independently validated for prophage genes. ₅ A technically executable pangenome-to-NMDC bridge can therefore require qualified interpretation.

The Co-fitness and Co-inheritance in Bacterial Pangenomes supports extending validation to biological value space: Fitness Browser co-fitness, KBase pangenome presence/absence, phylogenetic distances, independent component analysis (ICA; a decomposition of multivariate covariance) modules, and SEED annotations were joined and tested. Pairwise co-fitness produced delta = +0.003 with Mann-Whitney p=1.66e-29, whereas coordinated ICA modules produced delta phi=+0.053. ₆ The Ecotype Reanalysis — Environmental vs Human-Associated Species likewise integrated environment classifications, AlphaEarth embeddings, ANI distances, and gene-cluster memberships; environmental species did not have stronger partial correlations than human-associated species (U=1536, p=0.83). ₇ A validated join can establish a defensible null result.

The ENIGMA Carbon Census — Tiered Knowledge Census of 83 Enrichment Compounds supports this value-space criterion with a tiered compound-to-environment workflow: 83 compounds were structure-resolved, 54 linked to KEGG, 9 callable through ENIGMA-isolate predictions or measured fitness, and 74/83 (89%) remained organism-dark. ₈ The Metadata Correlates of Eukaryotic Contamination in NMDC Prokaryote-Targeted Metagenomes similarly linked 99%+ of classified runs and analyzed 2,759 workflow runs at run level, avoiding pseudo-replication from 1,067 pooled runs; environmental associations were largely study-confounded, while within-study vegetation and geography remained predictive. ₉

The Field vs Lab Gene Importance in *Desulfovibrio vulgaris* Hildenborough refines the framework by showing that tenant access is not organism-level coverage: ENIGMA CORAL contained 47 tables, 6,705 genomes, 15,015 genes, 4,346 field samples across 596 Oak Ridge locations, and 213,044 ASVs, but no Desulfovibrio vulgaris Hildenborough gene-level fitness data. Its single TnSeq library was FW300-N2E2 (Pseudomonas), while DubSeq covered E. coli, P. putida, and B. thetaiotaomicron. ₁₀

The Metabolic Consistency of Pseudomonas FW300-N2E3 Across Four KBase Data Lakehouse Databases supports organism-level four-resource validation: Web of Microbes exometabolomics, Fitness Browser gene fitness, BacDive utilization phenotypes, and GapMind predictions were joined for Pseudomonas FW300-N2E3. Of 21 testable metabolites, 17/21 (81%) were fully concordant, 4/21 (19%) partially concordant, none fully discordant, and mean concordance was 0.94; 37 metabolites (64%) were WoM-only. ₁₁ The WoM exploration refines this bridge by identifying 19 WoM-produced metabolites for pseudo3_N2E3 also present in Fitness Browser carbon- or nitrogen-source experiments: 5 were de novo products and 14 amplified metabolites. ₄ This is an actionable metabolite-to-gene path, but absent organism consumption actions prevent the stronger inference that production predicts utilization or gene essentiality. ₄

The Genotype × Condition to Phenotype Prediction from ENIGMA Growth Curves refines the framework by aligning growth curves, Genome Depot, RB-TnSeq (random barcode transposon sequencing), WoM, Carbon Source Phenotypes, pangenomes, and global 16S into 46,389 genome × condition pairs across 727 genomes and 363 conditions, including 4,293 shared kegg orthologs. Binary capability was predicted, but continuous kinetics were not under genus-blocked holdout. ₁₂

The Metagenome-Prioritized Phage Cocktails for Crohn’s Disease and IBD supports value-space validation across 8,489 curatedMetagenomicData samples, HMP2 and FRANZOSA metabolomics, PhageFoundry susceptibility pairs, viromics, and 23 UC Davis patient profiles. Its dominant taxonomy–metabolite canonical correlation had r = 0.964, but clinical efficacy and treatment rules remained hypotheses requiring prospective validation. ₁₃

Join-Key Semantics and Field-Level Traps

The pitfalls digest supports the rule that a shared key is not a shared identity. ENIGMA field isolates often have short strain names (MT20, MT42, GW460-LB6, FW507-14TSA) that are not globally unique. When matched to kbase.ke_pangenome.gtdb_metadata.ncbi_strain_identifiers, these names can hit completely unrelated NCBI organisms. ENIGMA MT20 is Rhodanobacter glycinis (Xanthomonadales, groundwater), but GTDB’s MT20 is Streptococcus pneumoniae (Lactobacillales, clinical), with 8,434 genomes, 1,751 of them clinical. The false match contaminated genus–environment profiles with spurious clinical data. In this linkage set, 12 of 32 pangenome linkages via ncbi_strain_identifiers were incorrect genus matches. The recommended fix is to cross-check the source-database genus (e.g. enigma.genome_depot_enigma.browser_taxon) against the GTDB-matched genus and reject linkages where they disagree. Any project linking ENIGMA CORAL or genome-depot strains to kbase_ke_pangenome should use assembly accession (GCF_*) matching when possible, because it is unambiguous. ₂ These checks support taxonomic-nomenclature-reconciliation.

The Genotype × Condition to Phenotype Prediction from ENIGMA Growth Curves describes the same MT20 linkage failure from the project side and reports its repair. MT20 matching injected 1,751 spurious clinical genomes into environmental profiles. Using CORAL brick 522 (GTDB-Tk assignments) for authoritative taxonomy, together with genus-consistency cross-checks, reduced verified linkages from 32 to 20 but eliminated all false matches. The report states that the pitfall affects any project linking ENIGMA strains to the pangenome by name. ₁₂ The correct bridge is therefore smaller than the naive bridge. Giving up coverage to gain validity is the expected cost of value-space validation, not a defect.

The same project shows how small a fully aligned multi-resource anchor set can be. The ENIGMA growth-curve corpus has 27,632 curves across 123 strains and 195 molecules. It aligns with Fitness Browser RB-TnSeq (random barcode transposon sequencing) data through 486 strain × condition anchor pairs, 7 strains x 72 conditions where growth curves, gene fitness, and genome annotations all coexist. Media names (RCH2_defined_noCarbon, LB, R2A, M9) match exactly between ENIGMA and Fitness Browser, and 42 molecules align by normalized compound name. Of the 486 anchor pairs, 275 (56.6%) show measurable growth, which provides both positive and negative labels for binary prediction. Five conditions (cytidine, glycine, inosine, thymidine, uridine) occur in all four datasets (ENIGMA, Fitness Browser, the carbon source phenotype corpus, and Web of Microbes), and 30 occur in three. ₁₂ This refines the 46,389-pair alignment described above. The broad genome × condition matrix narrows to a much smaller set in which measured fitness and growth coexist.

An unresolved name bridge can also block a planned analysis outright. Per-well uranium, nitrate, and metal concentrations are available in CORAL bricks 10/80, but sample-to-location name resolution is incomplete, so no geochemistry linkage was made. ₁₂ The report proposes linking strain-isolation-well coordinates (already in coral_strain_locations.tsv) to per-well geochemistry. That link would test whether Cluster B (acid-tolerant) strains come from higher-contamination wells locally, not just globally. The test has been proposed but not performed. ₁₂

Cross-cohort metabolite bridges need explicit matching tolerances. The IBD project bridged HMP2 to FRANZOSA_2019 (220 participants × 88 CD + 76 UC + 56 Control). It matched named HMP2 metabolites to Franzosa peaks by m/z (±0.005 Da, same method), which yielded 118 testable matched pairs. ₁₃ This supports cross-cohort-microbiome-portability in a limited sense: cross-cohort metabolite comparison depends on an explicit, tolerance-defined matching step. The matched-pair count describes which metabolite features can be compared. It is not a participant-level statistical denominator.

The Community Metabolic Ecology via NMDC × Pangenome Integration supports the warning that an apparent null overlap can be an artifact of a missing bridge. Its taxonomy-bridge notebook mapped NMDC Centrifuge taxa to GTDB pangenome species and computed per-sample bridge coverage. An earlier sample-overlap figure, generated before the omics_files_table bridge was discovered, showed 0 overlap. It was renamed nmdc_sample_coverage_SUPERSEDED.png and replaced by bridge_quality_distribution.png, which reflects the correct 220-sample cohort. ₁₄ Before a zero-overlap result is reported as a null, it should be checked against alternative bridge tables.

The NMDC audit refines the provenance argument by showing that co-hosting is often intentional and valuable rather than an error. The NCBI mirror exists because the KBase Data Lakehouse adds an attribute-harmonization layer that makes 51.7M raw NCBI samples analytically usable. The Arkin derivative adds embeddings and traits that do not exist upstream. The audit therefore recommends surfacing provenance and value together rather than relabeling or removing resources. ₃

Bridge coverage can be too thin for anything beyond spot checks. In the Gene Function Ecological Agora substrate audit, coverage was measured against 18,989 species. NMDC species-taxid overlap covered 9.4% (1,791 species) at sample level and was treated as supplementary. Fitness Browser BBH (bidirectional best hit) coverage was 0.18% (35 species) at gene-fitness level, which the report rates as suitable for point validation only. genomad_mobile_elements was not ingested, so geNomad mobile-element calls were not available as a bridge. ₁₅ The 0.18% figure limits how far Fitness Browser can arbitrate ecological claims in that project, which bears on lab-field-fitness-concordance.

Domain-annotation tables differ in identifier form and in coverage, so a null from one table is not a null for the domain. The eggnog_mapper_annotations.PFAMs column stores Pfam domain names (e.g. DUF4041, GIY-YIG) rather than accession IDs (e.g. PF13250, PF01541), and searching it by accession returns 0 results. ₂ The gene-function agora project’s v1 control detection relied on that column, which holds names such as HisKA rather than accessions such as PF00512. ₁₅ In the phage-defense project, interproscan_domains (833M rows) was the primary Pfam-based defense-detection table. It is far broader than bakta_pfam_domains (18.8M rows), which misses classical defense markers: CRISPR Cas1 PF01867 returns 0 in the Bakta table but ~25K in InterProScan. interproscan_domains.signature_acc uses version-free Pfam accessions (PF01867), whereas bakta_pfam_domains.pfam_id uses versioned accessions (PF01867.29), so cross-database Pfam joins must strip versions on the Bakta side. ₁₆ These cases support homology-search-negative-evidence: zero hits can reflect identifier format or table scope rather than biological absence.

Fitness Browser joins have specific key and type constraints. There is no single (orgId, locusId) → KO table in kescience_fitnessbrowser. besthitkegg maps each locus to a KEGG organism and gene (keggOrg, keggId) via best hit, and keggmember maps each (keggOrg, keggId) pair to its KEGG ortholog group (kgroup, the KO). Both joins are required. Joins on (orgId, locusId) are composite, because locusId alone collides across organisms. genefitness.fit and .t are stored as strings and must be cast (e.g. CAST(t AS DOUBLE)) before ABS(...) or any comparison. Other numeric columns (fit, t, begin, end, cofit, rank, ratio, protein_length, etc.) are also STRING and need CAST(fit AS FLOAT) or an equivalent cast before numeric comparison or ORDER BY. ₂ This refines the UC1 recipe below, which depends on the same composite key.

String-typed numerics are a lakehouse-wide hazard, not a Fitness Browser quirk. Many databases store numeric values as strings, which must be cast before comparisons. The digest lists kescience_fitnessbrowser (all columns), kbase_genomes (coordinates, lengths), and others as affected. ₂ This supports numeric-type-fidelity-across-engine-boundaries.

Several taxonomy-level joins fail silently. Joining kescience.fitnessbrowser.organism to kbase.ke_pangenome.gtdb_metadata via NCBI taxonomy IDs returns zero matches, because the candidate column holds the boolean string tokens "t" / "f" rather than numeric taxonomy IDs. In metabolic_capability_dependency, the fallback was to match organisms directly by orgId without a clade-level link. The genome and gtdb_taxonomy_r214v1 tables both have a gtdb_taxonomy_id column, but genome.gtdb_taxonomy_id is truncated at genus level while the taxonomy table includes species. Joining on that column returns zero rows, so the join must use genome_id. eggnog_mapper_annotations.query_name joins to gene_cluster.gene_cluster_id, not to gene.gene_id. ₂ Each of these joins executes but yields an empty or wrong result, which supports separating executability from validity.

Tenant and namespace names are not interchangeable. The tenant field in a data_lakehouse_ingest config is the MinIO governance group name, not a database-name prefix; the library auto-prepends the tenant name to the dataset field to form the namespace. The kbase_ke_pangenome database lives under the kbase tenant, not kbase_ke. ₂

ncbi_env is an entity–attribute–value table rather than a flat genome-metadata table. Its columns are accession (BioSample ID), attribute_name, content, display_name, harmonized_name, id, and package_content. Getting genome metadata takes three steps: join genome.ncbi_biosample_id → ncbi_env.accession, filter attribute_name IN ('isolation_source', 'collection_date', 'geo_loc_name', 'host'), and pivot from long to wide format. Analyses should pivot or filter by harmonized_name and must not assume one row per sample. ₂

Genome-level identifiers differ across collections. In kbase_genomes, all primary keys are CDM UUIDs, so an external gene_id (e.g. a pangenome gene_id) must be mapped through the name table (name.name = '<external_id>' → name.entity_id). There is no direct genome_id column: genomes are identified by contig_collection_id and cross-referenced via name or gtdb_taxon_id. Its numeric columns (length, start, end, gc_content, coordinates) are stored as STRING and must be cast before comparing. In GenomeDepot, browser_genome has no species column, so species filtering requires a join through browser_taxon. BacDive sequence_info.accession is GCA_*, whereas pangenome genome_id uses GB_GCA_* or RS_GCF_*, so joins must prefix-match or strip prefixes. ₂ The BacDive prefix mismatch refines the proposed GCA-accession recovery for the BacDive–pangenome bridge described below.

Biochemistry identifiers have their own format and quality rules. ModelSEED IDs use a prefix-colon form (seed.reaction:rxn00001, seed.compound:cpd00001), not bare rxnNNNNN. The deltag field has extreme outlier values and should be filtered with a sanity bound such as deltag > -10000000 before any aggregation. BiGG_Reaction is referenced by eggNOG annotations but is absent from the ModelSEED tables, so there is no biochemistry-side BiGG mapping. ₂

Protein and structure joins need string handling and method-aware filtering. A UniRef100 → UniProt join requires stripping the UniRef100_ prefix (REPLACE(uniref100, 'UniRef100_', '') = uniprot_accession). Because this is a per-row string operation, large joins may justify a materialized mapping table. In PDB tables, r_work and r_free are NULL for non-X-ray methods (EM, NMR) and resolution is NULL for NMR, so method-specific filters are needed before averaging. pdb_uniprot_mapping.pdb_beg / pdb_end are STRING and can literally be the string "None" for unmapped regions, so they must be coerced or filtered before casting. pdb_entries.organism reflects only the first polymer entity, so multi-organism complexes appear as single-organism entries. ₂

PaperBLAST fields carry similar traps. year is a STRING and needs CAST(year AS INT) for comparisons or ORDER BY. protein_length in curatedgene and uniq is a STRING and must be cast for numeric operations. geneId spans multiple namespaces (RefSeq NP_*, WP_*, VIMSS, etc.), so cross-namespace queries must go through seqtoduplicate. ₂

In nmdc_arkin, the abiotic_features numeric measurement columns (e.g. annotations_ammonium_has_numeric_value, annotations_ph, annotations_temp_has_numeric_value) are STRING and must be cast before numeric comparison. Embedding dimensions vary across tables: embeddings_v1 is 256-dim, while the abiotic, biochemical, taxonomy, and trait embeddings differ, so a fixed dimensionality cannot be assumed. ₂

The digest’s phagefoundry_*_genome_browser section lists five databases: phagefoundry_acinetobacter_genome_browser, phagefoundry_klebsiella_genome_browser_genomedepot, phagefoundry_paeruginosa_genome_browser, phagefoundry_pviridiflava_genome_browser, and phagefoundry_strain_modelling. The four genome browsers share the same GenomeDepot schema. phagefoundry_strain_modelling is separate and has an unknown schema. Because these databases have no unified table, cross-species queries must run the same SQL once per database. ₂

Table names can also overstate what a dataset measures. In the PROTECT Gold tables, fact_pairwise_interaction and fact_carbon_utilization contain identical values (correlation = 1.0, mean difference = 0.0). The endpoint OD data therefore do not capture co-culture metabolic interactions; only the RFU-based competition assay provides pairwise interaction effects. ₂ This null constrains competitive-exclusion-consortium-design: a joinable interaction table is not evidence of measured interactions.

The pitfalls digest’s Fitness Browser (kescience_fitnessbrowser) section refines the composite-key recipe above with more table and column traps. Every column is stored as a string, so values must be cast before any numeric comparison. orgId matching is case-sensitive: use WHERE orgId = 'Keio', not 'keio'. keggmember has no orgId or locusId columns, only keggOrg, keggId, and kgroup. The documented gene-to-KEGG query therefore runs from besthitkegg to keggmember on keggOrg and keggId, and can left-join kgroupdesc on kgroup for descriptions. kgroupec uses the column ecnum, not ec. The experiments table is experiment (not exps), and its columns are expName, expGroup, and condition_1 (not name, Group, or Condition_1). expGroup provides clean condition categories (“carbon source”, “nitrogen source”, “stress”, “pH”, “temperature”, etc.). It should be used to classify condition types instead of pattern matching on condition_1. ₂

Annotation tables in the same database are easy to misread. seedannotationtoroles has only seed_desc and seedrole, with no orgId or locusId. The SEED role hierarchy therefore needs a three-table chain: seedannotation (orgId, locusId, seed_desc) → seedannotationtoroles (seed_desc → seedrole) → seedroles (toplevel, category, subsystem, seedrole). Names such as seed_role, seed_subsystem, or seedroleId are not columns. seedclass (orgId, locusId, type, num) stores EC numbers, not SEED subsystem categories; gene-level functional descriptions come from seedannotation. The fitbyexp_* tables are long format (expName, locusId, fit, t), not the pre-pivoted wide tables that the schema docs suggest. Each is equivalent to a per-organism slice of genefitness, so genefitness should be queried directly with an orgId filter. ₂ The fitbyexp_* case supports provenance-aware-resource-discovery: schema documentation can misdescribe a table, so analysts need to inspect the live table. The seedclass case bears on ontology-and-category-schema-sensitivity, because a table whose name suggests one category system holds another. ₂

Locus identifiers can diverge even within one resource. For some organisms, the Fitness Browser protein sequences at fit.genomics.lbl.gov/cgi_data/aaseqs use RefSeq-style locus tags (e.g. ABZR86_RS00005 for Dyella79), while the gene table uses the original annotation tags (N515DRAFT_0001). A DIAMOND database built from aaseqs and joined back to the gene table therefore returned zero matches without raising an error. Dyella79 had 0% overlap, whereas the other organisms had >94% overlap, so locusId overlap should be checked before analysis. ₂ Type mismatches can drop whole organisms without warning. Fitness matrices from fitness_modules/data/matrices/ use integer-typed locusId indices (e.g. 206065), while the Metal Atlas metal_important_genes.csv stores locusIds as strings (e.g. '206065'). An uncast pandas lookup reports such a gene as absent instead of raising an error. This removed DvH (495 metal-important genes) and Btheta (276 genes) from the analysis entirely. The processing loop reported “SKIPPED (no fitness matrix or no non-metal experiments)” even though the matrix existed. Numeric Spark columns can also arrive as strings after .toPandas(). ₂ These failures support numeric-type-fidelity-across-engine-boundaries. They also bear on metal-cross-resistance analyses built on that lookup. ₂

GTDB prefixes cause another family of joins that execute but return nothing. phylogenetic_tree_distance_pairs uses bare NCBI accessions (GCA_001038305.1, GCF_000005845.2), whereas genome, gene, and gene_genecluster_junction use GTDB-prefixed IDs (GB_GCA_001038305.1, RS_GCF_000005845.2). Joining phylogenetic distances with presence matrices therefore produces zero matches. Stripping the GB_ or RS_ prefix (the first 3 characters) restores 100% overlap for all tested species (399/399 for Koxy, 287/287 for Btheta, etc.). ₂ GapMind’s kbase.ke_pangenome.gapmind_pathways likewise stores unprefixed genome IDs (e.g. GCF_001307155.1), while the pangenome genome table uses RS_GCF_001307155.1, so direct equality returns zero rows. The digest recommends a fallback chain: try the original ID first, then the RS_/GB_-stripped ID, then an accession-based partial match. ₂ Integer and string types cause the same failure. BacDive sequence_info.bacdive_id is INT but isolation.bacdive_id is STRING, and Spark does not implicitly cast across INT/STRING types. A direct join therefore returns zero rows, so the join must be written as CAST(si.bacdive_id AS STRING) = iso.bacdive_id. In kbase_genomes, all primary keys are CDM UUIDs, so external gene IDs must be mapped through the name table. ₂ These cases support taxonomic-nomenclature-reconciliation at the identifier-format level and refine the BacDive accession-prefix rule above. ₂

The Plant Microbiome Ecotypes — Compartment-Specific Functional Guilds and Their Genetic Architecture shows the prefix trap reaching a scientific result and then being repaired. Pairwise phylogenetic distances from phylogenetic_tree_distance_pairs (22.6M rows) were used to build distance matrices for multidimensional scaling (MDS) visualization and clustering. In the Phase 2 pass, all 5 species showed 0 testable subclade × plant-association enrichments, because a genome ID mismatch (bare NCBI accessions vs GTDB-prefixed IDs) had dropped the plant-associated subset to zero. That null was a data-plumbing artifact, not biological evidence that subclades do not segregate by plant association. ₁₇ In the Phase 2b revision, prepending GB_/RS_ prefixes to the bare accessions recovered 599 plant-associated genomes across the 5 species. A full 18-species scan then found 5/17 testable species passing Bonferroni-corrected Fisher’s exact tests, with P. avellanae, X. campestris, and X. vasicola the three most robust. An earlier 5-species draft had also flagged P. amygdali on a chi² test alone, but the report rejects that result because it fails Cochran’s rule and Fisher’s 2×2 collapse. ₁₇ The repaired join therefore yields qualified positive results where the broken join yielded none, which supports separating executability from validity. The same project’s Phase 1 NB02 query WHERE pfam_id IN ('PF00771', …) also returned zero rows, because bakta_pfam_domains stores versioned IDs (PF00771.22). This supports homology-search-negative-evidence alongside the version-stripping rule above. ₁₇ Name-based domain lookups have a parallel trap: eggnog_mapper_annotations.PFAMs holds comma-separated lists of domain names, so it must be searched with LIKE and % wildcards, not exact equality. ₂

The same project’s taxonomy bridges were only partial. Its NMDC taxonomy bridge matched 260 of 322 genera (80.7%) to GTDB and lost 62 genera (19.3%). The impact on the H3 complementarity analysis is unknown: if the unmatched genera include key complementary partners, the observed redundancy signal could be artifactual. The report lists the NMDC genus loss as documented and defers the sensitivity analysis to future work. ₁₇ Its MGnify bridge linked 20,473 MGnify species across 4 biomes to 1,185 pangenome genera, but only 9,603 MGnify species fell in matched genera. The per-biome figures describe the composition of the MGnify input, not successfully mapped counts: soil 19,472 species (4,229 genera), tomato rhizosphere 579 species (309 genera), maize rhizosphere 336 species (212 genera), and barley rhizosphere 86 species (52 genera). ₁₇ These genus-level matches support taxonomic-nomenclature-reconciliation as coverage figures, not as species-level identity. ₁₇

The digest supports the genus-level ceiling that the ENIGMA contamination project reports. ENIGMA taxonomy in ddt_brick0000454 currently includes only the Domain through Genus levels. Species-level bridge logic applied to it directly will either fail or silently reuse genus-level mappings while appearing to have higher resolution. One option the digest gives is an explicit species-proxy mode (unique genus → single GTDB clade) that reports the resulting loss of mapped coverage. ₂ This supports taxonomic-resolution-dependent-functional-inference. ₂

NMDC tables in kbase.nmdc_arkin have their own key traps. metabolomics_gold, kraken_gold, centrifuge_gold, and gottcha_gold use file_id and file_name as their primary identifiers, not sample_id. Queries with WHERE sample_id = ... or COUNT(DISTINCT sample_id) throw AnalysisException: UNRESOLVED_COLUMN and stop notebook execution. Joining a classifier table directly to metabolomics_gold on file_id always returns zero rows, because classifier files carry nmdc:dobj-11-* prefixes and metabolomics files carry nmdc:dobj-12-*. The two are different workflow output types for the same biosample and can be linked only through the biosample sample_id (e.g. nmdc:bsm-11-*). The bridge table is kbase.nmdc_arkin.omics_files_table (385,562 rows, confirmed), with file_id, sample_id, study_id, workflow_type, and file_type columns. ₂ This refines the earlier rule that a zero-overlap NMDC result should be checked against alternative bridge tables: a zero-row join here reflects key format, not an absence of shared biosamples. _2,14

taxonomy_features has no sample_id or file_id column. Its columns are numeric NCBI taxon IDs (e.g. 7, 11, 33, 34), and the digest describes its rows only as likely samples. It should not be used for tidy-format joins. The tidy classifier tables (kraken_gold, centrifuge_gold, gottcha_gold, which have file_id, rank, name/label, and abundance) should be used instead, and the row count of taxonomy_features should be checked against the expected number of samples. metabolomics_gold has kegg (string KEGG compound ID), chebi (double ChEBI ID), name, inchi, inchikey, and smiles, and column names containing spaces need backtick quoting. annotation_terms_unified is a gene-annotation lookup (COG/EC/GO/KEGG terms) and cannot serve as a metabolite compound lookup. ₂

In abiotic_features, most numeric columns use a _has_numeric_value suffix (e.g. annotations_tot_org_carb_has_numeric_value), but annotations_ph has no suffix. Using a bare name where the suffix is required raises UNRESOLVED_COLUMN. annotations_water_content does not exist; the digest suggests annotations_diss_org_carb_has_numeric_value and annotations_conduc_has_numeric_value instead. The table stores 0.0 rather than NULL for unmeasured variables, so zeros should be replaced with NaN before analysis. ₂ This refines the casting note above: a correctly cast column can still record a missing measurement as a measured zero. ₂

Query-format mismatches can shrink an NMDC bridge almost to nothing. In the SNIPE Defense System in the KBase Data Lakehouse Pangenome, the NMDC metagenome ecosystem breakdown was limited because only 2 samples matched the API query format. The report suggests that interactive MCP tool queries would improve coverage. ₁₈

The WoM exploration’s own tables refine its bridge claims. The 2018 snapshot encodes observations with a 4-action system whose meanings differ between control and organism entries. Control (“The Environment”) entries are D (detected in starting medium) 742 and N (not detected) 1,023. Organism entries are I (increased) 1,338, E (emerged, i.e. de novo production) 1,155, and N (no significant change) 7,509. The report states that E and I are mutually exclusive, with zero overlap across all 10,744 observations. E means the organism produced a metabolite that was absent from the starting medium, and I means an existing metabolite was amplified. ₄ The per-action table and the 10,744-observation total are reported separately and are not reconciled here. ₄

WoM’s Fitness Browser matches differ in strength. Pseudomonas sp. FW300-N2E3 (pseudo3_N2E3; 5,854 genes, 211 experiments) and Pseudomonas sp. GW456-L13 (pseudo13_GW456_L13; 5,243 genes, 106 experiments) are direct strain matches, and both are ENIGMA groundwater isolates with rich Fitness Browser data. E. coli BW25113 (Keio; 4,610 genes, 168 experiments) is the same strain, but its WoM data are limited to 12 observations in ZMMG medium with a sulfur-metabolism focus. Synechococcus PCC7002 matches SynE (PCC 7942; 2,722 genes, 129 experiments) at the genus level only. ₄ Of 257 identified WoM compounds, 69 (26.8%) have definitive ModelSEED links by exact name match. Another 107 (41.6%) match by formula only, 176 (68.5%) have any link, and 81 (31.5%) are unmatched. Each WoM formula maps to an average of 8.4 ModelSEED molecules; C5H11NO2, for example, matches valine, norvaline, betaine, 5-aminopentanoate, and others. Formula matches are therefore candidate sets for manual curation, not definitive identifications. The project’s output files contain 37 organism, 589 compound, 10 environment, 5 project, 2 direct strain-match, 75 exact-name-match, and 900 formula-match rows. The 75 exact-name rows should not be conflated with the 69 distinct exactly matched compounds. ₄ This identifier ambiguity supports cross-condition-metabolic-comparability. ₄

Not every planned WoM bridge executed. WoM ↔ GapMind matching was blocked by a naming-convention mismatch: GapMind pathway names are internal IDs that do not contain simple metabolite names, so a lookup table mapping GapMind pathways to substrate and product metabolites is needed. The report judges H1 only partially supported, while describing the cross-collection links that did execute as real and actionable. Kosina et al. (2018) introduced WoM with the same data and describe “decrease” as a valid action, so consumption data are part of the WoM schema but absent from this specific export. One test remains future work: for pseudo3_N2E3 and pseudo13_GW456_L13, identify the genes that are fitness-important on carbon sources the same organism produces, which would test whether metabolic output predicts gene essentiality. ₄ The missing action class supports provenance-aware-resource-discovery: it is a property of the snapshot’s provenance, not evidence that the organisms consume nothing. ₄

Potential Connectivity, Documented Use, and Observability

Cross-tenant reuse is common but its measured extent is partly an observability problem: the atlas mined project README files, so mentions located only in research plans, notebook source, query histories, or other artifacts may have been missed. ₁ Consequently, reported tenant breadth and reuse are lower bounds, not complete measurements. The audit found that 51 of 66 BERIL projects, or 77%, spanned multiple tenants; KBase appeared in 53/66 (80%) and KEScience in 35/66 (53%). The KBase × KEScience bridge accounts for 36 projects, mostly pangenome-by-fitness joins through genome_id and ncbi_taxon_id. ₁ These are documented-use counts, not a census of all activity. The atlas presents this as a realized-versus-theoretical overlay that identifies the 51 cross-tenant BERIL projects and the high-leverage bridges that remain unused. Realized tenant reuse is charted in nb03_tenant_frequency.png, and nb04_atlas_composite.png combines it with tenant × topic heat and theory-versus-practice views. ₁ The audit attributes kbase × kescience dominance to three features: both tenants are intra-DOE-BER, both are well documented, and together they cover the most cross-tenant topics (genome × phenotype). This is an explanation the audit offers, not a tested cause. ₁ Its working heuristic is that an analysis crossing topics such as genome × phenotype × environment almost certainly crosses tenants. The canonical bridge is whichever of genome_id, ncbi_taxon_id, sample_id, or feature_id both sides expose, and data/cross_tenant_bridges.csv lists the key sets for each pair. ₁

This observability limitation refines interpretation of the comparison between availability and use, which the atlas summarizes as heavy presence in the lakehouse not implying heavy reuse. KBase appeared in 53/66 projects (80%) and KEScience in 35/66 (53%). By contrast, ENIGMA contained 36% of the KBase Data Lakehouse tables but appeared in 6 projects, PhageFoundry contained 14% of tables but appeared in 5 projects, and PROTECT contained 4% but appeared in 2 projects. ₁ The ratios may reflect genuine differences in research use, incomplete documentation, or both; the audit does not distinguish those explanations. DOE-BER accounted for 63% of the KBase Data Lakehouse tables, so documentation gaps could alter conclusions about tenant centrality as well as bridge activity. ₁

Five high-leverage bridges had zero realized use at audit time: UC1 between KEScience and refdata with 12 shared keys; UC2 between ENIGMA and PhageFoundry with 11; UC3 between KBase and refdata with 11; UC4 between NMDC and PROTECT with 10; and UC5 between NMDC and refdata with 9. ₁ They concern structural fitness signatures; subsurface prophages, metal resistance, and the Oak Ridge contamination gradient; GTDB–KBase species-pangenome disagreement; environmental distributions of clinically relevant pathogens and associated biogeochemistry; and ENVO ontology completeness in NMDC biosamples, respectively. ₁ “Zero realized use” means zero use found in audited documentation, not proof that no researcher used the bridge in unobserved notebooks, plans, or other materials. ₁ The atlas refines the apparent zero-use gap because UC1 was later sample-validated, whereas UC2–UC5 still require live execution. ₁ The ranking covers the bridges with the highest schema-level join surface that no BERIL project had exercised at audit time. Each use case is posed as a question rather than a reported effect: UC2 does not show prophage-mediated metal-resistance mobilization, UC3 does not report GTDB–KBase disagreement or gene flow, UC4 does not report pathogen occurrence, and UC5 does not report an ENVO coverage fraction. ₁ The accompanying data/untapped_bridges.csv lists 20 highest-leverage unrealized bridges with ≥1 shared key and 0 realized projects. The figure nb03_theoretical_vs_realized.png plots shared keys against realized projects for each tenant pair, with untapped bridges in the bottom-right. ₁ The report estimates ~15–30 min of SQL probing per bridge to confirm value-space overlap and surface join-recipe corrections. It proposes UC3 as the next lowest-friction test because both of its tenants are intra-DOE-BER and well documented. ₁ The atlas is a meta-catalog of data systems. Its references are canonical citations for the catalogued systems, not support for the biological hypotheses that UC1–UC5 propose. ₁

The distinction is therefore among schema compatibility, documented project use, technically validated value-space overlap, and scientifically interpretable evidence. ₁ A bridge may be technically possible yet undocumented, documented yet unvalidated, or validated through an executed join. README mining provides a reproducible lower-bound inventory, but not a census of data reuse. ₁ Reliable reuse discovery supports provenance-aware-resource-discovery by requiring records of notebooks, research plans, executed queries, and other artifacts in addition to README descriptions; otherwise tenant-use rankings may be biased toward projects with more complete or standardized documentation. ₁

The atlas supports pangenome-integration by describing KBase computed pangenomes as the canonical genome reference and the cross-program join anchor. kbase_ke_pangenome contains 293,059 genomes, 27,690 GTDB species clades or pangenomes, and 132.5M gene clusters. ₁ Relevant tables are large—gene and gene-genecluster junction approximately 1B rows, genome_ani approximately 421M, eggNOG annotations approximately 93M, InterProScan domains approximately 833M, Bakta cross-references approximately 572M, and GapMind pathways approximately 305M—so Spark-native filtering and aggregation are required; a 1 GB driver cap makes large .toPandas() collections unsafe. ₂

The WoM exploration supports the realized-bridge pattern while refining its expected scope. WoM has two direct Fitness Browser strain matches—Pseudomonas FW300-N2E3 and GW456-L13—and two same-strain or genus-level matches, including E. coli BW25113/Keio and Synechococcus PCC7002/SynE. ₄ For pseudo3_N2E3, 19 produced metabolites matched Fitness Browser carbon- or nitrogen-source experiments, including de novo lactate and valine production and amplified amino acids and nucleotides. ₄ The bridge is actionable for three organisms, but the E. coli WoM record contains only 12 observations focused on sulfur metabolism in ZMMG medium despite the richness of Keio data. ₄

The Co-fitness and Co-inheritance in Bacterial Pangenomes supports a realized pathway across 9 organisms: 2,253,491 cofit pairs versus 22,534,910 prevalence-matched random pairs; 7/9 organisms had positive pairwise effects, but the across-organism Wilcoxon test was not significant (W=9, p=0.13). ICA modules had mean delta phi +0.053, with 21/195 significant at q<0.05; accessory modules had mean delta phi +0.108 versus +0.059 for core modules, with p=0.051. ₆

The ENIGMA census produced 569 isolate-utilizer rows across 8 compounds, placing 494 records representing 359 strains on GTDB taxonomy: 64 high-certainty, 387 medium-certainty Tier-2, and 43 medium-certainty Tier-3. ₈ Its atlas detected 83/86 genera in 1,719 NMDC metagenomes and used 3,825 taxonomy-bearing NMDC metagenomes plus 302 Planet Microbe runs, but measured occurrence rather than catabolic activity. ₈

The eukaryotic-contamination project supports controlled NMDC results-to-metadata joins: native results keyed by data_object_id or workflow_run_id were connected through biosample/workflow and study tables, and approximately 29M-row Kraken2 results were aggregated to one row per workflow run. ₉ Classifier and metabolomics file IDs (nmdc:dobj-11-* and nmdc:dobj-12-*) require a sample_id bridge through omics_files_table, which has 385,562 rows; wide taxonomy_features and zero-coded abiotic_features require reshaping and zero-to-NaN conversion. ₂

The Prophage Gene Modules and Terminase-Defined Lineages Across Bacterial Phylogeny and Environmental Gradients supports this integration pattern while refining its evidential requirements. Across 1,773 species, PERMANOVA (permutational multivariate analysis of variance) on Bray-Curtis module composition found significant genome-size, environment, and family-phylogeny effects (each p=0.01), with F=212.99, 30.04, and 6.17 respectively; genome size correlated with cluster count at rho=0.717. Environment remained significant within every genome-size quartile (all p < 6.5e-78), and AlphaEarth niche breadth correlated with module count after genome-size control (rho=0.468, p=8.41e-110). ₅ However, only 28% of genomes had AlphaEarth embeddings, and environmental categories plus the indirect NMDC taxonomy bridge limit causal interpretation. ₅

The same study found 57 significant module–abiotic correlations in 6,365 NMDC samples at FDR (false discovery rate) < 0.05: packaging–pH rho=0.519, all-modules–pH rho=0.474, all-modules–temperature rho=0.399, all-modules–depth rho=0.361, and all-modules–total-nitrogen rho=0.333. ₅ Concordance between pangenome enrichment and NMDC correlations was strongest for head morphogenesis, tail, and anti-defense, providing independent module-level value-space support but not direct prophage detection or causal validation. ₅

The field-versus-lab study found 2,725 genes with pangenome and fitness links; 76.3% of non-essential genes with fitness were core, while 678 essential genes were 80.1% core but lacked transposon data. Field-stress-important genes were 83.6% core, field-core-important 82.4%, heavy-metal-important 71.2%, and lab-antibiotic-important 73.4%. ₁₀ Field-plus-lab prediction had AUC 0.548 versus 0.645 with gene length, supporting condition-specific-fitness and pangenome-integration while limiting ecological inference. ₁₀

The FW300 bridge found 601 unique genes and 4,764 total significant gene-condition hits using |fit| > 1 and |t| > 4; comparisons were concordant for 21/21 matched metabolites with Fitness Browser and 13/13 with GapMind, but only 3/7 with BacDive. ₁₁ Malate was utilized by 49/49 BacDive strains, arginine by 40/48, and valine by 1/1, illustrating why strain counts must be retained. ₁₁

The ENIGMA LightGBM model achieved binary-growth AUC 0.620 under genus-blocked holdouts; KO × condition interactions raised mean AUC to 0.653 in 80 of 106 held-out genera, while continuous µmax, lag, and max_A had negative R². ₁₂ The IBD bridge likewise requires scope controls: PhageFoundry contained 96 phages, 188 E. coli strains, and 17,672 susceptibility pairs; 3,929 were susceptible and a five-phage cocktail covered 94.7% of tested strains, not UC Davis isolates or in-vivo efficacy. ₁₃

Identifier, Provenance, and Semantic Validation

The atlas refines bridge counts because schema compatibility does not prove value-space overlap. genome_id denotes a UPA in KBase, an accession in NCBI, and a hash in MAG (metagenome-assembled genome) pipelines. Value-space validity was established only for UC1, and UC2–UC5 each require their own first sample-execution before publication. ₁ The pitfalls supports this warning with concrete failures: ENIGMA MT20 was Rhodanobacter glycinis, whereas GTDB MT20 was Streptococcus pneumoniae; the erroneous match involved 8,434 genomes, including 1,751 clinical genomes, and 12 of 32 pangenome linkages through ncbi_strain_identifiers were incorrect genus matches. Genus checks and GCF_* assembly accessions are required. ₂

Pangenome taxonomy must use genome_id, not gtdb_taxonomy_id; GapMind clade_name uses the full gtdb_species_clade_id; species-specific gene-cluster IDs require COG, KEGG ortholog, or Pfam representations; eggnog_mapper_annotations.query_name joins to gene_cluster.gene_cluster_id; and ncbi_env requires entity-attribute-value extraction and pivoting. ₂

The NMDC audit supports the same warning: nmdc.ncbi_biosamples contains 51,711,888 biosamples and 756,112,544 attribute rows, versus 16,640 samples in genuine nmdc.metadata.biosample_set; kbase.nmdc_mags contains 62,346 MAGs. ₃ This is a ~3,000× scale trap. ₃ MetaPhlAn3 synonymy also requires taxid-backed, GTDB-version-aware reconciliation: failures involving Bacteroides vulgatus/Phocaeicola vulgatus, Eubacterium rectale/Agathobacter rectalis, and Ruminococcus gnavus/Mediterraneibacter gnavus produced log₂FC approximately 28, equivalent to approximately 2.68 × 10⁸ fold. ₂

WoM evidence supports compound-level identifier validation. Of 257 identified, non-unknown compounds, 69 (26.8%) had definitive ModelSEED links through exact name matching; 107 (41.6%) had formula-only matches, yielding 176 compounds with any link (68.5%) and leaving 81 unmatched (31.5%). Formula-only matching expanded those 107 compounds to 900 ModelSEED molecules, an average of 8.4 candidates per WoM compound; these are candidate sets for manual curation, not definitive identifications. ₄

The PaperBLAST analysis supports evidence-provenance validation: its 12.4 million rows span 14 tables; Homo sapiens accounts for 46.7% of gene-paper records, the top five organisms 72.8%, and 65.6% of 841K genes with text-mined links have exactly one paper. ₁₉ At 50% sequence identity, 31,653 protein families had zero papers and 159,046 exactly one; 5,218 multi-member families had no literature. ₁₉ PaperBLAST covers 1951–2026, with 845K genes linked to 1.1M papers through 3.2M associations, but 25.6% of genes have no text-mined link. ₁₉

The ENIGMA census supports validating biological direction as well as identifiers: it linked PubChem, KEGG, ModelSEED, Fitness Browser, Genome Depot, GTDB, SSO, NMDC, and Planet Microbe, but 74 compounds were not linkable to organismal utilization determinants. Xanthine’s reaction R02107 represented purine nitrogen acquisition rather than carbon catabolism, reducing the effective carbon-callable set from 9 to 8. ₈

The Regulatory and proteomic architecture of Δfur-permitted lipid A loss in *Caulobacter crescentus* supports this distinction: it bridged Fitness Browser, PaperBLAST, published datasets, transcript measurements, and proteomics, but PaperBLAST returned false negatives for known Caulobacter lipid A genes while NCBI supported absence claims. ₂₀ GOTTCHA2, unlike Kraken2 and Centrifuge, was usable for eukaryotic-fraction estimation because the latter reference databases were prokaryote-restricted. ₉

Database semantics also require validation. WoM uses I for increased, E for emerged, N for no change, and sometimes D for decreased; “produced” means action IN ('I','E'), except that for “The Environment” D means detected. ₂ In the 2018 export, D occurred only for the control, with 742 detected and 1,023 not-detected control observations; organism actions included 1,338 increased, 1,155 emerged, and 7,509 with no significant change across 10,744 observations, with no organism consumption or decreased action recorded. ₄ The discrepancy with the valid “decrease” action described by Kosina et al. (2018) indicates that the archived export may be incomplete or older than a GNPS2 version, rather than proving that consumption is absent from WoM generally. ₄

BacDive utilization is four-valued; for Pseudomonas fluorescens, indole had 60 “produced” entries but 1 actual utilization test, so only explicit + and - observations belong in utilization percentages. ₂ Fitness Browser, GapMind, and Carbon Source Phenotypes are also non-equivalent observables: matched-condition AUCs were 0.800 and 0.646, with approximately 76% of ENIGMA conditions lacking either GapMind coverage or training data. ₁₂

Cross-tenant deduplication is another interpretive constraint: the atlas did not perform it, and notes that Refdata and KBase may contain the same UniProt entries through different cluster indices, while ENIGMA and genome-depot tables share genome records with the ENIGMA SDT layer. ₁ Joined counts must therefore distinguish matched records, distinct biological entities, and duplicated representations.

Validated UC1 Integration

UC1 links FitnessBrowser measurements to AlphaFold models through SwissProt best hits. The validated path uses composite orgId, locusId between genefitness and besthitswissprot, then sprotAccession to uniprot_accession in alphafold_entries; FitnessBrowser does not expose protein_id. ₁ SQL probing corrected the proposed join by identifying this composite route. ₁

Live-cluster sample validation found 27,410,721 measurements, 79,180 genes with SwissProt hits, 241,070,489 AlphaFold entries, and 78,753/79,180 hits represented in AlphaFold, a SwissProt-best-hit coverage of 99.5%. The cohort contains 55,454 genes across 48 organisms and 22,303 models, including 6,635 essential genes (min_fit ≤ −4), 8,271 strong-defect, 10,950 moderate-defect, 29,467 mild-defect, and 131 no-defect genes, tested under an average of 121–187 conditions. ₁ E. coli sample rows confirmed semantic validity: thrA → AF-P00561-F1, thrB → AF-P00547-F1, thrC → AF-P00934-F1, and talB → AF-Q3Z606-F1. The report describes the cohort as strongly enriched for essentiality signal. UC1 is the atlas’s only sample-validated use case with a corrected, executable join recipe and live-cluster coverage statistics. ₁

UC1 supports condition-specific-fitness and gene-essentiality as a sample-validated route from condition-specific fitness to structure for the 55,454-gene cohort, but not residue-level structure-function analysis because per-residue pLDDT and structural features are absent. ₁ Fitness fields may be strings and require casting; orgId is case-sensitive; KO mapping is a two-hop besthitkegg–keggmember join; essential genes are absent from transposon records, so genefitness-only analyses miss approximately 14.3% of protein-coding genes. ₂

Bridge Coverage Rates and Identifier Mapping

The Acinetobacter baylyi ADP1 Data Explorer supports value-space validation with measured match rates for an external Acinetobacter baylyi ADP1 database. All 13 queried genome IDs matched the pangenome (100%). Against kbase_msd_biochemistry, 1,210 of 1,330 reactions (91%) and all 230 compounds (100%) matched; this collection supplied ModelSEED reaction and compound name resolution through its reaction and molecule tables. ₂₁ The 13 genomes belong to the clade s__Acinetobacter_baylyi--RS_GCF_000368685.1, whose pangenome has 3,207 core and 1,684 accessory gene clusters. The 120 unmatched reactions (9%) may be custom or draft reactions not yet in ModelSEED; this is a possibility, not a verified explanation. ₂₁ ADP1 is absent from the Fitness Browser, so the database’s mutant growth data are not available elsewhere in the lakehouse. This bridge therefore adds a phenotype layer rather than duplicating one. ₂₁

The same project refines the rule that a shared concept does not imply a shared key. ADP1 uses mmseqs2-style cluster IDs (NHSXFYEX_mmseqsCluster_NNNN), whereas the KBase pangenome uses centroid gene IDs (NC_005966.1_1024); the two systems had 0% direct string match. ₂₁ The bridge ran through gene_genecluster_junction. That table maps pangenome cluster IDs to member gene IDs, which match feature_id in the ADP1 pan_genome_features table, which in turn carries the ADP1-style cluster_id. Through this mapping, all 4,891 cluster IDs matched (100%). ₂₁

The AlphaFold MSA Depth as a Lens on the Bacterial Annotation Gap shows how far a protein-level bridge reaches across the whole pangenome. Of 132,531,501 gene clusters in kbase_ke_pangenome.gene_cluster, 38,804,903 (29.3%) had a real UniProt accession via bakta_annotations.uniref100, and 38,051,842 (28.7%) bridged to kescience_alphafold.alphafold_msa_depths. ₂₂ The report states that the remaining 70.7% lack UniRef100 IDs or carry UniParc-only (UPI-prefixed) identifiers without an AlphaFold entry, while its limitations section gives 71%. These figures, and the 29.3% it elsewhere calls “AlphaFold bridge coverage”, are not consistent with the stated 28.7% successful MSA-depth bridge rate. The excluded fraction should therefore be cited with its source wording rather than reconciled. ₂₂ By contrast, 111,035,431 gene clusters (83.8% of all clusters) had at least one InterProScan domain annotation. The report calls this substantially higher than the AlphaFold bridge coverage (29.3%), a label that differs from its 28.7% successful MSA-depth bridge rate, and takes it as showing that profile-based domain annotation reaches further into sequence space. It also proposes that the excluded clusters skew toward poorly characterised organisms with a larger annotation gap. That is a hypothesis, not a measured result, and it supports genomic-under-representation only as a hypothesis. ₂₂ This whole-pangenome rate contrasts with UC1’s 99.5% coverage, which applies only to genes that already have a SwissProt best hit. _22,1

The BacDive Phenotype Signatures of Metal Tolerance supports treating name-based joins as partial bridges. Using species-name matching, 37,368 BacDive strains (38.4%) matched both the pangenome and a metal score. ₂₃ The report proposes that GCA accession matching could recover 10–30% more species. That is a prospective estimate rather than a measured gain, and it relates to taxonomic-nomenclature-reconciliation. ₂₃

The discoveries digest records a similar ceiling for annotation cross-references. Only 33.3% of Bakta’s 17.6M distinct UniRef50 IDs exist in kbase_uniprot.uniprot_identifier. Among matched IDs, the bridge reaches RefSeq (71%), OrthoDB (46%), STRING (40%), and KEGG (36%). GO, EC, InterPro, and Pfam are missing from the bridge, and the digest proposes importing full UniProt to unlock them. ₂₄

The digest also supports the WoM–Fitness Browser bridge with exact strain matches: Pseudomonas sp. FW300-N2E3 = pseudo3_N2E3 (5,854 genes, 211 experiments), Pseudomonas sp. GW456-L13 = pseudo13_GW456_L13 (5,243 genes, 106 experiments), and E. coli BW25113 = Keio. The 109 metabolite–condition name overlaps it reports are name-level candidates for metabolite-to-gene-fitness linking, not tested links. ₂₄ The digest reports that 69 of 257 identified WoM compounds match ModelSEED molecules by exact name and 107 by formula, totaling 176 (68.5%). It describes this as enabling reaction-level annotation for most WoM metabolites. The WoM exploration refines that reading, because the 107 formula-only matches expand to 900 candidate molecules that require curation. _24,4

The ENIGMA Carbon Census — Tiered Knowledge Census of 83 Enrichment Compounds refines the denominator problem. Its NMDC abundance denominator is the 3825 covstats files that actually carry taxonomy, not the ~6700 sample_file_lookup rows, because using the lookup count deflates relative abundances by ~1.75×. ₈ Sample-level environment labels came from nmdc_metadata.biosample_set, joined to covstats sample_id via kbase.nmdc_arkin.sample_file_lookup. This gave 99% coverage across two ontologies, versus ~13% from study_table GOLD alone. ₈ Its pangenome-to-ENIGMA taxonomic bridge is genus-level at best. Isolate-specific transfer of pangenome Tier-2 reconstructions was therefore avoided in favor of direct Genome Depot annotation. ₈

The Contamination Gradient vs Functional Potential in ENIGMA Communities supports the same resolution limit and quantifies its coverage cost. Of 1,392 observed ENIGMA genera, 530 mapped to the pangenome bridge and 862 were unmapped. Of the mapped genera, 150 were unique-clade (species-proxy resolvable) and 380 were multi-clade ambiguous. ₂₅ Parsing 27,690 GTDB species rows yielded 530 strict clades, 7,380 relaxed clades, and 150 species-proxy clades. The many-to-many genus-to-clade expansion produced 8,242 bridge rows and 3,630 functional feature rows across three modes: strict_single_clade, relaxed_all_clades, and species_proxy_unique_genus. ₂₅ Mean mapped abundance fraction was 0.343 in strict/relaxed modes (range 0.031 to 0.854) but 0.031 in species-proxy mode (range 0.0002 to 0.543). The finer-resolution mode therefore covers much less of the community, which supports taxonomic-resolution-dependent-functional-inference. ₂₅

The Gene Function Ecological Agora adds a redundancy check for environment metadata. gtdb_metadata.ncbi_isolation_source has 74.6% non-trivial coverage but is redundant with ncbi_env, so it does not add an independent environmental bridge. ₁₅

Documentation can also carry provenance that bridges otherwise lose. The digest describes the UC Davis / Arkin CrohnsPhage data mart as shipping several provenance files: a lineage.yaml (ETL provenance, changelog, known gaps), a schema_overview.yaml, and per-table *.yaml dictionaries (columns, dtypes, null counts, unique counts, sample values). It also has a ref_missing_data_codes table whose sentinel codes, such as PENDING_DAVE_LAB, PENDING_KUEHL, and PENDING_HMP2_RAW, distinguish real NULLs from known-pending data. This supports provenance-aware-resource-discovery; the digest proposes the pattern as a BERIL convention but does not report it as adopted. ₂₄

Tensions

The atlas exposes a tension between 536 schema-level bridges and evidential validation: the original audit recorded five high-leverage bridges with zero realized use, but UC1 was subsequently sample-executed and UC2–UC5 remained untested. ₁ The realized-use count is also a lower bound because README mining may miss plans and notebooks. ₁ Thus “zero documented use” and “zero use” are not equivalent claims.

The NMDC audit contradicts any assumption that a catalog prefix, tenant, or name proves common authority: 20 entries resolve to 7 resources, with kbase.nmdc_neon a NEON namesake collision. ₃

The WoM snapshot introduces a related tension with broad cross-collection integration claims: the 19-metabolite Fitness Browser bridge is directly actionable, but missing consumption actions mean production cannot be treated as utilization, and 107 formula-only ModelSEED matches expand to 900 candidate molecules. ₄ This refines rather than overturns the framework: a join may be technically valid while its biological direction and chemical identity remain uncertain.

The prophage study refines this tension: module-level pangenome and NMDC associations were concordant, but prophage annotations came from eggNOG rather than geNomad or VIBRANT, so near-universal prevalence may include domesticated remnants, bacterial homologs, and integrases; the false-positive rate is uncharacterized. ₅ Its environmental effect exceeded family-level phylogeny in the reported PERMANOVA, but genome size was dominant (rho=0.717), only 28% of genomes had embeddings, and genus-level NMDC inference assumed conserved prophage content. ₅

The study also supports phage-defense-syndromes-and-arms-race while limiting mechanism: human-associated environments enriched tail (log2(OR)=2.21), head morphogenesis (1.98), and anti-defense (1.70), whereas anti-defense was depleted in freshwater (-0.74) and animal-associated environments (-0.24); these are consistent with arms-race theory but do not demonstrate coevolution. ₅ TerL lineages did not show independent enrichment in 0/500 FDR-corrected tests; among 824 lineages with at least 5 species, 325 were specialists and 499 generalists. ₅

Other tensions remain. PaperBLAST’s organism-level Gini coefficient is 0.967 and gene-level Gini 0.669; 9.2% of 50%-identity families have zero papers, 46.1% exactly one, and 4.3% at least 20, contradicting paper count as a uniform evidence proxy. ₁₉ The ecotype analysis found p=0.83 and rho=-0.085, p=0.25, while its median partial correlation was 0.081 across 183 species versus 0.003 originally, a reported 27x difference caused by different sampling. ₇

The co-fitness bridge had weak pairwise but stronger module-level effects; co-fitness strength was anti-correlated with co-occurrence (rho=-0.109, p<1e-300 across 1.04M pairs), requiring auxiliary-only comparisons below 95% prevalence. ₆ The field-versus-lab study likewise found weak condition-only prediction (AUC 0.548; field-only 0.517; lab-only 0.531) and no significant module-conservation correlation with field activity (rho=0.071, p=0.62). ₁₀

Semantic tensions persist: tryptophan increased in WoM and had 231 significant Fitness Browser genes and a complete GapMind pathway, yet 0/50 P. fluorescens strains used it as carbon; production therefore does not imply utilization. ₁₁ Binary growth could be predicted for tryptophan (AUC 0.933), phenylalanine (0.932), and valine (0.927), while continuous phenotypes had negative R². ₁₂

The IBD CCA result (r=0.964) and PhageFoundry coverage are hypothesis-generating rather than clinical validation; HMP2/FRANZOSA clustering had cross-cohort LOSO ARI=0.000, and the cocktail covered only tested strains. ₁₃ In ecotype analyses, pooled diagnosis was structurally unidentifiable because 45 sub-studies had at least 10 HC samples, 5 had at least 10 CD, and 0 had at least 10 of both; selection-on-outcome leakage reduced independent Tier-A candidates from 33 to 3. ₂

Open Directions

• Combine README files, research plans, notebook source, and query logs, then ask how many additional project–tenant and project–bridge uses are recovered beyond the documented baseline of 51 of 66 multi-tenant projects; preserve the original README-derived counts and deduplicate evidence at project–tenant–dataset and project–bridge levels. ₁
• Build a provenance-aware project–dataset graph and audit notebook and query artifacts for the 36 documented kbase × kescience projects, asking whether their pangenome-by-fitness joins are reproducible and whether additional join keys beyond genome_id and ncbi_taxon_id were used. ₁
• Execute UC2–UC5 on the live cluster with provenance, authority, currency, identifier, overlap, representative-record, duplicate, and evidence checks; compare validated value-space overlap with the README-derived finding of zero realized use, asking whether the bridges are unused, undocumented, or merely unvalidated. Use UC2’s 11, UC3’s 11, UC4’s 10, and UC5’s 9 candidate keys to test their proposed prophage, fitness, pathogen, biogeochemical, species-pangenome, and ENVO analyses. ₁
• Add authority, tenant, scale, provenance, and max(committed_at) to inventory; compare nmdc.metadata and nmdc.ncbi_biosamples with upstream counts and repair NMDC schema, skill, documentation, aliases, and user copies. ₃
• Build a live-catalog harness for dotted/underscore namespaces, canonical identifiers, data types, plausible row counts, join overlap, provenance, and biological representative records; use Spark-native workflows, parquet checkpoints, explicit casts, and finalization logs. ₂
• For prophage ecology, validate eggNOG calls with geNomad or VIBRANT, quantify false positives and domesticated remnants, repeat NMDC burden inference against directly detected prophages, and test whether pH and other abiotic correlations persist within study and genus. ₅
• Test whether human-associated tail, head, and anti-defense enrichment reflects phage exposure or counter-defense by integrating direct phage detection, host range, and matched environmental metadata; characterize whether TerL specialists persist after finer taxonomy and sampling controls. ₅
• Combine NMDC metabolomics, proteomics, and lipidomics with UC4/UC5 and prophage-module bridges, preserving the distinction between occurrence, activity, and mechanism. _1,5
• Ingest UC1 residue-level pLDDT and structural features and test them across the 55,454-gene cohort; restrict fitness-to-pangenome analyses to auxiliary-only pairs below 95% prevalence. _1,6
• Link the 129,823 PaperBLAST VIMSS cross-references to Fitness Browser values, stratify by organism and family literature coverage, and distinguish missing PMC coverage from missing biology. ₁₉
• Reproduce ecotype analyses with downsampled and full-genome extraction, genome-count covariates, structured ENVO terms, study-aware validation, and within-substudy contrasts. _7,2
• Apply GTDB-Tk-verified identifiers, quantify the 12 collision-derived mismatches, and retest environmental profiles; compare GTDB and KBase assignments through genome_id and ncbi_taxon_id. _12,1
• Pair environmental occurrence with compound-specific enrichment and measured fitness; expand WoM–BacDive matching with InChIKey or CHEBI identifiers and test tryptophan cross-feeding. _8,11
• Use exact-name ModelSEED links separately from formula-only candidate sets, manually curate the 900 formula-expanded molecules, and quantify how compound ambiguity changes pathway and fitness conclusions. ₄
• Obtain the current GNPS2 or Northen laboratory WoM dataset, determine whether consumption actions are available, and repeat the production–utilization–fitness analysis with versioned provenance. ₄
• Test whether E/(E+I) metabolic novelty associates with pangenome openness or accessory gene content, using strain-to-genome mappings and species-level rather than genus-only joins. ₄
• Apply GroupKFold by study and within-study contrasts to NMDC eukaryotic-fraction models, harmonize classifier databases, and test replication outside NEON soil. ₉
• Extend field-versus-lab analyses to organisms with both environmental and gene-level fitness data; quantify gene-cluster prevalence and acquisition history for metal and antibiotic resistance. ₁₀
• Query INPHARED and IMG/VR for phages targeting H. hathewayi, F. plautii, and M. gnavus, validating host range against patient isolates. ₁₃
• Reproduce the reduced IBD Tier-A list with held-out-feature clustering, leave-one-species-out refitting, study-aware validation, and within-substudy contrasts. ₂
• Extend the validated UC1 cohort with per-residue pLDDT and structural-feature data from PDB files, because kescience_alphafold.alphafold_entries lacks those fields, then test whether structure-derived variables explain condition-specific fitness patterns beyond the existing gene-level join. ₁
• Add GCA accession matching to the BacDive–pangenome species-name bridge and measure whether the proposed 10–30% species recovery materializes beyond the current 38.4% match. ₂₃
• Resolve the 862 unmapped ENIGMA genera, then compare functional conclusions across strict, relaxed, and species-proxy modes to quantify how bridge resolution changes contamination–function inferences. ₂₅
• Report AlphaFold bridge coverage against a single stated denominator, and test whether clusters lacking non-UPI UniProt accessions are enriched in poorly characterised organisms. ₂₂
• Complete sample-to-location name resolution for CORAL bricks 10/80, then link coral_strain_locations.tsv well coordinates to per-well uranium, nitrate, and metal concentrations to test whether acid-tolerant Cluster B strains come from more contaminated wells locally. ₁₂
• Rerun defense-marker and control-domain detection on interproscan_domains with version-stripped Bakta accessions, and quantify how many zero-hit calls from eggnog_mapper_annotations.PFAMs or bakta_pfam_domains were artifacts of identifier format or table scope. _16,2,15
• Run the deferred plant-ecotype sensitivity analysis to test whether the 62 unmatched NMDC genera change the H3 complementarity and redundancy conclusions, and report MGnify mapped counts per biome separately from catalog composition. ₁₇
• Build a GapMind pathway-to-substrate/product metabolite lookup, retry the blocked WoM ↔ GapMind bridge, and then test whether production predicts gene fitness for pseudo3_N2E3 and pseudo13_GW456_L13. ₄
• Audit Fitness Browser-derived analyses for locusId type and tag-format mismatches (e.g. Dyella79 aaseqs tags, integer-versus-string Metal Atlas locusIds), and re-include DvH and Btheta where they were silently skipped. ₂