Pangenome integration reconciles gene-cluster classifications across datasets with different identifiers, enabling annotations, comparative fitness analyses, and ecological metadata to be joined at gene level. 1 The pitfalls audit refines this goal: identifiers, namespaces, schemas, and access paths must be discovered from live catalogs, and every bridge must preserve provenance rather than treating a successful join as biological confirmation. 2
Identifier bridges and cross-dataset coverage
The acinetobacter-baylyi-adp1 exploration analyzed a database containing Acinetobacter baylyi ADP1 and 13 related genomes. 1 Its mmseqs2-style identifiers had 0% direct string match to KBase KE centroid identifiers. 1 Integration instead used the KBase KE gene_genecluster_junction table to connect KBase Data Lakehouse clusters to member genes matching the ADP1 pan_genome_features table. 1 All 4,891 KBase Data Lakehouse clusters mapped to 4,081 unique ADP1 clusters, producing a 100% gene-level match across 43,754 genes; this is gene-level completeness, not proof that clusters are identical, because clusters may split or merge. 1 The kbase_ke_pangenome collection supplied the A. baylyi pangenome context of 13 genomes and 4,891 clusters. All 4,891 cluster IDs matched the pangenome through this mapping. The saved table data/cluster_id_mapping.csv lets any KBase Data Lakehouse pangenome annotation, such as eggNOG or functional predictions, be joined to ADP1 genes. 1
The same kbase_ke_pangenome collection recurs across projects as the shared join anchor. The data atlas describes KBase computed pangenomes as the canonical genome reference and the cross-program join anchor. 3 The ADP1 deletion-phenotype study took pangenome cluster IDs and core/accessory status from it, cross-referenced through the pangenome_cluster_id column in genome_features. 4 The co-fitness/co-inheritance study built genome × cluster presence matrices from the gene_genecluster_junction and gene tables. 5 The conservation–fitness synthesis lists the KBase pangenome as a gene-cluster conservation dataset spanning 27,690 species. 6 The counter-ion analysis used gene_cluster data for core/accessory classification via the Fitness Browser–pangenome link. 7 The metal cross-resistance analysis likewise took core/accessory genome classification from kbase_ke_pangenome through ortholog groups. 8 The CF formulation study integrated 4,949 isolates from 175 CF/NCFB patient samples in the PROTECT CF Synbiotic Cocktail Study with a KBase Data Lakehouse pangenome, which it reported as containing 293,000 genomes, together with GapMind metabolic pathway predictions. 9
The conservation–fitness analyses extend this bridge from ADP1 to Fitness Browser–pangenome links. The Phase 1 table contained 177,863 gene-to-cluster links, with 100.0% median protein identity and 94.2% median gene coverage; 44 of 48 Fitness Browser organisms mapped to pangenome species clades. Of these, 34 organisms had >=90% mapping coverage, but only 33 entered downstream analysis because Dyella79 was excluded for a locus-tag mismatch. 10 Gene-to-cluster mapping used DIAMOND blastp protein-similarity search. 10 A related analysis cautions that the 90% identity threshold for DIAMOND matching may miss rapidly evolving genes. 11 The newer integration incorporated approximately 194,000 genes from 43 bacteria with RB-TnSeq (random-barcode transposon sequencing), essentiality, condition-specific phenotypes, and pangenome mappings. 12
The resulting Fitness Browser–pangenome link became reusable infrastructure. A cross-project file, conservation_vs_fitness/fb_pangenome_link.tsv, bridges Fitness Browser genes to pangenome clusters for downstream projects such as the AMR fitness-cost analysis. 13 The module-conservation study cites the same KBase pangenome link table, at conservation_vs_fitness/data/fb_pangenome_link.tsv, as its gene-to-cluster conservation mapping. 14 The conservation–fitness synthesis took RB-TnSeq mutant fitness for ~194K genes across 43 bacteria from the Fitness Browser. 6 The co-fitness/co-inheritance study took the top-20 co-fitness partners per gene from the Fitness Browser cofit table. 5 The Gene Function Ecological Agora used a Fitness Browser BBH (bidirectional best hit, i.e. reciprocal top-scoring homolog pairs between genomes) cross-walk covering 35 species, but only for point-validation. 15 The discoveries log refines this route for pathway questions. The Fitness Browser’s own KEGG tables (besthitkegg → keggmember → kgroupec → keggconf → GapMind pathway) give a direct gene-to-KEGG KO-to-EC-to-pathway mapping for all 48 Fitness Browser organisms. This chain is simpler than the DIAMOND blastp link-table pipeline and covers more organisms. 16
The module-conservation analysis supports and refines this gene-level bridge at the level of ICA (independent component analysis) fitness modules. Across 1,116 modules from 32 organisms, module genes were 86.0% core versus 81.5% for all genes (OR=1.46, p=1.6e-87). 14 Among 974 modules with at least 3 mapped genes, 577 (59%) were >90% core, 349 (36%) mixed, and 48 (5%) <50% core; the median module was 93.4% core. 14 Module-family breadth did not predict core fraction (Spearman rho=-0.01, p=0.914), so conservation remains better treated as a gene-level property than as a consequence of cross-organism module scope. 14
The Gene Function Ecological Agora scales and refines the bridge by combining GTDB-r214 phylogeny, KO presence, pangenome openness, environmental metadata, phenotypes, and gene-neighborhood evidence across 18,989 species representatives. 15 Its atlas contains 13.74M rank × clade × KO producer/participation scores, 17.07M Sankoff-parsimony gain events, and 3.94M KO × genus MGE-machinery records. 15 Producer × Participation classified clade–function tuples as Innovator-Isolated, Innovator-Exchange, Sink/Broker-Exchange, or Stable; Sankoff parsimony with M22 recipient-rank attribution replaced the earlier parent-rank dispersion metric, while D2 annotation-density residualization and leaf_consistency supplied bias and within-clade diagnostics. 15 The pangenome-internal environmental tables were richer than the plan’s external-substrate list suggested. They include 4.1M ncbi_env EAV (entity–attribute–value) rows and 64-dimensional quantitative geospatial AlphaEarth embeddings. 15 The report lists a presence-call caveat among the limitations of its earlier Phase 1A pilot: in that phase, a paralog fallback (n_gene_clusters used when UniRef90 was absent) covered 21.5% of presence rows, and the report states that Phase 1B should report sensitivity with and without the fallback. That percentage is scoped to Phase 1A; the evidence here does not show how much the fallback contributes to the later full-scale atlas. 15
The functional-dark-matter analysis further extends this bridge to an evidence-weighted catalog of 228,709 genes across 48 Fitness Browser organisms. Of 57,011 dark genes, 39,532 (69.3%) had pangenome links; 12,686 were accessory, 511 both accessory and strongly fitness-active, and 6,142 belonged to independent-component-analysis fitness modules. 17 Full GTDB r214 integration increased conservation coverage from 32,791 (57.5%) to 37,997 (66.6%) dark genes by querying 93.5M gene-cluster annotations and recovering 5,206 additional dark genes. 17 The report also attributes the recovery of 5,206 additional dark genes to ortholog-group identifier propagation. That step transferred root_og assignments from genes with known pangenome links to genes in the same 48-organism ortholog group. Part of this coverage gain therefore rests on ortholog-based inference rather than direct gene-to-cluster linkage. 17 This supports pangenome links as a scalable route for prioritizing measurable unknown genes, while taxonomic imbalance and conservation-by-ignorance classifications require caution. 17
The new truly-dark-gene census refines this interpretation by separating annotation lag from persistent functional uncertainty. Of 57,011 dark genes, 39,532 had pangenome links that enabled Bakta v1.12.0 reannotation; 33,105 (83.7%) were reclassified, whereas 6,427 remained hypothetical in both pipelines. A further 17,479 lacked pangenome links and could not be assessed. 18 Thus, a pangenome bridge can expose annotation-lag genes, but it cannot by itself resolve the unlinked coverage gap or establish function for genes that remain hypothetical. 18 The 6,427 persistent genes were shorter, less conserved, more taxonomically restricted, and lower in GC content than annotation-lag genes; 18.0% were essential versus 13.4% of annotation-lag genes. 18 This supports using pangenome linkage together with fitness, orthology, genomic context, and annotation evidence rather than treating cluster membership alone as functional evidence. 18
The fitness-module study further supports cross-organism integration by aligning 1.15M bidirectional-best-hit (BBH) pairs across 32 organisms into 13,402 ortholog groups. It identified 156 module families spanning 2+ organisms, including 28 spanning 5+, 7 spanning 10+, and 1 spanning 21 organisms; 145 families had consensus functional labels, representing 93% of the families. 19 BBH-family membership does not prove identical molecular function. 19 The functional-dark-matter study refines this interpretation: 65 ortholog groups present in at least three organisms showed concordant dark-gene effects under the same condition classes, with several concordance values of 1.00, and 998 operon pairs were double-validated by conserved synteny and strong co-fitness. 17
BacDive validation extends the bridge to strain-level ecology. Species-name matching linked 42,227 of 97,334 bacdive strains (43.4%) to scores for 6,426 GTDB species; exact matches covered 33,535 strains (34.5%), suffix removal added 8,692 (8.9%), and 55,107 (56.6%) remained unmatched. 20 Of matched strains, 25,089 had isolation metadata. 20 A phenotype-to-metal-score subset matched 37,368 strains (38.4%), representing 5,647 gtdb species, including 3,994 species with at least 5 phenotype features. 21 Thus, ADP1 demonstrates high coverage in a constrained test, whereas BacDive–GTDB provides useful but lossy large-scale coverage. 1,20,21 The pitfalls audit refines this result: short ENIGMA strain names are not globally unique, and one erroneous MT20 match involved 8,434 genomes, including 1,751 clinical genomes; of 32 pangenome linkages through ncbi_strain_identifiers, 12 were incorrect genus matches. 2 Genus consistency and, where possible, assembly accessions such as GCF_* are therefore required before interpreting strain-level joins. 2
The MicrobeAtlas metal-ecology analysis extends the bridge from genome-level metadata to a global amplicon atlas. It joined a pangenome database of 6,789 sequenced species across the GTDB bacterial tree with a global OTU (operational taxonomic unit) atlas of 98,919 OTUs × 463,972 samples. 22 The report describes this as the first analysis linking genus-level metal-type diversity, inferred from pangenome AMR annotations across 6,789 species, to global ecological niche breadth from a 464K-sample 16S atlas, with phylogenetic control via PGLS (phylogenetic generalized least squares, regression that accounts for shared ancestry). 22 Because the metal trait is aggregated to genus level, this join contrasts with the strain-level BacDive bridge: it gains taxonomic and geographic scope but does not resolve strain- or species-level variation. 22,20
The annotation-gap study supports and generalizes the bridge beyond ADP1 by joining pangenome clusters with model-based gapfilling, Fitness Browser phenotypes, gapmind pathway evidence, bakta annotations, and BLAST homology across 14 organisms. Pangenome conservation helped resolve 96 of 201 gapfilled enzymatic reaction-organism pairs (47.8%): 44 high-confidence, 19 medium-confidence, and 33 low-confidence assignments, leaving 105 unresolved. 23 BLAST alone resolved 70 (34.8%), NB03 EC matching 51 (25.4%), NB04 Bakta alternatives 22 (10.9%), and the full pipeline 96 (47.8%); no single stream exceeded 35% resolution. 23 The NB03 stream matched gapfilled-reaction EC numbers to Fitness Browser gene annotations via pangenome gene clusters. It resolved 51/201 pairs (25.4%) and generated 107 gene candidates. 23
The BERDL Data Atlas — Inventory, Topic Map, and Cross-Reference Synergies supports and scales this interpretation: the BERDL Data Atlas inventories 1,740 deduplicated tables across 119 databases, 17 tenants, and 10 funding agencies or programs, with 536 schema-level cross-tenant bridges defined by 29 canonical join keys. 3 KBase pangenome data contain 1,011,650,903 gene rows spanning 293,059 genomes, 27,690 GTDB species clades or pangenomes, and 132.5M gene clusters. 3 Among 66 audited BERIL projects, 51 (77%) span multiple tenants; 36 are realized KBase × KEScience bridges, mostly pangenome-by-fitness joins through genome_id and ncbi_taxon_id. 3
The Web of Microbes integration extends this bridge from gene and genome tables to metabolite-production phenotypes. The 2018 snapshot contained 37 organisms and 589 metabolites, with 332 (56.4%) unidentified; for Pseudomonas sp. FW300-N2E3, 19 produced metabolites matched Fitness Browser carbon- or nitrogen-source experiments, including 5 de novo products and 14 amplified metabolites. 24 WoM-to-pangenome links were available at genus level for all WoM genera, while species-level matching still required strain-to-genome mapping; the report therefore supports future comparisons of the E/(E+I) metabolic-novelty fraction with pangenome openness or accessory genes, but this remains a hypothesis rather than an established association. 24 Pangenome context (gene clusters, conservation, functional annotations) is available for all WoM organism genera. However, that project did not attempt the strain-to-genome mapping needed for species-level linkage. Its pangenome table also lists only the top five pangenome species per genus: Bacillus 5 species and 2,557 genomes, Rhizobium 5 and 449, Pseudomonas fluorescens 5 and 139, Synechococcus 5 and 88, Phenylobacterium 5 and 80, Acidovorax 5 and 79, Zymomonas mobilis 1 and 26, and Escherichia coli 1 and 2. Because of the top-five limit, those species counts are not exhaustive totals. 24 The same integration refines cross-collection interpretation because only 69 of 257 identified compounds (26.8%) had definitive ModelSEED links, while 107 (41.6%) had formula-only candidate sets and 81 (31.5%) were unmatched. 24
The plant-ecotype analysis extends this integration pattern from broad pangenome and fitness joins to compartment-specific plant ecology. Across 293,059 GTDB r214 genomes, it identified 1,136 plant-associated species and 7,995 plant-associated genomes by joining pangenome gene-cluster architecture, marker genes, eggNOG, GapMind, NMDC, MGnify, host metadata, and within-species phylogenetic data. 25 The refined root–rhizosphere–phyllosphere comparison explained approximately 6% of functional variance by db-RDA (distance-based redundancy analysis) location-only R² = 0.060, while the original R² = 0.527 fell to 0.072 after excluding three genome-rich species per compartment. 25 Thus, identifier integration can reveal a real but weak ecological signal while exposing taxonomic sampling as a major confounder.
The plant analysis also supports joining pangenome classes to pathway and trait architecture: beneficial plant-growth-promotion clusters were 64.6% core versus 45.2% for pathogenic clusters and 46.8% genome-wide, with U = 83,567,419 and p = 3.38e-125. 25 It refines simple accessory-trait interpretations because plant-associated marker singletons were 16 times more likely to co-occur with transposase or integrase singletons (Fisher OR = 15.95, p = 8.8e-20), yet marker singletons were 0.78 times as prevalent as the genomic average and MGnify mobilome enrichment occurred at genus rather than genome scale. 25
The plant study extends pathway integration through an 83.1% BacDive–GapMind consistency rate and corrected co-occurrence analysis, which found weakly negative rather than positive complementarity (Cohen’s d ≈ −0.4, permutation p < 0.001). 25 It also supports controlled functional enrichment: 50 eggNOG families remained enriched after phylum-level control, including COG3569 with Fisher OR = 8.92 and phylogenetically controlled OR = 6.01, q = 7.7e-242. 25 However, the near-saturated raw enrichment of 5,341/5,671 tests refines the claim by showing that broad taxonomic composition can dominate uncorrected pangenome comparisons. 25
The new T4SS–CAZy analysis extends this bridge to mobile gene neighborhoods and environmental HGT. Among 30,497 high-quality environmental MAGs, 6,652 (21.8%) carried T4SS or conjugative machinery under a multi-marker definition. 26 Ninety-two CAZy families showed elevated co-occurrence with T4SS loci within ≤10 kb, with GT2 glycosyltransferases the top hit in 767 genomes; threshold validation remains pending. 26 Associations were enriched in marine sediment (OR=5.5, q<10⁻⁹⁸), barley rhizosphere (OR=10.4), and maize rhizosphere (OR=4.1). 26 A GT2 gene tree contained 77 detected HGT events, including 32 normalized high-confidence cross-phylum events; Node_4915 spanned 8 phyla at Max_Divergence = 4.843. 26 This supports the existing use of pangenome neighborhoods to study gene mobility across genomic backgrounds, but refines it as an observational HGT hypothesis rather than a demonstrated T4SS mechanism: CAZy genes were not detected on plasmids by ICEfinder, 12 integrative mobilizable elements occurred among the top 100 accumulators, and T4SS-positive genomes had 10× higher MGE density than other genomes. 26
The pangenome taxonomy tables must be joined through genome_id, not gtdb_taxonomy_id, because the genome table stores a genus-level taxonomy string while gtdb_taxonomy_r214v1 stores a species-level string. 2 gapmind_pathways.clade_name uses the full gtdb_species_clade_id, including the representative accession, rather than the shorter GTDB_species value. 2 Pangenome gene-cluster identifiers are species-specific and cannot be compared directly across species; cross-species comparisons should use shared representations such as COG categories, KEGG orthologs, or Pfam domains. 2 The CF formulation study supports this rule across annotation pipelines. PROTECT genomes annotated by GenomeDepot and pangenome genomes annotated by Bakta/eggNOG made gene names unreliable for cross-database comparison. The study therefore used Pfam domain IDs as a pipeline-independent vocabulary for a phylogenetic tree of Pseudomonas aeruginosa (PA) genomes, with 2,850 domains shared between the two databases. The ~3,200 unique-domain figure is reported per PA genome only, not for all PROTECT genomes. 9 eggnog_mapper_annotations.query_name joins to gene_cluster.gene_cluster_id, not gene.gene_id; Bakta and InterProScan annotations likewise join at gene-cluster level. 2
The pitfalls audit adds further schema rules for these joins. Column names differ by table: kbase.ke_pangenome.gene_cluster carries gtdb_species_clade_id rather than clade_name, gapmind_pathways carries clade_name, and the gene table has only gene_id and genome_id, with no species or clade column. 2 gtdb_species_clade_id has the form s__<Genus_species>--<RS_GCF_XXXXXXXXX.X | GB_GCA_XXXXXXXXX.X> (e.g., s__Escherichia_coli--RS_GCF_000005845.2). genome_id has the form RS_GCF_XXXXXXXXX.X (RefSeq) or GB_GCA_XXXXXXXXX.X (GenBank) and should be matched by exact equality only. 2 The pangenome table references 12 species clades absent from gtdb_species_clade. These orphan single-genome species or symbionts are valid pangenomes that were filtered from species metadata. They should be handled with a LEFT JOIN, or an anti-join where needed, rather than silently dropped. 2 gene_genecluster_junction has 141 orphan genes relative to gene, which an INNER JOIN silently drops. genome_ani contains only within-species comparisons, so cross-species ANI is not stored. 2 These rules refine gene-level bridges that pass through gene_genecluster_junction, including the ADP1 bridge: completeness claims should state the join type used. 2,1
Because gene-cluster IDs are only meaningful within a species, identically named clusters must not be treated as equivalent across species. The audit’s example is that cluster X_123 in E. coli is unrelated to X_123 in Salmonella. 2 The same species specificity changes counts after joins: a DUF4041 query returned 2,962 eggNOG annotation records but 4,572 rows after joining to gene_cluster, because the same query_name can map to gene-cluster entries in multiple species pangenomes. 2 The post-join count is the number of species-level gene clusters, and the eggNOG count is the number of unique annotation records. Both are valid for different purposes, and reports must document which one they use. 2 This refines the SNIPE survey’s 4,572 DUF4041 clusters (below) as a species-level gene-cluster count rather than a count of distinct annotation records. 2,27
Conservation, fitness, metabolism, and pangenome classes
The conservation–fitness synthesis supports a positive relationship between laboratory fitness importance and pangenome conservation. Across 194,216 protein-coding genes from 43 bacteria, essential genes were 82% core whereas always-neutral genes were 66% core; 28,017 genes were both costly and conserved, while 5,526 were costly and dispensable. 6 The first group is evidence for, rather than a direct measurement of, purifying selection in nature. 6
The fitness-effects analysis refines this gradient: essential genes with no viable mutants were 82% core (n=27,693), often-sick genes 78% (n=15,989), mixed genes 70% (n=20,739), sometimes-sick genes 72% (n=25,201), always-neutral genes 66% (n=94,889), and sometimes-beneficial genes 70% (n=9,705). 12 Fitness breadth correlated positively but weakly with core status (Spearman rho=0.086, p=8.1e-230). 12
The essential-genome analysis supports and extends this association across 221,005 genes from 48 bacteria: 859 (5.0%) ortholog families were universally essential, 4,799 (27.9%) variably essential, and 11,564 (67.1%) never essential. 28 Universally essential genes were 91.7% core, variably essential genes 88.9%, never-essential genes 81.7%, and orphan essential genes 49.5%. 28
The truly-dark-gene analysis refines the interpretation of essentiality as a conservation signal. Truly dark genes had an essential fraction of 18.0%, compared with 13.4% for annotation-lag genes, and 34 of the top 100 prioritized candidates were essential. 18 This supports integrating essentiality with pangenome status to prioritize unknown genes, but the short length of truly dark genes and reduced transposon-insertion opportunities may affect both annotation and fitness measurement. 18
The PGP pangenome analysis supports and extends these conservation results to plant-growth-promoting genes. Across 27,702 species, 11,272 carried at least one of 13 PGP markers; all 13 had higher core fractions than the 46.8% genome-wide baseline, and mean PGP accessory fraction was 29.7% versus 53.2% genome-wide. 29 Highlighted core fractions included pqqC 81.5%, acdS 70.4%, nifH 63.8%, and pqqD 55.5%; PGP gene richness correlated negatively with openness (Spearman rho=-0.195, p=2.0e-97). 29 The plant study supports the core-encoded PGP pattern at a broader plant-associated scale, measuring 64.6% core beneficial clusters, but refines it by showing that pathogenic clusters were only 45.2% core and that a ten-strain Pseudomonas study found biocontrol traits largely accessory. 25
The PHB study qualifies any simple extension of conservation patterns from PGP markers to storage pathways. Across 27,690 GTDB species, 21.9% carried phaC and 21.7% the complete PHB pathway; 6,067 phaC-carrying species spanned 95 of 142 GTDB phyla, with 5,371 species having core phaC and 1,959 accessory phaC. 30 Putative 311 acquisition and 278 loss events remain proxy-based and require a phaC gene tree. 30
The metabolic-capability study contradicts any stronger inference that pathway conservation alone identifies dependency: among 755 active dependencies, 508 intermediate pathways, and 248 latent capabilities, mean conservation was 0.829, 0.907, and 0.869, respectively, with p=0.94 for active > latent. 31 The pathway analysis found mean core completeness of 0.986 for Active Dependencies versus 0.975 for Latent Capabilities. 32 The Pseudomonas carbon analysis supports the more cautious interpretation: pathway profiles can distinguish deep clade structure and carry a modest ecological signal, but lifestyle differences within Pseudomonas_E substantially overlap. 33
Condition-specific fitness and ecological context
The fitness-effects analysis contradicts the expectation that condition-specific genes are predominantly accessory: strong condition-specific genes were 77.3% core versus 70.3% without specific phenotypes (OR=1.78, p=1.8e-97). 12 The metabolic-capability analyses support and sharpen this interpretation because complete pathways classified as latent under laboratory thresholds may reflect untested environments. 31,32
The truly-dark-gene analysis refines interpretation of condition-specific evidence by rejecting stress enrichment. Among strongly fitness-phenotyped truly dark genes, stress conditions were reported at 28.7% versus 43.2% for annotation-lag genes (OR = 0.53, p < 0.001); the results also reported stress at 43.3% versus 54.7%, carbon-source conditions at 13.7% versus 21.7%, and motility conditions at 1.2% versus 3.2%. 18 Mixed-community, iron, nutrient time-series, and rich-media conditions were reported as enriched in the truly dark set. 18 These measurements contradict a generalized stress-enrichment hypothesis, while suggesting—but not establishing—that some persistent unknowns may participate in metabolic or community-interaction functions. 18
The annotation-gap study supports and extends phenotype integration with carbon-source RB-TnSeq measurements. Across 574 organism–carbon-source combinations, baseline FBA (flux-balance analysis, a constraint-based method for predicting metabolic flux and growth) achieved 42.5% accuracy, 86.5% recall, and 42.5% precision; conditional gapfilling for 38 false-negative cases added 219 reactions. 23
The Web of Microbes bridge supports testing gene-fitness relationships against organism-produced metabolites: 19 FW300-N2E3 products matched Fitness Browser carbon or nitrogen experiments, but the snapshot contained no organism consumption actions. 24 Therefore it refines the stronger dependency interpretation: produced-metabolite/fitness comparisons are actionable, whereas testing whether consumption predicts essentiality remains blocked until consumption data are available. 24
The PGP analysis extends condition-specific integration to ecological gene distributions: acdS, pqqC, and hcnC were soil-enriched, nifH was depleted, and acdS enrichment remained strong after phylum control (OR=6.98, p=4.8e-61) and in a strict rhizosphere analysis (OR=10.6, q=7.6e-38). 29 The plant analysis refines this ecological bridge by showing that compartment separation is statistically significant but weak (PERMANOVA pseudo-F = 23.2, p = 0.001, R² = 0.071; db-RDA location-only R² = 0.060), and that 878 of 1,115 plant-associated species (78.7%) were dual-nature under the refined marker scheme. 25 Marker presence therefore does not establish expression, phenotype, or a useful categorical ecological classifier. 25
The PHB study supports and refines this ecology bridge: plant-associated species had 44.0% phaC prevalence versus 43.6% in soil and 18.7% in marine environments, but the genome-size-controlled PHB–niche-breadth association was partial rho = -0.047 (p = 0.037), contrasting with the raw rho = 0.106. 30
The Pseudomonas analysis refines these pathway–ecology claims with a controlled carbon-capability example. Among 54 sufficiently sampled free-living and plant-associated species, pathway profiles were associated with isolation environment by a 999-permutation test (p = 0.006), but four-class prediction achieved only 0.408 +/- 0.169 balanced accuracy. 33 The result supports non-random environmental embedding while limiting claims that pathway completeness alone identifies a fine-grained niche. 33
Comparative pangenome and ecological context
The ADP1 example contained 3,207 core and 1,684 accessory gene clusters across 13 genomes, all assigned to s__Acinetobacter_baylyi and s__Acinetobacter_baylyi--RS_GCF_000368685.1; all 13 genome IDs matched the pangenome. 1 The same report noted 120 unmatched reactions (9%) that may be custom or draft reactions not yet in ModelSEED. ADP1 is absent from the Fitness Browser, so the database’s mutant growth data are a resource not available elsewhere in the KBase Data Lakehouse. 1 The atlas scales this example to 293,059 genomes, 27,690 GTDB species clades or pangenomes, and 132.5M gene clusters. 3,34
The ecotype analysis refines the ecological bridge by separating ancestry-wide from environment-associated gene-content similarity. Across 172 bacterial species, median partial correlation of environment with gene content was 0.0025 versus 0.0143 for phylogeny; phylogeny dominated in 60.5% and environment in 39.5%. 35 The pangenome-openness analysis found no significant relationship between openness and environment or phylogeny effects (rho=-0.05, p=0.54; rho=0.03, p=0.73). 36
The environmental-resistome analysis supports and scales the ecological bridge: KBase KE pangenomes connected AMR clusters with ecological classifications across 14,723 species and 293K genomes; clinical species had a median of 5 AMR clusters versus 2 for soil, aquatic, and host-associated environments, and clinical AMR was 68% accessory versus 43% in soil. 37 The AMR atlas found clinical AMR at 30.8% core versus 58.1% soil and 63.1% plant, with environmental diversity positively correlated with AMR count (rho=0.466, p=1.6e-144) and negatively with AMR core fraction (rho=-0.173, p=1.8e-19). 38
The Web of Microbes result extends this ecological framework to metabolite novelty: among ENIGMA isolates in R2A medium, E/(E+I) de novo-production fractions ranged from 15.2% to 32.4%, but the proposed relationship between this phenotype and pangenome openness or accessory genes has not yet been tested. 24
The new truly-dark-gene evidence supports and refines the accessory-genome component of this framework. Truly dark genes had a core-genome fraction of 43.1% versus 72.7% for annotation-lag genes, ortholog presence of 29.3% versus 63.7%, and median ortholog breadth of 1 versus 4 organisms. 18 Their mean absolute GC deviation was 0.047 versus 0.038, 9.2% versus 4.0% had strong GC deviation, and 12.0% were within 2 genes of a mobile genetic element. 18 These patterns are consistent with recent acquisition or rapid evolution, but GC deviation is an imperfect HGT proxy and the observations do not establish HGT or gene function. 18
The new T4SS evidence supports and refines this resistome framework by linking a transfer-associated CAZy neighborhood to metal-resistance breadth: GT2-neighborhood MAGs had 0.045 versus 0.004 mean metal-resistance types, with Mann–Whitney p=8.6e-27, and GT2 in T4SS-proximal neighborhoods was associated with 11× more metal-resistance genes. 26 This is consistent with environmental HGT hubs carrying multiple niche functions, but it is an association and does not prove co-transfer of CAZy and resistance genes. 26
The prophage-AMR atlas extends this resistome framework from defense-system inventories to prophage-associated gene neighborhoods and species-level mobile-element context. Across 27,702 species, 14,669 (52.9%) carried both AMR and prophage markers; among 36,041 AMR instances from 1,953 genomes across 100 species, 20,073 (55.7%) were on contigs carrying strict prophage markers and 3,731 (10.4%) were within 10 genes of one. 39 The proximity association was statistically significant but modest: AMR genes within 10 genes were 67.6% accessory versus 65.5% for distal genes (odds ratio 1.10, one-sided p=0.005, bootstrap 95% confidence interval [1.024, 1.185]). 39 More strongly, prophage marker density correlated with AMR repertoire breadth across 4,770 species (Spearman rho=0.572, p<10^-300), remaining associated after controlling for genome count (partial rho=0.464, p=1.0×10^-253). 39 This refines the interpretation of mobile-element integration: the strongest signal is species-level co-variation rather than a uniform, gene-level proximity effect, and it does not establish phage-mediated AMR transfer. 39
The phage-defense analysis extends this resistome framework to defense-system architecture. Across the KBase Data Lakehouse pangenome, markers for seven anti-phage defense families occurred in 27,626 of 27,690 species-level pangenomes (99.8%), producing 930,573 marker hits across 825,476 unique gene clusters. 40 Six systems were significantly enriched in auxiliary and singleton fractions, while DISARM detection was unreliable because its PF00176 anchor is widespread in housekeeping helicases. 40 In the ≥5-genome set, 27 of 28 defense-system pairs had significant positive co-occurrence; R-M Type II × Gabija had 2,429 observed co-occurrences versus a null mean of 1,555, z = 46.1, and OR = 24.0, whereas CRISPR-Cas × CBASS was nonsignificant. 40
The SNIPE analysis supports and qualifies this defense-system framework with a distinct, mostly accessory domain distribution. Its 4,572 DUF4041 clusters spanned 33 phyla and 1,696 species, but the inferred 86.7% accessory-plus-singleton fraction is based on DUF4041 clusters and does not by itself prove complete defense-system activity. 27 In the Fitness Browser, only Methanococcus maripaludis JJ carried a full two-domain SNIPE protein among the surveyed organisms; it had 129 fitness experiments, a minimum fitness of -1.16 on formate/acetate, and was dispensable under most conditions. 27
The lanthanide methylotrophy atlas supports and extends large-scale marker-to-environment integration across 293,059 genomes, joining xoxF and mxaF, lanmodulin, PQQ-biosynthesis annotations, and environmental metadata. 34 It found 3,690 xoxF genomes versus 195 mxaF genomes, a global xoxF:mxaF ratio of 18.92:1; Bakta identified 62 validated lanmodulin genomes whereas eggNOG produced 505 likely false-positive lanM calls. 34
The ENIGMA Carbon Census extends the bridge from genes and genomes to compounds and environments by integrating 83 enrichment compounds, 3109 genomes, utilization predictions, GTDB placement, SSO occurrence, NMDC metagenomes, and Planet Microbe runs; 74/83 compounds remained organism-dark. 41 The ENIGMA contamination analysis refines this approach by linking geochemistry and community taxa to KBase pangenome clades and eggnog proxies across 108 samples, 212 communities, and 1,392 genera. 42
The AlphaFold MSA analysis qualifies technically successful integration: only 38,051,842 of 132,531,501 clusters (28.7%) bridged successfully to MSA depth, while 70.7% lacked required identifiers or AlphaFold entries. 43 The discoveries log refines annotation integration: eggNOG had higher COG, KEGG, and Pfam coverage, Bakta had higher GO, product-description, and unique UniRef50 coverage, their union reached 77.3% any-functional-annotation coverage, and Bakta rescued 11.2M clusters among 39.2M missed by eggNOG. 16
The truly-dark-gene census further qualifies coverage-based interpretations. Among truly dark genes, 79.4% had UniRef50 links and 84.7% had database cross-references, but only 4.0% had Pfam hits and 4.6% had KEGG KOs; 43.5% had partial eggNOG signal and only 17.0% had COG categories. 18 Therefore, sequence-level linkage often exists without functional assignment, and pangenome integration should preserve separate evidence layers for identity, orthology, annotation, phenotype, and genomic context. 18
Limitations and tensions
The ADP1 bridge passes through 3 tables. Although it was reported as 100% complete, this indirection may conceal edge cases where clusters split or merge between the two pangenome analyses, and it covers only A. baylyi. 1 BacDive left 56.6% of strains unmatched. 20 Conservation–fitness integration could not map four organisms (Cola, Kang, Magneto, and SB2B) because their species had too few GTDB genomes for pangenome construction. It also lost the main E. coli clade, which was absent from the pangenome because it had too many genomes, and mapped Keio to a small s__Escherichia_coli_E clade at only 26.1% coverage. 10 Clades with only 2 genomes can have trivially high core fractions, and essentiality was measured under one growth condition. 10
Fitness Browser–pangenome attrition also came from identifier formats and alignment coverage. 10 organisms were excluded from Phase 2 for <90% DIAMOND coverage, which reduced taxonomic breadth. Dyella79 was excluded because its Fitness Browser gene-table locus tags (N515DRAFT_*) differed from its protein-sequence locus tags (ABZR86_RS*), producing a 0% join rate. 10 The discoveries log traces this failure to the Fitness Browser aaseqs file, which uses RefSeq-style locus tags for some organisms while the gene table uses the original annotation tags. Only 1 of 34 organisms was affected, but any pipeline joining aaseqs-derived data with gene-table data should verify locus-tag consistency. 16
Metal-focused analyses report additional Fitness Browser–pangenome attrition. The metal fitness atlas states that only 22 of 48 organisms have pangenome links, with 26 lacking Fitness Browser–pangenome mappings. Its conservation analysis covers 22 of 31 metal-tested organisms (71%), and the 9 excluded organisms (Putida, Keio, SynE, Miya, Kang, BFirm, Cola, Ponti, Dino) lack links. The report judges this exclusion unlikely to introduce systematic bias because the excluded organisms are taxonomically diverse. 44 The 22-of-48 figure is in tension with the conservation–fitness report that 44 of 48 Fitness Browser organisms mapped to pangenome species clades. The sources do not explain the difference, and the counts should not be merged or averaged. 44,10 Identifier formats caused a separate loss: 7 of 31 metal-tested organisms (ANA3, Dino, Keio, MR1, Miya, PV4, SB2B) could not be processed in the metal-specificity analysis because their metal-important gene locusIds did not match the fitness-matrix index format. The report attributes this to an integer-versus-string mismatch and says fixing it would recover the remaining 40% of gene records. 45 GapMind linkage is narrower still: of the 48 Fitness Browser organisms, only 7 had matching GapMind genome data, and 23 GapMind pathways had sufficient gene-level annotation to classify, yielding 161 organism–pathway pairs. 32 Like the Dyella79 case, these losses support verifying identifier types and link coverage before treating a Fitness Browser cohort as representative. 45,10
The truly-dark-gene results introduce a coverage and interpretation tension. The 17,479 dark genes without pangenome links cannot be classified as annotation-lag or truly dark; the report estimates that approximately 2,841 may be truly dark at the observed 16.3% rate. 18 This extrapolated set includes 2,208 genes with strong fitness phenotypes. The report estimates that its ranked list covers ~69% of the estimated total truly dark population and describes the 17,479 unlinked genes (31%) as a 31% coverage gap in the prioritized list. 18 Its proposed remedy, extending pangenome linkage to these genes and then running Bakta, has not yet been carried out. The additional truly dark candidates therefore remain an extrapolation rather than an established finding. 18 Conversely, Bakta may produce false-negative functional calls, and short genes are intrinsically harder to annotate and measure for fitness. 18 These limitations refine rather than overturn the use of pangenomes for unknown-gene prioritization: residual hypothetical status is evidence of annotation resistance, not proof of biological novelty.
The Web of Microbes integration creates a coverage and directionality tension with metabolite-to-fitness interpretation. WoM links produced metabolites to Fitness Browser utilization experiments, but this 2018 snapshot contains no organism consumption action; the report notes that a decrease action described for WoM may occur in newer GNPS2 data. 24 Thus, the bridge supports metabolite-to-gene hypothesis generation but does not yet test whether consumed metabolites predict essentiality. 24
The T4SS–CAZy findings create a mobile-transfer tension rather than resolving one. The 77 GT2 HGT events, including 32 normalized high-confidence cross-phylum events, and the 10× higher MGE density in T4SS-positive genomes are consistent with chromosomal or integrative transfer. 26 However, CAZy genes were not detected on plasmids by ICEfinder, threshold validation is pending, and the associations are observational; therefore the data do not establish that T4SS mediates GT2 transfer. 26 This refines the prophage-AMR tension: as with prophage marker density and AMR breadth, co-localization and MGE enrichment support a mobile-element context but do not prove direct gene mobilization. 39,26
The Pseudomonas carbon result creates a related scope tension. Carbon profiles showed a significant environment association (p = 0.006), but the four-class classifier achieved balanced accuracy of only 0.408 +/- 0.169, and the primary PCA axis separated Pseudomonas s.s. from Pseudomonas_E rather than lifestyles. 33 This qualifies any interpretation of pathway profiles as general ecological classifiers, while the plant analysis likewise found weak compartment separation after controls (db-RDA location-only R² = 0.060). 25
The prophage-AMR result introduces a specific mobile-element tension. Species-level prophage density was positively associated with AMR repertoire breadth across five major phyla, including Pseudomonadota (rho=0.54), Bacillota_A (rho=0.55), Bacillota (rho=0.40), Bacteroidota (rho=0.59), and Actinomycetota (rho=0.29), but the association does not prove phage-mediated AMR transfer. 39 The gene-level proximity effect was heterogeneous: the odds ratio was 0.78 at 3 genes, 0.92 at 5 genes, 1.10 at 10 genes, 1.19 at 15 genes, and 1.28 at 50 genes, with only 33 of 74 testable species above 1 and a median species-level odds ratio of 0.85. 39
The phage-defense integration has database-specific detection limitations: eggNOG description-based CRISPR detection produced 96% species prevalence versus approximately 55% for Cas1 PF01867, and InterProScan returned approximately 500K rows for curated Pfams where Bakta contained 0 Cas1 hits. 40 The PHB analysis likewise classified all 11,792 phaC clusters as “other_pfam” until eggNOG domain names were mapped to PF00561 and PF07167. 30
The pangenome-openness null result remains in tension with metabolic-capability associations: openness was positively associated with latent capability rate (rho=0.69, p=0.0004, n=22 clades), variable pathway count (partial rho=0.530, p=2.83e-203), and metabolic ecotype count (partial rho=0.322, p=8.0e-07), but not environment or phylogeny effect sizes (rho=-0.05, p=0.54; rho=0.03, p=0.73). 31,32,36 PGP gene richness added a negative openness association (rho=-0.195, p=2.0e-97), which is a trait-specific contrast rather than a direct contradiction. 29
The field-versus-lab result creates a conservation tension: DvH field-stress genes were 83.6% core and field-core genes 82.4% core, but heavy-metal genes were 71.2% core and lab-antibiotic genes 73.4% core; broader studies found essential genes 86.1% or 82% core depending on cohort and definition, while the metal-fitness atlas reported 87.4% core for broad metal-important genes. 46,10,6,44,8 These estimates should not be averaged.
The fitness-module benchmark creates a tension in treating cofitness modules as annotations: module-ICA and cofitness voting each had <1% strict KEGG KO precision, whereas ortholog transfer achieved 95.8% strict precision, 91.2% coverage, and 0.934 F1. 19 Conserved modules therefore support process-level hypotheses, not exact molecular-function assignments. 19
The identifier audit creates a provenance tension with apparent large-scale coverage: 12 of 32 pangenome linkages through ncbi_strain_identifiers were incorrect genus matches, and an erroneous ENIGMA MT20 match involved 8,434 genomes. 2 This contradicts interpreting raw linkage counts as validated biological integration; genus checks and accession-based joins are required. 2
Open Directions
- Test whether any of the 4,081 mapped ADP1 clusters split or merge relative to 4,891 KBase Data Lakehouse clusters using cluster membership and representative sequences. 1
- Test fitness–conservation relationships with matched organisms and conditions, controlling for fitness magnitude, breadth, gene length, annotation coverage, and transposon coverage. 12
- Align the 156 fitness-module families and 974 mapped modules to pangenome prevalence, testing core status against organism, experiment count, operon adjacency, and phylogeny. 19,14
- Reconstruct essential-gene modules using measurements that do not require viable transposon insertions. 14
- Extend pangenome linkage to the 17,479 unlinked dark genes and quantify how the estimated 2,841 additional truly dark genes alter prioritization. 18
- Test the top truly dark candidates with structure prediction, Foldseek searches, predicted-condition growth assays, mobile-CRISPRi, and genomic-island or prophage-remnant analyses. 18
- Join environmental AMR classifications to gene-level annotations and fitness, testing CARD ARO mapping, gene gain/loss, islands, insertion sequences, and integrons. 37,38
- Match BacDive GCA accessions directly to pangenome
genome_idvalues and repeat the heavy-metal analysis with enough matched isolates to distinguish a robust effect from the current n=10 estimate. 20 - Reanalyze ecotype cohorts with matched species and genome sets, structured ENVO terms, genome-count covariates, and phylogenetic controls. 47,35
- Directly map GapMind per-step genes to SEED and pangenome clusters and test whether latent-capability and pathway-variation associations persist after controls. 31,32
- Test whether openness predicts ecological effects only for particular gene-function classes, including PGP modules, by stratifying auxiliary fraction, Heap’s law alpha, pangenome fluidity, mobile/defense categories, and genome count. 36,29
- Reanalyze defense-system presence using PADLOC MacSyFinder-style multi-PFam and gene-order rules, including PF13250/DUF4041 plus PF13455 for SNIPE, then test whether syndrome and accessory-enrichment patterns persist after specificity-aware calls. 40,27
- Apply geNomad or PHASTER to the KBase Data Lakehouse genomes and use scaffold sequences for base-pair-resolution distances to test whether the prophage–AMR association persists after dedicated prophage calls and separation of phage, plasmid, and ICE neighborhoods. 39
- Classify the 11,792 phaC clusters after mapping eggNOG domain names to PF00561 and PF07167, then reconstruct a phaC gene tree to test the 311 putative acquisitions and 278 losses. 30
- Build a plant-ecotype integration analysis with expanded phylogenetic-tree coverage, accessory-gene and gene-content trees, host-stratified models, and direct mobile-element annotations to test whether the five significant species generalize. 25
- Extend the Pseudomonas analysis with KEGG aromatic-degradation modules and PGLS (phylogenetic generalized least squares) or phylogenetic logistic regression to test whether environment associations persist after ancestry control. 33
- Test the Web of Microbes hypothesis that E/(E+I) metabolic novelty associates with pangenome openness or accessory genes, using strain-resolved WoM records and species-matched pangenomes; separately test produced metabolites against Fitness Browser gene fitness. 24
- Obtain a current GNPS2 or Northen laboratory WoM release, add consumption actions, and test whether consumed metabolites predict gene essentiality and pathway capability. 24
- Validate T4SS–CAZy proximity with the pending synteny-threshold permutation test, BLAST Node_4915 against NCBI nr, and a housekeeping-gene null baseline; then factorize biome enrichment using θ = OR(T4SS-CAZy) / [OR(T4SS) × OR(CAZy)]. 26
- Test whether GT2–T4SS neighborhoods, IMEs, MGE density, and metal-resistance genes co-occur after controlling for genome quality, contig length, biome, and taxonomic structure, and determine whether transfer is chromosomal, integrative, or plasmid-associated. 26
- Build a live-catalog resolver that records dotted and underscore namespaces, tenant, dataset, schema version, and access outcome before every cross-tenant join. 2
- Recompute strain and species bridges with genus checks, NCBI-taxid-backed synonymy, GTDB-version-aware reconciliation, and assembly-accession joins; quantify how many existing ecological and fitness associations change. 2
- Benchmark Spark-side aggregation and key-filtered joins for billion-row pangenome tables, retaining only small final outputs and documenting schema/type checks, broadcast behavior, and checkpoint provenance. 2
- Resolve the integer-versus-string locusId mismatch for ANA3, Dino, Keio, MR1, Miya, PV4, and SB2B, rerun metal-specificity and conservation analyses, and test whether the recovered gene records change metal-specific conservation estimates. 45
- Recount Fitness Browser–pangenome link coverage from the shared link table under each project’s inclusion criteria to explain the 22-of-48 versus 44-of-48 discrepancy. 44,10
- Carry out the with-and-without paralog-fallback sensitivity analysis that the report recommended for Phase 1B, after the fallback covered 21.5% of Phase 1A pilot presence rows, and measure the fallback’s share in the full-scale presence calls. 15
See the source reports: Acinetobacter baylyi ADP1 Data Explorer, AlphaFold MSA Depth as a Lens on the Bacterial Annotation Gap, Environmental Resistome at Pangenome Scale, Fitness Cost of Antimicrobial Resistance Genes, Pan-Bacterial AMR Gene Landscape, Annotation-Gap Discovery via Phenotype-Fitness-Pangenome-Gapfilling Integration, BacDive Isolation Environment × Metal Tolerance Prediction, BacDive Phenotype Signatures of Metal Tolerance, BERDL Data Atlas — Inventory, Topic Map, and Cross-Reference Synergies, ADP1 Deletion Collection Phenotype Analysis, Rational Design of Protective Microbiome Formulations for Competitive Exclusion of *Pseudomonas aeruginosa* in Cystic Fibrosis Airways, Co-fitness and Co-inheritance in Bacterial Pangenomes, Core Gene Paradox — Why Are Core Genes More Burdensome?, Counter Ion Effects on Metal Fitness Measurements, discoveries, Ecotype Correlation Analysis, Ecotype Reanalysis — Environmental vs Human-Associated Species, ENIGMA Carbon Census — Tiered Knowledge Census of 83 Enrichment Compounds, Contamination Gradient vs Functional Potential in ENIGMA Communities, The Pan-Bacterial Essential Genome, Field vs Lab Gene Importance in *Desulfovibrio vulgaris* Hildenborough, Conservation vs Fitness — Linking FB Genes to Pangenome Clusters, Gene Conservation, Fitness, and the Architecture of Bacterial Genomes, Fitness Effects vs Conservation — Quantitative Analysis, Pan-bacterial Fitness Modules via Independent Component Analysis, Functional Dark Matter — Experimentally Prioritized Novel Genetic Systems, Gene Function Ecological Agora, Lanthanide Methylotrophy Atlas — Summary, Metabolic Capability vs Metabolic Dependency, Gene-Resolution Metal Cross-Resistance Across Diverse Bacteria, Pan-Bacterial Metal Fitness Atlas, Fitness Modules × Pangenome Conservation, Pangenome Openness Analysis, Metabolic Capability vs Dependency, PGP Gene Distribution Across Environments & Pangenomes, Pan-Bacterial Anti-Phage Defense Arsenal, Polyhydroxybutyrate Granule Formation Pathways — Distribution Across Clades and Environmental Selection, Plant Microbiome Ecotypes — Compartment-Specific Functional Guilds and Their Genetic Architecture, Prophage-AMR Co-mobilization Atlas, pitfalls, Carbon Source Utilization Predicts Ecology and Lifestyle in Pseudomonas, SNIPE Defense System in the KBase Data Lakehouse Pangenome, T4SS–CAZy Environmental HGT, Truly Dark Genes — What Remains Unknown After Modern Annotation?, and Web of Microbes Data Explorer.