Differential abundance (DA) — testing which taxa differ in abundance between sample groups — returns answers that depend on two choices made before any test is run: how counts are normalized, and how the contrast is designed. Both choices can reverse the direction of a call, not merely its significance. The methodological grounding is explicit in the corpus: cited work identifies microbiome datasets as compositional, i.e. constrained by a sum-to-constant 1; a cited review sets out the fundamentals, tools and challenges of compositional data analysis of the microbiome 1; and a cited method paper supplies analysis of compositions of microbiomes with bias correction 1. The direct evidence below is method-vs-method and design-vs-design comparison on the same data, which is strong for the comparison itself, but it comes from one human gut project plus the cross-project digest, so the magnitude of the effect elsewhere is untested.
Normalization can reverse the sign of a call
On a curated battery of protective and pathobiont species, a raw mann-whitney-u-test on relative abundance mis-directs 4+ of 8 protective species, and centered log-ratio (CLR) correction — rescaling each taxon against the sample’s geometric mean before testing — recovers them 1.
The case that motivated the re-analysis is clostridium-scindens, the named species central to the apparent Crohn’s-disease (CD) enrichment paradox: a preliminary pooled Mann–Whitney differential-abundance analysis called it CD-enriched at log₂FC +2.67, contradicting its established role as a bile-acid-producing protective species at ~79 % prevalence in healthy individuals, and three explanations were left open — compositional artifact, strain heterogeneity, or ecotype mixing in the pooled analysis 1.
Compositional correction recovers a depletion signal that the raw test misses: F. prausnitzii, A. muciniphila, R. hominis, L. eligens and A. rectalis are reported as flipping from “n.s.” under the raw test to “CD↓” under CLR 1.
Roseburia intestinalis shows an outright sign flip, raw CD↑ → CLR CD↓, reproducing the C. scindens artifact pattern on a second species 1.
The reversal is not universal: the top reported pathobionts agree across methods, with mediterraneibacter gnavus (R. gnavus), enterocloster bolteae, eggerthella lenta and hungatella hathewayi all CD-enriched under both raw and CLR analyses 1.
A null result bounds the fix: C. scindens remains CD-enriched under both methods (raw log₂FC +5.66; CLR Δ +1.25), so compositional correction alone is insufficient, and the strain-heterogeneity and ecotype-mixing explanations remained live at that stage 1. The two raw log₂FC values reported for this species in the project (+2.67 in the preliminary pooled analysis and +5.66 in the curated-battery comparison) come from different analyses and are recorded here as reported, not reconciled 1.
This refines rather than vindicates the normalization fix. Pooled Mann–Whitney on relative abundance is described as systematically under-sensitive for protective-species depletion, an artifact of compositional bias plus pooled-cohort heterogeneity; C. scindens was not resolvable at that level, and under a later within-IBD-substudy CD-vs-nonIBD design that eliminates study confounding the species is still CD↑ — read in that section not as a paradox but as the expected behavior of a species more prevalent in the IBD-cohort source studies than in the healthy-cohort source studies of the curated resource 1. The revised confound-free analysis reports the within-substudy result directly: across the four IBD sub-studies (242 CD / 369 nonIBD pooled) C. scindens has pooled CLR-Δ = +1.18 at false discovery rate (FDR, the expected share of false positives among calls) 1e-8 with 4/4 sign concordance, and is described as genuinely CD↑ in this design 1. See study-batch-confounding-of-environmental-associations and cross-cohort-microbiome-portability for the same confound seen from the design side.
Second-method concordance
The project reports reproducing the compositional-bias result directly on a curated battery and using linda — linear models for differential abundance analysis of microbiome compositional data, i.e. linear CLR regression with bias correction — as a second-method concordance check 1. LinDA was used as the second compositional DA method alongside CLR plus Mann–Whitney in NB04c §4 and implemented in pure NumPy for the project 1. Second-method agreement does not by itself rescue a confounded design: the revised report states that both compositional-bias-aware methods tried on the within-ecotype subsets (CLR-MW and LinDA) share the bias, so evidence counts from within-ecotype methods alone do not resolve the direction reversals 1.
The comparison is published as a figure: NB00_protective_species_paradox.png plots raw log₂FC against CLR Δ on 14 curated protective + pathobiont species 1.
Contrast design changes the target list
Stratifying the contrast prunes candidates as hard as changing the normalization does. In the cross-project digest’s account, within ecotype E3 — one data-driven stratum of gut microbiome samples, see gut-microbiome-ecotypes-as-patient-strata — 3 of 15 candidates passed all three filters: mediterraneibacter gnavus (within-substudy CD↑ +5.13, LinDA E3 +1.64), flavonifractor plautii (within-substudy +1.89, LinDA +2.91) and blautia wexlerae (within-substudy +0.91, LinDA +2.00); these were described there as the trustworthy Tier-A, the tier of candidates cleared for downstream targeting 2.
The scope consequence stated at that stage was that per-ecotype Tier-A reduces from “33 candidates across both E1 and E3” to “3 candidates, E3 only”, that E1 needs a reformulated analytical approach, and that the current within-ecotype DA is not a usable target list; the candidate fixes listed were to rebuild ecotypes on a functional (pathway / KEGG) feature matrix to break taxonomic-feature leakage, to run within-IBD-substudy CD-vs-nonIBD stratified by ecotype as the primary analysis — which requires substudy × ecotype × diagnosis cells with ≥ 10 samples each and, at the time of the digest, still needed checking — or to broaden to UC-vs-HC + CD-vs-HC combined as IBD-vs-control for power 2. The downstream consequence for intervention design is carried in ecological-cost-of-microbiome-target-depletion.
The revised project report supersedes that target list, which makes the digest’s counts historical rather than current. Under a within-ecotype × within-substudy meta-analysis on the four IBD sub-studies carrying both CD and nonIBD (HallAB_2017, LiJ_2014, IjazUZ_2017, NielsenHB_2014), E1 Tier-A = 51 candidates meta-analysed across 2 sub-studies (HallAB_2017 + NielsenHB_2014, 82 CD / 280 nonIBD), all 100 % sign-concordant, and E3 Tier-A = 40 candidates from single-study HallAB_2017 (22 CD / 31 nonIBD), flagged as provisional and needing replication 1. The cell-eligibility question raised by the digest has therefore been answered for these strata — E1 is eligible in two sub-studies while E3 is eligible only in HallAB_2017 — and the E3 list is to be treated as provisional until a second cMD-IBD sub-study populates E3 with ≥ 10 CD and ≥ 10 nonIBD samples 1. Within the stricter E3 × HallAB_2017 test, F. plautii replicates at +2.26 (FDR 0.02) while Blautia wexlerae does not replicate (+0.25, FDR 0.80) and is removed from the rock-solid set, the earlier triad having rested on cohort-level rather than E3-restricted within-substudy evidence 1. This contradicts the digest’s “3 candidates, E3 only” framing in both direction and size: tightening the design did not shrink E1 to nothing but produced a classical-pathobiont E1 list, and it removed one member of the three-species set the digest called trustworthy.
Tensions
Does compositional correction fix direction errors, or only some of them? One side: raw Mann–Whitney mis-directs 4+ of 8 protective species and CLR recovers them, with Roseburia intestinalis flipping outright from raw CD↑ to CLR CD↓ 1. The other side: C. scindens stays CD-enriched under both methods (raw log₂FC +5.66; CLR Δ +1.25), so compositional correction alone is insufficient there and strain heterogeneity and ecotype mixing remained live explanations 1. The project’s §4 interprets the surviving call as cohort-composition behavior — a species more prevalent in the IBD-cohort source studies than the healthy-cohort source studies 1 — but that interpretation is not established by the revised analysis, which finds the enrichment persisting within sub-studies after the study confound is removed (pooled CLR-Δ +1.18, FDR 1e-8, 4/4 sign concordance) and calls the species genuinely CD↑ 1. Between-study prevalence alone therefore cannot be the explanation, and which transformation to trust in general remains unsettled.
Which stratification is the target list? The digest’s within-ecotype DA yielded 33 candidates across both E1 and E3, and its stricter filter requiring within-substudy and LinDA agreement left 3, E3 only, declaring the within-ecotype list not usable 2. The revised within-ecotype × within-substudy meta-analysis instead reports 51 E1 candidates across two eligible sub-studies and 40 provisional E3 candidates from one 1. The sides are not averaged: the later, design-controlled count replaces the earlier one, with E3 explicitly provisional pending a second eligible sub-study 1, which also bears on whether the ecotype partition itself is stable — see ecotype-clustering-validity.
Open Directions
- Obtain a second cMD-IBD sub-study that populates E3 with ≥ 10 CD and ≥ 10 nonIBD samples and re-run the E3 × within-substudy test, to convert the 40 provisional E3 candidates into replicated calls or retire them 1.
- Re-test the former E3 triad under the stricter design on new data, since F. plautii replicates at +2.26 (FDR 0.02) while B. wexlerae does not (+0.25, FDR 0.80), to establish which of the digest’s three candidates survive at all 1.
- Rebuild ecotypes on a functional (pathway / KEGG) feature matrix to break taxonomic-feature leakage and re-run within-ecotype DA, to test whether the E1 and E3 candidate lists are stable to the feature space used for clustering 2.
- Run the combined UC-vs-HC + CD-vs-HC IBD-vs-control contrast proposed for power, and compare its call list against the design-controlled E1 and E3 Tier-A lists to see whether broader contrasts recover candidates or only add noise 2.
- Extend the raw-vs-CLR-vs-LinDA comparison beyond the 14-species curated battery of
NB00_protective_species_paradox.pngto the full species table, to measure how often sign flips occur outside hand-picked examples 1. - Separate strain heterogeneity from cohort composition for C. scindens with strain-level profiling inside single substudies, the explanation that compositional correction could not exclude and that the persistent within-substudy enrichment leaves open 1.