When pasting together the ROBIN summary and detailled results, including adjusted p values, I noticed that the 1, 0, -1 denotifiers in the summary were not in all cases conform with the respective fold change values and p values that I used as a cutoff. Especially by comparing metagoups I noticed that the 1,-1 denotifiers identified every gene as significantly regulated that was changed more than 2-fold, independent of the corresponding p value. Is there a statistical reason to do so, or is this a bug in ROBIN?

Dear Maik,

no there would be no statistical reason to do just use p-values for the flag calculation. There could however be some explanations for this.
However, can you please post a bit more details, which p-value decision strategy you used and how you identified p-values?
(this would be the expert tab in Robin)
And maybe just a few lines of you result file without your identifiers, so I get a clearer picture of the problem

Best Wishes,
Björn

Hi Björn,

thanks for your answer. The parameters for the analysis were the following:

# Normalization settings for quality control
normalization method: rma
P-value correction method: BH
analysis strategy: Limma

# Normalization settings for main analysis
normalization method: rma
P-value correction method: BH
Multiple testing strategy: nestedF
P-value cut-off value for significant differential expression: 0.05
Genes that showed a log2-fold change smaller than two ignored: yes

I analyzed a hormone treatment in the presence and absence of an inhibitor. Comparisons that were made were hormone vs. control, inhibitor vs. control, Hormone and inhibitor vs. inhibitor and Metagroup1 (hormone vs. control) vs. Metagroup 2 (Hormone and inhibitor vs. inhibitor). In the latter comparison the adjusted p-values did not fit the denotifiers in the summary tab.

Robin_results.txt.annotated (selected lines)

Hormone-control Hormone_and_inhibitor-inhibitor (Hormone-control)-(Hormone_and_inhibitor-inhibitor) Inhibitor-control

250500_at -8.535981648 -1 -5.99297812 -1 -2.543003528 -1 1.354121429 1
247522_at -1.193693344 -1 -0.008196252 0 -1.185497093 -1 0.981509861 0
267207_at -1.461743331 -1 -0.429277012 0 -1.032466319 -1 0.771483775 0


but if I look into the detailled results fr the metagroup comparisons, the p values are >0.05.

full table (condition1-control)-(condition3-condition2) (selected lines)

ID logFC AveExpr t P.Value adj.P.Val B

250500_at -2.543003528 8.353980695 -2.302276235 0.04167254 0.303593302 -4.253660369
247522_at -1.185497093 6.013773592 -1.486087537 0.165105914 0.553263683 -5.484515286
267207_at -1.032466319 7.507863881 -1.821346703 0.095598608 0.441328528 -5.012205202

For now, should I just ignore the denotifiers and filter the results of the metagroup comparison by the adjusted p value<0.05?

Thanks,

Maik

Dear Maik,

thanks for providing the details of your analysis. The difference in the
p-values that you are observing is caused by the fact that the underlying
limma methods decideTests and topTable can give different results depending
on the multiple testing strategy used. In your case you have used "nestedF"
for the main analysis which is the standard setting in Robin and a good method
to identify genes that respond simultaneously in several contrasts. Such genes
(when using nestedF) might be called significantly differentially expressed
even if the p-values for these genes in the individual separate contrasts are
above the threshold.

The topTable method used to generate the topTable lists for
each contrast defined, on the other hand, always gives the p-values for
each selected contrast separately (unless more than one contrast is selected,
but this is not implemented in Robin) analogous to using the method "separate"
as the multiple testing strategy for decideTests. Hence the discrepancy in your
analysis. If you selected "separate" in your analysis the p-values in the topTables
and the flags in the result file would be in agreement.

Concerning your question whether you should ignore the flags and filter only based
on the p-values in the topTables, the answer depends on your experiment: The nestedF
strategy is better at finding genes that respond to more than one condition while
it might not well pick up genes that respond to only one. Please see the Robin manual
section 3.1.2.2 and the limma user's guide section 10.3 for more details.

The bottom line is (as stated in the discussion referenced below) that different
analysis methods give different results. Depending on what your experimental
question is one method might be more suitable than the other but it might also
be a good idea to try several and compare the results to build a consensus set
of differentially expressed genes.

Best greetings,
Marc

Please see also this related forum discussion:
https://stat.ethz.ch/pipermail/bioconductor/2006-June/013420.html

Thanks Marc,

that answered my question. I did not realize there were two different algorithms working in parallel. Luckily functional classification of the resulting genes led to the same pathways under both multiple testing correction strategies, so whatever the 'real' setof genes, the genes I was interested in were always included.

Best regards,

Maik