gms | German Medical Science

GMDS 2012: 57. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie

16. - 20.09.2012, Braunschweig

A simulation study on the potential of permutation tests for differential network analysis

Meeting Abstract

  • Miriam Lohr - TU Dortmund, Deutschland
  • Marco Grzegorczyk - TU Dortmund, Deutschland
  • Jörg Rahnenführer - TU Dortmund, Deutschland

GMDS 2012. 57. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS). Braunschweig, 16.-20.09.2012. Düsseldorf: German Medical Science GMS Publishing House; 2012. Doc12gmds146

doi: 10.3205/12gmds146, urn:nbn:de:0183-12gmds1469

Published: September 13, 2012

© 2012 Lohr et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.



The reconstruction of genetic networks is a major goal and particularly difficult task in systems biology. An even greater challenge is to identify the differences of a genetic network in two different biological settings. For example, one might be interested in inferring the network differences between two groups of patients with different staging. In this situation it is highly relevant to study which regulatory or signaling pathways break down in a certain disease subtype or in an advanced disease stage, as such insights open opportunities to develop drugs for the corresponding patient groups.

This setup requires first estimating both networks accurately and then quantifying the differences with a suitable measure. Important parameters for the expected success of this approach are the level of noise in the data, the numbers of patients in the different disease subgroups, the number of genes belonging to the pathway of interest, the number of additional potential confounder genes in the considered gene set, and the magnitude of the network differences between the settings being compared.

We carry out a simulation study based on the topological structure of the MAPK/ERK pathway, as discussed in Sachs et al. (2005) [1]. Our gold standard obtained from this paper is a network of 11 nodes and 20 edges. For one patient group we generate data from this theoretical pathway, for another patient group we assume some edges to be deactivated. We vary all parameters mentioned above to assess their influence on the ability to identify the predetermined differences. In this controlled scenario it is possible to objectively compare the discrimination ability of different measures that quantify the difference between two genetic networks.

Our candidate measures are based on estimated correlations and partial correlations between edges in the network, with different degrees of robustness, for example by considering ranks, as introduced in Lohr et al. (2010) [2]. We consider these measures as test statistics and apply permutation tests evaluate the difference between the two compared biological settings. Depending on the number and position of the disabled nodes in the disease group, the noise level, the number of patients in the two settings, and the size of the observed network, we analyze the level and power of the respective tests. As a result we can order the measures by their discrimination quality as well as quantify the values for the different parameters that are required to achieve a desired performance.


Sachs K, Perez O, Peèr DA, Nolan GP. Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data. Science. 2005;308:523-9.
Lohr M, Godoy P, Hengstler JG, Rahnenführer J, Grzegorczyk M. Extracting differential regulatory sub-networks from genome-wide microarray expression data. In: 7th International Workshop on Computational Systems Biology; 2010 Jun 16-18; Luxembourg, Luxembourg. Tampere: TICSP series # 51; 2010. p. 63-6.