gms | German Medical Science

50. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds)
12. Jahrestagung der Deutschen Arbeitsgemeinschaft für Epidemiologie (dae)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie
Deutsche Arbeitsgemeinschaft für Epidemiologie

12. bis 15.09.2005, Freiburg im Breisgau

Image Processing for Medicine

Meeting Abstract

Search Medline for

  • Maria Petrou - Centre for Vision, Speech and Signal Processing, Guilford

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. Deutsche Arbeitsgemeinschaft für Epidemiologie. 50. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds), 12. Jahrestagung der Deutschen Arbeitsgemeinschaft für Epidemiologie. Freiburg im Breisgau, 12.-15.09.2005. Düsseldorf, Köln: German Medical Science; 2005. Doc05gmds633

The electronic version of this article is the complete one and can be found online at:

Published: September 8, 2005

© 2005 Petrou.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Image Processing may contribute to Medicine at three levels of increasing innovation: Allow the clinician to see what they wish to see; allow the clinician to measure things they want to measure; allow the clinician to see beyond the limitations of the human vision system. Inspite of the widespread use of Image Processing in many medical tasks, significant challenges remain, in particular with the adaptability of Image Processing systems to the varying imaging conditions within the same clinical environment, yet alone across different establishments.



Image Processing is concerned with the process of images with four major objectives:

To make images appear better in a subjective way. This is called image enhancement and it has several application areas including Medicine.
To make an image looking better in an objective way, by reversing the damage the image might have suffered during the process of capture or transmission. This is called image restoration.
To compress an image so it needs fewer bits to be stored or transmitted. Image compression may be lossy, when part of information in the image is lost because it is deemed unwanted or incorrect, or it may be lossless, when no information in the image is lost. In the former case very high image compression may be achieved, but the information that is left out is application specific. So, such an approach is not acceptable for long term generic image storage, as information that appears irrelevant at some stage in time, may become very relevant at a subsequent occasion.
To prepare an image for further processing by a vision system, by making explicit, for example, information which is implicit in it. Such information may be the identification and salienation of region boundaries in an image, or the segmentation of the image into distinct regions, each of which might subsequently be processed separately.

What Image Processing does not do is to proceed to a fully automatic vision system which will replace the human in a generic vision task. This is the subject of Computer Vision research. However, Image Processing often uses methodology from Pattern Recognition to identify and label specific regions in an image for specific tasks.

All aspects of Image Processing research may have significant input to the Biomedical Sciences. In particular, I will distinguish three levels at which Image Processing may contribute not only to clinical practice, but also to clinical research and knowledge discovery:

Image Processing may be used to allow the clinician see better what he/she wants to see. Such approaches are widely used in daily clinical practice.
Image Processing may be used to allow the clinician to measure what he/she wants to measure in an image. Such approaches are often available nowadays to the clinician, but they are not as widespread in practical applications as one might have thought.
Image Processing may be used to extent the capabilities of the clinician beyond the limitations of his/her human vision system. Such approaches are not currently in use. One might consider that the use of X-rays is an extension of the capabilities of the clinician beyond the limitations of the human vision system. However, I consider such advances advances of sensor development than advances of Image Processing. Here I mean limitations concerning the way our brain processes and sees pictures rather than limitations imposed by the structure of our visual sensor.

In what follows I will discuss each of the above stages in turn and I will give some examples of current practice and some challenges that lie in the future for each one of them.

To see better what we want to see

Image enhancement is one of the most widely used applications of Image Processing. Ultrasound images for example are routinely enhanced before being displayed. Some less well known applications concern the bias field correction in NMR imagery or the improvement of the resolution of a gamma camera using Wiener filtering. Figure 1 [Fig. 1] shows an example of a liver MRI image before and after bias field correction. Wiener filtering may increase the resolution of a gamma camera by about 30%.

The real challenge, however, is the development of automatic or semi-automatic techniques to extract organs from 3D data and build models from them which the clinician may manipulate for pre-surgical planing. Several such approaches have been tried and they work to a lesser or greater extent for certain image modalities. However, many challenges remain. For example, the extraction of the liver from MRI data, with all its vascularity in a reliable and repeatable way, and with no omission or commission errors (Omission error is when part of the organ is left out of the extracted region, while commission error is when tissue that does not belong to the organ is extracted as part of it.). The problems arise from many sources: Not just that the liver is different for different people, not just that the liver is in an area of the body congested with other nearby organs with similar properties as far as NMR imaging is concerned, but also the fact that the creation of an image is a synergy between the imaged object and the imaging device and environment: The created image depends very much on the scanner used, the imaging sequence employed, the settings chosen, the noise induced by the ambient environment etc. Really robust methods that can meet these challenges and be able to cope with such variations in clinical practice constitute a holy grail far in the horizon of Image Processing.

To measure objectively what we want to measure

A lot of clinical processes involve the quantification of observations made in images. For example, the clinician wants to know the volume of a tumour they see, or the pathologist needs to estimate the percentage of pleomorphic cells present in a slide. In some cases such estimates are crucial for the subsequent management of the patient. Image Processing may play a very significant role here, with the ability to produce results that are objective and repeatable. However, again the challenge remains: Can we have a system that can cope with different staining protocols followed by different laboratories? What about the freedom of the clinician to capture images at different resolutions and play with the white balance on the colour device? A dynamically adaptable system in the future should be able to cope with such changes with the minimum prompting from the user. An intelligent system will be able to identify the changes and adapt itself to operate under the new conditions. Such systems however, remain in the horizon of challenges for the future.

If we return back to the measurement of the volume of lesions, things are perhaps more tractable there: Tissue is an elastic medium that changes its shape with time. Such changes are more dramatic in malignant situations. To be able to extract the 3D volume of such an object and compare it with its previous observation is not trivial. Such problems are closely related to the problem of image registration. For example, to register the patient's body with the position it had during the previous radiotherapy session, so that the radiation is directed exactly where the tumour is, constitutes another non-trivial problem. In case of brain the problem has been routinely tackled by assuming rigid body transformation. The existence of the skull helps significantly in this case. However, in case of prostate cancer or cancers in other parts of the body the problem cannot really be tacked under the assumption that we deal with rigid bodies. Elastic image registration techniques are required. Such approaches effectively have to assign to each voxel a vector indicating its direction of movement from one imaging session to the next. Finding the optimal combination of motion vectors for all objects is an optimisation problem with as many unknowns as voxels in the image and it has more possible solutions than there are seconds in the life of the Universe! Exhaustive search for such solutions is beyond contemplation and stochastic optimisation approaches have to be used. Such approaches are notoriously slow, and therefore achieving optimal elastic volume registration in real time is a real challenge. One at the moment can only hope to accelerate the process significantly, eg to make it achieve good sub-optimal solutions in minutes as opposed to hours [1]. Achieving such registrations further allows the quantification of the deformation suffered by the patient over time, in ways that are different from the conventional way of measuring volumes. For example, it is possible to use the values of the objective functions used to perform the image registration here, as an extra indicator for the state of the patient. And this, brings me to the real challenge which I will discuss in the next section: Can we convince the clinicians to adopt something that is the result of processing that cannot be explained in terms of direct human perception, as a useful indicator for a familiar condition?

To see things we cannot see

The clinician who deals with real people every day is a practitioner. He deals with the real world and not the abstract world of mathematics. The clinician believes his eyes and his brain. It is very easy then to forget that both the human vision system and the human brain have their limitations in what they can see. Figure 2 [Fig. 2] shows three pairs of two regions next to each other. In the first case, the regions are made up from intensity values that have different mean, but the same standard deviation. In the second case, the two regions are made up from points that have the same mean but different standard deviations. In both these cases one may easily discern the boundary between them. In the third case the regions are made up from pixels that have the same mean, the same standard deviation, but different skewness. The boundary between them cannot be discerned now. In all three cases the difference in the mentioned statistic between the two regions was chosen to be as strong as the value the statistic had for one of the regions. In MRI data in the regions around malignant tumours the distribution of recorded values are clearly skewed. Is it possible then, that the invisible diffuse boundary of the tumour may be detected by looking for changes in the skewness of the distribution of the data along rays as we move away from the visible tumour? The answer is "maybe". Such boundaries have been detected, but detailed clinical studies are required to establish whether they correspond to boundaries that are physically related to the actual tumour [2].

If the human eye has such difficulties when coming to 3rd order statistics, then one can appreciate its limitations when it has to perceive variations in volume. Our vision system has not been constructed to see inside volumes, yet alone to measure how the data vary in 3D. Here one has to invoke more advanced tools of Image Processing, namely texture analysis in 3D [3]. Such analysis may reveal things that cannot be seen otherwise. For example, it has been known for some time that the shape of sulci of schizophrenic patients shows differences from that of normal controls. But what about the tissue structure that makes up the grey matter? Careful 3D texture analysis of MRI data concerning such cases revealed that the structural difference goes deeper than just the shape of the surface of the sulci and in fact statistically significant differences in the structure of the tissue of the two groups were identified in a recent study [4]. The real challenge then starts here: Can we make the connection between the measurements Image Processing may yield, with what is going on inside the body at the cellular or even the molecular level? Can we make the connection between the macro-scale of direct observation, the meso-scale that Image Processing is beginning to poke, and the micro-scale that molecular and cellular biology can tackle? The first level of connection is already happening: Image Processing practitioners may not understand disease, but they may easily test for correlations with diagnostic tools currently in use. For example, correlation was found between the score attained in the Mini Mental State test dementia patients take and the anisotropy measured in their brain MRI scans [5]. At the other end of the scale, such correlations are not easy, because they require specific studies and careful collection of relevant data.

In conclusion

Image Processing is playing and can play a very important role in clinical practice and biomedical research. It faces a lot of challenges if it is to become more wide spread. It can offer objectivity when subjectivity may be an issue, but most of all it has the potential to broaden the horizons of our understanding by discovering associations and information in the data that is only implicitly there and has to be made explicit in order to be appreciated.


V A Kovalev and M Petrou, 1998. "Non-rigid volume registration of medical images". Journal of Computing and Information Technology, Vol 6, pp 181-190.
M Petrou, Vassili Kovalev and Jiirgen R Reichenbach, 2005. "3D Non-linear invisible boundary detection". IEEE Transactions on Image Processing, under review.
V Kovalev and M Petrou, 2000 Chapter 15: "Texture analysis in Three Dimensions as a cue to Medical Diagnosis", in Handbook of Medical Imaging, Processing and Analysis, I Bankman, editor, Academic Press, ISBN 0-12-077790-8, pp 231-247.
V Kovalev, M Petrou and J Suckling, 2003. "Detection of structural differences between the brains of schizophrenic patients and controls". Psychiatry Research: Neuroimaging, Vol 124, pp 177-189.
M Segovia-Martinez, M Petrou and W Crum, 2001. "Texture features that correlate with the MMSE score". Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, October 25-28, Istanbul, Turkey .
E Kokkinou, K Wells and M Petrou, 2003. "Digital Autoradiography Imaging Using Direct Irradiation of a CCD Between 278 - 309K". IEEE Transactions on Nuclear Science, Vol 50, No 5, pp 1702-1707.