gms | German Medical Science

The radio frequency ablation is a gentle method in the therapy of primary and secondary liver tumours. It can be performed percutaneously. This however involves aiming at a moving soft tissue target inside the body. Conventionally, the ablation needle is placed into the tumour under guidance of an imaging method, such as computer tomography (CT), magnet resonance imaging (MR), or ultrasound (US) [1]. Most tumours (98%) are treated under ultrasound guidance [2]. In our opinion ultrasound is the preferable intra-interventional imaging method, because it keeps the patient's exposure to radiation low and is a low cost alternative to radiological imaging.

Several factors influence the local recurrence rate [2], tumour-dependent, e.g. size, type, location, and physician-dependent, e.g. experience, accuracy, spatial sense. One approach to increase the orientation and thus the precision of percutaneous interventions is to intra-interventionally use planning data, such as access route, target points, critical vessels and others. In the FUSION project (Future Environment for Gentle Liver Surgery Using Image-Guided Planning and Intra-Operative Navigation) in this sense we develop an intervention assistant for percutaneous, ablative methods in the therapy of liver tumours. The intervention assistant is a navigation system with facilities to introduce pre-interventional planning data and imaging for this purpose. It is essential to align the pre-interventional data with the intra-interventional data of the patient. To do so a geometric reference system, introduced via registration, is needed.

We carried out clinical evaluations of an interactive registration using external landmarks together with our project partner, the Department of Gastroenterology, Hepatology and Infectiology of the University Düsseldorf. This simple registration is very error prone. In many cases, however, the doctor is able to compensate the misalignment in his/her actions by identifying corresponding structures. But the analysis of liver therapies also revealed situations where the doctor could not compensate the registration error and no corresponding structures were found.

The aim of this work was the development of a more accurate, automatic registration. It is an addition to the simple registration using external landmarks. Within the FUSION project we evaluate the potential of higher registration accuracy to increase the spatial orientation of the intervention.

To register the pre-interventional imaging to the intra-interventional images several general conditions have to be considered. The algorithm has to cope with two very different image modalities: on the one hand the high resolution pre-interventional tomography images and on the other the noisy intra-interventional ultrasound images. Other challenges are the real-time requirements in the intra-interventional situation and the restricted possibility of the doctor to operate a computer during the intervention. Furthermore the algorithm should be intuitive to improve the acceptance of the whole support system.

To achieve a stable registration the idea is to bring as much information into the registration process as possible. Conventional registration methods used in this application area [3] are exclusively based on grey-value information with unstable results. For additional usage of structural information, a structure has to be identified that is stable and observable in both 3D image data sets. In our application the vascular tree of the liver turned out to be the only structure meeting these conditions [3].

We extract vascular models from both 3D image data sets that are graph based models, containing information about the branching points and the width of the corresponding vessel. These models are matched and thus the whole registration process is realized as a model-to-model approach.

The registration algorithm divides up into two major steps:

1.

Extraction of graph models representing the vascular system from the interventional 3D ultrasound and from the pre-interventional radiological imaging, respectively, called vascular graph models

2.

Iterative matching of the vascular graph models and corresponding calculation of the registration transformation

The vascular system can be extracted in very detail from the high resolution pre-interventional radiological image data. This is done by our project partner, MeVis AG, Bremen using a segmentation-based method [4]. The vascular system is modelled as a graph structure containing information about the branching points and the radius of the vessels.

The extraction of the vascular graph model from the 3D ultrasound is more difficult because ultrasound is very noisy. Furthermore the 3D ultrasound volume is build from slices of a 2D ultrasound that is manually moved and tracked in space. There are often gaps within the volume corrupting anatomical structures. We use 3D Doppler as well as contrast-enhanced ultrasound. The vessels are represented by areas of high intensity marked by a light grey value. To extract them, an extended and modified Growing Neural Gas [5] approach, called Vessel Extracting Gas (VEG) [6], was developed.

The following pre-processing and initialization for the extraction is done in the VEG:

1.

In a first step a set of voxels potentially representing areas of the vessels is extracted semi-automatically. The doctor adjusts a threshold in the ultrasound interactively. The VEG then identifies all voxels exceeding this threshold as candidates to belong to the vessels, called vessel voxels.

2.

The graph model is initialized by the VEG using two randomly chosen vessel voxels as potential branching points and an edge joining them. The VEG then assigns an initial width to the potential branching points using the average width in the graph model extracted beforehand from the radiological images.

The actual automatic extraction process of the VEG is an iteration of two major steps:

1.

The adaption of the potential branching points:
a) Random choice of a point P from the set of vessel voxels for next calculation step
b) Identification of two potential branching points, according to the Euclidean norm nearest to the point P, for adaption
c) Adaption of the nearest potential branching point and its topological neighbours in the current graph model in their position (towards P in the Euclidean space)
d) Adaption of the width of the nearest potential branching point (according to its current width and its Euclidean distance to P [6])
e) Adaption of the structure of the whole graph model joining the two nearest potential branching point and incorporating an ageing mechanism (see [4], [5])
f) Adaption of the error value at the nearest potential branching point (according to the Euclidean distance to P) to identify those areas better represented using additional potential branching points

2.

The insertion of additional potential branching points:
a) Regularly after a given number of adaptions of potential branching points:
Introduction of a new potential branching point into the graph model near the potential branching point with highest error value
b) Initialization of its edges to other potential branching points, its error value and width

As a criterion to stop the iteration a final number of potential branching points is used. The width of the branching points is averaged to get the width of the edges. Thus we get a graph model containing information about the branching points and the width of the vessels.

The doctor can now judge the quality of the model extracted by the VEG as representative of the reality according to the structures seen in the ultrasound. The model is displayed together with the set of vessel voxel. If the model is unrepresentative, i.e. if branching points are missing, if edges lie outside the vessels or if the width of the edges is wrong, the doctor has the possibility to start the extraction again with different parameter settings. If otherwise the doctor is pleased with the representation he/she can start the iterative matching of the vascular graph models.

This procedure of evaluative selection by the physician is not possible for the radiological vascular graph models, as the extraction is not done on-site.

The two graph models extracted from the 3D radiological image data and the 3D ultrasound are usually very different from one another. The model extracted from the high resolution radiological data is a detailed and total model of the real vascular system of the liver. The 3D ultrasound is prone to artefacts and gaps and comprises a small region of interest. The model build from the ultrasound is thus coarse and fragmentary compared to the real vascular system of the liver.

To cope with these differences we developed the Transform Learning (see [7]). In an automatic process the vascular graph models are iteratively registered. One vascular graph model is used as reference model, whereas the other (the moving model) is moved towards the reference. In a first evaluation we used the ultrasound as reference model and the radiological graph model as moving model. The effects of this choice were not evaluated yet.

As precondition for the Transform Learning an initial alignment T of the models from an interactive preregistration using external landmarks has to be created by the doctor.

The automatic learning process iteration consists of two major steps [7]:

1.

The “virtual” adaption of the branching points:
a) Random choice of a branching point P of the reference model for the next calculation step
b) Identification of branching point in the moving model transformed with T nearest to P, according to the Euclidean norm
c) “Virtual” adaption little towards P in the Euclidean space: The adaption is based on the “Winner-takes-all” principle [8] as well as on the step-wise adaption in the Growing Neural Gas by Fritzke [5]. It is called “virtual” because the position of the branching points does not change directly, but the old position and the “virtual” new position, respectively, are saved as corresponding movement positions.

2.

The adaption of the transformation:
a) Regularly after a given number of “virtual” adaptions of branching points:
Passing of corresponding movement points calculated by adaptions to landmark based registration method: In our case a Least-Square-Estimation is used. In this point of calculation the corresponding movement points act as corresponding landmarks for a small movement of the moving model towards the reference model.
b) Calculation of new valid transformation: The landmark based registration method (Least-Square-Estimation) returns a transformation adjustment ΔT that is appended to the last valid transformation.

This iteration is repeated until the virtual movement of the branching points falls below a specified threshold. In this case, the last valid transformation is applied to the complete moving model or even to the complete image from which the moving model had been derived.

Two mechanisms reduce the probability for the algorithm to fall into a local minimum. Our experience shows that moving the branching points just a little towards the nearest reference branching points reduces the influence of outliers. Averaging introduced through the collection of “virtual movements” before really adapting the transformation furthermore reduces this influence. Thus Transform Learning can also be used in cases where the two graph models differ significantly from one another.

There are two criteria for the evaluation of the registration:

• Mean accuracy of the registration:
  • For artificial ultrasonic image data sets this can directly be measured because the registration transformation is known. The reference model is given by the vascular graph model extracted from the radiological image data whereas the vascular graph model extracted from the ultrasound is used as moving model. The accuracy is measured as distance of the transformed moving branching points to the reference branching points.
  • For real data this judgement is not as straightforward. It can at least be done qualitatively. Bad registrations can be characterized by visible deviations (see Figure 1 [Fig. 1], left side). Good registrations will be identified if the structures are congruently aligned with each other (see Figure 1 [Fig. 1], right side). The actual success criterion in the target application is a gain of orientation for the doctor. To evaluate this, an environment supporting the visual evaluation of the registration quality is currently under development.
• Run time performance of the registration: In the target application described in chapter 1 the calculation has to be nearly real-time.

The extraction was extensively tested on artificial data sets. It showed good representations of the actual branching points as well as of the width of the vessels.

The whole process including extraction and matching was also extensively tested using artificial 3D ultrasound data (see [5], [6]). Registration results were achieved in about 1.8 s including extractions. A mean distance of about 1.2 mm between the ideal transformed branching points and the branching points transformed using the calculated registration transformation was reached.

These results are encouraging but put into perspective by the following aspects:

• The artificial ultrasound data did not incorporate noise or other artefacts.
• The artificial ultrasound data did not incorporate deformations, only rotations and translations.
• A registration accuracy measured in mm may not be critical for the orientation of the doctor.

In this sense the tests on artificial data can be seen as prove of concept. It was important to preliminary evaluate the registration on real data.

Preliminary tests of the extraction via VEG using real 3D ultrasound data, two Doppler as well as one contrast-enhanced ultrasound image, also produced acceptable representations. An example of the contrast-enhanced ultrasound is shown in Figure 2 [Fig. 2]; the extracted model shown in green, the vessel voxels from the ultrasound shown in orange. The extraction proofed to be quiet stable against the artefacts and gaps usually present in 3D ultrasound data (see [6]).

The whole registration process in the medical target situation was preliminarily evaluated on real data sets (one from a test person study comprising an MR and contrast-enhanced ultrasound as well as two patient data sets comprising a CT and Doppler ultrasound).

We evaluated the registration process in two different ways:

• Before the interventional registration for all data sets a preregistration via external landmark registration was available as well as the radiological vascular graph model. For these data sets the vascular graph models from the ultrasound were extracted using the VEG. The quality of the extraction was judged by computer experts using a combined display of the vessel voxels from the ultrasound and the extracted graph model. If necessary the process was started again with different parameter settings. After a successful extraction the Transform Learning was calculated using different parameter settings. Again the quality was judged by computer experts using a combined display of the registered vessel voxels from the ultrasound and the radiological vascular graph model.
• For the test person study an additional evaluation by the medical expert was carried out. The MR data as well as the radiological vascular graph model were acquired beforehand. A setting similar to that in the interventional situation was build up, with the test person in interventional position. A 3D ultrasound was acquired that the medical expert then interactively preregistered to the MR of the test person using external landmarks. The extraction of the ultrasound vascular graph model, as well as the iterative matching of both vascular graph models followed. The doctor had the possibility to evaluate the registration using the display described above. Furthermore he could navigate in a combined view of the whole MR and the whole 3D ultrasound.

In the test person as well as in one patient study the process yielded acceptable results. Corresponding structures could be identified and it could be seen that they were situated in closely related regions of the common geometric space.

In a second patient data set there were two problems (see Figure 3 [Fig. 3]):

• The interactively gained preregistration was not acceptable as no correspondences could be identified in the subjective evaluation.
• The ultrasound only comprised two branches of a vessel.

Under these conditions the Transform Learning did not lead to a registration that supported the identification of corresponding structures in the 3D image data sets. The process would have required more structural information in the ultrasound to match both images.

In Figure 1 [Fig. 1] an example for a qualitative evaluation of registration results on the subject study data set is shown. The figure shows structures from ultrasound and radiological imaging, respectively. They are reduced to the vascular system for a better overview.

The orange coloured shape in both pictures is the surface of the set of vessel voxels from the ultrasound. In the left part of Figure 1 [Fig. 1] you see the vascular graph model extracted from the radiological data shown in red after it has been preregistered interactively to the patient and thereby the 3D ultrasound data set using anatomical landmarks. The model is translated and rotated in relation to the ultrasound vessel voxels. The right part of Figure 1 [Fig. 1] shows in yellow the radiological model registered with the introduced approach. Here the model is significantly nearer to the ultrasound vessel voxels.

The evaluation of the test person study in a simulation of the interventional situation made clear that the evaluation of the registration quality is difficult for the doctor with the given support environment (combined display of vessel voxel and radiological vascular graph model, combined display of whole MR and whole 3D ultrasound). Therefore we decided to develop and evaluate further technical possibilities to support the visual evaluation of the registration quality.

We presented a method to intra-interventionally register a pre-interventional 3D radiological data set to an intra-interventional 3D ultrasound data set. The method employs vascular graph model information extracted based on grey value information. Tests on artificial data were promising but are put into perspective by the artificiality of the data. In preliminary evaluations on real data computer experts qualitatively judged the registration result as acceptable. Medical expert evaluation revealed the problem, that the current environment for the support of the visual evaluation of registration quality is not intuitive for medical experts.

As a result we will further stress our work on an intuitive environment for the visual environment. If the registration results are stable and accurate this environment also provides a tool to inspire confidence in the method. That is a prerequisite for the integration of the registration into a clinical workflow. Furthermore reliable registration results render possible to include information gained from the pre-interventional imaging into the intervention process. This procedure has the potential to improve the reliability of the radio frequency ablation or other percutaneous interventions.

The presented work took place in the context of the project FUSION – Future Environment for Gentle Liver Surgery Using Image-Guided Planning and Intra-Operative Navigation – funded by the Federal Ministry of Education and Research (BMBF). We want to thank the LOCALITE GmbH – Biomedical Visualization Systems – for the support, especially concerning the LOCALITE SonoNavigator. We would also like to thank MeVis GmbH, a partner in the FUSION project, for the extraction of the vessels from radiological data sets. Furthermore we would like to thank our project partner – the Düsseldorf University Hospital – especially Dr. med Ralf Kubitz for image data and expert evaluation of achieved results.