Image Understanding and Graphics

A common problem in many of our projects is to extract anatomical objects from medical images, and to visualize these objects, as realistically and as efficiently as possible. Because of their rich and varied shape, biological objects pose many interesting and difficult challenges for researchers interested in image understanding and graphics.

The approaches we have taken can be divided into three categories: reconstruction from serial sections, 3-D region-growing, and shape-based surface models.

Reconstruction from serial sections is primarily a manual method which is useful for extracting the most detailed anatomy from images. Because it is manual it is useful only for one or two datasets. However, it still gives the most detailed models. Research issues include handling branching structures, handling structures with more than one contour on a single cross-section, and automation of the procedures.

3-D region growing is a low-level image segmentation technique that is useful for extracting organs with high contrast from the surrounding images. We have used it successfully for cortex extraction for the brain project, and it is our current method. However, it can often fail when the border between organs is not distinct, a common occurrence. And, like deformable models, it is controlled by a set of non-intuitive parameters that make it difficult for a non-expert to use. Research issues include adding higher level knowledge to compensate for deficiencies in the images, developing methods to learn the control parameters from training sets, and providing better user control.

Shape-based surface models attempt to capture shape and range of variation for various organ classes, and to use this knowledge to guide low level segmentation. We have shown that this approach is also useful for brain modelling. The main advantages of this method over the region grower are that it is much more intuitive for a non-expert to control, it can be more readily integrated in our brain mapper package, and it can be generalized. We are therefore in the process of replacing the region grower with this method. However, the models are not as detailed as those obtained from other methods, so the models must be combined with other methods. Research issues include generalizing the models to handle more classes of structures, providing better methods to backtrack in case of errors by the low level methods, and representing the relationships among multiple structures.