Software Tools for Visualization and Analysis of Human Brain Data for Language Mapping

Andrew V. Poliakov, PhD, Rex M. Jakobovits, PhD, Richard F. Martin, PhD, David P. Corina, PhD, James F. Brinkley, MD, PhD
Structural Informatics Group, Department of Biological Structure, University of Washington
Department of Psychology, University of Washington
Department of Radiological Sciences, UCLA and Workhost.com
Abstract
As a part of the Human Brain Project, the Structural Informatics Group at the University of Washington is developing software tools for processing, integrating and visualizing multimodality data for language mapping. These data are primarily collected in preparation for, and during neurosurgery for intractable epilepsy. The epilepsy cases present a unique opportunity for better understanding of language function of the brain, as individual patients can be studied both using cortical stimulation of language areas during neurosurgery, and using functional MRI prior to surgery. Scientific implications of these studies for understanding of language function in the brain are discussed in companion presentations (D. P. Corina, R. F. Martin).
In this technical demonstration, we will present software tools developed by this group to aid these tasks. The tools were designed as general-purpose toolkits that provide a flexible environment for developing focused applications for processing and presenting particular types of data.

Skandha4
Skandha4 is a general purpose graphics toolkit, which combines a subset of Common Lisp - useful for fast interactive programming and prototyping - with the ability to add pre-compiled C-based primitive functions that significantly accelerate computationally demanding routines. Skandha4 supports 3D graphics, with drivers for IRIX GL and OpenGL available. It includes a module for processing and storing MRI data, with support for the Montreal MINC 3-D image volume file format added recently. Server capabilities make it possible to use Skandha4 as a backend graphics server for web-based applications, in which the server is used to generate images from 3D models to support interactive visualization on the web.

Skandha4 is currently used as a graphics engine for several applications: 1) Brain Mapper (Hinshaw et al., 1997) is an application that implements complex workflow of data processing for language mapping. The steps include aligning MRI data - structural, veins, arteries and functional, isolating the brain, reconstructing 3D models, and interactively integrating cortical stimulation sites with the 3D models using intra-operative photographs of the cortical surface; 2) Dynamic 3D scene navigation in web-based anatomy atlases (Wong et al., 1998) utilizes Skandha4 as its backend graphics server; and 3) Functional MRI visualization and analysis tool (Poliakov et al., 1999) is yet another example of using Skandha4 toolkit.

WIRM
The Web Interface Repository Manager (WIRM) is a Perl-based application server that provides a high-level programming environment for developing web-based multimedia database systems (Jakobovits et al., 1996). WIRM consists of an object-relational database, and a suite of Perl interfaces for visualizing, integrating and analyzing heterogeneous multimedia data. It provides facilities for creating context-sensitive views over a multimedia database, allowing developers to rapidly build dynamic web sites that adapt their content and presentation to multiple classes of end-users. These capabilities are illustrated with a Lab Library Demo - a system for loaning and returning books in the office, and by our Brain Mapper Experiment Management System. The Brain Map system is currently used by neuroscientists and clinicians to share language-related experimental data on the web, and to manage the workflow of complex data analysis systems for studying language.

Current Work
In our current work we continue improving performance and expanding features, as well as developing applications that utilize both the Skandha4 and WIRM toolkits. In particular, a web-accessible implementation of the Skandha4 functional MRI visualization and analysis tool (Albright, unpublished) has been prototyped using both an HTML forms-based and a Java-based approach. We are also working to integrate the 3D graphics server capabilities of Skandha4 with the web and database capabilities of WIRM, the result of which will be a visualization and management system for multimedia brain data that can be accessed from a simple web browser.

References

K. P. Hinshaw and J. F. Brinkley, Using 3-D shape models to guide segmentation of MR brain images, Proceedings, American Medical Informatics Association, Nashville, Tennessee, 1997.

B. A. Wong and J. F. Brinkley, Dynamic 3-D scene navigation in web-based anatomy atlases, Proceedings, American Medical Informatics Association Fall Symposium Orlando, Florida, 1998.

A. V. Poliakov, K. P. Hinshaw, C. Rosse and J. F. Brinkley, Integration and Visualization of Multimodality Brain Data for Language Mapping, Proceedings, American Medical Informatics Association Fall Symposium Washington, D.C., 1999.

R. M. Jakobovits, B. Modayur, and J. F. Brinkley, A Web-based repository manager for brain mapping data. Proceedings, American Medical Informatics Association Fall Symposium Fall Symposium Washington, D.C., 1996.