OsiriX: An Open-Source Software for Navigating in Multidimensional DICOM Images

The Open-Source Paradigm

The OsiriX software program is developed as a stand-alone application for the MacOS X operating system. It includes an image database that is updated automatically when new images are downloaded to a specific directory (Fig. 1). Images can either be pushed from the PACS using a DICOM “store” function or they can be “pulled” by a DICOM query-retrieve function of the program. Image files can also be manually copied from off-line media or from other network sources. The OsiriX software was developed based on an open architecture built on existing open-source components as described in Figure 2. The main components are these:

1. “GNUstep / Cocoa”^10,11: an object-oriented and cross-platform framework for the development of the graphical user interface. This framework allows the user to quickly design and develop complex graphic user interfaces. It is distributed in open-source format under the name of GNUstep. Cocoa is the name given by Apple Computer in the MacOS X environment. It is developed in Objective-C language.¹² Objective-C is an object-oriented language that has the benefits of C++ but without its complexity. Objective-C language also provides powerful memory management: it uses a retain/release scheme. It allows the user to automatically retain or release memory blocks. This prevents any undesired memory hugs and memory overflows (memory leaks) of the type that often occur in the development of graphic programs that manipulate large sets of image data. The open-source and cross-platform compiler “GNU Compiler Collection” (GCC)¹³ is used to compile this Objective-C framework.

2. “OpenGL”¹⁴: an industry standard graphic library for 3D image visualization functions. OpenGL was originally developed by SGI (Silicon Graphics) under the name of IrixGL and was only available to high-end dedicated Unix workstations that came with extremely high price tags. With the evolution of the 3D games market, OpenGL has been adapted by graphic boards manufacturers for the personal computing market, dominated by two main manufacturers: NVIDIA¹⁵ and ATI.¹⁶ All their graphic video boards are OpenGL compatible.

3. allows the user to take advantage of hardware acceleration by 3D graphic cards when available. It is the only truly cross-platform 3D library available that is designed for hardware acceleration. It is currently available for Linux, IRIX, Windows, and MacOS X; there is even an OpenGL version for the Palm operating system. OpenGL has been developed for 3D rendering, but it also performs extremely well in 2D rendering functions. On most operating systems 2D images are displayed faster through OpenGL than by using the standard graphic functions of the native operating system. By adopting OpenGL, large sets of over 1000 CT slices can be displayed in a few seconds.

4. “The Visualization Toolkit” (VTK)⁷: an object-oriented open-source and cross-platform library for 3D image processing and visualization that is widely adopted in the scientific community. This toolkit offers a large number of functions for the manipulation and display of 3D data sets. It is built over OpenGL to ensure hardware accelerated rendering on different platforms. VTK offers a complete set of functions for the manipulation and display of 3D data sets: multiplanar reconstruction (MPR), maximum intensity projection (MIP), volume rendering with transparency, and surface rendering with 2D or 3D texturing. VTK also offers a complete set of function to display and modify 3D vectorial graphics: it allows the display and modification of region-of-interest as well as other measures in 3D data sets. Finally, it fully supports all the blending functions of OpenGL to easily create 3D image fusion between two different data sets. VTK is actually the only true cross-platform and hardware accelerated 3D rendering library for personal computers. Under open-source licensing, VTK is used and developed by a large number of users, allowing its fast and robust evolution.

5. The “Insight Segmentation and Registration Toolkit” (ITK)⁸ is an extended set of libraries for specific medical image processing. It is an extension of the VTK library and is based on the same framework. It was developed to solve some of today’s problems in medical imaging such as image segmentation and multimodality image registration. It provides a full set of 2D and 3D processing algorithms adapted for these tasks. The different library components allow developers to easily add image fusion and organs segmentation in image processing software programs. This library is, like VTK, under active open-source licensing and benefits from updates and improvements provided by a large number of users.

6. “Papyrus toolkit”¹⁷ for DICOM files management (Fig. 1): This public domain library developed at the University of Geneva offers all the necessary functions to read and write DICOM files including all meta-data. This library is distributed with the entire ANSI C¹⁸ source code, easily adaptable to any platform. It provides a complete set of functions to manage the complex and cumbersome DICOM image file format. It allows for the extraction of image data but also for all numeric and textual meta-data associated to a DICOM file. The Papyrus toolkit is a convenient extension to VTK and ITK libraries.

7. “DICOM Offis”¹⁹ for DICOM network functions: This cross-platform library supports the DICOM communication protocols that make it possible to query, send, retrieve, and receive DICOM images within a PACS network. The complete C++ source code is also available. This library provides a complete set of functions that greatly facilitate the management and implementation of the extremely complex and convoluted DICOM network protocol.

8. “Altivec” of PowerPC²⁰ is a low-level assembly function for accelerating digital signal processing (DSP) functions. Because our goal is to provide the maximum level of acceptability and adoption by the users, we have emphasized the need for high performance and real time interaction. To achieve the best performance, we adopted Altivec as the only platform-specific and non-open-source technology that is available only for PowerPC platforms. We used this assembly language feature in very limited portions of our program to accelerate key features such as convolutions, image projections, and complex mathematical data conversions. We also wrote these functions in standard ANSI C¹⁸ in order to compile the project on non-PowerPC platforms. Altivec is a “Single Instruction, Multiple Data” (SIMD) unit of PowerPC processors. A SIMD system packs multiple data elements into a single register and performs the same calculation in all of them at the same time. It offers great performance when manipulating large sets of data such as 3D volume data. One disadvantage is the complexity of software development restricted to 128 bit registers for all operations. However, the advantage is the acceleration in performance of some functions by a factor of 10 or 20.

9. “Quicktime”²¹ for exchange of multimedia image file formats. Many commercial DICOM viewers or 3D workstations allow the user’s to see and manipulate images, but they lack a convenient way to export the images to standard multimedia file formats. This limits the ability to communicate the resulting images to other users and applications. To support the maximum possible number of existing image and graphic file formats in OsiriX, the cross-platform Quicktime library was used. It allows the export of any image data to all standard multimedia image formats such as TIFF, Photoshop, JPEG, JPEG2000, and BMP²² and any image sequence to movies file formats such as AVI, MPEG, and MPEG4. Furthermore we also used the QuickTime library to enable importing of non-DICOM images into OsiriX, thereby allowing users to benefit from the display and processing tools provided in OsiriX.

An important and challenging aspect of the development of this project was to integrate all of these technologies. OpenGL, VTK, ITK, DICOM Offis, Papyrus, and Quicktime are C/C++ cross-platform toolkits. These components were not designed to be fully compatible, and some inconsistencies and incompatibilities between header files or compiler flags made the integration somewhat difficult and laborious. Once past the steep learning curve in assessing the structure of the various components, the major advantage of the integrated product is the wealth of functions and features that are invaluable for any software developer of medical imaging applications.

The most important aspect of the project is that all the components function together within a simple and user-friendly graphic user interface. To achieve the best possible user interface design, we chose to develop the OsiriX program on a Macintosh platform to benefit from its well-known user interface features and convenience and ease of use. Additionally, the latest development environment provided by Apple greatly facilitates the rapid development of interactive graphic applications. The first version of OsiriX was developed in less than 6 months including complex 2D and 3D functions, DICOM files, and network management and multimedia file formats management. One of the key features of the latest Macintosh graphic user interface is the ability to interactively add or delete some of the software features by simply dragging icons from a retractable palette (Fig. 3) to and from a window toolbar. Each icon can represent a simple program function or a complex processing tool. This feature alone offers a major advantage over other computer platforms, allowing easy customization of the program adaptable to different users’ needs.

From Parallel Processing to Grid Computing

The OsiriX program offers all the basic image manipulation functions of zoom, pan, intensity adjustment, and filtering with real-time performance. Additional functions such as multiplanar projection, convolution filters, variable slice thickness adjustments, volume rendering, minimum and maximum intensity projections, and surface rendering are also accessible in quasi-real-time, depending on the hardware used, as well as on the number of slices to reconstruct. For example, on a dual-GS 2 GHz PowerMac computer, OsiriX can to render two images per second of a 400 CT slices set in volume-rendering mode. All these basic functions being handled by the OpenGL library essentially rely on the processing capabilities of the video hardware and require very minimal processing from the central processor unit (CPU). In essence the most basic image manipulations are processed in parallel by the video graphic processor unit (GPU), and the performance therefore does not depend on the computer processor unit (CPU).

The program enables the rapid review of very large sets of images and does not rely on pre-loading of the images from the disk to memory before review. As soon as an images serie is selected, images will automatically be displayed on the screen in a cine loop using a “streaming” technique, enabling direct display of the images at a very rapid pace. For example, on a G4 1 GHz mobile PowerBook computer, OsiriX is able to completely load a 355 CT images series in less than 10 seconds (loading rate: 35 images per second) while displaying the first image on the screen in less than 1 second. This streaming technique allows for an unlimited number of image series to be displayed simultaneously in separate windows. The software architecture was tested on very large image series consisting of thousands of images without any limitations. The OsiriX program was developed and mainly tested on the following standard off-the-shelf computers: PowerPC G4 laptop from Apple Computer, running at 1.25 GHz with 768 MB of RAM, with a built-in original 64 MB graphic board from ATI (ATI Mobility Radeon). Even on this standard laptop, the basic image manipulations were extremely fast and performed in real time without noticeable delays in response. For performance comparisons, we also implemented the software on a PowerPC dual G5 processors desktop computer from Apple Computer, running at 2 GHz with 1 GB of RAM and with a 128 MB graphic board from ATI (ATI Radeon 9600 Pro): performance was significantly better on the dual processor unit, with sometimes up to a factor of 6–10 faster performance for computer intensive tasks such as MIP and volume rendering, mostly due to the faster graphic board and dual processor architecture. The other basic functions of image manipulation and display such as intensity and contrast, zoom and pan, and navigation through large image sets were not noticeably different, as they are already fast in real time on the standard laptop computer. The program was also tested on lower end computers such as older generation laptops based on 800 MHz G4 PowerPC and 512 Mb of RAM, as well as a desktop 1.2 GHz G4 with 1 Gb of memory. No perceivable difference in performance could be identified except for moderately-slower 3D rendering and maximum intensity projection functions. The OsiriX software is designed to be multithreading compliant, taking advantage of parallel processing capabilities of multiprocessors when available. In particular, all graphics manipulations and rendering functions are tilled in separate threads and distributed to each available processor when possible. The support of multiprocessor architecture is seamless, performing particularly well on the new generation of dual processor G5 computers: the HyperTransport technology developed by Apple²³ greatly accelerates memory transfers from RAM to processors (up to 12.8 gigabytes/second). The RAM to processor transfer is particularly critical for all 3D medical imaging rendering techniques, and significant improvement in performance can be perceived on multiprocessor platforms.

Another potential improvement in performance could be expected for very computational-intensive image processing and rendering techniques by using Grid Computing technology federating several networked computers. Grid Computing technology would allow multiple computers to be networked to perform complex processing tasks such as high-resolution volume rendering and surface rendering of very large data sets.

Navigating the Fifth Dimension

The development of this new DICOM viewer was motivated by the necessity to a have a tool more suitable for multidimensional and multimodality imaging studies such as PET-CT and ultra-fast multidetector Cardiac-CT.²⁴ Essentially these new imaging techniques provide image data that are beyond the traditional 3D anatomical data. The combined PET-CT adds a new dimension to the data that represent the metabolic activity of the tracer, and blending this functional parameter represents a fourth dimension that the users need in order to navigate. The ultrafast multidetector CT scanners can add a temporal dimension to the data by sequentially acquiring images over time to display temporal behavior such as a beating heart or transit of a tracer or contrast agent across a vascular tree or an organ. The newer generation of PET-CT scanners can now provide data sets that represent all five dimensions for cardiac PET-CT studies. We therefore designed our software to allow users to interactively navigate in the five dimensions.

Although the software does not yet support any realignment technique for images obtained from different modalities, it does provide a very simple and very intuitive way to generate fused images by blending two image sets that are prealigned. For image fusion, the user needs to select and adjust a color scale for one of the image sets that will be overlaid over another image set. Both sets must be open at the same time on the screen, and the fusion can be initiated simply by dragging and dropping the title bar of the window containing the overlay image (PET for example) over the window of the basic images (CT for example) as shown in Figure 4. The program will automatically generate a set of fused images by color blending the two image sets. This new set is still fully synchronized with the original images, and any changes in contrast or intensity in either of the two sets will be reflected in the fused image set, allowing the user to adjust the best rendering setting of the fused image. The fused image set can also be manipulated like any other image data set, and image processing functions such as MIP rendering or MPR can be applied to the four-dimensional data (Fig 5).

To facilitate and improve the navigation and image manipulation functions in five-dimension exams, we explored innovative solutions using advanced joystick and multidimensional navigation devices that can be used in conjunction with the standard mouse and keyboard functions. We elected to use a special multipurpose video editing jog-wheel device (Fig 6) that is widely used by professional video editors. This allows rapid navigation in multiple data sets and multiple dimensions with a single hand. The addition of this low-cost pointing device increases the ability of the user to navigate rapidly and to switch among different functions in real time.

By using this jog-wheel device we were able to develop an interactive graphic user interface that allows navigation in these five dimensions in all rendering modes: the classic 2D viewer, but also in MPR or volume rendering modes (Fig 7). The user is able at any time to move in any of the five dimensions: for example it is possible to modify the dynamic and the fusion parameters while navigating in a 3D MPR view.