Advanced Visualization

Equipping physicians with the ability to see more and do more

In this roundtable feature, rt image gathers some of the leading developers of advanced visualization software to discuss how this technology has become so integrated, how it handles the massive amounts of data generated, and how it will change the imaging market.

The Participants:

• Don Barry, product manager, ContextVision
• Terry Chang, director of marketing, Ziosoft Inc.
• Mary Frances Feider, CEO, Fiatlux Imaging
• Ray Ghanbari, PhD, executive vice president of strategy and products, Vital Images
• Marianne Maffoni, clinical marketing manager, TeraRecon Inc.
• Kimberly Powell, business development manager, healthcare, NVIDIA
• Henri “Rik” Primo, strategic relations manager, image and knowledge management, Siemens Healthcare

rt image: It seems that advanced visualization (AV) has matured drastically over just a few years. Why has it experienced so much success within the medical imaging community?

Marianne Maffoni: In the past few years, new imaging modalities such as hybrid imaging, high-field MR scanners, and the 256-320 multidetector CT seem to continuously evolve, adding more coverage, faster speeds which are capable of producing superior image quality and a wealth of information due to their high spatial and temporal resolution capabilities.

The speed of these scanners is also capable of acquiring angiography-type results that rival traditional angiographic procedures, which provide monumental benefits to patients because they are noninvasive, less risky, and less expensive and often more economical for the imaging facility than using the more traditional angiography exams.

However, it comes at the expense of collecting and interpreting the large amounts of data that these scanners are capable of producing on an image-by-image basis, as these datasets can reach several gigabytes with acquisition times measured in seconds. AV tools are now a necessity to complement this technology and are able to present each of the thin axial slices in the form of a volume so it can be reviewed and manipulated in any plane in a variety of rendering modes such as MPR, MIP, 3-D, or 4-D.

This method of viewing is very beneficial as it provides a thorough and accurate view of the patient’s anatomy and can reach far beyond the one-dimensional reading methodology of reviewing each slice of the scan. It also decreases the risk of image overload in the radiology and healthcare enterprise and enhances efficiency of the interpreting physician.

Rik Primo: The widespread use of multislice CT and multisequence MR created huge image data sets. Viewing each slice individually became a challenge for the perception capabilities of the human brain and created “viewing fatigue,” diminishing attention of the viewer to the fine detail. AV, including CAD and 3-D, tackles these huge image datasets in a more efficient format for the observer and allows the users to zoom in from a larger overview on the subsets of important anatomical details. CAD algorithms (mammo, lung, etc.) will even highlight specific areas of interest and suspicion.

These advantages are not limited to the diagnostic process in the imaging department. Three-D volume rendering will help therapy providers like oncologists and surgeons to view the exact location of the pathology and assist them while planning the best possible treatment plan. Three-D rendering will also allow them to see and avoid critical anatomy surrounding the area of interest.

Mary Frances Feider: The community of AV users has grown to include non-radiology medical image-dependent physicians such as neurosurgeons, cardiologists, orthopedic surgeons, oncologists, and others. Each of these specialists relies on AV in different ways. One of the most common ways specialists use AV including 3-D is as a valuable tool to help them plan for surgery and other interventional treatments.

Also, physicians are discovering that AV provides a powerful communication and education tool that allows patients to see their own pathology so they can have an informed discussion regarding plans for intervention or other treatment.

We are also hearing from physicians that in many cases, AV in diagnostic imaging is a powerful tool for tracking the progress of cancer and cardiac patients – in many instances replacing the need for surgical procedures.

Terry Chang: AV has matured drastically because it’s not just about “pretty pictures” anymore. AV is now a necessary tool to optimize diagnosis, collaboration, and efficiency. A human body is not two-dimensional, so all clinicians can benefit from analyzing the body volumetrically. Without AV, imaging modalities cannot achieve their fullest potential. AV adds the intelligence to the images and helps to minimize analysis variability.

Don Barry: Until the last decade, OEM manufacturers focused on hardware improvements tied directly to the source of images, such as the flat panel detector/image intensifier and the gantry. As a result, we now have 3T MRI machines, 256-slice CT systems, and 1024 x 1024 detectors with pixel sizes below 0.5 mm (1/64 of an inch). So, in order to further optimize the imaging systems, the OEMs have turned to the improved visualization of medical images.

The drive has been to twist/turn/rotate/zoom/enhance the data to enable visualization from different perspectives. This wasn’t possible during the film-based era, when everything was a two-dimensional image. Instead, doctors did this manually (and often in the mind’s eye), but today’s software packages include these powerful diagnostic visualization tools. The analysis now happens within the image, using CAD tools (including the mouse and digital rulers) to measure distances, compare and contrast levels, etc.

Of course, the gold standards of medical imaging continue to change and grow. So doctors, scientists, researchers, and the R&D departments are constantly coming up with new ways to improve the diagnostic quality of images. Sometimes this is very successful and adds value, but often it is a repetition or a small incremental change of something already existing.

In recent years, technical breakthroughs in capacity and computational power have enabled the medical imaging industry to use multicore processors, extremely fast graphic processing units (GPUs), digital signal processors (DSP), and lately the CELL broadband engine (used in the Playstation 3). These ultra-fast processors allow advanced computations and extreme amounts of data to be processed in seconds, compared to the hours and minutes it took 10 years ago.

Ray Ghanbari: AV has pushed the technology envelope and at the same time benefited from the overall technology advances in healthcare. Several years ago, many physicians would rely on others to review imaging studies and take their diagnostic findings to make their treatment decisions.

As the technology got easier to use, showed its clinical value and the average physician became more accustomed to using technology, AV became a vehicle for more physicians to access, view, and make large quantities of complex image data easy to use and useful in addition to the diagnostic report that is still provided. Additionally, AV has grown from a value-added component for the radiologist to review the imaging data for diagnostic reporting to a treatment planning and analysis tool for specialists in many areas outside of traditional radiology.

Kimberly Powell: It’s an exciting time for the healthcare market. As with most industries, companies in healthcare are investing more in research and development, and are looking at ways to become more efficient, reduce medical equipment costs, and at the same time, advance applications and medical diagnosis.

A technology that is making a huge impact is the graphics processing unit (GPU), as it’s designed to advance real-time and off-line graphics. The inherently parallel architecture of a GPU has enabled a new level of programmability and performance, resulting in the capability to process much larger amounts of data and deliver an unprecedented level of graphics realism at ever-increasing speeds.

AV has been aggressively taking advantage of the GPU’s architectural benefits, leading healthcare professionals to better diagnosis quality and speed, earlier detection, better classification of disease, and enhanced treatment planning.

Moreover, due to the advancements in imaging scanners, larger and more complex data sets are being acquired, requiring computational processing. The GPU has evolved into not only a graphics engine but also a compute engine. Imaging modalities that feed AV – such as CT, MR, ultrasound, and PET – all benefit from the GPU’s massive parallel processing architecture, advanced graphics, and cost/performance in comparison to legacy architectures. Real-time AV is now within the realm of possibility and in some cases, like CT, already a reality.


image: What improvements are being made to the technology to handle the massive amounts of data generated to deliver the 3-D images?

Marianne Maffoni: For TeraRecon, our product provides a highly efficient, seamless, and unified workflow that begins upon the acquisition of images, continues through automated pre-processing, automated preparation for interpretation by a specialist, individualized diagnostic interpretation, Web distribution of rich interactive results to referring physicians and their patients, rapid access by surgeons or interventionalists, and provides these functions via seamless API and URL integration to the widest range of leading PACS and patient reporting environments.

Aquarius iNtuition leverages VolumePro™ technology to deliver more rendering power and more clinical tools to more locations across the enterprise.
 
Rik Primo: Advances in technology – such as faster and multiple CPU technology, high-performance graphic cards, higher and more affordable density storage media, faster network, and affordable high-definition display systems – have all contributed to bringing AV into the mainstream of clinical practice, rather than relying on the traditional high-end workstations in radiology or a 3-D lab.

SW and IT infrastructure developments, such as thin-client, have made 3-D reconstructions possible wherever a high broadband Internet or LAN connection is available to the DICOM CT or MR datasets in the PACS archive. Furthermore, we have seen significant improvements in 3-D rendering algorithms.

Mary Frances Feider: The imaging market is leapfrogging itself today with more powerful scanners coming on the market each year. The more powerful the scanner, the larger the dataset it outputs. The need for ever-more powerful software to handle the analysis of this ever-growing deluge of data is very real.

Radiologists must read the same number of cases each day even though the amount of data they need to review may have grown by as much four times in the last few years. Referring physicians and other clinicians involved with the patient’s care are at even more of a disadvantage to view the larger datasets in a meaningful way.

Fortunately, easily accessible AV software programs are now available that handle these datasets in realtime with unprecedented graphics quality at a fraction of the price of traditional, workstation-based technology. These new applications are easy to use and require little hardware investment.

Terry Chang: Fortunately, the advances of the computer and networking industry have started to meet the demanding computational needs of AV. You no longer need expensive proprietary hardware to run AV software. Network bandwidth continues to increase both inside and outside the facility. This has enabled the rise of centralized server-side rendering (thin-client) systems which has bolstered accessibility, unburdened the network infrastructure, and outdated the standalone “thick” workstation.

The Web is fast becoming the medium of choice for AV due to its ubiquity. As the performance requirements of AV applications continue to grow, new methods to load-balance sophisticated processes across server clusters will emerge to avoid compromising image integrity.

Don Barry: The volume of data from the imaging device is staggering. A complete CT examination typically generates several hundred megabytes. Going from 8-bit data format or 256 grayscale levels to 12-bit increases the data by four times. And all data must, according to the regulations, be stored for a certain period of time.

To a radiology department, a gigabyte of data is almost a speck of dust compared to a 3-D study where the image data will occupy terabytes of file space for just one patient or exam. Therefore, fast Ethernet and Local Area networks (LAN) that can push data at high speeds are essential. High-speed storage, such as hard disks and CD-disk system archives, also play important roles. There is no time for someone to retrieve patient folders from offsite storage.

Instead, everything must be online and available 24/7. So, hardware, PACS, and software that give the physician immediate access to the 3-D patient data are being developed and constantly being improved.

Ray Ghanbari: The data does keep growing as the imaging technology advances, and AV solutions consume it, make it useful, and add to the content very quickly. The transition from the dedicated workstation to server technology has been key in allowing the data, while still growing, to be centralized and processed once consistently rather than distributed and processed differently.

Advances in graphics technology and CPU processing as well as network speeds have helped, but it has really come down to advanced processing algorithms that parse through the data getting to the heart of what is needed based on the specifics of the imaging exam that has allowed AV to keep up. We are able to take large series of images of a part of the body and, before even looking at one of them, know what to look for, segment out and render the relevant views, thus reducing the amount of data to be processed to get to the desired view of the available information.

Kimberly Powell: Two major bottlenecks present themselves in the imaging chain when datasets increase: pre-processing of scanner data to create anatomical images and post-processing to render 3-D volumetric data. Historically, these processing steps were done on expensive, customized hardware such as field-programmable gate arrays (FPGAs) and clustered central processing units (CPUs), and still, most physicians and researchers experienced wait times of tens of minutes for pre-processing and hours for post-processing.

Due to the cost, complexity, size, and power consumption of the former technologies and their inability to scale well with the data explosion, the medical imaging community naturally gravitated toward the accelerated visual computing available through GPUs.

In addition to the advanced 3-D rendering technology inherent in GPUs, GPUs are also now programmable to take advantage of a many-core parallel processing architecture, allowing healthcare professionals to process large image data in a fraction of the time. Additionally, GPUs are a scalable architecture protecting the future of application investment when datasets continue to grow. These advances have reduced the imaging chain bottlenecks, giving birth to real-time visualization of large datasets, such as the beating heart.


image: What features and price points are PACS OEMs and facilities looking for before implementing the AV technology?

Marianne Maffoni: PACS systems were designed to store data and distribute the axial images on the viewing stations to facilitate report generation; however, they were never designed to allow for the influx of the current image data overflow for interactive image review, let alone the “on the fly” demand for volumetric post-processing.

For this to occur, it would be a significant upgrade purchase and the workflow architecture would have to be designed on existing PACS to accommodate AV technologies. TeraRecon’s client server technology can be seamlessly be integrated into existing PACS, providing immediate AV tools at a fraction of the cost.

Rik Primo: The answer really depends on the specific utilization requirements. Thick-client technologies and dedicated workstations play a significant role in the 3-D lab, especially in hospitals with a heavy imaging workload. Web-based thin clients running on a PC can be used in the radiology department in case of a lighter load in 3-D studies.

However, thin clients are a “conditio sine qua non” [indispensable condition] for users outside the imaging department at the point-of-care, such as the physicians in the ER or OR. A mix of both technologies is indicated in many of the use cases.

Mary Frances Feider: The good news is that there are visualization solutions available for many different needs and at all different price points. Facilities and individual purchasers should create a list of must-have features and capabilities. They should ensure that the system chosen is versatile and can grow as their needs grow to avoid obsolescence in just a few short years.

Users should be able to utilize “off-the-shelf” computers and video graphics cards to leverage the rapid advances of the computing industry. Software code that is intertwined with expensive proprietary hardware or specialized graphics cards limits the pace of software innovation and can complicate PACS integration.

The most flexible and cost-effective configuration is a software-only CPU-based system that allows independent procurement of hardware based on the vendor’s minimum specifications. In many cases, a laptop and robust graphics card can provide all the computing power and display needs for much of the viewing, manipulation, and analysis needs of image-dependent physicians.

Terry Chang: Today, customers are really scrutinizing return on investment for AV. They are asking questions like, “How can AV improve productivity? Will AV increase reimbursement or referrals? Are there alternative ways to pay for AV?” Customers are looking for an AV system that can grow as their needs grow and not have to be replaced or upgraded at high cost in a few years.

Common features such as volume rendering, MPR, and CPR may be sufficient for now, but in a few years, they may need more advanced features, such as MR cardiac function, dynamic brain perfusion, and colon analysis. Cost-effective, software-based scalability is something customers are demanding.

Don Barry: The old saying, “time is money,” still applies. If a doctor can save five minutes at the workstation per patient, more patients can be reviewed: 10 patients per day and five minutes per patient adds up to an additional hour of work each day. So, features are targeting “faster, more accurate diagnosis,” “semi-automatic diagnostics,” and the software that is being developed is able to process many different tasks at the same time (parallel processing), so that when the physician asks for data, it will be available.

Ray Ghanbari: Like all technology providers in a fiscally conscious market, PACS vendors are looking for AV solutions that complement their workflow while not competing for a large percentage of the budget. For the past several years, PACS has been about the enterprise. Well, now those same enterprise users want and need the advanced clinical capabilities that the right AV solution can bring to them at any location.

Integration, scalability, and performance become key AV features in augmenting the PACS story as well as depth of clinical tools in the subspecialty areas (neuro, cardiac, colon, orthopedic, oncology, etc.). What really sets the right solution combination apart is the ease of use of the advanced tools and how well the solution component vendors (PACS and AV) work together to support the customer’s needs.

Kimberly Powell: In my experience, most facilities and PACS OEMs are looking for a powerful AV solution that has the flexibility to suit the particular needs of hospitals and imaging centers. They are also looking for a seamless, modular, and flexible application integration that can create a continuous image application interface and workflow.

From a pricing perspective, the hardware to support advanced visualization must be considered. There are two models that exist today – 3-D workstations and client/server models where the 3-D volume rendering is done on a central server. A recent trend in healthcare is to allow for radiologists, cardiologists, and physicians to collaborate on patient studies, where a client/server solution is preferred.

Recently, GPU-based server solutions have significantly increased the number of concurrent users at a considerable price/user decrease. Similarly, PACS workstations are now more than ever able to be 3-D ready with professional 3-D capable graphics cards.


image: Looking ahead at the demands of radiology, including the need to offer anywhere access, how will AV need to adapt to remain a valuable and efficient part of radiology workflow?

Marianne Maffoni: TeraRecon understands the importance of AV and decision support with anywhere access, and is already delivering operational and diagnostic efficiencies with Aquarius iNtuition. TeraRecon is convinced that, within a short period of time, such advanced decision support and visualization will become the standard of care and an integral part of all enterprise health information management systems, with concomitant improvements in the quality, cost, and efficiency of care.

Rik Primo: In order to tackle the large imaging studies for volume rendering applications use of the latest technologies, such as high-speed networking, thin clients and a contemporary PACS solution are required. Additionally, the 3-D applications will need to be seamlessly integrated into the clinical workflow. Mechanisms such as single sign-on, access to 3-D through the EMR or PACS portal, and floating licenses will show to be essential.

Mary Frances Feider: Laptops and the next generation of tablet computers developed by Intel and available from manufacturers such as Panasonic and Motion Computing are making inroads in the healthcare industry. These portable platforms and the powerful AV software that runs on them will give a whole different meaning to the term anywhere, anytime access. The decision about whether to choose a thick client or thin client will become irrelevant as clinicians have the ability to view, manipulate, analyze, and even diagnose completely untethered from a server.

You will also see the total cost of ownership of advanced visualization being driven down by removing the dependency on hardware. More vendors will take a page from the software industry and develop user interfaces that will appeal to and have meaning for a broad population in the medical field.

As this happens, you will see total cost of ownership go down as the huge costs of scalability and support are reduced. And with the emphasis on software, PACS integration should become less of a headache. Ultimately, advanced visualization will improve clinical care and reduce costs across the enterprise and beyond.

Terry Chang: No matter how you slice it, AV requires more time from the clinician. In order for AV to remain valuable, it needs to add real value, both clinically and productively. AV must address the “pain points” of today, such as inadequate analysis tools, lack of accessibility to all the required tools, and the amount of time it takes to utilize these tools. This will require AV vendors to innovate on three fronts: clinical application development, the system architecture, and usability. Each is just as important as the other.

Don Barry: The availability of images from various sources has created a situation in which doctors order multiple images using multiple modalities. In the long term, this will not work. For instance, when treating a patient with occluded heart, the cardiologist uses ultrasound images to review the beating heart (it’s noninvasive and always readily available). A CT examination follows because the CT will add new information (less noise in the image, static images, volumetric images).

Then, an MR scan differentiates between different types of tissue. The necessity for fusing all of this information for easy medical access and increased diagnostic capability will continue to drive the need for enhanced visualization for a long time.

Finally, a fluoroscopy examination might be used for treatment. After treatment, follow-up requires going through all the modalities again, possibly several times. Think about a patient with a broken arm and the X-rays taken: emergency X-ray, before surgery, during surgery, after surgery, three-days post-op, two-weeks post-op, three-months follow-up, before/during/after surgery, and exacerbating the dose concerns, many of these X-rays will be done in multiple views.

The next generation of imaging software must integrate the specifics from each modality and a gold standard must be developed to reduce both the number of images taken and the patient’s exposure to radiation. The time and costs are just staggering.

Ray Ghanbari: The key to AV remaining valuable will be in its integration into not only the radiology workflow, but all workflow that benefits from the use of imaging. We will have been successful when people don’t even know they are using AV, it is just expected functionality, and people are accustomed to getting that level of information from the images being produced. Ease of use, speed, and levels of automation are what will continue to separate the solution providers much like they do today.

Kimberly Powell: AV is quickly evolving to accommodate the “anywhere-access” requirement. Most vendors now have a Web-based version of AV tools, enabling physicians to view complex 3-D data sets in a Web browser. Previously, Web-based AV had a slow adoption due to the lack of image quality and usability. With servers now being architected with GPUs, Web-based applications can harness the graphics power directly at the server to quickly deliver advanced imaging without compromising image quality.

Additionally, high-performance GPUs are findin