Celina Z. Imielinska, Ph.D.

Associate Research Scientist

Columbia University

Dimitris Metaxas, Ph.D.

Professor of Computer Science and Biomedical Engineering

Rutgers the State University of New Jersey

Half a Day

Developing computer-based training tools for minimally invasive medical procedures (e.g. training for basic skills in anesthesiology/surgery, and minimally invasive surgical procedures) requires new methodologies for modeling and simulation. Major medical centers are interested in forming programs to teach, train, certify and re-certify medical practitioners using new simulation technologies. This course presents modeling and simulation techniques along with their graphical output for medical applications. It covers the medical data acquisition, segmentation, organ modeling, rendering and visualization, registration, virtual surgery and simulation, augmented reality, and a variety of clinical and educational applications.

D. Expanded Statement

This course is intended to demonstrate the state-of-the-art in interactive and real-time modeling techniques, spanning a number of educational and clinical medical applications, where the use of graphical representations of organ anatomy and physiology is necessary. Applications include methods for the acquisition and representation of medical data, organ visualization and segmentation methods, surface-based reconstruction and digital surfaces, "intuitive manipulation" and "alternate viewing" of large volumetric models, data driven implicit models, virtual patient from Visible Human data, virtual anatomy atlases, augmented patient simulators, modeling the shape and motion of the heart (combined left and right ventricle heart model), breast modeling for minimally invasive procedure, real-time tracking and visualization of knee joint motion, and physiological model of real-time breathing lungs.

All the speakers have the necessary experience to present this course at the right mathematical and medical level for our audience.

1. Data acquisition procedures and visualization methods
2. Modeling Shape and Function
3. Segmentation methods
4. Deformable Implicit surfaces
5. Large Volumetric Manipulation
6. Visualization and registration methods for surgical applications
7. Augmented Reality
8. Virtual Anatomy Atlases
9. Minimally invasive procedures

Celina Imielinska, Ph.D.

Deborah Silver, Ph.D.

Terry Yoo, Ph.D.

Dimitris Metaxas, Ph.D.

Jannick Rolland, Ph.D.

9am Welcome and Overview (Metaxas)

9:15 The Virtual Patient from the Visible Human Data (Imielińska)

• 3D Visualization of Human Anatomy

• surface-based reconstruction
• digital surfaces

• importance of detail/accuracy of the Visible Human-based 3D visualization of anatomy in anatomy teaching
• Vesalius^TM Project Virtual anatomy lessons

9:45 Large Volumetric data Manipulation (Silver)

• Overview and Motivation
• Skeletonization of 3D Volumetric Models.
• Animating skeletons & Motion Capture.
• 3D Volume Reconstruction from Deformed Skeletons (Software and Hardware based algorithms)
• Reposing and unwinding data (unwinding the colon)
• Visualization & Results

10:15 Deformable Implicit Surfaces (Yoo)

• Introduction
• Implicit surfaces
• Data Driven Implicit Modeling
• Turk & O’Brien’s Variational Implicit Surfaces
• Methodology
• Linear Algebra for Implicit Surfaces
• Thin-Place Splines as a variational solution
• Radial Basis Functions (RFBs)
• Compact, local RFB’s
• Examples: Active Implicit Snakes
• 2D Implicit Snakes
• 2 ½ D Implicit Snakes
• 3D Active Implicit Surfaces

10:45 Modeling for the Integrated Recovery of Structure and Function from

Medical Images (Metaxas)

• Locating structure and function from medical images
• types of image data sources (MRI,CT)
• local and global geometrical model constraints
• Lagrangian Dynamics
• Finite Elements
• Modeling Tissue Properties using Finite Elements
• Segmentation Methods

• Application areas
• Combined left and right ventricle heart model
• Breast modeling for minimally invasive procedures

11:15 Augmented Reality: Aims and Challenges (Rolland)

• Selecting models for specific applications
• Types of models (Patient specific, generic, CT vs MRI)
• Scaling the models – examples studies of knees and mandibles
• Registration of real and virtual objects
• Visualization technology
• A Framework for Calibration
• A Framework for Registration – static and dynamic.
• Real-time dynamic models
• Real-time tracking and visualization of knee joint motion
• Physiological model of real-time breathing lungs
• A Framework for Registration.

11:45am: The Future of Modeling and Simulation Methods in Medicine (All Speakers)

A special emphasis will be placed on providing course notes that present the methodologies and applications in a clear and intuitive way. The presentation of the course will allow people with a general knowledge of graphics and interest in this area to be able to attend the course and comprehend the notes. We will also provide web pointers to relevant additional information and animations based on the presenter's work or related work by others.

A major goal of this course will be to stir increased interest in the graphics community to this exciting area of applications where graphics methods are of paramount importance.

The format will be classroom style with lectures and considerable video material and live presentations. A panel discussion by all speakers at the end will address future directions of modeling methods in medical applications, and allow questions from the attendees.

K. Presentation Media

L. Speaker Biosketches

Deborah Silver, Ph.D., is an Associate Professor in the Dept. of Electrical and Computer Engineering at Rutgers, The State University of New Jersey. She received a B.S. from Columbia University School of Engineering in 1984 and an MS. (1986) and a Ph.D. (1988) from Princeton University in Computer Science. Her area of research is Scientific Visualization and Computer Graphics and she is the PI of the Vizlab, which is part of the CAIP Center at Rutgers University. She has taught courses in Computer Graphics, Visualization, Data Structures, Software Engineering and Robotics. She has worked on many aspects of visualization and is currently involved in distributed visualization, volume graphics, oceanographic visualization, and multimedia research efforts. She has

Dimitris Metaxas, Ph.D., received his Diploma in EE from NTUA Athens, Greece in 1986, his MSc in CS from the University of Maryland in 1988, and his PhD in CS from the University of Toronto in 1992. He joined the Department of Computer and Information Science, University of Pennsylvania as an Assistant Professor in September of 1992 and was promoted to Associate Professor with tenure in January 1998. From September 2001 he is Professor in the Division of Computer and Information Sciences and the Dept of Biomedical Engineering at Rutgers Univ. Dr. Metaxas is the director of the CBIM (Computational Bio-Imaging and Modeling) Center. He specializes in physics-based modeling techniques with applications to computer vision, graphics, and medical image analysis and has published over 170 research papers in refereed journals and conferences. He is the author of the book ``Physics-Based Deformable Models: Applications to Computer Vision, Graphics, and Medical Imaging''. He has organized several conferences and workshops, is an associate editor of GMIP, and is on the editorial board of Medical Image Analysis. He has received an NSF Career Award (1996), and an ONR Young Investigator Award (1997). He has received 2 best papers awards in computer graphics conferences and a Gold Medal in the Ninth World Congress on Medical Informatics (1998). He is also a Fellow of AIMBE.