TVASurg produces educational surgery videos enhanced with 3D animations. We’ve previously discussed the production process for how we produce our 3D models, and this month we'd like to go over how we make those models come to life using 3D animation tools and techniques.
While looking through our archive of animations, it was quite a challenge to narrow down which images to use for this discussion, so to simplify things, we're going to focus on a few specific aspects of what makes 3D animation a particularly useful tool for teaching surgery, and use a case study from our atlas (living donor liver right posterior sectionectomy transplant donor procedure) to demonstrate how these principles are applied.
PLANNING
3D animation actually shares many parallels to surgical procedures, not the least of which is the importance of planning. To craft a motion-driven visualization, the intended narrative needs to be broken down into discrete steps, and this is done with a narrative script, and storyboards.
Shown below is an example of how we often make storyboards for our videos, which is actually not standard for most other studios. The best practice in most scenarios is draw out a series of thumbnail sketches that represent separate scenes or takes, which when strung together, produce a sequence or segment of a larger story. For us, reconstructing patient-specific 3D models from medical imaging scans gives us a specific arrangement of anatomical structures we can take screenshots of to put together our storyboards. This way, our surgeon reviewers don't get distracted by anatomical inaccuracies that may appear in otherwise swiftly composed sketches.
Storyboards provide a visual overview of the core components of a surgical narrative. Early in production we want to ensure we are capturing the key steps of a surgery, and will have the proper assets prepared to be used for these shots.
ORIENTATION
A primary goal of the animation content in our videos is orientation. In the animated intros to our videos, we demonstrate an overview of the sequence of steps of the procedure, and provide a perceptual legend to help the viewer navigate through footage. The animated intro and 3D demonstrations provide the viewer with visual landmarks they can connect with what they see in the footage.
Here we see the line of transection being marked on the liver surface. It may not be immediately obvious to the viewer though, due to the cramped nature of the surgical field, that the liver has in fact been rotated anteriorly.
By integrating 3D models with corresponding camera views, we can clarify to the viewer what they are actually seeing in the footage, and reinforce the correct perspective.
MOVEMENT
Sometimes the view the surgeon sees while operating is not the same view they would use to explain a concept; we try to accommodate both views in our animation to visually correlate footage and animation, and achieve educational objectives.
We use the advantages of 3D animation such as transparency and camera movements that would otherwise be impossible to accomplish during surgery. Here we move the camera around a complex group of hepatic veins. Sometimes it is difficult to find the best vantage point to view three-dimensional structures, so camera parallax can be useful to illuminate a structure's position and orientation.
CHANGES OVER TIME
Animation also allows us to demonstrate how spatial relationships of structures change throughout the procedure.
Almost every structure that plays a leading role in the story of a surgery will undergo change, and with so many anatomical features of similar visual appearance in the landscape of surgical footage, this can get confusing, even for the surgeons themselves when looking back on the footage after operating.
By pulling out of the surgical field and returning to the well-defined shapes and uniform colours of familiar anatomical models, we can clarify where and how structural changes have taken place, and bring perceptual relief to the audience, allowing them to more easily remain engrossed in the overall story.
Here we see an important artery that is carefully isolated during the porta dissection phase of the procedure.
Later in the procedure, the same artery is seen, but in a completely new orientation due to the mobilization of surrounding structures.
DRIVING THE STORY
Animated segments are also used to "fill in the gaps" between radically different camera orientations in the footage, and string together disparate shots into a cohesive and logical sequence. This lets us leave out distracting details that do not advance the plot of our narrative, but rather focus on the sequence as the surgeon sees several steps as being integrated into one "phase" of an operation.
By using 3D camera movements, we not only save time in the total video length but also maintain a sense of calm, consistent pacing in the story. Simultaneously, we are clarifying structures and explaining surgical technique. In this sequence we demonstrate how the umbilical tape sling is used to definite the liver transection plane.
When developing and integrating 3D animation content in to our videos, the surgeon’s involvement is just as important as on the day of operation. They determine the core educational concepts of a particular video, identify difficult concepts best depicted with 3D visualization, direct the pacing of the video and identify areas which require revision. Our role is to build the 3D assets and refine them, always keeping in mind the educational objective each asset is meant to address, and do so in a timely and efficient manner.
We hope you've enjoyed this month's installment of our process blogpost series. Is there a part of our production process you'd like to learn more about? Let us know!
Thanks so much for tuning in, and we hope you're staying safe and healthy.
-- TVASurg Team
REFERENCES
Brazina, D., Fojtik, R., & Rombova, Z. (2014). 3D visualization in teaching anatomy. Procedia-Social and Behavioral Sciences, 143, 367-371.
Cooper, K., Hatfield, E., & Yeomans, J. (2019). Animated stories of medical error as a means of teaching undergraduates patient safety: an evaluation study. Perspectives on medical education, 8(2), 118-122.
Crossingham, J. L., Jenkinson, J., Woolridge, N., Gallinger, S., Tait, G. A., & Moulton, C. A. E. (2009). Interpreting three‐dimensional structures from two‐dimensional images: a web‐based interactive 3D teaching model of surgical liver anatomy. HPB, 11(6), 523-528.
Guttmann, G. D. (2000). Animating functional anatomy for the web. The Anatomical Record: An Official Publication of the American Association of Anatomists, 261(2), 57-63.
Kadleček, P., Ichim, A. E., Liu, T., Křivánek, J., & Kavan, L. (2016). Reconstructing personalized anatomical models for physics-based body animation. ACM Transactions on Graphics (TOG), 35(6), 1-13.
Khalil, M. K., Paas, F., Johnson, T. E., & Payer, A. F. (2005). Interactive and dynamic visualizations in teaching and learning of anatomy: a cognitive load perspective. The Anatomical Record Part B: The New Anatomist: An Official Publication of the American Association of Anatomists, 286(1), 8-14.
Kim, M., Pons-Moll, G., Pujades, S., Bang, S., Kim, J., Black, M. J., & Lee, S. H. (2017). Data-driven physics for human soft tissue animation. ACM Transactions on Graphics (TOG), 36(4), 1-12.
Limano, F. (2019, August). Realistic or Iconic 3D Animation (Adaptation Study with Theory Uncanny Valley). In 2019 International Conference on Sustainable Engineering and Creative Computing (ICSECC) (pp. 36-41). IEEE.
Pandey, N., Anwer, E., Salam, A., Pandey, S., & Gupta, S. (2015). 3-D medical animation could be helpful tool for integration of medical curriculum. Medical teacher, 37(1), 99.
Upson-Taboas, C. F., Montoya, R., & O’Loughlin, V. D. (2019). Impact of cardiovascular embryology animations on short-term learning. Advances in physiology education, 43(1), 55-65.
Vernon, T., & Peckham, D. (2002). The benefits of 3D modelling and animation in medical teaching. Journal of Audiovisual media in Medicine, 25(4), 142-148.