Association of Medical Illustrators 2024 Annual Conference

The Association of Medical Illustrators 2024 annual conference just took place in Rochester NY, USA. TVASurg team member Paul Kelly, and host of the Medical Illustration Podcast attended, and presented a tech showcase on using the node-based compositing system of DaVinci Resolve. He also made a podcast about the conference!:

Salon opening

Shortly after that we reconvened at the conference venue for the Salon opening. Always a stimulating and inspiring experience to walk through the gallery of artwork from student and professional medical illustrators putting out their best, most recent work on display in a big room–I’m talking several aisles to walk up and down. One of the benefits you get from the Salon is paying attention to the trends you see. There are certain recurring themes, there are certain names that keep showing up, and you start to see patterns and get a feel for what other people are using. When new tools and techniques hit the scene you get a sense of what has lasting power and where the fundamentals shine. Such a great time and the absolute best time and place for networking–the opening night of the Salon.

Mia Brownell

After a short break we returned to see the McLatchie Fine Arts Lecture with “Exploring the Intersection of Art, Science, and Culture in Mia Brownell’s Surreal Still Life Paintings”. Mia’s work was predominantly oil paintings, many of which featured translucent orbs forming what could be molecules or floating bubbles or a suspended grapes kind of vibe. She would go on to explain they were intended to imitate grapes as often seen in renaissance paintings. She began with her  “Zoonotic Tonic“ series, one of several series of paintings she walked us through talking about her influences, motivations, and a bit of her process. I’ll share a link in the show notes so you can visit her site and see more of these. She had shared images from her “Plate to Platelets” series, which were paintings she did from 2016 - 2019 , “Delightful, Delicious, Disgusting” paintings from 2011-2015, “Stomach Acid Dreams” a series from 2009 - 2010, and “Adventures of a Reluctant Omnivore” 2006–2008, among others. I think it may have been her entire body of work she included in the talk!

I like this idea too of grouping work in a collection that revolves around a central theme, maybe this is a thing in the fine arts world, but it’s inspired me to look at my own work in this way as well. Mia mentioned that a recurring theme for her in her work is exploring what it means to be human in a biotech world. Food and physiology, protein-inspired forms, and momentum mori were visual elements she explored and employed in her compositions. She noted her use of dark backgrounds to enhance a sense of depth and pay homage to scientific conventions for the use of black backgrounds to study structure. Black backgrounds are something I’ve been playing with quite a bit in my 3D rendering experiments and it works well to disguise the scale and edges of forms and allows them to recede or emerge from that darkness in ways they can’t or won’t from a flat white background which implies a very different light and atmosphere. 

Mia points out that food is our ultimate interaction with nature, that food was the first medicine, and that food is the answer to the riddle of what can a woman successfully objectify? She pulled no punches on injecting political commentary into her work with the presence of bees and butterflies calling attention to our vulgar use of pesticides and our overindulgent meat consumption fueling the ever-concerning smoke signal of climate change. Brownell seeks to use still life as an expressive end, and opposed to an end to itself. The ubiquitous presence of grape-like spheres in her work being iconic of European still life paintings, she employs them to invoke protein-inspired forms connecting our hidden mental worlds of visual abstraction with the hidden microscopic worlds of molecular biology. The daughter of a sculptor and biophysicist, Mia Brownell has found a space within the overlap of art and science unique among us, and I found her work illuminating, as well as mesmerizing.

There are 3D color visualization tools and apps that can demonstrate this better but verbally, I think what he showed is, essentially you don’t get an even distribution of color choices available to you when you change between color hues. So in this talk Michael clued us all into an alternative color model that has addressed this problem, the OKLab, or OKLCH which is perceptually consistent. Unfortunately it is currently only available for a small number of platforms, but those platforms include Safari, Figma, and Nuke, which are some major players in their respective markets so we may see extended adoption in the future.

Dr Sarah Chen

Dr Sarah Chen presented The HeART of Surgery, unfortunately I didn’t catch this one. I know! I should be at every surgery talk. Please forgive me. This did sound really interesting, and similar to TVASurg in several aspects, so it really spoke to my heart: using visual aids to explain complex anatomy, customizing your visualizations to specific patients. But oh man, listen to this, this is from the program text: “This talk will cover the extensive utility of art—from simplified drawings to customized 3D printed models—in cardiothoracic surgery, from augmenting patient education, promoting research and clinical didactics, facilitating pre-operative planning, and fostering resident wellness. This talk will also provide a review of cardiac and thoracic anatomy, including intra-operative views that are not readily intuitive to understand when compared to the standardly drawn anteroposterior views.” I am bummed I missed this. Dr. Chen PLEASE do a webinar, this would be phenomenal!

Brittany Bennett

So I did see Brittany Bennett present “Establishing a Medical Illustration Practice in a Pediatric Radiology Department.” And again, kindred spirit in the world of Healthcare institutions. Brittany came to this talk from the perspective of someone who had really done some serious administrative management work, asking important questions such as “Who is my team?” and “Where’s my support?” Brittany shared some hard-earned advice and observations–I found myself bobbing my head to all of this: When you’re working inside a hospital, you need a white coat, you need that MD representation, and and you need a “day-to-day contact”, like a Surgical Fellow, or Admin, usually one of those two for us, but someone who is ever so slightly less booked than your “White coat”. Define your department scope. Network within your institution. High-impact short deadlines get triaged to the front of your production line.

David Aten

David Aten presented “Designing Patient-Specific Anatomical Models for Surgical Planning in Orthopedic Oncology.” Unfortunately I had to make a tough choice and missed this one in order to catch Brittany Bennett, but check this out from the program: “Managing pelvic tumors presents many challenges in both tumor resection and reconstruction of the bony anatomy…Using scan data, a 3D model can be produced that gives surgeons a tactile, 3-dimensional representation of the patient's anatomy. These models are of value to the surgeons for assessing tumor size and positioning, and give them an accurate depiction of anatomy that can be saved, and anatomy that must be removed..” Similar as we do at TVASurg, using scan data to produce patient-specific 3D models. OK. But it sounds like they are using 3D prints which is another layer of production stages, but that’s great if they’re doing lots of these cases. “This presentation will present participants with the challenges inherent in pelvic tumor resections...” I hope they put together an exhibit of their collection one day, that would be extraordinary. I’m there.

Roxy and Shay

Next up we began what would be a series of talks on the hot-topic of AI. “Generative Artificial Intelligence (AI) and its impact on biomedical visualization: Perspectives and Prospects” was presented by Roxanne Ziman and Shehryar Saharan (unfortunately Dr Gael McGill was unable to join us but he is the third player of the triumvirate involved in this research study) to help us form a broader and more neutral view of generative AI. The format of this talk was really engaging, they would occasionally pause and poll the audience, and show that data live on screen as they navigated their slides. It was a great way to pull the audience into this research. 

So the purpose of the larger study they were discussing was to answer 3 questions: 

1. What are the perceptions and attitudes towards generative AI?
2. How is genAI currently being used in the profession, and how will and should it be in the future?
3. What are the key concerns?

Some of the findings from their work: There is a clear preference for use of genAI in pre-production rather than later stages. Current use for pre-production work has been some use in ideation, general visual concepts, and scripts. In production it’s being used for coding and voice over, in the future we may see it being used more for 3D mesh cleanup, resolving technical issues, and uprezzing. Creating at low resolution and then scaling up without pixelating your image. Some of the perspectives they shared from the study: Human agency and control is #1. Task automation is a close second, this is where people see this going. Perhaps it can serve the role of a creative collaboration. There may be some subdomain specializations that remain to be an advantage. We have to define the line between creativity and fulfillment. I like how they ended this talk with a shout out to Todd Buck’s “C”s address. These are the core of what we do and will continue to be.

Dr Julia Lerner

Concurrently, Dr Julia Lerner, an Emergency Medicine Resident Physician in Newport News, VA, had been presenting “HALO:” Innovations in Accessible Simulation Models for Emergency Procedures.” Wow, an emergency medicine resident physician. Now that’s a title. Dr. Lerner demonstrated simulations for healthcare workers to practice and improve responses in emergency scenarios. From the conference program quote “The session will combine visual representations, hands-on demonstrations, and case studies to provide a comprehensive look at novel simulation models and their applications.” That’s really cool.

Niámh Mundy

Niámh Mundy, from Jeroen Claus’s company Phospho, gave what I think was the AI talk everyone was waiting to hear. I may have misunderstood a few aspects of this study so bear with me but I think what they had done here was they were looking to see if we can train an AI art model to have more accurate depictions of scientific topics, and they wanted to look at DNA specifically. So they made some animated clips of DNA using Houdini. They animated it in such a way so that over the course of this short animation clip, each frame of the animation would cover a wide range of possible orientations and compositions that DNA could appear as in a framed image. So the DNA strand in their animation would bend and twist and move at different angles, this results in many shots where there is a dramatic plane change, sort of like looking down the barrel of a gun if you were aiming a rifle let’s say, but imagine the rifle barrel is a strand of DNA and this is important because these dramatic angle compositions will then result in very strong depth of field blurs in medical animations, here I don’t think they used a DOF blur it was just that point of view but that makes it disorienting to look at, at first glance you might not recognize it as DNA. OK. And they also altered the appearance of this DNA on the animations to look more or less like a ribbon, so there was an adjustment to the degree of abstraction. And they ended up, after rendering all these animation clips out, with~27,000 images.

Joe Samson

Joe Samson presented “Summoning the AI Wizards: Navigating the Infinitely Evolving Terrain of NeRFS, GANS, Splats & MoCaps”. I know Joe has been experimenting with AI tools quite extensively over the past few years. I met up with him 2 years ago at the meeting in Des Moines Iowa I think it was and we got to chat briefly about what he’d been up to. So Joe had been doing this whole series of hypothetical organisms with generative AI, lots of alien-like microorganisms or something from a prehistoric era here on Earth. Joe is well-versed in this tech and well-qualified to share a perspective as a user. To give you a few definitions from his talk and understand what he was demo-ing:

• NeRFs are Neural Radiance Fields, these are sort of like photogrammetry but not, they do produce geometry, but they do it with as little as 2 images of the same thing as opposed to many that are required for photogrammetry. 
• GANs stands for General Adversarial Network. This is the technical term for a machine learning framework. The Diffusion models we are familiar with today evolved out of GANs.
• Splats refers to Gaussian Splatting, these are points clouds, what’s amazing is they can capture reflections and transparency, Joe had some examples that showed this,
• MoCaps refers to motion capture. Joe shared several examples of animation generators such as Runway Gen2, Luma AI’s Dream Machine, Deforum (a Stable Diffusion plugin), Kaiber (this one looked like it was catered more towards 2D animation) and Pika (more cinematic, photorealistic looking videos). 

Joe demonstrated how these tools can offer “rapid exploration, prototyping, and evolution of 3D modeling and environmental concepts”. Lastly, Joe also shared a few random AI tools that can be of immediate practical use such as TwoMinutePapers. He also recommends checking out the work of the AI artist who goes by mrk.ism. Really impressive, I applaud Joe for exploring and experimenting in a way that seeks a true marriage of AI capability with human art direction. He’s already there!

Hjordis Bjartmars

Running concurrently with Joe’s talk was (and I apologize because I’m totally going to mispronounce this, but I’ll give it my best shot) Hjordis Bjartmars (your-diss be-yardt-marrd), “The Use of Illustrations in Patient Education in Iceland”. I did not see this one, but I can read to you from the conference program: “an insightful session led by Hjördís Bjartmars, a Medical Illustrator specializing in patient education in Iceland through visual communication. Learn about her projects such as:

• Empowering foreign women during pregnancy with a wordless picture book.
• Facilitating communication for respirator-dependent patients.
• Exploring the Stroke Alert App (SLAG) for swift emergency response.” 

Well that’s great to see another international presenter, and that’s interesting that she’s creating patient education materials for immigrants to Iceland. Certainly an at-risk population from many aspects. It's great that there's someone with the passion and the skills to take on challenges like this and I’m glad Hjordis could come present at the AMI.

Kristen Browne

Kristen Browne from the NIH presented on “Using Python to Automate Tasks with a Little Help from AI” I did have the wonderful opportunity to chat with Kristen at one of the meal breaks earlier in the day so I got to hear a bit about this. Kristen Browne, if you don’t know her, Swiss army knife of medical illustration skills and scientific knowledge. Biomedical 3D Model Archivist. We should call her Kristen Brownian motion in a packed fact environment. 

In this session, Kristen shared some practical applications of the Python programming language, using it to build efficiencies into 3D production pipelines. Her experience designing and developing the NIH 3D model sharing and creation platform positions her as an expert in technical direction for 3D. She showed how she’s been using Python to quote “...facilitate smooth file transitions between Blender, pyMeshLab, Trimesh, and other modules, consolidating a myriad of functions and features under a single command.” Incredible. Oh, and by the way, all the code she discussed in this session is freely available on GitHub.

SJ Okemow

Meanwhile, concurrent to Kristen’s talk was SJ Okemow “An Artist’s Practice-Based PhD Journey: Fusing Molecular Animation and Indigenous Ways of Knowing” This was a fascinating and electrifying talk. There was a recurring theme here of finding a balance and resolution between our necessary activity in digital spaces and our soulful longing to reconnect with the natural world. SJ began with an autobiographical description of her understanding of self-in-relation and a brief historical context of western scientific practice imposing incredible harm on indigenous communities. SJ then drew parallels between western science beliefs and indigenous ways of knowing, which to me was reminiscent of the book “The Tao of Physics” by Fritjov Capra. I appreciated the wealth of book recommendations given throughout this talk, a sampling of which included:

• “Braiding Sweetgrass” by Robin Wall Kimmerer
• “The Molecular Gaze: Art in the Genetic Age” by Suzanne Anker
• “Pollution is Colonialism” by Max Liboiron
• “Objectivity” by Lorraine Daston and Peter Galison
• and “Research is Ceremony: Indigenous Research Methods” by Shawn Wilson

I appreciated the “Two-Eyed Seeing” section of this talk, the idea that 2 ways of knowing can coexist. I think we can all relate to that at some level coming into this profession where we have to balance, combine and contrast science and art, education and business. I also really liked towards the end the examples SJ gave of some of her experiments using audio, making her own recordings, using audio to set the mood of her workspace, and using audio to drive animations in Houdini. This talk was a captivating account of a researcher-artist’s still unfolding journey. Seeking out and implementing her own cultural heritage, revitalizing forgotten rhythms pulsing at the edges of her unique gaze. SJ’s talk indeed inspired, to quote her bio in the conference program: “...critical reflection on the ways in which we move through, exist within, and share our environments.”

Cameron Slayden

Following lunch, Microverse CEO and co-founder Cameron Slayden presented “How Diffusion Models Work, and other AI Discussions”. Cameron has previously presented on the topic of AI at AMI meetings, he’s also done a presentation for the BMCAA that you can find on YouTube, I’ll share a link to that in the show notes.

This was a good objective walk-through on the major species of AI tools we’re now interacting with and seeing results from. Cameron began by giving an overview of AI Diffusion models, which essentially see patterns in random noise. He also confirmed what I’m sure many of us suspected for years, which is that those “Capta” images we click on to prove we aren’t robots when creating new accounts or logging in to sites, were part of a training algorithm to help machine learning discern objects in scenes of everyday human life.

As I mentioned from Joe Samson’s talk, GANs or General Adversarial Networks were earlier machine learning frameworks, now falling into obsolescence. Diffusion models evolved from GANs, which use two neural networks communicating with each other to produce an end result: a generator and discriminator. The generator produces the noise patterns and the discriminator tries to identify patterns from within that noise to produce an image.

Cameron then went on to talk about RAG or (Retrieval Augmented Generation). This is, to quote an NVIDIA blog article written by Rick Merritt: “...a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from external sources.”  From the same article, quote “large language models (LLMs) can respond to a wide variety of human queries. But to deliver authoritative answers that cite sources, the model needs an assistant to do some research.” OK interesting, so it sounds like this is an additional feature one can add on top of a LLM that you’ve built an app around giving it the ability to go beyond its base dataset without the time and cost of retraining a model. I may be misunderstanding this, but maybe this is a pathway towards what people are calling agenic AI, or AI agents that can be given specific assignments or tasks. Cameron mentioned “One-shot learning” vs “Zero-shot learning” in this section of the talk, these are types of machine learning training.

If I understand this correctly, One-shot learning is when a machine learning algorithm is trained to recognize a new class of objects or things with only one example to go off of. It then has to generalize or extrapolate new classes based on single examples of each class. Zero-shot learning on the other hand is when you tell the machine learning algorithm what something is without giving it an example of the thing. You just describe it and it has to go off of your description alone.

Next up, LLMs (Large Language Models),  ChatGPT falls in this category. Also from the NVIDIA site, this one was a glossary page so no author listed, LLMs are: “deep learning algorithms that can recognize, summarize, translate, predict, and generate content using very large datasets.” So LLMs essentially track relationships between words, based on the word associations they’ve analyzed over a massive archive of text references. I think. I love how NVIDIA puts the effort into making these layman explainer pages on their site for knuckleheads like me who can barely understand what they’re talking about. Like you can keep up for a few sentences and then they get all technical and you’re like “Where am I? Am I awake? Am I inside the computer?” Haha. I can follow for only so long before my eyes start to glaze over. But hey, I’m trying. NVIDIA is trying. Cameron Slayden is trying. We’re all trying.

The second part of this talk was geared towards looking at what’s next. Cameron shared the work of several companies he’s either worked for or with and what they’re doing to use AI to drive and accelerate medical research. We are already seeing targeted drug delivery, the development of “safe” opioids, anti-aging drugs, and maybe even one day, re-growing teeth.

Cameron shared with us a glimpse into the future of AI development, what we may see especially in the biotech world. We’re already experiencing the 1st bottleneck which is figuring out what to fund? There is so much potential here but the resources and funding is limited so it’s a question of looking at potential impact, and the right fit between the tool and problem, which is always the case in scientific research. The 2nd bottleneck is what to review? We’re going to quickly see that the FDA can’t keep up with all the applications it’s going to be bombarded with. Government is notoriously slow when it comes to updating regulations and passing legislation. So, as always, we’ll have to wait and see.

In the meantime, I do agree with Cameron–we need to better understand AI so we can better understand its impact on our field and its impact on medicine, and in turn, its impact on our clients. As we look outward to the horizon into a maelstrom of change, may we rely on the consistent fact that the value we add, as always has been the case, is unique to our skills as medical illustrators and scientific communicators.

iso-Form

Rounding out the AI talks Nick Klein and Russ Adams presented “Harnessing the Power of AI: Boosting Efficiency, Productivity and Creativity Using AI Tools”. The iso-Form team always brings a lot of energy and humor to the stage. 

They began with framing AI as a disruptor in its current state. Similar to how photography was to painting. I can't disagree with that! I think we’re all feeling this. With several recent in-house examples, Nick and Russ invite us to view AI as a powerful and helpful set of production tools, which I also agree with. I did a podcast stating exactly this (March 31st 2023). So we’re on the same page there. There are a lot of things AI is particularly good at, for example:

• Procrastination busting
• Brainstorming
• Research (ELI5)
• Client Communications
• Reviewing or updating legal documents
• Programming
• Look & feel, character development, stylization, and inspiration
• and Writing

This talk got me thinking again about a question I’ve been asking a lot of people recently, which is, “If you could design or build a custom AI tool to help you in your day-to-day work, what would you want?” Well, for one, I’d love to make/use an AI tool that will help me remember things I looked up already or suggest videos/articles based on tabs I never close. There’s probably one out there already that does that. 

I’d also love something to help me ask clients the right questions to communicate that you care about their work. That was where I think Nick and Russ were steering in this talk and encouraging us to turn our perspective towards–making a better production experience and end result for our clients. How can we do this? How can we use powerful AI tools to get us there faster? iso-Form are the guys out there, asking the tough questions.

Melanie Connolly

The conference was concluded by a phenomenal presentation by Melanie Connolly, “Beyond the Canvas: Exploring New Avenues in Health Innovation for Medical Illustrators” Melanie’s career has taken new turns, and coupled with the challenges of raising a family have given her new insights about the American healthcare system. Melanie shared some of her new duties and roles that included

• marketing, innovation and branding
• web & app development, 
• project management,
• patient and clinical education,
• and international collaborations

I think this probably sounds familiar to a lot of us. Melanie then moved into the narrative of her talk, beginning with the need to find your “why” in what you do. I love the way she phrased this on one of her slides: “Scope creep will happen in your career-A “why” creates more directed choices.” This also speaks to the problem of burnout we know is common to our field and the adjacent healthcare and creative professional spaces. I really appreciated Melanie bringing this up to immediately offer a possible solution. She said to solve burnout, you need to lean into the problems you’re seeing and be able to affect change. I think that’s a wise observation.

Melanie spoke quite a bit about her own navigation and struggles with the US Healthcare system. A lot of this was personal so I don’t want to share too much of it without her having OK’d that. But in summary there were a lot of mishaps and misunderstandings, miscommunications and in a few cases straight up unprofessional conduct by healthcare workers she was interacting with and of course, then they send you a bill. It set a fire under her to take action and try to do something about it. Thankfully it sounds like things panned out such that no one was seriously harmed, but she shared some startling statistics such as “greater than 300,000 preventable deaths annually from 5 diseases” and “nearly 800,000 Americans die or are permanently disabled from diagnostic error each year” according to Johns Hopkins Researchers, as was reported in a BMJ Quality & Safety 2023 article. 

Getting chewed up and spit out of the healthcare system is enough to cause anyone to repeatedly ask the universe “why”, especially when your career is so closely related to the system you’ve just been mistreated by. And this is where I felt the narrative of this story reach a crescendo point. The light at the end of this tunnel of torment was an enhanced sense of self reliance and recognition in one’s own expertise. It sounded like Melanie came to see her own value as an educated member of the healthcare network, and she shared with us a thorough list of what we bring to the table. This is just a sampling of what Medical Illustrators are good at:

• Making order out of chaos
• Navigating complexity and complex ambiguity
• Being voracious learners
• Focusing on the audience
• Taking jumbled explanations from multiple experts and creating a succinct, actionable deliverable
• Not only taking critique well, but actively seeking it out
• staying on top of new technology
• Understanding contracts
• Being power connectors and KNOWING the value of a diverse network

These are skills and attitudes you don’t acquire overnight or without effort. I hope when listening to this list you started to form a mental picture, as I did, as I think Melanie has, of someone capable of working with a complex system, and initiating change from within. Debugging while the program is running. Yea, we are positioned to do that!

Melanie shared a wealth of resources in this talk. I want to share her booklist for personal development:

• “Build” by Tony Fadell
• “Thinking Fast and Slow” by Daniel Kahneman
• “Your Brain on Art” by Ivy Ross & Susan Magsamen
• “Never Split the Difference” by Chris Voss
• “4,000 Weeks” by Oliver Burkeman
• “Deep Work” by Cal Newport
• “Infinite Game” and “Start with Why” both by Simon Sinek
• “Dare to Lead” by Brene Brown
• “The Tools” by Phil Stutz

Resources for healthcare education included:

• FairHealthConsumer.org
• “Never Pay the First Bill” by Marshall Allen
• “The Price We Pay” by Dr Marty Makary
• and the YT channel “Glaucomflecken” by Dr Will Flanary

Thank you again to Melanie for this excellent talk, for sharing these resources, and for wrapping up the meeting with this inspiring and motivating window to a future we can build together as a community.