Stanford Assistant Professor, Dr. Jeanne Shen, on the struggles of maintaining a healthy work-life balance, recovering from setbacks and her excitement at the future of AI in medicine…


Dr. Jeanne Shen is Assistant Professor, Gastrointestinal and Hepatobiliary Pathology at Stanford University School of Medicine. Her research interests include gastrointestinal, liver, and pancreatobiliary pathology, with major emphasis on neoplasia, inflammatory bowel disease, and the application of artificial intelligence to digital pathologic image analysis.

Dr. Shen’s combined expertise in diagnostic pathology, cancer biology, and AI-related study design and data analysis has helped her serve as a principal investigator and collaborator on multiple academic and industry-sponsored studies in pathology and oncology.

She also contributed her expertise as a medicolegal, startup, and venture capital consultant, and serves as an associate director and executive committee member of the Stanford Center for Artificial Intelligence in Medicine & Imaging (AIMI).


What initially sparked your interest in medicine?

Biology was always one of my favorite subjects as a kid. I loved the plants and the fossils and the frogs! In high school, I joined the science bowl team and became the designated bio person. We had so much fun and even won the national championship that year. During my undergraduate at Stanford, I was a biological sciences major. Back then, I was not so much interested in medicine as I was in biotech. I wanted to do a PhD and maybe an MBA, combining biology and business. That changed after I volunteered at a clinic providing free basic healthcare services to the uninsured population in San Jose. It was called the Pacific Free Clinic and I believe it’s still around today.

It was founded by several Stanford physicians, and I would spend a couple of hours every other weekend helping provide primary care services to those in need. I remember meeting a woman who came to the clinic with uterine bleeding. She was around the same age as my mother, and ultimately was diagnosed with endometrial cancer. In assisting her through the diagnostic and referral process, that was the first time that I had a chance to bond and establish a personal connection with a patient. That became the turning point for me to go into medicine – being able to experience the human aspect of medicine, rather than just the science aspect.

How did you get into medical AI?

I moved back to the Bay Area and became an assistant professor at Stanford in 2017. The proximity to Silicon Valley, a place so rich with tech innovation, and the fact that Stanford is such a great institution for computer science and biomedical data science, made it the ideal setting for me to get into AI. My first AI project was with Andrew Ng’s lab in Computer Science. Andrew is an extremely inspiring person who has done so much for the field of AI and online education. He is also really nice to work with, and an incredibly humble guy who I think genuinely wants to make a positive impact on the world.

The first idea that I pitched was trying to predict microsatellite instability (MSI) in colorectal cancer. MSI used to be just a biomarker for determining whether a patient might have Lynch syndrome, which is a hereditary cancer predisposition syndrome. What we now know is that it might also help predict response to conventional chemotherapy and immune checkpoint inhibitors. I thought, wouldn’t it be cool if we could directly predict what the MSI status was from routine H&E-stained pathology slides? So we began working on that, but weren’t able to achieve a model performance that we were satisfied with. We decided that perhaps we were being too ambitious and abandoned the project altogether.

Ironically, later on during my time with another group, we managed to get it all to work out and published a paper in The Lancet Oncology. In retrospect, I think that our expectations for model performance were too ambitious in the beginning, and that this was partly a consequence of the strong positive publication bias in the field of medical AI. Particularly when you’re getting into clinician-AI performance comparisons, most journals at the time weren’t willing to publish anything less than headline-worthy results where the AI model significantly outperforms human experts. In fact, the very first AI manuscript that I submitted got rejected for that reason. The feedback was, “It is unfortunate that the model did not improve the performance of the pathologists.” I’m not sure if the situation has improved now. I really hope so, as I think that it’s extremely important to publish all of the negative findings as well, so that we can collectively learn what does and doesn’t work.

Did you ever consider a different career?

I started taking painting lessons when I was around 10 and so when I was in high school, I wanted to go to art school and become a portrait painter. That was my ‘secret’ first choice, because my parents were not supportive. Painting was a much less practical career choice compared to, for example, a biology degree. I believe that my life would have been quite different if I had gone down that route. But then again, there are so many potential directions in life that you can take, and there’s no sense in lingering over the ‘what-ifs.’ I still love art and art history and I may pick up painting again when I retire.

What are you most excited about with AI in pathology?

One of the things that got me excited initially and continues to excite me is the potential to derive novel insights from routine H&E-stained whole slide images. It was only after we started applying AI that we began to realize there’s a lot of hidden or sub-visual information present in the tissue that we may not be able to pick up, but that computer vision algorithms can pick up, and that this information might help us better predict a patient’s prognosis, or what treatments they might respond to. There are details and patterns that reflect the underlying disease biology which human eyes cannot appreciate or quantify well, but that machines can pick up on quite well. I’m hopeful that AI-based histomics will help us derive new insights and predict important clinical outcomes that we weren’t able to predict before.

I often think of pathology AI tasks as falling into two main buckets. One is helping pathologists do the things they already do, but more accurately or efficiently. Many of the tasks that pathologists perform are manual and very tedious, or subject to interobserver variability in diagnosis. For example, counting mitotic figures or Ki-67 stained nuclei is quite popular now, and it’s an area where many researchers and companies are working because it’s an essential part of grading for several tumors. It’s pretty time-intensive to hunt through every tumor slide to figure out which high-power field contains the highest proliferative rate and to do a manual count. I am quite sure that some companies are actively working on automating this process right now, and I look forward to seeing some tools come out that can actually be used for primary clinical diagnosis. The second type of AI task is helping pathologists do things that they currently can’t do very well, if at all, and much of this falls under the area of histomics. I believe that AI can help us with both types of tasks. All these potential benefits from AI excite me, and I feel that we will be able to see some real progress in the coming years.

As compared to radiology and cardiology, pathology is probably the least digitized domain. What other challenges do you see?

Digitization is actually the major barrier to overcome right now. Before we can start to think about applying AI to pathology, the slides need to be digitized so that we can run the models on them. Scanners are expensive and not many pathology labs can afford them. There’s also the need to consider the histology laboratory and pathologist workflow. A major re-organization of the workflow must occur when going digital. Just digitizing a full batch of slides is probably going to take six to eight hours – if this is not done on time, it’s going to disrupt the downstream clinical workflow and lead to delays in diagnostic turnaround time. Therefore, you need to re-organize work shifts in the histology lab or hire additional technicians so that histology processing and scanning can take place around the clock.

Also, if you were to review the pathologist workflow before and after digitization, you might notice one major disadvantage; it’s simply not as fast reviewing a case digitally as it is reviewing the glass slides on a microscope. Furthermore, the technology isn’t perfect yet, so some of the whole-slide images might be blurry or have parts that weren’t scanned. In such cases, the pathologist would still need to request that the glass slides be sent over for manual review. Internet connection speeds may also be an issue, as the whole-slide images may take a few seconds to fully load when you’re moving around and zooming in and out. All of these things can contribute to frustration when you are looking at a large case with many slides, especially if reviewing them digitally is taking twice as long as reviewing them on a microscope. I think that we are still in the early stages of digital pathology adoption, and that many of these bottlenecks will be resolved in the future.

Is there anything we can do better to address the challenges of gender and racial disparities in medical AI?

This is certainly an important topic, and I am glad that it’s getting more attention now, because biased datasets can cause AI models to yield misleading outputs. I believe that AI is capable of both contributing to and mitigating healthcare disparities. Being aware of the potential impact of gender and race on how AI models perform, and consciously designing our datasets, models, and analyses to take into consideration these variables, is a first step in reducing the risk of bias.

Who has been the biggest influence on your career?

Many people have positively influenced my career, so it’s hard to name just one person. But, for medical AI, Dr. Matt Lungren is a great friend and someone I often look to for advice, as an example of a successful AI physician scientist. There aren’t many clinicians who have also managed to establish a career in AI. I am amazed at how he manages to do it all, from managing his clinical duties to research to teaching – he is doing so much and achieving so much.

Dr. Lungren, together with Dr. Curt Langlotz, who has also been a great influence on my career trajectory, came up with the idea of setting up the Stanford AIMI Center. They essentially started from scratch and have managed to attract so many talented people who are excited about applying AI to healthcare. AIMI has grown tremendously over the past few years. When I first met Dr. Lungren, he hadn’t yet established his own lab, and we would often discuss how challenging it was to break into the field of medical AI, even though we were so excited about its potential. He has offered me a lot of good advice and showed that it’s possible to balance a career in clinical medicine and AI.

What do you consider your biggest achievement and conversely, failure?

To be perfectly honest, my biggest achievement is that I haven’t yet quit! I sometimes laugh about it, but there have been many times in my career when the future looked bleak and I questioned whether I was wasting all of my time and energy when I should just give up and do something easier. It’s definitely not been all smooth sailing in academia, and I am proud of the fact that I’ve managed to recover from some major setbacks and have made it so far without giving up.

As for failure, I’ve spent most of my life either preparing to get a job or working, and I hardly have time for anything outside of work. This has probably been the biggest disappointment. I thought that I’d at least have achieved a satisfactory balance of career and personal life by now. During the journey from undergraduate to medical school, residency, fellowship, and now professorship, I passed the milestones, but never felt as though I had space in between to take better care of myself, meet people, and settle down. I sometimes feel like this is a rat race that will only end when I retire.

Several female clinicians have mentioned that they find it hard to dedicate themselves to both work and family at the same time. Is that how you feel too?

Yes. For me, personally, I think that the pressures of the biological clock have contributed to stress surrounding work-life balance. Many of my female friends, regardless of whether they are single or married, worry either about fertility or childrearing responsibilities. Especially in medicine, the early-career stage is often critically important in setting you up for future success, and this unfortunately comes at a time when most women are trying to start or raise families. So they’re essentially forced to choose whether they want to focus on their careers or on their family.

Have you ever considered taking a short break from work to explore something for yourself?

Unfortunately, it’s never been a viable option for me. I’d love to take a year off and be a student again. There’s so much in the area of AI that I feel I still need to learn.

What’s the best piece of advice you’ve ever received?

Keep trying and don’t give up. Several of my mentors have told me that. Right now, this has been particularly applicable to applying for research funding. It’s so hard to get funding these days and many applications get rejected. The times that I didn’t give up, I eventually got funded.

Finally, what advice would you give someone just starting their career in medical AI?

Don’t try to do everything yourself all of the time. When you’re trying to learn the ropes, it might be ok to try doing a lot of the grunt work yourself, but that isn’t sustainable in the long run. If you can afford to pay someone else to do the same task that you’re currently doing, you should.

Dr. Jeanne Shen will be speaking at AIMed’s virtual multi-track CME-accredited event, ‘Imaging’ on 29th and 30th June.

View the full, exciting two day agenda and book here