A clinical visit is one of the most important events that physicians gather information from and about their patients to make a diagnosis. During clinic visits, physicians ask questions related to patient’s past history of medication and symptoms. Along with other information such as lab testing results (eg. EKGs or blood test) and past medical records, physicians adjust their treatment strategy and monitor the health condition of the patients for the long term. Some clinics have scribes assigned to physicians and have them record important information and changes to an individual’s medical record. While not all clinics have scribes, physicians are usually responsible for keeping a record of how their patients’ conditions are evolving over time.

Currently, physicians need to spend valuable extra time and effort on creating and maintaining these records. With many patients, physicians are easily buried into paperwork and lose track of important information over time. Also, human error and information loss might be introduced during the process leading to bad outcomes.

In order to make the note-taking process more efficient and effective, a speech recognition and text summarizing system can be developed to capture conversations between patients, patients’ family, and physicians. The conversations can be converted into analyzable text documents. More meaning can be extracted using a summarization system, which maps important information, to pre-design templates and keywords. These keywords and templates are generated beforehand by physicians and vary by different diseases or patient types.

The system is able to keep a long-term record of the evolution of the patient’s described symptoms and conditions. All the information will automatically be stored in a structured pre-defined form that physicians can refer to easily. It frees up physicians’ time spent on

maintaining the constantly updating medical record. In addition to the benefits in efficiency improvement, the system also allows better data tagging that creates structured data. Structured data enables more sophisticated data analysis and leads to actionable insights.

There are some concerns to the implementation of the system. The effectiveness of the system will heavily depend on the accuracy of the speech recognition and text summarization. Different from traditional speech recognition and text summarization system, the challenge of this clinical note-taking system is that it has to extract important clinical information while mapping it to the pre-defined note template.



Author: Dehua Liang

Coauthor(s): Dehua Liang

Status: Project Concept