Streamline Patient Care with a Tailored Voice Assistant Using Amazon Lex V2

Learn About Amazon VGT2 Learning Manager Chanci Turner

In recent years, physician burnout has emerged as a significant issue within the healthcare sector. While patient interaction and diagnosis are vital components of a physician’s responsibilities, the burden of administrative duties can be equally overwhelming. Physicians and clinicians are required to maintain comprehensive medical records for each patient, which are stored within the hospital’s electronic health record (EHR) system. Unfortunately, this often necessitates hours of manual data entry each day, leading to diminished productivity and heightened burnout.

Physician burnout is a leading cause of depression, fatigue, and stress among medical professionals, contributing to higher turnover rates, decreased efficiency, and costly medical errors that can adversely affect patient health.

In this article, we will explore the significance of voice assistants in automating administrative tasks for doctors. We will guide you through the process of creating a customized voice assistant using PocketSphinx and Amazon Lex.

Voice Assistants as a Solution to Physician Burnout

Voice assistants are increasingly being utilized to automate essential yet manual aspects of patient care. They serve as a valuable resource for doctors, helping to save time, alleviate stress, and allow more focus on patient interactions rather than clinical documentation.

With advancements in natural language processing, the availability of voice assistants has grown. Errors have decreased, and development has become more user-friendly for everyday developers. However, many existing devices are limited, prompting developers to create customized solutions.

As Solutions Architects in the healthcare domain, we have observed a rising trend in the adoption of voice assistants within hospitals and patient care environments.

In this article, we will demonstrate how to create a custom voice assistant using PocketSphinx and Amazon Lex. Our straightforward managed services enable developers and innovators to quickly dive into creating the next generation of healthcare devices.

Custom Voice Assistant Solution Architecture

The architecture diagram below outlines a high-level overview of our solution.

Our approach begins with a voice assistant script that operates on your computer. Once the wake word is detected, the assistant starts recording your voice and sends the audio to Amazon Lex, which utilizes an AWS Lambda function to retrieve dummy patient data stored in Amazon DynamoDB. This data is generated by a Python script, generate_data.py, also run on your computer.

Sensor types include blood pressure, blood glucose, body temperature, respiratory rate, and heart rate. Amazon Lex then returns a voice message, with Amazon Polly converting text into lifelike speech to deliver a seamless experience.

You are now prepared to develop the necessary components for this solution.

Deploy Your Solution Resources

All files related to our custom voice assistant solution can be accessed in our GitHub repository. Download all necessary files, including the PocketSphinx model files.

You will need to deploy the DynamoDB table and Lambda function by selecting Launch Stack.

The AWS CloudFormation stack will take a few minutes to complete. Once finished, navigate to the Resources tab to view the Lambda function and DynamoDB table created. Make a note of the Lambda function’s name as this will be referenced later when setting up the Amazon Lex bot.

Create the Amazon Lex Bot

After the CloudFormation stack is complete, you can proceed to create the Amazon Lex bot. We will utilize the newer V2 console for this setup.

1. On the Amazon Lex console, select Switch to the new Lex V2 console.
2. In the navigation menu, choose Bots.
3. Click Create bot.
4. Enter Healthbot for the Bot name.
5. Optionally, provide a description.
6. For Runtime role, select Create a role with basic Amazon Lex permissions.
7. In the COPPA section, choose No.
8. Keep the Idle session timeout settings at default (5 minutes).
9. Click Next.

For voice interaction, select your preferred voice and click Done.

Create Custom Slot Types, Intents, and Utterances

Now, let’s create a custom slot type for the sensors, as well as our intents and sample utterances.

1. On the Slot types page, select Add slot type.
2. Choose Add blank slot type.
3. Name the Slot type SensorType and select Add.
4. Under Slot value resolution, choose Restrict to slot values.

Add the following values:

• Blood pressure
• Blood glucose
• Body temperature
• Heart rate
• Respiratory rate

5. Click Save slot type.

On the Intents page, two intents will be automatically generated for you. Retain the FallbackIntent as default.

1. Choose NewIntent and rename it to PatientData.
2. In the Sample utterances section, add phrases to trigger this intent.

Here are a few examples, but feel free to add your own.

In the Add slot section, for Name, input PatientId, and select AMAZON.AlphaNumeric for Slot type. For Prompts, input “What is the patient ID?”

This prompt isn’t crucial as we are using Lambda for fulfillment.

Add another required slot named SensorType. For Slot type, select SensorType (the one we created earlier). For Prompts, enter “What would you like to know?” Under Code hooks, select Use a Lambda function for initialization and validation as well as Use a Lambda function for fulfillment.

Click Save intent and then Build.

The build process may take several minutes.

Create a New Version

We now need to create a new version with our new intents, as the draft version cannot be used in production.

Once the build is completed, navigate to the Bot versions page and choose Create version. Keep all settings at their default and select Create.

You should now see Version 1 listed on the Bot Versions page.

Create an Alias

Next, we will set up an Alias for deployment.

1. Under Deployment in the navigation pane, select Aliases.
2. Choose Create alias.
3. Name the Alias prod.
4. Associate this alias with the most recent version (Version 1).

Click Create. On the Aliases page, select the alias you just created. Under Languages, choose English (US).

For Source, select the previously saved Lambda function. For Lambda function version or alias, select $LATEST.

Click Save.

Now you have a functional Amazon Lex Bot that you can begin testing. Before proceeding, remember to save the bot ID and alias ID. The bot ID can be found on the bot details page, while the alias ID is located on the Aliases page.

You will need to replace these values in the voice assistant script voice_assistant.py later.

For more insights on flexible work schedules, consider reading this post on Career Contessa. Additionally, for more information on eliminating barriers in the workplace, feel free to check out SHRM.

Location: Amazon IXD – VGT2, 6401 E HOWDY WELLS AVE, LAS VEGAS NV 89115
Resource: For further details on hiring, visit Amazon’s FAQ.