This AI model is helping researchers detect disease based on coughs

From cough to speech and even breath, the sounds our bodies make are filled with information about our health. Subtle clues hidden within these bioacoustic sounds hold the potential to revolutionize how we screen, diagnose, monitor and manage a wide range of health conditions like tuberculosis (TB) or chronic obstructive pulmonary disease (COPD). As researchers at Google, we recognize the potential of sound as a useful health signal, and also that smartphone microphones are widely accessible. To that end, we’ve been exploring ways to use AI to extract health insights from acoustic data.

Earlier this year, we introduced Health Acoustic Representations, or HeAR, a bioacoustic foundation model designed to help researchers build models that can listen to human sounds and flag early signs of disease. The Google Research team trained HeAR on 300 million pieces of audio data curated from a diverse and de-identified dataset, and we trained the cough model in particular using roughly 100 million cough sounds.

HeAR learns to discern patterns within health-related sounds, creating a powerful foundation for medical audio analysis. We found that, on average, HeAR ranks higher than other models on a wide range of tasks and for generalizing across microphones, demonstrating its superior ability to capture meaningful patterns in health-related acoustic data. Models trained using HeAR also achieved high performance with less training data, a crucial factor in the often data-scarce world of healthcare research.

HeAR is now available to researchers to help accelerate development of custom bioacoustic models with less data, setup and computation. Our goal is to enable further research into models for specific conditions and populations, even if data is sparse or if cost or compute barriers exist.

Salcit Technologies, an India-based respiratory healthcare company, has built a product called Swaasa® that uses AI to analyze cough sounds and assess lung health. Now, the company is exploring how HeAR can help expand the capabilities of their bioacoustic AI models. To start, Swaasa® is using HeAR to help research and enhance their early detection of TB based on cough sounds.

TB is a treatable disease, but every year millions of cases go undiagnosed — often because people don’t have convenient access to healthcare services. Improving diagnosis is critical to eradicating TB, and AI can play an important role in improving detection and helping make care more accessible and affordable for people around the world. Swaasa® has a history of using machine learning to help detect diseases early, bridging the gap with accessibility, affordability and scalability by offering location-independent, equipment-free respiratory health assessment. With HeAR, they see an opportunity to extend screening for TB more widely across India by building on this research.

“Every missed case of tuberculosis is a tragedy; every late diagnosis, a heartbreak,” says Sujay Kakarmath, a product manager at Google Research working on HeAR. “Acoustic biomarkers offer the potential to rewrite this narrative. I am deeply grateful for the role HeAR can play in this transformative journey.”

We’re also seeing support for this approach from organizations including The StopTB Partnership, a United Nations-hosted organization that brings together TB experts and affected communities with the goal of ending TB by 2030.

“Solutions like HeAR will enable AI-powered acoustic analysis to break new ground in tuberculosis screening and detection, offering a potentially low-impact, accessible tool to those who need it most," said Zhi Zhen Qin, digital health specialist with the Stop TB Partnership.

HeAR represents a significant step forward in acoustic health research. We hope to advance the development of future diagnostic tools and monitoring solutions in TB, chest, lung and other disease areas, and help improve health outcomes for communities around the globe through our research. If you're a researcher interested in exploring HeAR, you can learn more and request access to the HeAR API.