COVID-19: Physician-Vetted Snippets in Search

The Pandemic's Information Crisis

The pandemic flooded the web with health information, but who ensured it was trustworthy?

During the COVID-19 pandemic, when misinformation was at its peak and lockdowns made online resources more crucial than ever, making sure the health information in Google's featured snippets and related questions was accurate became a top priority.

I took on the challenge of creating a system within Google sheets that would allow selected doctors to review and validate this information, ensuring users got the right advice when they needed it most.

My Role

As the operations lead and web developer for this project, I was responsible for:

• Designing and implementing a scalable review system with feedback loops
• Coordinating with medical professionals
• Working with lead MD to develop the validation process
• Reporting progress and insights to leadership

Google Sheets-based Review System

I built a set of tools around Google Sheets to onboard and guide doctors on how to evaluate the questions accurately, working closely with our lead MD to get everything right. The heart of the system was a setup in Google Sheets that was a combo of Apps Script and internal API's. This system:

• Fetched questions for review from our internal database
• Evenly allocated them among reviewers
• Ensured doctors only got questions in languages they were fluent in
• Tracked the review progress and status of each

Doctor Onboarding and Question Distribution

The process began by onboarding doctors, calibrating them on how to evaluate the accuracy of the questions and answers pairs. This onboarding process was developed in collaboration with the lead MD to ensure that each doctor understood the criteria for accepting or rejecting questions.

Once the doctors were onboarded, the system fetched new questions from our internal database using Google Sheets' integration with Apps Script. The questions were distributed equally among the reviewers using a priority queue, also making sure that questions were assigned to doctors who were fluent in the language.

Metrics Pipeline / Executive Reporting

I also built a metrics pipeline to track how fast we were processing everything and shared those insights with leadership every week. This way, we could ensure that the process was not only working but was doing so at a pace that met the urgent demands of the time.

To monitor the system's efficiency, I tracked key performance indicators (KPIs) such as daily throughput and the time taken from question allocation to review completion. This data was important to understand if the process was operating smoothly, especially during a period of high demand for accurate health information. Not only did I build the pipeline, but I also presented the results to leadership on a weekly basis, providing insights into the system's performance and areas for potential improvement.

• Established a system for maintaining high-quality, reliable online health information during the pandemic powered by human in the loop.
• Ensured high accuracy in COVID-19 related featured snippets, significantly reducing misinformation spread
• Expanded global reach by covering over 40 languages, providing critical health information worldwide