Chatbot Accuracy and Reliability Evaluation (CARE) Exploration Topic

May 16, 2024: Final CARE Q&A, dated May 16, 2024 is provided.

Amendment 01 (May 9, 2024): The purpose of this amendment is to clarify page count guidance for the Stage 1, Volume 1 proposal submission as stated in Section 5, Module Announcement Responses, and to correct a typo in Section 9, Questions & Answers. All changes are highlighted in yellow in the attached CARE module announcement. The CARE Q&A, dated May 9, 2024, is also provided.

ARPA-H is launching an Exploration Topic (ET) aimed at expanding the Resilient System Office’s (RSO) funding approach associated with the interest areas included within Appendix A to the Master Announcement Instructions, ARPA-H-MAI-24-01. Exploration Topics will be announced via Module Announcements issued under the Master Announcement Instructions (MAI), ARPA-H-MAI-24-01. Exploration Topics are short-duration, fast-paced efforts with smaller, targeted awards.

Each Exploration Topic will pursue topics that strategically align with the RSO mission and provide foundational proofs-of-concept for additional future research to be built upon. The mission of the Advanced Research Projects Agency for-Health (ARPA-H) is to accelerate better health outcomes for everyone by advancing innovative research that addresses society's most challenging health problems.

Awardees will develop groundbreaking new ways to tackle health-related challenges through high potential, high-impact biomedical and health research. ARPA-H seeks proposals to develop new evaluation technologies for detecting hallucinations and other inaccuracies in medical Large Language Model (LLM) output for patient-facing applications.

The Chatbot Accuracy and Reliability Evaluation (CARE) ET endeavors to produce tools and technology that evaluates output with the efficiency of computational methods and the accuracy of human experts. By creating scalable and cost-effective LLM evaluation technologies, CARE seeks to accelerate the creation of trustworthy medical chatbots and enable access to more reliable health-related information. See the attached CARE module announcement for full project and submission details. All question regarding CARE must be sent to Ask-A-Question at: https://solutions.arpa-h.gov/Ask-A-Question