Meta's AI Health Tool Poses Privacy Risks and Gives Dangerous Medical Advice

Meta's newly launched Muse Spark artificial intelligence model is asking users to share their raw health data, including laboratory results and fitness tracking information. While the company claims the model was designed with input from over 1,000 physicians to provide better health-related responses, testing reveals significant concerns about both privacy protection and the quality of medical advice provided.

When prompted about its capabilities, Muse Spark explicitly encouraged users to upload sensitive health information: "Paste your numbers from a fitness tracker, glucose monitor, or a lab report. I'll calculate trends, flag patterns, and visualize them." The bot even provided examples of how users might share personal health data, such as blood pressure readings. This approach mirrors similar features offered by competitors like OpenAI's ChatGPT, Anthropic's Claude, and Google's Fitbit integration, all of which allow users to connect personal health data for AI analysis.

However, medical experts and privacy advocates express serious reservations about this practice. Monica Agrawal, an assistant professor at Duke University and cofounder of Layer Health, warns that while providing more data context might improve AI responses, it creates "major privacy concerns" without proper protections. Critically, these popular AI tools are not compliant with HIPAA, the United States law that protects patient privacy in medical settings. Meta's privacy policy explicitly states that data shared with its AI may be stored and used to train future models, and the company may use interactions to tailor advertisements to users.

Medical professionals refuse to use these tools with their own health information. Dr. Gauri Agarwal, an associate professor at the University of Miami, emphasizes her reluctance: "I certainly wouldn't connect my own health information to a service that I'm not fully able to control, understand where that information is being stored, or how it's being utilized." Dr. Kenneth Goodman, founder of the University of Miami's Institute for Bioethics and Health Policy, advocates for caution, stating that using these tools "without due diligence is dangerous" and calling for research proving their health benefits before widespread adoption.

Beyond privacy concerns, the quality of medical advice from Muse Spark raises alarming questions. When tested with health questions, the bot demonstrated troubling tendencies to follow user suggestions without appropriate safeguards. In one test, when asked about extreme weight loss through intermittent fasting, Meta AI created a meal plan recommending only 500 calories daily—despite acknowledging risks of eating disorders. Such responses could prove catastrophic for vulnerable individuals.

Experts note that AI chatbots are inherently susceptible to being shaped by how users frame questions, potentially leading to biased or harmful recommendations. Additionally, Meta's previous public sharing of AI conversations revealed that some users inadvertently had sensitive medical questions and embarrassing prompts broadcast publicly, highlighting unexpected privacy vulnerabilities.

While Meta positions Muse Spark as an educational tool comparable to "a med school professor, not your doctor," this distinction provides little comfort to medical professionals who view the stakes as too high. The current regulatory environment leaves users largely unprotected when uploading sensitive health data to commercial AI platforms, creating a significant gap between the intimate nature of health information and the loose protections governing its use by artificial intelligence systems.