About Idem Labs
Independent post-deployment audit of clinical AI in the medical record.
Why we built this
Three observations drove the founding.
First, clinical AI is deploying faster than the validation literature is keeping up. By the end of 2025, roughly half of US non-federal acute care hospitals were using EHR-integrated generative AI. The pre-deployment validation conversation is dense and well-developed. The post-deployment audit conversation is thin.
Second, current evaluation methods can't reliably measure what actually matters on EHR data: hallucination rate, omission rate, and demographic parity. The standard metrics were built for news articles and capture word overlap, not clinical accuracy, so the failure modes that matter most go unmeasured. The biggest vendors report workflow and time-savings numbers, but no one is measuring whether the summaries are actually true. The capability to evaluate if clinical AI summarizations are accurate let alone contain important the most important information is functionally non-existent.
Third, the existing market for AI assurance is structurally mismatched to the post-deployment audit need. Big 4 sells frameworks. AI assurance platform vendors sell software seats. Mayo Platform validates models pre-procurement. CHAI's Assurance Labs program collapsed in early 2025 and pivoted to a vetted-partner model. Nothing in the commercial market audits the AI feature your hospital already has running.
We built Idem to occupy that gap.
Founders
Matt Allison
Matt directs AI Products and Regulatory Strategy at Quest Diagnostics, where he led the launch of Quest's first patient and provider facing GenAI products (AI Companion and Clinical Insights Companion, 100,000+ weekly active users), built the LLM evaluation pipeline (human-in-the-loop and LLM-as-judge), designed the pre-market red team validation protocol, runs post-launch observability for live AI in production, and leads the audit and quality programs across the AI portfolio. He represents Quest and the American Clinical Laboratory Association in federal AI, interoperability, privacy, and Information Blocking rulemaking with HL7, ONC, and CMS.
With over 20 years in clinical quality and health IT, his previous work included VP Quality and Compliance through Pack Health's $123M acquisition by Quest. He owned audit and oversight of the patient support programs and customer service quality operations supporting Novo Nordisk's Wegovy and Ozempic across the entire US, including the FDA-required pharmacovigilance reporting workflows for both products. He also ran research on patient needs and real-world effectiveness for multiple pharmaceutical sponsors, several of those studies registered as controlled clinical trials. Across the company, he led HITRUST certification, NCQA Population Health accreditation, and internal and third-party audit management. Earlier work included $15M CDC grant operations across hundreds of FQHCs at the American Cancer Society and CMS quality reporting across 74 facilities at the Alabama Quality Assurance Foundation.
Idem Labs is Matt's independent venture, structurally separated from Quest by documented firewall.
Josh Ward, Ph.D.
Josh Ward earned his PhD in Statistics from UCLA, where he worked in the AI Trustworthiness Lab under Professor Guang Cheng. His research focuses on auditing the privacy and safety of modern AI systems, particularly quantifying the privacy risk of generative models operating under regulations like FERPA and HIPAA. He developed a body of work on privacy quantification in synthetic and tabular data generation, built open testbeds for auditing that leakage, and published methods for group fairness and out-of-distribution risk control in high-stakes classification; his work on risk control for medical machine learning models received a best paper award from Clinical and Translational Science, and his research has appeared at AISTATS, PETs, and KDD.
Josh has paired that research with a track record of building production systems. He spent three summers as an applied scientist on Amazon's Search Experience team, where he designed a novel reinforcement learning algorithm for query auto-completion that improved recommendation recall by 10% and researched methods for debiasing language models. At Accenture's computer vision R&D group he built and deployed image-detection models for a $40B+ utility client to reduce wildfire liability, work that contributed to a patent in the firm's portfolio. Before his PhD he architected the survey and analytics platform at FIGS through its $5B IPO, sending over a million targeted surveys a year and authoring nearly half the company's committed data infrastructure code. He also worked as a statistician at a top public opinion research firm, where he designed the sampling and survey methodology and built the voter likelihood models and message-testing experiments behind campaigns for clients ranging from sitting senators to major technology companies during the 2018 election cycle.
Read the evidence.
Then let's talk.
The June 2026 white paper documents what EHR-native AI features are doing in the medical record and what your organization is on the hook for.