Get Your Medical Mobile App Certified by IEEE

Patients who use mobile apps to manage medical conditions including depression and chronic pain may assume that the apps have been tested by regulatory agencies to be safe and effective. But that’s not really the case.

Most of the more than 55,000 medical apps that claim to diagnose or treat a condition—or those that provide clinical decision support, known as “therapeutic” apps—have never been evaluated by trusted neutral bodies or regulatory agencies for technical validity, ethical design, or clinical benefit. Apps often don’t comply with regional data security and privacy laws to protect people’s sensitive health information.

Medical apps differ from traditional health apps, which provide users with information to improve their health, for example, by tracking fitness activities, monitoring blood pressure, and analyzing sleep patterns.

There is no reliable way to ensure that therapeutic applications deliver the results they show. To help ensure that such apps are reliable, the IEEE Standards Association (IEEE SA) recently launched the IEEE Global Medical Mobile App Assessment and Registry. The publicly searchable directory is designed to list expert-reviewed applications across several criteria including technical soundness, ethical design, compliance with data security and privacy laws, and clinical effectiveness, which is evidence of clinical benefit to the patient.

“Patients, physicians, payers, and health care systems often struggle to distinguish meaningful medical applications from well-marketed ones,” said IEEE Senior Member Yuri Quintana, chair of the evaluation and registration program. He is chief of clinical informatics at Beth Israel Deaconess Medical Center, Boston. “Our goal is to establish a standardized review method using methods developed by experts.”

Why is there no law

Because applications are intended for medical use without being part of a medical device, they fall under the designation of software as a medical device (SaMD), according to the International Medical Device Regulators Forum. SaMD is supposed to be regulated by public health agencies like the US Food and Drug Administration, but the apps have developed and grown in popularity so quickly that regulators haven’t been able to keep up, Quintana said. Some companies have received approval, but many have not, he said.

Most users don’t know about the regulatory gap, he says.

“Seeing an app from a well-known company often creates the impression that it has been vetted for safety and effectiveness, even if it isn’t,” he said.

Some companies use deceptive advertising to sell their products, he adds. Marketing materials may say that all of a company’s health apps are certified, even if only one app is approved by a regulatory body to treat a specific condition. Or the verbiage may say that the company has clinical evidence that its application works, even though the application has not been independently tested.

Another concern is that the revised applications are not being vetted, said Maria Palombini, IEEE SA’s director of global health care and life sciences.

“The original app may have received regulatory agency approval, but not the updated version,” Palombini said. “There may be significant changes from the beginning.”

“Not all medical-related apps trigger the same regulatory framework or reviews in all areas,” Quintana added. “That leaves a large gray area for clinically relevant but low-risk applications that have yet to be independently tested. The IEEE registry was created to help fill these gaps.

“IEEE is the best organization to deal with this problem because this is basically a standards challenge, trust, interoperability, and compatibility testing challenge,” he said. IEEE is “the world’s largest technology organization, with deep expertise in developing globally recognized standards including healthcare, cybersecurity, AI principles, and interoperability.”

“Through the IEEE Conformity Assessment Program, we are already conducting testing and registration programs,” said Palombini. “Our neutral, consensus-driven approach, which brings together practitioners, regulators, developers and ethicists without bias—puts IEEE in a unique position to create reliable global pathways that can reach all jurisdictions and support regulatory consensus.”

How the registry works

The evaluation framework was created by a multidisciplinary team of 35 volunteer experts from 10 countries, Quintana said. The panel includes academics, AI experts, application developers, doctors, mental health professionals, patient advocates, regulators, researchers, experts, and those who assess safety in health care.

Registration is for any application used for clinical care or treatment that claims to demonstrate medical benefit. That includes apps designed for heart disease, diabetes, mental health, neurology, oncology, rehabilitation, and respiratory diseases, Quintana said.

Initially, he says, the focus will be on programs that aim to treat mental health conditions, given the large number of donations in that area and the experience of the registration committee.

Submission of applications is voluntary. There is no government mandate that requires a company to use IEEE registration.

Products will be evaluated against approximately 150 consensus-based criteria in three major areas:

• Clinical practice including therapeutic efficacy, any sustained benefits, risk management, comparison with standard care, user engagement, and actual clinical value.
• Technical sound including accessibility, privacy and security, error handling, interoperability, AI management, usability, and performance quality.
• Behavioral design including prevention of bias, patient consent, data management, disclosure of conflicts of interest, responsible use of AI and linguistic models, and prioritization of public health benefits.

IEEE charges a non-refundable shipping fee that covers the cost of the evaluation and annual subscription for the first year’s subscription.

Developers must first prove that they are a legitimate business before they can fill out the application publisher registration form and submit documents and guarantees about the product.

IEEE review of the application is estimated to take six to eight weeks, Palombini said. Test results will be shared confidentially with the app’s publisher, he says, and to be listed on the registry, an app must achieve more than 85 percent compliance in each category.

Developed applications must be submitted and retested, Palombini said. Similar to how users are notified when an app is on their smart devices, subscriptions will be notified when listed apps have a new update available, he said.

Applicants who do not pass the test must receive a response explaining why. They will be given the opportunity to make changes or provide additional documents, said Palombini.

“It’s a good process, with checks and balances,” Quintana said. “We are transparent about the process.”

Accredited apps added to the registry receive an IEEE certification badge and submission indicator, which the company can display on its website, app store listings, and marketing materials.

“The badge serves as visual proof that the app has met independent evaluations, based on consistency of clinical value, technical rigor, and ethical design,” Quintana said.

He says the registry will be publicly available for free.

Patients and families looking for safe, reliable apps—with payers and insurers evaluating the potential for reimbursement—will find the registry useful, he said.

The application website is open. The public registration page does not yet list a specific number of approved applications because testing is ongoing. Approved applications and their unique identifiers will be published once the initial review is complete.

To learn more, you can watch the webinar recorded in March.

The assessment framework that supports registration supports the formal recognition of the IEEE P3962 Standard for Criteria Assessment Framework f