AI healthcare regulation: Protecting patients from data theft and malpractice

• Author:
• Author name

  Quantumrun Foresight
• May 6, 2022

Insight summary

The healthcare sector is witnessing a surge in the adoption of AI technologies, with applications ranging from diagnosing ailments to streamlining appointment scheduling. However, this rapid integration brings challenges, including potential biases in AI systems, increased cybercrime risks, and the urgent need for a regulatory framework to govern self-learning AI devices. To navigate these complexities, there is a concerted effort to foster informed policies through the inclusion of AI specialists in regulatory teams globally, and a call for educational curricula to evolve.

AI healthcare regulation context

Hospital networks are increasingly experimenting with artificial intelligence (AI) systems in healthcare settings for diagnostics, monitoring patients’ health, and helping to schedule appointments and treatments for patients. However, to maintain the public’s trust in these new tools, healthcare AI systems must be continuously monitored and refined to minimize bias and protect against malpractice. 

Applications of healthcare-tailored AI systems are experiencing increased adoption. For example, AI can diagnose skin cancer using deep learning techniques. AI systems can also interpret radiology scans, and surgeons can leverage AI-directed robotics in operating theaters. Wearable devices can monitor and report changes in patient vital signs and determine which patients require a COVID-19 test. 

Unfortunately, due to inherent bias (either programmed in or derived through how results are interpreted) and unnecessary emphasis on specific data points, issues can arise when AI is used in the healthcare industry, potentially lowering the quality of care received by some patients. Furthermore, the World Health Organization has noted that the increased use of AI further raises the risk of unethical data collection, as well as patients and institutions being targeted by cybercrime. By January 2022, the US Food and Drug Administration (FDA) regulated some AI-enabled healthcare products to maintain standards and ensure their safety and efficacy. 

However, there is no regulatory framework (2022) for self-learning AI-enabled devices that adapt rapidly to new data sets. Some experts believe that patients should not be given a choice between AI systems and human doctors due to the complexities and challenges inherent in AI healthcare platforms. In addition, to eliminate intrinsic bias, they argue that software developers should be required to use data from diverse patients when training AI-enabled devices. 

Disruptive impact

The FDA established the Digital Health Center of Excellence to oversee the development and implementation of AI technologies in the healthcare sector, emphasizing patient safety. This initiative could potentially scrutinize AI-driven tools such as chatbots that have the capacity to review patient information, diagnose ailments, and even recommend a visit to a doctor based on the data fed into them. For individuals, this means a more streamlined and possibly more accurate initial consultation process, reducing the time spent in waiting rooms and facilitating early interventions. However, it is crucial that these systems are meticulously vetted to prevent misdiagnoses and to ensure the privacy and security of sensitive personal data.

Looking globally, healthcare regulators in various countries are contemplating the expansion of their teams to include more AI specialists and professionals, aiming to craft policies that are both informed and effective. This strategy acknowledges the nuanced understanding required to govern rapidly evolving technology adeptly. Companies, especially those in the healthcare sector, stand to benefit from clear and well-thought-out regulations, which can foster a stable environment for innovation while delineating the boundaries of ethical AI use. Moreover, associations representing healthcare workers might advocate for clear demarcations in the roles of AI systems and human workers, ensuring that the integration of AI does not compromise the quality of healthcare services.

As AI becomes increasingly integrated into healthcare, there is a growing recognition of the need to adapt educational curricula to reflect this shift. Regulatory bodies may need to collaborate closely with educational institutions, including training colleges and medical schools, to determine the extent of AI incorporation in training courses. Governments and educational institutions have a pivotal role in shaping future healthcare professionals who are adept at leveraging AI tools while retaining a deep understanding of human-centric care. 

Implications of AI regulation in healthcare 

Wider implications of AI regulation within the healthcare sector may include:

• Improved healthcare equity as more patients from diverse backgrounds will experience less bias and improved healthcare services.
• Increased resources invested into healthcare technology infrastructure to protect patient data and AI-based equipment from security breaches. 
• Technology companies hiring more healthcare professionals to provide the necessary expertise to guide the development of new software systems.
• Healthcare industry professionals requiring upskilling or training courses to use AI-based equipment in their day-to-day roles. 
• The development of ethical or responsible AI use, not just in healthcare, but in other essential sectors like energy and transportation.
• Regulations that strictly evaluate the algorithms used in healthcare decision-making to ensure that there are no biases or tampered training data used.

Questions to consider

• Do you think regulators within the healthcare industry can cope with the high pace of innovation within the healthcare field? 
• Do you think AI-based healthcare systems can replace the need for a human doctor?