Image via Monash University

As the creation of both information and misinformation escalates, the role of Generative AI becomes increasingly significant. This cutting-edge facet of AI technology is redefining what’s possible, enabling the generation of new information and ideas at a pace and scale previously unimaginable. In the face of endless possibilities, it must be acknowledged that AI is only in its first generation and that we must let AI grow and mature as our digital world adapts.

Artificial intelligence today is not just an auxiliary tool, but a central player in daily operations across various sectors. Yet, this rapid advancement is not without its pitfalls. The unchecked use of Generative AI poses a substantial cybersecurity risk, potentially opening the door to a new wave of cyber-attacks.

It is crucial to examine the potential cyber risks from the adoption of AI for industries like healthcare and finance that are common targets for cyber attackers. In this white paper, we will explore the implications of Generative AI within these sectors and identify strategies to mitigate the associated cyber threats.

Generative AI in the Healthcare Industry

Image via Medical Device Network

In the realm of healthcare, Generative AI is becoming an indispensable tool, offering solutions that extend the capabilities of medical professionals and enhance patient care.

Sophisticated AI-powered chatbots are at the forefront of this transformation, providing a level of customer service that is both efficient and deeply human in its interactions. These virtual assistants are available around the clock, offering patients immediate responses to their health-related inquiries and support when they need it most. However, the intelligence that enables these chatbots to mimic human conversation also makes them targets for adversarial attacks. Malicious actors may exploit these systems, seeking to elicit sensitive information or spread misinformation, which could have serious repercussions for patient health, safety and trust.

Generative AI plays a pivotal role in drug discovery and the formulation of personalized treatment plans, elevating the prospects of positive patient health outcomes. Yet, this progress is not without its challenges. The intricate nature of drug discovery often involves handling sensitive and proprietary information. Without robust  protection measures, the door is open to potential unauthorized information disclosure through cyber attacks, jeopardizing valuable research, patient health and intellectual property.

In the healthcare professional’s realm, the implementation of Generative AI aims to ease the administrative burden on doctors and nurses. This involves the extraction of information from conversations between physicians and patients to generate medical notes, facilitating the integration with electronic health record (EHR) systems. However, the advancement comes with a cautionary note. Insecure information transmission channels pose a potential risk, exposing patient information to interception or eavesdropping, emphasizing the need for secure communication infrastructure.

Furthermore, the automation of clinical document creation, such as discharge summaries, showcases another facet of Generative AI’s impact on healthcare efficiency. This innovation saves valuable time for healthcare professionals, streamlining patient care. However, the potential for malicious tampering of generated clinical documents raises concerns. Ensuring the integrity of these documents is paramount to prevent inaccuracies in patient information that could have far-reaching consequences.

Generative AI in the Financial Industry

Image via PaymentsJournal

As the financial industry navigates digital transformation, Generative AI stands as a beacon of innovation, offering new ways to engage with customers and streamline complex financial processes.

Banks leverage Generative AI to deploy chatbots that assist customers in handling routine inquiries. Rather than navigating a menu of options, customers can ask questions to obtain precise solutions for their financial queries. However, this convenience could be compromised if rigorous input validation and filtering are not in place, potentially allowing attackers to manipulate these systems to extract sensitive information.

Beyond customer interactions, banks invest significant efforts in managing internal information and documents. Bank staff members spend a considerable amount of time searching for and synthesizing information from a huge amount of documents. Generative AI proves instrumental in assisting bank staff with the swift location and comprehension of information within various documents, from policies to regulatory papers and other unstructured documents. Nonetheless, maintaining strict gatekeeping is essential within a bank’s environment, as inadequate measures may lead to unauthorized personnel gaining access to sensitive financial information.

The dynamic nature of regulations demands constant adjustments to code for compliance. Generative AI plays a pivotal role in automating code adjustments and supporting documentation, ensuring seamless adherence to regulatory changes. However, vulnerabilities arise if the AI-generated code lacks proper validation and sanitization of inputs, potentially exposing it to injection attacks.

Now moving on to the capital markets, Generative AI tools empower the investment analysts by swiftly summarizing critical information from diverse sources like transcripts, company filings and reports. This allows the investment analysts to gain a comprehensive understanding of the global markets. However, the potential manipulation of source documents by malicious actors poses a risk, potentially distorting the AI-generated summaries and influencing investment insights. 

Insurance firms and high-net-worth fund managers leverage Generative AI tools to fine-tune and enhance financial products. These entities input customer personas to generate customized offerings, benefiting both customers and firms involved. Nevertheless, ongoing monitoring is essential to prevent sensitive information loss and the inadvertent disclosure of intellectual property throughout this process.

The Double-Edged Sword of Generative AI in Cybersecurity

AI is not only adding risks to companies who use it, but it is also empowering a new age of hackers. Generative AI is being used as a weapon by hackers across the world, making them more productive and effective.

Brian Roche, Chief Product Officer of application security company Veracode, explains the malicious side of AI in cybersecurity. “Hackers are using AI to automate attacks, evade detection systems, and even create malware that can mutate in real-time,” Roche said. “What’s more, Dark Web forums are already full of discussions on how to use Generative AI platforms, like ChatGPT, to execute spear phishing and social engineering attacks.”

Instead of purchasing pre-made ransomware code from the Dark Web, some attackers are now using Generative AI to create their own malicious code. This code can encrypt entire systems, making ransomware attacks even more dangerous. Additionally, Business Email Compromise (BEC) scams are on the rise, especially with the increase in remote work. The FBI has received nearly 20,000 complaints about these scams in the past year. Traditional security software relies on recognizing patterns to stop BEC attacks, but when powered by Generative AI, these scams can sometimes get past security filters. Generative AI also raises concerns about hackers using it to create convincing phishing emails that are harder to detect. Furthermore, Generative AI can generate fake content, which is a significant threat. Deepfake technology, a part of this AI-driven dark side, can create realistic images, videos, and audio, enabling impersonation, disinformation, and potential social engineering attacks. 

We’ve seen how companies are suffering from new AI based threat vectors that work to weaken their defenses from outside. Along with this, use of AI by businesses is leading to a new array of vulnerabilities, weakening the company’s defenses from within.

Image Copyright © BluOcean Digital 2023

This image depicts how defense in depth security layers are being weakened by AI enabled threat attacks from outside and by vulnerabilities from AI use within the company.

This unsettling development has forced the organizations into an era of unprecedented cyber risk, creating a formidable challenge for cybersecurity professionals. The conventional approaches to cybersecurity are proving insufficient in the face of these AI-driven threats. 

The New Age Information Security Framework

Both the threat landscape and vulnerabilities are fast changing, so how do companies equip themselves to protect their business better?

One way is to use AI to defeat AI.

The cybersecurity market has seen an immense rise of AI-enabled tools and technologies. These are faster, more efficient, and vastly more intelligent to prevent, detect, and in some cases, even respond to threat attacks.

Our Cybersecurity professionals at BluOcean Digital have leveraged the NIST AI Risk Management Framework, the NIST 800-53 framework, and secondary research to design an effective framework to manage the risk posed by Generative AI.

Image Copyright © BluOcean Digital 2023

This image depicts BluOcean’s proposed framework for addressing AI security risks.

The four program components we consider are:

• Govern – Establish clear and effective organizational policies, processes, procedures, and practices for dealing with AI risks.

• Map – Define the organization’s AI mission and goals, understand the business context, set limits on acceptable organizational risks, gather system requirements, and consider the socio-technical aspects to mitigate AI-related risks.

• Measure – Utilize various qualitative and quantitative tools and techniques to assess, benchmark, and analyze AI risks, including privacy risks.

• Manage – Ensure there are procedures, technologies, and skill sets put in place to manage and recover from AI risks.

Generative AI-based attacks, like other security threats, are expected to develop over time, emphasizing the need for businesses to be ready. Existing security controls that are currently in place can be improved with the integration of controls specifically targeting AI risk. Here we look at what we consider should be some key areas of focus: 

• Threat Detection: Cybersecurity experts can use trained machine learning models and AI systems to identify various hidden patterns and anomalies in the organizational network indicating potential threats. This will help in detecting malicious activities and potential breaches, thereby automatically taking appropriate action to mitigate them more quickly and accurately as compared to traditional security methods.

• Policies and Standards: Establish clear and effective organizational policies, processes, procedures, and practices for dealing with AI risks, covering the mapping, measurement, and management of these risks.

• Awareness and Training: Employee training is a crucial and proactive step towards mitigating the risk of cyber-attacks linked to Generative AI. 

• Risk Assessment:  An effective risk assessment approach prioritizes managing AI risks by considering their impact, likelihood, and available resources for mitigation. High-priority risks are identified, and response plans are created, which can involve mitigation, transfer, avoidance, or acceptance. Additionally, all remaining unmitigated risks are documented, highlighting potential harm to downstream users and end users.

• User Access Control: Security experts use machine learning models that are trained on massive amounts of unfiltered endpoint data, which can be used to identify patterns that deviate from the normal or malicious behavior through signatures of well-known attacks. Security experts can identify potential threats by observing and pinpointing unusual access used by attackers.

Conclusion

In the arms race of digital security, the key to gaining the upper hand lies in embracing the power of AI to counteract AI-centric threats. For businesses, this means a critical evaluation of current security measures and a strategic pivot towards AI-enhanced solutions that are emerging in the marketplace. These advanced solutions are not just upgrades; they are necessary evolutions to meet the sophistication of the threats we face today.

At BluOcean, we recognize that transformation is not just about adoption but adaptation. When the dust settles, the new world business leaders would be Adaptive CEO and Information Security professionals , who embrace AI with its full power and relook at their strategies and approach to drive the agenda for secured next-gen AI based business strategies.  Our approach to Business-Driven Cyber Risk Governance is rooted in rigorous research and deep expertise. We are not just responding to today’s threats; we are anticipating the challenges of tomorrow. Our comprehensive solution is designed to revolutionize existing cybersecurity frameworks, enabling companies to not only withstand but thrive against the onslaught of next-generation AI threats and vulnerabilities. This transformation encompasses a holistic enhancement of cybersecurity controls and an overhaul of the operational model. It’s about creating a synergy between the human element and advanced technology to forge a security posture that is dynamic, intelligent, and resilient.