As large language models (LLMs) become more common, a comprehensive understanding of the LLM threat landscape remains elusive. But this uncertainty doesn't mean progress should stand still: exploring AI is critical to staying competitive, which means CISOs are under tremendous pressure to understand and respond to emerging AI threats.
While the AI threat landscape changes every day, we know that there are some LLM vulnerabilities that pose a significant risk to today's business operations. If network teams have a strong grasp of what these vulnerabilities are and how to mitigate them, organizations can continue to innovate with LLMs without taking unnecessary risks.
For LLMs, the possibility of data breaches is a real and growing concern. LLMs can be "tricked" into revealing sensitive business or user information, leading to a range of privacy and security concerns. Timely leaks are another big problem. If a malicious user accesses the system prompt, the company's intellectual property may be compromised.
Both vulnerabilities are related to prompt injection, an increasingly popular and dangerous hacking technique. Both direct and indirect rapid injection attacks are becoming more common and come with serious consequences. A successful prompt injection attack can result in cross-plugin request forgery, cross-site scripting, and training data extracts, each of which puts company secrets, individual user data, and basic training data at risk.
As a result, enterprises need to implement inspection systems throughout the AI application development lifecycle. From ingesting and processing data to selecting and training applications, each step should include restrictions that reduce the risk of non-compliance. When dealing with LLMs, regular security practices such as sandboxing, whitelisting, and API gateways are just as valuable, if not more. In addition to this, teams should carefully review all plugins before integrating them with LLM applications, and human approval should still be essential for all highly privileged tasks.
The effectiveness of an AI model depends on the quality of the data. But throughout the model development process, from pre-training to fine-tuning and embedding, training datasets are vulnerable to hacking.
Most businesses utilize a third-party model where unknown people manage the data, and the network team can't blindly trust that the data hasn't been tampered with. Whether you use a third-party model or your own, there is a risk of "data poisoning" by bad actors, which can have a significant impact on model performance and damage brand reputation.
The open-source autopoison framework provides a clear overview of how data poisoning affects models during instruction tuning. In addition, here are a range of strategies that network teams can implement to reduce risk and maximize AI model performance.
Chain Review: Double-check the chain to verify that the data source is clean and has impeccable security measures in place. Ask questions such as "How is the data collected?" and "whether appropriate consent and ethical considerations have been taken into account." You can also ask who tagged and annotated the data, their qualifications, and if there were any biases or inconsistencies in the labels. Also, address data ownership and licensing issues, including who owns the data and what the license terms and conditions are.
Data Cleansing and Cleansing: Before entering the model, be sure to check all the data and **. For example, a PII must be edited before it can be placed in a model.
Red Team Exercises: LLM-focused red team exercises are conducted during the testing phase of the model lifecycle. Specifically, prioritize testing scenarios that involve manipulating training data to inject malicious**, biased, or harmful content, and employ a variety of attack methods, including adversarial input, poisoning attacks, and model extraction techniques.
Advanced models such as GPT-4 are often integrated into systems that communicate with other applications. However, whenever APIs are involved, downstream systems are at risk. This means that a malicious prompt can have a domino effect on interconnected systems. To reduce this risk, consider the following:
If LLMs are allowed to call external APIs, ask for user confirmation before performing potentially disruptive actions.
View LLM output before disparate systems are interconnected. Check them for potential vulnerabilities that could lead to risks such as Remote Execution (RCE).
Pay particular attention to the scenarios in which these outputs facilitate interaction between different computer systems.
Implement strong security measures for all APIs involved in the interconnected system.
Use strong authentication and authorization protocols to prevent unauthorized access and data breaches.
Monitor API activity for signs of unusual and suspicious behavior, such as unusual request patterns or attempts to exploit vulnerabilities.
Network bandwidth saturation vulnerabilities can be exploited by attackers as part of a denial-of-service (DoS) attack and can have a painful impact on the cost of LLM usage.
In a model denial-of-service attack, the attacker interacts with the model in a way that consumes excessive resources, such as bandwidth or system processing power, ultimately compromising the availability of the target system. Businesses, in turn, can expect service degradation and sky-high bills. Because DoS attacks are not new in the cybersecurity space, there are several strategies that can be used to defend against model denial-of-service attacks and reduce the risk of rapidly rising costs.
Rate limiting: Implement rate limiting to prevent the system from being overwhelmed with too many requests. Determining the correct rate limit for your application will depend on the model size and complexity, hardware and infrastructure, as well as the average number of requests and peak usage time.
Character limit: Set a limit on the number of characters a user can include in a query to protect LLM-based APIs from resource exhaustion.
Framework-provided approaches: Leverage the methodology provided by the framework provider to strengthen defenses against attacks. For example, if you're using langchain, consider using the max iterations parameter.
Securing LLMs requires a multifaceted approach that includes careful consideration of data processing, model training, system integration, and resource usage. However, by implementing the recommended strategies and remaining vigilant, businesses can harness the power of LLMs while minimizing the associated risks.