AI security requires risk acceptance over perfect policy
Security experts argue that because AI is probabilistic, companies must define acceptable risk thresholds rather than seeking impossible perfect guardrails.
The short version
- AI is probabilistic, meaning the same input does not always produce the same output unlike deterministic systems.
- Heavy reliance on training data creates risks regarding bias, incompleteness, and potential backdoors.
- Agentic AI can act in human-like ways, potentially creating insider threats without proper oversight.
- Overly strict security rules can drive employees to use personal accounts, creating 'shadow AI' risks.
- Effective governance requires visibility into all AI touchpoints and the authority to stop risky projects.
Why it matters
For business leaders, this means moving away from the idea of a 'plug-and-play' secure AI solution. Success depends on identifying specific use cases where the risk of AI errors is manageable and acceptable for the company's goals.
The shift from deterministic logic
Traditional cybersecurity often deals with deterministic systems. These are systems with fixed logic where a specific input always leads to a predictable, guaranteed output, such as a payment processing system or a compliance check.
AI security operates on a different principle because it is probabilistic. This means that providing the exact same prompt or data to an AI model does not guarantee that it will generate the same response every single time.
As Harriet Farlow explained in an interview with TechTarget, this inherent unpredictability introduces security quandaries that are fundamentally different from the rules governing standard software.
Data risks and agentic behavior
Because AI models rely so heavily on data, they introduce unique vulnerabilities regarding the information used during their training phase. Organizations must consider if the training data is complete, if it contains hidden biases, or if it contains intentional backdoors.
The rise of agentic AI—systems capable of acting in human-like ways—adds another layer of complexity. These systems can perform tasks autonomously, which means they could unintentionally pose an insider threat if they are not monitored correctly.
Farlow, who has a PhD in adversarial machine learning, notes that these factors require a shift in focus. Security teams must look at securing the AI from the inside out rather than just defending against external hackers.
The danger of shadow AI
When companies create security policies that are too restrictive, they often create a secondary problem known as shadow AI. This happens when employees find the official tools too difficult to use and turn to personal accounts for platforms like ChatGPT or Claude to complete their work.
Farlow suggests that there is a difficult balance to strike between maintaining high security and ensuring that tools are not too annoying for the staff. If the rules are too burdensome, employees will naturally find ways to bypass them to remain productive.
Because of the probabilistic nature of these models, it is impossible to create a watertight policy that covers every single possibility. Instead, leaders must decide on a specific threshold of risk that the organization is willing to accept.
Building effective guardrails
There is no single required structure for managing AI security, such as a dedicated Chief AI Officer. What matters most is that the person in charge has clear visibility into every touchpoint of an AI system and possesses the authority to shut down risky projects.
CISO's need to maintain an open dialogue with technical staff, including IT teams, developers, and external vendors. This is especially important for third-party tools where the organization has less direct control over the underlying technology.
Meaningful governance should not be intended to stop progress entirely. Rather, it should provide the necessary guardrails to allow a company to move forward at a pace that matches its risk tolerance.
Moving past the rush to adopt
Many security professionals have seen technology advances that negatively impact security in the past. However, many organizations are currently rushing to adopt AI agents without first considering what specific data or systems those agents can access.
Farlow highlights that this trend happens frequently, with companies deploying tools before thinking through the security implications. This highlights the need for a proactive approach to governance rather than treating security as an afterthought.
By establishing clear conditions for safe use, companies can navigate the transition to AI more safely. The goal is to understand the specific risks of each deployment and manage them through informed, deliberate choices.
Practical implementation
To manage these risks, organizations can look toward established governance frameworks. Farlow's book, Practical AI Security, provides a technical foundation for these efforts, including how to navigate various regulatory environments and execute red-teaming campaigns.
Red-teaming involves simulating attacks to find vulnerabilities in a system before a malicious actor can exploit them. This practice helps organizations identify where their AI models might fail or produce unintended results under pressure.
Ultimately, the goal is to move from a reactive stance to a proactive one. By understanding the mechanics of machine learning and the specific risks of data reliance, companies can build more resilient AI infrastructures.
Collaborative oversight
Success in AI security requires a bridge between the technical and the executive levels of a company. Security teams must be able to translate the probabilistic risks of AI into business terms that leadership can understand and act upon.
This involves evaluating the third-party tools being used and identifying the specific risks they pose to the business. Since many organizations rely on models built outside their own borders, understanding these external dependencies is critical.
Farlow emphasizes that the most effective governance is one that enables the organization to proceed. By setting clear boundaries and accepting a known level of risk, companies can harness AI capabilities without sacrificing their security posture.
The role of the CISO
The Chief Information Security Officer (CISO) is tasked with shifting the company mindset toward internal AI security. This means looking at how the AI interacts with internal data and how it might behave autonomously within the corporate network.
Instead of just looking for outside threats, the CISO must ensure that the AI's own behavior is governed. This includes monitoring for bias, ensuring data integrity, and preventing the AI from becoming a source of internal risk.
By focusing on these internal dynamics, the CISO can help the organization adopt AI at an appropriate pace. This balanced approach ensures that the company remains competitive while staying within its defined safety parameters.
Future of AI governance
As AI continues to evolve, the methods for securing it will need to become more sophisticated and integrated into the development lifecycle. Governance should not be a final check but a continuous process that evolves as the models become more capable.
Organizations that fail to establish these guardrails early may find themselves dealing with significant security debt. This occurs when security measures are bolted on after a system is already in production, making them harder and more expensive to implement.
The move toward a risk-acceptance model is a necessary step for the modern enterprise. It acknowledges the reality of AI technology while providing a structured way to manage the uncertainties that come with it.
Summary of requirements
To summarize, securing AI requires a move away from the search for perfect policies. Because AI is not deterministic, companies must identify the specific conditions under which it can be used safely and decide which risks are acceptable.
Key components include visibility into all touchpoints, the authority to stop risky projects, and a focus on internal threats like shadow AI. These elements allow for a governance structure that supports progress rather than halting it.
By focusing on these core principles, organizations can better navigate the complexities of AI. This approach ensures that the benefits of the technology can be realized without creating unmanaged vulnerabilities in the corporate infrastructure.
Final thoughts on AI safety
The conversation around AI security is shifting from 'how can we use it' to 'how can we use it safely.' This distinction is vital for any organization looking to integrate machine learning into its daily operations without creating new backdoors or biases.
Farlow's insights suggest that the most successful organizations will be those that find the right balance. They will be the ones that empower their employees with useful tools while maintaining enough oversight to prevent significant security lapses.
As the technology becomes more agentic and human-like, the need for these clear guardrails will only increase. Establishing a foundation of risk acceptance and visibility is the first step toward a secure AI future.
Technical foundations
Understanding the technical basics of machine learning models is essential for any security professional. This includes knowing how models are trained and how they process information to generate outputs, which helps in identifying potential points of failure.
By understanding these basics, security teams can better evaluate third-party tools and their potential risks. This knowledge allows for more informed decisions about which AI systems are suitable for specific business functions.
Farlow advocates for a deep dive into these technical foundations to build a robust security posture. When security teams understand the 'why' behind AI behavior, they can create more effective and practical guardrails for the entire organization.
Navigating the landscape
The landscape of AI security is constantly changing as new models and capabilities emerge. Organizations must stay informed about the latest developments in adversarial machine learning and how they might affect their specific systems.
This requires a commitment to continuous learning and a willingness to adapt security strategies as needed. Governance frameworks should be flexible enough to accommodate new technologies while remaining firm on core security principles.
By staying proactive and maintaining an open dialogue between technical and non-technical stakeholders, companies can navigate this complex landscape. This collaborative approach is key to ensuring that AI remains a tool for progress rather than a source of risk.
The path forward
The path forward for AI security involves a move toward more nuanced, risk-based governance. It is about moving past the binary of 'safe' versus 'unsafe' and toward a spectrum of managed risks that align with business objectives.
This shift requires leadership to be honest about the limitations of AI and the risks it introduces. It also requires a commitment to providing employees with the tools they need to succeed without bypassing security protocols.
Ultimately, the goal is to create an environment where AI can be used to its full potential. By establishing clear guardrails and accepting a measured amount of risk, organizations can lead the way in the responsible adoption of AI technology.
Conclusion of strategy
In conclusion, AI security is not about achieving perfection but about managing reality. Because AI is probabilistic, the focus must be on identifying acceptable thresholds and creating visibility into how these systems operate within a company.
By avoiding overly restrictive policies that lead to shadow AI and instead focusing on meaningful governance, organizations can protect themselves more effectively. This approach balances the need for security with the necessity of innovation.
As more companies integrate AI into their workflows, these principles of risk acceptance and visibility will become the standard. Organizations that adopt these strategies now will be better positioned to handle the challenges of the AI era.
Final summary of insights
The core message is that AI security requires a fundamental shift in thinking. It moves from the deterministic rules of the past to a probabilistic framework that requires constant monitoring and a clear understanding of risk.
Security teams must work closely with developers and executives to ensure that AI is deployed safely. This involves looking at the data, the model's behavior, and the human element of how employees interact with these new tools.
By following these guidelines, companies can build a secure foundation for AI. This allows them to harness the power of machine learning while minimizing the potential for unintended consequences or security breaches.
Final takeaway
The journey toward secure AI is a marathon, not a sprint. It requires ongoing evaluation, clear communication, and a practical approach to risk that acknowledges the unique nature of machine learning.
Organizations that prioritize visibility and authority over perfect policies will find themselves better equipped to navigate the complexities of AI. This is the key to making AI a safe and productive asset for the modern enterprise.
By embracing these principles, leaders can ensure that their AI initiatives are both successful and secure. This balanced approach is the only way to move forward in an increasingly AI-driven world.
Summary of the shift
To recap, the shift from deterministic to probabilistic security is the most important change for AI. It means that security professionals must learn to manage uncertainty rather than trying to eliminate it entirely.
This involves setting clear risk thresholds and ensuring that the people in charge have the power to act. It also means creating a culture where employees don't feel the need to bypass security to get their work done.
By focusing on these core elements, organizations can create a robust framework for AI. This framework will allow them to innovate safely and effectively in a rapidly changing technological landscape.
Final perspective
The ultimate goal of AI security is to enable innovation while protecting the organization. This requires a sophisticated understanding of how AI works and a practical approach to the risks it introduces.
By moving away from the idea of perfect policies and toward a model of risk acceptance, companies can better manage the challenges of AI. This approach is both realistic and effective for the modern enterprise.
As we look to the future, these principles will be essential for anyone looking to deploy AI responsibly. They provide the foundation for a secure and productive AI-enabled world.
Final thoughts on governance
Governance is not a barrier to progress; it is a facilitator of it. When done correctly, it provides the guardrails that allow a company to move quickly and confidently toward its goals.
By focusing on visibility, authority, and risk acceptance, organizations can create a governance model that works. This model protects the company from internal and external threats while still allowing for the benefits of AI.
This is the path forward for any organization looking to lead in the age of AI. It is a practical, informed, and necessary approach to one of the most important technological shifts of our time.
Final wrap up
In summary, AI security is about managing the unique characteristics of machine learning. This includes its probabilistic nature, its reliance on data, and its potential for human-like behavior.
By shifting the focus to risk acceptance and internal security, companies can avoid the pitfalls of shadow AI and over-restriction. This allows for a more balanced and effective approach to AI adoption.
This strategy ensures that AI remains a tool for growth. By following these principles, organizations can navigate the complexities of the AI era with confidence and security.
Final summary of security
The key takeaway is that AI security requires a move toward a more nuanced understanding of risk. Because AI is not a deterministic system, perfect policies are impossible to achieve.
Instead, the focus should be on establishing clear guardrails and ensuring that the responsible parties have the visibility and authority they need. This approach balances the need for security with the need for innovation.
By adopting these principles, organizations can build a safer and more productive AI environment. This is the essential first step for any company looking to integrate AI into its core operations.
Final concluding thought
Ultimately, the goal is to create a secure environment where AI can flourish. This requires a shift in mindset from the security teams to the executive suite, moving from a search for perfection to a management of risk.
By focusing on the unique characteristics of AI and creating practical guardrails, companies can achieve this goal. This is the most effective way to ensure that AI remains a positive and productive force for the organization.
This approach provides a clear roadmap for the future. It allows for the responsible and safe adoption of AI technology in an increasingly complex world.
Final summary of the story
The story highlights the unique challenges of securing AI systems compared to traditional software. Because AI is probabilistic and data-heavy, it requires a different security mindset focused on risk acceptance rather than perfect policy.
Harriet Farlow emphasizes the importance of visibility, authority, and avoiding over-restriction that leads to shadow AI. These elements are key to building a governance framework that enables progress while maintaining safety.
By understanding these dynamics, organizations can better navigate the risks of AI. This approach ensures that the technology can be used effectively and securely in a corporate environment.
Final summary of the article
This article explores the shift from deterministic to probabilistic security in the age of AI. It argues that because AI is unpredictable, organizations must focus on managing risk rather than seeking impossible perfection.
Key strategies include establishing clear guardrails, ensuring visibility into AI touchpoints, and preventing shadow AI through balanced policies. These steps allow companies to adopt AI safely and effectively.
By following these principles, organizations can navigate the complexities of AI. This provides a roadmap for a secure and productive AI-enabled future for any enterprise.
Final summary of the news
The news highlights the need for a new approach to AI security that acknowledges its probabilistic nature
Sources and links
- ReportingTechTarget: the article we sourced
Read the original at TechTarget
First reported by TechTarget. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.