ISO 42001: How AI Governance Makes Companies Safer

AI sends phishing emails, accesses third-party systems, or injects vulnerabilities into software. What sounds like a worst-case scenario has already happened in tests of powerful AI models.

How can ISO 42001, the EU AI Act, and ISO 27001 help companies establish effective AI governance? This article shows how the use of AI, information security, accountability, and skills development can be combined to manage risks such as shadow IT, data breaches, and uncontrolled access.

In this post:

In a nutshell: ISO 42001 and AI Governance in the Workplace

• The more autonomously AI systems act, the more important it is to have clear rules governing what they are allowed to do and what data they can access.
• ISO 42001 helps organizations systematically manage responsibilities, processes, risks, and opportunities associated with the use of AI.
• ISO 27001 and the EU AI Act supplement AI governance with information security and binding regulatory requirements.
• An AI inventory, data classification, guardrails, approval processes, and clear lines of responsibility are among the key components of effective AI governance.
• AI literacy is becoming a security factor: Employees must understand the capabilities, limitations, and risks of the AI systems in use.
• For companies in the defense sector in particular, the goal is not to prevent the use of AI, but to ensure it can be used in a controlled and secure manner.

Artificial intelligence is intended to speed up processes, support decision-making, and lighten the workload for employees. However, the more powerful and autonomous AI systems become, the more important it is to determine what they are allowed to do, what data they can access, and who is responsible for them.

AI security, therefore, cannot be addressed through technical measures alone. The use of AI, information security, responsibilities, and capacity building must be considered together. This is precisely where AI governance comes into play: ISO/IEC 42001 establishes an international management system standard specifically for artificial intelligence. Together with ISO 27001 and the requirements of the EU AI Act, it provides companies with a framework for using AI in a controlled yet productive manner.

This is particularly relevant for defense-related companies: Where classified or export-controlled information is processed, an AI agent with the wrong permissions or an unverified shadow IT tool can have security implications. At the same time, the industry cannot afford to do without AI.

When AI Suddenly Becomes a Security Risk Itself

Several incidents from cybersecurity tests in the summer of 2026 demonstrate just how real the risks posed by powerful AI systems already are.

• At the British AI Security Institute (AISI), AI agents carried out unauthorized actions on the real Internet during security tests. Among other things, one agent attempted to inject malicious code into an open-source project and persuade a maintainer to approve it by using false identities. Internet access had been deliberately granted for the tests.
• At Anthropic, the situation was different: Claude models were supposed to operate in a isolated test environment, but due to a misconfiguration, they gained access to the open internet. In several cases, this resulted in unauthorized access to real third-party systems.
• A similar configuration error occurred during a security test of Meta’s Muse Spark 1.1. The model inadvertently gained Internet access and was directed to a real website. There, it identified a security vulnerability, exploited it, and made changes to the database.

These cases are not comparable and occurred under different testing conditions. However, they share one crucial commonality: powerful AI was combined with real-world access capabilities.

The more an AI system is able to act autonomously, the more important fundamental governance issues become:

Not just: What can our AI do? But: What is our AI allowed to do?

What the Recent Incidents Mean for Businesses

These examples demonstrate one thing above all: What matters is not only how powerful an AI is, but also the conditions under which it is used and how the test environment is configured.

Is an AI system allowed to access the Internet? Can it access internal data, applications, or email inboxes? Is it allowed to execute code or trigger actions on its own? And at what point is human approval required?

With each additional permission, the potential scope of damage increases, and with it, the importance of effective controls. This is especially true for AI agents, which can not only generate content but also plan and execute tasks independently.

It is therefore becoming increasingly important for companies to establish clear guidelines for the use of AI. These include, for example:

• defined access and authorization concepts,
• clear rules regarding permitted data and applications,
• technical safeguards and approval processes,
• Human-in-the-loop for critical decisions and
• Clearly defined responsibilities.

This doesn’t just apply to officially implemented AI systems. Employees have long been using chatbots, AI assistants, and other AI tools on their own. When this happens without authorization and without the knowledge of IT and information security, it results in shadow IT.

In the process, sensitive company data may end up in systems whose use, data processing, or security mechanisms the company has not adequately reviewed.

Not every unknown tool is automatically dangerous. However, there must be no gray area. AI governance, therefore, does not mean banning as many AI applications as possible. It is about knowing which AI is being used in the company, what risks are associated with it, and what rules apply to its use.

This is precisely why we need frameworks that go beyond individual technical security measures. ISO 42001, ISO 27001, and the EU AI Act address different aspects—and can complement each other effectively.

How ISO 42001, ISO 27001, and the EU AI Act Interact

Using AI safely and in compliance with regulations requires more than just technical safeguards. Companies must manage AI risks, protect sensitive information, and comply with legal requirements. This is precisely where ISO 42001, ISO 27001, and the EU AI Act complement one another: These three sets of standards do not necessarily mean three separate systems. They address different aspects of AI use but can be effectively integrated in practice.

An AI Management System (AIMS) thus transforms AI from a collection of individual projects into a systematically managed task.

ISO 42001 and ISO 27001: AI Governance Meets Information Security

Organizations that already use an information security management system in accordance with ISO 27001 do not have to start from scratch when establishing AI governance.

Both management systems have areas of overlap: risk management, responsibilities, documented processes, controls, continuous improvement, and the management of external service providers all play an important role.

However, the perspective is different.

ISO 27001 focuses on information security. ISO 42001 broadens the scope to include AI-specific risks and issues.

Because this raises new questions: What kinds of AI systems are actually being used? For what purpose, and with what data? What risks arise from models, providers, or autonomous functions? And when must a human intervene?

This allows information security and AI governance to be integrated with one another, rather than establishing parallel processes. This not only makes practical implementation easier for companies but also helps them integrate AI risks into existing management and security processes.

ISO 42001 and the EU AI Act: Management Systems Meet Regulation

ISO 42001 and the EU AI Act also complement each other, but serve different purposes. The EU AI Act is currently in force. It establishes specific regulatory requirements based on role, system, and risk category.

ISO 42001, on the other hand, is a voluntary international management system standard. It helps organizations manage AI through clear responsibilities, standardized processes, and systematic risk management.

The advantage: AI requirements do not have to be reorganized from scratch for every use case. Companies can establish processes to identify AI systems, assess risks, define responsibilities, and regularly review measures.

ISO 42001 can thus support the organizational implementation of responsible and compliant AI use. However, it does not replace the requirements of the EU AI Act or the need to demonstrate compliance with them.

In practice, this means that regulations set out obligations, while a management system helps to permanently organize responsibilities, processes, and controls within the company.

The Key Components of Effective AI Governance

AI governance doesn’t start with a policy. It begins with a seemingly simple question: What kind of AI do we actually have in our company?

AI is no longer limited to obvious applications such as chatbots. It is being integrated into existing software, connected via APIs, or deployed as an agent that interacts with other applications and data sources. Companies need to know what kind of AI they are using, what it is used for, and what risks are associated with it.

Building on this, clear rules for safe and responsible use can be established.

The most important components include:

Good AI governance creates a reliable framework within which companies can take advantage of the opportunities offered by AI without losing sight of the risks.

This makes it clear that governance is not the responsibility of a single department.

IT and information security must secure technical access. Business units must assess usage scenarios and business risks. Compliance and, where applicable, legal departments must evaluate regulatory requirements. Executives must establish responsibilities.

And the people who work with these systems need to understand what they’re doing.

AI Literacy: Safe AI Starts with Competence

Even the best rules are of little help if employees don’t understand why they’re important. Anyone who enters confidential data into an unauthorized AI tool, accepts AI responses without verifying them, or grants an AI agent too broad of permissions can create security risks—even without malicious intent.

This is exactly where AI literacy comes in. Employees don’t need to become AI experts. However, they should understand the risks associated with the systems in use and know how to use AI safely and responsibly within their respective areas of responsibility.

This includes, for example:

• to determine what data may be entered into an AI system,
• To critically evaluate results and identify possible errors or hallucinations,
• Assess the risks posed by automated actions and broad access rights,
• be familiar with internal rules and approval processes, and
• knowing when human oversight or consultation is required.

AI literacy is therefore an important part of AI governance—and has now also become a regulatory consideration. Article 4 of the EU AI Act requires providers and operators of AI systems to take measures to develop the AI literacy of their employees and other individuals who work with AI systems on their behalf. Among other factors, technical knowledge, experience, education, and training, as well as the specific context of use, must be taken into account.

For companies, this means that a one-size-fits-all AI training program is insufficient. Role-based skill development is crucial. Employees who use generative AI for writing or research need different knowledge than developers, administrators, managers, or employees who work with sensitive data and security-critical systems.

The ML Gruppe’s training courses on the EU AI Act and the safe use of AI provide support in developing these skills in a targeted and practical manner.

In defense-related companies in particular, AI literacy thus becomes a security measure in its own right: Those who understand the capabilities and limitations of AI can identify risks earlier and act more responsibly.

Why AI Governance Is Especially Important for Defense-Related Companies

For defense-related companies, the use of AI creates a particular area of tension: On the one hand, development, project, customer, and supplier data is often highly sensitive. Complex software supply chains, external service providers, and dual-use components further increase the number of touchpoints where information must be protected.

On the other hand, the industry can hardly afford to do without AI.

This is already evident today in potential areas of application. For example, AI can analyze large amounts of data to generate situational awareness, support decision-making, detect anomalies, or be used in autonomous and robotic systems. The German Research Center for Artificial Intelligence (DFKI) focuses specifically on such applications at its Defense AI Competence Center . Controllability, robustness, reliability, and explainability are among the key requirements for trustworthy AI systems.

The conclusion for security-critical companies cannot, therefore, be: “Prevent AI as much as possible.” It must be: “Enable AI in a controlled manner.”

The key is to make AI usable in areas where it offers real added value—while at the same time setting clear limits on data, access, and autonomous actions.

Especially in a security-critical environment, AI governance thus serves as the link between innovation and security. A systematic approach helps companies not only evaluate new AI applications based on their performance, but also take into account information protection, risks, responsibilities, and human oversight from the very beginning.

What NIS2 and BSIG Have to Do with AI Governance

For companies that fall within the scope of NIS2 or the German BSI Act (BSIG), there is an additional consideration. Section 30 of the BSIG requires critical and highly critical infrastructure operators to implement appropriate technical and organizational measures to protect their IT systems, components, and processes.

These include, among other things , risk analyses, supply chain security, access controls, the secure use of ICT systems, and training and awareness initiatives. When AI systems are integrated into such processes and systems, their specific risks should therefore also be taken into account in existing cybersecurity risk management.

This highlights the connection to AI governance: AI is not an isolated technological issue. It must be integrated into existing processes for information security, risk management, and compliance.

Important for defense-related companies: Simply being part of the defense industry does not automatically mean that a company is subject to the relevant NIS2/BSIG obligations. Whether these requirements apply—and, if so, which ones—must be determined based on the company’s specific activities and classification.

You can learn more about this in the ML Magazine article “Sabotage, Cyberattacks, Hybrid Threats: Why NIS2 Will Fail Without Training.”

Safe AI requires technology, rules, and people who understand it

AI is becoming increasingly integrated into business processes and is acting more and more autonomously. As its capabilities grow, so do the requirements for its safe and responsible use.

After all, one key lesson from recent AI security incidents is this: The more an AI can do on its own, the more important it is to define in advance what it is allowed to do. Using AI safely, therefore, means considering technology, processes, and people as a unified whole.

For companies in the defense sector in particular, the answer therefore cannot be to keep AI out of critical areas as much as possible and to stifle innovation. Rather, the key is to use AI in a controlled manner.

ISO 42001 provides a systematic management approach for this. Together with ISO 27001, the requirements of the EU AI Act, and existing cybersecurity processes, it helps companies avoid viewing AI risks in isolation. Organizations and their employees need to know: What systems are we using? What data and access rights do they have? Where do we need human oversight? Who is responsible? And do the employees involved have the necessary AI expertise?

This makes it clear that AI governance is a management responsibility. This is also evident in other areas of regulation. For companies subject to the relevant provisions of the BSIG, Section 38 requires management to implement and monitor risk management measures. Management must also participate in regular training sessions.

“Those who view AI solely through the lens of risk will fail to fully harness its potential. Those who focus only on its potential underestimate its risks. Good AI governance brings both together: It provides the security that companies need to be able to use AI boldly and effectively in the first place.”—Jaber Kakar, Head of Information Security

AI governance should therefore not be viewed as an additional barrier to innovation. When implemented correctly, it creates the conditions necessary to deploy AI where it offers real added value—without losing sight of security, compliance, and control.

Build AI Competence in a Targeted Manner

The ML Gruppe helps companies build the necessary skills in a targeted manner and translate the requirements of the EU AI Act, AI literacy, and information security into effective training programs.