Open source AI in the company: Gamechanger or risk?

Increasingly powerful open source AI models such as LLaMA or Mistral are available, can be customized and used free of charge. This is tempting, but brings with it an area of tension: maximum freedom on the one hand and a lack of control on the other. How can companies use open source AI without taking risks in terms of security, compliance and application?

In this post:

Briefly summarized: Open source AI in the company

• Open source AI enables maximum control, but brings more responsibility
• It offers flexibility, cost benefits and independence from providers
• At the same time, risks arise in terms of data protection, shadow AI and expertise
• The decisive factor is not the technology, but how it is used within the company
• Companies benefit in particular when they develop specific skills

What is open source AI anyway?

Open source AI refers to AI models that companies can operate, adapt and control themselves. Source code, training data and model weights are publicly accessible. Model weights determine how inputs are processed and outputs are generated.

These characteristics of open source AI mean that developers and companies can freely adapt and further develop such models and integrate them into their own infrastructure. “Free” here means not only without license fees, but also independent of a single provider.

In contrast, there are proprietary, i.e. closed, systems such as ChatGPT or Microsoft Copilot. They are provided as ready-made services. The provider determines how the model is trained, what data may be processed and what costs are incurred.

With open source models, however, this control lies with the respective company that uses them. Well-known examples include Meta’s LLaMA family, the European model Mistral, the Chinese DeepSeek-R1 and Google DeepMind’s Gemma 4.

Open source AI is often developed collaboratively. This means that innovation is not created in isolation, but through the collaboration of a global developer community that continuously improves, develops and reviews models.

Typical application scenarios in the corporate context include internal chatbots and knowledge databases, the automation of recurring processes and document-based knowledge management.

The decisive factor is that open source does not automatically mean “easy to use”. Technical freedom also requires well-founded decisions in use.

Differentiation: Open source vs. proprietary AI – no either-or

The question is not: Which solution is fundamentally better? But rather: Which one is suitable for which application?

In practice, many companies will adopt hybrid strategies in the future. This means that they will use open source models where data protection, customizability and cost control are crucial. Proprietary solutions, on the other hand, will be used where quick implementation and support are paramount.

Open source is not a replacement, but a supplement. The challenge lies in making sensible decisions.

Why open source AI is now becoming an issue in companies

Open source AI is no longer a topic of the future: in 2025, open source LLMs closed the gap to proprietary models. In 2026, they are on a par or better in many areas and have long since arrived in corporate reality.

There are three central drivers for this:

Technological maturity The performance of open models such as Llama 4, Mistral Large and DeepSeek-R1 reaches a performance level that enables productive applications. At the same time, they run entirely on their own infrastructure.

Lower barriers to entry Tools, frameworks and infrastructure are becoming more accessible: what used to require supercomputers can now be run on standard server hardware or in your own cloud environment.

Increasing competitive pressure The market never sleeps: companies that use open source AI strategically can automate processes, reduce costs and react more quickly to market changes.

The result: companies need to position themselves. Not at some point – but now.

The opportunities of open source AI for companies

Open source AI offers companies three key advantages: Flexibility, cost control and independence.

1. maximum flexibility and adaptability Open source models can be specifically adapted and fine-tuned to your own data, technical terms and processes. The result: an AI system that speaks your own language and is very close to the reality of your company.

2. cost savings without dependence on providers Compared to API-based models, there are no ongoing usage fees. Analyses show: This can mean considerable savings, especially for scaling applications.

3. independence from providers Those who operate a voice model on their own infrastructure retain full control over their data and reduce dependencies (vendor lock-in). This is a key argument, especially for companies in regulated industries.

Open source AI creates strategic freedom. But only if companies use it responsibly.

The risks: Where open source AI becomes a challenge

In addition to the opportunities, open source AI entails three key risks: data protection, lack of control and skills gaps.

If you want to use technology strategically and wisely, you need to be aware of these challenges and actively address them.

1. data protection and compliance Using an AI model on your own infrastructure does not automatically solve data protection issues. The responsibility for self-operated models in particular lies entirely with the company.

Who is responsible for processing personal data? Which certifications are required? Does the architecture meet the requirements of the GDPR and the EU AI Act?

The latter has been in force since February 2025 and stipulates that all employees who work with AI must be demonstrably trained. Anyone who does not know the actual use of AI in their company cannot fulfill this obligation.

2. shadow AI: lack of control Shadow AI refers to the uncontrolled use of AI tools by employees without official approval from the company. This is not just a problem with open source models, but it should nevertheless be mentioned among the risks.

This is a particularly underestimated problem: employees use AI tools independently – without coordination.

According to a survey of over 3,500 knowledge workers worldwide, 78% openly admit to using AI tools that their employer has not approved. A Bitkom study from May 2025 shows that the unofficial use of AI in German companies has doubled within a year (from 4 to 8 percent).

Shadow AI is the logical development of the familiar Shadow IT – with one key difference: while unauthorized software remains local, company data flows uncontrolled to external servers with AI tools. This affects everything that employees enter into public AI platforms: Customer data, contracts, internal source code. This can quickly become critical. According to the IBM Cost of a Data Breach Report 2025, shadow AI-related security incidents have increased the cost of data breaches worldwide by an average of 670,000 US dollars.

3. skills gaps in dealing with AI The biggest risk is not the technology itself, but how it is used.

AI models deliver results that sound plausible but may be incorrect or incomplete. Anyone who is unable to recognize and classify this will make poor decisions based on AI outputs – without knowing it. The crucial question for companies is therefore not just: “Can our employees use AI?” – but: “Can they use, assess and control AI sensibly?” The aim here is to develop skills in a targeted manner so that employees gain the necessary caution and confidence in dealing with AI:

• Understanding the functionality and limitations of the models,
• the ability to critically evaluate results and
• knowing when and how AI can be used safely in your own work context and what security risks there are.

Companies that invest in these skills are the ones that actually benefit from open source AI.

It is crucial to build and anchor this capability in the long term through a combination of

• practical learning formats that are close to the working reality of your teams
• Continuous development instead of one-off training
• a combination of training, enablement and guidelines.

Classification: game changer or risk? Open source AI is both: a tool that creates enormous value in the right hands and becomes a liability risk in the wrong ones. The difference lies in the ability to understand, control and use it responsibly.

What companies should do now: 6 specific recommendations

To use open source AI successfully, companies should implement six key measures:

1. make AI use in the company transparent Get a clear picture of the current situation. Which AI tools are already being used – officially and unofficially? Where do needs arise that are not covered by existing systems? This transparency is the basis for everything else.

Only those who know what is being used can control it sensibly.

2. clearly define guidelines and governance Guidelines, policies and responsibilities must be established at an early stage, not just after the first incident. This means: define clear rules on which tools are permitted and for which tasks. Define responsibilities in IT, Legal and HR and ensure close coordination between these areas. Compliance and security are not an afterthought, but part of the architecture.

Clear guidelines provide orientation.

3. set up a targeted technical infrastructure If you want to use open source models, you need to deal with a number of crucial questions in advance. Clarify all requirements for data storage, access rights, encryption and monitoring before you start. A well thought-out setup is the prerequisite for ensuring that employees work with AI securely – and not in secret.

Safe environments are the basis for responsible use.

4. start pilot projects with clear guidelines Don’t do everything at once! It makes more sense to consciously select areas of application, launch pilot projects with clear goals and limits and learn from these experiences. This results in successes that build trust internally – and mistakes that are still manageable.

Small, clearly defined use cases provide quick insights.

5. train employees in a targeted manner – not just inform them Introducing a tool and issuing an information sheet on the AI usage guidelines is not competence building. What works are practical learning formats that are close to the real working context.

Employees must understand

• how AI systems work
• how results are evaluated
• where risks lie.

In other words, they teach your employees how AI can be used in their own tasks, where the limits are and how results can be critically evaluated. The EU AI Act has made this training mandatory since February 2025. But regardless of this, the following also applies: only those who really understand what AI can and cannot do can work with it safely and profitably.

Only real empowerment can turn use into safe and effective application.

6. anchoring competence development as a continuous process Competence is not developed in a one-off training course. Companies that want to benefit in the long term anchor skills development as an integral part of their learning culture. In addition, AI is constantly evolving. What applies today may be outdated in six months’ time.

What works:

• Practical learning formats
• Learning in the work context
• Continuous further development

This creates security in dealing with AI and turns technology into real added value.

Conclusion: Open source AI is neither a risk nor a solution – but a tool

Open source AI offers enormous potential. Companies have the opportunity to benefit from real advantages such as flexibility, cost savings and data sovereignty.

At the same time, it also brings challenges.

Whether it becomes a game changer does not depend on the technology, but on how companies deal with it.

Companies that focus on building genuine AI expertise now will benefit from this development.

After all, the success of open source AI is not determined by IT – but by the skills of the employees.

Technology changes little – if people cannot use it effectively.

We help companies do just that: with learning formats that work – practical, contextualized and tailored to the reality of your company. This is how the new succeeds. So that knowledge arrives.

Get in touch with us – we’re listening.

Are you planning to use AI in your company?

We support you in quickly and legally anchoring the innovative benefits of AI systems in your company. Get in touch with us.