Working with local AI models: easier than you think

What is a local AI model?  

Local AI models are programs that run entirely on your own computer, without an internet connection. These programs work the same way as online AI tools like ChatGPT, Gemini, or Claude: you have a chat window where you can ask your question or give your task. You can also upload files.  

There are local AI models for various applications: text, image, video, audio, speech, etc. In this article, we primarily describe text applications because they are the most commonly used.  

What can you do with local AI models? 

Local models work best when you give them a concrete and not too complex task or assignment. For example:  

• Rewriting a grant application 
• Elaborating notes on confidential information  
• Adjusting or drafting a contract with a supplier  
• Asking questions about internal confidential documents (project plan, grant application, employee handbook) 
• Elaborating notes from a performance review 

Where local AI is less strong: tasks that require a lot of creativity, complex reasoning, or current knowledge. Local AI is also less strong in fetching information because its database is smaller than online models. For extensive data analysis or strategic advice, it’s wise to use a powerful online model.  

Local AI can thus be an alternative for specific tasks. For a large portion of daily writing and thinking work, it is good enough. 

The three most important programs 

A program is the environment in which you work, comparable to a media player. An AI model is the content you load into it, comparable to a music file. 

There are three programs that are easiest to start with. Within these programs, you can easily download the most commonly used AI models.  

Which AI models are available? 

The most commonly used and free open-source language models at the moment (summer 2026) are: 

New open-source language models are regularly launched, and their quality is improving. So make sure to stay updated on new developments. Language models are offered in different sizes: the larger the model, the ‘smarter’ it is. But also, the more computing power your computer needs.  

How to get started 

Want to start working with local AI models yourself? Follow this simple step-by-step plan: 

Step 1. Choose a program and install it. Ollama is the most commonly used.  

Step 2. Open the program and search for a model. Start with Llama 3, Gemma 4, or Mistral. Choose the largest possible model that works with your RAM.   

Step 3. Download the model, it will be stored on your computer. 

Step 4. Ask a question or give a task, just like with ChatGPT or Copilot. 

Step 5. Experiment and discover what works and what doesn’t.  

Conclusion 

Make a conscious decision: when do you use local models, and when do you choose an online service? For high quality, complex analyses, or tasks where you need the best quality, online models are better. For sensitive or confidential tasks — personnel information, grant applications, contracts — local is the logical choice.  

Document this choice in an AI policy. As an organization, make agreements about when employees use which type of AI. This provides clarity and helps everyone work responsibly. 

The threshold is lower than you think. You can get started within half an hour!