Skip to main content
Ollama enables you to run powerful open-source AI models locally on your own hardware, ensuring your code never leaves your machine.

​
Why Use Ollama?

Complete Privacy

Your code and commits stay on your local machine - never sent to external servers

No API Costs

No usage fees or API quotas - unlimited commits once set up

Offline Access

Works without internet connection after initial model download

Full Control

Choose from dozens of open-source models and customize parameters

​
Prerequisites

Before using Ollama with GitWhisper, you need to:
  1. Install Ollama: Download from ollama.com
  2. Pull a model: Download at least one AI model
  3. Start the server: Ollama must be running locally

​
Installation

# Download from ollama.com or use Homebrew
brew install ollama

# Start Ollama
ollama serve

​
Download Models

Pull models you want to use: 
# Popular models for code
ollama pull llama3.2
ollama pull codellama
ollama pull deepseek-coder
ollama pull qwen2.5-coder

# Smaller, faster models
ollama pull phi3
ollama pull gemma2

# Verify available models
ollama list
Start with llama3.2 or codellama - they offer a good balance of quality and speed for commit messages.

​
Available Models

Ollama supports hundreds of models. Popular choices for GitWhisper:
  • llama3.2:latest ⭐ (default) - Great all-around choice
  • codellama - Optimized for code understanding
  • deepseek-coder - Excellent code generation and analysis
  • qwen2.5-coder - Strong coding capabilities
  • granite-code - IBM’s code-focused model

​
Fast & Lightweight

  • phi3 - Microsoft’s compact model
  • gemma2 - Google’s efficient model
  • tinyllama - Ultra-fast but basic

​
Large & Powerful

  • llama3:70b - More capable, requires more RAM
  • mixtral - High quality, mixture of experts
  • command-r - Cohere’s advanced model
Browse all available models at ollama.com/library

​
Usage

​
Basic Usage

Make sure Ollama is running, then: 
# Use Ollama with default model (llama3.2:latest)
gitwhisper commit --model ollama

# Shorthand
gw commit -m ollama

​
Specify Model

Use a specific Ollama model: 
# Use CodeLlama
gitwhisper commit --model ollama --model-variant codellama

# Use DeepSeek Coder
gitwhisper commit --model ollama --model-variant deepseek-coder

# Use specific version
gitwhisper commit --model ollama --model-variant llama3.2:3b

​
Set as Default

Configure Ollama as your default model: 
# Set Ollama with default model
gitwhisper set-defaults --model ollama

# Set Ollama with specific model
gitwhisper set-defaults --model ollama --model-variant codellama

# Set custom base URL (if not using default)
gitwhisper set-defaults --model ollama --model-variant llama3.2 --base-url http://localhost:11434

​
Custom Base URL

If Ollama is running on a different host or port: 
# Custom port
gitwhisper commit --model ollama --base-url http://localhost:8080

# Remote Ollama server
gitwhisper commit --model ollama --base-url http://192.168.1.100:11434

# Save as default
gitwhisper set-defaults --model ollama --base-url http://localhost:8080
See Custom Endpoints for more details.

​
No API Key Required

Ollama doesn’t require an API key: 
# Just use it directly
gitwhisper commit --model ollama

# No save-key needed
# No environment variables needed

​
Hardware Requirements

Model performance depends on your hardware:
Basic laptop/desktop
  • 8GB RAM
  • Any modern CPU
  • No GPU required
Recommended models:
  • phi3
  • tinyllama
  • llama3.2:3b
Response time: 5-15 seconds
Larger models (70B parameters) require significant RAM (32GB+) and will be slow without a GPU.

​
Model Comparison

ModelQualitySpeedSizeRAM Needed
llama3:70b⭐⭐⭐⭐⭐⭐40GB32GB+
codellama⭐⭐⭐⭐⭐⭐⭐7GB8GB
llama3.2⭐⭐⭐⭐⭐⭐⭐⭐2GB8GB
deepseek-coder⭐⭐⭐⭐⭐⭐⭐7GB8GB
phi3⭐⭐⭐⭐⭐⭐⭐⭐2GB4GB
tinyllama⭐⭐⭐⭐⭐⭐⭐700MB2GB

​
Code Analysis

Use Ollama for local code analysis: 
# Analyze changes locally
gitwhisper analyze --model ollama

# Use specialized code model
gitwhisper analyze --model ollama --model-variant codellama

​
Analysis Benefits

  • Private: Code never leaves your machine
  • Fast: No network latency
  • Detailed: Generate comprehensive reports
  • Unlimited: No API quotas or rate limits

​
GPU Acceleration

Ollama automatically uses your GPU if available:

​
NVIDIA GPUs

# Ollama detects NVIDIA GPU automatically
# Install CUDA drivers if not detected

# Check GPU usage
watch -n 1 nvidia-smi

​
Apple Silicon (M1/M2/M3)

# Metal acceleration is automatic
# No additional setup needed

​
AMD GPUs

# ROCm support on Linux
# Check Ollama documentation for setup
GPU acceleration can make models 10-50x faster. For best results, use a GPU with 8GB+ VRAM.

​
Troubleshooting

Error: Connection refused
Solution: Start Ollama:
# macOS/Linux
ollama serve

# Or check if already running
ps aux | grep ollama

Error: Model not found
Solution: Pull the model first:
ollama pull llama3.2

# List available models
ollama list

Error: Out of memory
Solution: Use a smaller model:
# Try a lighter model
ollama pull phi3
gitwhisper commit --model ollama --model-variant phi3
Solutions:
  1. Use a smaller model (phi3, gemma2)
  2. Enable GPU acceleration
  3. Close other applications
  4. Use quantized models (models with :q4 or :q8 suffix)
If Ollama is on a different port:
# Set custom URL
gitwhisper set-defaults --model ollama --base-url http://localhost:8080

​
Best Practices

  1. Start with recommended models: Use llama3.2 or codellama first
  2. Monitor resource usage: Watch RAM/GPU with htop or Activity Monitor
  3. Keep models updated: Run ollama pull <model> periodically
  4. Use GPU when available: Much faster than CPU-only
  5. Match model to hardware: Don’t use 70B models on 8GB RAM

​
Comparison with Cloud Models

Advantages

  • Complete privacy
  • No API costs
  • Offline capability
  • Unlimited usage
  • No rate limits

Trade-offs

  • Requires local hardware
  • Setup complexity
  • May be slower
  • Quality varies by model
  • Uses system resources

​
Example Workflow

# 1. Install and start Ollama
ollama serve

# 2. Pull a model (in another terminal)
ollama pull codellama

# 3. Set as default in GitWhisper
gitwhisper set-defaults --model ollama --model-variant codellama

# 4. Use for commits
git add .
gitwhisper commit

# 5. Analyze code changes
gitwhisper analyze
Explore models on Ollama’s library:

​
Next Steps

Custom Endpoints

Configure custom Ollama URLs

Model Variants

View all available models

Cloud Models

Compare with cloud-based options

Configuration

Set up default preferences

Build docs developers (and LLMs) love

Get started for freeTalk to us