grepai-embeddings-ollama

GrepAI Embeddings with Ollama

This skill covers using Ollama as the embedding provider for GrepAI, enabling 100% private, local code search.

When to Use This Skill

• Setting up private, local embeddings
• Choosing the right Ollama model
• Optimizing Ollama performance
• Troubleshooting Ollama connection issues

Why Ollama?

Prerequisites

1. Ollama installed and running
2. An embedding model downloaded

# Install Ollama
brew install ollama  # macOS
# or
curl -fsSL https://ollama.com/install.sh | sh  # Linux
 
# Start Ollama

Configuration

Basic Configuration

With Custom Endpoint

With Explicit Dimensions

Available Models

Recommended: nomic-embed-text

Configuration:

Multilingual: nomic-embed-text-v2-moe

Best for codebases with non-English comments/documentation.

Configuration:

High Quality: bge-m3

Best for large, complex codebases where accuracy is critical.

Configuration:

Maximum Quality: mxbai-embed-large

Configuration:

Model Comparison

Performance Optimization

Memory Management

Models load into RAM. Ensure sufficient memory:

GPU Acceleration

Ollama automatically uses:

• macOS: Metal (Apple Silicon)
• Linux/Windows: CUDA (NVIDIA GPUs)

Check GPU usage:

Keeping Model Loaded

By default, Ollama unloads models after 5 minutes of inactivity. Keep loaded:

Verifying Connection

Check Ollama is Running

List Available Models

Test Embedding

Running Ollama as a Service

macOS (launchd)

Ollama app runs automatically on login.

Linux (systemd)

Manual Background

Remote Ollama Server

Run Ollama on a powerful server and connect remotely:

On the Server

On the Client

Common Issues

❌ Problem: Connection refused ✅ Solution:

❌ Problem: Model not found ✅ Solution:

❌ Problem: Slow embedding generation ✅ Solutions:

• Use a smaller model (nomic-embed-text)
• Ensure GPU is being used (ollama ps)
• Close memory-intensive applications
• Consider a remote server with better hardware

❌ Problem: Out of memory ✅ Solutions:

• Use a smaller model
• Close other applications
• Upgrade RAM
• Use remote Ollama server

❌ Problem: Embeddings differ after model update ✅ Solution: Re-index after model updates:

Best Practices

1. Start with nomic-embed-text: Best balance of speed/quality
2. Keep Ollama running: Background service recommended
3. Match dimensions: Don't mix models with different dimensions
4. Re-index on model change: Delete index and re-run watch
5. Monitor memory: Embedding models use significant RAM

Output Format

Successful Ollama configuration: