swadhin / opengrammar public

swadhinbiswas

[email protected]
2026-03-14 19:12:57 +0600 committed NaN years ago
0ec38d0
commit 0ec38d0
1 file changed
+483 -0
Showing 1 changed file+483-0
docs/07-ai-providers.md
▼docs/07-ai-providers.md
+483-0
@@ -0,0 +1,483 @@
1
+# 🤖 AI Provider Setup Guide
2
+
3
+Configure AI providers for advanced grammar checking and tone rewriting.
4
+
5
+---
6
+
7
+## 📋 Overview
8
+
9
+OpenGrammar supports **6 AI providers**, giving you flexibility in cost, quality, and privacy.
10
+
11
+| Provider | Speed | Quality | Cost | Best For |
12
+|----------|-------|---------|------|----------|
13
+| **Groq** | ⚡⚡⚡ | ⭐⭐⭐⭐ | Free tier | Fast & free |
14
+| **OpenAI** | ⚡⚡ | ⭐⭐⭐⭐⭐ | $$ | Best quality |
15
+| **OpenRouter** | ⚡⚡ | ⭐⭐⭐⭐⭐ | Varies | Model variety |
16
+| **Together AI** | ⚡⚡ | ⭐⭐⭐⭐ | $ | Open-source |
17
+| **Ollama** | ⚡⚡⚡ | ⭐⭐⭐ | Free | Privacy, offline |
18
+| **Custom** | - | - | - | Your own API |
19
+
20
+---
21
+
22
+## 🎯 Quick Setup
23
+
24
+### Step 1: Choose Your Provider
25
+
26
+**For Free Usage:**
27
+- Provider: **Groq**
28
+- Free tier: 100 requests/day
29
+- Quality: Excellent (Llama 3.1 70B)
30
+
31
+**For Best Quality:**
32
+- Provider: **OpenAI**
33
+- Model: GPT-4o-mini
34
+- Cost: ~$0.15 per 1K requests
35
+
36
+**For Privacy:**
37
+- Provider: **Ollama (Local)**
38
+- 100% offline
39
+- Free (your hardware)
40
+
41
+### Step 2: Get API Key
42
+
43
+Follow the guide for your chosen provider below.
44
+
45
+### Step 3: Configure Extension
46
+
47
+1. Click OpenGrammar icon
48
+2. Click **Settings** (gear icon)
49
+3. Select **Provider** from dropdown
50
+4. Enter **API Key**
51
+5. Select **Model**
52
+6. Click **Save**
53
+
54
+---
55
+
56
+## ⚡ Groq (Recommended - Free)
57
+
58
+### Overview
59
+- **Free Tier:** 100 requests/day
60
+- **Speed:** Fastest (LPU technology)
61
+- **Quality:** Excellent (Llama 3.1 70B)
62
+- **Setup Time:** 2 minutes
63
+
64
+### Step 1: Create Account
65
+1. Visit [Groq Console](https://console.groq.com)
66
+2. Click **Sign Up**
67
+3. Complete registration
68
+
69
+### Step 2: Get API Key
70
+1. Go to **API Keys** in left sidebar
71
+2. Click **Create API Key**
72
+3. Give it a name (e.g., "OpenGrammar")
73
+4. Copy the key (starts with `gsk_`)
74
+5. ⚠️ **Save it now** - you can't see it again!
75
+
76
+### Step 3: Configure in Extension
77
+1. Click OpenGrammar icon → Settings
78
+2. Provider: **Groq**
79
+3. API Key: `gsk_xxx` (paste your key)
80
+4. Model: **llama-3.1-70b-versatile**
81
+5. Click **Save**
82
+
83
+### Available Models
84
+| Model | Speed | Quality | Best For |
85
+|-------|-------|---------|----------|
86
+| `llama-3.1-70b-versatile` | ⚡⚡⚡ | ⭐⭐⭐⭐⭐ | **Recommended** |
87
+| `llama-3.1-8b-instant` | ⚡⚡⚡⚡ | ⭐⭐⭐⭐ | Fast checks |
88
+| `gemma2-9b-it` | ⚡⚡⚡ | ⭐⭐⭐⭐ | Balanced |
89
+| `mixtral-8x7b-32768` | ⚡⚡ | ⭐⭐⭐⭐⭐ | Complex writing |
90
+
91
+### Testing
92
+```bash
93
+# Test Groq API
94
+curl https://api.groq.com/openai/v1/chat/completions \
95
+ -H "Authorization: Bearer gsk_xxx" \
96
+ -H "Content-Type: application/json" \
97
+ -d '{
98
+ "model": "llama-3.1-70b-versatile",
99
+ "messages": [{"role": "user", "content": "Hello!"}]
100
+ }'
101
+```
102
+
103
+### Troubleshooting
104
+- **Error: Invalid API key** - Check key starts with `gsk_`
105
+- **Error: Rate limit** - Free tier is 100/day, wait 24 hours
106
+- **Error: Model not found** - Use exact model name
107
+
108
+---
109
+
110
+## 🟢 OpenAI (Best Quality)
111
+
112
+### Overview
113
+- **Quality:** Best available
114
+- **Speed:** Good
115
+- **Cost:** $0.15 per 1K requests (GPT-4o-mini)
116
+- **Setup Time:** 5 minutes
117
+
118
+### Step 1: Create Account
119
+1. Visit [OpenAI Platform](https://platform.openai.com)
120
+2. Click **Sign Up**
121
+3. Complete registration
122
+4. Add phone number (required)
123
+
124
+### Step 2: Add Credits
125
+1. Go to **Settings** → **Billing**
126
+2. Click **Add payment method**
127
+3. Add minimum $5 credit (recommended)
128
+
129
+### Step 3: Get API Key
130
+1. Go to **API Keys** section
131
+2. Click **Create new secret key**
132
+3. Give it a name (e.g., "OpenGrammar")
133
+4. Copy the key (starts with `sk-`)
134
+5. ⚠️ **Save it now** - can't see it again!
135
+
136
+### Step 4: Configure in Extension
137
+1. Click OpenGrammar icon → Settings
138
+2. Provider: **OpenAI**
139
+3. API Key: `sk-xxx` (paste your key)
140
+4. Model: **gpt-4o-mini** (recommended)
141
+5. Click **Save**
142
+
143
+### Available Models
144
+| Model | Speed | Quality | Cost/1K | Best For |
145
+|-------|-------|---------|---------|----------|
146
+| `gpt-4o-mini` | ⚡⚡ | ⭐⭐⭐⭐⭐ | $0.15 | **Recommended** |
147
+| `gpt-4o` | ⚡ | ⭐⭐⭐⭐⭐⭐ | $2.50 | Premium quality |
148
+| `gpt-3.5-turbo` | ⚡⚡⚡ | ⭐⭐⭐⭐ | $0.05 | Budget option |
149
+
150
+### Cost Estimates
151
+- **Light user** (50 checks/day): ~$0.25/month
152
+- **Medium user** (200 checks/day): ~$1/month
153
+- **Heavy user** (1000 checks/day): ~$5/month
154
+
155
+### Testing
156
+```bash
157
+# Test OpenAI API
158
+curl https://api.openai.com/v1/chat/completions \
159
+ -H "Authorization: Bearer sk-xxx" \
160
+ -H "Content-Type: application/json" \
161
+ -d '{
162
+ "model": "gpt-4o-mini",
163
+ "messages": [{"role": "user", "content": "Fix: me and him went"}]
164
+ }'
165
+```
166
+
167
+---
168
+
169
+## 🌐 OpenRouter (100+ Models)
170
+
171
+### Overview
172
+- **Models:** 100+ (Claude, Llama, Mistral, etc.)
173
+- **Cost:** Pay-per-use, varies by model
174
+- **Speed:** Good
175
+- **Setup Time:** 5 minutes
176
+
177
+### Step 1: Create Account
178
+1. Visit [OpenRouter](https://openrouter.ai)
179
+2. Click **Sign In**
180
+3. Sign in with Google or GitHub
181
+
182
+### Step 2: Get API Key
183
+1. Go to **Keys** section
184
+2. Click **Create Key**
185
+3. Give it a name
186
+4. Copy the key
187
+
188
+### Step 3: Add Credits
189
+1. Go to **Credits** section
190
+2. Add minimum $5 credit
191
+3. Choose payment method
192
+
193
+### Step 4: Configure in Extension
194
+1. Click OpenGrammar icon → Settings
195
+2. Provider: **OpenRouter**
196
+3. API Key: (paste your key)
197
+4. Model: **anthropic/claude-3.5-sonnet** (recommended)
198
+5. Click **Save**
199
+
200
+### Popular Models
201
+| Model | Provider | Quality | Cost/1K | Best For |
202
+|-------|----------|---------|---------|----------|
203
+| `anthropic/claude-3.5-sonnet` | Anthropic | ⭐⭐⭐⭐⭐⭐ | $3.00 | Best overall |
204
+| `meta-llama/llama-3.1-70b` | Meta | ⭐⭐⭐⭐⭐ | $0.40 | Great value |
205
+| `mistralai/mistral-large` | Mistral | ⭐⭐⭐⭐⭐ | $2.00 | European AI |
206
+| `google/gemini-pro` | Google | ⭐⭐⭐⭐⭐ | $0.25 | Google quality |
207
+
208
+### Testing
209
+```bash
210
+# Test OpenRouter API
211
+curl https://openrouter.ai/api/v1/chat/completions \
212
+ -H "Authorization: Bearer xxx" \
213
+ -H "Content-Type: application/json" \
214
+ -d '{
215
+ "model": "anthropic/claude-3.5-sonnet",
216
+ "messages": [{"role": "user", "content": "Hello!"}]
217
+ }'
218
+```
219
+
220
+---
221
+
222
+## 🔷 Together AI (Open-Source)
223
+
224
+### Overview
225
+- **Models:** Open-source (Llama, Mistral, etc.)
226
+- **Cost:** Very cheap ($0.01-0.10 per 1K)
227
+- **Speed:** Fast
228
+- **Setup Time:** 5 minutes
229
+
230
+### Step 1: Create Account
231
+1. Visit [Together AI](https://api.together.ai)
232
+2. Click **Sign Up**
233
+3. Complete registration
234
+
235
+### Step 2: Get API Key
236
+1. Go to **Settings** → **API Keys**
237
+2. Click **Create API Key**
238
+3. Copy the key
239
+
240
+### Step 3: Add Credits
241
+1. Go to **Billing** section
242
+2. Add credits (minimum $5)
243
+3. Get $25 free credit on signup
244
+
245
+### Step 4: Configure in Extension
246
+1. Click OpenGrammar icon → Settings
247
+2. Provider: **Together AI**
248
+3. API Key: (paste your key)
249
+4. Model: **meta-llama/Meta-Llama-3.1-70B-Instruct**
250
+5. Click **Save**
251
+
252
+### Available Models
253
+| Model | Quality | Cost/1K | Best For |
254
+|-------|---------|---------|----------|
255
+| `meta-llama/Meta-Llama-3.1-70B-Instruct` | ⭐⭐⭐⭐⭐ | $0.40 | **Recommended** |
256
+| `mistralai/Mixtral-8x7B-Instruct` | ⭐⭐⭐⭐⭐ | $0.60 | Complex tasks |
257
+| `Qwen/Qwen2.5-72B-Instruct` | ⭐⭐⭐⭐⭐ | $0.40 | Alternative |
258
+
259
+---
260
+
261
+## 🦙 Ollama (Local - Offline)
262
+
263
+### Overview
264
+- **Privacy:** 100% local, offline
265
+- **Cost:** Free (your hardware)
266
+- **Speed:** Depends on hardware
267
+- **Setup Time:** 15 minutes
268
+
269
+### Prerequisites
270
+- 4GB+ RAM (for smaller models)
271
+- 8GB+ RAM (for larger models)
272
+- NVIDIA GPU (optional, for acceleration)
273
+
274
+### Step 1: Install Ollama
275
+
276
+**Linux:**
277
+```bash
278
+curl -fsSL https://ollama.com/install.sh | sh
279
+```
280
+
281
+**macOS:**
282
+```bash
283
+brew install ollama
284
+```
285
+
286
+**Windows:**
287
+1. Download from [ollama.com](https://ollama.com)
288
+2. Run installer
289
+
290
+### Step 2: Start Ollama Service
291
+```bash
292
+# Start Ollama (runs in background)
293
+ollama serve
294
+```
295
+
296
+### Step 3: Pull Models
297
+```bash
298
+# Small & fast (recommended for grammar)
299
+ollama pull qwen2.5:0.5b # 400MB, very fast
300
+ollama pull qwen2.5:1.5b # 1GB, balanced
301
+
302
+# Medium quality
303
+ollama pull phi4-mini:3.8b # 2.5GB, great quality
304
+ollama pull llama3.2:3b # 2GB, good balance
305
+
306
+# High quality (requires GPU)
307
+ollama pull llama3.2:7b # 4GB, requires 8GB+ RAM
308
+ollama pull mistral:7b # 4GB, requires 8GB+ RAM
309
+```
310
+
311
+### Step 4: Test Ollama
312
+```bash
313
+# Test the model
314
+ollama run qwen2.5:1.5b "Fix this: me and him went to store"
315
+```
316
+
317
+Expected output:
318
+```
319
+"He and I went to the store."
320
+```
321
+
322
+### Step 5: Configure in Extension
323
+1. Click OpenGrammar icon → Settings
324
+2. Provider: **Ollama (Local)**
325
+3. Base URL: `http://localhost:11434/v1`
326
+4. Model: `qwen2.5:1.5b` (or your chosen model)
327
+5. API Key: (leave empty - not needed)
328
+6. Click **Save**
329
+
330
+### Model Recommendations
331
+
332
+| Use Case | Model | RAM | Speed | Quality |
333
+|----------|-------|-----|-------|---------|
334
+| **Basic grammar** | qwen2.5:0.5b | 1GB | ⚡⚡⚡⚡⚡ | ⭐⭐⭐ |
335
+| **Better accuracy** | qwen2.5:1.5b | 2GB | ⚡⚡⚡⚡ | ⭐⭐⭐⭐ |
336
+| **Professional writing** | phi4-mini:3.8b | 4GB | ⚡⚡⚡ | ⭐⭐⭐⭐⭐ |
337
+| **Best quality** | llama3.2:7b | 8GB | ⚡⚡ | ⭐⭐⭐⭐⭐ |
338
+
339
+### GPU Acceleration (NVIDIA)
340
+```bash
341
+# Install NVIDIA Container Toolkit
342
+sudo apt-get install -y nvidia-container-toolkit
343
+
344
+# Configure Docker
345
+sudo nvidia-ctk runtime configure --runtime=docker
346
+sudo systemctl restart docker
347
+
348
+# Ollama will automatically use GPU
349
+ollama run llama3.2:7b
350
+```
351
+
352
+### Troubleshooting
353
+- **Error: Connection refused** - Make sure Ollama is running: `ollama serve`
354
+- **Error: Model not found** - Pull the model: `ollama pull qwen2.5:1.5b`
355
+- **Slow performance** - Use smaller model or enable GPU
356
+
357
+---
358
+
359
+## 🔧 Custom Provider
360
+
361
+### Overview
362
+Use your own API endpoint or any OpenAI-compatible API.
363
+
364
+### Configuration
365
+1. Click OpenGrammar icon → Settings
366
+2. Provider: **Custom**
367
+3. Base URL: `https://your-api.com/v1`
368
+4. API Key: (your API key)
369
+5. Model: (your model name)
370
+6. Click **Save**
371
+
372
+### Example: LocalAI
373
+```bash
374
+# Run LocalAI with Docker
375
+docker run -p 8080:8080 localai/localai
376
+
377
+# Configure in extension
378
+Provider: Custom
379
+Base URL: http://localhost:8080/v1
380
+Model: llama-3.1-70b
381
+```
382
+
383
+### Example: LM Studio
384
+```bash
385
+# LM Studio runs locally
386
+# Configure in extension
387
+Provider: Custom
388
+Base URL: http://localhost:1234/v1
389
+Model: local-model
390
+```
391
+
392
+---
393
+
394
+## 📊 Provider Comparison
395
+
396
+### Speed Comparison
397
+```
398
+Groq (Llama-70B) ████████████████████ 200ms
399
+OpenAI (GPT-4o-mini) ██████████ 1s
400
+Together (Llama-70B) ████████████ 800ms
401
+OpenRouter (Claude) ██████████████████ 2s
402
+Ollama (Qwen-1.5B) ██████████████ 500ms (local)
403
+```
404
+
405
+### Quality Comparison
406
+```
407
+OpenAI (GPT-4o) ████████████████████ 96%
408
+OpenRouter (Claude) ████████████████████ 96%
409
+OpenAI (GPT-4o-mini) ██████████████████ 95%
410
+Groq (Llama-70B) ████████████████ 92%
411
+Ollama (Phi-4-mini) ██████████████ 88%
412
+Ollama (Qwen-1.5B) ████████████ 82%
413
+```
414
+
415
+### Cost Comparison (per 1K requests)
416
+```
417
+Ollama (Local) $0.00 (free)
418
+Groq (Free tier) $0.00 (100/day free)
419
+Together (Llama-70B) $0.40
420
+OpenAI (GPT-4o-mini) $0.15
421
+OpenRouter (Claude) $3.00
422
+```
423
+
424
+---
425
+
426
+## 🎯 Provider Selection Guide
427
+
428
+### Choose Groq If:
429
+- ✅ You want free usage
430
+- ✅ You need fast responses
431
+- ✅ You're okay with 100 requests/day limit
432
+- ✅ You want good quality
433
+
434
+### Choose OpenAI If:
435
+- ✅ You want the best quality
436
+- ✅ You don't mind paying
437
+- ✅ You need reliability
438
+- ✅ You want GPT-4 level intelligence
439
+
440
+### Choose OpenRouter If:
441
+- ✅ You want model variety
442
+- ✅ You want to try Claude
443
+- ✅ You want flexibility
444
+- ✅ You need specific models
445
+
446
+### Choose Together AI If:
447
+- ✅ You want cheap open-source
448
+- ✅ You like Llama models
449
+- ✅ You want good value
450
+- ✅ You need fast inference
451
+
452
+### Choose Ollama If:
453
+- ✅ Privacy is your priority
454
+- ✅ You have good hardware
455
+- ✅ You want offline access
456
+- ✅ You don't want API costs
457
+
458
+---
459
+
460
+## 🔄 Switching Providers
461
+
462
+You can switch providers anytime:
463
+
464
+1. Click OpenGrammar icon
465
+2. Click **Settings**
466
+3. Change **Provider** dropdown
467
+4. Enter new API key (if required)
468
+5. Select new **Model**
469
+6. Click **Save**
470
+
471
+Your settings are saved automatically.
472
+
473
+---
474
+
475
+## 📚 Related Documentation
476
+
477
+- [Backend Deployment](05-backend-deployment.md) - Deploy your backend
478
+- [Docker Self-Hosting](06-docker-self-hosting.md) - Run locally
479
+- [Troubleshooting](18-troubleshooting.md) - Common issues
480
+
481
+---
482
+
483
+**Your AI provider is now configured! Start writing better! ✨**
0
484