OAuth Token Refresh Issues

The Problem

Your OAuth-authenticated data source randomly disconnects, requiring manual re-authorization, despite refresh tokens being configured.

Symptoms

• ❌ "Authentication expired" after 1 hour
• ❌ Sync works, then fails days later with 401 errors
• ❌ "Refresh token invalid" despite recent setup
• ❌ Must re-authorize every week
• ❌ Token refresh succeeds but still get 401 errors

Real-World Example

Google Drive connected via OAuth
Day 1: Syncs perfectly
Day 7: "401 Unauthorized - Invalid credentials"

Logs show:
✓ Access token expired (expected after 1 hour)
✗ Refresh token request failed: "invalid_grant"
✗ Stored refresh token: null (disappeared?)

User must: Disconnect and reconnect integration

---

Deep Technical Analysis

OAuth 2.0 Token Lifecycle

OAuth uses two tokens with different lifespans:

Access Token (short-lived):

Lifespan: 1-2 hours (varies by provider)
Purpose: Make API requests
When expired: API returns 401

Example:
Authorization: Bearer eyJhbGc...  ← access token

Refresh Token (long-lived):

Lifespan: Days to months (or indefinite)
Purpose: Obtain new access token
When expired: Must re-authorize

Flow:
1. Access token expires
2. Use refresh token to get new access token:
   POST /oauth/token
   {
     "grant_type": "refresh_token",
     "refresh_token": "1//abc123...",
     "client_id": "...",
     "client_secret": "..."
   }
3. Receive new access token (and sometimes new refresh token)

The Refresh Token Rotation Problem:

Some providers (e.g., Google) use rotating refresh tokens:

Initial authorization:
→ access_token: "token1" (expires in 1h)
→ refresh_token: "refresh1"

After 1 hour, refresh:
POST /oauth/token with refresh_token="refresh1"
→ New access_token: "token2"
→ New refresh_token: "refresh2"  ← Different!

Old refresh_token="refresh1" is INVALIDATED

If Twig doesn't update stored refresh token:
→ Next refresh attempts with "refresh1"
→ Error: "invalid_grant"
→ Integration breaks permanently

Storage Race Condition:

Concurrent sync processes:

Process A:
1. Access token expired
2. Refresh token → get token2, refresh2
3. Make API call with token2
4. Store refresh2 (overwrites refresh1)

Process B (concurrent):
1. Access token expired (same time)
2. Refresh with refresh1 → get token3, refresh3
3. Make API call with token3
4. Store refresh3 (overwrites refresh2)

Result:
→ Token2 valid, but refresh2 discarded
→ Token3 valid, refresh3 stored
→ Next refresh uses refresh3: works
→ But if Process A refreshes first next time:
  → Uses refresh2 (not stored) → fails

Offline Access and Consent Prompts

OAuth scopes determine token longevity:

Online vs Offline Access:

Google OAuth scopes:
→ access_type=online: No refresh token (session-based)
→ access_type=offline: Refresh token granted

If Twig requests:
OAuth URL: ?scope=drive.readonly&access_type=online

Result:
→ User authorizes
→ Twig receives: access_token only (no refresh_token)
→ Token expires in 1 hour
→ No way to refresh
→ Integration breaks after 1 hour

Common mistake: Forgetting access_type=offline

Incremental Authorization:

Initial auth:
→ Scope: drive.readonly
→ User grants access
→ Refresh token: "refresh1"

Later, Twig needs more permissions:
→ Scope: drive.readonly+drive.metadata

Reauthorization:
→ User must approve again
→ New refresh token: "refresh2"
→ Old "refresh1" invalidated (sometimes)

Problem:
→ Twig doesn't trigger reauth flow automatically
→ Uses old "refresh1"
→ Fails with "invalid_grant"

Consent Screen Re-prompts:

Some OAuth providers force re-consent periodically:

Google:
→ If app is "Testing" status (not verified)
→ Refresh tokens expire after 7 days
→ User must re-authorize weekly

Microsoft:
→ Conditional Access policies can force re-auth
→ MFA required every 30 days
→ Refresh fails until user logs in again

Provider may not notify Twig that reauth needed
→ Just returns: "invalid_grant"
→ User sees: "Integration broken"

Refresh Token Revocation

Tokens can be revoked externally:

User-Initiated Revocation:

User action:
→ Opens Google Account settings
→ "Manage third-party access"
→ Finds "Twig AI"
→ Clicks "Remove access"

Google revokes:
→ All access tokens for this user+app
→ All refresh tokens for this user+app

Next Twig sync:
→ Attempts refresh
→ Error: "invalid_grant"
→ No way to recover automatically
→ Must prompt user to reauthorize

Admin-Initiated Revocation:

Google Workspace admin:
→ Reviews OAuth app permissions
→ Decides Twig has too much access
→ Revokes app access for entire organization

Effect:
→ 500 users' integrations break simultaneously
→ All refresh tokens invalidated
→ Twig sees: Mass 401 errors
→ Must notify all 500 users to reauthorize

Automatic Revocation (security):

OAuth provider's security system:
→ Detects unusual activity (e.g., API calls from new IP)
→ Suspects account compromise
→ Automatically revokes all tokens
→ Forces password reset

Or:
→ User changes password
→ Provider invalidates all refresh tokens (security best practice)
→ Twig integration breaks
→ User doesn't connect the dots

Token Storage Security vs Availability

Refresh tokens are sensitive credentials:

Storage Requirements:

Security:
→ Encrypt at rest
→ Encrypt in transit
→ Never log refresh tokens
→ Rotate encryption keys periodically
→ Access control (only token service can read)

Availability:
→ Must survive server restarts
→ Replicate across regions
→ Backup and recovery
→ Fast retrieval (needed for every API call)

The Encryption Key Rotation Problem:

Day 1:
→ Refresh token: "abc123"
→ Encrypt with key_v1: "encrypted_xyz"
→ Store in database

Day 30:
→ Rotate encryption key (security best practice)
→ New key: key_v2

Day 31:
→ Read encrypted_xyz from database
→ Decrypt with key_v2: FAILS (encrypted with key_v1)
→ Can't decrypt refresh token
→ Integration breaks

Solution:
→ Re-encrypt all tokens during rotation
→ Or: Store key version with token
→ But: Massive operation for millions of tokens

Client ID and Secret Management

OAuth requires client credentials:

The Client Secret Problem:

OAuth app registration:
→ Client ID: "abc123" (public)
→ Client Secret: "secret456" (private)

Token refresh request needs both:
POST /oauth/token
{
  "client_id": "abc123",
  "client_secret": "secret456",
  "refresh_token": "..."
}

But:
→ Client secret leaked in Git commit
→ Attacker has access to secret
→ Can impersonate Twig app
→ Must rotate secret immediately

Secret rotation:
→ Generate new secret: "secret789"
→ Old secret "secret456" invalidated
→ All existing refresh attempts with old secret fail
→ Must update all deployments simultaneously
→ Zero-downtime rotation is complex

Per-Customer OAuth Apps:

Enterprise customers may require:
→ Custom OAuth app (their branding)
→ Their own client ID/secret
→ Different redirect URLs

Twig must:
→ Store N sets of client credentials
→ Route refresh requests to correct credentials
→ Handle different provider configurations
→ Complexity: Multiple apps per provider

Token Expiry vs Actual Invalidity

Token expiry times aren't always accurate:

The Early Expiration Problem:

Token response:
{
  "access_token": "token123",
  "expires_in": 3600  ← Says 1 hour
}

Twig computes:
expiry_time = now() + 3600 seconds

But:
→ Provider may have issued token 10 minutes ago
→ expiry_in is relative to issuance, not now
→ Twig's expiry_time is 10 minutes off

Or:
→ Provider's clock is off by 5 minutes
→ Token expires 5 minutes early from Twig's perspective

Preemptive Refresh:

Best practice: Refresh before expiry

If expires_in = 3600:
→ Refresh at: now() + 3000 (50 minutes)
→ 10-minute buffer

But:
→ If processing takes 15 minutes
→ Access token used at minute 55
→ Token expired at minute 50 (from provider's perspective)
→ API call fails with 401
→ Must handle refresh mid-operation

Lazy vs Eager Refresh:

Lazy (current):
→ API call fails with 401
→ Detect expiry
→ Refresh token
→ Retry API call

Eager (better):
→ Check expiry before API call
→ If < 10 min remaining: refresh proactively
→ Make API call with fresh token
→ No retry needed

Tradeoff:
→ Eager: Extra checks, but fewer failures
→ Lazy: Simpler, but retries on every expiry

---

How to Solve

Request access_type=offline + handle refresh token rotation by updating stored token + implement preemptive refresh (10 min buffer) + gracefully handle revocation with reauth prompt. See OAuth Configuration.

---

Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the ask query parameter:

GET /dev/rag-scenarios-and-solutions/data-integration/oauth-refresh.md?ask=<question>

The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.