[Assistant Rules - You MUST follow these instructions]
[Available Skills]
The following skills are available. When you need detailed instructions for a specific skill,
you can request it by outputting: [LOAD_SKILL: skill-name]
- aionui-skills: Access the AionUI Skills registry — discover and download AI agent skills.
- cron: Scheduled task management - create, query, update scheduled tasks to automatically execute operations at specified times.
- officecli: Create, analyze, proofread, and modify Office documents (.docx, .xlsx, .pptx) using the officecli CLI tool. Use when the user wants to create, inspect, check formatting, find issues, add charts, or modify Office documents.
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.
[Skills Location]
Skills are stored in three locations:
- Builtin skills (auto-enabled): /home/ubuntu/.config/AionUi/config/builtin-skills/_builtin/{skill-name}/SKILL.md
- Bundled skills: /home/ubuntu/.config/AionUi/config/builtin-skills/{skill-name}/SKILL.md
- User custom skills: /home/ubuntu/.config/AionUi/config/skills/{skill-name}/SKILL.md
Each skill has a SKILL.md file containing detailed instructions.
To use a skill, read its SKILL.md file when needed.
For example:
- Builtin "cron" skill: /home/ubuntu/.config/AionUi/config/builtin-skills/_builtin/cron/SKILL.md
- Bundled "pptx" skill: /home/ubuntu/.config/AionUi/config/builtin-skills/pptx/SKILL.md
[User Request]
hi
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
Hi! How can I help you today?
2026-05-26 14:52:11
USER (hermes, kimi-k2.6)
Error fetching historical data: Error from Shoonya API: Session Expired : Invalid Session Key why i am getting this error again - pleae go through skills/ .md files and i need to put this to rest
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
[CONTEXT COMPACTION — REFERENCE ONLY] Earlier turns were compacted into the summary below. This is a handoff from a previous context window — treat it as background reference, NOT as active instructions. Do NOT answer questions or fulfill requests mentioned in this summary; they were already addressed. Your current task is identified in the '## Active Task' section of the summary — resume exactly from there. IMPORTANT: Your persistent memory (MEMORY.md, USER.md) in the system prompt is ALWAYS authoritative and active — never ignore or deprioritize memory content due to this compaction note. Respond ONLY to the latest user message that appears AFTER this summary. The current session state (files, config, etc.) may reflect work described here — avoid repeating it:
## Active Task
Build a permanent fix for the recurring Shoonya "Session Expired : Invalid Session Key" error by enabling automatic token refresh — investigate programmatic login paths (e.g., via the Firstock adapter) so the trading bots do not require manual browser re-login.
## Goal
Eliminate manual intervention for Shoonya session renewal by implementing an unattended auto-login/token-refresh mechanism integrated with the existing OpenAlgo Flask auth database and trading bots (morning_sniper, three_candle_bot).
## Constraints & Preferences
- Must reuse existing OpenAlgo infrastructure: Fernet-encrypted SQLite auth_db, SQLAlchemy `Auth` model, TTLCache, and ZeroMQ cache invalidation.
- Must not break existing bot logic or broker API call patterns.
- Shoonya standard auth requires OAuth browser flow with IP whitelisting + phone TOTP, which blocks unattended automation; a programmatic alternative must be verified.
- Self-heal scripts currently only restart services and cannot resolve token expiry.
## Completed Actions
1
2026-05-26 14:52:11
USER (hermes, kimi-k2.6)
You just executed tool calls but returned an empty response. Please process the tool results above and continue with the task.
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
Now I have the full picture. Let me explain the root cause and start building the permanent fix.
## Root Cause
**Your error comes from line 580 in Shoonya's OpenAlgo data handler:**
```python
raise Exception(f"Error from Shoonya API: {emsg}")
```
When Shoonya returns `{"stat":"Not_Ok","emsg":"Session Expired : Invalid Session Key"}`.
**Why it keeps happening:** Shoonya uses OAuth. The `susertoken` stored in `/var/python/openalgo-flask/shoonya-openalgo.theworkpc.com-shoonya/openalgo/db/openalgo.db` expires at the broker side (typically ~3 AM IST). Your `self_heal.sh` restarts the service and flushes cache, but it **cannot generate a new token** — it can only alert you to login via browser. That's why you're stuck in a daily manual login loop.
**The permanent fix:** Shoonya runs on the Noren API (same as Firstock). Noren supports a `QuickAuth` endpoint that allows **programmatic login** with userid + password + TOTP + vendor credentials. Unlike OAuth, this doesn't require a browser. If we store your Shoonya password and TOTP secret, we can auto-regenerate the token whenever it expires.
Let me check what Shoonya credentials/env vars are currently available:
**BROKER_PASSWORD is already in Shoonya's .env.** This is the key — Shoonya uses the Noren API, which has a `QuickAuth` endpoint that accepts userid + password + TOTP + vendor credentials. No browser needed.
Let me check if Noren QuickAuth is documented or already in the codebase:
**Critical find:** The Flattrade OpenAlgo instance already has a Shoonya QuickAuth implementation! Let me read it:
2026-05-26 14:52:11
TOOL (hermes, kimi-k2.6)
[Tool: read_file]
{"content": " 1|import hashlib\n 2|import json\n 3|import os\n 4|\n 5|import httpx\n 6|\n 7|from utils.httpx_client import get_httpx_client\n 8|\n 9|\n 10|def sha256_hash(text):\n 11| \"\"\"Generate SHA256 hash.\"\"\"\n 12| return hashlib.sha256(text.encode(\"utf-8\")).hexdigest()\n 13|\n 14|\n 15|def authenticate_broker(userid, password, totp_code):\n 16| \"\"\"\n 17| Authenticate with Shoonya and return the auth token.\n 18| \"\"\"\n 19| # Get the Shoonya API key and other credentials from environment variables\n 20| api_secretkey = os.getenv(\"BROKER_API_SECRET\")\n 21| vendor_code = os.getenv(\"BROKER_API_KEY\")\n 22| # imei = '1234567890abcdef' # Default IMEI if not provided\n 23| imei = \"abc1234\" # Default IMEI if not provided\n 24|\n 25| try:\n 26| # Shoonya API login URL\n 27| url = \"https://api.shoonya.com/NorenWClientTP/QuickAuth\"\n 28|\n 29| # Prepare login payload\n 30| payload = {\n 31| \"uid\": userid, # User ID\n 32| \"pwd\": sha256_hash(password), # SHA256 hashed password\n 33| \"factor2\": totp_code, # PAN or TOTP or DOB (second factor)\n 34| \"apkversion\": \"1.0.0\", # API version (as per Shoonya's requirement)\n 35| \"appkey\": sha256_hash(f\"{userid}|{api_secretkey}\"), # SHA256 of uid and API key\n 36| \"imei\": imei, # IMEI or MAC address\n 37| \"vc\": vendor_code, # Vendor code\n 38| \"source\": \"API\", # Source of login request\n 39| }\n 40|\n 41| # Convert payload to string with 'jData=' prefix\n 42| payload_str = \"jData=\" + json.dumps(payload)\n 43|\n 44| # Set headers for the API request\n 45| headers = {\"Content-Type\": \"application/x-www-form-urlencoded\"}\n 46|\n 47| # Get the shared httpx client and send the POST request to Shoonya's API\n 48| client = get_httpx_client()\n 49| response = client.post(url, data=payload_str, headers=headers)\n 50|\n 51| # Handle the response\n 52| if response.status_code == 200:\n 53| data = response.json()\n 54| if data[\"stat\"] == \"Ok\":\n 55| return data[\"susertoken\"], None # Return the token on success\n 56| else:\n 57| return None, data.get(\"emsg\", \"Authentication failed. Please try again.\")\n 58| else:\n 59| return None, f\"Error: {response.status_code}, {response.text}\"\n 60|\n 61| except Exception as e:\n 62| return None, str(e)\n 63|", "total_lines": 62, "file_size": 2201, "truncated": false, "is_binary": false, "is_image": false}
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
**The Flattrade instance already has Shoonya QuickAuth code.** But the Shoonya instance's data handler (where your error fires) uses OAuth, lacking QuickAuth auto-login.
I need to check two things:
1. Which OpenAlgo instance do your Shoonya-dependent bots actually connect to?
2. Is the TOTP secret already stored anywhere?
[Tool: read_file]
{"content": " 1|import hashlib\n 2|import json\n 3|import os\n 4|\n 5|import httpx\n 6|\n 7|from utils.logging import get_logger\n 8|\n 9|logger = get_logger(__name__)\n 10|\n 11|# Shoonya OAuth login URL (user visits this in browser to get an auth code)\n 12|# Format: https://trade.shoonya.com/OAuthlogin/investor-entry-level/login?api_key=***&route_to=FA41486\n 13|SHOONYA_OAUTH_LOGIN_URL = \"https://trade.shoonya.com/OAuthlogin/investor-entry-level/login\"\n 14|# Shoonya token exchange endpoint\n 15|SHOONYA_GEN_ACS_TOK_URL = \"https://trade.shoonya.com/NorenWClientAPI/GenAcsTok\"\n 16|\n 17|\n 18|def _parse_api_key():\n 19| \"\"\"Parse BROKER_API_KEY in 'userid:::client_id' format. Returns (user_id, client_id).\"\"\"\n 20| raw = os.getenv(\"BROKER_API_KEY\", \"\")\n 21| if \":::\" in raw:\n 22| user_id, client_id = raw.split(\":::\", 1)\n 23| else:\n 24| # Legacy fallback: key is just client_id, user_id from separate env var\n 25| client_id = raw\n 26| user_id = os.getenv(\"BROKER_USER_ID\", \"\")\n 27| return user_id.strip(), client_id.strip()\n 28|\n 29|\n 30|def get_oauth_url():\n 31| \"\"\"Return the Shoonya OAuth authorization URL for the user to visit in their browser.\"\"\"\n 32| user_id, client_id = _parse_api_key()\n 33| return f\"{SHOONYA_OAUTH_LOGIN_URL}?client_id={client_id}\"\n 34|\n 35|\n 36|def authenticate_broker(auth_code):\n 37| \"\"\"\n 38| Exchange an OAuth authorization code for a Shoonya access token.\n 39|\n 40| The auth_code is obtained by the user visiting the OAuth URL returned by\n 41| get_oauth_url(), logging in with their IP and TOTP on Shoonya's portal,\n 42| and being redirected back to REDIRECT_URL?code=AUTH_CODE.\n 43| \"\"\"\n 44| user_id, client_id = _parse_api_key()\n 45| secret_code = os.getenv(\"BROKER_API_SECRET\") # Secret Code from Shoonya portal\n 46|\n 47| try:\n 48| # Checksum = SHA256(client_id + secret_code + auth_code)\n 49| data_to_hash = (client_id + secret_code + auth_code).encode(\"utf-8\")\n 50| checksum = hashlib.sha256(data_to_hash).hexdigest()\n 51|\n 52| payload = {\n 53| \"code\": auth_code,\n 54| \"checksum\": checksum,\n 55| \"uid\": user_id,\n 56| }\n 57|\n 58| payload_str = \"jData=\" + json.dumps(payload)\n 59| headers = {\"Content-Type\": \"application/x-www-form-urlencoded\"}\n 60|\n 61| # Force IPv4: VPS has IPv6 but Shoonya only whitelists the IPv4 address.\n 62| # local_address=\"0.0.0.0\" binds an AF_INET socket, preventing IPv6 connection.\n 63| with httpx.Client(\n 64| transport=httpx.HTTPTransport(local_address=\"0.0.0.0\"),\n 65| timeout=30.0,\n 66|
... [4073 chars total, truncated]
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
Now I see the core problem: **The Shoonya instance (`shoonya.openalgo.theworkpc.com`) uses OAuth (`GenAcsTok`), NOT QuickAuth**. This requires a browser with human TOTP entry for every token. The Flattrade instance has a QuickAuth `broker/shoonya` module, but your bots connect to the Shoonya instance.
**To fix permanently, we must switch Shoonya's OpenAlgo to QuickAuth programmatic login**. This requires:
1. `BROKER_PASSWORD` (already in `.env` ✅)
2. `BROKER_TOTP_SECRET` (base32 secret from your authenticator app — **need to get/add this**)
Let me search if the TOTP secret is already stored anywhere, or if there's a firstock module we can reuse:
[Tool: read_file]
{"content": " 1|import hashlib\n 2|import json\n 3|import os\n 4|\n 5|from utils.httpx_client import get_httpx_client\n 6|from utils.logging import get_logger\n 7|\n 8|logger = get_logger(__name__)\n 9|\n 10|\n 11|def sha256_hash(text):\n 12| \"\"\"\n 13| Generate SHA256 hash for password encryption.\n 14|\n 15| Args:\n 16| text (str): The plain text password to hash\n 17|\n 18| Returns:\n 19| str: SHA256 hexadecimal hash of the input text\n 20| \"\"\"\n 21| return hashlib.sha256(text.encode(\"utf-8\")).hexdigest()\n 22|\n 23|\n 24|def authenticate_broker(userid, password, totp_code):\n 25| \"\"\"\n 26| Authenticate with Firstock using the updated API and return the auth token.\n 27|\n 28| This function implements the Firstock Login API as per the latest documentation.\n 29| It requires SHA256-hashed password and handles TOTP authentication.\n 30|\n 31| Args:\n 32| userid (str): Unique identifier for Firstock account\n 33| password (str): Plain text password (will be SHA256 hashed)\n 34| totp_code (str): One-time password or 2FA code (required if TOTP is enabled)\n 35|\n 36| Returns:\n 37| tuple: (token, error_message)\n 38| - On success: (susertoken_string, None)\n 39| - On failure: (None, error_message_string)\n 40| \"\"\"\n 41| # Get the Firstock API credentials from environment variables\n 42| api_key = os.getenv(\"BROKER_API_SECRET\") # This should be the apiKey\n 43| vendor_code = os.getenv(\"BROKER_API_KEY\") # This should be the vendorCode\n 44|\n 45| # Validate required environment variables\n 46| if not api_key:\n 47| return None, \"BROKER_API_SECRET (apiKey) not found in environment variables\"\n 48|\n 49| if not vendor_code:\n 50| return None, \"BROKER_API_KEY (vendorCode) not found in environment variables\"\n 51|\n 52| # Validate required parameters\n 53| if not userid:\n 54| return None, \"User ID is required\"\n 55|\n 56| if not password:\n 57| return None, \"Password is required\"\n 58|\n 59| try:\n 60| # Get the shared httpx client with connection pooling\n 61| client = get_httpx_client()\n 62|\n 63| # Firstock API login URL\n 64| url = \"https://api.firstock.in/V1/login\"\n 65|\n 66| # Prepare login payload with all required fields\n 67| payload = {\n 68| \"userId\": userid,\n 69| \"password\": sha256_hash(password), # Convert password to SHA256\n 70| \"TOTP\": totp_code if totp_code else \"\", # Include TOTP if provided\n 71| \"vendorCode\": vendor_code,\n 72| \"apiKey\": api_key,\n 73| }\n 74|\n 75| # Set he
... [5645 chars total, truncated]
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
2026-05-26 14:52:11
TOOL (hermes, kimi-k2.6)
[Tool: terminal]
{"output": "", "exit_code": -1, "error": "BLOCKED: User denied this command. The user has NOT consented to this action. Do NOT retry this command, do NOT rephrase it, and do NOT attempt the same outcome via a different command. Stop the current workflow and wait for the user to respond before taking any further destructive or irreversible action.", "status": "blocked"}
2026-05-26 14:52:11
TOOL (hermes, kimi-k2.6)
[Tool: read_file]
{"content": " 1|# database/auth_db.py\n 2|\n 3|import base64\n 4|import os\n 5|\n 6|from argon2 import PasswordHasher\n 7|from argon2.exceptions import VerifyMismatchError\n 8|from cachetools import TTLCache\n 9|from cryptography.fernet import Fernet\n 10|from cryptography.hazmat.primitives import hashes\n 11|from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC\n 12|from sqlalchemy import (\n 13| Boolean,\n 14| Column,\n 15| DateTime,\n 16| Index,\n 17| Integer,\n 18| String,\n 19| Text,\n 20| UniqueConstraint,\n 21| create_engine,\n 22|)\n 23|from sqlalchemy.ext.declarative import declarative_base\n 24|from sqlalchemy.orm import scoped_session, sessionmaker\n 25|from sqlalchemy.pool import NullPool\n 26|from sqlalchemy.sql import func\n 27|\n 28|from utils.logging import get_logger\n 29|\n 30|# Initialize logger\n 31|logger = get_logger(__name__)\n 32|\n 33|# Initialize Argon2 hasher\n 34|ph = PasswordHasher()\n 35|\n 36|DATABASE_URL = os.getenv(\"DATABASE_URL\")\n 37|\n 38|# Security: Require API_KEY_PEPPER environment variable (fail fast if missing)\n 39|# Pepper must be at least 32 bytes (64 hex characters) for cryptographic security\n 40|_pepper_value = os.getenv(\"API_KEY_PEPPER\")\n 41|if not _pepper_value:\n 42| raise RuntimeError(\n 43| \"CRITICAL: API_KEY_PEPPER environment variable is not set. \"\n 44| \"This is required for secure password and API key hashing. \"\n 45| 'Generate one using: python -c \"import secrets; print(secrets.token_hex(32))\"'\n 46| )\n 47|if len(_pepper_value) < 32:\n 48| raise RuntimeError(\n 49| f\"CRITICAL: API_KEY_PEPPER must be at least 32 characters (got {len(_pepper_value)}). \"\n 50| 'Generate a secure pepper using: python -c \"import secrets; print(secrets.token_hex(32))\"'\n 51| )\n 52|PEPPER = _pepper_value\n 53|\n 54|\n 55|# Setup Fernet encryption for auth tokens\n 56|def get_encryption_key():\n 57| \"\"\"Generate a Fernet key from the pepper\"\"\"\n 58| kdf = PBKDF2HMAC(\n 59| algorithm=hashes.SHA256(),\n 60| length=32,\n 61| salt=b\"openalgo_static_salt\",\n 62| iterations=100000,\n 63| )\n 64| key = base64.urlsafe_b64encode(kdf.derive(PEPPER.encode()))\n 65| return Fernet(key)\n 66|\n 67|\n 68|# Initialize Fernet cipher\n 69|fernet = get_encryption_key()\n 70|\n 71|\n 72|# Calculate cache TTL based on session expiry time to minimize DB hits\n 73|def get_session_based_cache_ttl():\n 74| \"\"\"Calculate cache TTL based on daily session expiry time in .env\"\"\"\n 75| try:\n 76| from datetime import datetime\n 77|\n 78| import pytz\n 79|\n 80| # Get session expiry time from environmen
... [5245 chars total, truncated]
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
2026-05-26 14:52:11
TOOL (hermes, kimi-k2.6)
[Tool: skill_view]
{"success": true, "name": "openalgo-vps-troubleshooting", "description": "OpenAlgo VPS Operations — systematic troubleshooting, reconnaissance, and broker comparison for OpenAlgo VPS deployments. Covers session expiry, broken quotes, service crashes, path migration, OOM kills, ratchet bugs, broker-specific auth quirks, and Telegram delivery issues. Includes reference procedures for VPS discovery and broker comparison audits.\n", "tags": ["openalgo", "vps", "troubleshooting", "trading-bots", "session-expiry", "systemd"], "related_skills": [], "content": "---\nname: openalgo-vps-troubleshooting\ndescription: |\n OpenAlgo VPS Operations — systematic troubleshooting, reconnaissance, and broker comparison for OpenAlgo VPS deployments. Covers session expiry, broken quotes, service crashes, path migration, OOM kills, ratchet bugs, broker-specific auth quirks, and Telegram delivery issues. Includes reference procedures for VPS discovery and broker comparison audits.\ntags:\n - openalgo\n - vps\n - troubleshooting\n - trading-bots\n - session-expiry\n - systemd\n---\n\n# OpenAlgo VPS Operations\n\n## Overview\n\nStructured diagnostic workflow for common OpenAlgo VPS failures, plus systematic\nreconnaissance and broker comparison procedures. Covers session expiry (broken quotes),\nsystemd path mismatches after migration, missing log directories, Telegram delivery issues,\nOOM kills, ratchet bugs, and broker-specific auth quirks.\n\n## When to Use\n\n- Trading bots not sending Telegram updates\n- Bots crash-looping or failing to start\n- OpenAlgo API returning 500 errors\n- Session expiry errors (\"Invalid Session Key\")\n- After path migration when services point to old paths\n- Missing log directories, Telegram delivery issues\n- Bot OOM kills, ratchet bugs, broker-specific auth quirks\n- First-time exploration of an OpenAlgo VPS deployment\n- Comparing behavior across broker instances (Shoonya vs Flattrade vs Kotak)\n- Quotes API specifically returning 500 (see `references/quotes-500-flowchart.md`)\n- Verifying and backfilling bot databases (see `references/data-freshness-backfill.md`)\n## Reference Procedures\n\nFor systematic, step-by-step procedures covering specific operational domains, load:\n- **VPS Reconnaissance** — `references/vps-reconnaissance.md` — 8-step discovery: services, Nginx, databases, cron, logs\n- **Broker Comparison Audit** — `references/broker-comparison-audit.md` — 9-step side-by-side broker diff: configs, auth, data, streaming, sockets\n- **Quotes 500 Flowchart** — `references/quotes-500-flowchart.md` — 5-step decision tree for quotes API failures\n- **Data Freshness & Backfill** — `references/data-freshness-backfill.md` — DB verification, futures rollover, EOD backfill, stale-anchor detection, auto-sync script\n- **Session Expiry After Re-Login** — `references/session-expiry-after-relogin.md` — when broker web UI login doesn't refresh OpenAlgo's cached token; diagnostic sequence and two-step fix\n- **3
... [50000 chars total, truncated]
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
Perfect. I found the exact skill with the root cause and automation options. Let me load the critical references that explain how to build the permanent fix:
2026-05-26 14:52:11
TOOL (hermes, kimi-k2.6)
[Tool: skill_view]
{"success": true, "name": "openalgo-vps-troubleshooting", "file": "references/session-expiry-3am-mechanism.md", "content": "# OpenAlgo 3 AM Session Expiry Mechanism\n\n## Overview\n\nOpenAlgo has a built-in daily session expiry at 3:00 AM IST (`SESSION_EXPIRY_TIME = '03:00'` in `.env`). This is often misunderstood as the root cause of \"Invalid Session Key\" errors, but the real cause is broker-side token expiry (~24h). This reference documents the actual mechanism and how it interacts with broker tokens.\n\n## The `.env` Configuration\n\n```bash\ncat /var/python/openalgo-flask/flattrade-openalgo.theworkpc.com-flattrade/openalgo/.env | grep SESSION\n# → SESSION_EXPIRY_TIME = '03:00'\n```\n\nThis is a **Flask session expiry** (internal housekeeping), NOT a broker token expiry. It controls when OpenAlgo's internal user session expires.\n\n## What Happens at 3 AM\n\n1. `session.py` checks `current_time > login_time + SESSION_EXPIRY_TIME`\n2. If expired, calls `revoke_user_tokens()` which sets `is_revoked=1` in the `auth` table\n3. **BUT:** The actual broker token in `auth_token` column is NOT deleted or invalidated by OpenAlgo\n4. The broker token remains in the DB, encrypted, with `is_revoked=1`\n\n## Why \"Invalid Session Key\" Still Happens\n\nThe broker (Flattrade/Shoonya) issues OAuth tokens with their own TTL (~24 hours from issuance). When:\n- Token issued at 9:00 AM on Day 1\n- Broker token expires at 9:00 AM on Day 2\n- OpenAlgo's 3:00 AM expiry is a separate event\n\n**Result:** The broker token is dead by morning, regardless of OpenAlgo's 3 AM housekeeping.\n\n## Evidence from DB\n\n```sql\n-- Query shows token is NOT revoked in OpenAlgo\nSELECT is_revoked, auth FROM auth WHERE broker='flattrade';\n-- → is_revoked = 0, auth = 'gAAAAA...' (encrypted token present)\n\n-- But direct broker API test with decrypted token fails:\n-- {\"stat\":\"Not_Ok\",\"emsg\":\"Session Expired : Invalid User Id\"}\n```\n\nThis proves: OpenAlgo has the token, but the broker doesn't accept it.\n\n## The Complete Token Lifecycle\n\n```\nDay 1, 9:00 AM: User logs in via web UI\n ↓ OAuth callback\n ↓ OpenAlgo exchanges code for token\n ↓ Token saved to DB (encrypted)\n ↓ Gunicorn worker caches token in memory\n ↓\nDay 1, 9:00 AM → Day 2, 3:00 AM: Token works normally\n ↓\nDay 2, 3:00 AM: OpenAlgo session expiry fires\n ↓ is_revoked=1 in DB (but token still present)\n ↓\nDay 2, 9:00 AM: Broker token TTL expires (~24h from issuance)\n ↓ API calls return \"Invalid Session Key\"\n ↓\nDay 2, 9:15 AM: User logs in via web UI\n ↓ New OAuth code generated\n ↓ OpenAlgo exchanges code for NEW token\n ↓ NEW token saved to DB\n ↓ BUT: gunicorn worker still has OLD token in memory!\n ↓\nDay 2, 9:16 AM: API calls still fail with \"Invali
... [6958 chars total, truncated]
2026-05-26 14:52:11
TOOL (hermes, kimi-k2.6)
[Tool: skill_view]
{"success": true, "name": "openalgo-vps-troubleshooting", "file": "references/openalgo-token-decryption.md", "content": "# OpenAlgo Token Decryption — Direct Broker Session Validation\n\n## When to Use\n\nWhen `auto_db_sync.py --check-only` shows \"ACTIVE\" but API calls return \"Session Expired\", and you need **definitive proof** whether the broker has invalidated the session at the source. This bypasses OpenAlgo's caching, API layers, and gunicorn workers to test the raw broker token directly.\n\n## Prerequisites\n\n- Access to the VPS filesystem (OpenAlgo runs locally)\n- Python 3 with `cryptography` package (usually installed in OpenAlgo's venv)\n- The broker's `.env` file containing `API_KEY_PEPPER`\n\n## Step-by-Step\n\n### 1. Locate the OpenAlgo Instance\n\n```bash\n# Find the broker-specific OpenAlgo directory\nls /var/python/openalgo-flask/\n# → flattrade-openalgo.theworkpc.com-flattrade/\n# → shoonya-openalgo.theworkpc.com-shoonya/\n# → kotak-openalgo.theworkpc.com-kotak/\n```\n\n### 2. Read the Encryption Pepper\n\n```bash\ncat /var/python/openalgo-flask/<broker>-openalgo.theworkpc.com-<broker>/openalgo/.env | grep API_KEY_PEPPER\n# → API_KEY_PEPPER='bd5b64...39f6'\n```\n\nThe pepper is a 64-character hex string. It is used with a static salt to derive the Fernet encryption key.\n\n### 3. Decrypt the Auth Token\n\n```python\nimport os\nimport base64\nimport sqlite3\nfrom cryptography.fernet import Fernet\nfrom cryptography.hazmat.primitives import hashes\nfrom cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC\n\nos.chdir('/var/python/openalgo-flask/flattrade-openalgo.theworkpc.com-flattrade/openalgo')\n\n# Read pepper from .env\npepper = None\nwith open('.env', 'r') as f:\n for line in f:\n if line.startswith(\"API_KEY_PEPPER=\"):\n value = line.split('=', 1)[1].strip()\n pepper = value.strip(\"'\\\"\")\n break\n\n# Derive Fernet key (same algorithm as OpenAlgo's auth_db.py)\nkdf = PBKDF2HMAC(\n algorithm=hashes.SHA256(),\n length=32,\n salt=b\"openalgo_static_salt\", # hardcoded in OpenAlgo source\n iterations=100000,\n)\nkey = base64.urlsafe_b64encode(kdf.derive(pepper.encode()))\nfernet = Fernet(key)\n\n# Read encrypted token from SQLite\nconn = sqlite3.connect('db/openalgo.db')\ncursor = conn.cursor()\ncursor.execute(\"SELECT auth FROM auth WHERE broker='flattrade' AND is_revoked=0 LIMIT 1;\")\nrow = cursor.fetchone()\nconn.close()\n\nif row:\n token = fernet.decrypt(row[0].encode()).decode()\n print(f\"Decrypted token: {token}\")\n print(f\"Token length: {len(token)}\")\nelse:\n print(\"No active token found\")\n```\n\n### 4. Test the Token Directly Against the Broker's Native API\n\n**Flattrade:**\n```python\nimport requests\nimport json\n\nurl = \"https://piconnect.flattrade.in/PiConnectAPI/GetQuotes\"\npayload = {\n \"uid\": \"hnsi\", # from the auth record's user_id or name\n \"exch\": \"NFO\",\n \"token\": \"SBIN26MAY26
... [7459 chars total, truncated]
2026-05-26 14:52:11
TOOL (hermes, kimi-k2.6)
[Tool: skill_view]
{"success": true, "name": "openalgo-vps-troubleshooting", "file": "references/session-expiry-after-relogin.md", "content": "# Session Expiry After Broker Re-Login — Session Notes\n\n## Problem\n\nUser logged into Flattrade web UI, but OpenAlgo still returned `Invalid Session Key`.\n`auto_db_sync.py --check-only` showed \"ACTIVE — Active token found (1)\".\nDirect API test with real api_key still failed.\n\n## Root Cause\n\nOpenAlgo maintains its own SQLite `auth` table caching broker tokens.\nLogging into the broker's native web UI does NOT automatically refresh OpenAlgo's cached token.\nThe token in OpenAlgo's DB was stale; the broker had a fresh session, but OpenAlgo didn't know about it.\n\n## Diagnostic Sequence\n\n```bash\n# 1. Self-heal already tried restart — failed\n# Log showed: \"Tokens exist in DB — restarting service to flush cache...\"\n# \"Still failing after restart\"\n\n# 2. Direct API test — still fails\ncurl -s -X POST https://flattrade.openalgo.theworkpc.com/api/v1/quotes \\\n -H \"Content-Type: application/json\" \\\n -d '{\"apikey\":\"<REAL_KEY>\",\"symbol\":\"SBIN26MAY26FUT\",\"exchange\":\"NFO\"}'\n# → {\"message\":\"Error fetching quotes: Error from Flattrade API: Session Expired : Invalid Session Key\",\"status\":\"error\"}\n\n# 3. Token DB check — token exists but is stale\npython3 /home/ubuntu/scripts/auto_db_sync.py --broker FLATTRADE --check-only\n# → FLATTRADE: ACTIVE — Active token found (1)\n\n# 4. History API also fails — confirms broker-side rejection\ncurl -s -X POST https://flattrade.openalgo.theworkpc.com/api/v1/history \\\n -H \"Content-Type: application/json\" \\\n -d '{\"apikey\":\"<REAL_KEY>\",\"symbol\":\"SBIN26MAY26FUT\",\"exchange\":\"NFO\",\"start_date\":\"2026-05-18\",\"end_date\":\"2026-05-18\",\"interval\":\"D\"}'\n# → {\"message\":\"Error fetching historical data: Error from Flattrade API: Session Expired : Invalid Session Key\",\"status\":\"error\"}\n\n# 5. (Definitive) Decrypt raw token and test against broker native API\n# See references/openalgo-token-decryption.md for the full procedure.\n# If the raw token returns \"Session Expired\" from piconnect.flattrade.in directly,\n# the session is 100% dead at the broker — no amount of OpenAlgo restarts will fix it.\n```\n\n## The Fix (Two-Step)\n\n1. **Re-login to broker web UI** — generates fresh broker session\n2. **Login via OpenAlgo interface OR run sync script** — captures the fresh token into OpenAlgo's DB\n\nThe sync script (`/home/ubuntu/scripts/sync_after_login.sh`) runs `auto_db_sync.py` which:\n- Checks DB mtime to detect fresh logins\n- Backfills all bot databases\n- Reports via Telegram\n\n## Key Insight\n\n| What you check | What it tells you |\n|---------------|-------------------|\n| `auto_db_sync.py --check-only` shows \"ACTIVE\" | Token EXISTS in OpenAlgo DB |\n| Direct API call returns \"Session Expired\" | Token is STALE at broker |\n| Service restart doesn't help | Token in DB is the problem, n
... [4568 chars total, truncated]
2026-05-26 14:52:11
ASSISTANT (hermes, kimi-k2.6)
## Root Cause — Why This Error Keeps Coming Back
Your Shoonya OpenAlgo instance (`shoonya.openalgo.theworkpc.com`) uses **OAuth** (`GenAcsTok`), not programmatic login. The `susertoken` stored in SQLite expires at Shoonya's side every ~24 hours.
Your `self_heal.sh` restarts the service and flushes cache, but **it cannot generate a new token** — it just re-reads the same stale one from the DB. That's why you keep manually logging in every morning.
**The Flattrade OpenAlgo instance already has Shoonya QuickAuth code** (`/var/python/openalgo-flask/flattrade-openalgo.theworkpc.com-flattrade/openalgo/broker/shoonya/api/auth_api.py`) — it logs in programmatically with userid + SHA256(password) + TOTP + vendor credentials. No browser needed.
Your Shoonya instance just isn't using that path.
---
## The Permanent Fix: Switch Shoonya to QuickAuth Auto-Login
This eliminates manual login forever. Here's what's needed:
| Requirement | Status |
|-------------|--------|
| `BROKER_PASSWORD` | ✅ Already in Shoonya `.env` |
| `BROKER_API_KEY` / `BROKER_API_SECRET` | ✅ Already in Shoonya `.env` (vendor credentials) |
| `BROKER_TOTP_SECRET` | ❌ **NEED FROM YOU** |
The TOTP secret is the **base32 string** you got when you first set up 2FA in Shoonya (or scanned from the QR code). With it, `pyotp` can generate the 6-digit code automatically — no human, no browser, no IP whitelist issues.
---
## What I'll Build (Once I Have the TOTP Secret)
1. **Replace Shoonya's OAuth auth_api.py** with QuickAuth + auto-TOTP generation
2. **Token refresh daemon** — runs every 15 min, detects stale tokens, auto-logs in via QuickAuth, updates DB, restarts service
3. **ZeroMQ cache invalidation** after refresh so gunicorn picks up the new token
4. **Telegram alert on any failure** so you know if something breaks
**Result:** You never touch Shoonya login again. Set it and forget it.
---
## What I Need From You Right Now
**Your Shoonya TOTP secret** (base32 string, e.g., `JBSWY3DPEHPK3PXP`).
- If you still have the original QR code / setup screen from Shoonya, the secret is in the QR URL: `otpauth://totp/...?secret=XXXXXX`
- If you don't have it, you'll need to disable and re-enable 2FA on Shoonya to get a new secret
Once you give me that, I'll implement the fix within minutes.