Spaces:
Running
Running
Khanna, Videh Rakesh Rakesh commited on
Commit ·
a1620df
1
Parent(s): 25ec864
Fix F&O search with automatic Dhan contract discovery
Browse files- README.md +19 -3
- app.py +43 -11
- fo.py +175 -5
- static/app.js +25 -3
- static/style.css +1 -0
- templates/index.html +2 -2
- tests/conftest.py +6 -0
- tests/test_api_contract.py +82 -1
- tests/test_fo.py +131 -1
- tests/test_watchlist_frontend.cjs +25 -0
README.md
CHANGED
|
@@ -19,8 +19,21 @@ filter. Search index names or aliases such as `Nifty 50` and `BANKNIFTY`, or con
|
|
| 19 |
details such as `RELIANCE futures` and `RELIANCE 2500 CE`. Contract results show expiry,
|
| 20 |
strike and Call/Put; selecting one retains its exact identity when adding or removing it.
|
| 21 |
|
| 22 |
-
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
watchlist cards use contract quotes only, explicitly label stale/unavailable prices, and open
|
| 25 |
the F&O popup with the selected contract. They do not run equity AI/ML forecasts, use stock
|
| 26 |
prices, or enter equity cache-warming and prediction-refresh jobs. Other stock-prediction
|
|
@@ -51,7 +64,8 @@ their original semantics; the legacy `shares` column stores derivative quantity
|
|
| 51 |
| Variable | Default | Purpose |
|
| 52 |
| --- | --- | --- |
|
| 53 |
| `FO_PROVIDER` | `manual` | Optional registered live adapter; built-in choices are `manual` and `dhan` |
|
| 54 |
-
| `
|
|
|
|
| 55 |
| `FO_AUTO_MONITOR` | `1` | With a live provider, check up to five derivative positions per 15-second pass; `0` disables background checks |
|
| 56 |
|
| 57 |
The reference file must be maintained from a trusted current exchange/provider source.
|
|
@@ -69,6 +83,8 @@ properly CSV-escaped JSON mappings. For the optional Dhan adapter, its `dhan` ob
|
|
| 69 |
`security_id`; option chains also need `underlying_security_id` and `underlying_segment`
|
| 70 |
(`IDX_I` for indices, `NSE_EQ` for stocks). These are public identifiers, never credentials.
|
| 71 |
Other adapters implement `fo.FoProvider` and register a factory with `fo.register_provider`.
|
|
|
|
|
|
|
| 72 |
|
| 73 |
Live Dhan use additionally requires its existing server configuration and browser-data
|
| 74 |
permission. It remains optional and read-only. Quote and chain controls show their capability
|
|
|
|
| 19 |
details such as `RELIANCE futures` and `RELIANCE 2500 CE`. Contract results show expiry,
|
| 20 |
strike and Call/Put; selecting one retains its exact identity when adding or removing it.
|
| 21 |
|
| 22 |
+
Queries such as `NIFTY 19th OCT 23000 CE`, `NIFTY 19 October 2026 23000 call` and ISO
|
| 23 |
+
expiry dates are supported. A fully specified typed query can be added directly when it
|
| 24 |
+
matches exactly one listed contract; ambiguous expiries require selecting a search result.
|
| 25 |
+
Strike, option side and complete underlying symbols are matched exactly.
|
| 26 |
+
|
| 27 |
+
Futures/options discovery defaults to Dhan's public detailed instrument master, without
|
| 28 |
+
credentials or a data subscription. Yahoo-backed stock/index data paths are unchanged;
|
| 29 |
+
Yahoo stock prices are never substituted for derivative contracts. The catalog loads in
|
| 30 |
+
the background, and search displays its state and retries while loading. Verified references
|
| 31 |
+
are cached for 24 hours in `paper_trading.db` (`app_result_cache`) on HF persistent storage;
|
| 32 |
+
at most 48 hours of verified cache is retained during an outage, with a stale label. Expired
|
| 33 |
+
contracts, including expiry-day contracts after 15:30 IST, are excluded. A configured
|
| 34 |
+
`FO_REFERENCE_CSV` overrides automatic discovery (configuration below).
|
| 35 |
+
|
| 36 |
+
No contracts or lot sizes are fabricated when the catalog is unavailable. Derivative
|
| 37 |
watchlist cards use contract quotes only, explicitly label stale/unavailable prices, and open
|
| 38 |
the F&O popup with the selected contract. They do not run equity AI/ML forecasts, use stock
|
| 39 |
prices, or enter equity cache-warming and prediction-refresh jobs. Other stock-prediction
|
|
|
|
| 64 |
| Variable | Default | Purpose |
|
| 65 |
| --- | --- | --- |
|
| 66 |
| `FO_PROVIDER` | `manual` | Optional registered live adapter; built-in choices are `manual` and `dhan` |
|
| 67 |
+
| `FO_AUTO_REFERENCE` | `1` | Discover NSE stock/index futures and options from the public Dhan master; `0` disables discovery |
|
| 68 |
+
| `FO_REFERENCE_CSV` | Unset | Override discovery with a server-managed, validated neutral CSV; use `/data/fo_contracts.csv` on HF Spaces |
|
| 69 |
| `FO_AUTO_MONITOR` | `1` | With a live provider, check up to five derivative positions per 15-second pass; `0` disables background checks |
|
| 70 |
|
| 71 |
The reference file must be maintained from a trusted current exchange/provider source.
|
|
|
|
| 83 |
`security_id`; option chains also need `underlying_security_id` and `underlying_segment`
|
| 84 |
(`IDX_I` for indices, `NSE_EQ` for stocks). These are public identifiers, never credentials.
|
| 85 |
Other adapters implement `fo.FoProvider` and register a factory with `fo.register_provider`.
|
| 86 |
+
Automatic public references include Dhan contract quote IDs. Underlying mappings needed
|
| 87 |
+
for the optional live option-chain feature still require a configured reference override.
|
| 88 |
|
| 89 |
Live Dhan use additionally requires its existing server configuration and browser-data
|
| 90 |
permission. It remains optional and read-only. Quote and chain controls show their capability
|
app.py
CHANGED
|
@@ -1233,6 +1233,7 @@ def search_ticker():
|
|
| 1233 |
if not q:
|
| 1234 |
return jsonify({"results": []})
|
| 1235 |
|
|
|
|
| 1236 |
candidates = []
|
| 1237 |
for ticker, (name, aliases) in _WATCHLIST_INDICES.items():
|
| 1238 |
candidates.append(({"ticker": ticker, "name": name, "instrument_type": "INDEX"}, aliases))
|
|
@@ -1247,24 +1248,33 @@ def search_ticker():
|
|
| 1247 |
except Exception:
|
| 1248 |
pass
|
| 1249 |
|
|
|
|
| 1250 |
if kind in {"", "OPTION", "FUTURE"}:
|
| 1251 |
-
|
|
|
|
|
|
|
| 1252 |
candidates.append(({"ticker": contract["contract_id"], "name": fo.contract_label(contract),
|
| 1253 |
"instrument_type": contract["instrument_type"], "contract": contract}, "FUT"))
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1254 |
|
| 1255 |
-
terms = [{"FUT": "FUTURE", "FUTURES": "FUTURE", "OPTIONS": "OPTION"}.get(term, term)
|
| 1256 |
-
for term in q.split()]
|
| 1257 |
matches = []
|
| 1258 |
for row, aliases in candidates:
|
| 1259 |
if kind and row["instrument_type"] != kind:
|
| 1260 |
continue
|
| 1261 |
bare = _bare(row["ticker"])
|
| 1262 |
haystack = f"{bare} {row['name']} {row['instrument_type']} {aliases}".upper()
|
| 1263 |
-
|
|
|
|
| 1264 |
rank = 0 if q == bare or q in aliases.split() else 1 if bare.startswith(q) else 2
|
| 1265 |
matches.append((rank, row))
|
| 1266 |
matches.sort(key=lambda match: match[0])
|
| 1267 |
-
|
|
|
|
|
|
|
|
|
|
| 1268 |
|
| 1269 |
|
| 1270 |
# ── MARKET STATUS ─────────────────────────────────────────────────────────────
|
|
@@ -2206,7 +2216,28 @@ def watchlist_add():
|
|
| 2206 |
name = data.get("name", _bare(ticker))
|
| 2207 |
if not ticker:
|
| 2208 |
return jsonify({"error": "ticker required"}), 400
|
|
|
|
|
|
|
|
|
|
| 2209 |
t = _normalise(ticker)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2210 |
if t.startswith("NSE:") and _watchlist_instrument_type(t) not in {"OPTION", "FUTURE"}:
|
| 2211 |
return jsonify({"error": "Invalid derivative contract identifier"}), 400
|
| 2212 |
if _watchlist_instrument_type(t) in {"OPTION", "FUTURE"}:
|
|
@@ -2220,12 +2251,13 @@ def watchlist_add():
|
|
| 2220 |
return jsonify({"error": "Unknown index symbol"}), 400
|
| 2221 |
# Membership in the cached official NSE equity list is sufficient to add. Requiring
|
| 2222 |
# a live price and 200 bars here makes adding fail whenever a market-data provider is slow.
|
| 2223 |
-
|
| 2224 |
-
|
| 2225 |
-
|
| 2226 |
-
|
| 2227 |
-
|
| 2228 |
-
|
|
|
|
| 2229 |
if not name or name == _bare(ticker):
|
| 2230 |
name = _WATCHLIST_INDICES[t][0] if t in _WATCHLIST_INDICES else TICKER_NAMES.get(t, _bare(t))
|
| 2231 |
try:
|
|
|
|
| 1233 |
if not q:
|
| 1234 |
return jsonify({"results": []})
|
| 1235 |
|
| 1236 |
+
terms = fo.search_terms(q)
|
| 1237 |
candidates = []
|
| 1238 |
for ticker, (name, aliases) in _WATCHLIST_INDICES.items():
|
| 1239 |
candidates.append(({"ticker": ticker, "name": name, "instrument_type": "INDEX"}, aliases))
|
|
|
|
| 1248 |
except Exception:
|
| 1249 |
pass
|
| 1250 |
|
| 1251 |
+
reference = None
|
| 1252 |
if kind in {"", "OPTION", "FUTURE"}:
|
| 1253 |
+
service = fo.get_service()
|
| 1254 |
+
contracts = service.contracts(instrument_type=kind)
|
| 1255 |
+
for contract in fo.matching_contracts(contracts, terms):
|
| 1256 |
candidates.append(({"ticker": contract["contract_id"], "name": fo.contract_label(contract),
|
| 1257 |
"instrument_type": contract["instrument_type"], "contract": contract}, "FUT"))
|
| 1258 |
+
status = service.status()
|
| 1259 |
+
if isinstance(status, dict):
|
| 1260 |
+
reference = {"state": status.get("reference_state"), "source": status.get("reference_source"),
|
| 1261 |
+
"count": status.get("reference_count", 0)}
|
| 1262 |
|
|
|
|
|
|
|
| 1263 |
matches = []
|
| 1264 |
for row, aliases in candidates:
|
| 1265 |
if kind and row["instrument_type"] != kind:
|
| 1266 |
continue
|
| 1267 |
bare = _bare(row["ticker"])
|
| 1268 |
haystack = f"{bare} {row['name']} {row['instrument_type']} {aliases}".upper()
|
| 1269 |
+
matched = "contract" in row or all(term in haystack for term in terms)
|
| 1270 |
+
if matched:
|
| 1271 |
rank = 0 if q == bare or q in aliases.split() else 1 if bare.startswith(q) else 2
|
| 1272 |
matches.append((rank, row))
|
| 1273 |
matches.sort(key=lambda match: match[0])
|
| 1274 |
+
payload = {"results": [row for _, row in matches[:20]]}
|
| 1275 |
+
if reference is not None:
|
| 1276 |
+
payload["reference"] = reference
|
| 1277 |
+
return jsonify(payload)
|
| 1278 |
|
| 1279 |
|
| 1280 |
# ── MARKET STATUS ─────────────────────────────────────────────────────────────
|
|
|
|
| 2216 |
name = data.get("name", _bare(ticker))
|
| 2217 |
if not ticker:
|
| 2218 |
return jsonify({"error": "ticker required"}), 400
|
| 2219 |
+
kind = data.get("instrument_type", "").upper()
|
| 2220 |
+
if kind not in {"", "EQUITY", "INDEX", "OPTION", "FUTURE"}:
|
| 2221 |
+
return jsonify({"error": "Invalid instrument_type"}), 400
|
| 2222 |
t = _normalise(ticker)
|
| 2223 |
+
terms = fo.search_terms(ticker)
|
| 2224 |
+
if not ticker.startswith(("NSE:", "^")) and (kind in {"OPTION", "FUTURE"} or any(term in {"CE", "PE", "OPTION", "FUTURE"} for term in terms)):
|
| 2225 |
+
service = fo.get_service()
|
| 2226 |
+
matches = fo.matching_contracts(service.contracts(instrument_type=kind if kind in {"OPTION", "FUTURE"} else ""), terms)
|
| 2227 |
+
if not matches:
|
| 2228 |
+
state = service.status().get("reference_state")
|
| 2229 |
+
if state in {"loading", "unavailable", "not_configured"}:
|
| 2230 |
+
message = "Contract catalog is loading. Retry shortly." if state == "loading" else "Contract catalog is unavailable. Retry after it refreshes."
|
| 2231 |
+
return jsonify({"error": message, "retryable": state == "loading"}), 503
|
| 2232 |
+
return jsonify({"error": "No listed contract matches that underlying, expiry, strike and option type."}), 400
|
| 2233 |
+
if len(matches) != 1:
|
| 2234 |
+
return jsonify({"error": "Multiple contracts match. Select the exact expiry and contract from search."}), 400
|
| 2235 |
+
t = matches[0]["contract_id"]
|
| 2236 |
+
elif kind in {"", "INDEX"}:
|
| 2237 |
+
t = next((symbol for symbol, (label, aliases) in _WATCHLIST_INDICES.items()
|
| 2238 |
+
if ticker == label.upper() or ticker in aliases.split()), t)
|
| 2239 |
+
if kind and kind != _watchlist_instrument_type(t):
|
| 2240 |
+
return jsonify({"error": "Selected instrument type does not match the ticker or contract."}), 400
|
| 2241 |
if t.startswith("NSE:") and _watchlist_instrument_type(t) not in {"OPTION", "FUTURE"}:
|
| 2242 |
return jsonify({"error": "Invalid derivative contract identifier"}), 400
|
| 2243 |
if _watchlist_instrument_type(t) in {"OPTION", "FUTURE"}:
|
|
|
|
| 2251 |
return jsonify({"error": "Unknown index symbol"}), 400
|
| 2252 |
# Membership in the cached official NSE equity list is sufficient to add. Requiring
|
| 2253 |
# a live price and 200 bars here makes adding fail whenever a market-data provider is slow.
|
| 2254 |
+
if _watchlist_instrument_type(t) == "EQUITY":
|
| 2255 |
+
try:
|
| 2256 |
+
universe = get_universe()
|
| 2257 |
+
if universe and t.endswith(".NS") and t not in universe:
|
| 2258 |
+
return jsonify({"error": f"Ticker {t} is not in the current NSE equity list."}), 400
|
| 2259 |
+
except Exception:
|
| 2260 |
+
pass # A universe refresh problem must not prevent adding a valid ticker.
|
| 2261 |
if not name or name == _bare(ticker):
|
| 2262 |
name = _WATCHLIST_INDICES[t][0] if t in _WATCHLIST_INDICES else TICKER_NAMES.get(t, _bare(t))
|
| 2263 |
try:
|
fo.py
CHANGED
|
@@ -8,12 +8,19 @@ import json
|
|
| 8 |
import math
|
| 9 |
import os
|
| 10 |
import re
|
|
|
|
| 11 |
import threading
|
| 12 |
import time
|
| 13 |
from typing import Protocol
|
| 14 |
|
|
|
|
|
|
|
| 15 |
|
| 16 |
IST = timezone(timedelta(hours=5, minutes=30))
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
|
| 19 |
def number(value, name, *, allow_zero=False):
|
|
@@ -84,6 +91,53 @@ def contract_label(contract):
|
|
| 84 |
return label + (f" {contract['strike']:g} {contract['option_type']}" if contract["option_type"] else " FUT")
|
| 85 |
|
| 86 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
def validate_price(trade, value):
|
| 88 |
contract = contract_for_trade(trade)
|
| 89 |
price = number(value, "contract price", allow_zero=trade["instrument_type"] == "OPTION")
|
|
@@ -140,12 +194,19 @@ class FoProvider(Protocol):
|
|
| 140 |
|
| 141 |
|
| 142 |
def parse_reference_csv(text):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
contracts, ambiguous = {}, set()
|
| 144 |
-
for
|
| 145 |
-
row = {(key or "").strip(): (value or "").strip() for key, value in raw.items()}
|
| 146 |
try:
|
| 147 |
contract = validate_contract(row, row.get("instrument_type"))
|
| 148 |
-
mappings =
|
|
|
|
|
|
|
| 149 |
if not isinstance(mappings, dict):
|
| 150 |
continue
|
| 151 |
allowed = {"security_id", "underlying_security_id", "underlying_segment", "symbol"}
|
|
@@ -161,6 +222,30 @@ def parse_reference_csv(text):
|
|
| 161 |
return [contract for identity, contract in contracts.items() if identity not in ambiguous]
|
| 162 |
|
| 163 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 164 |
def normalize_quote(raw, *, now=None):
|
| 165 |
now = time.time() if now is None else now
|
| 166 |
result = {"source": str(raw.get("source") or "unknown")[:40]}
|
|
@@ -194,11 +279,91 @@ class FoService:
|
|
| 194 |
self._catalog = []
|
| 195 |
self._catalog_key = None
|
| 196 |
self._reference_state = "not_configured"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 197 |
self._quotes = {}
|
| 198 |
self._chains = {}
|
| 199 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 200 |
def contracts(self, underlying="", instrument_type="", expiry=""):
|
| 201 |
path = os.getenv("FO_REFERENCE_CSV", "")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
try:
|
| 203 |
info = os.stat(path) if path else None
|
| 204 |
key = (path, info.st_mtime_ns, info.st_size) if info else None
|
|
@@ -214,9 +379,13 @@ class FoService:
|
|
| 214 |
except (OSError, ValueError):
|
| 215 |
self._reference_state = "unavailable"
|
| 216 |
rows = []
|
|
|
|
|
|
|
|
|
|
| 217 |
symbol = underlying.upper().removesuffix(".NS").strip()
|
| 218 |
-
|
| 219 |
-
|
|
|
|
| 220 |
and (not symbol or row["underlying"] == symbol)
|
| 221 |
and (not instrument_type or row["instrument_type"] == instrument_type)
|
| 222 |
and (not expiry or row["expiry"] == expiry)]
|
|
@@ -226,6 +395,7 @@ class FoService:
|
|
| 226 |
caps = self.provider.capabilities() if self.provider else {}
|
| 227 |
return {"provider": self.name, "manual_available": True,
|
| 228 |
"reference_state": self._reference_state, "reference_count": len(rows),
|
|
|
|
| 229 |
"capabilities": {key: caps.get(key) is True for key in ("quotes", "option_chain", "margin")}}
|
| 230 |
|
| 231 |
def resolve(self, contract_id):
|
|
|
|
| 8 |
import math
|
| 9 |
import os
|
| 10 |
import re
|
| 11 |
+
import sqlite3
|
| 12 |
import threading
|
| 13 |
import time
|
| 14 |
from typing import Protocol
|
| 15 |
|
| 16 |
+
import requests
|
| 17 |
+
|
| 18 |
|
| 19 |
IST = timezone(timedelta(hours=5, minutes=30))
|
| 20 |
+
PUBLIC_REFERENCE_URL = "https://images.dhan.co/api-data/api-scrip-master-detailed.csv"
|
| 21 |
+
_PUBLIC_REFERENCE_KEY = "fo_public_reference_v1"
|
| 22 |
+
_REFERENCE_TTL = 86400
|
| 23 |
+
_REFERENCE_MAX_AGE = 2 * _REFERENCE_TTL
|
| 24 |
|
| 25 |
|
| 26 |
def number(value, name, *, allow_zero=False):
|
|
|
|
| 91 |
return label + (f" {contract['strike']:g} {contract['option_type']}" if contract["option_type"] else " FUT")
|
| 92 |
|
| 93 |
|
| 94 |
+
def search_terms(query):
|
| 95 |
+
def expiry_term(match):
|
| 96 |
+
day, month, year = match.groups()
|
| 97 |
+
for pattern in ("%d %b %Y", "%d %B %Y"):
|
| 98 |
+
try:
|
| 99 |
+
expiry = datetime.strptime(f"{day} {month} {year or '2000'}", pattern)
|
| 100 |
+
return expiry.strftime("%Y-%m-%d" if year else "-%m-%d")
|
| 101 |
+
except ValueError:
|
| 102 |
+
continue
|
| 103 |
+
return match.group()
|
| 104 |
+
|
| 105 |
+
query = re.sub(r"\b(\d{1,2})(?:ST|ND|RD|TH)?\s+([A-Z]{3,9})(?:\s+(\d{4})\b(?!\s+(?:CE|PE|CALLS?|PUTS?)\b))?",
|
| 106 |
+
expiry_term, query.upper())
|
| 107 |
+
aliases = {"FUT": "FUTURE", "FUTURES": "FUTURE", "OPTIONS": "OPTION",
|
| 108 |
+
"CALL": "CE", "CALLS": "CE", "PUT": "PE", "PUTS": "PE"}
|
| 109 |
+
return [aliases.get(term, term) for term in query.split()]
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
def contract_matches(contract, terms):
|
| 113 |
+
for term in terms:
|
| 114 |
+
if re.fullmatch(r"\d+(?:\.\d+)?", term):
|
| 115 |
+
if contract["strike"] is None or Decimal(term) != Decimal(str(contract["strike"])):
|
| 116 |
+
return False
|
| 117 |
+
elif re.fullmatch(r"(?:\d{4})?-\d{2}-\d{2}", term):
|
| 118 |
+
if not contract["expiry"].endswith(term):
|
| 119 |
+
return False
|
| 120 |
+
elif term in {"CE", "PE"}:
|
| 121 |
+
if contract["option_type"] != term:
|
| 122 |
+
return False
|
| 123 |
+
elif term in {"OPTION", "FUTURE"}:
|
| 124 |
+
if contract["instrument_type"] != term:
|
| 125 |
+
return False
|
| 126 |
+
elif term not in contract["underlying"] and term != contract["contract_id"]:
|
| 127 |
+
return False
|
| 128 |
+
return True
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def matching_contracts(contracts, terms):
|
| 132 |
+
symbols = {contract["underlying"] for contract in contracts}
|
| 133 |
+
exact_symbol = next((term for term in terms if term in symbols), None)
|
| 134 |
+
matches = [contract for contract in contracts
|
| 135 |
+
if (not exact_symbol or contract["underlying"] == exact_symbol)
|
| 136 |
+
and contract_matches(contract, terms)]
|
| 137 |
+
return sorted(matches, key=lambda contract: (contract["expiry"], contract["underlying"],
|
| 138 |
+
contract["strike"] or 0, contract["option_type"] or ""))
|
| 139 |
+
|
| 140 |
+
|
| 141 |
def validate_price(trade, value):
|
| 142 |
contract = contract_for_trade(trade)
|
| 143 |
price = number(value, "contract price", allow_zero=trade["instrument_type"] == "OPTION")
|
|
|
|
| 194 |
|
| 195 |
|
| 196 |
def parse_reference_csv(text):
|
| 197 |
+
rows = ({(key or "").strip(): (value or "").strip() for key, value in raw.items()}
|
| 198 |
+
for raw in csv.DictReader(io.StringIO(text)))
|
| 199 |
+
return _validated_references(rows)
|
| 200 |
+
|
| 201 |
+
|
| 202 |
+
def _validated_references(rows):
|
| 203 |
contracts, ambiguous = {}, set()
|
| 204 |
+
for row in rows:
|
|
|
|
| 205 |
try:
|
| 206 |
contract = validate_contract(row, row.get("instrument_type"))
|
| 207 |
+
mappings = row.get("provider_ids") or {}
|
| 208 |
+
if isinstance(mappings, str):
|
| 209 |
+
mappings = json.loads(mappings)
|
| 210 |
if not isinstance(mappings, dict):
|
| 211 |
continue
|
| 212 |
allowed = {"security_id", "underlying_security_id", "underlying_segment", "symbol"}
|
|
|
|
| 222 |
return [contract for identity, contract in contracts.items() if identity not in ambiguous]
|
| 223 |
|
| 224 |
|
| 225 |
+
def parse_public_reference_csv(lines):
|
| 226 |
+
def references():
|
| 227 |
+
kinds = {"OPTIDX": ("OPTION", "INDEX"), "OPTSTK": ("OPTION", "STOCK"),
|
| 228 |
+
"FUTIDX": ("FUTURE", "INDEX"), "FUTSTK": ("FUTURE", "STOCK")}
|
| 229 |
+
for row in csv.DictReader(lines):
|
| 230 |
+
kind = kinds.get(row.get("INSTRUMENT"))
|
| 231 |
+
security_id = row.get("SECURITY_ID", "")
|
| 232 |
+
if row.get("EXCH_ID") != "NSE" or row.get("SEGMENT") != "D" or not kind or not security_id.isdigit():
|
| 233 |
+
continue
|
| 234 |
+
try:
|
| 235 |
+
lot = number(row.get("LOT_SIZE"), "lot_size")
|
| 236 |
+
if lot != lot.to_integral_value():
|
| 237 |
+
continue
|
| 238 |
+
expiry = datetime.fromisoformat(row.get("SM_EXPIRY_DATE", "")).date().isoformat()
|
| 239 |
+
yield {"underlying": row.get("UNDERLYING_SYMBOL"), "underlying_type": kind[1],
|
| 240 |
+
"instrument_type": kind[0], "expiry": expiry, "lot_size": int(lot),
|
| 241 |
+
"tick_size": number(row.get("TICK_SIZE"), "tick_size") / 100,
|
| 242 |
+
"strike": row.get("STRIKE_PRICE"), "option_type": row.get("OPTION_TYPE"),
|
| 243 |
+
"provider_ids": {"dhan": {"security_id": security_id}}}
|
| 244 |
+
except (ValueError, TypeError):
|
| 245 |
+
continue
|
| 246 |
+
return _validated_references(references())
|
| 247 |
+
|
| 248 |
+
|
| 249 |
def normalize_quote(raw, *, now=None):
|
| 250 |
now = time.time() if now is None else now
|
| 251 |
result = {"source": str(raw.get("source") or "unknown")[:40]}
|
|
|
|
| 279 |
self._catalog = []
|
| 280 |
self._catalog_key = None
|
| 281 |
self._reference_state = "not_configured"
|
| 282 |
+
self._reference_source = "csv"
|
| 283 |
+
self._reference_lock = threading.Lock()
|
| 284 |
+
self._reference_loading = False
|
| 285 |
+
self._reference_loaded = False
|
| 286 |
+
self._reference_at = 0.0
|
| 287 |
+
self._reference_retry_at = 0.0
|
| 288 |
self._quotes = {}
|
| 289 |
self._chains = {}
|
| 290 |
|
| 291 |
+
def _refresh_public_reference(self):
|
| 292 |
+
try:
|
| 293 |
+
deadline = time.monotonic() + 60
|
| 294 |
+
with requests.get(PUBLIC_REFERENCE_URL, stream=True, timeout=(5, 10)) as response:
|
| 295 |
+
response.raise_for_status()
|
| 296 |
+
response.encoding = "utf-8-sig"
|
| 297 |
+
if int(response.headers.get("Content-Length", "0")) > 64_000_000:
|
| 298 |
+
raise ValueError("Reference response is too large")
|
| 299 |
+
|
| 300 |
+
def bounded_lines():
|
| 301 |
+
size = 0
|
| 302 |
+
for line in response.iter_lines(decode_unicode=True):
|
| 303 |
+
size += len(line.encode("utf-8")) + 1
|
| 304 |
+
if size > 64_000_000 or time.monotonic() > deadline:
|
| 305 |
+
raise ValueError("Reference download exceeded its budget")
|
| 306 |
+
yield line
|
| 307 |
+
|
| 308 |
+
rows = parse_public_reference_csv(bounded_lines())
|
| 309 |
+
if not rows:
|
| 310 |
+
raise ValueError("Public reference contains no valid NSE contracts")
|
| 311 |
+
from database import save_app_result_cache
|
| 312 |
+
saved_at = time.time()
|
| 313 |
+
try:
|
| 314 |
+
save_app_result_cache(_PUBLIC_REFERENCE_KEY, {"contracts": rows}, saved_at)
|
| 315 |
+
except (sqlite3.Error, OSError):
|
| 316 |
+
pass
|
| 317 |
+
with self._reference_lock:
|
| 318 |
+
self._catalog = rows
|
| 319 |
+
self._reference_at = saved_at
|
| 320 |
+
self._reference_state = "available"
|
| 321 |
+
except (requests.RequestException, OSError, ValueError, TypeError):
|
| 322 |
+
with self._reference_lock:
|
| 323 |
+
self._reference_state = "unavailable"
|
| 324 |
+
self._reference_retry_at = time.time() + 60
|
| 325 |
+
finally:
|
| 326 |
+
with self._reference_lock:
|
| 327 |
+
self._reference_loading = False
|
| 328 |
+
|
| 329 |
+
def _public_contracts(self):
|
| 330 |
+
self._reference_source = "dhan_public"
|
| 331 |
+
now = time.time()
|
| 332 |
+
start_refresh = False
|
| 333 |
+
with self._reference_lock:
|
| 334 |
+
if not self._reference_loaded:
|
| 335 |
+
from database import get_app_result_cache
|
| 336 |
+
try:
|
| 337 |
+
cached = get_app_result_cache(_PUBLIC_REFERENCE_KEY)
|
| 338 |
+
if cached and 0 <= now - cached["saved_at"] <= _REFERENCE_MAX_AGE:
|
| 339 |
+
rows = cached["result"].get("contracts", [])
|
| 340 |
+
if isinstance(rows, list) and all(isinstance(row, dict) and "expiry" in row for row in rows):
|
| 341 |
+
self._catalog = _validated_references(rows)
|
| 342 |
+
if self._catalog:
|
| 343 |
+
self._reference_at = cached["saved_at"]
|
| 344 |
+
except (sqlite3.Error, OSError, ValueError, TypeError, KeyError):
|
| 345 |
+
pass
|
| 346 |
+
self._reference_loaded = True
|
| 347 |
+
age = now - self._reference_at
|
| 348 |
+
if self._reference_at and 0 <= age < _REFERENCE_TTL:
|
| 349 |
+
self._reference_state = "available"
|
| 350 |
+
elif not self._reference_loading and now >= self._reference_retry_at:
|
| 351 |
+
self._reference_loading = True
|
| 352 |
+
self._reference_state = "loading"
|
| 353 |
+
start_refresh = True
|
| 354 |
+
if self._catalog and self._reference_at and 0 <= age <= _REFERENCE_MAX_AGE and age >= _REFERENCE_TTL:
|
| 355 |
+
self._reference_state = "stale"
|
| 356 |
+
rows = self._catalog if self._reference_at and 0 <= age <= _REFERENCE_MAX_AGE else []
|
| 357 |
+
if start_refresh:
|
| 358 |
+
threading.Thread(target=self._refresh_public_reference, daemon=True, name="fo-reference-refresh").start()
|
| 359 |
+
return rows
|
| 360 |
+
|
| 361 |
def contracts(self, underlying="", instrument_type="", expiry=""):
|
| 362 |
path = os.getenv("FO_REFERENCE_CSV", "")
|
| 363 |
+
if not path and os.getenv("FO_AUTO_REFERENCE", "1") == "1":
|
| 364 |
+
rows = self._public_contracts()
|
| 365 |
+
return self._filter_contracts(rows, underlying, instrument_type, expiry)
|
| 366 |
+
self._reference_source = "csv"
|
| 367 |
try:
|
| 368 |
info = os.stat(path) if path else None
|
| 369 |
key = (path, info.st_mtime_ns, info.st_size) if info else None
|
|
|
|
| 379 |
except (OSError, ValueError):
|
| 380 |
self._reference_state = "unavailable"
|
| 381 |
rows = []
|
| 382 |
+
return self._filter_contracts(rows, underlying, instrument_type, expiry)
|
| 383 |
+
|
| 384 |
+
def _filter_contracts(self, rows, underlying, instrument_type, expiry):
|
| 385 |
symbol = underlying.upper().removesuffix(".NS").strip()
|
| 386 |
+
now = datetime.now(IST)
|
| 387 |
+
today = now.date().isoformat()
|
| 388 |
+
return [dict(row) for row in rows if (row["expiry"] > today or (row["expiry"] == today and (now.hour, now.minute) < (15, 30)))
|
| 389 |
and (not symbol or row["underlying"] == symbol)
|
| 390 |
and (not instrument_type or row["instrument_type"] == instrument_type)
|
| 391 |
and (not expiry or row["expiry"] == expiry)]
|
|
|
|
| 395 |
caps = self.provider.capabilities() if self.provider else {}
|
| 396 |
return {"provider": self.name, "manual_available": True,
|
| 397 |
"reference_state": self._reference_state, "reference_count": len(rows),
|
| 398 |
+
"reference_source": self._reference_source,
|
| 399 |
"capabilities": {key: caps.get(key) is True for key in ("quotes", "option_chain", "margin")}}
|
| 400 |
|
| 401 |
def resolve(self, contract_id):
|
static/app.js
CHANGED
|
@@ -2964,6 +2964,28 @@ function _searchWatchlistSuggestions() {
|
|
| 2964 |
});
|
| 2965 |
wlSuggest.appendChild(item);
|
| 2966 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2967 |
} catch (error) {
|
| 2968 |
if (seq !== _wlSuggestSeq || error.name === 'AbortError') return;
|
| 2969 |
wlSuggest.textContent = 'Search unavailable. Try again.';
|
|
@@ -2982,15 +3004,15 @@ wlAddInput?.addEventListener('blur', () => {
|
|
| 2982 |
document.getElementById('wl-add-btn')?.addEventListener('click', async () => {
|
| 2983 |
const ticker = wlAddInput.value.toUpperCase().trim();
|
| 2984 |
if (!ticker) return;
|
| 2985 |
-
await addToWatchlist(ticker, _wlSelectedName);
|
| 2986 |
});
|
| 2987 |
|
| 2988 |
-
async function addToWatchlist(ticker, name) {
|
| 2989 |
try {
|
| 2990 |
const res = await fetch('/api/watchlist', {
|
| 2991 |
method: 'POST',
|
| 2992 |
headers: { 'Content-Type': 'application/json' },
|
| 2993 |
-
body: JSON.stringify({ ticker, name }),
|
| 2994 |
});
|
| 2995 |
const data = await res.json();
|
| 2996 |
if (!res.ok) throw new Error(data.error || `Server error ${res.status}`);
|
|
|
|
| 2964 |
});
|
| 2965 |
wlSuggest.appendChild(item);
|
| 2966 |
});
|
| 2967 |
+
const state = data.reference?.state;
|
| 2968 |
+
let message = '';
|
| 2969 |
+
if (state === 'loading') {
|
| 2970 |
+
message = 'Loading listed futures/options contracts...';
|
| 2971 |
+
_wlSuggestTimer = setTimeout(() => {
|
| 2972 |
+
if (seq === _wlSuggestSeq) _searchWatchlistSuggestions();
|
| 2973 |
+
}, 2000);
|
| 2974 |
+
} else if (state === 'unavailable' || state === 'not_configured') {
|
| 2975 |
+
message = 'Contract catalog is unavailable. Retry shortly.';
|
| 2976 |
+
} else if (state === 'stale') {
|
| 2977 |
+
message = 'Cached contract references; refreshing...';
|
| 2978 |
+
} else if (!results.length) {
|
| 2979 |
+
message = 'No matching listed instruments.';
|
| 2980 |
+
}
|
| 2981 |
+
if (message && results.length) {
|
| 2982 |
+
const status = document.createElement('div');
|
| 2983 |
+
status.className = 'suggestion-status';
|
| 2984 |
+
status.textContent = message;
|
| 2985 |
+
wlSuggest.appendChild(status);
|
| 2986 |
+
} else if (message) {
|
| 2987 |
+
wlSuggest.textContent = message;
|
| 2988 |
+
}
|
| 2989 |
} catch (error) {
|
| 2990 |
if (seq !== _wlSuggestSeq || error.name === 'AbortError') return;
|
| 2991 |
wlSuggest.textContent = 'Search unavailable. Try again.';
|
|
|
|
| 3004 |
document.getElementById('wl-add-btn')?.addEventListener('click', async () => {
|
| 3005 |
const ticker = wlAddInput.value.toUpperCase().trim();
|
| 3006 |
if (!ticker) return;
|
| 3007 |
+
await addToWatchlist(ticker, _wlSelectedName, wlSearchType?.value || '');
|
| 3008 |
});
|
| 3009 |
|
| 3010 |
+
async function addToWatchlist(ticker, name, instrumentType = '') {
|
| 3011 |
try {
|
| 3012 |
const res = await fetch('/api/watchlist', {
|
| 3013 |
method: 'POST',
|
| 3014 |
headers: { 'Content-Type': 'application/json' },
|
| 3015 |
+
body: JSON.stringify({ ticker, name, instrument_type: instrumentType }),
|
| 3016 |
});
|
| 3017 |
const data = await res.json();
|
| 3018 |
if (!res.ok) throw new Error(data.error || `Server error ${res.status}`);
|
static/style.css
CHANGED
|
@@ -275,6 +275,7 @@ input::placeholder { color: var(--text-dim); }
|
|
| 275 |
.watchlist-contract-card .pick-name { overflow-wrap: anywhere; }
|
| 276 |
#wl-search-type { width: auto; max-width: 100%; }
|
| 277 |
.suggestion-name { font-size: 12px; color: var(--text-muted); }
|
|
|
|
| 278 |
|
| 279 |
.tag-list { display: flex; flex-wrap: wrap; gap: 6px; margin-top: 8px; min-height: 0; }
|
| 280 |
.tag {
|
|
|
|
| 275 |
.watchlist-contract-card .pick-name { overflow-wrap: anywhere; }
|
| 276 |
#wl-search-type { width: auto; max-width: 100%; }
|
| 277 |
.suggestion-name { font-size: 12px; color: var(--text-muted); }
|
| 278 |
+
.suggestion-status { padding: 10px 12px; font-size: 12px; color: var(--text-muted); }
|
| 279 |
|
| 280 |
.tag-list { display: flex; flex-wrap: wrap; gap: 6px; margin-top: 8px; min-height: 0; }
|
| 281 |
.tag {
|
templates/index.html
CHANGED
|
@@ -33,7 +33,7 @@
|
|
| 33 |
@keyframes apl-spin { to { transform: rotate(360deg); } }
|
| 34 |
.apl-hint { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; font-size: 11px; color: #5a6278; }
|
| 35 |
</style>
|
| 36 |
-
<link rel="stylesheet" href="/static/style.css?v=
|
| 37 |
<script src="/static/vendor/lightweight-charts-4.2.0.js"></script>
|
| 38 |
</head>
|
| 39 |
<body>
|
|
@@ -537,6 +537,6 @@
|
|
| 537 |
</a>
|
| 538 |
</nav>
|
| 539 |
|
| 540 |
-
<script src="/static/app.js?v=
|
| 541 |
</body>
|
| 542 |
</html>
|
|
|
|
| 33 |
@keyframes apl-spin { to { transform: rotate(360deg); } }
|
| 34 |
.apl-hint { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; font-size: 11px; color: #5a6278; }
|
| 35 |
</style>
|
| 36 |
+
<link rel="stylesheet" href="/static/style.css?v=20261015watch2">
|
| 37 |
<script src="/static/vendor/lightweight-charts-4.2.0.js"></script>
|
| 38 |
</head>
|
| 39 |
<body>
|
|
|
|
| 537 |
</a>
|
| 538 |
</nav>
|
| 539 |
|
| 540 |
+
<script src="/static/app.js?v=20261015watch2"></script>
|
| 541 |
</body>
|
| 542 |
</html>
|
tests/conftest.py
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import pytest
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
@pytest.fixture(autouse=True)
|
| 5 |
+
def isolate_public_contract_discovery(monkeypatch):
|
| 6 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "0")
|
tests/test_api_contract.py
CHANGED
|
@@ -374,7 +374,7 @@ class ApiContractTests(unittest.TestCase):
|
|
| 374 |
response = self.client.get("/api/search?q=NOTLISTED")
|
| 375 |
|
| 376 |
self.assertEqual(response.status_code, 200)
|
| 377 |
-
self.assertEqual(response.get_json()
|
| 378 |
mock_urlopen.assert_not_called()
|
| 379 |
|
| 380 |
@patch("app.get_universe", return_value={"RELIANCE.NS": "Reliance Industries Limited"})
|
|
@@ -417,6 +417,87 @@ class ApiContractTests(unittest.TestCase):
|
|
| 417 |
filtered = self.client.get("/api/search?q=reliance&instrument_type=OPTION").get_json()
|
| 418 |
self.assertEqual([row["instrument_type"] for row in filtered["results"]], ["OPTION"])
|
| 419 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 420 |
@patch("app.get_universe", side_effect=RuntimeError("Equity catalog unavailable"))
|
| 421 |
@patch("fo.get_service")
|
| 422 |
def test_search_keeps_indices_when_equity_catalog_is_unavailable(self, service, _universe):
|
|
|
|
| 374 |
response = self.client.get("/api/search?q=NOTLISTED")
|
| 375 |
|
| 376 |
self.assertEqual(response.status_code, 200)
|
| 377 |
+
self.assertEqual(response.get_json()["results"], [])
|
| 378 |
mock_urlopen.assert_not_called()
|
| 379 |
|
| 380 |
@patch("app.get_universe", return_value={"RELIANCE.NS": "Reliance Industries Limited"})
|
|
|
|
| 417 |
filtered = self.client.get("/api/search?q=reliance&instrument_type=OPTION").get_json()
|
| 418 |
self.assertEqual([row["instrument_type"] for row in filtered["results"]], ["OPTION"])
|
| 419 |
|
| 420 |
+
@patch("app.get_universe", return_value={})
|
| 421 |
+
@patch("fo.get_service")
|
| 422 |
+
def test_option_search_matches_natural_expiry_dates_and_exact_strikes(self, service, _universe):
|
| 423 |
+
contracts = [_app_module.fo.validate_contract({
|
| 424 |
+
"underlying": "NIFTY", "underlying_type": "INDEX", "expiry": expiry,
|
| 425 |
+
"strike": strike, "option_type": side, "lot_size": 25, "tick_size": 0.05,
|
| 426 |
+
}, "OPTION") for expiry, strike, side in (
|
| 427 |
+
("2099-10-19", 23000, "CE"), ("2099-10-20", 23000, "CE"),
|
| 428 |
+
("2099-10-19", 23000, "PE"), ("2099-10-19", 123000, "CE"),
|
| 429 |
+
)]
|
| 430 |
+
service.return_value.contracts.return_value = contracts
|
| 431 |
+
for query in ("nifty 19th OCT 23000 CE", "NIFTY 19 October 23000 call",
|
| 432 |
+
"NIFTY 19 OCT 2099 23000 CE", "NIFTY 2099-10-19 23000 CE"):
|
| 433 |
+
with self.subTest(query=query):
|
| 434 |
+
response = self.client.get("/api/search", query_string={"q": query, "instrument_type": "OPTION"})
|
| 435 |
+
self.assertEqual([row["ticker"] for row in response.get_json()["results"]],
|
| 436 |
+
[contracts[0]["contract_id"]])
|
| 437 |
+
response = self.client.get("/api/search?q=NIFTY%2018th%20OCT%2023000%20CE&instrument_type=OPTION")
|
| 438 |
+
self.assertEqual(response.get_json()["results"], [])
|
| 439 |
+
|
| 440 |
+
@patch("app.get_universe", return_value={})
|
| 441 |
+
@patch("fo.get_service")
|
| 442 |
+
def test_search_reports_public_catalog_loading_without_hiding_indices(self, service, _universe):
|
| 443 |
+
service.return_value.contracts.return_value = []
|
| 444 |
+
service.return_value.status.return_value = {"reference_state": "loading", "reference_source": "dhan_public", "reference_count": 0}
|
| 445 |
+
payload = self.client.get("/api/search?q=NIFTY").get_json()
|
| 446 |
+
self.assertTrue(any(row["ticker"] == "^NSEI" for row in payload["results"]))
|
| 447 |
+
self.assertEqual(payload["reference"]["state"], "loading")
|
| 448 |
+
|
| 449 |
+
@patch("app.get_universe")
|
| 450 |
+
@patch("app.db.add_to_watchlist")
|
| 451 |
+
@patch("fo.get_service")
|
| 452 |
+
def test_typed_option_add_resolves_one_exact_contract(self, service, add, universe):
|
| 453 |
+
raw = {**self._watchlist_reference(), "underlying": "NIFTY", "underlying_type": "INDEX",
|
| 454 |
+
"expiry": "2099-10-19", "strike": 23000}
|
| 455 |
+
valid = _app_module.fo.validate_contract(raw, "OPTION")
|
| 456 |
+
other = _app_module.fo.validate_contract({**raw, "underlying": "BANKNIFTY"}, "OPTION")
|
| 457 |
+
service.return_value.contracts.return_value = [valid, other]
|
| 458 |
+
service.return_value.resolve.return_value = valid
|
| 459 |
+
add.return_value = {"ticker": valid["contract_id"], "name": "NIFTY call"}
|
| 460 |
+
response = self.client.post("/api/watchlist", json={"ticker": "NIFTY 19th OCT 23000 CE"})
|
| 461 |
+
self.assertEqual(response.status_code, 201, response.get_json())
|
| 462 |
+
add.assert_called_once_with(valid["contract_id"], "NIFTY 2099-10-19 23000 CE")
|
| 463 |
+
universe.assert_not_called()
|
| 464 |
+
|
| 465 |
+
@patch("app.db.add_to_watchlist")
|
| 466 |
+
@patch("fo.get_service")
|
| 467 |
+
def test_typed_derivative_add_rejects_ambiguous_and_unlisted_contracts(self, service, add):
|
| 468 |
+
raw = self._watchlist_reference()
|
| 469 |
+
service.return_value.status.return_value = {"reference_state": "available"}
|
| 470 |
+
service.return_value.contracts.return_value = [raw, {**raw, "expiry": "2099-11-29"}]
|
| 471 |
+
for query in ("RELIANCE 2500 CE", "NIFTY 18th OCT 23000 CE"):
|
| 472 |
+
response = self.client.post("/api/watchlist", json={"ticker": query})
|
| 473 |
+
self.assertEqual(response.status_code, 400, response.get_json())
|
| 474 |
+
add.assert_not_called()
|
| 475 |
+
service.return_value.contracts.return_value = []
|
| 476 |
+
service.return_value.status.return_value = {"reference_state": "loading"}
|
| 477 |
+
response = self.client.post("/api/watchlist", json={"ticker": "NIFTY 19th OCT 23000 CE"})
|
| 478 |
+
self.assertEqual(response.status_code, 503)
|
| 479 |
+
self.assertTrue(response.get_json()["retryable"])
|
| 480 |
+
|
| 481 |
+
@patch("app.get_universe", return_value={"RELIANCE.NS": "Reliance"})
|
| 482 |
+
@patch("app.db.add_to_watchlist")
|
| 483 |
+
@patch("fo.get_service")
|
| 484 |
+
def test_all_derivative_underlying_and_instrument_types_can_be_searched_and_added(self, service, add, _universe):
|
| 485 |
+
for kind in ("OPTION", "FUTURE"):
|
| 486 |
+
for symbol, underlying_type in (("NIFTY", "INDEX"), ("RELIANCE", "STOCK")):
|
| 487 |
+
valid = _app_module.fo.validate_contract({**self._watchlist_reference(),
|
| 488 |
+
"underlying": symbol, "underlying_type": underlying_type}, kind)
|
| 489 |
+
service.return_value.contracts.return_value = [valid]
|
| 490 |
+
service.return_value.resolve.return_value = valid
|
| 491 |
+
add.return_value = {"ticker": valid["contract_id"], "name": "Contract"}
|
| 492 |
+
query = f"{symbol} 29th OCT " + ("2500 CE" if kind == "OPTION" else "futures")
|
| 493 |
+
with self.subTest(kind=kind, underlying_type=underlying_type):
|
| 494 |
+
result = self.client.get("/api/search", query_string={"q": query, "instrument_type": kind}).get_json()
|
| 495 |
+
self.assertEqual([row["ticker"] for row in result["results"]], [valid["contract_id"]])
|
| 496 |
+
response = self.client.post("/api/watchlist", json={"ticker": query, "instrument_type": kind})
|
| 497 |
+
self.assertEqual(response.status_code, 201, response.get_json())
|
| 498 |
+
response = self.client.post("/api/watchlist", json={"ticker": valid["contract_id"], "instrument_type": "OPTION"})
|
| 499 |
+
self.assertEqual(response.status_code, 400)
|
| 500 |
+
|
| 501 |
@patch("app.get_universe", side_effect=RuntimeError("Equity catalog unavailable"))
|
| 502 |
@patch("fo.get_service")
|
| 503 |
def test_search_keeps_indices_when_equity_catalog_is_unavailable(self, service, _universe):
|
tests/test_fo.py
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
import json
|
| 2 |
-
from datetime import date
|
| 3 |
from unittest.mock import Mock, patch, mock_open
|
| 4 |
|
| 5 |
import pytest
|
|
@@ -81,6 +81,136 @@ def test_reference_csv_is_provider_independent_and_filters_bad_rows():
|
|
| 81 |
assert rows[1]["instrument_type"] == "FUTURE"
|
| 82 |
|
| 83 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
def test_quote_freshness_rejects_invalid_old_future_and_crossed_quotes():
|
| 85 |
raw = {"price": 100, "bid": 99, "ask": 101, "observed_at": 1000,
|
| 86 |
"trade_time": 999, "source": "fake"}
|
|
|
|
| 1 |
import json
|
| 2 |
+
from datetime import date, datetime
|
| 3 |
from unittest.mock import Mock, patch, mock_open
|
| 4 |
|
| 5 |
import pytest
|
|
|
|
| 81 |
assert rows[1]["instrument_type"] == "FUTURE"
|
| 82 |
|
| 83 |
|
| 84 |
+
@pytest.fixture
|
| 85 |
+
def public_master():
|
| 86 |
+
return ("EXCH_ID,SEGMENT,SECURITY_ID,INSTRUMENT,UNDERLYING_SYMBOL,SM_EXPIRY_DATE,"
|
| 87 |
+
"STRIKE_PRICE,OPTION_TYPE,LOT_SIZE,TICK_SIZE\n"
|
| 88 |
+
"NSE,D,47161,OPTIDX,NIFTY,2099-10-19,23000,CE,65.0,5.0000\n"
|
| 89 |
+
"NSE,D,47162,OPTIDX,NIFTY,2099-10-19,23000,PE,65.0,5.0000\n"
|
| 90 |
+
"NSE,D,47163,FUTIDX,NIFTY,2099-10-27,,XX,65.0,10.0000\n"
|
| 91 |
+
"NSE,D,47164,OPTSTK,RELIANCE,2099-10-27,2500,CE,250.0,5.0000\n"
|
| 92 |
+
"NSE,D,47165,FUTSTK,RELIANCE,2099-10-27,,XX,250.0,10.0000\n"
|
| 93 |
+
"BSE,D,47166,OPTIDX,SENSEX,2099-10-19,80000,CE,20.0,5.0000\n"
|
| 94 |
+
"NSE,E,2885,EQUITY,RELIANCE,,,,1.0,5.0000\n"
|
| 95 |
+
"NSE,D,47167,OPTIDX,NIFTY,2020-10-19,23000,CE,65.0,5.0000\n")
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
def test_public_master_validates_all_nse_contract_types(public_master):
|
| 99 |
+
rows = fo.parse_public_reference_csv(public_master.splitlines())
|
| 100 |
+
assert len(rows) == 5
|
| 101 |
+
assert {row["instrument_type"] for row in rows} == {"OPTION", "FUTURE"}
|
| 102 |
+
assert {row["underlying_type"] for row in rows} == {"INDEX", "STOCK"}
|
| 103 |
+
assert rows[0]["lot_size"] == 65
|
| 104 |
+
assert rows[0]["tick_size"] == 0.05
|
| 105 |
+
assert rows[2]["tick_size"] == 0.1
|
| 106 |
+
assert rows[0]["provider_ids"] == {"dhan": {"security_id": "47161"}}
|
| 107 |
+
assert fo.parse_public_reference_csv(public_master.replace("65.0", "65.5").splitlines()) == rows[3:]
|
| 108 |
+
|
| 109 |
+
|
| 110 |
+
def test_public_discovery_starts_once_without_waiting_for_network(monkeypatch):
|
| 111 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
|
| 112 |
+
monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
|
| 113 |
+
service = fo.FoService()
|
| 114 |
+
with patch("database.get_app_result_cache", return_value=None), patch("fo.threading.Thread") as worker:
|
| 115 |
+
assert service.contracts() == []
|
| 116 |
+
assert service.status()["reference_state"] == "loading"
|
| 117 |
+
assert service.status()["reference_source"] == "dhan_public"
|
| 118 |
+
worker.return_value.start.assert_called_once()
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def test_public_discovery_restores_fresh_sqlite_cache_without_network(monkeypatch):
|
| 122 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
|
| 123 |
+
monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
|
| 124 |
+
valid = {**fo.validate_contract(contract(), "OPTION"), "provider_ids": {}}
|
| 125 |
+
cached = {"result": {"contracts": [valid]}, "saved_at": 1000}
|
| 126 |
+
with patch("database.get_app_result_cache", return_value=cached), patch("fo.time.time", return_value=1001), patch("fo.threading.Thread") as worker:
|
| 127 |
+
service = fo.FoService()
|
| 128 |
+
assert service.resolve(valid["contract_id"]) == valid
|
| 129 |
+
assert service.status()["reference_state"] == "available"
|
| 130 |
+
worker.assert_not_called()
|
| 131 |
+
|
| 132 |
+
|
| 133 |
+
def test_public_discovery_fetches_and_persists_verified_rows(monkeypatch, public_master):
|
| 134 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
|
| 135 |
+
response = Mock()
|
| 136 |
+
response.__enter__ = Mock(return_value=response)
|
| 137 |
+
response.__exit__ = Mock(return_value=False)
|
| 138 |
+
response.headers = {"Content-Length": "1000"}
|
| 139 |
+
response.iter_lines.return_value = public_master.splitlines()
|
| 140 |
+
with patch("fo.requests.get", return_value=response) as fetch, patch("database.save_app_result_cache") as save:
|
| 141 |
+
service = fo.FoService()
|
| 142 |
+
service._refresh_public_reference()
|
| 143 |
+
assert len(service._catalog) == 5
|
| 144 |
+
assert service._reference_state == "available"
|
| 145 |
+
assert service._reference_loading is False
|
| 146 |
+
assert save.call_args.args[1]["contracts"] == service._catalog
|
| 147 |
+
assert fetch.call_args.args[0] == fo.PUBLIC_REFERENCE_URL
|
| 148 |
+
assert "headers" not in fetch.call_args.kwargs
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
def test_public_discovery_outage_is_bounded_and_does_not_fabricate_contracts(monkeypatch):
|
| 152 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
|
| 153 |
+
monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
|
| 154 |
+
with patch("fo.requests.get", side_effect=OSError), patch("fo.time.time", return_value=1000), patch("database.get_app_result_cache", return_value=None), patch("fo.threading.Thread") as worker:
|
| 155 |
+
service = fo.FoService()
|
| 156 |
+
service._refresh_public_reference()
|
| 157 |
+
assert service.contracts() == []
|
| 158 |
+
assert service.status()["reference_state"] == "unavailable"
|
| 159 |
+
worker.assert_not_called()
|
| 160 |
+
|
| 161 |
+
|
| 162 |
+
@pytest.mark.parametrize("saved_at,rows", [(1000, [{"expiry": "2099-10-19"}]), (2000, []), (-200000, [])])
|
| 163 |
+
def test_public_discovery_discards_corrupt_future_and_expired_cache(monkeypatch, saved_at, rows):
|
| 164 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
|
| 165 |
+
monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
|
| 166 |
+
cached = {"result": {"contracts": rows}, "saved_at": saved_at}
|
| 167 |
+
with patch("database.get_app_result_cache", return_value=cached), patch("fo.time.time", return_value=1001), patch("fo.threading.Thread") as worker:
|
| 168 |
+
service = fo.FoService()
|
| 169 |
+
assert service.contracts() == []
|
| 170 |
+
assert service.status()["reference_state"] == "loading"
|
| 171 |
+
worker.return_value.start.assert_called_once()
|
| 172 |
+
|
| 173 |
+
|
| 174 |
+
def test_public_discovery_uses_only_bounded_stale_cache_during_refresh(monkeypatch):
|
| 175 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
|
| 176 |
+
monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
|
| 177 |
+
valid = {**fo.validate_contract(contract(), "OPTION"), "provider_ids": {}}
|
| 178 |
+
cached = {"result": {"contracts": [valid]}, "saved_at": 1000}
|
| 179 |
+
with patch("database.get_app_result_cache", return_value=cached), patch("fo.time.time", return_value=90000), patch("fo.threading.Thread") as worker:
|
| 180 |
+
service = fo.FoService()
|
| 181 |
+
assert service.contracts() == [valid]
|
| 182 |
+
assert service.status()["reference_state"] == "stale"
|
| 183 |
+
worker.return_value.start.assert_called_once()
|
| 184 |
+
with patch("fo.time.time", return_value=200000):
|
| 185 |
+
assert service.contracts() == []
|
| 186 |
+
|
| 187 |
+
|
| 188 |
+
def test_public_discovery_rejects_oversized_and_empty_responses(monkeypatch):
|
| 189 |
+
monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
|
| 190 |
+
response = Mock()
|
| 191 |
+
response.__enter__ = Mock(return_value=response)
|
| 192 |
+
response.__exit__ = Mock(return_value=False)
|
| 193 |
+
response.iter_lines.return_value = []
|
| 194 |
+
with patch("fo.requests.get", return_value=response), patch("database.save_app_result_cache") as save:
|
| 195 |
+
for size in ("65000000", "0"):
|
| 196 |
+
response.headers = {"Content-Length": size}
|
| 197 |
+
service = fo.FoService()
|
| 198 |
+
service._refresh_public_reference()
|
| 199 |
+
assert service._reference_state == "unavailable"
|
| 200 |
+
assert service._reference_loading is False
|
| 201 |
+
save.assert_not_called()
|
| 202 |
+
|
| 203 |
+
|
| 204 |
+
def test_contract_catalog_removes_expiry_day_contracts_at_market_close():
|
| 205 |
+
raw = fo.validate_contract(contract(expiry="2026-10-19"), "OPTION", today=date(2026, 10, 15))
|
| 206 |
+
service = fo.FoService()
|
| 207 |
+
with patch("fo.datetime") as clock:
|
| 208 |
+
clock.now.return_value = datetime(2026, 10, 19, 15, 29, tzinfo=fo.IST)
|
| 209 |
+
assert service._filter_contracts([raw], "", "", "") == [raw]
|
| 210 |
+
clock.now.return_value = datetime(2026, 10, 19, 15, 30, tzinfo=fo.IST)
|
| 211 |
+
assert service._filter_contracts([raw], "", "", "") == []
|
| 212 |
+
|
| 213 |
+
|
| 214 |
def test_quote_freshness_rejects_invalid_old_future_and_crossed_quotes():
|
| 215 |
raw = {"price": 100, "bid": 99, "ask": 101, "observed_at": 1000,
|
| 216 |
"trade_time": 999, "source": "fake"}
|
tests/test_watchlist_frontend.cjs
CHANGED
|
@@ -377,6 +377,31 @@ test('watchlist search sends the selected instrument filter', async () => {
|
|
| 377 |
assert.match(requestedUrl, /instrument_type=OPTION/);
|
| 378 |
});
|
| 379 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 380 |
test('derivative watchlist cards show contract quotes without equity controls', () => {
|
| 381 |
const context = vm.createContext({ num: value => value.toFixed(2) });
|
| 382 |
vm.runInContext(section('function _foEscape(', 'function _foCancelRequests('), context);
|
|
|
|
| 377 |
assert.match(requestedUrl, /instrument_type=OPTION/);
|
| 378 |
});
|
| 379 |
|
| 380 |
+
test('watchlist search reports catalog loading and retries the current query', async () => {
|
| 381 |
+
let calls = 0;
|
| 382 |
+
const fixture = searchContext([], async () => ({ ok: true, json: async () => {
|
| 383 |
+
calls += 1;
|
| 384 |
+
return { results: [], reference: { state: calls === 1 ? 'loading' : 'available' } };
|
| 385 |
+
} }));
|
| 386 |
+
fixture.input.value = 'NIFTY 19th OCT 23000 CE';
|
| 387 |
+
fixture.handlers.input();
|
| 388 |
+
await fixture.runTimer();
|
| 389 |
+
assert.match(fixture.suggestions.textContent, /Loading.*contract/);
|
| 390 |
+
fixture.runTimer();
|
| 391 |
+
await fixture.runTimer();
|
| 392 |
+
assert.equal(calls, 2);
|
| 393 |
+
assert.match(fixture.suggestions.textContent, /No matching listed/);
|
| 394 |
+
});
|
| 395 |
+
|
| 396 |
+
test('watchlist search explains unavailable contract catalogs', async () => {
|
| 397 |
+
const fixture = searchContext([], async () => ({ ok: true, json: async () => ({
|
| 398 |
+
results: [], reference: { state: 'unavailable' },
|
| 399 |
+
}) }));
|
| 400 |
+
fixture.handlers.input();
|
| 401 |
+
await fixture.runTimer();
|
| 402 |
+
assert.match(fixture.suggestions.textContent, /catalog.*unavailable/i);
|
| 403 |
+
});
|
| 404 |
+
|
| 405 |
test('derivative watchlist cards show contract quotes without equity controls', () => {
|
| 406 |
const context = vm.createContext({ num: value => value.toFixed(2) });
|
| 407 |
vm.runInContext(section('function _foEscape(', 'function _foCancelRequests('), context);
|