Khanna, Videh Rakesh Rakesh commited on
Commit
a1620df
·
1 Parent(s): 25ec864

Fix F&O search with automatic Dhan contract discovery

Browse files
README.md CHANGED
@@ -19,8 +19,21 @@ filter. Search index names or aliases such as `Nifty 50` and `BANKNIFTY`, or con
19
  details such as `RELIANCE futures` and `RELIANCE 2500 CE`. Contract results show expiry,
20
  strike and Call/Put; selecting one retains its exact identity when adding or removing it.
21
 
22
- Futures/options search requires a current trusted `FO_REFERENCE_CSV` catalog (configuration
23
- below). No contracts or lot sizes are fabricated when the catalog is unavailable. Derivative
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  watchlist cards use contract quotes only, explicitly label stale/unavailable prices, and open
25
  the F&O popup with the selected contract. They do not run equity AI/ML forecasts, use stock
26
  prices, or enter equity cache-warming and prediction-refresh jobs. Other stock-prediction
@@ -51,7 +64,8 @@ their original semantics; the legacy `shares` column stores derivative quantity
51
  | Variable | Default | Purpose |
52
  | --- | --- | --- |
53
  | `FO_PROVIDER` | `manual` | Optional registered live adapter; built-in choices are `manual` and `dhan` |
54
- | `FO_REFERENCE_CSV` | Unset | Server-managed, validated neutral contract CSV; use `/data/fo_contracts.csv` on HF Spaces |
 
55
  | `FO_AUTO_MONITOR` | `1` | With a live provider, check up to five derivative positions per 15-second pass; `0` disables background checks |
56
 
57
  The reference file must be maintained from a trusted current exchange/provider source.
@@ -69,6 +83,8 @@ properly CSV-escaped JSON mappings. For the optional Dhan adapter, its `dhan` ob
69
  `security_id`; option chains also need `underlying_security_id` and `underlying_segment`
70
  (`IDX_I` for indices, `NSE_EQ` for stocks). These are public identifiers, never credentials.
71
  Other adapters implement `fo.FoProvider` and register a factory with `fo.register_provider`.
 
 
72
 
73
  Live Dhan use additionally requires its existing server configuration and browser-data
74
  permission. It remains optional and read-only. Quote and chain controls show their capability
 
19
  details such as `RELIANCE futures` and `RELIANCE 2500 CE`. Contract results show expiry,
20
  strike and Call/Put; selecting one retains its exact identity when adding or removing it.
21
 
22
+ Queries such as `NIFTY 19th OCT 23000 CE`, `NIFTY 19 October 2026 23000 call` and ISO
23
+ expiry dates are supported. A fully specified typed query can be added directly when it
24
+ matches exactly one listed contract; ambiguous expiries require selecting a search result.
25
+ Strike, option side and complete underlying symbols are matched exactly.
26
+
27
+ Futures/options discovery defaults to Dhan's public detailed instrument master, without
28
+ credentials or a data subscription. Yahoo-backed stock/index data paths are unchanged;
29
+ Yahoo stock prices are never substituted for derivative contracts. The catalog loads in
30
+ the background, and search displays its state and retries while loading. Verified references
31
+ are cached for 24 hours in `paper_trading.db` (`app_result_cache`) on HF persistent storage;
32
+ at most 48 hours of verified cache is retained during an outage, with a stale label. Expired
33
+ contracts, including expiry-day contracts after 15:30 IST, are excluded. A configured
34
+ `FO_REFERENCE_CSV` overrides automatic discovery (configuration below).
35
+
36
+ No contracts or lot sizes are fabricated when the catalog is unavailable. Derivative
37
  watchlist cards use contract quotes only, explicitly label stale/unavailable prices, and open
38
  the F&O popup with the selected contract. They do not run equity AI/ML forecasts, use stock
39
  prices, or enter equity cache-warming and prediction-refresh jobs. Other stock-prediction
 
64
  | Variable | Default | Purpose |
65
  | --- | --- | --- |
66
  | `FO_PROVIDER` | `manual` | Optional registered live adapter; built-in choices are `manual` and `dhan` |
67
+ | `FO_AUTO_REFERENCE` | `1` | Discover NSE stock/index futures and options from the public Dhan master; `0` disables discovery |
68
+ | `FO_REFERENCE_CSV` | Unset | Override discovery with a server-managed, validated neutral CSV; use `/data/fo_contracts.csv` on HF Spaces |
69
  | `FO_AUTO_MONITOR` | `1` | With a live provider, check up to five derivative positions per 15-second pass; `0` disables background checks |
70
 
71
  The reference file must be maintained from a trusted current exchange/provider source.
 
83
  `security_id`; option chains also need `underlying_security_id` and `underlying_segment`
84
  (`IDX_I` for indices, `NSE_EQ` for stocks). These are public identifiers, never credentials.
85
  Other adapters implement `fo.FoProvider` and register a factory with `fo.register_provider`.
86
+ Automatic public references include Dhan contract quote IDs. Underlying mappings needed
87
+ for the optional live option-chain feature still require a configured reference override.
88
 
89
  Live Dhan use additionally requires its existing server configuration and browser-data
90
  permission. It remains optional and read-only. Quote and chain controls show their capability
app.py CHANGED
@@ -1233,6 +1233,7 @@ def search_ticker():
1233
  if not q:
1234
  return jsonify({"results": []})
1235
 
 
1236
  candidates = []
1237
  for ticker, (name, aliases) in _WATCHLIST_INDICES.items():
1238
  candidates.append(({"ticker": ticker, "name": name, "instrument_type": "INDEX"}, aliases))
@@ -1247,24 +1248,33 @@ def search_ticker():
1247
  except Exception:
1248
  pass
1249
 
 
1250
  if kind in {"", "OPTION", "FUTURE"}:
1251
- for contract in fo.get_service().contracts():
 
 
1252
  candidates.append(({"ticker": contract["contract_id"], "name": fo.contract_label(contract),
1253
  "instrument_type": contract["instrument_type"], "contract": contract}, "FUT"))
 
 
 
 
1254
 
1255
- terms = [{"FUT": "FUTURE", "FUTURES": "FUTURE", "OPTIONS": "OPTION"}.get(term, term)
1256
- for term in q.split()]
1257
  matches = []
1258
  for row, aliases in candidates:
1259
  if kind and row["instrument_type"] != kind:
1260
  continue
1261
  bare = _bare(row["ticker"])
1262
  haystack = f"{bare} {row['name']} {row['instrument_type']} {aliases}".upper()
1263
- if all(term in haystack for term in terms):
 
1264
  rank = 0 if q == bare or q in aliases.split() else 1 if bare.startswith(q) else 2
1265
  matches.append((rank, row))
1266
  matches.sort(key=lambda match: match[0])
1267
- return jsonify({"results": [row for _, row in matches[:20]]})
 
 
 
1268
 
1269
 
1270
  # ── MARKET STATUS ─────────────────────────────────────────────────────────────
@@ -2206,7 +2216,28 @@ def watchlist_add():
2206
  name = data.get("name", _bare(ticker))
2207
  if not ticker:
2208
  return jsonify({"error": "ticker required"}), 400
 
 
 
2209
  t = _normalise(ticker)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2210
  if t.startswith("NSE:") and _watchlist_instrument_type(t) not in {"OPTION", "FUTURE"}:
2211
  return jsonify({"error": "Invalid derivative contract identifier"}), 400
2212
  if _watchlist_instrument_type(t) in {"OPTION", "FUTURE"}:
@@ -2220,12 +2251,13 @@ def watchlist_add():
2220
  return jsonify({"error": "Unknown index symbol"}), 400
2221
  # Membership in the cached official NSE equity list is sufficient to add. Requiring
2222
  # a live price and 200 bars here makes adding fail whenever a market-data provider is slow.
2223
- try:
2224
- universe = get_universe()
2225
- if universe and t.endswith(".NS") and t not in universe:
2226
- return jsonify({"error": f"Ticker {t} is not in the current NSE equity list."}), 400
2227
- except Exception:
2228
- pass # A universe refresh problem must not prevent adding a valid ticker.
 
2229
  if not name or name == _bare(ticker):
2230
  name = _WATCHLIST_INDICES[t][0] if t in _WATCHLIST_INDICES else TICKER_NAMES.get(t, _bare(t))
2231
  try:
 
1233
  if not q:
1234
  return jsonify({"results": []})
1235
 
1236
+ terms = fo.search_terms(q)
1237
  candidates = []
1238
  for ticker, (name, aliases) in _WATCHLIST_INDICES.items():
1239
  candidates.append(({"ticker": ticker, "name": name, "instrument_type": "INDEX"}, aliases))
 
1248
  except Exception:
1249
  pass
1250
 
1251
+ reference = None
1252
  if kind in {"", "OPTION", "FUTURE"}:
1253
+ service = fo.get_service()
1254
+ contracts = service.contracts(instrument_type=kind)
1255
+ for contract in fo.matching_contracts(contracts, terms):
1256
  candidates.append(({"ticker": contract["contract_id"], "name": fo.contract_label(contract),
1257
  "instrument_type": contract["instrument_type"], "contract": contract}, "FUT"))
1258
+ status = service.status()
1259
+ if isinstance(status, dict):
1260
+ reference = {"state": status.get("reference_state"), "source": status.get("reference_source"),
1261
+ "count": status.get("reference_count", 0)}
1262
 
 
 
1263
  matches = []
1264
  for row, aliases in candidates:
1265
  if kind and row["instrument_type"] != kind:
1266
  continue
1267
  bare = _bare(row["ticker"])
1268
  haystack = f"{bare} {row['name']} {row['instrument_type']} {aliases}".upper()
1269
+ matched = "contract" in row or all(term in haystack for term in terms)
1270
+ if matched:
1271
  rank = 0 if q == bare or q in aliases.split() else 1 if bare.startswith(q) else 2
1272
  matches.append((rank, row))
1273
  matches.sort(key=lambda match: match[0])
1274
+ payload = {"results": [row for _, row in matches[:20]]}
1275
+ if reference is not None:
1276
+ payload["reference"] = reference
1277
+ return jsonify(payload)
1278
 
1279
 
1280
  # ── MARKET STATUS ─────────────────────────────────────────────────────────────
 
2216
  name = data.get("name", _bare(ticker))
2217
  if not ticker:
2218
  return jsonify({"error": "ticker required"}), 400
2219
+ kind = data.get("instrument_type", "").upper()
2220
+ if kind not in {"", "EQUITY", "INDEX", "OPTION", "FUTURE"}:
2221
+ return jsonify({"error": "Invalid instrument_type"}), 400
2222
  t = _normalise(ticker)
2223
+ terms = fo.search_terms(ticker)
2224
+ if not ticker.startswith(("NSE:", "^")) and (kind in {"OPTION", "FUTURE"} or any(term in {"CE", "PE", "OPTION", "FUTURE"} for term in terms)):
2225
+ service = fo.get_service()
2226
+ matches = fo.matching_contracts(service.contracts(instrument_type=kind if kind in {"OPTION", "FUTURE"} else ""), terms)
2227
+ if not matches:
2228
+ state = service.status().get("reference_state")
2229
+ if state in {"loading", "unavailable", "not_configured"}:
2230
+ message = "Contract catalog is loading. Retry shortly." if state == "loading" else "Contract catalog is unavailable. Retry after it refreshes."
2231
+ return jsonify({"error": message, "retryable": state == "loading"}), 503
2232
+ return jsonify({"error": "No listed contract matches that underlying, expiry, strike and option type."}), 400
2233
+ if len(matches) != 1:
2234
+ return jsonify({"error": "Multiple contracts match. Select the exact expiry and contract from search."}), 400
2235
+ t = matches[0]["contract_id"]
2236
+ elif kind in {"", "INDEX"}:
2237
+ t = next((symbol for symbol, (label, aliases) in _WATCHLIST_INDICES.items()
2238
+ if ticker == label.upper() or ticker in aliases.split()), t)
2239
+ if kind and kind != _watchlist_instrument_type(t):
2240
+ return jsonify({"error": "Selected instrument type does not match the ticker or contract."}), 400
2241
  if t.startswith("NSE:") and _watchlist_instrument_type(t) not in {"OPTION", "FUTURE"}:
2242
  return jsonify({"error": "Invalid derivative contract identifier"}), 400
2243
  if _watchlist_instrument_type(t) in {"OPTION", "FUTURE"}:
 
2251
  return jsonify({"error": "Unknown index symbol"}), 400
2252
  # Membership in the cached official NSE equity list is sufficient to add. Requiring
2253
  # a live price and 200 bars here makes adding fail whenever a market-data provider is slow.
2254
+ if _watchlist_instrument_type(t) == "EQUITY":
2255
+ try:
2256
+ universe = get_universe()
2257
+ if universe and t.endswith(".NS") and t not in universe:
2258
+ return jsonify({"error": f"Ticker {t} is not in the current NSE equity list."}), 400
2259
+ except Exception:
2260
+ pass # A universe refresh problem must not prevent adding a valid ticker.
2261
  if not name or name == _bare(ticker):
2262
  name = _WATCHLIST_INDICES[t][0] if t in _WATCHLIST_INDICES else TICKER_NAMES.get(t, _bare(t))
2263
  try:
fo.py CHANGED
@@ -8,12 +8,19 @@ import json
8
  import math
9
  import os
10
  import re
 
11
  import threading
12
  import time
13
  from typing import Protocol
14
 
 
 
15
 
16
  IST = timezone(timedelta(hours=5, minutes=30))
 
 
 
 
17
 
18
 
19
  def number(value, name, *, allow_zero=False):
@@ -84,6 +91,53 @@ def contract_label(contract):
84
  return label + (f" {contract['strike']:g} {contract['option_type']}" if contract["option_type"] else " FUT")
85
 
86
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
  def validate_price(trade, value):
88
  contract = contract_for_trade(trade)
89
  price = number(value, "contract price", allow_zero=trade["instrument_type"] == "OPTION")
@@ -140,12 +194,19 @@ class FoProvider(Protocol):
140
 
141
 
142
  def parse_reference_csv(text):
 
 
 
 
 
 
143
  contracts, ambiguous = {}, set()
144
- for raw in csv.DictReader(io.StringIO(text)):
145
- row = {(key or "").strip(): (value or "").strip() for key, value in raw.items()}
146
  try:
147
  contract = validate_contract(row, row.get("instrument_type"))
148
- mappings = json.loads(row.get("provider_ids") or "{}")
 
 
149
  if not isinstance(mappings, dict):
150
  continue
151
  allowed = {"security_id", "underlying_security_id", "underlying_segment", "symbol"}
@@ -161,6 +222,30 @@ def parse_reference_csv(text):
161
  return [contract for identity, contract in contracts.items() if identity not in ambiguous]
162
 
163
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
164
  def normalize_quote(raw, *, now=None):
165
  now = time.time() if now is None else now
166
  result = {"source": str(raw.get("source") or "unknown")[:40]}
@@ -194,11 +279,91 @@ class FoService:
194
  self._catalog = []
195
  self._catalog_key = None
196
  self._reference_state = "not_configured"
 
 
 
 
 
 
197
  self._quotes = {}
198
  self._chains = {}
199
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
  def contracts(self, underlying="", instrument_type="", expiry=""):
201
  path = os.getenv("FO_REFERENCE_CSV", "")
 
 
 
 
202
  try:
203
  info = os.stat(path) if path else None
204
  key = (path, info.st_mtime_ns, info.st_size) if info else None
@@ -214,9 +379,13 @@ class FoService:
214
  except (OSError, ValueError):
215
  self._reference_state = "unavailable"
216
  rows = []
 
 
 
217
  symbol = underlying.upper().removesuffix(".NS").strip()
218
- today = datetime.now(IST).date().isoformat()
219
- return [dict(row) for row in rows if row["expiry"] >= today
 
220
  and (not symbol or row["underlying"] == symbol)
221
  and (not instrument_type or row["instrument_type"] == instrument_type)
222
  and (not expiry or row["expiry"] == expiry)]
@@ -226,6 +395,7 @@ class FoService:
226
  caps = self.provider.capabilities() if self.provider else {}
227
  return {"provider": self.name, "manual_available": True,
228
  "reference_state": self._reference_state, "reference_count": len(rows),
 
229
  "capabilities": {key: caps.get(key) is True for key in ("quotes", "option_chain", "margin")}}
230
 
231
  def resolve(self, contract_id):
 
8
  import math
9
  import os
10
  import re
11
+ import sqlite3
12
  import threading
13
  import time
14
  from typing import Protocol
15
 
16
+ import requests
17
+
18
 
19
  IST = timezone(timedelta(hours=5, minutes=30))
20
+ PUBLIC_REFERENCE_URL = "https://images.dhan.co/api-data/api-scrip-master-detailed.csv"
21
+ _PUBLIC_REFERENCE_KEY = "fo_public_reference_v1"
22
+ _REFERENCE_TTL = 86400
23
+ _REFERENCE_MAX_AGE = 2 * _REFERENCE_TTL
24
 
25
 
26
  def number(value, name, *, allow_zero=False):
 
91
  return label + (f" {contract['strike']:g} {contract['option_type']}" if contract["option_type"] else " FUT")
92
 
93
 
94
+ def search_terms(query):
95
+ def expiry_term(match):
96
+ day, month, year = match.groups()
97
+ for pattern in ("%d %b %Y", "%d %B %Y"):
98
+ try:
99
+ expiry = datetime.strptime(f"{day} {month} {year or '2000'}", pattern)
100
+ return expiry.strftime("%Y-%m-%d" if year else "-%m-%d")
101
+ except ValueError:
102
+ continue
103
+ return match.group()
104
+
105
+ query = re.sub(r"\b(\d{1,2})(?:ST|ND|RD|TH)?\s+([A-Z]{3,9})(?:\s+(\d{4})\b(?!\s+(?:CE|PE|CALLS?|PUTS?)\b))?",
106
+ expiry_term, query.upper())
107
+ aliases = {"FUT": "FUTURE", "FUTURES": "FUTURE", "OPTIONS": "OPTION",
108
+ "CALL": "CE", "CALLS": "CE", "PUT": "PE", "PUTS": "PE"}
109
+ return [aliases.get(term, term) for term in query.split()]
110
+
111
+
112
+ def contract_matches(contract, terms):
113
+ for term in terms:
114
+ if re.fullmatch(r"\d+(?:\.\d+)?", term):
115
+ if contract["strike"] is None or Decimal(term) != Decimal(str(contract["strike"])):
116
+ return False
117
+ elif re.fullmatch(r"(?:\d{4})?-\d{2}-\d{2}", term):
118
+ if not contract["expiry"].endswith(term):
119
+ return False
120
+ elif term in {"CE", "PE"}:
121
+ if contract["option_type"] != term:
122
+ return False
123
+ elif term in {"OPTION", "FUTURE"}:
124
+ if contract["instrument_type"] != term:
125
+ return False
126
+ elif term not in contract["underlying"] and term != contract["contract_id"]:
127
+ return False
128
+ return True
129
+
130
+
131
+ def matching_contracts(contracts, terms):
132
+ symbols = {contract["underlying"] for contract in contracts}
133
+ exact_symbol = next((term for term in terms if term in symbols), None)
134
+ matches = [contract for contract in contracts
135
+ if (not exact_symbol or contract["underlying"] == exact_symbol)
136
+ and contract_matches(contract, terms)]
137
+ return sorted(matches, key=lambda contract: (contract["expiry"], contract["underlying"],
138
+ contract["strike"] or 0, contract["option_type"] or ""))
139
+
140
+
141
  def validate_price(trade, value):
142
  contract = contract_for_trade(trade)
143
  price = number(value, "contract price", allow_zero=trade["instrument_type"] == "OPTION")
 
194
 
195
 
196
  def parse_reference_csv(text):
197
+ rows = ({(key or "").strip(): (value or "").strip() for key, value in raw.items()}
198
+ for raw in csv.DictReader(io.StringIO(text)))
199
+ return _validated_references(rows)
200
+
201
+
202
+ def _validated_references(rows):
203
  contracts, ambiguous = {}, set()
204
+ for row in rows:
 
205
  try:
206
  contract = validate_contract(row, row.get("instrument_type"))
207
+ mappings = row.get("provider_ids") or {}
208
+ if isinstance(mappings, str):
209
+ mappings = json.loads(mappings)
210
  if not isinstance(mappings, dict):
211
  continue
212
  allowed = {"security_id", "underlying_security_id", "underlying_segment", "symbol"}
 
222
  return [contract for identity, contract in contracts.items() if identity not in ambiguous]
223
 
224
 
225
+ def parse_public_reference_csv(lines):
226
+ def references():
227
+ kinds = {"OPTIDX": ("OPTION", "INDEX"), "OPTSTK": ("OPTION", "STOCK"),
228
+ "FUTIDX": ("FUTURE", "INDEX"), "FUTSTK": ("FUTURE", "STOCK")}
229
+ for row in csv.DictReader(lines):
230
+ kind = kinds.get(row.get("INSTRUMENT"))
231
+ security_id = row.get("SECURITY_ID", "")
232
+ if row.get("EXCH_ID") != "NSE" or row.get("SEGMENT") != "D" or not kind or not security_id.isdigit():
233
+ continue
234
+ try:
235
+ lot = number(row.get("LOT_SIZE"), "lot_size")
236
+ if lot != lot.to_integral_value():
237
+ continue
238
+ expiry = datetime.fromisoformat(row.get("SM_EXPIRY_DATE", "")).date().isoformat()
239
+ yield {"underlying": row.get("UNDERLYING_SYMBOL"), "underlying_type": kind[1],
240
+ "instrument_type": kind[0], "expiry": expiry, "lot_size": int(lot),
241
+ "tick_size": number(row.get("TICK_SIZE"), "tick_size") / 100,
242
+ "strike": row.get("STRIKE_PRICE"), "option_type": row.get("OPTION_TYPE"),
243
+ "provider_ids": {"dhan": {"security_id": security_id}}}
244
+ except (ValueError, TypeError):
245
+ continue
246
+ return _validated_references(references())
247
+
248
+
249
  def normalize_quote(raw, *, now=None):
250
  now = time.time() if now is None else now
251
  result = {"source": str(raw.get("source") or "unknown")[:40]}
 
279
  self._catalog = []
280
  self._catalog_key = None
281
  self._reference_state = "not_configured"
282
+ self._reference_source = "csv"
283
+ self._reference_lock = threading.Lock()
284
+ self._reference_loading = False
285
+ self._reference_loaded = False
286
+ self._reference_at = 0.0
287
+ self._reference_retry_at = 0.0
288
  self._quotes = {}
289
  self._chains = {}
290
 
291
+ def _refresh_public_reference(self):
292
+ try:
293
+ deadline = time.monotonic() + 60
294
+ with requests.get(PUBLIC_REFERENCE_URL, stream=True, timeout=(5, 10)) as response:
295
+ response.raise_for_status()
296
+ response.encoding = "utf-8-sig"
297
+ if int(response.headers.get("Content-Length", "0")) > 64_000_000:
298
+ raise ValueError("Reference response is too large")
299
+
300
+ def bounded_lines():
301
+ size = 0
302
+ for line in response.iter_lines(decode_unicode=True):
303
+ size += len(line.encode("utf-8")) + 1
304
+ if size > 64_000_000 or time.monotonic() > deadline:
305
+ raise ValueError("Reference download exceeded its budget")
306
+ yield line
307
+
308
+ rows = parse_public_reference_csv(bounded_lines())
309
+ if not rows:
310
+ raise ValueError("Public reference contains no valid NSE contracts")
311
+ from database import save_app_result_cache
312
+ saved_at = time.time()
313
+ try:
314
+ save_app_result_cache(_PUBLIC_REFERENCE_KEY, {"contracts": rows}, saved_at)
315
+ except (sqlite3.Error, OSError):
316
+ pass
317
+ with self._reference_lock:
318
+ self._catalog = rows
319
+ self._reference_at = saved_at
320
+ self._reference_state = "available"
321
+ except (requests.RequestException, OSError, ValueError, TypeError):
322
+ with self._reference_lock:
323
+ self._reference_state = "unavailable"
324
+ self._reference_retry_at = time.time() + 60
325
+ finally:
326
+ with self._reference_lock:
327
+ self._reference_loading = False
328
+
329
+ def _public_contracts(self):
330
+ self._reference_source = "dhan_public"
331
+ now = time.time()
332
+ start_refresh = False
333
+ with self._reference_lock:
334
+ if not self._reference_loaded:
335
+ from database import get_app_result_cache
336
+ try:
337
+ cached = get_app_result_cache(_PUBLIC_REFERENCE_KEY)
338
+ if cached and 0 <= now - cached["saved_at"] <= _REFERENCE_MAX_AGE:
339
+ rows = cached["result"].get("contracts", [])
340
+ if isinstance(rows, list) and all(isinstance(row, dict) and "expiry" in row for row in rows):
341
+ self._catalog = _validated_references(rows)
342
+ if self._catalog:
343
+ self._reference_at = cached["saved_at"]
344
+ except (sqlite3.Error, OSError, ValueError, TypeError, KeyError):
345
+ pass
346
+ self._reference_loaded = True
347
+ age = now - self._reference_at
348
+ if self._reference_at and 0 <= age < _REFERENCE_TTL:
349
+ self._reference_state = "available"
350
+ elif not self._reference_loading and now >= self._reference_retry_at:
351
+ self._reference_loading = True
352
+ self._reference_state = "loading"
353
+ start_refresh = True
354
+ if self._catalog and self._reference_at and 0 <= age <= _REFERENCE_MAX_AGE and age >= _REFERENCE_TTL:
355
+ self._reference_state = "stale"
356
+ rows = self._catalog if self._reference_at and 0 <= age <= _REFERENCE_MAX_AGE else []
357
+ if start_refresh:
358
+ threading.Thread(target=self._refresh_public_reference, daemon=True, name="fo-reference-refresh").start()
359
+ return rows
360
+
361
  def contracts(self, underlying="", instrument_type="", expiry=""):
362
  path = os.getenv("FO_REFERENCE_CSV", "")
363
+ if not path and os.getenv("FO_AUTO_REFERENCE", "1") == "1":
364
+ rows = self._public_contracts()
365
+ return self._filter_contracts(rows, underlying, instrument_type, expiry)
366
+ self._reference_source = "csv"
367
  try:
368
  info = os.stat(path) if path else None
369
  key = (path, info.st_mtime_ns, info.st_size) if info else None
 
379
  except (OSError, ValueError):
380
  self._reference_state = "unavailable"
381
  rows = []
382
+ return self._filter_contracts(rows, underlying, instrument_type, expiry)
383
+
384
+ def _filter_contracts(self, rows, underlying, instrument_type, expiry):
385
  symbol = underlying.upper().removesuffix(".NS").strip()
386
+ now = datetime.now(IST)
387
+ today = now.date().isoformat()
388
+ return [dict(row) for row in rows if (row["expiry"] > today or (row["expiry"] == today and (now.hour, now.minute) < (15, 30)))
389
  and (not symbol or row["underlying"] == symbol)
390
  and (not instrument_type or row["instrument_type"] == instrument_type)
391
  and (not expiry or row["expiry"] == expiry)]
 
395
  caps = self.provider.capabilities() if self.provider else {}
396
  return {"provider": self.name, "manual_available": True,
397
  "reference_state": self._reference_state, "reference_count": len(rows),
398
+ "reference_source": self._reference_source,
399
  "capabilities": {key: caps.get(key) is True for key in ("quotes", "option_chain", "margin")}}
400
 
401
  def resolve(self, contract_id):
static/app.js CHANGED
@@ -2964,6 +2964,28 @@ function _searchWatchlistSuggestions() {
2964
  });
2965
  wlSuggest.appendChild(item);
2966
  });
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2967
  } catch (error) {
2968
  if (seq !== _wlSuggestSeq || error.name === 'AbortError') return;
2969
  wlSuggest.textContent = 'Search unavailable. Try again.';
@@ -2982,15 +3004,15 @@ wlAddInput?.addEventListener('blur', () => {
2982
  document.getElementById('wl-add-btn')?.addEventListener('click', async () => {
2983
  const ticker = wlAddInput.value.toUpperCase().trim();
2984
  if (!ticker) return;
2985
- await addToWatchlist(ticker, _wlSelectedName);
2986
  });
2987
 
2988
- async function addToWatchlist(ticker, name) {
2989
  try {
2990
  const res = await fetch('/api/watchlist', {
2991
  method: 'POST',
2992
  headers: { 'Content-Type': 'application/json' },
2993
- body: JSON.stringify({ ticker, name }),
2994
  });
2995
  const data = await res.json();
2996
  if (!res.ok) throw new Error(data.error || `Server error ${res.status}`);
 
2964
  });
2965
  wlSuggest.appendChild(item);
2966
  });
2967
+ const state = data.reference?.state;
2968
+ let message = '';
2969
+ if (state === 'loading') {
2970
+ message = 'Loading listed futures/options contracts...';
2971
+ _wlSuggestTimer = setTimeout(() => {
2972
+ if (seq === _wlSuggestSeq) _searchWatchlistSuggestions();
2973
+ }, 2000);
2974
+ } else if (state === 'unavailable' || state === 'not_configured') {
2975
+ message = 'Contract catalog is unavailable. Retry shortly.';
2976
+ } else if (state === 'stale') {
2977
+ message = 'Cached contract references; refreshing...';
2978
+ } else if (!results.length) {
2979
+ message = 'No matching listed instruments.';
2980
+ }
2981
+ if (message && results.length) {
2982
+ const status = document.createElement('div');
2983
+ status.className = 'suggestion-status';
2984
+ status.textContent = message;
2985
+ wlSuggest.appendChild(status);
2986
+ } else if (message) {
2987
+ wlSuggest.textContent = message;
2988
+ }
2989
  } catch (error) {
2990
  if (seq !== _wlSuggestSeq || error.name === 'AbortError') return;
2991
  wlSuggest.textContent = 'Search unavailable. Try again.';
 
3004
  document.getElementById('wl-add-btn')?.addEventListener('click', async () => {
3005
  const ticker = wlAddInput.value.toUpperCase().trim();
3006
  if (!ticker) return;
3007
+ await addToWatchlist(ticker, _wlSelectedName, wlSearchType?.value || '');
3008
  });
3009
 
3010
+ async function addToWatchlist(ticker, name, instrumentType = '') {
3011
  try {
3012
  const res = await fetch('/api/watchlist', {
3013
  method: 'POST',
3014
  headers: { 'Content-Type': 'application/json' },
3015
+ body: JSON.stringify({ ticker, name, instrument_type: instrumentType }),
3016
  });
3017
  const data = await res.json();
3018
  if (!res.ok) throw new Error(data.error || `Server error ${res.status}`);
static/style.css CHANGED
@@ -275,6 +275,7 @@ input::placeholder { color: var(--text-dim); }
275
  .watchlist-contract-card .pick-name { overflow-wrap: anywhere; }
276
  #wl-search-type { width: auto; max-width: 100%; }
277
  .suggestion-name { font-size: 12px; color: var(--text-muted); }
 
278
 
279
  .tag-list { display: flex; flex-wrap: wrap; gap: 6px; margin-top: 8px; min-height: 0; }
280
  .tag {
 
275
  .watchlist-contract-card .pick-name { overflow-wrap: anywhere; }
276
  #wl-search-type { width: auto; max-width: 100%; }
277
  .suggestion-name { font-size: 12px; color: var(--text-muted); }
278
+ .suggestion-status { padding: 10px 12px; font-size: 12px; color: var(--text-muted); }
279
 
280
  .tag-list { display: flex; flex-wrap: wrap; gap: 6px; margin-top: 8px; min-height: 0; }
281
  .tag {
templates/index.html CHANGED
@@ -33,7 +33,7 @@
33
  @keyframes apl-spin { to { transform: rotate(360deg); } }
34
  .apl-hint { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; font-size: 11px; color: #5a6278; }
35
  </style>
36
- <link rel="stylesheet" href="/static/style.css?v=20261015watch">
37
  <script src="/static/vendor/lightweight-charts-4.2.0.js"></script>
38
  </head>
39
  <body>
@@ -537,6 +537,6 @@
537
  </a>
538
  </nav>
539
 
540
- <script src="/static/app.js?v=20261015watch"></script>
541
  </body>
542
  </html>
 
33
  @keyframes apl-spin { to { transform: rotate(360deg); } }
34
  .apl-hint { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; font-size: 11px; color: #5a6278; }
35
  </style>
36
+ <link rel="stylesheet" href="/static/style.css?v=20261015watch2">
37
  <script src="/static/vendor/lightweight-charts-4.2.0.js"></script>
38
  </head>
39
  <body>
 
537
  </a>
538
  </nav>
539
 
540
+ <script src="/static/app.js?v=20261015watch2"></script>
541
  </body>
542
  </html>
tests/conftest.py ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ import pytest
2
+
3
+
4
+ @pytest.fixture(autouse=True)
5
+ def isolate_public_contract_discovery(monkeypatch):
6
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "0")
tests/test_api_contract.py CHANGED
@@ -374,7 +374,7 @@ class ApiContractTests(unittest.TestCase):
374
  response = self.client.get("/api/search?q=NOTLISTED")
375
 
376
  self.assertEqual(response.status_code, 200)
377
- self.assertEqual(response.get_json(), {"results": []})
378
  mock_urlopen.assert_not_called()
379
 
380
  @patch("app.get_universe", return_value={"RELIANCE.NS": "Reliance Industries Limited"})
@@ -417,6 +417,87 @@ class ApiContractTests(unittest.TestCase):
417
  filtered = self.client.get("/api/search?q=reliance&instrument_type=OPTION").get_json()
418
  self.assertEqual([row["instrument_type"] for row in filtered["results"]], ["OPTION"])
419
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
420
  @patch("app.get_universe", side_effect=RuntimeError("Equity catalog unavailable"))
421
  @patch("fo.get_service")
422
  def test_search_keeps_indices_when_equity_catalog_is_unavailable(self, service, _universe):
 
374
  response = self.client.get("/api/search?q=NOTLISTED")
375
 
376
  self.assertEqual(response.status_code, 200)
377
+ self.assertEqual(response.get_json()["results"], [])
378
  mock_urlopen.assert_not_called()
379
 
380
  @patch("app.get_universe", return_value={"RELIANCE.NS": "Reliance Industries Limited"})
 
417
  filtered = self.client.get("/api/search?q=reliance&instrument_type=OPTION").get_json()
418
  self.assertEqual([row["instrument_type"] for row in filtered["results"]], ["OPTION"])
419
 
420
+ @patch("app.get_universe", return_value={})
421
+ @patch("fo.get_service")
422
+ def test_option_search_matches_natural_expiry_dates_and_exact_strikes(self, service, _universe):
423
+ contracts = [_app_module.fo.validate_contract({
424
+ "underlying": "NIFTY", "underlying_type": "INDEX", "expiry": expiry,
425
+ "strike": strike, "option_type": side, "lot_size": 25, "tick_size": 0.05,
426
+ }, "OPTION") for expiry, strike, side in (
427
+ ("2099-10-19", 23000, "CE"), ("2099-10-20", 23000, "CE"),
428
+ ("2099-10-19", 23000, "PE"), ("2099-10-19", 123000, "CE"),
429
+ )]
430
+ service.return_value.contracts.return_value = contracts
431
+ for query in ("nifty 19th OCT 23000 CE", "NIFTY 19 October 23000 call",
432
+ "NIFTY 19 OCT 2099 23000 CE", "NIFTY 2099-10-19 23000 CE"):
433
+ with self.subTest(query=query):
434
+ response = self.client.get("/api/search", query_string={"q": query, "instrument_type": "OPTION"})
435
+ self.assertEqual([row["ticker"] for row in response.get_json()["results"]],
436
+ [contracts[0]["contract_id"]])
437
+ response = self.client.get("/api/search?q=NIFTY%2018th%20OCT%2023000%20CE&instrument_type=OPTION")
438
+ self.assertEqual(response.get_json()["results"], [])
439
+
440
+ @patch("app.get_universe", return_value={})
441
+ @patch("fo.get_service")
442
+ def test_search_reports_public_catalog_loading_without_hiding_indices(self, service, _universe):
443
+ service.return_value.contracts.return_value = []
444
+ service.return_value.status.return_value = {"reference_state": "loading", "reference_source": "dhan_public", "reference_count": 0}
445
+ payload = self.client.get("/api/search?q=NIFTY").get_json()
446
+ self.assertTrue(any(row["ticker"] == "^NSEI" for row in payload["results"]))
447
+ self.assertEqual(payload["reference"]["state"], "loading")
448
+
449
+ @patch("app.get_universe")
450
+ @patch("app.db.add_to_watchlist")
451
+ @patch("fo.get_service")
452
+ def test_typed_option_add_resolves_one_exact_contract(self, service, add, universe):
453
+ raw = {**self._watchlist_reference(), "underlying": "NIFTY", "underlying_type": "INDEX",
454
+ "expiry": "2099-10-19", "strike": 23000}
455
+ valid = _app_module.fo.validate_contract(raw, "OPTION")
456
+ other = _app_module.fo.validate_contract({**raw, "underlying": "BANKNIFTY"}, "OPTION")
457
+ service.return_value.contracts.return_value = [valid, other]
458
+ service.return_value.resolve.return_value = valid
459
+ add.return_value = {"ticker": valid["contract_id"], "name": "NIFTY call"}
460
+ response = self.client.post("/api/watchlist", json={"ticker": "NIFTY 19th OCT 23000 CE"})
461
+ self.assertEqual(response.status_code, 201, response.get_json())
462
+ add.assert_called_once_with(valid["contract_id"], "NIFTY 2099-10-19 23000 CE")
463
+ universe.assert_not_called()
464
+
465
+ @patch("app.db.add_to_watchlist")
466
+ @patch("fo.get_service")
467
+ def test_typed_derivative_add_rejects_ambiguous_and_unlisted_contracts(self, service, add):
468
+ raw = self._watchlist_reference()
469
+ service.return_value.status.return_value = {"reference_state": "available"}
470
+ service.return_value.contracts.return_value = [raw, {**raw, "expiry": "2099-11-29"}]
471
+ for query in ("RELIANCE 2500 CE", "NIFTY 18th OCT 23000 CE"):
472
+ response = self.client.post("/api/watchlist", json={"ticker": query})
473
+ self.assertEqual(response.status_code, 400, response.get_json())
474
+ add.assert_not_called()
475
+ service.return_value.contracts.return_value = []
476
+ service.return_value.status.return_value = {"reference_state": "loading"}
477
+ response = self.client.post("/api/watchlist", json={"ticker": "NIFTY 19th OCT 23000 CE"})
478
+ self.assertEqual(response.status_code, 503)
479
+ self.assertTrue(response.get_json()["retryable"])
480
+
481
+ @patch("app.get_universe", return_value={"RELIANCE.NS": "Reliance"})
482
+ @patch("app.db.add_to_watchlist")
483
+ @patch("fo.get_service")
484
+ def test_all_derivative_underlying_and_instrument_types_can_be_searched_and_added(self, service, add, _universe):
485
+ for kind in ("OPTION", "FUTURE"):
486
+ for symbol, underlying_type in (("NIFTY", "INDEX"), ("RELIANCE", "STOCK")):
487
+ valid = _app_module.fo.validate_contract({**self._watchlist_reference(),
488
+ "underlying": symbol, "underlying_type": underlying_type}, kind)
489
+ service.return_value.contracts.return_value = [valid]
490
+ service.return_value.resolve.return_value = valid
491
+ add.return_value = {"ticker": valid["contract_id"], "name": "Contract"}
492
+ query = f"{symbol} 29th OCT " + ("2500 CE" if kind == "OPTION" else "futures")
493
+ with self.subTest(kind=kind, underlying_type=underlying_type):
494
+ result = self.client.get("/api/search", query_string={"q": query, "instrument_type": kind}).get_json()
495
+ self.assertEqual([row["ticker"] for row in result["results"]], [valid["contract_id"]])
496
+ response = self.client.post("/api/watchlist", json={"ticker": query, "instrument_type": kind})
497
+ self.assertEqual(response.status_code, 201, response.get_json())
498
+ response = self.client.post("/api/watchlist", json={"ticker": valid["contract_id"], "instrument_type": "OPTION"})
499
+ self.assertEqual(response.status_code, 400)
500
+
501
  @patch("app.get_universe", side_effect=RuntimeError("Equity catalog unavailable"))
502
  @patch("fo.get_service")
503
  def test_search_keeps_indices_when_equity_catalog_is_unavailable(self, service, _universe):
tests/test_fo.py CHANGED
@@ -1,5 +1,5 @@
1
  import json
2
- from datetime import date
3
  from unittest.mock import Mock, patch, mock_open
4
 
5
  import pytest
@@ -81,6 +81,136 @@ def test_reference_csv_is_provider_independent_and_filters_bad_rows():
81
  assert rows[1]["instrument_type"] == "FUTURE"
82
 
83
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84
  def test_quote_freshness_rejects_invalid_old_future_and_crossed_quotes():
85
  raw = {"price": 100, "bid": 99, "ask": 101, "observed_at": 1000,
86
  "trade_time": 999, "source": "fake"}
 
1
  import json
2
+ from datetime import date, datetime
3
  from unittest.mock import Mock, patch, mock_open
4
 
5
  import pytest
 
81
  assert rows[1]["instrument_type"] == "FUTURE"
82
 
83
 
84
+ @pytest.fixture
85
+ def public_master():
86
+ return ("EXCH_ID,SEGMENT,SECURITY_ID,INSTRUMENT,UNDERLYING_SYMBOL,SM_EXPIRY_DATE,"
87
+ "STRIKE_PRICE,OPTION_TYPE,LOT_SIZE,TICK_SIZE\n"
88
+ "NSE,D,47161,OPTIDX,NIFTY,2099-10-19,23000,CE,65.0,5.0000\n"
89
+ "NSE,D,47162,OPTIDX,NIFTY,2099-10-19,23000,PE,65.0,5.0000\n"
90
+ "NSE,D,47163,FUTIDX,NIFTY,2099-10-27,,XX,65.0,10.0000\n"
91
+ "NSE,D,47164,OPTSTK,RELIANCE,2099-10-27,2500,CE,250.0,5.0000\n"
92
+ "NSE,D,47165,FUTSTK,RELIANCE,2099-10-27,,XX,250.0,10.0000\n"
93
+ "BSE,D,47166,OPTIDX,SENSEX,2099-10-19,80000,CE,20.0,5.0000\n"
94
+ "NSE,E,2885,EQUITY,RELIANCE,,,,1.0,5.0000\n"
95
+ "NSE,D,47167,OPTIDX,NIFTY,2020-10-19,23000,CE,65.0,5.0000\n")
96
+
97
+
98
+ def test_public_master_validates_all_nse_contract_types(public_master):
99
+ rows = fo.parse_public_reference_csv(public_master.splitlines())
100
+ assert len(rows) == 5
101
+ assert {row["instrument_type"] for row in rows} == {"OPTION", "FUTURE"}
102
+ assert {row["underlying_type"] for row in rows} == {"INDEX", "STOCK"}
103
+ assert rows[0]["lot_size"] == 65
104
+ assert rows[0]["tick_size"] == 0.05
105
+ assert rows[2]["tick_size"] == 0.1
106
+ assert rows[0]["provider_ids"] == {"dhan": {"security_id": "47161"}}
107
+ assert fo.parse_public_reference_csv(public_master.replace("65.0", "65.5").splitlines()) == rows[3:]
108
+
109
+
110
+ def test_public_discovery_starts_once_without_waiting_for_network(monkeypatch):
111
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
112
+ monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
113
+ service = fo.FoService()
114
+ with patch("database.get_app_result_cache", return_value=None), patch("fo.threading.Thread") as worker:
115
+ assert service.contracts() == []
116
+ assert service.status()["reference_state"] == "loading"
117
+ assert service.status()["reference_source"] == "dhan_public"
118
+ worker.return_value.start.assert_called_once()
119
+
120
+
121
+ def test_public_discovery_restores_fresh_sqlite_cache_without_network(monkeypatch):
122
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
123
+ monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
124
+ valid = {**fo.validate_contract(contract(), "OPTION"), "provider_ids": {}}
125
+ cached = {"result": {"contracts": [valid]}, "saved_at": 1000}
126
+ with patch("database.get_app_result_cache", return_value=cached), patch("fo.time.time", return_value=1001), patch("fo.threading.Thread") as worker:
127
+ service = fo.FoService()
128
+ assert service.resolve(valid["contract_id"]) == valid
129
+ assert service.status()["reference_state"] == "available"
130
+ worker.assert_not_called()
131
+
132
+
133
+ def test_public_discovery_fetches_and_persists_verified_rows(monkeypatch, public_master):
134
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
135
+ response = Mock()
136
+ response.__enter__ = Mock(return_value=response)
137
+ response.__exit__ = Mock(return_value=False)
138
+ response.headers = {"Content-Length": "1000"}
139
+ response.iter_lines.return_value = public_master.splitlines()
140
+ with patch("fo.requests.get", return_value=response) as fetch, patch("database.save_app_result_cache") as save:
141
+ service = fo.FoService()
142
+ service._refresh_public_reference()
143
+ assert len(service._catalog) == 5
144
+ assert service._reference_state == "available"
145
+ assert service._reference_loading is False
146
+ assert save.call_args.args[1]["contracts"] == service._catalog
147
+ assert fetch.call_args.args[0] == fo.PUBLIC_REFERENCE_URL
148
+ assert "headers" not in fetch.call_args.kwargs
149
+
150
+
151
+ def test_public_discovery_outage_is_bounded_and_does_not_fabricate_contracts(monkeypatch):
152
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
153
+ monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
154
+ with patch("fo.requests.get", side_effect=OSError), patch("fo.time.time", return_value=1000), patch("database.get_app_result_cache", return_value=None), patch("fo.threading.Thread") as worker:
155
+ service = fo.FoService()
156
+ service._refresh_public_reference()
157
+ assert service.contracts() == []
158
+ assert service.status()["reference_state"] == "unavailable"
159
+ worker.assert_not_called()
160
+
161
+
162
+ @pytest.mark.parametrize("saved_at,rows", [(1000, [{"expiry": "2099-10-19"}]), (2000, []), (-200000, [])])
163
+ def test_public_discovery_discards_corrupt_future_and_expired_cache(monkeypatch, saved_at, rows):
164
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
165
+ monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
166
+ cached = {"result": {"contracts": rows}, "saved_at": saved_at}
167
+ with patch("database.get_app_result_cache", return_value=cached), patch("fo.time.time", return_value=1001), patch("fo.threading.Thread") as worker:
168
+ service = fo.FoService()
169
+ assert service.contracts() == []
170
+ assert service.status()["reference_state"] == "loading"
171
+ worker.return_value.start.assert_called_once()
172
+
173
+
174
+ def test_public_discovery_uses_only_bounded_stale_cache_during_refresh(monkeypatch):
175
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
176
+ monkeypatch.delenv("FO_REFERENCE_CSV", raising=False)
177
+ valid = {**fo.validate_contract(contract(), "OPTION"), "provider_ids": {}}
178
+ cached = {"result": {"contracts": [valid]}, "saved_at": 1000}
179
+ with patch("database.get_app_result_cache", return_value=cached), patch("fo.time.time", return_value=90000), patch("fo.threading.Thread") as worker:
180
+ service = fo.FoService()
181
+ assert service.contracts() == [valid]
182
+ assert service.status()["reference_state"] == "stale"
183
+ worker.return_value.start.assert_called_once()
184
+ with patch("fo.time.time", return_value=200000):
185
+ assert service.contracts() == []
186
+
187
+
188
+ def test_public_discovery_rejects_oversized_and_empty_responses(monkeypatch):
189
+ monkeypatch.setenv("FO_AUTO_REFERENCE", "1")
190
+ response = Mock()
191
+ response.__enter__ = Mock(return_value=response)
192
+ response.__exit__ = Mock(return_value=False)
193
+ response.iter_lines.return_value = []
194
+ with patch("fo.requests.get", return_value=response), patch("database.save_app_result_cache") as save:
195
+ for size in ("65000000", "0"):
196
+ response.headers = {"Content-Length": size}
197
+ service = fo.FoService()
198
+ service._refresh_public_reference()
199
+ assert service._reference_state == "unavailable"
200
+ assert service._reference_loading is False
201
+ save.assert_not_called()
202
+
203
+
204
+ def test_contract_catalog_removes_expiry_day_contracts_at_market_close():
205
+ raw = fo.validate_contract(contract(expiry="2026-10-19"), "OPTION", today=date(2026, 10, 15))
206
+ service = fo.FoService()
207
+ with patch("fo.datetime") as clock:
208
+ clock.now.return_value = datetime(2026, 10, 19, 15, 29, tzinfo=fo.IST)
209
+ assert service._filter_contracts([raw], "", "", "") == [raw]
210
+ clock.now.return_value = datetime(2026, 10, 19, 15, 30, tzinfo=fo.IST)
211
+ assert service._filter_contracts([raw], "", "", "") == []
212
+
213
+
214
  def test_quote_freshness_rejects_invalid_old_future_and_crossed_quotes():
215
  raw = {"price": 100, "bid": 99, "ask": 101, "observed_at": 1000,
216
  "trade_time": 999, "source": "fake"}
tests/test_watchlist_frontend.cjs CHANGED
@@ -377,6 +377,31 @@ test('watchlist search sends the selected instrument filter', async () => {
377
  assert.match(requestedUrl, /instrument_type=OPTION/);
378
  });
379
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
380
  test('derivative watchlist cards show contract quotes without equity controls', () => {
381
  const context = vm.createContext({ num: value => value.toFixed(2) });
382
  vm.runInContext(section('function _foEscape(', 'function _foCancelRequests('), context);
 
377
  assert.match(requestedUrl, /instrument_type=OPTION/);
378
  });
379
 
380
+ test('watchlist search reports catalog loading and retries the current query', async () => {
381
+ let calls = 0;
382
+ const fixture = searchContext([], async () => ({ ok: true, json: async () => {
383
+ calls += 1;
384
+ return { results: [], reference: { state: calls === 1 ? 'loading' : 'available' } };
385
+ } }));
386
+ fixture.input.value = 'NIFTY 19th OCT 23000 CE';
387
+ fixture.handlers.input();
388
+ await fixture.runTimer();
389
+ assert.match(fixture.suggestions.textContent, /Loading.*contract/);
390
+ fixture.runTimer();
391
+ await fixture.runTimer();
392
+ assert.equal(calls, 2);
393
+ assert.match(fixture.suggestions.textContent, /No matching listed/);
394
+ });
395
+
396
+ test('watchlist search explains unavailable contract catalogs', async () => {
397
+ const fixture = searchContext([], async () => ({ ok: true, json: async () => ({
398
+ results: [], reference: { state: 'unavailable' },
399
+ }) }));
400
+ fixture.handlers.input();
401
+ await fixture.runTimer();
402
+ assert.match(fixture.suggestions.textContent, /catalog.*unavailable/i);
403
+ });
404
+
405
  test('derivative watchlist cards show contract quotes without equity controls', () => {
406
  const context = vm.createContext({ num: value => value.toFixed(2) });
407
  vm.runInContext(section('function _foEscape(', 'function _foCancelRequests('), context);