Data flow

​

1. Overview

​

2. Real-Time Market Data Pipeline

​

2.1 Instrument & symbol sync → DB

1. Fetches instruments from Kite for NSE, NFO, MCX, CDS via getInstruments(exchange).
2. For each symbol, builds or updates a document in the indexSetting collection:
   • cash — NSE EQ instrumentToken, lot size, tradingsymbol
   • futuresChainByExpiry — all FUT expiries mapped to token, tradingsymbol, expiryType (weekly/monthly)
   • optionChainByExpiry — per expiry: strikeList, baseDiff, strikes map (CE/PE tokens per strike)
   • optionExpiriesByFuturesExpiry — which option expiries are valid for each futures expiry
   • optionsUnderlyingType — SPOT or FUTURES (options priced off cash or linked futures)
3. Upserts by index (lowercase symbol). Preserves admin-controlled fields: isActive, tradingModes, recurring/trailing settings, strike differences.
4. Emits indexSettingEmitter on updates so the live processor can reload subscriptions.

​

2.2 Active symbols → token registration

1. Loads indexSettingModel.find({ isActive: true }) — only active indices participate in streaming.
2. For each active index, enriches optionExpiriesByFuturesExpiry via enrichOptionExpiriesMap().
3. Registers base tokens when the corresponding tradingModes.* flag is enabled:

4. Subscribes all base tokens through ZerodhaConnectionManager.subscribe().

​

2.3 Real-time LTP (Kite WebSocket)

1. ZerodhaConnectionManager creates a Kite Ticker instance, connects, and subscribes tokens in FULL mode (depth + OI).
2. Every incoming tick is stored in the liveData Map keyed by instrument_token — this holds all subscribed tokens including option legs.
3. Each tick is forwarded to ZerodhaEventProcessor.processTick().
4. Base segment ticks (cash, futures, options underlying) update segmentTicks[indexName][segment].
5. An options underlying tick additionally triggers handleOptionUnderlyingTick() for strike recalculation.
6. Option leg ticks do not enter segmentTicks. They remain in liveData and are read at snapshot time via zerodhaConnection.getLatest(token) and getBidAsk(token).

​

2.4 Option strike calculation & dynamic subscribe

​

2.5 Snapshot build (buildSnapshot)

1. Reads latest ticks from segmentTicks for cash, futures, and options underlying.
2. Sets baseValue from activeSegment setting:
   • cash → cash.lastPrice
   • futures → futures.lastPrice
   • options → options.underlyingLastPrice
   • Fallback chain: options underlying → futures → cash
3. Builds nested segment objects (cash, futures, options) with quote fields (lastPrice, bidPrice, askPrice, oi, volumeTraded).
4. For options, reads leg premiums from liveData via getOptionQuote() and attaches an options.legs[] array (role, strike, right, prices).
5. Merges legacyFields (flat strike numbers + bid/ask premiums) onto the snapshot root for backward compatibility with automation and UI.

​

2.6 Persist, emit, fan-out

​

2.7 Streaming lifecycle

• handlePostZerodhaLogin() → ensureMarketDataRunning({ forceRestart: true }) → restartMarketData() → ZerodhaEventProcessor.start()
• Stopped on disconnect, 6 AM token refresh, or server shutdown via handleZerodhaDisconnect()

​

3. Snapshot Schema Reference

​

Root metadata

​

Legacy option fields (root level)

​

Nested segment objects

​

Example snapshot (trimmed)

{
  "type": "nifty",
  "date": "01/09/2026",
  "time": "10:15:01",
  "dayName": "Tuesday",
  "dataProvider": "zerodha",
  "segments": ["cash", "options"],
  "baseValue": 24532.5,
  "call": 24550,
  "put": 24550,
  "callPut": 24600,
  "putCall": 24500,
  "call1": 24650,
  "put1": 24450,
  "callAskPrice": 125.5,
  "callBidPrice": 124.0,
  "putAskPrice": 98.2,
  "putBidPrice": 97.5,
  "putCallAskPrice1": 102.0,
  "callPutAskPrice1": 118.0,
  "cash": {
    "instrumentToken": 256265,
    "tradingsymbol": "NIFTY 50",
    "lastPrice": 24530.0,
    "bidPrice": 24529.5,
    "askPrice": 24530.5
  },
  "options": {
    "underlyingLastPrice": 24532.5,
    "expiryDate": "2026-09-04",
    "legs": [
      { "role": "baseCall", "right": "CE", "strike": 24550, "lastPrice": 125.0, "askPrice": 125.5, "bidPrice": 124.0 },
      { "role": "putCall1", "right": "CE", "strike": 24500, "lastPrice": 155.0, "askPrice": 102.0, "bidPrice": 101.0 }
    ]
  }
}

​

4. Application Consumers

​

React UI (Socket.io direct)

​

Automation (latestDataMap)

• executeBuyOrder reads the latest snapshot for tradingsymbol, strike, and price:
  • Options: putCall1 / callPut1 for strike; putCallAskPrice1 / callPutAskPrice1 for price
  • Cash: cash.lastPrice or baseValue
  • Futures: futures.lastPrice or baseValue
• Skips if tick data is missing or exchange is closed.

• startInterval() runs every 1 second, iterating active symbols.
• Reads latestDataMap[\$TickData`]and passes it totrailingSell()`.
• For candle min/max over the sell duration window, also aggregates from indexDataModel (and an in-memory tick buffer at candle end).
• On price match, calls handlePlaceOrder for MARKET SELL.

• Resolves order price and strike from latestDataMap when placing limit or protected market orders.

​

5. Order Placement Flow

​

After placement

​

6. Order Status Update Flow (Webhook)

​

Step-by-step

1. Kite POSTs order status (e.g. COMPLETE, REJECTED, partial fills).
2. Backend parses payload via parseZerodhaPostback.
3. Order document is upserted in MongoDB; final statuses (COMPLETE, REJECTED) and isSold are preserved on update.
4. orderEmitter.emit("orderUpdated", updatedOrder) and orderEmitter.emit("notification", …) fire.
5. Socket bridge emits fetchOrderData and fetchFinancialData to all clients.
6. Frontend order tables listen on fetchOrderData and re-fetch order lists via REST.

​

7. Socket.io Bridge (Internal → Client)

​

Event mapping table

​

Connection setup

• Frontend: socketConnect.ts connects with query: { userId } and a 15s heartbeat (ping / pong).
• Backend: userSocketMap maps userId → array of socket IDs for targeted emits (updateUserSettingData).
• Most refresh events use io.emit (broadcast to all connected clients).

​

8. Financial / PnL Flow

​

Step-by-step

1. Every 5 seconds, startFinacialInterval fetches net positions from Kite REST.
2. getLivePnl() opens (or reuses) a dedicated Kite Ticker WebSocket for position instrument tokens.
3. On each tick batch, realized/unrealized PnL is recalculated per position.
4. orderEmitter.emit("fetchFinancialSocketData", financialData) pushes the array to clients.
5. Admin dashboard listens on fetchFinancialSocketData and updates the financial table.
6. fetchFinancialData (generic refresh signal) is also emitted on order place/update events.

​

9. Settings Sync Flow

​

Index settings

• Emitted from indexSetting.controller.ts, order.controller.ts, indexSettingGenerator.ts, and recurringOrder.controller.ts.
• indexSettingUpdated triggers:
  • Socket: fetchIndexSettingData → all clients refetch settings
  • Market data: processor reloads active settings and resubscribes tokens (see §2.2)
• indexRecurringSettingUpdated also triggers fetchIndexSettingData.

​

User settings

• Emitted from user.controller.ts and indexSetting.controller.ts when page access changes.
• userSettingUpdated sends populated user document to only that user’s socket IDs via updateUserSettingData.

​

10. Session Lifecycle vs Data Flow

​

11. End-to-End Combined Flow

1. Admin logs into Zerodha → WebSocket streaming starts (after active index settings are loaded).
2. Ticks broadcast to UI and populate latestDataMap.
3. Recurring buy reads latest tick → places BUY via handlePlaceOrder.
4. Kite webhook confirms BUY complete → order record updated → UI refreshes.
5. Schedule order created for trailing sell → 1s loop watches baseValue vs strike/target.
6. On match, MARKET SELL placed → webhook updates → UI refreshes again.

​

12. Code Map

​

13. Related Documentation

​

Summary

• Instrument sync pulls NSE/NFO/MCX/CDS instruments from Kite into IndexSetting; only isActive: true indices stream live data.
• Market ticks flow through base-token LTP → dynamic option strike subscribe → 1s snapshot → three outputs: Socket.io, MongoDB, and latestDataMap.
• Option leg prices live in liveData (not segmentTicks); snapshots read them at build time into nested options.legs[] and flat legacy fields.
• latestDataMap is the shared hub — recurring buy and trailing sell read from it every interval, not from Kite directly.
• Orders always flow through handlePlaceOrder; internal orderEmitter events are bridged to client refresh signals by socket-manager.ts.
• Kite webhooks update order status asynchronously; the UI refetches order lists on fetchOrderData rather than receiving full order payloads over Socket.io.
• Financial PnL runs on a separate 5s loop with its own Kite Ticker subscription, pushed via fetchFinancialSocketData.
• Settings changes propagate via emitters, trigger market-data resubscription, and cause targeted or broadcast UI refreshes.