NASDAQ Python SDK Client - How to resolve Market Open/Close Data Delay?

I am using the NASDAQ Python SDK client to consume market data for the NLSUTP topic. The data is fetched using the NCDSClient and processed in real-time. I send the data received from the consumer via WebSocket. During normal market hours, I get multiple responses from WebSocket within 1 second, which works as expected. However, during market opening and closing timestamps, I experience significant delays in receiving messages. These delays are sometimes more than 15 seconds, and occasionally up to a few minutes, which is problematic for my real-time application.

Code Snippet

Below is the relevant portion of my code:

def init_nasdaq_kafka_connection(topic):      security_cfg = {  "oauth.token.endpoint.uri": os.getenv("NASDAQ_KAFKA_ENDPOINT"),  "oauth.client.id": os.getenv("NASDAQ_KAFKA_CLIENT_ID"),  "oauth.client.secret": os.getenv("NASDAQ_KAFKA_CLIENT_SECRET"),      }      kafka_cfg = {  "bootstrap.servers": os.getenv("NASDAQ_KAFKA_BOOTSTRAP_URL"),  "auto.offset.reset": "latest",  "socket.keepalive.enable": True,      }      ncds_client = NCDSClient(security_cfg, kafka_cfg)      consumer = ncds_client.ncds_kafka_consumer(topic)      logger.info(f"Success to connect NASDAQ Kafka server for topic {topic}.")      return consumer  # Usage  consumer = init_nasdaq_kafka_connection("NLSUTP")  while True:      messages = consumer.consume(num_messages=1000000, timeout=0.25)      if messages:          response = makeRespFromKafkaMessages(messages)          # Sending response via WebSocket  

I wrote a script with connect to the websocket and save data in a format {current_datetime}_{number_of_messages}.jsonDuring normal hours I received data like this where you can see I get multiple records in 1 second:

Data saved to ./websocket_data/2024-12-31_15-52-00-256_13.jsonData saved to ./websocket_data/2024-12-31_15-52-00-754_12.jsonData saved to ./websocket_data/2024-12-31_15-52-01-458_45.jsonData saved to ./websocket_data/2024-12-31_15-52-01-956_26.jsonData saved to ./websocket_data/2024-12-31_15-52-02-556_48.jsonData saved to ./websocket_data/2024-12-31_15-52-03-310_45.json

But during market close I get this:

Data saved to ./websocket_data/2024-12-31_15-54-48-756_405.jsonData saved to ./websocket_data/2024-12-31_15-54-56-198_500.jsonData saved to ./websocket_data/2024-12-31_15-55-05-019_1033.jsonData saved to ./websocket_data/2024-12-31_15-55-21-788_2057.jsonData saved to ./websocket_data/2024-12-31_15-55-42-214_1318.jsonData saved to ./websocket_data/2024-12-31_15-56-00-091_1324.jsonData saved to ./websocket_data/2024-12-31_15-56-15-703_1200.jsonData saved to ./websocket_data/2024-12-31_15-56-32-602_1120.jsonData saved to ./websocket_data/2024-12-31_15-56-46-802_1149.jsonData saved to ./websocket_data/2024-12-31_15-57-00-099_940.jsonData saved to ./websocket_data/2024-12-31_15-57-13-380_875.jsonData saved to ./websocket_data/2024-12-31_15-57-24-969_936.jsonData saved to ./websocket_data/2024-12-31_15-57-36-150_789.jsonData saved to ./websocket_data/2024-12-31_15-57-49-312_1202.jsonData saved to ./websocket_data/2024-12-31_15-58-03-939_1068.jsonData saved to ./websocket_data/2024-12-31_15-58-22-238_1290.jsonData saved to ./websocket_data/2024-12-31_15-58-43-967_1553.jsonData saved to ./websocket_data/2024-12-31_15-59-12-348_1734.jsonData saved to ./websocket_data/2024-12-31_15-59-42-903_2091.jsonData saved to ./websocket_data/2024-12-31_16-00-11-254_2853.jsonData saved to ./websocket_data/2024-12-31_16-01-17-148_5911.jsonData saved to ./websocket_data/2024-12-31_16-01-49-192_2566.jsonData saved to ./websocket_data/2024-12-31_16-02-05-338_5035.jsonData saved to ./websocket_data/2024-12-31_16-02-27-056_2343.jsonData saved to ./websocket_data/2024-12-31_16-02-39-615_36.json

In the file 2024-12-31_16-02-27-056_2343.json the timestamp of lastmessage is 2024-12-31 16:00:01.072991 which shows we got the data after 2:26 minutes.

Observations

1. During normal market hours, the WebSocket receives multiple responses within 1 second, which works as expected.
2. During market opening and closing timestamps:
   • The data flow is delayed significantly.
   • Delays range from 15 seconds to a few minutes.
   • The WebSocket responses also experience delays during this period.

Expected Behavior

The data should have minimal delay, even during the high-activity periods of market opening and closing. The WebSocket should receive multiple responses within 1 second, similar to normal market hours.

Questions

1. Is this delay expected behavior for the NASDAQ Python SDK client during market open/close times?
2. Are there any configuration options or optimizations to reduce these delays?
3. Could this issue be related to server-side throttling, Kafka settings, or network latency?

Environment Details

• NASDAQ Python SDK version: Latest
• Python version: 3.10.6
• OS: Windows 10