nconsistent JSON Key Transmission to Splunk HEC Collector Across Various Pages

[ebrahimghane]

Description

We have identified a critical issue affecting the consistent transmission of JSON keys to the Splunk HEC Collector, particularly under conditions of slow internet connectivity, such as when using a VPN. This inconsistency has been observed across various pages and events originating from the user's browser, notably within event groups like doctor profile for doctor profile page events. The issue is characterized by the absence of vital keys (terminal_id, location, city_id) in events sent upon the initial page load, with these keys appearing in events upon page refresh. This suggests a potential problem with the timing of data storage in cookies, affecting not only search metrics but potentially all functionalities involving the Splunk HEC.

Steps to Reproduce

1. Connect to the internet using a VPN with reduced speed.
2. Perform a search on Google for a specific query such as متخصص گوش و حلق و بینی در خوی and navigate to the resulting Paziresh24 link, e.g., https://www.paziresh24.com/s/khoy/otorhinolaryngology/.
   • Expected Result: The event sent to Splunk includes keys terminal_id, location, and city_id.
   • Actual Result: These keys are missing from the initial event sent to Splunk.
3. Refresh the page.
   • Expected Result: Subsequent events maintain the same key set as the initial event.
   • Actual Result: The previously missing keys are now included in the events.

This behavior indicates that the issue could be replicated in other contexts where Splunk HEC is utilized, such as on doctor profile pages.

Expected Outcome

All events, regardless of the user's internet speed or the page context, should consistently include all necessary keys to ensure accurate data transmission and analysis.

Actual Outcome

Initial page load events often lack crucial information (terminal_id, location, city_id), which could be attributed to delays in cookie data storage or retrieval. This inconsistency affects various pages and functionalities beyond the initial search.

Suggested Developer Action

Developers are advised to first hypothesize potential causes of this inconsistency, focusing on the event handling mechanism during slow network conditions and the timing/sequence of cookie data storage and retrieval. A thorough investigation should be conducted into how data is prepared for transmission to the Splunk HEC and the role of network performance in data integrity. Proposed solutions should aim to ensure consistent data transmission across all pages and functionalities where Splunk HEC is invoked, addressing both the immediate issue and preventing similar problems in the future.

[hamidsarani]

Estimate: 6h
Deadline: 1 Day after start

what we should do:

• Fix splunk event call issue on first load from google with vpn and slow internet