Wisp units crashing during batch send

[sarveshta]

Wisp units have been crashing during batch send.

Wisp 52

• x1 crashed halfway through (RSSI at -99dB, fully plugged in)
• The same chime failed with RSSI -22 and fully plugged in (20 days & connection refused on 576, reconnected next upload, connection refused on 4392, never reconnected, and finally crashed on 5401).
  The hologram says it's still connected for multiple days.

Wisp 51

• x1 Wisp died during packet Teros10 computeDP formula needs fixed to this: #40/72 (battery voltage at 3.64)

Soren speculates its 4G related or with Batch.
Soren will send Wisp 52 and 51 file.

[sarveshta]

Wisp 54

[2025.01.18 14:16:31] [DEBUG] [Loom_MongoDB.cpp:publish:148] Batch not ready to publish: 71/72
[2025.01.18 14:16:31] [DEBUG] [Loom_Hypnos.cpp:setInterruptDuration:448] Current Time (Local): 2025.01.18 06:16:31
[2025.01.18 14:16:31] [DEBUG] [Loom_Hypnos.cpp:setInterruptDuration:451] Next Interrupt Alarm Set For: 2025.01.18 06:21:31
[2025.01.18 14:16:31] [DEBUG] [Loom_Hypnos.cpp:reattachRTCInterrupt:231] Interrupt successfully reattached!
[2025.01.18 14:16:31] [DEBUG] [Loom_Hypnos.cpp:sleep:470] Entering Standby Sleep...
[2025.01.18 14:21:32] [DEBUG] [Loom_LTE.cpp:power_up:134] Powering up GPRS Modem. This should take about 10 seconds...
[2025.01.18 14:21:44] [DEBUG] [Loom_LTE.cpp:power_up:144] Powering up complete!
[2025.01.18 14:21:44] [DEBUG] [Loom_LTE.cpp:connect:196] Waiting for network...
[2025.01.18 14:21:45] [DEBUG] [Loom_LTE.cpp:connect:209] Connected to network!
[2025.01.18 14:21:45] [DEBUG] [Loom_LTE.cpp:connect:213] Attempting to connect to LTE Network: hologram
[2025.01.18 14:21:45] [DEBUG] [Loom_LTE.cpp:connect:215] Successfully Connected!
[2025.01.18 14:21:51] [DEBUG] [Loom_Hypnos.cpp:post_sleep:550] Device has awoken from sleep!
[2025.01.18 14:21:51] [DEBUG] [Loom_Hypnos.cpp:networkTimeUpdate:329] Attempting to set RTC time to the current network time...
[2025.01.18 14:21:52] [DEBUG] [Loom_Hypnos.cpp:networkTimeUpdate:335] Network time successfully set to: 2025.01.18 14:21:52
[2025.01.18 14:21:52] [DEBUG] [Loom_Manager.cpp:measure:68] ** Measuring **
[2025.01.18 14:21:52] [DEBUG] [Loom_SEN55.cpp:measure:95] Waiting for data to be ready... If not ready in 10 seconds we will stop trying
[2025.01.18 14:21:52] [ERROR] [Loom_SEN55.cpp:measure:124] No new data was ready within the given time period.
[2025.01.18 14:21:52] [DEBUG] [Loom_Manager.cpp:measure:85] ** Measuring Complete **
[2025.01.18 14:21:52] [DEBUG] [Loom_Manager.cpp:package:95] ** Packaging **
[2025.01.18 14:21:52] [DEBUG] [Loom_Manager.cpp:package:125] ** Packaging Complete **
[2025.01.18 14:21:52] [DEBUG] [Loom_Manager.cpp:display_data:198] Data Json: 

{
  "type": "data",
  "id": {
    "name": "Whisp_brd_v0p4_",
    "instance": 52
  },
  "contents": [
    {
      "module": "Packet",
      "data": {
        "Number": 1584
      }
    },
    {
      "module": "Analog",
      "data": {
        "Vbat": 4.365087986,
        "Vbat_MV": 4439.208984
      }
    },
    {
      "module": "SEN55",
      "data": {
        "PM1_0": 4.400000095,
        "PM2_5": 2.700000048,
        "PM4_0": 3.900000095,
        "PM10_0": 4.400000095,
        "AmbientHumidity": 41.84000015,
        "AmbientTemperature": 21.79999924,
        "VocIndex": 106,
        "NoxIndex": 1
      }
    },
    {
      "module": "SHT31",
      "data": {
        "Temperature": 20.56999969,
        "Humidity": 34.27999878
      }
    },
    {
      "module": "LTE",
      "data": {
        "RSSI": 24
      }
    }
  ],
  "timestamp": {
    "time_utc": "2025-01-18T14:21:52Z",
    "time_local": "2025-01-18T6:21:52Z"
  }
}
[2025.01.18 14:21:52] [DEBUG] [SDManager.cpp:log:159] Successfully logged data to Whisp_brd_v0p4_0.csv
[2025.01.18 14:21:52] [DEBUG] [MQTTComponent.cpp:connectToBroker:46] Attempting to connect to broker: cas-mosquitto.biossys.oregonstate.edu:1883
[2025.01.18 14:21:54] [DEBUG] [MQTTComponent.cpp:connectToBroker:58] Successfully connected to broker!
[2025.01.18 14:21:55] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 1 of 72
[2025.01.18 14:21:57] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:21:58] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 2 of 72
[2025.01.18 14:22:01] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:01] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 3 of 72
[2025.01.18 14:22:04] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:04] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 4 of 72
[2025.01.18 14:22:07] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:08] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 5 of 72
[2025.01.18 14:22:10] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:11] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 6 of 72
[2025.01.18 14:22:14] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:14] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 7 of 72
[2025.01.18 14:22:17] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:18] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 8 of 72
[2025.01.18 14:22:21] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:21] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 9 of 72
[2025.01.18 14:22:24] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:24] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 10 of 72
[2025.01.18 14:22:27] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:28] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 11 of 72
[2025.01.18 14:22:30] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:31] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 12 of 72
[2025.01.18 14:22:34] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:34] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 13 of 72
[2025.01.18 14:22:37] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:37] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 14 of 72
[2025.01.18 14:22:40] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:40] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 15 of 72
[2025.01.18 14:22:43] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:44] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 16 of 72
[2025.01.18 14:22:46] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:47] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 17 of 72
[2025.01.18 14:22:50] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:50] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 18 of 72
[2025.01.18 14:22:53] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:53] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 19 of 72
[2025.01.18 14:22:56] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:22:57] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 20 of 72
[2025.01.18 14:22:59] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:00] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 21 of 72
[2025.01.18 14:23:02] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:03] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 22 of 72
[2025.01.18 14:23:05] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:06] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 23 of 72
[2025.01.18 14:23:08] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:09] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 24 of 72
[2025.01.18 14:23:11] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:12] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 25 of 72
[2025.01.18 14:23:15] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:15] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 26 of 72
[2025.01.18 14:23:18] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:18] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 27 of 72
[2025.01.18 14:23:21] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:21] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 28 of 72
[2025.01.18 14:23:24] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:24] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 29 of 72
[2025.01.18 14:23:27] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:28] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 30 of 72
[2025.01.18 14:23:30] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:31] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 31 of 72
[2025.01.18 14:23:33] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:34] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 32 of 72
[2025.01.18 14:23:36] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:37] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 33 of 72
[2025.01.18 14:23:39] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:40] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 34 of 72
[2025.01.18 14:23:42] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:43] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 35 of 72
[2025.01.18 14:23:45] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:46] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 36 of 72
[2025.01.18 14:23:48] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:49] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 37 of 72
[2025.01.18 14:23:52] [DEBUG] [MQTTComponent.cpp:publishMessage:99] Data has been successfully sent!
[2025.01.18 14:23:52] [DEBUG] [Loom_MongoDB.cpp:publish:111] Publishing Packet 38 of 72

I was looking at the last couple packets, found it interesting that RSSI was 99 for a good portion of the packets except the one right before the batch send:

{"type":"data","id":{"name":"Whisp_brd_v0p4_","instance":52},"contents":[{"module":"Packet","data":{"Number":1579}},{"module":"Analog","data":{"Vbat":4.329638481,"Vbat_MV":4353.808594}},{"module":"SEN55","data":{"PM1_0":4.400000095,"PM2_5":2.700000048,"PM4_0":3.900000095,"PM10_0":4.400000095,"AmbientHumidity":41.84000015,"AmbientTemperature":21.79999924,"VocIndex":106,"NoxIndex":1}},{"module":"SHT31","data":{"Temperature":20.54999924,"Humidity":34.27999878}},{"module":"LTE","data":{"RSSI":99}}],"timestamp":{"time_utc":"2025-01-18T13:56:18Z","time_local":"2025-01-18T5:56:18Z"}}
{"type":"data","id":{"name":"Whisp_brd_v0p4_","instance":52},"contents":[{"module":"Packet","data":{"Number":1580}},{"module":"Analog","data":{"Vbat":4.363476753,"Vbat_MV":4363.476563}},{"module":"SEN55","data":{"PM1_0":4.400000095,"PM2_5":2.700000048,"PM4_0":3.900000095,"PM10_0":4.400000095,"AmbientHumidity":41.84000015,"AmbientTemperature":21.79999924,"VocIndex":106,"NoxIndex":1}},{"module":"SHT31","data":{"Temperature":20.53000069,"Humidity":34.33000183}},{"module":"LTE","data":{"RSSI":99}}],"timestamp":{"time_utc":"2025-01-18T14:1:21Z","time_local":"2025-01-18T6:1:21Z"}}
{"type":"data","id":{"name":"Whisp_brd_v0p4_","instance":52},"contents":[{"module":"Packet","data":{"Number":1581}},{"module":"Analog","data":{"Vbat":4.365087986,"Vbat_MV":4392.480469}},{"module":"SEN55","data":{"PM1_0":4.400000095,"PM2_5":2.700000048,"PM4_0":3.900000095,"PM10_0":4.400000095,"AmbientHumidity":41.84000015,"AmbientTemperature":21.79999924,"VocIndex":106,"NoxIndex":1}},{"module":"SHT31","data":{"Temperature":20.56999969,"Humidity":34.41999817}},{"module":"LTE","data":{"RSSI":99}}],"timestamp":{"time_utc":"2025-01-18T14:6:24Z","time_local":"2025-01-18T6:6:24Z"}}
{"type":"data","id":{"name":"Whisp_brd_v0p4_","instance":52},"contents":[{"module":"Packet","data":{"Number":1582}},{"module":"Analog","data":{"Vbat":4.345751762,"Vbat_MV":4389.257813}},{"module":"SEN55","data":{"PM1_0":4.400000095,"PM2_5":2.700000048,"PM4_0":3.900000095,"PM10_0":4.400000095,"AmbientHumidity":41.84000015,"AmbientTemperature":21.79999924,"VocIndex":106,"NoxIndex":1}},{"module":"SHT31","data":{"Temperature":20.56999969,"Humidity":34.38000107}},{"module":"LTE","data":{"RSSI":99}}],"timestamp":{"time_utc":"2025-01-18T14:11:27Z","time_local":"2025-01-18T6:11:27Z"}}
{"type":"data","id":{"name":"Whisp_brd_v0p4_","instance":52},"contents":[{"module":"Packet","data":{"Number":1583}},{"module":"Analog","data":{"Vbat":4.395703316,"Vbat_MV":4358.642578}},{"module":"SEN55","data":{"PM1_0":4.400000095,"PM2_5":2.700000048,"PM4_0":3.900000095,"PM10_0":4.400000095,"AmbientHumidity":41.84000015,"AmbientTemperature":21.79999924,"VocIndex":106,"NoxIndex":1}},{"module":"SHT31","data":{"Temperature":20.56999969,"Humidity":34.24000168}},{"module":"LTE","data":{"RSSI":99}}],"timestamp":{"time_utc":"2025-01-18T14:16:30Z","time_local":"2025-01-18T6:16:30Z"}}
{"type":"data","id":{"name":"Whisp_brd_v0p4_","instance":52},"contents":[{"module":"Packet","data":{"Number":1584}},{"module":"Analog","data":{"Vbat":4.365087986,"Vbat_MV":4439.208984}},{"module":"SEN55","data":{"PM1_0":4.400000095,"PM2_5":2.700000048,"PM4_0":3.900000095,"PM10_0":4.400000095,"AmbientHumidity":41.84000015,"AmbientTemperature":21.79999924,"VocIndex":106,"NoxIndex":1}},{"module":"SHT31","data":{"Temperature":20.56999969,"Humidity":34.27999878}},{"module":"LTE","data":{"RSSI":24}}],"timestamp":{"time_utc":"2025-01-18T14:21:52Z","time_local":"2025-01-18T6:21:52Z"}}

[sarveshta]

I plan to follow up with Will to get access or get info about the server MQTT is hosted on. I'm interested in looking at logs from the server side as well.