Inconsistant meter reading

[hindseb]

Hi
First, thank you so much for creating this tool to read the coned meter data and get this into Home Assistant!

I noticed that the tool does not constantly read all meter updates published by ConEd. Somehow, it is only successful in ~1 of 8 updates available.

Below are some screenshots.
I am not sure where exactly the issue is, but maybe it helps to fix this so that the tool does catch all updates.

Thanks a lot for your help!

`DevTools listening on ws://127.0.0.1:47855/devtools/browser/743a81eb-383d-4ab4-8d0b-b12366843dbc
[0820/223427.304942:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/223428.612899:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/223431.125800:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049270747 (3)
[0820/223431.141310:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049270737 (3)
[0820/223500.242722:INFO:CONSOLE(2)] "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.", source: https://acdn-prod-bsdce-ngcx-va.azureedge.net/va/coned-va-build.min.js (2)
[0820/223501.557375:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/223502.520911:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/223503.017514:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin%3Furl%3D%252fen%252faccounts-billing%252fdashboard%253ftab1%253dbillingandusage-1%2526tab3%253dsectionRealTimeData-3%26returnUrl%3D%252Fen%252Faccounts-billing%252Fdashboard%253Ftab1%253Dbillingandusage-1%2526tab3%253DsectionRealTimeData-3&t=1661049302844 (3)
[0820/223506.979122:INFO:CONSOLE(1)] "::-webkit-details-marker pseudo element selector is deprecated. Please use ::marker instead. See https://chromestatus.com/feature/6730096436051968 for more details.", source: https://nextweb-static.opower.com/maestro/themes/maestro-theme-cned/291e529f859e134f140a21f4b32a070d/pages/apps/maestro-widget-library-core/index.generated.embedded-api.40.min.js (1)
Task exception was never retrieved
future: <Task finished coro=<resp() done, defined at /usr/local/lib/python3.7/dist-packages/coned/meter.py:225> exception=NetworkError('Protocol error (Network.getResponseBody): No resource with given identifier found')>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 231, in resp
raw_data = await res.text()
File "/usr/local/lib/python3.7/dist-packages/pyppeteer/network_manager.py", line 669, in text
content = await self.buffer()
File "/usr/local/lib/python3.7/dist-packages/pyppeteer/network_manager.py", line 654, in _bufread
'requestId': self._request._requestId
pyppeteer.errors.NetworkError: Protocol error (Network.getResponseBody): No resource with given identifier found
[0820/223512.896145:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Faccounts-billing%2Fdashboard%3Ftab1%3Dbillingandusage-1%26tab3%3DsectionRealTimeData-3&t=1661049310566 (3)
Creating Meter
Calling meter.last_read()..
res.url: https://cned.opower.com/ei/edge/apis/cws-real-time-ami-v1/cws/cned/accounts/8ff0f432-7638-11eb-99d8-02001700dbdb/meters/13798110/usage
res.status: 200
res.url: https://cned.opower.com/ei/edge/apis/cws-real-time-ami-v1/cws/cned/accounts/8ff0f432-7638-11eb-99d8-02001700dbdb/meters/13798110/usage
res.status: 200
message: {'startTime': '2022-08-20T21:45:00-04:00', 'endTime': '2022-08-20T22:00:00-04:00', 'value': 0.021, 'uom': 'KWH'}
Connecting to mqtt 192.168.5.149 as mqtt_user
Publishing to mqtt
Publishing electric_meter/value: 0.021
Publishing electric_meter/uom: KWH
Publishing electric_meter/startTime: 2022-08-20T21:45:00-04:00
Publishing electric_meter/endTime: 2022-08-20T22:00:00-04:00
Publishing electric_meter/message: {"startTime": "2022-08-20T21:45:00-04:00", "endTime": "2022-08-20T22:00:00-04:00", "value": 0.021, "uom": "KWH"}
Disconnectig from mqtt
DONE
Sleeping 5 mins..

[0820/224118.863236:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
DevTools listening on ws://127.0.0.1:50145/devtools/browser/9b452db1-1157-4498-bb8e-6eeb69116340
[0820/224119.702062:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
DevTools listening on ws://127.0.0.1:42163/devtools/browser/a60da16c-a6cb-401b-b8d8-982c2ecf5749
[0820/224124.877010:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/224127.177833:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049686882 (3)
[0820/224155.751674:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/224156.594159:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049716339 (3)
[0820/224210.437321:INFO:CONSOLE(2)] "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.", source: https://acdn-prod-bsdce-ngcx-va.azureedge.net/va/coned-va-build.min.js (2)
Task exception was never retrieved
future: <Task finished coro=<Meter.browse() done, defined at /usr/local/lib/python3.7/dist-packages/coned/meter.py:120> exception=TimeoutError('Navigation Timeout Exceeded: 10000 ms exceeded.')>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 199, in browse
await page.goto(usage_page_url, {'waitUntil': 'domcontentloaded', 'timeout': 10000})
File "/usr/local/lib/python3.7/dist-packages/pyppeteer/page.py", line 837, in goto
raise error
pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 10000 ms exceeded.
[0820/224216.089391:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/224218.489546:INFO:CONSOLE(2)] "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.", source: https://acdn-prod-bsdce-ngcx-va.azureedge.net/va/coned-va-build.min.js (2)
[0820/224219.567349:INFO:CONSOLE(1)] "::-webkit-details-marker pseudo element selector is deprecated. Please use ::marker instead. See https://chromestatus.com/feature/6730096436051968 for more details.", source: https://nextweb-static.opower.com/maestro/themes/maestro-theme-cned/291e529f859e134f140a21f4b32a070d/pages/apps/maestro-widget-library-core/index.generated.embedded-api.40.min.js (1)
Creating Meter
Calling meter.last_read()..
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 96, in last_read
await self.browse()
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 199, in browse
await page.goto(usage_page_url, {'waitUntil': 'domcontentloaded', 'timeout': 10000})
File "/usr/local/lib/python3.7/dist-packages/pyppeteer/page.py", line 837, in goto
raise error
pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 10000 ms exceeded.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run.py", line 26, in
startTime, endTime, value, uom = asyncio.get_event_loop().run_until_complete(meter.last_read())
File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
return future.result()
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 118, in last_read
raise MeterError("Error requesting meter data")
coned.meter.MeterError: Error requesting meter data
[0820/224250.029086:ERROR:database.cc(1707)] Cookie sqlite error 1032, errno 0: attempt to write a readonly database, sql: DELETE FROM cookies WHERE name=? AND host_key=? AND path=?
[0820/224250.031757:ERROR:database.cc(1707)] Cookie sqlite error 2067, errno 0: UNIQUE constraint failed: cookies.host_key, cookies.name, cookies.path, sql: INSERT INTO cookies (creation_utc, host_key, name, value, path, expires_utc, is_secure, is_httponly, last_access_utc, has_expires, is_persistent, priority, encrypted_value, samesite, source_scheme, source_port, is_same_party) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
[0820/224250.033149:ERROR:database.cc(1707)] Cookie sqlite error 1032, errno 0: attempt to write a readonly database, sql: DELETE FROM cookies WHERE name=? AND host_key=? AND path=?
[0820/224250.034569:ERROR:database.cc(1707)] Cookie sqlite error 2067, errno 0: UNIQUE constraint failed: cookies.host_key, cookies.name, cookies.path, sql: INSERT INTO cookies (creation_utc, host_key, name, value, path, expires_utc, is_secure, is_httponly, last_access_utc, has_expires, is_persistent, priority, encrypted_value, samesite, source_scheme, source_port, is_same_party) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
[0820/224250.036206:ERROR:database.cc(1707)] Cookie sqlite error 1032, errno 0: attempt to write a readonly database, sql: DELETE FROM cookies WHERE name=? AND host_key=? AND path=?
[0820/224250.037849:ERROR:database.cc(1707)] Cookie sqlite error 2067, errno 0: UNIQUE constraint failed: cookies.host_key, cookies.name, cookies.path, sql: INSERT INTO cookies (creation_utc, host_key, name, value, path, expires_utc, is_secure, is_httponly, last_access_utc, has_expires, is_persistent, priority, encrypted_value, samesite, source_scheme, source_port, is_same_party) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
Sleeping 5 mins..`

[0820/225413.007690:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
DevTools listening on ws://127.0.0.1:34941/devtools/browser/9c9bc563-97a8-4425-ab97-e46dd448fc07
[0820/225413.896444:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
DevTools listening on ws://127.0.0.1:36045/devtools/browser/221b9a78-f2af-482e-8b6d-8cc4a08f1bea
[0820/225419.102500:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/225419.448758:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/225421.218994:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661050460803 (3)
[0820/225441.969077:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661050481436 (3)
[0820/225458.051161:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/225459.603692:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24)
[0820/225459.924115:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin%3Furl%3D%252fen%252faccounts-billing%252fdashboard%253ftab1%253dbillingandusage-1%2526tab3%253dsectionRealTimeData-3%26returnUrl%3D%252Fen%252Faccounts-billing%252Fdashboard%253Ftab1%253Dbillingandusage-1%2526tab3%253DsectionRealTimeData-3&t=1661050499607 (3)
[0820/225501.157648:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed.
(error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin%3Furl%3D%252fen%252faccounts-billing%252fdashboard%253ftab1%253dbillingandusage-1%2526tab3%253dsectionRealTimeData-3%26returnUrl%3D%252Fen%252Faccounts-billing%252Fdashboard%253Ftab1%253Dbillingandusage-1%2526tab3%253DsectionRealTimeData-3&t=1661050500951 (3)
Creating Meter
Calling meter.last_read()..
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 96, in last_read
await self.browse()
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 214, in browse
self._LOGGER.debug(f"raw_data = {raw_data}")
NameError: name 'raw_data' is not defined
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run.py", line 26, in
startTime, endTime, value, uom = asyncio.get_event_loop().run_until_complete(meter.last_read())
File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
return future.result()
File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 118, in last_read
raise MeterError("Error requesting meter data")
coned.meter.MeterError: Error requesting meter data
Task was destroyed but it is pending!
task: <Task pending coro=<sleep() running at /usr/lib/python3.7/asyncio/tasks.py:568> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fa7bb6ee8>()]> cb=[<TaskWakeupMethWrapper object at 0x7fa7af97c8>()]>
Task was destroyed but it is pending!
task: <Task pending coro=<Meter.browse() running at /usr/local/lib/python3.7/dist-packages/coned/meter.py:205> wait_for=<Task pending coro=<sleep() done, defined at /usr/lib/python3.7/asyncio/tasks.py:555> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fa7bb6ee8>()]> cb=[<TaskWakeupMethWrapper object at 0x7fa7af97c8>()]>>
Sleeping 5 mins..