Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cacheable-lookup is not working properly in some cases #2321

Closed
2 tasks done
T0bst3r opened this issue Nov 15, 2022 · 17 comments
Closed
2 tasks done

cacheable-lookup is not working properly in some cases #2321

T0bst3r opened this issue Nov 15, 2022 · 17 comments
Labels
area:monitor Everything related to monitors blocked-upstream upstream (i.e. a dependency we depend on will have to do this work) bug Something isn't working

Comments

@T0bst3r
Copy link

T0bst3r commented Nov 15, 2022

⚠️ Please verify that this bug has NOT been raised before.

  • I checked and didn't find similar issue

🛡️ Security Policy

Description

I have recently upgraded from I believe 1.17.x to 1.18.5.

Now my previously set up Slack notification channel is no longer working. When I test the notification I get the following error popup: "Error: Error: queryAaaa ESERVFAIL hooks.slack.com"

If I'm interpreting my google search correctly there is some issue for Kuma resolving the IP-address for hooks.slack.com, tho I believe that issue does not lie with slack or the DNS because I was able to send a message to the slack channel through the URL in my terminal.

I am running Kuma behind a reverse proxy and inside the official docker container, although that has never been an issue so far, as the notifications had worked flawlessly before.

I also tested with a second slack app just to be on the safe side, but same result.

Screenshot 2022-11-15 at 16 19 35

👟 Reproduction steps

  1. Got to Settings->Notifications
  2. Click on Setup Notification
  3. Select Slack
  4. Enter the Webhook URL provided by your existing or new Slack App
  5. Click test

👀 Expected behavior

I would expect Kuma to successfully send a notification in to the channel configured in the Slack App

😓 Actual Behavior

A popup is displayed reading "Error: Error: queryAaaa ESERVFAIL hooks.slack.com" and no notification arrives in the channel

🐻 Uptime-Kuma Version

1.18.5

💻 Operating System and Arch

Ubuntu 22.04 LTS

🌐 Browser

Google Chrome Version 107.0.5304.87 (Official Build) (arm64)

🐋 Docker Version

Docker version 20.10.12, build 20.10.12-0ubuntu4; docker-compose version 1.29.2, build unknown

🟩 NodeJS Version

No response

📝 Relevant log output

No response

@T0bst3r T0bst3r added the bug Something isn't working label Nov 15, 2022
@louislam
Copy link
Owner

You slack domain is resolvable in my Uptime Kuma container.

@T0bst3r
Copy link
Author

T0bst3r commented Nov 17, 2022

Okay, good to know, thank you.

I have tried gathering some more information. I'm not sure it will help with the reproducibility, but it might help finding a cause none the less.

The reason Uptime Kuma isn't able to find a AAAA-record (ipv6) for hooks.slack.com is because it has none. There are plenty of A-records (ipv4) tho and they seem to be working.

I was able to open a terminal inside the Uptime Kuma Docker container to verify that the proxy isn't the issue. After installing curl inside the container I was able to run the hello world demo request slack offers on their site, and successfully received a new message in the channel.
Kuma_bug_report_slack_request

The question I'm now asking myself is why Uptime Kuma is requesting the ipv6 address and not the ipv4 and why it is not trying to request the ipv4 after failing to get an ipv6. While I considered the Docker network being at fault here, potentially forcing Uptime Kuma to use ipv6, I found the opposite when inspecting it. As the second screenshot shows, not only does the Uptime Kuma container not have an ipv6 address, ipv6 is disabled for the entire network.
Kuma_bug_report_docker_network

Now I wasn't able to find anything in your code myself yet, not have I had the time to have a deeper look, but my guess would be that an ipv6 is requested first and an ipv4 should be requested later on as a backup and something isn't working there. Or perhaps there is a setting forcing Uptime Kuma to use ipv6 that I have not yet found (Although all monitoring pings seem to be running fine and on ipv4). What is confusing me now tho is why it's working on your side, but not on mine.

I will have a further look if I'm able to find any further issues on my side and I appreciate the effort you are putting in too.

@louislam
Copy link
Owner

louislam commented Nov 17, 2022

Your info is helpful.

For your information, since Uptime Kuma 1.18.0, I added https://github.com/szmarczak/cacheable-lookup to cache dns records.

I guess maybe it was not configured well in the Uptime Kuma side or it is a bug of cacheable-lookup.

@T0bst3r
Copy link
Author

T0bst3r commented Nov 17, 2022

Just as a quick update from my side. I downgraded back to 1.17.1 and the notification test ran successfully. I'll be updating step by step to see the update causing issues for me

@T0bst3r
Copy link
Author

T0bst3r commented Nov 17, 2022

1.18.0 Seems to be the one causing issues

@T0bst3r
Copy link
Author

T0bst3r commented Nov 17, 2022

I have rummaged through the code, as far as I can tell, nothing related to Slack notifications has been changed. Neither the test, nor the notifications in general, or the Slack specific implementation. Even the version of axios used for the outwards request seems to have stayed identical.
I am at my wits end. I'll make sure my data is still properly backed up and will set up a fresh instance, otherwise I'll have to stay with version 1.17.1 for now.
Thank you very much for your effort and the software itself tho.

@T0bst3r
Copy link
Author

T0bst3r commented Nov 17, 2022

I have however just noticed, that you responded to my earlier updates already, must have been too quick in scrolling down or too tired eyes. What ever it is. I'll take a break for now. I'm glad my additional info helped, so good luck to you in finding the issue and remember to take breaks as well :D

@bytesWhisperer
Copy link

Unfortunately I also have the same error with the same error message

@louislam louislam changed the title Slack notifications not working cacheable-lookup is not working properly in some cases Nov 21, 2022
@louislam
Copy link
Owner

It seems that there is no a perfect cache dns library for Node.js.

Maybe the best way to handle it is, make it optional.

  • Enable => working like 1.18.X
  • Disable => working like 1.17.X

@T0bst3r
Copy link
Author

T0bst3r commented Nov 21, 2022

I haven't had a chance yet to look in to your code at the cacheable-lookups there, or at their plugin itself. But is there the possibility that it's just poorly configured on your end, or a bug on theirs? Or are you sure that both sides work as intended and just seem to be incompatible with one another?

@chakflying
Copy link
Collaborator

Someone has this exact issue in their repo: cacheable-lookup#68

@louislam louislam added the blocked-upstream upstream (i.e. a dependency we depend on will have to do this work) label Dec 12, 2022
@louislam
Copy link
Owner

Workaround: Added a setting for enable/disable dns cache. In upcoming release, if you encounter the issue, you can disable it.

image

@pseacrest
Copy link

Same issue here, tried the work around but maybe best way to handle it is simply to make it optional.

Enable => working like 1.18.X
Disable => working like 1.17.X

@louislam
Copy link
Owner

Same issue here, tried the work around but maybe best way to handle it is simply to make it optional.

Enable => working like 1.18.X Disable => working like 1.17.X

Yes, it is.

@lokesh411
Copy link

lokesh411 commented Jan 3, 2023

I have the same problem, when i query for some domains like staging.camsrepository.com, CF DNS resolver throws ESERVFAIL. Since the implementation is querying for both resolve4 and resolve6, should we use promise.allSettled instead of promise.all
Code reference: https://github.com/szmarczak/cacheable-lookup/blob/master/source/index.mjs#L251

@louislam
Copy link
Owner

louislam commented Jan 3, 2023

I have the same problem, when i query for some domains like staging.camsrepository.com, CF DNS resolver throws ESERVFAIL. Since the implementation is querying for both resolve4 and resolve6, should we use promise.allSettled instead of promise.all Code reference: https://github.com/szmarczak/cacheable-lookup/blob/master/source/index.mjs#L251

You may need to share your thought in szmarczak/cacheable-lookup#52.

Or make a pull request on their repo.

@CommanderStorm CommanderStorm added the area:monitor Everything related to monitors label Dec 5, 2023
@CommanderStorm
Copy link
Collaborator

Closing as cachable-lookup was dropped in v2.0 by #4178

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:monitor Everything related to monitors blocked-upstream upstream (i.e. a dependency we depend on will have to do this work) bug Something isn't working
Projects
None yet
Development

No branches or pull requests

7 participants