JMAP - DNS requests during Account export & import

[infrequently]

Hi there,
it is recommended for the upgrade from 0.4.2 to 0.5.0 to export all accounts and import them later.

I tried that on my test environment, witch is a VM running on my PC at home. The VM is using my pi.hole as DNS server, just like all other devices do.
My VM is named "debian". But my pi.hole has an DNS record identifying the VM as mail.***.. I can copy all data to my test environment and simulate the upgrade with my real domain name and data.

During the first export, die VM submitted about 6000 DNS requests per minute towards my pi.hole, queuing mail.***.. I found the reason myself: pi.hole uses a TTL of zero in all DNS replies, no caching.
To make my test environment more realistic, I increased TTL to 600 and expected just a few DNS requests. Because the reply should be cached for a while. But still about 1500 DNS requests per minute were submitted.
The same thing happened during import.

I added the servers dns-name & IP-Address in /etc/hosts (both, the "real" and the one inside docker), but that changed nothing.

Why the import and export are triggering so many DNS requests?
Or is it just a fault in my test environment, causing the amount of DNS requests?

Thank you in advance for any suggestions.

[mdecimus]

Hi,

Both the 'export' and 'import' commands use JMAP to download and then insert the records. Since JMAP is an HTTP based protocol, all those DNS requests you are seeing are for each HTTP request sent to the server. The CLI tool could be improved to use JMAP over WebSockets which will use just one connection rather than many.

[infrequently]

Thank you very much for your fast answer.
I checked my findings again and the changed TTL has no effect. It was just a coincidence, the number of requests increased to the same amount after a few minutes.

Fewer DNS requests would be nice. I run my own mail server and use the external DNS server provided by my hoster. If the external DNS server has a rate limit, the export can be slowed down and/or canceled.

[dimejo]

I run my own mail server and use the external DNS server provided by my hoster. If the external DNS server has a rate limit, the export can be slowed down and/or canceled.

Using your own DNS resolver is generally recommended when running a mail server because spam filters tend to make lots of DNS request.

[infrequently]

Using your own DNS resolver is generally recommended when running a mail server because spam filters tend to make lots of DNS request.

That may be true depending on the amount of connections and emails processed.
But I never counted the DNS requests done by stalwart for one incoming or rejected email or any connection. My test environment doesn't allow any incoming mail.
If you did count the amount of DNS requests, would you please share your findings?

I just would like to point out, that a local export/import are triggering that amount of DNS requests, and that the import/export are not taking advantage of DNS caching. Less DNS requests would be desirable if an import or export is executed locally. Or taking advantage of the DNS caching mechanism.
At least that is my opinion about that matter, witch of course can be discussed.

[mdecimus]

The CLI tool uses the reqwest library to send HTTP requests to the server. Although I haven't looked at their code, I'm quite certain that they're delegating hostname resolution to the OS.
If you are seeing all those queries it is probably due to an issue with your local resolver that is not caching DNS requests.

[infrequently]

I did some more tests.

• A "curl mail.[mydomain]" executed on the linux machine itself triggers two DNS requests the first time (A record and AAAA record). Any following "curl mail.[mydomain]" triggers no DNS request until the TTL is reached.
• The same curl command executed inside the docker container triggers two DNS requests (A record and AAAA record) every time. The container itself has an ipv4-Address 172.17.x.x but no ipv6-Address.
• If the container has a static ipv4-Address and user defined bridge network, the curl command executed inside the docker container only triggers one DNS request the first time (A record). Any following request triggers no DNS request until the TTL is reached.

The behavior seems to be docker-related. Either it is a default behavior or I accidentally modified a relevant setting.
I use the static ipv4-Address and user defined bridge network as a solution for me.

Edit 14.01.2024: Closed, because it seems to be an unique problem and there is a solution as described in this post.