Err initial load: DNS_ERROR

[islamyakin]

mirroring postgres to clickhouse,

with instalation peerdb ./run-peerdb.sh.

i have a error

failed to sync records: code: 499, message: Failed to get object info: Code: 198. DB::NetException: Not found address of host: host.docker.internal. (DNS_ERROR) (version 24.9.2.42 (official build)). HTTP response code: 18446744073709551615: while reading clone_click_exa_ak_rezas_e13ff3d0_c93b_4187_9e81_7157a11ee53f/clone_click_exa_ak_rezas_e13ff3d0_c93b_4187_9e81_7157a11ee53f/f38ac276-6d49-4b92-9314-8844a7e336dc.avro.zst: While executing S3Source

[islamyakin]

has been config in my virtual machine

127.0.0.1 host.docker.internal
127.0.0.1 gateway.docker.internal

[iamKunalGupta]

Hi can you please provide some more information:

• How is ClickHouse setup? (Cloud/VM/Docker)

[islamyakin]

Hi can you please provide some more information:

• How is ClickHouse setup? (Cloud/VM/Docker)

setup in VM

[islamyakin]

Clickhouse and peerdb in different host

[iamKunalGupta]

Minio needs to be accessible by both PeerDB and ClickHouse (via the same URL). In that case I'd recommend setting the AWS Endpoint for S3 to maybe the url of the minio VM and make the minio VM accessible to ClickHouse.

[islamyakin]

i see, clickhouse can access minio in docker-compose with peerdb?
this is my docker-compose

name: peerdb-quickstart

x-minio-config: &minio-config
PEERDB_CLICKHOUSE_AWS_CREDENTIALS_AWS_ACCESS_KEY_ID: _1
PEERDB_CLICKHOUSE_AWS_CREDENTIALS_AWS_SECRET_ACCESS_KEY: _1
PEERDB_CLICKHOUSE_AWS_CREDENTIALS_AWS_REGION: us-east-1
PEERDB_CLICKHOUSE_AWS_CREDENTIALS_AWS_ENDPOINT_URL_S3: http://minio:9000
PEERDB_CLICKHOUSE_AWS_S3_BUCKET_NAME: peerdbbucket1

[iamKunalGupta]

i see, clickhouse can access minio in docker-compose with peerdb?

Yes ClickHouse needs to access the object storage that PeerDB also has access to.

In this case you can set PEERDB_CLICKHOUSE_AWS_CREDENTIALS_AWS_ENDPOINT_URL_S3 to http://: and make sure your security groups allow for the CH VM to access the minio VM. Alternatively you can use an S3 Bucket and use that instead.