Message:A general system error occurred: The parent of this virtual disk could not be opened

[qxmips]

Could you advice what could cause this?

Summary

SOmetimes when performing docker-compose pull the operation fails with:

Pulling app (registryl/staging/smb:latest)...
latest: Pulling from staging/smb
Failed to write to image store: [POST /storage/{store_name}][500] WriteImage default  &{Code:500 Message:A general system error occurred: The parent of this virtual disk could not be opened}

Environment information

VIC 1.5.2

vSphere and vCenter Server version

6.7.0 build 13007421

VIC version

v1.5.2.1500
VCH installer version v1.5.2-20879-30b67a14

VCH configuration

    --target=https://*******
    --thumbprint=******
    --name=vch-prod
    --compute-resource=******
    --ops-user=****@vsphere.local
    --image-store=ds://****
    --container-name-convention={name}-prod
    --volume-store=ds://*****/VIC:default
    --volume-store=ds://*****/VIC:san-prod-02
    --volume-store=ds://*****/VIC:san-prod-03
    --dns-server=*****
    --bridge-network=vlan72-vic-bridge
    --bridge-network-range=172.20.0.0/16
    --public-network=vlan72-10.4.72-WebApps
    --public-network-gateway=10.4.72.254
    --public-network-ip=10.4.72.10/24
    --container-network=vlan72-vic-containers:vlan72-vic-containers
    --container-network-gateway=vlan72-vic-containers:10.4.72.254/24
    --container-network-ip-range=vlan72-vic-containers:10.4.72.151-10.4.72.199
    --container-network-dns=vlan72-vic-containers:10.4.34.80
    --container-network-dns=vlan72-vic-containers:10.4.34.81
    --container-network-firewall=vlan72-vic-containers:open
    --syslog-address=tcp://******l:5140

Details

as a workaround I copied the same image layer (fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36) from another VCH . then it pulled sussessfully.

Steps to reproduce

Actual behavior

Expected behavior

Logs

port-layer logs:

Sep  5 2019 12:58:24.623Z INFO  op=245.8942.8943.35507:   Creating directory [san-prod-03] vic-staging-imagestore/VIC/422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36
Sep  5 2019 12:58:24.635Z INFO  op=245.8942.8943.35507:   Writing metadata 422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36/imageMetadata/metaData
Sep  5 2019 12:58:25.145Z INFO  op=245.8942.8943.35507:   Creating image fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36 ([san-prod-03] vic-staging-imagestore/VIC/422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36.vmdk)
Sep  5 2019 12:58:25.145Z INFO  op=245.8942.8943.35507:   Create/attach vmdk [san-prod-03] vic-staging-imagestore/VIC/422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36.vmdk from parent [san-prod-03] vic-staging-imagestore/VIC/422a873c-08bb-2e20-c5fd-42be31e8f49c/images/c84398f205d3d9ee4b6cada86a51d08a4b80ea98b95b046a1daa01dfbd3f1925/c84398f205d3d9ee4b6cada86a51d08a4b80ea98b95b046a1daa01dfbd3f1925.vmdk
Sep  5 2019 12:58:25.153Z INFO  op=245.8942.8943.35507:   No disks match name and persistence: [san-prod-03] vic-staging-imagestore/VIC/422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36.vmdk, true
Sep  5 2019 12:58:25.683Z ERROR op=245.8942.8943.35507:   unexpected fault on task retry: &types.SystemError{RuntimeFault:types.RuntimeFault{MethodFault:types.MethodFault{FaultCause:(*types.LocalizedMethodFault)(nil), FaultMessage:[]types.LocalizableMessage(nil)}}, Reason:"The parent of this virtual disk could not be opened"}
Sep  5 2019 12:58:25.691Z ERROR op=245.8942.8943.35507:   vmdk storage driver failed to attach disk: A general system error occurred: The parent of this virtual disk could not be opened
Sep  5 2019 12:58:25.691Z ERROR op=245.8942.8943.35507:   Cleaning up failed image fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36
Sep  5 2019 12:58:25.691Z INFO  op=245.8942.8943.35507:   Removing 422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36/manifest
Sep  5 2019 12:58:25.731Z INFO  op=245.8942.8943.35507:   Removing 422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36.vmdk
Sep  5 2019 12:58:25.943Z INFO  op=245.8942.8943.35507:   Removing 422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36
Sep  5 2019 12:58:26.017Z ERROR op=245.8942.8943.35507:   ImageStore: delete image error: Cannot delete file [san-prod-03] vic-staging-imagestore/VIC/422a873c-08bb-2e20-c5fd-42be31e8f49c/images/fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36
Sep  5 2019 12:58:26.017Z ERROR op=245.8942.8943.35507:   WriteImage of fd0baa9d66fc6b9a758a607a015fa544b6080d45b04c3c2291c1aea09ba96e36 failed with: A general system error occurred: The parent of this virtual disk could not be opened

[renmaosheng]

Jun,
could you please take a look at this issue? thanks.