MD cluster add new device might failed

[t460p]

When adding a member disk in an MD (Multiple Devices) cluster, each node clears the superblock and uses a new device_uuid？
If each node clears the superblock and uses a new device_uuid in an MD cluster, it might lead to a failure in locating member disks using blkid on some nodes。

[mtkaczyk]

Discussed offline:
It is clustered raid array with multiple nodes, and it is failing --cluster-confirm after --add to raid array on one node:

blkid -o device -t UUID_SUB=xxxxxxx in 69-md-clustered-confirm-device.ruler failed

The best I can do is ask @GuoqingJiang for advice here.

@GuoqingJiang could you please take a look?

[mtkaczyk]

Looks like @GuoqingJiang is stale account and we may not receive answer, maybe @mwilck?

[mwilck]

Sorry, no clue about clustered MD.@colyli, perhaps.

[t460p]

In the MD cluster ,adding a new Device, just like this:

other nodes may occur like this "blkid -o device -t UUID_SUB=xxxxxxx in 69-md-clustered-confirm-device.ruler failed".

Through debugging, it was discovered that other nodes will modify the super block and generate new device_UUID during doing the commad of “mdadm --manage /dev/mdX --confirm-cluster /dev/sdZZ:$slot”. So, if node2 modifed the super block and generate new device_UUID, the node3 will failed in 69-md-clustered-confirm-device.ruler

[mtkaczyk]

So looks like it is broken because we are writing new metadata to fast, correct?

I don't have environment and possibility to support you with investigation right now, best I can do is asking you for fix that I can merge.