[SPARK-48389][INFRA] Remove obsolete workflow cancel_duplicate_workfl…

…ow_runs

### What changes were proposed in this pull request?

This PR proposes to remove obsolete workflow cancel_duplicate_workflow_runs

### Why are the changes needed?

After apache@93d83c3, we don't need this anymore. In fact, this has not been working for very long time because of the workflow name mismatch.

### Does this PR introduce _any_ user-facing change?

No, dev-only.

### How was this patch tested?

Manually.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#46703 from HyukjinKwon/SPARK-48389.

Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>