E2E labelled with "Schedule", "BackupCreation", "SKIP_KIND", "LongTime" occasionally failed

[blackpiglet]

What steps did you take and what happened:

velero/test/e2e/e2e_suite_test.go

Lines 201 to 202 in 732b87b

	var _ = Describe("Schedule controller wouldn't create a new backup when it still has pending or InProgress backup",
	Label("Schedule", "BackupCreation", "SKIP_KIND", "LongTime"), ScheduleBackupCreationTest)

The scheduled backup creation testing occasionally fails due to the backup may created twice by schedule in a time window.

�[38;5;243m------------------------------�[0m
�[0mSchedule controller wouldn't create a new backup when it still has pending or InProgress backup �[0m�[1mSchedule controller wouldn't create a new backup when it still has pending or InProgress backup�[0m �[38;5;204m[Schedule, BackupCreation, SKIP_KIND, LongTime]�[0m
�[38;5;243m/velero/workspace/E2E/e2e/velero/test/e2e/test/test.go:95�[0m
Running test case Schedule controller wouldn't create a new backup when it still has pending or InProgress backup 2024-10-18 19:43:12
velero is ready for case schedule-backup-creation-test55016685 
CreateResources 2024-10-18 19:43:13
  �[1mSTEP:�[0m Create namespace schedule-backup-creation-test55016685 �[38;5;243m@ 10/18/24 19:43:13.533�[0m
  �[1mSTEP:�[0m Create pod pod-1 in namespace schedule-backup-creation-test55016685 �[38;5;243m@ 10/18/24 19:43:13.608�[0m
Pod pod-1 is in state Pending waiting for it to be Running
Pod pod-1 is in state Pending waiting for it to be Running
Pod pod-1 is in state Pending waiting for it to be Running
Pod pod-1 is in state Pending waiting for it to be Running
Pod pod-1 is in state Pending waiting for it to be Running
Pod pod-1 is in state Pending waiting for it to be Running
Pod pod-1 is in state Pending waiting for it to be Running
Pod pod-1 is in state Pending waiting for it to be Running
Backup 2024-10-18 19:43:53
  �[1mSTEP:�[0m Creating schedule schedule-schedule-backup-creation-test55016685 ......
   �[38;5;243m@ 10/18/24 19:43:53.953�[0m
velero cmd =/velero/workspace/E2E/e2e/velero/test/../_output/bin/linux/amd64/velero --namespace velero create schedule schedule-schedule-backup-creation-test55016685 --include-namespaces schedule-backup-creation-test55016685 --schedule=*/3 * * * *
Schedule "schedule-schedule-backup-creation-test55016685" created successfully.
  �[1mSTEP:�[0m Delay one more minute to make sure the new backup was created in the given period �[38;5;243m@ 10/18/24 19:45:25.309�[0m
  �[1mSTEP:�[0m Get backups every 3 minute, and backups count should increase 1 more step in the same pace
   �[38;5;243m@ 10/18/24 19:46:25.312�[0m
Start to sleep 300000000000 minute #1 time...
[/velero/workspace/E2E/e2e/velero/test/../_output/bin/linux/amd64/velero get backups -l velero.io/storage-location=default /usr/bin/awk {print $1,"," $5,$6,$7,$8} /usr/bin/tail -n +2]
line: schedule-schedule-backup-creation-test55016685-20241018195138 ,2024-10-18 19:51:38 +0000 UTC
line: schedule-schedule-backup-creation-test55016685-20241018194838 ,2024-10-18 19:48:38 +0000 UTC
velero cmd =/velero/workspace/E2E/e2e/velero/test/../_output/bin/linux/amd64/velero --namespace velero delete schedule schedule-schedule-backup-creation-test55016685 --confirm
Schedule deleted: schedule-schedule-backup-creation-test55016685
  �[1mSTEP:�[0m Clean namespace with prefix schedule-backup-creation-test55016685 after test �[38;5;243m@ 10/18/24 19:52:26.564�[0m
  �[1mSTEP:�[0m Clean backups after test �[38;5;243m@ 10/18/24 19:52:27.071�[0m
Backup schedule-schedule-backup-creation-test55016685-20241018194838 is going to be deleted...
velero cmd =/velero/workspace/E2E/e2e/velero/test/../_output/bin/linux/amd64/velero --namespace velero delete backup schedule-schedule-backup-creation-test55016685-20241018194838 --confirm
Request to delete backup "schedule-schedule-backup-creation-test55016685-20241018194838" submitted successfully.
The backup will be fully deleted after all associated data (disk snapshots, backup files, restores) are removed.
Backup schedule-schedule-backup-creation-test55016685-20241018195138 is going to be deleted...
velero cmd =/velero/workspace/E2E/e2e/velero/test/../_output/bin/linux/amd64/velero --namespace velero delete backup schedule-schedule-backup-creation-test55016685-20241018195138 --confirm
Request to delete backup "schedule-schedule-backup-creation-test55016685-20241018195138" submitted successfully.
The backup will be fully deleted after all associated data (disk snapshots, backup files, restores) are removed.
  �[38;5;9m[FAILED]�[0m in [It] - /velero/workspace/E2E/e2e/velero/test/e2e/schedule/schedule-backup-creation.go:111 �[38;5;243m@ 10/18/24 19:52:28.449�[0m
�[38;5;9m• [FAILED] [556.444 seconds]�[0m
�[0mSchedule controller wouldn't create a new backup when it still has pending or InProgress backup �[38;5;9m�[1m[It] Schedule controller wouldn't create a new backup when it still has pending or InProgress backup�[0m �[38;5;204m[Schedule, BackupCreation, SKIP_KIND, LongTime]�[0m
�[38;5;243m/velero/workspace/E2E/e2e/velero/test/e2e/test/test.go:95�[0m

  �[38;5;9m[FAILED] Expected
      <[]string | len:2, cap:2>: [
          "schedule-schedule-backup-creation-test55016685-20241018195138 ,2024-10-18 19:51:38 +0000 UTC",
          "schedule-schedule-backup-creation-test55016685-20241018194838 ,2024-10-18 19:48:38 +0000 UTC",
      ]
  to have length 1�[0m
  �[38;5;9mIn �[1m[It]�[0m�[38;5;9m at: �[1m/velero/workspace/E2E/e2e/velero/test/e2e/schedule/schedule-backup-creation.go:111�[0m �[38;5;243m@ 10/18/24 19:52:28.449�[0m
�[38;5;243m------------------------------�[0m
�[38;5;14mS�[0m�[38;5;14mS�[0m
�[38;5;243m------------------------------�[0m

What did you expect to happen:

The following information will help us better understand what's going on:

If you are using velero v1.7.0+:
Please use velero debug --backup <backupname> --restore <restorename> to generate the support bundle, and attach to this issue, more options please refer to velero debug --help

If you are using earlier versions:
Please provide the output of the following commands (Pasting long output into a GitHub gist or other pastebin is fine.)

• kubectl logs deployment/velero -n velero
• velero backup describe <backupname> or kubectl get backup/<backupname> -n velero -o yaml
• velero backup logs <backupname>
• velero restore describe <restorename> or kubectl get restore/<restorename> -n velero -o yaml
• velero restore logs <restorename>

Anything else you would like to add:

Environment:

• Velero version (use velero version):
• Velero features (use velero client config get features):
• Kubernetes version (use kubectl version):
• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "I would like to see this bug fixed as soon as possible"
• 👎 for "There are more important bugs to focus on right now"