Skip to content

e2e-flakiness-detector #230

e2e-flakiness-detector

e2e-flakiness-detector #230

Triggered via schedule October 25, 2024 14:04
Status Failure
Total duration 1h 28m 55s
Artifacts 2
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

17 errors and 2 notices
attribution.test.ts:10:1 › attribution search enabled in chat: vscode\test\e2e\helpers.ts#L377
1) attribution.test.ts:10:1 › attribution search enabled in chat ───────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVSGVB0 after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVSGVB0\Crashpad\reports\83c2fb55-b476-4001-926b-88dd77abab90.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
attribution.test.ts:20:1 › attribution search disabled in chat: vscode\test\e2e\helpers.ts#L377
2) attribution.test.ts:20:1 › attribution search disabled in chat ──────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscekMfX3z after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscekMfX3z\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode\test\e2e\helpers.ts#L377
3) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceMjIowH after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceMjIowH\Crashpad\reports\9a4f9018-7ff0-4572-b3cc-4772aec74ef2.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-atFile.test.ts:218:3 › editing a chat message with @-mention: vscode\test\e2e\helpers.ts#L377
4) chat-atFile.test.ts:218:3 › editing a chat message with @-mention ───────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscecSehjX after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscecSehjX\Crashpad\reports\4d60cbca-c631-4b74-97a8-2340c45f73c4.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L365
5) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 4 + Received + 1 - Array [ - "buzz.ts", - "buzz.ts:2-5", - ] + Array [] Call log: - expect.toHaveText with timeout 2000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements 363 | 364 | const [, lastChatInput] = await createEmptyChatPanel(page) > 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], { | ^ 366 | timeout: 2_000, 367 | }) 368 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L365
6) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 4 + Received + 1 - Array [ - "buzz.ts", - "buzz.ts:2-5", - ] + Array [] Call log: - expect.toHaveText with timeout 2000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements 363 | 364 | const [, lastChatInput] = await createEmptyChatPanel(page) > 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], { | ^ 366 | timeout: 2_000, 367 | }) 368 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L1
6) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\2f52d20c0dfcdbbb3b23f8c25eda1f63.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_2_retry_0_failure.webm'
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L365
7) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 4 + Received + 1 - Array [ - "buzz.ts", - "buzz.ts:2-5", - ] + Array [] Call log: - expect.toHaveText with timeout 2000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements 363 | 364 | const [, lastChatInput] = await createEmptyChatPanel(page) > 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], { | ^ 366 | timeout: 2_000, 367 | }) 368 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L1
7) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\197e7d4278500f6430a71b96124214c1.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_3_retry_0_failure.webm'
chat-context.test.ts:11:5 › chat followup context: vscode\test\e2e\helpers.ts#L377
8) chat-context.test.ts:11:5 › chat followup context ───────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceRuOUII after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceRuOUII\Crashpad\reports\18ffa883-14b0-4a2d-bfc5-756b54ece183.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode/test/e2e/chat-atFile.test.ts#L89
1) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) 87 | await chatInput.fill('') 88 | await chatInput.pressSequentially('@mj', { delay: 350 }) > 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible() | ^ 90 | await chatInput.fill('clear') 91 | 92 | // Searching and clicking at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
2) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Test timeout of 30000ms exceeded.
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/common.ts#L179
2) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Error: locator.fill: Target page, context or browser has been closed Call log: - waiting for getByPlaceholder('Type the name of a command to run.') - locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/> - fill("buzz.ts") - attempting fill action - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #1 - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #2 - waiting 20ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #3 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #4 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #5 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #6 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #7 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #8 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #9 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #10 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #11 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #12 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #13 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #14 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #15 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #16 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #17 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #18 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #19 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #20 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible
command-edit.test.ts:133:1 › edit (fixup) input - model selection: vscode/test/e2e/command-edit.test.ts#L154
3) command-edit.test.ts:133:1 › edit (fixup) input - model selection ───────────────────────────── Error: expect(locator).toBeVisible() Locator: getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic') - locator resolved to <div role="option" data-index="0" aria-level="1" aria-setsize="4" id="list_id_4_0" aria-posinset="1" draggable="false" data-parity="even" data-last-element="false" class="monaco-list-row focused" aria-label="check anthropic-logo Claude 3.5 Sonnet, by anthropic">…</div> - unexpected value "hidden" 152 | await modelItem.click() 153 | const selectedModelItem = page.getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic') > 154 | expect(selectedModelItem).toBeVisible() | ^ 155 | }) 156 | at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:154:31
uninstall.test.ts:7:5 › uninstall extension: vscode/test/e2e/uninstall.test.ts#L1
4) uninstall.test.ts:7:5 › uninstall extension ─────────────────────────────────────────────────── Test timeout of 30000ms exceeded.
uninstall.test.ts:7:5 › uninstall extension: vscode/test/e2e/uninstall.test.ts#L1
5) uninstall.test.ts:7:5 › uninstall extension ─────────────────────────────────────────────────── Test timeout of 30000ms exceeded.
test-e2e (ubuntu)
Process completed with exit code 1.
🎭 Playwright Run Summary
21 failed attribution.test.ts:10:1 › attribution search enabled in chat ────────────────────────────────── attribution.test.ts:20:1 › attribution search disabled in chat ───────────────────────────────── chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:218:3 › editing a chat message with @-mention ────────────────────────────── chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-context.test.ts:11:5 › chat followup context ────────────────────────────────────────────── chat-input.test.ts:194:3 › chat model selector ───────────────────────────────────────────────── chat-input.test.ts:194:3 › chat model selector ───────────────────────────────────────────────── chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users ─────────── command-commit.test.ts:24:3 › use terminal output as context ─────────────────────────────────── command-commit.test.ts:24:3 › use terminal output as context ─────────────────────────────────── command-commit.test.ts:24:3 › use terminal output as context ─────────────────────────────────── command-edit.test.ts:95:1 › edit (fixup) input - range selection ─────────────────────────────── command-edit.test.ts:28:3 › edit (fixup) task ────────────────────────────────────────────────── command-edit.test.ts:133:1 › edit (fixup) input - model selection ────────────────────────────── command-edit.test.ts:133:1 › edit (fixup) input - model selection ────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── 12 skipped 155 passed (40.7m)
🎭 Playwright Run Summary
5 failed chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ────────────────────────── command-edit.test.ts:133:1 › edit (fixup) input - model selection ────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── 30 skipped 435 passed (1.4h)

Artifacts

Produced during runtime
Name Size
playwright-recordings ubuntu
1.47 MB
playwright-recordings windows
1.04 MB