Skip to content

e2e-flakiness-detector #242

e2e-flakiness-detector

e2e-flakiness-detector #242

Triggered via schedule November 6, 2024 14:04
Status Failure
Total duration 1h 51m 54s
Artifacts 2
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

20 errors and 2 notices
attribution.test.ts:20:1 › attribution search disabled in chat: vscode\test\e2e\helpers.ts#L380
1) attribution.test.ts:20:1 › attribution search disabled in chat ──────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscesuJFet after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscesuJFet\Crashpad\reports\c05d6094-b340-4800-9f66-e61c4cb374b3.dmp' at helpers.ts:380 378 | } catch (error) { 379 | if (attempts === 1) { > 380 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 381 | } 382 | 383 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:380:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:261:13)
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode\test\e2e\chat-atFile.test.ts#L89
2) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) 87 | await chatInput.fill('') 88 | await chatInput.pressSequentially('@mj', { delay: 350 }) > 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible() | ^ 90 | await chatInput.fill('clear') 91 | 92 | // Searching and clicking at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:89:77
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode\test\e2e\chat-atFile.test.ts#L1
2) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\__mention_file_in_chat\temp-videos\38edacf1103d96d4708e3d885d8b0cb6.webm' -> 'D:\a\cody\cody\playwright\__mention_file_in_chat\videos\run_0_retry_0_failure.webm'
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L365
3) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 4 + Received + 1 - Array [ - "buzz.ts", - "buzz.ts:2-5", - ] + Array [] Call log: - expect.toHaveText with timeout 2000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements 363 | 364 | const [, lastChatInput] = await createEmptyChatPanel(page) > 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], { | ^ 366 | timeout: 2_000, 367 | }) 368 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
command-custom.test.ts:283:17 › use terminal output as context: vscode\test\e2e\helpers.ts#L380
4) command-custom.test.ts:283:17 › use terminal output as context ──────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscerzDcxW after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscerzDcxW\Crashpad\reports\8e7cdc32-b2d9-4728-b907-b265bf98bf08.dmp' at helpers.ts:380 378 | } catch (error) { 379 | if (attempts === 1) { > 380 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 381 | } 382 | 383 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:380:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:261:13)
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode\test\e2e\command-edit.test.ts#L123
5) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect(locator).toBeVisible() Locator: getByLabel('check file-code Nearest Code Block') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('check file-code Nearest Code Block') 121 | await rangeItem.click() 122 | const selectedRangeItem = page.getByLabel('check file-code Nearest Code Block') > 123 | expect(selectedRangeItem).toBeVisible() | ^ 124 | 125 | // Open the symbols input and check it has the correct item selected 126 | const symbolitem = page.getByText('Select a Symbol...') at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:123:31
command-edit.test.ts:138:1 › edit (fixup) input - model selection: vscode\test\e2e\command-edit.test.ts#L154
6) command-edit.test.ts:138:1 › edit (fixup) input - model selection ───────────────────────────── Error: expect(locator).toBeVisible() Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" 152 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') 153 | await nap() > 154 | expect(modelItem).toBeVisible() | ^ 155 | 156 | // Open the model input and check it has the correct item selected 157 | await modelItem.click() at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:154:23
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode\test\e2e\command-edit.test.ts#L118
7) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect(locator).toBeVisible() Locator: getByText('Nearest Code Block') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 116 | // Check the correct range item is auto-selected 117 | const rangeItem = page.getByText('Nearest Code Block') > 118 | expect(rangeItem).toBeVisible() | ^ 119 | 120 | // Open the range input and check it has the correct item selected 121 | await rangeItem.click() at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:118:23
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error: vscode\test\e2e\helpers.ts#L380
8) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce1Jf0zi after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce1Jf0zi\Crashpad\reports\eac4656b-9cec-4c7e-acf2-f8eb47ac2643.dmp' at helpers.ts:380 378 | } catch (error) { 379 | if (attempts === 1) { > 380 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 381 | } 382 | 383 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:380:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:261:13)
uninstall.test.ts:7:5 › uninstall extension: vscode\test\e2e\common.ts#L54
9) uninstall.test.ts:7:5 › uninstall extension ─────────────────────────────────────────────────── TimeoutError: page.click: Timeout 30000ms exceeded. Call log: - waiting for locator('[aria-label="Cody"]') at common.ts:54 52 | // In case the cody sidebar isn't focused, select it. 53 | while (!(await isSidebarVisible(page))) { > 54 | await page.click('[aria-label="Cody"]') | ^ 55 | } 56 | } 57 | at focusSidebar (D:\a\cody\cody\vscode\test\e2e\common.ts:54:20) at signin (D:\a\cody\cody\vscode\test\e2e\uninstall.test.ts:65:5) at D:\a\cody\cody\vscode\test\e2e\uninstall.test.ts:50:5
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode/test/e2e/chat-atFile.test.ts#L89
1) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) 87 | await chatInput.fill('') 88 | await chatInput.pressSequentially('@mj', { delay: 350 }) > 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible() | ^ 90 | await chatInput.fill('clear') 91 | 92 | // Searching and clicking at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode/test/e2e/chat-atFile.test.ts#L89
2) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) 87 | await chatInput.fill('') 88 | await chatInput.pressSequentially('@mj', { delay: 350 }) > 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible() | ^ 90 | await chatInput.fill('clear') 91 | 92 | // Searching and clicking at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode/test/e2e/chat-atFile.test.ts#L89
3) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) 87 | await chatInput.fill('') 88 | await chatInput.pressSequentially('@mj', { delay: 350 }) > 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible() | ^ 90 | await chatInput.fill('clear') 91 | 92 | // Searching and clicking at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
4) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Test timeout of 30000ms exceeded.
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/common.ts#L179
4) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Error: locator.fill: Target page, context or browser has been closed Call log: - waiting for getByPlaceholder('Type the name of a command to run.') - locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/> - fill("buzz.ts") - attempting fill action - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #1 - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #2 - waiting 20ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #3 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #4 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #5 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #6 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #7 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #8 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #9 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #10 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #11 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #12 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #13 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #14 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #15 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #16 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #17 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #18 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #19 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #20 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible
command-core.test.ts:106:3 › Explain Command from Prompts Tab: vscode/test/e2e/command-core.test.ts#L1
5) command-core.test.ts:106:3 › Explain Command from Prompts Tab ───────────────────────────────── Test timeout of 30000ms exceeded.
command-core.test.ts:106:3 › Explain Command from Prompts Tab: vscode/test/e2e/command-core.test.ts#L127
5) command-core.test.ts:106:3 › Explain Command from Prompts Tab ───────────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('option', { name: 'Explain Code' }) 125 | const sidebarTab = sidebarChat.getByTestId('tab-prompts') 126 | await sidebarTab.click() > 127 | await sidebarChat.getByRole('option', { name: 'Explain Code' }).click() | ^ 128 | 129 | // Click on a command from the sidebar should not start a new Editor window when sidebar is empty. 130 | await expect(sidebarChat.getByText('hello from the assistant')).toBeVisible() at /home/runner/work/cody/cody/vscode/test/e2e/command-core.test.ts:127:69
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L118
6) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements: 1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block') 2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' }) Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 116 | // Check the correct range item is auto-selected 117 | const rangeItem = page.getByText('Nearest Code Block') > 118 | expect(rangeItem).toBeVisible() | ^ 119 | 120 | // Open the range input and check it has the correct item selected 121 | await rangeItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:118:23
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L118
7) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements: 1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block') 2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' }) Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 116 | // Check the correct range item is auto-selected 117 | const rangeItem = page.getByText('Nearest Code Block') > 118 | expect(rangeItem).toBeVisible() | ^ 119 | 120 | // Open the range input and check it has the correct item selected 121 | await rangeItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:118:23
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L129
7) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for getByLabel('symbol-method fizzbuzz') - locator resolved to <div role="option" data-index="0" aria-level="1" aria-setsize="3" id="list_id_4_0" aria-posinset="1" draggable="false" data-parity="even" class="monaco-list-row" data-last-element="false" aria-label="symbol-method fizzbuzz, symbols">…</div> - attempting click action - waiting for element to be visible, enabled and stable - element is not visible - retrying click action, attempt #1 - waiting for element to be visible, enabled and stable 127 | await symbolitem.click() 128 | const selectedSymbolItem = page.getByLabel('symbol-method fizzbuzz') > 129 | await selectedSymbolItem.click() | ^ 130 | 131 | // Check that the range input updated correctly to reflect the selected symbol 132 | const inputBox = page.getByPlaceholder(/^Enter edit instructions \(type @ to include code/) at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:129:30
🎭 Playwright Run Summary
11 failed attribution.test.ts:20:1 › attribution search disabled in chat ───────────────────────────────── chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── command-custom.test.ts:283:17 › use terminal output as context ───────────────────────────────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── command-edit.test.ts:138:1 › edit (fixup) input - model selection ────────────────────────────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── 12 skipped 165 passed (50.0m)
🎭 Playwright Run Summary
7 failed chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ────────────────────────── command-core.test.ts:106:3 › Explain Command from Prompts Tab ────────────────────────────────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── 30 skipped 433 passed (1.8h)

Artifacts

Produced during runtime
Name Size
playwright-recordings ubuntu
4.66 MB
playwright-recordings windows
1.04 MB