e2e-flakiness-detector #248
Annotations
19 errors and 2 notices
auth.test.ts:21:3 › requires a valid auth token and allows logouts:
vscode\test\e2e\auth.test.ts#L30
1) auth.test.ts:21:3 › requires a valid auth token and allows logouts ────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByRole('alert').getByText('Authentication failed.')
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByRole('alert').getByText('Authentication failed.')
28 | await page.getByRole('combobox', { name: 'input' }).press('Enter')
29 |
> 30 | await expect(page.getByRole('alert').getByText('Authentication failed.')).toBeVisible()
| ^
31 |
32 | await sidebar?.getByRole('button', { name: 'Sign In to Your Enterprise Instance' }).click()
33 | await page.getByRole('option', { name: 'Sign In with URL and Access Token' }).click()
at D:\a\cody\cody\vscode\test\e2e\auth.test.ts:30:79
|
auth.test.ts:21:3 › requires a valid auth token and allows logouts:
vscode\test\e2e\auth.test.ts#L1
1) auth.test.ts:21:3 › requires a valid auth token and allows logouts ────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\requires_a_valid_auth_token_and_allows_logouts\temp-videos\520e28f9519efcf97356eeccc81e12b1.webm' -> 'D:\a\cody\cody\playwright\requires_a_valid_auth_token_and_allows_logouts\videos\run_0_retry_0_failure.webm'
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L365
2) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 4
+ Received + 1
- Array [
- "buzz.ts",
- "buzz.ts:2-5",
- ]
+ Array []
Call log:
- expect.toHaveText with timeout 2000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 0 elements
- locator resolved to 0 elements
- locator resolved to 0 elements
- locator resolved to 0 elements
363 |
364 | const [, lastChatInput] = await createEmptyChatPanel(page)
> 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], {
| ^
366 | timeout: 2_000,
367 | })
368 |
at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L1
2) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\2b1ddb28158cb52dd01fe278da49572f.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_1_retry_0_failure.webm'
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L365
3) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 4
+ Received + 1
- Array [
- "buzz.ts",
- "buzz.ts:2-5",
- ]
+ Array []
Call log:
- expect.toHaveText with timeout 2000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 0 elements
- locator resolved to 0 elements
- locator resolved to 0 elements
- locator resolved to 0 elements
363 |
364 | const [, lastChatInput] = await createEmptyChatPanel(page)
> 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], {
| ^
366 | timeout: 2_000,
367 | })
368 |
at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
|
chat-keyboard-shortcuts.test.ts:14:5 › chat keyboard shortcuts for sidebar chat:
vscode\test\e2e\chat-keyboard-shortcuts.test.ts#L25
4) chat-keyboard-shortcuts.test.ts:14:5 › chat keyboard shortcuts for sidebar chat ───────────────
Error: Timed out 3000ms waiting for expect(locator).toContainText(expected)
Locator: frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).first()
Expected string: "buzz.ts"
Received string: " "
Call log:
- expect.toContainText with timeout 3000ms
- waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).first()
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value " "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value " "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value " "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value " "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value " "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value " "
23 | await clickEditorTab(page, 'buzz.ts')
24 | await page.keyboard.press('Shift+Control+l')
> 25 | await expect(chatSidebarInput).toContainText('buzz.ts', { timeout: 3_000 })
| ^
26 |
27 | await executeCommandInPalette(page, 'View: Close Primary Sidebar')
28 |
at D:\a\cody\cody\vscode\test\e2e\chat-keyboard-shortcuts.test.ts:25:36
|
chat-keyboard-shortcuts.test.ts:14:5 › chat keyboard shortcuts for sidebar chat:
vscode\test\e2e\chat-keyboard-shortcuts.test.ts#L1
4) chat-keyboard-shortcuts.test.ts:14:5 › chat keyboard shortcuts for sidebar chat ───────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\chat_keyboard_shortcuts_for_sidebar_chat\temp-videos\f5586a3ccb5c182ef92e5f8ce3bda6ff.webm' -> 'D:\a\cody\cody\playwright\chat_keyboard_shortcuts_for_sidebar_chat\videos\run_1_retry_0_failure.webm'
|
command-custom.test.ts:228:3 › open and delete cody.json from the custom command menu:
vscode\test\e2e\helpers.ts#L381
5) command-custom.test.ts:228:3 › open and delete cody.json from the custom command menu ─────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceHTf92M after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceHTf92M\Crashpad\reports\da3dc5a2-aaea-465c-b9ea-32b0f0e1fe94.dmp'
at helpers.ts:381
379 | } catch (error) {
380 | if (attempts === 1) {
> 381 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
382 | }
383 |
384 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:381:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:262:13)
|
command-edit.test.ts:144:1 › edit (fixup) input - model selection:
vscode\test\e2e\command-edit.test.ts#L160
6) command-edit.test.ts:144:1 › edit (fixup) input - model selection ─────────────────────────────
Error: expect(locator).toBeVisible()
Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
158 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
159 | await nap()
> 160 | expect(modelItem).toBeVisible()
| ^
161 |
162 | // Open the model input and check it has the correct item selected
163 | await modelItem.click()
at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:160:23
|
command-edit.test.ts:106:1 › edit (fixup) input - range selection:
vscode\test\e2e\command-edit.test.ts#L124
7) command-edit.test.ts:106:1 › edit (fixup) input - range selection ─────────────────────────────
Error: expect(locator).toBeVisible()
Locator: getByText('Nearest Code Block')
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('Nearest Code Block')
122 | // Check the correct range item is auto-selected
123 | const rangeItem = page.getByText('Nearest Code Block')
> 124 | expect(rangeItem).toBeVisible()
| ^
125 |
126 | // Open the range input and check it has the correct item selected
127 | await rangeItem.click()
at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:124:23
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
1) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Test timeout of 30000ms exceeded.
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/common.ts#L179
1) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Error: locator.fill: Target page, context or browser has been closed
Call log:
- waiting for getByPlaceholder('Type the name of a command to run.')
- locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/>
- fill("buzz.ts")
- attempting fill action
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #1
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #2
- waiting 20ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #3
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #4
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #5
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #6
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #7
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #8
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #9
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #10
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #11
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #12
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #13
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #14
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #15
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #16
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #17
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #18
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #19
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #20
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
2) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Test timeout of 30000ms exceeded.
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/common.ts#L179
2) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Error: locator.fill: Target page, context or browser has been closed
Call log:
- waiting for getByPlaceholder('Type the name of a command to run.')
- locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/>
- fill("buzz.ts")
- attempting fill action
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #1
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #2
- waiting 20ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #3
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #4
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #5
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #6
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #7
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #8
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #9
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #10
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #11
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #12
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #13
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #14
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #15
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #16
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #17
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #18
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #19
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #20
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
|
command-core.test.ts:106:3 › Explain Command from Prompts Tab:
vscode/test/e2e/command-core.test.ts#L1
3) command-core.test.ts:106:3 › Explain Command from Prompts Tab ─────────────────────────────────
Test timeout of 30000ms exceeded.
|
command-core.test.ts:106:3 › Explain Command from Prompts Tab:
vscode/test/e2e/command-core.test.ts#L127
3) command-core.test.ts:106:3 › Explain Command from Prompts Tab ─────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('option', { name: 'Explain Code' })
125 | const sidebarTab = sidebarChat.getByTestId('tab-prompts')
126 | await sidebarTab.click()
> 127 | await sidebarChat.getByRole('option', { name: 'Explain Code' }).click()
| ^
128 |
129 | // Click on a command from the sidebar should not start a new Editor window when sidebar is empty.
130 | await expect(sidebarChat.getByText('hello from the assistant')).toBeVisible()
at /home/runner/work/cody/cody/vscode/test/e2e/command-core.test.ts:127:69
|
command-edit.test.ts:106:1 › edit (fixup) input - range selection:
vscode/test/e2e/command-edit.test.ts#L124
4) command-edit.test.ts:106:1 › edit (fixup) input - range selection ─────────────────────────────
Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements:
1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block')
2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' })
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('Nearest Code Block')
122 | // Check the correct range item is auto-selected
123 | const rangeItem = page.getByText('Nearest Code Block')
> 124 | expect(rangeItem).toBeVisible()
| ^
125 |
126 | // Open the range input and check it has the correct item selected
127 | await rangeItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:124:23
|
command-edit.test.ts:106:1 › edit (fixup) input - range selection:
vscode/test/e2e/command-edit.test.ts#L135
4) command-edit.test.ts:106:1 › edit (fixup) input - range selection ─────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for getByLabel('symbol-method fizzbuzz')
- locator resolved to <div role="option" data-index="0" aria-level="1" aria-setsize="3" id="list_id_4_0" aria-posinset="1" draggable="false" data-parity="even" class="monaco-list-row" data-last-element="false" aria-label="symbol-method fizzbuzz, symbols">…</div>
- attempting click action
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- element is not visible
- retrying click action, attempt #1
- waiting for element to be visible, enabled and stable
133 | await symbolitem.click()
134 | const selectedSymbolItem = page.getByLabel('symbol-method fizzbuzz')
> 135 | await selectedSymbolItem.click()
| ^
136 |
137 | // Check that the range input updated correctly to reflect the selected symbol
138 | const inputBox = page.getByPlaceholder(/^Enter edit instructions \(type @ to include code/)
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:135:30
|
test-e2e (ubuntu)
Process completed with exit code 1.
|
🎭 Playwright Run Summary
13 failed
auth.test.ts:21:3 › requires a valid auth token and allows logouts ─────────────────────────────
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-keyboard-shortcuts.test.ts:14:5 › chat keyboard shortcuts for sidebar chat ────────────────
command-custom.test.ts:228:3 › open and delete cody.json from the custom command menu ──────────
command-edit.test.ts:144:1 › edit (fixup) input - model selection ──────────────────────────────
command-edit.test.ts:106:1 › edit (fixup) input - range selection ──────────────────────────────
command-edit.test.ts:106:1 › edit (fixup) input - range selection ──────────────────────────────
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
12 skipped
163 passed (50.5m)
|
🎭 Playwright Run Summary
4 failed
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ──────────────────────────
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ──────────────────────────
command-core.test.ts:106:3 › Explain Command from Prompts Tab ──────────────────────────────────
command-edit.test.ts:106:1 › edit (fixup) input - range selection ──────────────────────────────
30 skipped
436 passed (1.8h)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
playwright-recordings ubuntu
|
3.18 MB |
|
playwright-recordings windows
|
1.06 MB |
|