Skip to content

e2e-flakiness-detector #243

e2e-flakiness-detector

e2e-flakiness-detector #243

Triggered via schedule November 7, 2024 14:04
Status Failure
Total duration 1h 52m 13s
Artifacts 2
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

15 errors and 2 notices
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L365
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 4 + Received + 1 - Array [ - "buzz.ts", - "buzz.ts:2-5", - ] + Array [] Call log: - expect.toHaveText with timeout 2000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements 363 | 364 | const [, lastChatInput] = await createEmptyChatPanel(page) > 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], { | ^ 366 | timeout: 2_000, 367 | }) 368 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L1
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\7fcdca28c95ff0dcd0102adf9376afa3.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_1_retry_0_failure.webm'
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L365
2) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 2000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 4 + Received + 1 - Array [ - "buzz.ts", - "buzz.ts:2-5", - ] + Array [] Call log: - expect.toHaveText with timeout 2000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 0 elements 363 | 364 | const [, lastChatInput] = await createEmptyChatPanel(page) > 365 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5'], { | ^ 366 | timeout: 2_000, 367 | }) 368 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:365:52
command-custom.test.ts:141:3 › execute custom commands with context defined in cody.json: vscode\test\e2e\command-custom.test.ts#L198
3) command-custom.test.ts:141:3 › execute custom commands with context defined in cody.json ────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByRole('tab', { name: 'var.go' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByRole('tab', { name: 'var.go' }) 196 | .getByRole('button', { name: withPlatformSlashes('var.go:1 lib/batches/env') }) 197 | .click() > 198 | await expect(page.getByRole('tab', { name: 'var.go' })).toBeVisible() | ^ 199 | 200 | /* Test: context.openTabs with openTabs command */ 201 | at D:\a\cody\cody\vscode\test\e2e\command-custom.test.ts:198:61
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode\test\e2e\command-edit.test.ts#L118
4) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect(locator).toBeVisible() Locator: getByText('Nearest Code Block') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 116 | // Check the correct range item is auto-selected 117 | const rangeItem = page.getByText('Nearest Code Block') > 118 | expect(rangeItem).toBeVisible() | ^ 119 | 120 | // Open the range input and check it has the correct item selected 121 | await rangeItem.click() at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:118:23
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode\test\e2e\command-edit.test.ts#L118
5) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByText('Nearest Code Block') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 116 | // Check the correct range item is auto-selected 117 | const rangeItem = page.getByText('Nearest Code Block') > 118 | expect(rangeItem).toBeVisible() | ^ 119 | 120 | // Open the range input and check it has the correct item selected 121 | await rangeItem.click() at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:118:23
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error: vscode\test\e2e\enterprise-cody-ignore.test.ts#L1
6) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error Test timeout of 30000ms exceeded.
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error: vscode\test\e2e\enterprise-cody-ignore.test.ts#L1
6) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\using_actively_invoked_commands_and_autocomplete_shows_a_error\temp-videos\19ba78fb963b7eca972da9c7f7638f57.webm' -> 'D:\a\cody\cody\playwright\using_actively_invoked_commands_and_autocomplete_shows_a_error\videos\run_2_retry_0_failure.webm'
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error: vscode\test\e2e\enterprise-cody-ignore.test.ts#L73
6) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error Error: locator.hover: Target page, context or browser has been closed Call log: - waiting for getByRole('button', { name: 'ignored' }) 71 | name: 'ignored', //TODO(rnauta): migrate this test to e2ev2 and use StatusBar tags 72 | }) > 73 | await statusBarButton.hover() | ^ 74 | await expect(statusBarButton).toBeVisible() 75 | 76 | // Clicking on the Cody icon shows a message at D:\a\cody\cody\vscode\test\e2e\enterprise-cody-ignore.test.ts:73:31
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled: vscode\test\e2e\enterprise-server-sent-models.test.ts#L27
7) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('combobox', { name: 'Select a model' }).last() Expected pattern: /^Titan/ Received string: "Opus" Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('combobox', { name: 'Select a model' }).last() - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_rqosg_5 _button--secondary_rqosg_54 _button--no-icon-start_rqosg_45">…</button> - unexpected value "Opus" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_rqosg_5 _button--secondary_rqosg_54 _button--no-icon-start_rqosg_45">…</button> - unexpected value "Opus" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_rqosg_5 _button--secondary_rqosg_54 _button--no-icon-start_rqosg_45">…</button> - unexpected value "Opus" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_rqosg_5 _button--secondary_rqosg_54 _button--no-icon-start_rqosg_45">…</button> - unexpected value "Opus" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_rqosg_5 _button--secondary_rqosg_54 _button--no-icon-start_rqosg_45">…</button> - unexpected value "Opus" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_rqosg_5 _button--secondary_rqosg_54 _button--no-icon-start_rqosg_45">…</button> - unexpected value "Opus" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_rqosg_5 _button--secondary_rqosg_54 _button--no-icon-start_rqosg_45">…</button> - unexpected value "Opus" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r9:" aria-label="Select a model" data-testid="chat-model-selector" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
1) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Test timeout of 30000ms exceeded.
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/common.ts#L179
1) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Error: locator.fill: Target page, context or browser has been closed Call log: - waiting for getByPlaceholder('Type the name of a command to run.') - locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/> - fill("buzz.ts") - attempting fill action - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #1 - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #2 - waiting 20ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #3 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #4 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #5 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #6 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #7 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #8 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #9 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #10 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #11 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #12 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #13 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #14 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #15 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #16 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #17 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #18 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #19 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #20 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L118
2) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements: 1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block') 2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' }) Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 116 | // Check the correct range item is auto-selected 117 | const rangeItem = page.getByText('Nearest Code Block') > 118 | expect(rangeItem).toBeVisible() | ^ 119 | 120 | // Open the range input and check it has the correct item selected 121 | await rangeItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:118:23
command-edit.test.ts:100:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L118
3) command-edit.test.ts:100:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements: 1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block') 2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' }) Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 116 | // Check the correct range item is auto-selected 117 | const rangeItem = page.getByText('Nearest Code Block') > 118 | expect(rangeItem).toBeVisible() | ^ 119 | 120 | // Open the range input and check it has the correct item selected 121 | await rangeItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:118:23
test-e2e (ubuntu)
Process completed with exit code 1.
🎭 Playwright Run Summary
13 failed chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── command-custom.test.ts:141:3 › execute custom commands with context defined in cody.json ─────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled initial-context.test.ts:83:82 › chat try-again actions ───────────────────────────────────────── support-menu-items.test.ts:9:1 › shows support link for free users ───────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── 12 skipped 163 passed (50.0m)
🎭 Playwright Run Summary
3 failed chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ────────────────────────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── command-edit.test.ts:100:1 › edit (fixup) input - range selection ────────────────────────────── 30 skipped 437 passed (1.8h)

Artifacts

Produced during runtime
Name Size
playwright-recordings ubuntu
2.18 MB
playwright-recordings windows
2.46 MB