…rc#485)

* Add an exclude list for known failing Hifive1 tests

This commit adds a list of known failing tests based on: riscv-collab/riscv-openocd#869 (comment)

* Fix name of the HiFive1 flash target 

Signed-off-by: Marek Vrbka <[email protected]>

---------

Signed-off-by: Marek Vrbka <[email protected]>

flushregs is deprecated.

…shregs

debug: flushregs -> maintenance flush register-cache

Just so it's easier to quickly comment out code and hard-code the target
to use without pylint complaining. This really should be a command line
option.

It's a bit messy to read the log file to get the output, but it seems to
be flushed often so that this works.

Also, added the `targets` method for retrieving the list of targets,
and `wait_until_running` method to wait until all targets are in a
running state.

Use the new spike mechanism to test OpenOCD behavior when a hart becomes
unavailable while running.

Create CommandException.

Use new spike mechanism to test OpenOCD behavior when the current hart
becomes unavailable while running.

Create ThreadTerminated exception.

Use new spike mechanism to test OpenOCD behavior when a hart becomes
unavailable, and then available again.

…er_dance

debug: Test OpenOCD behavior when harts become unavailable, using new spike mechanism

They have issues when run in a github workflow.

…able_unavailable

debug: Disable unavailable tests.

Just doing this to make a change in the debug files, which should now
cause the pylint workflow to execute.

…int_workflow

debug: Only run pylint if debug files changed.

This should avoid problems like we just had where bad tests can break
the OpenOCD workflow. These tests only run if any debug files are
changed, so should have no impact at all on non-debug tests in this
repo.

This file is copied and then slightly changed from riscv-openocd. New
changes are that cacheable steps (building spike, OpenOCD) are stored to
the cache even if running the tests fails.

…ug_workflow

debug: Actually run tests in github workflow.

During the github workflow this character is \n, while on my computer
it's ' '. I'm sure there's a good reason for that, but it doesn't seem
worth figuring out what that reason is.

…vailable

debug: Re-enable unavailable tests, and fix them for github

This lets you reproduce a test running on a specific hart.

Actually wait for the command to be echoed back. This means we won't be
confused if there are extra newlines in gdb output.

…ug_hart

debug: Add --hart command line option to gdbserver.py

…errupt_all

debug: Fix interrupt_all() to restore state.

…gets

debug: Make Openocd.targets() tolerate blank lines.

This helper uses dmi_write commands to mark harts
available/unavailable.

…erlock

debug: Better interlock when interacting with gdb CLI.

…_available

debug: Add Openocd.set_available()

Signed-off-by: liangzhen <[email protected]>

Disable timer interrupt to fix some bugs

Resolves riscv-software-src#510.

…pect

debug: Document that pexpect is needed.

Test behavior when a hart becomes unavailable while halted.

Signed-off-by: liangzhen <[email protected]>

Signed-off-by: liangzhen <[email protected]>

…vailable_halted2

debug: Add UnavailableHaltedTest

Make CLINT address configurable

Make the non-existent csr configurable

This is taking into account that the hardware limits count to 1.

Signed-off-by: liangzhen <[email protected]>

Support limits_icount_to_one in IcountTest

…rsing log file

Quick and dirty fix for riscv-software-src#520

…ble_fixup

debug: use TCL-RPC to fetch results of OpenOCD commands instead of parsing log file

Signed-off-by: liangzhen <[email protected]>

Uses appropriate addi instruction in lrsc test.

Previously the seed was not printed and this created problems with
reproduction of the issues. It's still not an ideal - meaning
interactions between spike/gdb/openocd are inherently non-determistic
(since time is involved), but at least we should get the same sources
for the same seed now.

introduce a new option to log communications over GDB remote serial
protocol which is helpful for debugging some tests.

fixes setting of `remotetimeout`. It was silently overwritten by default
values from platform definition even if user specified one.

Signed-off-by: liangzhen <[email protected]>
Change-Id: Ida1490338d204541c5c7f143aec3b8d79d83d7f4

Signed-off-by: liangzhen <[email protected]>
Change-Id: I7a4a24972cfa2ddc307a5f06fe3fd5380794719f

If Svnapot is not implemented, a page fault will occur when accessing a page with napot specified.
In this case, let the test pass.

If Svnapot is not implemented, skip the test.

Signed-off-by: liangzhen <[email protected]>
Change-Id: Iac914aef8080411e6acd9039c4bdfa728533103c

Signed-off-by: Roger Chang <[email protected]>

Signed-off-by: Roger Chang <[email protected]>

Signed-off-by: Roger Chang <[email protected]>

Signed-off-by: Roger Chang <[email protected]>

Add bitmanip test cases

See https://github.com/riscv/riscv-debug-spec/blob/f510a7dd33317d0eee0f26b4fa082cd43a5ac7ea/Sdtrig.tex#L213-L214

Restore $stap and $mstatus after test translate

Add virtual memory synchronization after completing the page tables

…ents

 improvements to debug tests infrastructure to help with triaging process

Clear breakpoints so that gdb will not single step

Signed-off-by: 梁镇 <[email protected]>
Change-Id: I4657a0417b79d515655f6ad4a5ba4465ca58061f

Check the mcontrol triggers, no other triggers.

…cv-software-src#553)

Spike simulator is very demanding to CPU resources. This causes debug
tests to sporadically fail on slower machines. Increasing of gdb's
`remotetimeout` should get rid of such failures, unless we run the
testsuite on a potato.

The only downside is that if OpenOCD is broken, tests can run longer.
However, I think this is the sacrifice we can make, since execution time
is not affected if everything works as expected.

This patch fixes the case when we are using an empty exception list (for example just a YAML file with comments but without any test items to skip).

…n-fix

debug: Fix loading of empty exclude lists with comments

…tu-20.04

Use Ubuntu 20.04 for debug tests

debug: Fix nonexistent trigger registers trap handle in entry.S

…peat-read

Remove old warning check in RepeatReadTest

…ed data present in pexpect match

Problem was observed on UnavailableMultiTest - this test was sporadically failing.
When the failure was observed the log of the failing test looked as follows:

```
  File "/whatever/RISCVTests/debug/testlib.py", line 504, in <genexpr>
    if all(targets[hart.id]["State"] == "running" for hart in harts):
           ~~~~~~~~~~~~~~~~^^^^^^^^^
KeyError: 'State'
```

Adding this modification to testlib.py

```
--- a/debug/testlib.py
+++ b/debug/testlib.py
@@ -498,6 +498,10 @@ class Openocd:
         for line in lines[2:]:
             if line.strip():
                 data.append(dict(zip(headers, line.split()[1:])))
+        str_data = str(data)
+        sys.stdout.flush()
+        sys.stdout.write(f"parsed targets:\n{result}\n===\n{str_data}\n---\n")
+        sys.stdout.flush()
         return data
```

Allowed me to root cause the issue. Namely we have the following
situation:

```
parsed targets:
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f2dee64d1c0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/tempfile.py", line 450, in __del__
    self.close()
  File "/usr/local/lib/python3.11/tempfile.py", line 446, in close
    unlink(self.name)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/[email protected]'
...
    TargetName         Type       Endian TapName            State
--  ------------------ ---------- ------ ------------------ ------------
 0  riscv.cpu0         riscv      little riscv.cpu          running
 1* riscv.cpu1         riscv      little riscv.cpu          running
===
[{'Exception': '"/usr/local/lib/python3.11/tempfile.py",', 'ignored': 'line', 'in:': '450,', ...
```

The only reasonable (to me) explanation for the observed behavior is below.

Here is how we connect to TCL-RPC server:

```
  self.openocd_cli = pexpect.spawn(f"nc localhost {self.tclrpc_port}")
  tty.setraw(self.openocd_cli.child_fd)
```

Later we request target list by issuing "targets" command:

```
  self.command("targets")
```

Internally, pexpect.spawn implemented as follows:

- we fork the process
- set up pty and then call execve
- all these steps are written in python

"Exception ignored" messages are result of exceptions thrown from
finalizers of NamedTemporaryFile objects. When exception is thrown from
the finalizer - python unconditionally prints a "warning" to stderr. It
seems that these messages are polluting our output from "nc" since python
GC can be invoked before the execve syscall.

The workaround is just to make sure that execve was executed before we
rely on the format of command output. To have such a guarantee we just
issue a dummy "echo" command and check that we have a proper reply in the
output stream.

While this explanation looks convincing, the behavior above still looks
strange, given that we have https://bugs.python.org/issue14548 which
was resolved long ago.

However, the proposed workaround fixes the issue.

Memory sampling tests fail sporadically for spike targets. A typical
failure looks as follows (ROI from test log):

```
---------------------------------[ Message ]----------------------------------
139670831 not less than 124104544
--------------------------------[ Traceback ]---------------------------------
    ... SECTION IS SKIPPED FOR READABILITY ...
    raise TestFailed(f"{a!r} not less than {b!r}", comment)
testlib.TestFailed
```

Few observations:

- 139670831 is 0x0853352f in hex, while 124104544 is 0x0765af60
- Now, the assert which is failing corresponds to the following
  expression:

```
  assertLess(value, previous_value + tolerance)
```

- tolerance is `0x500000`. (124104544 - 0x500000) is 0x0715af60

If we look at the sampling output for such failing test, we'll see:

```
...
0x1212340c5c: 0x0715af60
timestamp after: 878087500
timestamp before: 878088133
0x1212340c5c: 0x0853352f
...
```

The log above demonstrates the reason for the failure. Since memory
sampling occures every poll (which by default happens approximately
every 100ms) a value of the counter may exceed the threshold if the time
between subsequent polls is increased (for whatever reason).

In my opinion the failing assert can be safely removed, since the checks
it perform are quite brittle and cannot be generalized. The assert
violation is affected by CPU performance and sporadic delays between
polls.

For now, instead of assert removal we just avoid checks in-between
memory sample bursts. This way we still can be certain that memory
samples are frequent enough and hopefully this will avoid sporadic
failures.

…_sporadic_failure

debug: fix sporadic failures of memory sampling tests

…_faulure_fix

debug: workaround for sporadic failures of some tests due to unexpected data present in pexpect match

HwbpManual test was broken:
* Value read back from `tselect` was compared with `tdata1` value.
https://github.com/riscv-software-src/riscv-tests/blob/408e461da11e0b298c4b69e587729532787212f5/debug/gdbserver.py#L701-L703
This resulted in the test being reported as not supported, after all the
triggers were checked.
* `tdata1.type` field was not set to `mcontrol`.
* `tselect` value used to be changed by `handle_reset` and not restored.
https://github.com/riscv-software-src/riscv-tests/blob/408e461da11e0b298c4b69e587729532787212f5/debug/programs/entry.S#L79-L84
* Manual breakpoint used to be left behind.

Signed-off-by: Evgeniy Naydanov <[email protected]>

By using Zca-friendly registers, we can reuse the existing tests to get
quick-and-dirty coverage of Zca, when the assembler is told to use Zca.
(This doesn't break non-Zca targets.)

Continuation of fee361f

debug: fix HwbpManual test

… see riscv-software-src#531 (comment)

…ib.py:run_all_tests()

…LEN (riscv-software-src#566)

…software-src#567)

…list-tests

Debug: suppress `PRNG seed ...` log messages when `gdbserver.py --list-tests <target>` used

…eef (riscv-software-src#565)

Signed-off-by: Daniel Maslowski <[email protected]>

…-software-src#572)