Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate buildroot and add support for QEMU Linux target #667

Merged
merged 5 commits into from
Mar 20, 2024

Conversation

metin-arm
Copy link
Contributor

@metin-arm metin-arm commented Jan 23, 2024

Commits:

  • 918c19c - utils/ssh: Try to free up resources during client creation
  • 3b7fb5b - target: Introduce make_temp() for creating temp file/folder on target
  • 1253903 - tests/test_target: Test more targets
  • 82a722d - tools/buildroot: Add support for generating Linux target system images
  • a4b3547 - target: Implement target runner classes

devlib/target.py Outdated Show resolved Hide resolved
devlib/target.py Outdated
if not os.path.exists(kernel_image):
raise FileNotFoundError(f'{kernel_image} does not exist!')

qemu_cmd = f'sudo {qemu_path} -kernel {kernel_image} -append "{cmdline}" ' \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is sudo really required here ? I don't need it on my laptop when running qemu

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does if (id -nG "$USER" | grep -qw "kvm"); then echo "$USER is in kvm group"; else echo "$USER is not in kvm group"; fi print for you? I guess you are in kvm group and able to access /dev/kvm.
I'm working on another task such as Android emulator requires the user to be in kvm group.
Maybe I should remove this sudo and mention this requirement in doc/target.rst?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you are in kvm group and able to access /dev/kvm.

Indeed. I tried to have a peek at how this is typically configured (e.g. on Debian) and it's quite complicated: https://unix.stackexchange.com/questions/599651/whats-the-purpose-of-kvm-libvirt-and-libvirt-qemu-groups-in-linux
Basically with systemd (well, logind) you also get the permission added to you if you logged in on a console even if you are not part of the kvm group. Considered all that, maybe we should remove the sudo and let the user enable kvm, which seems to be automatically done in a typical desktop system.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Removed sudo already.

devlib/target.py Outdated
if not os.path.exists(kernel_image):
raise FileNotFoundError(f'{kernel_image} does not exist!')

qemu_cmd = f'sudo {qemu_path} -kernel {kernel_image} -append "{cmdline}" ' \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating a shell command requires proper escaping of the values. This can be achieved with shlex.quote()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Will fix it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shlex.quote() converts CPU list to '-cpu cortex-a72 -cpu cortex-a53' and QEMU is unhappy with those '.

Copy link
Collaborator

@douglas-raillard-arm douglas-raillard-arm Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you use it ? You need to apply it to each parameter individually, otherwise it will add escaping so that the entire string is a single argument. That's partly why it's not recommended to use shell=True in subprocess. It's typically a much better idea to simply pass an iterable, which won't require any escaping at all.

EDIT: each parameter that is a user input. -cpu does not need quoting.

devlib/target.py Outdated Show resolved Hide resolved
devlib/target.py Outdated Show resolved Hide resolved
devlib/target.py Outdated Show resolved Hide resolved
tests/spin_targets.py Outdated Show resolved Hide resolved
ll_target.remove(ll_target.working_directory)

q_target = QEMULinuxTarget(kernel_image='/home/user/devlib/tools/buildroot/buildroot-v2023.11.1/output/images/Image',
connection_settings={'host': '127.0.0.1',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think their is a way to infer automatically the connection settings (IP/port) ? After all, devlib launched QEMU so we shouldn't have to tell it how to reach it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

devlib launches QEMU, but the kernel / rootfs image may be built externally.
The only interesting bit of connection_settings is 'strict_host_check': False. Rest of the parameters can be inferred from default settings.

tools/buildroot/generate-kernel-initrd.sh Outdated Show resolved Hide resolved
devlib/target.py Outdated
@@ -2892,6 +2895,99 @@ def _resolve_paths(self):
if self.executables_directory is None:
self.executables_directory = '/tmp/devlib-target/bin'

class QEMULinuxTarget(LinuxTarget):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's really an issue but I was wondering if we really want to make that QEMU runner a subclass of LinuxTarget. If we do that, we can easily end up with a combinatorial explosion of the number of subclasses with just a few "modifier" classes like that, and the naming nightmare that goes with it.

Maybe a better approach to the problem is to make that QEMU thingy a separate class (e.g. let's say a TargetRunner class with a QEMUTargetRunner subclass), and then just pass the target class to use as a parameter. The created target can be accessed as a .target attribute. This way we also avoid any possible clash between methods defined here and current + future methods of Target classes.


"""
Module for testing different targets.
Forked from tests/test_target.py.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@douglas-raillard-arm , I think spin_targets.py now can replace test_target.py. What do you think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I don't think anyone really cared about test_target.py, it was an embryo of tests that never got fleshed out

@metin-arm metin-arm force-pushed the staging branch 4 times, most recently from d518e70 to 28ac822 Compare January 25, 2024 11:33
@setrofim
Copy link
Collaborator

This should probably be implemented as a Platform rather than a Target, as this handles "hardware" rather than OS-level functionality; And that way, the same code can be used for Android targets running on QEMU as well.

@douglas-raillard-arm
Copy link
Collaborator

@setrofim I'm not sure platform would be appropriate either, we want an object that starts its life before the Target and survives after. Platform seems to be more for managing platform-specific interaction with a device (such as rebooting) more than spawning the device itself.

@setrofim
Copy link
Collaborator

we want an object that starts its life before the Target and survives after

A Platform is passed a a parameter on Target creation, so it is already assumed to exist before the Target (only if one is not provided, the Target will then create a "default" one). And the Target doesn't do anything to "terminate" a Platform, so there is no reason why the latter can't persist after the former stopped existing.

Platform seems to be more for managing platform-specific interaction with a device (such as rebooting) more than spawning the device itself.

Only because a device is not normally "spawned". However the interactions can include the initial target setup prior to execution (e.g. flashing images -- see VExpress platform). Support for Gem5 simulation (conceptually similar to QEMU) is implemented as a Platform and that includes optionally starting the simulation.


"""
Module for testing different targets.
Forked from tests/test_target.py.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I don't think anyone really cared about test_target.py, it was an embryo of tests that never got fleshed out

devlib/target.py Outdated
Comment on lines 2906 to 2910
banner = None
boot_timeout = None
runner_cmd = None
target = None
connection_settings = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all of those are class attributes and as such should be immutable (as class attributes are singletons), and therefore have capitalized names as per PEP8.
However, I think most of them should not be a class attribute and just be __init__ parameters instead

devlib/target.py Outdated
target = None
connection_settings = None

def __init__(self):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing parameters to super().__init__() implicitly via attributes is not a good idea. It doesn't bring anything and obfuscates the flow of information in the code. It's especially bad in a language like Python where you can very easily call super().__init__() without having set an attribute it relies on, leading to an AttributeError.

devlib/target.py Outdated
except Exception as ex:
raise HostError(f'Error while running "{self.runner_cmd}": {ex}') from ex

atexit.register(self.terminate_runner)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make TargetRunner a context manager and call that from __exit__()

devlib/target.py Outdated

self.wait_boot_complete()

self.target.connect(check_boot_completed=False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What fails if we leave check_boot_complete=True (which I assume is the default ?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, default is True. Nothing fails since previous line (self.wait_boot_completed()) already waits for boot completion.
Leaving check_boot_completed as True just causes an extra function call.
I'll revisit this after addressing #667 (comment).

'test3': '3\n\n4\n\n',
}

print(f'{method_name}: target={target.__class__.__name__} os={target.os} ' \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally it's better to avoid breaking a string literal, as this prevents from using grep easily when looking for the source of a given message

dirname = os.path.join(target.working_directory,
f'devlib-test-{get_random_string(12)}')
print(f'{method_name}: creating {dirname}...')
target.makedirs(dirname)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed before, this should go in a context manager as a Target method

raw_result = target.read_tree_values_flat(dirname)
result = {os.path.basename(k): v for k, v in raw_result.items()}
except Exception as ex:
raise TargetStableError('Error while processing the tree!') from ex
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this adds much, the original exception will have all the details we need to debug.

target.remove(target.working_directory)

def create_targets():
targets = []
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this whole function could just be:

return [
    AndroidTarget(connection_settings={'device': 'emulator-5554'},
                                     working_directory='/data/local/tmp/devlib-target'),
...
]

instead of loads of append() calls

method_name = self.__class__.test_read_multiline_values.__qualname__

targets = create_targets()
for target in targets:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's probably good enough for now but when we write more tests, we probably don't want to create a new set of target every time, so we will want to use the unittest infrastructure to keep those alive. Or maybe not if it stinks too much, let's see later

@metin-arm metin-arm force-pushed the staging branch 4 times, most recently from 244abe0 to e9e48b1 Compare February 5, 2024 16:04
@metin-arm metin-arm force-pushed the staging branch 6 times, most recently from 125abbf to 0504f1c Compare February 8, 2024 08:27
@metin-arm metin-arm force-pushed the staging branch 4 times, most recently from 84050ed to e044b4c Compare February 12, 2024 14:52
@metin-arm metin-arm force-pushed the staging branch 2 times, most recently from 13308a6 to a3d58e3 Compare February 23, 2024 15:40
@@ -1071,6 +1072,38 @@ async def write_value(self, path, value, verify=True, as_root=True):
else:
raise

@contextmanager
def make_temp(self, is_directory=True, directory='', prefix='devlib-test'):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Would path be a more descriptive parameter name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it's working directory?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my mind the function is designed to create a new temp directory/file under the path provided from the user. The default location is just set to the working directory.

Having said this, after checking the documentation for tempfile.mkstemp I see they use dir as parameter name so not a strong opinion on this.

@@ -37,23 +36,57 @@ def build_targets():

targets = []

if target_configs.get('AndroidTarget') is not None:
print('> Android targets:')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be using logging within these tests rather then print statements?

Copy link
Contributor Author

@metin-arm metin-arm Feb 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are dummy prints. Not sure if it's worth. Please let me know if you have a strong opinion to use logger.
Also pytest does not show any outputs unless -s parameter is passed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point, if they are not shown by default this should be ok.

print(f'target={target.__class__.__name__} os={target.os} hostname={target.hostname}')

with target.make_temp() as tempdir:
print(f'created {tempdir}.')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Capitalization and same for prints below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -1,5 +1,30 @@
AndroidTarget:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if it would be worth renaming this file to be target_configs_exmaple or something similar? This would allow us to provide a default file that only contains entries for configurations that work out of the box i.e. so that a default run of pytest doesn't require manually starting the android emulators etc. as well as providing the full information?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a default run of pytest doesn't require manually starting the android emulators

I'm unsure how renaming this file can help in automating Android emulators.
run_tests.sh may be a good reference for how to run emulators, example config file, etc.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I didn't mean it would help in automating android emulators.

I was thinking that for a user who just wants to check out devlib and run the tests to validate some changes they made, it could be useful to provide a "default" config that just works out of the box with no additional setup required. (I'm thinking going forward if we have a greater set of tests to run).

The "example" version would then provide details of all the available options including the android emulators that requrie manually starting to provide a more comprehensive testing setup.

QEMUTargetRunner:
entry-0:
qemu_params:
kernel_image: '/devlib/tools/buildroot/buildroot-v2023.11.1-aarch64/output/images/Image'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we were to provide this file as an example, would it be worth changing this slightly to something to make it more obvious to a user that it needs to be configured to their own location? e.g. /path/to/devlib/tools...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

{
if [[ "${1}" == "aarch64" || "${1}" == "x86_64" ]]; then
ARCH=${1}
KERNEL_IMAGE_NAME="bzImage"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The help lists aarch64 as the default architecture however if a user doesn't explicitly set the value to aarch64 then this generates a different file name to what is expected by the test scripts.

Could we set ARCH variable before checking this condition for consistency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Thanks.

@@ -0,0 +1,118 @@
#!/usr/bin/env bash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there scope for listing the required packages to run this somewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dockerfile may give an idea about the requirements, but no, I don't have an explicit list of necessary packages.

devlib/target.py Outdated
@@ -2893,6 +2928,249 @@ def _resolve_paths(self):
self.executables_directory = '/tmp/devlib-target/bin'


class TargetRunner:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Would it be worth splitting the TargetRunner related code out into a separate file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, makes sense.

devlib/target.py Outdated
QEMU performance.
:type qemu_params: Dict

:param connection_settings: he dictionary which stores connection settings
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Typo he dictionary -> Dict to store..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Thanks.

devlib/target.py Outdated
}

# Update default QEMU parameters with :param:`qemu_params`.
qemu_default_args.update(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Would making qemu_default_args to just be named qemu_args be a bit clearer in the following code that it is taking in account the user provided information as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

SshConnection._make_client() may throw exceptions for several reasons
(e.g., target is not ready yet). The client should be closed if that is
the case. Otherwise Python unittest like tools report resource warning
for 'unclosed socket', etc.

Signed-off-by: Douglas Raillard <[email protected]>
Signed-off-by: Metin Kaya <[email protected]>
``Target.make_temp()`` employs ``mktemp`` command to create a temporary
file or folder.

This method will be used in unit tests.

Signed-off-by: Metin Kaya <[email protected]>
Test Android and Linux targets as well in addition to LocalLinux target.
In order to keep basic verification easy, list complete list of test
targets in tests/target_configs.yaml.example and keep the default
configuration file for targets simple.

Also:
- Create a test folder on target's working directory.
- Remove all devlib artefacts after execution of the test.
- Add logs to show progress of operations.

Signed-off-by: Metin Kaya <[email protected]>
Integrate buildroot into devlib in order to ease building kernel and
root filesystem images via 'generate-kernel-initrd.sh' helper script.

As its name suggests, the script builds kernel image which also includes
an initial RAM disk per default config files located under
configs/<arch>/.

Provide config files for buildroot and Linux kernel as well as a
post-build.sh script which tweaks (e.g., allowing root login on SSH)
target's root filesystem.

doc/tools.rst talks about details of kernel and rootfs configuration.

Signed-off-by: Metin Kaya <[email protected]>
Add support for launching emulated targets on QEMU. The base class
``TargetRunner`` has groundwork for target runners like
``QEMUTargetRunner``.

``TargetRunner`` is a contextmanager which starts runner process (e.g.,
QEMU), makes sure the target is accessible over SSH (if
``connect=True``), and terminates the runner process once it's done.

The other newly introduced ``QEMUTargetRunner`` class:
- performs sanity checks to ensure QEMU executable, kernel, and initrd
  images exist,
- builds QEMU parameters properly,
- creates ``Target`` object,
- and lets ``TargetRunner`` manage the QEMU instance.

Also add a new test case in ``tests/test_target.py`` to ensure devlib
can run a QEMU target and execute some basic commands on it.

While we are in neighborhood, fix a typo in ``Target.setup()``.

Signed-off-by: Metin Kaya <[email protected]>
@marcbonnici marcbonnici merged commit 228baeb into ARM-software:master Mar 20, 2024
@metin-arm metin-arm deleted the staging branch March 22, 2024 09:10
@@ -0,0 +1,267 @@
# Copyright 2024 ARM Limited
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to expose that module publicly. This is certainly not needed for the "normal devlib operation", and for now is really only needed for testing devlib/WA/LISA. These 3 projects could just use a private module on a best-effort basis without having to fully commit in front of the entire world to backward compatibility in any of these features.

This is especially important given that android-related things tend to change on a regular basis, and that overall this emulation business is pretty complex, so we may want at some point to use it in a different way, or switch to a different backend (e.g. use VirtualBox) or whatever. We don't want to be constrained by backward compat when doing such changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Creates temporary file/folder on target and deletes it once it's done.

:param is_directory: Specifies if temporary object is a directory, defaults to True.
:type is_directory: bool, optional
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This , optional part is not recognized by Sphinx and results in a broken class reference:

foobar.py:1: WARNING: py:class reference target not found: optional

The typical way an optional parameter is implemented in Python is by using None as default value. It follows that the type to give with Sphinx would become bool or None.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do see , optional on https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html. Is that wrong doc for Sphinx?
How do you verify docstring?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems to be the right doc, but I think that either the doc is wrong or it's actually the doc on how to contribute to Sphinx itself, which may use something else than plain autodoc. That syntax is part of the numpy style, which requires the napoleon Sphinx extension. Apparently that napoleon extension pre-processes the docstring before autodoc sees it.

How do you verify docstring?

You build the doc and check for warnings (lisa-doc-build for LISA). I tried by modifying a docstring in LISA. Since devlib's doc is not configured with autodoc, building the doc won't do anything as nothing will touch those docstrings.

Either way, if the function accepts a type T or None (which is the standard way of making a parameter "optional" in the common understanding of the term[1]), then it's actually more accurate to state T or None.

[1] You can make a parameter actually optional without any default value with the variable keyword argument syntax, but that's definitely not something you'd do on a daily basis.

Copy link
Collaborator

@douglas-raillard-arm douglas-raillard-arm Mar 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You build the doc and check for warnings

Another way would be to setup a test sphinx project, which is very easy too. Or port devlib to use autodoc. It's not particularly difficult, just an invasive PR (all existing function documentations need to be moved back as docstrings). If you were to do that, it would probably take ~1week (or maybe less) and I think it would be reasonable to as @marcbonnici to freeze merging any other stuff so you don't end up with conflicts while working on it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will not go for updating entire devlib repo now :)
Would you be OK if I just remove , optional part from docstring? Otherwise, it does not seem trivial to verify docstrings' Sphinx compatibility in devlib.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes just go for replacing , optional by or None

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If implementing a temporary code freeze to make such changes would be beneficial I'm happy to coordinate to do so.


:param directory: Temp object will be created under this directory,
defaults to :attr:`Target.working_directory`.
:type directory: str, optional
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

:type is_directory: bool, optional

:param directory: Temp object will be created under this directory,
defaults to :attr:`Target.working_directory`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you checked that

:attr:`Target.working_directory`

actually works when building a Sphinx project ? This would requires working_directory to be an item with its own docstring so that it's a valid reference target, which is very unlikely considered it's an instance attribute. Short of having a proper reference, you could just use:

 ``Target.working_directory``

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

devlib/target.py Show resolved Hide resolved
for entry in target_configs['QEMUTargetRunner'].values():
pp(entry)
qemu_settings = entry.get('qemu_settings') and entry['qemu_settings']
connection_settings = entry.get(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

qemu_settings=qemu_settings,
connection_settings=connection_settings,
)
targets.append((qemu_runner.target, qemu_runner))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a point in returning tuples rather than just the runner, considered you can get the target by using runner.target ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You cannot if we want to test ChromeOS with QEMU, i.e., targets.append((c_target, qemu_runner)).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As commented below/offline, this should be achievable with an implementation of a runner that allows sharing a target with another runner and only disposing of the Target once both are finished with it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I do have such mechanism in the new PR (#677 ).

if target_configs.get('LocalLinuxTarget') is not None:
print('> LocalLinux targets:')
for entry in target_configs['LocalLinuxTarget'].values():
pp(entry)
ll_target = LocalLinuxTarget(connection_settings=entry['connection_settings'])
targets.append(ll_target)
targets.append((ll_target, None))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning None is a bad idea: https://github.com/ARM-software/devlib/pull/669/files#r1541156876
I do note that you avoided your own trap, so hats off :) , but I can guarantee you'll fall in it at one point or another without type checking (I've never seen anyone not fall in it, me included)

The cleanest approach here is to simply create a no-op target runner that implements the API and does nothing apart from exposing a .target attribute. This way, the rest of the code does not need to care about special cases, and if part of the API is impossible to implement, that will just have revealed a design problem that needs to be addressed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - kind of.
Let's continue discussion on the PR that I'm gonna file for addressing these comments.


shutil.rmtree(tempdir)
if target_runner is not None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the None test can be dropped once the case disappears.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

shutil.rmtree(tempdir)
if target_runner is not None:
print('Terminating target runner...')
target_runner.terminate()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

target_runner is supposed to be used as a context manager. There might be some legitimate uses for having a separate terminate() method so I'm not saying we should necessarily remove it, but that specific use-site is not one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but TargetRunner is created in build_targets() and terminated at the end of test_read_multiline_values().

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point where it is created is not a problem, test_read_multiline_values() could be implemented as:

def test_read_multiline_values(..., target_runner):
    with target_runner as target_runner:
       do_stuff()

That also makes me realize that currently, we will spin a fresh set of target runners for every single test, so we will need to fix that. I'm not sure of how it's best done with pytest, but you can probably find some inspiration online on how to deal with database connections (which are typically used as a context manager).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented separate initialize / tear down routines and made other improvements on #677 . However, I still don't use context manager. Because targets would be terminated after the first test case (I know there is only 1 test case for now). Let's continue reviewing there if you don't have any other open comments here.

metin-arm added a commit to metin-arm/devlib that referenced this pull request Mar 28, 2024
PR#667: ARM-software#667

- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
@metin-arm
Copy link
Contributor Author

@douglas-raillard-arm, #677 is filed to address your comments except open questions here.

metin-arm added a commit to metin-arm/devlib that referenced this pull request Mar 28, 2024
PR#667: ARM-software#667

- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Mar 28, 2024
PR#667: ARM-software#667

- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Mar 28, 2024
PR#667: ARM-software#667

- Implement separate initialize / tear down methods and make various
  cleanups in test/test_target.py
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Apr 2, 2024
PR#667: ARM-software#667

- Implement separate initialize / tear down methods and make various
  cleanups in test/test_target.py
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Apr 2, 2024
PR#667: ARM-software#667

- Implement separate initialize / tear down methods and make various
  cleanups in test/test_target.py
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Apr 2, 2024
PR#667: ARM-software#667

- Implement separate initialize / tear down methods and make various
  cleanups in test/test_target.py
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Apr 2, 2024
PR#667: ARM-software#667

- Implement separate initialize / tear down methods and make various
  cleanups in test/test_target.py
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Apr 15, 2024
PR#667: ARM-software#667

- Implement a test module initializer with a tear down method in
  test/test_target.py
- Make various cleanups in test/test_target.py
- Improve structure of test/test_config.yml (previously
  target_configs.yaml)
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Apr 15, 2024
PR#667: ARM-software#667

- Implement a test module initializer with a tear down method in
  test/test_target.py
- Make various cleanups in test/test_target.py
- Improve structure of test/test_config.yml (previously
  target_configs.yaml)
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request Apr 17, 2024
PR#667: ARM-software#667

- Implement a test module initializer with a tear down method in
  test/test_target.py
- Make various cleanups in test/test_target.py
- Improve structure of test/test_config.yml (previously
  target_configs.yaml)
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request May 15, 2024
PR#667: ARM-software#667

- Implement a test module initializer with a tear down method in
  test/test_target.py
- Make various cleanups in test/test_target.py
- Improve structure of test/test_config.yml (previously
  target_configs.yaml)
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
metin-arm added a commit to metin-arm/devlib that referenced this pull request May 16, 2024
PR#667: ARM-software#667

- Implement a test module initializer with a tear down method in
  test/test_target.py
- Make various cleanups in test/test_target.py
- Improve structure of test/test_config.yml (previously
  target_configs.yaml)
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
marcbonnici pushed a commit that referenced this pull request May 29, 2024
PR#667: #667

- Implement a test module initializer with a tear down method in
  test/test_target.py
- Make various cleanups in test/test_target.py
- Improve structure of test/test_config.yml (previously
  target_configs.yaml)
- Make docstrings Sphinx compatible
- Make ``TargetRunner`` and its subclasses private
- Cleanup tests/test_target.py
- Replace print()'s with appropriate logging calls
- Implement ``NOPTargetRunner`` class for simplifying tests
- Improve Python v3.7 compatibility
- Relax host machine type checking
- Escape user input strings

and more..

Signed-off-by: Metin Kaya <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants