Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory:Add memory mapping tool #3772

Merged
merged 1 commit into from
Aug 29, 2023
Merged

Conversation

zhenyzha
Copy link
Contributor

When referring to memory mapping, one is usually referring
to mmap(2).
There are 2 categories for using mmap.
One category is SHARED vs PRIVATE mappings.
The other category is FILE vs ANONYMOUS mappings.
Mixed together you get the 4 following combinations:

  1. PRIVATE FILE MAPPING
  2. SHARED FILE MAPPING
  3. PRIVATE ANONYMOUS MAPPING
  4. SHARED ANONYMOUS MAPPING Add a memory mapping tool
    to support the above four scenarios.
    Updates numa_stress with mmap tool.

ID: 2221632
Signed-off-by: zhenyzha [email protected]

@zhenyzha zhenyzha force-pushed the mmaping branch 3 times, most recently from bce1a13 to dccda6e Compare July 11, 2023 08:15
@zhenyzha
Copy link
Contributor Author

git am 3772.patch
Applying: Memory:Add memory mapping tool

(1/1) Host_RHEL.m9.u3.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.aarch64.page_64k.io-github-autotest-qemu.numa.numa_stress.arm64-pci: PASS (423.82 s)

@YongxueHong @yanan-fu @mcasquer Could you pls help review when you are free. thanks.

The current patch is to prove the availability of the tool,
I think I will build a unified external interface for the tool in the next patch.

@mcasquer
Copy link
Contributor

numa_stress case failed in x86_64:

 (1/1) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.numa.numa_stress.q35: STARTED
 (1/1) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.numa.numa_stress.q35: FAIL: Memory still stick in node 0 (173.36 s)
RESULTS    : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

@mcasquer
Copy link
Contributor

I've published the debug.log on the bz. Apart from that I've also noticed the following error:

[stdlog] 2023-07-13 02:40:42,175 avocado.virttest.utils_test INFO | Uninstall mem_mapping
[stdlog] 2023-07-13 02:40:42,176 aexpect.client DEBUG| [192.168.122.146] Sending command: ./configure && make uninstall
[stdlog] 2023-07-13 02:40:42,277 aexpect.client DEBUG| [192.168.122.146] Sending command: echo $?
[stdlog] 2023-07-13 02:40:42,377 avocado.virttest.utils_test ERROR| Uninstall stress failed with error: -bash: ./configure: No such file or directory

qemu/tests/numa_stress.py Outdated Show resolved Hide resolved
@zhenyzha
Copy link
Contributor Author

I've published the debug.log on the bz. Apart from that I've also noticed the following error:

[stdlog] 2023-07-13 02:40:42,175 avocado.virttest.utils_test INFO | Uninstall mem_mapping
[stdlog] 2023-07-13 02:40:42,176 aexpect.client DEBUG| [192.168.122.146] Sending command: ./configure && make uninstall
[stdlog] 2023-07-13 02:40:42,277 aexpect.client DEBUG| [192.168.122.146] Sending command: echo $?
[stdlog] 2023-07-13 02:40:42,377 avocado.virttest.utils_test ERROR| Uninstall stress failed with error: -bash: ./configure: No such file or directory

Thanks for reminding me, I will modify it:)

@zhenyzha zhenyzha force-pushed the mmaping branch 5 times, most recently from b17c596 to 2babf1f Compare July 26, 2023 07:57
@zhenyzha
Copy link
Contributor Author

git am 3772.patch
Applying: Memory:Add memory mapping tool

(1/1) Host_RHEL.m9.u3.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.aarch64.page_4k.io-github-autotest-qemu.numa.numa_stress.arm64-pci: PASS (393.88 s)

@mcasquer Could you pls help review again when you are free. thanks.

@mcasquer
Copy link
Contributor

git am 3772.patch Applying: Memory:Add memory mapping tool

(1/1) Host_RHEL.m9.u3.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.aarch64.page_4k.io-github-autotest-qemu.numa.numa_stress.arm64-pci: PASS (393.88 s)

@mcasquer Could you pls help review again when you are free. thanks.

@zhenyzha numa_stress case still fails

 (1/2) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads.q35: STARTED
 (1/2) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads.q35: PASS (944.97 s)
 (2/2) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.numa.numa_stress.q35: STARTED
 (2/2) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.numa.numa_stress.q35: ERROR: Timeout expired while waiting for shell command to complete: 'pidof -s ./mem_mapping'    (output: '') (845.32 s)
RESULTS    : PASS 1 | ERROR 1 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

@mcasquer
Copy link
Contributor

debug.log attached to the bz

@zhenyzha
Copy link
Contributor Author

debug.log attached to the bz

It seems that the memory pressure is too high, and the host killed some processes during the second round of testing. I need an x86 host environment, to adjust the pressure. The good news, on the other hand, is that this proves that the script works :)

@mcasquer
Copy link
Contributor

After an offline discussion, if pressure is reduced in x86_64, the test case finish successfully.

 (1/1) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.numa.numa_stress.q35: STARTED
 (1/1) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.numa.numa_stress.q35: PASS (315.58 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

Copy link
Contributor

@mcasquer mcasquer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhenyzha
Copy link
Contributor Author

@YongxueHong @yanan-fu Could you pls help review when you are free? thanks.

qemu/tests/cfg/numa.cfg Outdated Show resolved Hide resolved
qemu/tests/cfg/numa.cfg Outdated Show resolved Hide resolved
qemu/tests/numa_stress.py Outdated Show resolved Hide resolved
qemu/tests/numa_stress.py Outdated Show resolved Hide resolved
qemu/tests/numa_stress.py Outdated Show resolved Hide resolved
When referring to memory mapping, one is usually referring
to mmap(2). There are 2 categories for using mmap.
One category is SHARED vs PRIVATE mappings.
The other category is FILE vs ANONYMOUS mappings.
Mixed together you get the 4 following combinations:
1. PRIVATE FILE MAPPING
2. SHARED FILE MAPPING
3. PRIVATE ANONYMOUS MAPPING
4. SHARED ANONYMOUS MAPPING
Add a memory mapping tool to support the above four scenarios.
Updates numa_stress with mmap tool.

Signed-off-by: zhenyzha <[email protected]>
@zhenyzha
Copy link
Contributor Author

@YongxueHong Could you pls help review when you are free? thanks.

git am 3772.patch
Applying: Memory:Add memory mapping tool

(1/1) Host_RHEL.m9.u3.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.3.0.x86_64.io-github-autotest-qemu.numa.numa_stress.q35: PASS (795.11 s)

Copy link
Contributor

@YongxueHong YongxueHong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@YongxueHong YongxueHong merged commit 6332157 into autotest:master Aug 29, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants