RHCEPHQE-17658: [IO Tool Integration] - Including dbench & postgresIO workflow #4406

Manimaran-MM · 2025-01-28T09:04:53Z

Description

Please include Automation development guidelines. Source of Test case - New Feature/Regression Test/Close loop of customer BZs

Steps to configure dbench(from client node):

dnf config-manager --add-repo=https://download.fedoraproject.org/pub/epel/9/Everything/x86_64/
dnf install dbench
Authorise client to use the fs (From Mon nodes)
Run command "dbench 32 -t 1200 -D /mnt/mani1/"
Run 32 clients in parallel for duration of 1200 seconds against the mount path

Steps to configure PostgresIO (from client node):

Create a directory for PostgreSQL data on Client node under the mounted directory
sudo mkdir -p /mnt/mani_fs_1/postgres_data
Provide postgres user access to the folder
chown -R postgres:postgres /mnt/mani_fs_1/postgres_data
chmod 700 /mnt/mani_fs_1/postgres_data
Install postgresql and postgres-server
dnf install postgresql postgresql-server
Point PostgreSQL to use the CephFS directory for its data
sudo -u postgres /usr/bin/initdb -D /mnt/mani_fs_1/postgres_data
Edit the PostgreSQL configuration file to use the CephFS data directory - vi /usr/lib/systemd/system/postgresql.service
Environment=PGDATA=/mnt/mani_fs_1/postgres_data
systemctl daemon-reload
systemctl restart postgresql
Restart will fail. Fix by disabling the enforce option
Since you are running on RHEL, SELinux might be blocking access
sestatus
setenforce 0 (temporarily disable it for testing)
systemctl restart postgresql
If the service starts successfully, set the correct SELinux context for the data directory
chcon -R -t postgresql_db_t /mnt/mani_fs_1
setenforce 1
systemctl restart postgresql
Create a db
sudo -u postgres psql
CREATE DATABASE testdb;
Run the initialization command
pgbench -i --scale=100 -U postgres -d testdb
The default pgbench transaction mix includes SELECT, INSERT, UPDATE, and DELETE operations.
pgbench -c 50 -j 4 -T 300 -U postgres -d testdb
This simulates 50 users performing mixed read/write transactions for 5 minutes

Code Changes - Summary

tests/cephfs/cephfs_vol_management/cephfs_vol_mgmt_fs_create_delete_loop.py
- Include additional subvolume
- Run dbench and health check in parallel on the newly created subvolume
- Handle Cleanup
- Include additional subvolume
- Run PostgresIO and health check in parallel on the newly created subvolume
- Handle Cleanup
tests/cephfs/cephfs_utilsV1.py
- Add new entry for dbench and postgresIO in the existing io tools function
- Include installation of dbench and postgres in the client nodes
- Add a monitor function to check the health status while running IO parallely

Pass Logs:

click to expand checklist

Create a test case in Polarion reviewed and approved.
Create a design/automation approach doc. Optional for tests with similar tests already automated.
Review the automation design
Implement the test script and perform test runs
Submit PR for code review and approve
Update Polarion Test with Automation script details and update automation fields
If automation is part of Close loop, update BZ flag qe-test_coverage “+” and link Polarion test

openshift-ci-robot · 2025-01-28T09:04:56Z

@Manimaran-MM: This pull request references RHCEPHQE-17658 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "odf-4.18" version, but no target version was set.

In response to this:

Description

Please include Automation development guidelines. Source of Test case - New Feature/Regression Test/Close loop of customer BZs

Steps to configure dbench(from client node):

dnf config-manager --add-repo=https://download.fedoraproject.org/pub/epel/9/Everything/x86_64/

dnf install dbench

Authorise client to use the fs (From Mon nodes)

Run command "dbench 32 -t 1200 -D /mnt/mani1/"

Run 32 clients in parallel for duration of 1200 seconds against the mount path

Code Changes - Summary

tests/cephfs/cephfs_vol_management/cephfs_vol_mgmt_fs_create_delete_loop.py

Include additional subvolume

Run dbench and health check in parallel on the newly created subvolume

Handle Cleanup

tests/cephfs/cephfs_utilsV1.py

Add new entry for dbench in the existing io tools function

Include installation of dbench and postgres in the client nodes

Add a monitor function to check the health status while running IO parallely

Pass Logs:

http://magna002.ceph.redhat.com/ceph-qe-logs/manim/logs/RHCEPHQE-17658/

click to expand checklist

Create a test case in Polarion reviewed and approved.

Create a design/automation approach doc. Optional for tests with similar tests already automated.

Review the automation design

Implement the test script and perform test runs

Submit PR for code review and approve

Update Polarion Test with Automation script details and update automation fields

If automation is part of Close loop, update BZ flag qe-test_coverage “+” and link Polarion test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

neha-gangadhar · 2025-01-28T11:36:49Z

tests/cephfs/cephfs_vol_management/cephfs_vol_mgmt_fs_create_delete_loop.py

@@ -202,6 +208,58 @@ def run(ceph_cluster, **kw):
                long_running=True,
            )

+            # Mounting Subvolume4 on Kernel - Run dbench
+            log.info(f"Mount subvolume {subvolume_name}_4 on Kernel Client")


can we add support to run dbench on fuse mounts too?

Added for dbench. For PosgresIO, not able to include FUSE mount. Need some more time to check on that

hkadam134 · 2025-01-28T16:54:36Z

tests/cephfs/cephfs_utilsV1.py

+                    sudo=True,
+                    cmd=(
+                        "dnf config-manager "
+                        "--add-repo=https://download.fedoraproject.org/pub/epel/9/Everything/x86_64/"


the repo file is specific to rhel-9, I think we need to check for the rhel versions and then replace 8 or 9 depending on the version installed.

openshift-ci-robot · 2025-01-29T10:39:46Z

@Manimaran-MM: This pull request references RHCEPHQE-17658 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "odf-4.18" version, but no target version was set.

In response to this:

Description

Please include Automation development guidelines. Source of Test case - New Feature/Regression Test/Close loop of customer BZs

Steps to configure dbench(from client node):

dnf config-manager --add-repo=https://download.fedoraproject.org/pub/epel/9/Everything/x86_64/

dnf install dbench

Authorise client to use the fs (From Mon nodes)

Run command "dbench 32 -t 1200 -D /mnt/mani1/"

Run 32 clients in parallel for duration of 1200 seconds against the mount path

Steps to configure PostgresIO (from client node):

Create a directory for PostgreSQL data on Client node under the mounted directory

sudo mkdir -p /mnt/mani_fs_1/postgres_data

Provide postgres user access to the folder

chown -R postgres:postgres /mnt/mani_fs_1/postgres_data

chmod 700 /mnt/mani_fs_1/postgres_data

Install postgresql and postgres-server

dnf install postgresql postgresql-server

Point PostgreSQL to use the CephFS directory for its data

sudo -u postgres /usr/bin/initdb -D /mnt/mani_fs_1/postgres_data

Edit the PostgreSQL configuration file to use the CephFS data directory - vi /usr/lib/systemd/system/postgresql.service

Environment=PGDATA=/mnt/mani_fs_1/postgres_data

systemctl daemon-reload

systemctl restart postgresql

Restart will fail. Fix by disabling the enforce option

Since you are running on RHEL, SELinux might be blocking access

sestatus

setenforce 0 (temporarily disable it for testing)

systemctl restart postgresql

If the service starts successfully, set the correct SELinux context for the data directory

chcon -R -t postgresql_db_t /mnt/mani_fs_1

setenforce 1

systemctl restart postgresql

Create a db

sudo -u postgres psql

CREATE DATABASE testdb;

Run the initialization command

pgbench -i --scale=100 -U postgres -d testdb

The default pgbench transaction mix includes SELECT, INSERT, UPDATE, and DELETE operations.

pgbench -c 50 -j 4 -T 300 -U postgres -d testdb

This simulates 50 users performing mixed read/write transactions for 5 minutes

Code Changes - Summary

tests/cephfs/cephfs_vol_management/cephfs_vol_mgmt_fs_create_delete_loop.py

Include additional subvolume

Run dbench and health check in parallel on the newly created subvolume

Handle Cleanup

Include additional subvolume

Run PostgresIO and health check in parallel on the newly created subvolume

Handle Cleanup

tests/cephfs/cephfs_utilsV1.py

Add new entry for dbench and postgresIO in the existing io tools function

Include installation of dbench and postgres in the client nodes

Add a monitor function to check the health status while running IO parallely

Pass Logs:

http://magna002.ceph.redhat.com/ceph-qe-logs/manim/logs/RHCEPHQE-17658-rerun1/

http://magna002.ceph.redhat.com/ceph-qe-logs/manim/logs/RHCEPHQE-17658/

click to expand checklist

Create a test case in Polarion reviewed and approved.

Create a design/automation approach doc. Optional for tests with similar tests already automated.

Review the automation design

Implement the test script and perform test runs

Submit PR for code review and approve

Update Polarion Test with Automation script details and update automation fields

If automation is part of Close loop, update BZ flag qe-test_coverage “+” and link Polarion test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

mergify · 2025-01-29T10:44:24Z

"This pull request now has conflicts with the target branch. Could you please resolve conflicts and force push the corrected changes?"

Signed-off-by: Manimaran M <[email protected]>

neha-gangadhar

Approved

openshift-ci · 2025-01-30T05:17:00Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Manimaran-MM, neha-gangadhar

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Manimaran-MM added Ceph-FS Ceph File System test-gap-analysis Test Cases Identified and Automated as part of Test Gaps found from Customer Issues or Upstream Test labels Jan 28, 2025

Manimaran-MM self-assigned this Jan 28, 2025

Manimaran-MM requested review from neha-gangadhar and AmarnatReddy as code owners January 28, 2025 09:04

Manimaran-MM requested review from julpark-rh, hkadam134 and sumabai January 28, 2025 09:13

neha-gangadhar reviewed Jan 28, 2025

View reviewed changes

hkadam134 reviewed Jan 28, 2025

View reviewed changes

Manimaran-MM changed the title ~~RHCEPHQE-17658: [IO Tool Integration] - Including dbench workflow~~ RHCEPHQE-17658: [IO Tool Integration] - Including dbench & postgresIO workflow Jan 29, 2025

Manimaran-MM force-pushed the RHCEPHQE-17658 branch from fe063fb to b0529e8 Compare January 29, 2025 10:43

Manimaran-MM requested review from vdas-redhat, pranavprakash20, pdhiran, Manohar-Murthy, mkasturi18, ckulal, TejasC88 and a team as code owners January 29, 2025 10:43

openshift-merge-robot added the needs-rebase label Jan 29, 2025

Manimaran-MM marked this pull request as draft January 29, 2025 10:44

openshift-ci bot added the do-not-merge/work-in-progress label Jan 29, 2025

Manimaran-MM force-pushed the RHCEPHQE-17658 branch 2 times, most recently from 68a1797 to 3601117 Compare January 29, 2025 10:54

openshift-merge-robot removed the needs-rebase label Jan 29, 2025

Manimaran-MM removed request for a team, vdas-redhat, TejasC88, mkasturi18, Manohar-Murthy, pranavprakash20, pdhiran and ckulal January 29, 2025 10:54

Manimaran-MM force-pushed the RHCEPHQE-17658 branch 2 times, most recently from fe063fb to 9ff3a30 Compare January 29, 2025 11:12

Manimaran-MM removed the do-not-merge/work-in-progress label Jan 29, 2025

Manimaran-MM marked this pull request as ready for review January 29, 2025 11:46

Manimaran-MM added 2 commits January 29, 2025 10:20

dbench workflow

1fe1ffc

Signed-off-by: Manimaran M <[email protected]>

dbench workflow

18f1fac

Signed-off-by: Manimaran M <[email protected]>

Manimaran-MM force-pushed the RHCEPHQE-17658 branch from 9ff3a30 to 18f1fac Compare January 29, 2025 15:20

revert client

6f33c5f

Signed-off-by: Manimaran M <[email protected]>

neha-gangadhar approved these changes Jan 30, 2025

View reviewed changes

openshift-ci bot assigned neha-gangadhar Jan 30, 2025

openshift-ci bot added the lgtm Add this label when the PR is good to be merged label Jan 30, 2025

mergify bot merged commit 60b08ca into red-hat-storage:main Jan 30, 2025
5 of 6 checks passed

Manimaran-MM mentioned this pull request Feb 4, 2025

RHCEPHQE-17760: [IO Tool Integration] - Including postgresIO on FUSE #4419

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RHCEPHQE-17658: [IO Tool Integration] - Including dbench & postgresIO workflow #4406

RHCEPHQE-17658: [IO Tool Integration] - Including dbench & postgresIO workflow #4406

Manimaran-MM commented Jan 28, 2025 •

edited

Loading

openshift-ci-robot commented Jan 28, 2025 •

edited by openshift-ci bot

Loading

Description

Code Changes - Summary

Pass Logs:

neha-gangadhar Jan 28, 2025

Manimaran-MM Jan 29, 2025

hkadam134 Jan 28, 2025

Manimaran-MM Jan 29, 2025

openshift-ci-robot commented Jan 29, 2025 •

edited by openshift-ci bot

Loading

Description

Code Changes - Summary

Pass Logs:

mergify bot commented Jan 29, 2025

neha-gangadhar left a comment

openshift-ci bot commented Jan 30, 2025

RHCEPHQE-17658: [IO Tool Integration] - Including dbench & postgresIO workflow #4406

RHCEPHQE-17658: [IO Tool Integration] - Including dbench & postgresIO workflow #4406

Conversation

Manimaran-MM commented Jan 28, 2025 • edited Loading

Description

Code Changes - Summary

Pass Logs:

openshift-ci-robot commented Jan 28, 2025 • edited by openshift-ci bot Loading

Description

Code Changes - Summary

Pass Logs:

neha-gangadhar Jan 28, 2025

Choose a reason for hiding this comment

Manimaran-MM Jan 29, 2025

Choose a reason for hiding this comment

hkadam134 Jan 28, 2025

Choose a reason for hiding this comment

Manimaran-MM Jan 29, 2025

Choose a reason for hiding this comment

openshift-ci-robot commented Jan 29, 2025 • edited by openshift-ci bot Loading

Description

Code Changes - Summary

Pass Logs:

mergify bot commented Jan 29, 2025

neha-gangadhar left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Jan 30, 2025

Manimaran-MM commented Jan 28, 2025 •

edited

Loading

openshift-ci-robot commented Jan 28, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jan 29, 2025 •

edited by openshift-ci bot

Loading