Trace message: pid could not be transfered: Invalid argument

[thanh-lam]

Describe the bug
On the CSM compute node, when csmapi trace is turned on, there are thousands of these messages in csm_compute.log:

[COMPUTE]2019-04-05 14:28:59.307304     csmapi::trace    | pid 82130 could not be transfered: Invalid argument

A count of these messages in the above time 14:28:59 resulted in 5532:

# grep "Invalid argument" csm_compute.log.save.for.later | grep "14:28:59"| wc -l
5532

To Reproduce
Steps to reproduce the behavior:

1. Make sure csmd-compute is active: systemctl status csmd-compute
2. Enable trace: /opt/ibm/csm/sbin/csm_ctrl_cmd --log.csmapi trace
3. Monitor compute log: tail -f /var/log/ibm/csm/csm_compute.log
4. Create an allocation: /opt/ibm/csm/bin/csm_allocation_create -j 1 -l <launchnode> -n <node> --isolated_core 1 --smt_mode 4
5. See messages in /var/log/ibm/csm/csm_compute.log

Expected behavior
These messages should not be seen in trace output.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• Machine [IST alt. cluster compute nodes]
• Version [CSM 1.5.3]

Additional context
Before turning off the cores for core isolation, CSM has to move processes to isolated cores. These messages could mean something was wrong when a process being moved or transferred.

Issue Source:
These messages might indicate some kind of error. Or, they should not be displayed in trace output. They make debugging with trace more difficult.