writes to numpy arrays returned through py::array_t don't seem to work

[iourielementary]

I have a process that uses shared memory. The goal is to have memory allocated in shared memory and then return this memory as numpy array to calling python process.

Shared memory is managed by Boost.Interprocess. When a call is made from python to get numpy array, a tuple is created containing numpy array and Boost.Interprocess handle that identifies allocated shared memory.

This is a snippet of code where numpy array is created, inserted into tuple and returned:

                 std::cout << "start = " << std::hex << static_cast<void*>(start) << std::endl;                                  
                 return py::make_tuple(py::array_t<char>({ size },  { 1 },  (const char *)start),  local_hdl);

On python side numpy and handle are extracted from the tuple:

test_array = result[0]
shmem_handle = result[1]

The issue that I have is when new values are assigned to numpy array, they are not written to shared memory. I did a little bit of troubleshooting to confirm this:

1. Prior to returning numpy array first 3 bytes of shared memory are written:

                 std::cout << "start = " << std::hex << static_cast<void*>(start) << std::endl;
                                                                                                                                                                                                                                                                      
                 start[0] = 1;                                                   
                 start[1] = 2;                                                   
                 start[2] = 3;                                                   
                 return py::make_tuple(py::array_t<char>({ size },  { 1 },  (const char *)start),   local_hdl);

2. Python code that gets numpy array, prints it out original contents, then attempts to overwrite the first 3 bytes:

              test_array = result[0]                                                                                                                                                                                                                                       
              shmem_handle = result[1]                                                

              pointer, read_only_flag = test_array.__array_interface__['data']        
              print("pointer = ", hex(pointer))                                       
              print("read_only_flag = ", read_only_flag)                              
              print("Type = ", type(test_array))                                      

              print("ShMem Handle =", result[1])              
              print(test_array)                                                       

              print("setting new numpy values")                                       
 
              test_array.dtype = numpy.uint8;                                         
              test_array.shape = (24,1)                                               
              test_array[0] = 11;                                                     
              test_array[1] = 22;                                                     
              test_array[2] = 33;                                                     

             print(test_array)

3. This is the output that I get from running the program:

start = 0x7f74084c21e0                                                                                                                                                                                                                                                   
pointer =  0x17a8c90                                                                                                                                                                                                                                                     
read_only_flag =  False                                                                                                                                                                                                                                                  
Type =  <class 'numpy.ndarray'>                                                                                                                                                                                                                                          
ShMem Handle = 432                                                                                                                                                                                                                                                       
[1 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]                                                                                                                                                                                                                        
setting new numpy values                                                                                                                                                                                                                                                 
[11 22 33  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]                                                                                                                                                                                                
Hit any key to exit

So it looks like numpy array has changed and has new values 11,22,33. However I need to verify that new values are written to shared memory as well.

Looking at the memory mapping of the running process, I confirm that start address 0x7f74084c21e0 is indeed within shared memory.

7f74084c2000-7f74084c3000 rw-s 00000000 00:1c 540208                     /dev/shm/shmem

4. Next thing would be to attach gdb to a running process and check memory contents at localtion 0x7f74084c21e0, which is buffer start address

(gdb) x/24bx 0x7f74084c21e0
0x7f74084c21e0:	0x01	0x02	0x03	0x00	0x00	0x00	0x00	0x00
0x7f74084c21e8:	0x00	0x00	0x00	0x00	0x00	0x00	0x00	0x00
0x7f74084c21f0:	0x00	0x00	0x00	0x00	0x00	0x00	0x00	0x00
(gdb) 

this shows me that actual values stored in shared memory are still 1,2,3, and new values were never written.

Does anybody have an idea of what I am doing wrong here? Any comments/suggestion/queries are welcome.

[leslinice]

Okay

[leslinice]

Yuo