GETXATTR "security.capability" is called on every write

[rfjakob]

Looks like GETXATTR "security.capability" is called for every write that comes in from the user application. Note that a user application write can be larger than 128kB, and then is split up to 128kB by the kernel. Then we only get one GETXATTR for multiple WRITE commands.

dd bs=128k count=2 (GETXATTR after each WRITE):

17:37:54.485059 rx 2364: GETXATTR n1 {sz 24} ["security.capability"] 20b
17:37:54.485136 tx 2364:     95=operation not supported
17:37:54.485186 rx 2366: LOOKUP n1 ["foo"] 4b
17:37:54.485288 tx 2366:     OK, {n4 g1 tE=1s tA=1s {M0100644 SZ=12288 L=1 1026:1026 B32*4096 i0:3747177 A 1603035325.725516 M 1603035325.725516 C 1603035325.725516}}
17:37:54.485357 rx 2368: GETXATTR n4 {sz 24} ["security.capability"] 20b
17:37:54.485418 tx 2368:     95=operation not supported
17:37:54.485544 rx 2370: OPEN n4 {WRONLY,0x8000} 
17:37:54.485603 tx 2370:     OK, {Fh 1 }
17:37:54.485631 rx 2372: GETXATTR n4 {sz 0} ["security.capability"] 20b
17:37:54.485673 tx 2372:     95=operation not supported
17:37:54.485694 rx 2374: SETATTR n4 {size 0} 
17:37:54.485755 tx 2374:     OK, {tA=0s {M0100644 SZ=0 L=1 1026:1026 B0*4096 i0:3747177 A 1603035325.725516 M 1603035474.484960 C 1603035474.484960}}
17:37:54.485783 rx 2376: FLUSH n4 {Fh 1} 
17:37:54.485790 tx 2376:     OK
17:37:54.485863 rx 2378: GETXATTR n4 {sz 0} ["security.capability"] 20b
17:37:54.485895 tx 2378:     95=operation not supported
17:37:54.485964 rx 2380: WRITE n4 {Fh 1 [0 +131072)  L 0 WRONLY,0x8000} "\x00\x00\x00\x00\x00\x00\x00\x00"... 131072b
17:37:54.486233 tx 2380:     OK
17:37:54.486275 rx 2382: GETXATTR n4 {sz 0} ["security.capability"] 20b
17:37:54.486320 tx 2382:     95=operation not supported
17:37:54.486398 rx 2384: WRITE n4 {Fh 1 [131072 +131072)  L 0 WRONLY,0x8000} "\x00\x00\x00\x00\x00\x00\x00\x00"... 131072b
17:37:54.486581 tx 2384:     OK
17:37:54.486615 rx 2386: FLUSH n4 {Fh 1} 
17:37:54.486623 tx 2386:     OK
17:37:54.486648 rx 2388: RELEASE n4 {Fh 1 WRONLY,0x8000  L0} 
17:37:54.486660 tx 2388:     OK

dd bs=256k count=1 (GETXATTR called only once after two WRITEs):

17:38:55.328698 rx 2390: GETXATTR n1 {sz 24} ["security.capability"] 20b
17:38:55.328823 tx 2390:     95=operation not supported
17:38:55.328905 rx 2392: LOOKUP n1 ["foo"] 4b
17:38:55.328988 tx 2392:     OK, {n4 g1 tE=1s tA=1s {M0100644 SZ=262144 L=1 1026:1026 B520*4096 i0:3747177 A 1603035474.484960 M 1603035474.485960 C 1603035474.485960}}
17:38:55.329064 rx 2394: GETXATTR n4 {sz 24} ["security.capability"] 20b
17:38:55.329139 tx 2394:     95=operation not supported
17:38:55.329163 rx 2396: OPEN n4 {WRONLY,0x8000} 
17:38:55.329194 tx 2396:     OK, {Fh 1 }
17:38:55.329260 rx 2398: GETXATTR n4 {sz 0} ["security.capability"] 20b
17:38:55.329297 tx 2398:     95=operation not supported
17:38:55.329318 rx 2400: SETATTR n4 {size 0} 
17:38:55.329408 tx 2400:     OK, {tA=0s {M0100644 SZ=0 L=1 1026:1026 B0*4096 i0:3747177 A 1603035474.484960 M 1603035535.328142 C 1603035535.328142}}
17:38:55.329440 rx 2402: FLUSH n4 {Fh 1} 
17:38:55.329453 tx 2402:     OK
17:38:55.329561 rx 2404: GETXATTR n4 {sz 0} ["security.capability"] 20b
17:38:55.329594 tx 2404:     95=operation not supported
17:38:55.329672 rx 2406: WRITE n4 {Fh 1 [0 +131072)  L 0 WRONLY,0x8000} "\x00\x00\x00\x00\x00\x00\x00\x00"... 131072b
17:38:55.329890 tx 2406:     OK
17:38:55.329962 rx 2408: WRITE n4 {Fh 1 [131072 +131072)  L 0 WRONLY,0x8000} "\x00\x00\x00\x00\x00\x00\x00\x00"... 131072b
17:38:55.330166 tx 2408:     OK
17:38:55.330201 rx 2410: FLUSH n4 {Fh 1} 
17:38:55.330210 tx 2410:     OK
17:38:55.330232 rx 2412: RELEASE n4 {Fh 1 WRONLY,0x8000  L0} 
17:38:55.330243 tx 2412:     OK

See also

• Ticket for the same issue in MarFS:
  • the system calls getxattr("security.capability") on every FUSE write()! mar-file-system/marfs#205
• Discussion on fuse-devel about this behavoir:
  • https://sourceforge.net/p/fuse/mailman/fuse-devel/thread/[email protected]/

Quoting a message from the thread above :

"I believe that this is occurring because in mm/
filemap.c:__generic_file_aio_write_nolock() there's a call to
file_remove_suid() which down the call graph a ways tries to remove
the capabilities, but does so by checking to see if the capabilities
xattr is there before it attempts to delete it."

--jmike

[rfjakob]

Rejecting the calls gives a nice speedup.

VERSION           WRITE    READ     UNTAR  MD5   LS     RM 
v2.0-beta1-5-gc943ed3 417 1000      30.4  12.7  9.9   16.4
v2.0-beta1-6      529     1100      17.5   9.0  3.6    9.0        <-------- 6697ffd