From 517be9cd02a6509b2512637505d6c93163a26151 Mon Sep 17 00:00:00 2001
From: Jim Garlick <garlick.jim@gmail.com>
Date: Tue, 24 Sep 2024 15:45:20 -0700
Subject: [PATCH 1/2] rfc42: add credit based input flow control

Problem: rfc42 does not specify anything about channel flow control,
so it is difficult to design clients and servers that operate without
dropping data.

Add credit based flow control to the protocol in the write direction.

The client receives credit for writes to the subprocess via the
exec add-credit response.
---
 spec_42.rst  | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 spell.en.pws |  1 +
 2 files changed, 56 insertions(+), 1 deletion(-)

diff --git a/spec_42.rst b/spec_42.rst
index 88e444a..ae08812 100644
--- a/spec_42.rst
+++ b/spec_42.rst
@@ -85,6 +85,8 @@ Goals
 
 - Optionally forward additional I/O channels.
 
+- Provide flow control on channels.
+
 - Provide signal delivery capability.
 
 - Protect against unauthorized use.
@@ -121,6 +123,10 @@ are defined as follows:
     channel (4)
       Forward auxiliary channel output to the client.
 
+    write-credit (8)
+      Send ``add-credit`` exec responses when buffer space is available
+      for standard input or writable auxiliary channels.
+
 Several response types are distinguished by the type key:
 
 .. object:: exec started response
@@ -179,6 +185,25 @@ Several response types are distinguished by the type key:
 
     See `I/O Object`_ below.
 
+.. object:: exec add-credit response
+
+  The subprocess server has more buffer space available to receive data
+  on the specified channel.  The response SHALL consist of a JSON object
+  with the following keys:
+
+  .. object:: type
+
+    (*string*, REQUIRED) The response type with a value of ``add-credit``.
+
+  .. object:: channels
+
+    (*object*, REQUIRED) An object with channel names as keys and
+    integer credit counts as values.
+
+  The server's initial response contains the full buffer size(s).
+
+  These messages are suppressed if the write-credit flag was not specified.
+
 .. object:: exec error response
 
 The :program:`exec` response stream SHALL be terminated by an error
@@ -219,6 +244,26 @@ This request receives no response, thus the request message SHOULD set
 FLUX_MSGFLAG_NORESPONSE.  Write Requests to invalid channel names MAY be
 ignored by the subprocess server.
 
+Input Flow Control
+------------------
+
+A client MAY track a channel's free remote buffer space :math:`L`:
+
+- :math:`L` is initialized to zero
+
+- Each :program:`exec add-credit` response adds credits to :math:`L`
+
+- Each :program:`write` request subtracts the unencoded data length
+  from :math:`L`.
+
+A client MAY avoid the risk of overrunning the remote buffer by ensuring
+a :program:`write` request never sends more than :math:`L` bytes of data.
+
+To avoid unnecessary start-up delays, a client MAY "borrow credit" up to
+the remote buffer size before the first :program:`exec add-credit` response.
+In that case :math:`L` would have a negative value until the first
+:program:`exec add-credit` response is received.
+
 kill
 ====
 
@@ -338,12 +383,21 @@ exec request
       "opts": {},
       "channels": []
     },
-    "flags": 3
+    "flags": 11
   }
 
 exec responses
 --------------
 
+.. code:: json
+
+  {
+    "type": "add-credit",
+    "channels": {
+      "stdin": 4096
+    }
+  }
+
 .. code:: json
 
   {
diff --git a/spell.en.pws b/spell.en.pws
index 8437516..b808622 100644
--- a/spell.en.pws
+++ b/spell.en.pws
@@ -494,3 +494,4 @@ chu
 unsatisfiable
 cgroup
 procs
+unencoded

From ffca60eb8f3d028841f1f27bc19a2070c7182177 Mon Sep 17 00:00:00 2001
From: Jim Garlick <garlick.jim@gmail.com>
Date: Wed, 23 Oct 2024 15:33:09 -0700
Subject: [PATCH 2/2] rfc42: set minimum default input buffer size

Problem: clients cannot send data to a remote subprocess before
receiving the first exec add-credit response unless they know how
big the remote buffer is.

State that the minimum default buffer size is 4096 bytes.
The default may be greater, but an eager client that doesn't know the
remote buffer size should know it can safely send that much data
initially without receiving credit.
---
 spec_42.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/spec_42.rst b/spec_42.rst
index ae08812..0987dde 100644
--- a/spec_42.rst
+++ b/spec_42.rst
@@ -264,6 +264,11 @@ the remote buffer size before the first :program:`exec add-credit` response.
 In that case :math:`L` would have a negative value until the first
 :program:`exec add-credit` response is received.
 
+Servers SHALL implement a default input buffer size of at least 4096 bytes.
+Unless the client explicitly requests a different input buffer size for the
+channel, it MAY assume that 4096 bytes can be borrowed before the first
+:program:`exec add-credit` response.
+
 kill
 ====