TMTP Protocol

"Trusted Messaging Transfer Protocol" defines a simple client/server scheme for reliable store-and-forward message delivery. It needs no other protocol in the way that POP & IMAP need SMTP. TMTP may be conveyed by any secure, reliable means, e.g. TCP+TLS. TMTP sessions are typically long-duration, and may idle for extended periods. After the client completes a login or register request, either side may contact the other. A client may simultaneously contact multiple TMTP servers via separate connections.

The project homepage is mnmnotmail.org.

Status: Version 1 is a work in progress, and aspects need to be revised. All of the elements defined herein are implemented in the mnm server.

Todo: Collect relevant items from the todo list into a Future section. Define types for msgid, uid, datetime, etc.

Conventions: Header definitions herein use these conventions:

  "nameT": type,           // field gives a certain type
  "nameV": value,          // field gives a certain value
  "nameW": value | value,  // field gives one of many certain values
 ["nameA": t/v, ...        // alternative fieldsets
 |"nameB": t/v, ... ],     //   nth alternative
 <"nameO": t/v, ... >,     // optional fieldset
 <[ ... | ... ]>,          // optional alternative fieldsets
  (ref),                   // previously defined fieldset

Definitions

Each message starts with a header, wherein four hex digits give the size of a JSON metadata object, which may be followed by arbitrary 8-bit data if .datalen is non-zero. The data may begin with a secondary JSON metadata object if .datahead is non-zero.

hhhh{ ... <"datalen":uint, <"datahead":uint>> }datahead octets, datalen - datahead octets

Messages are not interleaved/multiplexed within a link; that should be considered for a later revision.

Link errors & timeouts cause the server to close the connection without notice. Protocol errors or login failure by TMTP clients cause the server to close the connection after emitting a quit response:

{ "op":    "quit",
  "error": string}   // reason the connection was closed

Ack responses (ack) from the server have the following format:

{ "op":     "ack",
  "id":     string,  // matches id on the message being ack'd
 <"msgid":  string,  // id assigned to the message being ack'd
  "posted": string   // datetime assigned to the message being ack'd
 |"error":  string>} // reason the message was not delivered

Messages from the server typically have the following required headers (std):

  "from":    string, // uid of the sender
  "id":      string, // id assigned to the message
  "posted":  string, // datetime assigned to the message
  "headsum": uint    // checksum for the message header

Messages carrying arbitrary data have the following headers (data):

  "datalen":  uint,  // count of octets following the header
 <"datahead": uint>, // count <= datalen of octets which comprise a secondary header
 <"datasum":  uint>, // checksum for datalen octets

Id strings generated by the server ((ack) .msgid and (std) .id) must be sequential, but need not be contiguous. An easy way to generate them is to contact an NTP (network time protocol) service on startup for the current time, calculate the number of nanoseconds since the epoch, and increment that value for each id string generated. This supports generation of up to one billion ids per second, and doesn't require a persistent record of the last id generated before shutdown.

Human-readable text must be UTF-8. For Ping and GroupInvite messages, the text must be displayed verbatim. For Post and PostNotify messages, the text should be displayed per the CommonMark specification, with extensions for strikethrough, tables, slide decks, charts, forms, and hyperlinks to attachments & messages.
todo: document mnm CommonMark extensions

Client messages and resulting server messages

After the client completes a Login or Register sequence, either side may contact the other.

TmtpRev gives the latest recognized protocol version; it must be the first message.

{ "op": 0,
  "id": "1"}                                // protocol version string

Response:

{ "op":     "tmtprev",
  "id":     "1",                            // protocol version string
  "name":   string,                         // site-specific name
 <"auth":   1 | 2,                          // 1 registration, 2 registration & login
  "authby": [{"label": string,              // OpenID Connect provider
              "login": [string, <string>],  // authentication URL, URL-encoded params
              "token": [string, <string>]}, // token URL, URL-encoded params
             ... ]>}

Register creates a user account with a single node.
todo: accept credentials for third party authentication services

{ "op":       1,
  "newnode":  string,            // user label for a client device
 <"newalias": string>,           // user alias, must be 8+ printable characters
 <"oidc":                        // OpenID Connect token result
   { "token_type":    "Bearer",
     "expires_in":    uint,
     "id_token":      string,
     "access_token":  string,
     "refresh_token": string}>}

Response: same as Login
To sender's node:

{ "op":     "registered",
  "uid":    string,              // permanent id for new user
  "nodeid": string,              // password for first node
 <"error":  string>}             // reason alias was not allowed

Login connects a client to a user node.
todo: response provides connection timeout period

{ "op":   2,
  "uid":  string,        // permanent user id
  "node": string}        // password for this node

Response:

{ "op":   "info",
  "info": "login ok",    // todo: drop this
  "ohi":  [string, ...]} // list of uids now online

To sender's nodes:
todo: replace with messages on ohi channel?

{ "op":   "login",
  (std),
  "node": "tbd"}

UserEdit updates a user account to add an alias or node.
todo: disallow node label reuse?
todo: dropnode and dropalias; prevent account hijacking from stolen client/nodeid
todo: cork pauses delivery to user's nodes, params date & for-list

{ "op":       3,
  "id":       string,  // referenced by (ack) response
 ["newnode":  string   // user label for a client device
 |"newalias": string]} // user alias, must be 8+ printable characters

Response: (ack) // without .msgid or .posted
To sender's nodes:

{ "op":       "user",  // these have higher priority than normal messages
  (std),
 ["newnode":  string,  // from client request
  "nodeid":   string   // password for new node
 |"newalias": string]} // from client request

OhiEdit notifies selected contacts of a user's presence.

{ "op":   4,
  "id":   string,                  // referenced by (ack) response
  "for":  [{"id": string}, ...],   // list of uids
  "type": "add" | "drop"
        | "init"}                  // no "ohiedit" message is sent to sender's nodes

Response: (ack) // without .msgid or .posted
To sender's nodes:

{ "op":   "ohiedit",
  (std),
  "for":  [{"id": string}, ...],   // from client request
  "type": "add" | "drop"}          // from client request

To recipients:

{ "op":     "ohi",
  "from":   string,                // uid
  "status": 1 | 2}                 // 1 online, 2 offline

GroupInvite performs a Ping, and authorizes the recipient to join a group, creating it if necessary.
todo: closed groups; invitations by moderators

{ "op":   5,
  "id":   string,       // referenced by (ack) response
  "gid":  string,       // group name, must be 8+ printable characters
  "from": string,       // sender alias
  "to":   string,       // invitee alias
  (data)}               // .datalen max is limited by server; data must be valid UTF-8

Response: (ack)
To recipient:

{ "op":    "invite",
  (std),
  "gid":   string,      // from client request
  "alias": string,      // from client request .from
  "to":    string,      // from client request
  (data)}               // from client request

To group members:

{ "op":       "member",
  (std),
  "act":      "invite",
  "gid":      string,   // from client request
  "alias":    string}   // from client request .to

GroupEdit updates a group, allowing an invitee to join, or a member to be dropped, or a member to update their recorded alias.

{ "op":        6,
  "id":        string,  // referenced by (ack) response
  "act":       "join"
             | "alias"
             | "drop",
  "gid":       string,  // group name
 [<"newalias": string>  // user alias, for "join"
 |"newalias":  string   // user alias, for "alias"
 |"to":        string]} // user alias, for "drop"

Response: (ack) // without .msgid or .posted
To group members:

{ "op":       "member",
  (std),
  "act":      string,   // from client request
  "gid":      string,   // from client request
  "alias":    string,   // last reported value
 <"newalias": string>}  // from client request

Post sends a message to users and/or groups.

{ "op":  7,
  "id":  string,               // referenced by (ack) response
  "for": [{"id":   string,     // uid or group name
           "type": 1 | 2 | 3}, // 1 uid, 2 gid (include self), 3 gid (exclude self)
          ... ]
       | [],                   // only sender's nodes
  (data)}
                               // .datahead segment
{ "threadid":      string,     // empty for new thread
  "alias":         string,     // user alias
  "subject":       string,     // optional if .threadid set
 <"confirmid":     string,     // id of confirmed forwarded message, see PostNotify
  "confirmposted": string>,    // posted date of confirmed forwarded message
 <"cc":                        // subscribers, omit unless .threadid empty
   [{ "who":       string,     // user alias
      "whouid":    string,     // uid
      "by":        string,     // user alias that included .who
      "byuid":     string,     // uid
      "date":      string,     // datetime when .who was included
      "note":      string,     // comment from .by
      "subscribe": bool},      // .who receives new thread messages
    ... ]>,
 <"attach":                    // file contents appear in the data segment following the message body
   [{ "name":      string,     // unique in .attach, prefix u:/f:/r:
      "size":      uint,       // count of octets
     <"ffn":       string>},   // filled-form name, omit for .name u:*
    ... ]> }
                               // todo: filled-form attachment

{ "nodesync": true}            // .datahead segment for node sync
                               // todo: data segment for node sync

Response: (ack)
To recipients:

{ "op":  "delivery",
  (std),
  (data)}                      // from client request

PostNotify sends a message to the .for list and a separate notification to the .for and .notefor lists.
todo: PostNotify synchronization; see Todo.txt

This enables a decentralized subscriber list per-thread, and forwarding a thread to new subscribers. The (data) message delivers a copy of the thread, without attachments, to new subscribers. Recipients flag its messages as unconfirmed. The notification delivers the list of new subscribers to current & new subscribers. The current subscribers then Post a copy of each thread message authored by them, with attachments, to the new subscribers, including the .confirmid & .confirmposted fields of .datahead.

{ "op":         8,
  "id":         string,  // referenced by (ack) response
  "for":        [ ... ], // list of uids and/or group names, see Post
 <"fornotself": true>,   // exclude sender's nodes (normally implicit in .for list)
  "notefor":    [ ... ], // list of uids and/or group names, see Post
  "notelen":    uint,    // count of octets following the header for notification
 <"notehead":   uint>,   // analagous to .datahead for notification
 <"notesum":    uint>,   // analagous to .datasum for notification
  (data)}                // .datahead & .datasum pertain to octets following notification

                         // .notehead segment
{ "threadid": string,    // id of forwarded thread
  "cc":       [ ... ]}   // new subscribers, see Post

                         // .datahead segment
{ "threadid": string}    // id of forwarded thread
                         // todo: data segment for forwarded thread

Response: (ack)
To recipients:

{ "op":     "delivery",
  (std),
  "notify": uint,        // count of items in .for & .notefor, including self
  (data)}                // from client request (.datalen - .notelen), .datahead, .datasum

To notification recipients:

{ "op":     "notify",
  (std),
  "postid": string,      // (ack) .msgid
  (data)}                // from client request .notelen, .notehead, .notesum

Ping sends a short text message via a user's alias. On receipt, the recipient may thereafter contact the sender by uid, given in (std) .from. A server may limit the number of pings and consecutive failed pings per 24h.

{ "op":   9,
  "id":   string,  // referenced by (ack) response
  "from": string,  // sender alias
  "to":   string,  // invitee alias
  (data)}          // .datalen max is limited by server; data must be valid UTF-8

Response: (ack)
To recipient:

{ "op":    "ping",
  (std),
  "alias": string, // from client request .from
  "to":    string, // from client request
  (data)}          // from client request

Ack acknowledges receipt of a message.

{ "op":   10,
  "id":   string, // from .id of message being ack'd
  "type": string} // unused

Pulse resets the connection timeout.
```
{ "op": 11}
```
Quit performs logout.
```
{ "op": 12}
```

License

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Protocol.md

Protocol.md

TMTP Protocol

Definitions

Client messages and resulting server messages

License

Files

Protocol.md

Latest commit

History

Protocol.md

File metadata and controls

TMTP Protocol

Definitions

Client messages and resulting server messages

License