Discussion

Why SOA?

central, well defined place to perform saga-pilot level scheduling
saga-pilot users always see a 'Queue' to submit jobs to ('big queue', 'application queues' (there is one per application instance))
use dedicated queues toward the pilot layer, to easily support early and late binding ('small queues' / 'pilot queues')
task of a scheduler (pluggable) is to map CUs from application queues to pilot queues.
easy to define sensible default schedulers
easy to support research schedulers on the fly (per application instance, for example)
well defined information flow
persistency beyond application lifetime
map one application queue to one pilot queue with default scheduler to disable saga-pilot level scheduling, with near zero perf penalty (e.g. for Troy, which does pilot level scheduling on its own)

BTW: The combined semantics of the pilot-service and the queuing-service are equivalent to the P* 'pilot manager' entity.

the name is probably misleading, but we need an information and notification hub which can serve a varying number of producers / consumers
connectivity issues (NAT, Firewall) make N:N communication difficult / impossible, thus the need for a communication hub
information flow needs to be configurable: too much will limit scalability, too little will limit ability to reason
logging configurable per component (what / how often / how fine grained)
example: ability to tag a single CU to get a full trace of activities related to that CU, w/o flooding the whole system
example: determine global congestion points

implementing full fledged multi-user capabilities with credential delegation etc. seems beyond our capabilities (expertise, manpower, time)
support for science production use cases required
multi-user, multi-application and multi-DCI functionality required

Proposal:

all components and operations are tagged with session IDs, system enforces auth based on those session IDs.
front layer translates auth tokens to session IDs, backend layer translates session IDs to backend auth tokens (this is a very inexact statement, but kind of represents the idea)

Example:

gateway bootstraps, and owns, service layer
users register ssh keys in gateway, and also registers backend credentials (X509, myproxy, ssh, whatever)
users auth via https (ssh keys), and get session token
session token is used for all interactions to saga-pilot, and within saga-pilot (--> multi-user, multi-app)
for backend interactions, like running a pilot agent or a connection proxy, the service layer selects the proper user's ssh credentials as registered (selection by session ID)