sign_view and get_configuration

[keighrim]

Because

A typical clams app based on the python SDK will implement _annotate() as something like this;

class AwesomeClamsApp(ClamsApp):
    ...  # preamble including __init__() 
    def _annotate(self, mmif, **kwargs):

        mmif = Mmif(mmif)
        # `kwargs` is the raw input parameters passed in the URL
        config = self.get_configuration(**kwargs)
        # `get_configuration`'s job is 1. validate k-v pairs in kwargs , 2. fill the k-v dict with default values (if not given)
        # now config has only "valid" values and all the default values
        # HENCE kwargs != config 
        
        for doc in self.find_relevant_docs(mmif): 
            view = mmif.new_view()
            # at this point the dev can sign this new view, but they has liberty to sign the view either with `kwargs` or `config`
            self.sign_view(view, kwargs)
            ... # continue processing 

        return mmif
    ... # code continues

And I don't like the idea that some views are signed with raw user input params, while other are signed with "refined" input params.

Done when

The issue will be resolved when

1. we decided which param dict to use to sign views
2. (optionally) SDK implement some automated signing process

Additional context

No response

[keighrim]

To the first question, I think the output MMIF should include the param dict before it's validated by calling get_configuration, so that we would have a record of the raw user input (kwargs dict in the above snippet) as a logging. This is because that config dict above can always be replicated using the same get_configuration in the same version of app, while the raw input will be lost if we don't write that down somewhere.

[marcverhagen]

How about signing with the raw/original parameters but leave a warning when parameters were refined? I actually think I may have recently read this somewhere so this may not be my idea.

[keighrim]

I guess we all have some consensus that raw parameters are the ones that need to be recorded.

---

Implementation-wise, a few problems I've been struggling with;

1. sign_view only sees a dict and it doesn't care if the input's raw or refined.
2. new_view() can be called at any time during _annotate(), so we can't reliably "inject" automatic sign_view() after each call to new_view() (and even if we do so, it might seem too "magic-y")
3. Most importantly, we can't keep the "raw" dict in some self. instance variable, because that will affect concurrent POST requests.

One way to resolve those issues is to use self. and to disallow concurrent requests. meaning

1. all POST requests are now queued, and processed one by one (this could be at flask-level, or could be via a MQ component in orchestrated containerized runtime)
2. if there's an absolute necessity for concurrent process, it's now container orchestration's (e.g., k8s) responsibility, assuming all apps are always running as containers under an orchestration implementation (which I don't think realistic or practical assumptions).

The other way is to call get_configuration in the wrapper annotate() method (at the SDK level, so devs don't have to worry about it) and pass two dicts (raw and refined) to the internal _annotate() call, and then somehow we inject sign_view to every new_view call automatically with the raw one from the two. The main problem here is that it will change the signature of the _annotate() method, and it will break all existing apps (plus, we don't know how to do that "somehow" yet).

[keighrim]

#181 is following the non-self. way proposed in the above, except for automatic association between new_view and sign_view. However, since #179 , any "unsigned" views will result in errors during serialization, it will work as a forcing function for devs to use sign_view methods .

Keeping this issue open for now, until clamsproject/mmif#208 is closed due to the following TODO item:

clams-python/clams/app/__init__.py

Lines 207 to 208 in 38fd45f

	# TODO (krim @ 8/2/23): add "refined" parameters as well
	# once https://github.com/clamsproject/mmif/issues/208 is resolved

[keighrim]

closing the issue as clamsproject/mmif#208 is recently closed.

[keighrim]

re-opening this issue since the TODO mentioned above hasn't really been fixed;
TODO item:

clams-python/clams/app/__init__.py

Lines 207 to 208 in ded8505

	# TODO (krim @ 8/2/23): add "refined" parameters as well
	# once https://github.com/clamsproject/mmif/issues/208 is resolved

[keighrim]

reopening issue as clamsproject/mmif#208 was re-opened and fixed by clamsproject/mmif#223. Here we need to add support for appConfiguration (instead of refinedParameters).