External operations

[asbjornu]

In discussions related to Introspected REST with @vasilakisfil on HTTPAPIs Slack, a use case we might want to support in Hydra materialised that I haven't thought about before: A way to reference external operations. It could look this:

{
  "operation": "https://example.api/resource-state-specific-operations.json"
}

{
  "operation": [
    "https://example.api/more-static-operations.json",
    {
      "target": "...",
      "etc": "..."
    }
  ]
}

So, operation could, as @context, be either a URI, an object, or an array with the mix of both. Thoughts?

[tpluscode]

I don't see why not. Distributed to the core. It's not something that has been discussed as of yet but in general sounds good.

The only difference from your snippet would be to have the representation of resource-state-specific-operations.json conform to the Hydra Core vocabulary.

Otherwise yes, why not allow them to be linked and dereferenced. The obvious drawback is of course increased chattiness.

[elf-pavlik]

Otherwise yes, why not allow them to be linked and dereferenced. The obvious drawback is of course increased chattiness.

I think we can not force including operations inline and must also allow IRI + dereference approach. Also with HTTP/2 I woudn't worry that much about 'chattiness' any more.

[alien-mcl]

Well - every resource can be dereferenced according to linked data, so can the operations. It makes clients more complex, but still - doable

[lanthaler]

I think the only question here is whether we want to force conformant clients to go and dereference operations (and thus move additional burden from the server to the client).

[tpluscode]

By force you mean exposing an API to dereference it on demand or doing so automatically? I'd go with the former.

[lanthaler]

In order to implement an API it needs to be known what can be expected to be supported by clients. The Hydra spec can either require clients to support such things or stay silent about it. The latter means API providers could not assume that clients would support this and would probably not use such a feature. The former means all clients would become slightly more complex as they would need to dereference operations.

[elf-pavlik]

The former means all clients would become slightly more complex as they would need to dereference operations.

How much more complexity do you see in dereferencing operations? I think if client lazy loads those operations it would just force code that selects them to stay asynchronous. Clients which would load them eagerly could keep code that selects them synchronous. Parsing of JSON-LD already has to happen asynchronous because of potential external @context so I think that external operations would not introduce additional complexity.

What we might have to address - operations shared by more than one resource. Given "target": { "@reverse": "operation" }, multiple resources referencing the same operation could result in that operation 'having multiple targets'.

[lanthaler]

How much more complexity do you see in dereferencing operations? I think if client lazy loads those operations it would just force code that selects them to stay asynchronous. Clients which would load them eagerly could keep code that selects them synchronous.

Implementation complexity is just one factor. Runtime complexity/performance and failure modes are another aspect to keep in mind.

Parsing of JSON-LD already has to happen asynchronous because of potential external @context so I think that external operations would not introduce additional complexity.

Yeah, we should probably also "restrict" that somehow for the reasons mentioned above. There are various machine clients (the Google crawler being one of them) that don't support remote contexts at all for instance.

What we might have to address - operations shared by more than one resource. Given "target": { "@reverse": "operation" }

Yeah, I think this is a pretty good argument for, generally, not giving operations an IRI.

[asbjornu]

While I agree with all of your concerns, I don't see any of them as blockers for Hydra providing this as a feature. The problem with reverse operations, for instance, only exists for APIs that use reverse operations, which I assume most won't.

I also assume most APIs won't use external operations, but those who do might do it to increase cache hits on operations that are similar across many different responses, for instance.

@vasilakisfil's use-case is to simplify the responses so much so that they look just like "ordinary", hypermedia-free JSON. While I disagree with that design goal, I can see why it exists and how it may attract certain kinds of developers.

[elf-pavlik]

I also assume most APIs won't use external operations, but those who do might do it to increase cache hits on operations that are similar across many different responses, for instance.

Here we get into

What we might have to address - operations shared by more than one resource. Given "target": { "@reverse": "operation" }

Yeah, I think this is a pretty good argument for, generally, not giving operations an IRI.

@asbjornu can you provide an example of operation that you might want to reference from multiple resources to 'increase cache hits`? Operations tend to stay very small so IRI vs. the whole operation inline doesn't seem to make much difference.

[vasilakisfil]

Hi, thanks for your input! Actually, my idea about Hydra's operations was to somehow decouple the operations part of the spec, from other parts like the structure of the data or the structure of other metadata (like links). The idea comes from my conceptual idea of MicroTypes (more info on [1] and [2]).

I can see 2 improvements: In the first step (described in (1)), is to have dereferencable links. The second step (explained in (2)), is to support different, alternative operations descriptions, using different relation types (as explained in RFC8288).

To elaborate more:

1. Today I have a client that understands pretty well Hyda's operations. By providing the operations on the side, using an IRI, the client could fetch them whenever is needed and we don't have to provide them always on the same document. I think this alone has many advantages that are obvious and personally I think all APIs should be like that (that's a personal opinion though, expressed in detail in [3]), but if it's not obvious to everyone let me know, I can expand on that :)

2. Tomorrow I want to have a client that supports more advanced operations. Hydra limits me to do that. I mean, Hydra is really good now, but as always nothing is timeless. So we need to provide a way to the client to let it know that hey! we also support operations V2, that is more advanced and provide more cool things. But if you want V1, you can also have that. I think by adding such capabiltities to the Hydra spec, we make it much more robust. Basically, dereferencable links, each one with different relation type (for distinguishing operations V1, V2 etc).

The idea of (2) is to make the spec extensible, but not arbitrarily but instead provide a platform for the extensions to step on. (because in theory, I can just have another attribute key, like operationsV2 and have there all my new logic, but that's not a good solution because we want to have a common API for all operations specs)

I hope I made my self clear, if not let me know where I should expand :)

[1] https://introspected.rest/#92-microtypes-reusable-modules-composing-a-media-type
[2] https://www.youtube.com/watch?v=pc8ZyFjJY4A
[3] https://introspected.rest

[vasilakisfil]

@asbjornu can you provide an example of operation that you might want to reference from multiple resources to 'increase cache hits`? Operations tend to stay very small so IRI vs. the whole operation inline doesn't seem to make much difference.

Not sure if it will increase cache hits, but being able to dereference operations allows you to have different caching policies from your main resource/document and this is very important, I think, because not all API parts have the same volatility. Also, the IRIs are much smaller from operation details in the general case, but especially in some extreme cases that you want to specify a bunch of operations along with supported types etc. Providing such capabilities are vital for having a good spec that adapts to the API designer's needs.