Support "IN" operator in govaluate statements

[palourde]

At this moment, the IN operator can't be used in govaluate.

For example, if I use the statement "foo" in entity.Subscriptions, govaluate will return:

failed to evaluate statement: "foo" in entity.Subscriptions error="Value '[bar foo]' cannot be used with the comparator 'in', it is not a number"

I believe it's because the value would need to be in the form of ("foo", "bar") instead of [foo bar] so we will probably need to create a custom function that users can call to support that format.

[grepory]

That seems... Strange. This feels more like a govaluate bug. Can we make sure that govaluate just doesn’t support arrays?

[palourde]

That's definitively a bug in govaluate IMO (see Knetic/govaluate#79) but considering that we forked that package and that Eric's PR is still not merged, I would be tempted to say we need to either fix that with a custom function or by modifying govaluate itself to support that.

[grepory]

We could just keep maintaining our fork and merge upstream changes into it over time.

[palourde]

Additionally, I just realized we can't use a comma in the filter statements, in the current version of sensuctl, since we use them to separate statements, e.g.:

? Statements (comma separated list): weekday(timestamp) in (1, 2, 3, 4, 5)
Error: invalid statement 'weekday(timestamp) in (1': Unbalanced parenthesis

[grepory]

Can we change the way we input filters to not use commas to separate statements?

[jamesdphillips]

Didn't find this issue when I previously searched and created a similar ticket. 😬

Copying over my investigation:

When using the IN operator the right side (RHS) of the expression must be an instance of []interface. This, for instance, makes it frustrating when working the Subscriptions field ([]string) on the Check and Entity types.

Expected Behavior

I would expect to be able write the expression 'unix' IN entity.Subscriptions, however since the RHS of the expression is a slice of strings, this will always evaluate to false.

Possible Solution

A few possible avenues.

a) Expose a function

• We could expose a function (eg. sliceContains) that allows the user to check if an identifier has a specified value.
• In this case, to avoid any confusion. it may make sense to simply remove IN.

// Example expression
entity.LastSeen > 12345678 && sliceContains(entity.Subscriptions, 'unix')

Drawbacks

• Lots of type assertions, I guess
• Functions require extra documentation
• Without any sort of module/packages/namespaces, we run the risk of identifiers colliding

b) Add to types package

I believe, we potentially could add a type with some helpful methods familiar to those who write Python and Ruby.

type MyStringSlice []string

func (MyStringSlice) Count() int {
  // ...
}

func (MyStringSlice) Contains(interface{}) {
  // ...
}

type Entity struct {
  // ...
  Subscriptions MyStringSlice
  // ...
}

// example expression
entity.LastSeen > 12345678 && entity.Subscriptions.Contains('unix')

Drawbacks

• Is protobuf going to make this needlessly painful?
• Accessors feature makes heavy use of reflection
• Methods require extra documentation

c) Expand govaluate

Update govaluate so that IN operator can handle greater range of expressions.

I figure this could partially be optimized at the planning stage;

• if the RHS of the expression is an array literal we know the RHS should be cast to []interface
• if the LHS of the expression is a string, integer, float literal, etc. we know that the RHS should be cast to []string, []int, []float, etc.
• becomes more tricky when the type of the LHS isn't as easy to infer; eg. (1 + 3 / 6.0) IN myIdentifier
• (in the planner) becomes nearly impossible when the LHS is an identifier. myIdentifier IN otherIdentifer

// the follow example expression becomes viable
entity.LastSeen > 12345678 && 'unix' IN entity.Subscriptions

Drawbacks

• Planning and in some cases evaluation likely becomes a bit slower.
• Potentially tricky edge cases

Context

• Initially ran into this while trying to filter entities by subscriptions for the web UI.
• For what it's worth this behavior is documented in govaluate's MANUAL.
• Blocks [Web] Entities page: filter #839

... I don't have a particularly strong opinion with regards to what is the best solution and open to any thoughts.

[echlebek]

I made a release of github.com/sensu/govaluate that fixes this issue.