Ordering dictionary does not yield expected results when using multiple startswith entries

[jmcgill298]

I do not get a match under ordering when I use a nested lineage (interface subcommands); but it works when I only match the parent line.

Here is my config data:
Running:

interface GigabitEthernet1/0/18
 ip access-group TEST in
!
ip access-list extended TEST
 permit tcp host 10.1.1.1 host 10.1.1.2 eq 443
!
end

Intended:

interface GigabitEthernet1/0/18
 ip access-group TESTING in
!
ip access-list extended TESTING
 permit tcp host 10.1.1.2 host 10.1.1.1 eq 443
!
end

Here is a non-working ordering

Options:

ordering:
- lineage:
  - startswith:
    - ip access-list
    - access-list
  order: 100
- lineage:
  - startswith: interface
  - startswith: ip access-group
  order: 200
- lineage:
  - startswith:
    - no ip access-list
    - no access-list
  order: 300

idempotent_commands:
- lineage:
  - startswith: interface
  - startswith:
    - ip access-group

Result:

"ip access-list extended TESTING",
"  10 permit tcp host 10.1.1.2 host 10.1.1.1 eq 443",
"no ip access-list extended TEST",    ## This should have a weight of 300 (it actually does based on teh next results) ##
"interface GigabitEthernet1/0/18",   ## The next two lines should have a weight of 200 (does not since the above is 300) ##
"  ip access-group TESTING in"

I tested without using a nested lineage and I get the expected results

Options:

ordering:
- lineage:
  - startswith:
    - ip access-list
    - access-list
  order: 100
- lineage:
  - startswith: interface      ## Only matching the parent line ##
  order: 200
- lineage:
  - startswith:
    - no ip access-list
    - no access-list
  order: 300

idempotent_commands:
- lineage:
  - startswith: interface
  - startswith:
    - ip access-group

Result:

"ip access-list extended TESTING",
"  10 permit tcp host 10.1.1.2 host 10.1.1.1 eq 443",
"interface GigabitEthernet1/0/18",
"  ip access-group TESTING in"
"no ip access-list extended TEST",

[aedwardstx]

ordering:
- lineage:
  - startswith: interface
  - startswith: ip access-group
  order: 200

The behavior:
In the above example, the order weight will apply to the second level matching object startswith: ip access-group but not the parent object startswith: interface which will remain at the default of 500. We do not support conditional ordering of parents based on children using the lineage rules mechanism. Also, we don't natively support cases where remediation would have two or more tree views of the same object rendered at different order points.

Recommendation:

ordering:
- lineage:
  - startswith:
    - ip access-list
    - access-list
  order: 499
- lineage:
  - startswith:
    - no ip access-list
    - no access-list
  order: 501

The default ordering weight is 500. This should result in:

ip access-list extended TESTING
  10 permit tcp host 10.1.1.2 host 10.1.1.1 eq 443
interface GigabitEthernet1/0/18
  ip access-group TESTING in
no ip access-list extended TEST

Which is what you likely want.

As your use of HConfig grows, you will find it necessary to have fix up methods that fire after running and startup config is loaded to make various corrections/normalizations. The same is true for post remediation command generation. In this case, you are looking to clean up unused objects; an ACL in this case. We have unused object removal methods in our internal project. We will discuss open sourcing them. If you really really really need conditional parent ordering, I suggest creating a remediation config fix up method that conditionally sets the order weight prior to returning the textual remediation config. We haven't found a need to do this yet.

Actions:

• I think we need to better document the ordering behavior Update documentation on ordering weight #16
• Consider open sourcing unused object removal methods Add unused object removal functions #15

[jmcgill298]

What is a use case for not giving the parent line the same weight as the child line(s)?

[aedwardstx]

Ordering is required at different levels. Consider the below example:

ip access-list example !order weight 499 <- you want this to happen first
  permit tcp any 22 !order weight 500 default
interface eth1/1 !order weight 500 default <- you want this to happen after ACL update/creation
  vrf forwarding DMZ !order weight 499 <- you want VRF modification to happen first
  ip address 1.2.3.4/24 !order weight 500 default
  ip access-group example in !order weight 501 <- likely want ACL application to happen last

[jmcgill298]

This is how I would have thought to implement that:

ordering:
- lineage:
  - startswith:
    - ip access-list
    - access-list
  order: 100
- lineage:
  - startswith: interface
  - startswith: vrf
  order: 110
- lineage:
  - startswith: interface
  - startswith: ip address
  order: 120
- lineage:
  - startswith: interface
  - startswith: ip access-group
  order: 130

The child line cannot be written without the parent line coming first, so it seems more intuitive to me that parent inherits the lowest weight from all of its children.

[aedwardstx]

When considering this, think of the remediation config as whole. There are cases where the remediation will be hundreds(we've had cases in the thousands) of lines long going down several levels of hierarchy. The ordering ruleset needs to be able to set ordering on these levels of hierarchy independent of one another. Setting the ordering weight of a child of interface x needs to be a separate action then setting the ordering weight of interface x.