--interact is not working

[ataymano]

Describe the bug

--interact is supposed to produce elementwise product features (so equivalent to --interactions in single features namespace case?), but looks like it is doing something else.

How to reproduce

data:

import numpy as np

def datagen(n, path):
    with open(path, 'w') as f:
        for _ in range(n):
            first = np.random.randint(0, 2)
            second = np.random.randint(0, 2)
            f.write(f'{first * second} |a x:{first} |b x:{second}\n')

datagen(10000, 'data.txt')

no interactions:

vw --coin --noconstant -d data.txt --readable_model model.txt

output:

...
finished run
number of examples = 10000
weighted example sum = 10000.000000
weighted label sum = 2443.000000
average loss = 0.085632
best constant = 0.244300
best constant's loss = 0.184618
total feature number = 9938

outer product:

vw --coin --noconstant -q ab -d data.txt --readable_model model.txt

output:

...
finished run
number of examples = 10000
weighted example sum = 10000.000000
weighted label sum = 2443.000000
average loss = 0.000205
best constant = 0.244300
best constant's loss = 0.184618
total feature number = 12381

elementwise products:

vw --coin --noconstant --interact ab -d data.txt --readable_model model.txt

output:

...
finished run
number of examples = 10000
weighted example sum = 10000.000000
weighted label sum = 2443.000000
average loss = 0.128257
best constant = 0.244300
best constant's loss = 0.184618
total feature number = 9938

If we look at readable model (for --interact run):

initial_t 0
norm normalizer 7495
t 10000
sum_loss 1282.57
sum_loss_since_last_dump 237.63
dump_interval 16384
min_label 0
max_label 1
weighted_labeled_examples 10000
weighted_labels 2443
weighted_unlabeled_examples 0
example_number 10000
total_features 9938
total_weight 10000
sd::oec.weighted_labeled_examples 8192
current_pass 1
l1_state 0
l2_state 1
:55432:0.407074 160.891 5010.24 1 14.6625 2
b^x:143313:0 0 0 1 0 0

Both a^x and interaction feature are missing.

Version

9.8.0

OS

Windows

Language

CLI

Additional context

No response

[ataymano]

1. It only works on dense namespaces with first feature has to be equal to 1 (from here)
2. explicit 0 values are getting ignored by parser
   So, dataset should be changed to (in order to demonstrate happy path):

0.24 |a 0:1 1:0.88 |b 0:1 1:0.28
0.06 |a 0:1 1:0.46 |b 0:1 1:0.14
...

[ataymano]

Tried to cleanup anchor features here but cancelled PR (#4646) after conversation with @jackgerrits: not having anchor features is breaking offsets calculations if first feature turned out to be 0.
Leaving unchanged for now but should be used very carefully.
Counter-intuitive effects with current implementation:

1. Working on feature group level, so having a lot of unexpected behavior when more than one namespace with same first letter exists.
2. Destination hash is calculated based on first namespace (from commandline) => non-commutative, "--interact ab" and "--interact ba" are producing different hashes
3. Destination hashes non sensitive to hash shift of second argument:
   for "--interact ab" configuration:
   "|a 0:1 1:1 |b 0:1 1:1" is producing the same hashes as "|a 0:1 1:1 |b 3:1 4:1", but dufferent from "|a 3:1 4:1 |b 0:1 1:1

[ataymano]

wiki page is created:
https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Element%E2%80%90wise-interaction