Handle deprecation of tf.batch_matmul

[milost]

During the execution of TransR, the following error occurs:

Traceback (most recent call last):
  File "/Users/milost/anaconda/envs/pyKE/bin/pyke", line 11, in <module>
    load_entry_point('pyKE', 'console_scripts', 'pyke')()
  File "/Users/milost/anaconda/envs/pyKE/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/milost/anaconda/envs/pyKE/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/milost/anaconda/envs/pyKE/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/milost/anaconda/envs/pyKE/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/milost/anaconda/envs/pyKE/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/milost/Code/Python/pyKE/pyke/bin/pyke.py", line 68, in transr
    margin=margin,
  File "/Users/milost/Code/Python/pyKE/pyke/embedding.py", line 52, in __init__
    self.__init_config()
  File "/Users/milost/Code/Python/pyKE/pyke/embedding.py", line 78, in __init_config
    con.set_model(self.model_class)
  File "/Users/milost/Code/Python/pyKE/pyke/openke/Config.py", line 257, in set_model
    self.trainModel = self.model(config=self)
  File "/Users/milost/Code/Python/pyKE/pyke/models/Model.py", line 87, in __init__
    self.loss_def()
  File "/Users/milost/Code/Python/pyKE/pyke/models/TransR.py", line 51, in loss_def
    p_h = tf.reshape(self._transfer(pos_matrix, pos_h_e), [-1, config.rel_size])
  File "/Users/milost/Code/Python/pyKE/pyke/models/TransR.py", line 10, in _transfer
    return tf.batch_matmul(transfer_matrix, embeddings)
AttributeError: module 'tensorflow' has no attribute 'batch_matmul'

According to this issue the tf.batch_matmul() op was removed in 3a88ec0. So we need to replace to the call to tf.batch_matmul with another call, probably something like tf.linalg.matmul

[milost]

Simply replacing tf.batch_matmul with tf.matmul results in nan values for the loss.

[Epoch 0] Loss: nan
[Epoch 1] Loss: nan
[Epoch 2] Loss: nan
[Epoch 3] Loss: nan
...

Since this straight forward solution doesn't work, a closer look is necessary.

[Mayo42]

AFAIK tf.batch_matmul was designed to handle matrix multiplications over a batch. For example, mat-multiplying tensors with shape [100, 20, 10] and [100, 10, 20] would result in a tensor with shape [100, 10, 10] where 100 is the batch size and 10, 10 the resulting tensors foreach sample.

Looking at the code for TransR, i would try to change the reshaping of the h_e and t_e tensors from [-1, config.ent_size, 1] to [-1, config.ent_size] and instead of performing tf.batch_matmul simply use tf.multiply and tf.reduce_sum to perform a simple dot product. This should work with the following _transfer function:

    def _transfer(self, transfer_matrix, embeddings):
        multiplied = tf.multiply(transfer_matrix, embeddings)
        return tf.reduce_sum(multiplied, axis=-1)

EDIT:

I'm not sure if the broadcasting semantics will work in the _transfer function above.
If this happens to not work, i suggest to expand the dimensions of embeddings and tile them.
This should definitely work:

    def _transfer(self, transfer_matrix, embeddings):
        tiled_embeddings = tf.expand_dims(embeddings, axis=1)
        tiled_embeddings = tf.tile(tiled_embeddings, [1, config.rel_size, 1])
        multiplied = tf.multiply(transfer_matrix, tiled_embeddings)
        return tf.reduce_sum(multiplied, axis=-1)

Hope this helps