You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, @saudet , compare with python pytorch offical doc ,javacpp pytorch Transformer Encoder Decoder lack some parameters ,
python need full four parameters : [ layer_norm_eps, batch_first,norm_first, bias, ] [enable_nested_tensor]
but the javacpp Transformer Encoder Decoder options dan't have ,I not know if cpp libtorch have these parameters or not.
layer_norm_eps ([float](https://docs.python.org/3/library/functions.html#float)) – the eps value in layer normalization components (default=1e-5).
batch_first ([bool](https://docs.python.org/3/library/functions.html#bool)) – If True, then the input and output tensors are provided as (batch, seq, feature). Default: False (seq, batch, feature).
norm_first ([bool](https://docs.python.org/3/library/functions.html#bool)) – if True, layer norm is done prior to attention and feedforward operations, respectively. Otherwise it’s done after. Default: False (after).
bias ([bool](https://docs.python.org/3/library/functions.html#bool)) – If set to False, Linear and LayerNorm layers will not learn an additive bias. Default: True.
·```
enable_nested_tensor
(bool) – if True, input will automatically convert to nested tensor (and convert back on output). This will improve the overall performance of TransformerEncoder when padding rate is high. Default: True (enabled).
The text was updated successfully, but these errors were encountered:
Hi, @saudet , compare with python pytorch offical doc ,javacpp pytorch Transformer Encoder Decoder lack some parameters ,
python need full four parameters : [ layer_norm_eps, batch_first,norm_first, bias, ] [enable_nested_tensor]
but the javacpp Transformer Encoder Decoder options dan't have ,I not know if cpp libtorch have these parameters or not.
https://pytorch.org/docs/2.3/generated/torch.nn.TransformerEncoderLayer.html
https://pytorch.org/docs/2.3/generated/torch.nn.TransformerDecoderLayer.html
https://pytorch.org/docs/2.3/generated/torch.nn.TransformerEncoder.html
https://pytorch.org/docs/2.3/generated/torch.nn.TransformerDecoder.html
·```
enable_nested_tensor
(bool) – if True, input will automatically convert to nested tensor (and convert back on output). This will improve the overall performance of TransformerEncoder when padding rate is high. Default: True (enabled).
The text was updated successfully, but these errors were encountered: