Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Lack parameter] javacpp-pytorch Transformer Encoder Decoder lack parameter #1595

Open
mullerhai opened this issue Mar 9, 2025 · 2 comments

Comments

@mullerhai
Copy link

Hi, @saudet , compare with python pytorch offical doc ,javacpp pytorch Transformer Encoder Decoder lack some parameters ,

python need full four parameters : [ layer_norm_eps, batch_first,norm_first, bias, ] [enable_nested_tensor]
but the javacpp Transformer Encoder Decoder options dan't have ,I not know if cpp libtorch have these parameters or not.

https://pytorch.org/docs/2.3/generated/torch.nn.TransformerEncoderLayer.html
https://pytorch.org/docs/2.3/generated/torch.nn.TransformerDecoderLayer.html
https://pytorch.org/docs/2.3/generated/torch.nn.TransformerEncoder.html
https://pytorch.org/docs/2.3/generated/torch.nn.TransformerDecoder.html

layer_norm_eps ([float](https://docs.python.org/3/library/functions.html#float)) – the eps value in layer normalization components (default=1e-5).

batch_first ([bool](https://docs.python.org/3/library/functions.html#bool)) – If True, then the input and output tensors are provided as (batch, seq, feature). Default: False (seq, batch, feature).

norm_first ([bool](https://docs.python.org/3/library/functions.html#bool)) – if True, layer norm is done prior to attention and feedforward operations, respectively. Otherwise it’s done after. Default: False (after).

bias ([bool](https://docs.python.org/3/library/functions.html#bool)) – If set to False, Linear and LayerNorm layers will not learn an additive bias. Default: True.

·```
enable_nested_tensor

(bool) – if True, input will automatically convert to nested tensor (and convert back on output). This will improve the overall performance of TransformerEncoder when padding rate is high. Default: True (enabled).

@saudet
Copy link
Member

saudet commented Mar 9, 2025

The C++ API doesn't have all the features of the Python API, that's probably normal.

@mullerhai
Copy link
Author

The C++ API doesn't have all the features of the Python API, that's probably normal.

maybe need myself coding

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants