Running on z/OS to receive file from z/OS failed SSH_FX_BAD_MESSAGE

[drimmeer]

Please help!
I just realized that when I ran the code on a z/OS to receive a file from another z/OS, it always failed when trying to open the sftpClient, with error message:
sftp: "Bad message" (SSH_FX_BAD_MESSAGE)

Please note that sending file from z/OS to z/OS is successful;
And receiving/sending file between Linux and z/OS is also successful!

I tested this on 4 z/OS machines already...

I am using the 'master' branch, which is v1.13.6

Here is the related code:

	sshConfig := &ssh.ClientConfig{
		User:            argStruct.RemoteUserName,
		HostKeyCallback: ssh.InsecureIgnoreHostKey(),
		Auth: []ssh.AuthMethod{
			ssh.Password(argStruct.RemotePassword),
		},
	}
	sshClient, err := ssh.Dial(sshDailNetwork, remote_hostname_and_port, sshConfig)
	concurrentWriteOption := sftp.UseConcurrentWrites(true)
	concurrentReadOption := sftp.UseConcurrentReads(true)
	maxConcurrentOption := sftp.MaxConcurrentRequestsPerFile(100) // 100 was found to be the best during some testing
	sftpClient, err := sftp.NewClient(sshClient, concurrentReadOption, concurrentWriteOption, maxConcurrentOption)
	fSource, err := sftpClient.Open(srcNameRemote)

[puellanivis]

Are you trying to receive the files from a remote z/OS ssh implementation? Or from our server implementation?

Do you have any logs from the z/OS machine’s ssh about the packets?

Could you maybe try out the v2 branch? The NewClient call changes slightly, and doesn’t take any options yet. Though, it definitely won’t need either of the WithConcurrentXY calls in the future, but I’ll probably need to adjust it to fit the MaxConcurrentRequestsPerFile design principle for your needs. Right now, it just has a global maximum of in-flight packets, and it’s hardcoded to 64. (YAGNI principle, don’t build a feature until there’s a real-world need.)

If the two implementations differ somehow, that could be a hint into what’s breaking…

[drimmeer]

Thank you for the response.

1. Yes, I am trying to receive the files from a remote z/OS ssh implementation.

2. I managed to capture the sshd log on debug level, on the remote z/OS, and attach it here.
   The log for receiving files from z/OS to z/OS (which was failed):
   server.debuglog.receive.part.txt

The log for sending files from z/OS to z/OS (which was successful):
server.debuglog.send.part.txt

3. I am trying to get the v2 branch as suggested, but got this error when running go get "github.com/pkg/sftp@v2":
   go: github.com/pkg/sftp@v2: no matching versions for query "v2"

[drimmeer]

Hi Cassondra , good news!
After I took the master branch, the problem is gone!
Don't know why, but 'go get github.com/pkg/sftp' got the 1.13.6 version, which doesn't have the z/OS support now! while 'go get github.com/pkg/sftp@master' got the 1.13.7 version, which has the z/OS support.

So 1.13.7 solved the problem!

[puellanivis]

Huh, it could be that you already had the v1.13.6 version local, so it just used the cached value. 🤔

[drimmeer]

maybe...
Is there a way for me to fix this version so that I won't be impacted if it is changed?
I found that sometime ago I was able to wrap it to calculate the digest but recently I found that it didn't work! So I had to find another way to wrap it.

[puellanivis]

Ah, OK, I’ve released a new version, so we can ensure that z/OS support is in a released version. I hadn’t realized that we hadn’t released a version since.

[drimmeer]

Thank you Cassondra. What version is it? What do I put in my 'go get' and 'import' path ?

[puellanivis]

It is v1.13.7, you should be able to update your go.mod to have github.com/pkg/sftp v.1.13.7 the import path is the same as all of the v1 versions have used.

[puellanivis]

I’ve also corrected the issue with trying to use v2 as the branch name. I just renamed it to dev-v2 and it works now.

This is what I get for not testing things before I try and release them. 🤦‍♀️

[drimmeer]

When I tried this branch, I got different error.

When running go get github.com/pkg/sftp@dev-v2 on Mac, I got:
go: github.com/pkg/sftp@dev-v2: github.com/pkg/[email protected]: invalid version: go.mod has post-v0 module path "github.com/pkg/sftp/v2" at revision 5839350cfcba

But when running go get github.com/pkg/sftp@dev-v2 on z/OS, I got:
go: github.com/pkg/sftp@dev-v2: git init --bare in /u/jamiel/go/pkg/mod/cache/vcs/74e1affce0467ac134276b4073f643881e0cb81f11fcefd0045fc639aae1e3d1: exec: "git": executable file not found in $PATH

[puellanivis]

You need to use github.com/pkg/sftp/v2 as the import path, since this will be a v2 version.

[drimmeer]

When I tried this command, I still got error (sorry, I am quite dumb at using Go...):

go get github.com/pkg/sftp/v2    
go: module github.com/pkg/sftp@upgrade found (v1.13.7), but does not contain package github.com/pkg/sftp/v2

[puellanivis]

You need to still use @dev-v2 to target the branch.