kit/sync-modules fails without id_rsa, and fails with a different error after I create it or set id_ed25519

[patrixl]

I use ed25519 instead of rsa, so when I try to dync modules:

user> (kit/sync-modules)
failed to clone module: [email protected]:kit-clj/modules.git 
cause: [email protected]:kit-clj/modules.git: java.io.FileNotFoundException: /Users/xxx/.ssh/id_rsa (No such file or directory)

So reading the code I found out about kit.git-config.edn where I can give the name of my key, but replacing id_rsa with id_ed25519, I get:

user> (kit/sync-modules)
failed to clone module: [email protected]:kit-clj/modules.git 
cause: [email protected]:kit-clj/modules.git: invalid privatekey: [B@4cf19be

[nikolap]

I think this might be down to the underlying JGit library that clj-jgit is using. I'll see about upgrading it and testing with ed25519. There was a historical issue with the lib that it had issues handling ed25519 and ECDSA-384/521 but was resolved in v5.2

[nikolap]

Will need to investigate some more, looks like not such a simple fix... To summarize so far: kit-generator uses an old version of clj-jgit (0.8.10) due to issues with the latest version being unable to handle password protected SSH keys. However, this version cannot implement the fixes that jgit 5.2 provides, which includes ed25519 support.

The latest version of clj-jgit would support jgit up to the latest 5.x version (6.0 seems unlikely since there are breaking API changes), but we need to figure out how to get it to work with password protected keys first before using it. It also requires some code changes because the clj-jgit API breaks from 0.8 -> 1.0

[patrixl]

OK, so ed25519 is out for the moment. I followed up by creating an RSA key, and restarting the repl, now I get:

user> (kit/sync-modules)
2022-01-09 17:06:54,655 [nREPL-session-590ea496-1654-4758-9e2d-f36504906f97] DEBUG org.eclipse.jgit.util.FS - readpipe [git, --version],/opt/local/bin 
2022-01-09 17:06:54,675 [nREPL-session-590ea496-1654-4758-9e2d-f36504906f97] DEBUG org.eclipse.jgit.util.FS - readpipe may return 'git version 2.34.1' 
2022-01-09 17:06:54,676 [nREPL-session-590ea496-1654-4758-9e2d-f36504906f97] DEBUG org.eclipse.jgit.util.FS - remaining output:
 
2022-01-09 17:06:54,676 [nREPL-session-590ea496-1654-4758-9e2d-f36504906f97] DEBUG org.eclipse.jgit.util.FS - readpipe [git, config, --system, --edit],/opt/local/bin 
2022-01-09 17:06:54,684 [nREPL-session-590ea496-1654-4758-9e2d-f36504906f97] DEBUG org.eclipse.jgit.util.FS - readpipe may return '/opt/local/etc/gitconfig' 
2022-01-09 17:06:54,684 [nREPL-session-590ea496-1654-4758-9e2d-f36504906f97] DEBUG org.eclipse.jgit.util.FS - remaining output:
 
failed to clone module: [email protected]:kit-clj/modules.git 
cause: [email protected]:kit-clj/modules.git: invalid privatekey: [B@1bcad448

[nikolap]

Is the kit.git-config.edn still pointing to the old key? This should be the default config from the template which points to an RSA key:

{:name       "~/.ssh/id_rsa"
 :passphrase ""
 :options    {"StrictHostKeyChecking" "no"
              "UserKnownHostsFile"    "/dev/null"}
 :exclusive  true}

[patrixl]

yeah, I nuked the test app and re-created with clj-new, and I also created a new rsa ssh key with ssh-keygen -t rsa

% ls ~/.ssh/id_rsa
/Users/xxx/.ssh/id_rsa

[nikolap]

Think found the issue...

Does this key generation work for you?

ssh-keygen -t rsa -m PEM

I can also reproduce it now by generating a new key. Apparently the header below isn't supported in SSH keys with jgit which is the newer one produced by OpenSSH 7.8+.

-----BEGIN OPENSSH PRIVATE KEY-----

Separately... while we needed the key authentication for when the modules repo was private we don't at the moment. Though people should be able to host their own private modules repos, so the feature should be supported but not required.

@yogthos So two things:

• We'll need to try to either upgrade clj-jgit or find some other alternative. Looks like the version we're using has some unfortunate bugs.
• Also let's see if we can make the git ssh optional -- resolved in io.github.kit-clj/lein-template 0.1.1 using HTTPS instead of SSH

[patrixl]

yup, the PEM format worked! Thanks!

And agreed that a key should not be a requirement to get something that is public..

[nikolap]

Np, thanks for reporting/helping troubleshoot it 👍 Will leave the ticket open until we resolve the issues

[fonghou]

using https repository url worked for me (even without any .ssh keys).

kit.edn

{:full-name "kit/guestbook"
 :ns-name   "kit.guestbook"
 :sanitized "kit/guestbook"
 :name      "guestbook"
 :modules   {:root         "modules"
             :repositories [{:url  "https://github.com/kit-clj/modules.git"
                             :tag  "master"
                             :name "kit-modules"}]}}

kit.git-config.edn

{:name       ""
 :passphrase ""
 :options    {"StrictHostKeyChecking" "no"
              "UserKnownHostsFile"    "/dev/null"}
 :exclusive  true}

[patrixl]

@fonghou that works for me too, and I didn't need to blank out my ssh key name or passphrase in kit.git-config.edn.

And it looks like https is the default now instead.

[tragiclifestories]

I'm afraid I'm having this issue and none of the proposed changes make any difference. Whether the key is generated as a PEM or not, I get

failed to clone module: https://github.com/kit-clj/modules.git
cause: [email protected]:kit-clj/modules.git: Couldn't find any key file(s), tried: ~/.ssh/id_rsa in /Users/tragiclifestories/.ssh

Also, despite the fact that the modules URL is a https url in the config file, something or other is rewriting it as an SSH url (as you can see from the error).

Is there a way I can just clone this in the right place manually with git to unblock myself?

[tragiclifestories]

I finally got a workaround going - set the contents of the git credentials file to {}:

echo '{}' > kit.git-config.edn

This obviously doesn't help in the case where one actually does have a private repo full of kit modules, of course. But I expect setting HTTPS user/pass credentials in that file will work, since clj-jgit certainly has support for that. Not sure what the schema would be though.

If I get a moment I'll take a closer look at what exactly is failing to load my private key and why ...

PS: I also wonder if a more appropriate default credential here would be {}. I deleted the credentials file to try and get kit not to authenticate, and that feels like a more logical expectation. But maybe that's just me.

[sebastian-palma]

fonghou's advice seems to work.

getting a similar error when executing sync-modules;

user=> (kit/sync-modules)
failed to clone module: https://github.com/kit-clj/modules.git
cause: [email protected]:kit-clj/modules.git: Couldn't find any key file(s), tried: ~/.ssh/id_rsa in /Users/yonomellamojavier/.ssh
org.eclipse.jgit.api.errors.TransportException: [email protected]:kit-clj/modules.git: Couldn't find any key file(s), tried: ~/.ssh/id_rsa in /Users/yonomellamojavier/.ssh

leaving the name empty in kit.git-config.edn does the trick;

{:name       ""
 :passphrase ""
 :options    {"StrictHostKeyChecking" "no"
              "UserKnownHostsFile"    "/dev/null"}
 :exclusive  true}

[yogthos]

Might make sense to update the module to give a better error.