gpg throws "You may want to update to a newer pinentry" error during signing git commit (fresh install)

[Dentrax]

After I applied the installation instructions step by step, I got an error while signing commit. The problem is in the communication between the gpg-agent and the pinentry program as the following debug output shows: ¹

$ git commit -v -s -S

gpg-agent[20618]: You may want to update to a newer pinentry
gpg-agent[20618]: failed to unprotect the secret key: Operation cancelled
gpg-agent[20618]: failed to read the secret key
gpg-agent[20618]: command 'PKSIGN' failed: Operation cancelled <Pinentry>
error: gpg failed to sign the data
fatal: failed to write commit object

I reload gpg-agent daemon:

$ gpg-agent --daemon; killall gpg-agent && gpg-agent --daemon --use-standard-socket --pinentry-program /usr/local/bin/pinentry-touchid

This one works:

$ echo GETPIN | pinentry
OK Pleased to meet you

gpg-agent:

$ gpg-agent --help
gpg-agent (GnuPG/MacGPG2) 2.2.27
libgcrypt 1.8.7

pinentry-mac:

$ pinentry --help
pinentry-mac (pinentry) 1.1.1

MacBook Pro (16-inch, Late 2019) - macOS 11 - Intel

But the pinentry-mac is working:

$ gpg-agent --daemon; killall gpg-agent && gpg-agent --daemon --use-standard-socket --pinentry-program /usr/local/bin/pinentry-mac

My output logs:

$ cat /tmp/pinentry-touchid.log

2021/10/24 00:09:50 main.go:105: Ready!
2021/10/24 00:09:50 main.go:256: Error calling pinentry-mac: unexpected response: ERR 83918950 Inappropriate ioctl for device <Pinentry>
2021/10/24 00:09:50 main.go:260: pinentry-mac didn't return a password

Tried this solution but no luck.

Any ideas here? 🤔

Footnotes

1. https://bugs.launchpad.net/ubuntu/+source/pinentry-qt4/+bug/281487 ↩

[Gby56]

I think my issue #11 is the same :/
But I get DBG: error calling pinentry: Operation cancelled <Pinentry>
Any updates on this ? @Dentrax

[jorgelbg]

Hi @Dentrax I was able to reproduce this locally when invoking the gpg-agent using the command from the terminal _even when the GPG_TTY environment variable is set in my environment.

If instead I specify the pinentry-program via the ~/.gnupg/gpg-agent.conf file and use gpg-connect-agent reloadagent /bye to restart the process:

❯ killall -9 gpg-agent; gpg-connect-agent reloadagent /bye
gpg-connect-agent: no running gpg-agent - starting '/usr/local/Cellar/gnupg/2.3.3_1/bin/gpg-agent'
gpg-connect-agent: waiting for the agent to come up ... (5s)
gpg-connect-agent: connection to the agent established
OK

Then pinentry-touchid is able to successfully call pinentry-mac.

That being said I'm not sure why it does not work when starting the daemon from the terminal.

[Dentrax]

Thanks, @jorgelbg! I tried this one but still no luck.

$ cat ~/.gnupg/gpg-agent.conf
pinentry-program /usr/local/opt/pinentry-touchid/bin/pinentry-touchid

$ gpg-agent --daemon; killall gpg-agent && gpg-agent --daemon --use-standard-socket --pinentry-program /usr/local/bin/pinentry-touchid

$ killall -9 gpg-agent; gpg-connect-agent reloadagent /bye

$ git commit -S
error: gpg failed to sign the data
fatal: failed to write commit object

$ cat /tmp/pinentry-touchid.log
2021/11/17 14:27:05 main.go:105: Ready!
2021/11/17 14:27:05 main.go:285: Duplicated entry in the keychain

[Dentrax]

@jorgelbg Hey! Any advices or help here? 🙏

[SunsetYe66]

@Dentrax You may need to:

• Open Keychain Access.app
• Search for gnupg related item
• Del it/them and make sure:
  • have executed defaults write org.gpgtools.common DisableKeychain -bool yes
  • have executed killall -9 gpg-agent; gpg-connect-agent reloadagent /bye

Then try again. This workaround saved me from this bug.

cc @jorgelbg Hey! You might want to check what's wrong with it?

[Dentrax]

I don't know exactly what defaults write org.gpgtools.common DisableKeychain -bool yes does under the hood so I didn't run the command to try. Can you please explain a bit these, and why I have to delete gnupg key?

cc @jorgelbg

[SunsetYe66]

Since it's too long for me to recall every detail, I may try my best to explain it.

I think pinentry-touchid is acted as the agent to translate between fingerprint and REAL GPG PASSWORD. This procedure can be securely done by macOS's built-in Keychain Access function, and the GPG suite's save in keychain does the same thing in the same way.

In my scenario, when the first time pinentry-touchid was executed, it fallbacks to pinentry-mac and ask you for GPG key password. The password will be stored in Keychain and you will be asked for access permission: pinentry-touchid -> GPG key password Keychain item.

But for an unknown reason, the GPG key password item is already stored in Keychain Access, and it can not be accessed by pinentry-touchid but is available for pinentry-mac. So it comes to my solution, defaults write org.gpgtools.common DisableKeychain -bool yes, delete GnuPG keychain access item, then try again.

For removing the keychain item, this may be relevant: #11 (comment)

Since you want more explanation and to be honest, I don't really know the principles in these procedures, I did a little more research. For reference, defaults write org.gpgtools.common DisableKeychain -bool yes means:

GPG Suite defaults to store OpenPGP passwords in macOS keychain. If you prefer to never store your OpenPGP passwords in macOS keychain, use the following setting. The option to store your password will be removed from the pinentry dialog and the setting in System Preferences > GPG Suite > Settings > Password will be disabled. source

And other people already mentioned this command, but in the opposite way: #3 (comment) . Based on the information we have known, maybe you can delete the item and try again. If it doesn't work, maybe the defaults write - delete item - try again procedure works.

BTW, I encountered the same error log blocks as yours, so I assume my solution may also work for you:

gpg-agent[20618]: failed to read the secret key
gpg-agent[20618]: command 'PKSIGN' failed: Operation cancelled <Pinentry>

[Dentrax]

Just remembered this one, any updates on this? @jorgelbg? 🙏

[jorgelbg]

@Dentrax I released v0.0.3 a few minutes ago, would you mind giving it a try? Just to be sure:

$ defaults write org.gpgtools.common DisableKeychain -bool yes

should indicate pinentry-mac (the program that we use to get your GPG password when it is not there) to not use the the keychain, as in it acts as a proxy just for getting your password. After pinentry-touchid gets a password from pinentry-mac it stores it by itself in the keychain. This helps prevent the password prompt for accessing the keychain item the next time that pinentry-touchid needs to get the password.

In either case if the item in the keychain is there pinentry-touchid should be able to fetch it (give or take one additional password prompt at some point).

Also regarding the "You may want to update to a newer pinentry" error I think this happens because pinentry-touchid is not responding to the GETINFO command. I will implement this to get rid of the error (as it creates confusion) but it shouldn't be the issue of the error.

[Dentrax]

Hi @jorgelbg, thank you for your efforts on this! I just tried the v0.3.0 release by downloading macos-amd64 binary.

Here is what I have tried so far:

1. Create a new test repo:

$ cd /tmp
$ mkdir foo; cd foo
$ git init
$ touch foo
$ git add --all

2. Checked my ~/.gnupg/gpg-agent.conf file to ensure pinentry-program is set to /usr/local/bin/pinentry-touchid:

$ cat ~/.gnupg/gpg-agent.conf

pinentry-program /usr/local/bin/pinentry-touchid
default-cache-ttl 600
max-cache-ttl 7200
default-cache-ttl-ssh 600
max-cache-ttl-ssh 7200

3. Disable the keychain for gpgtools:

$ defaults write org.gpgtools.common DisableKeychain -bool yes

4. Restart the agent:

$ gpg-agent --daemon; killall gpg-agent && gpg-agent --daemon --use-standard-socket --pinentry-program /usr/local/bin/pinentry-touchid

gpg-agent: a gpg-agent is already running - not starting a new one
gpg-agent[6312]: SIGTERM received - shutting down ...
gpg-agent[6312]: gpg-agent (GnuPG/MacGPG2) 2.2.32 stopped
gpg-agent[6344]: WARNING: "--use-standard-socket" is an obsolete option - it has no effect
gpg-agent[6345]: gpg-agent (GnuPG/MacGPG2) 2.2.32 started

5. Restart the process:

$ killall -9 gpg-agent; gpg-connect-agent reloadagent /bye

gpg-connect-agent: no running gpg-agent - starting '/usr/local/MacGPG2/bin/gpg-agent'
gpg-connect-agent: waiting for the agent to come up ... (5s)
gpg-connect-agent: connection to agent established
OK

6. Ensure $GPG_TTY is set:

$ echo $GPG_TTY

/dev/ttys000

7. Commit:

$ git commit -v -s -S

gpg-agent[7251]: You may want to update to a newer pinentry
gpg-agent[7251]: failed to unprotect the secret key: Operation cancelled
gpg-agent[7251]: failed to read the secret key
gpg-agent[7251]: command 'PKSIGN' failed: Operation cancelled <Pinentry>
error: gpg failed to sign the data
fatal: failed to write commit object

Log:

$ cat /tmp/pinentry-touchid.log

2022/08/04 21:35:35 main.go:109: Ready!

It seems the issue still persists. What I'm missing here? 🤔

[Dentrax]

Ohhh, according to this comment and as @SunsetYe66 mentioned deleting it above, I deleted my GnuPG key, and finally it works like a charm! 🎉

To make UX better here, pinentry-touchid should check the GnuPG key in the keychain: security find-generic-password -s 'GnuPG' whether it's correct. Or would better to refresh this key on first run.

We can also log some useful instructions such as deleting key in keychain and disabling the gpgtools: defaults write org.gpgtools.common DisableKeychain -bool yes

WDYT? What should we do to increase the overall UX here?

[jorgelbg]

Glad that it is working 🥳

I'm all in favour of improving the UX here, I'm just not sure why this is happening in the first place.

Did you, by any chance, kept the details of the item that you found when executing the commands in the terminal? I'm curious about why pinentry-touchid tried to add the item again. The first thing that we do is check if, for the given GPG key, we can find a matching item in the keychain (this is based on the generated keychainLabel). If the item is not found then we invoke pinentry-mac to get a pin.

At this point one of two things should happen: a) the item is persisted directly by pinentry-mac (when the org.gpgtools.common DisableKeychain is not set) or we get the pin/password from pinentry-mac and then pinentry-touchid tries to store it directly in the keychain.

In theory, if the item is already present in the keychain we shouldn't try to create it again (pinentry-mac shouldn't be even called) but instead you should get the system password dialog for allowing pinentry-touchid to access the keychain item.

It would seem as if at some point the item is not found in the keychain but still pinentry-touchid is trying to save it 🤔 and worse, the keychain is complaining about the item already being present 🤷‍♂️.

This is why we have 2 checks in the code:

pinentry-touchid/main.go

Lines 284 to 333 in 42fe314

	exists, err := checkEntryInKeychain(keychainLabel)
	if err != nil {
	logger.Printf("error checking entry in keychain: %s", err)
	return "", assuanError(err)
	}
	// If the entry is not found in the keychain, we trigger `pinentry-mac` with the option
	// to save the pin in the keychain.
	//
	// When trying to access the newly created keychain item we will get the normal password prompt
	// from the OS, we need to "Always allow" access to our application, still the access from our
	// app to the keychain item will be guarded by Touch ID.
	//
	// Currently I'm not aware of a way for automatically adding our binary to the list of always
	// allowed apps, see: https://github.com/keybase/go-keychain/issues/54.
	if !exists {
	pin, err := promptFn(s)
	if err != nil {
	logger.Printf("Error calling pinentry program (%s): %s", pinentryBinary.GetBinary(), err)
	}
	if len(pin) == 0 {
	logger.Printf("pinentry-mac didn't return a password")
	return "", assuanError(fmt.Errorf("pinentry-mac didn't return a password"))
	}
	// s.KeyInfo is always in the form of x/cacheId
	// https://gist.github.com/mdeguzis/05d1f284f931223624834788da045c65#file-info-pinentry-L357-L362
	keyInfo := strings.Split(s.KeyInfo, "/")[1]
	// pinentry-mac can create an item in the keychain, if that was the case, the user will have
	// to authorize our app to access the item without asking for a password from the user. If
	// not, we create an entry in the keychain, which automatically gives us ownership (i.e the
	// user will not be asked for a password). In either case, the access to the item will be
	// guarded by Touch ID.
	exists, err = checkEntryInKeychain(keychainLabel)
	if err != nil {
	logger.Printf("error checking entry in keychain: %s", err)
	return "", assuanError(err)
	}
	if !exists {
	// pinentry-mac didn't create a new entry in the keychain, we create our own and take
	// ownership over the entry.
	err = storePasswordInKeychain(keychainLabel, keyInfo, pin)
	if err == keychain.ErrorDuplicateItem {
	logger.Printf("Duplicated entry in the keychain")
	return "", assuanError(err)
	}

Because between checking for the item the first time and after invoking pinentry-mac there is a chance that the item is already in the keychain.

We can also log some useful instructions such as deleting key in keychain and disabling the gpgtools: defaults write org.gpgtools.common DisableKeychain -bool yes

We suggest to not allow pinentry-mac to create the item in the keychain by itself in the README, at the very least this causes that (after the cached pin expires) you need to re-authorize pinentry-touchid to access the item (via the system password dialog).

Maybe we can set check/modify org.gpgtools.common DisableKeychain automatically 🤔.