Hello,
I have a question -- I am writing an article on Sequoia for LWN (http://lwn.net) and as part of that I wanted to compile and try out the command line tool (specifically sq).
One thing I noticed was that, while I was able to import my PGP keys, etc.. I couldn't seem to be able to USE those PGP keys in the store to encrypt or decrypt messages. The only way I could get it to work was to specify the keyfile using things like --secret-key-file.
Is this intended behavior, or am I just missing how this is supposed to work?
John
Hi John,
On Thu, 10 Sep 2020 09:38:30 +0200, John Coggeshall wrote:
I have a question
Thanks for reaching out!
One thing I noticed was that, while I was able to import my PGP keys, etc.. I couldn't seem to be able to USE those PGP keys in the store to encrypt or decrypt messages. The only way I could get it to work was to specify the keyfile using things like --secret-key-file.
Is this intended behavior, or am I just missing how this is supposed to work?
There are a few things going on here.
First, for our upcoming 1.0 release (anytime now (tm), although the API and feature set have been stable for months; we are just documenting everything, and carefully reviewing the API & code), we are only releasing the low-level API, i.e., the sequoia-openpgp crate [1], and its dependencies. The high-level library [2], the services (the sequoia key store [3] & network component [4]), and the command-line tool sq [5] are functionally incomplete, and their interfaces are still experimentally.
[1] https://docs.sequoia-pgp.org/sequoia_openpgp/ [2] https://docs.sequoia-pgp.org/sequoia_core/index.html [3] https://docs.sequoia-pgp.org/sequoia_store/index.html [4] https://docs.sequoia-pgp.org/sequoia_net/index.html [5] https://docs.sequoia-pgp.org/sq/index.html
(FWIW, some experience suggests that our low-level API is actually pretty high level. For instance, we replaced the gpg backend in the pep engine with a Sequoia backend. This replaced 3.1 kLOC [6] with 3.3 kLOC [7]. That sounds like a small increase, but just under 1 kLOC are for implementing a key store. So, in this case, implementing equivalent functionality using our low-level API required less code (~900 fewer LOC) than using gpgme's high-level API.)
[6] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c1920c/src/pgp_gpg.c [7] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c1920c/src/pgp_sequoia...
One of the main bits of missing functionality in the key store is support for handling secret key material. So, as you correctly observed, it is currently only possible to use secret key material when it is stored in a file using the --secret-key-file option to sq. We actually have some code to manage secret keys, but it is at such an early stage, that we haven't checked it in yet. Finishing up the key store is one of our top priorities after the 1.0 release.
Modifying sq to take advantage of that functionality should be straightforward. However, we aren't releasing sq either, because we want to reserve the right to change its interface: sq is missing quite a bit of functionality. And, as we expose more of the low-level functionality, experience has taught us that we'll discover deficiencies in our current API. We want to be able to fix those issues. Also, we don't yet have a machine readable interface (we plan to implement a REST-like JSON-based API so users won't have to implement complex state machines and parse custom formats), and we don't want people to screenscrape sq.
That said, encrypting to keys stored in the key store (and verifying messages signed by keys in the key store) should work. In this case, you need to address them by their label. For instance, to create a mapping and then encrypt a message to that key, this should work:
$ sq mapping import neal neal.asc $ echo foo | sq encrypt -r neal -----BEGIN PGP MESSAGE-----
wcBMA8K4GQVsZSWY...
But, this won't work:
$ sq encrypt -r neal@walfield.org Error: No such key found
Caused by: Key not found
even though my key has that email address:
$ sq inspect neal.asc neal.asc: OpenPGP Certificate.
Fingerprint: 8F17 7771 18A3 3DDA 9BA4 8E62 AACB 3243 6300 52D9 ... UserID: Neal H. Walfield neal@walfield.org
because the email address is not authenticated.
Right now, the key store only implements a petname interface [8] for addressing certificates ("PGP keys"). In the future, we'll add support for the WoT.
[8] https://en.wikipedia.org/wiki/Petname
One reason why you can only use authenticated identifiers is that we view the key store as a cache of all keys, not a curated keyring. That means, just because you have a key with an email address locally, there isn't any more reason to use it than if you found it on an SKS key server. We have ideas for how to simplify authentication. In particular, we take an address-book view of key management where identifiers are central rather than a key-centric view like gpg does, which is confusing when there are multiple keys with the same identifier.
Since you are writing about Sequoia, you might also want to check out our affiliated projects:
- Hagrid, the verifying key server that powers https://keys.openpgp.org
https://gitlab.com/hagrid-keyserver/hagrid
- OpenPGP CA, a tool for simplifying key management and authentication in groups and organizations
https://gitlab.com/openpgp-ca/openpgp-ca
- Koverto, a tool for adding a encrypion / signing gateway in front of a service, like a wiki
https://gitlab.com/koverto/koverto
- The OpenPGP Interoperability Test Suite A tool for exploring edge cases in many OpenPGP implementations https://tests.sequoia-pgp.org/
- Sequoia Dump A web frontend to 'sq packet dump', a tool for exploring OpenPGP messages
If you have any more questions, feel free to follow up here or ask on our irc channel (Freenode, #sequoia).
:) Neal
Neal,
Thank you for your detailed response to my question! It was very helpful in clarifying the state of things for the article, which just ran on LWN:
https://lwn.net/SubscriberLink/830902/ebcaaf43af6834d2/
It is good to see you all making solid progress, and I'm looking forward to seeing how things pan out. Outside of my work with LWN I am a core contributor to the PHP project, so if you'd like to discuss trying to make PHP bindings for Sequoia sometime just let me know.
Cheers,
John
Neal H. Walfield wrote on 9/10/20 8:51 AM:
Hi John,
On Thu, 10 Sep 2020 09:38:30 +0200, John Coggeshall wrote:
I have a question
Thanks for reaching out!
One thing I noticed was that, while I was able to import my PGP keys, etc.. I couldn't seem to be able to USE those PGP keys in the store to encrypt or decrypt messages. The only way I could get it to work was to specify the keyfile using things like --secret-key-file.
Is this intended behavior, or am I just missing how this is supposed to work?
There are a few things going on here.
First, for our upcoming 1.0 release (anytime now (tm), although the API and feature set have been stable for months; we are just documenting everything, and carefully reviewing the API & code), we are only releasing the low-level API, i.e., the sequoia-openpgp crate [1], and its dependencies. The high-level library [2], the services (the sequoia key store [3] & network component [4]), and the command-line tool sq [5] are functionally incomplete, and their interfaces are still experimentally.
[1] https://docs.sequoia-pgp.org/sequoia_openpgp/ [2] https://docs.sequoia-pgp.org/sequoia_core/index.html [3] https://docs.sequoia-pgp.org/sequoia_store/index.html [4] https://docs.sequoia-pgp.org/sequoia_net/index.html [5] https://docs.sequoia-pgp.org/sq/index.html
(FWIW, some experience suggests that our low-level API is actually pretty high level. For instance, we replaced the gpg backend in the pep engine with a Sequoia backend. This replaced 3.1 kLOC [6] with 3.3 kLOC [7]. That sounds like a small increase, but just under 1 kLOC are for implementing a key store. So, in this case, implementing equivalent functionality using our low-level API required less code (~900 fewer LOC) than using gpgme's high-level API.)
[6] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c1920c/src/pgp_gpg.c [7] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c1920c/src/pgp_sequoia...
One of the main bits of missing functionality in the key store is support for handling secret key material. So, as you correctly observed, it is currently only possible to use secret key material when it is stored in a file using the --secret-key-file option to sq. We actually have some code to manage secret keys, but it is at such an early stage, that we haven't checked it in yet. Finishing up the key store is one of our top priorities after the 1.0 release.
Modifying sq to take advantage of that functionality should be straightforward. However, we aren't releasing sq either, because we want to reserve the right to change its interface: sq is missing quite a bit of functionality. And, as we expose more of the low-level functionality, experience has taught us that we'll discover deficiencies in our current API. We want to be able to fix those issues. Also, we don't yet have a machine readable interface (we plan to implement a REST-like JSON-based API so users won't have to implement complex state machines and parse custom formats), and we don't want people to screenscrape sq.
That said, encrypting to keys stored in the key store (and verifying messages signed by keys in the key store) should work. In this case, you need to address them by their label. For instance, to create a mapping and then encrypt a message to that key, this should work:
$ sq mapping import neal neal.asc $ echo foo | sq encrypt -r neal -----BEGIN PGP MESSAGE-----
wcBMA8K4GQVsZSWY...
But, this won't work:
$ sq encrypt -r neal@walfield.org Error: No such key found
Caused by: Key not found
even though my key has that email address:
$ sq inspect neal.asc neal.asc: OpenPGP Certificate.
Fingerprint: 8F17 7771 18A3 3DDA 9BA4 8E62 AACB 3243 6300 52D9 ... UserID: Neal H. Walfield <neal@walfield.org>because the email address is not authenticated.
Right now, the key store only implements a petname interface [8] for addressing certificates ("PGP keys"). In the future, we'll add support for the WoT.
[8] https://en.wikipedia.org/wiki/Petname
One reason why you can only use authenticated identifiers is that we view the key store as a cache of all keys, not a curated keyring. That means, just because you have a key with an email address locally, there isn't any more reason to use it than if you found it on an SKS key server. We have ideas for how to simplify authentication. In particular, we take an address-book view of key management where identifiers are central rather than a key-centric view like gpg does, which is confusing when there are multiple keys with the same identifier.
Since you are writing about Sequoia, you might also want to check out our affiliated projects:
Hagrid, the verifying key server that powers https://keys.openpgp.org
OpenPGP CA, a tool for simplifying key management and authentication in groups and organizations
Koverto, a tool for adding a encrypion / signing gateway in front of a service, like a wiki
The OpenPGP Interoperability Test Suite A tool for exploring edge cases in many OpenPGP implementations https://tests.sequoia-pgp.org/
Sequoia Dump A web frontend to 'sq packet dump', a tool for exploring OpenPGP messages
If you have any more questions, feel free to follow up here or ask on our irc channel (Freenode, #sequoia).
:) Neal
Hi John,
Thanks for the nice write up! I really appreciate the fact checking before publishing the article!
:) Neal
On Fri, 11 Sep 2020 18:11:22 +0200, John Coggeshall wrote:
Neal,
Thank you for your detailed response to my question! It was very helpful in clarifying the state of things for the article, which just ran on LWN:
https://lwn.net/SubscriberLink/830902/ebcaaf43af6834d2/
It is good to see you all making solid progress, and I'm looking forward to seeing how things pan out. Outside of my work with LWN I am a core contributor to the PHP project, so if you'd like to discuss trying to make PHP bindings for Sequoia sometime just let me know.
Cheers,
John
Neal H. Walfield wrote on 9/10/20 8:51 AM:
Hi John,
On Thu, 10 Sep 2020 09:38:30 +0200, John Coggeshall wrote:
I have a question
Thanks for reaching out!
One thing I noticed was that, while I was able to import my PGP keys, etc.. I couldn't seem to be able to USE those PGP keys in the store to encrypt or decrypt messages. The only way I could get it to work was to specify the keyfile using things like --secret-key-file.
Is this intended behavior, or am I just missing how this is supposed to work?
There are a few things going on here.
First, for our upcoming 1.0 release (anytime now (tm), although the API and feature set have been stable for months; we are just documenting everything, and carefully reviewing the API & code), we are only releasing the low-level API, i.e., the sequoia-openpgp crate [1], and its dependencies. The high-level library [2], the services (the sequoia key store [3] & network component [4]), and the command-line tool sq [5] are functionally incomplete, and their interfaces are still experimentally.
[1] https://docs.sequoia-pgp.org/sequoia_openpgp/ [2] https://docs.sequoia-pgp.org/sequoia_core/index.html [3] https://docs.sequoia-pgp.org/sequoia_store/index.html [4] https://docs.sequoia-pgp.org/sequoia_net/index.html [5] https://docs.sequoia-pgp.org/sq/index.html
(FWIW, some experience suggests that our low-level API is actually pretty high level. For instance, we replaced the gpg backend in the pep engine with a Sequoia backend. This replaced 3.1 kLOC [6] with 3.3 kLOC [7]. That sounds like a small increase, but just under 1 kLOC are for implementing a key store. So, in this case, implementing equivalent functionality using our low-level API required less code (~900 fewer LOC) than using gpgme's high-level API.)
[6] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c1920c/src/pgp_gpg.c [7] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c1920c/src/pgp_sequoia...
One of the main bits of missing functionality in the key store is support for handling secret key material. So, as you correctly observed, it is currently only possible to use secret key material when it is stored in a file using the --secret-key-file option to sq. We actually have some code to manage secret keys, but it is at such an early stage, that we haven't checked it in yet. Finishing up the key store is one of our top priorities after the 1.0 release.
Modifying sq to take advantage of that functionality should be straightforward. However, we aren't releasing sq either, because we want to reserve the right to change its interface: sq is missing quite a bit of functionality. And, as we expose more of the low-level functionality, experience has taught us that we'll discover deficiencies in our current API. We want to be able to fix those issues. Also, we don't yet have a machine readable interface (we plan to implement a REST-like JSON-based API so users won't have to implement complex state machines and parse custom formats), and we don't want people to screenscrape sq.
That said, encrypting to keys stored in the key store (and verifying messages signed by keys in the key store) should work. In this case, you need to address them by their label. For instance, to create a mapping and then encrypt a message to that key, this should work:
$ sq mapping import neal neal.asc $ echo foo | sq encrypt -r neal -----BEGIN PGP MESSAGE-----
wcBMA8K4GQVsZSWY...
But, this won't work:
$ sq encrypt -r neal@walfield.org Error: No such key found
Caused by: Key not found
even though my key has that email address:
$ sq inspect neal.asc neal.asc: OpenPGP Certificate.
Fingerprint: 8F17 7771 18A3 3DDA 9BA4 8E62 AACB 3243 6300 52D9 ... UserID: Neal H. Walfield <neal@walfield.org>because the email address is not authenticated.
Right now, the key store only implements a petname interface [8] for addressing certificates ("PGP keys"). In the future, we'll add support for the WoT.
[8] https://en.wikipedia.org/wiki/Petname
One reason why you can only use authenticated identifiers is that we view the key store as a cache of all keys, not a curated keyring. That means, just because you have a key with an email address locally, there isn't any more reason to use it than if you found it on an SKS key server. We have ideas for how to simplify authentication. In particular, we take an address-book view of key management where identifiers are central rather than a key-centric view like gpg does, which is confusing when there are multiple keys with the same identifier.
Since you are writing about Sequoia, you might also want to check out our affiliated projects:
Hagrid, the verifying key server that powers https://keys.openpgp.org
OpenPGP CA, a tool for simplifying key management and authentication in groups and organizations
Koverto, a tool for adding a encrypion / signing gateway in front of a service, like a wiki
The OpenPGP Interoperability Test Suite A tool for exploring edge cases in many OpenPGP implementations https://tests.sequoia-pgp.org/
Sequoia Dump A web frontend to 'sq packet dump', a tool for exploring OpenPGP messages
If you have any more questions, feel free to follow up here or ask on our irc channel (Freenode, #sequoia).
:) Neal
-
John Coggeshall -
Neal H. Walfield