2024

2023

2022

2021

2020

2019

2018

List overview
Download

Devel December 2020

devel@lists.sequoia-pgp.org

1 participants
1 discussions

Start a nNew thread

Sequoia PGP v1.0 Released: The Seedling's a Sapling
by Neal H. Walfield 16 Dec '20

16 Dec '20

Version 1.0. It's here. After three and a half years of development, we are happy to announce the release of version 1.0 of Sequoia! The release includes the low-level crate [`sequoia-openpgp`], and a program to verify detached signatures geared towards software distribution systems called [`sqv`]. [`sequoia-openpgp`]: https://crates.io/crates/sequoia-openpgp [`sqv`]: https://crates.io/crates/sequoia-sqv We will support this API with security updates for at least one year. In 9 months, we will announce whether we will extend this commitment. The two main criteria will be our financial situation (please [donate], or sponsor a developer or two), and the number of users. [donate]: https://pep.foundation/support-pep/index.html # Getting to Version 1.0 We actually [almost released] version 1.0 about a year ago. All of the features that we had planned for version 1.0 were implemented, we had good test coverage, and we even had a few users. But, we decided to wait. We decided to wait not because we thought of another feature, or because we become aware of a significant bug, but because [we][] [decided][] [to][] [take][] [some][] [time][] [to][to2] [improve][] [our][] [documentation][]. [almost released]: https://mastodon.social/@sequoiapgp/103364362621954545 [we]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/465 [decided]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/466 [to]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/467 [take]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/468 [some]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/469 [time]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/470 [to2]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/471 [improve]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/472 [our]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/473 [documentation]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/474 Our goal was to make sure that every module had a helpful introduction, and all public methods had a useful description, a link to [the standard], when appropriate, and a meaningful example. Dividing the task between five people, we figured it would delay the release by a month, perhaps two. In the end, well, it took nearly a year, and we had to scale our ambitions back a bit. Nevertheless, we're quite happy with [the][] [result][]. [the]: https://docs.sequoia-pgp.org/sequoia_openpgp/cert/index.html [the standard]: https://tools.ietf.org/html/rfc4880 [result]: https://docs.sequoia-pgp.org/sequoia_openpgp/parse/stream/trait.DecryptionH… First, the documentation is much better. It's of course hard to quantify its quality. But, we can distill a few numbers. When we started our documentation effort, shortly after we released version 0.14, the Sequoia library had just over 11k lines of comments including 53 documentation test, and 37 kSLOC including 12 kSLOC of unit tests. The 1.0 release has over 33k lines of comments (190% more) including 464 documentation tests (780% more), and 44 kSLOC (21% or 7.5 kSLOC more) including 8k SLOC of additional unit tests. Second, in the process of documenting our public API and writing examples, we discovered many minor annoyances, some inconsistencies, and more than a few bugs. We took the opportunity to fix them. Finally, as the rate of change of the API had decreased, more projects were willing to try out Sequoia. They provided additional useful feedback, which we integrated. All in all, we feel that with version 1.0 we've not only checked the [right boxes], but we also have a high-quality API and implementation that we can be proud of. [right boxes]: https://www.tomsguide.com/news/cyberpunk-2077-is-a-disaster-on-ps4-and-xbox… # Background Sequoia was started 3.5 years ago by Justus Winter, Kai Michaelis and me, Neal Walfield. Prior to working on Sequoia, the three of us had worked together at [g10code] on [GnuPG]. The [p≡p foundation] hired us not only to create a new OpenPGP implementation using a new architecture and programming language, but to improve the ecosystem around privacy-preserving tools as a whole. [g10code]: http://www.g10code.de/ [GnuPG]: https://gnupg.org/ [p≡p foundation]: https://pep.foundation The Sequoia library is a first step in that direction. But it is not our goal. Indeed, over the past three years, we've help other OpenPGP implementations. We've reported bugs that we've found (thanks in particular to our [OpenPGP interoperability test suite]), and even contributed some fixes to other OpenPGP implementations. [OpenPGP interoperability test suite]: https://tests.sequoia-pgp.org/ And, we've invested in tooling. We developed [Hagrid], a new verifying OpenPGP key server, which powers [keys.openpgp.org] and is now maintained by Vincent Breitmoser. We've helped [OpenPGP CA], a tool written by Heiko Schaefer to create *federated* CAs for groups like activists, lawyers, and journalists, but also companies, who don't want to trust centralized infrastructure with monetary incentives. OpenPGP CA significantly simplifies key discovery and authentication for unsophisticated OpenPGP users. We've developed [Koverto], an SMTP proxy, which makes it easy to sign and encrypt mails sent by services that don't support OpenPGP out of the box, like most CMSes. We developed a tool, [`sq-keyring-linter`] ([Debian]), to help users update their OpenPGP Certificates, so that we can [finally get rid of SHA-1]. [Hagrid]: https://gitlab.com/hagrid-keyserver/hagrid [keys.openpgp.org]: https://keys.openpgp.org/ [OpenPGP CA]: https://openpgp-ca.gitlab.io/openpgp-ca/ [Koverto]: https://gitlab.com/koverto/koverto [finally get rid of SHA-1]: https://mailarchive.ietf.org/arch/msg/openpgp/Rp-inhYKT8A9H5E34iLTrc9I0gc/ [`sq-keyring-linter`]: https://gitlab.com/sequoia-pgp/keyring-linter [Debian]: https://packages.debian.org/sid/sq-keyring-linter We're thinking big. We're thinking not only about mail encryption or even encryption in general, but also about integrity and authentication. And, we're thinking in particular, about [PKI]. If users can't easily find the **right** certificate for a communication partner, encryption and digital signatures are worthless, and possibly even dangerous. [PKI]: https://en.wikipedia.org/wiki/Public_key_infrastructure # Sequoia, the Library In designing Sequoia, we took a library-first approach. Although we have a command-line tool, [`sq`], which we are not yet releasing, we intend for the library to always provide a richer, more expressive interface. We agree that there is value in process separation, but we want to avoid the dangerous complexity of *safely* shelling out to another program. [`sq`]: https://docs.sequoia-pgp.org/sq/index.html The sequoia-openpgp crate (Rust's terminology for a library) is a low-level, policy-free OpenPGP implementation. Our goal was to implement all of [RFC 4880], and provide an API that can be used to access and modify pretty much everything, but is simultaneously secure by default. We understand low-level to mean not only an API that provides getters and setters, but an API that provides interfaces to parse and serialize those fields, and can combine them in ways intended by the standard, and needed by users. For instance, the [`Cert`] data structure encapsulates an OpenPGP certificate (casually referred to as an OpenPGP key). It canonicalizes the structure, and makes it easy to [query its properties]. But, it does so in such a way that it is still possible for a user to inspect and modify the low-level bits themselves without reimplementing the rest of the functionality. Another example is the [`DecryptionHelper`], which makes it easy to parse and decrypt an OpenPGP message. [`Cert`]: https://docs.sequoia-pgp.org/sequoia_openpgp/struct.Cert.html [query its properties]: https://docs.sequoia-pgp.org/sequoia_openpgp/cert/struct.ValidCert.html#met… [`DecryptionHelper`]: https://docs.sequoia-pgp.org/sequoia_openpgp/parse/stream/trait.DecryptionH… An example of how we make the API safe by default is that it is hard to accidentally export secret key material. In Sequoia, you have to explicitly [opt-in to export it]. Similarly, when [updating a signature], the creation time, hash algorithm, and issuer are automatically updated. This is usually what the user wants, but is easy to forget, and hard to debug when forgotten. Critically, it is easy to opt out when that behavior is not desired. [opt-in to export that]: https://docs.sequoia-pgp.org/sequoia_openpgp/struct.Cert.html#secret-keys [updating a signature]: https://docs.sequoia-pgp.org/sequoia_openpgp/packet/signature/struct.Signat… While developing Sequoia, we spent a lot of time thinking about extremes and corner cases. For instance, OpenPGP supports notarizations (signatures over signatures), but as far as we know no OpenPGP implementation supports them. We implemented support for it anyway, and it improved the ergonomics of the common case. ## Notable Details The devel is in the details. And while deviloping Sequoia, we paid attention. Here are a few noteworthy details: [SHA-1] has been broken since 2005. And, in 2011 NIST deprecated its use. Initially, we decided to simply reject any signature that used SHA-1. However, we were recently forced to [reevaluate][] [that decision][]: 22% of Debian developers use a certificate that relies on SHA-1 as do 63% of Arch developers. Even the Fedora release keys use SHA-1. [SHA-1]: https://en.wikipedia.org/wiki/SHA-1 [reevaluate]: https://mailarchive.ietf.org/arch/msg/openpgp/Rp-inhYKT8A9H5E34iLTrc9I0gc/ [that decision]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/595 We decided that we couldn't simply reenable SHA-1. After some consideration, we've opted to [permit it] in contexts where collision attacks similar to the one presented in [SHA-1 is a Shambles] are harder. We also use a variant of SHA-1 called [SHA1CD] (SHA-1 Collision Detection), which detects and neutralizes the known attacks against SHA-1. Among others, [GitHub uses it]. And, we have also decided to **[start rejecting SHA-1 by default] at the beginning of 2023**, i.e., in a bit more than two years. This will hopefully give Debian developers and others sufficient time to [fix] or replace their certificates. [permit it]: https://docs.sequoia-pgp.org/sequoia_openpgp/policy/enum.HashAlgoSecurity.h… [SHA-1 is a Shambles]: https://sha-mbles.github.io/ [SHA1CD]: https://gitlab.com/sequoia-pgp/sha1collisiondetection [GitHub uses it]: https://github.blog/2017-03-20-sha-1-collision-detection-on-github-com/ [start rejecting SHA-1 by default]: https://docs.sequoia-pgp.org/sequoia_openpgp/policy/struct.StandardPolicy.h… [fix]: https://gitlab.com/sequoia-pgp/keyring-linter When we create a signature, we include a [salt]. This makes it harder for an attacker to predict what data a user will sign. And, it foils attacks where an attacker needs multiple signatures over the same message. [salt]: https://gitlab.com/sequoia-pgp/sequoia/-/issues/597 Similar to [OpenSSH], we [encrypt secret key material] while it is in memory. This frustrates side-channel attacks. [OpenSSH]: https://www.undeadly.org/cgi?action=article;sid=20190621081455 [encrypt secret key material]: https://docs.sequoia-pgp.org/sequoia_openpgp/crypto/mem/struct.Encrypted.ht… Sequoia supports [padding messages] to obfuscate an encrypted message's length. We include support for the [padmé] scheme, but other schemes can be plugged in. [padding messages]: https://docs.sequoia-pgp.org/sequoia_openpgp/serialize/stream/padding/index… [padmé]: https://bford.info/pub/sec/purb.pdf To allow users to control the policy while still using higher-level functionality, Sequoia uses a [policy object]. A policy object is passed to any method that checks something for validity. For instance, when a method needs to determine whether a binding signature should be used, it invokes the policy object's [signature callback]. Our experience suggests that this approach greatly simplifies dealing with this [cross-cutting concern] in a highly flexible manner. [policy object]: https://docs.sequoia-pgp.org/sequoia_openpgp/policy/index.html [signature callback]: https://docs.sequoia-pgp.org/sequoia_openpgp/policy/trait.Policy.html#metho… [cross-cutting concern]: https://en.wikipedia.org/wiki/Cross-cutting_concern Policy objects can also be embedded in other objects. For instance, a [`ValidCert`] encapsulates a [`Cert`] and a policy object. This ensures that the application of the policy is consistent, and hard to forget to apply. [`ValidCert`]: https://docs.sequoia-pgp.org/sequoia_openpgp/cert/struct.ValidCert.html [`Cert`]: https://docs.sequoia-pgp.org/sequoia_openpgp/cert/struct.Cert.html In Sequoia, we prefer the use of formal grammars rather than ad-hoc parsing when doing any non-trivial parsing. For instance, when verifying the structure of [OpenPGP Certificates] and [OpenPGP Messages], we use [LALRPOP], a parser generator, to generate the parser. [OpenPGP Certificates]: https://tools.ietf.org/html/rfc4880#section-11.1 [OpenPGP Messages]: https://tools.ietf.org/html/rfc4880#section-11.3 [LALRPOP]: https://github.com/lalrpop/lalrpop Sequoia implements a streaming API. If not careful, this can lead to a consumer processing unauthenticated data, which was exploited by [EFAIL]. To mitigate this type of failure, the [`DecryptionHelper`] withholds the last `O(1)` bytes of data, and only releases it if the message can be authenticated. This makes it harder for an attacker to control what is released. And for short messages, nothing is released since the whole message is buffered. [EFAIL]: https://efail.de/ [`DecryptionHelper`]: https://docs.sequoia-pgp.org/sequoia_openpgp/parse/stream/trait.DecryptionH… We've tried to ensure that data structures that may be used in a side-channel sensitive context use constant time comparisons. Where possible, we use a device driver-style API so that it is straightforward to add new backends. For instace, our [`Signer`] and [`Decryptor`] traits make it easy to implement alternative signing and decryption backends. In addition to the in memory implementations, we already have implementations that use secret key material managed by [gpg agent]. [`Signer`]: https://docs.sequoia-pgp.org/sequoia_openpgp/crypto/trait.Signer.html [`Decryptor`]: https://docs.sequoia-pgp.org/sequoia_openpgp/parse/stream/struct.Decryptor.… [gpg agent]: https://docs.sequoia-pgp.org/sequoia_ipc/gnupg/struct.KeyPair.html We tried hard to provide helpful error messages. This is particularly difficult when looking for a valid self signature, for instance: if all self signatures are ignored, because they are invalid, what should be shown? In this case, we remember why signatures were rejected, and then show one of those error messages. This is less confusing. The following output illustrates this idea: ``` $ sq inspect md5.pgp only-md5-pub.pgp: OpenPGP Certificate. Fingerprint: 526F D3F8 4EC7 DA9D E565 1B06 C4C9 7B7F A453 52C3 Invalid: No binding signature at time 2020-12-16T14:18:21Z Public-key algo: RSA (Encrypt or Sign) Public-key size: 2048 bits Creation time: 2020-11-05 09:16:25 UTC UserID: MD5 All Around <md5-all-around(a)example.org> Invalid: Policy rejected non-revocation signature (PositiveCertification) requiring second pre-image resistance because: MD5 is not considered secure since 2004-02-01T00:00:00Z ``` Finally, we thought a lot about forward compatability. The OpenPGP standard makes provisions for this by using extensible types, and versioning. We are careful to preserve what we don't understand, and respect these packets as much as possible. ## What's Missing Sequoia provides a low-level API. As such, we don't provide a public or private key store. We do, however, plan to develop them (in fact, we already have prototypes). But, they will be released separately. In particular, we are not convinved that a single design is universally appropriate. For instance, a server application may not care about sharing keys with other applications, and it may have its own database. Sequoia implements most of [RFC 4880]. But, it is still missing a few features. In particular, we don't yet support [cleartext signatures]. The [`Cert`] data structure does not yet respect [certification revocations]. And, we still need to implement OpenPGP [regular expressions], which are used to scope [trust signatures]. We will add support for these features in the near future. [RFC 4880]: https://tools.ietf.org/html/rfc4880 [cleartext signatures]: https://tools.ietf.org/html/rfc4880#section-7 [certification revocations]: https://tools.ietf.org/html/rfc4880#section-5.2.1 [regular expressions]: https://tools.ietf.org/html/rfc4880#section-8 [trust signatures]: https://tools.ietf.org/html/rfc4880#section-5.2.3.13 Sequoia does not implement the cryptographic primitives that it uses, but relies on a library. By default, Sequoia uses [Nettle]. But, on Windows it is also possible to use [Windows CNG], which has the advantage that it is already part of the user's trusted computing base. A consequence of this is that Sequoia is limited to what the libraries support, and neither supports Elgamal keys, or [brainpool] keys. (For 25519 keys on Windows, we use the [ed25519-dalek] and associated crates.) [Nettle]: http://www.lysator.liu.se/~nisse/nettle/ [Windows CNG]: https://docs.microsoft.com/en-us/windows/win32/seccng/cng-portal [brainpool]: https://en.wikipedia.org/wiki/Elliptic-curve_cryptography [ed25519-dalek]: https://crates.io/crates/ed25519-dalek We have also decided to not support AEAD, as [its specification] for OpenPGP is still a draft, and consensus has not yet been reached. [its specification]: https://www.ietf.org/id/draft-ietf-openpgp-rfc4880bis-10.html For further details, please consult our [status page]. [status page]: https://sequoia-pgp.org/status/ # What's Next Our plans for the near future includes: - Adding support for the few [missing pieces from RFC 4880], as outlined above. [missing pieces from RFC 4880]: https://sequoia-pgp.org/status/ - Adding high-level services like a public key store and a private key store. We've already begun work on [prototypes], so we are confident that our low-level API is sufficient. A noteworthy goal of the private key store is to use a device driver-like framework so that different backends can be used. The main motivation for this is to handle smartcards. But, it can also be used to transparently access remote keys, and gpg's agent, which will simplify interoperability. [prototypes]: https://docs.sequoia-pgp.org/sequoia_store/index.html - Continuing to work on our CLI tool, [`sq`]. Although the tool is already quite usable, and useful, there are still some important [features missing]. In particular, we want to add a REST-style, JSON-based, versioned API as we expect people will end up writing scripts that use `sq`, and we want to avoid having them screen scrape its output, which will undoubtedly result in security issues. [`sq`]: https://docs.sequoia-pgp.org/sq/index.html [features missing]: https://gitlab.com/sequoia-pgp/sequoia/-/issues?label_name%5B%5D=cli More generally, we plan to continue our work in the ecosystem with a particular focus on tooling. # Financial Support Currently, the there are six people paid to work on Sequoia. They are all paid by the [p≡p foundation]. Additional financial support has came from a one-time donation from the [Wau Holland Foundation] in 2018. [Phil Zimmermann], the creator of PGP, donated 1000 Euros. And, [Proton Mail] donated 1000 Euros after our interoperability test suite found several issues in OpenPGP.js and GopenPGP, which they maintain. [p≡p foundation]: https://pep.foundation/ [Wau Holland Foundation]: https://www.wauland.de/de/ [Phil Zimmermann]: https://pep.foundation/blog/phil-zimmermann-donates-eur-1000-to-the-pep-fou… [Proton Mail]: https://protonmail.com/blog/openpgp-test-suite/ Although the p≡p foundation's finances are secure in the mid term, it is essential for Sequoia's long-term health that we diversify our funding. You don't need to directly use Sequoia to be positively impacted by it. For instance, OpenPGP CA is a new tool for creating federated CAs targeted at simplifying the use of OpenPGP for activists, lawyers, and journalists who can't rely on centralized authentication solutions. So, [consider donating]. Of course, if your company is using Sequoia, consider sponsoring a developer (or two). Note: if you want to use Sequoia under a license other than the GPLv2+, please [contact the foundation]. [consider donating]: https://pep.foundation/support-pep/index.html [contact the foundation]: https://pep.foundation/contact-us/index.html # Thanks Sequoia is a team effort. The following people have worked directly on Sequoia: - Azul - Daniel Kahn Gillmor - Daniel Silverstone - Heiko Schaefer - Igor Matuszewski - Jonas Bernoulli - juga - Justus Winter - Kai Michaelis - Neal H. Walfield - Nora Widdecke - Wiktor Kwapisiewicz The following people contributed code: - Alexander Kjäll - amesgen - Azul - Björn Petersen - Damian Poddebniak - Daniel Kahn Gillmor - Daniel Silverstone - David Sastre Medina - Doron Behar - Gerhard Bräunlich - Hartmut Goebel - Hussein - Igor Matuszewski - Jakub Kądziołka - Jakub Onderka - Jann Röder - Jonas Bernoulli - juga - Justus Winter - Kai Michaelis - Leonhard Markert - Levente Polyak - Neal H. Walfield - Nora Widdecke - phryk - Ruben Pollan - Sebastian Thiel - Tobias Mueller - Wesley Moore - Wiktor Kwapisiewicz And, the following people contributed on our [issue tracker]: [issue tracker]: https://gitlab.com/sequoia-pgp/sequoia/-/issues - [adulau](https://gitlab.com/adulau) - [alexanderkjall](https://gitlab.com/alexanderkjall) - [anarcat](https://gitlab.com/anarcat) - [ashkan.kiani](https://gitlab.com/ashkan.kiani) - [assarbad](https://gitlab.com/assarbad) - [azul](https://gitlab.com/azul) - [bme](https://gitlab.com/bme) - [chocol4te](https://gitlab.com/chocol4te) - [cluck](https://gitlab.com/cluck) - [curiousleo](https://gitlab.com/curiousleo) - [darkfox](https://gitlab.com/darkfox) - [darthmama](https://gitlab.com/darthmama) - [DBLouis](https://gitlab.com/DBLouis) - [dirkz](https://gitlab.com/dirkz) - [dkg](https://gitlab.com/dkg) - [doronbehar](https://gitlab.com/doronbehar) - [Ekleog](https://gitlab.com/Ekleog) - [erictapen](https://gitlab.com/erictapen) - [fdik](https://gitlab.com/fdik) - [flanfly](https://gitlab.com/flanfly) - [ghost1](https://gitlab.com/ghost1) - [hanno](https://gitlab.com/hanno) - [HeikoStamer](https://gitlab.com/HeikoStamer) - [hkos](https://gitlab.com/hkos) - [htgoebel](https://gitlab.com/htgoebel) - [husspEp](https://gitlab.com/husspEp) - [joakimlundborg](https://gitlab.com/joakimlundborg) - [juga0](https://gitlab.com/juga0) - [kalt](https://gitlab.com/kalt) - [kinnison](https://gitlab.com/kinnison) - [kushaldas](https://gitlab.com/kushaldas) - [lo48576](https://gitlab.com/lo48576) - [lu-zero](https://gitlab.com/lu-zero) - [mehmeteribol](https://gitlab.com/mehmeteribol) - [meskio](https://gitlab.com/meskio) - [mexus](https://gitlab.com/mexus) - [mishajw](https://gitlab.com/mishajw) - [mlboogerd](https://gitlab.com/mlboogerd) - [muelli](https://gitlab.com/muelli) - [nathankleyn](https://gitlab.com/nathankleyn) - [nimtiazm](https://gitlab.com/nimtiazm) - [nwalfield](https://gitlab.com/nwalfield) - [P-E-Meunier](https://gitlab.com/P-E-Meunier) - [pfaffm](https://gitlab.com/pfaffm) - [puzzlewolf](https://gitlab.com/puzzlewolf) - [Reisen](https://gitlab.com/Reisen) - [rockdaboot](https://gitlab.com/rockdaboot) - [roederja](https://gitlab.com/roederja) - [simonrepp](https://gitlab.com/simonrepp) - [teythoon](https://gitlab.com/teythoon) - [thomas__](https://gitlab.com/thomas__) - [TrashMacNugget](https://gitlab.com/TrashMacNugget) - [valodim](https://gitlab.com/valodim) - [vivapolonium](https://gitlab.com/vivapolonium) - [wezm](https://gitlab.com/wezm) - [wiktor-k](https://gitlab.com/wiktor-k) - [WolfgangMK](https://gitlab.com/WolfgangMK) - [Xanewok](https://gitlab.com/Xanewok)

1 0