Commit Graph

1757 Commits

Author SHA1 Message Date
Riyaz Faizullabhoy
67d1847cb3 Remove authkey usage
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:03 -07:00
David Lawrence
66d6b347d8 cert fixtures for rethink
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2016-04-14 19:38:03 -07:00
David Lawrence
db9fbc3e4b adding rethink compose setup
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2016-04-14 19:38:03 -07:00
Riyaz Faizullabhoy
d86227642a Add bootstrapper for signer
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:03 -07:00
Riyaz Faizullabhoy
ee02a50e31 Add privatekeys table to bootstrapping, some cleanup
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:02 -07:00
Riyaz Faizullabhoy
d95891e54f Caching logic for TufMetaStorage
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:02 -07:00
David Lawrence
b196a803e2 updating use of indices and queries
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2016-04-14 19:38:02 -07:00
Riyaz Faizullabhoy
30f356f940 Use consistent meta storage for server binary
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:02 -07:00
Riyaz Faizullabhoy
6d3b017c56 Consistent metastore walks from current timestamp to checksums of snapshot/other roles
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:02 -07:00
Riyaz Faizullabhoy
f8a67f8af0 Add rethink init for signer config
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:02 -07:00
David Lawrence
b8c62731a6 adding bootstrapping and config update for notary server
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2016-04-14 19:38:02 -07:00
David Lawrence
045721250f rethink server implementation
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2016-04-14 19:38:02 -07:00
Riyaz Faizullabhoy
72135193b0 Skeleton for notary signer rethinkdb driver
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 19:38:02 -07:00
David Lawrence
fbe0626a9d common time tracking fields
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2016-04-14 19:38:01 -07:00
David Lawrence
860714c9d3 shared connection code for rethinkdb
Signed-off-by: David Lawrence <david.lawrence@docker.com> (github: endophage)
2016-04-14 19:38:01 -07:00
David Lawrence
6c524b2500 Merge pull request #681 from docker/doc-version-note
Add note about versioning
2016-04-14 17:10:04 -07:00
Ying Li
2968fa7407 Merge pull request #648 from cyli/cert-rotation
Root rotation (previously root cert rotation)
2016-04-14 11:06:05 -07:00
Riyaz Faizullabhoy
d49544587b Include example of version numbers
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 10:45:49 -07:00
Riyaz Faizullabhoy
76e265935a Add note about versioning
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-14 10:11:07 -07:00
Ying Li
f8cd53cf2f When signing the root, modify and sign a temporary root that gets assigned back into
the repo if signing was successful.  This way, we don't mutate the existing root
in a failed attempt to sign it.

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 22:25:43 -07:00
Ying Li
cea46f7c3e Change root cert rotation to be root key rotation instead
Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 22:12:53 -07:00
Ying Li
708507adde Require signing with all previous roles, instead of just the immediately previous role
Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 22:10:58 -07:00
Ying Li
54d1cb1855 Revert "Implement new (notary cert rotate) command."
This reverts commit 684c17867740e77460f2940d3d76023f7a9647ed, and extra cert rotate test changes

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 11:48:36 -07:00
Ying Li
160ea2bc54 Address review comments and improve docstrings
Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 11:48:36 -07:00
Ying Li
6dfeca463f Add another test for finding old root roles in the root.json
Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 11:48:36 -07:00
Ying Li
7bc485faae Change root signing such that when root keys change, the role is stored as a
versioned root role in the root.json.  That way we can figure out which keys were
previously root keys.

Update tuf.Repo.sign to take a list of required roles (at most two, for root
rotations, because only the immediate root after a rotation absolutely needs
to correctly validate against the previous root role and the new root role)
instead of just a single role.

tuf.Repo.sign now ensures that the number of signatures on the metadata satisfy
role requirements for every required role.  Then it tries to sign with
whatever optional keys it can, ignoring errors and not requiring that any
particular number of signatures were produced with the optional keys.

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 11:48:36 -07:00
Ying Li
8447e0e1da Move the optional signing key logic from tuf/signed to tuf.Repo.sign
Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 11:48:36 -07:00
Ying Li
8be82b31f0 Revert "Modify signed.Sign to replace, not add, signatures."
This reverts commit aca1cf6d43ce9aa423a9de21ac5bcdca35cbe811, and modifies a test to expect 2 signatures.

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
5a14044574 Implement new (notary cert rotate) command.
This is a trivial wrapper around the NotaryRepository functionality. The
UI is simplest possible, a single (notary cert rotate) rotates all
certificates.

This handles the common case (only a single certificate)
perfectly

If there were multiple certificates, rotating all of them
regardless of age does not really hurt; we can easily extend this to
give the user more control (having the user specify a cert ID, for
example) later if necessary.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
d835fbbca2 Implement root certificate rotation in NotaryRepository
NotaryRepository can now list root certificates, and
generate new versions (as changelists to be applied
on Publish).

This is a pretty mechanical encapsulation of the
root certificate rotation support in Repo.AddBaseKeys
and Repo.RemoveBaseKeys. The only slightly interesting
part is ListRootCert, which requires on-line access
to ensure fresh data, and depends on CertStore doing
some verification for us.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
34aa149cbc Implement root certificate rotation in tuf.Repo
Repo.RotateBaseKeys() can now be used to replace the root keypair
(i.e. primarily the expiring certificate), and Repo.SignRoot will make
signatures using the old keypairs if they are available (but not fail if
they are not; only the keypairs listed in the role as trusted are
mandatory).

To do this, we need to keep even the old, possibly currently untrusted,
keypairs around (to allow rollover from clients which trust these old
keys):

The private keys are simply not deleted from the repo's CryptoService;
this means no change for the current setup with a long-term private key
and periodically expiring certificates, but a true rotation of the
private key will eventually require explicit management of the preserved
long-term private keys (if we are keeping several of them around, but
most are either obsolete and non-preferred or possibly even known to be
compromised, we will want to make sure that we always use the
new/preferred private key for new certificate generation).

The public keys are tricker:

1) We need to keep a list of them; the private keys can be looked up
by their IDs, and that allows extracting the public part as well,
but we need a list of the key IDs.  We can't just keep the key IDs
included in the role's list of authorized keys, that would make it
impossible to rotate away from a suspect or known compromsied key.

2) With the X.509 certificate “public keys”, key ID is not actually
sufficient to retrieve the full public key even if we have access to the
private key; we actually need to store the full public key ==
certificate somewhere.  And preferably without having tuf.Repo depend on
a certs.Manager, designed to deal with concepts of trust at a higher
level than TUF cares about.  Actually, to the extent certs.Manager's
purpose is to manage and verify trust, storing old, possibly suspect or
known compromised certificates would be explicitly contrary to its
mission.

So, this patch keeps around full copies of the certificates in the
root.json “keys” map (not the “roles” map of trusted keys). It means
sending to clients a little data which they don't need but it is
otherwise harmless; and keeping the certificates within the
structured and managed tuf.data.Root format could allow us to build nice
UI (e.g. show me all certificates we still carry and keep signing with, let
me drop two of them now that our company has changed a name and does not
want to advertise the history) if we ever needed to something like this

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
816c1c980c Modify signed.Sign to replace, not add, signatures.
The only thing depending on signed.Sign keeping old signatures was two
tests; all real users were modifying the signed data without clearing
the old signatures, and therefore implicitly relying on signing with
the keys which were used for the old signatures as well.

This broke signing an updated root with a new certificate when the old
certificate was no longer available.

It could have been fixed by keeping signed.Sign as is and adding the
clearing to all users, but noting actually needs the appending
semantics, the appending semantics is surprising, and switching to
replacing signatures is less code.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
925e956fca Pass through any errors from signed.Sign
Instead of silently ignoring any signing errors, and failing with a
generic ErrInsufficientSignatures, just pass through any signing errors.

As long as the role's threshold is met, some primary keys may be
missing. But if any key does exist (whether primary or optional), failures
using that key are fatal. (This is why signed.Sign needs to know the
role's threshold; making any missing key fatal would break signatures
of roles with 2 valid keys and threshold of 1.)

As a side effect, this improves the “snapshot key is not available”
detection in NotaryRepository.Publish, to trigger only when a key is not
available, not also when it should be available but using it is failing.

Similarly, the ErrInsufficientSignatures error now mentions only the
missing keys, not the successfully used keys now that present but
failing keys are handled by using other error types.

(This removes the second ErrInsufficientSignatures check because it is
no longer necessary, which also fixes the fact that it was broken by
introducing optional keys. The first, now only, ErrInsufficientSignatures
check does correctly distinguish between primary and optional keys.)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
aa596338a6 Add an optionalKeys parameter to signed.Sign
Ordinarily we don't want to continue operating on signed data if the
role's threshold of signatures cannot be me and the signature is unsuable.

OTOH we want to keep signing root.json with all older keys if they are
available (to allow migration), but in that case a missing key is not a
fatal error.

So, split the keys passed to signed.Sign into primary and optional,
treating all current uses as primary and enforcing the role's threshold
only on primary keys.  Also update the single existing test which uses
a missing/unusable key to use the optionalKeys parameter.

Note that only the _presence_ of optionalKeys is optional; if an
optional key exists but signing using it fails, the function will fail.

This temporarily breaks the second ErrInsufficientSignatures check
(optional keys count against the role threshold), but that will be fixed
soon.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
f23f2093e3 Create enough signatures as role's threshold requires
Tell signed.Sign how many signatures are necessary to sign a role, and
have it fail if it cannot create that many.

For most uses this does not make much of a difference because the
threshold tends to be 1 and signed.Sign was already failing if no key
could be found or if no signature could be created; only >1-threshold
roles now (correctly) fail in additional situations. But the knowledge
of a role’s threshold will be useful in a future commit.

Always use ErrInsufficientSignatures for this failure, whether this is
when loading the keys or actually using them (also fixing
ErrInsufficentSignature documentation to refer to signing and not
verification). ErrNoKeys is no longer returned by signed.Sign.

So, adjust the “snapshot key is not available” logic in
NotaryRepository.Publish accordingly, which also makes it more precise
(actually triggering only when no snapshot key is available).

Now that role's threshold is enforced when signing, update
TestValidateRootInvalidTimestampThreshold to create the second key
necessary to correctly sign the timestamp role.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2016-04-13 11:48:36 -07:00
Miloslav Trmač
a5da680e90 Modify signed.Sign to use a slice instead of varargs
This is a refactoring with no behavior change, but it will allow adding
more parameters to the function in the future.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2016-04-13 11:48:36 -07:00
Riyaz Faizullabhoy
6a3c5d7cc7 Merge pull request #680 from HuKeping/comments
Update some comments
2016-04-13 10:12:43 -07:00
HuKeping
ef6dac2bfe Update code comments
Change to a more readable format.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2016-04-13 22:00:36 +08:00
HuKeping
fd0b628e70 Update some comments
If we update the maximum value for "infinite" in the future, it is possible
that we forget to update this comments and thus leads misunderstanding.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2016-04-13 21:23:51 +08:00
Ying Li
812acce15f Merge pull request #678 from cyli/lint-fix
fix "make lint" to lint every file
2016-04-12 22:54:41 -07:00
Ying Li
39d312dac6 While we're at cleanups, change "name" in keydbstore to be keyID to more
accurately reflect what the name of the key is.

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-12 22:28:32 -07:00
Ying Li
cf4e726514 "make lint" wasn't actually linting every file in the repo. golint ./...
ignores buildtags, for instance, and somehow didn't pick up some code in
the signer.

This calls golint on every go file in the repo and also fixes some linting
issues, which involves renaming two yubikey functions to avoid stuttering.

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-12 22:28:32 -07:00
David Lawrence
d01d666771 Merge pull request #656 from docker/build-run-integration-test
Build run integration test
2016-04-12 21:57:01 -07:00
David Lawrence
a301086be0 Merge pull request #676 from docker/make-integration-fix
Write to file to signal success for make integration
2016-04-12 21:51:46 -07:00
David Lawrence
18fa8a2fc6 Merge pull request #677 from cyli/mtrmac-cleanups
@mtrmac's cleanups, split from root/cert rotation feature
2016-04-12 20:56:15 -07:00
Riyaz Faizullabhoy
20711a4128 Pipe build output to dev null
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-12 18:09:55 -07:00
Ying Li
dcc41be3e1 Rename cert function, and also check if newCertMap is empty.
Also do not check for CKR_FUNCTION_FAILED for error translation

Signed-off-by: Ying Li <ying.li@docker.com>
2016-04-12 16:07:37 -07:00
Riyaz Faizullabhoy
23a1cf3c60 Make file accessible to CI
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-12 16:03:04 -07:00
Riyaz Faizullabhoy
cb080ede06 Fix test_output path and remove
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-12 15:32:58 -07:00
Riyaz Faizullabhoy
4216a37149 Write to file to signal success for make integration
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-04-12 15:32:58 -07:00