Commit Graph

1656 Commits

Author SHA1 Message Date
Michael Crosby
2acb856dff Merge pull request #9233 from inatatsu/fix-pkg-units-size-for-gccgo
Fix to avoid a compile error due to float to int truncation with GCCGO
2014-12-18 18:33:01 -08:00
unclejack
e7c0587d54 pkg/tarsum: delete the logging code
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2014-12-18 23:38:26 +02:00
Josh Hawn
35a22c9e12 Refactor to optimize storage driver ApplyDiff()
To avoid an expensive call to archive.ChangesDirs() which walks two directory
trees and compares every entry, archive.ApplyLayer() has been extended to
also return the size of the layer changes.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-17 21:54:23 -08:00
Srini Brahmaroutu
c16e41245e Fix to avoid a compile error due to float to int truncation with GCCGO
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2014-12-17 22:44:55 +00:00
Álex González
be27d97118 Log when truncindex.Get returns >1 container
When the user is not using the full has to retrieve a container it's
possible that we find conflicts with the ids of other containers.

At the moment it's just failing saying that it can not find a container,
but it doesn't say why. Adding a small log saying that duplicates where
found is going to help the user.

Closes #8098

Signed-off-by: Alex Gonzalez <agonzalezro@gmail.com>
2014-12-13 17:43:05 +00:00
Alexander Morozov
a7ae7fed73 Fix vet errors about formatting directives
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2014-12-12 10:58:56 -08:00
Alexander Morozov
e7d086c2be Fix vet errors about unkeyed fields
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2014-12-12 10:44:59 -08:00
Michael Crosby
001ac15b54 Decompress archive before streaming the unpack in a chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/archive.go
	pkg/chrootarchive/archive.go

Conflicts:
	pkg/archive/archive.go
2014-12-11 16:29:13 -05:00
Michael Crosby
7862f831fe Update chroot apply layer to handle decompression outside chroot
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Conflicts:
	pkg/archive/diff.go
	pkg/chrootarchive/archive.go

Conflicts:
	pkg/archive/diff.go
	pkg/chrootarchive/diff.go
2014-12-11 16:28:24 -05:00
Tibor Vass
5f0f0c228d symlink: add more tests
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-11 16:26:01 -05:00
Tibor Vass
ff4d05ed73 symlink: cleanup names and break big test into multiple smaller ones
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-11 16:26:01 -05:00
Tibor Vass
b7f352b0fb Refactor of symlink tests to remove testdata dir
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-12-11 16:26:01 -05:00
Tianon Gravi
fa3ec89515 Simplify FollowSymlinkInScope based on Go 1.3.3's EvalSymlinks
Signed-off-by: Andrew Page <admwiggin@gmail.com>
2014-12-11 16:26:01 -05:00
Vincent Batts
74bbb93571 Merge pull request #9258 from rhvgoyal/transaction-id-improvements
devmapper fix usage of pool transaction id
2014-12-11 12:58:18 -05:00
Michael Crosby
428c263a6c Merge pull request #9099 from vbatts/vbatts-tarsum_spec
pkg/tarsum: specification on TarSum checksum algorithm
2014-12-10 10:24:48 -08:00
Michael Crosby
d1535131d2 Flush stdin from within chroot archive
This makes sure that we don't buffer in memory and that we also flush
stdin from diff as well as untar.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2014-12-08 15:41:07 -08:00
Tibor Vass
c9285a08b1 Merge pull request #9539 from flynn/chroot-untar-empty-archive-slow
Fix chroot untar for zero padded archive from slow reader
2014-12-08 17:59:21 -05:00
Erik Dubbelboer
46b1194d99 Removed race condition
If two interrupts were fired really quickly interruptCount could have been incremented twice before the LoadUint32 making cleanup not being called at all.

Signed-off-by: Erik Dubbelboer <erik@dubbelboer.com>
2014-12-06 22:52:19 +08:00
Lewis Marshall
c128c134d7 Fix chroot untar for zero padded archive from slow reader
Signed-off-by: Lewis Marshall <lewis@lmars.net>
2014-12-06 02:44:31 +00:00
Alexander Morozov
984b072a37 Merge pull request #9479 from jlhawn/tarsum_test_fix
Correct TarSum benchmarks: 9kTar and 9kTarGzip
2014-12-05 14:47:54 -08:00
Josh Hawn
385917e22c Correctly close generated benchmark archives
Another update to TarSum tests, this patch fixes an issue where
the benchmarks were generating archives incorrectly by not closing
the tarWriter.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-03 22:26:06 -08:00
Jessica Frazelle
6fd818f3ef Fix output format where no variable specified in mount pkg
Docker-DCO-1.1-Signed-off-by: Jessica Frazelle <jess@docker.com> (github: jfrazelle)
2014-12-03 19:02:51 -08:00
Jessie Frazelle
8520fd55b6 Merge pull request #9451 from erikh/fix-sqlite
graphdb: initialize the database semi-idempotently on every connection, not just new installs.
2014-12-03 17:17:20 -08:00
Andy Goldstein
48ec176cd5 Fix invalid argument error on push
With 32ba6ab from #9261, TempArchive now closes the underlying file and
cleans it up as soon as the file's contents have been read. When pushing
an image, PushImageLayerRegistry attempts to call Close() on the layer,
which is a TempArchive that has already been closed. In this situation,
Close() returns an "invalid argument" error.

Add a Close method to TempArchive that does a no-op if the underlying
file has already been closed.

Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2014-12-03 16:12:24 -05:00
Josh Hawn
92fd49f7ca Correct TarSum benchmarks: 9kTar and 9kTarGzip
These two cases did not actually read the same content with each iteration
of the benchmark. After the first read, the buffer was consumed. This patch
corrects this by using a bytes.Reader and seeking to the beginning of the
buffer at the beginning of each iteration.

Unfortunately, this benchmark was not actually as fast as we believed. But
the new results do bring its results closer to those of the other benchmarks.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-03 10:44:05 -08:00
Vivek Goyal
7b2b15d3e9 devmapper: Use device id as specified by caller
Currently devicemapper CreateDevice and CreateSnapDevice keep on retrying
device creation till a suitable device id is found. 

With new transaction mechanism we need to store device id in transaction
before it has been created.

So change the logic in such a way that caller decides the devices Id to
use. If that device Id is not available, caller bumps up the device Id
and retries.

That way caller can update transaciton too when it tries a new Id. Transaction
related patches will come later in the series.
  
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2014-12-03 13:06:43 -05:00
Josh Hawn
8d9e25dbdd Fix TarSum iteration test
I noticed that 3 of the tarsum test cases had expected a tarsum with
a sha256 hash of
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
As I've been working with sha256 quite a bit lately, it struck me that
this is the initial digest value for sha256, which means that no data
was processed. However, these tests *do* process data. It turns out that
there was a bug in the test handling code which did not wait for tarsum
to end completely. This patch corrects these test cases.

I'm unaware of anywhere else in the code base where this would be an issue,
though we definitily need to look out in the future to ensure we are
completing tarsum reads (waiting for EOF).

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2014-12-02 15:28:13 -08:00
Erik Hollensbe
fb06e1c6eb graphdb: initialize the database semi-idempotently on every connection.
Docker-DCO-1.1-Signed-off-by: Erik Hollensbe <github@hollensbe.org> (github: erikh)
2014-12-02 03:02:25 -08:00
Michael Crosby
42522d29c7 Merge pull request #9407 from hqhq/hq_fix_comments
flag: fix the comments
2014-12-01 11:27:06 -08:00
Qiang Huang
9cc73c62e6 flag: fix the comments
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2014-11-30 01:58:16 +08:00
Alexandr Morozov
be5bfbe221 Change path breakout detection logic in archive package
Fixes #9375

Signed-off-by: Alexandr Morozov <lk4d4@docker.com>
2014-11-28 09:44:47 -08:00
Vincent Batts
b47ff77b5c Merge pull request #9006 from snitm/thin-pool-improvements
Thin pool improvements
2014-11-26 14:44:09 -05:00
Vincent Batts
79064a2820 Merge pull request #9279 from yoheiueda/termios-via-cgo
Use termios via CGO
2014-11-25 20:04:26 -05:00
Brian Goff
bd4fe9b986 Cleanup ParseHost
Current implementation is comingling things that ought not be together.
There are _some_ similarities between parsing for the different proto
types, but they are more different than alike, making the code extremely
difficult to reason about.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2014-11-25 18:55:40 -05:00
unclejack
3db5ea1617 pkg/tarsum: add maintainers & add missing s
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2014-11-26 00:44:16 +02:00
Yohei Ueda
dfcf05a5d0 Use termios via CGO only on Linux
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2014-11-25 20:49:01 +09:00
Tatsushi Inagaki
82a5cd0d37 Fix to avoid a compilation error of size_test.go with GCCGO due to float to int truncation
Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>
2014-11-25 18:12:22 +09:00
Tatsushi Inagaki
36560a76d7 Revert "Fix to avoid a compile error due to float to int truncation with GCCGO"
This reverts commit 967a42f116.

Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>

Roll back the change to fix the parameter of HumanSize from int64 to float64
2014-11-25 16:48:09 +09:00
Vincent Batts
7f84174109 tarsum: updates for jamtur01 comments
Signed-off-by: Vincent Batts <vbatts@redhat.com>
2014-11-24 23:18:48 -05:00
Vincent Batts
3d6e63e0c4 pkg/tarsum: review cleanup
Signed-off-by: Vincent Batts <vbatts@redhat.com>
2014-11-24 23:18:48 -05:00
Vincent Batts
3e08fb5ad2 pkg/tarsum: review amendments
(separate commit to preserve github conversation)

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2014-11-24 23:18:48 -05:00
Vincent Batts
f30fee69b1 pkg/tarsum: specification on TarSum checksum
Signed-off-by: Vincent Batts <vbatts@redhat.com>
2014-11-24 23:18:47 -05:00
Tibor Vass
e6ec703df3 Merge pull request #9323 from unclejack/merge_release_v1.3.2
Merge release v1.3.2 to master
2014-11-24 19:26:36 -05:00
Jessie Frazelle
2ad98b0cfa Merge pull request #9324 from crosbymichael/git-urls
Move IsGIT and IsURL to urlutil package
2014-11-24 16:07:42 -08:00
Michael Crosby
fc66a22df6 Merge pull request #9173 from vbatts/vbatts-dm_clarify_TaskCreate
pkg/devicemapper: clarify TaskCreate and createTask
2014-11-24 15:45:54 -08:00
Michael Crosby
feca1b1780 Move git and url checks into pkg
This moves the IsGIT and IsURL functions out of the generic `utils`
package and into their own `urlutil` pkg.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2014-11-24 18:10:37 -05:00
Tibor Vass
f6d9780229 archive: do not call FollowSymlinkInScope in createTarFile
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-11-25 01:03:41 +02:00
unclejack
330171e1d9 pkg/chrootarchive: provide TMPDIR for ApplyLayer
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-11-25 01:03:41 +02:00
Tibor Vass
31d1d73303 archive: prevent breakout in ApplyLayer
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-11-25 01:03:41 +02:00
Tibor Vass
1852cc3841 archive: prevent breakout in Untar
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2014-11-25 01:03:41 +02:00