Skip to content
This repository has been archived by the owner on Oct 2, 2024. It is now read-only.

fix test suite for Docker BuildKit #1707

Merged
merged 3 commits into from
Aug 29, 2023
Merged

fix test suite for Docker BuildKit #1707

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6128c04
deal with Docker’s changed COPY behavior
reidpr Aug 18, 2023
204fd52
fix the rest of the BuildKit test suite failures
reidpr Aug 22, 2023
3fb6841
respond to feedback [skip ci]
reidpr Aug 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 10 additions & 13 deletions doc/ch-image.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -843,6 +843,12 @@ Can be repeated.
:code:`COPY`
~~~~~~~~~~~~

.. note:: The behavior described here matches Docker’s `now-deprecated legacy
builder
<https://docs.docker.com/engine/deprecated/#legacy-builder-for-linux-images>`_.
Docker’s new builder, BuildKit, has different behavior in some
cases, which we have not characterized.

Especially for people used to UNIX :code:`cp(1)`, the semantics of the
Dockerfile :code:`COPY` instruction can be confusing.

Expand Down Expand Up @@ -885,19 +891,10 @@ bug-compatible.
at the 2nd level or deeper, the source directory’s metadata (e.g.,
permissions) are copied to the destination directory. (Not documented.)

5. If an object appears in both the source and destination, and is at the 2nd
level or deeper, and is of different types in the source and destination,
then the source object will overwrite the destination object. (Not
documented.) For example, if :code:`/tmp/foo/bar` is a regular file, and
:code:`/tmp` is the context directory, then the following Dockerfile
snippet will result in a *file* in the container at :code:`/foo/bar`
(copied from :code:`/tmp/foo/bar`); the directory and all its contents will
be lost.

.. code-block:: docker

RUN mkdir -p /foo/bar && touch /foo/bar/baz
COPY foo /foo
5. If an object (a) appears in both the source and destination, (b) is at the
2nd level or deeper, and (c) is different file types in source and
destination, the source object will overwrite the destination object. (Not
documented.)

We expect the following differences to be permanent:

Expand Down
77 changes: 41 additions & 36 deletions examples/copy/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,14 +164,17 @@ COPY dirA dir08a
# Dest: new directory, one level, trailing slash
COPY dirA dir08b/

# Source: one directory
# Dest: existing file, 2nd level
# NOTE: Not currently tested (see #1707). We keep it around but commented out
# to illustrate what we really would like to test.
#
# Note: While this fails if the existing file is at the top level (which we
# verify in test/build/50_dockerfile.bats), if the existing file is at the 2nd
# level, it's overwritten by the directory.
RUN touch dir08a/dirCb
COPY dirCa dir08a
# # Source: one directory
# # Dest: existing file, 2nd level
# #
# # Note: While this fails if the existing file is at the top level (which we
# # verify in test/build/50_dockerfile.bats), if the existing file is at the 2nd
# # level, it's overwritten by the directory.
# RUN touch dir08a/dirCb
# COPY dirCa dir08a

# Source: two directories, explicit
# Dest: existing directory
Expand Down Expand Up @@ -269,37 +272,39 @@ RUN test $(stat -c '%A' dir19/dir19a2 | cut -c8-) != 'rwx' \

## Destination: Symlink, 2nd level.

# Note: This behavior is DIFFERENT from the symlink at 1st level tests above
# (recall we are trying to be bug-compatible with Docker).

# Set up destination.
RUN mkdir dir20 \
&& echo new > dir20/filex \
&& mkdir dir20/dirx \
&& for i in $(seq 4); do \
echo file$i > dir20/file$i \
&& ln -s file$i dir20/s_file$i \
&& mkdir dir20/dir$i \
&& echo dir$i/file_ > dir20/dir$i/file_ \
&& ln -s dir$i dir20/s_dir$i; \
done \
&& ls -lR dir20

# Copy in the new directory tree. In all of these cases, the source simply
# overwrites the destination; symlinks are not followed.
# NOTE: Not currently tested (see #1707).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to add some extra context for this comment chunk, particularly the commented out Dockerfile instruction that's still included.

#
# # Note: This behavior is DIFFERENT from the symlink at 1st level tests above
# # (recall we are trying to be bug-compatible with Docker).
#
# name source destination
# ------- ------------ ------------
# 1. s_file1 file link to file
# 2. s_dir1 file link to dir
# 3. s_file2 link to file link to file
# 4. s_dir2 link to file link to dir
# 5. s_file3 link to dir link to file
# 6. s_dir3 link to dir link to dir
# 7. s_file4 directory link to file
# 8. s_dir4 directory link to dir
# # Set up destination.
# RUN mkdir dir20 \
# && echo new > dir20/filex \
# && mkdir dir20/dirx \
# && for i in $(seq 4); do \
# echo file$i > dir20/file$i \
# && ln -s file$i dir20/s_file$i \
# && mkdir dir20/dir$i \
# && echo dir$i/file_ > dir20/dir$i/file_ \
# && ln -s dir$i dir20/s_dir$i; \
# done \
# && ls -lR dir20
#
COPY dirG dir20/
# # Copy in the new directory tree. In all of these cases, the source simply
# # overwrites the destination; symlinks are not followed.
# #
# # name source destination
# # ------- ------------ ------------
# # 1. s_file1 file link to file
# # 2. s_dir1 file link to dir
# # 3. s_file2 link to file link to file
# # 4. s_dir2 link to file link to dir
# # 5. s_file3 link to dir link to file
# # 6. s_dir3 link to dir link to dir
# # 7. s_file4 directory link to file
# # 8. s_dir4 directory link to dir
# #
# COPY dirG dir20/


## Wrap up; this output helps to build the expectations in test.bats.
Expand Down
74 changes: 0 additions & 74 deletions examples/copy/test.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ dir16/
dir17/
dir18/
dir19/
dir20/
file1a
file1b
file2
Expand Down Expand Up @@ -131,13 +130,7 @@ fileAa
fileAa

./dir08a:
dirCb/
fileAa
symlink-to-dirCb@

./dir08a/dirCb:
fileCba
fileCbb

./dir08b:
fileAa
Expand Down Expand Up @@ -221,51 +214,6 @@ file19c1

./dir19/dir19a3:
file19b1

./dir20:
dir1/
dir2/
dir3/
dir4/
dirx/
diry/
file1
file2
file3
file4
filex
filey
s_dir1
s_dir2@
s_dir3@
s_dir4/
s_file1
s_file2@
s_file3@
s_file4/

./dir20/dir1:
file_

./dir20/dir2:
file_

./dir20/dir3:
file_

./dir20/dir4:
file_

./dir20/dirx:

./dir20/diry:
file_

./dir20/s_dir4:
file_

./dir20/s_file4:
file_
EOF
}

Expand Down Expand Up @@ -298,8 +246,6 @@ f: ./dir07c/fileAa: dirA/fileAa
f: ./dir07d/fileAa: dirA/fileAa
f: ./dir07e/fileAa: dirA/fileAa
f: ./dir07f/fileAa: dirA/fileAa
f: ./dir08a/dirCb/fileCba: dirCa/dirCb/fileCba
f: ./dir08a/dirCb/fileCbb: dirCa/dirCb/fileCbb
f: ./dir08a/fileAa: dirA/fileAa
f: ./dir08b/fileAa: dirA/fileAa
f: ./dir09/fileAa: dirA/fileAa
Expand Down Expand Up @@ -336,21 +282,6 @@ f: ./dir19/dir19a3/file19b1: new
f: ./dir19/file19a1: old
f: ./dir19/file19a2: new
f: ./dir19/file19a3: new
f: ./dir20/dir1/file_: dir1/file_
f: ./dir20/dir2/file_: dir2/file_
f: ./dir20/dir3/file_: dir3/file_
f: ./dir20/dir4/file_: dir4/file_
f: ./dir20/diry/file_: diry/file_
f: ./dir20/file1: file1
f: ./dir20/file2: file2
f: ./dir20/file3: file3
f: ./dir20/file4: file4
f: ./dir20/filex: new
f: ./dir20/filey: new
f: ./dir20/s_dir1: new
f: ./dir20/s_dir4/file_: s_dir4/file_
f: ./dir20/s_file1: new
f: ./dir20/s_file4/file_: s_file4/file_
f: ./file1a: fileA
f: ./file1b: fileA
f: ./file2: fileB
Expand All @@ -367,14 +298,9 @@ EOF

diff -u - <(ch-run --cd /test "$ch_img" \
-- find . -type l -printf '%y: %p -> %l\n' | sort) <<EOF
l: ./dir08a/symlink-to-dirCb -> dirCb
l: ./dir14/symlink-to-fileDa -> fileDa
l: ./dir15/symlink-to-fileDa -> fileDa
l: ./dir16/symlink-to-dirEb -> dirEb
l: ./dir20/s_dir2 -> filey
l: ./dir20/s_dir3 -> diry
l: ./dir20/s_file2 -> filey
l: ./dir20/s_file3 -> diry
l: ./symlink-to-dir01c -> dir01c
l: ./symlink-to-dir01d -> /test/dir01d
l: ./symlink-to-dir01e -> dir01e
Expand Down
Loading