From: Apple <opensource@apple.com>
Date: Tue, 22 Sep 2020 23:24:34 +0000 (+0000)
Subject: copyfile-173.40.2.tar.gz
X-Git-Tag: macos-1101^0
X-Git-Url: https://git.saurik.com/apple/copyfile.git/commitdiff_plain?ds=inline
copyfile-173.40.2.tar.gz
---
diff --git a/copyfile.3 b/copyfile.3
index 55d4454..08d07a2 100644
--- a/copyfile.3
+++ b/copyfile.3
@@ -125,13 +125,15 @@ file had extended attributes but no ACLs, the return value would be
.It Dv COPYFILE_PACK
Serialize the
.Va from
-file. The
+file.
+The
.Va to
file is an AppleDouble-format file.
.It Dv COPYFILE_UNPACK
Unserialize the
.Va from
-file. The
+file.
+The
.Va from
file is an AppleDouble-format file; the
.Va to
@@ -168,7 +170,8 @@ file. (This is only applicable for the
.Fn copyfile
function.) No error is returned if
.Xr remove 3
-fails. Note that
+fails.
+Note that
.Xr remove 3
removes a symbolic link itself, not the
target of the link.
@@ -185,8 +188,8 @@ This flag is equivalent to (COPYFILE_EXCL | COPYFILE_ACL | COPYFILE_STAT | COPYF
| COPYFILE_NOFOLLOW_SRC).
Note that if cloning is successful, progress callbacks will not be invoked.
Note also that there is no support for cloning directories: if a directory is provided as the source,
-an error will be returned. Since this flag implies COPYFILE_NOFOLLOW_SRC, symbolic links themselves will
-be cloned instead of their targets.
+an error will be returned.
+Since this flag implies COPYFILE_NOFOLLOW_SRC, symbolic links themselves will be cloned instead of their targets.
(This is only applicable for the
.Fn copyfile
function.)
@@ -197,9 +200,9 @@ This flag is equivalent to (COPYFILE_EXCL | COPYFILE_ACL | COPYFILE_STAT | COPYF
| COPYFILE_NOFOLLOW_SRC).
Note that if cloning is successful, progress callbacks will not be invoked.
Note also that there is no support for cloning directories: if a directory is provided as the source and
-COPYFILE_CLONE_FORCE is not passed, this will instead copy the directory. Since this flag implies COPYFILE_NOFOLLOW_SRC,
-symbolic links themselves will be cloned instead of their targets. Recursive copying however is
-supported, see below for more information.
+COPYFILE_CLONE_FORCE is not passed, this will instead copy the directory.
+Since this flag implies COPYFILE_NOFOLLOW_SRC, symbolic links themselves will be cloned instead of their targets.
+Recursive copying however is supported, see below for more information.
(This is only applicable for the
.Fn copyfile
function.)
@@ -216,16 +219,19 @@ if sparse copying cannot be performed for any reason; otherwise, an error is ret
This is a convenience macro, equivalent to
.Dv (COPYFILE_NOFOLLOW_DST | COPYFILE_NOFOLLOW_SRC) .
.It Dv COPYFILE_RUN_IN_PLACE
-If the src file has quarantine information, add the QTN_FLAG_DO_NOT_TRANSLOCATE flag to the quarantine information of the dst file. This allows a bundle to run in place instead of being translocated.
+If the src file has quarantine information, add the QTN_FLAG_DO_NOT_TRANSLOCATE flag to the quarantine information of the dst file.
+This allows a bundle to run in place instead of being translocated.
.It Dv COPYFILE_PRESERVE_DST_TRACKED
Preserve the UF_TRACKED flag at
.Va to
when copying metadata, regardless of whether
.Va from
-has it set. This flag is used in conjunction with COPYFILE_STAT, or COPYFILE_CLONE (for its fallback case).
+has it set.
+This flag is used in conjunction with COPYFILE_STAT, or COPYFILE_CLONE (for its fallback case).
.El
.Pp
-Copying files into a directory is supported. If
+Copying files into a directory is supported.
+If
.Va to
is a directory,
.Va from
@@ -269,7 +275,8 @@ parameters are pointers to
.It Dv COPYFILE_STATE_SRC_FILENAME
.It Dv COPYFILE_STATE_DST_FILENAME
Get or set the filename associated with the source (or destination)
-file. If it has not been initialized yet, the value will be
+file.
+If it has not been initialized yet, the value will be
.Dv NULL .
For
.Fn copyfile_state_set ,
@@ -385,7 +392,8 @@ entered. (That is, none of the filesystem objects contained
within the directory have been copied yet.)
.It Dv COPYFILE_RECURSE_DIR_CLEANUP
The object being copied is a directory, and all of the
-objects contained have been copied. At this stage, the destination directory
+objects contained have been copied.
+At this stage, the destination directory
being copied will have any extra permissions that were added to
allow the copying will be removed.
.It Dv COPYFILE_RECURSE_ERROR
@@ -403,15 +411,15 @@ The second argument to the call-back function will indicate
the stage of the copy, and will be one of the following values:
.Bl -tag -width COPYFILE_FINISH
.It Dv COPYFILE_START
-Before copying has begun. The third
-parameter will be a newly-created
+Before copying has begun.
+The third parameter will be a newly-created
.Vt copyfile_state_t
object with the call-back function and context pre-loaded.
.It Dv COPYFILE_FINISH
After copying has successfully finished.
.It Dv COPYFILE_ERR
-Indicates an error has happened at some stage. If the
-first argument to the call-back function is
+Indicates an error has happened at some stage.
+If the first argument to the call-back function is
.Dv COPYFILE_RECURSE_ERROR ,
then an error occurred while processing the source hierarchy;
otherwise, it will indicate what type of object was being copied,
@@ -437,13 +445,14 @@ values:
The copy will continue as expected.
.It Dv COPYFILE_SKIP
This object will be skipped, and the next object will
-be processed. (Note that, when entering a directory.
-returning
+be processed.
+(Note that, when entering a directory, returning
.Dv COPYFILE_SKIP
from the call-back function will prevent the contents
of the directory from being copied.)
.It Dv COPYFILE_QUIT
-The entire copy is aborted at this stage. Any filesystem
+The entire copy is aborted at this stage.
+Any filesystem
objects created up to this point will remain.
.Fn copyfile
will return -1, but
@@ -474,11 +483,13 @@ Note that recursive cloning is also supported with the
.Dv COPYFILE_CLONE
flag (but not the
.Dv COPYFILE_CLONE_FORCE
-flag). A recursive clone operation invokes
+flag).
+A recursive clone operation invokes
.Fn copyfile
with
.Dv COPYFILE_CLONE
-on every entry found in the source file-system object. Because
+on every entry found in the source file-system object.
+Because
.Fn copyfile
does not allow the cloning of directories, a recursive clone will
instead copy any directory it finds (while cloning its contents).
@@ -503,10 +514,12 @@ Note that if the source path ends in a
.Va /
its contents are copied rather than the directory itself (like cp(1)).
The behavior of a recursive copy on a directory hierarchy also depends
-on the contents of the destination. If the destination is a directory,
-the source directory (or its contents, if the source path ends in a
+on the contents of the destination.
+If the destination is a directory, the source directory (or its contents,
+if the source path ends in a
.Va /
-) will be copied into it. If the destination exists but is not a
+) will be copied into it.
+If the destination exists but is not a
directory, and the source is a non-empty directory, the copy will fail;
the exact error set depends on the flags provided to
.Fn copyfile
@@ -518,9 +531,11 @@ and
.Fn fcopyfile
will also use a callback to report data (e.g.,
.Dv COPYFILE_DATA )
-progress. If given, the callback will be invoked on each
+progress.
+If given, the callback will be invoked on each
.Xr write 2
-call. The first argument to the callback function will be
+call.
+The first argument to the callback function will be
.Dv COPYFILE_COPY_DATA .
The second argument will either be
.Dv COPYFILE_PROGRESS
@@ -560,7 +575,8 @@ when finished with each individual attribute.
may be called for all of the extended attributes, before
the first callback with
.Dv COPYFILE_PROGRESS
-is invoked.) Any attribute skipped by returning
+is invoked.)
+Any attribute skipped by returning
.Dv COPYFILE_SKIP
from the
.Dv COPYFILE_START
@@ -621,6 +637,28 @@ changes while the copy is occurring, the results are undefined.
does not reset the seek position for either source or destination.
This can result in the destination file being a different size
than the source file.
+.Sh EXAMPLES
+.Bd -literal -offset indent
+/* Initialize a state variable */
+copyfile_state_t s;
+s = copyfile_state_alloc();
+/* Copy the data and extended attributes of one file to another */
+copyfile("/tmp/f1", "/tmp/f2", s, COPYFILE_DATA | COPYFILE_XATTR);
+/* Convert a file to an AppleDouble file for serialization */
+copyfile("/tmp/f2", "/tmp/tmpfile", NULL, COPYFILE_ALL | COPYFILE_PACK);
+/* Release the state variable */
+copyfile_state_free(s);
+/* A more complex way to call copyfile() */
+s = copyfile_state_alloc();
+copyfile_state_set(s, COPYFILE_STATE_SRC_FILENAME, "/tmp/foo");
+/* One of src or dst must be set... rest can come from the state */
+copyfile(NULL, "/tmp/bar", s, COPYFILE_ALL);
+/* Now copy the same source file to another destination file */
+copyfile(NULL, "/tmp/car", s, COPYFILE_ALL);
+copyfile_state_free(s);
+/* Remove extended attributes from a file */
+copyfile("/dev/null", "/tmp/bar", NULL, COPYFILE_XATTR);
+.Ed
.Sh ERRORS
.Fn copyfile
and
@@ -688,40 +726,18 @@ parameter.
In addition, both functions may set
.Dv errno
via an underlying library or system call.
-.Sh EXAMPLES
-.Bd -literal -offset indent
-/* Initialize a state variable */
-copyfile_state_t s;
-s = copyfile_state_alloc();
-/* Copy the data and extended attributes of one file to another */
-copyfile("/tmp/f1", "/tmp/f2", s, COPYFILE_DATA | COPYFILE_XATTR);
-/* Convert a file to an AppleDouble file for serialization */
-copyfile("/tmp/f2", "/tmp/tmpfile", NULL, COPYFILE_ALL | COPYFILE_PACK);
-/* Release the state variable */
-copyfile_state_free(s);
-/* A more complex way to call copyfile() */
-s = copyfile_state_alloc();
-copyfile_state_set(s, COPYFILE_STATE_SRC_FILENAME, "/tmp/foo");
-/* One of src or dst must be set... rest can come from the state */
-copyfile(NULL, "/tmp/bar", s, COPYFILE_ALL);
-/* Now copy the same source file to another destination file */
-copyfile(NULL, "/tmp/car", s, COPYFILE_ALL);
-copyfile_state_free(s);
-/* Remove extended attributes from a file */
-copyfile("/dev/null", "/tmp/bar", NULL, COPYFILE_XATTR);
-.Ed
.Sh SEE ALSO
-.Xr listxattr 2 ,
.Xr getxattr 2 ,
+.Xr listxattr 2 ,
.Xr setxattr 2 ,
.Xr acl 3
+.Sh HISTORY
+The
+.Fn copyfile
+API was introduced in Mac OS X 10.5.
.Sh BUGS
Both
.Fn copyfile
functions lack a way to set the input or output block size.
.Pp
Recursive copies do not honor hard links.
-.Sh HISTORY
-The
-.Fn copyfile
-API was introduced in Mac OS X 10.5.
diff --git a/copyfile.c b/copyfile.c
index 443dbc5..a745d17 100644
--- a/copyfile.c
+++ b/copyfile.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2004-2019 Apple, Inc. All rights reserved.
+ * Copyright (c) 2004-2020 Apple, Inc. All rights reserved.
*
* @APPLE_LICENSE_HEADER_START@
*
@@ -202,6 +202,41 @@ does_copy_protection(int fd)
return ((sfs.f_flags & MNT_CPROTECT) == MNT_CPROTECT);
}
+static bool
+path_does_copy_protection(const char *path)
+{
+ struct statfs sfs;
+
+ if (statfs(path, &sfs) == -1) {
+ char parent_path[MAXPATHLEN];
+
+ if (errno != ENOENT)
+ return false;
+
+ // If the path doesn't exist,
+ // try to get its parent path and re-attempt the statfs().
+ if (dirname_r(path, parent_path) == NULL)
+ return false;
+
+ if (statfs(parent_path, &sfs) == -1)
+ return false;
+ }
+
+ return ((sfs.f_flags & MNT_CPROTECT) == MNT_CPROTECT);
+}
+
+static int
+do_copy_protected_open(const char *path, int flags, int class, int dpflags, int mode)
+{
+ // The passed-in protection class is meaningful, so use open_dprotected_np().
+ if (path_does_copy_protection(path)) {
+ return open_dprotected_np(path, flags, class, dpflags, mode);
+ }
+
+ // Fall-back to regular open().
+ return open(path, flags, mode);
+}
+
static void
sort_xattrname_list(void *start, size_t length)
{
@@ -1077,6 +1112,74 @@ static int copyfile_clone(copyfile_state_t state)
return ret;
}
+/*
+ * Check if two provided paths are identical,
+ * and if we're able to determine that, return true.
+ */
+static bool copyfile_paths_identical(const char *src, const char *dst)
+{
+ struct attrlist attrs;
+ struct statfs sfs;
+ struct stat src_sb, dst_sb;
+ char volroot[MAXPATHLEN + 1];
+ struct {
+ uint32_t length;
+ vol_capabilities_attr_t volAttrs;
+ } volattrs;
+ char *real_src_path = NULL, *real_dst_path = NULL;
+
+ // Common case: the destination does not exist.
+ if ((stat(dst, &dst_sb) == -1) || (stat(src, &src_sb) == -1))
+ return false;
+
+ // If both files exist, then we next try to check file IDs.
+ // This requires that the underlying filesystem support persistent file IDs.
+ if (statfs(src, &sfs) == -1)
+ return false;
+
+ strlcpy(volroot, sfs.f_mntonname, sizeof(volroot));
+ memset(&attrs, 0, sizeof(attrs));
+ attrs.bitmapcount = ATTR_BIT_MAP_COUNT;
+ attrs.volattr = ATTR_VOL_CAPABILITIES;
+
+ if (getattrlist(volroot, &attrs, &volattrs, sizeof(volattrs), 0) == -1)
+ return false;
+
+ // If the underlying devices are not the same, then the files are not the same.
+ if (src_sb.st_dev != dst_sb.st_dev)
+ return false;
+
+ if ((volattrs.volAttrs.capabilities[VOL_CAPABILITIES_FORMAT] & VOL_CAP_FMT_PERSISTENTOBJECTIDS) &&
+ (volattrs.volAttrs.valid[VOL_CAPABILITIES_FORMAT] & VOL_CAP_FMT_PERSISTENTOBJECTIDS)) {
+ // The underlying source filesystem supports persistent file IDs,
+ // so if our two files have the same file ID on the same device,
+ // they are identical.
+ return (src_sb.st_ino == dst_sb.st_ino);
+ }
+
+ // Finally, if we don't support persistent file ID's,
+ // we fall back to path comparisons.
+ real_src_path = realpath(src, NULL);
+ if (real_src_path == NULL)
+ goto exit;
+
+ real_dst_path = realpath(dst, NULL);
+ if (real_dst_path == NULL)
+ goto exit;
+
+ if (strncasecmp(src, dst, MAXPATHLEN) == 0)
+ return true;
+
+exit:
+ if (real_src_path)
+ free(real_src_path);
+
+ if (real_dst_path)
+ free(real_dst_path);
+
+ return false;
+}
+
/*
* the original copyfile() routine; this copies a source file to a destination
* file. Note that because we need to set the names in the state variable, this
@@ -1134,6 +1237,20 @@ int copyfile(const char *src, const char *dst, copyfile_state_t state, copyfile_
COPYFILE_SET_FNAME(src, s);
COPYFILE_SET_FNAME(dst, s);
+ if (!(s->flags & COPYFILE_CHECK)) {
+ // We have no work to do if `src` and `dst` point to the same place.
+ if (copyfile_paths_identical(src, dst)) {
+ // ...but return an error if requested to do so.
+ if (s->flags & COPYFILE_EXCL) {
+ s->err = EEXIST;
+ goto error_exit;
+ }
+
+ ret = 0;
+ goto exit;
+ }
+ }
+
if (s->flags & COPYFILE_RECURSIVE) {
ret = copytree(s);
goto exit;
@@ -1678,7 +1795,7 @@ static int copyfile_unset_acl(copyfile_state_t s)
*/
static int copyfile_open(copyfile_state_t s)
{
- int oflags = O_EXCL | O_CREAT | O_WRONLY;
+ int oflags = O_EXCL | O_CREAT;
int islnk = 0, isdir = 0, isreg = 0;
int osrc = 0, dsrc = 0;
int prot_class = PROTECTION_CLASS_DEFAULT;
@@ -1749,6 +1866,26 @@ static int copyfile_open(copyfile_state_t s)
if (s->dst && s->dst_fd == -2)
{
+ /*
+ * Per <rdar://60074298>, only open files for writing if we expect
+ * to modify the file's content. This avoids undesirable side effects
+ * of O_WRONLY when we're only modifying metadata/attributes.
+ *
+ * The calls needed by COPYFILE_METADATA (e.g. fchown(), fchmod(),
+ * fchflags(), futimes(), fsetattrlist(), f{set,remove}xattr(), and
+ * acl_set_fd()) are safe to use with O_RDONLY descriptors. They
+ * operate on the underlying vnode_t, as path-based variants do.
+ * These usually only need write permissions in st_mode or ACLs.
+ */
+ const copyfile_flags_t writable_flags = (COPYFILE_DATA | COPYFILE_DATA_SPARSE);
+ if (COPYFILE_PACK & s->flags) {
+ oflags |= O_WRONLY; // always writes file content
+ } else if (COPYFILE_UNPACK & s->flags) {
+ oflags |= O_RDONLY; // only updates metadata
+ } else {
+ oflags |= (writable_flags & s->flags) ? O_WRONLY : O_RDONLY;
+ }
+
/*
* COPYFILE_UNLINK tells us to try removing the destination
* before we create it. We don't care if the file doesn't
@@ -1841,7 +1978,8 @@ static int copyfile_open(copyfile_state_t s)
return -1;
}
set_cprot_explicit = 1;
- } else while((s->dst_fd = open_dprotected_np(s->dst, oflags | dsrc, prot_class, 0, s->sb.st_mode | S_IWUSR)) < 0)
+ } else while((s->dst_fd = do_copy_protected_open(s->dst, oflags | dsrc, prot_class,
+ 0, s->sb.st_mode | S_IWUSR)) < 0)
{
/*
* We set S_IWUSR because fsetxattr does not -- at the time this comment
@@ -2053,10 +2191,13 @@ static int copyfile_data_sparse(copyfile_state_t s, size_t input_blk_size, size_
if (!(s->flags & COPYFILE_DATA_SPARSE)) {
// Don't attempt this unless the right flags are passed.
return ENOTSUP;
- } else if (src_size <= 0) {
+ } else if (src_size < 0) {
// The file size of our source is invalid; there's nothing to copy.
errno = EINVAL;
goto error_exit;
+ } else if (src_size == 0) {
+ // This is a zero-length file: no work to do.
+ goto exit;
}
// Since a major underlying filesystem requires that holes are block-aligned,
@@ -2761,6 +2902,8 @@ static int copyfile_stat(copyfile_state_t s)
return 0;
}
+#define MAX_GETXATTR_RETRIES 3
+#define MAX_XATTR_BUFFER_SIZE (32 * 1024 * 1024) // 32 MiB
/*
* Similar to copyfile_security() in some ways; this
* routine copies the extended attributes from the source,
@@ -2774,31 +2917,53 @@ static int copyfile_stat(copyfile_state_t s)
static int copyfile_xattr(copyfile_state_t s)
{
char *name;
- char *namebuf, *end;
+ char *namebuf = NULL;
+ char *end;
ssize_t xa_size;
void *xa_dataptr;
- ssize_t bufsize = 4096;
+ ssize_t xa_bufsize = 4096;
+ ssize_t namebuf_size = 0;
ssize_t asize;
- ssize_t nsize;
+ ssize_t list_size;
int ret = 0;
int look_for_decmpea = 0;
+ int tries_left = MAX_GETXATTR_RETRIES;
/* delete EAs on destination */
- if ((nsize = flistxattr(s->dst_fd, 0, 0, 0)) > 0)
+dst_restart:
+ if ((list_size = flistxattr(s->dst_fd, 0, 0, 0)) > 0)
{
- if ((namebuf = (char *) malloc(nsize)) == NULL)
- return -1;
- else
- nsize = flistxattr(s->dst_fd, namebuf, nsize, 0);
+ /* this is always true on the first call: no buffer yet (namebuf_size == 0) */
+ if (list_size > namebuf_size) {
+ if (list_size > MAX_XATTR_BUFFER_SIZE) {
+ copyfile_warn("destination's xattr list size (%zu) exceeds the threshold (%d); trying to allocate", list_size, MAX_XATTR_BUFFER_SIZE);
+ }
+ namebuf_size = list_size;
+ void *tdptr = namebuf;
+
+ if ((namebuf =
+ (void *) realloc((void *) namebuf, namebuf_size)) == NULL)
+ {
+ if (tdptr) {
+ free(tdptr);
+ }
+ return -1;
+ }
+ }
+ list_size = flistxattr(s->dst_fd, namebuf, namebuf_size, 0);
- if (nsize > 0) {
+ if ((list_size < 0) && (errno == ERANGE) && (tries_left > 0)) {
+ /* `namebuf` is too small - try again */
+ tries_left--;
+ goto dst_restart;
+ } else if (list_size > 0) {
/*
* With this, end points to the last byte of the allocated buffer
* This *should* be NUL, from flistxattr, but if it's not, we can
* set it anyway -- it'll result in a truncated name, which then
* shouldn't match when we get them later.
*/
- end = namebuf + nsize - 1;
+ end = namebuf + list_size - 1;
if (*end != 0)
*end = 0;
for (name = namebuf; name <= end; name += strlen(name) + 1) {
@@ -2809,15 +2974,24 @@ static int copyfile_xattr(copyfile_state_t s)
fremovexattr(s->dst_fd, name,0);
}
}
- free(namebuf);
- } else
- if (nsize < 0)
- {
- if (errno == ENOTSUP || errno == EPERM)
- return 0;
- else
- return -1;
+ }
+ else if (list_size < 0)
+ {
+ if (namebuf) {
+ free(namebuf);
+ }
+ if (errno == ENOTSUP || errno == EPERM) {
+ return 0;
+ } else {
+ return -1;
}
+ }
+ if (namebuf) {
+ free(namebuf);
+ }
+ namebuf = NULL;
+ namebuf_size = 0;
+ tries_left = MAX_GETXATTR_RETRIES;
#ifdef DECMPFS_XATTR_NAME
if ((s->flags & COPYFILE_DATA) &&
@@ -2829,24 +3003,50 @@ static int copyfile_xattr(copyfile_state_t s)
#endif
/* get name list of EAs on source */
- if ((nsize = flistxattr(s->src_fd, 0, 0, look_for_decmpea)) < 0)
+src_restart:
+ if ((list_size = flistxattr(s->src_fd, 0, 0, look_for_decmpea)) <= 0)
{
- if (errno == ENOTSUP || errno == EPERM)
+ if (namebuf) {
+ free(namebuf);
+ }
+
+ if (list_size == 0) {
return 0;
- else
- return -1;
- } else
- if (nsize == 0)
+ } else if (errno == ENOTSUP || errno == EPERM) {
return 0;
+ } else {
+ return -1;
+ }
+ }
- if ((namebuf = (char *) malloc(nsize)) == NULL)
- return -1;
- else
- nsize = flistxattr(s->src_fd, namebuf, nsize, look_for_decmpea);
+ /* this is always true on the first call: no buffer yet (namebuf_size == 0) */
+ if (list_size > namebuf_size) {
+ if (list_size > MAX_XATTR_BUFFER_SIZE) {
+ copyfile_warn("source's xattr list size (%zu) exceeds the threshold (%d); trying to allocate", list_size, MAX_XATTR_BUFFER_SIZE);
+ }
+ namebuf_size = list_size;
+ void *tdptr = namebuf;
+ if ((namebuf =
+ (void *) realloc((void *) namebuf, namebuf_size)) == NULL)
+ {
+ if (tdptr) {
+ free(tdptr);
+ }
+ return -1;
+ }
+ }
- if (nsize <= 0) {
- free(namebuf);
- return (int)nsize;
+ list_size = flistxattr(s->src_fd, namebuf, namebuf_size, look_for_decmpea);
+
+ if ((list_size < 0) && (errno == ERANGE) && (tries_left > 0)) {
+ /* `namebuf` is too small - try again */
+ tries_left--;
+ goto src_restart;
+ } else if (list_size <= 0) {
+ if (namebuf) {
+ free(namebuf);
+ }
+ return (int)list_size;
}
/*
@@ -2855,11 +3055,11 @@ static int copyfile_xattr(copyfile_state_t s)
* set it anyway -- it'll result in a truncated name, which then
* shouldn't match when we get them later.
*/
- end = namebuf + nsize - 1;
+ end = namebuf + list_size - 1;
if (*end != 0)
*end = 0;
- if ((xa_dataptr = (void *) malloc(bufsize)) == NULL) {
+ if ((xa_dataptr = (void *) malloc(xa_bufsize)) == NULL) {
free(namebuf);
return -1;
}
@@ -2875,17 +3075,22 @@ static int copyfile_xattr(copyfile_state_t s)
if (strncmp(name, XATTR_QUARANTINE_NAME, end - name) == 0)
continue;
+ tries_left = MAX_GETXATTR_RETRIES;
+get_restart:
if ((xa_size = fgetxattr(s->src_fd, name, 0, 0, 0, look_for_decmpea)) < 0)
{
continue;
}
- if (xa_size > bufsize)
+ if (xa_size > xa_bufsize)
{
+ if (xa_size > MAX_XATTR_BUFFER_SIZE) {
+ copyfile_warn("xattr named %s has size (%zu), which exceeds the threshold (%d); trying to allocate", name, list_size, MAX_XATTR_BUFFER_SIZE);
+ }
void *tdptr = xa_dataptr;
- bufsize = xa_size;
+ xa_bufsize = xa_size;
if ((xa_dataptr =
- (void *) realloc((void *) xa_dataptr, bufsize)) == NULL)
+ (void *) realloc((void *) xa_dataptr, xa_bufsize)) == NULL)
{
free(tdptr);
ret = -1;
@@ -2893,8 +3098,13 @@ static int copyfile_xattr(copyfile_state_t s)
}
}
- if ((asize = fgetxattr(s->src_fd, name, xa_dataptr, xa_size, 0, look_for_decmpea)) < 0)
+ if ((asize = fgetxattr(s->src_fd, name, xa_dataptr, xa_bufsize, 0, look_for_decmpea)) < 0)
{
+ if ((errno == ERANGE) && (tries_left > 0)) {
+ /* `xa_dataptr` is too small - try again */
+ tries_left--;
+ goto get_restart;
+ }
continue;
}
diff --git a/copyfile.xcodeproj/project.pbxproj b/copyfile.xcodeproj/project.pbxproj
index f282750..4cd88ec 100644
--- a/copyfile.xcodeproj/project.pbxproj
+++ b/copyfile.xcodeproj/project.pbxproj
@@ -7,7 +7,9 @@
objects = {
/* Begin PBXBuildFile section */
+ 096213F7239827D0005847FC /* identical_test.c in Sources */ = {isa = PBXBuildFile; fileRef = 096213F6239827D0005847FC /* identical_test.c */; };
098AF3B622692BF300F9BA42 /* stat_test.c in Sources */ = {isa = PBXBuildFile; fileRef = 098AF3B522692BF300F9BA42 /* stat_test.c */; };
+ 3EF9FA5F2418B6BA003B43E8 /* readonly_fd_test.c in Sources */ = {isa = PBXBuildFile; fileRef = 3EF9FA5E2418B6BA003B43E8 /* readonly_fd_test.c */; };
721D4F071EA95283000F0555 /* copyfile.c in Sources */ = {isa = PBXBuildFile; fileRef = FCCE17C1135A658F002CEE6D /* copyfile.c */; };
721D4F081EA95290000F0555 /* xattr_flags.c in Sources */ = {isa = PBXBuildFile; fileRef = 72406E621676C3C80099568B /* xattr_flags.c */; };
72406E631676C3C80099568B /* xattr_flags.c in Sources */ = {isa = PBXBuildFile; fileRef = 72406E621676C3C80099568B /* xattr_flags.c */; };
@@ -18,6 +20,7 @@
72B4C0F41676C47D00C13E05 /* copyfile_private.h in Headers */ = {isa = PBXBuildFile; fileRef = 72B4C0F31676C47D00C13E05 /* copyfile_private.h */; settings = {ATTRIBUTES = (Private, ); }; };
72EAA3B016A72F4500833E98 /* xattr_flags.h in Headers */ = {isa = PBXBuildFile; fileRef = 72EAA3AF16A72F4500833E98 /* xattr_flags.h */; settings = {ATTRIBUTES = (Public, ); }; };
86EF9F0A1834018C00AAB3F3 /* xattr_properties.h in Headers */ = {isa = PBXBuildFile; fileRef = 86EF9F091834018C00AAB3F3 /* xattr_properties.h */; settings = {ATTRIBUTES = (Private, ); }; };
+ D11048A22455AE7900E8F465 /* xattr_test.c in Sources */ = {isa = PBXBuildFile; fileRef = D11048A12455AE7900E8F465 /* xattr_test.c */; };
FCCE17C3135A658F002CEE6D /* copyfile.c in Sources */ = {isa = PBXBuildFile; fileRef = FCCE17C1135A658F002CEE6D /* copyfile.c */; };
FCCE17C4135A658F002CEE6D /* copyfile.h in Headers */ = {isa = PBXBuildFile; fileRef = FCCE17C2135A658F002CEE6D /* copyfile.h */; settings = {ATTRIBUTES = (Public, ); }; };
/* End PBXBuildFile section */
@@ -35,9 +38,13 @@
/* End PBXCopyFilesBuildPhase section */
/* Begin PBXFileReference section */
+ 096213F5239827D0005847FC /* identical_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = identical_test.h; sourceTree = "<group>"; };
+ 096213F6239827D0005847FC /* identical_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = identical_test.c; sourceTree = "<group>"; };
098AF3B422692BF300F9BA42 /* stat_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = stat_test.h; sourceTree = "<group>"; };
098AF3B522692BF300F9BA42 /* stat_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = stat_test.c; sourceTree = "<group>"; };
098AF3B7226A510E00F9BA42 /* copyfile_test.entitlements */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.entitlements; path = copyfile_test.entitlements; sourceTree = "<group>"; };
+ 3EF9FA5D2418B6BA003B43E8 /* readonly_fd_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = readonly_fd_test.h; sourceTree = "<group>"; };
+ 3EF9FA5E2418B6BA003B43E8 /* readonly_fd_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = readonly_fd_test.c; sourceTree = "<group>"; };
3F1EFD4C185C4EB400D1C970 /* copyfile.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = copyfile.xcconfig; path = xcodescripts/copyfile.xcconfig; sourceTree = "<group>"; };
721D4F051EA95008000F0555 /* libcopyfile.tbd */ = {isa = PBXFileReference; lastKnownFileType = "sourcecode.text-based-dylib-definition"; name = libcopyfile.tbd; path = Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/lib/system/libcopyfile.tbd; sourceTree = DEVELOPER_DIR; };
72406E621676C3C80099568B /* xattr_flags.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = xattr_flags.c; sourceTree = "<group>"; };
@@ -53,6 +60,8 @@
72EAA3AF16A72F4500833E98 /* xattr_flags.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = xattr_flags.h; sourceTree = "<group>"; };
861E1C14180F0AF900E65B9A /* xattr_name_with_flags.3 */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = xattr_name_with_flags.3; sourceTree = "<group>"; };
86EF9F091834018C00AAB3F3 /* xattr_properties.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = xattr_properties.h; sourceTree = "<group>"; };
+ D11048A02455AE7900E8F465 /* xattr_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = xattr_test.h; sourceTree = "<group>"; };
+ D11048A12455AE7900E8F465 /* xattr_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = xattr_test.c; sourceTree = "<group>"; };
FCCE17BB135A6444002CEE6D /* libcopyfile.dylib */ = {isa = PBXFileReference; explicitFileType = "compiled.mach-o.dylib"; includeInIndex = 0; path = libcopyfile.dylib; sourceTree = BUILT_PRODUCTS_DIR; };
FCCE17C0135A658F002CEE6D /* copyfile.3 */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = copyfile.3; sourceTree = "<group>"; };
FCCE17C1135A658F002CEE6D /* copyfile.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = copyfile.c; sourceTree = "<group>"; };
@@ -89,6 +98,10 @@
isa = PBXGroup;
children = (
726EE9DA1E9423E50017A5B9 /* main.c */,
+ 096213F6239827D0005847FC /* identical_test.c */,
+ 096213F5239827D0005847FC /* identical_test.h */,
+ 3EF9FA5E2418B6BA003B43E8 /* readonly_fd_test.c */,
+ 3EF9FA5D2418B6BA003B43E8 /* readonly_fd_test.h */,
726EE9DE1E9425160017A5B9 /* sparse_test.c */,
726EE9DF1E9425160017A5B9 /* sparse_test.h */,
098AF3B522692BF300F9BA42 /* stat_test.c */,
@@ -98,6 +111,8 @@
726EE9E21E946B320017A5B9 /* systemx.c */,
726EE9E31E946B320017A5B9 /* systemx.h */,
098AF3B7226A510E00F9BA42 /* copyfile_test.entitlements */,
+ D11048A02455AE7900E8F465 /* xattr_test.h */,
+ D11048A12455AE7900E8F465 /* xattr_test.c */,
);
path = copyfile_test;
sourceTree = "<group>";
@@ -201,6 +216,7 @@
developmentRegion = English;
hasScannedForEncodings = 0;
knownRegions = (
+ English,
en,
);
mainGroup = FCCE17AB135A5FFB002CEE6D;
@@ -238,9 +254,12 @@
files = (
098AF3B622692BF300F9BA42 /* stat_test.c in Sources */,
721D4F081EA95290000F0555 /* xattr_flags.c in Sources */,
+ 3EF9FA5F2418B6BA003B43E8 /* readonly_fd_test.c in Sources */,
+ D11048A22455AE7900E8F465 /* xattr_test.c in Sources */,
721D4F071EA95283000F0555 /* copyfile.c in Sources */,
726EE9DB1E9423E50017A5B9 /* main.c in Sources */,
726EE9E61E946D590017A5B9 /* test_utils.c in Sources */,
+ 096213F7239827D0005847FC /* identical_test.c in Sources */,
726EE9E41E946B320017A5B9 /* systemx.c in Sources */,
726EE9E01E9425160017A5B9 /* sparse_test.c in Sources */,
);
@@ -366,6 +385,7 @@
"-ldispatch",
"-lxpc",
);
+ OTHER_TAPI_FLAGS = "-umbrella System";
SDKROOT = macosx.internal;
"SIM_SUFFIX[sdk=iphonesimulator*]" = _sim;
SUPPORTS_TEXT_BASED_API = YES;
diff --git a/copyfile_test/identical_test.c b/copyfile_test/identical_test.c
new file mode 100644
index 0000000..19f8dd6
--- /dev/null
+++ b/copyfile_test/identical_test.c
@@ -0,0 +1,119 @@
+//
+// identical_test.c
+// copyfile_test
+//
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <removefile.h>
+#include <sys/fcntl.h>
+#include <sys/stat.h>
+#include <sys/xattr.h>
+
+#include "../copyfile.h"
+#include "identical_test.h"
+#include "test_utils.h"
+
+#define REGULAR_FILE_NAME "regular_file"
+#define REGULAR_DIR_NAME "regular_dir"
+#define DUMMY_XATTR_NAME "dummy_xattr"
+#define DUMMY_XATTR_DATA "drell"
+#define TEST_FILE_DATA "krogan"
+
+static bool verify_src_dst_identical(const char *apfs_test_directory, __unused size_t block_size) {
+ char regular_file[BSIZE_B] = {0}, folder[BSIZE_B] = {0}, file_inside_folder[BSIZE_B] = {0};
+ int regular_file_fd, file_inside_folder_fd;
+ bool success = true;
+
+ // The idea here is to verify that copyfile(file1, file1) returns success
+ // without doing anything.
+ // There are a few wrinkles - COPYFILE_CHECK still needs to work on these files,
+ // and we need to make sure that our identity check works on filesystems without
+ // persistent object identifiers. The first we can easily verify but the second
+ // is not tested here. Nor are negative tests included (there are an infinite
+ // number of those, so we rely on the other tests to verify that behavior).
+
+ // Create path names.
+ assert_with_errno(snprintf(regular_file, BSIZE_B, "%s/" REGULAR_FILE_NAME, apfs_test_directory) > 0);
+ assert_with_errno(snprintf(folder, BSIZE_B, "%s/" REGULAR_DIR_NAME, apfs_test_directory) > 0);
+ assert_with_errno(snprintf(file_inside_folder, BSIZE_B, "%s/" REGULAR_FILE_NAME, folder) > 0);
+
+ // First, verify copyfile(file1, file1),
+ // where file1 is a regular file.
+
+ // Create our regular file.
+ regular_file_fd = open(regular_file, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
+ assert_with_errno(regular_file_fd >= 0);
+
+ // Write some data to the test file so that we can verify it is not empty.
+ assert(write(regular_file_fd, TEST_FILE_DATA, sizeof(TEST_FILE_DATA)) > 0);
+
+ // Verify copyfile(file1, file1) does nothing.
+ assert_no_err(copyfile(regular_file, regular_file, NULL, COPYFILE_ALL));
+ success = success && verify_contents_with_buf(regular_file_fd, 0, (const char *)TEST_FILE_DATA, sizeof(TEST_FILE_DATA));
+
+ // Verify copyfile(file1, file1, COPYFILE_EXCL) returns an error.
+ assert(copyfile(regular_file, regular_file, NULL, COPYFILE_ALL|COPYFILE_EXCL) == -1);
+ assert(errno == EEXIST);
+
+ // Write an dummy xattr to the file to verify COPYFILE_CHECK.
+ assert_no_err(fsetxattr(regular_file_fd, DUMMY_XATTR_NAME, DUMMY_XATTR_DATA, sizeof(DUMMY_XATTR_DATA), 0, XATTR_CREATE));
+
+ // Verify copyfile(file1, file1, ..., COPYFILE_CHECK) works.
+ assert_no_err(copyfile(regular_file, regular_file, NULL, COPYFILE_CHECK) == COPYFILE_XATTR);
+
+ // Now, verify that copyfile(dir1, dir1, COPYFILE_RECURSIVE)
+ // also returns early. Do this by making sure the contents of a file inside the directory
+ // do not change after copyfile(COPYFILE_RECURSIVE).
+
+ // Create our directory.
+ assert_no_err(mkdir(folder, DEFAULT_MKDIR_PERM));
+
+ // Create a regular file inside that directory.
+
+ file_inside_folder_fd = open(file_inside_folder, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
+ assert_with_errno(file_inside_folder_fd >= 0);
+
+ // Write some data to the interior file so that we can verify it is not empty.
+ assert(write(file_inside_folder_fd, (const char *)TEST_FILE_DATA, sizeof(TEST_FILE_DATA)) > 0);
+
+ // Verify copyfile(dir1, dir1, ... COPYFILE_RECURSIVE).
+ assert_no_err(copyfile(folder, folder, NULL, COPYFILE_RECURSIVE));
+ success = success && verify_contents_with_buf(file_inside_folder_fd, 0, TEST_FILE_DATA, sizeof(TEST_FILE_DATA));
+
+ // Post-test cleanup.
+ assert_no_err(close(file_inside_folder_fd));
+ assert_no_err(close(regular_file_fd));
+ (void)removefile(folder, NULL, REMOVEFILE_RECURSIVE);
+ (void)removefile(regular_file, NULL, 0);
+
+ return success;
+}
+
+bool do_src_dst_identical_test(const char *apfs_test_directory, __unused size_t block_size) {
+ char test_dir[BSIZE_B] = {0};
+ int test_folder_id;
+ bool success = true;
+
+ printf("START [identical]\n");
+
+ // Get ready for the test.
+ test_folder_id = rand() % DEFAULT_NAME_MOD;
+ create_test_file_name(apfs_test_directory, "identical", test_folder_id, test_dir);
+ assert_no_err(mkdir(test_dir, DEFAULT_MKDIR_PERM));
+
+ success = verify_src_dst_identical(test_dir, block_size);
+
+ if (success) {
+ printf("PASS [identical]\n");
+ } else {
+ printf("FAIL [identical]\n");
+ }
+
+ (void)removefile(test_dir, NULL, REMOVEFILE_RECURSIVE);
+
+ return success ? EXIT_SUCCESS : EXIT_FAILURE;
+}
\ No newline at end of file
diff --git a/copyfile_test/identical_test.h b/copyfile_test/identical_test.h
new file mode 100644
index 0000000..e4085bd
--- /dev/null
+++ b/copyfile_test/identical_test.h
@@ -0,0 +1,14 @@
+//
+// identical_test.h
+// copyfile_test
+//
+
+#ifndef identical_test_h
+#define identical_test_h
+
+#include <stdbool.h>
+#include <stdlib.h>
+
+bool do_src_dst_identical_test(const char *apfs_test_directory, size_t block_size);
+
+#endif /* identical_test_h */
diff --git a/copyfile_test/main.c b/copyfile_test/main.c
index a5384e9..af5ec06 100644
--- a/copyfile_test/main.c
+++ b/copyfile_test/main.c
@@ -11,8 +11,11 @@
#include <sys/stat.h>
#include <removefile.h>
+#include "identical_test.h"
+#include "readonly_fd_test.h"
#include "sparse_test.h"
#include "stat_test.h"
+#include "xattr_test.h"
#include "test_utils.h"
#define DISK_IMAGE_SIZE_MB 512
@@ -51,11 +54,14 @@ int main(__unused int argc, __unused const char * argv[]) {
// Run our tests.
sranddev();
+ failed |= do_readonly_fd_test(TEST_DIR, stb.f_bsize);
failed |= do_sparse_test(TEST_DIR, stb.f_bsize);
failed |= do_sparse_recursive_test(TEST_DIR, stb.f_bsize);
failed |= do_fcopyfile_offset_test(TEST_DIR, stb.f_bsize);
failed |= do_preserve_dst_flags_test(TEST_DIR, stb.f_bsize);
failed |= do_preserve_dst_tracked_test(TEST_DIR, stb.f_bsize);
+ failed |= do_src_dst_identical_test(TEST_DIR, stb.f_bsize);
+ failed |= do_xattr_test(TEST_DIR, stb.f_bsize);
// Cleanup the disk image we ran our tests on.
if (USING_DISK_IMAGE) {
diff --git a/copyfile_test/readonly_fd_test.c b/copyfile_test/readonly_fd_test.c
new file mode 100644
index 0000000..ee5a0b3
--- /dev/null
+++ b/copyfile_test/readonly_fd_test.c
@@ -0,0 +1,124 @@
+//
+// Copyright (c) 2020 Apple Inc. All rights reserved.
+//
+
+#include <err.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <paths.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/acl.h>
+#include <sys/attr.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/xattr.h>
+#include <unistd.h>
+#include "readonly_fd_test.h"
+#include "test_utils.h"
+
+
+static
+bool test_readonly_fd_metadata(const char *basedir)
+{
+ char filename[] = ".readonly-ops-XXXXXX";
+ bool created = false;
+ bool success = true;
+ int dirfd = -1;
+ int tmpfd = -1;
+ int fd = -1;
+ acl_t acl = NULL;
+
+ static const char test_name[] = "readonly_fd_metadata";
+ printf("START [%s]\n", test_name);
+
+ assert_with_errno((dirfd = open(basedir, O_RDONLY | O_DIRECTORY)) != -1);
+ assert_with_errno((tmpfd = mkstempsat_np(dirfd, filename, 0)) != -1);
+ created = true;
+
+ assert_with_errno((fd = openat(dirfd, filename, O_RDONLY)) != -1);
+ close(tmpfd);
+ tmpfd = -1;
+
+ // confirm that writes are disallowed
+ const char data[] = "failure";
+ assert(write(fd, data, sizeof(data) - 1) == -1);
+
+ // check fchown()
+ const uid_t uid = geteuid();
+ const gid_t gid = getegid();
+ assert_no_err(fchown(fd, uid, gid));
+
+ // check fchmod()
+ assert_no_err(fchmod(fd, 0644));
+ assert_no_err(fchmod(fd, 0600));
+
+ // check fchflags()
+ assert_no_err(fchflags(fd, UF_HIDDEN));
+
+ // check setting timestamps with fsetattrlist
+ const time_t mtime = 978307200;
+ const time_t atime = mtime + 1;
+
+ struct timeval matimes_usec[] = {{mtime, 0}, {atime, 0}};
+ assert_no_err(futimes(fd, matimes_usec));
+
+ struct attrlist attrlist = {
+ .bitmapcount = ATTR_BIT_MAP_COUNT,
+ .commonattr = ATTR_CMN_MODTIME | ATTR_CMN_ACCTIME,
+ };
+ struct {
+ struct timespec mtime;
+ struct timespec atime;
+ } matimes_nsec = {{mtime, 0}, {atime, 0}};
+ assert_no_err(fsetattrlist(fd, &attrlist, &matimes_nsec, sizeof(matimes_nsec), 0));
+
+ // check adding and removing xattrs
+ static const char key[] = "local.test-xattr";
+ static const char value[] = "local.test-xattr.value";
+ assert_no_err(fsetxattr(fd, key, value, sizeof(value)-1, 0, 0));
+ assert_no_err(fremovexattr(fd, key, 0));
+
+ // check setting ACLs
+ assert_with_errno((acl = acl_init(1)) != NULL);
+ assert_no_err(acl_set_fd(fd, acl));
+
+ // log pass/fail before cleanup
+ if (success) {
+ printf("PASS [%s]\n", test_name);
+ } else {
+ printf("FAIL [%s]\n", test_name);
+ }
+
+ // clean up resources
+ if (acl) {
+ acl_free(acl);
+ }
+ if (fd != -1) {
+ close(fd);
+ }
+ if (tmpfd != -1) {
+ close(tmpfd);
+ }
+ if (created) {
+ unlinkat(dirfd, filename, 0);
+ }
+ if (dirfd != -1) {
+ close(dirfd);
+ }
+
+ return success;
+}
+
+
+bool do_readonly_fd_test(const char *apfs_test_directory, size_t block_size __unused)
+{
+ // These tests verify the underlying calls needed for COPYFILE_METADATA
+ // operations are safe with O_RDONLY file descriptors. If this fails,
+ // expect <rdar://60074298> to cause many other copyfile() failures.
+ bool success = true;
+ success = success && test_readonly_fd_metadata(apfs_test_directory);
+ return !success; // caller expects nonzero to mean failure
+}
diff --git a/copyfile_test/readonly_fd_test.h b/copyfile_test/readonly_fd_test.h
new file mode 100644
index 0000000..339230f
--- /dev/null
+++ b/copyfile_test/readonly_fd_test.h
@@ -0,0 +1,13 @@
+//
+// Copyright (c) 2020 Apple Inc. All rights reserved.
+//
+
+#ifndef readonly_fd_test_h
+#define readonly_fd_test_h
+
+#include <stdbool.h>
+#include <stddef.h>
+
+bool do_readonly_fd_test(const char *apfs_test_directory, size_t block_size);
+
+#endif /* readonly_fd_test_h */
diff --git a/copyfile_test/sparse_test.c b/copyfile_test/sparse_test.c
index fc126d7..0a75df5 100644
--- a/copyfile_test/sparse_test.c
+++ b/copyfile_test/sparse_test.c
@@ -182,12 +182,16 @@ static off_t write_diff_adj_holes(int fd, off_t block_size) {
return 0;
}
+static off_t write_nothing(__unused int fd, __unused off_t block_size) {
+ return 0;
+}
+
typedef struct {
creator_func func; // pointer to function to create a sparse file
const char * name; // null terminated string
} sparse_test_func;
-#define NUM_TEST_FUNCTIONS 10
+#define NUM_TEST_FUNCTIONS 11
sparse_test_func test_functions[NUM_TEST_FUNCTIONS] = {
{write_start_and_end_holes, "start_and_end_holes"},
{write_middle_hole, "middle_hole"},
@@ -198,7 +202,8 @@ sparse_test_func test_functions[NUM_TEST_FUNCTIONS] = {
{write_no_sparse, "no_sparse"},
{write_sparse_odd_offset, "write_sparse_odd_offset"},
{write_sparse_bs_offset, "write_sparse_bs_offset"},
- {write_diff_adj_holes, "write_diff_adj_holes"}
+ {write_diff_adj_holes, "write_diff_adj_holes"},
+ {write_nothing, "write_nothing"},
};
bool do_sparse_test(const char* apfs_test_directory, size_t block_size) {
diff --git a/copyfile_test/stat_test.c b/copyfile_test/stat_test.c
index 0ab2944..784b046 100644
--- a/copyfile_test/stat_test.c
+++ b/copyfile_test/stat_test.c
@@ -174,8 +174,8 @@ bool do_preserve_dst_tracked_test(const char *test_directory, __unused size_t bl
assert_no_err(copyfile(file_src, file_dst, NULL, COPYFILE_DATA|COPYFILE_STAT|COPYFILE_PRESERVE_DST_TRACKED));
assert_no_err(stat(file_dst, &dst_stb));
- success &= (dst_stb.st_size == src_fsize);
- success &= (dst_stb.st_flags & UF_TRACKED);
+ success = success && (dst_stb.st_size == src_fsize);
+ success = success && (dst_stb.st_flags & UF_TRACKED);
if (success) {
printf("PASS [preserve_dst_tracked]\n");
} else {
diff --git a/copyfile_test/test_utils.c b/copyfile_test/test_utils.c
index 95741b1..0da0b87 100644
--- a/copyfile_test/test_utils.c
+++ b/copyfile_test/test_utils.c
@@ -11,10 +11,78 @@
#include <unistd.h>
#include <sys/fcntl.h>
#include <sys/stat.h>
+#include <sys/xattr.h>
#include "test_utils.h"
#include "systemx.h"
+static bool verify_xattr_content(int fd, const char *xattr_name, const char *expected, ssize_t size) {
+ // Verify that the file referenced by `fd` has an xattr named `xattr_name`
+ // of size `size` and with contents equal to `expected`.
+ char *actual = NULL;
+ bool equal;
+
+ assert(fd > 0 && xattr_name && expected);
+ assert_with_errno(actual = malloc(size));
+ assert_with_errno(fgetxattr(fd, xattr_name, actual, size, 0, 0) == size);
+
+ equal = (memcmp(actual, expected, size) == 0);
+ if (!equal) {
+ printf("xattr %s: content does not match expected\n", xattr_name);
+ }
+
+ free(actual);
+ return equal;
+}
+
+bool verify_fd_xattr_contents(int orig_fd, int copy_fd) {
+ // Verify that both fd's have the same xattrs.
+ // We do so by first verifying that `flistxattr()` returns the same size
+ // for both, and then validating that `copy_fd` has each of the xattrs
+ // that `orig_fd` has.
+ char *namebuf = NULL, *xa_buf = NULL, *name, *end;
+ ssize_t orig_size, copy_size, xa_size;
+ bool equal = true;
+
+ assert((orig_fd > 0) && (copy_fd > 0));
+
+ orig_size = flistxattr(orig_fd, 0, 0, XATTR_SHOWCOMPRESSION);
+ copy_size = flistxattr(copy_fd, 0, 0, XATTR_SHOWCOMPRESSION);
+ if (orig_size != copy_size) {
+ printf("xattrlist size: orig_size(%zu) != (%zu)copy_size\n", orig_size, copy_size);
+ return false;
+ }
+
+ if (orig_size == 0) {
+ return true;
+ }
+
+ assert_with_errno(namebuf = malloc(orig_size));
+
+ assert_with_errno(flistxattr(orig_fd, namebuf, orig_size, 0) == orig_size);
+
+ end = namebuf + orig_size - 1;
+ if (*end != 0) {
+ *end = 0;
+ }
+
+ for (name = namebuf; name <= end; name += strlen(name) + 1) {
+ xa_size = fgetxattr(orig_fd, name, 0, 0, 0, 0);
+ assert(xa_size >= 0);
+ assert_with_errno(xa_buf = malloc(xa_size));
+ assert_with_errno(fgetxattr(orig_fd, name, xa_buf, xa_size, 0, 0) == xa_size);
+ equal = equal && verify_xattr_content(copy_fd, name, xa_buf, xa_size);
+ free(xa_buf);
+
+ if (!equal) {
+ break;
+ }
+ }
+ free(namebuf);
+
+ return equal;
+}
+
bool verify_st_flags(struct stat *sb, uint32_t flags_to_expect) {
// Verify that sb's flags include flags_to_expect.
if (((sb->st_flags & flags_to_expect)) != flags_to_expect) {
@@ -26,6 +94,37 @@ bool verify_st_flags(struct stat *sb, uint32_t flags_to_expect) {
return true;
}
+bool verify_contents_with_buf(int orig_fd, off_t orig_pos, const char *expected, size_t length)
+{
+ // Read *length* bytes from a file descriptor at a specified position
+ // and assert that they match *length* bytes from expected.
+ char orig_contents[length];
+ bool equal;
+
+ assert(orig_fd > 0 && orig_pos >= 0);
+ memset(orig_contents, 0, length);
+
+ errno = 0;
+ ssize_t pread_res = pread(orig_fd, orig_contents, length, orig_pos);
+ assert_with_errno(pread_res == (off_t) length);
+ equal = (memcmp(orig_contents, expected, length) == 0);
+ if (!equal) {
+ printf("fd (%lld - %lld) did not match expected contents\n", orig_pos, orig_pos + length);
+
+ // Find the first non-matching byte and print it out.
+ for (size_t bad_off = 0; bad_off < length; bad_off++) {
+ if (orig_contents[bad_off] != expected[bad_off]) {
+ printf("first mismatch is at offset %zu, original 0x%llx expected 0x%llx\n",
+ bad_off, (unsigned long long)orig_contents[bad_off],
+ (unsigned long long)expected[bad_off]);
+ break;
+ }
+ }
+ }
+
+ return equal;
+}
+
bool verify_fd_contents(int orig_fd, off_t orig_pos, int copy_fd, off_t copy_pos, size_t length) {
// Read *length* contents of the two fds and make sure they compare as equal.
// Don't alter the position of either fd.
@@ -33,6 +132,7 @@ bool verify_fd_contents(int orig_fd, off_t orig_pos, int copy_fd, off_t copy_pos
bool equal;
assert(orig_fd > 0 && copy_fd > 0);
+ assert(orig_pos >= 0);
memset(orig_contents, 0, length);
memset(copy_contents, 0, length);
@@ -119,7 +219,7 @@ int create_hole_in_fd(int fd, off_t offset, off_t length) {
void create_test_file_name(const char *dir, const char *postfix, int id, char *string_out) {
- // Make a name for this new file and put it in out_name, which should be BSIZE_B bytes.
+ // Make a name for this new file and put it in string_out, which should be BSIZE_B bytes.
assert_with_errno(snprintf(string_out, BSIZE_B, "%s/testfile-%d.%s", dir, id, postfix) > 0);
}
diff --git a/copyfile_test/test_utils.h b/copyfile_test/test_utils.h
index 4cd7faa..decbe1e 100644
--- a/copyfile_test/test_utils.h
+++ b/copyfile_test/test_utils.h
@@ -35,7 +35,9 @@
#define DIFF_PATH "/usr/bin/diff"
// Test routine helpers.
+bool verify_fd_xattr_contents(int orig_fd, int copy_fd);
bool verify_st_flags(struct stat *sb, uint32_t flags_to_expect);
+bool verify_contents_with_buf(int orig_fd, off_t orig_pos, const char *expected, size_t length);
bool verify_fd_contents(int orig_fd, off_t orig_pos, int copy_fd, off_t copy_pos, size_t length);
bool verify_copy_contents(const char *orig_name, const char *copy_name);
bool verify_copy_sizes(struct stat *orig_sb, struct stat *copy_sb, copyfile_state_t cpf_state,
diff --git a/copyfile_test/xattr_test.c b/copyfile_test/xattr_test.c
new file mode 100644
index 0000000..472a4a7
--- /dev/null
+++ b/copyfile_test/xattr_test.c
@@ -0,0 +1,84 @@
+//
+// xattr_test.c
+// copyfile_test
+//
+
+#include <unistd.h>
+#include <removefile.h>
+#include <sys/fcntl.h>
+#include <sys/stat.h>
+#include <sys/xattr.h>
+
+#include "xattr_test.h"
+#include "test_utils.h"
+
+#define SRC_FILE_NAME "src_file"
+#define DST_FILE_NAME "dst_file"
+#define SMALL_XATTR_NAME "small_xattr"
+#define SMALL_XATTR_DATA "drell"
+#define BIG_XATTR_NAME "big_xattr"
+#define BIG_XATTR_SIZE (20 * 1024 * 1024) // 20MiB
+
+#define DEFAULT_CHAR_MOD 256
+
+static bool copy_and_verify_xattr_contents(const char *src_file, const char *dst_file, int src_file_fd, int dst_file_fd) {
+ assert_no_err(copyfile(src_file, dst_file, NULL, COPYFILE_XATTR));
+
+ return verify_fd_xattr_contents(src_file_fd, dst_file_fd);
+}
+
+bool do_xattr_test(const char *apfs_test_directory, __unused size_t block_size) {
+ char test_dir[BSIZE_B] = {0};
+ char src_file[BSIZE_B] = {0}, dst_file[BSIZE_B] = {0};
+ char *big_xattr_data = NULL, buf[4096] = {0};
+ int test_folder_id;
+ int src_file_fd, dst_file_fd;
+ bool success = true;
+
+ printf("START [xattr]\n");
+
+ // Get ready for the test.
+ test_folder_id = rand() % DEFAULT_NAME_MOD;
+ create_test_file_name(apfs_test_directory, "xattr", test_folder_id, test_dir);
+ assert_no_err(mkdir(test_dir, DEFAULT_MKDIR_PERM));
+
+ // Create path names.
+ assert_with_errno(snprintf(src_file, BSIZE_B, "%s/" SRC_FILE_NAME, test_dir) > 0);
+ assert_with_errno(snprintf(dst_file, BSIZE_B, "%s/" DST_FILE_NAME, test_dir) > 0);
+
+ // Create our files.
+ src_file_fd = open(src_file, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
+ assert_with_errno(src_file_fd >= 0);
+ dst_file_fd = open(dst_file, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
+ assert_with_errno(dst_file_fd >= 0);
+
+ // Sanity check - empty copy
+ success = success && copy_and_verify_xattr_contents(src_file, dst_file, src_file_fd, dst_file_fd);
+
+ // Write a small xattr to the source file.
+ assert_no_err(fsetxattr(src_file_fd, SMALL_XATTR_NAME, SMALL_XATTR_DATA, sizeof(SMALL_XATTR_DATA), 0, XATTR_CREATE));
+ success = success && copy_and_verify_xattr_contents(src_file, dst_file, src_file_fd, dst_file_fd);
+
+ // Create big xattr data
+ assert_with_errno(big_xattr_data = malloc(BIG_XATTR_SIZE));
+ for (int i = 0; i * sizeof(buf) < BIG_XATTR_SIZE; i++) {
+ memset(buf, rand() % DEFAULT_CHAR_MOD, sizeof(buf));
+ memcpy(big_xattr_data + (i * sizeof(buf)), buf, sizeof(buf));
+ }
+
+ // Write a big xattr to the source file.
+ assert_no_err(fsetxattr(src_file_fd, BIG_XATTR_NAME, big_xattr_data, BIG_XATTR_SIZE, 0, XATTR_CREATE));
+ success = success && copy_and_verify_xattr_contents(src_file, dst_file, src_file_fd, dst_file_fd);
+
+ if (success) {
+ printf("PASS [xattr]\n");
+ } else {
+ printf("FAIL [xattr]\n");
+ }
+
+ free(big_xattr_data);
+ (void)removefile(test_dir, NULL, REMOVEFILE_RECURSIVE);
+
+ return success ? EXIT_SUCCESS : EXIT_FAILURE;
+}
+
diff --git a/copyfile_test/xattr_test.h b/copyfile_test/xattr_test.h
new file mode 100644
index 0000000..2749e97
--- /dev/null
+++ b/copyfile_test/xattr_test.h
@@ -0,0 +1,14 @@
+//
+// xattr_test.h
+// copyfile_test
+//
+
+#ifndef xattr_test_h
+#define xattr_test_h
+
+#include <stdbool.h>
+
+bool do_xattr_test(const char *apfs_test_directory, size_t block_size);
+
+#endif /* xattr_test_h */
+
diff --git a/xattr_name_with_flags.3 b/xattr_name_with_flags.3
index 35afc2a..cd8a3b5 100644
--- a/xattr_name_with_flags.3
+++ b/xattr_name_with_flags.3
@@ -7,6 +7,7 @@
.Sh NAME
.Nm xattr_preserve_for_intent , xattr_name_with_flags , xattr_name_without_flags ,
.Nm xattr_flags_from_name , xattr_intent_with_flags
+.Nd obtain properties related to extended attributes, for use in copying
.Sh LIBRARY
.Lb libc
.Sh SYNOPSIS
@@ -23,7 +24,8 @@
.Fn xattr_intent_with_flags "xattr_operation_intent_t" "xattr_flags_t"
.Sh DESCRIPTION
These functions are used in conjunction with copying extended attributes from
-one file to another. Various types of copying (an "intent") check flags to
+one file to another.
+Various types of copying (an "intent") check flags to
determine which is allowed or not.
.Pp
The
@@ -32,7 +34,8 @@ function returns an extended attribute name with the appropriate flags encoded
as a string; the
.Fn xattr_name_without_flags
undoes this, giving the name of the extended attribute without the flags
-encoding. The slight inverse of that is
+encoding.
+The slight inverse of that is
.Fn xattr_flags_from_name ,
which will return the flags encoded in a name.
.Pp
@@ -62,30 +65,30 @@ named extended attribute should be preserved during a copy for
the given intent.
.Sh INTENT
The type
-.Dt xattr_operation_intent_t
-is an integral type, which is used to indicate what the intent for the operation
-is. The following intent values are defined:
+.Vt xattr_operation_intent_t
+is an integral type, which is used to indicate what the intent for the operation is.
+The following intent values are defined:
.Bl -tag -width XATTR_OPERATION_INTENT_SHARE
.It Dv XATTR_OPERATION_INTENT_COPY
Indicates that the intent is to simply copy from the source to the destination.
-E.g., with cp. Most extended attributes should generally be preserved in this
-case.
+E.g., with cp.
+Most extended attributes should generally be preserved in this case.
.It Dv XATTR_OPERATION_INTENT_SAVE
Indicates that intent is to perform a save (perhaps as in a "safe save").
This differs from a copy in that the content may be changing; the destination
may be over-writing or replacing the source, and some extended attributes should
not be preserved during this process.
.It Dv XATTR_OPERATION_INTENT_SHARE
-Indicates that the intent is to share, or export, the object. For example,
-saving as an attachment in an email message, or placing in a public folder.
+Indicates that the intent is to share, or export, the object.
+For example, saving as an attachment in an email message, or placing in a public folder.
Sensitive information should probably not be preserved in this case.
.It Dv XATTR_OPERATION_INTENT_SYNC
Indicates that the intent is to sync the object to a service like iCloud Drive.
.El
.Sh FLAGS
Various flags are defined by the type
-.Dt xattr_flags_t ;
-the currently-defined values for this are
+.Vt xattr_flags_t ;
+the currently-defined values for this are:
.Bl -tag -width XATTR_FLAG_CONTENT_DEPENDENT
.It Dv XATTR_FLAG_NO_EXPORT
This indicates that the extended attribute should not be exported, or shared.
@@ -94,15 +97,16 @@ This is used with
.It Dv XATTR_FLAG_CONTENT_DEPENDENT
This indicates that the extended attribute is tied to the contents of the
file (or vice versa), such that it should be re-created when the contents
-are changed. A checksum, for example, should not be copied, and would thus
-be marked with this flag.
+are changed.
+A checksum, for example, should not be copied, and would thus be marked with this flag.
.It Dv XATTR_FLAG_NEVER_PRESERVE
This indicates that the extended attribute should never be copied from a
source object to a destination, no matter what the given intent is.
.It Dv XATTR_FLAG_SYNCABLE
This indicates that the extended attribute should be copied when the file
-is synced on services like iCloud Drive. Sync services may enforce additional
-restrictions on the acceptable size and number of extended attributes.
+is synced on services like iCloud Drive.
+Sync services may enforce additional restrictions on the acceptable size and number
+of extended attributes.
.El
.Sh EXAMPLE
The following example is a simple function that, given an extended attribute