shutil.move vs. os.replace

I came across an interesting issue while working on a reproducer. pip ran out of disk space while writing a cached wheel¹. The operation stopped, but the incomplete file remained and could later be used as a cache entry. pip#9964

pip uses shutil.move to populate the wheel cache. The same situation can be reproduced as follows.

# (Disk limited to 1 MB, on a different filesystem)
>>> import shutil, os
>>> os.path.getsize("distribution.whl")
10000000

# Move a 10 MB distribution.whl file to /mnt/small, a 1 MB cache filesystem.
>>> shutil.move("distribution.whl", "/mnt/small/distribution.whl")
Traceback (most recent call last):
  ...
  File "/usr/local/lib/python3.12/shutil.py", line 867, in move
    copy_function(src, real_dst)
  File "/usr/local/lib/python3.12/shutil.py", line 475, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  ...
OSError: [Errno 28] No space left on device: 'distribution.whl' -> '/mnt/small/distribution.whl'

>>> os.path.getsize("/mnt/small/distribution.whl")
1048576

The operation stopped, but a 1 MB file remained. Why was it not cleaned up? (Source: CPython - shutil.py)

shutil.move(src, dst)
└─ os.rename(src, dst)      # try
   ├─ success                # same filesystem, atomic
   └─ OSError               # different filesystem
      ├─ copy2(src, dst)    # interruption during I/O leaves an incomplete dst
      └─ os.unlink(src)

shutil.move calls os.rename first and falls back to copy2 if that fails. If copy2 stops midway because the process is interrupted or the disk runs out of space, it leaves an incomplete file behind. Another process reading the file at the same time may treat it as a cache entry before the copy finishes.

How can we avoid leaving incomplete files behind? First, look at how os.replace behaves.

>>> os.replace("distribution.whl", "/mnt/small/distribution.whl")
Traceback (most recent call last):
  ...
OSError: [Errno 18] Invalid cross-device link: 'distribution.whl' -> '/mnt/small/distribution.whl'
>>> os.listdir("/mnt/small")
[]

The os.replace call failed, but /mnt/small remained empty. Within the same filesystem, os.replace performs an atomic replacement². Across filesystems, it raises an exception instead of falling back to a copy.

shutil.move continues by copying across filesystems, while os.replace raises an error if it cannot perform an atomic replacement. Use shutil.move when the file has to move across filesystems, and os.replace when the final path must be swapped atomically within the same filesystem.

	`shutil.move`	`os.replace`
Same filesystem	Atomic move with `os.rename`	Atomic replacement
Different filesystems	Copy, then delete the source	Fail with `EXDEV`
Destination after failure	May contain an incomplete file if the copy fails	Previous state is preserved

To avoid leaving an incomplete file when the filesystems differ, first copy the file under a temporary name at the destination. Once the copy finishes, use os.replace to give it the final name and then delete the source file.

from tempfile import TemporaryDirectory

with TemporaryDirectory(dir="/mnt/small") as temp_dir:
    tmp = os.path.join(temp_dir, "distribution.whl")
    shutil.copy2("distribution.whl", tmp)
    os.replace(tmp, "/mnt/small/distribution.whl")

A wheel is a ZIP-based Python package distribution with a .whl extension. A truncated copy cannot be read as a valid wheel. Wheel specification ↩
With an atomic os.replace, another process sees either the old file or the new file, never a partially replaced file. os.replace documentation ↩

Footnotes