File Helpers

Files (sillo.helpers.files)

Accepting a file upload means accepting a name, a size, and a byte stream from someone you do not trust. Each of the three is an attack surface. The name can escape your upload directory. The declared size can lie. The extension can claim to be a JPEG while the bytes are a shell script.

This module covers the mechanical parts of handling that: formatting sizes for humans, parsing size limits from configuration, classifying extensions, and — most importantly — turning a hostile filename into one that is safe to write to disk.

from sillo.helpers import files

Stdlib-only, built on pathlib and mimetypes.

Formatting sizes

format_size(bytes_value) and format_size_binary(bytes_value)

Turn a byte count into something a person can read:

files.format_size(0)          # '0 B'
files.format_size(999)        # '999 B'
files.format_size(1024)       # '1.0 KB'
files.format_size(1536000)    # '1.5 MB'

files.format_size_binary(1024)      # '1.0 KiB'
files.format_size_binary(1536000)   # '1.5 MiB'

Both functions divide by 1024; only the label differs

The names suggest decimal (SI) versus binary units. In the current implementation both compute the same number, dividing by 1024 at each step, and differ only in the suffix string:

files.format_size(1_000_000_000)         # '953.7 MB'
files.format_size_binary(1_000_000_000)  # '953.7 MiB'

A billion bytes is 1.0 GB under the SI definition that “MB” implies. format_size reports 953.7 MB, which is the binary magnitude wearing a decimal label.

In practice this matters when your number must line up with someone else’s. Disk manufacturers, cloud storage billing, and network transfer quotas are all decimal. If you display “953.7 MB” next to a provider’s “1.0 GB” for the same object, users will file a bug.

format_size_binary is correct and unambiguous — use it. If you need true decimal output, compute it yourself:

def format_size_si(n: int) -> str:
    for unit in ("B", "KB", "MB", "GB", "TB"):
        if abs(n) < 1000:
            return f"{n:.1f} {unit}" if unit != "B" else f"{n:.0f} {unit}"
        n /= 1000
    return f"{n:.1f} PB"

Negative values format with a sign (format_size(-1024) gives '-1.0 KB'), which is occasionally useful for showing a delta.

parse_size(size_str) -> int

The inverse, for reading limits out of configuration files and environment variables:

files.parse_size("10 MB")    # 10485760
files.parse_size("10MB")     # 10485760  — space optional
files.parse_size("1.5 GB")   # 1610612736
files.parse_size("512")      # 512       — bare number means bytes

Malformed input raises ValueError:

files.parse_size("banana")   # ValueError: Invalid size string: banana

This is what makes size limits readable in config rather than magic numbers:

Readable upload limits

import os
from sillo.helpers.files import parse_size, format_size

MAX_UPLOAD = parse_size(os.environ.get("MAX_UPLOAD_SIZE", "25 MB"))

@app.post("/upload")
async def upload(request, response):
    declared = int(request.headers.get("content-length", 0))
    if declared > MAX_UPLOAD:
        return response.json(
            {"error": f"File too large. Maximum is {format_size(MAX_UPLOAD)}."},
            status_code=413,
        )
    ...

Parse at import time, not per request, so a malformed value crashes at boot rather than on the first upload.

Content-Length is a claim, not a measurement

The header above is supplied by the client and can say anything. A malicious client sends Content-Length: 100 and then streams ten gigabytes.

The header check is a cheap early rejection that saves you reading a body you were going to refuse anyway. It is not the limit. Enforce the real one while reading, and stop when you exceed it:

total = 0
async for chunk in request.stream():
    total += len(chunk)
    if total > MAX_UPLOAD:
        return response.json({"error": "File too large"}, status_code=413)
    await destination.write(chunk)

See File Uploads for the full streaming pattern.

Extensions and types

files.get_extension("photo.jpg")             # '.jpg'
files.get_extension("photo.JPG")             # '.jpg'  — lowercased
files.get_extension("noext")                 # ''
files.get_extension_clean("archive.tar.gz")  # 'gz'    — no dot, last part only

files.guess_mime_type("photo.png")           # 'image/png'
files.guess_mime_type("weird.xyzzy")         # None

get_extension keeps the dot and lowercases. get_extension_clean drops the dot. Note that for a double extension such as .tar.gz, both return only the final component, so a .tar.gz looks like a gzip file rather than a tar archive.

Three classifiers:

files.is_dangerous_extension("script.exe")   # True
files.is_dangerous_extension("notes.txt")    # False
files.is_image_extension("photo.jpg")        # True
files.is_media_extension("song.mp3")         # True

An extension is a claim by the uploader, not a fact about the bytes

is_dangerous_extension is a blocklist, and blocklists fail in one direction: everything not on the list is permitted. New dangerous types appear (.jse, .wsf, .hta, .lnk, .iso, .svg with embedded script), platform-specific ones are easy to forget, and a plain .html upload served from your domain is a stored XSS.

Renaming payload.exe to payload.jpg defeats it entirely. is_image_extension returns True and guess_mime_type says image/jpeg, because both functions only look at the string after the last dot.

For untrusted uploads, do all of the following:

1. Allowlist, never blocklist. Name the handful of extensions you accept and reject everything else.
2. Check the actual bytes. Read the magic number and confirm it matches the claimed type. python-magic or filetype does this.
3. Rename on write. Store under an ID you generate. Keep the original name in the database as a display label only.
4. Serve from a separate origin, or with Content-Disposition: attachment and X-Content-Type-Options: nosniff. A file served from your main domain runs in your origin.
5. Never execute an uploaded file, and make sure the upload directory is outside any path your web server will interpret.

ALLOWED = {".jpg", ".jpeg", ".png", ".webp", ".pdf"}

ext = files.get_extension(upload.filename)
if ext not in ALLOWED:
    return response.json({"error": f"Type not allowed: {ext or 'none'}"}, status_code=400)

guess_mime_type maps an extension to a type using the stdlib mimetypes table, and returns None for anything unrecognised. It never inspects content. Treat its output as a hint for setting a response header on files you already trust, never as verification of an upload.

Safe filenames

safe_filename(filename, replacement="_") -> str

The most security-relevant function in the module. It strips path separators and characters that are illegal or dangerous in a filename:

files.safe_filename("user input: file?.txt")   # 'user_input__file_.txt'
files.safe_filename("../../etc/passwd")        # '.._.._etc_passwd'
files.safe_filename("")                        # 'file'

The path-traversal case is the one that matters. Without it, a filename of ../../etc/passwd joined to your upload directory writes outside it:

# Catastrophic
path = os.path.join("/var/uploads", request_filename)
# "/var/uploads/../../etc/passwd"  ->  "/etc/passwd"

Separators become the replacement character, so the result stays inside the directory you join it to. An empty or fully stripped name falls back to 'file' rather than returning an empty string that would resolve to the directory itself.

The .. characters survive; only the separators are replaced

safe_filename("../../etc/passwd") returns '.._.._etc_passwd', not 'etc_passwd'. The dots are still there — what makes the result safe is that no / or \ remains, so the whole thing is a single filename component.

That is sufficient for traversal, but it means the result can still be a name you did not expect, including a leading dot (a hidden file on Unix). If that matters, strip leading dots yourself.

Sanitizing the name is not enough on its own

safe_filename handles the name. It does not handle:

• Length. Most filesystems cap a component at 255 bytes. A longer name raises OSError on write. Truncate, and preserve the extension when you do.
• Reserved Windows names. CON, PRN, AUX, NUL, COM1–COM9, LPT1–LPT9 are reserved regardless of extension. Writing CON.txt on Windows fails or behaves strangely.
• Case-insensitive collisions. Report.PDF and report.pdf are different names on Linux and the same file on macOS and Windows.
• Unicode normalization. Two visually identical names can differ in bytes, and macOS normalizes on write, so a name you stored may not be the name you get back.

The reliable answer is not to use the user’s name as the stored name at all:

Store under a generated ID, display the original

from sillo.helpers.strings import random_token
from sillo.helpers.files import get_extension, safe_filename

ext = get_extension(upload.filename)
stored_name = f"{random_token(16)}{ext}"          # what goes on disk
display_name = safe_filename(upload.filename)     # what the user sees

await upload.save(f"/var/uploads/{stored_name}")
await Attachment.create(stored_name=stored_name, display_name=display_name)

This sidesteps traversal, collisions, length limits, reserved names, and normalization in one move.

unique_filename(directory, filename) -> str

Returns a name that does not already exist in the directory, appending a counter if needed:

files.unique_filename("/tmp", "log.txt")   # 'log (1).txt' when log.txt exists
files.unique_filename("/tmp", "new.txt")   # 'new.txt'     when it does not

The suffix goes before the extension, so the file type is preserved.

Check-then-write is a race

The function checks for existence and returns a name. Between that check and your write, another request can create the same file. Under concurrent uploads you will occasionally overwrite.

For a single-process development server this is fine. For anything concurrent, use an atomic create (open(path, "x"), which fails if the file exists) and retry, or generate a random name and skip the check entirely.

Directories and ages

ensure_directory(path) -> Path

Creates a directory and any missing parents, returning the Path. Idempotent:

uploads = files.ensure_directory("/data/uploads/2026/01")

Useful at startup and for date-partitioned upload trees. It does not raise if the directory already exists.

list_files(directory, pattern="*", recursive=False) -> list[Path]

Globs a directory:

files.list_files("/data")                             # everything, top level
files.list_files("/data", "*.csv")                    # CSVs, top level
files.list_files("/data", "*.csv", recursive=True)    # CSVs, all subdirectories

Returns Path objects, not strings.

list_files is synchronous and eager

It builds the whole list in memory before returning and performs blocking disk I/O. On a directory with a hundred thousand files, both the memory and the stall are real, and inside an async handler the stall blocks every other request.

Thread it, or better, keep file metadata in the database and query that instead of walking the filesystem per request.

file_age(path) and file_age_human(path)

Seconds since last modification, and a human-readable version:

files.file_age("/var/log/app.log")         # 10842.3
files.file_age_human("/var/log/app.log")   # '3h ago'

The obvious use is cleanup:

Purging stale temporary uploads

from sillo.helpers.files import list_files, file_age

TWENTY_FOUR_HOURS = 24 * 60 * 60

async def purge_stale_uploads():
    for path in list_files("/tmp/uploads", "*"):
        if file_age(path) > TWENTY_FOUR_HOURS:
            path.unlink(missing_ok=True)

Run it from the scheduler (Scheduled Jobs) rather than in a request. Both functions call stat() and raise OSError for a missing path, so guard against a file deleted between listing and checking.

What not to do

Do not join a user-supplied filename to a path. Sanitize it, or better, do not use it as the stored name at all.

Do not trust an extension. It is a string the uploader chose. Check magic bytes.

Do not use is_dangerous_extension as your upload filter. It is a blocklist. Allowlist instead.

Do not trust Content-Length. Enforce the limit while streaming.

Do not serve uploads from your application’s origin. A separate domain, or Content-Disposition: attachment plus nosniff.

Do not rely on unique_filename under concurrency. It is check-then-write.

Do not call list_files or hash_file in an async handler. Blocking I/O stalls the whole event loop.

Do not display format_size output next to a cloud provider’s number. The units disagree.

Do not assume get_extension understands double extensions. .tar.gz reports as gz.

Performance

Formatting, parsing, and extension functions are pure string arithmetic — nanoseconds, no I/O.

The filesystem functions are the ones to watch. file_age, file_age_human, and unique_filename each perform a stat(). list_files walks the directory and, with recursive=True, the entire tree beneath it. ensure_directory is a mkdir syscall. All are synchronous and block the event loop.

The rule for an async handler is simple: string functions are free, anything that touches the disk is not. Push disk work to a thread with anyio.to_thread.run_sync, or out of the request entirely with Background Tasks.

Function reference

Function	Signature	Returns
format_size	(bytes_value: int | float)	Human size, 1024-based with decimal labels
format_size_binary	(bytes_value: int | float)	Human size with KiB/MiB labels
parse_size	(size_str: str)	Byte count; raises ValueError
get_extension	(filename: str)	Lowercased extension with dot, or ''
get_extension_clean	(filename: str)	Extension without the dot
guess_mime_type	(filename: str)	MIME type or None. Extension-based only.
is_dangerous_extension	(filename: str)	bool. Blocklist — do not rely on it.
is_image_extension	(filename: str)	bool
is_media_extension	(filename: str)	bool
safe_filename	(filename: str, replacement: str = "_")	Single-component safe name
unique_filename	(directory: str | Path, filename: str)	Non-colliding name. Racy.
file_age	(path: str | Path)	Seconds since mtime; raises OSError
file_age_human	(path: str | Path)	Human string such as '3h ago'
ensure_directory	(path: str | Path)	Path, created with parents
list_files	(directory, pattern: str = "*", recursive: bool = False)	list[Path], eager

Related

• File Uploads — receiving and streaming uploads safely
• Static Files — serving files, and why not from the upload directory
• Strings — generating stored filenames
• Hashing — content-addressed storage and deduplication
• Background Tasks — moving disk work out of the request