Skip to content

swath documentation ​

swath lists object stores and filesystems in parallel, prints size totals and text rows, and writes sorted Parquet inventories. Releases are published on the releases page (Install).

Start

  • Start here: pick a job, and when swath fits.
  • Install: release binaries and their verification, the container image, a source build.
  • Your first inventory: total a local directory, list it, save and inspect an inventory.
  • Understand your results: scope, sizes, coverage, observation time, live and saved results.

Use

  • Sources: Amazon S3 and S3-compatible stores, Google Cloud Storage, Azure Blob Storage, and local and mounted filesystems: URLs, credentials, permissions, endpoints, requests.
  • Output: text rows (table, TSV, JSONL), the fields of swath du's rows, reading them in DuckDB, when to write a dataset.
  • Total a source: swath du, the view of where the bytes are, totals per prefix, totals from a saved dataset, and how sizes, hard links and unreadable entries are counted.
  • Filter rows: globs, regular expressions, sizes, times and storage classes.
  • Run large listings: list a bucket of your own start to finish: the command, what it prints, stopping and resuming it, where the result is.
  • Run on a schedule: a scheduled run under a supervisor, a dataset written to a bucket, preemptible machines.
  • Troubleshooting: symptom, cause, fix; the known limits.

Read the output

Reference

Released under the Apache-2.0 license · by Varve