Skip to content

Start here ​

swath lists what an object store or a filesystem holds, and reports what it found. It reads listing metadata only: names, sizes, times and storage fields. It never reads, changes or deletes the contents.

A source is a local or mounted directory, s3://bucket/prefix, gs://bucket/prefix or az://account/container/prefix (sources).

Pick a job ​

You want toRunRead
Find where the space wentswath du SOURCE, or -d N for totals to a fixed depthYour first inventory
List every entry, or export rows for a scriptswath list SOURCE, or -o rows.tsv.zstOutput
Save a sorted Parquet inventory to query laterswath list SOURCE -o DIRQuery an inventory

New to swath? Install it, then follow Your first inventory on a local directory. It needs no account and no credentials.

When swath fits ​

  • du and find answer most questions about a small directory. swath walks a directory with many workers at once, which helps on a large directory or a slow network mount, and it can save the walk as a dataset.
  • aws s3 ls --recursive, gcloud storage ls --recursive and rclone lsf -R --fast-list list a bucket one page of keys after another. swath divides the key space into ranges and lists them in parallel, which matters once a bucket holds tens of millions of keys.
  • Scheduled inventories (S3 Inventory, GCS Storage Insights, Azure Blob inventory) can cost less than listing the whole bucket again and again, when a daily or weekly report is fresh enough. On AWS, S3 Metadata's live inventory table keeps every object of a bucket in a queryable table, typically within an hour of a change, once its initial backfill is done. Use swath when you need the answer now, for a bucket or prefix that has neither set up, for the same dataset across providers, or for a source without one, such as a local or mounted filesystem.

swath only lists: it does not copy data, schedule runs, compare inventories or send alerts.

Released under the Apache-2.0 license · by Varve