Git-LFS

Swarmfile can act as a git-LFS server. Keep your source in the git host you already use - GitHub, GitLab, or self-hosted - and route only the large binary assets to Swarmfile, where they get real storage, deduplication, encryption (on most upload paths - see below), and quota instead of bloating your git remote.

This is the lowest-friction way to start: nothing about your VCS or your team's workflow changes. You keep git push; git lfs push/pull/checkout move the big files through Swarmfile.

The fuller destination is making Swarmfile the home for the whole project - native version control on a drive you mount, with no clone at all (see Coming from Git and Version Control). The LFS bridge is where you start; you can move the rest at your own pace.

Which projects it works on#

git-LFS works on managed-key and unencrypted projects. It is not available on end-to-end-encrypted projects: a stock git-lfs client can't hold a per-user encryption key, and the hub structurally can't read end-to-end content, so there is no one to encrypt or decrypt for. An LFS request against an E2E project returns a per-object 422 with a clear message rather than failing silently. See End-to-End Encryption for what that tier trades off.

Encryption at rest depends on which upload path the object takes, not only on the project's tier - git-LFS is the one place where a managed project can hold plaintext, by design (LFS is an add-on surface, and paying worker-side encrypt/decrypt on multi-GB artifacts isn't worth it). The paths, verified against the code:

An unencrypted (Free-plan) project is plaintext on every path by definition, and end-to-end projects don't support git-LFS at all (above). Objects pushed through the chunked block path also deduplicate against the same content on your mounted drive; the direct paths trade that cross-surface dedup for speed.

If a binary must be encrypted at rest, keep it on the mounted drive (managed or E2E) rather than routing it through LFS.

Setup#

1. Generate a git-LFS credential#

In the dashboard, open your project → the git-LFS tab → Generate git-LFS credential. This mints a project-scoped API key (an sf_key_… string). Treat it like a password; it grants access to this one project's files. It counts against your plan's headless-key allowance (Starter 2 per seat, minimum 5; Pro 5 per seat, minimum 10) - if the mint is refused, revoke an unused key or add a key pack under Settings → Billing. One key per project is the natural pattern: reuse it for that project's LFS, CI, and scripts rather than minting a fresh one each time.

2. Commit .lfsconfig#

The panel shows an .lfsconfig block. Commit it to the root of your repo so every clone points its LFS traffic at Swarmfile:

[lfs]
  url = https://hub.swarmfile.com/orgs/<org>/lfs/<project>

The URL encodes your org and project, so there's no coupling to your git remote - the git remote stays exactly where it is.

Keep it to just the url: git-lfs only accepts a small safe allowlist of keys in a repo-committed .lfsconfig and prints a warning: These unsafe '.lfsconfig' keys were ignored line for anything else - per-machine settings (the large-download retry window, credentials) go in git config in the next step instead.

3. Store the credential#

git-lfs authenticates over HTTP Basic. Run the setup once per clone: register git-lfs's filters (without this, git add commits the raw bytes instead of an LFS pointer), store the sf_key_… in your git credential store as the password (the username can be anything), tell git-lfs to use basic auth directly so it skips a 401-probe round trip on every call, and widen the large-download retry window (see Large files):

git lfs install

# Store the credential (username arbitrary, password = your sf_key_…)
git credential approve <<EOF
protocol=https
host=hub.swarmfile.com
username=swarmfile
password=sf_key_…
EOF

# Use basic auth for this endpoint without the 401 probe
git config lfs.https://hub.swarmfile.com/orgs/<org>/lfs/<project>.access basic

# Per-machine: how many times a large download retries while it's prepared.
# (.lfsconfig can't carry this - git-lfs ignores non-allowlisted keys there.)
git config lfs.transfer.maxretries 240

The "Generate git-LFS credential" panel emits this snippet with your real org, project, and key filled in - copy it from there; it has a Windows (PowerShell) tab for machines where bash is not available. A ~/.netrc entry (machine hub.swarmfile.com login swarmfile password sf_key_…) works too if you prefer.

On Windows, run the same setup from PowerShell instead - the credential step uses a here-string, since <<EOF is bash-only:

git lfs install

@'
protocol=https
host=hub.swarmfile.com
username=swarmfile
password=sf_key_…
'@ | git credential approve

git config lfs.https://hub.swarmfile.com/orgs/<org>/lfs/<project>.access basic
git config lfs.transfer.maxretries 240

4. Track and push#

From here it's stock git-LFS - nothing Swarmfile-specific:

git lfs track "*.psd" "*.exr" "*.uasset"   # writes .gitattributes
git add .gitattributes
git add my-big-file.psd
git commit -m "Add art source"
git push            # source objects to your git host
git lfs push        # big files to Swarmfile

Teammates who clone the repo get the committed .lfsconfig, add their own credential once (step 3), and git lfs pull fetches the big files from Swarmfile.

Or let the GitHub App do it#

Everything above works with any git host and any git-lfs client. If your remote is GitHub, the project's git-LFS tab can do part of it for you (this is being rolled out - the card appears once the GitHub App is registered on your deployment; the manual steps above always work):

• Install the GitHub App - click the button in the tab, pick the repositories to let Swarmfile see, and come back. One installation per Swarmfile organization.
• Route a repository - in the same tab, choose the repository and click Route this repo to <project>. That links it to the project (each repo gets its own .lfsconfig, because the URL embeds the project id).
• Open setup PR - Swarmfile opens a small pull request that adds .lfsconfig to the repository. Merge it (or close it and use the manual steps above - nothing is forced).
• Mint git-LFS credential - the tab mints a project-scoped sf_key_… for you and shows the same per-clone commands as step 3, prefilled.

After that, git lfs push/pull work exactly as described above, and pushes show up on the mounted drive automatically - a push webhook reflects the pushed branch's objects onto the project's matching Swarmfile branch, so you don't need to run swarmfile-lfs reflect by hand. The App is provisioning-only: it never sees or proxies your LFS objects, and the credential stays the sf_key_… you minted - uninstalling the App doesn't break clones that already have their credential.

Deduplication and the mounted drive#

Swarmfile stores content once. If two people push the same bytes - or the same object already exists - the second upload is skipped, exactly as LFS dedup is meant to work. One exception: when the bytes so far exist only as a file saved on the drive, the first push uploads them anyway, and Swarmfile checks the drive copy in the background; once it's confirmed, later pushes of the same bytes are skipped.

It goes one step further: a file saved to the mounted Swarmfile drive through the desktop app is git lfs pull-able by its object id, with zero duplicate storage. (So is a file added through the web dashboard's uploader. A file saved before object ids were recorded can be given one with swarmfile git backfill-oids.) The LFS object is the native content - the same bytes the drive already holds, addressed two ways. So a file your team produced on the drive can be pulled by a git-LFS client that only knows its oid, without storing it twice.

That holds for any version the project still keeps: a drive version that has aged out of your history retention is gone for LFS too, and a pull of it reports the object as missing. On the default Keep every saved version projects a drive version doesn't age out, so its LFS object stays pull-able while the version is retained. Pushing or re-pushing an object with git lfs push makes it an LFS object in its own right, kept independently of drive retention. It doesn't apply to end-to-end encrypted projects (no LFS at all), nor to content encrypted with a static key you supply yourself (SWARMFILE_ENCRYPTION_KEY): in both cases the hub holds no key, so Swarmfile never sends it a file's SHA-256 object id.

The reverse direction works too, with one manual step. An object pushed with git lfs push can be projected onto the mounted drive as a first-class file at its repo path - reusing the blocks the push already stored (again, zero duplicate storage). Because the LFS protocol only ever sends an object's id, never its path, the file's path comes from your git working tree, so you run one command after pushing:

git lfs push origin main
swarmfile-lfs reflect            # projects the just-pushed objects onto the drive

reflect reads git lfs ls-files, sends the oid→path map to Swarmfile, and the hub creates the drive entries. It mirrors your current git branch onto a same-named Swarmfile branch (auto-created; override with --branch), and it never overwrites a file already on the drive - a path already occupied by different content is reported as a conflict and left alone. Re-running it is safe (already-projected files report as unchanged). (If you run the desktop app rather than the standalone binary, swarmfile-engine reflect is the identical command.) To make it automatic, wire it into a git push alias:

git config --global alias.pushr '!git push "$@" && swarmfile-lfs reflect'
# then: git pushr origin main

After reflection, that path has a real drive entry - so it's reachable through both surfaces (drive + LFS), participates in mount worksharing locks, and appears for teammates on the drive.

Reflection is additive: it creates or leaves-alone, and never deletes. Removing or renaming a file in git won't remove or move its drive entry - do that on the drive (or via the dashboard). It enforces the same write permission a drive write does: on a project with per-folder access rules, reflecting a file into a folder you can't write to is refused (reported as denied) and skipped.

Large files#

Small and mid-sized objects download immediately. A very large object (roughly a gigabyte or more) is prepared on its first download: Swarmfile assembles it into a single downloadable copy in the background, and until that finishes git lfs pull waits and retries automatically - you'll see it pause, then complete. This is why step 3 sets git config lfs.transfer.maxretries 240: it gives the client enough patience to wait through preparation. Once prepared, that copy is reused, so subsequent pulls of the same object are immediate.

If a pull of a very large object ever gives up with a "being prepared, retry" message, just run git lfs pull again - preparation continues in the background and the retry will pick up the finished copy. (Raising lfs.transfer.maxretries further with git config widens the wait window if you routinely pull multi-gigabyte objects. It has to be git config, not .lfsconfig - git-lfs ignores that key in a committed config file.)

Pushing files over 100 MB#

A plain git lfs push sends each object in a single request, which the network edge caps at ~100 MB - so a stock push of a larger object is rejected. To push bigger files, install the Swarmfile transfer agent: a small (~2 MB), standalone, open-source binary (swarmfile-lfs) that speaks git-lfs's documented custom-transfer protocol. No desktop app required - but if you already run the desktop app, it's already installed (every desktop installer bundles it on PATH), so skip the download and go straight to swarmfile-lfs install.

# 1. Install the agent (pick one):
brew install swarmfile/tap/swarmfile-lfs      # macOS / Linux (Homebrew)
cargo install swarmfile-lfs-transfer          # any Rust toolchain
# …or download the binary for your platform from the releases page.
# …or skip this step entirely if the Swarmfile desktop app is installed.

# 2. Wire git-lfs to it, once per machine:
swarmfile-lfs install

The Homebrew tap, cargo crate, and curl | sh installer (get.swarmfile.com/lfs) are being rolled out - check the git-LFS tab in your project dashboard for the install command that's live for your account. If you already run the desktop app, none of that is needed: swarmfile-lfs is installed on PATH with the app, and the app also bundles an equivalent built-in agent (swarmfile-engine lfs-transfer; its encryption path differs slightly - see below).

swarmfile-lfs install just writes the git-lfs custom-transfer adapter config for you (add --local to scope it to the current repo instead of your global config; swarmfile-lfs uninstall undoes it in the same scope). With it in place, git lfs push transparently uploads large files as small content-addressed blocks - encrypted on a managed project when they take the chunked path (see the plaintext exceptions above) - up to 256 GiB per object (and your project quota), deduped against the same content on your Swarmfile drive. Nothing else changes: git lfs push/pull work exactly as before, the agent only kicks in for uploads, and it authenticates with the same credential you stored in step 3 (so make sure that's done on each machine that pushes large files).

Without the agent, pushes still work for files up to ~100 MB, and any file is always available by writing it to the mounted drive instead. If you already run the Swarmfile desktop app, swarmfile-lfs is already on PATH - swarmfile-lfs install is all you need. The engine also exposes an equivalent built-in agent as swarmfile-engine lfs-transfer, if you'd rather point git-lfs at that (git config lfs.customtransfer.swarmfile-chunked.path swarmfile-engine / .args lfs-transfer). (The built-in one encrypts managed blocks client-side and takes a multipart path for ≥ 64 MiB objects; the standalone binary relies on the hub's server-side encryption - see the table above.)

An interrupted large push is safe to re-run. Just run git lfs push again: content-addressed blocks the agent already uploaded are deduplicated, and the desktop app's built-in agent additionally resumes a ≥ 64 MiB multipart upload from the parts that already landed instead of starting over. Nothing is left behind either way - an upload nobody resumes is reaped server-side after 7 days on Swarmfile-managed storage.

What's supported#

• The LFS Batch API (upload and download) and the basic transfer adapter - the operations git lfs push, pull, checkout, and fetch use.
• Verify, so a truncated upload is caught rather than silently stored.
• Locking (git lfs lock / git lfs locks / git lfs unlock), for every tracked path - including files that live only in your git host and have never been written to the Swarmfile drive. When a path does also exist on the drive, its LFS lock and its desktop/mount worksharing lock are the same server-side lock, so locking via git-lfs and locking on the drive mutually exclude, across machines and across both surfaces. When a path exists only in git, it takes a standalone advisory lock that other git-lfs clients see; unlocking is owner-gated (use --force to break another user's lock). Locks hold for 30 days and nothing renews a git-side lock, so a lapsed one releases itself rather than staying stuck - take it again if the work outlives that. See Working with Files.

Performance#

There's a reproducible benchmark in the repo - run it against your own project and uplink rather than trusting a number from ours:

SWARMFILE_API_KEY=sf_key_… \
SWARMFILE_LFS_URL=https://hub.swarmfile.com/orgs/<ORG>/lfs/<PROJECT> \
SIZES_MB="20 100" STOCK_COMPARE=1 \
  ./tests/integration/bench-git-lfs.sh

Absolute MiB/s is bounded by your connection to Cloudflare, so it's not a useful headline number. The ratios below are what hold regardless of uplink; these are what we measured from a single client:

• The chunked agent uploads faster than stock git-lfs, because it sends many ~1 MiB blocks in parallel instead of one large request - roughly 1.5-2× the throughput of the stock basic transfer on the same connection at 20-100 MiB.
• Stock git-lfs can't push past ~100 MB to Swarmfile - a single-request upload of a 100 MiB object is rejected at the edge, while the agent pushes it fine (and on up to 256 GiB). The agent isn't an optimization here; it's the only way.
• Re-pushing an unchanged object transfers nothing - git-lfs sees the server already has the object and skips it, so a re-push costs one batch round-trip (well under a second) no matter the file's size.

The benchmark verifies a byte-for-byte round trip on every row, prints a table, and writes a CSV (git-lfs-bench.csv) you can attach to a report.

Continuous integration (CI)#

Two things make CI painless, and they split by direction:

• Pulling assets needs no agent. git lfs pull/fetch/checkout of any size works with the stock git-lfs that's already on every runner - large objects just materialize and retry (that's what the lfs.transfer.maxretries setting below is for). A build or test job needs only credentials.
• Pushing objects over 100 MB needs the agent (the ~100 MB edge cap applies to stock single-request uploads). A release or asset-generating job adds one install step.

In both cases, store the project's sf_key_… as a CI secret (SWARMFILE_API_KEY) and feed it to git through a credential helper. Never bake the key into .lfsconfig or a command line.

GitHub Actions#

Pull-only job (build/test - no agent):

env:
  SWARMFILE_API_KEY: ${{ secrets.SWARMFILE_API_KEY }}
steps:
  - uses: actions/checkout@v4
    with: { lfs: false }            # skip the auto-smudge; we pull explicitly below
  - name: Configure Swarmfile LFS credentials
    run: |
      git config --global credential.helper \
        '!f() { test "$1" = get && printf "username=lfs\npassword=%s\n" "$SWARMFILE_API_KEY"; }; f'
      git config --global lfs.transfer.maxretries 240   # wait through large-object preparation
  - run: git lfs pull                # any size - materialize + retry handles big objects

Push job (release/asset generation - installs the agent):

env:
  SWARMFILE_API_KEY: ${{ secrets.SWARMFILE_API_KEY }}
  SWARMFILE_LFS_URL: https://hub.swarmfile.com/orgs/<ORG>/lfs/<PROJECT>
steps:
  - uses: actions/checkout@v4
  - name: Install the Swarmfile transfer agent
    run: |
      curl -fsSL https://get.swarmfile.com/lfs | SWARMFILE_LFS_BINDIR="$HOME/.local/bin" sh
      echo "$HOME/.local/bin" >> "$GITHUB_PATH"
      swarmfile-lfs install         # writes the git-lfs custom-transfer config (global)
  - name: Configure Swarmfile LFS credentials
    run: |
      git lfs install                 # register the clean/smudge filters on this runner
      git config --global credential.helper \
        '!f() { test "$1" = get && printf "username=lfs\npassword=%s\n" "$SWARMFILE_API_KEY"; }; f'
      git config --global "lfs.${SWARMFILE_LFS_URL}.access" basic
      git config --global lfs.transfer.maxretries 240
  - run: git lfs push origin HEAD   # >100 MB objects now go through the agent

GitLab CI#

push-assets:
  image: ubuntu:22.04
  variables:
    SWARMFILE_LFS_URL: https://hub.swarmfile.com/orgs/<ORG>/lfs/<PROJECT>
  before_script:
    - apt-get update && apt-get install -y git git-lfs curl ca-certificates && git lfs install
    - curl -fsSL https://get.swarmfile.com/lfs | SWARMFILE_LFS_BINDIR=/usr/local/bin sh
    - swarmfile-lfs install
    - git config --global credential.helper
        '!f() { test "$1" = get && printf "username=lfs\npassword=%s\n" "$SWARMFILE_API_KEY"; }; f'
    - git config --global "lfs.${SWARMFILE_LFS_URL}.access" basic
    - git config --global lfs.transfer.maxretries 240
  script:
    - git lfs push origin HEAD
  # Set SWARMFILE_API_KEY as a *masked, protected* CI/CD variable - not in this file.

The credential helper reads SWARMFILE_API_KEY at the moment git invokes it, so the secret never lands on a command line or in the log. For a pull-only job you can drop the agent install and the access basic line entirely.

Leaving Swarmfile#

Your source lives in your own git host and never touches Swarmfile - so leaving is only ever about pulling your binary objects back out, and that works with stock git-lfs and no Swarmfile software installed at all. There is no export API to learn and no proprietary format to unwind.

# In a clone that points at Swarmfile, pull every version of every object
# referenced by any branch or tag into your local .git/lfs/objects store:
git lfs fetch --all

# Confirm you hold complete, uncorrupted copies (each object hashes to its oid):
git lfs fsck

After git lfs fetch --all, every object is on your disk. To move them to another LFS server, point the repo at the new endpoint and push:

git config -f .lfsconfig lfs.url https://your-new-lfs-server/…   # GitHub, GitLab, S3-backed, self-hosted, anything
git lfs push --all <remote>

This is deliberately the standard git-lfs export path - the custom-transfer agent is only an optional accelerator for large uploads, and downloads never need it. You can walk away with a single command, which is exactly the point: your binaries are as portable as your git history.

Troubleshooting#

A warning: These unsafe '.lfsconfig' keys were ignored line. Something other than lfs.url is in the committed .lfsconfig - older setups put lfs.transfer.maxretries there. git-lfs only accepts a safe allowlist in a repo-committed config file and ignores the rest by design; move per-machine settings to git config (step 3 shows the retry one). The warning is about the ignored key, not a failed transfer.

A 401 on every LFS call. Credential helpers disagree about whether the key belongs in the username or the password field; Swarmfile accepts the sf_key_… in either, but the safest placement is the password. Re-run the git credential approve step above, and confirm you set …access basic so git-lfs sends the header on the first request.

A 422 for every object. The project is end-to-end encrypted, which git-LFS can't support (see above). Use a managed or unencrypted project, or move the source-plus-assets to a mounted drive instead.

git lfs pull fetches nothing / objects are missing. Confirm .lfsconfig is committed and its url names the right org and project, and that the credential you stored is for that same project.

git lfs unlock fails on a lock someone else holds. Path locks are owner-gated: only the user who took a lock can release it normally. Pass --force (or --id <id> --force) to break another user's lock, or ask them to unlock it. git lfs locks shows the current holder of every live lock.