Backups

Kopia

  • Fast, incremental backups, client-side end-to-end encryption, compression and data deduplication.

  • CLI and GUI included.

  • Kopia

  • Kopia

  • Kopia Tutorial

  • Platforms :

    • Linux.

    • Windows.

    • Mac.

    • FreeBSD.

Ignores

  • Check the Policies -> Files -> Ignore Files / Ignore Rule Files .

  • .kopiaignore  is great.

Restic

BorgBackup

RSync

  • rsync

  • rsync  originated on Unix-like systems and is considered a standard utility on Linux.

  • It's a command-line tool used to efficiently copy and synchronize files and directories between locations. It's commonly used for:

    • Backups

    • File transfers between computers

    • Keeping folders in sync

    • Deploying websites or applications

  • A key feature of rsync is that it only transfers the differences between files, making it much faster than copying everything every time.

  • Platforms :

    • Linux.

    • BSD.

    • Mac.

Version Control

Git (It's a "Snapshot-based versioning with content-addressable deduplicated storage")

  • It is a content-addressable version control system, not a backup system, although it can serve as one in some situations.

  • It does Snapshots of the repository, deduplicated by content.

  • Conceptually, each commit is a snapshot of the entire project. If you check out an old commit, Git reconstructs exactly what the repository looked like at that point.

  • Internally, however, Git doesn't duplicate unchanged files.

  • It's not incremental in the traditional backup sense.

  • A commit only introduces new objects (new blobs, trees, commits), so storage growth is incremental.

  • However:

    • each commit represents a complete project state

    • you don't need a "base backup" plus incrementals to understand a commit

    • Git computes the snapshot by following tree objects, not by replaying a chain of file deltas

  • Git is:

    • Interface: snapshot-based version control

    • Storage: content-addressable object database with deduplication

    • Compression: optional delta compression inside packfiles

  • Diference between Incremental Backups :

    • An incremental backup says:

      • "To reconstruct version N, start from an earlier version and apply all the changes."

    • Git says:

      • "Version N is a complete directory tree. Some pieces of that tree happen to be shared with previous versions."

    • Example : 

      A.txt
B.txt

      • Commit 1: 

      A = "hello"
B = "world"

      • Git stores: 

      Blob A1  -> "hello"
Blob B1  -> "world"

Tree T1
├── A.txt -> A1
└── B.txt -> B1

Commit C1 -> T1

      • The tree is just a directory listing that points to blobs.

      • Commit 2: 

      A = "hello!!!" // only this file is edited.
B = "world"

      • Git stores: 

      Blob A1  -> "hello"
Blob A2  -> "hello!!!"
Blob B1  -> "world"

Tree T2
├── A.txt -> A2
└── B.txt -> B1   <-- exactly the same blob

Commit C2 -> T2

      • Git did not say: "Commit 2 = Commit 1 then replace A.txt". Instead, Commit 2 has its own tree. The only reason it doesn't duplicate B.txt is because it can simply reuse the existing blob.

      • "But doesn't Commit 2 depend on Blob B1?" Yes—but that's object sharing, not change replay.

  • Advantages :

    • Every snapshot is independent.

    • Restoring any snapshot is simple.

    • No "replay" of history.

    • Corruption of one snapshot usually doesn't invalidate the others (assuming the shared objects remain intact).

    • Branching is extremely cheap (one reason Git branches are so fast).

    • This is why systems like Git, ZFS, Btrfs, APFS, and some object stores are based on snapshots.

  • Disadvantages :

    • Metadata can become expensive:

      • Imagine a VM image that's 100 GB.

      • If one byte changes, Git's object model treats the entire file as a different blob. 

      100 GB blob
↓
100 GB blob'

      • Git later delta-compresses them, but initially they're separate objects.

      • For example, imagine a 10 GB video; if you trim 5 seconds off the end, it cannot efficiently represent "remove the last 5 seconds" in its object model. It may later delta-compress them, but that process is expensive and not always effective for compressed media.

      • Traditional incremental backups would simply record:

        • offset 72,384 changed from X to Y

      • which is much smaller.

      • Large binary files are Git's weak point.

    • Backup goals differ:

      • A backup system often wants to answer:

        • "What blocks changed since yesterday?"

      • Storage arrays and cloud backups can copy only those changed blocks over the network.

      • A snapshot model might require hashing lots of data to determine that.

      • Incremental backups can simply track changed blocks using filesystem journals, dirty block maps, or copy-on-write metadata.

      • A good backup system usually provides:

        • scheduled backups

        • automatic retention (daily, weekly, monthly)

        • encryption

        • integrity verification

        • easy restore

        • multiple destinations

        • support for external drives/cloud

      • Git provides almost none of these.

    • Delete policies:

      • Incremental chains can be discarded fairly easily.

      • With deduplicated snapshots: 

      Snapshot A
Snapshot B
Snapshot C

      • they may all share objects.

      • Deleting Snapshot A requires reference counting or garbage collection to determine which objects are still needed. Git has git gc  for exactly this reason.

  • Alternatives for backup :

    • Snapshot/deduplicated backups:

      • They all expose snapshots much like Git, but they're designed for arbitrary files, large datasets, encryption, and efficient deduplication.

      • Ex:

        • BorgBackup

        • Restic

        • Kopia

    • Disk imaging

      • If you want to restore an entire machine after an SSD failure, disk imaging tools are often the better choice.