Metadata-Version: 2.1
Name: btrfsgit
Version: 90.0.0
Summary: B-tree Filesystem Git attempts to enable git-like workflow for BTRFS subvolumes. Commit, push, checkout, stash, pull...
Home-page: https://github.com/koo5/bfg
License: GNU Affero General Public License v3 or later (AGPLv3+)
Author: koo5
Author-email: kolman.jindrich@gmail.com
Requires-Python: >=3.6.1,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
Classifier: License :: Other/Proprietary License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Provides-Extra: dev
Provides-Extra: doc
Provides-Extra: test
Requires-Dist: PyInquirer (>=1.0.3,<2.0.0)
Requires-Dist: black (==20.8b1); extra == "test"
Requires-Dist: fire (==0.4.0)
Requires-Dist: flake8 (==3.8.4); extra == "test"
Requires-Dist: flake8-docstrings (>=1.6.0,<2.0.0); extra == "test"
Requires-Dist: isort (==5.6.4); extra == "test"
Requires-Dist: mkdocs (>=1.1.2,<2.0.0); extra == "doc"
Requires-Dist: mkdocs-autorefs (==0.1.1); extra == "doc"
Requires-Dist: mkdocs-include-markdown-plugin (>=1.0.0,<2.0.0); extra == "doc"
Requires-Dist: mkdocs-material (>=6.1.7,<7.0.0); extra == "doc"
Requires-Dist: mkdocs-material-extensions (>=1.0.1,<2.0.0)
Requires-Dist: mkdocstrings (>=0.13.6,<0.14.0); extra == "doc"
Requires-Dist: pathvalidate (>=2.5.0,<3.0.0)
Requires-Dist: pip (>=20.3.1,<21.0.0); extra == "dev"
Requires-Dist: pre-commit (>=2.12.0,<3.0.0); extra == "dev"
Requires-Dist: pytest (==6.1.2); extra == "test"
Requires-Dist: pytest-cov (==2.10.1); extra == "test"
Requires-Dist: toml (>=0.10.2,<0.11.0); extra == "dev"
Requires-Dist: tox (>=3.20.1,<4.0.0); extra == "dev"
Requires-Dist: twine (>=3.3.0,<4.0.0); extra == "dev"
Requires-Dist: virtualenv (>=20.2.2,<21.0.0); extra == "dev"
Description-Content-Type: text/markdown

# BFG
B-tree Filesystem Git attempts to enable git-like workflow for subvolumes. Commit, push, checkout, stash, pull..

<p align="center">
<a href="https://pypi.python.org/pypi/bfg">
    <img src="https://img.shields.io/pypi/v/bfg.svg"
        alt = "Release Status">
</a>

<a href="https://bfg.readthedocs.io/en/latest/?badge=latest">
    <img src="https://readthedocs.org/projects/bfg/badge/?version=latest" alt="Documentation Status">
</a>

</p>

* Documentation: <https://bfg.readthedocs.io>


## user install:
```pip install --user BtrFsGit```


## dev install:
```
pip install --user poetry
`cd BtrFsGit
poetry install # only installs the executable into somewhere like `/.cache/pypoetry/virtualenvs/bfg-iXQCHChq-py3.6/bin/`. It just doesn't have a "development mode" like setuptools have with `pip install -e .`. So find that directory, and copy the `bfg` into your `~/.local/bin/`. But that's about to be [fixed soon](https://github.com/python-poetry/poetry/issues/34).
```



## status
Undertested, but `commit_and_push_and_checkout`, `remote_commit_and_pull` and other commands work.

## why
I built this because my scenario is not just simple backup, but also transfering subvolumes back and forth between multiple machines, where no one machine is a single source of truth. In other words, a desktop computer and a notebook, and a subvol with a bunch of VM images. And then maybe a bunch of external backup HDDs. 

## cool features
* It tries to figure out shared parents smartly, by walking the uuids of subvolumes of both filesystems. It doesn't just expect the last transferred snapshot to "be there", in a fixed location, like other tools do.
* No config files, just specify a source subvol and a target subvol (and the ID 5 mount point) on the command line, and in case of a remote machine, a ssh command to use.

## what this doesn's do (yet?)
* snapshot pruning
* cleanup after failure
* config files

## what this isn't yet
* a proper python package that you can install into your $PATH

## planned features
* automatically saving and propagating `sub list` dumps - to allow finding shared parents also for offine generating of send streams, even across multiple machine hops
* Generating a send stream, and applying it later.

## wishlist
* some kind of integration with https://github.com/csirac2/snazzer/#snazzer for integrity checks
* maybe some automation for non-BTRFS backups, ie, create a snapshot, rsync it to an ext4, (and apply snazzer..)

## what this will probably never be
* an attempt to immitate more of git, like merging, exact same command syntax, commit messages (well maybe commit messages would make sense, maybe as a backend to datalad?)..


## example workflow
this is how i ping-pong my data between my two machines:
```
./main.py   \
  --YES=true  \  #  no confirmations
  --LOCAL_FS_TOP_LEVEL_SUBVOL_MOUNT_POINT=/nvme0n1p6_crypt_root  \  # ugly hack
  --sshstr='/opt/hpnssh/usr/bin/ssh   -p 2222   -o TCPRcvBufPoll=yes -o NoneSwitch=yes  -o NoneEnabled=yes     koom@10.0.0.20'  \
  commit_and_push_and_checkout  \  # the command
  --SUBVOLUME=/d \  # source
  --REMOTE_SUBVOLUME=/mx500data/lean  # target
```
...this:
* makes a read-only snapshot of /d/ in /.bfg_snapshots.d/<timestamp>_from_<hostname>
* finds the best shared parent and sends the snapshot to the other machine over ssh
* receives it on the other machine in /mx500data/.bfg_snapshots.lean
* makes a read-only snapshot of /mx500data/lean in /mx500data/.bfg_snapshots.lean/<timestamp>_stash
* deletes /mx500data/lean
* makes a read-write snapshot of the received snapshot, in /mx500data/lean

 
And back:
```
./main.py   --YES=true    --REMOTE_FS_TOP_LEVEL_SUBVOL_MOUNT_POINT=/mx500data    --sshstr='/opt/hpnssh/usr/bin/ssh   -p 2222   -o TCPRcvBufPoll=yes -o NoneSwitch=yes  -o NoneEnabled=yes     koom@10.0.0.20'   remote_commit_and_pull   --SUBVOLUME=/d  --REMOTE_SUBVOLUME=/mx500data/lean
```
full output:
[example_session.md](misc/example_session.md)

see also:
[test1](tests/test1.sh)

## available commands
[docs](docs/)

## prerequisites

### install
This isnt a proper python package yet. Python3.8 is expected. Checkout the repo, do
```
 virtualenv -p /usr/bin/python3.8 venv
 pip install -r requirements.txt 
 
```
### mount the root
#### problem
If you want to work with subvolumes mounted with `subvol=..`: This is how linux distributions set up your system by default. In this case, BFG would not be able to automatically find the filesystem path of a subvolume given its UUID, so, it wouldn't be able to call `btrfs send` with correct `-p` parents.
#### solution
make sure that the root subvolume of your BTRFS filesystem is always mounted. For example my fstab entry:
```
/dev/mapper/nvme0n1p6_crypt /nvme0n1p6_crypt_root  btrfs   defaults,subvol=   0   2
```
For some operations, you will need to pass this mountpoint like so: `--LOCAL_FS_TOP_LEVEL_SUBVOL_MOUNT_POINT=...` or `--REMOTE_FS_TOP_LEVEL_SUBVOL_MOUNT_POINT=...`.
### avoid nested subvolumes
#### problem
To be able to make use of stash and checkout, the subvolume that you want to manage with BFG should not contain other subvolumes, so that it can be `btrfs subvolume delete`'d without affecting your snapshots or other subvolumes. (or possibly we could just `mv`?)
#### solution
As an example, i have a subvolume `/data`, and by default, BFG will store all snapshots in `/.bfg_snapshots.data`, and i don't have snapper doing stuff in `/data/.snapshots`.

### prevent writes to incomplete snapshots
#### problem
BTRFS doesn't make a subvolume read-only when it's `btrfs receive`-ing. If another program writes into it at that time, something bad will happen..
#### solution
don't do it!



