Checkpoint archive name longer than cli argument validator allows

[SpiritInAShell]

user too tired to evaluate this properly (and doesn't want to spam Issues (again) with user error)

Still, if this was a BUG, it should be addressed soon.

tl;dr:

with borg2 v2.0.0b9 it is possible to (unpurposefully) create archive names exceeding length of 200chars, by interrupting the process, where archive name will be extended by string .checkpoint which causes length to exceed 200chars. That archive cannot be accessed by borg list <too long archive name>.

I roughly tried this with borg import-tar aswell as borg create where I interrupted the process after a minute to force creating a .checkpoint

I did some short testing with newly created empty borg2 repos. It seems to be true for these, too.

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG (likely)

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

> borg2beta version # just a symbolic link
2.0.0b9 / 2.0.0b9

Operating system (distribution) and version.

> uname -a
Linux hp-server 5.16.0-0.bpo.4-amd64 #1 SMP PREEMPT Debian 5.16.12-1~bpo11+1 (2022-03-08) x86_64 GNU/Linux

How much data is handled by borg?

A: about 400GiB on a sftp share
B: empty new created repos on ext4 filesystem

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg2beta import-tar <...> "<archive name total length 197> -  < export.tar
# pressed ctrl+c or other cause of interruption
borg2beta list --bypass-lock --short --repo=/<path to a read-only snapshot>/<myRepo>.borg2/ <archivename length total 208 chars ending with .checkpoint.checkpoint>

Describe the problem you're observing.

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Yes.

Include any warning/errors/backtraces from the system logs

[ThomasWaldmann]

That's a bit unpretty indeed. But why do you need to list the contents of a checkpoint archive?

Usually, their main purpose is to hold references to chunks already transferred into the repo and borg prune will get rid of them as soon as a more recent non-checkpoint archive exists.

[SpiritInAShell]

Consciously knowing what is already included in that archive. Unwise naming of archives and not having a clue what is inside. Starting archive creation but (interactively) interrupted by external causes (network, system load, etc)

In a sane world, not neccessary. From a technical perfect world perspective it should be totally unimportant. Referencing an archive not by a human readable string but by its id (like in --format=...{id}) could be a workaround.

[ThomasWaldmann]

BTW, what do you put into your archive names to even reach 197 chars? :-)

[SpiritInAShell]

Normally I wouldn't care about the .checkpoint archives. Removing them would be fine.

This situation is very artificial. I used borg for many years and I considered switching to duplicacy or kopia (non-locking, unified, deduplicating remote storage simultanous from many devices), but there are plenty (practical) reasons that I trust borg more (availability of sane documentation is one important reason).

Now borg2 is near release and I am doing funny stuff to learn about its limits. Also, I might stumble upon bugs that way.

Converting old repositories, either by transfer, or by transfer, then export/import-tar is part of my tests. There - within this unstructured testing - I create new (temporary) archive names like coming from repo1,disk1,machine1:oldarchivename1. In a sane use case my archive names have a (measured) length of about 120chars. (I have never used --comment so I should put parts of the description in there)

Also, I plan to transfer much (1TiB+) "production/useful" data to a online sftp accessible storage (Hetzner storagebox). As uploading with 50MBits takes quite some time, I do something very nasty like "PRE-SEEDING:"

That is pushing large directories to the remote server in times...

• where I have all the bandwidth,
• where disks are accessible
• and system has no load/is unattended (eg overnight)

A current goal is to use all available bandwidth with least-possible user interaction. But that way transfers need to be interrupted, and sometimes I won't be able to continue them. Hence .checkpoints in this artificial use case are created but not always completed.

That way "the data" is already there (chunked, compressed), when I upload the "sane" archives that I want to keep. When doing a "productive" create to the online storage, almost nothing is transferred.

I consider borg like an arbitrary deduplicating data storage for data hoarding. I have my whole data in 2x 10GiB+ borg1 repos and I consider some to be off-site on the storagebox (but have not fully decided what).

Yes, my motivations and use-cases come from personal, autistic, adhd, and very bad executive function disorder.