Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
fbnielsen
@fbnielsen
ok I see that it is unable to reach the destination, would that prevent cleanup in general?
Tobias Oetiker
@oetiker
well it prevents cleanup for the src if one of the dsts of the src fails
fbnielsen
@fbnielsen
alright thanks, I will fix it - will it followup on the cleanup when dst is working again?
Tamás Gérczei
@tgerczei
@oetiker I've packaged Your last release for Gentoo, are You interested in it?
Jim Klimov
@jimklimov
@oetiker : I've update the PRs oetiker/znapzend#439 and oetiker/znapzend#386 with more tested codepaths, both green now and coverage is up :)
Tobias Oetiker
@oetiker
@jimklimov great ... looking at them now
Jim Klimov
@jimklimov
thanks! :)
Tobias Oetiker
@oetiker
@tgerczei if you have files to add to the distro to simplify packing on gentoo ... sure create a PR
Jim Klimov
@jimklimov
@oetiker : PRs revisited and green again. Did not merge one of the suggestions yet, not sure if it was right :)
Tobias Oetiker
@oetiker
true ... fixed it ... if (not @backupSets) {
merged one ... but the other now has a conflict
Jim Klimov
@jimklimov
thanks, on it again
posted the update(s)
Jim Klimov
@jimklimov
thanks ;_
Tamás Gérczei
@tgerczei
@oetiker well that's not how Gentoo works but I've sent a PR nevertheless in order to have my package mentioned: oetiker/znapzend#440
I'd also seize the opportunity to ask about oetiker/znapzend#323 which isn't mine but I second the necessity of the feature
Tobias Oetiker
@oetiker
@tgerczei sure ... the problem is the API design ... positional parameters are becoming a problem here ...
Tamás Gérczei
@tgerczei
@oetiker yeah, I've seen the remark - unfortunately that's where it all stalled, but I'd still need the raw send feature for encrypted backups... since I guess the PR got orphaned, how do we go on about it? I don't want to inherit it unless I have to, nor do I know whether you'd be willing to add the feature another way you deem proper?
if there's a bounty system in place, I can try to sponsor it somewhat, should that help
Tobias Oetiker
@oetiker
I will be glad to accept a PR if it is done in a way that fits ... the first step would be to come up with a sensible way to configure that feature
Tamás Gérczei
@tgerczei
@oetiker considering the use-case and context I think this should be safe to assume enabled if the encryption feature is enabled on the pool containing the source dataset and the source dataset is encrypted, with an option to override (disable) this in the plan - in practice this would mean that encrypted data is always sent raw with the destination not having its encryption key at all, thereby ensuring security of remote data at rest - what do you think?
Jim Klimov
@jimklimov
@oetiker maybe it is the adrenaline rush from recent birthday celebration (gotta achieve something in the remaining years) or whatnot, but got a few more PRs green and pending :)
As for the "inherited origins" PR, I think by now it does what I intended so want to drop the "WIP" tag and let you merge it :-) Though maybe more people giving it a run first and finding any unfortunate regressions would also help.
Tobias Oetiker
@oetiker
well :) not sure about the other people ... but yes ... lets merge it
Jim Klimov
@jimklimov
As commented in the code, there are some nuances that could be handled better (e.g. processing/tracking of datasets we would ignore) but these refinements might be in some other PR
I gotta switch to dayjob at some point :-)
Tobias Oetiker
@oetiker
great work !
thank you
Jim Klimov
@jimklimov
I feel itchy... and scratchy.... way too often when I use stuff :)
Tobias Oetiker
@oetiker
glad you scratched it
Jim Klimov
@jimklimov
for a couple of decades now, and probably still, my "Why FOSS?" motto is "Forging the tools of my trade" which is not easy or possible in other ecosystems ;)
and since the late 90's, never had a chance to say directly: Thank you for the goodnesses of RRD(tool) and MRTG! (by the way)
Would you visit FOSDEM sometime so I can repeat that in person (assuming I'm approved this year too)?
Tobias Oetiker
@oetiker
it tends to clash with my winter holidays unfortunately ... for the last few years ... otherwhise I would yes
Jim Klimov
@jimklimov
Oh, yes... Andy said that the last time...
then if not visiting, be sure to have a good family time then :)
Jim Klimov
@jimklimov
@oetiker : I posted some ideas about recursive send into oetiker/znapzend#438 - does that seem reasonable or did I miss something glaring? :-)
also seems oetiker/znapzend#437 can be closed
kevdogg
@kevdogg
Hi - just installed znapsend on Arch Linux (installed with ZFS on Root), backing up to FreeNAS. Just a few questions since I'm a little confused about things. I believe I made my first backup correctly, however is there a way to verify? Where is configuration file stored? Web page states in the zfs file structure itself -- so how to read? Although I have the systemd znapzend service started and enabled -- what program is involved with the scheduling? I ran the first task with a runonce command - how to make this run automatically?
Tobias Oetiker
@oetiker
znapzend schedules itself
settings are stored in zfs properties
tuxwielder
@tuxwielder
Hi, I could use some help on the new feature that should allow excluding part of an recursive dataset?
tuxwielder
@tuxwielder
Looks like excluded snapshots are being deleted (after https://github.com/oetiker/znapzend/pull/379/files/8e65186d9b8a98502f192c615055d8c35aebf460 ), but we still send them. This causes ERRORs on later runs, since common snapshots are missing). Are we missing some logic?:
--- ZnapZend.pm.dist    2019-11-28 17:07:17.927267363 +0100
+++ ZnapZend.pm    2019-11-28 17:44:33.203399136 +0100
@@ -336,6 +336,22 @@
         #from being snapshot/sent by setting property "org.znapzend:enabled"
         #to "off" on them
         for my $srcDataSet (@$srcSubDataSets){
+
+            # get the value for org.znapzend property
+            my @cmd = (@{$self->zZfs->priv}, qw(zfs get -H -o value org.znapzend:enabled), $srcDataSet);
+            print STDERR '# ' . join(' ', @cmd) . "\n" if $self->debug;
+            open my $prop, '-|', @cmd;
+
+            # if the property does not exist, the command will just return. In this case,
+            # the value is implicit "on"
+            $prop = <$prop> || "on";
+            chomp($prop);
+            if ( $prop eq 'off' ) {
+                $self->zLog->debug('Skipping ' . $srcDataSet . ' due to it being explicitly disabled.');
+                next;
+            }
+
+
             my $dstDataSet = $srcDataSet;
             $dstDataSet =~ s/^\Q$backupSet->{src}\E/$backupSet->{$dst}/;
Please note the missing "-s local", so it also works on inherited properties in the recursive dataset.
tuxwielder
@tuxwielder
Also it looks like we are not properly destroying disabled snapshots when we have multiple disabled datasets:
--- lib/ZnapZend.pm.dist    2019-11-29 00:29:36.980904411 +0100
+++ lib/ZnapZend.pm    2019-11-29 00:21:08.546796322 +0100
@@ -615,7 +615,13 @@
             # removal here is non-recursive to allow for fine-grained control
             if ( @dataSetsExplicitelyDisabled ){
                $self->zLog->info("Requesting removal of marked datasets: ". join( ", ", @dataSetsExplicitelyDisabled));
-               $self->zZfs->destroySnapshots(@dataSetsExplicitelyDisabled, 0);
+
+               # We need to explicitly call destroySnapshots for each dataSet we found
+               # because it is only designed to destroy sets of snapshots _for the same filesystem_.
+               # Since we are/have a top-level recursive snapshot, recurse here too.
+               for my $dataSetToDestroy (@dataSetsExplicitelyDisabled){
+                   $self->zZfs->destroySnapshots($dataSetToDestroy, 1);
+               }
            }
         }
     }

As a side note, it think

$self->zZfs->destroySnapshots(@dataSetsExplicitelyDisabled, 0);

should have been:

$self->zZfs->destroySnapshots(\@dataSetsExplicitelyDisabled, 0);

but my "Perl"-ish is rusty :)

Tobias Oetiker
@oetiker
Hi @tuxwielder mabe its better to log these on github and tag @jimklimov
tuxwielder
@tuxwielder
Hi Tobias, thought to discuss first, but sure will do :)
Tobias Oetiker
@oetiker
yea :) it seems that the whole recursive and include/exclude/override topic proves to be quite involved