Skip to main content

peertube - Link to source

Using rsync for backups, because it's not shiny and new

You might not even like rsync. Yeah it's old. Yeah it's slow. But if you're working with Linux you're going to need to know it.

In this video I walk through my favorite everyday flags for rsync.

Support the channel:
patreon.com/VeronicaExplains
ko-fi.com/VeronicaExplains
thestopbits.bandcamp.com

Here's a companion blog post, where I cover a bit more detail: vkc.sh/everyday-rsync

Also, @BreadOnPenguins made an awesome rsync video and you should check it out:

Lastly, I left out all of the ssh setup stuff because I made a video about that and the blog post goes into a smidge more detail. If you want to see a video covering the basics of using SSH, I made one a few years ago and it's still pretty good:

Chapters:
1:18 Invoking rsync
4:05 The --delete flag for rsync
5:30 Compression flag: -z
6:02 Using tmux and rsync together
6:30 but Veronica... why not use (insert shiny object here)

Return From Interrupt, by The Stop Bits

5 track album
The Stop Bits
#Linux #sysadmin #rsync #homelab #Self-Hosting
This entry was edited (10 hours ago)

reshared this

in reply to vkc via PeerTube

mausmalone
mastodon - Link to source
mausmalone
• •
shoot - that reminds me I need to set up rsync again to copy my DAS to the NAS. And as a fallback in case deja-dup for my home directory ever fails.
in reply to vkc via PeerTube

Nimona RESISTS
mastodon - Link to source
Nimona RESISTS
• •

rsync is great for syncing 2 directories. Have you heard of rsnapshot? It's a perl script that runs rsync, and uses hard links (if available) to have multiple old versions available. You can have an hourly backup, a daily backup, a weekly backup, and if you cd into the backups, you can see what the directories looked like at that time. It's neat!

As always, thanks for the great video!

rsnapshot.org/

#linux #backup

rsnapshot | rsnapshot

rsync-based backup utility
rsnapshot.org
#backup #Linux
in reply to vkc via PeerTube

fyrfaras
mastodon - Link to source
fyrfaras
• •
Thank you for this lovely video ^_^ I will try to use rsync instead of sftp in my daily use. The --dry-run option is pretty cool. Looking forward to your next videos :3
in reply to vkc via PeerTube

WTL
mastodon - Link to source
WTL
• •
Great video. Rsync is a tool I use all the time. Super handy, and I know it will work.
in reply to vkc via PeerTube

mausmalone
mastodon - Link to source
mausmalone
• •

Just a quick question - when you use a shell script to copy data to vkc@backup.lan - does that user just not have a password?

Having to specify the username and password and mount through fstab has always been my problem with rsync backups - but that's also because I really only know SMB.

in reply to mausmalone

Luna Lactea
glitchsoc - Link to source
Luna Lactea
• •
You can set up a key pair for logging in to that user, then disable password authentication for that user so you don't get a password prompt when trying to log in. That way you don't have to deal with a password prompt but there is still key based authentication to keep random people from logging in. Specify that host in ~/.ssh/config like this:
Host [Host nickname]<br>        User [username]<br>        Hostname [IP address or domain name]<br>        IdentityFile [file containing private key]<br>

Once this is done, any time you try to connect to the remote host, you can simply do ssh [Host nickname] without having to specify which user & without having to type in a password. Your SSH client will read ~/.ssh/config & assume that you meant [username]. You can still log in as other users by specifying them as normal, & you'll still be able to log in with a password for those users unless you decide to also set up key based authentication for them & disable password authentication for them.
in reply to Luna Lactea

Luna Lactea
glitchsoc - Link to source
Luna Lactea
• •
I have password authentication disabled for my entire system & key based authentication required for all users. It's convenient because no one needs to type their password, so they can automate things like backups, & it's probably more secure too.
in reply to Luna Lactea

Luna Lactea
glitchsoc - Link to source
Luna Lactea
• •
Here's a step by step guide on how to do this.
digitalocean.com/community/tut…

How to Create an SSH Key in Linux: Easy Step-by-Step Guide

Learn how to generate SSH keys in Linux with our detailed guide. Includes step-by-step instructions, troubleshooting tips, and practical examples for secure connections.
Justin Ellingwood (DigitalOcean)
in reply to Luna Lactea

mausmalone
mastodon - Link to source
mausmalone
• •
awesome! I don't think I can do this with my WD MyCloud but it's like 10 years old now and maybe it's time to replace it with a simple Debian server.
in reply to vkc via PeerTube

dbc5radio
peertube - Link to source
dbc5radio
• •
I've never used Rsync I'm very interested what I've missed out on.
in reply to vkc via PeerTube

William B Peckham
mastodon - Link to source
William B Peckham
• •
I have used our sync for a couple of decades, I guess, but never for backups. I've always depended on backup and archive software for that. Love BURP. Used ADSM back in the day, but prefer FOSS solutions now. Rsync is sync software, not a backup solution. I need generational backups with point in time recovery, and real backup solutions give me that.
Unknown parent

eleijeep
piefed - Link to source
eleijeep
• •

if your cp fails for any reason then you're not going to be able to resume the transfer in an efficient way.

rsync makes resuming an interrupted transfer very easy, because in most cases you can just re-run the same command and it will not re-transfer files which were already successfully completed.

in reply to vkc via PeerTube

Iαη 🍺
mastodon - Link to source
Iαη 🍺
• •
For those who don't want to use the command line, there is a GUI version, Grsync. It helps you set all the options and folders without needing to research and memorise it all. And you can keep sync sessions for repeated use.
This GUI could help avoid some from giving up on Linux.
in reply to vkc via PeerTube

Evan Freyer
mastodon - Link to source
Evan Freyer
• •
Thanks for the awesome video! Is rsync a viable option to automatically upload / sync a website generated by a SSG to a webhost?
in reply to vkc via PeerTube

gigachad
piefed - Link to source
gigachad
• •
Just came here to say I love your videos about Linux commands. They are short, concise, and I am actually able to memorize Linux commands lol. And it's so refreshing to not listen to the stereotype male Linux nerd. Keep up the good work, cheers from PieFed!
in reply to vkc via PeerTube

antoniovr
mastodon - Link to source
antoniovr
• •
It is one of the tools I usually used. Usually I use #unison to synchronise "manually" folders on different computers (I could make it automatically but I prefer to have the control and check it before proceeding), #rsync to back up/update usually in a single direction and finally #syncthing to automatically sync things like pictures, org agenda files and so on. I look forward to the rclone video, that I hadn't used it and perhaps I'll try
#rsync #syncthing #unison
in reply to vkc via PeerTube

Calandiel
mastodon - Link to source
Calandiel
• •
One of the best channels on peertube, I hope to follow for many years to come 🙂
in reply to vkc via PeerTube

Jonathan Lamothe
friendica - Link to source
Jonathan Lamothe
• •
@vkc via PeerTube I love rsync and use it daily. For my off-site backups though, I use borgbackup because I don't want to hand my data over unencrypted to a third party. Can rsync be made to do this?
@vkc via PeerTube
in reply to vkc via PeerTube

Mina
mastodon - Link to source
Mina
• •

Excellent!

Haven't used it in a long while, but just set up my new backup strategy alongside your tutorial.

⇧