Articles Hierarchy

Articles Home » RPI » Image Backup

Apr 08 2018

Image Backup

Posted by kll · In RPI · 1039 Reads · Print

for RPI the Image Backup is a big question, not so much for your work ( privat files...)
but for system, setup: installs and tweaks, change of SD cards and USB system drives.

as i start usually from a Windows 7 PC, download RASPBIAN desktop .img.zip or NOOBS / PINN .zip
i am used to the win7 tools
++ SD Formatter (v5.0)
++ win32 disk imager (v1.0)
++ etcher (v1.3.1)
+++ guiformat32 and h2testw ( for special jobs )

i was used to use win32diskimager to read back SD cards to .img files and zip it.
but much changed my working style when i find the
RASPBIAN desktop menu SD card copier tool (piclone)
( or rpi_clone CLI version )
it copies a running system to a SD card or USB stick / makes it bootable,
works for smaller drives ( as long the content fits ) and for bigger drives ( with automatic enlarging the last partition )

i now have a MASTER SD card / my slowest 8GB SD card / with the RASPBIAN image, updated and customized.
and when i need a new system i boot that / update again / use SD card copier tool from desktop menu.
* * the 2 variable points are later to be changed manually:
RPI_IP: change in dhcpcd.conf eth0: 192.168.1.100 to 192.168.1.103 and wlan0: 192.168.1.200 to 192.168.1.203
name: change with sudo raspi-config RPI0 to RPI3..
reboot

but still wanting a .img file i read it back to PC by win32 disk imager. ( a 8GB file ) and zip it down.

all runs well, but at forum more people have problems with the windows tools
possible all related with the latest
windows 10 / anniversary update / to
Windows 10, version 1607
+ + now can read / make partitions on USB drives and SD cards
- - but the old tools what are used to see only the first partition / but dealing with the full drive
now not work any more?
( i only have win7 SP0, so its lot of guesswork what i say about windows 10 )
but see this interesting update:


one of the latest forum posts is here
so i start to play about creating a .img file by RPI instead by win32 disk imager.

-1- a pre condition is that you have that space, minimal the size of the content of the system drive. use
* example on a system on a 32GB USB stick:
df -Th and lsblk
used partition
pi@RPI3:~ $ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 29G 0 disk
├─sda1 8:1 1 41.8M 0 part /boot
└─sda2 8:2 1 28.9G 0 part /
pi@RPI3:~ $ df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/root ext4 29G 5.1G 22G 19% /
devtmpfs devtmpfs 434M 0 434M 0% /dev
tmpfs tmpfs 438M 0 438M 0% /dev/shm
tmpfs tmpfs 438M 12M 427M 3% /run
tmpfs tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs tmpfs 438M 0 438M 0% /sys/fs/cgroup
/dev/sda1 vfat 42M 22M 21M 52% /boot
tmpfs tmpfs 88M 0 88M 0% /run/user/1000
pi@RPI3:~ $

so minimal needed 6GB / a image file on windows by win32diskimager would be 32GB / .zip 2.2GB /

-2- DD
the official way with linux dd
-- on PC or RPI
-- with the source in a USB card reader ( offline )
-- with the target a big empty system SD card or a other USB drive
and i found some smart command...
do not use that dd command line
# the source SD card is in usb card reader, auto mounted, so first need
umount /dev/sdb1
umount /dev/sdb2
sudo dd bs=4M if=/dev/sdb | zip > MY_master_raspbian.img.zip
adding: -1915+1 records in
1915+1 records out
8035237888 bytes (8.0 GB, 7.5 GiB) copied, 2033.66 s, 4.0 MB/s
(deflated 67%)

copy to USB stick(cleaned by SDFormatter)
move USBstick to PC
try burn with etcher but that zip file contains a image file called "-",
so etcher not work!
unzip MY_master_raspbian.img.zip ( get "-" )
rename "-" to "MY_master_raspbian.img"
just to have full check:
zip again to MY_master_raspbian.img.zip
burn with etcher and boot in RPI OK

so need to test with a 2 commandline thing with separate .img and .img.zip again.
what i not have clear if the required space is different?
p.s. i got that command from official docu backup

-3- image-backup
here RONR introduces us to ?his? backup bash script
what can work on a running system and makes .img files.
so he give the warning:
Do NOT create the backup image file on the running SD card
i not find any homepage / related forum post /
also CLI image-backup -h --help not work
and i learned the hard way
-a- if the target is a USB stick in FAT32 the created .img file will only be possible up to 4GB
-b- if i make ( gparted) it a EXT4 file system the win7 PC can not read it
-c- if i later format it back to FAT32 ( gparted) and just store the zipped file ( under 4GB )
the windows PC still refuse to read it ( and want format the stick )
-d- the image-backup tool does only
RonRimage-backup assumes you're backing up a normal two-partition Raspbian system (fat + ext4/f2fs).
too late, i already trashed my NOOBS/PINN system camera tools setup.

but now i think i got it:
here at try 4th i do:
SOURCE: system SD 8GB ( my master RASPBIAN desktop ( from image / 2 partitions ) running
TARGET: 8GB USB stick (FAT32)
#___-a- format USB stick to ext4
#___-b- image-backup
#___-c- incremental backup
#___-d- look inside that backup .img file ( like for restore one file only )
#___-e- reconfig SAMBA so windows PC can read that .img file
#___-f- network transfer and zip to PC
#___-g- burn to 16GB SD card with etcher and boot in RPI and increase file system

8GB master SD boot again in RPI3cd /home/pi/projects/image_backup
# check on USB stick
# see /dev/sda1 vfat 7.5G 32K 7.5G 1% /media/pi/CLEAN
#___-a- format USB stick to ext4
pi@RPI0:~/projects/image_backup $ sudo umount /dev/sda1
pi@RPI0:~/projects/image_backup $ sudo mkfs.ext4 -F -L "CLEANext4" /dev/sda1
mke2fs 1.43.4 (31-Jan-2017)
/dev/sda1 contains a vfat file system labelled 'CLEAN'
Creating filesystem with 1954816 4k blocks and 489600 inodes
Filesystem UUID: 6937888d-1ad0-4c13-b79e-d42324965422
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

pi@RPI0:~/projects/image_backup $ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 7.5G 0 disk
└─sda1 8:1 1 7.5G 0 part
mmcblk0 179:0 0 7.5G 0 disk
├─mmcblk0p1 179:1 0 41.8M 0 part /boot
└─mmcblk0p2 179:2 0 7.3G 0 part /
pi@RPI0:~/projects/image_backup $

# usb stick replug

pi@RPI0:~/projects/image_backup $ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 7.5G 0 disk
└─sda1 8:1 1 7.5G 0 part /media/pi/CLEANext4
mmcblk0 179:0 0 7.5G 0 disk
├─mmcblk0p1 179:1 0 41.8M 0 part /boot
└─mmcblk0p2 179:2 0 7.3G 0 part /
pi@RPI0:~/projects/image_backup $
#___-b- image-backup
pi@RPI0:~/projects/image_backup $ sudo bash image-backup

Image file to create? /media/pi/CLEANext4/my_raspbian.img

Image ROOT filesystem size (MB) [7517]? 0

Requested image ROOT filesystem size (0) is too small (Minimum = 5473)

Image ROOT filesystem size (MB) [7517]? 7517

Create /media/pi/CLEANext4/my_raspbian.img [7517 MB] (y/n)? y

Starting full backup (for incremental backups, run: image-backup /media/pi/CLEANext4/my_raspbian.img)
pi@RPI0:~/projects/image_backup $

#(19min)

pi@RPI0:~/projects/image_backup $ ls -lh /media/pi/CLEANext4/
total 5.2G
drwx------ 2 root root 16K Apr 8 19:00 lost+found
-rw-r--r-- 1 root root 7.4G Apr 8 19:26 my_raspbian.img
pi@RPI0:~/projects/image_backup $

#___-c- incremental backup
# now i want test the incremental backup ( already making a little script )

pi@RPI0:~/projects/image_backup $ nano ibackup
pi@RPI0:~/projects/image_backup $ cat ibackup
#!/bin/bash
# add tool for incremental backup using image-backup
date
CMD='sudo bash /home/pi/projects/image_backup/image-backup /media/pi/CLEANext4/my_raspbian.img'
echo $CMD
$CMD
date
#end
pi@RPI0:~/projects/image_backup $

pi@RPI0:~/projects/image_backup $ chmod +x ibackup
pi@RPI0:~/projects/image_backup $ ./ibackup
Sun 8 Apr 19:36:14 +07 2018
sudo bash /home/pi/projects/image_backup/image-backup /media/pi/CLEANext4/my_raspbian.img
Sun 8 Apr 19:37:27 +07 2018
pi@RPI0:~/projects/image_backup $
#___-d- look inside that backup .img file ( like for restore one file only )
# now i want look inside that backup .img file if this script is stored there already ( as it is the one new file after full backup )
# but actually to check the in image-backup used rsync command i should also check on changed files and on deleted files
pi@RPI0:~/projects/image_backup $ cd /media/pi/CLEANext4/
pi@RPI0:/media/pi/CLEANext4 $ sudo losetup /dev/loop0 my_raspbian.img
pi@RPI0:/media/pi/CLEANext4 $ sudo losetup -a
/dev/loop0: [2049]:12 (/media/pi/CLEANext4/my_raspbian.img)
pi@RPI0:/media/pi/CLEANext4 $ sudo fdisk -l /dev/loop0
Disk /dev/loop0: 7.4 GiB, 7927234560 bytes, 15482880 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x93e79d45

Device Boot Start End Sectors Size Id Type
/dev/loop0p1 2048 88063 86016 42M c W95 FAT32 (LBA)
/dev/loop0p2 88064 15482879 15394816 7.3G 83 Linux

# you might need: sudo apt install kpartx

pi@RPI0:/media/pi/CLEANext4 $ sudo kpartx -l /dev/loop0
loop0p1 : 0 86016 /dev/loop0 2048
loop0p2 : 0 15394816 /dev/loop0 88064
pi@RPI0:/media/pi/CLEANext4 $ sudo kpartx -a /dev/loop0
pi@RPI0:/media/pi/CLEANext4 $ sudo file -sL /dev/mapper/loop0p1
/dev/mapper/loop0p1: DOS/MBR boot sector, code offset 0x3c+2, OEM-ID "mkfs.fat", sectors/cluster 4, reserved sectors 4, root entries 512, Media

descriptor 0xf8, sectors/FAT 84, sectors/track 32, heads 64, sectors 86016 (volumes > 32 MB) , serial number 0x60eaaba1, unlabeled, FAT (16 bit)
pi@RPI0:/media/pi/CLEANext4 $ sudo file -sL /dev/mapper/loop0p2
/dev/mapper/loop0p2: Linux rev 1.0 ext4 filesystem data, UUID=b2e10051-c433-4810-9a28-10365d32009a (extents) (64bit) (large files) (huge files)
pi@RPI0:/media/pi/CLEANext4 $ cd /tmp

pi@RPI0:/tmp $ mkdir imgrootfs
pi@RPI0:/tmp $ sudo mount /dev/mapper/loop0p2 /tmp/imgrootfs
pi@RPI0:/tmp $
pi@RPI0:/tmp $ ls -lh /tmp/imgrootfs/home/pi/projects/image_backup/
total 12K
-rwxr-xr-x 1 pi pi 191 Apr 8 19:36 ibackup
-rw-r--r-- 1 pi pi 6.7K Sep 11 2017 image-backup
pi@RPI0:/tmp $ cat /tmp/imgrootfs/home/pi/projects/image_backup/ibackup
#!/bin/bash
# add tool for incremental backup using image-backup
date
CMD='sudo bash /home/pi/projects/image_backup/image-backup /media/pi/CLEANext4/my_raspbian.img'
echo $CMD
$CMD
date
#end
pi@RPI0:/tmp $

# ok, that worked out, now shut that down:

pi@RPI0:/tmp $
pi@RPI0:/tmp $ sudo umount imgrootfs
umount: /tmp/imgrootfs: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)

#need to close the MC window looking into that mounted .img file

pi@RPI0:/tmp $ sudo umount imgrootfs
pi@RPI0:/tmp $ sudo kpartx -d /dev/loop0
pi@RPI0:/tmp $ sudo losetup -d /dev/loop0
pi@RPI0:/tmp $ rm -r imgrootfs
pi@RPI0:/tmp $

#___-e- reconfig SAMBA so windows PC can read that .img file
# last time i used zip from .img to (big) system SD it took 44min
# and here is no space at all
# new way:

pi@RPI0:~ $ sudo nano /etc/samba/smb.conf
# KLL share
[share]
path = /home/pi/Desktop/share
valid users = pi
read only = no
# KLL new added
[share_backupimg]
path = /media/pi/CLEANext4
valid users = pi
read only = yes

sudo reboot

#___-f- network transfer and zip to PC
# use win7PC explorer network RPI0
# or i use totalcommander zip ( incl network transfer LAN 30Mbps from RPI3 WIFI connection )

# wifi copy + zip + verify (37min)
#___-g- burn to 16GB SD card with etcher and boot in RPI and increase file system
# etcher1.3.1 burn (8GB) to 16GB card (15min)
# boot that in RPI3 OK
sudo raspi-config [7][A1] resize filesystem and reboot




but i consider that all as TEMPORARY:
i hope that one day there might be a option in SD card copier tool "create .img file"
what even would work for NOOBS/PINN systems.