[Devel] [PATCH RHEL8 COMMIT] dm: Interpreter of ploop1 format (ploop driver)

[Konstantin Khorenko]

Applied, thank you!

--
Best regards,

Konstantin Khorenko,
Virtuozzo Linux Kernel Team

On 10/17/2019 03:22 PM, Kirill Tkhai wrote:
> The idea is to add simple layer on top of block device,
> which dispatches the requests regarding of ploop1 format.
> The most bios are dispatched by ploop_map(), that just
> replaces bio's target cluster with the corresponding
> bat_entries[cluster] value.
>
> In case of a cluster is not presented in BAT, kwork
> is waked up, and it serves all the work of allocation
> of a new cluster and updating BAT. Requests going
> to RO deltas are served by the kwork too (since it's
> impossible to start aio from ploop_map() context).
>
> Service operations are also made from kwork, so often
> we may avoid additional synchronization between them
> and cluster allocation code. See do_ploop_work() for
> details of all of this.
>
> ----
> Full backup, snapshots attaching and merging, tracking, cluster COW
> and device resize are implemented.
>
> TODO: partial backup and preemption of cached bat.
>
> ----
> 1)Start:
> losetup /dev/loop0 delta_N+1.img  (current image)
> dmsetup create dm_ploop --table "0 <size_in_sectors> ploop <cluster_log> /dev/loop0"
> ./add_delta.sh delta_0.img   (oldest snapshot)
> ...
> ./add_delta.sh delta_N.img   (newest snapshot)
> mount /dev/mapper/dm_ploop /mount_point
>
> Where: add_delta.sh is:
>
> #!/bin/bash
> if [ "$#" -ne 1 ];
> then
>         echo "Wrong arguments number"
>         exit 1
> fi
> if [ ! -f "$1" ]; then
>         echo "$1 does not exist"
>         exit 1
> fi
> exec {var}<$1
> dmsetup message dm_ploop 0 add_delta $var
>
> (add_delta requires open fd of new delta. This bash script
>  opens the file and passes fd to dmsetup util).
>
> 2)Start in second way. You may pass deltas fd to dmsetup create (extern bash script is required -- not attached):
>
> losetup /dev/loop0 delta_N+1.img  (current image)
> dmsetup create dm_ploop --table "0 <size_in_sectors> ploop <cluster_log> /dev/loop0 <delta_0_fd> .. <delta_N_fd>"
> mount /dev/mapper/dm_ploop /mount_point
>
> 3)Resize of running device.
>
> dmsetup message dm_ploop 0 resize <new_size_in_sectors>
> dmsetup suspend dm_ploop
> dmsetup reload dm_ploop --table "0 <new_size_in_sectors> ploop <cluster_log> /dev/loop0 <delta_0_fd> .. <delta_N_fd>"
> dmsetup resume dm_ploop
>
> (Currently dm-core can't change size of mapped device w/o reloading --
>  despite there is no strict limitations of this -- it should be implemented).
>
> 4)Merge of latest snapshot forward
>
> $dmsetup message dm_ploop 0 merge
>
> 5)Notify that userspace merged intermediate snapshot
>
> $dmsetup message dm_ploop 0 notify_merged_forward <index>
>
> or
>
> $dmsetup message dm_ploop 0 notify_merged_backward <index>
>
> 6)Snapshot (switch top_delta)
>
> $dmsetup message dm_ploop 0 snapshot <old_top_delta_fd> <loop_attached_to_new_top_delta>
>
> 7)Tracking
>
> $dmsetup message dm_ploop 0 tracking_start
> $dmsetup message dm_ploop 0 tracking_stop
> $dmsetup message dm_ploop 0 tracking_get_next (returns cluster or -EAGAIN if no dirty blocks)
>
> 8)Update RO delta index
>
> $dmsetup message dm_ploop 0 update_delta_index <level> <map>
>
> where <map> is cluster1:dst_cluster1;cluster2:dst_cluster2;
>
> 9)Prohibit resume
>
> $dmsetup message dm_ploop 0 set_noresume 1
>
> (or set_noresume 0 to enable it back)
>
> 10)Flip upper delta
>
> Let we have Base.img and TopDelta.img
>
> $dmsetup message dm_ploop 0 flip_upper_deltas <loop-attached-to-Base.img> <TopDelta-fd>
>
> 11)Push backup
>
> Start:	$dmsetup message dm_ploop 0 push_backup_start <UUID> <addr_of_user_mask>
> 	(currenly only <addr_of_user_mask> == 0 is supported)
>
> Stop:	$dmsetup message dm_ploop 0 push_backup_stop <UUID>
>
> Get UUID: $dmsetup message dm_ploop 0 push_backup_get_uuid
>
> Read:	$dmsetup message dm_ploop 0 push_backup_read <UUID>
>
> Write:	$dmsetup message dm_ploop 0 push_backup_read <UUID> <cluster>
>
> Signed-off-by: Kirill Tkhai <ktkhai at virtuozzo.com>
> ---
>  drivers/md/Kconfig           |    6
>  drivers/md/Makefile          |    3
>  drivers/md/dm-ploop-bat.c    |  302 +++++++
>  drivers/md/dm-ploop-cmd.c    | 1615 +++++++++++++++++++++++++++++++++++++++
>  drivers/md/dm-ploop-map.c    | 1719 ++++++++++++++++++++++++++++++++++++++++++
>  drivers/md/dm-ploop-target.c |  318 ++++++++
>  drivers/md/dm-ploop.h        |  389 ++++++++++
>  7 files changed, 4352 insertions(+)
>  create mode 100644 drivers/md/dm-ploop-bat.c
>  create mode 100644 drivers/md/dm-ploop-cmd.c
>  create mode 100644 drivers/md/dm-ploop-map.c
>  create mode 100644 drivers/md/dm-ploop-target.c
>  create mode 100644 drivers/md/dm-ploop.h
>
> diff --git a/drivers/md/Kconfig b/drivers/md/Kconfig
> index 3db222509e44..46c55982133e 100644
> --- a/drivers/md/Kconfig
> +++ b/drivers/md/Kconfig
> @@ -529,6 +529,12 @@ config DM_INTEGRITY
>  	  To compile this code as a module, choose M here: the module will
>  	  be called dm-integrity.
>
> +config DM_PLOOP
> +	tristate "Ploop target support"
> +	depends on BLK_DEV_DM
> +	---help---
> +          This is ploop1 format interpreter on device-mapper rails.
> +
>  config DM_ZONED
>  	tristate "Drive-managed zoned block device target support"
>  	depends on BLK_DEV_DM

<skipped>