[Devel] [PATCH RHEL8 COMMIT] dm: Interpreter of ploop1 format (ploop driver)

[Konstantin Khorenko]

reverted, v3 will be committed

--
Best regards,

Konstantin Khorenko,
Virtuozzo Linux Kernel Team

On 10/18/2019 12:16 PM, Konstantin Khorenko wrote:
> Applied, thank you!
>
> --
> Best regards,
>
> Konstantin Khorenko,
> Virtuozzo Linux Kernel Team
>
> On 10/17/2019 03:22 PM, Kirill Tkhai wrote:
>> The idea is to add simple layer on top of block device,
>> which dispatches the requests regarding of ploop1 format.
>> The most bios are dispatched by ploop_map(), that just
>> replaces bio's target cluster with the corresponding
>> bat_entries[cluster] value.
>>
>> In case of a cluster is not presented in BAT, kwork
>> is waked up, and it serves all the work of allocation
>> of a new cluster and updating BAT. Requests going
>> to RO deltas are served by the kwork too (since it's
>> impossible to start aio from ploop_map() context).
>>
>> Service operations are also made from kwork, so often
>> we may avoid additional synchronization between them
>> and cluster allocation code. See do_ploop_work() for
>> details of all of this.
>>
>> ----
>> Full backup, snapshots attaching and merging, tracking, cluster COW
>> and device resize are implemented.
>>
>> TODO: partial backup and preemption of cached bat.
>>
>> ----
>> 1)Start:
>> losetup /dev/loop0 delta_N+1.img  (current image)
>> dmsetup create dm_ploop --table "0 <size_in_sectors> ploop <cluster_log> /dev/loop0"
>> ./add_delta.sh delta_0.img   (oldest snapshot)
>> ...
>> ./add_delta.sh delta_N.img   (newest snapshot)
>> mount /dev/mapper/dm_ploop /mount_point
>>
>> Where: add_delta.sh is:
>>
>> #!/bin/bash
>> if [ "$#" -ne 1 ];
>> then
>>         echo "Wrong arguments number"
>>         exit 1
>> fi
>> if [ ! -f "$1" ]; then
>>         echo "$1 does not exist"
>>         exit 1
>> fi
>> exec {var}<$1
>> dmsetup message dm_ploop 0 add_delta $var
>>
>> (add_delta requires open fd of new delta. This bash script
>>  opens the file and passes fd to dmsetup util).
>>
>> 2)Start in second way. You may pass deltas fd to dmsetup create (extern bash script is required -- not attached):
>>
>> losetup /dev/loop0 delta_N+1.img  (current image)
>> dmsetup create dm_ploop --table "0 <size_in_sectors> ploop <cluster_log> /dev/loop0 <delta_0_fd> .. <delta_N_fd>"
>> mount /dev/mapper/dm_ploop /mount_point
>>
>> 3)Resize of running device.
>>
>> dmsetup message dm_ploop 0 resize <new_size_in_sectors>
>> dmsetup suspend dm_ploop
>> dmsetup reload dm_ploop --table "0 <new_size_in_sectors> ploop <cluster_log> /dev/loop0 <delta_0_fd> .. <delta_N_fd>"
>> dmsetup resume dm_ploop
>>
>> (Currently dm-core can't change size of mapped device w/o reloading --
>>  despite there is no strict limitations of this -- it should be implemented).
>>
>> 4)Merge of latest snapshot forward
>>
>> $dmsetup message dm_ploop 0 merge
>>
>> 5)Notify that userspace merged intermediate snapshot
>>
>> $dmsetup message dm_ploop 0 notify_merged_forward <index>
>>
>> or
>>
>> $dmsetup message dm_ploop 0 notify_merged_backward <index>
>>
>> 6)Snapshot (switch top_delta)
>>
>> $dmsetup message dm_ploop 0 snapshot <old_top_delta_fd> <loop_attached_to_new_top_delta>
>>
>> 7)Tracking
>>
>> $dmsetup message dm_ploop 0 tracking_start
>> $dmsetup message dm_ploop 0 tracking_stop
>> $dmsetup message dm_ploop 0 tracking_get_next (returns cluster or -EAGAIN if no dirty blocks)
>>
>> 8)Update RO delta index
>>
>> $dmsetup message dm_ploop 0 update_delta_index <level> <map>
>>
>> where <map> is cluster1:dst_cluster1;cluster2:dst_cluster2;
>>
>> 9)Prohibit resume
>>
>> $dmsetup message dm_ploop 0 set_noresume 1
>>
>> (or set_noresume 0 to enable it back)
>>
>> 10)Flip upper delta
>>
>> Let we have Base.img and TopDelta.img
>>
>> $dmsetup message dm_ploop 0 flip_upper_deltas <loop-attached-to-Base.img> <TopDelta-fd>
>>
>> 11)Push backup
>>
>> Start:	$dmsetup message dm_ploop 0 push_backup_start <UUID> <addr_of_user_mask>
>> 	(currenly only <addr_of_user_mask> == 0 is supported)
>>
>> Stop:	$dmsetup message dm_ploop 0 push_backup_stop <UUID>
>>
>> Get UUID: $dmsetup message dm_ploop 0 push_backup_get_uuid
>>
>> Read:	$dmsetup message dm_ploop 0 push_backup_read <UUID>
>>
>> Write:	$dmsetup message dm_ploop 0 push_backup_read <UUID> <cluster>
>>
>> Signed-off-by: Kirill Tkhai <ktkhai at virtuozzo.com>
>> ---
>>  drivers/md/Kconfig           |    6
>>  drivers/md/Makefile          |    3
>>  drivers/md/dm-ploop-bat.c    |  302 +++++++
>>  drivers/md/dm-ploop-cmd.c    | 1615 +++++++++++++++++++++++++++++++++++++++
>>  drivers/md/dm-ploop-map.c    | 1719 ++++++++++++++++++++++++++++++++++++++++++
>>  drivers/md/dm-ploop-target.c |  318 ++++++++
>>  drivers/md/dm-ploop.h        |  389 ++++++++++
>>  7 files changed, 4352 insertions(+)
>>  create mode 100644 drivers/md/dm-ploop-bat.c
>>  create mode 100644 drivers/md/dm-ploop-cmd.c
>>  create mode 100644 drivers/md/dm-ploop-map.c
>>  create mode 100644 drivers/md/dm-ploop-target.c
>>  create mode 100644 drivers/md/dm-ploop.h
>>
>> diff --git a/drivers/md/Kconfig b/drivers/md/Kconfig
>> index 3db222509e44..46c55982133e 100644
>> --- a/drivers/md/Kconfig
>> +++ b/drivers/md/Kconfig
>> @@ -529,6 +529,12 @@ config DM_INTEGRITY
>>  	  To compile this code as a module, choose M here: the module will
>>  	  be called dm-integrity.
>>
>> +config DM_PLOOP
>> +	tristate "Ploop target support"
>> +	depends on BLK_DEV_DM
>> +	---help---
>> +          This is ploop1 format interpreter on device-mapper rails.
>> +
>>  config DM_ZONED
>>  	tristate "Drive-managed zoned block device target support"
>>  	depends on BLK_DEV_DM
>
> <skipped>
>