kernel/drivers/md/dm-verity-target.c, branch 0x221E-v0.0.1-v6.19

dm-verity: disable recursive forward error correction

2025-12-10T18:27:59Z

There are two problems with the recursive correction: 1. It may cause denial-of-service. In fec_read_bufs, there is a loop that has 253 iterations. For each iteration, we may call verity_hash_for_block recursively. There is a limit of 4 nested recursions - that means that there may be at most 253^4 (4 billion) iterations. Red Hat QE team actually created an image that pushes dm-verity to this limit - and this image just makes the udev-worker process get stuck in the 'D' state. 2. It doesn't work. In fec_read_bufs we store data into the variable "fio->bufs", but fio bufs is shared between recursive invocations, if "verity_hash_for_block" invoked correction recursively, it would overwrite partially filled fio->bufs. Signed-off-by: Mikulas Patocka Reported-by: Guangwu Zhang Reviewed-by: Sami Tolvanen Reviewed-by: Eric Biggers

dm verity fec: Expose corrected block count via status

2025-11-18T17:24:26Z

Enhance visibility into dm-verity Forward Error Correction (FEC) activity. While FEC can correct on-disk corruptions, the number of successful correction events is not readily exposed through a standard interface. This change integrates FEC statistics into the verity target's .status handler for STATUSTYPE_INFO. The info output now includes count of corrected block by FEC. The counter is a per-device instance atomic64_t, maintained within the struct dm_verity_fec, tracking blocks successfully repaired by FEC on this specific device instance since it was created. This approach aligns with the standard Device Mapper mechanism for targets to report runtime information, as used by other targets like dm-integrity. This patch also updates Documentation/admin-guide/device-mapper/verity.rst to reflect the new status information. Tested: Induced single-bit errors on a block device protected by dm-verity with FEC on android phone. Confirmed 'dmctl status ' on Android reports an incrementing 'fec_corrected_blocks' count after the corrupted blocks were accessed. Signed-off-by: Shubhankar Mishra Signed-off-by: Mikulas Patocka

dm-verity: use 2-way interleaved SHA-256 hashing when supported

2025-10-20T13:47:42Z

When the crypto library provides an optimized implementation of sha256_finup_2x(), use it to interleave the hashing of pairs of data blocks. On some CPUs this nearly doubles hashing performance. The increase in overall throughput of cold-cache dm-verity reads that I'm seeing on arm64 and x86_64 is roughly 35% (though this metric is hard to measure as it jumps around a lot). For now this is done only on data blocks, not Merkle tree blocks. We could use sha256_finup_2x() on Merkle tree blocks too, but that is less important as there aren't as many Merkle tree blocks as data blocks, and that would require some additional code restructuring. Signed-off-by: Eric Biggers Signed-off-by: Mikulas Patocka

dm-verity: reduce scope of real and wanted digests

2025-10-20T13:47:35Z

In preparation for supporting interleaved hashing where dm-verity will need to keep track of the real and wanted digests for multiple data blocks simultaneously, stop using the want_digest and real_digest fields of struct dm_verity_io from so many different places. Specifically: - Make various functions take want_digest as a parameter rather than having it be implicitly passed via the struct dm_verity_io. - Add a new tmp_digest field, and use this instead of real_digest when computing a digest solely for the purpose of immediately checking it. The result is that real_digest and want_digest are used only by verity_verify_io(). Signed-off-by: Eric Biggers Signed-off-by: Mikulas Patocka

dm-verity: use SHA-256 library for SHA-256

2025-10-20T13:47:27Z

When the hash algorithm is SHA-256 and the verity version is not 0, use the SHA-256 library instead of crypto_shash. This is a prerequisite for making dm-verity interleave the computation of SHA-256 hashes for increased performance. That optimization is available in the SHA-256 library but not in crypto_shash. Even without interleaved hashing, switching to the library also slightly improves performance by itself because it avoids the overhead of crypto_shash, including indirect calls and other API overhead. (Benchmark on x86_64, AMD Zen 5: hashing 4K blocks gets 2.1% faster.) SHA-256 is by far the most common hash algorithm used with dm-verity. It makes sense to optimize for the common case and fall back to the generic crypto layer for uncommon cases, as suggested by Linus: https://lore.kernel.org/r/CAHk-=wgp-fOSsZsYrbyzqCAfEvrt5jQs1jL-97Wc4seMNTUyng@mail.gmail.com Signed-off-by: Eric Biggers Signed-off-by: Mikulas Patocka

dm-verity: remove log message with shash driver name

2025-10-20T13:47:17Z

I added this log message in commit bbf6a566920e ("dm verity: log the hash algorithm implementation"), to help people debug issues where they forgot to enable the architecture-optimized SHA-256 code in their kconfig or accidentally enabled a slow hardware offload driver (such as QCE) that overrode the faster CPU-accelerated code. However: - The crypto layer now always enables the architecture-optimized SHA-1, SHA-256, and SHA-512 code. Moreover, for simplicity the driver name is now fixed at "sha1-lib", "sha256-lib", etc. - dm-verity now uses crypto_shash instead of crypto_ahash, preventing the mistake of accidentally using a slow driver such as QCE. Therefore, this log message generally no longer provides useful information. Remove it. Signed-off-by: Eric Biggers Signed-off-by: Mikulas Patocka

dm-verity: remove support for asynchronous hashes

2025-07-31T13:39:55Z

The support for asynchronous hashes in dm-verity has outlived its usefulness. It adds significant code complexity and opportunity for bugs. I don't know of anyone using it in practice. (The original submitter of the code possibly was, but that was 8 years ago.) Data I recently collected for en/decryption shows that using off-CPU crypto "accelerators" is consistently much slower than the CPU (https://lore.kernel.org/r/20250704070322.20692-1-ebiggers@kernel.org/), even on CPUs that lack dedicated cryptographic instructions. Similar results are likely to be seen for hashing. I already removed support for asynchronous hashes from fsverity two years ago, and no one ever complained. Moreover, neither dm-verity, fsverity, nor fscrypt has ever actually used the asynchronous crypto algorithms in a truly asynchronous manner. The lack of interest in such optimizations provides further evidence that it's only the CPU-based crypto that actually matters. Historically, it's also been common for people to forget to enable the optimized SHA-256 code, which could contribute to an off-CPU crypto engine being perceived as more useful than it really is. In 6.16 I fixed that: the optimized SHA-256 code is now enabled by default. Therefore, let's drop the support for asynchronous hashes in dm-verity. Tested with verity-compat-test. Acked-by: Ard Biesheuvel Signed-off-by: Eric Biggers Signed-off-by: Mikulas Patocka

dm-verity: fix a memory leak if some arguments are specified multiple times

2025-06-03T17:01:42Z

If some of the arguments "check_at_most_once", "ignore_zero_blocks", "use_fec_from_device", "root_hash_sig_key_desc" were specified more than once on the target line, a memory leak would happen. This commit fixes the memory leak. It also fixes error handling in verity_verify_sig_parse_opt_args. Signed-off-by: Mikulas Patocka Cc: stable@vger.kernel.org

dm: Allow .prepare_ioctl to handle ioctls directly

2025-05-04T09:35:05Z

This adds a 'bool *forward' parameter to .prepare_ioctl, which allows device mapper targets to accept ioctls to themselves instead of the underlying device. If the target already fully handled the ioctl, it sets *forward to false and device mapper won't forward it to the underlying device any more. In order for targets to actually know what the ioctl is about and how to handle it, pass also cmd and arg. As long as targets restrict themselves to interpreting ioctls of type DM_IOCTL, this is a backwards compatible change because previously, any such ioctl would have been passed down through all device mapper layers until it reached a device that can't understand the ioctl and would return an error. Signed-off-by: Kevin Wolf Reviewed-by: Benjamin Marzinski Signed-off-by: Mikulas Patocka

dm-verity: use softirq context only when !need_resched()

2025-05-04T09:35:05Z

Further limit verification in softirq (a.k.a. BH) context to cases where rescheduling of the interrupted task is not pending. This helps prevent the CPU from spending too long in softirq context. Note that handle_softirqs() in kernel/softirq.c already stops running softirqs in this same case. However, that check is too coarse-grained, since many I/O requests can be processed in a single BLOCK_SOFTIRQ. Signed-off-by: Eric Biggers Signed-off-by: Mikulas Patocka