kernel/drivers/virtio/virtio_ring.c, branch linux-6.9.y

virtio: packed: fix unmap leak for indirect desc table

2024-03-19T07:19:22Z

When use_dma_api and premapped are true, then the do_unmap is false. Because the do_unmap is false, vring_unmap_extra_packed is not called by detach_buf_packed. if (unlikely(vq->do_unmap)) { curr = id; for (i = 0; i < state->num; i++) { vring_unmap_extra_packed(vq, &vq->packed.desc_extra[curr]); curr = vq->packed.desc_extra[curr].next; } } So the indirect desc table is not unmapped. This causes the unmap leak. So here, we check vq->use_dma_api instead. Synchronously, dma info is updated based on use_dma_api judgment This bug does not occur, because no driver use the premapped with indirect. Fixes: b319940f83c2 ("virtio_ring: skip unmap for premapped") Signed-off-by: Xuan Zhuo Message-Id: <20240223071833.26095-1-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin

virtio_ring: fix syncs DMA memory with different direction

2023-12-04T13:36:25Z

Now the APIs virtqueue_dma_sync_single_range_for_{cpu,device} ignore the parameter 'dir', that is a mistake. [ 6.101666] ------------[ cut here ]------------ [ 6.102079] DMA-API: virtio-pci 0000:00:04.0: device driver syncs DMA memory with different direction [device address=0x00000000ae010000] [size=32752 bytes] [mapped with DMA_FROM_DEVICE] [synced with DMA_BIDIRECTIONAL] [ 6.103630] WARNING: CPU: 6 PID: 0 at kernel/dma/debug.c:1125 check_sync+0x53e/0x6c0 [ 6.107420] CPU: 6 PID: 0 Comm: swapper/6 Tainted: G E 6.6.0+ #290 [ 6.108030] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014 [ 6.108936] RIP: 0010:check_sync+0x53e/0x6c0 [ 6.109289] Code: 24 10 e8 f5 d9 74 00 4c 8b 4c 24 10 4c 8b 44 24 18 48 8b 4c 24 20 48 89 c6 41 56 4c 89 ea 48 c7 c7 b0 f1 50 82 e8 32 fc f3 ff <0f> 0b 48 c7 c7 48 4b 4a 82 e8 74 d9 fc ff 8b 73 4c 48 8d 7b 50 31 [ 6.110750] RSP: 0018:ffffc90000180cd8 EFLAGS: 00010092 [ 6.111178] RAX: 00000000000000ce RBX: ffff888100aa5900 RCX: 0000000000000000 [ 6.111744] RDX: 0000000000000104 RSI: ffffffff824c3208 RDI: 00000000ffffffff [ 6.112316] RBP: ffffc90000180d40 R08: 0000000000000000 R09: 00000000fffeffff [ 6.112893] R10: ffffc90000180b98 R11: ffffffff82f63308 R12: ffffffff83d5af00 [ 6.113460] R13: ffff888100998200 R14: ffffffff824a4b5f R15: 0000000000000286 [ 6.114027] FS: 0000000000000000(0000) GS:ffff88842fd80000(0000) knlGS:0000000000000000 [ 6.114665] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 6.115128] CR2: 00007f10f1e03030 CR3: 0000000108272004 CR4: 0000000000770ee0 [ 6.115701] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 6.116272] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 6.116842] PKRU: 55555554 [ 6.117069] Call Trace: [ 6.117275] [ 6.117452] ? __warn+0x84/0x140 [ 6.117727] ? check_sync+0x53e/0x6c0 [ 6.118034] ? __report_bug+0xea/0x100 [ 6.118353] ? check_sync+0x53e/0x6c0 [ 6.118653] ? report_bug+0x41/0xc0 [ 6.118944] ? handle_bug+0x3c/0x70 [ 6.119237] ? exc_invalid_op+0x18/0x70 [ 6.119551] ? asm_exc_invalid_op+0x1a/0x20 [ 6.119900] ? check_sync+0x53e/0x6c0 [ 6.120199] ? check_sync+0x53e/0x6c0 [ 6.120499] debug_dma_sync_single_for_cpu+0x5c/0x70 [ 6.120906] ? dma_sync_single_for_cpu+0xb7/0x100 [ 6.121291] virtnet_rq_unmap+0x158/0x170 [virtio_net] [ 6.121716] virtnet_receive+0x196/0x220 [virtio_net] [ 6.122135] virtnet_poll+0x48/0x1b0 [virtio_net] [ 6.122524] __napi_poll+0x29/0x1b0 [ 6.123083] net_rx_action+0x282/0x360 [ 6.123612] __do_softirq+0xf3/0x2fb [ 6.124138] __irq_exit_rcu+0x8e/0xf0 [ 6.124663] common_interrupt+0xbc/0xe0 [ 6.125202] We need to enable CONFIG_DMA_API_DEBUG and work with need sync mode(such as swiotlb) to reproduce this warn. Fixes: 8bd2f71054bd ("virtio_ring: introduce dma sync api for virtqueue") Reported-by: "Ning, Hongyu" Closes: https://lore.kernel.org/all/f37cb55a-6fc8-4e21-8789-46d468325eea@linux.intel.com/ Suggested-by: Jason Wang Signed-off-by: Xuan Zhuo Message-Id: <20231201033303.25141-1-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin Reviewed-by: Parav Pandit Acked-by: Jason Wang Tested-by: Hongyu Ning

virtio_ring: fix avail_wrap_counter in virtqueue_add_packed

2023-09-03T22:10:24Z

In current packed virtqueue implementation, the avail_wrap_counter won't flip, in the case when the driver supplies a descriptor chain with a length equals to the queue size; total_sg == vq->packed.vring.num. Let’s assume the following situation: vq->packed.vring.num=4 vq->packed.next_avail_idx: 1 vq->packed.avail_wrap_counter: 0 Then the driver adds a descriptor chain containing 4 descriptors. We expect the following result with avail_wrap_counter flipped: vq->packed.next_avail_idx: 1 vq->packed.avail_wrap_counter: 1 But, the current implementation gives the following result: vq->packed.next_avail_idx: 1 vq->packed.avail_wrap_counter: 0 To reproduce the bug, you can set a packed queue size as small as possible, so that the driver is more likely to provide a descriptor chain with a length equal to the packed queue size. For example, in qemu run following commands: sudo qemu-system-x86_64 \ -enable-kvm \ -nographic \ -kernel "path/to/kernel_image" \ -m 1G \ -drive file="path/to/rootfs",if=none,id=disk \ -device virtio-blk,drive=disk \ -drive file="path/to/disk_image",if=none,id=rwdisk \ -device virtio-blk,drive=rwdisk,packed=on,queue-size=4,\ indirect_desc=off \ -append "console=ttyS0 root=/dev/vda rw init=/bin/bash" Inside the VM, create a directory and mount the rwdisk device on it. The rwdisk will hang and mount operation will not complete. This commit fixes the wrap counter error by flipping the packed.avail_wrap_counter, when start of descriptor chain equals to the end of descriptor chain (head == i). Fixes: 1ce9e6055fa0 ("virtio_ring: introduce packed ring support") Signed-off-by: Yuan Yao Message-Id: <20230808051110.3492693-1-yuanyaogoog@chromium.org> Acked-by: Jason Wang Signed-off-by: Michael S. Tsirkin

virtio_ring: introduce dma sync api for virtqueue

2023-09-03T22:10:23Z

These API has been introduced: * virtqueue_dma_need_sync * virtqueue_dma_sync_single_range_for_cpu * virtqueue_dma_sync_single_range_for_device These APIs can be used together with the premapped mechanism to sync the DMA address. Signed-off-by: Xuan Zhuo Message-Id: <20230810123057.43407-12-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin

virtio_ring: introduce dma map api for virtqueue

2023-09-03T22:10:23Z

Added virtqueue_dma_map_api* to map DMA addresses for virtual memory in advance. The purpose is to keep memory mapped across multiple add/get buf operations. Signed-off-by: Xuan Zhuo Message-Id: <20230810123057.43407-11-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin

virtio_ring: introduce virtqueue_reset()

2023-09-03T22:10:23Z

Introduce virtqueue_reset() to release all buffer inside vq. Signed-off-by: Xuan Zhuo Acked-by: Jason Wang Message-Id: <20230810123057.43407-10-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin

virtio_ring: separate the logic of reset/enable from virtqueue_resize

2023-09-03T22:10:23Z

The subsequent reset function will reuse these logic. Signed-off-by: Xuan Zhuo Acked-by: Jason Wang Message-Id: <20230810123057.43407-9-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin

virtio_ring: correct the expression of the description of virtqueue_resize()

2023-09-03T22:10:23Z

Modify the "useless" to a more accurate "unused". Signed-off-by: Xuan Zhuo Acked-by: Jason Wang Message-Id: <20230810123057.43407-8-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin

virtio_ring: skip unmap for premapped

2023-09-03T22:10:23Z

Now we add a case where we skip dma unmap, the vq->premapped is true. We can't just rely on use_dma_api to determine whether to skip the dma operation. For convenience, I introduced the "do_unmap". By default, it is the same as use_dma_api. If the driver is configured with premapped, then do_unmap is false. So as long as do_unmap is false, for addr of desc, we should skip dma unmap operation. Signed-off-by: Xuan Zhuo Message-Id: <20230810123057.43407-7-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin

virtio_ring: introduce virtqueue_dma_dev()

2023-09-03T22:10:23Z

Added virtqueue_dma_dev() to get DMA device for virtio. Then the caller can do dma operation in advance. The purpose is to keep memory mapped across multiple add/get buf operations. Signed-off-by: Xuan Zhuo Acked-by: Jason Wang Message-Id: <20230810123057.43407-6-xuanzhuo@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin