kernel/drivers/net/ethernet/sfc/efx_channels.c, branch linux-5.14.y

sfc: add logs explaining XDP_TX/REDIRECT is not available

2021-07-13T17:02:41Z

If it's not possible to allocate enough channels for XDP, XDP_TX and XDP_REDIRECT don't work. However, only a message saying that not enough channels were available was shown, but not saying what are the consequences in that case. The user didn't know if he/she can use XDP or not, if the performance is reduced, or what. Signed-off-by: Íñigo Huguet Signed-off-by: David S. Miller

sfc: ensure correct number of XDP queues

2021-07-13T17:02:41Z

Commit 99ba0ea616aa ("sfc: adjust efx->xdp_tx_queue_count with the real number of initialized queues") intended to fix a problem caused by a round up when calculating the number of XDP channels and queues. However, this was not the real problem. The real problem was that the number of XDP TX queues had been reduced to half in commit e26ca4b53582 ("sfc: reduce the number of requested xdp ev queues"), but the variable xdp_tx_queue_count had remained the same. Once the correct number of XDP TX queues is created again in the previous patch of this series, this also can be reverted since the error doesn't actually exist. Only in the case that there is a bug in the code we can have different values in xdp_queue_number and efx->xdp_tx_queue_count. Because of this, and per Edward Cree's suggestion, I add instead a WARN_ON to catch if it happens again in the future. Note that the number of allocated queues can be higher than the number of used ones due to the round up, as explained in the existing comment in the code. That's why we also have to stop increasing xdp_queue_number beyond efx->xdp_tx_queue_count. Signed-off-by: Íñigo Huguet Signed-off-by: David S. Miller

sfc: fix lack of XDP TX queues - error XDP TX failed (-22)

2021-07-13T17:02:41Z

Fixes: e26ca4b53582 sfc: reduce the number of requested xdp ev queues The buggy commit intended to allocate less channels for XDP in order to be more unlikely to reach the limit of 32 channels of the driver. The idea was to use each IRQ/eventqeue for more XDP TX queues than before, calculating which is the maximum number of TX queues that one event queue can handle. For example, in EF10 each event queue could handle up to 8 queues, better than the 4 they were handling before the change. This way, it would have to allocate half of channels than before for XDP TX. The problem is that the TX queues are also contained inside the channel structs, and there are only 4 queues per channel. Reducing the number of channels means also reducing the number of queues, resulting in not having the desired number of 1 queue per CPU. This leads to getting errors on XDP_TX and XDP_REDIRECT if they're executed from a high numbered CPU, because there only exist queues for the low half of CPUs, actually. If XDP_TX/REDIRECT is executed in a low numbered CPU, the error doesn't happen. This is the error in the logs (repeated many times, even rate limited): sfc 0000:5e:00.0 ens3f0np0: XDP TX failed (-22) This errors happens in function efx_xdp_tx_buffers, where it expects to have a dedicated XDP TX queue per CPU. Reverting the change makes again more likely to reach the limit of 32 channels in machines with many CPUs. If this happen, no XDP_TX/REDIRECT will be possible at all, and we will have this log error messages: At interface probe: sfc 0000:5e:00.0: Insufficient resources for 12 XDP event queues (24 other channels, max 32) At every subsequent XDP_TX/REDIRECT failure, rate limited: sfc 0000:5e:00.0 ens3f0np0: XDP TX failed (-22) However, without reverting the change, it makes the user to think that everything is OK at probe time, but later it fails in an unpredictable way, depending on the CPU that handles the packet. It is better to restore the predictable behaviour. If the user sees the error message at probe time, he/she can try to configure the best way it fits his/her needs. At least, he/she will have 2 options: - Accept that XDP_TX/REDIRECT is not available (he/she may not need it) - Load sfc module with modparam 'rss_cpus' with a lower number, thus creating less normal RX queues/channels, letting more free resources for XDP, with some performance penalty. Anyway, let the calculation of maximum TX queues that can be handled by a single event queue, and use it only if it's less than the number of TX queues per channel. This doesn't happen in practice, but could happen if some constant values are tweaked in the future, such us EFX_MAX_TXQ_PER_CHANNEL, EFX_MAX_EVQ_SIZE or EFX_MAX_DMAQ_SIZE. Related mailing list thread: https://lore.kernel.org/bpf/20201215104327.2be76156@carbon/ Signed-off-by: Íñigo Huguet Signed-off-by: David S. Miller

sfc: adjust efx->xdp_tx_queue_count with the real number of initialized queues

2021-04-27T22:38:29Z

efx->xdp_tx_queue_count is initially initialized to num_possible_cpus() and is later used to allocate and traverse efx->xdp_tx_queues lookup array. However, we may end up not initializing all the array slots with real queues during probing. This results, for example, in a NULL pointer dereference, when running "# ethtool -S ", similar to below [2570283.664955][T4126959] BUG: kernel NULL pointer dereference, address: 00000000000000f8 [2570283.681283][T4126959] #PF: supervisor read access in kernel mode [2570283.695678][T4126959] #PF: error_code(0x0000) - not-present page [2570283.710013][T4126959] PGD 0 P4D 0 [2570283.721649][T4126959] Oops: 0000 [#1] SMP PTI [2570283.734108][T4126959] CPU: 23 PID: 4126959 Comm: ethtool Tainted: G O 5.10.20-cloudflare-2021.3.1 #1 [2570283.752641][T4126959] Hardware name: [2570283.781408][T4126959] RIP: 0010:efx_ethtool_get_stats+0x2ca/0x330 [sfc] [2570283.796073][T4126959] Code: 00 85 c0 74 39 48 8b 95 a8 0f 00 00 48 85 d2 74 2d 31 c0 eb 07 48 8b 95 a8 0f 00 00 48 63 c8 49 83 c4 08 83 c0 01 48 8b 14 ca <48> 8b 92 f8 00 00 00 49 89 54 24 f8 39 85 a0 0f 00 00 77 d7 48 8b [2570283.831259][T4126959] RSP: 0018:ffffb79a77657ce8 EFLAGS: 00010202 [2570283.845121][T4126959] RAX: 0000000000000019 RBX: ffffb799cd0c9280 RCX: 0000000000000018 [2570283.860872][T4126959] RDX: 0000000000000000 RSI: ffff96dd970ce000 RDI: 0000000000000005 [2570283.876525][T4126959] RBP: ffff96dd86f0a000 R08: ffff96dd970ce480 R09: 000000000000005f [2570283.892014][T4126959] R10: ffffb799cd0c9fff R11: ffffb799cd0c9000 R12: ffffb799cd0c94f8 [2570283.907406][T4126959] R13: ffffffffc11b1090 R14: ffff96dd970ce000 R15: ffffffffc11cd66c [2570283.922705][T4126959] FS: 00007fa7723f8740(0000) GS:ffff96f51fac0000(0000) knlGS:0000000000000000 [2570283.938848][T4126959] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [2570283.952524][T4126959] CR2: 00000000000000f8 CR3: 0000001a73e6e006 CR4: 00000000007706e0 [2570283.967529][T4126959] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [2570283.982400][T4126959] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [2570283.997308][T4126959] PKRU: 55555554 [2570284.007649][T4126959] Call Trace: [2570284.017598][T4126959] dev_ethtool+0x1832/0x2830 Fix this by adjusting efx->xdp_tx_queue_count after probing to reflect the true value of initialized slots in efx->xdp_tx_queues. Signed-off-by: Ignat Korchagin Fixes: e26ca4b53582 ("sfc: reduce the number of requested xdp ev queues") Cc: # 5.12.x Signed-off-by: David S. Miller

sfc: reduce the number of requested xdp ev queues

2021-01-23T03:16:45Z

Without this change the driver tries to allocate too many queues, breaching the number of available msi-x interrupts on machines with many logical cpus and default adapter settings: Insufficient resources for 12 XDP event queues (24 other channels, max 32) Which in turn triggers EINVAL on XDP processing: sfc 0000:86:00.0 ext0: XDP TX failed (-22) Signed-off-by: Ivan Babrou Acked-by: Edward Cree Link: https://lore.kernel.org/r/20210120212759.81548-1-ivan@cloudflare.com Signed-off-by: Jakub Kicinski

sfc: decouple TXQ type from label

2020-09-12T00:15:22Z

Make it possible to have an arbitrary mapping from types to labels, because when we add inner-csum-offload TXQs there will no longer be a convenient nesting hierarchy of NIC types (EF10 will have inner-csum TXQs, while Siena will have HIGHPRI). Correct a misleading comment on efx_hard_start_xmit(). Signed-off-by: Edward Cree Signed-off-by: David S. Miller

sfc: cleanups around efx_alloc_channel

2020-09-11T21:55:14Z

The old_channel argument is never used, so remove it. The function is only called from elsewhere in efx_channels.c, so make it static and remove the declaration from the header file. Signed-off-by: Edward Cree Signed-off-by: David S. Miller

sfc: assign TXQs without gaps

2020-07-02T21:47:40Z

Since we only allocate VIs for the number of TXQs we actually need, we cannot naively use "channel * TXQ_TYPES + txq" for the TXQ number, as this has gaps (when efx->tx_queues_per_channel < EFX_TXQ_TYPES) and thus overruns the driver's VI allocations, causing the firmware to reject the MC_CMD_INIT_TXQ based on INSTANCE. Thus, we distinguish INSTANCE (stored in tx_queue->queue) from LABEL (tx_queue->label); the former is allocated starting from 0 in efx_set_channels(), while the latter is simply the txq type (index in channel->tx_queue array). To simplify things, rather than changing tx_queues_per_channel after setting up TXQs, make Siena always probe its HIGHPRI queues at start of day, rather than deferring it until tc mqprio enables them. Signed-off-by: Edward Cree Signed-off-by: David S. Miller

sfc: commonise netif_set_real_num[tr]x_queues calls

2020-07-02T21:47:40Z

While we're at it, also check them for failure. Signed-off-by: Edward Cree Signed-off-by: David S. Miller

sfc: make tx_queues_per_channel variable at runtime

2020-07-02T21:47:40Z

Siena needs four TX queues (csum * highpri), EF10 needs two (csum), and EF100 only needs one (as checksumming is controlled entirely by the transmit descriptor). Rather than having various bits of ad-hoc code to decide which queues to set up etc., put the knowledge of how many TXQs a channel has in one place. Signed-off-by: Edward Cree Signed-off-by: David S. Miller