kernel/fs/jbd2/checkpoint.c, branch linux-5.1.y

jbd2: fix deadlock while checkpoint thread waits commit thread to finish

2019-02-01T04:42:11Z

This issue was found when I tried to put checkpoint work in a separate thread, the deadlock below happened: Thread1 | Thread2 __jbd2_log_wait_for_space | jbd2_log_do_checkpoint (hold j_checkpoint_mutex)| if (jh->b_transaction != NULL) | ... | jbd2_log_start_commit(journal, tid); |jbd2_update_log_tail | will lock j_checkpoint_mutex, | but will be blocked here. | jbd2_log_wait_commit(journal, tid); | wait_event(journal->j_wait_done_commit, | !tid_gt(tid, journal->j_commit_sequence)); | ... |wake_up(j_wait_done_commit) } | then deadlock occurs, Thread1 will never be waken up. To fix this issue, drop j_checkpoint_mutex in jbd2_log_do_checkpoint() when we are going to wait for transaction commit. Reviewed-by: Jan Kara Signed-off-by: Xiaoguang Wang Signed-off-by: Theodore Ts'o

jbd2: fix use after free in jbd2_log_do_checkpoint()

2018-10-05T22:44:40Z

The code cleaning transaction's lists of checkpoint buffers has a bug where it increases bh refcount only after releasing journal->j_list_lock. Thus the following race is possible: CPU0 CPU1 jbd2_log_do_checkpoint() jbd2_journal_try_to_free_buffers() __journal_try_to_free_buffer(bh) ... while (transaction->t_checkpoint_io_list) ... if (buffer_locked(bh)) { <-- IO completes now, buffer gets unlocked --> spin_unlock(&journal->j_list_lock); spin_lock(&journal->j_list_lock); __jbd2_journal_remove_checkpoint(jh); spin_unlock(&journal->j_list_lock); try_to_free_buffers(page); get_bh(bh) <-- accesses freed bh Fix the problem by grabbing bh reference before unlocking journal->j_list_lock. Fixes: dc6e8d669cf5 ("jbd2: don't call get_bh() before calling __jbd2_journal_remove_checkpoint()") Fixes: be1158cc615f ("jbd2: fold __process_buffer() into jbd2_log_do_checkpoint()") Reported-by: syzbot+7f4a27091759e2fe7453@syzkaller.appspotmail.com CC: stable@vger.kernel.org Reviewed-by: Lukas Czerner Signed-off-by: Jan Kara Signed-off-by: Theodore Ts'o

ext4: fix up remaining files with SPDX cleanups

2017-12-18T03:00:59Z

A number of ext4 source files were skipped due because their copyright permission statements didn't match the expected text used by the automated conversion utilities. I've added SPDX tags for the rest. While looking at some of these files, I've noticed that we have quite a bit of variation on the licenses that were used --- in particular some of the Red Hat licenses on the jbd2 files use a GPL2+ license, and we have some files that have a LGPL-2.1 license (which was quite surprising). I've not attempted to do any license changes. Even if it is perfectly legal to relicense to GPL 2.0-only for consistency's sake, that should be done with ext4 developer community discussion. Signed-off-by: Theodore Ts'o

block,fs: use REQ_* flags directly

2016-11-01T15:43:26Z

Remove the WRITE_* and READ_SYNC wrappers, and just use the flags directly. Where applicable this also drops usage of the bio_set_op_attrs wrapper. Signed-off-by: Christoph Hellwig Signed-off-by: Jens Axboe

jbd2: fix checkpoint list cleanup

2015-10-18T02:35:09Z

Unlike comments and expectation of callers journal_clean_one_cp_list() returned 1 not only if it freed the transaction but also if it freed some buffers in the transaction. That could make __jbd2_journal_clean_checkpoint_list() skip processing t_checkpoint_io_list and continue with processing the next transaction. This is mostly a cosmetic issue since the only result is we can sometimes free less memory than we could. But it's still worth fixing. Fix journal_clean_one_cp_list() to return 1 only if the transaction was really freed. Fixes: 50849db32a9f529235a84bcc84a6b8e631b1d0ec Signed-off-by: Jan Kara Signed-off-by: Theodore Ts'o Cc: stable@vger.kernel.org

jbd2: avoid infinite loop when destroying aborted journal

2015-07-28T18:57:14Z

Commit 6f6a6fda2945 "jbd2: fix ocfs2 corrupt when updating journal superblock fails" changed jbd2_cleanup_journal_tail() to return EIO when the journal is aborted. That makes logic in jbd2_log_do_checkpoint() bail out which is fine, except that jbd2_journal_destroy() expects jbd2_log_do_checkpoint() to always make a progress in cleaning the journal. Without it jbd2_journal_destroy() just loops in an infinite loop. Fix jbd2_journal_destroy() to cleanup journal checkpoint lists of jbd2_log_do_checkpoint() fails with error. Reported-by: Eryu Guan Tested-by: Eryu Guan Fixes: 6f6a6fda294506dfe0e3e0a253bb2d2923f28f0a Signed-off-by: Jan Kara Signed-off-by: Theodore Ts'o

jbd2: fix ocfs2 corrupt when updating journal superblock fails

2015-06-15T18:36:01Z

If updating journal superblock fails after journal data has been flushed, the error is omitted and this will mislead the caller as a normal case. In ocfs2, the checkpoint will be treated successfully and the other node can get the lock to update. Since the sb_start is still pointing to the old log block, it will rewrite the journal data during journal recovery by the other node. Thus the new updates will be overwritten and ocfs2 corrupts. So in above case we have to return the error, and ocfs2_commit_cache will take care of the error and prevent the other node to do update first. And only after recovering journal it can do the new updates. The issue discussion mail can be found at: https://oss.oracle.com/pipermail/ocfs2-devel/2015-June/010856.html http://comments.gmane.org/gmane.comp.file-systems.ext4/48841 [ Fixed bug in patch which allowed a non-negative error return from jbd2_cleanup_journal_tail() to leak out of jbd2_fjournal_flush(); this was causing xfstests ext4/306 to fail. -- Ted ] Reported-by: Yiwen Jiang Signed-off-by: Joseph Qi Signed-off-by: Theodore Ts'o Tested-by: Yiwen Jiang Cc: Junxiao Bi Cc: stable@vger.kernel.org

jbd2: use GFP_NOFS in jbd2_cleanup_journal_tail()

2015-06-15T04:18:02Z

jbd2_cleanup_journal_tail() can be invoked by jbd2__journal_start() So allocations should be done with GFP_NOFS [Full stack trace snipped from 3.10-rh7] [] dump_stack+0x19/0x1b [] warn_slowpath_common+0x61/0x80 [] warn_slowpath_null+0x1a/0x20 [] slab_pre_alloc_hook.isra.31.part.32+0x15/0x17 [] kmem_cache_alloc+0x55/0x210 [] ? mempool_alloc_slab+0x15/0x20 [] mempool_alloc_slab+0x15/0x20 [] mempool_alloc+0x69/0x170 [] ? _raw_spin_unlock_irq+0xe/0x20 [] ? finish_task_switch+0x5d/0x150 [] bio_alloc_bioset+0x1be/0x2e0 [] blkdev_issue_flush+0x99/0x120 [] jbd2_cleanup_journal_tail+0x93/0xa0 [jbd2] -->GFP_KERNEL [] jbd2_log_do_checkpoint+0x221/0x4a0 [jbd2] [] __jbd2_log_wait_for_space+0xa7/0x1e0 [jbd2] [] start_this_handle+0x2d8/0x550 [jbd2] [] ? __memcg_kmem_put_cache+0x29/0x30 [] ? kmem_cache_alloc+0x130/0x210 [] jbd2__journal_start+0xba/0x190 [jbd2] [] ? lru_cache_add+0xe/0x10 [] ? ext4_da_write_begin+0xf9/0x330 [ext4] [] __ext4_journal_start_sb+0x77/0x160 [ext4] [] ext4_da_write_begin+0xf9/0x330 [ext4] [] generic_file_buffered_write_iter+0x10c/0x270 [] __generic_file_write_iter+0x178/0x390 [] __generic_file_aio_write+0x8b/0xb0 [] generic_file_aio_write+0x5d/0xc0 [] ext4_file_write+0xa9/0x450 [ext4] [] ? pipe_read+0x379/0x4f0 [] do_sync_write+0x90/0xe0 [] vfs_write+0xbd/0x1e0 [] SyS_write+0x58/0xb0 [] system_call_fastpath+0x16/0x1b Signed-off-by: Dmitry Monakhov Signed-off-by: Theodore Ts'o Cc: stable@vger.kernel.org

jbd2: simplify calling convention around __jbd2_journal_clean_checkpoint_list

2014-09-18T04:58:12Z

__jbd2_journal_clean_checkpoint_list() returns number of buffers it freed but noone was using the value so just stop doing that. This also allows for simplifying the calling convention for journal_clean_once_cp_list(). Signed-off-by: Jan Kara Signed-off-by: Theodore Ts'o

jbd2: avoid pointless scanning of checkpoint lists

2014-09-18T04:42:16Z

Yuanhan has reported that when he is running fsync(2) heavy workload creating new files over ramdisk, significant amount of time is spent in __jbd2_journal_clean_checkpoint_list() trying to clean old transactions (but they cannot be cleaned up because flusher hasn't yet checkpointed those buffers). The workload can be generated by: fs_mark -d /fs/ram0/1 -D 2 -N 2560 -n 1000000 -L 1 -S 1 -s 4096 Reduce the amount of scanning by stopping to scan the transaction list once we find a transaction that cannot be checkpointed. Note that this way of cleaning is still enough to keep freeing space in the journal after fully checkpointed transactions. Reported-and-tested-by: Yuanhan Liu Signed-off-by: Jan Kara Signed-off-by: Theodore Ts'o