In the Linux kernel, the following vulnerability has been resolved:<\/p>\n
mm\/huge_memory: fix huge_zero_pfn race<\/p>\n
Patch series “mm\/huge_memory: fix huge_zero_pfn race”, v2.<\/p>\n
There is a subtle race in the reference-counted huge_zero_folio
\nimplementation.<\/p>\n
The fast path atomic logic fails to account for the fact that the shrinker
\n(which drops the final huge_zero_refcount pin) can overwrite huge_zero_pfn
\nwith the ~0UL sentinel value in shrink_huge_zero_folio_scan() after a
\nracing get_huge_zero_folio() installed a valid value there.<\/p>\n
This results in huge_zero_folio being correctly set but huge_zero_pfn
\nbeing set incorrectly and thus is_huge_zero_pfn() and consequently
\nis_huge_zero_pmd() will misidentify the huge zero folio as being an
\nordinary THP folio.<\/p>\n
This can result in the huge zero folio being split and otherwise treated
\nincorrectly.<\/p>\n
The solution to this is very subtle as there is an atomic fast path, and
\nthus ordering in weakly ordered architectures has to be treated very
\ncarefully.<\/p>\n
The first commit fixes the issue by introducing a spinlock around
\nhuge_zero_[pfn, folio, refcount] write, with careful consideration paid to
\nload\/store ordering in the fast path. It is placed first and kept as
\nsmall as possible so that it can be backported on its own.<\/p>\n
The second commit is a pure cleanup which reworks the
\nCONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic to better separate the persistent
\nlogic from the dynamically allocated one.<\/p>\n
This patch (of 2):<\/p>\n
If !CONFIG_PERSISTENT_HUGE_ZERO_FOLIO, the huge_zero_folio is refcounted
\nby huge_zero_refcount and returned by mm_get_huge_zero_folio().<\/p>\n
When the caller is done with the huge zero page, its reference count is
\ndecremented. Only a shrinker can set the reference count to zero.<\/p>\n
A race can unfortunately occur between a shrinker decrementing the
\nreference count to zero and a concurrent page fault.<\/p>\n
This is because shrink_huge_zero_folio_scan() might, if very unlucky, be
\npreempted between setting huge_zero_refcount to zero and writing an
\ninvalid value.<\/p>\n
During this time get_huge_zero_folio() could write to huge_zero_pfn before
\nshrink_huge_zero_folio_scan() resumes.<\/p>\n
In this event the huge zero folio will be persistently misidentified
\ncausing the THP code path to be entered inappropriately for the huge zero
\nfolio:<\/p>\n
CPU 0 CPU 1
\n=======================================|=================================
\nshrink_huge_zero_folio_scan() |
\n atomic_cmpxchg() sets refcount to 0 |
\n xchg() sets huge_zero_folio to NULL | get_huge_zero_folio()
\n | | atomic_inc_not_zero() -> zero
\n preempted for a long time | Allocate new huge zero folio
\n | | Write valid huge_zero_folio
\n v | Write valid huge_zero_pfn
\n Overwrite huge_zero_pfn with ~0UL <— Invalid overwrite!<\/p>\n
This results in is_huge_zero_pfn() and is_huge_zero_pmd() incorrectly
\nreturning false for a huge zero page which could result in issues like the
\nhuge zero folio being incorrectly split.<\/p>\n
Note that the issue is with huge_zero_pfn not huge_zero_folio, as
\nget_huge_zero_folio() uses cmpxchg() gated on huge_zero_folio being NULL
\nwith a retry loop and shrink_huge_zero_folio_scan() uses xchg() to set
\nhuge_zero_folio.<\/p>\n
Fix the issue by introducing a spinlock, huge_zero_lock, to prevent
\nconcurrent write of huge_zero_folio, huge_zero_pfn and huge_zero_refcount.<\/p>\n
There needs to be significant care taken here to ensure correctness:<\/p>\n
The fast path in get_huge_zero_folio() uses atomic_inc_not_zero(), which
\nis outside of the critical section, and means huge zero allocation is
\ngated on zero huge_zero_refcount.<\/p>\n
The fast path doesn't use huge_zero_lock, so the critical section is
\nirrelevant to it.<\/p>\n
So invariants are required – huge_zero_refcount MUST:<\/p>\n
* Only be set in the huge_zero_lock critical section to ensure
\n serialisation of huge_zero_pfn, huge_zero_folio and
\n—truncated—<\/p>\n","protected":false},"excerpt":{"rendered":"
In the Linux kernel, the following vulnerability has been resolved:<\/p>\n
mm\/huge_memory: fix huge_zero_pfn race<\/p>\n
Patch series “mm\/huge_memory: fix huge_zero_pfn race”, v2.<\/p>\n
There is a subtle race in the r<\/p>\n","protected":false},"template":"","meta":[],"categories":[],"tags":[],"class_list":["post-11233","vuln","type-vuln","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.54iter.cn\/index.php?rest_route=\/wp\/v2\/vuln\/11233","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.54iter.cn\/index.php?rest_route=\/wp\/v2\/vuln"}],"about":[{"href":"https:\/\/www.54iter.cn\/index.php?rest_route=\/wp\/v2\/types\/vuln"}],"wp:attachment":[{"href":"https:\/\/www.54iter.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=11233"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.54iter.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=11233"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.54iter.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=11233"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}