Skip to content

Speculative page faults

Vinayak Menon linux-mm 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v11 Latest0 of 26 patches 18 repliesIn review

Patches in v11 No applies recorded yet

None of the 26 patches the cover letter announces are in this archive.

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v11 yet.

Read the v11 thread

Cover letter [PATCH v11 00/26] Speculative page faults

Hi Laurent,

We are observing an issue with speculative page fault with the following test code on ARM64 (4.14 kernel, 8 cores).

Steps:

1) Run the program

2) The 2 threads will try lock/unlock and prints messages, and main thread does fork. In around 1 minute time, lock/unlock threads will hang on pthread_mutex_lock.

Initially the issue was reported when ptrace was tried on apps. Later it was discovered that the write protect done by fork is causing the issue and the below test code was created.

Observations:

1) We have tried disabling SPF and the issue disappears.

2) Adding this piece of code in __handle_speculative_fault also fixes the problem.

if (flags & FAULT_FLAG_WRITE && !pte_write(vmf.orig_pte))

    return VM_FAULT_RETRY;

3) As an experiment we tried encapsulating handle_speculative_fault with down_read(mmap_sem) and that too fixes the problem.

4) It is observed that while in wp_page_copy, the contents of the old_page changes which should not ideally happen as the pte is !pte_write.

5) To prove that it is a race, we tried affining the threads to single core, and the issue disappears.

Let us know if you want us to try out any experiments.

Thanks,
Vinayak

/**test.c***/

#include <stdio.h>
#include <pthread.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>

#define UNUSED(expr) do { (void)(expr); } while(0) #define print printf

pthread_mutex_t m;
pthread_cond_t c;

static void* cond_do(void* arg)
{
  UNUSED(arg);
  do {
    print("%s:%d state=%u addr: %lx\n", __func__, __LINE__, *(uint32_t*)(&m), (unsigned long)&m); pthread_mutex_lock(&m);

    print("%s:%d state=%u addr: %lx\n", __func__, __LINE__, *(uint32_t*)(&m), (unsigned long)&m); pthread_mutex_unlock(&m);
    print("%s:%d state=%u addr: %lx\n", __func__, __LINE__, *(uint32_t*)(&m), (unsigned long)&m); } while (true);

  return NULL;
}

static void* sig_do(void* arg)
{
  UNUSED(arg);
  do {
    print("%s:%d state=%u addr: %lx\n", __func__, __LINE__, *(uint32_t*)(&m), (unsigned long)&m); pthread_mutex_lock(&m);

    print("%s:%d state=%u addr: %lx\n", __func__, __LINE__, *(uint32_t*)(&m), (unsigned long)&m); pthread_mutex_unlock(&m);
    print("%s:%d state=%u addr: %lx\n", __func__, __LINE__, *(uint32_t*)(&m), (unsigned long)&m); } while (true);
  return NULL;
}

int main()
{
  pthread_t sig;
  pthread_t cond;
  pthread_mutex_init(&m, NULL);

  pthread_create(&cond, NULL, cond_do, NULL);
  pthread_create(&sig, NULL, sig_do, NULL);

  while(1) {
      if (!fork()) {
          usleep(500);
          abort();
      }
      usleep(550);
  }

  pthread_join(sig, NULL);
  pthread_join(cond, NULL);

  return 0;
}