Handle kernel page faults using task gate
Subsystems (7)
I am trying to handle page faults exceptions in the kernel using the task gate mechanism. I succeeded to transfer the execution to my page fault handler using a new TSS and updates to the GDT and IDT tables (similar to the double fault mechanism in 2.6). After handling the fault and allocating the physical page I use the iret instruction to switch back to the previous task. The problem is that I got a double fault with the same address that cause the fault (although the physical page is allocated and mapped). Any clues?
The new page fault TSS:
struct tss_struct pagefault_tss __cacheline_aligned = {
.esp0 = STACK_START,
.ss0 = __KERNEL_DS,
.ldt = 0, .io_bitmap_base = INVALID_IO_BITMAP_OFFSET,
.eip = (unsigned long) pagefault_fn,
.eflags = X86_EFLAGS_SF|0x2,/* 0x2 bit is always set */
.esp = STACK_START,
.es = __USER_DS,
.cs = __KERNEL_CS,
.ss = __KERNEL_DS,
.ds = __USER_DS,
.__cr3 = __pa(swapper_pg_dir)
};
Quoted 8 lines · eliad lubovsky
I am trying to handle page faults exceptions in the kernel using the task gate mechanism. I succeeded to transfer the execution to my page fault handler using a new TSS and updates to the GDT and IDT tables (similar to the double fault mechanism in 2.6). After handling the fault and allocating the physical page I use the iret instruction to switch back to the previous task. The problem is that I got a double fault with the same address that cause the fault (although the physical page is allocated and mapped). Any clues?
are you clearing the 'nested task' (NT) flag of the new TSS once you have switched to it?
Ingo
"...The NT flag indicates that the previous task link field of the TSS has been loaded with a saved TSS segment selector. If software uses an IRET instruction to suspend the new task, the processor uses the value in the previous task link field and the NT flag to return to the previous task; that is, if the NT flag is set, the processor performs a task switch to the task specified in the previous task link field."
"If the task switch was initiated with an exception, or an interrupt, the processor sets the NT flag in the EFLAGS image stored in the new task’s TSS;"
Intel Architecture Software Developer’s Manual Volume 3: System Programming
Not sure I need to clear the NT flag.
my page fault handler:
static void pagefault_fn(void)
{
unsigned int address, aligned_page_fault_address; struct vm_struct *area;
/* retrieve the page fault address */
__asm__("movl %%cr2,%0":"=r" (address));
aligned_page_fault_address = ((address+PAGE_SIZE)&(~(4096-1))); area = find_vm_area((void*)(aligned_page_fault_address));
/* allocate a new physical page, expand the stack size */ expend_stack_size(area);
// asm ("pushf; orl $0x00004000, (%esp); popf; iret"); /* sets NT */
// asm ("pushf; andl $0xffffbfff, (%esp); popf; iret"); /* clears NT */ asm ("iret");
}
Quoted 15 lines · Ingo Molnar
Quoted 8 lines · eliad lubovsky
I am trying to handle page faults exceptions in the kernel using the task gate mechanism. I succeeded to transfer the execution to my page fault handler using a new TSS and updates to the GDT and IDT tables (similar to the double fault mechanism in 2.6). After handling the fault and allocating the physical page I use the iret instruction to switch back to the previous task. The problem is that I got a double fault with the same address that cause the fault (although the physical page is allocated and mapped). Any clues?
are you clearing the 'nested task' (NT) flag of the new TSS once you have switched to it?
Ingo
Quoted 11 lines · Ingo Molnar
Quoted 8 lines · eliad lubovsky
I am trying to handle page faults exceptions in the kernel using the task gate mechanism. I succeeded to transfer the execution to my page fault handler using a new TSS and updates to the GDT and IDT tables (similar to the double fault mechanism in 2.6). After handling the fault and allocating the physical page I use the iret instruction to switch back to the previous task. The problem is that I got a double fault with the same address that cause the fault (although the physical page is allocated and mapped). Any clues?
Quoted 2 lines
are you clearing the 'nested task' (NT) flag of the new TSS once you have switched to it?
correction - i meant the 'busy' bit in the TSS descriptor. It's being set by TSS-switch operations, and needs to be cleared.
Ingo
How do I clear the 'busy' bit?
I set my TSS descriptor with__set_tss_desc(cpu, GDT_ENTRY_PAGE_FAULT_TSS, &pagefault_tss);
Eliad
Quoted 20 lines · Ingo Molnar
Quoted 11 lines · Ingo Molnar
Quoted 8 lines · eliad lubovsky
I am trying to handle page faults exceptions in the kernel using the task gate mechanism. I succeeded to transfer the execution to my page fault handler using a new TSS and updates to the GDT and IDT tables (similar to the double fault mechanism in 2.6). After handling the fault and allocating the physical page I use the iret instruction to switch back to the previous task. The problem is that I got a double fault with the same address that cause the fault (although the physical page is allocated and mapped). Any clues?
Quoted 2 lines
are you clearing the 'nested task' (NT) flag of the new TSS once you have switched to it?
correction - i meant the 'busy' bit in the TSS descriptor. It's being set by TSS-switch operations, and needs to be cleared.
Ingo
Quoted 3 lines · eliad lubovsky
How do I clear the 'busy' bit?
I set my TSS descriptor with__set_tss_desc(cpu, GDT_ENTRY_PAGE_FAULT_TSS, &pagefault_tss);
i suspect you have to clear the busy bit in the pagefault handler itself. The CPU marks it as busy upon fault. I guess it would be OK to just do the above __set_tss_desc() for _every_ pagefault, that too will clear the busy bit, but you are probably better off just clearing that bit manually:
cpu_gdt_table[cpu][GDT_ENTRY_TSS].b &= 0xfffffdff; Ingo
attached a patch, it may be more clear to understand what I have done. To cause a page fault there is a need to compile the kernel with 8KB stack, and use the following module and application:
the module is a char device "my_device", the application opens a fd to the module and call ioctl, the module respond in calling to an overloaded stack function which cause a page fault.
the task gate is being set dynamically by the ioctl sys_call.
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <asm/io.h>
#include <linux/vmalloc.h>
#include <asm/irq.h>
#include <asm/uaccess.h>
int simple_open (struct inode *inode, struct file *filp); int simple_release(struct inode *inode, struct file *filp); int simple_ioctl (struct inode *inode, struct file *filp, unsigned int cmd, unsigned long arg);
static int simple_major = 0;
//extern void my_set_task_gate(unsigned int n, unsigned int gdt); //extern void my_set_page_fault_intr_gate();
struct file_operations simple_nopage_ops = {
open: simple_open,
release: simple_release,
ioctl: simple_ioctl
};
struct file_operations *simple_fops = {
&simple_nopage_ops,
};
int simple_open (struct inode *inode, struct file *filp) {
filp->f_op = simple_fops;
filp->private_data = 0;
return 0;
}
int simple_release(struct inode *inode, struct file *filp)
{
printk(KERN_INFO "Driver: simple release\n");
return 0;
}/* declare a stack that is larger then a PAGE_SIZE
*/ static void call_overloaded_stack(void)
{
char buff[1420];
char buff1[1420];
char buff2[1500];
printk(KERN_INFO "overloaded stack called\n");
buff[1400] = 1;
buff1[1400] = 1;
buff2[1450] = 1;
}
/* cause a page fault by calling to call_overloaded_stack() method */ int simple_ioctl (struct inode *inode, struct file *filp,
unsigned int cmd, unsigned long arg)
{
printk(KERN_INFO "Driver: Pre simple ioctl\n");
/* set task gate */
// my_set_task_gate(14, GDT_ENTRY_PAGE_FAULT_TSS); /* it is being set by the ioctl sys_call */
call_overloaded_stack();
/* return to interrupt gate */
// my_set_page_fault_intr_gate();
return 0;
}
static int __init mm_init(void)
{
int result;
printk("Driver: initialized\n"); result = register_chrdev(simple_major, "my_device",&simple_nopage_ops);
if (result < 0)
{
printk(KERN_WARNING "Driver: my_device: unable to get major
%d\n", simple_major);
return result;
}
if (simple_major == 0)
simple_major = result;
return 0;
}static void __exit mm_exit(void)
{
printk("Driver: exited\n");
unregister_chrdev(simple_major, "my_device");
}
MODULE_AUTHOR("Eliad Lubovsky");MODULE_DESCRIPTION("Memory Test");MODULE_LICENSE("GPL");
module_init(mm_init);module_exit(mm_exit);
/***********************************************************/
Application: open device and implement ioctl to cause a page fault by an overloaded stack.
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <fcntl.h>
int main(int argc, char ** argv) { int ret;
unsigned int i=0;
int fd = open("/dev/my_device", O_RDWR);
if(fd<0) {
printf("Failed to open my_device\n");
exit(1);
} ret = ioctl(fd, 0, i);
if(ret != 0) {
printf("Ioctl failed\n");
exit(1);
}
close(fd); return 0;
}
Quoted 12 lines · Ingo Molnar
Quoted 3 lines · eliad lubovsky
How do I clear the 'busy' bit?
I set my TSS descriptor with__set_tss_desc(cpu, GDT_ENTRY_PAGE_FAULT_TSS, &pagefault_tss);i suspect you have to clear the busy bit in the pagefault handler itself. The CPU marks it as busy upon fault. I guess it would be OK to just do the above
__set_tss_desc()for _every_ pagefault, that too will clear the busy bit, but you are probably better off just clearing that bit manually:
Quoted 3 lines
cpu_gdt_table[cpu][GDT_ENTRY_TSS].b &= 0xfffffdff; Ingo
arch/i386/kernel/cpu/common.c +3 -1
@@ -565,11 +565,13 @@ void __init cpu_init (void)/* Set up doublefault TSS pointer in the GDT */ __set_tss_desc (cpu,GDT_ENTRY_DOUBLEFAULT_TSS , &doublefault_tss); +/* Set up page fault TSS pointer in the GDT */ +__set_tss_desc (cpu,GDT_ENTRY_PAGE_FAULT_TSS , &pagefault_tss); +/* Clear %fs and %gs. */ asmvolatile ("xorl %eax, %eax; movl %eax, %fs; movl %eax, %gs" );/* Clear all 6 debug registers: */ -#define CD (register) __asm__("movl %0,%%db" #register ::"r"(0) );CD (0 );CD (1 );CD (2 );CD (3 );/* no db4 and db5 */ ;CD (6 );CD (7 );
arch/i386/kernel/Makefile +1 -1
@@ -7,7 +7,7 @@ extra-y := head.o init_task.o vmlinux.ld obj-y := process.o semaphore.o signal.o entry.o traps.o irq.o vm86.o \ ptrace.o i8259.o ioport.o ldt.o setup.o time.o sys_i386.o \ pci-dma.o i386_ksyms.o i387.o dmi_scan.o bootflag.o \ - doublefault.o + doublefault.o pagefault.o obj-y += cpu/ obj-y += timers/
arch/i386/kernel/pagefault.c +79 -0
79 lines not on this page · raw message
arch/i386/kernel/traps.c +14 -0
@@ -1025,6 +1025,20 @@ static void __init set_task_gate(unsigne_set_gate (idt_table+n,5 ,0 ,0 ,(gdt_entry<<3 )); } +void my_set_task_gate (unsigned int n,unsigned int gdt_entry) +{ +set_task_gate (n, gdt_entry); +} + +EXPORT_SYMBOL (my_set_task_gate); + +void my_set_page_fault_intr_gate () +{ +set_intr_gate (14 ,&page_fault); +} + +EXPORT_SYMBOL (my_set_page_fault_intr_gate); +void __init trap_init (void ) {
fs/ioctl.c +6 -0
@@ -57,6 +57,9 @@ asmlinkage long sys_ioctl(unsigned int funsigned int flag;int on, error = -EBADF ; +/* set task gate entry in the idt */ +my_set_task_gate (14 ,GDT_ENTRY_PAGE_FAULT_TSS ); + filp =fget (fd);if (!filp) goto out;
@@ -133,6 +136,9 @@ asmlinkage long sys_ioctl(unsigned int ffput (filp); out: +/* set intr gate entry in the idt */ +my_set_page_fault_intr_gate (); +return error; }
include/asm-i386/processor.h +1 -0
@@ -86,6 +86,7 @@ struct cpuinfo_x86 {extern struct cpuinfo_x86 boot_cpu_data;extern struct cpuinfo_x86 new_cpu_data;extern struct tss_struct doublefault_tss; +extern struct tss_struct pagefault_tss; DECLARE_PER_CPU(struct tss_struct, init_tss);#ifdef CONFIG_SMP
include/asm-i386/segment.h +2 -1
@@ -42,7 +42,7 @@* 27 - unused * 28 - unused * 29 - unused -* 30 - unused +* 30 - TSS for page fault handler * 31 - TSS for double fault handler */ #define GDT_ENTRY_TLS_ENTRIES 3
@@ -71,6 +71,7 @@#define GDT_ENTRY_PNPBIOS_BASE (GDT_ENTRY_KERNEL_BASE + 6)#define GDT_ENTRY_APMBIOS_BASE (GDT_ENTRY_KERNEL_BASE + 11) +#define GDT_ENTRY_PAGE_FAULT_TSS 30#define GDT_ENTRY_DOUBLEFAULT_TSS 31/*
include/asm-i386/smp.h +2 -1
@@ -50,7 +50,8 @@ extern u8 x86_cpu_to_apicid[];* from the initial startup. We map APIC_BASE very early in page_setup(), * so this is correct in the x86 case. */ -#define smp_processor_id () (current_thread_info()->cpu) +//#define smp_processor_id() (current_thread_info()->cpu) +#define smp_processor_id () 0extern cpumask_t cpu_callout_map;#define cpu_possible_map cpu_callout_map
include/asm-i386/thread_info.h +1 -1
@@ -55,7 +55,7 @@ struct thread_info {#ifdef CONFIG_4KSTACKS #define THREAD_SIZE (4096)#else -#define THREAD_SIZE (8192) +#define THREAD_SIZE (8192*2)#endif #define STACK_WARN (THREAD_SIZE/8)
include/asm-i386/vmalloc_thread_info.h +15 -0
@@ -0,0 +1,15 @@ +#ifndef _LINUX_VMALLOC_THREAD_INFO_H +#define _LINUX_VMALLOC_THREAD_INFO_H + +#include <linux/spinlock.h> +#include <asm/page.h> /* pgprot_t */ + + +/* +* Highlevel APIs for driver use +*/ +extern void *vmalloc_thread_info (unsigned long size); +extern void extend_stack_size (struct vm_struct *area); + +#endif /* _LINUX_VMALLOC_THREAD_INFO_H */ +
include/linux/gfp.h +16 -0
@@ -91,6 +91,19 @@ static inline struct page *alloc_pages_n NODE_DATA(nid)->node_zonelists + (gfp_mask & GFP_ZONEMASK)); } +extern struct page * +FASTCALL (__alloc_thread_info_pages (unsigned int ,unsigned int ,struct zonelist *)); + +static inline struct page *alloc_thread_info_pages_node (int nid,unsigned int gfp_mask, +unsigned int order) +{ +if (unlikely (order >=MAX_ORDER )) +return NULL ; + +return __alloc_thread_info_pages (gfp_mask, order, +NODE_DATA (nid)->node_zonelists + (gfp_mask &GFP_ZONEMASK )); +} +#ifdef CONFIG_NUMA extern struct page *alloc_pages_current (unsigned gfp_mask,unsigned order);
@@ -107,9 +120,12 @@ extern struct page *alloc_page_vma(unsig#else #define alloc_pages (gfp_mask, order) \ alloc_pages_node(numa_node_id(), gfp_mask, order) +#define alloc_thread_info_pages (gfp_mask, order) \ + alloc_thread_info_pages_node(numa_node_id(), gfp_mask, order)#define alloc_page_vma (gfp_mask, vma, addr) alloc_pages(gfp_mask, 0)#endif #define alloc_page (gfp_mask) alloc_pages(gfp_mask, 0) +#define alloc_thread_info_page (gfp_mask) alloc_thread_info_pages(gfp_mask, 0)extern unsigned long FASTCALL (__get_free_pages (unsigned int gfp_mask,unsigned int order));extern unsigned long FASTCALL (get_zeroed_page (unsigned int gfp_mask));
kernel/fork.c +9 -0
@@ -79,7 +79,11 @@ static kmem_cache_t *task_struct_cachep;void free_task (struct task_struct *tsk) { +#if 0 free_thread_info (tsk->thread_info); +#else +vfree_thread_info (tsk->thread_info); +#endif free_task_struct (tsk); }EXPORT_SYMBOL (free_task);
@@ -264,7 +268,12 @@ static struct task_struct *dup_task_struif (!tsk)return NULL ; +#if 0 ti =alloc_thread_info (tsk); +#else + ti =vmalloc_thread_info (THREAD_SIZE ); +#endif +if (!ti) {free_task_struct (tsk);return NULL ;
mm/Makefile +1 -1
@@ -5,7 +5,7 @@ mmu-y := nommu.o mmu-$(CONFIG_MMU) := fremap.o highmem.o madvise.o memory.o mincore.o \ mlock.o mmap.o mprotect.o mremap.o msync.o rmap.o \ - vmalloc.o + vmalloc.o vmalloc_thread_info.o obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ page_alloc.o page-writeback.o pdflush.o prio_tree.o \
mm/page_alloc.c +154 -0
157 lines not on this page · raw message
mm/vmalloc_thread_info.c +502 -0
502 lines not on this page · raw message