Skip to contentSkip to conversation

Questions about User Mode Linux

DISCUSSION jiangtao.jit 4 participants 8 messages Oct 12, 2011 – Oct 14, 2011 2 messages not in the archive

lore
Share & tools
lore

jiangtao.jit

author
#0 lore raw

Hi:

Need help with UML issue

1.is User Mode Linux still supported?
2.where can I find the latest Docs and guidelines?
3.compile error?
on kernel version 2.6.39
gcc 4.1.2
I compile with: make ARCH=um linux
errors ocurred
/usr/bin/ld:arch/um/kernel/vmlinux.lds:237: syntax error
collect2: ld returned 1 exit status
KSYM .tmp_kallsyms1.S
nm: '.tmp_vmlinux1': No such file
No valid symbol.
make: *** [.tmp_kallsyms1.S] Error 1

Thanks.

2011-10-12
jiangtao.jit

#1 lore raw

Hi!

Quoted 5 lines · jiangtao.jit

Hi:

Need help with UML issue

1.is User Mode Linux still supported?

I think so

Quoted 1 line

2.where can I find the latest Docs and guidelines?

I usually refer to http://user-mode-linux.sourceforge.net/old/index.html . It probably is not really the latest version, but it has been good enough for me so far.

Quoted 11 lines

3.compile error?
on kernel version 2.6.39
gcc 4.1.2
I compile with: make ARCH=um linux
errors ocurred
/usr/bin/ld:arch/um/kernel/vmlinux.lds:237: syntax error
collect2: ld returned 1 exit status
KSYM .tmp_kallsyms1.S
nm: '.tmp_vmlinux1': No such file
No valid symbol.
make: *** [.tmp_kallsyms1.S] Error 1

1) gcc 4.1.2 looks very old. Even my debian stable here already has 4.4.5.
2) I use the command "ARCH=um make linux". It will probably not make any difference, but ...
3) Sometimes UML is a real pain. When I upgrade my testing kernel, I usually have to spend some time searching the internet for fixes. I do not remember this error, tough. You can find my current (3.0) kernel at
http://repo.or.cz/w/cor.git , if you are interested.

-Michi

Signature · 2 lines
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com
#2 lore raw

Hi!

On 17:25 Wed 12 Oct , michi1 at michaelblizek.twilightparadox.com wrote:
...

Quoted 4 lines

3) Sometimes UML is a real pain. When I upgrade my testing kernel, I usually have to spend some time searching the internet for fixes. I do not remember this error, tough. You can find my current (3.0) kernel at
http://repo.or.cz/w/cor.git , if you are interested.

Oh, no. After migrating to 64bit my UML is broken again...

-Michi

Signature · 2 lines
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com
#3 lore raw
Quoted 10 lines · On Wed, Oct 12, 2011 at 07:32:26PM +0200, michi1 at…

Hi!

On 17:25 Wed 12 Oct , michi1 at michaelblizek.twilightparadox.com wrote:
...

Quoted 4 lines

3) Sometimes UML is a real pain. When I upgrade my testing kernel, I usually have to spend some time searching the internet for fixes. I do not remember this error, tough. You can find my current (3.0) kernel at
http://repo.or.cz/w/cor.git , if you are interested.

Oh, no. After migrating to 64bit my UML is broken again...

Sounds interesting :)

Please report this kind of things, if you think you've hit a bug. (MAINTAINERS or the output of "scripts/get_maintainer.pl -f arch/um" tells you where to send your mails)

Thanks,
Jonathan Neusch?fer

#4 lore raw

Hi!

After migrating from i386 to x86_64, my uml started to segfault in weird ways. In the end, I figured that "make clean/mrproper/distclean" does not really clean up properly. I had to use "ARCH=um make distclean". The command "make distclean" did not remove these files:

arch/um/include/shared/kern_constants.h
arch/um/include/shared/user_constants.h
arch/um/kernel/config.c
arch/um/kernel/config.tmp
arch/um/kernel/vmlinux.lds
arch/um/sys-x86_64/vdso/vdso.lds
arch/um/sys-x86_64/vdso/vdso.so
arch/um/sys-x86_64/vdso/vdso.so.dbg
arch/um/sys-x86_64/vdso/vdso-syms.lds

Why is this the case?

-Michi

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw

Hi!

Quoted 5 lines · Richard Weinberger

Am 13.10.2011 20:10, schrieb michi1 at michaelblizek.twilightparadox.com:

Quoted 1 line

After migrating from i386 to x86_64, my uml started to segfault in weird ways.

What exactly is the problem?
Without any details nobody can and will help you.

I have attached the trace below, if you are interested. It happened during/after /sbin/init was executed. Version is f2c0d0266cc5eb36a4aa44944b4096ec121490aa of linus'es git tree. However, everything is running now. It was caused by uncleaned files. I am just reporting, because this was suprising to me.

Quoted 15 lines
Quoted 4 lines

In the end, I figured that "make clean/mrproper/distclean" does not really clean up properly. I had to use "ARCH=um make distclean". The command "make distclean" did not remove these files:

Quoted 10 lines

arch/um/include/shared/kern_constants.h
arch/um/include/shared/user_constants.h
arch/um/kernel/config.c
arch/um/kernel/config.tmp
arch/um/kernel/vmlinux.lds
arch/um/sys-x86_64/vdso/vdso.lds
arch/um/sys-x86_64/vdso/vdso.so
arch/um/sys-x86_64/vdso/vdso.so.dbg
arch/um/sys-x86_64/vdso/vdso-syms.lds

Quoted 1 line

"make clean ARCH=um" works fine on my system.

Yes, the point is "ARCH=um". I do not really understand why it is needed for clean operations?

trace:

$ gdb --args ./linux ubda=root.img
Core dump limits :
 soft - 0
 hard - NONE
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking for tmpfs mount on /dev/shm...nothing mounted on /dev/shm
Checking PROT_EXEC mmap in /tmp/...OK
Checking for the skas3 patch in the host:
  - /proc/mm...not found: No such file or directory
  - PTRACE_FAULTINFO...not found
  - PTRACE_LDT...not found
UML running in SKAS0 mode
Adding 22114304 bytes to physical memory to account for exec-shield gap
Initializing cgroup subsys cpu
Linux version 3.1.0-rc1-106673-g7208f79-dirty (michi at grml) (gcc version 4.4.5 (Debian 4.4.5-8) ) #1570 Thu Oct 13 18:43:52 CEST 2011
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 13405
Kernel command line: ubda=root.img root=98:0
PID hash table entries: 256 (order: -1, 2048 bytes)
Dentry cache hash table entries: 8192 (order: 4, 65536 bytes)
Inode-cache hash table entries: 4096 (order: 3, 32768 bytes)
Memory: 27880k available
NR_IRQS:15
------------------------
| Locking API testsuite:
----------------------------------------------------------------------------
                                 | spin |wlock |rlock |mutex | wsem | rsem |
  --------------------------------------------------------------------------
                     A-A deadlock:failed|failed|  ok  |failed|failed|failed|
                 A-B-B-A deadlock:failed|failed|  ok  |failed|failed|failed|
             A-B-B-C-C-A deadlock:failed|failed|  ok  |failed|failed|failed|
             A-B-C-A-B-C deadlock:failed|failed|  ok  |failed|failed|failed|
         A-B-B-C-C-D-D-A deadlock:failed|failed|  ok  |failed|failed|failed|
         A-B-C-D-B-D-D-A deadlock:failed|failed|  ok  |failed|failed|failed|
         A-B-C-D-B-C-D-A deadlock:failed|failed|  ok  |failed|failed|failed|
                    double unlock:failed|failed|failed|  ok  |failed|failed|
                  initialize held:failed|failed|failed|failed|failed|failed|
                 bad unlock order:  ok  |  ok  |  ok  |  ok  |  ok  |  ok  |
  --------------------------------------------------------------------------
              recursive read-lock:             |  ok  |             |failed|
           recursive read-lock #2:             |  ok  |             |failed|
            mixed read-write-lock:             |failed|             |failed|
            mixed write-read-lock:             |failed|             |failed|
  --------------------------------------------------------------------------
     hard-irqs-on + irq-safe-A/12:failed|failed|  ok  |
     soft-irqs-on + irq-safe-A/12:failed|failed|  ok  |
     hard-irqs-on + irq-safe-A/21:failed|failed|  ok  |
     soft-irqs-on + irq-safe-A/21:failed|failed|  ok  |
       sirq-safe-A => hirqs-on/12:failed|failed|  ok  |
       sirq-safe-A => hirqs-on/21:failed|failed|  ok  |
         hard-safe-A + irqs-on/12:failed|failed|  ok  |
         soft-safe-A + irqs-on/12:failed|failed|  ok  |
         hard-safe-A + irqs-on/21:failed|failed|  ok  |
         soft-safe-A + irqs-on/21:failed|failed|  ok  |
    hard-safe-A + unsafe-B #1/123:failed|failed|  ok  |
    soft-safe-A + unsafe-B #1/123:failed|failed|  ok  |
    hard-safe-A + unsafe-B #1/132:failed|failed|  ok  |
    soft-safe-A + unsafe-B #1/132:failed|failed|  ok  |
    hard-safe-A + unsafe-B #1/213:failed|failed|  ok  |
    soft-safe-A + unsafe-B #1/213:failed|failed|  ok  |
    hard-safe-A + unsafe-B #1/231:failed|failed|  ok  |
    soft-safe-A + unsafe-B #1/231:failed|failed|  ok  |
    hard-safe-A + unsafe-B #1/312:failed|failed|  ok  |
    soft-safe-A + unsafe-B #1/312:failed|failed|  ok  |
    hard-safe-A + unsafe-B #1/321:failed|failed|  ok  |
    soft-safe-A + unsafe-B #1/321:failed|failed|  ok  |
    hard-safe-A + unsafe-B #2/123:failed|failed|  ok  |
    soft-safe-A + unsafe-B #2/123:failed|failed|  ok  |
    hard-safe-A + unsafe-B #2/132:failed|failed|  ok  |
    soft-safe-A + unsafe-B #2/132:failed|failed|  ok  |
    hard-safe-A + unsafe-B #2/213:failed|failed|  ok  |
    soft-safe-A + unsafe-B #2/213:failed|failed|  ok  |
    hard-safe-A + unsafe-B #2/231:failed|failed|  ok  |
    soft-safe-A + unsafe-B #2/231:failed|failed|  ok  |
    hard-safe-A + unsafe-B #2/312:failed|failed|  ok  |
    soft-safe-A + unsafe-B #2/312:failed|failed|  ok  |
    hard-safe-A + unsafe-B #2/321:failed|failed|  ok  |
    soft-safe-A + unsafe-B #2/321:failed|failed|  ok  |
      hard-irq lock-inversion/123:failed|failed|  ok  |
      soft-irq lock-inversion/123:failed|failed|  ok  |
      hard-irq lock-inversion/132:failed|failed|  ok  |
      soft-irq lock-inversion/132:failed|failed|  ok  |
      hard-irq lock-inversion/213:failed|failed|  ok  |
      soft-irq lock-inversion/213:failed|failed|  ok  |
      hard-irq lock-inversion/231:failed|failed|  ok  |
      soft-irq lock-inversion/231:failed|failed|  ok  |
      hard-irq lock-inversion/312:failed|failed|  ok  |
      soft-irq lock-inversion/312:failed|failed|  ok  |
      hard-irq lock-inversion/321:failed|failed|  ok  |
      soft-irq lock-inversion/321:failed|failed|  ok  |
      hard-irq read-recursion/123:  ok  |
      soft-irq read-recursion/123:  ok  |
      hard-irq read-recursion/132:  ok  |
      soft-irq read-recursion/132:  ok  |
      hard-irq read-recursion/213:  ok  |
      soft-irq read-recursion/213:  ok  |
      hard-irq read-recursion/231:  ok  |
      soft-irq read-recursion/231:  ok  |
      hard-irq read-recursion/312:  ok  |
      soft-irq read-recursion/312:  ok  |
      hard-irq read-recursion/321:  ok  |
      soft-irq read-recursion/321:  ok  |
--------------------------------------------------------
144 out of 218 testcases failed, as expected. |
----------------------------------------------------
Calibrating delay loop... 967.47 BogoMIPS (lpj=4837376)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 256
Initializing cgroup subsys blkio
Checking that host ptys support output SIGIO...Yes
Checking that host ptys support SIGIO on close...No, enabling workaround
Using 2.6 host AIO
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
Switching to clocksource itimer
NET: Registered protocol family 1
mconsole (version 2) initialized on /home/michi/.uml/CkIDmD/mconsole
Checking host MADV_REMOVE support...
MADV_REMOVE failed, err = -38
Can't release memory to the host - memory hotplug won't be supported
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 55
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
loop: module loaded
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <[email protected]>
NET: Registered protocol family 17
Initialized stdio console driver
Console initialized on /dev/tty0
Switched to NOHz mode on CPU #0
console [tty0] enabled
Initializing software serial port version 1
console [mc-1] enabled
 ubda: unknown partition table
kjournald starting.  Commit interval 5 seconds
EXT3-fs (ubda): mounted filesystem with writeback data mode
VFS: Mounted root (ext3 filesystem) readonly on device 98:0.

Program received signal SIGSEGV, Segmentation fault. bio_list_add (bio=0x61c5ba40) at include/linux/bio.h:414 414 if (bl->tail)
(gdb) backtrace

#0  bio_list_add (bio=0x61c5ba40) at include/linux/bio.h:414
#1  generic_make_request (bio=0x61c5ba40) at block/blk-core.c:1574
#2  0x000000006015c27d in submit_bio (rw=2, bio=0x61c5ba40) at block/blk-core.c:1645
#3  0x00000000600ccff3 in submit_bh (rw=0, bh=0x61808130) at fs/buffer.c:2946
#4  0x00000000600d0e83 in __bread_slow (bdev=<value optimized out>, block=<value optimized out>, size=<value optimized out>)
    at fs/buffer.c:1220

#5 __bread (bdev=<value optimized out>, block=<value optimized out>, size=<value optimized out>) at fs/buffer.c:1413 #6 0x000000006012a0c0 in sb_bread (inode=<value optimized out>, depth=1, offsets=0x61c224d0, chain=0x61c22470, err=0x61c224fc)
at include/linux/buffer_head.h:289
#7 ext3_get_branch (inode=<value optimized out>, depth=1, offsets=0x61c224d0, chain=0x61c22470, err=0x61c224fc) at fs/ext3/inode.c:428 #8 0x000000006012affc in ext3_get_blocks_handle (handle=<value optimized out>, inode=0x6180d740, iblock=<value optimized out>,
maxblocks=<value optimized out>, bh_result=<value optimized out>, create=0) at fs/ext3/inode.c:880
#9 0x000000006012b5f4 in ext3_get_block (inode=0x6180d740, iblock=14, bh_result=0x61c226b0, create=16843009) at fs/ext3/inode.c:1038 #10 0x00000000600d729d in do_mpage_readpage (bio=0x61808198, page=<value optimized out>, nr_pages=<value optimized out>,

    last_block_in_bio=<value optimized out>, map_bh=0x61c226b0, first_logical_block=0x101010101010101, get_block=0x6012b530 <ext3_get_block>)
    at fs/mpage.c:219
#11 0x00000000600d773a in mpage_readpages (mapping=0x6180d878, pages=0x61c22880, nr_pages=18, get_block=<value optimized out>)
    at fs/mpage.c:387
#12 0x000000006012a986 in ext3_readpages (file=<value optimized out>, mapping=0x61c5ba40, pages=0x61c5ba40, nr_pages=0)
    at fs/ext3/inode.c:1804

#13 0x0000000060081996 in read_pages (mapping=0x6180d878, filp=<value optimized out>, pages=0x61c22880, nr_pages=18) at mm/readahead.c:119 #14 0x0000000060081b94 in __do_page_cache_readahead (mapping=<value optimized out>, filp=<value optimized out>, offset=14,
nr_to_read=<value optimized out>, lookahead_size=<value optimized out>) at mm/readahead.c:199
#15 0x0000000060081bfc in ra_submit (ra=<value optimized out>, mapping=0x2, filp=0x61c5ba40) at mm/readahead.c:256 #16 0x0000000060079869 in do_sync_mmap_readahead (vma=0x61d0b720, vmf=<value optimized out>) at mm/filemap.c:1616 #17 filemap_fault (vma=0x61d0b720, vmf=<value optimized out>) at mm/filemap.c:1681
#18 0x000000006008ebc9 in __do_fault (mm=0x61d08b80, vma=0x61d0b720, address=0, pmd=0x0, pgoff=104, flags=16843009, orig_pte=...)

    at mm/memory.c:3185
#19 0x000000006008fde6 in do_linear_fault (mm=0x61c5ba40, vma=0x61d0b720, address=1075965856, pte=<value optimized out>, pmd=0x0, 
    flags=16843009) at mm/memory.c:3338
#20 handle_pte_fault (mm=0x61c5ba40, vma=0x61d0b720, address=1075965856, pte=<value optimized out>, pmd=0x0, flags=16843009)
    at mm/memory.c:3398
#21 0x00000000600909f8 in handle_mm_fault (mm=0x61d08b80, vma=0x61d0b720, address=1075965856, flags=16843009) at mm/memory.c:3503
#22 0x00000000600164f2 in handle_page_fault (address=1075965856, ip=<value optimized out>, is_write=<value optimized out>, 
    is_user=<value optimized out>, code_out=0x68) at arch/um/kernel/trap.c:68

#23 0x0000000060017321 in maybe_map (addr=<value optimized out>, len=<value optimized out>, is_write=1, op=<value optimized out>,
arg=<value optimized out>) at arch/um/kernel/skas/uaccess.c:47
#24 do_op_one_page (addr=<value optimized out>, len=<value optimized out>, is_write=1, op=<value optimized out>, arg=<value optimized out>)
at arch/um/kernel/skas/uaccess.c:66
#25 0x00000000600174de in buffer_op (addr=1640348224, len=2, is_write=0, op=0, arg=0x68) at arch/um/kernel/skas/uaccess.c:97 #26 0x0000000060017674 in clear_user (mem=0x61c5ba40, len=2) at arch/um/kernel/skas/uaccess.c:227
#27 0x00000000600e5015 in padzero (elf_bss=1640348224) at fs/binfmt_elf.c:106
#28 0x00000000600e57c7 in load_elf_interp (interp_elf_ex=0x61ce5200, interpreter=<value optimized out>,

    interp_map_addr=<value optimized out>, no_base=2218976) at fs/binfmt_elf.c:507

---Type <return> to continue, or q <return> to quit---
#29 0x00000000600e6636 in load_elf_binary (bprm=0x61c04280, regs=0x61c20c20) at fs/binfmt_elf.c:883
#30 0x00000000600ab8e9 in search_binary_handler (bprm=0x61c04280, regs=0x61c20c20) at fs/exec.c:1396
#31 0x00000000600ad3f7 in do_execve_common (filename=0x60227e0b "/sbin/init", argv=..., envp=..., regs=0x61c20c20) at fs/exec.c:1522 #32 0x00000000600ad4e9 in do_execve (filename=0x61c5ba40 "\340\\\016", __argv=0x61c5ba40, __envp=0x0, regs=0x0) at fs/exec.c:1569 #33 0x00000000600129b9 in execve1 (file=0x61c5ba40 "\340\\\016", argv=0x61c5ba40, env=0x0) at arch/um/kernel/exec.c:51 #34 0x0000000060012a4d in um_execve (file=0x61c5ba40 "\340\\\016", argv=0x61c5ba40, env=0x0) at arch/um/kernel/exec.c:67 #35 0x0000000060014eab in kernel_execve (filename=0x61c5ba40 "\340\\\016", argv=0x61c5ba40, envp=0x0) at arch/um/kernel/syscall.c:63 #36 0x000000006001256a in run_init_process (init_filename=0x61c5ba40 "\340\\\016") at init/main.c:739
#37 0x00000000600125ca in init_post () at init/main.c:774
#38 0x0000000060001251 in kernel_init (unused=<value optimized out>) at init/main.c:835
#39 0x0000000060024d65 in run_kernel_thread (fn=<value optimized out>, arg=<value optimized out>, jmp_ptr=<value optimized out>)

    at arch/um/os-Linux/process.c:268

#40 0x0000000060013f98 in new_thread_handler () at arch/um/kernel/process.c:153 #41 0x0000000000000000 in ?? ()
(gdb) print bl->tail
Cannot access memory at address 0x8
(gdb) list
409
410 static inline void bio_list_add(struct bio_list *bl, struct bio *bio) 411 {
412 bio->bi_next = NULL;
413
414 if (bl->tail)
415 bl->tail->bi_next = bio;
416 else
417 bl->head = bio;
418
(gdb) frame 1
#1 generic_make_request (bio=0x61c5ba40) at block/blk-core.c:1574
1574 bio_list_add(current->bio_list, bio);
(gdb) list
1569 {
1570 struct bio_list bio_list_on_stack;
1571
1572 if (current->bio_list) {

1573    /* make_request is active */
1574    bio_list_add(current->bio_list, bio);
1575    return;

1576 }
1577 /* following loop may be a bit non-obvious, and so deserves some
1578 * explanation.
(gdb)

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw

Ooops, sorry, forget the full CC:

Quoted 6 lines · Richard Weinberger

Am 14.10.2011 07:29, schrieb michi1 at michaelblizek.twilightparadox.com:

Quoted 5 lines

I have attached the trace below, if you are interested. It happened during/after /sbin/init was executed. Version is f2c0d0266cc5eb36a4aa44944b4096ec121490aa of linus'es git tree. However, everything is running now. It was caused by uncleaned files. I am just reporting, because this was suprising to me.

[....]

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

"make clean ARCH=um" works fine on my system.

Yes, the point is "ARCH=um". I do not really understand why it is needed for clean operations?

Otherwise kbuild will use x86 as arch.

The question is IMHO more in the direction of:
Shouldn't make mrproper delete *all* generated files from all architectures (and not only the current one) to get the tree in a (or actually the) pristine state?

Hmm, how are other arches handling that?

Bernd

Signature · 2 lines
Bernd Petrovitsch                  Email : bernd at petrovitsch.priv.at
                     LUGA : http://www.luga.at
author
#9 lore raw
In reply to michi1 at michaelblizek.twilightparadox.com

michi:

Thank you.

2011/10/12 <[email protected]>:

Quoted 42 lines

Hi!

Quoted 5 lines · jiangtao.jit

Hi:

Need help with UML issue

1.is User Mode Linux still supported?

I think so

Quoted 1 line

2.where can I find the latest Docs and guidelines?

I usually refer to http://user-mode-linux.sourceforge.net/old/index.html . It probably is not really the latest version, but it has been good enough for me so far.

Quoted 11 lines

3.compile error?
? ? ? on kernel version 2.6.39
? ? ? gcc 4.1.2
? ? ? I compile with: make ARCH=um linux
? ? errors ocurred
? ? ? ? ? ? ? /usr/bin/ld:arch/um/kernel/vmlinux.lds:237: syntax error ? ? ? ? ? ? ? collect2: ld returned 1 exit status
? ? ? ? ? ? ? KSYM ? ?.tmp_kallsyms1.S
? ? ? ? ? ? ? nm: '.tmp_vmlinux1': No such file
? ? ? ? ? ? ? No valid symbol.
? ? ? ? ? ? ? make: *** [.tmp_kallsyms1.S] Error 1

1) gcc 4.1.2 looks very old. Even my debian stable here already has 4.4.5.
2) I use the command "ARCH=um make linux". It will probably not make any ? difference, but ...
3) Sometimes UML is a real pain. When I upgrade my testing kernel, I usually ? have to spend some time searching the internet for fixes. I do not remember ? this error, tough. You can find my current (3.0) kernel at
? http://repo.or.cz/w/cor.git , if you are interested.

? ? ? ?-Michi
--
programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com