Virtual Framebuffer on UM

Hey, all!

I've been hacking on the UM kernel to reenable framebuffer support, in preparation for writing a driver using SDL (http://libsdl.org). The only things I have modified (at least with this tree, heh heh) so far were to add 'source drivers/video/Kconfig' to arch/um's Kconfig, and patch fbmem.c to disable caching on the UM arch. This is in the fb_mmap function, in all those messy #ifdefs around the arch specific calls to disable caching on the video memory.

I have a 2.6.6 UM kernel with the aforementioned patches that has been booted with video=vfb:, and displays:
fb0: Virtual frame buffer device, using 1024K of video memory
in the kernel messages.

I've ensured /dev/fb0 device node has major 29 and minor 0.

However, when I do 'cp /dev/fb0 /root/blob', I get a file of size 0. I was expecting a 1 meg file.

Is this normal behavior?

p.s. Geert, where can the source for your old GDK UM driver be found, the one that had the -lpthread trouble? I have a design that will (hopefully) not have those issues, but I'm curious as to how you tackled the rest of it. (SDL itself has a -lpthread dependency, but I can get around this. The graphical component of SDL does not require pthreads, so as long as I don't init the audio or threading components, I should be OK)

Regards,
Andrew Clunis

http://orospakr.is-a-geek.org/



-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com

What does `fbset -i' say about the length of the frame buffer?

I never finished it. I started using SKAS mode (in that case UML is dynamically linked instead of statically, and libX11 doesn't need libpthread) and managed to get something to work. IIRC at one moment of time even fbtest worked a bit.

However, when trying to get the logo and a real frame buffer console, I got stuck in the conflicts between the standard virtual console subsystem and the UML virtual consoles. With Gerd Knorr's recent patches for a special stderr console, you can probably solve (a few of) these conflicts.

What I had in mind:
- Split gdkfb in a kernel (gdkfb_kern) part that talks fbdev, and a user (gdkfb_user) part that takes care of X/GDK/GTK,
- Use a ring buffer for transfering drawing commands (from fbcon) between the two parts [not yet implemented],
- Use a separate thread to periodically update the screen contents if the user has mmap()ed the virtual frame buffer [not yet implemented].

Anyway, here's the patch for my current work. Note that it includes 2 versions of gdkfb and that you have to append gdkfb' or gdkfb2' to the kernel command line to make it initialize gdkfb or gdkfb2.

Good luck!

@@ -149,7 +149,7 @@
 # Default value for CROSS_COMPILE is not to prefix executables
 # Note: Some architectures assign CROSS_COMPILE in their arch/*/Makefile
 
-ARCH		?= $(SUBARCH)
+ARCH		?= um
 CROSS_COMPILE	?=
 
 # Architecture as present in compile.h

@@ -225,6 +225,8 @@
 
 source "drivers/base/Kconfig"
 
+source "drivers/input/Kconfig"
+
 source "arch/um/Kconfig_char"
 
 source "arch/um/Kconfig_block"

@@ -239,6 +241,8 @@
 
 source "fs/Kconfig"
 
+source "drivers/video/Kconfig"
+
 source "security/Kconfig"
 
 source "crypto/Kconfig"

@@ -19,6 +19,8 @@
 ubd-objs := ubd_kern.o ubd_user.o
 port-objs := port_kern.o port_user.o
 harddog-objs := harddog_kern.o harddog_user.o
+gdkfb-objs := gdkfb_kern.o gdkfb_user.o
+gdkfb2-objs := gdkfb2_kern.o gdkfb2_user.o
 
 obj-y =
 obj-$(CONFIG_SSL) += ssl.o

@@ -41,8 +43,10 @@
 obj-$(CONFIG_UML_WATCHDOG) += harddog.o
 obj-$(CONFIG_BLK_DEV_COW) += cow_kern.o
 obj-$(CONFIG_BLK_DEV_COW_COMMON) += cow_user.o
+obj-$(CONFIG_FB_GDK) += gdkfb.o gdkfb2.o
 
-obj-y += stdio_console.o $(CHAN_OBJS)
+obj-$(CONFIG_STDIO_CONSOLE) += stdio_console.o
+obj-y += $(CHAN_OBJS)
 
 USER_SINGLE_OBJS = $(foreach f,$(patsubst %.o,%,$(obj-y) $(obj-m)),$($(f)-objs))
 

@@ -50,5 +54,8 @@
 	null.o pty.o tty.o xterm.o
 USER_OBJS := $(foreach file,$(USER_OBJS),$(obj)/$(file))
 
+CFLAGS_gdkfb_user.o += $(shell gtk-config --cflags)
+CFLAGS_gdkfb2_user.o += $(shell gtk-config --cflags)
+
 $(USER_OBJS) : %.o: %.c
 	$(CC) $(CFLAGS_$(notdir $@)) $(USER_CFLAGS) -c -o $@ $<

@@ -0,0 +1,23 @@
+/*
+ *  linux/drivers/video/gdkfb.c -- GDK/GTK based frame buffer device
+ *
+ *	Copyright (C) 2003 Geert Uytterhoeven
+ *
+ *  This file is subject to the terms and conditions of the GNU General Public
+ *  License. See the file COPYING in the main directory of this archive for
+ *  more details.
+ */
+
+#include <linux/fb.h>
+#include <linux/init.h>
+
+struct gdkfb_user;
+
+extern int __init gdkfb_user_init(void);
+extern int __init gdkfb_user_init_one(struct gdkfb_user **gdkfb,
+				      struct fb_fix_screeninfo *fix,
+				      struct fb_var_screeninfo *var);
+extern void gdkfb_user_update(int x, int y, int width, int height,
+			      struct gdkfb_user *gdkfb);
+extern void gdkfb_user_setcolreg(unsigned int regno, unsigned int rgb,
+				 struct gdkfb_user *gdkfb);

@@ -0,0 +1,23 @@
+/*
+ *  linux/drivers/video/gdkfb2.c -- GDK/GTK based frame buffer device
+ *
+ *	Copyright (C) 2003 Geert Uytterhoeven
+ *
+ *  This file is subject to the terms and conditions of the GNU General Public
+ *  License. See the file COPYING in the main directory of this archive for
+ *  more details.
+ */
+
+#include <linux/fb.h>
+#include <linux/init.h>
+
+struct gdkfb2_user;
+
+extern int __init gdkfb2_user_init(void);
+extern int __init gdkfb2_user_init_one(struct gdkfb2_user **gdkfb2,
+				      struct fb_fix_screeninfo *fix,
+				      struct fb_var_screeninfo *var);
+extern void gdkfb2_user_update(int x, int y, int width, int height,
+			      struct gdkfb2_user *gdkfb2);
+extern void gdkfb2_user_setcolreg(unsigned int regno, unsigned int rgb,
+				 struct gdkfb2_user *gdkfb2);

@@ -147,8 +147,11 @@
         }
 	else if(check_remapped_addr(address & PAGE_MASK, is_write))
 		return(0);
-	else if(current->mm == NULL)
+	else if(current->mm == NULL) {
+printk("address = %lx, ip = %lx, is_write = %c, is_user = %c, sc = %p\n",
+       address, ip, is_write ? 'W' : 'R', is_user ? 'U' : 'K', sc);
 		panic("Segfault with no mm");
+	}
 	err = handle_page_fault(address, ip, is_write, is_user, &si.si_code);
 
 	catcher = current->thread.fault_catcher;

@@ -17,6 +17,7 @@
 #include "linux/sysrq.h"
 #include "linux/seq_file.h"
 #include "linux/delay.h"
+#include "linux/console.h"
 #include "asm/page.h"
 #include "asm/pgtable.h"
 #include "asm/ptrace.h"

@@ -412,6 +413,9 @@
  	strcpy(command_line, saved_command_line);
  	*cmdline_p = command_line;
 	setup_hostinfo();
+#ifdef CONFIG_DUMMY_CONSOLE
+	conswitchp = &dummy_con;
+#endif
 }
 
 void __init check_bugs(void)

@@ -146,7 +146,7 @@
        */
 static void disable_lcall(void)
 {
-	struct modify_ldt_ldt_s ldt;
+	struct user_desc ldt;
 	int err;
 
 	bzero(&ldt, sizeof(ldt));

@@ -59,7 +59,7 @@
 
 config HW_CONSOLE
 	bool
-	depends on VT && !S390 && !UM
+	depends on VT && !S390
 	default y
 
 config SERIAL_NONSTANDARD

@@ -1359,7 +1359,7 @@
 		/* noctty = 1; */
 		goto got_driver;
 	}
-#ifdef CONFIG_VT
+#ifdef CONFIG_VT_CONSOLE
 	if (device == MKDEV(TTY_MAJOR,0)) {
 		extern int fg_console;
 		extern struct tty_driver *console_driver;

@@ -660,6 +660,7 @@
 
 static void visual_init(int currcons, int init)
 {
+printk("visual_init\n");
     /* ++Geert: sw->con_init determines console size */
     sw = conswitchp;
 #ifndef VT_SINGLE_DRIVER

@@ -751,6 +752,7 @@
 	unsigned int new_cols, new_rows, new_row_size, new_screen_size;
 	unsigned short *newscreen;
 
+printk("vc_resize\n");
 	WARN_CONSOLE_UNLOCKED();
 
 	if (!vc_cons_allocated(currcons))

@@ -2501,6 +2503,7 @@
 {
 	int j, k ;
 
+printk("vc_init\n");
 	video_num_columns = cols;
 	video_num_lines = rows;
 	video_size_row = cols<<1;

@@ -2602,6 +2605,7 @@
 
 int __init vty_init(void)
 {
+#ifdef CONFIG_VT_CONSOLE
 	console_driver = alloc_tty_driver(MAX_NR_CONSOLES);
 	if (!console_driver)
 		panic("Couldn't allocate console driver\n");

@@ -2617,6 +2621,7 @@
 	tty_set_operations(console_driver, &con_ops);
 	if (tty_register_driver(console_driver))
 		panic("Couldn't register console driver\n");
+#endif /* CONFIG_VT_CONSOLE */
 
 	kbd_init();
 	console_map_init();

@@ -884,6 +884,12 @@
 	  Say Y here if you want to support the built-in frame buffer of
 	  the Motorola 68328 CPU family.
 
+config FB_GDK
+	tristate "GDK/GTK support"
+	depends on FB && USERMODE
+	---help---
+	  This is a virtual frame buffer device on top of GDK/GTK.
+
 config FB_VIRTUAL
 	tristate "Virtual Frame Buffer support (ONLY FOR TESTING!)"
 	depends on FB

@@ -89,3 +89,4 @@
 obj-$(CONFIG_FB_LEO)               += leo.o sbuslib.o cfbimgblt.o cfbcopyarea.o \
 				      cfbfillrect.o
 
+obj-$(CONFIG_FB_GDK)		  += cfbfillrect.o cfbcopyarea.o cfbimgblt.o

@@ -164,6 +164,10 @@
 extern int leo_setup(char*);
 extern int kyrofb_init(void);
 extern int kyrofb_setup(char*);
+extern int gdkfb_init(void);
+extern int gdkfb_setup(char*);
+extern int gdkfb2_init(void);
+extern int gdkfb2_setup(char*);
 
 static struct {
 	const char *name;

@@ -385,6 +389,11 @@
 	{ "resolverfb", NULL, resolver_video_setup },
 #endif
 
+#ifdef CONFIG_FB_GDK
+	{ "gdkfb", gdkfb_init, gdkfb_setup },
+	{ "gdkfb2", gdkfb2_init, gdkfb2_setup },
+#endif
+
 #ifdef CONFIG_FB_VIRTUAL
 	/*
 	 * Vfb must be last to avoid that it becomes your primary display if

@@ -1190,9 +1199,15 @@
 #else
 #warning What do we have to do here??
 #endif
+#ifndef CONFIG_USERMODE
 	if (io_remap_page_range(vma, vma->vm_start, off,
 			     vma->vm_end - vma->vm_start, vma->vm_page_prot))
 		return -EAGAIN;
+#else
+	if (remap_page_range(vma, vma->vm_start, off,
+			     vma->vm_end - vma->vm_start, vma->vm_page_prot))
+		return -EAGAIN;
+#endif
 #endif /* !__sparc_v9__ */
 	return 0;
 #endif /* !sparc32 */

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com

root@darkstar:~# fbset -i

mode "640x400-70"

    # D: 25.176 MHz, H: 31.469 kHz, V: 69.932 Hz
    geometry 640 400 640 400 8
    timings 39721 40 24 39 9 96 2
    rgba 8/0,8/0,8/0,0/0

endmode

Frame buffer device information:

    Name        : Virtual FB
    Address     : (nil)
    Size        : 0
    Type        : PACKED PIXELS
    Visual      : PSEUDOCOLOR
    XPanStep    : 1
    YPanStep    : 1
    YWrapStep   : 1
    LineLength  : 0

Accelerator : No

Hm. No address or size. That does not look right. However, I performed a quick experiment to see if it was in fact UML and/or the caching hack that was causing this. I built a simple, standard x86 2.6.6 with vfb enabled and booted it on my laptop with the same boot arguments, and fbset -i returned almost exactly the same values! The *only* difference was Address was reported as '0' rather than '(nil)'. I guess this was due to a slight difference in the fbset tool on my laptop and the one in the slackware filesystem I'm using with UML. Either I am just making a mistake somewhere, or vfb is intended to behave this way. Perhaps I'm missing a step and not initialising the framebuffer all the way? :S

I should have been more clear here. SDL is an simple but awesome multimedia library. It's at http://www.libsdl.org/. Not only does it support 2D accel., but it works with a plethora of platforms (it even uses DirectDraw when you use it on win32!). Only problem with it is it indeed uses -lpthread, but I have a number of tricks that could be used to get around that (I list them below).

Interesting. The -lpthread overriding weirdness will not jump across from a shared lib linked against pthread?

I have actually been through a couple of attempts at this project over the last few months I am glad I have not been totally barking up the wrong tree. ;). I have tried a number of tricks such as adding pthread stubs to the kernel, POSIX shared memory, and writing a shared library as an intermediary.

This time around I'm just been starting from the basics again, which is why I'm bothering to make sure basic fb subsystem and vfb are working properly before I attack anything else again. :P

Could I just build UML without the stdio console support? If the UML framebuffer works properly with fbcon, then we don't really need the stdio console anyway. SDL can handle keyboard and mouse input as well, and I will eventually add input support to my driver. Maybe. A good excuse to learn the Linux input system, anyway. ;)

argh! I had not thought of that... I wonder if there is some hack I could do to make it more efficient that just polling. yuck. Oh well. I think I am going to focus on getting a picture out of the thing first. ;)

Oh, one last dumb question. Are the acceleration imgblit, cpyrect, etc. functions in the drivers used by the fbcon system only and can't be leveraged by userspace things like an X11 fbdev driver? Because if that is so I am probably just going to use the cfb* functions for now.

Thanks again! :D

Are you going to OLS next week? It would be awesome to meet you and perhaps pick your brain some more. ;)

Regards,
Andrew Clunis

http://orospakr.is-a-geek.org/



-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com

Different C library, I guess.

Yep, smem_len is 0 on purpose, since accessing vfb from userspace definitely doesn't work on real hardware due to the vmalloc()ed frame buffer. On UML it may work, you can try changing smem_len.

Apparently not.

Yes, you can disable the stdio console. But then you couldn't see kernel messages nor login, which is annoying for debugging :-( Mileage may vary after Gerd's patches.

In a first version these commands would just be `update this region'. Later we can add mode change (resize GTK window), etc...

Map the frame buffer read-only, and trap writes. But to make this efficient, and not redraw on every single byte that's changed, you probably need something like this in your trap handler:
- make everything writeable
- schedule refresh in x jiffies

and in the scheduled refresh:
- refresh screen
- make everything read-only again

The acceleration routines can be used inside the kernel (i.e. fbcon) only. If they would work from user space we wouldn't need the periodic thread or so.

Hmm, one more reason to add sending of those commands to the fbdev core using writev(). Then the app (e.g. X) can keep a shadow buffer and send the appropriate commands to update the screen. Would solve non-standard frame buffer layouts as well, since the app doesn't need to care anymore (imgblit always uses cfb).

Yes I will! CU!

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com