Skip to contentSkip to conversation

Linux OS boilerplate

DISCUSSION Scott Long 17 participants 26 messages Feb 18, 2001 – May 21, 2001

lore
Share & tools
lore

Scott Long

author
#0 lore raw

I've been poring over the x86 boot code for a while now and I've been considering writing a FAQ on the boot process (mostly for my own use, but maybe others will be interested). This would include all relevant information on setting up the x86 hardware for a boot (timers, PIC, A20, protected mode, GDT, initial page tables, initial TSS, etc).

The motivation behind this is that I would like to use the Linux boot system as a boilerplate booter for some experimental code. It's probably much cleaner and more robust than any boot loader I might come up with.

Does there exist an outline (detailed or not) of the boot process from the point of BIOS bootsector load to when the kernel proper begins execution? If not would anyone be willing to help me understand bootsect.S and setup.S enough so that I might write such an outline?

If no one can help me, would you consider it appropriate for me to send email to the people listed in bootsect.S and setup.S asking for assistance?

Thanks,
Scott

#1 lore raw
Quoted 4 lines · Scott Long

Does there exist an outline (detailed or not) of the boot process from the point of BIOS bootsector load to when the kernel proper begins execution? If not would anyone be willing to help me understand bootsect.S and setup.S enough so that I might write such an outline?

I have been over it, and would be willing to help. You should first read all

of the LILO documentation, and check out some of the LinuxBIOS project at www.linuxbios.org.

#2 lore raw
In reply to Scott Long
Quoted 4 lines · Scott Long

Does there exist an outline (detailed or not) of the boot process from the point of BIOS bootsector load to when the kernel proper begins execution? If not would anyone be willing to help me understand bootsect.S and setup.S enough so that I might write such an outline?

It might be a little fundamental but there is *a* boot loading process documented here: http://www.eecs.wsu.edu/~cs460/

Look over all the lecture notes and lab assignments up to the booter lab.

I'd offer up my own explanation, but I'm just starting to learn this stuff.

-karl

#3 lore raw
In reply to Scott Long
Quoted 5 lines

I've been poring over the x86 boot code for a while now and I've been considering writing a FAQ on the boot process (mostly for my own use, but maybe others will be interested). This would include all relevant information on setting up the x86 hardware for a boot (timers, PIC, A20, protected mode, GDT, initial page tables, initial TSS, etc).

It would certainly be a valuable piece for the kernel Documentation dir. Paticularly as people with embedded x86 grow keener and keener to boot biosless to save money and flash.

#6 lore raw
In reply to Scott Long
Quoted 3 lines · Scott Long

I've been poring over the x86 boot code for a while now and I've been considering writing a FAQ on the boot process (mostly for my own use,

[...]

Quoted 2 lines

Does there exist an outline (detailed or not) of the boot process from the point of BIOS bootsector load to when the kernel proper begins

IIRC, there is some useful info contained within loadlin. Also, I found a doc by hpa called "THE LINUX/I386 BOOT PROTOCOL" in my local archive of cruft - I just assumed it was in Documentation/ but apparently it never made it there (yet).

Paul.

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

#4 lore raw
In reply to Alan Cox
Quoted 9 lines · Alan Cox
Quoted 5 lines

I've been poring over the x86 boot code for a while now and I've been considering writing a FAQ on the boot process (mostly for my own use, but maybe others will be interested). This would include all relevant information on setting up the x86 hardware for a boot (timers, PIC, A20, protected mode, GDT, initial page tables, initial TSS, etc).

It would certainly be a valuable piece for the kernel Documentation dir. Paticularly as people with embedded x86 grow keener and keener to boot biosless to save money and flash.

Actually biosless is nice. You don't actually need to do things like enable a20 as there isn't a BIOS to disable it...

As for working code see the linuxBIOS project. http://www.linuxbios.org
There aren't a lot of chipsets supported yet but progress is being made. And you do need something like a BIOS to enable memory and cache before you jump to the linux kernel.

Alan working on all booting x86 biosless, keeps making me chuckle over your hatred towards BIOSes, it is absolutely amazing what linux assumes the BIOS will setup correctly. With linux-2.4 able to do a complete PCI bus setup it isn't as bad it used to be, but it's still pretty significant.

You wouldn't happen to know a good place to put drivers for superio chips, and pci-isa bridges with the superio functions integrated?

I should have a proposal written up (and the code to implement it) in the next month, on how to change the boot process to be more friendly
to BIOS less situations. What we do with the ``empty_zero_page'' is
nasty, and it isn't even the empty_zero_page.

And if anyone has any questions on how the boot process works, I'd be glad to answer specific questions. As I've traced it fairly thoroughly.

Eric

#7 lore raw
In reply to Scott Long
Quoted 8 lines · Paul Gortmaker
Quoted 3 lines · Scott Long

I've been poring over the x86 boot code for a while now and I've been considering writing a FAQ on the boot process (mostly for my own use,

[...]

Quoted 8 lines
Quoted 2 lines

Does there exist an outline (detailed or not) of the boot process from the point of BIOS bootsector load to when the kernel proper begins

IIRC, there is some useful info contained within loadlin. Also, I found a doc by hpa called "THE LINUX/I386 BOOT PROTOCOL" in my local archive of cruft - I just assumed it was in Documentation/ but apparently it never made it there (yet).

It's in there (Documentation/i386/boot.txt).

-hpa

Signature · 3 lines
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
#5 lore raw
In reply to Alan Cox
Quoted 2 lines · Eric W. Biederman

With linux-2.4 able to do a complete PCI bus setup it isn't as bad it used to be, but it's still pretty significant.

For an incomplete subset of chipsets. Serverworks doesn't work correctly for a start (see the threads relating to having to kill the Serverworks fixup code and rely on the BIOS to see all the PCI busses on larger systems). Of course, this is due to Serverworks refusal to release documentation (yes, I've heard the excuses regarding protection of IP), and it's a worrisome. What else are we potentially failing to setup on this chipset ?

Tim

Signature · 4 lines
Tim Wright - [email protected] or [email protected] or [email protected]
IBM Linux Technology Center, Beaverton, Oregon
Interested in Linux scalability ? Look at http://lse.sourceforge.net/
"Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI
#8 lore raw
In reply to Scott Long
Followup to:  <[email protected]>
By author:    Paul Gortmaker <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 9 lines
Quoted 3 lines

Does there exist an outline (detailed or not) of the boot process from the point of BIOS bootsector load to when the kernel proper begins

IIRC, there is some useful info contained within loadlin. Also, I found a doc by hpa called "THE LINUX/I386 BOOT PROTOCOL" in my local archive of cruft - I just assumed it was in Documentation/ but apparently it never made it there (yet).

Documentation/i386/boot.txt

-hpa

Signature · 3 lines
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
#9 lore raw

Linux scalability?

Hi!

I am just writing an essay, an have mentioned TUX as a performance and scalability linearity recort holder with TUX, referencing the specweb99 website summary page:

http://www.spec.org/osg/web99/results/web99.html

However, taking a closer look, it turns out, that the above statement holds true only for 1 and 2 processor machines. Scalability already suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten by IIS 5.0 (8001), and these were measured on the same kind of box!

How come, TUX is soooo good at the lowend (1 and 2 CPUs), and scales this bad?

Signature · 1 line
SaPE - Peter, Sasi - mailto:[email protected] - http://sape.iq.rulez.org/
#10 lore raw

Re: Linux scalability?

Quoted 4 lines

However, taking a closer look, it turns out, that the above statement holds true only for 1 and 2 processor machines. Scalability already suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten by IIS 5.0 (8001), and these were measured on the same kind of box!

not really the same box
look at the disk subsystem
7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0

this is sufficient for a such difference

-David

#22 lore raw

Re: Linux scalability?

In reply to H. Peter Anvin

Why would you want to run a web server with 8 processors rather than four webservers with 2 each?

Sean

Quoted 25 lines · Sasi Peter

Hi!

I am just writing an essay, an have mentioned TUX as a performance and scalability linearity recort holder with TUX, referencing the specweb99 website summary page:

http://www.spec.org/osg/web99/results/web99.html

However, taking a closer look, it turns out, that the above statement holds true only for 1 and 2 processor machines. Scalability already suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten by IIS 5.0 (8001), and these were measured on the same kind of box!

How come, TUX is soooo good at the lowend (1 and 2 CPUs), and scales this bad?

Signature · 7 lines
SaPE - Peter, Sasi - mailto:[email protected] - http://sape.iq.rulez.org/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#11 lore raw

Re: Linux scalability?

In reply to reiser.angus
Quoted 10 lines · reiser.angus
Quoted 4 lines

However, taking a closer look, it turns out, that the above statement holds true only for 1 and 2 processor machines. Scalability already suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten by IIS 5.0 (8001), and these were measured on the same kind of box!

not really the same box
look at the disk subsystem
7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0

this is sufficient for a such difference

I read an article about TUX in the dutch C'T a few months ago (nov/dec 2000, I think) - the real difference (according to the article) was the 2.2.x kernel used in TUX. Look at the stats of the website, they used Redhat 7.0 as base, with kernel 2.2.16. In the C'T, they also used a 2.4-test kernel for TUX, and this one didn't have these "scalibility problems". The problem seemed to be SMP problems with systems with more than two cpus in the 2.2.x-based kernel series. 2.4.x kernels didn't seem to have this problem.

And as far as I know, TUX with 2.4.x kernel was faster than win2k on all SMP-combinations.

--
Ronald

#12 lore raw

Re: Linux scalability?

Quoted 4 lines

I read an article about TUX in the dutch C'T a few months ago (nov/dec 2000, I think) - the real difference (according to the article) was the 2.2.x kernel used in TUX. Look at the stats of the website, they used Redhat 7.0 as base, with kernel 2.2.16.

TUX does not exist on 2.2 kernel
They use a RedHat 7.0 with a 2.4 kernel patched by RedHat (with TUX, zerocopy, etc..)

-David

#13 lore raw

Re: Linux scalability?

Quoted 3 lines · reiser.angus

TUX does not exist on 2.2 kernel
They use a RedHat 7.0 with a 2.4 kernel patched by RedHat (with TUX, zerocopy, etc..)

I am pretty sure the C'T article mentioned that TUX did use a 2.2.x kernel - so it does exist. How else could they make a 2.2.x-kernel based TUX vs. 2.4-test-kernel based TUX comparison?

Too bad the website doesn't mention the kernel number.

--
Ronald

#15 lore raw

[OT] Re: Linux scalability?

In reply to Sasi Peter
Quoted 12 lines · Ronald Bultje
Quoted 10 lines · reiser.angus
Quoted 4 lines

However, taking a closer look, it turns out, that the above statement holds true only for 1 and 2 processor machines. Scalability already suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten by IIS 5.0 (8001), and these were measured on the same kind of box!

not really the same box
look at the disk subsystem
7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0

this is sufficient for a such difference

Quoted 8 lines

I read an article about TUX in the dutch C'T a few months ago (nov/dec 2000, I think) - the real difference (according to the article) was the 2.2.x kernel used in TUX. Look at the stats of the website, they used Redhat 7.0 as base, with kernel 2.2.16. In the C'T, they also used a 2.4-test kernel for TUX, and this one didn't have these "scalibility problems". The problem seemed to be SMP problems with systems with more than two cpus in the 2.2.x-based kernel series. 2.4.x kernels didn't seem to have this problem.

All Tux webservers have run on a 2.4 or 2.4-pre kernel.

Quoted 2 lines

And as far as I know, TUX with 2.4.x kernel was faster than win2k on all SMP-combinations.

Tux held the record for most of the time since last summer, when Linux vaulted into 1st place

Microsoft finally managed to get a better result using an all-out, "bet the farm", "benchmark buster" setup with a special web cache in front of iis.

However, they haven't heard the last of Linux either.

cu

jjs

#16 lore raw

Re: [OT] Re: Linux scalability?

Quoted 3 lines · J Sloan

Microsoft finally managed to get a better result using an all-out, "bet the farm", "benchmark buster" setup with a special web cache in front of iis.

I haven't heard anyone talk about the fact that their 8-cpu numbers got disqualified and aren't even mentioned on the SPEC site on the main tables anymore.

Later,
David S. Miller
[email protected]

#17 lore raw

Re: [OT] Re: Linux scalability?

In reply to Sasi Peter
Quoted 9 lines · David S. Miller
Quoted 3 lines · J Sloan

Microsoft finally managed to get a better result using an all-out, "bet the farm", "benchmark buster" setup with a special web cache in front of iis.

I haven't heard anyone talk about the fact that their 8-cpu numbers got disqualified and aren't even mentioned on the SPEC site on the main tables anymore.

Really? I just checked and it's still there from what I see. We're talking about the Dell 8450/700 w/ IIS & SWC 3.0 result, right? I'm hoping that they're deemed NC, but I don't see it yet...

- Pete

#18 lore raw

Re: [OT] Re: Linux scalability?

Quoted 3 lines · Peter Rival

Really? I just checked and it's still there from what I see. We're talking about the Dell 8450/700 w/ IIS & SWC 3.0 result, right? I'm hoping that they're deemed NC, but I don't see it yet...

Sorry, they are there in the table, but marked as NC.

Maybe you need to hit reload in your browser :-)

Later,
David S. Miller
[email protected]

#19 lore raw

Re: [OT] Re: Linux scalability?

In reply to Sasi Peter
Quoted 9 lines · David S. Miller
Quoted 3 lines · Peter Rival

Really? I just checked and it's still there from what I see. We're talking about the Dell 8450/700 w/ IIS & SWC 3.0 result, right? I'm hoping that they're deemed NC, but I don't see it yet...

Sorry, they are there in the table, but marked as NC.

Maybe you need to hit reload in your browser :-)

Yup, one of them is marked as NC. But the other one is still there (and I hit reload and even shift-reload). So either you're missing the second one or something is not behaving nicely with our web proxies here. While I'd probably be more inclined to believe the latter... ;)

- Pete

#20 lore raw

Re: [OT] Re: Linux scalability?

Quoted 1 line · Peter Rival

Yup, one of them is marked as NC. But the other one is still there

Right, I missed the second one.

That's an interesting trick, the old one gets disqualified because of the 3-month rule, so just submit new results :-)

Later,
David S. Miller
[email protected]

#25 lore raw

Re: Linux scalability?

In reply to H. Peter Anvin

On Fri, May 18, 2001 at 09:17:11AM +0100, Sean Hunter wrote:

[Discussion of SPECWeb results]

Quoted 2 lines

Why would you want to run a web server with 8 processors rather than four webservers with 2 each?

Because you want to win benchmarketing exercises, not demonstrate that your architecture has any value in the real world whatsoever. Because you know that you can induce people with financial approval to make stupid and irrational decisions based on irrelevant data.

Signature · 4 lines
Rodger Donaldson		[email protected]
Klingons do *not* make good programmers.  They make good PFWs and abuse
staff, though.  "You have dishonoured our Ascend!  I should kill you
where you stand!"	-- Malcolm Ray
#21 lore raw

Re: [OT] Re: Linux scalability?

In reply to Sasi Peter
Quoted 7 lines · Peter Rival
Quoted 5 lines · David S. Miller
Quoted 3 lines · J Sloan

Microsoft finally managed to get a better result using an all-out, "bet the farm", "benchmark buster" setup with a special web cache in front of iis.

Quoted 9 lines
Quoted 4 lines

I haven't heard anyone talk about the fact that their 8-cpu numbers got disqualified and aren't even mentioned on the SPEC site on the main tables anymore.

Really? I just checked and it's still there from what I see. We're talking about the Dell 8450/700 w/ IIS & SWC 3.0 result, right? I'm hoping that they're deemed NC, but I don't see it yet...

IIRC they did have some results disqualified, but them these latest results have been submitted since then - perhaps they will be disqualified as well, once the facts come to light...

cu

jjs

#14 lore raw

Re: Linux scalability?

In reply to reiser.angus
Quoted 5 lines · reiser.angus

not really the same box
look at the disk subsystem
7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0
this is sufficient for a such difference

Don't you think that all the really needed stuff could just fit in the enormous ram of the boxes?

Signature · 1 line
SaPE - Peter, Sasi - mailto:[email protected] - http://sape.iq.rulez.org/
#23 lore raw

Re: Linux scalability?

In reply to Sean Hunter
Quoted 2 lines · Sean Hunter

Why would you want to run a web server with 8 processors rather than four webservers with 2 each?

As you might already know, after the interviews to Mingo I assumed, that a major portion of the achievements was enabled by the 2.4 scalability enhacements. That is why I wrote to LKML, to ask about the 2.4 scalability, if anybody out there could tell us about the linux kernel's scalability possibily compared to W2k scalability...

Signature · 1 line
SaPE - Peter, Sasi - mailto:[email protected] - http://sape.iq.rulez.org/
#24 lore raw

Re: Linux scalability?

In reply to Sean Hunter

Yup. The problem is that you're trying to measure scalability in performance of an i/o-bound task by comparing a machine with greater i/o resource but less processing power with one with greater processing but poorer i/o. Surprisingly enough, the one with the best i/o wins. This isn't really a fair comparison between the two platforms.

If you put the same disk array on both machines and got the same results, then you'd have a point.

My point was that in the real world having this configuration for a webserver is unlikely to be sensible at all.

Sean

Quoted 20 lines · Sasi Peter
Quoted 2 lines · Sean Hunter

Why would you want to run a web server with 8 processors rather than four webservers with 2 each?

As you might already know, after the interviews to Mingo I assumed, that a major portion of the achievements was enabled by the 2.4 scalability enhacements. That is why I wrote to LKML, to ask about the 2.4 scalability, if anybody out there could tell us about the linux kernel's scalability possibily compared to W2k scalability...

Signature · 7 lines
SaPE - Peter, Sasi - mailto:[email protected] - http://sape.iq.rulez.org/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/