High network usage over network filesystem
I have a weird problem. I'm using dosbox to access a mounted network partition. The problem is that every time a command or a program is launched, if your current working directory (of msdos, not linux) is in the novell partition, it takes forever to start. The weird thing was that the cpu was idle all the time, so I looked into it and saw that the network usage would skyrocket. This happens specially when you're on a directories with lots of files in them, and I need to run a lot of commands in directories that have over 4000 files. This means that every time I try to launch something it will usually sit there for 5-6 seconds before doing anything, and that means that it's impossible to execute some bats we now execute under dos (they invoke a program more than a thousand times). Doing an strace, I can see it "hangs" after doing a getdents call, so I suppose that for some reason it's retrieving info of the files on the current directory or something.
Why it's doing that? Is it a dosemu thing or a freedos thing? Any idea how I can get around it?
Quoted 2 lines · On Tue, 27 Jan 2009 10:13:28 +0100 Javier Tarifa wrote:
I have a weird problem. I'm using dosbox to access a mounted network partition.
1. Are you having problems using dosbox or dosemu? Despite the similarity of the names, they are two different dos emulators.
2. "mounted network partition". Do you mean nfs? Samba? Something else?
Quoted 3 lines
The problem is that every time a command or a program is
launched, if your current working directory (of msdos, not linux) is in the novell partition,
Are you talking about Novell Netware?
Quoted 3 lines
it takes forever to start. The weird thing was
that the cpu was idle all the time, so I looked into it and saw that the network usage would skyrocket.
What if you just type dir at a dosemu prompt?
Describe more about your setup, how you're mounting your network partition and what it consists of and how it's structured, and how you're accessing your data files.
Signature · 2 lines
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com DRY CLEANER BUSINESS FOR SALE ~ http://www.canadadrycleanerforsale.com
Quoted 4 lines
1. Are you having problems using dosbox or dosemu? Despite the similarity of the names, they are two different dos emulators.
Sorry about the confusion dosbox/dosemu, i'm definitely using dosemu.
Quoted 2 lines
2. "mounted network partition". Do you mean nfs? Samba? Something else?
The computer dosemu is running on is a fedora that mounts a remote novell netware partition using ncpmount, and I use lredir to make it my f:
Quoted 6 lines
What if you just type dir at a dosemu prompt?
Describe more about your setup, how you're mounting your network partition and what it consists of and how it's structured, and how you're accessing your data files.
It doesn't matter how I acces my files. After every command I type there are several seconds of high network usage before doing anything, even running "dir" or running an "unix" command. The only thing that is not slowed down is changing directory and changing the device This time of doing nothing seems to be proportional to the amount of files in the current directory.
Here's an example:
test.bat contains this:
call time /t
call dir > null
call time /t
call unix "echo hi"
call time /t
running it on the root of the network device, on f:, where there are only 44 files, gives this:
F:\>test.bat
F:\>call time /t
Current time is 6:47:34.97 pm
F:\>call dir > null
F:\>call time /t
Current time is 6:47:35.68 pm
F:\>call unix "echo hi"
hi
F:\>call time /t
Current time is 6:47:35.84 pm
But then running on the directory where everything needs to be run, with 3830 files on the current directory, gives you this:
F:\somedir>test.bat
F:\somedir>call time /t
Current time is 6:51:20.16 pm
F:\somedir>call dir > null
F:\somedir>call time /t
Current time is 6:51:41.20 pm
F:\somedir>call unix "echo hi"
hi
F:\somedir>call time /t
Current time is 6:51:44.22 pm
On a windows prompt it runs like this:
F:\somedir>test.bat
F:\somedir>call time /t
18:53
F:\somedir>call dir 1>null
F:\somedir>call time /t
18:53
F:\soemdir>call unix "echo hi"
"unix" no se reconoce como un comando interno o externo, programa o archivo por lotes ejecutable.
F:\somedir>call time /t
18:53
Any ideas why is this slow? I wouldn't mind if doing a dir or something like this lasted a little more than normal, but those 3-4 seconds that is "hanged" everytime anything is trying to execute makes that, for example, compiling can take hours with dosemu and before were a matter of a couple of minutes or so (the compiler is called 2000 times more or less, but it finishes at the moment)
If I can give any more info, please tell me.
Quoted 85 lines · Javier Tarifa
Quoted 4 lines
1. Are you having problems using dosbox or dosemu? Despite the similarity of
the names, they are two different dos emulators.Sorry about the confusion dosbox/dosemu, i'm definitely using dosemu.
Quoted 1 line
2. "mounted network partition". Do you mean nfs? Samba? Something else?
The computer dosemu is running on is a fedora that mounts a remote novell netware partition using ncpmount, and I use lredir to make it my f:
Quoted 7 lines
What if you just type dir at a dosemu prompt?
Describe more about your setup, how you're mounting your network partition and
what it consists of and how it's structured, and how you're accessing your data
files.It doesn't matter how I acces my files. After every command I type there are several seconds of high network usage before doing anything, even running "dir" or running an "unix" command. The only thing that is not slowed down is changing directory and changing the device This time of doing nothing seems to be proportional to the amount of files in the current directory.
Here's an example:
test.bat contains this:call time /t
call dir > null
call time /t
call unix "echo hi"
call time /trunning it on the root of the network device, on f:, where there are only 44 files, gives this:
F:\>test.bat
F:\>call time /t
Current time is 6:47:34.97 pm
F:\>call dir > null
F:\>call time /t
Current time is 6:47:35.68 pm
F:\>call unix "echo hi"
hi
F:\>call time /t
Current time is 6:47:35.84 pmBut then running on the directory where everything needs to be run, with 3830 files on the current directory, gives you this:
F:\somedir>test.bat
F:\somedir>call time /t
Current time is 6:51:20.16 pm
F:\somedir>call dir > null
F:\somedir>call time /t
Current time is 6:51:41.20 pm
F:\somedir>call unix "echo hi"
hi
F:\somedir>call time /t
Current time is 6:51:44.22 pmOn a windows prompt it runs like this:
F:\somedir>test.bat
F:\somedir>call time /t
18:53F:\somedir>call dir 1>null
F:\somedir>call time /t
18:53F:\soemdir>call unix "echo hi"
"unix" no se reconoce como un comando interno o externo, programa o archivo por lotes ejecutable.F:\somedir>call time /t
18:53Any ideas why is this slow? I wouldn't mind if doing a dir or something like this lasted a little more than normal, but those 3-4 seconds that is "hanged" everytime anything is trying to execute
makes that, for example, compiling can take hours with dosemu and before were a matter of a couple of minutes or so (the compiler is called 2000 times more or less, but it finishes at the moment)If I can give any more info, please tell me.
Signature · 16 lines
It look as Novell NW access is slow - and as DOS (in contrast in Unix, which look for command wo full path only in PATH environ var) first look for command in current directory, then when there is several thousands files, this search will take a while. When You know about command full path, then it may help: C:\bin\command CMDLINE However I not know, what it does in case of internal command.com commands (dir, cd, etc). Franta Hanzlík -- To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · On Tue, 27 Jan 2009 19:02:34 +0100 Javier Tarifa wrote:
If I can give any more info, please tell me.
It would be interesting to see what happens if you put your data on a nfs filesystem and then try running your compiler on that.
Then put it on a local filesystem and see what happens there.
Signature · 2 lines
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com DRY CLEANER BUSINESS FOR SALE ~ http://www.canadadrycleanerforsale.com
Quoted 9 lines
Quoted 3 lines
If I can give any more info, please tell me.
It would be interesting to see what happens if you put your data on a nfs filesystem and then try running your compiler on that.
Then put it on a local filesystem and see what happens there.
Nevermind, I've got it.
When you try to execute something, scan_dir scans all the files on your path and on your current directory searching for it.
Even more, it scans multiple times searching for different name combinations (exe/com/bat and with different capitalization), and that's what causing huge bandwidth consumption and stalls when you're on a big directory. Luckily for me, as I said in another mail I'm mounting the remote novell partition with ncpmount that makes all acceses to the mounted device case-insensitive, so I just made scan_dir return true always, and now I think it determines if a file it's there or not by stating it in find_file. I have encountered no problems with this yet.
Another thing, I don't know if this is the proper place to ask for it, but I've found that dosemu doesn't work right with dtach. When you reattach you simply get a black screen and all the text, windows, pictures, etc are gone. I can't use screen because for some reason some programs just hang when they're executed in dosemu inside a screen, but they work flawlessly in dtach. It would be great if you could do something about this. To get around it I added this at the end of sigwinch:
SLsmg_touch_lines(0,vga.text_height);SLsmg_refresh();
It does work now, but I know nothing of slang and the docs say you should not use SLsmg_touch_lines, so it has to be a better way. It would be great if in the next version dosemu and dtach could get along.
Javier Tarifa escreveu:
Quoted 7 lines
Quoted 1 line
Nevermind, I've got it.
When you try to execute something,scan_dirscans all the files on your path and on your current directory searching for it.
Even more, it scans multiple times searching for different name combinations (exe/com/bat and with different capitalization), and that's what causing huge bandwidth consumption and stalls when you're on a big directory. Luckily for me, as I said in another mail I'm mounting the remote novell partition with ncpmount that makes all acceses to the mounted device case-insensitive, so I just madescan_dirreturn true always, and now I think it determines if a file it's there or not by stating it infind_file. I have encountered no problems with this yet.
Could you explain this a little more... It seems that you have found the reason why FreeDOS is slow sometimes.
Maybe that scan_dir could be re-written so as to scan only once ant test all name combinations for each file in turn.
Alain
Quoted 13 lines
Another thing, I don't know if this is the proper place to ask for it, but I've found that dosemu doesn't work right with dtach. When you reattach you simply get a black screen and all the text, windows, pictures, etc are gone. I can't use screen because for some reason some programs just hang when they're executed in dosemu inside a screen, but they work flawlessly in dtach. It would be great if you could do something about this. To get around it I added this at the end of sigwinch:
SLsmg_touch_lines(0,vga.text_height);SLsmg_refresh();It does work now, but I know nothing of slang and the docs say you should not use
SLsmg_touch_lines, so it has to be a better way. It would be great if in the next version dosemu and dtach could get along.