About reading /proc/*/mem

Ok, so as a rehash, the ptrace & open(),lseek() on /proc/*/mem should both work about the same. After a lot of struggling, I've gotten the ptrace to work right & spit out the data I want/need. However there is one small problem, SIGSTOP.

ptrace() appears to set up the child process to do a SIGSTOP whenever any interrupt is received. Which is kind of a bad thing for what I'm looking to do. I guess I'm trying to write a non-intrusive debugger that can be used to view static variables stored in the heap of an application.

On other OS's, this can be done just by popping open /proc/*/mem, and reading the data as needed, allowing the child process to continue processing away as if nothing is going on. I'm looking to do the same sort of task under Linux.

Unfortunately, ptrace() probobally isn't going to allow me to do that. So my next question is does opening /proc/*/mem force the child process to stop on every interrupt (just like ptrace?)

Second, I would imagine opening /dev/mem (or /proc/kcore) would get me into the physical memory of the system itself. How would I know what the starting physical memory addresses of a processes data is to start at:

What I do see is under /proc/*/maps is the entries:

08048000-08049000 r-xp 00000000 03:01 718368     /devel/mem_probe/child
08049000-0804a000 rw-p 00000000 03:01 718368     /devel/mem_probe/child
40000000-40015000 r-xp 00000000 03:01 310089     /lib/ld-2.2.so
40015000-40016000 rw-p 00014000 03:01 310089     /lib/ld-2.2.so

40016000-40017000 rwxp 00000000 00:00 0
40017000-40018000 rw-p 00000000 00:00 0
40027000-4013f000 r-xp 00000000 03:01 310096 /lib/libc-2.2.so
4013f000-40144000 rw-p 00117000 03:01 310096 /lib/libc-2.2.so
40144000-40148000 rw-p 00000000 00:00 0
bfffe000-c0000000 rwxp fffff000 00:00 0

I would assume that this tells me that memory addresses 0x08049000-0x804a000 are mapped to the physical address of 0x718368. However Going to this address, and then doing an lseek(SEEK_CUR)out to my expected variable offset doesn't get me the result I'm expecting. Is the 0x718368 the right location to be looking at, or is there some translation that needs to get done (* by page size, translate into hex/from hex, etc.) I can't find any documentation indicating what each column represents so it's just a stab on my part.

Thanks for the good information so far.

--Rich

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

The not stopping the child should be the major difference between /proc/*/mem and ptrace.

You don't even want to go there. You've got the wrong model in your head.

Nope 0x718368 is the inode of an on-disk file.

man proc or reading the source works.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

See this is where I start seeming to have problems. I can open /proc/*/mem & lseek, but reads come back as "No such process". However, if I first do a ptrace(PTRACE_ATTACH), then I can read the data, but the process stops. I've kind of dug through the sys_ptrace() code under /usr/src/linux/arch/i386/kernel/ptrace.c, and can see and understand generally what it's doing, but that's getting into serious kernel-land stuff. I wouldn't expect it to be this difficult to just open up another processes /proc/*/mem file to read data from.

Is there something obvious I'm missing? It seems to keep pointing back to ptrace & /proc/*/mem are very closely related (ie: the same) including stopping of the child.

Thanks.

--Rich

OK, here's something I really don't understand. Suppose that I tell your debugger to tell me when in the executed program foo becomes greater than bar[0] + 14. Or when cyclic list foo becomes longer than 1 element (i.e. foo.next != foo.prev).

How do you do that if program is running? If you don't guarantee that it doesn't run during the access to its memory (moreover, between sever such accesses) - the data you get is worthless.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Actually what we are looking to do is a lot more simplistic than that what you are suggesting. The RT applications is the Host portion of a flight Sim. So for example, we want to see what the system thinks it's current altitude is at, or what position the throttle is at, etc. So there isn't a lot of dynamic list generation, or pointer manipulation going on. Having dynamic data generation and/or pointers is another nasty hole that gets opened up that nobody here is quite ready to step into.

So the majority of the data is static, as well as small enough to be atomic. Mostly 32-bit floats & ints with occaisional calls to 64-bit floats & ints.

Don't forget too, that since this is running in a real-time environment, that alot of things are expected to be guaranteed. We know a certain process is going to be running at 50 Hz, or 200Hz. So when it's timeslice is complete it really better be done or else we have architecture problems. The child will be getting interrupts & starting, running, & finishing 50 times a second, meanwhile, the debug process may be running at 1 Hzand updating relatively slowly. Assuming this is a uniprocessor, the child process should be totally done running when the debug process goes in to get updates. On an SMP there will probobally be some synchronization problems, but we'll worry about that when we get there.

What I'm trying to do is port an application that was working on Suns, SGI's & Concurrent PowerHawks over to Linux. All three used the /proc to access the memory, and Linux doesn't seem to offer the same kind of support. Which brings me back to my original question of how to tie into the process's memory without interrupting it's execution.

--Rich

Could somebody tell me what would one do with data read from memory of process that is currently running?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

The main thing I'm looking to do is examine data that's part of a real-time process. The process's execution can't be interrupted, otherwise it makes debugging it inaccurate. The applications is certainly not looking to see every line of code get executed, but rather have a real-time monitor of a symbol as it gets modified. Now viewing/selecting the symbol is done through a combination of nm's & a console based util (hopefully GTK Based in the future). Other applications include recording this data to disk for later playback & analysis.

Now the next logical step would be to create a debug module in the RT system itself that dumps out the values we care about. The problem with this is we are looking at a lot of legacy code (done in fortran, C & Ada) as well as tons of variables. By peeking at the memory on the fly we can dynamically decide which values are important for this run, without having to record all possible data to the disk (which in itself would be quite painful since disk accesses would make debugging again difficult).

Granted, it's probobally not a very popular application, but it's still one which is present in many of the Big Unixes, and so far has me stumped as to how to get it working correctly under Linux.

--Rich

You want the data in each frame be taken at the same moment. Otherwise you are going to see inconsistent data. I.e. tons of false warnings saying that data corruption is going on when there's none.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Hi,

As long as we have the appropriate page table lock while doing the physical page lookup, and grab a refcount on the page with the lock held, we'll get a valid physical memory location to read to the user. We don't need any stronger guarantee than that --- if the target process is playing mmap games or modifying the memory while the read happens, the result is unpredictable but safe.

Cheers,
Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

After doing some digging around, I found this URL: (Sorry, the original page at nsa seems to have disappeared).

http://www.google.com/search?q=cache:nsa.gov/selinux/slinux-200101020953/node57.html+access+physical+memory+/proc+/mem&hl=en

In any case, it indicates the the /proc/*/mem file can only be read by the process itself on the fly, or by a parent process that has stopped execution of the child via SIGSTOP. So it seems so far that the behavior of /proc/*/mem is exactly the same as the behavior of ptrace in that it forces stoppage of execution in order to read memory. Bummer.(Or has this changed from v 2.2.x to v2.4.x?)

So how else can I access the process memory? I'm wondering if it'd be feasible to hack the kernel to add an extra ptrace_nostop() fn that would allow ptracing without forcing a stop in the process. I'd really rather not have to hack the kernel unless I really have to though.

My second thought is again going back to finding the translation of where the process physically lives in the virtual memory itself. So this way I can just go and directly look at the memory of the process. But then this runs into problems as to finding where that process lives, and being told my thoughts on this are totally wrong.

I suppose the question is I know the process exists, I know the process is pinned into memory so it can't get swapped out, I just need to know how to translate the process's virtual address of 0x080495998 to some chunk of physical memory, and then take a look at it.

Thanks.

--Rich

Well, just wanted to send an e-mail to let the group know what I've found & what my current workaround is, as well as make sure that it's recorded somewhere that will get searchable to save future hackers the trouble.

So far the story is I'm trying to get access to another processes memory structure for non-intrusive debugging (mainly to just track statically allocated variables to see how they change). The technique used on other Operating systems was to simply open /proc/pid/mem, lseek to the proper location, and read the value directly from the processes mapped memory.

Unfortunately, on Linux this is not the case. It appears that /proc/pid/mem is not available to another process, unless there is a ptrace(PTRACE_ATTACH) to bind the two processes together. However, the problem with this method is that ptrace forces the child process to do a SIGSTOP whenever any signal is received. In a real-time system, this isn't a good method to debug a process when it's important for the process being debugged to run correctly.

This brings up an interesting feature though, if you access /proc/self/mem, you can access anywhere in that process. A few people suggested using a debugging process, or debugging routines which would again be a bit intrusive for use. However, if you make a child thread, the child thread will get executed independently, but still have full access to the native application's memory. This will basically provide the functionality I'm looking for.

So in conclusion, I have a call to pthread_create(thread_ptr,NULL,debug_fn,NULL) which is used to create a new thread that is designed to run the debug_fn function, which is the main routine from the original debug application.

Thanks to everyone for answering my questions. I think a few people suggested I look at threads, but I don't remember who you were. Thanks again for your help.

BTW: As a future feature suggestion, what about having a /proc/pid/nbmem that would provide non-blocking access to a processes memory. I would think it'd be kind of a redundant hack in the kernel to allow this, and since it will be a security concern make it an option you have to turn on in the kernel for recompilation (of course if it was a module that'd be sweeter still).

--Rich

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/