Wednesday, January 19, 2011

*nix measure memory usage

Hi to all, i'd like to measure the total memory (like RAM) used by a program during its execution. Ideally, something like time that runs the program and print stats when it exits.

Is there such a standard tool in the unix / linux / bsd world? Thanks.

EDIT:

the program i'm trying to profile takes fractions of a second to execute, so anything that implies "while running check this" won't work. that's why i'm asking for a "time" like tool.

  • On Linux you can read /proc/[pid]/status; the VmPeak field is the maximum virtual memory size, VmHWM is the maximum resident set size.

    The getrusage() syscall might, or might not, help. The struct rusage contains e.g. a maxrss field, but at least on Linux this is never filled in.

    From janneb
  • No standard tool except ps (which is not so standard in regards to options/output too). This will not give you time-like usability.

    You can use nmon to gather system-wide statistics, it captures memory usage of processes (it will be probably useless for short-lived processes). It supports many platforms.

    From kubanczyk
  • The strace command might be of some use. It will show you in what system calls the time is being taken up:

    $ strace -cTv zip /tmp/test.zip /usr/share/dict/words
      adding: usr/share/dict/words (deflated 73%)
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
    100.00    0.000055           2        34           read
      0.00    0.000000           0        21           write
      0.00    0.000000           0        12         3 open
      0.00    0.000000           0         9           close
      0.00    0.000000           0         3         3 access
      0.00    0.000000           0         1           rename
      0.00    0.000000           0         3           brk
      0.00    0.000000           0         1           gettimeofday
      0.00    0.000000           0         4           munmap
      0.00    0.000000           0         3           mprotect
      0.00    0.000000           0         6           _llseek
      0.00    0.000000           0         6           rt_sigaction
      0.00    0.000000           0        12           mmap2
      0.00    0.000000           0         8         1 stat64
      0.00    0.000000           0         1         1 lstat64
      0.00    0.000000           0         9           fstat64
      0.00    0.000000           0         1           fcntl64
      0.00    0.000000           0         1           set_thread_area
    ------ ----------- ----------- --------- --------- ----------------
    100.00    0.000055                   141         8 total
    
    pistacchio : how can i then relate time with memory usage?
    davey : If you've run it with the `-c`. summary you now have a list of what calls are being made. Calls like `mmap2`, `brk` indicate memory assignment (there may be others). Run under `strace` again but with something like `strace -o /tmp/program.trc ...` then you will be able to see what `brk`, `mmap` are assigning. If you are having problems with something like this you should refer it to the developer of the programs you are running, they should be able to help.
    From davey
  • If you don't mind repeating the execution a few times you can use ulimit -Hv to set the memory limit for the shell (in Bash) and then binary search the minimum when the application successfully exits.

    From pehrs
  • You could try writing your own script that could do this. It could get the PID of the launched program and use ps or /proc/[pid]/status to get memory information about it. Of course this wouldn't totally characterize the memory usage but it should give you a good idea.

  • I'd suggest valgrind. The massif app will report heap usage stats, at least.

  • Just use process accounting (acct or psacct depending on your distribution). The process accounting log includes "memory used" statistics.

    From womble

0 comments:

Post a Comment