How To Find and Overcome Shell Command Line Length Limitations

12:58:00 PM 0 Comments

Q. While using mv or rm command I get command line length error (Argument list too long error). How do I find out current running shell command line length limitations? How do I overcomes these limitations while writing UNIX / BSD / Linux shell utilities?

A. All shell have / has a limit for the command line length. UNIX / Linux / BSD system has a limit on how many bytes can be used for the command line argument and environment variables. When you start a new process or type a command these limitations are applied and you will see an error message as follows on screen:

Argument list too long

How do I find out current command line length limitations?

Type the following command (works under Linux / UNIX / BSD operating systems):
$ getconf ARG_MAX
Sample output:

  262144

BSD operating system also supports following command:
$ sysctl kern.argmax
Sample output:

  kern.argmax=262144

To get accurate picture about limitation type the following command (hat tip to Jeff):
$ echo $(( $(getconf ARG_MAX) - $(env | wc -c) ))
Output:

  261129

How do overcome shell command line length?

You have following option to get around these limitations:

  • Use find or xargs command
  • Use shell for / while loop

find command example to get rid of "argument list too long" error

$ find /nas/data/accounting/ -type f -exec ls -l {} \;
$ find /nas/data/accounting/ -type f -exec /bin/rm -f {} \;

xargs command example to get rid of "argument list too long" error

$ echo /nas/data/accounting/* | xargs ls -l
$ echo /nas/data/accounting/* | xargs /bin/rm -f

while loop example to get rid of "argument list too long" error

  ls -1 /nas/data/accounting/ | while read file; do mv /nas/data/accounting/$file /local/disk/ ; done

Alternatively, you can combine above methods:

  find /nas/data/accounting/ -type f |    while read file    do      mv /nas/data/accounting/$file /local/disk/    done

time command - give resource usage

Use time command to find out exact system resource usage for each command:
$ time find blah blah
$ time ls -1 blah | while read file; do #blah on $file; done

Further readings:

  • Your shell documentation
  • man pages ksh, bash, getconf, sysconf, sysctl, find, and xargs

Some say he’s half man half fish, others say he’s more of a seventy/thirty split. Either way he’s a fishy bastard.