How To Find and Overcome Shell Command Line Length Limitations
Q. While using mv or rm command I get command line length error (Argument list too long error). How do I find out current running shell command line length limitations? How do I overcomes these limitations while writing UNIX / BSD / Linux shell utilities?
A. All shell have / has a limit for the command line length. UNIX / Linux / BSD system has a limit on how many bytes can be used for the command line argument and environment variables. When you start a new process or type a command these limitations are applied and you will see an error message as follows on screen:
Argument list too long
How do I find out current command line length limitations?
Type the following command (works under Linux / UNIX / BSD operating systems):
$ getconf ARG_MAX
Sample output:
262144
BSD operating system also supports following command:
$ sysctl kern.argmax
Sample output:
kern.argmax=262144
To get accurate picture about limitation type the following command (hat tip to Jeff):
$ echo $(( $(getconf ARG_MAX) - $(env | wc -c) ))
Output:
261129
How do overcome shell command line length?
You have following option to get around these limitations:
- Use find or xargs command
- Use shell for / while loop
find command example to get rid of "argument list too long" error
$ find /nas/data/accounting/ -type f -exec ls -l {} \;
$ find /nas/data/accounting/ -type f -exec /bin/rm -f {} \;
xargs command example to get rid of "argument list too long" error
$ echo /nas/data/accounting/* | xargs ls -l
$ echo /nas/data/accounting/* | xargs /bin/rm -f
while loop example to get rid of "argument list too long" error
ls -1 /nas/data/accounting/ | while read file; do mv /nas/data/accounting/$file /local/disk/ ; done
Alternatively, you can combine above methods:
find /nas/data/accounting/ -type f | while read file do mv /nas/data/accounting/$file /local/disk/ done
time command - give resource usage
Use time command to find out exact system resource usage for each command:
$ time find blah blah
$ time ls -1 blah | while read file; do #blah on $file; done
Further readings:
- Your shell documentation
- man pages ksh, bash, getconf, sysconf, sysctl, find, and xargs