Background Jobs on the HPs

• Policies on background jobs
• Foreground vs. background jobs
• Output and input for background jobs
• Redirecting input and output
• Unattended background jobs
• Running multiple unattended background jobs is not allowed
• Commands for manipulating jobs

• If you are actually sitting at a machine, there are no limits on the number of background jobs you may have on that machine.
• You are limited to one unattended background job, on one machine, at a time.
• That job must be `nice'd (that is, it must be set to run at a low priority) to minimize the impact on other users of the machine.
• Locking the screen and leaving the machine is not an acceptable way of running unattended jobs.
• If you violate these rules, your background jobs will be killed and you may lose your computing privileges.

When you type a command at a UNIX shell prompt, normally the shell will wait until that command has completed before prompting you for another command. This is called foreground processing. Only one command at a time can be running in the foreground in any window, because the shell won't prompt for another until the command that is running has finished.

If you don't want to wait for a command to finish before you see the prompt again, you can run the command in the background. Any shell command can be run in the background by typing an ampersand (&) at the end of the command line. The following command:

wc /hp-ux &

will count the number of bytes in the UNIX kernel, and it will take a second or two to complete. Try it with and without the ampersand and notice how the shell doesn't wait for the command to complete when you execute it in the background. When you start a background job, you'll get a message that looks like this:

[2] 8397

The number in brackets is a `job number' assigned by the shell to the background job, and the second number is a `process number' assigned by the operating system. When a background job is completed, you'll get a notice from the shell that looks like this:

[2] Done wc /hp-ux

When background jobs have output, they write it to the screen. When the current foreground job and one or more background jobs are all writing the the screen at once, the outputs are mingled together and the result can be very confusing.

When background jobs require input from the keyboard, they stop running. You must bring them into the foreground in order to supply them with input from the keyboard.

Because of this, it is usually a good idea to redirect the input and output of background jobs.

The output of any shell command can be redirected to a file (instead of the screen) by using a greater than (>) character, followed by the file name. The less than (<) character is an input redirection operator - it will cause the command to read its input from a file rather than from the keyboard.

Output redirection can be used for any command, not just background commands. For instance, the command

ls -R ~ >listing

will do a directory listing of your home directory and all of its subdirectories and save the output in a file called listing. You do not need to create the output file in advance---it will be created if it does not already exist. Caution! When you redirect output to an existing file, that file is overwritten. The previous contents will be lost. If you wish to append the output of a command to a file, use the redirection operator >> instead of >. To use input redirection, you must first create the file and put some appropriate input into it. You can do this with an editor such as emacs or vi. For instance, if the file testrun contains a series of Matlab commands, then the command

matlab <testrun >testrun.out

will cause Matlab to take its commands from the file testrun and to put the results in the file testrun.out.

When you logout, any jobs you had running in the background will continue to run until they are done. If you redirected the output to a file, you will be able to look at that output later. This is very convenient for jobs involving lots of numerical calculations, or any other task which takes minutes or hours to complete, as you don't have to sit and wait for the result. These jobs are called unattended background jobs, because you are not sitting at the computer, waiting on the result.

However, it is a rule of this computing site that unattended jobs must be nice'd. This means that they run at a lower priority. The jobs will still run to completion, but they will not interfere with other people using the machine. For example,

nohup nice matlab < inputfile > outputfile &

will run Matlab in the background at a lower priority (with input and output redirected to files). csh and tcsh have a built in nice command that follows the syntax listed above. If you are using a different shell, or if you are explicitly invoking /bin/nice, see the man page for the correct syntax.

If you forget to nice a background job when you start, or if the job takes longer than you anticipated and you decide to log out and go to lunch, you can use the command renice -n 19 to lower the priority of already-running jobs. The argument to the renice command is a process number (the second of the two numbers reported when you started the job.) For example, the command:

renice -n 19 8397

will lower the priority for the job whose process number is 8397.

If you've forgotten the process number, you can use the command

jobs -l

to list all of your jobs and their process numbers. Note: the jobs command only knows about jobs started from the same window. See the manual page for the ps command to learn how to find out about jobs started from a different window.

It is a rule of this computer site that you can only run one unattended background job, on only one machine, at a time. However, often it's necessary to run several calculations, or run the same lengthy calculation several times with different inputs. In this case, you must arrange for the calculations to run one at a time instead of starting up many separate background jobs.

To run your commands one at a time, create a file containing a list of the commands you want to run. This is called a `shell script'. Then you can run the shell script, and each of the commands in it will be executed in turn. You run a shell script by typing its file name at the shell prompt (don't forget the & if you want it to run in the background and nice if you plan to leave before it finishes.) You may need to set execute permission on the file so that the script is runnable---use the chmod command, like this:

chmod u+x filename

The shell provides a number of commands for manipulating background jobs:

Consult the csh manual page for more details on how to use these commands. The renice and ps commands have their own manual pages.