NHFSSTONE(8) NHFSSTONE(8)
NAME
nhfsstone - Network File System benchmark program
SYNOPSIS
nhfsstone [ -v ] [[ -t secs ] | [ -c calls ]] [ -l load ] [ -p nprocs ]
[ -m mixfile ] [ dir ]...
DESCRIPTION
nhfsstone (pronounced n-f-s-stone, the "h" is silent) is used on a NFS
client to generate an artificial load with a particular mix of NFS
operations. It reports the average response time of the server in mil-
liseconds per call and the load in calls per second. The program
adjusts its calling patterns based on the client’s kernel NFS statis-
tics and the elapsed time. Load can be generated over a given time or
number of NFS calls.
Because it uses the kernel NFS statistics to monitor its progress, nhf-
sstone cannot be used to measure the performance of non-NFS filesys-
tems.
The nhfsstone program uses file and directory manipulation in an
attempt to generate particular NFS operations in response to particular
system calls. To do this it uses several tricks that are based on a
knowledge of the implementation of the NFS client side reference port.
For example, it uses long file names to circumvent the kernel name
lookup cache so that a stat(2) system call generates an NFS lookup
operation.
The mix of NFS operations can be set with a mix file, which is the out-
put of the nfsstat(8C) command (see the "-m" option below). The per-
centages taken from the mix file are calculated based on the number of
NFS calls, not on the percentages printed by nfsstat. Operations with
0% in the mix will never get called by nhfsstone. In a real server
load mix, even though the percentage of call for a particular NFS oper-
ation may be zero, the number of calls is often nonzero. Nhfsstone
makes the assumption that the number of calls to these 0 percent opera-
tions will have an insignificant effect on server response.
Normally nhfsstone should be given a list of two or more test directo-
ries to use (default is to use the current directory). The test direc-
tories used should be located on different disks and partitions on the
server to realistically simulate typical server loads. Each nhfsstone
process looks for a directory <dir>/testdir<n> (where <n> is a number
from 0 to nprocs - 1). If a process directory name already exists, it
is checked for the correct set of test files. Otherwise the directory
is created and populated.
OPTIONS
-v Verbose output.
-t secs Sets calls based on the given running time (in seconds) and
the load.
-c calls Total number of NFS calls to generate (default is 5000).
-l load Load to generate in NFS calls per second (default is 30).
-p nprocs Number of load generating sub-processes to fork (default is
7). This can be used to maximize the amount of load a sin-
gle machine can generate. On a small client machine (slow
CPU or small amount of memory) fewer processes might be
used to avoid swapping.
-m mixfile Mix of NFS operations to generate. The format of mixfile
is the same as the output of the nfsstat(8C) program. A
mix file can be created on a server by typing "nfsstat -s >
mixfile". The default mix of operations is: null 0%,
getattr 13%, setattr 1%, root 0%, lookup 34%, readlink 8%,
read 22%, wrcache 0%, write 15%, create 2%, remove 1%,
rename 0%, link 0%, symlink 0%, mkdir 0%, rmdir 0%, readdir
3%, fsstat 1%.
USING NHFSSTONE
As with all benchmarks, nhfsstone can only provide numbers that are
useful if experiments that use it are set up carefully. Since it is
measuring servers, it should be run on a client that will not limit the
generation of NFS requests. This means it should have a fast CPU, a
good ethernet interface and the machine should not be used for anything
else during testing. A Sun-3/50 can generate about 60 NFS calls per
second before it runs out of CPU.
Nhfsstone assumes that all NFS calls generated on the client are going
to a single server, and that all of the NFS load on that server is due
to this client. To make this assumption hold, both the client and
server should be as quiescent as possible during tests.
If the network is heavily utilized the delays due to collisions may
hide any changes in server performance. High error rates on either the
client or server can also cause delays due to retransmissions of lost
or damaged packets. netstat(8C) -i can be used to measure the error
and collision rates on the client and server.
To best simulate the effects of NFS clients on the server, the test
directories should be set up so that they are on at least two of the
disk partitions that the server exports and the partitions should be as
far apart as possible. The dkinfo(8) command can be used to find the
physical geometry of disk on BSD based systems. NFS operations tend to
randomize access the whole disk so putting all of the nhfsstone test
directories on a single partition or on two partitions that are close
together will not show realistic results.
On all tests it is a good idea to run the tests repeatedly and compare
results. The number of calls can be increased (with the -c option)
until the variance in milliseconds per call is acceptably small. If
increasing the number of calls does not help there may be something
wrong with the experimental setup. One common problem is too much mem-
ory on the client test machine. With too much memory, nhfsstone is not
able to defeat the client caches and the NFS operations do not end up
going to the server at all. If you suspect that there is a caching
problem you can use the -p option to increase the number of processes.
The numbers generated by nhfsstone are most useful for comparison if
the test setup on the client machine is the same between different
server configurations. Changing nhfsstone parameters between runs will
produce numbers that can not be meaningfully compared. For example,
changing the number of generator processes may affect the measured
response time due to context switching or other delays on the client
machine, while changing the mix of NFS operations will change the whole
nature of the experiment. Other changes to the client configuration
may also effect the comparability of results. While nhfsstone tries to
compensate for differences in client configurations by sampling the
actual NFS statistics and adjusting both the load and mix of opera-
tions, some changes are not reflected in either the load or the mix.
For example, installing a faster CPU or mounting different NFS filesys-
tems may effect the response time without changing either the load or
the mix.
To do a comparison of different server configurations, first set up the
client test directories and do nhfsstone runs at different loads to be
sure that the variability is reasonably low. Second, run nhfsstone at
different loads of interest and save the results. Third, change the
server configuration (for example, add more memory, replace a disk con-
troller, etc.). Finally, run the same nhfsstone loads again and compare
the results.
SEE ALSO
The nhfsstone.c source file has comments that describe in detail the
operation of of the program.
ERROR MESSAGES
illegal calls value
The calls argument following the -c flag on the command line is
not a positive number.
illegal load value
The load argument following the -l flag on the command line is
not a positive number.
illegal time value
The time argument following the -t flag on the command line is
not a positive number.
bad mix file
The mixfile file argument following the -m flag on the command
line could not be accessed.
can’t find current directory
The parent process couldn’t find the pathname of the current
directory. This usually indicates a permission problem.
can’t fork
The parent couldn’t fork the child processes. This usually
results from lack of resources, such as memory or swap space.
can’t open log file
can’t stat log
can’t truncate log
can’t write sync file
can’t write log
can’t read log
A problem occurred during the creation, truncation, reading or
writing of the synchronization log file. The parent process cre-
ates the log file in /tmp and uses it to synchronize and commu-
nicate with its children.
can’t open test directory
can’t create test directory
can’t cd to test directory
wrong permissions on test dir
can’t stat testfile
wrong permissions on testfile
can’t create rename file
can’t create subdir
A child process had problems creating or checking the contents
of its test directory. This is usually due to a permission prob-
lem (for example the test directory was created by a different
user) or a full filesystem.
bad mix format: unexpected EOF after ’nfs:’
bad mix format: can’t find ’calls’ value
bad mix format: unexpected EOF after ’calls’
bad mix format: can’t find %d op values
bad mix format: unexpected EOF
A problem occurred while parsing the mix file. The expected for-
mat of the file is the same as the output of the nfsstat(8C)
command when run with the "-s" option.
op failed:
One of the internal pseudo-NFS operations failed. The name of
the operation, e.g. read, write, lookup, will be printed along
with an indication of the nature of the failure.
select failed
The select system call returned an unexpected error.
BUGS
Running nhfsstone on a non-NFS filesystem can cause the program to run
forever because it uses the kernel NFS statistics to determine when
enough calls have been made.
Nhfsstone uses many file descriptors. The kernel on the client may have
to be reconfigured to increase the number of available file table
entries.
Shell scripts that used nhfsstone will have to catch and ignore SIGUSR1
(see signal(3)). This signal is used to synchronize the test pro-
cesses. If the signal is not caught the shell that is running the
script will be killed.
FILES
/vmunix system namelist
/dev/kmem kernel virtual memory
./testdir* per process test directory
/tmp/nhfsstone%d process synchronization log file
4 October 1989 NHFSSTONE(8)
