fsw: a File System Event Monitor for OS X and *BSD Systems

A very often demanded feature by many different types of users is the possibility of receiving an event when a change in a specific file or directory occurs. One of the most common way to consume such an event in a UNIX(-like) system would be:

• Having a command which lets you watch a set of files or directories outputting on standard output a detailed record for each event received.
• Piping such a process to a shell script that reacts accordingly.

Many UNIX and UNIX-like systems provide kernel facilities, subsystems and programs providing the possibility of receiving file system change events:

• Silicon Graphic's portable FAM.
• FreeBSD's kqueue, originated in FreeBSD and available on many *BSD systems.
• Linux inotify.
• Apple OS X FSEvents API.

However, unlike Linux (where inotifywatch is available), neither OS X nor kqueue-enabled *BSD systems provide a user-land program for users to take advantage of those APIs. That's why I decided to fill the gap and write a small C++ program: fsw, a shell-friendly file system event notifier for OS X and kqueue-enabled *BSD systems, hosted on GitHub.

fsw

fsw will let you subscribe to file system change events on multiple files or directories:

$ fsw file-0 ... file-n

When a file system event is received, fsw prints a complete record of the event such as

Sat Feb 15 00:26:30 2014 - /full/path/to/file:created renamed modified changeOwner isFile 

Sat Feb 15 00:26:31 2014 - /full/path/to/file:created renamed modified changeOwner xattrMod isFile

that include the following information:

• The timestamp when the event was received.
• The full path of the changed file or directory.
• The list of event flags, describing the type of change that occurred.

Watchers

fsw implements three kind of watchers:

• A watcher based on the File System Events API, available only on Apple OS X.
• A watcher based on kqueue, an event notification interface introduced in FreeBSD 4.1 and supported on most *BSD systems (including OS X).
• The poll watcher, a watcher which periodically stats the file system, saves file modification times in memory and manually calculates file system changes.

The limitations of fsw depend largely on the watcher being used:

• The FSEvents watcher, available only on Apple OS X, has no known limitations and scales very well with the number of files being observed.
• The kqueue watcher, available on any *BSD system featuring kqueue, requires a file descriptor to be opened for every file being watched. As a result, this watcher scales badly with the number of files being observed and may begin to misbehave as soon as the fsw process runs out of file descriptors. In this case, fsw dumps one error on standard error for every file that cannot be opened.
• The poll watcher, available on any platform, only relies on available CPU and memory to perform its task. The performance of this watcher degrades linearly with the number of files being watched.

Recommendations

• On OS X, use only the FSEvents watcher.
• If the number of files to observe is sufficiently small, use the kqueue watcher. Beware that on some systems the maximum number of file descriptors that can be opened by a process is set to a very low value (values as low as 256 are not uncommon), even if the operating system may allow a much larger value. In this case, check your OS documentation to raise this limit on either a per process or a system-wide basis.
• If feasible, watch directories instead of watching files.
• If none of the above applies, use the poll watcher. The authors' experience indicates that fsw requires approximately 150 MB or RAM memory to observe a hierarchy of 500.000 files with a minimum path length of 32 characters. A common bottleneck of the poll watcher is disk access, since stat()-ing a great number of files may take a huge amount of time. In this case, the latency should be set to a sufficiently large value in order to reduce the performance degradation that may result from frequent disk access.

Configuring the Latency

Events are passed to the watching process via a callback and every callback call pass an array of events. When creating the event stream, a latency is specified and it defines how many seconds d (a double value) elapse between a callback invocation and another. fsw lets users specify the desired latency using the -l/--latency options:

$ fsw -l 5 .

In this case, fsw receives change events every 5 seconds.

Other Options

fsw currently supports the following options:

• -f, --format-time: Print the event time using the specified format.
• -h, --help: Show this message.
• -k, --kqueue: Use the kqueue watcher.
• -l, --latency=DOUBLE: Set the latency.
• -n, --numeric: Print numeric event mask.
• -p, --poll: Use the poll watcher.
• -r, --recursive: Recurse subdirectories.
• -t, --time-format: Print the event time using the specified format.
• -u, --utc-time: Print the event time as UTC time.
• -v, --verbose: Verbose output.

Installing fsw

fsw is a C++ project using the GNU Build System and can be installed on any supported system (OS X, v. >= 10.6 or any *BSD system supported kqueue).

Regular users should download a release tarball, uncompress it, configure it and build it:

$ ./configure
$ make

If you're a developer and have the GNU Build System installed on your machine, you can clone the git repository to get the sources:

$ git clone https://github.com/emcrisostomo/fsw.git

Then, cd into fsw directory, bootstrap the GNU Build System, configure it and build it:

$ ./autogen.sh
$ ./configure
$ make

Finally, no matter you used a release tarball or the repository sources, you can install it system-wide to /usr/local by running the following command:

$ sudo make install

If you cannot install it on /usr/local or you just prefer to install it on a private directory, you can use the following command instead:

$ DESTDIR=/installation/path make install

Documentation

fsw ships with a detailed man page you can consulting using the following command:

$ man fsw

Getting Help or Giving Feedback

If you experience some issues or simply want to give your feedback, please contact me.