Polling a Log Directory
In .Net the defining feature of the libraries is that there is at least 2 different ways you can perform any one task. Each one usually has it’s own subtle uniqueness to it but in the end they all get you from Point A to Point B. The problem is figuring out which approach works best and combining all the different framework elements into a coherent application. In my line of work I write a lot of middlewear. Applications that interfaces with systems made before XML was a blip on the horizon and basically act as glue. One of the most common ways I interact with these applications is by reading their log files and looking for key phrases and elements. I have written more log-watchers than I can count so I’ve finally made the end-all piece of code to watch a log file.
In writing this piece of code you might instantly light up and remember that .Net supports event-driven file watching. There is a very nice IO.FileSystemWatcher class out there that does in fact provide most of the functionality required to implement this design. I used this class for about a week myself. What I came to find is that it is simply riddled with bugs, especially when watching files over the network. The object will simply loose touch with the network folder you are watching. This resulted in a mysterious mandelbug that appears only when mission-critical files need processing. The event-firer will grow eerily quite and nothing will ever be watched again. This is a problem. Upon ripping the FileSystemWatcher from the code and replacing it with a simple thread-based polling piece of code the problem was solved. The application that runs this code is now one of the most stable in the building, with an uptime approaching 2 years and counting.
So without further discussion here is my code for a simple watcher class that will watch either a directory with rolling log files or a single file.
WatcherPhrase.vb
FileWatcherEventArgs.vb
FileWatcher.vb
And how does it all fit together? Here is a very simple example of the classes' usage.
FileWatcherTest.vb
And there you go, one fully functional log watcher with each and every bug worked out. The following code is in use every day with hundreds of log files over many different testing expressions!
