Diary

• April 21, 2017
  
  • FreeBSD
    My new automation using mcblockd and dwmrdapd is working well for blocking nefarious users. There is more to come (botnet stuff), but I'm now relieved of reviewing my pf tables on a regular basis. I can now trust that ssh login attempts will be blocked immediately and effectively, with policy of my choosing. It's all hands off now, and secure. For more see my blog posts "dwmrdapd nearing production-ready: RDAP cache for IDS/IPS applications", "mcblockd automation progress" and "more on mcblockd automation progress".

• April 20, 2017
  
  • FreeBSD
    My web server is now running FreeBSD 11.0-STABLE, for the same reasons I'm using it elsewhere. I ran into a very mild hitch in the upgrade, beyond some problems with a couple of port conflicts: my ZFS pool wouldn't import due to a hostid change. Easy to fix by forcing the import, all is well.

• April 14, 2017
  
  • FreeBSD
    My gateway is now running FreeBSD 11.0-STABLE. I needed this in order to utilize some C++17 features in a newer version of clang++ than what was included in the base of 10.3-STABLE. On 10.3-STABLE I was using clang++-3.9 from ports for some of this, but it's beneficial to have the base system's compiler. On the latest 11.0-STABLE, that's clang++-4.0. I needed <shared_mutex>, and also wanted to clean up some of my use of regex_search() that is changing for C++17.

• March 26, 2017
  
  • FreeBSD
    My ZFS snapshotting is working fine. The last snapshot dates are stored in /etc/snapdates in the same format as /etc/dumpdates used by dump. Today it contains:

            zfs1/svn                       0 Mon Mar 13 17:47:22 2017
            zfs1/svn                       1 Mon Mar 20 11:00:00 2017
            zfs1/svn                       2 Sun Mar 26 04:00:00 2017
            

• March 25, 2017
  
  • FreeBSD
    I have been working on rearchitecting my intrusion detection/prevention system to provide more automation. Thus far, I've been working on building blocks. There is now a DwmPf C++ library to interact with pf (packet filter), making pf table modifications easier from application code (a higher level API than the ioctl() calls). I also now have my own RDAP C++ library (DwmRDAP) to query the registries for network information. I have revived parts of mcflow (written in 2005!) to collect some data, and will likely use pieces of it to automatically block some networks. The data on SYNs I don't acknowledge has been very enlightening.

    I wrote a simple server to listen on suspect ports like 23 and those used by Mirai and its descendants, just so I can allow a full TCP handshake before banning a network. This helps prevent SYN DoS attacks from spoofed addresses causing me to block unintended networks.

    mcblock now has the ability to grok mail logs for suspicious activity. This is wrapped up in a class just like the auth log parser, and later will use the in-development secure network API to my gateway to add addresses to be blocked.

    Speaking of which... I've put a little work into mcblockd, a secure service that will allow remote manipulation of pf tables and my database of blocked networks. This will allow me to stop doing some things by hand, like grokking my web and mail server logs and activating blocks on my gateway from the results.

• March 12, 2017
  
  • FreeBSD
    I finally finished a new ZFS backup program named dwmzfsback. It is using my preferred level scheme: a full followed by incrementals of level 1 and 2. The transfer to my backup host happens over ssh, and is incremental except when I transfer a full backup (once every 4 weeks). This is a C++ program, but execs the zfs and ssh commands as needed. Datasets to be snapshotted and transferred are specified on the command line.

• March 5, 2017
  
  • FreeBSD
    I finally finished a new program named dwmfstodot that creates a dot file containing a graph of filesystems, ZFS pools, ZFS VDEVs, partitions and disks. This is handy for keeping track of disk arrangement and utilization. This was somewhat difficult since libzfs is essentially undocumented. This is a C++ program utilizing libzfs.

• April 9, 2016
  
  • FreeBSD
    I submitted a bug and patch for demangling of wchar_t on FreeBSD. Bug 208661. I applied my patch on kiva and www.

• March 12, 2016
  
  • FreeBSD
    I submitted a bug and a patch for the C++ library on FreeBSD that had been annoying me for a long time. It had been causing the unit tests for my StringConverter class to fail. The problem is in /usr/src/contrib/libc++/include/locale, but the symptom would show up when using an ostream << operator with std::oct and std::showbase. The output would be truncated by one digit for unsigned long and unsigned long long.

    FreeBSD was the only platform on my list where this bug was present. Hopefully someone will commit my patch or an equivalent patch to fix the issue. I'm a happier camper when the unit tests for libDwm don't fail due to an OS library bug without an easy workaround. The bug I submitted is 207918.

• libDwm
  I finally started removing pthread dependencies from libDwm, since I can now use std::thread, std::mutex, et. al. on the platforms I use. This was overdue, but I had some old machines for a while which didn't have an up-to-date C++ standard library. In fact I still have such machines, but at least they're all up to C++11 or better. I'd love to be at C++14 across the board, but I'll have to wait.

  I also wrote more unit tests for libDwm.

• March 6, 2016
  
  • pigdo
    I created some state diagrams for several of the state machines I'm using in mcpigdo, and added them to the documentation. I also added doxygen comments where they were missing from header files, and created skeletal Doxyfiles to generate the documentation. The state diagrams were done with the dot tool from GraphViz.

• February 28, 2016
  
  • pigdo
    I created a RotaryEncoder class that encapsulates a RotaryEncoderReader object for FreeBSD on the Raspberry Pi 2. It's basically just a state machine with four states: Initial, MovingClockwise, MovingCounterClockwise and Stopped. It has Start() and Stop() methods that start and stop a thread that monitors a Dwm::Thread::Queue that is fed by the RotaryEncoderReader class. It is using a condition variable and timed wait, so it can go to the Stopped state when the RotaryEncoderReader stops sending movements for 500 milliseconds.

    I moved these clases and a Gpio class into a separate DwmPi library since I may need them for projects other than my garage door opener accessory at some point.

• February 27, 2016
  
  • pigdo
    I don't see GPIO interrupt support for userspace on the horizon for FreeBSD on the Raspberry Pi 2. I'm OK with this for the moment, since I intend to use a low resolution rotary encoder on my garage doors. Say 30 ppr. I can always make my own FreeBSD kernel modifications to support interrupts, if I deem it necessary at some point. The sysfs setup on Linux is terrible enough that I'd rather run a polling thread in FreeBSD than use sysfs/gpio on Linux just to get interrupt support.

    I created a new RotaryEncoderReader C++ class for FreeBSD, and it works fine. It monitors the A and B channels of the encoder in a thread and can signal another object when the encoder moves clockwise or counterclockwise. Internally it's using a state machine to suppress incorrect transitions. Since the signalling of another object is done in thread context, the receiving object needs to be thread safe. In my case, I will use an instance of my Thread::Queue class template to receive the events. As a bonus, this makes it easy to determine if the encoder has stopped since I can call TimedWaitForNotEmpty() from my Thread::Queue class template. If it returns false, I know that no events were added during the timeout I specified.

    So now I'm moving ahead with using FreeBSD on the Raspberry Pi 2 for my garage door opener accessory.

• February 25, 2016
  
  • pigdo
    I compiled emacs-nox11 on my Raspberry Pi 2 with FreeBSD 11.0-CURRENT. I saved all of the packages I've built in my ZFS pool on www so I won't have to do it again when I upgrade the microSD card for FreeBSD on the Raspberry Pi 2.

    I created a new Dwm::Thread::Queue C++ class template in the Dwm library to use to feed encoder data from one thread to another. This class template is essentially an update of my old Pthread::Queue class from the year 2000, but using standard C++ mutex and condition_variable facilities instead of pthread. I wrote unit tests for it and it appears to work fine. In the process I discovered that I had never updated my UnitAssert framework for multithreading. For now I've fixed it with one mutex with poor granularity. This should be fine for now, my unit tests do not need to be high performance. I ran my unit testsing for my new class template and UnitAssert on my web server and Raspberry Pi 2.

            dwm@rpi2:/home/dwm/src/dwm/libDwm/tests% ./TestThreadQueue
            6003/6003 passed
            

• February 24, 2016
  
  • pigdo
    For a number of reasons, I installed FreeBSD on a microSD card for my Raspberry Pi 2 Model B and have been playing with it. It's still rough around the edges for GPIO work; there's no way to get interrupts into user space. For reading a rotary encoder, that's somewhat of an issue. I can of course just read the pins at high frequency, but it's far from optimal since it gobbles CPU.

    The bigger problem at the moment is that there is no emacs package available, which means I can't really do development work directly on the Raspberry Pi with FreeBSD. I'll work on that soon.

    The good news for FreeBSD on the Raspberry Pi is that there's a reasonably friendly API for GPIO basics. And given that the RPi2 has a 4-core processor, it's not a huge problem to create a thread that reads the pins connected to the rotary encoder and pushes changes into a queue to be consumed by another thread. The pin-reading thread could use a state machine to avoid sending noise to the consumer.

    The other upside to FreeBSD is that I don't have to deal with sysfs stupidity like needing to export/unexport, wait for filesystem entries to appear and disappear, etc. While there are alternatives to sysfs with linux on the Raspberry Pi like PIGPIO, they're not part of the kernel nor included in a base distribution.

• February 21, 2016
  
  • pigdo
    I created a Gpio class for my garage door opener software, and refactored other classes to utilize it.

• February 20, 2016
  
  • pigdo
    I am close to completing the schematic and layout for a Raspberry Pi add-on board to use for a WiFi garage door opener accessory. It will handle two garage door openers. There are inputs for two open collector NPN rotary encoders and two limit switches. Two relays are used as outputs.

    

    Software-wise, today I completed the first pass at a configuration file lexer and parser. I also wrote the unit tests for them.

• February 14, 2016
  
  • pigdo
    I have been working a bit on a garage door opener accessory, using the Raspberry Pi as the platform. The idea here is to have a means of opening and closing my garage doors from my iPhone.

    So far I've spend most of my time on state machines for rotary encoders. It took me a while to find the documentation I needed for GPIO access under Raspbian "jessie" on the Raspberry Pi 2 Model B. And the interface is somewhat clumsy, but I have code working now. I have two state machines: one that handles the gray code from the encoder (using edge interrupts) and a higher-level state machine with states Stopped, Closing and Opening. I've tested it on the Rapsberry Pi 2 I have on hand, and it works.

• April 17, 2015
  
  • www
    I have drag and drop from the album view to the tree view working in my new gallery software. It needs some tweaking but it's functional. It will be handy for reorganizing photos.

• April 16, 2015
  
  • www
    I finished adding cookie support for the tree view in my new gallery software. When returning to the gallery, I restore the tree view hidden/visible state and restore the expanded/collapsed paths.

    I'm at about 7,700 lines of code, and still trying to refactor as I go to keep it reasonable. Once I have drang and drop in the tree view, I'll consider the code quite usable for my own purposes. It should easily wind up being less than 10,000 lines of code at that point. It might still be there after I add support for video.

• April 12, 2015
  
  • www
    I finally got around to adding a tree view to my photo gallery software. Its visibility may be toggled. Below is a screen snapshot with it visible and the Tools album expanded. Each entry in the tree is drop-enabled, but I haven't added the drop handling yet. The main purpose of the tree view, other than navigation, is to make it easy to copy or move photos from the main view to any location within the gallery.

    

• March 30, 2015
  
  • www
    My new photo gallery software is now running as a FastCGI application on my test machine. This mostly alleviates all of the headaches I've had with wthttpd server issues. Sadly this means that file upload progress isn't reported, but it's a worthwhile tradeoff for stability. I imported about 700 of my car photos tonight and didn't have any problems.

    Over the weekend I implemented password changing. I haven't yet added auth token invalidation/recreation when a password is changed, but it should be easy to do. Much easier than all the work I sunk into trying to make the wthttpd connector work in a stable manner.

    I also did a little bit of restyling to flatten the interface. I still need to flatten the montage images for albums.

    Here's a video taken while importing some of my car photos. This is running as a FastCGI application, hence the upload progress indicator stays at 0%. But at least the import progress widget works correctly, and that's more important to me. Import begins after the files are uploaded.

• March 19, 2015
  
  • www
    I did a little bit more work on my new gallery software's simple database. Most of it was done a while ago, but wasn't tested. I wrote a simple utility program to generate the database and search it with a regular expression. It seems to work fine.

    There are two ways to search the database. The first involves loading the whole database into memory and searching in memory. This is good for avoiding some I/O overhead when performing multiple searches. The second method searches the database file by loading each entry one at a time. Given filesystem cache and the relatively small size of the database file, there isn't a ton of overhead with the second method, and it conserves memory in user space.

    Unlike some other programs, I do not intend to store any metadata that is not contained in the JPEG files themselves. This allows me to completely wipe out the database and regenerate it from the JPEG files themselves with no external metadata needed. At the moment I haven't decided where I'll keep things in various EXIF and IPTC data, but I know I don't want to depend on external metadata. Doing things this way provides the highest level of portability for JPEG files; if I want to move my photos elsewhere, or perform a backward-incompatible upgrade of my software, it will be relatively painless.

    I have yet to put anything in the UI that involves the database.

• March 18, 2015
  
  • www
    I have authentication tokens implemented in my new gallery software. These are passed to/from browsers as cookies. They are comprised of the username, a 128-bit random string (generated from reading /dev/random) and a timestamp. On the server side, there's a simple database using my Dwm::IO namespace to store and retrieve them. Tokens expire every 2 weeks. All tokens for a user are deleted from the database if the user explicity logs out. I have not implemented password changing yet, but when I do it will require the old password and will remove all authentication tokens for the user and issue a new one.

• March 14, 2015
  
  • www
    I decided I want to run my new gallery software in a FreeBSD jail. This avoids the headaches of getting devfs working correctly at boot time, and allows me to get rid of the chroot() call. I have the jail set up on my test machine, but I haven't yet modified my Makefile to install to it nor tested running in the jail. I'm installing emacs and some other things inside the jail first. I am using ezjail to administer the jail.

• March 13, 2015
  
  • www
    I now have login and logout working in my new gallery software. Initially I tried using the Auth stuff in Wt, but I had issues with Dbo (dirty objects). I also really don't need many of its features, and the AuthWidget isn't well-documented. I rolled my own password database that's essentially the same as a FreeBSD password database, using SHA-512 with 16-byte salts that are generated via /dev/random. I'm using my own login dialog.

    Next I'd like to start adding search functionality, for which I need a simple data store. And I want an abstract album view so I can do things like show photos from a certain date regardless of how they're organized into albums. And I still need a tree view of albums.

• March 8, 2015
  
  • www
    After fighting with GraphicsMagick-1.3.21 locking problems (deadlock from using Image::write() from more than one thread), as well as memory leaks from its montage methods, I gave up trying to use it internally in my gallery software. Instead I now launch some small utility programs I wrote. The good news is that photo imports are now faster since I don't have to wrap all my calls to Image::write() with locking/unlocking of a singleton mutex to deal with GraphicsMagick problems.

    I'm now at just over 4,000 lines of code.

    Below is a video showing me importing 20 photos, creating a new album and moving the new photos into that album.

• March 4, 2015
  
  • www
    More updates to my new gallery software...

    I'm at about 3,100 lines of code. Still very reasonable, given that quite a bit of the intended functionality is in place.

    I fixed an issue with UI updates during file upload that would occasionally cause a crash. When I switched to using multiple threads for import processing, I missed a case where I was updating a variable from more than one thread. It's now fixed.

    I added a new album name dialog class, just so I could have a maintainable destructor. I tweaked some of the styling for album and photo selection.

    I'm now using WebSockets. Many operations are much faster, which is a good thing.

• March 3, 2015
  
  • www
    Several updates to my gallery software...

    I now allow albums to be moved or copied. The drag widget shows the number of albums (including subalbums) being moved or copied, as well as the total number of photos being moved or copied. Multiple albums and photos can be selected for copy or move at one time.

    I fixed a nit with dragging... in order to never have the drag widget screw up the AlbumBox or ThumbBox by appearing in the layout, its parent is now the album. I could not find any other solution.

    Since I run chrooted, I now require that the gallery binary be setuid root, and I change back to the real user ID right after calling chroot().

    I have been unable to resolve the massive memory leaks from GraphicsMagick. I will likely create some small utility programs and just call them to rotate and convert photos.

    I'm looking at adding video support so I can upload videos from my iPhone. I'll use ffmegthumbnailer to generate thumbnails.

• February 25, 2015
  
  • www
    My new gallery software now uses EXIF Orientation during import to try to get the orientation correct. When it's done, it rewrites the EXIF orientation tag. EXIF data is often botched, but this is better than many of the major websites that accept photo uploads. It works perfectly for my own photos taken with my DSLR or iPhone 6 Plus.

    Multi-photo rotations are now multithreaded: one thread per photo. Later I'll probably use a thread pool, but this works fine for me for now and it's much faster than using a single thread when running on a multi-core processor.

    I really should make photo uploads multi-threaded, at least in terms of the import process (orientation fix, creation of medium and thumb sized images).

• February 24, 2015
  
  • www
    I am now using a nicer drag widget in my gallery software when moving or copying photos to another album. Files are copied with a normal drag, and moved with a shift drag. I also fixed some photo selection issues. I disabled the double-click-on-the-image selection; it's too difficult to do reliably.

    The new code is at 2,636 lines (not counting comments and blank lines):

    % mcloc .
               309 ./DwmWWWGalleryAlbum.cc
                52 ./DwmWWWGalleryAlbum.hh
                32 ./DwmWWWGalleryAlbumBox.cc
                60 ./DwmWWWGalleryAlbumBox.hh
                58 ./DwmWWWGalleryAlbumBoxToolsDialog.cc
                26 ./DwmWWWGalleryAlbumBoxToolsDialog.hh
               135 ./DwmWWWGalleryApp.cc
                29 ./DwmWWWGalleryApp.hh
                28 ./DwmWWWGalleryConfig.hh
                28 ./DwmWWWGalleryConfigLex.lex
               108 ./DwmWWWGalleryConfigParse.yy
                11 ./DwmWWWGalleryCopyOrMoveDialog.hh
                19 ./DwmWWWGalleryDragPhoto.cc
                18 ./DwmWWWGalleryDragPhoto.hh
               156 ./DwmWWWGalleryFileUploader.cc
                60 ./DwmWWWGalleryFileUploader.hh
                21 ./DwmWWWGalleryFolderNameValidator.cc
                22 ./DwmWWWGalleryFolderNameValidator.hh
                42 ./DwmWWWGalleryImageSelector.cc
                18 ./DwmWWWGalleryImageSelector.hh
               244 ./DwmWWWGalleryMediumBox.cc
                57 ./DwmWWWGalleryMediumBox.hh
               202 ./DwmWWWGalleryPhoto.cc
                39 ./DwmWWWGalleryPhoto.hh
               232 ./DwmWWWGalleryPhotoAlbum.cc
                34 ./DwmWWWGalleryPhotoAlbum.hh
                31 ./DwmWWWGallerySubAlbumSelector.cc
                17 ./DwmWWWGallerySubAlbumSelector.hh
               201 ./DwmWWWGalleryThumbBox.cc
                79 ./DwmWWWGalleryThumbBox.hh
                50 ./DwmWWWGalleryThumbBoxToolsDialog.cc
                20 ./DwmWWWGalleryThumbBoxToolsDialog.hh
               118 ./DwmWWWGalleryUtilities.cc
                18 ./DwmWWWGalleryUtilities.hh
                62 ./dwmgallery.cc
              2636 TOTAL
            

• February 23, 2015
  
  • www
    I refactored my new gallery software to allow deletion of multiple photos at once from a single album. I also now allow a photo to be selected by clicking its group box. An image can still be selected by double-clicking on the image, but a single click on the group box is easier for the user to perform reliably.

    In the queue:

    • A nicer looking drag image for copying photos, and a dialog to choose between copying and moving.
    • Allow multiple photos to be commented at once (with the same comment).
    • Start work with WAuth.

• February 22, 2015
  
  • www
    My new gallery software...

    I added basic drag-and-drop copying of photos from an album to any visible sub-album in the thumbnail view. Double-clicking a photo enables and disables dragging of that photo. More than one photo can be dragged at once to an album.

    I refactored to allow bulk rotation of photos from the thumbnail view. I'll likely do the same for commenting and deleting photos. It will save mouse clicks and keystrokes for the user.

• February 21, 2015
  
  • www
    I'm making more progress on my gallery software. Below is a video of it in action.

• February 17, 2015
  
  • www
    I started adding internal path navigation to my gallery software. This turned out to be more work than expected, because Wt::WAnchor is a class with no descendats and only a few of the other widgets and other classes in the Wt heirarchy provide internal path functionality. In my case, I want the image inside my AlbumBox class to allow navigation to an internal path. The only way I've found to provide this functionality is to add a Wt::WArea to the image in my AlbumBox. For now this will work, but it's slow because I can't determine the size of the area until after layout has finished. Hence I'm now reading the thumbnail image file to get its geometry so I can set up the area before layout, which means additional filesystem access. Not a big issue at the moment, but once I have any albums with a lot of subalbums...

    At any rate, the reason I want the internal path navigation is to allow navigation to an album via a URL. Without this functionality, my gallery is a single-path application. When I want to tell someone to look at an album, I don't want to be forced to tell them how to navigate to it from the top level album; I want to just send them the album's URL.

    Lesson learned here: think about internal path navigation from the start so it's part of the design from the very beginning.

• February 16, 2015
  
  • www
    My new gallery software now allows the deletion of an album from its AlbumBox tools dialog.

• February 15, 2015
  
  • www
    My new gallery software now allows new albums to be added anywhere in the heirarchy. I also ditched the multiple tool buttons in each thumbnail box, and now have a single tool button that brings up a modal dialog. For album thumbnails, I added the ability to regenerate the montage image. Next up: allow the deletion of albums.

• February 14, 2015
  
  • www
    My new gallery software now allows the deletion of photos. I also fixed a problem with uploading to make sure uploaded photos go into the currently viewed album.

• February 13, 2015
  
  • www
    My new gallery software now has navigation from subalbums to parent albums. It also now has a rudimentary configuration file.

    Next up: the ability to add new albums, and to regenerate an album's montage image.

• February 12, 2015
  
  • www
    My new gallery software now has subalbum navigation. A montage image is generated and used to indicate a subalbum. The montage generation needs some tweaks, and I need to use a default image for empty subalbums. The good news is that it's very fast; MUCH faster than gallery3.

• February 10, 2015
  
  • www
    It's always good to fix minor issues as you find them, especially when they impact the user experience...

    I finally have scroll location restoration working the way I want in my new gallery software. I wound up just inserting some javascript with setJavaScriptMember() and calling it with callJavaScriptMember(). This avoids having to deal with a zillion client-server roundtrips tracking scroll position when the user scrolls, which would crush the user experience. I now save the scroll position when the user switches from the thumbnail to medium view, and restore it when they return. One round trip that was already needed and hence doesn't add any delay to the user experience.

• February 9, 2015
  
  • www
    I kept working on my own photo gallery software. I added the ability to change a photo caption (comment) from the thumbnail view.

• February 8, 2015
  
  • www
    I continued working on my own photo gallery software. I cleaned up the upload code, and added the automatic generation of thumbnails and medium-sized images. It works fine. I also added the ability to rotate images from the thumbnail or medium view.

• January 28, 2015
  
  • www
    I started working on my own photo gallery software. I'm using Wt. At the moment I'm just playing around with things so I have an idea how some of the components in wt-3.3.4-rc1 work and whether or not it's stable enough to use versus wt-3.3.3.

    By the time I'm ready to deploy my gallery, I will be using FreeBSD 10.x on my web server, and will be using WebSockets instead of FastCGI. Hence my playing around is on a local machine without apache. When deploying behind apache, I'll be using the proxy_wstunnel_module.

    When linking with shared libraries, I wind up with a huge list of dependencies and a large memory footprint. I built and installed Wt static libraries, and chased down remaining shared libraries so I could link statically. The memory consumption dropped by more than 50%. There appears to be a memory leak in Wt somewhere, I'll figure that out later.

    I have file upload working now, but I need to clean up my example so it will be useful later.

• December 21, 2012
  
  • www
    I finished simple front-end code for my UPS monitoring. There's now a chart for UPS status on the Site Health page. This is useful to me for monitoring battery charge level, expected battery runtime and of course my power consumption.

• December 13, 2012
  
  • depot
    Since I'll soon be using ZFS to store data on depot, I made changes to code used by sitehealthd and my Wt-based front-end code to track filesystem utilization for ZFS filesystems in addition to UFS filesystems. The main change was replacing getfsent() calls with getmntinfo() calls, since ZFS filesystems are not in /etc/fstab. This also allows me to run the code on OS X which doesn't use /etc/fstab.

    I'm also nearly done adding UPS monitoring to sitehealthd. I'll later add front-end code to view UPS status and history (battery charge level, expected runtime on battery, load, input AC voltage, etc.).

• December 8, 2012
  
  • www
    I've been remiss in updating this diary. I've worked on more software than I have time to write about here, but some recent updates...

    I deployed my sitehealthd and sitetrafficd on depot, to record disk usage, CPU temperatures, CPU utilization and IP traffic. depot is my new backup and media server, which is a work in progress.

    In the process of deploying sitetrafficd, I remembered that its packet processing code assumed that the pcap filter being used would only pass TCP packets. I corrected that assumption, it will now work with all IP traffic. I also discovered that GetLocalInterfaces() from libDwm did not work correctly on FreeBSD 9.1 and OS X Mountain Lion. I fixed it for both of these platforms by switching to using getifaddrs(). It is interesting to note that getifaddrs() behaves differently for root and non-root users; non-root users don't get the inet aliases for interfaces while the root user gets them. This is true on both FreeBSD and OS X.

• April 28, 2012
  
  • www
    I finally got around to fixing the Wt apps on my web server that were broken when I upgraded to FreeBSD 8.3-STABLE earlier this week. The list included the Site Health page, the Site Traffic page and the Calendar pages.

• April 23, 2012
  
  • dwmqping
    Recently I had the need for a program to collect and plot round-trip times from my desktop. I wrote such a program in 2008 for FreeBSD, called dwmqping. I decided to revive it this month.

    The old user interface was somewhat ugly (was I aiming for a Star Trek color scheme?), and at some point I'll probably change it. However, it is functional, and I had forgotten how useful it can be. It can ping one or more destinations, and it uses TCP packets for transmission and BPF for timing, so it's fairly accurate and not as dramatically affected by process scheduling, etc. as timestamping in application code. Clicking on the plot causes a snapshot to be saved in /tmp; this is useful but also a nuisance since it doesn't let you supply a filename and is easily triggered when you don't want a snapshot.

    I used Qt for the GUI, and I've no plans to change to something other than Qt.

    The buttons needs some work, but here are some screenshots of what it looks like right now.

    This first screenshot is with 2 destinations on my local network: ria and www. Here we have the live plot showing for ria. The packet rate was 1 packet per second. The blue lines and cyan dots in the plot are raw round-trip samples. The yellow line is the median of the last N points, whenre N was 1600 for this picture. If packet loss is oberved, it appears as a red area plot using the Y-axis on the right. This axis is logarithmic because TCP begins to suffer significantly before 10% packet loss.

    

    The next screenshot is the history plot for a single destination on the other side of the country at 20 packets per second. The history plot is a box or candle plot. The top of the blue line represents the 95th percentile of the samples in the box's set. I intentionally don't display the maximum sample; I don't want a single outlier to skew the Y-axis scale. The top of the cyan box represents the 75th percentile. The line through the cyan box represents the 50th percentile (median). The bottom of the cyan box represents the 25th percentile. The bottom of the blue line represents the minimum sample.

    

    The next screenshot is the distribution plot. It shows the round trip time distribution for the last N samples, where N is 1600 in this case.

    

• April 7, 2012
  
  • dwmqlaunch
    I updated dwmqlaunch a bit. It now may have a small time/date widget and remote control for xmms. The remote control for xmms is to make xmms usable; since my desktop is 3840x1200, xmms is too small to use at its default size and is too big/ugly when doubled.

    

• March 20, 2011
  
  • sitecal
    I finished a functional site calendar on the web site.

• March 13, 2011
  
  • sitemap
    The first pass at my sitemap is done. The interactive tree is a Wt application running as a WidgetSet. The application reads a file that's written by my mksitemap program once a day.

    My web site indexer is working fine and I don't think it needs any tweaks for now. A bunch of things are using its results to get information... search, the menu generator, the sitemap and the ancestry line shown just underneath the site logo.

• March 6, 2011
  
  • Wt
    I installed Wt 3.1.8 on www, and compiled the Wt examples so I can test mod_fastcgi.

• March 5, 2011
  
  • www
    I installed mod_fastcgi on www in the interest of using it for Wt applications. I installed the FastCGI Development Kit (fcgi-devkit-2.4.0), Boost libraries (boost-libs-1.45.0_1) and libharu (libharu-2.2.1) in preparation for building Wt 3.1.8, and then started the Wt 3.1.8 build before I went to bed.

• Wt
  It's been a long time since I've used Wt. However, I have several applications in mind for my web site and Wt nicely abstracts things away that I don't have time to author myself for every application (javascript, AJAX, etc.), especially for applications where I'd like stateful behavior.

  I've written a fair share of applications using Qt. Wt has a somewhat similar set of abstractions, including a signal/slot abstraction.

  The first application I have in mind is a very simple web site map.