web site traffic monitoring: part 2

I’m done with the first pass of traffic monitoring. The traffic page can show graps of web site traffic on port 80 for the current day, current week, previous week, current month, previous month, current year and previous year. Good enough for now.

web site traffic monitoring: part 1

Last weekend I wrote a simple program to store traffic statistics (packets and bytes for both input and output) for port 80 of my new web server. Actually, any pcap filter expression can be used, I just happen to only be tracking port 80 at the moment. I save data every 5 minutes.

This weekend I spent a little time writing an application using Wt to graph the traffic data. The graph works, but it needs some cleanup. In the process, I wound up making some changes to the Wt::Chart classes because it didn’t let me use the desired colors for axis labels, titles and legend text. I’ll have to check these changes into my repository so I can diff them and send a patch to the Wt author.

I now just need some utility classes and functions to allow graphing various time periods of interest.

sub-par electronic design in GE appliances

Two days ago my 1994-vintage GE countertop microwave oven started misbehaving. It would not start unless I jiggled the door. I assumed the door switches needed cleaning.

Last night it went further awry. The blower would run when I _opened_ the door and stop when I closed the door. I’m assuming the interlock switches prevented it from running the magnetron, but nonetheless it’s disturbing for the microwave to seem to be running while the door is open.

The root cause: a small, low quality Korean-made microswitch with 120V across the contacts and 10A of current (more during start-up). The switch is rated for 15A and 250V, but… every time the microwave door is opened while the microwave is running, arcing will occur. What happened in my case: the contact that is supposed to be depressed by the plunger when the door is closed had deteriorated to the point where a piece of it fell off. That piece had positioned itself such than when I opened the door, the piece would straddle the remains of the contacts and the microwave would start. Closing the door moved it out of the way, opening the circuit.

I can’t really complain since this was an inexpensive microwave that lasted nearly 17 years. However… it goes to show what can happen when a designer thinks he’s doing the right thing for the target price range but uses a sub-par part. Why a mechanical switch instead of a relay or contactor? Because in theory it tends to have safer modes of failure (high mechanical leverage against the contact spring to prevent contact sticking, etc.). But if you use an el-cheapo part and don’t test it for the many thousands of cycles it’s expected to see over the long haul, you’re gambling.

Arcing wasn’t the only issue. The microswitch housing was deformed from heat. If you ask me, a switch of this size and design shouldn’t be carrying 10A of current on a regular basis. Once the contacts are dirty from arcing, there’s more heat being generated than the housing is designed to handle, which causes the contacts to melt the housing. This accelerates the demise of the switch. When I tried to remove the 2-pole connector from the switch, one of the contacts came right out of the housing instead of the connector coming off of the contact. And it was black from arcing and melted plastic (the switch housing is gray).

Since I’m an electrical engineer by training (but not by profession), I diagnosed the problem and ordered a replacement part. Hopefully I can make it a habit to turn the microwave off from the front panel before opening the door to prevent the switch arcing. There’s nothing I can do about the fact that it carries 10 amps of current when the microwave is on, unless I get motivated to replace it with a normally-open relay and energize the relay coil (low voltage, low current) from the switch instead of letting the switch carry the load current.

I wonder if there are any microwaves with the smarts to open and close the load relay at the zero-crossing of the AC input voltage…

sitemap deployed

I’ve deployed my web site map. This is the result of the work discussed in my post on Using Wt (C++ Web Toolkit) for a web site map. The original application mentioned in that post can now run as a WidgetSet application, and is now embedded in my php page wrappers.

I’m starting to think that it’d be nice to play with replacing my search page with a Wt application. Not because there’s anything wrong with the existing search, but because I’d like to free myself from maintaining the javascript. The javascript for my search page isn’t large nor terrible since I’m using jQuery, but replacing it with Wt would permit graceful degradation. And while javascript is handy, I’m still a non-expert in it.

Using Wt (C++ Web Toolkit) for a web site map

It had been many years since I had looked at Wt, a C++ library for developing interactive web applications. Last weekend, I started using it again for a simple site map that’s automatically generated so I don’t have to manually update the map.

The first pass resulted in sitemap. I’m using a TreeView with Wt::WStandardItem objects.

The sitemap application initially generated the TreeView on the fly by traversing the filesystem under my document root, using my existing SiteIndexConfig class (used by my menu generators and my site indexer) for configuration. I knew from the start that this wouldn’t work for deployment, since it’d be too slow with all of the content from my old web server. But I already had classes to traverse the filesystem and pluck out web pages, find their titles (already stored in Xapian by my indexer), etc. It worked for a throwaway prototype.

I then wrote a new application to generate the data for sitemap, which I named (with no originality) mksitemap. It dumps a small binary file from a class named Dwm::WWW:DirectoryTree (derived from Dwm::DirectoryEntry), which can be directly read to populate a Dwm::WWW::SiteMap::DirectoryTree. From the instance of Dwm::WWW::SiteMap::DirectoryTree, I directly create Wt::WStandardItem objects for the Wt::TreeView.

The sitemap application itself is fairly small in terms of lines-of-code:

dwm@www2:/home/dwm/src/dwm/www/apps/sitemap% mcloc .
54 ./DwmWWWSiteMapApp.cc
22 ./DwmWWWSiteMapApp.hh
43 ./DwmWWWSiteMapDirectoryTree.cc
28 ./DwmWWWSiteMapDirectoryTree.hh
20 ./sitemap.cc

Obviously this isn’t counting the more general-purpose classes used by sitemap:

dwm@www2:/home/dwm/src/dwm/www/classes% mcloc src include
14 include/DwmWWW.hh
27 include/DwmWWWDirectoryTree.hh
41 include/DwmWWWSiteIndexConfig.hh
17 include/DwmWWWUtils.hh
97 src/DwmWWWDirectoryTree.cc
243 src/DwmWWWSiteIndexConfig.ll
130 src/DwmWWWUtils.cc

I’m not going to count the lines of code from libDwm; it’s a significant library, I’m only using a small part of it in this application, and it’s no effort to use it since I’ve been working on it for a decade. I’m using the Dwm::IO templates, the Dwm::SysLogger class, the Dwm::StringUtils templates, the Dwm::DirectoryEntry class and the Dwm::Signal class since I’m running under mod_fastcgi.

web-based bug tracking for the solo developer

It’s more than a little disappointing that there isn’t a clear winner in this space for the solo developer that wants to host their own bug reporting system, project tracking, various agile enablers, etc. Some of them look nice but wind up being huge resource consumers (Jira consumes 1G of RAM right out of the gate), some are getting crusty (Trac), some have little to no SCM integration, etc.

I’ve installed bugzilla and applied a whole bunch of CSS, but I’m not thrilled with using it. It’s more than I need on some fronts but less than I need on others, the HTML is inconsistent which makes it more work to style or process with javascript, etc. But it works; we’ll see how long I keep it.

Creating new Subversion repository part 1

In order to retire my old CVS and Subversion server, I need to migrate everything to my new web server.  In the process, I intend to migrate the old code in CVS to Subversion.

I installed Subversion 1.6.15 from ports on all of my FreeBSD machines.  I then created an svn user on www, which will be my new Subversion server.

I need to think about how to organize the new repository (or repositories).  All I know at the moment is that I want libDwm to have its own repository.

Migrating my web site to new hardware

Tonight I fished the first pass at migrating my new web site from its development machine to its new home on a nearly identical machine. More information is here.

I’m really enjoying the Supermicro MBD-X7SPA-H-O motherboards in the new web server and gateway.  With 6 SATA ports and dual ethernet, and Intel D510 Atom processors that are easy to keep cool, they work well in a reasonable case.  I’m also really liking the Supermicro CSE-731i-300B cases.  Though they’re significantly larger than I need, they’re inexpensive for the features.  The only modifications I made: I cut an intake hole adjacent the hard drive cage for an 80mm Noctua NF-R8 fan, populated the front intake location with another Noctua NF-R8 fan, and replaced the stock 92mm fan with a Noctua NF-B9.  The Noctua fans are all rated for 150,000 hours, and are quiet.

Replacing ‘imageindex’ with my own software: part 2

Over the weekend, I finished the first pass at ‘mcphotoweb’. It generates basic photo albums. On the web server side, I’m using javascript to expand the medium-sized images from the thumbs and the full-size images from the medium-sized images. It works, and the presentation is nicer than what ‘imageindex’ produced. The javascript is hackish, but it works. For an example, see my August 30, 2007 album from the 2007 BMW Z Homecoming.

The next round will probably emit XML files with more meta-information, which will enable some of the EXIF details of photos to be shown.

Replacing ‘imageindex’ with my own software

I’ve grown somewhat tired of ‘imageindex’. It doesn’t do what I want without modifying my ~/.imageindexrc file each time, which is annoying. I modified it to permit reading in a separate configuration, but the perl code is spaghetti and I don’t like the structure of the index file (tables… ugh!). I also want output that fits into my site’s style.

There are options other than imageindex, but I haven’t found any that work exactly as I’d like. It’s particularly important that I be able to easily bring in galleries from my old web site, and I can’t do that with Gallery3 or any of the freely available generators I’ve found while also fitting into my site’s style.

I’m almost done with a replacement written in C++, using the GraphicsMagick library. It’s smart enough to avoid unnecessary image regeneration, and automatically recurses into subdirectories to generate ‘medium’ images, thumbnails and montages. It also generates an index.php with all of the thumbnails and subdirectory montages as links. I need to decide how I want to display the medium images (via a slide file or via javascript) and how I want to link to the full-size image from the medium image.