Posted on

Measuring TCP round-trip times, part 4: plotting the data

I added a new plot to the Site Traffic page. This is just another Wt widget in my existing Wt application that displays the traffic chart. Since Wt does not have a box plot, I’m displaying the data as a simple point/line chart. There’s a data series for each of the minimum, 25th percentile, median, 75th percentile and 95th percentile. These are global round-trip time measurements across all hosts that accessed my web site. In a very rough sense, they represent the network distance of the clients of my web site. It’s worth noting that the minimum line typically represents my own traffic, since my workstation shares an ethernet connection with my web server.

Clicking on the chart will display the values (in a table below the chart) for the time that was clicked. I added the same function to the traffic chart while I was in the code. I also started playing with mouse drag tracking so I can later add zooming.

Posted on

sitemap deployed

I’ve deployed my web site map. This is the result of the work discussed in my post on Using Wt (C++ Web Toolkit) for a web site map. The original application mentioned in that post can now run as a WidgetSet application, and is now embedded in my php page wrappers.

I’m starting to think that it’d be nice to play with replacing my search page with a Wt application. Not because there’s anything wrong with the existing search, but because I’d like to free myself from maintaining the javascript. The javascript for my search page isn’t large nor terrible since I’m using jQuery, but replacing it with Wt would permit graceful degradation. And while javascript is handy, I’m still a non-expert in it.

Posted on

Using Wt (C++ Web Toolkit) for a web site map

It had been many years since I had looked at Wt, a C++ library for developing interactive web applications. Last weekend, I started using it again for a simple site map that’s automatically generated so I don’t have to manually update the map.

The first pass resulted in sitemap. I’m using a TreeView with Wt::WStandardItem objects.

The sitemap application initially generated the TreeView on the fly by traversing the filesystem under my document root, using my existing SiteIndexConfig class (used by my menu generators and my site indexer) for configuration. I knew from the start that this wouldn’t work for deployment, since it’d be too slow with all of the content from my old web server. But I already had classes to traverse the filesystem and pluck out web pages, find their titles (already stored in Xapian by my indexer), etc. It worked for a throwaway prototype.

I then wrote a new application to generate the data for sitemap, which I named (with no originality) mksitemap. It dumps a small binary file from a class named Dwm::WWW:DirectoryTree (derived from Dwm::DirectoryEntry), which can be directly read to populate a Dwm::WWW::SiteMap::DirectoryTree. From the instance of Dwm::WWW::SiteMap::DirectoryTree, I directly create Wt::WStandardItem objects for the Wt::TreeView.

The sitemap application itself is fairly small in terms of lines-of-code:


dwm@www2:/home/dwm/src/dwm/www/apps/sitemap% mcloc .
54 ./DwmWWWSiteMapApp.cc
22 ./DwmWWWSiteMapApp.hh
43 ./DwmWWWSiteMapDirectoryTree.cc
28 ./DwmWWWSiteMapDirectoryTree.hh
20 ./sitemap.cc
167 TOTAL

Obviously this isn’t counting the more general-purpose classes used by sitemap:


dwm@www2:/home/dwm/src/dwm/www/classes% mcloc src include
14 include/DwmWWW.hh
27 include/DwmWWWDirectoryTree.hh
41 include/DwmWWWSiteIndexConfig.hh
17 include/DwmWWWUtils.hh
97 src/DwmWWWDirectoryTree.cc
243 src/DwmWWWSiteIndexConfig.ll
130 src/DwmWWWUtils.cc
569 TOTAL

I’m not going to count the lines of code from libDwm; it’s a significant library, I’m only using a small part of it in this application, and it’s no effort to use it since I’ve been working on it for a decade. I’m using the Dwm::IO templates, the Dwm::SysLogger class, the Dwm::StringUtils templates, the Dwm::DirectoryEntry class and the Dwm::Signal class since I’m running under mod_fastcgi.

Posted on

Replacing ‘imageindex’ with my own software: part 2

Over the weekend, I finished the first pass at ‘mcphotoweb’. It generates basic photo albums. On the web server side, I’m using javascript to expand the medium-sized images from the thumbs and the full-size images from the medium-sized images. It works, and the presentation is nicer than what ‘imageindex’ produced. The javascript is hackish, but it works. For an example, see my August 30, 2007 album from the 2007 BMW Z Homecoming.

The next round will probably emit XML files with more meta-information, which will enable some of the EXIF details of photos to be shown.

Posted on

Replacing ‘imageindex’ with my own software

I’ve grown somewhat tired of ‘imageindex’. It doesn’t do what I want without modifying my ~/.imageindexrc file each time, which is annoying. I modified it to permit reading in a separate configuration, but the perl code is spaghetti and I don’t like the structure of the index file (tables… ugh!). I also want output that fits into my site’s style.

There are options other than imageindex, but I haven’t found any that work exactly as I’d like. It’s particularly important that I be able to easily bring in galleries from my old web site, and I can’t do that with Gallery3 or any of the freely available generators I’ve found while also fitting into my site’s style.

I’m almost done with a replacement written in C++, using the GraphicsMagick library. It’s smart enough to avoid unnecessary image regeneration, and automatically recurses into subdirectories to generate ‘medium’ images, thumbnails and montages. It also generates an index.php with all of the thumbnails and subdirectory montages as links. I need to decide how I want to display the medium images (via a slide file or via javascript) and how I want to link to the full-size image from the medium image.

Posted on

Automatic menu generation for my web site

It’s tedious to need to update the menu on each of my web pages when I add new children of a page, new siblings of a page, or move a page or directory. Especially since I keep my content organized via the underlying filesystem; why should I have to do more than that to get a menu that shows child directories, siblings and ancestors? I shouldn’t. One could argue that the easiest way to achieve what I want is to not have any index files, but that’d be quite ugly.

I’ve instead written a CGI program in C++ that will automatically add a ‘Genealogy’ menu to a page’s side menu. Within the ‘Genealogy’ menu are submenus: ‘Children’, ‘Siblings’ and ‘Ancestors’. I call the CGI from the RenderHeader() member of my PageStart PHP class to add to the side menu. Simple, always works. The only downside is that it’s a little slow since it accesses the Xapian backend to get page titles to use in the menu. I can live with it for now. I may later write the menus to files in batch mode and remove the CGI.

Posted on

Indexing my web site for search

I’ve been looking for site indexing code to support searching my web site.  Yes, you can certainly use Google for some of my content.  But Google won’t typically see content that’s not linked in.  Then there’s the data hidden away in databases instead of HTML.

Looking around at this time, nothing really excites me.  I want something fairly light weight, fast, and capable of indexing content from my HTML, PHP, blogs and photos.  I also want something that easily integrates into the look and feel of my site, today and 2 years from now.

Conceptually, I like xapian.  There’s also the ‘omega’ package built on xapian.  However, omega’s ‘omindex’ doesn’t work for me; I’ve got considerable textual content in PHP files and ‘omindex’ skips PHP.  It also doesn’t handle WordPress blogs, gallery3 databases, etc.  I’m also not terribly fond of the output of omega (though that’s not difficult to change).

I’ve decided I’ll write my own indexer and search using xapian as the back end.  In fact I’ve already written the code to index HTML and PHP files, and have a design for the code to index my blog posts.  I have a test search program that emits simple HTML.  All looks good so far; after indexing a handful of pages, searches yield appropriate document weights and rankings.