Posted on

Making my own office furniture: part 6

I continued making filter panels for the under-desk rack. The frames are 3/4″ wide solid oak, with a recess cut in the back to hold the screen. The screen is 22 squares per linear inch (484 squares per square inch) 316 stainless steel. It’s supported by 1/4″ square galvanized mesh, just to make it more difficult to damage the stainless steel screen when vacuuming them clean. The screen and mesh are held in the recess with a bead of hot glue which won’t be visible.

Below is a picture of one of the side panels. Obviously I’m not looking to keep fine dust from passing. I don’t want a lot of restriction. I just want to keep things like pet hair and the like out of the fans in the gear in the rack. The stainless steel screen is 70% open.

I’ve finished assembling three of the four filters. I need more stainless steel screen to complete the final filter. It’s McMaster-Carr 9230T51.

Posted on

Making my own office furniture: part 5

One of the desk bases is complete. The second one is built but needs stain and topcoat.

I’ve been working on the top for one of the desks.  As a reminder, here’s the desk design from SketchUp.

The base of the top is 3/4″ oak plywood. Above that is a piece of 1/2″ BCX plywood underneath the porcelain insert. The border of the top is 1″ thick solid oak, with four 5/16″ diameter oak dowels at each joint. The border sits atop the oak plywood. Hence the total top thickness is roughly 1.75″. Solid oak trim pieces cover the edges of the oak plywood.

Below is a picture from when I was testing the fitment of the porcelain insert.  There’s a 3/16″ gap between the porcelain and the frame.  This is to allow for flexible grout or caulk, since wood isn’t dimensionally stable compared to porcelain.

Yes, that’s a big piece of porcelain. Nominally 30″ x 60″. It’s 6.5mm thick. It has a PEI rating of 5 (the highest rating), so it’s unlikely that I’ll ever scratch it. If it’s good enough for commercial flooring, it’s good enough for my desktops. The design is a marble look, and I continue to be amazed at how far along this stuff has come in 10 years. It’s pretty, and zero maintenance (unlike real stone). Plus it’s manmade; it doesn’t involve carving up the planet. And I can move it by myself.

The downside… my desk base doesn’t meet the deflection requirements for tile or porcelain. That means that until the desk is in place, I will likely not install the insert. And I’m not going to use thinset to set it, since it’d likely just crack. I considered Ditra, but I don’t really trust it to hold the porcelain if I ever move the top later. While I love Ditra, I’ve never tried it in an inverted situation which might occur if two people are moving the top. Plus thinset would make the whole thing even heavier. Note that the porcelain alone is 38 pounds. I’m guessing that the whole top will be more than 80 pounds. My intent at the moment is to use Loctite PL Premium MAX to adhere the porcelain to the plywood. I may change my mind, but I think it’s my best option.

At any rate, I’ve installed the frame on the plywood base with TiteBond III and many cabinet screws from underneath. I also pin nailed and glued the trim pieces (not seen in picture above), which are clamped and curing.

Posted on

mcrover now monitors Plex server

mcrover is now monitoring my Plex server.
This was more work than expected. A big part of the issue here is that the REST API uses XML. I’ve always disliked using XML. It’s a nice technology, but when it comes to open source libraries for C++, it’s always been lacking.

Long ago, I used Xerces. Not because it’s the best, but because it was the only liberally licensed library with support for DTD and Schema validation. That is still the case today. Unfortunately, it’s very cumbersome to use and is written in old C++ (as in C++ 1998). There’s a lot of boilerplate, a considerable amount of global state (very bad for multithreaded applications), and a lot of the memory management is left to the application. I can’t imagine anyone today is using it in production.

But I plunged ahead anyway. Sadly, it was a mistake. Somewhere it was stomping on the stack, often in ways that caused problems deep inside openssl (which I don’t use directly, instead using boost::beast and boost::certify). The stack corruption caused problems trying to debug, and I didn’t have the time to figure it out. And of course I’m always suspicious of openssl, given the fact that it’s written in C and many of us lived through Heartbleed and many other critical openssl vulnerabilities. To be honest, we’ve been in desperate need of a really good, modern (and in at least C++11) C++ implementation of TLS for more than a decade. Of course I could rant about our whole TLS mess for hours, but I’ll spare you.

Time being of the essence, I switched to pugixml. I don’t need DTD/Schema validation. Problem gone, a lot less code, and a much more modern API (much harder to shoot yourself in the foot).

Inside mcrover, I’m using XPath on the XML returned by Plex. The internal code is generic; it would not be much work to support other Web applications with XML interfaces. The XPath I look for in the XML is a configuration item, and really the only reason I have something specific for Plex is their use of an API token. But the configuration is generic enough that supporting other XML Web applications shouldn’t be difficult.

At any rate, what I have now works. So now I don’t get blackholed fixing a Plex issue when I haven’t used it for months and something has gone wrong. I know ahead of time.

Posted on

UPS fiasco and mcrover to the rescue

I installed a new Eaton 5PX1500RT in my basement rack this week. I’d call it “planned, sort of…”. My last Powerware 5115 1U UPS went into an odd state which precipitated the new purchase. However, it was on my todo list to make this change.

I already own an Eaton 5PX1500RT, which I bought in 2019. I’ve been very happy with it. It’s in the basement rack, servicing a server, my gateway, ethernet switches and broadband modem. As is my desire, it is under 35% load.

The Powerware 5115 was servicing my storage server, and also under 35% load. This server has dual redundant 900W power supplies.

Installation of the new UPS… no big deal. install the ears, install the rack rails, rack the UPS.

Shut down the devices plugged into the old UPS, plug them in to the new UPS. Boot up, check each device.

Install the USB cable from the UPS to the computer that will monitor the state of the UPS. Install Network UPS Tools (nut) on that computer. Configure it, start it, check it.

This week, at this step things got… interesting.

I was monitoring the old Powerware 5115 from ‘ria’. ‘ria’ is a 1U SuperMicro server with a single Xeon E3-1270 V2. It has four 1G ethernet ports and a Mellanox 10G SFP+ card. Two USB ports. And a serial port which has been connected to the Powerware 5115 for… I don’t know, 8 years?

I can monitor the Eaton 5PX1500RT via a serial connection. However, USB is more modern, right? And the cables are less unwieldy (more wieldy). So I used the USB cable.

Trouble started here. The usbhid-ups driver did not reliably connect to the UPS. When it did, it took a long time (in excess of 5 seconds, an eternity in computing time). ‘ria’ is running FreeBSD 12.3-STABLE on bare metal.

I initially decided that I’d deal with it this weekend. Either go back to using a serial connection or try using a host other than ‘ria’. However…

I soon noticed long periods where mcrover was displaying alerts for many services on many hosts. Including alerts for local services, whose test traffic does not traverse the machine I touched (‘ria’). And big delays when using my web browser. Hmm…

Poking around, I seemed to only be able to reliably reproduce a network problem by pinging certain hosts with ICMPv4 from ria and observing periods where the round trip time would go from .05 milliseconds to 15 or 20 seconds. No packets lost, just periods with huge delays. These were all hosts on the same 10G ethernet network. ICMPv6 to the same hosts: no issues. Hmm…

I was eventually able to correlate (in my head) what I was seeing in the many mcrover alerts. On the surface, many didn’t involve ‘ria’. But under the hood they DO involve ‘ria’ simply because ‘ria’ is my primary name server. So, for example, tests that probe via both IPv6 and IPv4 might get the AAAA record but not the A record for the destination, or vice versa, or neither, or both. ‘ria’ is also the default route for these hosts. I honed in on the 10G ethernet interface on ‘ria’.

What did IPV4 versus IPv6 have to do with the problem? I don’t know without digging through kernel source. What was happening: essentially a network ‘pause’. Packets destined for ‘ria’ were not dropped, but queued for later delivery. As many as 20 seconds later! The solution? Unplug the USB cable for the UPS and kill usbhid-ups. In the FreeBSD kernel, is USB hoarding a lock shared with part of the network stack?

usbhid-ups works from another Supermicro server running the same version of FreeBSD. Different hardware (dual Xeon L5640). Same model of UPS with the same firmware.

This leads me to believe this isn’t really a lock issue. It’s more likely an interrupt routing issue. And I do remember that I had to add hw.acpi.sci.polarity="low" to /boot/loader.conf on ‘ria’ a while ago to avoid acpi0 interrupt storms (commented out recently with no observed consequence). What I don’t remember: what were all the issues I found that prompted me to add that line way back when?

Anyway… today’s lesson. Assume the last thing you changed has high probability of cause, even if there seems to be no sensible correlation. My experience this week: “Unplug the USB connection to the UPS and the 10G ethernet starts working again. Wait, what?!”.

And today’s thanks goes to mcrover. I might not have figured this out for considerably longer if I did not have alert information in my view. Being a comes-and-goes problem that only seemed to be reproducible between particular hosts using particular protocols might have made this a much more painful problem to troubleshoot without reliable status information on a dedicated display. Yes, it took some thinking and observing, and then some manual investigation and backtracking. But the whole time, I had a status display showing me what was observable. Nice!

Posted on

mcrover updates

I spent some weekend time updating mcrover this month. I had two motivations:

  1. I wanted deeper tests for two web applications: WordPress and piwigo. Plex will be on the list soon, I’m just not thrilled that their API is XML-based (Xerces is a significant dependency, and other XML libraries are not as robust or complete).
  2. I wanted a test for servers using libDwmCredence. Today that’s mcroverd and mcweatherd. dwmrdapd and mcblockd will soon be transitioned from libDwmAuth to libDwmCredence.

I created a web application alert, which allows me to create code to test web applications fairly easily and generically. Obviously, the test for a given application is application-specific. Applications with REST interfaces that emit JSON are pretty easy to add.

The alerts and code to check servers using libDwmCredence allow me to attempt connection and authentication to any service using Dwm::Credence::Peer. These are the steps that establish a connection; they are the prologue to application messaging. This allows me to check the status of a service using Dwm::Credence::Peer in lieu of a deeper (application-specific) check, so I can monitor new services very early in the deployment cycle.

Posted on

mcweather first pass completed

I’ve completed my first pass at ‘mcweather’, a caching weather server and command line client.

The main thrust here was to decouple weather condition and forecast fetching from qmcrover (a GUI client that is part of mcrover). I wanted a single server at home to fetch the weather information and provide it to local clients (command line and qmcrover). This avoids unnecessary traffic to the NWS (National Weather Service) web sites from local clients. This makes it faster as a consequence, since the local clients get the information from the local server.

It is secured using libDwmCredence. I changed qmcrover to use it, and in the process I updated two of my Raspberry Pis from buster to bullseye. One hurdle I encountered here: I was previously using a hand-built version of Qt 5.12. I didn’t want to continue doing so with bullseye, because Qt is very big and takes a long time to build. But the Raspberry Pi repositories don’t have two of the Qt libraries I need: libqt5webengine5 and libqt5webenginewidgets5. I hence added the debian repository to my list of apt repositories by adding it to /etc/apt/source.list:

deb http://deb.debian.org/debian bullseye main contrib non-free

I also created /etc/apt/preferences:

Package: *
Pin: origin raspbian.raspberrypi.org
Pin-Priority: 800

Package: *
Pin: origin deb.debian.org
Pin-Priority: 600

So far, so good. I’m running the server (mcweatherd) on the Raspberry Pi 4 that runs qmcrover in full screen mode in my office.

dwm@pi4e:/home/dwm% psg mcweatherd

USER   PID %CPU %MEM   VSZ  RSS TTY STAT START TIME COMMAND
root  8877  0.0  0.2 32832 8680 ?   Sl   01:10 0:02 /usr/local/sbin/mcweatherd

Posted on

libDwmCredence integrated into mcrover

I’ve completed my first pass at libDwmCredence (a replacement for libDwmAuth). As previously mentioned, I did this work in order to drop my use of Crypto++ in some of my infrastructure. Namely mcrover, my personal host and network monitoring software.

I’ve integrated it into mcrover, and deployed the new mcrover on 6 of my internal hosts. So far, so good.

Posted on

Replacing libDwmAuth due to Crypto++ problems

This has been a while in the making, but…

I am in the process of replacing libDwmAuth with a new library I’m calling libDwmCredence. Why?

Mainly due to poor maintenance of Crypto++, which I used in libDwmAuth. Quite some time ago I tried to explain two problems with Crypto++ to the maintainers that led to memory leaks and corruption on some platforms. I gave up due to the current maintainers not resolving the problem, and growing tired of maintaining my own fork in a private repository. The root of the issue was an incorrect optimization and attempt at forced alignment. In essence, an attempt at optimization that critically destroyed the integrity of the library on some platforms (notably an important IoT platform I use, Raspbian). The last place you want code to stomp on the stack or heap is in your crypto library!

Way back when, I used Crypto++ because it was FIPS validated. That hasn’t been true for a while, and it doesn’t appear that it will be true in the future. Then when elliptic curve solutions came along, Crypto++ was very slow to adopt due to design constraints. This was troublesome for me since some IoT platforms don’t have hardware acceleration for AES, and hence I wanted to be able to use XChaCha20poly1305 and the like to improve performance, as well as Ed25519 and X25519 keys. There was a period where I couldn’t do this at all with Crypto++.

Finally, for my own work, I don’t really need much of what Crypto++ provides. And parts of the interface use C++ in atypical ways, making code hard to read.

So, I’ve switched to libsodium and wrapped it with C++. While there are many things I don’t like about libsodium, it provides what I need and I am able to make it safer by wrapping it. Furthermore, I am using boost::asio and a semi-elegant wrapper that allows me to send and receive just about anything on the wire easily. It does what I need it to do, and no more. Of course the boost::asio interfaces are considerably different than socket()/bind()/select()/accept()/connect()/etc. but I mostly only have to think about that on the server side.

Posted on

2021 end of year ramblings

Another year of COVID-19 pandemic. Tragic, yes. Much of it preventable, yes (get vaccinated, please). But as the year closes, now is as good a time as any to reflect…

I was recently reminded of the “Why am I here?” question. It’s not as if it isn’t always staring us in the face. It is in fact central to being an adult human being. Mortality is real. But some have seen a lot of mortality this year.

Before I go to my main topic, I just want to say that my heart goes out to those who’ve endured day after day of preventable tragedies in 2021. Health care workers in particular. To those who’ve been pushed to questioning their hippocratic oath and other ethics: thank you, and stand strong. You are not alone; far from it. I’ll give you my hand if you’ll take it.

“Why am I here?”

For me, the simple answer: to build things. The complete answer is much more complex, but…

One day the remnants of humanity on this planet will just be a layer of remains. It’ll likely be a thin layer; we haven’t been here very long and I suspect our presence here will be shorter than the dinosaurs. However, we’ll leave behind an interesting layer.

Some of it utterly confounding in the grand scheme of things. Ungodly amounts of empty plastic containers. Megatons of polymer cases with silicon, glass, rare earth metals, etc. inside. Paper and more paper. An odd carbon transfer.

Some of it fascinating. Music in some form. Mathematics. Literature. Law. Medicine. Computing. Science in general. AI.

Some of it sad. Greed. Avarice. War. Poverty. Starvation. Exploitation. We’ve a long list of evils.

What do you want to leave behind? There’s a saying among software craftspeople, “Leave the code better than you found it.”. It’s attributed to Bob Martin. I’ve read a lot of Bob’s work, and listened to most of his publicly available talks. I’d be stunned if he claimed to have invented this idea. This idea predates software by… I don’t know, several millennia? It’s as old as the craftsperson, and is not specific to software. But it’s definitely a really good thing to do in software. Just as it is in plumbing, carpentry, medicine, law, etc.

I want to leave behind good, useful things that wouldn’t have existed without me. It’s weird. It feels humble and prideful at the same time.

Posted on

Making my own office furniture: Part 5

The under-desk rack cabinet is done except for the porcelain top insert. I haven’t yet decided which marble-look porcelain I want to use here. For the foreseeable future, this top won’t even be visible, which argues for low cost. On the other side is my penchant for “do things once, do them right” and “cry once now, or cry a lot later”. Thus far I’ve kept to “if you can make a decision late, do so”.

I’ve assembled the base of the first desk. My friend Randy Forbes noted that the feet of the legs are overkill. He then said, “I miss you.”. Having friends that appreciate who you are (and knowing that you’re gonna be that person even if it’s quirky/annoying/spooky) is SUCH a wonderful thing!

Now that I have the base of the first desk assembled and what I know is a solid desk design (with overkill thrown in), I thought I’d reflect a moment. This is after all a design/engineering/craftsmanship/whatever web site.

I’m not a fine woodworker. It’s not that I don’t have the patience (I do); I don’t have the time. Maybe someday.

My woodworking creations lean heavily toward utilitarian. I tend to choose function over form. My desk design reflects it. It’s sort of mission/craftsman style, but I have a LOT of fasteners that are NOT in those styles and none that typically are. I’ve got dowels, pocket hole screws, threaded inserts, long bolts going into dowel nuts, guide pins and leaded bronze guide bushings. No mortises or tenons, no biscuits, no dominos. The only time I’ve used my chisels thus far is to scrape off half-dried glue. Nevermind the 1″ thick Delrin feet or the single piece of porcelain for most of the top. I’m gonna call it “my style”. Which is again, utilitarian. It’s VERY strong; the base will likely easily hold 1,000 pounds. It has gaps for cabling. It’s designed to be able to be disassembled into 4 parts, since it’ll be heavy in total. The thick Derlin feet will allow me to slide it easily on the porcelain floor, and they can be replaced since they’re bolted to threaded inserts in the legs. Maybe someone will be using these desks long after I’m gone.

In terms of artistry, it’s not that far from a some-assembly-required MDF desk. Just with much nicer materials and beefier fastening. On the other hand, it has some qualities of a commissioned work: it’s designed specifically for the space it will inhabit and the people who will be using it. It’s actually designed by its primary user. In software and other technology fields, we call this dogfooding. I’ll be using what I’ve created.