Shit you can't make up... March 23, 2010
In early 2000, I worked for a small consulting firm (who will refer to as bleahterna blot blet – SEE WHAT I DID THERE?) where things were… on a shoestring budget. Like a lot of small companies rolling the dice and trying to find fertile ground to expand, decisions were often made without due consideration or sensible forward planning.
Unfortunately, as time went on, these events occurred rather frequently. One of these events gave me one of my most favorite parables about how, when you really start thinking radically, you can come up with a solution using the materials at hand.
This company was a live-work space, where affordable internet hosting was one of the services offered to our clients. Unfortunately, this was not long after Northpoint DSL bit it hard, and took thousands of businesses suddenly offline – us being one of them. So, to ensure that we weren’t at the mercy of a single ISP (remember, this was towards the beginning of the decade, and the vacuum caused by the tech bubble bursting sucked many other ISP’s underneath the waves), we ended up with 2 DSL lines, a T1, and a line-of-site broadband connection with a dish on the top of the building pointed atop the Sears Tower. While we never quite got the hang of configuring all of this for redundancy, everything worked pretty well…
… until Fall rolled around.
After the first thunderstorm, we lost both DSL lines. There had been some lightning strikes not far from us, so it wasn’t unreasonable to assume that some sort of surge had fried equipment, or lightning had damaged the integrity of the pair between us and the CO. Since we had enough spare DSL routers lying around, we ended up swapping them in and sometimes we’d get a sync, and sometimes we wouldn’t. After one final thunderstorm that turned into our first snow, we lost sync entirely, and just didn’t get it back at all.
Anyone who has ever spent any time with Ameritech (before they became SBC) knows that their best customer service efforts rivaled Comcast on their worst days, and after lots of calls back and forth with them, the ISP, and my boss, I was basically pulled off everything to chase down someone to come out and fix it (this literally took 3 and a half weeks – so much for “business class”). With both the ISP and the telco fingerpointing wildly at each-other, I scheduled an on-site inspection. Ameritech made it out first, spent time tromping around, and pronounced that everything was fine with their copper pair and the connection into the building. Next, we got the ISP to send an on-tech out, where after 5 minutes, he asked me to follow him to the outside of the building to show me the problem.
My then-boss, showing a stunning lack of foresight, had, during the course of renovating the building, the telco box and the downspout for the gutter perfectly lined up. During light rainfall, it wasn’t bad – the water dripped behind the box and everything was fine. During heavy rainfall, there was a short and sync was lost. And what did I see, when the tech opened the box (the one that Ameritech claimed to have inspected)?
A block of frozen ice – totally solid on the inside. Of course, this being a problem with the layout of the building, the ISP tech chuckled, wished me well, and headed off. So, what to do? Gotta melt the ice…
And that’s how I restored two DSL lines using only the following: Two extension cords, a clamp, and my wife’s hairdryer.
I couldn’t even make that one up – too weird to be fake.
As an aside: My then-boss refused to believe that this could be the result of bad planning, and then remained mysteriously quiet when the hairdryer trick restored connectivity.
