Mother Nature is Spiteful!

Now, for those of you who don't know, I have a great deal of respect for Mother Nature. I am an avid skier, and have skiied the BC mountain backcountry more than once, where you must give her respect or you're likely to have a front row seat at your own funeral. She and I have always got along fine, but over the last few weeks, and specifically 18 hours, I wonder if I did something to tick her off. 😉

As I start writing this, it's 5:30AM. I'm up, showered, dressed, and have a cup of coffee in front of me. This after a restful night of sleep which started at 10:00PM. Again, for those of you who know me, you'll be aware that this is an absolute absurdity. My prime forum and programming expermintation time is between 8:30 and 11:30 each night, after my daughter goes to bed. I let virtually nothing get in the way of that time.... except maybe my wife... she wins over the ribbon 😉

I get up at 7:00... ish... and am out the door around 7:30... ish... to head off to work. I'm NOT an early riser, nor am I a great morning person. (Please don't give me tips on how to be one, as I have no interest in learning.) In fact, I keep the same schedule even when I'm getting up to go skiing!

So what gives? It's Mother Nature, I'm telling you... Let me start by telling you at little about last week, and yesterday...

Over the last week it's been a particular busy time at work, and on my home development front. At work we have just begun preparing for our membership renewal cycle, in which we bill all of our exisitng membership for their next year's dues. The thing is that last year was the first year we used the system we're using, so a lot of stuff was done on the fly, and the superstar that managed it is now on maternity leave. Naturally, this is a pretty large concern to us, as it incoporates a very big piece of our annual cash flow.

Being the systems guy/accountant, I am the person who knows the system second best. So the first three days of last week were spent flowcharting, meeting about, explaining and adjusting the system. And following that I had to dig into a database to pull out the existing information to send to people for review... wait... did I mention that the link to actually get data from this database was only made possible a week before? Right... so I used my considerable experience with the database (a whole week's worth) to leverage my considerable SQL skills... wait... I only actually have rudimentary SQL skills... to create a report to show these folks what they needed.

Now, I'm not complaining here at all. It's good to be busy, but there are limits. During this whole time, we were (actually still are) processing our month end reports. Every month end, we have five business days to create financial statements for our ownership. We've actually become quite good at this, and manage to pull it off most of the time. (Quarter and month ends actually have a shorter time frame, but fortunately we're not at one of those.)

Oh... remember our snow storm? Well, it shut the resort down for about a week. You'd think that would be easy to do the accounting for, but let me tell you... Our night auditors couldn't get in, so they couldn't roll the hotel over to the next day for a three day period. Once that was finally done, they just did the hotel (by my direction), and left our five point of sale partitions. So now one of my staff is there trying to post them and balance for what transactions needed to be balance. And, of course, she was away during snow time as well, which is forcing her to catch up on her own work, so now she's WAY behind...

So due to the time lost to the billing process and the snow storm, we put ownership on notice that we'd be late this month. Fortunately they were okay with it, and we started plugging along, but not really getting into the full month end swing of things until last Thursday. So if you ran that out, you'd expect statements to be done by the following Wednesday evening (tomorrow). Let's not get carried away here now... Vancouver didn't suffer as much as we did under the snow, so that is pushing it a bit. 😉

At any rate, here we are pushing ahead on month end. Oh yeah! And I'm doing a server upgrade this week too! This week we are virtualizing two Novell servers from our Novell cluster, as well as at least one Citrix server from our server farm. So I'm planning that with my third party techinician as well, to make sure it all goes smoothly. It looks like we're going to be installing Ubuntu as the host OS on this server, and using VMWare server to run the Novell and Citrix server on one box. The idea here is that if it works well, we'll move to a full VMWare ESX machine (dropping the Unbuntu interface), and have everything needed to run our systems on each server box. But regardless, the plannign is just one more added thing. 😉 I should say that normally I won't touch our systems during month end, but in this case he would be working on a server that was available for, but not in production, so we figured that it would probably be safe.

We all left work on Friday, feeling pretty good about things. The accounting side was coming along very nicely, with only about a day's work left. The systems side was planned, and we were ready to go. There was only one thing we forgot to think about... Mother Nature doesn't like Mondays. 😉

So along comes Monday morning. For my own part, I get up (at 7:00 ish, NOT 5:00) and carpool in to work with my boss. He tells me the weather forecast for the day: Rain warning of up to 50mm and a wind warning of 70-90 kmph winds. One of the ferry terminals is already shut down due to the wind, and they haven't even come close to peaking yet. Great. Just flipping wonderful. We want to finish our month end, and we already know that the power goes out every time the wind blows where I work. Oh... and better yet... remember the "power bar" thing that I bought for my servers? It's at my contractor's place, but not installed yet. 🙁

So we get to work and start scrambling. Push those pencils! Hammer on those keys! Flood those printers! Let's get this month end finished before the power goes out! Then in walks my systems guy... smacking his head as he forgot the power bar in his office. 🙁

If this were a novel, (well it kind of is now,) it would be the most pathetic piece of foreshadowing every written. I'm sure you see that. But regardless, when we're in the middle of life, somehow we can't read that...

At any rate, we conferred and he assured me that there were no real "if the power goes out here we're dead" parts to this setup, so I let him fly at it.

So there we all are, all pushing to get our jobs done. 10:00 comes and goes... we still have power. The eaves are overflowing and it looks like I'm working behind a waterfall, but we still have power. 11:00 comes and goes... we still have power. There is a waterfall cascading down the cliff behind our office, but we still have power. 12:00 comes and goes... we still have power! There are pieces of trees flying past the windows, but we still have power! We decided to break for lunch. After lunch and more hard work, the clock rolls past 2:00 and... we still have power? Something is wrong, but we're almost ready to print statements. Don't look a gift horse in the mouth! Work harder! Focus turns back to computer. 3:00... I press the print button on the statements, and they start churning off the printer... my boss heads out for a meeting... the lights start to flicker every now and then. But we still have power!!!

3:30 on Monday afternoon, I have 70 pages of financial statements in my hand, still warm from the printer... but I can't see them...

It was actually quite a scene. We have three phase power in our building, and one of the phases went out. Then it came back on... a different one went out... and came back on... and then another... it was like we were standing in the middle of a rolling blackout. Heaving a great big sigh of relief that I have printed statements in my hand, I decided to check on the IT guy.

He's scrambling to shut down servers left and right. The power is going up and down like a yo-yo, so he's working under strobe-light like conditions. Why is he in a panick? He's trying to get VMWare server to install and having issues! Of course it had to happen that way, and we'd even joked about it earlier. When will the power go out? When we're half way through statments and the server migration, right?

Then we lose power completely. The emergency lights are on, and the glow of all the PC monitors being fed by their UPS's. Those get shut down, and now the only light there is shines through the windows at 3:45 on a very dark and rainy December afternoon. Needless to say, there isn't much... and there is NONE in the server room, since it's in the middle of the building. The light of the IT guy's laptop screen glimmers faintly from there.

We quickly shut down all the servers and SAN array, and leave him with only a monitor, the server he's working on, and his laptop running off it's own battery. The intention, of course, is to let it complete the installation complete. Knowing the stress he's under, I backed away to let him concentrate as much as possible until I hear him yell for help. "You got a flashlight? I can't see the keyboard!" He was using a text based HTML browser for soemthing, which had a black background. 🙂

Just then my boss comes back from his meeting. It's now about 4:15, so we sit down to review statements by flashlight... and wouldn't you know that we need to make changes? So we bailed for home at about 4:30, as power was not forecasted to come back on until at least 11:00PM. And despite the perfect opportunity to install that power bar... it's in the IT guy's shop!

An exciting day, to be sure. 🙂 But as you're aware if you read my blog before, until I get that power bar installed, I have to go in to restart servers. 🙁

I had the power bar delivered to my home, so it's ready to go in with me when the power goes back on. I called the hotel rooms supervisor, and discussed the occupancy. With only 1 guest in house tonight, we decide that it's prudent for my to skip going in to restore the servers in the middle of the night, and I'll do it early in the morning. If I get in there by 6:00 and get things going, night audit should still have enough time to complete their function before the guest checks out.

So I apologize for not posting a new Ribbon article last night. I'm part way through one, but there just wasn't enough time to finish my beloved article before I went to bed. Sadly, work had to come first... So the alarm got set for 5:00AM (shudder) and off to sleep I went.

At shortly after 5:00, I stumled into the shower, which is where I tend to wake up. In there, though, I started thinking... "that's weird"... My cell phone never rang in the night. I can't remember if I mentioned this in my last blog post on the subject, but my UPS sends me a text message whenever an "event" occurs. (A full power off and back on cycle sends about 11 in all, and two cycles fills the memory on my phone!) I did not recall hearing that phone. Not unusual for me, maybe, as I'm a pretty heavy sleeper, but I'm sure my wife would have kicked me to shut it off, as she isn't, necessarily. No kicks that I recall either... hmm...

I get out, I dry off, I check my phone. Nope. No messages. That can't mean... surely the power... am I up at 5:00 for NOTHING????? grrr....

I phone the hotel. Sure enough, they still do not have power.

So there we are... Mother Nature, I'm not sure what I did to irk you, but I'm puttin in the power bar today. That's the last time you're going to have this much fun with me!

5 thoughts on “Mother Nature is Spiteful!

  1. 10:30 AM... power restored, powerbar installed and resort services restored. Powerbar configuration to start shortly.

    Known casualties: One UPS fried during the "rolling blackout"

    Another storm expected tonight, then another the day after. So much fun!

    Back to get those statements out before we lose power again...

  2. Wow Ken!

    Glad to see you are surviving!

    I guess I should cancel my tickets and wait for the gentle rains you are most famous for during the winter!

    All the best!

  3. Ken, normally I would tease you about this but in view of the circumstances I will refrain (for now)

    Hope all works out ASAP 🙂

  4. It's okay, Andrew. I still managed to maintain my sense of humour about it all. 🙂

    The next big storm is raging outside right now, with another on the way for Thursday, I think. Tonight's is supposed to be worse than yesterday's, but at least I've got my power bar installed now.

    And better yet... our long range forecast is calling for snow next week! LOL!

  5. Yay! The power bar works! It got it's first test this morning, as the power took another dive at about 12:40AM. It's back now, though, and the servers came up properly. A thing of beauty, truly! 😀

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.