It’s been another week of Mondays these past two days at work.
The people in charge of our Very Expensive Financial App have spent huge amounts of time running around, trying to figure out how to make the app work.
This is in addition to the fact that the only fellow who has a piece of a clue about this program is on vacation this week. This means that troubleshooting has been left in the hands of the Able Assistant (juggling chainsaws) and.. well.. nobody. Apparently because I had a well-practiced look of concern on my face (along with my `I’m not touching that software’ attitude) I wound up helping.
When I say helping, I really mean displaying the aforementioned look, shaking my head a lot, and repeatedly denying that it has anything to do with the network.
So, in short, they have given loaded guns and blindfolds to children so they can go practice in the dark (after the rollercoaster ride).
In the meantime, we moved our domain controller to a virtual machine. Whether or not that makes this a virtual domain controller I refuse to speculate.
So all of the sudden the DNS crashes into the DC, with predictable results: Very Expensive Financial App fails. It usually doesn’t require anything so high level as domain issues… everything, including my dog sneezing, will crash the App.
This was perfect timing on the network side, as the fellow who migrated the domain controller got a call from HIS controller (the wife) and suddenly required this particular half of the day off. Can you say incommunicado? I knew you could.
Meanwhile the Very Expensive App Patrol came up with a few bizarre entries in the logs they wanted looked at from the network side. Just when we thought we had reached the Maximum Amount of Fun recommended by the FDA, the Boss shows up. [sharp intake of breath]
The Boss is universally acknowledged as a nice lady. And she is. Unfortunately she has risen above the level of her own incompetence and rides herd on all of the company’s technical wizards (and me). She decided that since things weren’t going well, she was going to HELP us.
Help comes in many forms, none of them the Boss. Her troubleshooting skills are only slightly above those of the owner’s guinea pig. Perhaps even, who knows.
She decided that this had to be a RAM problem. We somehow managed not to choke, guffaw, or loudly explain why it couldn’t be a RAM problem while listening to why it HAD TO BE a RAM problem. At this point we did manage to get off a few cogent points as to why it wasn’t RAM but she was the Boss; her job is to take all the expert opinions from those under her and ignore them completely.
We were saved by the bell, which happened to be attached to the cell phone which was attached to the Boss. Her boss apparently needed her on an urgent mission: his laptop couldn’t hit certain sites on the web and this needed to be rectified immediately. (!)
Left to our own devices, we went back to troubleshooting inside and outside of the virtual environment. We were virtually troubleshooting. Fortunately we were still physically employed.
By the time it hit only one hour after time to leave, we had managed to make the Very Expensive App work. It was some portion of the following:
* switching onboard NICs
* disjoining a server from the domain then rejoining it
* joining the foreign legion
* booting seven servers in the correct order while standing on our left legs
* Able Assistant had to operate the system from underneath his desk because when he sat down, it crashed again
* learning via email that one of our number is getting (un)hitched
* having a fight with Support because they Don’t
* desperately attempting to get somebody, anybody, to sexually harass us
