Me: “Jack, i’m interested in learning about your best practices in patch management, so we can figure out how to stay current with patches and at the same time not spend a lot of time on patch management.”

Jack: “Here’s a link to IBM support site [link omitted]“

Me: “Thanks for the link, Jack. I was more interested in best practices, recommendations, et cetera.”

Jack: “We tell customers to install fix packs”

 Me: “Thanks. I looked at the link you provided for the RSS feed, and i get an error message when i try to subscribe. can you help?”

Jack: “I have nothing to do with that. Try web support”

Jack: “Try this help file for RSS readers [link omitted]“

Me: “Thanks, Jack. I’ve been using RSS for years now. I’d like to investigate a better way of staying up-to-date with patch announcements and so on. What would you suggest?”

 Jack: “Contact your Account Manager.”

Me: “Do you know who that is?”

Jack: “Contact your Sales Rep.”

At this point, i’ve given up. Are we playing 20 questions here, or what?! I know he’s a tech guy. I know i’m not asking him about SQL or CLI commands. I also know that he’s pretty high-up on the tech guy totem pole, and i know that when he’s involved, something’s usually broken, which means that customer satisfaction is an issue. Wouldn’t you think that from a CSat perspective he’d have a better idea of how to handle this situation? Something that offers perhaps even an illusion that he cares about our production environment and is invested in reduced down-time for his company’s software platform?

Here’s how this entire exchange could have been shortened to 2 emails and turned to an actionable, positive next step:

Me: “Jack, i’m interested in learning about your best practices in patch management, so we can figure out how to stay current with patches and at the same time not spend a lot of time on patch management.”

Jack: “We don’t have anything specific about that. You can try the IBM Tech support RSS feed, and i’ll put you in touch with your Account Manager so you can provide your feeback to her and see if we can come up with a good way to get your systems patched and current.”

I hope i remember this for next time i’m in Jack’s shoes…

It seems as though all my recent blog posts are preoccupied with email. Perhaps, as a “solution architect”, I see that email misuse is an impediment to efficiency, productivity, and technology platform evolution. Reliance on email slows development and adoption of systems that are better suited for information delivery and processing. Email is the duct tape of system integration and monitoring/alerting infrastructure. When email is misused, system reporting is crippled and process analysis is next-to-impossible. Worse than that, the opportunity to build an efficient technology foundation, the benefits of which can be reaped over and over again, is missed as effort is directed towards an inferior architectural approach. I can put email misuse into 3 major categories:

1. Email as a monitoring solution
2. Email as a reporting solution
3. Email as a document management solution

I sort of covered the third one in my previous blog (Email Attachments are Evil), so I’ll focus on the first two in the sections below.

Email as a monitoring solution

Instead of a proper monitoring infrastructure, many systems rely on email as a vehicle for communicating alerts, status information, notifications, et cetera. It’s a “quick and dirty” way of putting something in place when a ‘real’ monitoring solution is missing or inadequate. It seems to be a very common scenario to write scripts that, as part of, their workflow, send an email to indicate the completion (or failure) of a particular step or execution milestone. What ends up happening, especially in a heterogeneous environment with many systems, is that you get an unmanageable collection of scripts (written by different authors) with point-to-point emails (addresses embedded in scripts) being sent to an unknown number of individuals or distribution lists for undocumented reasons. Visibility into issues is greatly reduced, ability to react quickly and appropriately is compromised, ability to report accurately to customers is also compromised (which becomes ever more important if you have any kind of an SLA to maintain). Data collection becomes a manual process – expensive, not scalable, and error-prone. A “master” view into the state of the systems is near impossible.

It’s as classic an argument for a “service bus” approach as one could think of. Wouldn’t it be better to have a central “interceptor” for system events generated by any system that could then provide an appropriate alert delivery channel (which would, in fact, be email – or maybe instant messaging, or maybe a web service call, or…)? Build it once, and all other monitoring “subscribers” would “plug” into this infrastructure – perhaps even using email as a delivery channel of alert information to the monitoring system.

Email as a reporting solution

Another common misuse of email is the delivery of source data for analysis. Say, you need to provide previous day’s usage statistics to your customer. A common approach would be to have your DBA write a query that runs at 12:01am on the new day, and emails the results to your technology team or a distribution list. An analyst on the team takes the data (usually in CSV format), copies them to a spreadsheet (possibly an existing one), performs whatever analysis magic that needs to happen, and then sends out a chart, pivot table, whatever, to the customer. This last step is performed daily, and if any changes are needed to the source data, date range, whatever – the DBA again is involved and (at best) modifies the original query or (most likely) creates yet another query, which follows the same process. Let’s say you also have to do the same thing monthly, and for more than one customer, and across multiple systems – and you can probably guess (or maybe don’t even have to guess) that it quickly becomes quite unsustainable and expensive.

In other words, as soon as you have to scale, all hell breaks loose. After a while, the management of the copy/paste process takes up more and more time, and as the system and reporting requirements grow, errors are introduced, the number of queries becomes unmanageable, and someone gets yelled at because nothing reconciles with anything else.

Again, a service-bus approach is the way out of this reporting spaghetti. Create a data warehouse, or a pseudo-data warehouse if OLAP isn’t your thing yet, then make all your data dumps go into a separate reporting database. At the very least, you’ll reduce the number of copy/paste operations if your analysts can connect their spreadsheets directly to the reporting database. You also reduce some complexity, as even through such a simple step you could reduce the number of queries run on the system (run only dailies, for example, and sum things up by date – instead of running dailies plus the monthly reports). The addition of dimensional modeling (conformed dimensions, fact tables, et cetera) and OLAP will take this yet another step towards a pan-enterprise, cross-system, ad hoc, reliable reporting.

In conclusion…

Hopefully you didn’t recognize your organization when you read the description of email misuse in the above two sections. If you nodded your head and thought “yep, that’s us” – at least you’ve either found a confirmation of what you already suspected, or now have an idea of an alternative to the world of hurt in which you or someone in your organization probably find yourselves.

In my last blog post (in which I extolled the virtues of blogging for project communication instead of using email), I wrote this:

By the way, if any of you out there routinely use attachments to share documents, i implore you to cease and desist. If your corporation doesn’t have a document management strategy, it needs to get into the 21st century, especially since 2009 is likely to favor companies that go after efficiency…

What I should have said was that there should be a good document management strategy. Without explicit rules that drive culture change, encourage behavior that corresponds to long term vision, and discourage behavior that goes against it, the inertia will take you exactly where you already are, which, for all intents and purposes, is akin to going backwards.

Here’s why attachments should be banned, or at least discouraged, by any forward-thinking company (the order isn’t indicative of importance or impact):

1. Email becomes your document management system
2. There’s no security (unless you implement IRM [Information Rights Management], which is even more rare [rarer?] than the jackalope)
3. There is no document versioning
4. There is no change control
5. There’s no access auditing
6. There’s no record retention
7. There’s no metadata management
8. Information contained in the document can’t be discovered (since it’s locked to individual mailboxes/computers, promoting silos)
9. Infrastructure investment is directed towards a mismatched category of IT (communication channels instead of information management)
10. Any investment in a real document management system becomes “money down the drain”, because if you’re emailing documents, clearly you’re not using your “real” document management solution

There are a few other reasons that can be added to my list (inefficient server space usage, additional administrative overhead, clogged mailboxes, reduced performance, and so on), but 10′s a good number, so I’ll stop there.

If you ever dream of liberating useful information out of the prison cells that documents really are, the first step is to end our dependence on email attachments – at least as we know them now…

So, why are we still emailing attachments around? Unfortunately, the answer is simple: it’s too damn convenient. The technology has been around for a long time, the concept has become part of our “collective subconscious”, and the supporting applications (both on the client and the server) have been fine-tuned to near perfection. Anything new is none of the above. Elegant integration with existing corporate email solutions is the real answer to the problem, and, as far as I know (am I wrong?), there’s no such thing out there.

Another reason for email dominance is kind of a chicken-and-egg problem. We spend a lot of time in email simply because that’s where our stuff is. That’s where the information lives, that’s where our documents are, et cetera. What if the information no longer lived in email? What if blog posts were how we learned things? Would we still email around so many attachments? Or would we start using email as a notification channel? I think the answer is yes – although I don’t have any corporate examples to back it up.

The real solution to the problem must include three components – server component that is a real document management platform, client software that is aware of the special capabilities of the server, cultural awareness of the attachment problem and an explicit agreement to address it. Those three components can be implemented separately, although in order for a true end-to-attachments to begin, the first two must be well designed and executed.

One (partial) solution I’ve been evangelizing is the consistent use of SharePoint (and DocuShare, where appropriate) as the real repository for documents of any kind, and the sending of links instead of attachments. This is only a partial solution (for several reasons), and even the most progressive of my colleagues will lapse back into attachment addiction when the pressure is high and time is short – and I can’t blame them. The extra steps required to upload a document and then email it, are an impediment to an efficient workflow. This is kind of a brute force solution, in which I taunt, cajole, and preach to bring about behavior change.

Moving information radiation to team/project blogs is another good way to start reducing the number of attachments emailed. As people spend time in blogs, the concept of document uploads becomes much more accessible and easier understood when the familiar “crutch” email tools aren’t under hand to subvert the move of information radiation online.

Who knows, maybe 2009 will be the year email attachments are eradicated?

Tagging all the emails belonging to this project or that project is a good way to get through the email landfill, as is focusing on the latest threads first and skimming over the rest to find offshoots & branches that didn’t quite develop into anything. Still, it took me a good 3 hours to go through all the updates and figure out where things are, after two weeks of not paying attention.

There is, of course, a better way… It does require a bit of a culture change, which immediately makes it an uphill battle, but hear me out.

What if, instead of sending out emails to the team, each individual team member updated (or sent an update email to) the project/team blog? In this beautiful utopian world of mine, all i’d have to do is open up the RSS reader for my project blogs, and have the evolution/storyline of my project unfold in front of my eyes as if i were reading the project’s diary (which, of course, is exactly what i would have been doing). There would have been no need to search my inbox for related messages, no need to waste time identifying messages i’ve already looked through as part of a newer message’s trail, categories would have been applied by authors for easier segmenting, and i probably would have spent less than half the time catching up on all the stuff that happened.

As a side benefit (and a major one), i would have had an online history of the project for all to view – instead of locking that history down in several mailboxen of individual team members. Of course, this history would have been keyword searcheable and could be integrated into project documentation (wiki) to make project data all the richer.

The blogging process can be made easier if the project blog were email-enabled. This way the familiar email becomes a tool in the new information radiation infrastructure, rather than remaining an instrument of information silos and mass document confusion. By the way, if any of you out there routinely use attachments to share document, i implore you to cease and desist. If your corporation doesn’t have a document management strategy, it needs to get into the 21st century, especially since 2009 is likely to favor companies that go after efficiency…