Email Abuse and System Architecture
Posted by ilyal on January 18, 2009
Email misuse?
It seems as though all my recent blog posts are preoccupied with email. Perhaps, as a “solution architect”, I see that email misuse is an impediment to efficiency, productivity, and technology platform evolution. Reliance on email slows development and adoption of systems that are better suited for information delivery and processing. Email is the duct tape of system integration and monitoring/alerting infrastructure. When email is misused, system reporting is crippled and process analysis is next-to-impossible. Worse than that, the opportunity to build an efficient technology foundation, the benefits of which can be reaped over and over again, is missed as effort is directed towards an inferior architectural approach. I can put email misuse into 3 major categories:
- Email as a monitoring solution
- Email as a reporting solution
- Email as a document management solution
I sort of covered the third one in my previous blog (Email Attachments are Evil), so I’ll focus on the first two in the sections below.
Email as a monitoring solution
Instead of a proper monitoring infrastructure, many systems rely on email as a vehicle for communicating alerts, status information, notifications, et cetera. It’s a “quick and dirty” way of putting something in place when a ‘real’ monitoring solution is missing or inadequate. It seems to be a very common scenario to write scripts that, as part of, their workflow, send an email to indicate the completion (or failure) of a particular step or execution milestone. What ends up happening, especially in a heterogeneous environment with many systems, is that you get an unmanageable collection of scripts (written by different authors) with point-to-point emails (addresses embedded in scripts) being sent to an unknown number of individuals or distribution lists for undocumented reasons. Visibility into issues is greatly reduced, ability to react quickly and appropriately is compromised, ability to report accurately to customers is also compromised (which becomes ever more important if you have any kind of an SLA to maintain). Data collection becomes a manual process – expensive, not scalable, and error-prone. A “master” view into the state of the systems is near impossible.
It’s as classic an argument for a “service bus” approach as one could think of. Wouldn’t it be better to have a central “interceptor” for system events generated by any system that could then provide an appropriate alert delivery channel (which would, in fact, be email – or maybe instant messaging, or maybe a web service call, or…)? Build it once, and all other monitoring “subscribers” would “plug” into this infrastructure – perhaps even using email as a delivery channel of alert information to the monitoring system.
Email as a reporting solution
Another common misuse of email is the delivery of source data for analysis. Say, you need to provide previous day’s usage statistics to your customer. A common approach would be to have your DBA write a query that runs at 12:01am on the new day, and emails the results to your technology team or a distribution list. An analyst on the team takes the data (usually in CSV format), copies them to a spreadsheet (possibly an existing one), performs whatever analysis magic that needs to happen, and then sends out a chart, pivot table, whatever, to the customer. This last step is performed daily, and if any changes are needed to the source data, date range, whatever – the DBA again is involved and (at best) modifies the original query or (most likely) creates yet another query, which follows the same process. Let’s say you also have to do the same thing monthly, and for more than one customer, and across multiple systems – and you can probably guess (or maybe don’t even have to guess) that it quickly becomes quite unsustainable and expensive.
In other words, as soon as you have to scale, all hell breaks loose. After a while, the management of the copy/paste process takes up more and more time, and as the system and reporting requirements grow, errors are introduced, the number of queries becomes unmanageable, and someone gets yelled at because nothing reconciles with anything else.
Again, a service-bus approach is the way out of this reporting spaghetti. Create a data warehouse, or a pseudo-data warehouse if OLAP isn’t your thing yet, then make all your data dumps go into a separate reporting database. At the very least, you’ll reduce the number of copy/paste operations if your analysts can connect their spreadsheets directly to the reporting database. You also reduce some complexity, as even through such a simple step you could reduce the number of queries run on the system (run only dailies, for example, and sum things up by date – instead of running dailies plus the monthly reports). The addition of dimensional modeling (conformed dimensions, fact tables, et cetera) and OLAP will take this yet another step towards a pan-enterprise, cross-system, ad hoc, reliable reporting.
In conclusion…
Hopefully you didn’t recognize your organization when you read the description of email misuse in the above two sections. If you nodded your head and thought “yep, that’s us” – at least you’ve either found a confirmation of what you already suspected, or now have an idea of an alternative to the world of hurt in which you or someone in your organization probably find yourselves.
