Read an Excerpt
Extract from Chapter 1 of 'Enterprise Application Architecture'.
Delivering an Enterprise
If you are anything like me, you're probably sick and tired of hearing the UNIX and Mainframe pros rattling off the reasons why our PC based systems can't handle the tough jobs their overpriced boxes perform. For quite a while now, we have had to bide our time, bite our tongues, and bear with their bloated senses of superiority. That time has passed. Our venerable PC is all grown up. It has matured into a powerful creature, more than capable of standing toe-to-toe with any of its alternatives. But be warned. This newfound prowess is really a double-edged sword. The fact that our equipment is up to the task doesn't mean that we are completely ready to step into the ring with the big boys yet. We still have a lot of work to do. This book is a step-by-step guide designed to give you all the tools you need to create an Enterprise Caliber System for your organization, no matter what size it is.
Using this BookI want to avoid any confusion or misconceptions, so I want to make a couple of things clear at the start. First, the main focus of this book is the design, development, and deployment of world-class applications across a distributed architecture. What that really means, is that we can use as many machines as we need in order to accomplish a given task. So if we were collecting and managing information from every cash register across the globe for a major retail company, then we might need 80 or 100 machines to handle that task. But, if our task was to design and develop software, then we can accomplish that task with a single machine.
Then you can take that code, and distribute it across 2, 3, 4 or even 100 or more machines if that is what the application calls for.
You will not need to change a single line of code to re-deploy your application.
The reason I have emphasized the multi-machine platform in this book is so that I could give you the concepts and specific instructions you need to deploy and use your application across multi-machine platforms. Most of the books I have read go to great lengths to say that this can be done, but then they fail to give you a clue about how to actually do it. This book is intended as a practical guide for delivering an enterprise. So, it contains the real-world instructions you are likely to need.
Second, you might notice that there is a lot of code in this book. Please don't let the volume of code cause you to believe that the coding techniques in this book are difficult to accomplish. My teams use the code in this book every day to get real work accomplished quickly and efficiently. In other words, sometimes I know that the actual code itself is really more valuable than anything I might have to say about the code. I think a good programming book should be, kind of, like having a smart friend sitting next to you - one that you can count on. In other words, what you will learn in this book is not some 'pie-in-the-sky' theory of how things should be done. It's a practical guide that explains exactly how to do it.
I have presented the material in an order that I believe will give you the tools you need to be able to "read" the code more and more easily as the chapters progress. I have also provided several tools that will actually do the job of writing some of the more nasty sections of code for you. I think about it like this. If I can handle some of the coding chores for you then I have sort of paid a price for your time. I hope that price gives you the freedom you need to stop, take some time, and really consider some of the interesting ideas in this book without any unnecessary aggravation.
An Enterprise Caliber SystemDistributed Architecture, ActiveX, Java Beans, OLE, DNA, ASP, XYZ, 123, blah blah blah. Right about now you are probably wondering if it will ever end. Don't worry so is everyone else. Every once in a while, just when I think I've got it all figured out, someone goes and throws another new twist at me, and I to begin to wonder if it is all worth it. How can I ever hope to build a reliable, stable system if someone keeps changing the rules? I will tell you how, take a hint from the classics. Even in these turbulent technical times, there are some timeless design principles that we can rely upon to guide us through the confusion. They are the same principles that good programmers and engineers have been using for years.
The cornerstone of this design is that the entire system is constructed in a modular fashion, which divides the processing into atomic units that can be tuned with pinpoint accuracy. This same atomicity allows the system to be flexible enough to handle anything else Uncle Bill at Microsoft may be planning to throw at us.
The Physical SystemIt is not possible to buy the perfect piece or pieces of hardware that will magically give an organization an Enterprise Caliber system. This is a hard notion for many companies to swallow. In the past, it may have been a perfectly reasonable approach to buy the biggest and best mainframe (or midrange) available and depreciate the beast over the next X number of years. If a company needed more computing power, they just bought a bigger box - problem solved. Today, while I imagine it's still possible to just go out and buy the biggest and best mainframe, most companies have come to the conclusion that this monolithic approach is probably not the best way to solve an organization's long term information management problems.
MainframesWhat I do want to talk about is the problems that occur when you try to apply the mainframe (or buy a bigger box mentality) to the enterprise design problem. This image is supposed to represent a big mainframe connected to a large number of client workstations. When most people envision a physical information system this image, or one rather like it, is probably what they conjure up:
So, it is not surprising that when companies first started creating enterprise level client-server solutions that they first envisioned the client-server model as something of a scaled down mainframe. It seemed like a perfectly reasonable solution to buy a single, powerful, server and connect a bunch of clients to it. The problem with this approach is that a mainframe is really very much more than just a big box. It is really a finely tuned orchestration of processes. The fact that these processes are executed, more or less, in the same box made the management of these different processes an expected, understood, and vital responsibility of the system's keepers.
Enterprise Servers - The Wrong WayWhen companies took their first run at designing an enterprise using servers in place of the mainframe, the structured environment that the mainframe had provided was missing. When the performance of these systems turned out to be less than anticipated everyone naturally assumed that this was due to the new server's physical lack of processing power:
Of course the real problem was not the less powerful machine as much as it was the system's lack of structure. This was due to the fact that the overall system management functions, that the mainframe world takes for granted, were often not identified, misunderstood, or just plain overlooked. In other words, in these server-based systems, there were few if any reins placed upon the clients. There was nothing stopping any, or all of the clients from initiating 1,000,000 row queries against a database simultaneously. You can imagine the results - the machines choked and died. Of course, everyone attributed the server's inability to handle the load to its most noticeable feature - its diminutive CPU. In other words, most people failed to realize that the real reason mainframes can handle so many clients is because mainframe systems don't let their clients run amok in the system.
Anyway it seems that the industry as a whole concluded that, because of their lack of processing power, server-based systems could only handle a small number of users. What followed was an understandable, but incorrect impression of how to create an Enterprise Caliber system.
Take a look at the next image and see if it looks familiar:
I bet your organization's infrastructure is set up something like this. Instead of having a single centralized resource, you probably have quite a few databases or servers spread throughout your organization. If you ask just about anyone why this is so, they will almost certainly answer that the smaller systems (UNIX and NT based) can only manage X number of users.
It is entirely possible to divide and secure data on a single server that can be used by multiple departments.
If you work for a large company, then you know the other discouraging facet of this design. This facet is perhaps the most expensive one - the myriad of "data bridges" most companies employ in an heroic but nearly futile attempt to make some sense from all of the different pockets of information they have collected. Unfortunately, when most people think about distributed computing, they just take it for granted that the system must be designed as shown above - several separate lines of communication between different user groups and their associated data stores. This is so wrong! This design is nothing more than the illegitimate child of a reasonable but incorrect delusion the industry adopted some years ago.
Distributed architecture does not have to mean distributed data.
Enterprise Servers - The Correct WayLet me make my major point clear. Although enterprise level servers are not as powerful as mainframes, the reason that mainframes seem to be able to handle so many more clients is because, inherently, the mainframe system actively manages the clients' access to its available resources. At some point, this management is more important to the overall system performance than the amount of raw processing power. Think about your friends or acquaintances for a minute; I bet you know two people that earn about the same amount of money, but one of them seems to be more financially secure than the other one. For both individuals the available resource, money, is limited and essentially the same. How do you explain the difference? Like every other engineering problem, it always boils down to one thing - the management (or better, the optimization) of a limited resource.
What this means is that while it doesn't hurt to have more powerful machines, what we really need to do at the enterprise level is to hire a smarter machine. We need a machine (read machine to include the operating system) that can monitor the demands on the system and respond accordingly. In other words, we do need to do what the mainframes do. But fortunately for us, we don't need to do it the same way. While the mainframe world has essentially one option "I think I need a bigger box", we have a nearly infinite number of choices at our disposal.
The real trick here is learning how to integrate the additional resources into the system in a manner that will enhance the system as a whole. This problem cannot be solved with hardware alone. It requires the same finely tuned orchestration of processes that the mainframe world applies to the problem. In real terms, this means that when an organization purchases or deploys hardware it has to do more than the typical polite interchange that too often occurs between the hardware and development teams.
Developers really need to understand what hardware options are available and just as importantly, the hardware team really needs to understand the nature of distributed architecture from the system architect's perspective. The good news is that even if your organization has made every mistake in the book, this problem is really more about the deployment and management of physical resources rather than a hardware specific issue. In other words, as long as the hardware/network team has purchased basically sound equipment, it can always be re-organized into a more efficient configuration that can grow to meet the needs of any number of clients.