The BizTalk 2000 Developer's Guide is written for developers who are responsible for installing, configuring, and deploying the BizTalk Server in their organizations IT infrastructure. The opening chapters of the book discuss the benefits of seamless business-to-business application integration, summarize the features and enhancements of BizTalk Server 2000, and offer an introduction to XML - the driving force behind BizTalk. The next chapters explore the multiple tools that are incorporated into BizTalk that will transform the way in which information is created, transmitted and maintained in the server environment. Other chapters include complete coverage of the security considerations for BizTalk, and an examination of the various third-party plug-ins for enhancing BizTalk Server 2000. The book also includes a companion CD with demo software plug-ins from the BizTalk Developer’s Organization, and the complete ready-to-use source code from the book.
About the Authors: Scott Roberts (MCSE+I 4.0, MCSE 2000, MSF, MCDBA, MCT, MCP + Site Building) was one of the first 1600 MCPs in the world. He has a long history with Microsoft products and technology and is currently employed as a Senior Consultant within the Microsoft Consulting Services, Platform Consulting Organization. This group develops and deploys solutions for Enterprise customers focused on the .NET Server platform. Prior to joining Microsoft, Scott was the President/CEO of Enterprise Technology Group Inc., a Windows 2000 and e-commerce development, consulting, and training company. He has also been a featured conference speaker on messaging and e-commerce topics throughout the country.
ChrisFarmer (Ph.D., MCSD) is a consultant at SciTegic in San Diego, CA where he specializes in integration of scientific applications for pharmaceutical and biotech companies using SOAP and other XML-based technologies. Chris’s recent background includes design and development of .NET-based Web applications and extensive e-commerce database development and integration with legacy systems using XML with IT-Age Corporation in Atlanta, GA. Chris holds a bachelor’s degree from the University of Virginia and a Ph.D. from the University of Georgia. Chris currently lives in sunny San Diego, CA with his wife, Michelle.
Robert J. Shimonski (CCDP, CCNP, NNCSS, MCSE, MCP+I, Master CNE, CIP, CIBS, CWP, CIW, GSEC, GCIH, Server+, Network+, Inet+, A+) is a Lead Network and Security Engineer for Thomson Industries Inc. Thomson Industries is the leading manufacturer and provider of linear motion products and engineering. Robert’s specialties include: network infrastructure design with the Cisco and Nortel product line; network security design and management with CiscoSecure and PIX Firewalls; network management and troubleshooting with CiscoWorks and Sniffer-based technologies; systems engineering and administration with Microsoft NT/2000/XP, UNIX, Linux, Apple, and Novell Netware technologies; and developing a host of Web-based solutions for companies securing their market on the Web. He has also contributed to hundreds of articles, study guides, and certification preparation software for Web sites and organizations worldwide, including Brainbuzz.com and SANS.Org. Robert’s background includes positions as a Network Architect at Avis and Cendant Information Technology. Robert holds a bachelor’s degree from SUNY, NY and is a part-time Licensed Technical Instructor for Computer Career Center in Garden City, NY teaching Windows-based and networking technologies. Robert has previously contributed to the Syngress Publishing title, Configuring and Troubleshooting Windows XP Professional (ISBN: 1-928994-80-6), and is the Technical Editor of the forthcoming Sniffer Network Optimization and Troubleshooting Guide (ISBN: 1-931836-57-4).
Milton Todd is a software engineer at InterKnowlogy, LLC in Carlsbad, CA. InterKnowlogy is a consulting firm and Microsoft partner providing custom software and infrastructure solutions for secure and effective use over the Internet. Milton has focused the last year on developing BizTalk solutions, primarily to the insurance industry. Previously, he developed front- and back-end applications in the e-commerce and manufacturing industries, depending heavily on Microsoft technologies. Milton holds a bachelor’s of science in Mechanical Engineering and spent several years in the design and construction field, experience that has provided a firm grounding in practical problem solving and the design process. When possible, he continues to teach mathematics. Milton currently resides in Diamond Bar, CA with his wife, Lida.
|Product dimensions:||7.42(w) x 9.20(h) x 1.39(d)|
Read an Excerpt
Capturing TrafficAs you already know, Sniffer Pro is a great tool to help you capture traffic. Network problems are not a rare thing for a network administrator to experience. Unfortunately, broadcast storms, slow responses, or network attacks happen quite frequently. You have to choose the best way to identify each problem, analyze it, and sort it out. Using Sniffer Pro to capture traffic is one of the fastest ways to obtain a complete picture of what is happening on your network, analyze captured information, and resolve the issue. It is also possible to capture traffic and to experiment with it, to analyze in a test environment the ways your network would react to specific groups of data.
When you use Sniffer Pro, all captured traffic goes straight to the capture buffer. Captured data can be saved on a hard drive to be used for future reference and review.
You can also send the captures to your colleagues to share some ideas, or you can open captures made by somebody else for you. In some cases, you might also want to use captured data for baselining purposes. Besides the ability to capture all the data that is flowing on your network, Sniffer Pro has broad filtering capabilities that greatly facilitate troubleshooting on highly loaded networks. You can perform filtering by station addresses, data pattern, or different protocols. You will learn more about filtering in Chapter 8.
How to Capture Traffic
When you capture traffic, it is important for you to make up your mind whether you want to capture all the packets Sniffer Pro can see and select interesting ones using display filters, or whether you want to define a capture filter beforehand and capture only the packets that are related to the problem you are exploring. Both methods have their advantages and disadvantages.
In the first case, when you capture all the traffic, you have more flexibility afterward because you have a full snapshot of your network’s traffic. There is one drawback here: Thousands of packets per second can flow through your network, carrying many megabytes of data. Regardless of how big your hard drive, you probably do not want to store gigabytes of almost useless information on it. This does not mean capturing all the data is a completely ineffective choice. Capturing all the data on the network without a filter applied allows you to see all the traffic passed over the transmission media, thus giving you a very clear picture of exactly what is there. It allows you to “feel” the customer’s network and, in some cases, even resolve the problem that the customer is complaining about, without using sophisticated filtering and troubleshooting techniques.
You can always apply a filter to a capture buffer after you’ve stopped the capture process to filter out data that is relevant to the problem you are working on. You can even apply another filter to the data you to which have already applied a filter, to get more granular information. We discuss how to do this later in this chapter.
After you have taken a snapshot of your customer’s network, you might want to get a more precise picture and start capturing only the data related to the problem you are troubleshooting. In this case, you can then define a specific capture filter so that you can find particular things you are looking for, making the capture considerably shorter. This also means that you won’t have to worry about your PC resources. Keep in mind the main disadvantage of this process: You might miss something very important if you define an incorrect filter.
Taking Captures from the Menu and the Toolbar
There are a few different ways of taking captures:
By choosing Capture | Start from the main menu
By pressing the F10 key
By pressing the Start button on the main toolbar (it looks like the Play button on your VCR)
You must understand how to use a number of other buttons on the main toolbar and Capture menu as well (see Figure 6.1). The first four buttons along the top are the familiar buttons to open, save, print, and stop printing. The functions of the next eight buttons are described in Table 6.1.
Figure 6.1 The Main Menu and the Toolbar
Table 6.1 The Main Toolbar and Capture Menu Buttons
Start capture By pressing this icon, you can start the capturing process. You can also start the capturing process by pressing the F10 key.
Pause capture By pressing this icon, you can stop the capturing process at any time and resume it later.
Stop capture Terminates the capturing process. You can stop the process to view the information or save it to a file. You can also stop capturing by pressing the F10 key.
Stop and Display Stops capturing and displays the frames captured. You can do the same thing by pressing the F9 key.
Display Displays a stopped capture. You can get the same result by pressing the F5 key.
Define filter Defines the filter used to capture the frames. Although
Chapter 8 is dedicated to the detailed discussion of filters, we define a few simple filters in this chapter.
Select filter Chooses a filter from the list of filters you have defined.
Capture Panel Brings up the Capture Panel, which we discuss in greater detail later in this chapter.
Address Book Lets you to assign recognized names for your network nodes.
Pulling Up the Capture Panel
The Capture Panel is at the center of your capturing process. It gives you all the information about the capturing process, such as how many packets have been captured, how much space is left in your buffer, and much, much more.
To pull up the Capture Panel, you can either go to the Capture menu and select Capture Panel or click the Capture Panel button in the main toolbar. The Capture Panel is very important because it is used to view the status of the capture process. At the bottom of the panel are two tabs: Gauge (see Figure 6.2) and Detail (see Figure 6.3). On the Gauge tab, you can see two gauges that show the following:
The number of packets captured
How full the buffer is
Figure 6.2 The Capture Panel’s Gauge Tab
Figure 6.3 The Capture Panel’s Detail Tab
Note that when the buffer is 100-percent full, the packets can be dropped or the capturing process can cease, depending on the settings, which we discuss a little later in the chapter. The Detail tab shows you additional details:
# Seen The number of frames Sniffer Pro sees.
# Dropped The number of frames dropped due to the lack of performance of the computer on which you are running Sniffer Pro. Packets are often dropped during periods of high network activity.
# Accepted Shows the number of frames that were put into the capturing buffer.
# Rejected Indicates how many frames did not satisfy the filtering rules you have defined. Frames can also be rejected if your buffer is 100-percent full.
Buffer size The size of the capturing buffer you have defined. We discuss the process of buffer definition in the following section.
Buffer Action Indicates the status of the buffer. Wrap means that the buffer will wrap as soon at it becomes full. Wrapped means that the buffer has wrapped. Stop means that the capture process will stop as soon as the buffer becomes full. Stopped means that the capture has stopped because the buffer is full.
Saved file # Shows to which file the capturing is being saved.
Slice size Shows whether Sniffer Pro captures the whole frame or just a part of it.
Elapsed time Indicates how long ago Sniffer Pro was started.
File wrap Wrap indicates that the files have been overwritten as the number of saved files has been reached. We talk about this option in the following section.
Please pay attention to the fact that the Capture Panel we have just discussed is not the same as the Sniffer Pro Dashboard, although their gauge tabs look alike. To access the Capture Panel, select Capture | Capture Panel. To access the Dashboard, select Monitor | Dashboard.
As a Sniffer Pro expert, you should understand that the Capture Panel can be quite useful. You can use it to easily see how many packets have traversed your network since you have started capturing, how many frames were filtered out (rejected), and how many frames Sniffer Pro dropped because your computer did not have enough resources to capture them.
Saving and Using Captures
It is very important to know how to save the information you have captured, because you will definitely need to open these captures later for future analysis. As a network analyst, you can spend hours looking into the data that took you only a few minutes or even seconds to capture! Sometimes you might decide to send the capture to your colleagues to get a second opinion on a problem you are investigating.
Throughout Sniffer Pro’s evolution, a variety of captured files formats have been used. Some of them could support compression; others could not. In addition to the file formats used to save captures, Sniffer Pro uses some other file formats for additional information. Table 6.2 lists all these formats.
Table 6.2 Sniffer Pro File Extensions
.cap Uncompressed capture files
.caz Compresses capture files; Sniffer automatically compresses data if you select this format
.enc Original format for Ethernet traces
.trc Original format for Token Ring traces
.fdc Original format for FDDI traces
.etm Broadcast and functional addresses
.trm Broadcast and functional addresses
.hst Saved history samples
.csv Saved history samples
.btr Token Ring Sniffer Pro table of assigned manufacturer IDs
.bet Ethernet Sniffer Pro table of assigned manufacturer IDs
In addition to understanding various Sniffer Pro formats, you should be able to distinguish among them and use the formats of other packet analyzers so that you can open files captured using other tools. We discuss all these processes in detail in the following sections.
Now that you know why it is important to save captured data, you need to understand how to save this data for future analysis. When using Sniffer Pro, you could come across a troubleshooting scenario in which you need to remotely capture data from multiple locations. If this is the case, the versions of Sniffer Pro covered in this book (versions 3 and 4) will not allow you to natively perform this task. You can't capture traffic on different remote segments with this product, so if you need to do that, you might need to purchase an enhanced version of Sniffer Pro called Sniffer Distributed. This product will allow you to capture traffic on all key segments of your network using Sniffer Distributed agents. Is there a way you can circumvent this issue for now and capture the remote traffic? Yes, there is a way, but the logistics of doing so could become quite a hassle. You can always ask someone to capture that data for you (or you can do that with a remotely controlled workstation). Once the data is captured, you can upload the capture files and analyze the captures from the comfort of your own machine.
After you or somebody else has captured the traffic, you need to be able to save it for future analysis. There are two ways of saving data captures:
Automatic saving when the capturing buffer is full
Manual saving is very popular because you can view the data you have captured and save it only if you find it necessary to do so. To perform manual saving, you must stop capturing and display the capture buffer (select Capture | Stop and Display or simply press the F9 button). Then you can actually save the data in your capture buffer. To do so, from the main menu, choose File | Save or Save As. Another alternative is to click the Floppy icon in the main toolbar (refer back to Figure 6.1). A standard Windows Save As dialog window appears on your screen. From here you can select the directory into which you want to save your capture, the filename, and the extension or type of file. (Refer back to Table 6.2 for the list of known extensions.)
Automatic capture is useful if you want to capture a great deal of data, such as a volume that would not fit into your computer’s memory. Automatic capture is also helpful if you definitely know that the data you are capturing is required for future analysis and you want to save it on your hard drive right away, without going through the manual-saving process. Before you begin capturing, you must define a special filter profile (although no actual filtering is done here; you save all the packets you have received). The filter profile allows Sniffer Pro to save the buffer content to a file.
It is usually not a good idea to modify a Default profile because it is used as a starting point for any new profile you create on your computer. For that reason, you should always create a new profile for new filters.
Let’s create a new a new capture profile by following these steps (see Figure 6.4):
1. Select Define Filter from the Capture menu.
2. In the Define Filter window that appears on your screen, press the Profiles button.
3. In the Capture Profiles window, press the New button.
4. Choose an appropriate name for your profile (for example, LightPave), and select OK.
5. Press Done to close the Capture Profiles window.
Figure 6.4 Creating a New Capture Profile
Now you are ready to modify the new profile you have just created. Switch to the Buffer tab. The Buffer tab window is divided into four main areas (see
When buffer is full
Let’s take a close look at each of these sections.
Figure 6.5 The Buffer Tab in the Define Filter Window
Buffer size allows you to select how much memory on your computer is actually used for the capture buffer. If your computer has a very limited amount of memory and you overset the buffer size, you can crash your computer or freeze Sniffer Pro. To avoid this situation, you should decrease the size of the buffer and use the Save to File option we discuss shortly. Note that the capture buffer’s default size is 8MB. You can manually modify the buffer size in a range from 256KB to 40MB. The available buffer sizes are:
You can check how much memory is available for a buffer on your computer by executing Task Manager on Windows 2000/NT or System Monitor on Windows 98/95. Make sure that the buffer size you have configured does not exceed available memory on your computer. We also recommend you close background applications (such as ICQ, Office Panel, or Real Player) to maximize the memory available for Sniffer Pro. Disabling unnecessary Sniffer Pro Expert Objects also allows you to optimize memory usage.
For example, if you have a notebook with 96MB of RAM, your Windows 2000 can use approximately 46MB of the available memory and Sniffer Pro can use approximately 30MB, so your buffer size can be anywhere between 256KB and 20MB. The standard 8MB buffer size looks like a good choice in this case. The Packet size option allows you to choose if the whole packet should be captured (the default option) or only some part of it (between 32 bytes and 18,432 bytes).
The When buffer is full option allows you to modify Sniffer Pro’s behavior in the event that the capture buffer becomes full. The program can either Stop capture or Wrap buffer and keep capturing data.
To enable automatic saving, choose the Save to File option and specify the filename prefix as well as the number of files you want to be created on your hard drive. Indicate the directory to which you want the files to be saved.
Other options you should specify to complete your setup are as follows:
Filename prefix Defines a common prefix of saved capture files.
Unique names This option specifies whether the analyzer must use a unique filename for each saved file. Sniffer Pro will make sure that the filenames are unique by assigning three random letters prior to the extension, as shown see in the following example. This option can be useful if you want to be sure that you don’t overwrite the files you have previously captured. Check to make sure that you have enough space on your hard drive to accommodate all the files.
Number of files This option sets the maximum number of files Sniffer Pro will create on the hard drive.
Wrap filenames This option specifies whether the files for this capture can be overwritten as soon as the number of saved files has been reached. Disabling of this option tells Sniffer Pro that it should stop capturing as soon as it fills its buffer and saves the number of files you have specified.
To better understand what these options actually do, perform the following exercise. Modify the new profile you have just created using these options:
1. Type LightPave as the filename prefix.
2. Select 3 as the number of files.
3. Enable the Unique Names option. Do not enable the Wrap filenames option, so Sniffer Pro will stop after the files become full.
4. Specify C:\Capture as the capture buffer directory.
5. Start the capturing process by pressing the F10 key. Sniffer Pro will automatically stop capturing as soon as three files are filled.
Now if you look into the C:\Capture directory to which you saved the captures, you will see three files that will look like the following:
LightPave here is the file prefix you chose; 001, 002, and 003 are the file numbers; and ajr is the randomly generated unique file identifier, so it can be different if you repeat this exercise.
Table 6.2 summarized different file types used by Sniffer Pro. Now let’s talk about the file types that are directly related to saving captured data—the ones you can select while saving your captured data on a hard drive.
When Sniffer Pro was introduced, capture files had extensions that depended on the type of network adapter used. Ethernet files had an extension *.ENC, Token Ring files had *.TRC, and FDDI files had *.FDC.
With the release of the Windows version of Sniffer Pro, new file formats were invented. Now Sniffer Pro uses the same *.CAP format for all types of interfaces. Sniffer Pro saves files in a unified uncompressed format, so the files can grow dramatically if you capture too much data. To prevent this situation, you can save your captures with the *.CAZ extension. In this case, Sniffer Pro automatically compresses your data. In the majority of cases, this extension will significantly reduce the drive space needed to save your captures.
For backward compatibility with other versions, Sniffer Pro permits you to save captures in the original Sniffer formats (*.ENC, *.TRC, and *.FDC).
Table of ContentsChapter 1
Introduction to Sniffer Pro
Installing Sniffer Pro
Exploring the Sniffer
Configuring Sniffer Pro to Monitor Network Applications
Using Sniffer Pro to Monitor the Performance of a Network
Capturing Network Data for Analysis
Analyzing Network Issues
Understanding and Using Triggers and Alarms
Detecting and Performing Security Breaches with Sniffer Pro
Troubleshooting Traffic for Network Optimization
- BizTalk Messaging Services
- Document Definitions
- Distribution Lists
- Messaging Services Object Model
- Solutions Fast Track
- Frequently Asked Questions
Introduction If you have ever been part of a project in which your product was responsible for trading information between multiple systems, in multiple locations, between multiple recipients, you will appreciate the robust and feature-rich capabilities found within BizTalk Server. The Messaging Services of BizTalk Server provide administrators and developers alike the ability to configure and manage document exchange between different operating systems, different types of networks, and even different languages. How, you might ask? Microsoft has made a strong effort with their latest .NET servers and framework to support open architecture and to adhere to industry standards. BizTalk Server is no exception to this rule, and provides support for the latest XML-based technologies and operating on industry-aware and publicly available standards such SOAP and the BizTalk 2.0 Specification.
The chapter talks about the various components that constitute the Messaging Services of BizTalk Server 2000. We will discuss items such as specifications and build open the knowledge of channels, ports, etc. introduced in Chapter 3, "Testing the Installation" Upon completion of this chapter, you will have a solid foundation of the main components of BizTalk Server, and have a better understanding of the newconcepts that BizTalk Server 2000 brings to the .NET server family.
BizTalk Messaging Services
The BizTalk Server framework is fundamentally based on messages. In any business process that is automated by BizTalk Server, messages must be received from their source, processed, possibly transformed into another format, and then sent to their destination. While BizTalk Orchestration services are responsible for any processing of the messages that arrive, BizTalk Messaging Services manage both the process of moving messages from their source to their destination and the process of converting documents between different message formats appropriate for each transaction. BizTalk Messaging consists of several key objects that must be both understood and properly configured in order to correctly route messages. The main objects are document definitions, document specifications, organizations, channels, messaging ports, and distribution lists. Each of these objects depends on one or more other types of object, so it’s important to understand their purpose. A brief description of these objects is given here, and more detail is provided for each later in the chapter:
- Document definitions provide information on the type of document used by a channel.
- Document specifications describe the exact structure and layout of the document.
- Organizations represent trading partners involved in exchange of business documents.
- Channels are used to receive source documents from other organizations or other internal applications, and act as "gateways" into your BizTalk Server.
- Messaging ports are used to route documents from a channel to another organization or application.
- Distribution lists combine messaging ports into a single entity, allowing one message to be distributed to multiple ports.
Even from these short descriptions, you can see that these different objects rely heavily on one another, since BizTalk Messaging cannot perform its job without interactions between these different types of objects. While it might seem initially cumbersome to have so many different entities act in combination to provide these messaging services, it is this decoupling that truly gives BizTalk Server Messaging Services its flexibility and power. Business processes are rarely etched in stone and change frequently. This loosely coupled messaging services architecture allows you to build up your messaging system from smaller parts to accomplish your business needs. When some aspect of your business process changes, you can react by simply updating only those objects that require it (modifying a port to send an invoice to a different URL, for example), rather than redesigning your entire message transport process. The details of the interactions between the different types of messaging objects will become clearer in the sections to follow.
The BizTalk Messaging objects can all be created from the BizTalk Messaging Manager interface, which can be launched from the Start menu via your BizTalk installation. The interface is rather basic and mainly serves as a launching pad for various wizards and property pages from which you can configure your messaging objects. The Messaging Manager interface is shown in Figure 4.1.
Figure 4.1 The Messaging Manager Interface
A document definition describes a specific type of document that’s involved in a business process and any tracking or selection criteria that go along with it. It could describe a class of XML document that is received by your BizTalk Server from a business partner, or it could describe a flat-file text document that represents the output of an old legacy system in your enterprise. Document definitions reinforce the flexibility of BizTalk to understand vastly different types of documents. In order to examine how a document definition is created and what kinds of information it contains, let’s create one.
1. Choose File | New document definition from the Messaging Manager interface.
2. Name the new definition Invoice, make sure document specification is checked, and specify CommonInvoice.xml from the Microsoft subfolder in the WebDAV repository on your BizTalk Server machine.
3. The new document definition property sheet is shown in Figure 4.2.
4. Click OK to save this document definition.
Figure 4.2 Creating a New Document Definition
The most important items when creating a document definition are the definition name and the document specification. The name allows different messaging channels to reference this document definition. You should choose a name that is descriptive for this type of document. The document specification provides the structure of the XML document referenced by this document definition. The difference between a document definition and its document specification is initially confusing. However, it is important to remember that a document specification provides XML structure and content information in the form of an XDR schema, and the document definition acts as a pointer to that specification. The separation of these two concepts allows multiple document definitions to reference a single document specification, reducing redundancy in your system. Although it is not strictly required that you select a document specification for a document definition, there are few good reasons not to do so. When you select a document specification, you get the following benefits that you would not otherwise have:
- Validation of the XML document against a schema to reduce errors.
- The ability to use the Messaging Services mapping functionality to transform one document type into another.
- The ability to specify tracking information and selection criteria for the XML document.
Considerations for Tracking and Querying Document-Specific Data In addition to the schema validation benefits obtained by using a document specification within your document definition, you gain the ability to track specific fields in your documents to the BizTalk tracking database in SQL Server. As a messaging channel receives a document, it looks to a document definition to describe the format of the document. The document definition describes the location of key document fields, or the important data structures contained by your document. The fields you select here are known as global tracking fields, because these fields are tracked for all documents imported by all channels that reference this document definition. This feature is useful in cases where your business needs require you to have easy access to fields.
In order to better see how to accomplish this and what benefits it yields, we will now create a new document definition. This document definition will represent an invoice document that we plan to receive from a trading partner. Because the company management requires access to important information about the invoices that pass through our system, we will need to track certain pieces of information for each invoice document that arrives. This document definition will be named Tracking Invoice and reference the same "CommonInvoice.xml" document specification from before. Field tracking options are available in the Global Tracking tab of this property sheet, as seen in Figure 4.3.
Figure 4.3 Selecting Global Tracking Fields
On the Global Tracking tab, the tree view on the left is populated with schema information from the document specification referenced on the General tab. To select a particular field for tracking, perform the following steps: 1. Click the field you want to track in the tree view on the left (note that it must be a leaf node).
2. Click the button corresponding to the desired data type. (Note that you are not specifically required to use strict data typing, but if you plan to use this information in business calculations later, it is wise to do so.) The chosen fields appear on the right, denoted by their data type and an XPath expression specifying where to find the field in the source document. In this example, we have specified two fields from the document to track: the "Date" and "Number" fields from the "InvoiceHeader" record. After this document definition is saved and subsequently referenced from one or more channels, any documents that reference this document definition will have these fields logged in the SQL Server tracking database.
Channel configuration gives you the option of tracking documents at the field level. These settings override any settings specified in your referenced document definition and are used in place of the fields in your document definition.
Global tracking fields specified in the document definition are saved to the SQL Server "InterchangeDTA" database in the "dta_outdoc_details" table. This database can be queried directly by your applications in order to retrieve summaries or information about specific document transactions. Note that the database schema for this table restricts you to tracking only two fields for each of the real, integer, date, and string data types. You can, however, track additional fields using the "custom" data type, which stores your additional fields (name, XPath, and value) in an XML structure in the same database record in "dta_outdoc_details." Since retrieving those values from the XML structure is more computationally intensive than reading single values from a database, you should take care to use your strongly typed values for the fields that are most frequently accessed by your applications, leaving the "custom" structure for the less common information. If you do choose to use the custom structures for commonly accessed information, you will be responsible for parsing the custom XML structure information from the database to retrieve your values of interest and then inserting it into your downstream business logic. It is far easier for both the developer and the processor to use a single, strongly typed field from the database.
In addition to directly accessing the SQL Server database, you can view query-tracking data via the BizTalk Document Tracking Web interface. Unlike the several other Windows-based applications included in the BizTalk Server product, the Document Tracking application is accessed via a Web browser, as seen in Figure 4.4. Internet Explorer 4 or later is required to access the Web-based tracking application, since ActiveX controls are used to provide much of the functionality. This is unfortunate, since by doing things this way, Microsoft has achieved the worst of both worlds. First, due to the nature of HTML-based interfaces, Web applications are inherently feature-limited compared to their desktop counterparts. Second, by including ActiveX controls in the application, they have restricted access to users of Internet Explorer on the Windows platform. If Microsoft had gone fully either way—either a truly portable Web-based application or a full-featured Windows-based desktop client—their rationale would have been easier to understand.
Figure 4.4 The BizTalk Document Tracking Application
This document tracking page can be used to form virtual queries against the tracking database by selecting the date range of the interchange, source and/or destination organizations, and document specifications for those transactions of interest. The results can be viewed on another screen by clicking Query. The Query Results screen is shown in Figure 4.5. Figure 4.5 Document Tracking Query Results
Individual interchanges are the main row entries, while individual documents are shown by expanding the + icon on the left of the interchange rows. The data detail contains information about the document itself, including all tracked field data.
Tracking can also be enabled at the document level, in place of or in addition to tracking at the field level. Tracking at the document level is enabled from the BizTalk Server Administration application. Right-click on the BizTalk Server Group, and choose Properties. The Tracking tab allows you to control the logging at the document level, as shown in Figure 4.6. Figure 4.6 Specifying Document-Level Tracking Options for the Server
Document specifications describe the internal structure of your document instances. In the previous section, we learned that document definitions reference document specifications in order to provide document validation and mapping capabilities to BizTalk Messaging Services. When a channel initially receives a document, its document definition is queried for (among other things) its document specification. The document specification exists as an extended XDR file, and this file is used to validate the document instance arriving at the channel. XDR stands for "XML Data-Reduced" and is a method for describing the schema (or layout) of an XML document. Use the BizTalk Editor to create new specifications and edit existing document specifications. The editor makes it easy to graphically build and visualize complicated document specifications for various instance document formats, including XML, EDIFACT, X.12, and custom positional or delimited text file formats, although there are some tricks to generating specifications based on flat-file formats.
Here are a sample flat-file format representing a simple purchase order, and some tips to using the editor to model this format:
Joe Smith 123 Main Street Somewhere, CA 92121 123-456-7890
20 56789 Apple Pie 9.99
5 23232 Box of plastic forks 2.50
In the preceding code, the individual lines are delimited by carriage returns, while the fields within each line are tab-delimited. The steps to take to model this file in the BizTalk Editor are as follows:
1. Create an XML tree in your specification to look like Figure 4.7. Figure 4.7 Modeling a Flat-File Format in the BizTalk Editor
2. Select the FlatOrder (root) node in the tree view on the left and the Reference tab on the right, then change the Standard property to CUSTOM. Changing this property is required to parse and validate any non-XML documents.
3. Change the Default Record Delimiter value to CR (0xd).
4. Change the Default Field Delimiter value to TAB (0x9).
5. Next, select the Parse tab on the right pane, and change the Field Order property to Infix. This tells the validator to look for the delimiters between different records or fields, but not before the first field or after the last field.
6. Change the Skip Carriage Return property’s value to No.
7. Duplicate these property settings for the Buyer node and the Item node to explicitly set the validation properties for these nodes. 8. Finally, select the Item node and its Reference tab. Change the Maximum Occurrences value from 1 to *. This is necessary to allow multiple items to be ordered in one instance of a document.
The document specification is now complete, and an instance document can be validated from the Tools … Validate instance menu option. Selecting the simple order flat file (Figure 4.7), we see the internal XML representation of the file resulting from BizTalk reading this document instance and parsing and validating it against our newly created document specification. There is a substantial benefit to separating the document definition from its document specification. The document specification provides abstract information about the types and structures of document instances, while the document definition provides specific information about the document instance. One of the benefits lies in the ability to have multiple document definitions that reference the same document specification. This might be necessary if you sometimes need to enable global tracking fields, but not always. For example, the specification contains metadata information about the document standard (i.e., XML, EDIFACT, etc.), document type, and version, in addition to the document schema information inherent in an XDR schema. Furthermore, a simple built-in versioning mechanism is provided here, since the document specification is imported into the internal configuration database whenever you create a document definition. For example, you reference a version of a specification in a document definition and use that definition to execute your business rules. Several weeks later, you update the specification to accommodate a slightly different schema based on requirements from a new business partner, and then you create another document definition to reference the revised specification. Even when the underlying original specification file changes, your original document definition remains valid. This allows you to accommodate different versions of a document specification, each of them actively participating in your business practices at any given time.
Sometimes the term "BizTalk Independent Document Specification" is used in information describing BizTalk. This term is more accurately the "BizTalk Framework," since it describes the internal, SOAP-based formatting of BizTalk documents in transit. "Document specifications," as we’re discussing, are concerned with the body of the internal SOAP message, which represents our original business document.
Industry awareness and acceptance is growing constantly for XML-based transfer of business documents and for the BizTalk initiative in particular. At the time of this writing, over 500 unique schemas and specifications have been deposited in public registries such as those at www.biztalk.org and www.xml.org. These documents have been published publicly to encourage adoption of standards for data exchange within particular industries, and to facilitate communication between disparate organizations within those industries. For example, various specifications exist for human resources, transportation, and banking, and are freely available for organizations within that industry to use to enable communication between them. One of the greatest benefits of the BizTalk initiative is that the framework itself makes no demands on the types of information that can be transferred with it. Those choices are left to those best able to determine what’s best for a particular industry or organization: the industry leaders themselves. The BizTalk Framework only specifies the packaging of the business message, requiring nothing in particular from the message itself.
Source and Destination
Another powerful feature of BizTalk Server 2000 is the ability to convert one document specification into another. This functionality is provided by the BizTalk Mapper, accessible from the BizTalk Server group in the Programs folder in the Start menu. The BizTalk Mapper allows you to create a "mapping" between a source and destination document specification, thus enabling BizTalk Server to translate one format to another as required by your business. This mapping functionality is implemented internally as an XSLT style sheet, an XML-based way to facilitate transformations between different documents. Simple one-to-one transformations can be done with simple drag-and-drop operations. More complex operations (e.g., totaling the prices for all items in an invoice and applying the result to a single field in the destination format) can be modeled by using what are known as functoids by the BizTalk Mapper tool. A functoid encapsulates some bit of logic to aid in the transformation of your document formats. Functoids are represented in the resulting XSLT as VBScript functions. BizTalk Mapper ships with 65 different functoids, divided between several tabs on the Functoid palette, ranging in function from string manipulations to math functions to extracting information from a database. For any function that’s off the beaten path and is not covered by the other included functoids, there exists a "scripting" functoid that enables you to write your own custom function to perform your transformation logic. Figure 4.8 demonstrates a map of the simple order form that was used earlier to the "PaymentSpec.xml" specification that’s included with BizTalk Server. Figure 4.8 Using the BizTalk Mapper to Map One Document Specification to Another
Each mapped field is a simple one-to-one mapping, except for the highlighted functoid mapping. The functoid shown is the "cumulative sum" functoid, and in this case, it sums the prices for each of the items in the source document and applies that value to the Total field in the destination specification. Developing & Deploying…
BizTalk Mapper Pitfalls
To open a source and destination specification from within the BizTalk Mapper, go to File | New. You will then be prompted to select a source specification and a destination specification. If you first attempt to use File | Open to open a source and destination specification, you will be prompted with an unintuitive error message similar to the one in Figure 4.9, even though the file is a valid document specification. Figure 4.9 BizTalk Mapper Warning
The File | Open option is used for opening an existing Mapper output file, which internally contains the source specification, the destination specification, and the XSLT to transform between the two. The functionality of document mapping and its use within BizTalk Messaging is discussed in greater detail later in this chapter, specifically in the discussion on Channels.