Biztalk Explorer Extension

Wouldnt it be nice to click a DLL and see it's internal schemas and objects contained in a Biztalk DLL , IF you have used ildasm for .net asemblies you know what I am talking about . There is a feature in Biztalk for that , YOu ned to register that dll to get the view in your explorer


The extension is the BtsAsmExt.dll and is located in the \Program Files\Microsoft BizTalk Server 2004\Developer Tools subdirectory.
To enable this functionality
regsvr32 "C:\Program Files\Microsoft BizTalk Server 2004\Developer Tools\BtsAsmExt.dll"

You will see the BizTalk Server Assemblies icon in Windows Explorer alongside the drive letters.



When you click on the BizTalk Server Assemblies you will a list of all of the deployed assemblies. You can then click on each of the assemblies and you will see the BizTalk artifacts that are used in that specific assembly. These includes Orchestrations, Schemas, Maps and Pipelines. In addition, if you double click on an individual artifact, the Type Content Viewer windows pops up and will show you an XML representation.



This tool also includes additional functionality. It adds a BizTalk Server Search pane (found under the View ->Explorer Bar->BizTalk Server Search) which lets you search across the deployed assemblies for any of the BizTalk types. So, I could do a search for any schemas that are referenced by pipelines across all assemblies or Correlation Types referenced by Orchestrations found in assembly X.

Monitoring Biztalk performance

Biztalk applications have now become matured enough and are now getting scrutinized for performance. Often this scrutinizing of results in capturing some key performance counters and you will eventually chose a set of counters to measure and run your tests. But often on later analysis you realize that you missed capturing certain key counters . So what is a good subset to be captured. The counters below offers a good start as any

Biztalk Box General

\\BTSBOX\.NET CLR Exceptions(_Global_)\*
\\BTSBOX\.NET CLR Memory(_Global_)\# Bytes in all Heaps
\\BTSBOX\.NET CLR Memory(_Global_)\# GC Handles
\\BTSBOX\.NET CLR Memory(_Global_)\# Gen 0 Collections
\\BTSBOX\.NET CLR Memory(_Global_)\# Gen 1 Collections
\\BTSBOX\.NET CLR Memory(_Global_)\# Gen 2 Collections
\\BTSBOX\.NET CLR Memory(_Global_)\# Total committed Bytes
\\BTSBOX\.NET CLR Memory(_Global_)\# Total reserved Bytes
\\BTSBOX\.NET CLR Memory(_Global_)\% Time in GC
\\BTSBOX\.NET CLR Memory(_Global_)\Allocated Bytes/sec
\\BTSBOX\.NET CLR Memory(_Global_)\Gen 0 heap size
\\BTSBOX\.NET CLR Memory(_Global_)\Gen 1 heap size
\\BTSBOX\.NET CLR Memory(_Global_)\Gen 2 heap size
\\BTSBOX\.NET CLR Memory(_Global_)\Large Object Heap size
\\BTSBOX\ASP.NET Applications(_LM_w3svc_1_ROOT_BTARNApp)\Anonymous Requests/Sec
\\BTSBOX\ASP.NET Applications(_LM_w3svc_1_ROOT_BTARNApp)\Requests In Application Queue
\\BTSBOX\ASP.NET Applications(_LM_w3svc_1_ROOT_BTARNApp)\Requests Total
\\BTSBOX\ASP.NET Applications(_LM_w3svc_1_ROOT_BTARNApp)\Requests/Sec
\\BTSBOX\BizTalk:Messaging(*)\*

Memory

\\BTSBOX\Memory\Available MBytes
\\BTSBOX\Memory\Available MBytes
\\BTSBOX\Memory\Committed Bytes

DISK

\\BTSBOX\PhysicalDisk(_Total)\% Disk Time
\\BTSBOX\PhysicalDisk(_Total)\% Idle Time

PROCESS
\\BTSBOX\Process(BTSNTSvc#1)\Handle Count
\\BTSBOX\Process(BTSNTSvc#1)\ID Process
\\BTSBOX\Process(BTSNTSvc#1)\Private Bytes
\\BTSBOX\Process(BTSNTSvc#1)\Thread Count
\\BTSBOX\Process(BTSNTSvc#2)\Handle Count
\\BTSBOX\Process(BTSNTSvc#2)\ID Process
\\BTSBOX\Process(BTSNTSvc#2)\Private Bytes
\\BTSBOX\Process(BTSNTSvc#2)\Thread Count
\\BTSBOX\Process(BTSNTSvc#3)\Handle Count
\\BTSBOX\Process(BTSNTSvc#3)\ID Process
\\BTSBOX\Process(BTSNTSvc#3)\Private Bytes
\\BTSBOX\Process(BTSNTSvc#3)\Thread Count
\\BTSBOX\Process(BTSNTSvc#4)\ID Process
\\BTSBOX\Process(BTSNTSvc#4)\Private Bytes
\\BTSBOX\Process(BTSNTSvc#5)\ID Process
\\BTSBOX\Process(BTSNTSvc#5)\Private Bytes
\\BTSBOX\Process(BTSNTSvc)\% Processor Time
\\BTSBOX\Process(BTSNTSvc)\Handle Count
\\BTSBOX\Process(BTSNTSvc)\Handle Count
\\BTSBOX\Process(BTSNTSvc)\ID Process
\\BTSBOX\Process(BTSNTSvc)\ID Process
\\BTSBOX\Process(BTSNTSvc)\Private Bytes
\\BTSBOX\Process(BTSNTSvc)\Private Bytes
\\BTSBOX\Process(BTSNTSvc)\Thread Count
\\BTSBOX\Process(BTSNTSvc)\Virtual Bytes
\\BTSBOX\Process(w3wp#1)\Private Bytes
\\BTSBOX\Process(w3wp)\ID Process
\\BTSBOX\Process(w3wp)\Private Bytes

\\BTSBOX\Processor(_Total)\% Processor Time
\\SQLBOX\PhysicalDisk(_Total)\% Disk Time
\\SQLBOX\PhysicalDisk(_Total)\% Idle Time
\\SQLBOX\PhysicalDisk(2 D:)\% Disk Time
\\SQLBOX\PhysicalDisk(2 D:)\% Idle Time
\\SQLBOX\PhysicalDisk(2 D:)\Avg. Disk Queue Length
\\SQLBOX\PhysicalDisk(2 D:)\Disk Read Bytes/sec
\\SQLBOX\PhysicalDisk(2 D:)\Disk Write Bytes/sec
\\SQLBOX\Processor(_Total)\% Processor Time


\\SQLBOX\SQLServer:General Statistics\User Connections
\\SQLBOX\SQLServer:Locks(_Total)\Average Wait Time (ms)
\\SQLBOX\SQLServer:Locks(_Total)\Lock Requests/sec
\\SQLBOX\SQLServer:Locks(_Total)\Lock Timeouts/sec
\\SQLBOX\SQLServer:Locks(_Total)\Lock Wait Time (ms)
\\SQLBOX\SQLServer:Locks(_Total)\Lock Waits/sec
\\SQLBOX\SQLServer:Locks(_Total)\Number of Deadlocks/sec



I am sure the list is quite extensive but will give you a good start on what to monitor or not monitor

Debatching and Batching in Biztalk 2004

One of the fairly common issue that most Biztalk designers face is the issue of Debatching and then Batching . Having faced a similar situation myself I must say that there is no best solution for this problme that I have so far come up with . Though there are different solutions that you could employ
Just to explain the problem first you have a set of files coming in say a folder ( transport does not matter ) and then each file consists fo a set of transcations . you need to process them indicvidually and then send them out as one file

The first instict is to use a Biztalk XML or flatfile dissasembler and then send the message to Messagebox ,and then process them independently. THe problem now comes of batching them back together.

Solution 1

Now if you have one file comeing in during an EOD process , doing ths split and then writing it to a file with append mode TRUE would do the trick ost of the times

But this is not a clean solution If you have more than one file coming in almost simultaneously this solution will not hold water as once Biztalk does thwe split it is not possile to identify the transactions to it's original source

Solution 2
A solution to this is to promote a context property in the pipeline and then in the orchestration decide which instance it belongs to create a unique guid based on the the context and then set the SourceFileName property and then use the append method
This is slightly more complicated as writing pipeline components is not the easiest things to do and you have to take care of your compoent to be streamig . Why it needs to be streamsing Read This

Pros: This is real Fast and great for CBR uses streaming XPATH

Cons: All or nothing. If something fails in the pipeline or map the entire message will be lost.


Solution 3

Another solution is not to split the message using the Docuement /Envelope schema and do this using XPATH . This solution makes us of XPATH to count and then uses XPATH to loop around message elements .To see a sample and a write up on this implementation try this


Pros: Excellent flexibility inside the Orchestration. Great control of the processing of your document! This process is sequential and ordered by default. Ability to loop over anything you can XPath and can easily (this is always a relative term) build mini-batches if needed. This is something Receive Pipelines are not able to do without extensive custom code.

Cons: Performance degrades quickly as message size increases. Downright slow on large messages and a resource hog. In some cases, the sequential and ordered processing of this type of debatching may be limiting to the process.



Solution 4
Ths issue with solution 3 above is performence when it comes to large messages ,Read the story of Biztalk and large files from Lee

The solution that we tried for our issue was to use a two stage orchestration. The first orchestration contains a map that promotes a unique value to an additional Node (effectively creating an intermediate schema) for every instance of the file.
This is then fed to a dissasembler but now each file retains information about its parent using the guid and then write out the guid value as file name with append mode on .

Solution 5

Send your output file out tho SQL and promote business fields that help you identify the batch use a SQL Receive port at periodic intervals to aggregate the result using some intelligent querying



So what is the best solution , It depends , as I said before there is no clean solution or a magic bullet .Depends really on the app that you are trying to build
but a thumb rule is for small messages you can get by with XPATH and go for pipeline based splitting for large messages

I welcome comments and opinions
Ciao

Accessing Standard Context properties in Biztalk pipelines like FileName

If you wnated to access context properties in pipelines and have spent needless time figuring out the correct schema for each property here is a comprehensive list

This content is taken as is from Stephen Kaufman's WebLog

There are a number of 'built in' promoted properties that provide important information. There are different items for the receive and send pipelines.



The 14 receive pipeline items are (including the associated namespace):

1. ReceivedFileName (http://schemas.microsoft.com/BizTalk/2003/file-properties)

2. InboundTransportLocation (http://schemas.microsoft.com/BizTalk/2003/system-properties)

3. InterchangeID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

4. ReceivePortID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

5. ReceivePortName (http://schemas.microsoft.com/BizTalk/2003/system-properties)

6. WasSolicitResponse (http://schemas.microsoft.com/BizTalk/2003/system-properties)

7. AuthenticationRequiredOnReceivePort (http://schemas.microsoft.com/BizTalk/2003/system-properties)

8. InboundTransportType (http://schemas.microsoft.com/BizTalk/2003/system-properties)

9. LRPMsgBodyTracking (http://schemas.microsoft.com/BizTalk/2003/system-properties)

10. MessageExchangePattern (http://schemas.microsoft.com/BizTalk/2003/system-properties)

11. PortName (http://schemas.microsoft.com/BizTalk/2003/messagetracking-properties)

12. ReceivePipelineID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

13. MessageType (http://schemas.microsoft.com/BizTalk/2003/system-properties)

14. SchemaStrongName (http://schemas.microsoft.com/BizTalk/2003/system-properties)



While the 31 send pipeline items are

1. CopyMode (http://schemas.microsoft.com/BizTalk/2003/file-properties)

2. LTPMsgBodyTracking (http://schemas.microsoft.com/BizTalk/2003/system-properties)

3. ReceivedFileName (http://schemas.microsoft.com/BizTalk/2003/file-properties)

4. SPID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

5. ActualRetryCount (http://schemas.microsoft.com/BizTalk/2003/system-properties)

6. FileName (http://schemas.microsoft.com/BizTalk/2003/file-properties)

7. PartyName (http://schemas.microsoft.com/BizTalk/2003/messagetracking-properties)

8. ReceivePortName (http://schemas.microsoft.com/BizTalk/2003/system-properties)

9. WasSolicitResponse (http://schemas.microsoft.com/BizTalk/2003/system-properties)

10. AllowCacheOnWrite (http://schemas.microsoft.com/BizTalk/2003/file-properties)

11. RetryInterval (http://schemas.microsoft.com/BizTalk/2003/system-properties)

12. OutboundTransportCLSID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

13. SPName (http://schemas.microsoft.com/BizTalk/2003/system-properties)

14. InboundTransportLocation (http://schemas.microsoft.com/BizTalk/2003/system-properties)

15. InterchangeID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

16. ReceivePortID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

17. SPTransportID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

18. TransmitPipelineID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

19. AuthenticationRequiredOnReceivePort (http://schemas.microsoft.com/BizTalk/2003/system-properties)

20. InboundTransportType (http://schemas.microsoft.com/BizTalk/2003/system-properties)

21. LRPMsgBodyTracking (http://schemas.microsoft.com/BizTalk/2003/system-properties)

22. MessageExchangePattern (http://schemas.microsoft.com/BizTalk/2003/system-properties)

23. OutboundTransportLocation (http://schemas.microsoft.com/BizTalk/2003/system-properties)

24. PortName (http://schemas.microsoft.com/BizTalk/2003/messagetracking-properties)

25. ReceivePipelineID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

26. SourcePartyID (http://schemas.microsoft.com/BizTalk/2003/system-properties)

27. MessageType (http://schemas.microsoft.com/BizTalk/2003/system-properties)

28. OutboundTransportType (http://schemas.microsoft.com/BizTalk/2003/system-properties)

29. PartNames (http://schemas.microsoft.com/BizTalk/2003/messageagent-properties)

30. RetryCount (http://schemas.microsoft.com/BizTalk/2003/system-properties)

31. SchemaStrongName (http://schemas.microsoft.com/BizTalk/2003/system-properties)





To get access to these properties you need to get access to the context object. The following method shell shows how we can do that.



public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)

{

IBaseMessageContext context = pInMsg.Context;
.
.
.
}


Now that we have a method to gain access to the context, what if we want to get access to the internal promoted properties? We can iterate through all of the default promoted properties by using the context.ReadAt method (which produces the list of the items above). This method takes an index and returns, through 2 out parameters, the name and namespace of the properties.

We can also use the context.Read method to access the value of each of these promoted properties. The Read method returns an object type containing the value of the promoted property when passing in the name and namespace of the property.


So, to retrieve the source file name we would use this following line of code:



string srcFileName = context.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties").ToString();

Biztalk Cool stuff

Some of the good Biztalk cool Stuff available out there

BizTalk Server 2004 "deploy with Nant" template
Integrating BizTalk Server with Nant

BizTalk Server 2004 Pipeline Component Wizard 
VS.NET addin for quickly building pipeline components

BizTalk AutoDeploy Utility
VS.NET addin for quickly (re)deploying BizTalk Server solutions.  No need for Ant

BizTalk Server 2004 Performance Tool
Tool that allows for BizTalk Server tuning.

BizTalk Server 2004 Transactional .NET Adapter
adapter that allows for transactional message transmission to any .NET component.

BizTalk Server 2004 Tracking Playground
Tracking demo application showing how to make use of the tracking db views and WMI.

BizTalk Server 2004 Management Tool
This is good...

BizTalk Explorer
Doing what VS.NET allows you to do, without VS.NET :-)

BizTalk Server Service Windows
Allows for more flexible service windows.

BizTalk Server POP3 Adapter
Inbound email messages in BizTalk Server

BizTalk Server TIBCO Adapter
Rudimentary sample

BizTalk Server SQL BLOB Adapter
Readingand writing message to binary SQL columns.

Biztalk Accelerator for Rosettanet Releases

Biztalk 2004 Accelerator for Rosettanet released.I have been involved with the accelerator for some time in 2003 so I feel very good personally about this. It was really a great experience working at microsoft although I later moved on from seatlle

I think this is the last of the Biztalk 2004 accelerators from Microsoft

The key features
Certified RosettaNet Ready™ RNIF 2.0. Full compliance with the RosettaNet Implementation Framework (RNIF) versions 1.1 and 2.
Rapid development. Out-of-the-box implementation of all existing Partner Interface Processes (PIPs)
Support for CIDX. Support for the Chemical Industry Data Exchange (CIDX) Chem eStandards version 2.0.2
Built-in management console. Built-in management tools for managing all RosettaNet-based trading partners and existing processes from a single console
Orchestration of RosettaNet processes. BizTalk Orchestration Designer makes it easy for users to visually modify private processes and orchestrations that integrate with public RosettaNet business processes.
Definition and documentation of schema. BizTalk Editor enables users to easily define and edit document schema for any structured document type, including all RosettaNet messages.
Flexibility for custom RosettaNet solutions. The software development kit and comprehensive documentation help users develop tailored RosettaNet solutions.

Finding Orchestration ID

Recently I had to do some reporting for which I was not able to use BAM . So i had to write some custom code to track data from promoted properties by populating a seperate DB . One of the interesting problem was to also report the stat of the orchestration.

This is possible in BizTalk as each orchestration
has a unique Instance ID you can take that out in an expression shape

OrchInstGuid = System.Convert.ToString(Microsoft.XLANGs.Core.Service.RootService.InstanceId)

We can then check for status of the orchestrations using this instance ID

The QuickStart Guide to Installing & Configuring BizTalk Server 2004 for Windows Server 2003

There was lot of times when I wanted a good documentation for installign Biztalk and I feel the Installation guide but every other minute you eil be downloading some service pack or the other . this is one concrete place where everything is listedThe QuickStart Guide to Installing & Configuring BizTalk Server 2004 for Windows Server 2003

BizTalk Feed

I like this feed linkBizTalk FeedThis has some great tips and great links

Biztalk 2004 pipeline components

If you are thinking of writing a Custom BTS 2004 pipeline component . This would be my number on recommended article written by Christof

It goes ito how to promote properties from inside your pipeline ;It discusses why your component needs to be streamed and some of those nifty things; that you tend to forget and even if you have written one before I still think this is a good read.

Biztalk 2004 Partner Competency Exam Training Guide

Biztalk 2004 Partner Competency Exam Training Guide was released .
Grab it here

So my next plan should be to see if can put myself through the certification engine. I have never done this certifiaction thing . IT certification value is largely at Debate mostly due to issue with dumps. How do you feel about certifications .
I would like to hear your opinion
abhilashms2--AT--GMAIL-NOSP@M-DOT-COM

Mini Visual Studio

Ever waited like a minute or so for Visual studio to load up and all you wanted to do was to test out some small console code to prove a concept, I am sure most of us have . and If you have you are gonna love this Snippet compiler

That reminds me while I was working in Redmond I have seen a stripped down verson of the IDE called RASCAL . I am not able to find any link to that now

Explaining Biztalk to your Grandma

I have heard this was apopular question at MSFT interviews "How do you sell excel to your grandmother ?" well No this blog is not about that but then this blog is about how to explain Biztalk to a Business Analyst.One who is not so comfortable with all the geeky terms that we are so used to . I recently had to do one of these myself and I had to kind of get creative with ti . So thought I would just blog about that and see how it goes
Let us first take a look at the architecture of Biztalk from The MSFT Site


To put things in perspective let us consider this Huge Shipping Port were there is lots of ships , barges coming and going . Now I am trying to draw a parallel here

The Port basically deals in containers to carry Goods and Biztalk in XML to carry data

Containers or goods could be brought in using different modes of transport
Big cargo ships , small passenger ships , By Road at times , small Boats, It does not matter how the cargo is delivered into the port , it does not change the later part of processing of the cargo . Same is true for Biztalk which has a number of transport mechanisms like FTP , HTTP, SQL , SMTP , as in the port it does not matter how the data came in what matters is their routing information or who tgat data needs to go .

For identifying the destination of the Goods , This may go through Customs , Transit Inspection and maybe finally broken down to individual goods at times and goes to a cleared warehouse from where they are dispatched based on destination . The same thing happens in the recieve pipeline stage of Biztalk . The file goes through Dissasembly , Party resolution to the SQL server Messagebox . By this time incoming XMLSchema#RootNode is promoted based on which the message is picked up Orchestrations which do some processing on them and return them back to message box .
Consider this as importing some raw material like wood and doing some value add ( make them chairs for instance )
and then on the send side pretty much the reverse happens they are prepacked assembled and encoded and send out using approporiate transports available

What I described above is a publish-subscribe model . Now you can have messages in Biztalk follow a CBR ( Content Based Routing MOdel ) . In that case tha anology is a a conatiner which reaches a port for forwarding . or a transit port , here there is no signinficant value add on the goods but you can change some fringe level properties or change the way the data is represented

Epilogue
There are lot of flaws in this theory but for a person who is wuite new to Biztalk this is as good as a start as any.


PS: The views expressed in these essays are those of the author, and in no way represent, nor are they endorsed by Microsoft oranyone else, as far as I know.

Boy you never know these days , this legal-mumbo jumbo

HAT and Biztalk Debugging

Biztalk Server 2004 follows a Hub-Bus hybrid model and why it is a better model is because it takes the best of each of these individual styles namely Scaling out , throughput and golden copy of data( Centralized data store ) . Every message gets tored in the messagebox . This is how Biztalk achieves Statelessness. This is one of the reason why the Debugging tool ( HAT) is like the way it is .
Although I have heard lot of people complaining about HAT and comparing them to Visual studio debugging support which are excellent many dont realise the nature of Biztalk . Unlike traditional debugging enterprise Stateless application like Biztalk being stateless in nature writes out events to a Database . and a tool like Kat is just a reporting screen of those events . ( You can do live debuggging -but only if you explicitly set it to)

Another way to get to see errors in Biztalk is to run Debugview from
www.sysinternals.com . Now you can write out your Debug messages using System.Debug Namespace

Biztalk 2004

It has been sometime since I started working with Biztalk and I have been thinking about blogging for quite sometime well quite a long time actually . I am not new to blogging. But I kind of thought of separating the tech stuff apart, I just wanted to get started So there.. That's it my first Post