Biztalk Server Best Practices Analyzer

I had heard about a new Biztalk 2006 Best Practices Analyzer tool . This was supposed to scan your biztalk implementation and point out irregularities.
This could be stuff like your Biztalk purgejobs are not running .This could seriously slow you down after some time. Or your Master secret server is not backed up .

There is some good posting and screenshots in Richard's Blog.

Failed to Create SQL Login for Biztalk Administrators Group

Failed to Create SQL Login for Biztalk Administrators Group on Database Server <> for Windows NT user or Group "DOMAIN\BTS2004 Biztalk Administrator"

You get the above message and you are already logged on as the user who has full permission to the SQL Server group . We got this error when we were configuring a new server in our production environement . I had faced this problem before but I had forgotten the solution . So I blog it here so that any lost souls get the benefit . This is a know problem when configuring Biztalk in a multiple computer configuration . You need to enable network client access DTC before you start configuration or your configuration WILL FAIL !!

This is documented in MSDN Biztalk trouble shooting section .

Multiple computer configuration using Windows Server 2003 or Windows XP
Problem: If you attempt to run the Configuration Wizard on Windows Server 2003, or Windows XP, and point to a remote SQL server, the configuration will fail.

Resolution: You must enable network clients DTC access before configuring BizTalk Server. This is required for the distributed transactions performed by BizTalk Server components to work.

Important The following procedure must be run on each of the computers in the multiple computer configuration.

To turn on DTC Access

From the Start menu, double-click Control Panel, double-click Administrative Tools, and then double-click Component Services.
Expand Component Services, and then expand Computers.
Right-click the computer you need to set, and from the context menu select Properties.
In the Properties window, select the MSDTC tab.
In the Transaction Configuration area, click Security Configuration.
In the Security Settings area, select the Network DTC Access check box.
In the Security Settings area, check the Network Clients check box.
Click OK on each open dialog box.
Restart your computer to enable the remote DTC option.
Warning This configuration could take up to 15 minutes.


Hope that gets you back on track .. Leave me a comment in any case

Biztalk 2006 hits RTM

Biztalk 2006 RTM released !!! .By the way RTM stands for Ready To Market I guessd this is one day earlier than announced .Very exceited at this product releasing in this time frame. Its out there for those of you who have MSDN subscribtion , on MSDN Downloads.

Biztalk Property Schema issue when Deploying

We recently had a problem with a Biztalk property schema deployment .

When we tried to deploy the schema , We came across this error and had to spend some time trying to debug this one.

The error you get would be a variation of the one below

Property "ns0:Your_Promoted_Property" (msgType="YourSchema#RootNode") not found in Configuration database.

Quick Solution

Delete the Node which it is complaining from the property schema and recreate it again

Reasoning

What was odd on our case was this same DLL was deploying in vertain machine but not on some other ones.

After aome amount of gogling I tried to think about the error for some time and this was my line of reasoning . The deployment was complaining about a promoted property not available in the configuration database . This was a bit weird because the property that Biztalk was complaining about was contained in the same Assembly.

But once I realized that somewhere in the deployment process the properties that were being promoted were first stored into SQL tables for eventual usage in CBR scenario . I realized that this insertion would be failing . The only reason this could be failing is if the variable already existed.

Since Biztalk being a Globalized product I figured the names would not be used as it is but rather some unique value a Identity or GUID . So I went ahead and changed the guid value of all the properties that it was complaining , I had edited the XML for this values . (You can delete and recreate the Node if you are not comfortable ) .

Then I rebuilt and it deployed like a charm !

PS: What had happened is one of our Dev's had copied the property schema from another project and changed the namespace and the field name of certain fileds.

This worked on certain machines without problems because these devemachines did not have the solution from which the property schema was copied deployed on them . Wheras we had this prior solution deployed on our servers.

So I guess Copy-Paste fanatics take care. !

Biztalk Podcasts

Listen to Biztalk Server Podcasts from Scott where he talks about Enterprise Service Bus and Windows Workflow Foundation .

Scott answers lot of intersting question here .. Like how did he become product manager of Latinum (Biztalk 2000).

While on code names these are the list so far..

Latinum --- BizTalk 2000
Bizet --- BizTalk 2002
Voyager --- BizTalk 2004
Pathfinder --- Biztalk 2006
Avalon --- Windows WorkFlow Foundation(WF)
??? --- Biztalk 2008


Now that Biztalk Server 2006 is out there and after having seen the demo of Windows Workflow Foundation (WF ) at PDC lot of us are asking what will be future of Biztalk Server 2008 ? .

Scott speaks out that the design surface of workflow for WF and BTS was built by the same team and the similarities end there Biztalk would still be needed to solve problmes of EAI, BPM ,B2B, Adaptor management .
Scott also mentions that the workflow for Biztalk 2008 would most probanly share the workflow piece with WF and there would be a upgrade to current customers. This also make me believer the news I heard about the Biztalk and Indigo Team got merged into one group .

Listen to the Biztalk Server Podcasts... here

PublishMessageException - A batch item failed persistence

I recently had to debug one of the errors for a buddy of mine and it take me some time to fix it ,It was quite intersting and i thought was worth noting down

You get the Error below


--------------------------------------------------------
Event Type: Error
Event Source: XLANG/s
Event Category: None
Event ID: 10034

Description:
Uncaught exception terminated service Biztalk.Reusable.FileMover.FileMoveOrch(edf0c1f0-c242-60c9-5b56-41ab42b9ee5d), instance e756bd88-5966-48f9-be40-8a55451d2bf4

Exception occurred when persisting state to the database.

Exception type: PersistenceException
Source: Microsoft.XLANGs.BizTalk.Engine
Target Site: Void Commit()
Help Link:
Additional error information:

A batch item failed persistence Item-ID f64a095a-eed6-451c-86ee-9aac941b935b OperationType MAIO_CommitBatch Status -1061151998 ErrorInfo The published message could not be routed because no subscribers were found. Please use the Health and Activity Monitoring (HAT) tool to diagnose this situation. .

--------------------------------------------------------

There are a couple of things you should check
1.) Is the Orchestration and all corresponding objects send ports started
2.) Is Biztalk 2004 Service Pack Installed
3.) Is your Orchestration Calling another Orchestration whihc is not started
4.) Are you using Dynamic Send Ports , If yes are you setting the Address location
Microsoft.XLANGs.BaseTypes.Address properly


In the problem I was debugging, the value of this variable was set from an external call to a .net function and this value was not properly intitalized.

Fixing the function and regacing solved the issue.

Biztalk Password Resets

Well we have our enterprise password Resets that happen every 45 days ; This causes Havoc to my local biztlak server installation.

THIS information Applies if you have NOT backed up your master secret
( Thanks Patrick for pointing out this ) .
if you have in C:\Program Files\Common Files\Enterprise Single Sign-On
>ssoconfig.exe -restoresecret YouBTSSSOBackupfile


So one day you reset that password and suddenly you discover that your biztalk service is not running.

First - Dont Panic !

ok Now you are not panicking :-)
so these are the places you need to reset

Goto ->Start->Run-> Services.msc
and update the password for the following

BizTalk Base EDI service
BiztalkServer Application
Enterprise Single Sign-On Service
MSSQLSERVER
Rule Engine Update Service
SQLSERVERAGENT

Next Open Biztalk Administrator and change passwords for

BizTalk Host Instance Account
BizTalk Isolated Host Instance Account ( In case you set it up same which is not the recommendation btw )

Next Open IIS Manager
Goto ->Start->Run-> inetmgr.msc ->Application pools->Properties ->Identities
and change the password for all the onee below ( You might not have some based on your configuration )

HwsMessagesAppPool
HwsWSAppPool
StsAdminAppPool
STSWebServiceAppPool
TpmWSAppPool (If you are using BAS)
WSSAppPool (Sharepoint app Pool )
BAMAppPool


Whew !! That should get you back on track .
In case it did not leave me a comment or Ping me !

Loading Dynamic Maps in Biztalk Orchestrations

I had a situation in which I had to map into a single schema when messages are send by different parties . Normally this would be done at the recieve port level and document is standardized , but one drawback of this approach is exception handling which is a pain in these situations . It would be great if we can dynamically load a map inside the orchestration and then normalize and we can have or exception handling code written there. I came across and excellent post from Steven Thomas on the same topic .


Inside the expression shape /Message Assignment Shape



objMapType = System.Type.GetType("Map Name ,( Full strong Name of Assembly)");


construct Output_Xml

{
transform (Output_Xml) = objMapType (Input_Xml);
}

Error while calling Orchestration Exposed as Webservice

If you are trying to call the Orchestration that are exposed as webservice , You will come acros this error below . This usually happens in Dev machines where Biztalk as well as IIS is on the same machine.



The Messaging Engine failed to register the adapter for "SOAP" for the receive location "/VirtDir/App.asmx". Please verify that the receive location is valid, and that the isolated adapter runs under an account that has access to the BizTalk databases.

An attempt to connect to "BizTalkMgmtDb" SQL Server database on server "Server" failed with error: "Cannot open database requested in login 'BizTalkMgmtDb'. Login fails.".

The "SOAP" adapter is suspending an outbound message going to destination URL


1 ) This issue occurs because the App pool identity that is used by the application pool for the exposed Web service is not added as to the BizTalk Isolated Host Users group. This is needed because the message needs to get properly authenticated to get submitted.

2 ) Also this account needs permission to the %systemroot%\Temp folder on the server that exposes the Web service .This is needed as the web service (.asmx) files are JIT compiled to create DLLs in the %systemroot%\Temp folder.

Giving these correct access and permissions should solve the issue .

BTS 2006 MSI Issues

I have built a Biztalk 2006 Project and I added a Setup project and added the output of the Biztalk project, but when I try to import the MSI that I created using the Biztalk administration console. I get an error saying this is not a valid Biztalk MSI ?

Has anybody encountered this error ?

PS : But if I export a already deployed application as an MSI , I am able to import it into anothe BTS2006 VPC machine. So does Biztalk not support the Windows MSI Setup route ?

Calling receive Pipelines from Inside orchestrations BTS2006

I had some Aggregator Orchestration which were running into Zombie situations ,. So when I got the biztalk 2006 beta version I was anxious to try out the new functionality of being able to call your receive pipeline from inside your Orchestration .

You can enumerate through the EnvelopeMessage passed into the recieve pipeline just like you would enumerate in an ICollection or Recordset using .MoveNext()

This is how my built Orchestration looked like







I will briefly try to capture the steps that I took

1.) Create a Document Schema SingleOrder.xsd

2.) Create an Envelope Schema TotalOrder.xsd

3.) Create a receive Pipeline SplitOrder.btp

4.) Add the XML Dissasembler and select the Document and envelope Schemas as above.

5.) Create your Recieve Port and set the document type to your Envelope Schema , This Port now has XMLPassthru transmit , I am calling this message IncomingFullMessage

6.) Now we enter Splitting scope ( Pipelines can only be invoked from Atomic scopes, Also it does not support Recoverable interchange processing when called this way )

7.) Add Microsoft.XLANGs.Pipeline as Reference and then ;declare a variable for the pipeline in my case
OrderSplitter AS "Microsoft.XLANGs.Pipeline.ReceivePipelineOutputMessages"

8) In the expression Shape Call Order Splitter ,

OrderSplitter = Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(SplitOrderReceivePipeline),IncomingFullMessage);


9) In the loop Get_Next-Order Shape iterate through each message using

OrderSplitter.MoveNext()

10.) Create an Orchestration variable "SingleOrder" of type Document Schema

11.) Create the Single Message you want in the message assignment Shape and set

SingleOrder = new System.Xml.XmlDocument();
OrderSplitter.GetCurrent(SingleOrder);

12) Now you have the single Message that you want to process. You can send it to whatever process you want , In this example I am just sending the message to a Output folder.

Biztalk Server 2006 Review - Installation

I started this install on a Virtual PC and I was a bit skeptic if I would be able to finish installation that day , Configuring our development server last time required almost 3 days and another instance I remember configuring a machine with two SQL servers with Analysis server on a different machine almost took a week and a call to MS help desk for some gotcha's .So I was a bit skeptic





One of the first things that you will notice is that there is a set of ways to download all the pre-requisites. Often times in Biztalk Server 2004 , You will have to go and hunt lot of required files like SQL XML , MSXML 4 and SP1 and office Web Components Update and ... Oh You know what I mean . Now the cool thing in BTS 2006 is that you have an one click option of asking Biztalk to download whatever it wants directly off the web or download ONE Cab file containing everything in case your machine is not on the internet or in my case like a VPC




The next thing that one notices is that the whole installation is not one big piece of ALL or NOTHING , wherein some small change or issue would roll back the whole process and you need to go now and manually delete the BizTalk databases at times and that often leads the beginner Biztalk user to give up in frustration .. Also before installation it checks for pre-requisites and tell you what is available and not . Also it asks for one user credentials only if you have a single box scenario. rather than typing the same user name a zillion times




It also shows a smoother progress indication and unlike BTS 2004 , It shows you what is not installed and lets you reconfigure that piece alone later



Overall I must say I am impressed with the setup . The biztalk Server Explorer is completely revamped and
now there is a whole concept of artifacts grouped by applications ( Duh !!)
, our administrator's are already looking forward to Biztalk 200^

TO "UNDEPLOY" OR "NOT TO UNDEPLOY"

If you have been working with Biztalk for a considerable amount of time like me
you would have gone through the pains of the Undeploy Regac - Deploy loop , Sometime when you are making changes to Schema , it is quite unavoidable but at lot of time you really don't need to , Just re-gacing your dll and restarting the Biztalk service will do the trick . I have not seen a comprehensive documentation for the same . So the list below is mostly from my own experiences

Schemas
=======

Any changes to Schemas like Namespace , Root Node Name , Promoted properties require a UNDEPLOY - DEPLOY Cycle.

Changing a node which is not a promoted one and not a root node , you can get by with just a GAC and restart

The reason for the same is Your Schema#rootnode is used in subscription evaluation , A changes to this without deploy undeploy will give you an subscription not found error


Maps
====
Changing mapping or Functoids in maps DO NOT require a UNDEPLOY - DEPLOY unless the Schema itself has changed ( See schema Above )


Orchestrations
==============
Changes in Orchestration DO NOT require a UNDEPLOY - DEPLOY Cycle unless Send/receive Port shapes are changed

This is mostly because Schemas would change or you add a new port based on a new schema which needs to get updated in the subscription viewer and this requires undeploy - deploy

Pipelines
=========
Changes in Pipeline DO Require a UNDEPLOY - DEPLOY


Assembly
========

External Assembly change DO NOT require a UNDEPLOY - DEPLOY Cycle unless you change the version number

In VB assembly info if the assembly version is set to 1.1.* it will increment the assembly number for every small change. Though it is good from a certain point of view , it can quickly become a pain if you just want to make a little minor change .
So I suggest you control the version number manually , So you can just re-gac the assembly

Please note Gac-ing the change and restart of the BTSservice is needed for all steps

Last but not the least , you should learn ,try and use BTSNANT tool , It is well worth the effort you spent

Biztalk FlatFile Disassembler Processing behavior

Recently I had a scenario which involves processing a large amount of flat files; each of whihc went througha Flatfile Dissasembler before the business processing on that started. Here I record some points of interest of FFDASM behaviour. When FlatFile Disassembler is fed with a large amount of files biztalk waits for all files to unpack before it starts processing.

Our test environment had two boxes each both of which configured under the same host and having identical machine config with SQL on a seperate box was used for the test
For our Scenario we had little over 500 files being dropped simultaneously .The file batch size was set at 20 the default.. It was noticed that after picking up the first 20 and initiating disassembling this batch before the next set is picked up , it goes on picks up all the files.( See graph)

Essentially Biztalk seemed to wait for all the messages to get picked up and then flat file disassembler unpacks all of the flat files from the interchange before it starts processing. This is evident from the apparent lack of activity form the time the last file was picked up and when the first set of files started coming out ( See Graph) . I had seen this in one of prior instance and I recently ran some test and got similar results which made me write this post.



Biztalk Flat File Dissasembler Behaviour  

Path Finder , Yukon and Whidbey

After a long wait and amongst lot of confusion on Whidbey not releasing in 2005 , Now it seems all the three VisualStudio 2005 , SQL Server 2005 and Biztalk Server 2006 is all going to come out in the Week of November 7. This was announced at TechED 2005

Also the Community Tech Preview (CTP) editions are out for BTS 2006.

Biztalk and File Masks Bug /Feature ?

Recently I was debugging a Biztalk solution with a friend of mine Rajesh ,and we noticed a curious Masking issue with BizTalk

Suppose you give a mask

*.XML

Which of these files are picked up ?

A) Orders.xml
B) Orders.XMLTEST
C) Orders.XML.BAK
D) Orders.TXT
E) ORDERS.XMLINFO
F) ORDERS.XML_BACKUP

I am sure your answer is only A , but in reality Biztalk picks up all files except C and D for a mask *.xml

So instead of looking for a file which starts with * and ending with .xml,It seems Biztalk picks up any file which has wild character starts and has .xml in the name not necessarily ending

But the contradiction to that is if there is one more DOT [.] Case C which is not picked up

So the way the mask gets applied is wild character then matches the last Dot and then matches charecters followwing the dot

Is this an intended behaviour or Bug ?

Error Lookup Tool for BizTalk

You can lookup your Biztalk Error codes using this tool ,You can look at this tool from inside VS.NET .

Download it form Got Dot Net here

Once you unzip the tool . You can open up your VStudio->Tools ->ExternalTools and Add this Exe and give a descriptive name like "Biztalk Error Code Lookup"
So next time you get an Error , It is right there

Separate Host For Tracking

A recommended Biztalk best practice for performance is to have a separate Tracking Host . So when you add a separate Tracking host , you might encounter this scenario


Event Type: Error
Event Source: BAM EventBus Service
Event Category: None
Event ID: 25
Date: 5/7/2005
Time: 1:53:25 AM
User: N/A
Computer: BiztalkProd1
Description:
Either another TDDS is processing the same data or there is an orphaned session in SQL server holding TDDS lock.EXECUTE permission denied on object 'TDDS_Lock', database 'BizTalkDTADb', owner 'dbo'. SQLServer: mysql, Database: BizTalkDTADb.


Soln: This means that your one of your Hosts other than the Tracking Host has the Host Tracking property Checked . Uncheck this and you should be all set

SEHException While Enlisting Orchestration

I am posting one of the errors we encountered over which considerable time was spent debugging.

You will get the below Error in Event Log


Event Type: Error
Event Source: XLANG/s
Event Category: None
Event ID: 0
Date: 5/5/2005
Time: 3:22:32 PM
User: N/A
Computer: Machine12

Description:
An SEHException exception occurred while the XLANG/s runtime enlisted a service.
Error message:External component has thrown an exception.
Call stack: at Microsoft.BizTalk.MetaDataOM. ICLRType.GetFields(UInt32 dwBindingFlags) at Microsoft.BizTalk.XLANGs.BTXEngine. BTXServiceStaticState._createStringLookup(String svcFullTypeName,
IAssemblyManager asmMgr) at Microsoft.BizTalk. XLANGs.BTXEngine.BTXServiceStaticState. PersistState (String mgmtDBServer, String mgmtDBName, String MsgBoxGroupName, String ApplicationName, String serviceAssembly, String serviceTypeName, Guid[] PortIDs, Guid[] LrpIDs, ITransaction transaction)
at Microsoft.BizTalk.XLANGs. BTXEngine. BTXService.GoLive (String configDBServer, String configDBName, String msgBoxGroupName, String applicationName, String servicePath, String serviceTypeName, Guid[] portIDs, Guid[] lrpIDs, ITransaction transaction)

Exception type: BTXEnlistmentException
Source:
Target Site:
Help Link:
Additional error information:

External component has thrown an exception.

Exception type: SEHException
Source: Microsoft.XLANGs.BizTalk.Engine
Target Site: System.Collections.IEnumerator GetFields(UInt32)
Help Link:
Additional error information:

This normally occurs when you don't have a dependent file of the assembly not in GAC . That is an orchestration which references other Dll's must have the dependencies already GAC'd

Another variation of this enlistment error is OutOfMemory exception which is due to some limitation of WMI , which can be increased with a custom Script


But in our case this was not the case I checked and rechecked the dependencies of our project they were all there .

What we found out was that our Production server did not have Biztalk SP1 applied whereas the dev machine had SP1.

So apparently the issue is still I guess of DLL's Missing in GAC but not of the application but that of Biztalk Later version Assemblies !!

Biztalk Subscription

I have often been asked What is the big difference between Biztalk 2002 and 2004 , There are a lot to be frank But if I have to pick one , It will be the much better and unified subscribtion mechanism . If you want to learn more about Biztalk subscribtion Charles Young's Blog Post is a good start