Returning Data from SAP Tables.

After spending a few frustrating hours on why my SAP RFC call was not returning data and the call was successful . We founf this little gem here from the LOB adapter team blog




If you're sending messages using the channel model instead of using a .NET proxy (for example, in BizTalk) - if the table node is missing from the input XML, we interpret that as if you're not interested in that table. If you do want the rows from that table in the response message, you should pass in the node in the input XML (0 rows, just the starting and ending parameter tag).

BizTalk HTTP Chunked Encoding issues

You are probably reading this snippet because you have a perfectly working Biztalk App that works and performs in development and unit test but fails during volume tests with a downstream LOB app like say Peoplesoft with which you are trying to post via HTTP.

As you may have realized already this due to a chunked encoding issue with BizTalk . BizTalk uses chunked encoding when a message is larger than 48 KB by default.


Well when the message payload size is large HTTP/1.1 RFC implementation allows you to specify to the listener that I am going to send you my message in a series of packets each with it's own size indicator with an Optional trailer containing entity header fields
This is to allow the recipient to verify that it has received the whole message.

To do this the sender must set the in the HTTP Header a property called
transfer-coding = "chunked"

It can then send data in chunks and the recipient if HTTP 1.1 compliant will start collecting data until it gets a chunk with chunk-size set to zero indicating end of transmission.

Great , How does it affect me

BizTalk HTTP Adapter does not set this header when sending chunked data causing downstream listeners to assume this is a whole transmission for a partial data packet. Now you know why your app works in your development box where one tends to test with smaller test files not in server at load :-)

What do I do to resolve this

Since you don't have a way of setting this HTTP Header , the other option is to disable chunking altogether in 2004 with a QFE

BizTalk 2004 SP1 and BizTalk 2006 already has the QFE so in this case all you need to do is to set the registry key DWORD DisableChunkEncoding to 1
as specified here

If you want to learn more about chunked encoding read the HTTP RFC section 3.6.1 here

If you had a scenario this post helped leave me a note

BizTalk on VM machines( Virtual Server , Vmware)

Though it is known that BizTalk Server 2006 was supported on Virtual Server 2005 . There was a silence on its support for othe VM's . It is quite hard to get some good documentation on what is Microsoft's stand on other VM's . This KB I recently saw throws more light

As per latest documentation (June 7 , 2007 ) BizTalk 2004 and 2006 is supported on Virtual Server and from a guarded silence on VMWARE now it has been clarified as commercially reasonable support . Which means as long as the issue on the virtual platform does not require a code fix as such, Microsoft would support it .

If the issue is on say VMware, and if it can be reproduced on a non VM box Microsoft would support it. I have heard from fellow MVP's that some customers have already gone and deployed BizTalk solution on VMWare as Vmware was consistent with their overall IT virtualization strategy that included non-ms applications and superior features for eg. Until Virtual Server 2007 ther was not 64 bit support which VMWare already had.

Also there does not seem any case study on customers going live on the BIZTALk + Virtualk Server platform that I have been able to find.

The full text of the KB is reproduced below


SUMMARY

Microsoft BizTalk Server 2006 and BizTalk Server 2004 are fully supported when they are installed on a supported operating system that is running on Microsoft Virtual Server 2005. Neither BizTalk Server 2006 nor BizTalk Server 2004 may perform as expected if you have installed it on a supported operating system that is running on a virtual machine (VM) other than Virtual Server 2005. BizTalk Server 2006 and BizTalk Server 2004 have not been tested on operating systems that are running on VMs other than Virtual Server 2005.

MORE INFORMATION

Product support for BizTalk Server 2006 and for BizTalk Server 2004 on a VM other than Virtual Server 2005 is provided as commercially reasonable support. Commercially reasonable support is defined as all reasonable support efforts by Microsoft Customer Support Services that do not require BizTalk Server 2006 or BizTalk Server 2004 code fixes.

If you experience a problem in BizTalk Server 2006 or in BizTalk Server 2004 on a VM other than Virtual Server 2005, and the problem cannot be resolved through commercially reasonable support, the problem must be reproducible in a non-VM environment or on Virtual Server 2005 for the problem to be escalated to the BizTalk Server Product group as a bug.

If you are planning a Virtual Biztalk Installtion . Do drop me a line , I am very much interested

BizTalk or WF vs BizTalk and WF

Well it is interesting to imagine what the future of a product will be that you love very passionaltely . I am of course talking about Biztalk . Ever since WF was announced and was shipped there was a question
Is WF going to replace Biztalk .
Is R2 going to be the last release of biztalk .
What is after R2 V next . What after VNext .
I have heard V Next is going to be based on WF .

That is a lot of questions and No I am not trying to answer them nor am I making any statements . This is just some random spaghetti thoughts that I feel I need to put out of my head .
Well for one I believe BizTalk as a product is NOT going away . It is very much here and it will be here to stay .I am no way discounting the argument that WF lets you do many of the things that Biztalk does today .But my point is how easy is it for you to do something that is already available in Biztalk . BizTalk today is much more than an orchestation engine . It has a very rich adapter story . Building a workflow engine is one thing having the surround story where you can integerate that engine easily with other systems is a totaaly different ballgame . Writing an EDI adapter for your project to connect using workflow project using hand built WF vs what you can achieve with BizTalk and the EDI Adpater that ships out of the box is two different set of efforts and they are very much different . Of course when you see the 23+ adapters shipping out of the box which includes SAP , MQ , Oracle, Peoplesoft , JD Edwards to name a few it is easy to see why I feel that way . Another importnat reason is today enterprises want to make thier integeration as smooth and integerated while being very cost effective . For this they need an integerated help desk . Case in point some enterprise EDI customers who had the Covast accelerator was not happy with the covast support that they recived compared to the level of support that they get for BizTalk . Considering EDI is not going anywhere with the advent of new standards like Rosettanet and eb-XML on the contarary EDI is growing in momentum and is growing about 6% YOY . So these customers will definiely buy a more integerated suite like Biztalk and not probably build or buy external party adapters .

The next release of Biztalk after R2 code named V Next is based on .net 3.0 stack where the WF that you build in Biztlak and the Visual studio will both be the same in fact the designer could be a single one . Also with adpters moving to the new WCF Framework . One could potentially see future adapters built on WCF being consumed by Biztalk ad well as WF . So will biztalk be replaced then , in future . Probably not again becuase Biztalk also does guaranteed delivery and has a high availability model buuilt in which is scalable vertically and horizontally which is again hard to replicate . But if you are a small shop and you want to send some SAP data to your big customer . You probably can use your SAP adapter based on WCF to send data while compromising on lot of enterprise class features at a much lower cost .

Now with R2's EDI support there would be massive RFID deployment at the edge and there should be some new RFID SKU like the previous partner edition whihc wil make the Biztalk footprint much larger . I know there are other RFID early movers out there but the reason I am willing to bet on BizTalk is I remember back in the 90's every game that I used to play from Wolfenstein to Jazz Jack Rabbit you need to install the sound driver for every game . same with a word processing you needed to install the driver for every printer which means the app maker has to figure out support for each printer /sound card vendor . But microsoft came out with a unified printer and sound driver model for which you could write your app and now you can plug and play any printer that you like . This is pretty much the model on RFID now so the RFID printer manufactures I am sure is gonna love that . The fact that RFID will be packeged with Biztalk is a master stroke that I gotta give it to Microsoft .

So in relaity both WF and BizTalk is going to remain playing to their own individual strengths .
WF in the smaller integeration efforts with lower cost for trimmed basic functionality and BizTalk giving a true integeration platform or the Service bus for the much larger enterprise.

Encryption Decryption using BizTalk

Many have asked me how to configure biztalk with certificates and I tell them it is really easy . But many seems to have a big challenge with it. Sometimes it is the lack of proper understanding of the secrurity infrastructure . sometimes it is things like not setting up the thumbprint. I was planning on writing an article on the same . But today i saw a good post from Richard and I dont think I need to do that now .

This is a well documented excellent post

BIZTALK R2 and RFID

Biztalk R2 is nearing release and with support of RFID being delivered it will open a new area for biztalk apps altogether. I am excited about RFID as a technology though the technology is under fire including boycott by many activist group . Some of the concerns regarding privacy are valid but the potential of the technology is amazing. RFID or Radio frequency identification (RFID) is a data collection system based on tiny microchips attached to a box, pallet or individual item that communicate with other devices using radio waves. The RFID readers obtain data from the tags ( even write ) in bulk. Your RFID client software then collects , ortganizes and distributes this data . The tags can be active or passive meaning ones that are activated by sedning a signal or active which continously screams out its existence .
The benefits of RFID over existing technologies like Barcoding are many including more data capacity , do not need line of sight , bulk read , automated reading ,counting and life history of the objects using read/write tags
The applications are tremendous end to end visibility from factory to consumer , medicines, international shipments , animal tracking . From the Biztallk perspective we have an RFID Services running on the client platform with an RFID client on WCF (possibly ) shipping out of the bax whihc biztalk can leverage. Since this would be on WCF you should be able to consume this with not biztalk applications also .
The Event processing layer which hosts the Biztalk rules engine in some format will support things like eliminating duplicate reads , aggregation events or some custom threshhold rules.
I will be posting soon about Biztalk RFID soon . Meanwhile you can visit the RFID resource centre

BizTalk Server 2006 Large Maps

There is a little gem in 2006 that lets you control how large messages are transformed .

Previouslt in 2004 if you had a fairly large transformation , there i s nothing much you could do than scale up the server with more memory. In 2006 you could let BizTalk us the disk after a particular threshold , By default that value us set to 1 MB . That means if your XSLT transformation (which you realize is a tree transformation in memory, Your XML is loaded as the source tree and your declartive Xpath in the xslt is used to match the source tree matches to get the result tree) is larger than 1 MB BizTalk will use the large message transformation engine which will page to disk . This is nice feature for large maps but will slow you down ,Hey it is better than getting an OutofMemoryException and that Administrator's pager going off . You might want to tweak this value (disk is slower than memory ) , Of course you dont want it too high either to beat the whole purpose . You need to figure this value on your available capacity.

To do this you need to change the threshold in the registry
Goto 'HKLM\\Software\\Microsoft\\BizTalk Server\\3.0\\Administration'
and set the DWORD 'TransformThreshold'