BizTalk 2006 Notes

I've been on a ton of BizTalk 2006 courses and training days recently, so I thought I'd write up some notes and stick them in google. However I've got so many now that I've got to stop typing or I'll never post this so... Enjoy!

Orchestrations

• Derive from BTXService


• Segment 0, 1, 2 etc in the precompiled C# output are the steps of the statemachine the orchestration represents. These segments do not have
  an easy 1:1 correlation with the shapes you drop on an orchestration.


• The orchestration engine calls these methods on the orchestration class


• Orchestrations and BPEL are translatable but you can't do full roundtrip without losing information


• In an orchestration you can use multipart message types as a level or indirection, on the basis that all messages are multipart under the hood anyway.
  Setting things up like this allows you to change a schema without editing every port, parameter, etc that's using it. However some things are less convenient and seem a bit buggier done this way.


• Port types do not specify a direction, they specify a message type. Therefore you just need the one port type for sending and receiving a message of a given type.


• A message typed for a .NET class is serialised using the XmlSerializer
  
  A variable typed for a .NET class is serialised using the Binary Serializer


• In .NET 1.1, an assembly would be generated at runtime which would serialise your type. In .NET 2.0 you can use sgen
  to generate and GAC this serialiser type, which is then automatically used at runtime.


• This serialiser assembly is written to the temp directory - the IIS process identity might need permissions on this directory if you are exposing an HTTP based receive location


• Use the XmlDocument type as a generic message type for ports and messages in orchestrations to do typeless messaging scenarios (even if the underlying data is not xml)


• In 2006 the orchestration designer remembers which groups were expanded/contracted when you close the file. This is about the only enhancement for 2006, as the orchestration investment will be for WF in 2008.


• Persistence points include:
  
  - After message send
  
  - After atomic scope
  
  - Parallel shape
  
  - Break points
  


• Dehydration behaviour can be tuned in BTSNTSvc.config, using the Dehydration element. Set the ConstantThreshold attribute to -1 to turn it off.


• In a listen shape, always set the delay shape in the right-most branch. Otherwise if the orchestration wakes up to execute the delay when the message has just been received, the left branches are executed
  before the delay can fire so that you get a chance to process the inbound message, otherwise you'll get a zombie ("completed with discard messages" - a message that is accepted to the messagebox on a subscription, but the subscription is gone by the time it comes to run).


• In a parallel convoy you have 2 activating receives with initiating correlations. When one comes in the other is treated like it follows the other correlation. This can cause a race
  condition if the second message comes in before the subscription can be set up (you'd get two orchestration instances created in this scenario)


• A parallel shape is not generally executed on multiple threads (although this is a possiblity), and creates extra persistence points. You also need to take the effort to synchronise access on the branches because of the theoretical possiblity of concurrent processing.


• Can only nest orchestrations and scopes 44 levels deep


• Compensations are for completed transactions only - compensations are not called automatically although nested compensations are called by the default compensation. Call a compensation using the compensate shape.


• Resource compensation is a "best efforts" design - there's no guarantee. Sometimes you won't be able to rollback, at which point you generally involve a human


• Delivery Notification on a solicit-response port returns a SOAP fault (regardless of transport) rather than throwing a DeliveryNotificationException. If you right click the operation in the R-R port, and select "Add Fault" you can react to this instead of the exception. You don't receive the soap fault as a message from the port, you add a catch for this in the scope around your send/receive. The fault will be listed as an available exception to catch once you've added the fault to the port


• When you enter a Message Construct shape, the messages you say you are going to construct are nulled out - therefore you cannot, for example, add to a message in a loop.


• XLANGPipelineManager.ExecuteReceivePipeline & XLANGPipelineManager.ExecuteSendPipeline can be used to call a send/receive pipeline from an orchestration.
  Although this allows you to use an assembler to combine multiple messages into another message based on an envelope schema this didn't work very well in beta 2 and I'd still be tempted to use custom code to create the XML you're after.
  Jack found a bug in this that he says he'll blog at some point. Its more useful for getting the text of a flatfile message in the orchestration - it would otherwise only be available to you after the pipeline in the port, which can be too late.


• By default all orchestrations run in the same app domain in a host ("_XDomain"). In BTSNTSvc.config you can assign assemblies to run in different app domains with their own config.
  Use the ExactAssignmentRule & PatternAssignmentRule elements to assign assemblies based on name to the app domains.

Adapters, Ports and Pipelines

• When a port is enlisted with a host, the host runs the adapter, which picks up messages and hands them to the messaging engine, which runs them through the pipeline, then optionally through a map, then into the messagebox
  
  HOST{ PORT{ Adapter -> Messaging Engine -> Pipeline -> [Map] } } -> MessageBox (immutable)
  


• spTransportID is the GUID for the send port and is the implied default subsription for a send port. When you bind to a send port this GUID is ORed into the rest of the filter you create with the binding.


• Routing failure messages are now resumable from the MMC


• Error messages can now be routed. Any previously promoted context properties are demoted and ErrorReport.ErrorType = FailedMessage is promoted (along with other error information) - this can be routed on.
  When you turn on failed message routing you're telling BizTalk to publish rather than suspend failed messages


• Send Port Groups - a distribution list. Send port priority can be used to determine order of sending. You can set a filter on the send port group and this is used to then broadcast to the send port members. The member's filters are not applied on top of the group, however their subscriptions are still valid as normal.


• Pipeline configuration - i.e. the configuration applied to each pipeline component in the pipeline can be overriden using the MMC on a per port basis.
  This was always the case in 2004 however it wasn't exposed in the UI. The configuration given in the compiled pipeline is used unless a replacement XML string is provided in the appropriate field in the appropriate table of the management database. On learning this my colleague Jack excitedly started writing a tool to expose this functionality for 2004, but then I had to piss on his chips as I discovered Jon Flanders had already done it.


• A handler is a mapping of an adapter to a host. You can have many handlers for a receive adapter, and in 2006 many handlers for a send adapter. You can pick the handler in the port configuration, and therefore the credentials with which the send or receive will be performed.


• The "Recoverable Interchange Processing" property on the XmlDasm or FFDasm can allow good records from an inbound batch to flow through whilst bad records are suspended (or routed elsewhere). In 2004 the entire batch would be thrown out.


• XmlDasm "Validate Schema" validates a message against a schema if you provide the schema, otherwise it does nothing. This XmlValidator component does exactly the same.
  Putting in an XmlDisassembler with no schemas causes the component to look the schema up in the management database. The validator flag/component does not do this (it does nothing instead)


• In 2004 Pipeline Components are only loaded from the pipeline components folder. In 2006 if you put them in the GAC, they'll be loaded from the GAC (see the Assembly.LoadFrom() issue below in "General & .NET 2.0 changes affecting BizTalk")


• If you don't decorate a PC with CatID_PipelineComponent, you can't even add it to the toolbox (I've done this)


• The IPipelineContextEx interface in 2006 & 2004 SP1 allows you to hook into the transaction the adapter's using - wrap this in a TransactionScope class on which you can vote


• Use the "Add processing instruction" parameter on the XML assembler to inject a processing instruction element into XML. Processing instructions are interrogated by the shell and are what make xml files launch automatically in office applications, for example InfoPath. This makes it trivially easy to output xml files that users can edit in business friendly apps.
  
  For example, design a form and save it in an accessible network or intranet location. Open this .xsn file and save an instance of the form (.xml). 
  Crack it open in notepad (because of the processing instruction if you double click
  it the shell will launch it in infopath) and you'll see a processing instruction
  similar to this
  
  href="file:///C:\temp\myform.xsn" name="urn:schemas-microsoft-com:office:infopath:myform:-myXSD-2006-04-10T18-43-30"
  ?><?mso-infoPathSolution solutionVersion="1.0.0.1" productVersion="11.0.6565"
  PIVersion="1.0.0.0" href="file:///C:\temp\myform.xsn" name="urn:schemas-microsoft-com:office:infopath:myform:-myXSD-2006-04-10T18-43-30"
  ?><?mso-application progid="InfoPath.Document"?> solutionVersion="1.0.0.1" productVersion="11.0.6565" PIVersion="1.0.0.0" href="file:///C:\temp\myform.xsn"
  name="urn:schemas-microsoft-com:office:infopath:myform:-myXSD-2006-04-10T18-43-30"
  ?> productVersion="11.0.6565" PIVersion="1.0.0.0" href="file:///C:\temp\myform.xsn"
  name="urn:schemas-microsoft-com:office:infopath:myform:-myXSD-2006-04-10T18-43-30"
  ?> productVersion="11.0.6565" PIVersion="1.0.0.0" href="file:///C:\temp\myform.xsn"
  name="urn:schemas-microsoft-com:office:infopath:myform:-myXSD-2006-04-10T18-43-30"
  ?>
  
  This tells the shell to launch it in infopath and where to find the design so it looks pretty.
  


• Partner Ports means "these two orchestrations are talking to each other"


• The SOAP adapter only works with the HTTP protocol - you can't run this over any other transport


• The SOAP adapter can be replaced with the use of the HTTP adapter and an XML Disassembler


• The class generated from the webservices wizard inherits from WebService, but also has code in the baseclass which can read from the management database to get receive location properties, and implements the adapter


• A port must be marked as public on an orchestration to be exposed as a webservice


• A BizTalk webservice consumes errors for security reasons, just returning "General Error" - you can turn this off in development, asking it to throw detailed errors in the web.config file


• You can promote any name of property you like in a pipeline component - its just a name-value pair. To select this property in the various dropdowns, etc, you have to create the propertyschema for it and deploy this to BizTalk.


• The "public address" property on the adapter-port configuration for most adapters is just for documentation - used by the SEED tool to create packages allowing partners to talk to your interfaces

Schemas and Maps

• Selecting the "repeating record" type in the flat file schema wizard causes an element with MaxOccurs = "unbounded" to be emitted. The normal record type does not so that each record becomes a message


• Setting the value field in a map writes that value into the output of the map. An alternative is to use a concatenate functoid to output a constant string, this can then be used to output the same string to multiple destinations


• New functoids in BizTalk 2006: Assert, Nil, IsNil, Logical Not.


• Maps are more performant in 2006 because of the performance enhancements in .NET 2.0. Also for large messages (> 1MB) the transform is performed against a rewritten engine which is able to cache data to disk. This prevents OOM errors - stops the transform failing rather than increases performance.

The Message Box & Subscriptions

• Still have a master messagebox to control message flow to other message boxes


• Still should scale from 1->3 messageboxes as the step from 1->2 is consumed with the overhead of moving to DTC transactions


• The master messagebox can easily get maxed out on the CPU


• Subscribers are free to process messages as they see fit. This is therefore an observer pattern rather than a push.
  A single service on each messagebox server calls DeQueue on the database until messages are found. Each subscriber then gets a copy of the message. This is a new copy in memory until a persistence point is reached, at which point its written to the messagebox.


• The message body (ie the contents) is deleted from the database when the service terminates (this includes when the message is routed to one or more subscribers)


• It follows that the message body is available to view from HAT, etc, when a message is suspended - its not removed from the database until terminated or successfully resumed


• The message is immutable outside of the adapters, pipelines and the create message shape. This seems to be more a matter of making subscription logic/routing straightforward rather than for any technical or legal reason


• Enlisting an orchestration/send port (ie a service) sets up the subscriptions


• A temporary xlang subscription is set up to hangle receive correlations - its set up at send time and removed when the subsription is satisfied

Applications

• Group, deploy and create MSI's based on an application


• This is just a property of a BizTalk artifact. You can't do anything deeper than that with them - for example run an application in a host.


• The Default Application: Set up to receive applications upgraded from BTS2004, or artifacts deployed from the Visual Studio BizTalk Explorer.
  The BizTalk explorer has not been changed for 2006 and so doesn't have the concept of an application. You can rename the default Default Application to something better than
  BizTalk Application 1, or create a new application and mark it as default.


• You can start, stop, etc an entire application as a whole in the new BizTalk MMC.

Deployment & the new MMC administration console

• BTSTask is the replancement command line tool for BizTalk deployments, replacing BTSDeploy, although BTSDeploy is installed with BizTalk 2006 for backwards compatibility


• "Resources" are the new "assemblies" in the new BizTalk MMC


• The Cache interval is an editable parameter in the MMC. This is the interval with which the host polls the management database for updates. You can reduce this from 60s in a development environment to, say, 2s so that changes you make show up immediately.


• Although Kevin Smith told us that the updates are fed to the hosts using an eventing mechanism now. Not sure what the truth is here.


• Visual Studio tries to remember your binding settings when making bindings in the BizTalk explorer. This is still no substitution for a decent developer's deployment script.

BAM

• Major enhancements for 2006, tight integration with messaging (new for 2006) and orchestrations


• Non-BizTalk access through a .NET API


• Defined activities are compiled into generated tables, etc, in the BAMPrimaryImport DB


• BM tool in BizTalk install directory is used to deploy and manage the DB - use bm deploy -all -definitionfile:*.xml to generate tables, etc, from the output of the tracking profile editor


• To change an activity, undeploy and redeploy it


• You can use SQL Notification Services for business alerting - this can be configured in the BAM portal tool that comes with 2006.

Enterprise Single Sign-on (SSO)

• Used to exchange tokens for authentication credentials


• Used as a secure storage mechanism by the adapter framework


• Adapter framework is SSO aware. Receive adapters can authenticate the inbound message - for example the HTTP receive, check the "Use Single Sign-on" checkbox. This is then swapped for an SSO ticket and put in the message context. Then send adapter can then use this ticket to look up the alternative credentials for the target system (using the affiliate).


• An affiliate is an external party - this is based on AD creds inbound - can't use client certificates, etc, although you can use specifically client certs to map to windows accounts on the way in.


• There are 3 versions of SSO: BTS, Sharepoint, HIS. All are incompatible and BizTalk has the strategic version.


• SSO has a UI in 2006!

Business Activity Services (BAS)

• A sharepoint portal for trading partners to view activity regarding their messages

Hosts

• An isolated host is just configuration. For example an HTTP receive running in an isolated host instance is simply the receive adapter ISAPI filter given permissions to access configuration data about the receive location and to post messages to the messagebox.

General & .NET 2.0 changes affecting BizTalk

• When a BizTalk service wants a schema instance, for example, it looks up the details of the type in the management database, and then loads the type (from the GAC) and reads the schema as a string from a field on the type.
  This applies to the other compiled and deployed artifacts, and allows you to do the GAC swap rather than re-deploy every time.


• A new group for 2006 - BizTalk Server Operators have permissions to view context properties & the message body for suspended messages - this previously required BizTalk Administrators group access.


• .NET 2.0 introduces new GAC folders (which you can see with the fusion viewer disabled)
  
  GAC - .NET 1.1 assemblies
  
  GAC 32 - assemblies rely on 32bit native code
  
  GAC 64 - assemblies rely on 64bit native code
  
  GAC MSIL - assemblies compiled against .NET 2.0 and don't rely on native code
  


• Versioning - multiple versions of assemblies can run side by side, but you must configure seperate receive ports for seperate orchestration versions. Only recommended if side by side is planned (even for a short time).


• Assembly.LoadFrom now reads the information from the dll at the path provided and actually loads the assembly from the GAC in .NET 2.0 (unlike in 1.1). This means that the webservice wizard may be looking in the GAC at your assembly rather than at the path you provided.


• Debugging webservices in Visual Studio 2005 are run in the integrated web server for debugging, but this does not run under the correct credentials (it runs as you) and potentially not against the correct version of .NET. Turn this feature off to go back to testing under IIS.