INDWAY-Integration Services

Using Wireshark to capture the SOAP messages

2011-02-22T18:41:00.001Z

Again this blog is my knowledge repository/diary. I have done something new which I haven’t done before; I have blog it here so that I can refer it when I want to reuse it again.

I was working with a webservice to which I would post a request message from my test harness and expected the response / resultset from it. But the response was an unexpected resultset. So I wanted to capture the outgoing request to the webservice and incoming response from it by intercepting and examining the SOAP messages exchanged to and from the webservice.

Before I was using an excellent tool to monitor traffic over a machine NIC called Packet sniffer. Now I wanted to use Wireshark (http://www.wireshark.org/download.html) another excellent network sniffing tool.

I have installed and running the Wireshark on my test machine where I am going to post the request message using the test harness to the webservice. Off course you can use Wireshark on any end, either at test harness end or at Webservice end. This is just a network sniffer, which can sniff both the incoming and outgoing messages.

Click “Capture” and select “Options”

Below is the screen shot of “Wireshark:Capture Options”.

Then do the following steps:

Step 1: In “Interface” dropdown list, select the appropriate network card to capture data from. There will be only one option if you have have only one ethernet card. If you’re not sure, no worries try by selecting the different network card and see whether the tool captures any data for the selected card. If it doesn’t, then change the option.

Step 2: Enter “tcp port 8080” in the “Capture Filter:” This filter instructs the Wireshark to capture TCP packets over the port 8080. You can get the port number from the webservice object’s URL property or from the Endpoint definition of your webservice.

Step 3: Click “Start”

Now Wireshark is ready to capture the network traffic over the port 8080. Send some calls to the webservice. After sending the request and receving the response, stop the capture by selecting Capture->Stop or press Control+E.

By default you will have three panel view: Packet List, Packet Details and Packet Bytes.

Packet List is at the top, Packet Details is in the middle and PacketBytes is at the bottom of the window.

Now you have captured the all the network traffics over the port 8080. All of them would be listed in the Packet List panel. Select one by one and see their details in the Packet Details panel. In the Packet Detail panel you will have tree node like structure. Expand the Hypertext Transfer Protocol, and then look for a node called “Data:” under it. Select the node “Data:” and in the Packet Bytes panel you would see the actual data captured, as shown in the above picture. To read the data in a better way, right-click on the “Data:” node in the Packet Details panel, select Copy->Bytes (Printable Text only) and paste it in a note pad to see the exact xml message being passed to the webservice.

<?xml version="1.0" encoding="utf-8"?>

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<soap:Body>………………………………………………..

This is the SOAP request sent across to the webservice and we will capture the SOAP response from the webservice. In the Packet List panel, go through the list after the SOAP request. And you will see the SOAP response on the Packet Details panel itself.

VBScript to convert the comma-delimited file to a carriage-return delimited file

2011-01-31T18:56:00.001Z

'This script will convert the comma-delimited file to a carriage-return delimited file.

Const ForReading = 1
Const ForWriting = 2

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFSOWrite = CreateObject("Scripting.FileSystemObject")

'Set folder
objStartFolder = "C:\Folder"

Set objFolder = objFSO.GetFolder(objStartFolder)

Set colFiles = objFolder.Files

'Loop through all the files in the folder
For Each objFile in colFiles
    'This line will output the name of the file including the extension
    objFileNameParts = split(objFile.Name,".")
    'Remove the extension from the file name
    objFileName = objFileNameParts(0)
    'msgbox("FileName: " & objFileName)
    objReadFilePath = objFolder & "\" & objFileName & ".csv"
    objWriteFilePath = objFolder & "_OUTPUT\" & objFileName & ".csv"
    'msgbox("ReadFilePath: "& objReadFilePath)
    'msgbox("WriteFilePath: "& objWriteFilePath)
    Set objFile = objFSO.OpenTextFile(objReadFilePath, ForReading)
    Do Until objFile.AtEndOfStream
        strLine = objFile.ReadLine
        'Replace commas(,) with carriage return(vbCrLF)
        strLine = Replace(strLine, ",", vbCrLF)
        strNewText = strLine
    Loop
    objFile.Close
    'Create a new file in not created before, otherwise rewrite in the existing file
    If objFSO.FileExists(objWriteFilePath) Then
        Set objFile = objFSO.OpenTextFile(objWriteFilePath, ForWriting)
        MSGBOX("FILE OPENED")
    Else
        set objFile = objFSO.CreateTextFile(objWriteFilePath)
        MSGBOX("FILE CREATED")
    End If
    objFile.WriteLine strNewText

objFile.Close
Next

KB: The following error occurred while saving message from the message box tracking spool. Class not registered

2010-10-07T20:07:00.001+01:00

Today when I tried to save the tracked message in HAT, I had the following error:

The following error occurred while saving message from the message box tracking spool. Class not registered

The solution which worked for us is to restart the “Windows Management Instrumentation” service in the services tray.

HAT uses WMI to query the MessageBox database. There can be two possible causes for this issue. Connectivity issue to the database or WMI. Since the message transmission is normal and its only the problem with HAT, we didn’t doubt the connectivity. Anyway to ensure that the things are right, we checked the MS-DTC setting as suggested here..http://msdn.microsoft.com/en-us/library/aa544733(CS.70).aspx .

Then WMI, in our case we have HP-Openview (like MOM/SCOM) to monitor the BizTalk environment and its agent service runs in BizTalk server to check the status of the server. We have some rules in Openview to monitor the status of the BizTalk artefacts like Orchestrations, receive locations , send ports etc. which uses WMI. Since Openview continuously polls for the BizTalk servers for the status the WMI services is battered. So for us recycling the WMI service (Windows Management Instrumentation) resolved the issue.

Errors List

2010-10-06T19:33:00.001+01:00

In our day to day work, if we face any errors we either resolve the error by our experience or google it to find the correct solution. When you search for the correct solution often we find various potential solutions. So we go through each of them and apply the most suitable for our situation. Why not list the errors which I have encounters here. Instead of searching for the right solution why not first look here and see whether these solution which I have used sometime earlier when I had errors. This may not be useful to others, but might help me again in saving some time. So this is going to be a running list, I’ll update as when I encounter any errors.

Error	The following error occurred while saving message from the message box tracking spool. Class not registered
Cause	WMI service has been battered
Solution	Recycle the “Windows Management Instrumentation” service in the services tray
Reference	http://indway-is.blogspot.com/2010/10/kb-following-error-occurred-while.html

Error	Cannot load source/destination schema: “schema_name”. Either the file/type does not exist, or if a project dependency exists, the dependent project is not built
Cause	For some reason the BizTalk designer loses the schema reference and it is a little hard trying to “trick” the designer into remembering the schema’s location
Solution	Unload/load project of refresh the VS shell with the references.
Reference	http://sandroaspbiztalkblog.wordpress.com/2010/04/26/biztalk-mapper-cannot-load-sourcedestination-schema/

Error	Error <<Error Number>> Exception Caught: Mapper Compiler: Functoid not found: guid (<<GUID>>) with functoid id (<<functiod ID>>)
Cause	For some reason the BizTalk designer loses the schema reference and it is a little hard trying to “trick” the designer into remembering the schema’s location
Solution	Copy the custom functiod’s assembly to Drive:\Program Files\Microsoft BizTalk Server 2006\Developer Tools\Mapper And GAC the custom functiod’s assembly
Reference	http://social.msdn.microsoft.com/Forums/en-US/biztalkgeneral/thread/f9d43d64-84a3-4116-95cb-f4cca19a7d82

Error	sgen.exe exited with code 1
Cause	When you build the solution in release mode. Exact cause??
Solution	Remove the previous version of the assembly from the GAC
Reference	http://chrissyblanco.blogspot.com/2006/10/sgenexe-exited-with-code-1.html

BizTalk Server 2010 - RTM

2010-09-28T18:58:00.002+01:00

BizTalk Server 2010 has been released to manufacturing (RTM)and available for purchase from October 1^st, 2010.

This seventh major release of Microsoft’s enterprise integration product’s developer edition is free. So you can start develop BizTalk Server 2010 applications if you have Visual Studio and SQl Server licenses.

Details about its new features: http://blogs.msdn.com/b/biztalk_server_team_blog/archive/2010/09/22/biztalk-server-2010-released-for-manufacturing.aspx

The evaluation and free developer editions are now available for download from: http://www.microsoft.com/biztalk/en/us/blogs.aspx

Which to use: BizTalk or SQL-Integration Service (SSIS) ?

2010-09-06T19:09:00.001+01:00

I had a situation where I could have used any of these tools, but after some reading about the difference I have nailed down to use SSIS. So this blog is just a collection of my readings which helped me to differentiate and choose the product in my project.

BizTalk and SSIS are different tools and fit different scenarios.

SSIS is designed to move and manipulate very large amount of data over extremely high performance batch processing where BizTalk has been designed to move, process, validate, transform, and route low rate of transactions consisting of tiny amounts of business data

Use SSIS for:

Large ETL processes like de-batching the large CSV file into SQL database, or cleansing the large record before inserting in to database.

Batch Oriented Scenarios i.e. when the files need to be processed in specific time of the day as opposed to file needs to be processed as and when the files are available.

Use BizTalk:

When you have to do the Business Process Management (BPM). i.e. some business processes.

Near real-time scenarios i.e. as and when the files are available and it has to be processed.

SSIS and BizTalk can work together:

Though SSIS and BizTalk are used in different place depends on the scenario, it’s important to note that we can use SSIS with BizTalk. They can work together. For Example we have a project that would receive a batch of data once a day - containing 1000s of records. SSIS would receive the data, and then scrub the data (or validate the format). Once the data was acceptable, BizTalk would pick up each record and do the business processing.

BizTalk can do SQL-BULK insert but..

For bulk data processing we can use “SQL Bulk Load adapter ” available from http://www.biztalkgurus.com/blogs/biztalksyn/archive/2005/10/23/Processing-a-Large-Flat-File-Message-with-BizTalk-and-the-SqlBulkInsert-Adapter.aspx which can efficiently insert a large amount of data into SQL you are still stuck with the issues of transmitting the MessageBox database and the memory issues of dealing with really large messages.

Reference:

http://social.msdn.microsoft.com/forums/en-US/biztalkgeneral/thread/745b3b31-a83a-4629-bec2-49800e266c7e/

http://geekswithblogs.net/wmichel/archive/2007/04/11/111470.aspx

http://social.msdn.microsoft.com/forums/en-US/biztalkgeneral/thread/745b3b31-a83a-4629-bec2-49800e266c7e/

http://social.msdn.microsoft.com/forums/en-us/biztalkgeneral/thread/261D0FBC-EE91-4211-B811-F1D29A4A38D4

http://social.msdn.microsoft.com/Forums/en-GB/biztalkgeneral/thread/0338b17e-f9af-476c-984f-4982b65f230f

Installed BizTalk Server 2006 R2 – Service Pack 1 (SP1)

2010-02-09T18:16:00.001Z

Installed the SP1 in our test server yesterday. It was pretty much a smooth ride with minor hurdles. I started off without reading the installation notes (as it’s in *.docx version and we still need to upgrade our MS-Word to latest version. We never felt the need of upgrading the MS-Word until now)

And these error messages didn’t provide much information to resolve. And same as the 13.6 MB log file.

Since our servers are not maintained properly in the recent time with regular reboots, I just rebooted the server and followed the instructions in the installation guide this time (Thought I would perform the installation properly this time, though the document doesn’t provide any clues for the above error).

I’ll emphasis on the followings to be considered before starting the installation:

SP1-Updates the database schemas:
- All BizTalk servers in a server farm or group should be updated with the service pack as this updates the database schemas.
- Backing-up the BizTalk databases are important (as it’s the case for every service pack). If you want to uninstall the SP1 later for any reason, you must restore the BizTalk Server databases with your backup.
I would always suggest to backup the master secret file.
All BizTalk Server 2006 R2 hotfixes that are currently installed on the computer will be automatically removed by the service pack installation. Some of the hotfixes are part of the service pack, where some are not. So keep a note of all the BizTalk hotfixes in your server before the installation and compare it against the hotfixes which are part of this service pack(http://support.microsoft.com/kb/974563). If any of your hotfixes are not part of this service pack, you can install the SP1-compatible version later..atleast your note will help you to double check later.
Installing BizTalk Server 2006 R2 SP1 does not change the version number of your BizTalk Server installation. Ours is still “3.6.1404.0” after the service pack installation. (the service packs of previous versions of BizTalk server updates the product version number) So to identify its existence later, Go to “Add or Remove Programs” check the “Show updates” check box and look under “Microsoft BizTalk Server 2006 <<Developer Edition>>”

GAC it

2010-02-01T21:22:00.001Z

Have you ever come across this error; I had an orchestration and a new requirement to update the orchestration with the BRE came in. After developing the orchestration, built the solution and deployed it in the test server. But the new code (executing the rule through Call Rule shape) didn’t reflect in the output. When I opened the orchestration debugger, there was a strange behaviour of the control flow. The control flow just jumped over the new code –the Call Rules shape, as shown in the below picture.

I have checked everything, and then later found that it’s because of the build error. When creating the executable for the application in the BizTalk administrator console, I have not added the updated assemblies in the “Resources” folder. This caused the mismatch to the User interface (UI) of the orchestration to the code behind of the orchestration. UI had reflected the newer version of the code but not the code behind!!!

Then created a new build, by properly including the updated assemblies in the “Resources” folder and GACed the updated assemblies. It worked as expected. So lesson leant is, do the basic things properly -“GAC IT”.

Calling .NET assembly or any out-of-box static methods in BizTalk Rule Engine (BRE)

2010-02-01T18:51:00.001Z

Calling .NET assembly or any out-of-box static methods (like System.String.Concat etc) in BizTalk Rule Engine (BRE) is a simple stuff; there is not much documentation available for beginners.

For example, if the rule is

If CustomerType = ‘new’

THEN

WelcomeMessage = String.Concat(“Hello”, CustomerName)

In the above example, I need to use the “System.String.Concat” method from out-of-box mscrolib assembly (or any of your custom method from your helper class).

Executing from Orchestration:

Option 1:

By default, the rule engine requires you to assert all the facts, before they ca be executed. So to execute the rule which contains the .NET assembly or any out-of-box static methods, we must create an instance of the helper class and pass it as a parameter to the rules engine in the Call Rules shape.

So for the above rule, in the Orchestration

Create an instance variable of type “String” - since we are calling the String.Concat() ( or the type of your helper class if you want to call a .NET assembly in your case)

Passing the instance type as the parameter: In the “CallRules Policy Configuration” window, select the above created variable in the “Parameter Name” column.

Option2:

Modify the BRE’s behaviour of expecting us to assert the facts by changing the value of the StaticSupport registry key as specified in MSDN (http://msdn.microsoft.com/en-gb/library/aa950269%28BTS.20%29.aspx).

Registry key for 32-bit OS: [HKEY_LOCAL_MACHINE\Software\Microsoft\BusinessRules\3.0 ]

Registry key for 64-bit OS:[HKEY_LOCAL_MACHINE/SOFTWARE/Wow6432Node/Microsoft/BusinessRules/3.0]

For testing the rule in the BRE composer:

Follow: http://geekswithblogs.net/nsthompson/articles/ClassAndBusinessRulesEngineIntegration.aspx

I have not used the steps mentioned by Neil (in the referenced blog). I am not sure how we will use the fact retriever specific for “integrating the .NET” if you already using a fact retriever in your rule for some other purpose like caching the datatable/database connectionstring.

I have always used a test orchestration which invoked my rule. Yes, I have to deploy my rule before using the rule in the orchestration. That's the major downside of this method.

KB: Errors while configuring SQLXML

2010-01-26T21:08:00.001Z

This post is just a knowledge base reference for me with the issues I have encountered while configuring the SQLXML.

We use SQLXML to execute the SQL query and display the xml document in the web browser.

(As part of the BizTalk BAM portal, users can see the actual message being processed by BizTalk. For example on clicking a link “Original message” in BAM portal, the actual message which is being processed by BizTalk to track the data for BAM has to be shown in the portal. This message(XML) can be retrieved from BAMPrimaryImport database. On clicking the link(Original message) we post the URL (“http://webserver/MsgView?sql=select%20LongReferenceData%20from%20dbo.bam_HireExtension_AllRelationships%20where%20ActivityID='18271dc1-6cd6-4b44-a2db-0bc37b2926c1'%20and%20ReferenceType='Message” ) which executes the SQL query contained in it as part of the querystring. Though this approach is prone to SQL injections, this is just a quick and dirty way to get the work done-The above description may be irrelevant to others, but will remind me the scenario where I have used SQLXML)

To setup the SQLXML, follow the instructions as in:http://sqlxml.org/faqs.aspx?faq=97

The steps in the above URL create a virtual directory in IIS. The only configuration change it performs to the default virtual directory is, mapping to the “sqlis3.dll” from C:\Program Files\Common Files\System\Ole DB directory.

Apparently it just performs simple steps with the virtual directory setting. But it is strictly suggested not to create the above using the IIS wizard, rather do it using the SQLXML wizard.

After configuring the virtual directory, when I executed the above query in the web browser, I received the following errors.

Error 1:

HResult: 0x80046000
Source: Microsoft SQL isapi extension
Description: Invalid connection settings: access denied

This error is due to the invalid SQL user credentials supplied when executing the query by using the virtual directory configured in the above steps.

Follow the instructions from Microsoft to solve the issue: http://support.microsoft.com/kb/820874/en-us

What I did was, created a separate login in SQL Server (Under Security à Logins) and provided this newly created SQL-User in the SQLXML configuration as specified in the article.

After the above, I received the

Error 2:

“Login failed for user ‘UserNameJustCreated’. The password of the account must be changed.”

Then I executed the following query

ALTER LOGIN UserNameJustCreated

WITH PASSWORD = 'password123',

CHECK_POLICY = OFF,

CHECK_EXPIRATION = OFF

Then I had the following error

Error 3:

The CHECK_POLICY and CHECK_EXPIRATION options cannot be turned OFF when MUST_CHANGE is ON

Executed the following to solve this error

ALTER LOGIN UserNameJustCreated WITH PASSWORD = 'password123'

ALTER LOGIN UserNameJustCreated WITH

CHECK_POLICY = OFF,

CHECK_EXPIRATION = OFF;

(So I should have not executed the query as specified in the Error-2 section)

Then when I executed the SQL query hoping it would work, but I received another error

Error 4:

XML document must have a top level element. Error processing resource 'http://nfp-util01/MsgView?sql=select%20LongReference...

To know the exact error, select “View Source” option and following is the error:

<?MSSQLError HResult="0x80040e09" Source="Microsoft OLE DB Provider for SQL Server" Description="The SELECT permission was denied on the object 'bam_HireExtension_AllRelationships', database 'BAMPrimaryImport', schema 'dbo'."?>

So the error was, it required the execute permission for the SQL-user (UserNameJustCreated) I have created in the one of the above step. Gave the execute permission, then I can see the XML in the browser as a sweety J

Error: Exception has been thrown by the target of an invocation

2010-01-20T18:00:00.001Z

In our BizTalk project, I have used a C# helper class, whose method uses Microsoft’s Enterprise Library to invoke a SQL-stored procedure. The SQL stored procedure does some data validation and raises errors for business validation errors using its RAISERROR keyword.

When our tester was testing the above application, they had this strange error

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> <<SQL Server’s error message>>

But the expected error is the <<SQL Server’s error message>> only which has been appended wrongly with the strange error “System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation”

After spending sometime, I found out that the problem could be due to “Instrumentation” in Enterprise Library. So I installed the “Instrumentation” from

Start -> All Programs -> Microsoft pattern & practices -> Enterprise Library 3.1 – May 2007-> Install Instrumentation

It solved the issue.

Resources on testing with BizTalk Server

2010-01-09T15:30:00.001Z

Michael Stephenson has shared his resources and reference on testing with BizTalk Server.

http://code.msdn.microsoft.com/BTSTestGuide

Thanks Michael.

Cheers

Powershell providers for BizTalk Server

2010-01-07T20:40:00.001Z

I was reading the latest edition of BizTalk Hotrod magazine, one interesting article which captured my eye was PowerShell provider for BizTalk developed by Maxime Labelle, Michel Hubert and Randal van Splunteren.(Page 42 onwards: http://biztalkhotrod.com/Documents/Issue8_Q4_2009.pdf)

Microsoft also announced its out of box support to PowerShell for management tasks in its BizTalk Server 2009 R2 release which is expected this year. (http://www.microsoft.com/biztalk/en/us/roadmap.aspx#2009r2)

Interesting to see Microsoft’s effort to next generation technologies to replace scripting languages and batch files for the management task. The obvious benefits of PowerShell over the scripts and batch file is object oriented programming model and much better debugging facilities.

I am going to try the PowerShell provider for my BizTalk Server 2006 projects before Microsoft’s PowerShell providers in 2009 R2.

BizTalk (BRE)Business Rules Engine “Failed to Load” error

2010-01-07T19:00:00.001Z

One of the common error, we would encounter while working with BRE –Business Rules Engine is “Failed to Load”. Most possible cause for this error is the reference assembly is missing in GAC. It’s obvious to identify this cause, as the reason is given very descriptive as shown.

I had this same issue, so GACed the missed the assembly, then reloaded the policy (either by pressing F5 or by physically re-loading the policy). But the error still persisted. Then it reminded me the IDE cache error we all would have encountered during the pipeline development. So the solution is “Just close and open the IDE-Visual Studio again”

I am sure this no-blogging error, but for me. As I have mentioned in my profile, this blog is a knowledge base repository for me, where I store all the things I learn at work so that I can reference latest. If its useful for you as well, I am happy otherwise you have the whole world of web to spend.

Microsoft BizTalk Server 2009 released

2009-04-27T20:31:00.000+01:00

Microsoft officially released the BizTalk Server 2009 for public.

Offical Press Release

New Features

Free ASP.NET MVC eBook Tutorial from ScottGuthrie

2009-03-21T09:52:00.004Z

If you're interested to learn about the Microsoft's new ASP.NET MVC framework, this ebook is for you and its completely FREE (legal also). This is the first chapter to the Professional ASP.NET MVC 1.0 book from Wrox. The 185 pager guides you to create a simple, but complete ASP.NET MVC application from scratch. I have just started to read this ebook, I should say that this is an excellent ebook which I have read in the recent time. Not only ASP.NET MVC framework, even If you're new to

Using Visual Studio 2008
LINQ to SQL

This step-by-step ebook is a must read for people who want to start learn ASP.NET MVC framework. At the end of this ebook I am sure to get satisfaction of complete ene-to-end learning experience for a new concept. Excellent work Scott and Wrox for sharing with all of us for FREE.

eBook: http://aspnetmvcbook.s3.amazonaws.com/aspnetmvc-nerdinner_v1.pdf
Details from Scott's Blog: http://weblogs.asp.net/scottgu/archive/2009/03/10/free-asp-net-mvc-ebook-tutorial.aspx

Cheers

UK SOA/BPM User Group Meeting - 26th May 2009, London

2009-03-21T09:36:00.004Z

The UK SOA/BPM User Group Meeting has been announced which is scheduled to take place on Tuesday, 26th May 2009 at Microsoft’s London office between 6pm - 9pm.

Date & Time: 26th May 2009, 6PM - 9PM
Agenda: 6:00 - 6.10 Introduction
A quick catchup on user group related things.
6:15-7:10 Welcome to Ascentn
Ascentn will be introducing their ISV offering and what it offers UK Professionals in the development of BPM Solutions.
7:30 - 8.30: Still open for volunteers, interested people can contact Mick
Registration: http://sbug20090526.eventbrite.com/
Details: http://sbug.org.uk/forums/p/122/174.aspx#174

I have registered a place there, meet you there..

Thanks Nick for sharing this info, I have found it in your blog.

C# Generics works with BizTalk Server 2009 (Beta) :)

2009-02-13T22:19:00.000Z

I have tried yesterday and it worked. I am really one of the happiest persons to have C# Generics support in BizTalk.

Though Generics was introduced in Version 2.0 of C# and CLR, BizTalk till its version 2006 R2 doesn't support it. But now the story has changed. Yes, BizTalk Server 2009-Beta can support C# helper class with typed parameters i.e. Generics.

In the BizTalk Server version till 2006 R2, though we can't use the C# helper class which has Generics defined in it, we can utilise them indirectly by creating a wrapper class around them. By making this way, we can still abide the rules of BizTalk Server i.e. of not exposing the class with Generics in it directly. But now in BizTalk Server 2009 Beta, we don't have to search for any workaround for using Generics.

Lets see a real time example where we will use C# Generics with BizTalk. I have a database table called "Employee" which contains Employee details (obviously). And as part of the message processing in BizTalk I have to retrieve the Employee name for the given Employee ID. So I'll use a C# helper class to retrieve the EmployeeName for the EmployeeID which will be passed in as a input parameter to a method in it. I have to design the C# helper project in a object-oriented way something like

In the EmployeesDataProvider class, I have defined the Generics as..

public List<Employee> employeeList = new List<Employee>();

to populate it with the result of "select * from Employee" then I'll add this to cache. So next time I'll populate the "employeeList" only if the cache is NULL. Thus avoids the database trips. And use Generics search functionality to get the result from the "employeeList" collection as..

Employee cusTemp = employeeList.Find(delegate(Employee _cus)
           {
               if (_cus.EmployeeID == strEmployeeID)
               {
                   return true;

               }
               return false;
           });

           if (cusTemp != null)
           {
               ret = cusTemp.EmployeeName.ToString();
           }

So the functionality of my helper class is:

Build the EmployeesList (typed class with Generics)from database if its not in the cache / other wise use the cached employee list and use the Generics's Find method to get the EmployeeName for the given EmployeeID.

Now in to BizTalk Server (till 2006 R2), inside the Orchestration's expression shape you may not be able to initiate the EmployeesDataProvider class. Reason being, this class has got the "Generics" type declaration. But now, the same works in BizTalk Sever 2009. Here you can see in BizTalk Server 2009-Beta, the class containing Generics recognised.

Download source code for "Generics in BizTalk Server 2009": http://cid-01845bf64da73117.skydrive.live.com/self.aspx/INDWAY-Integration%20Services/TryGenerics.zip

Thanks to Connected Systems for providing us this feature.

Cheers

M.R.ASHWINPRABHU

BizTalk poster updated for BizTalk Server 2009

2009-02-06T19:55:00.001Z

Microsoft technical editors have updated the BizTalk Server poster for their new BizTalk Server 2009.

http://blogs.msdn.com/nabeelp/archive/2009/02/04/biztalk-posters-updated-for-biztalk-2009.aspx

BizTalk Server 2004: Mainstream support expires soon

2009-02-06T19:40:00.001Z

Microsoft's mainstream support for BizTalk Server 2004 applications is expiring on 14/07/2009. But the extended support is available till 08/07/2014. In the extended support phase

Microsoft will not provide new non-security hotfixes.For those who require new non-security hotfixes during the Extended Support phase, they have to enrol themselves to the Extended Hotfix Support (EHS).
“No-charge” support options are no longer available.This means that customers will need to open paid support cases (such as Premier, Pro or Pay Per Incident cases) to obtain support for their product.For Premier and Pro customers who already open paid cases with Microsoft will notice little difference in the support during extended support phase.

Support Provided	Till 14/07/2009(Mainstream support)	Till 08/07/2014(Extended Support )
Paid support (per-incident, per hour, and others)	Yes	Yes
Security update support	Yes	Yes
Non-security hotfix support	Yes	Requires extended hotfix agreement, purchased within 90 days of mainstream support ending.
No-charge incident support	Yes	No
Warranty claims	Yes	No
Design changes and feature requests	Yes	No
Product-specific information that is available by using the online Microsoft Knowledge Base	Yes	Yes
Product-specific information that is available by using the Support site at Microsoft Help and Support to find answers to technical questions	Yes	Yes

BizTalk Server - Product Life cycle: http://support.microsoft.com/lifecycle/search/default.aspx?sort=PN&alpha=BizTalk+Server+2004&Filter=FilterNO

Microsoft is scheduled to release BizTalk Server 2009 by later this year (Its beta is already available for public:http://indway-is.blogspot.com/2008/12/biztalk-server-2009-beta-for-public.html). So if you plan to migrate your BizTalk server 2004 application, its advisable to wait for some time till the release of 2009 and migrate to 2009 to utilise the longer Microsoft support. And there is no direct upgrade path from BizTalk Server 2004 to BizTalk Server 2009. BizTalk server 2009 supports direct upgrade only from 2006 or 2006 R2.

Cheers

M.R.ASHWINPRABHU

BizTalk 2006 Documenter

2009-01-26T20:18:00.001Z

Documentation is always an important part of any project. Automating the documentation hasn’t been achieved its excellence. But various documentation tools are available for public use which will help us to create a decent base. One of them is Darren Jefford’s BizTalk documenter.

Download BizTalk 2006 Documenter:

http://www.codeplex.com/BizTalkDocumenter/

I have always found this tool handy whenever I have been asked to work on an existing BizTalk solution. For instance lets say you have been asked to work on an existing BizTalk Solution which comprises of 40 projects!!! Navigating through its reference projects, to understand the flow is always a painful process. I always use this tool to generate a help-file out of these BizTalk projects.

In another project, since my client was very much satisfied with the document generated by this tool, I have configured to run this tool as part of automated build (used MS-Build). so whenever our project changes, the project document will also be updated accordingly.

This tool is simple self-describing tool. But I have faced some problems initially and the solution is as follows.

Make sure that you have the HTML-Help compiler when you have opt to generate HTML-Help version (MS-Word version is also available) of document. Download the HTML-compiler from here:

http://www.microsoft.com/downloads/details.aspx?familyid=00535334-c8a6-452f-9aa0-d597d16580cc&displaylang=en

“Unable to locate the help compiler executable”

Ensure the “HelpComplieLocation” variable is assigned to the exact path of the Help compiler “*.exe” file in the BizTalkDocumenter.exe.config file

“There was an error generating the XML document.”

This error is due to tracking enabled in any of your send port. Just for documentation sake disables the tracking in your send port and rerun the tool again.

“Item has been already added. Key in dictionary: 'xxx.xxx.xxx.Orchestration' Key being added:'xxx.xxx.xxx.Orchestration'”

In rare case this error might occur. Ensure you have selected only the required BizTalk Applications in “Select Documentation Type”.

Host, Host Instance, Host Group

2009-01-25T01:49:00.001Z

Recently I have met some developers who were new to BizTalk. They were having very vague idea about BizTalk host, host group and host instance. We all learnt this by experience but for a beginner, understanding the host model in BizTalk might take some time. So let me post an article which explains these for beginners.

Host:

The BizTalk host is a logical, process boundary for BizTalk artifacts like Adapters, Orchestrations, Send Ports, Receive Locations and Pipelines.

BizTalk Host is not a physical entity; it’s a logical boundary
We assign the host to Adapters and Orchestrations, Transport Types and its corresponding handlers to Send Ports and Receive Locations.

Ok, so what is the actual use of the BizTalk Host? These separate process boundaries (Host) enable us to create security boundaries and facilitate management.

Let's say that your company where the BizTalk application is going to be deployed is

· Having different set of security policies for sending and receiving message from their stakeholders – by using different security certificates for receiving and sending any messages or by using different policies in their firewall.

· Having different security access rights to receive/process/send – this can be achieved by providing different windows group to a host, which controls the access to the host and its associated host instance(s) to database and other resources

· Having different set of throttling configurations for send and receive messages

· More receive intensive where your messaging system receives more messages, then you can isolate the receive locations in a separate host.

Microsoft’s recommendation for the HOST model is to have separate Receiving, Sending and orchestration hosts both logically and physically. This model has been proved to be the most optimal configuration for achieving the highest throughput.

Based on the physical configuration and type of adapter hosted within, there are two types of hosts:

1. In-process hosts.

2. Isolated hosts.

In-Process host:

The logical process boundary which gets executed and managed within the runtime process of BizTalk Server engine.

Isolated host:

The logical process boundary which gets executed and managed from outside the runtime process of BizTalk Server engine. In most of the cases, they run under IIS process boundary. Hence the adapters such as HTTP and SOAP (The adapters that runs outside of the normal BizTalk Server runtime process) are managed under isolated host.

Since the Isolated host exist outside the BizTalk environment, the BizTalk Administrator tool is not designed to show the status of the host (like Started/Stopped)

Host Group:

Host group is nothing but a Windows-group which will be assigned to the host to control its associated host instance(s) access to the database and other resources.

When you initially configure the host, you will provide a Windows group which will be used to control access of the host and its associated host instance(s) to database and other resources

Make sure that the Windows-group which will be assigned to the host has got the “Login access” to the BizTalk-SQL databases like:

BizTalkMgmtDb
BizTalkMsgBoxDb
BAMPrimaryImport
BizTalkDTADb
BizTalkRulesEngineDb

A host group requires the following privileges:

It must be a member of the BTS_HOST_USERS SQL Server role in the following databases:
- Configuration
- MessageBox
- Rule Engine
- Tracking
- BAM Primary Import

It must be a member of the BTS_<<in-process host name>>_USERS SQL Server role for the MessageBox database
It must be a member of the BAM_EVENT_WRITER SQL Server role in the BAM Primary Import database.

If you use the Configuration Wizard to create hosts, and if you specify a local Windows group for the hosts, the Configuration Wizard automatically creates two Windows groups. The default names of these groups are the BizTalk Application Users group and the BizTalk Isolated Host Users group.

Host Instance:

A host instance is a run-time instance of host in a server. Host instance runs within the security boundaries of the host and executes message processing on each machine belonging to the host.

The host instance executes in the associated server inheriting the logical boundary defined in the host.

When you create a host instance of a host associated with the host group, the host instance inherits the database privileges of the host group.

BAM in Depth for Developers

2009-01-22T23:32:00.001Z

I have been reading an excellent white paper on BAM, I thought I should share this info. This white paper provides a deep, low-level review of how BAM works and how BAM can be extended. This white paper can be found at:

http://msdn.microsoft.com/en-us/library/dd320628.aspx

Cheers
M.R.ASHWINPRABHU

LoadGen-Testharness for SOAP transport

2009-01-22T22:31:00.000Z

I have been using the LoadGen in the recent weeks. I have been using it as part of creating test harness for performance and stress testing in the BizTalk applications. I thought I’ll write a post about configuring LoadGen for SOAP transport.

A sample scenario: I have a WebService (which I have been using it with my SOAP adapter), which will initiate the message flow in BizTalk. I would like to create a harness which will simulate the source application to post the request message and to receive the response message back from the WebService.

Steps involved:

Create the WebService.
Configure the LoadGen.
Create the source file which LoadGen use to post).
Execute the LoadGen.

1. Create WebService

Following is the sample webservice to be used for the testing:

Build and complie the webservice project. Access the webservice from the browser. Browse to the page where you can see the SOAP message formats. Following is the SOAP Request message for the above webmethod.

2.Configure the LoadGen.

Download and install the latest version of LoadGen (2007) from the below Microsoft link (http://www.microsoft.com/downloads/details.aspx?familyid=C8AF583F-7044-48DB-B7B9-969072DF1689&displaylang=en)

Open the sample configuration file from <<installation drive>>\ LoadGen\ConfigFiles\ConsoleConfigFiles

In this case since we are going to test the webservice which inturn return the response, we will select the “SoapToSoapTwoWayLG.xml” – it’s for SOAP two-way transport. In the config file, the root element <LoadGenFramework> contains two key elements

<CommonSection>
<Section>

<CommonSection> element configures the default settings common to all LoadGen tasks in a LoadGen scenario. Some of the key settings are

<NumThreadsPerSection>: Number of concurrent threads that LoadGen will use to send messages.

<SleepInterval>:Interval between the subsequent threads.

<LotSizePerInterval>: Number of messages that will be sent as a lot in each thread.

<RetryInterval>: Interval time to reattempt in case of failure.

<OptimizeLimitFileSize>: If file size becomes greater than the specified limit (in bytes), LoadGen will perform streaming for optimization.

<Transport>: Transport protocol to be used for the LoadGen tasks. In this case SOAP.

<Section> element configures the setting specific to a LoadGen task.

<SrcFilePath>: Path of the source file for a LoadGen task

<DstLocation> Element contains the configuration settings for the LoadGen transport component.

<URL>: Specifies the URL of the WebService.

<SOAPHeader>: Specifies the SOAP action header for the request message to deliver.

<SOAPPrefixEnv>: Specifies the start tags of the SOAP envelope and the SOAP body.

<SOAPPostfixEnv>: Specifies the end tags of the SOAP envelope and the SOAP body.

<ResponseMsgPath>: Specifies the directory location for the response message.

Coming back to configuring our sample webservice in LoadGen configuration file, perform the following steps (We are going to refer the SOAP Request message):

Specify the input file path for the LoadGen in <SrcFilePath>

Specify the WebService URL in <Section>\<DstLocation>\<Parameters>\<URL> element as

Specify the SOAPHeader in the format of "SOAPAction:[value]".

To specify the SOAPPrefixEnv, copy the start tags of SOAP envelope and the SOAP body in a text-editor from our example:

Replace the “<” with “<” and “>” with “>”, now we should have

Copy the above content and paste it in <SOAPPrefixEnv></SOAPPrefixEnv> element, like

To specify the <SOAPPostfixEnv>, perform the similar replacements with the end tags of the SOAP envelope and SOAP body. Then we should have

Specify the directory location for the response message in <ResponseMsgPath>

Following figure could help in easily identifying the LoadGen sections from the SOAP message

Create the Source file:

While configuring the <SOAPPrefixEnv> and <SOAPPostfixEnv> elements in the LoadGen configuration, we have specified the start and end tags of the SOAP envelope and SOAP body. We have not used the actual body of the message. Now we will use this to create the source file for the LoadGen. Copy the SOAP body content and paste it in a text-editor with sample value in it. (I have saved the file as InputSrcFile.txt as specified in the <SrcFilePath>)

As per your test criteria, configure the <CommonSection> element.

I have configured it to 1 file in 1 thread per section to execute the try run

For the complete LoadGen Configuration file for this sample webservice (my skydrive folder location):

http://p9grea.bay.livefilestore.com/y1pVA4-qMhhNrBuLy7yUCv2AyVKjBHI8htB-3b-fwMopx11lrCuSDv9g91MP-3c9RiPJv5YIJmH1KodvcTNlSDwkQ/LGSOAPTwoway.xml

Execute the LoadGen:

From the command prompt, navigate to the installation directory of the LoadGen and type “LoadGenConsole” with its only parameter, the name of the LoadGenConfiguration file

[installation directory]\LoadGen\Bins>LoadGenConsole TryLGSOAPToSOAPTwoway.xml

If you want to enable tracing for your LoadGen (its very essential in my experience), Uncomment the follwing lines from your LoadGenConsole.exe.config file from “[Installation directory]\LoadGen\Bins\”

After executing the LoadGen, check for the response in the directory specified in <ResponseMsgPath>

BizTalk Server 2009 beta documentation

2009-01-22T20:29:00.001Z

BizTalk Server 2009 - beta documentation is now available for download at: http://www.microsoft.com/downloads/details.aspx?familyid=38923F29-167F-497D-80A5-899602FEF8D7&displaylang=en

Cheers
M.R.ASHWINPRABHU

Technorathi Tags: BizTalk Server, BizTalk Server 2009

BizTalk Server 2009 (Beta) for Public

2008-12-18T16:40:00.005Z

Hi BizTalkers,

On the 8th of December 2008 - Microsoft announced the availability of BizTalk 2009 Beta for public.
https://connect.microsoft.com/site/sitehome.aspx?SiteID=218

Try your hands with the beta version of the new BizTalk Server, looking at other release cycles the actual production version is expected to release early next year. I have been waiting for this release since I am not a beta tester and now it’s available for public.

Along with the BizTalk Server 2009 beta, ESB Guidance 2.0 CTP is also available now http://msdn.microsoft.com/esb

Multiple MessageBox databases in a BizTalk Server group

2008-11-21T15:53:00.001Z

My current client is one of the retail giants in UK. During the Christmas period, their numbers of business transactions are always very heavy. So, all of their IT applications will face heavy load during October - December period. As part of preparing for that period, the peak readiness activity will start in their initial part of the business year which starts from February. I have been employed to prepare their integration services applications to handle their projected high business transactions during Christmas. Many ideas were discussed for the integration applications which uses BizTalk server. One of them is, empowering the existing BizTalk servers inventory with more servers for scalability and reliability. The scalability of the BizTalk Server databases can be achieved by adding multiple Messagebox databases. The reliability for the BizTalk Server databases can be achieved by using clustered server mechanism. In this post, I am going to discuss about using multiple Messagebox databases in a BizTalk Server group.

First, we will see when we need to use multiple MessageBox databases.

We use the multiple MessageBox databases, when we have

More database transactions which increases the Messagebox database size and results in lesser performance. For example, when the number of messages to the MessageBox is more.
Heavy message interactions which is database-intensive and results in lesser performance. For example, if the messages interact with the MessageBox are heavy in size.

The MessageBox database performs three main functionalities. They are:

Evaluate subscriptions.
Message publication.
Save the message state.

So when we decide to add multiple MessageBox databases, our intention should be to distribute the functionalities to multiple databases/servers rather than sharing these functionalities with multiple databases/servers. We will see more detail of this in the following section.

When we add the first MessageBox database to the BizTalk processing servers using the Configuration Wizard, we need to make this MessageBox database as Master MessageBox database. Master MessageBox database is one of the MessageBox databases in your BizTalk Server group which holds the master subscription information, performs the subscription evaluation functionality of the MessageBox and routes the messages to the appropriate MessageBox databases for active message publication. Yes, Master MessageBox database doesn't perform the message publication. It only performs the "Evaluate subscriptions". This is done by disabling the new message publication in the Master MessageBox database.

Figure 1: Disabling new message publication for Master MessageBox Database.

A Master Messagebox database can receive two types of messages:

A new activation message.
An active instance's message/Correlation message.

Master MessageBox database with new activation message:

When the new activation message received by the Master MessageBox, after evaluating the subscription the Master MessageBox database distributes the activation message to the next available MessageBox database(secondary). When you have more than one secondary MessageBox database, Master MessageBox database uses it own build-in logic for load balancing the message for the publication with secondary MessageBox database.

Note: Secondary MessageBox databases are the MessageBox databases which are other than Master MessageBox database in your BizTalk Server group. Though "Secondary" is not the official term, it is used to refer the non-Master MessageBox databases in the group.

Master MessageBox database with Correlation message:

When the correlation message is received by Master MessageBox database, it performs the lookup operation in the database for the secondary MessageBox databases that contains the state for the correlation message. And it distributes the message instance to the secondary Messagebox database which is waiting for this message. In this case, the Master MessageBox database doesn't perform round-robin pattern for message distribution. Rather it distributes to the specific secondary MessageBox database which is already expecting for this message from Master MessageBox database.

During dehydration, the Orchestration engine stores the in-memory representation of the Orchestration, its associated messages and state in the secondary MessageBox database which activated its instance.

Ok, we have decided to add multiple MessageBox databases in the BizTalk Server group. But how many is "multiple"? A minimum of 3 MessageBox databases (as recommended by Microsoft) for a BizTalk Server group. Out of these one should act as the Master MessageBox database (needs to disable the message publication on this MessageBox database) and other two should act as the secondary MessageBox databases. This recommendation is made because additional MessageBox databases incurs overhead by the Master MessageBox database for routing between the MessageBox databases. If you have only two MessageBox databases for a BizTalk Server group then most of the benefits gained by the additional MessageBox database is offset by the overhead consumed by the Master MessageBox for message routing. You have another option of using 2 MessageBox databases both performing the subscription evaluation and message publication with your own load balancing logic. In this case distributed transactional consistent with the MessageBoxes cannot be achieved, and hence this method should not be implemented.

So it is always

Master MessageBox database + n- Secondary MessageBox databases.

Another colleague of mine asked can we have multiple MessageBox databases in the same server. Yes, we can have this, but ensure that the existing database server has enough CPU and I/O resources and is bottlenecked only by the lock contention. In most of the enterprises situation this may not be the case. So by adding the secondary MessageBox database on the same physical server can cause server-intensive problems rather than solving the overhead problems involved in the high and heavy message transactions with the MessageBox database server. So it’s always advisable and better to use Messagebox databases in a separate server. For reliability use both the Master and Secondary MessageBox databases in the Clustered server environment.

Since we have started our peak readiness activity much before the Christmas peak time, we had a sufficient time to test the new BizTalk Server group with multiple MessageBox databases in terms of performance and scalability. So thorough testing of your updated server infrastructure is vital in achieving the desired result.

BizTalk Server 2006 - Performance tuning

2008-11-13T20:23:00.001Z

Kelvin has shared his excellent collection of resources on performance tuning for BizTalk server 2006.

I have started to read through his entire resources. This shall be useful to many other like me.

Thanks Kelvi for sharing your collection.

http://intltechventures.blogspot.com/2008/11/2008-11-01-saturday-biztalk-2006-r2.html

BizTalk testing - Reference

2008-10-15T20:38:00.001+01:00

Recently I have read a series of blogs on "BizTalk Testing Guidance". It details the testing strategies and contains sample test harness code for schema, map, pipleline component, orchestration and .NET component in BizTalk Server. I felt it is interesting and useful, so I thought to refer it.

http://geekswithblogs.net/michaelstephenson/archive/2008/04/27/121687.aspx

BizTalk Server 2009 scheduled for release

2008-10-15T20:16:00.001+01:00

The next version of the BizTalk server will be called "BizTalk Server 2009" not BizTalk Server R3. The newer version is scheduled for release on first half of 2009.

BizTalk Server 2009 will be full release of the product. Some of the updates on the newer version are:

As one of the expected update, it supports the latest Microsoft platform technologies, including Windows Server 2008, Visual Studio 2008 SP1, SQL Server 2008 and the .NET Framework 3.5 SP1
Hyper-V Virtualisation Support: Microsoft's official embracement of the Virtualisation of the servers. Microsoft was reluctant to provide support for its products on virtualised servers. In BizTalk Server R2, this feature was added in (http://msdn.microsoft.com/en-us/library/cc768518.aspx) and in the newer version it has been extended to take complete advantage of the latest Virtualization improvements included as part of Windows Server 2008 Hyper-V.
BizTalk Server can be deployed in multi-site cluster scenarios, where cluster nodes could reside on separate IP subnets and avoid complicated VLANs.
Two new adapters for Oracle E-Business Suites and SQL Server. And updates to the existing adapters.
Expanding the BAM functionality with SQL Server 2008 Analysis Services.
Enhanced support for EDI and AS2 protocols, updates for SWIFT support.
New platform for RFID (Radio Frequency Identification).

Reference: http://www.microsoft.com/biztalk/en/us/roadmap.aspx

Schema (XSD) from XML or DTD file

2008-10-01T20:45:00.001+01:00

In software development often the reverse engineering comes handy, creating the design document from the code, drawing a database diagram the existing database etc. Similarly I have often faced situation where I need to generate the schema files from the XML or DTD file. Though we have various options to achieve the same, we were always using some utility outside the standard IDE's.

Visual studio (from 2005) has provided an option to generate the schema file from the XML or DTD file using the following ways:

Right click on the project name
Select "Add', choose "Add Generated Items.." option.
Choose "Generate Schemas" and click "Add" button.

For XML file:

Select the "Document type" as "Well-Formed XML" and browse for the input XML file

If this option is being used for the first time in your machine,you will see "Well-Formed XML (Not Loaded)" in the document type dropdown list, so you have to execute the "InstallWFX.vbs" script, from "InstallDir:\Program Files\Microsoft BizTalk Server 2006\sdk\Utilities\Schema Generator\" before you use the above method.

For DTD file:

Select the "Document type" as "DTD" and browse for the input XML file

If this option is been used for the first time in your machine "DTD(Not Loaded)" in the document type dropdown list, so you have execute the "InstallDTD.vbs" script, from "InstallDir:\Program Files\Microsoft BizTalk Server 2006\sdk\Utilities\Schema Generator\" before you use the above method.

Click "Finish" to get your schema file.

RSOR - MoM Rules/Alerts targeted to a particular agent

2008-06-18T11:29:00.007+01:00

RSOR - Resultant Set of Rules is an excellent utility to get the set of MoM rules/alerts targeted to a specific client. Thanks to Tom Keane from Microsoft who is the author of this utility. We use this utility more often in our development servers where we frequently add rules/alerts. In development, this utility can help us to ensure the newly added rules/alerts have been reflected in the MoM database or in production this can help us to identify the list the rules/alerts targeted for a specific agent without using the admin-console.

Syntax:
RSOR.exe <MOMDBServer[\instance]> <TargetAgent>

Ensure that you have enough permission to read the MoM database or use a login which has got at least read access to the MoM database to execute this utility.

After executing the above command, check for “ResultantSetOfRules” folder in the root directory where you have copied this utility.

Reference:
http://technet.microsoft.com/en-us/library/cc180050(TechNet.10).aspx

Read the following article for essential tool for MoM:
http://technet.microsoft.com/en-us/magazine/cc160922.aspx
Reference:
http://technet.microsoft.com/en-us/library/cc180050(TechNet.10).aspx

Read the following article for essential tool for MoM:
http://technet.microsoft.com/en-us/magazine/cc160922.aspx

Remove a BizTalk server from BizTalk server 2004 group

2008-06-17T18:13:00.003+01:00

You must have SQL 2000 Server system administrator rights or BizTalk Server 2004 administrator rights to follow these steps. These steps would only be performed on the BizTalk Processing server has failed as is being replaced.

1. Start the BizTalk Server 2004 Administration snap-in.
2. Expand Servers.
3. Click the computer that is not working.
A list of the host instances that are associated with the computer that is not working appears in the results pane.
4. Right-click each host instance and then click Delete. Repeat this step until all the host instances that are associated with the computer are deleted.

Note: You receive the following error message when you try to delete a host instance that is isolated;

Failed while deleting the Windows NT service BTSSvc {F46166EC-B4B8-4129-AB86-F3BB212CEC50}

You receive the following error message when you try to delete a host instance that is in process;

The network path was not found

If you receive one of these error messages click OK. After you click OK you receive the following error message;

You can forcefully delete this host instance. This may leave orphan orchestrations and prevent cleanup of COM+ applications and NT services on machine ‘computer name’. Do you want to proceed with forceful host instance deletion?

If you receive this error message click Yes.
5. Click Start and then click Run and type wbemtest.exe and then click OK. Hope wbemtest.exe should have already installed in your server.
6. In the Windows Management Instrumentation Tester dialog box click Connect.
7. In the Namespace text box type root\microsoftbiztalkserver and click Connect and the click Enum Instances.
8. In the Class Info dialog box type MSBTS_ServerSetting under Enter superclass name and click to select Immediate only and then click OK.
9. In the Query Result dialog box click the computer that is not working and then click Delete.

Note: You cannot delete the server instance if the server is associated with any host instances.

Problems faced while installing BizTalk Server 2004 – Service Pack 2 (SP2)

2008-06-17T17:41:00.010+01:00

It always had been a tough time when we installed BizTalk Server 2004 – SP2 on our servers. Thank god we have a good pre-production environment similar to the production server to try our hands. The lessons learnt are really a good experience.

* Before you apply the SP2, make sure that you have backup for all, off-course this should be the norm for any service pack installation. In BizTalk 2004 SP2, for some reason if you want to uninstall the SP2, it uninstalls the SP2 binaries and restores the BizTalk Server binaries to their SP1 state. SP2 contains major database schema changes and uninstalling SP2 does not restore BizTalk Server databases to their SP1 state. So you must restore the BizTalk Server databases from your backup.

* One at a time. If you have farm based set-up for your BizTalk Processing servers and all access a single cluster for the BizTalk database, ensure that you install the SP2 at one BizTalk server at a time. As said, SP2 contains major database schema changes and if you do the installation of SP2 on more than one server at a time (just to save sometime) there are high chances for your BizTalk databases to crash.

If the installation fails with some error, the SP2 executable package is expected to roll-back all the changes it had done till the failure. It doesn’t seem to be proper always. I have seen scenarios where the host instances were disturbed and all the in-process instance will appear as isolated host instance. In this case to recreate the host instance, you have to delete the instance from both the admin console and also from the services tray (services.msc). To delete the services from the service tray use the following command:

SC delete <<BizTalk Service Name>>

* "bts_CleanupMsgbox" is empty (or just with stored procedure structure without and process code in it) by default. Microsoft doesn't suggest by default to use this SP because of its seriousness in purging the "live" data, but for our operational reasons we decide to go ahead in using this SP, this has to be created explicitly. Microsoft provides the SQL-script to create this SP which can be located at

<BizTalk installation directory>\Schema\msgbox_cleanup_logic.sql

from the BizTalk processing servers.

So when we install BizTalk-SP2, this stored procedure is brought back to default which is empty.

The above are just the few of the issues we faced. And its always advisable to plan properly for restore while installing the BizTalk Server 2004 – Service Pack 2.

Read the following Microsft KB article on issue in BizTalk Server 2004 Service Pack 2 that are not documented in the Readme file:
<<http://support.microsoft.com/kb/940519>>

Debugging is not supported under current trust level settings

2008-06-10T09:50:00.002+01:00

When you try to open any web service/web-application from Visual Studio, you may get the following error:

Debugging is not supported under current trust level settings.

The problem is with the trust level setting of your application. Modifying the web.config file of your application would solve this problem:

Solution:
Locate the <system.web> section.
Insert the tag: <trust level="Full" />

Cleared Microsoft certification on BizTalk Server 2006 - 70-235

2008-05-12T09:29:00.003+01:00

Hi All,

Yesterday I have cleared the Microsoft certification on BizTalk Server 2006 (70-235). It’s a great relief after some good hard work.

Cheers
M.R.ASHWINPRABHU

MoM - Consolidation and Timer based rule

2008-05-04T21:27:00.003+01:00

I have a requirement to create a MoM rule which would automate a task based on some specific eventlog entires in production servers. The automatic task is to recycle some of the COM+ components based on the eventlog. So the eventlog entry is the evidence for me to identify that the issue has occurred. I am going not going to talk about the scripts which I have used to restart the COM+ componet but how I have configured the MoM rule for the same.

So I have to create a MoM rule which would monitor the production server and perform the resolution activity. But the crux is to perform the resolution activity only when the continuous 5 entries of the eventlog are identified in the span of 30 seconds in the production server. So I have to create MoM rule which should act based on the number of the entries of an event in the given time window.

I have achieved this by creating an event-rule which would monitor the specified eventlog for the “Repeat Count” of 5 and I have created another consolidation rule which would maintain a counter for event-rule in 30 seconds. This counter will be reset to zero, if the specified count of logs doesn’t met in specified period (30 seconds). i.e. if the application logs 4 event entries for the specified criteria in 30 seconds. The event-rule’s repeat count would be set back to 1. But if the application logs event entry for 5 times in 30 seconds (or in lesser than 30 seconds), the event-rule’s response entries would trigger.

Following are the snapshot of the MoM rules:

Figure 1: Event-rule for the event entry of 5 entries.

Figure 2: Consolidation rule for 30 seconds.

MoM (Microsoft Operations Manager) Design

2008-04-28T18:11:00.002+01:00

We have been using MoM (Microsoft Operations Manager) in our BizTalk projects for operational support purposes. I am already very much impressed with the advantages it provides for any Wintel applications in its production support arena. Few months back I got an opportunity to design another MoM package for a messaging product based out of Java for Windows environments. The messaging product has been in use for nearly a year in our customer’s production environment.

When I mean by messaging product, we can’t not compare its capabilities with BizTalk, as it doesn’t perform any business processing upon the messages while transferring the messages. But its intended objective is to just transfer very large files in very high frequency to different platform like UNIX, Mainframe etc. This product is yet to evolve in many ways for an enterprise usage as we all know no application is complete in this universe.

Our customer was already using an in-house operational tool to handle transfer failures (and also to handle the products open issues). This product has very minimal capability in throwing/publishing the exceptions (file transfer failures). Any transfer failures from this product would be simply logged in the event log with same Event ID, Event Category, Event Source and the only difference is the description of the error. Our customer is using an alert monitor tool, which provides the dashboard sort of view to the 1^st line support. So, all the operation tools should provide their output to dashboard tool. They call that tool as Mangers of Manger.

Exceptions in this messaging scenario can be like:

· File not found at source.

· Configured pattern of files are not available for transfer.

· Configured file path / server unavailability.

· File has been occupied by another process

· Server disk space full

· Staging folders maintenance issues

· Many product related issue

· Unknown.

Currently used operational tool doesn’t have the intelligence to classify the exception and act accordingly/show the classified alerts to the Monitoring tool. So the challenge is to create an efficient MoM-package which would identify the different categories of the alerts and give the output accordingly to the Managers of manager tool.

I have classified the exception as below:

The exceptions are classified into different category since each category requires different actions:

Subscriber issues require Call-out to the Subscriber production-support team.
Messaging service issue requires Call-out to our production team.
Product issue requires some workarounds which can be automated again using MoM.
Off course we can’t always capture all the exceptions, so a generic catch to capture these exceptions also and throw them with lower severity to the monitoring tool.

In MoM with its ability to use custom scripts, I have configured the categories of the exceptions, by throwing them with Custom Exception Number and Custom event description lets say exception category appended at the starting of the every event description. So I have configured a generic alert called “Event-Filter” which swallows all events from the application. In the event-filter I have used a script which would read the event description and in turn throws another alert with different event ID(specific to category) and the event category append in the event description. For example: if the exception is “File not found”

The messaging product would throw the exception as following in the event log:

Event Type : Error

Event Source : ABCD

Event Category : None

Event ID : 123 (Always same for all error)

Date : 28/04/2008

Time : 15:21:48

User :

Computer : DummyMach123

Description : Files not found at the configured path \\mshsrmsappp0008\P1102747\TestFile.txt>

This event log would be swallowed by the Event-Filter script to throw a MoM event:

Event-ID: 1001 (Custom defined)

Event Category: Subscriber

Event-Decription: Event_Category [Subscriber]: Files not found at the configured path file:////mshsrmsappp0008/P1102747/TestFile.txt

And for the alerts which came out of the event-filter I have configured a rule to action the category related actions. So the new MoM-package would enable the application to behave something like the following:

Following is the part of the event-filter script which is in if-condition, checks the event description and calls another function (CreateEvent) to throw the event with different event criteria’s.

If InStr(strParentEvtDesc,strSCNoMatchPattern) Then

'match the pattern

strParentEvtDesc= strECSubscriber & objContextEvent.Message

CreateEvent EVENT_ID_NoMatchPattern,EVENT_TYPE_ERROR,SCRIPT_EVENTSOURCE,strParentEvtDesc

Elseif…

Following is the method used to throw the event from event-filter:

Sub CreateEvent(intEventNumber,intEventType,strEventSource,strEventMessage)

Set objEvent = ScriptContext.CreateEvent()

objEvent.EventSource = strEventSource

objEvent.EventNumber = intEventNumber

objEvent.EventType = intEventType

objEvent.Message = strEventMessage

ScriptContext.Submit objEvent

End Sub

Following is the event rule in MoM for the subscriber related issue and marked is the criteria for the rule:

As you would have noticed, in the new-MoM package design diagram (showed above) I have showed a XML- file next to the subscriber issues box. Okay I am able to classify the exceptions and now what I am going to do with it? If it’s a subscriber related issue, as I mentioned earlier we want throw a MoM alter which would contain the details of the specific subscriber. I am going to maintain the subscriber contacts in the XML file which can be used a configuration file for the alert for my subscriber related issues. How I am going to accomplish this requirement? We will see it in another day.

Special Thanks to Mr.Lee Nichol for whatsoever little knowledge I have in MoM

IProbeMessage interface in BizTalk.

2008-03-24T22:25:00.003Z

If the pipeline component's execution mode is set as "First recognized" or "All recognized", Pipeline Manager iterate through all the components in the stage to find a component which has implemented IProbeMessage interface; then it executes that component leaving others in First recognized mode and executes all the components that recognise the format of the data in the All recognized mode. i.e. IProbeMessage interface enables the pipeline component to decide do they want to process the message or not.

In BizTalk pipeline, there is concept called Message Probing. This means pipeline component intercepts (probes) the message and determine the format of the message. Disassemble stage's message probing capability enables it receive different format of the message to be received from same endpoint, enables it to identify the format of the message that is being processed dynamically.

IProbeMessage interface contains a method Probe which returns true if the disassembler decides to process the message. Probe method read firs few bytes of the message data stream in order to decide whether it is capable of processing the message or not.

public interface IProbeMessage
{
bool Probe(IPipelineContext pContext, IBaseMessage pInMsg);
}

BizTalk in .NET 3.5 Framework

2008-03-02T14:34:00.002Z

To find out, What's in .NET 3.5 framework for BizTalk.

http://blogs.breezetraining.com.au/mickb/2008/02/08/Feb27thSydneyBizTalkUserGroupALookAtTheNET35Framework.aspx

Adapter version details under host process domain.

2008-03-02T13:36:00.002Z

Our production support guys came up with an interesting requirement, How to find the version of the custom BizTalk adapter assembly being executed under particular host. This has to be achieved without disturbing the production servers.

The answer is pretty simple, use the Process Explore (procexp.exe). One of the gems from SysInternal (Acquired by Microsoft). Process Explore doesn’t require installation, just works with the XCOPY of the procexp.exe as many of the SysInternals tools (satisfied our ChangeManagement since this doesn’t require any installation in the production server :) )

Start it up…
In the View menu, ensure that Show Lower Pane option is checked
In the View\Lower Pane View submenu choose DLLs
Highlight the relevant BTSNTSvc.exe process in the upper pane
Locate your adapter’s runtime assembly and you can get the version details

Process Explore: http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

PS: Assemble name has been hidden for companies privacy. And Thanks for Chris Dickson in helping us.

Technorati Tags: BizTalk

Oslo, BizTalk Mapper & Surface – Fjord

2008-02-27T23:08:00.003Z

I came across a very interesting initiative from Microsoft and thought of blog it. It is about Microsoft's Fjord.

Fjord is the code name for the Microsoft Surface integration pieces to the Modeling commitments of Oslo. BizTalk Mapper, a BizTalk component is the first piece of Oslo to be integrated with Surface.

As Oslo’s one of the key focus is going to be on Modeling, I am expecting the orchestration designer can also be integrated with the Microsoft’s surface(lets see). For expression or for the editor requirement, they may come up with Tablet-PC kind of the keyboard in Surface for Orchestration. More surprises and wonders can be expected as part of Oslo.

Fjord – The code name for BizTalk mapper’s integration with Surface. As fjord means a long, narrow bay with steep sides from the top region of the mountains to the sea, the source schema and destination schema on the two sides on the functoids in the BizTalk Map could have resembled as mountains in the two side of the sea. Aptly named.

And another interesting fact about the Microsoft’s Fjord and Mother Nature’s fjord is Norway has got many deep fjords. Again I donn know why Microsoft’s Oslo is more linked with Norway (Norway’s capital is Oslo).

Reference:

http://www.biztalkgurus.com/blogs/biztalksyn/archive/2007/11/29/project-fjord-biztalk-mapper-to-require-surface-in-oslo.aspx

Microsoft OSLO.

2008-02-21T23:10:00.002Z

OSLO is a codename for the Microsoft’s technical strategy which would provide the platform and tools to make and manage the cross boundaries integrated enterprise applications easily. When they mean easier, they mean by

Reducing the development code be 10xth portion and hence 10X increase in productivity.
And by hosting the services online, integration and the maintenance of these applications could be made simpler.

(I donn why Microsoft code-named their technical strategy for their next version of SOA after Norway’s capital)

Oslo is a initiative from Microsoft’s CSD – Connected Systems Division., CSD is where service-oriented application platform components of Microsoft like BizTalk Server, WCF, WF, messaging and queuing products, and much of the identity and authorization services, such as Windows CardSpace are created.

Microsoft has targeted to print its footprint on SOA by heavily investing in the five primary areas like Framework, Server, Services, Tools and Repository. And in these areas the strategy would be focused mainly on two key elements: Model driven and Service Based.

Model-Driven:

Microsoft is building a general-purpose modelling language, tools and repository to bridge all the models within an application, moving models to the centre of application development. So they have planned to create a modelling tool which bring the gap existing between the various modelling methodologies used by business analyst, architects, developers and non-it professionals who provide requirements to IT. We would be the more mature universal modelling editor.

Service-Based:

Continue to build on our service-oriented architecture (SOA) platform spanning client, server and cloud. BizTalk Services "1", which will feature messaging, identity and workflow for hosting composite applications

The first version of “Oslo” will be delivered through the next versions of our application platform products such as those code-named Microsoft Visual Studio “10,” Microsoft System Center “5,” BizTalk Server “6,” BizTalk Services “1” and Microsoft .NET Framework “4.”

In short Oslo would enable us to simplify modelling, developing, maintenance of enterprises integrated applications.

References:

Microsoft Oslo: http://www.microsoft.com/soa/products/oslo.aspx

Whats Oslo: http://blogs.msdn.com/asmitaw/archive/2007/12/27/whats-oslo.aspx

INDWAY-Integration Services

2008-02-18T12:56:00.000Z