LoadGen-Testharness for SOAP transport

I have been using the LoadGen in the recent weeks. I have been using it as part of creating test harness for performance and stress testing in the BizTalk applications. I thought I’ll write a post about configuring LoadGen for SOAP transport. 

A sample scenario: I have a WebService (which I have been using it with my SOAP adapter), which will initiate the message flow in BizTalk. I would like to create a harness which will simulate the source application to post the request message and to receive the response message back from the WebService. 

Steps involved:

1. Create the WebService.
2. Configure the LoadGen.
3. Create the source file which LoadGen use to post).
4. Execute the LoadGen.

1. Create WebService

Following is the sample webservice to be used for the testing: 

Build and complie the webservice project. Access the webservice from the browser. Browse to the page where you can see the SOAP message formats. Following is the SOAP Request message for the above webmethod.  

2.Configure the LoadGen.

 Download and install the latest version of LoadGen (2007) from the below Microsoft link (http://www.microsoft.com/downloads/details.aspx?familyid=C8AF583F-7044-48DB-B7B9-969072DF1689&displaylang=en)

 Open the sample configuration file from <<installation drive>>\ LoadGen\ConfigFiles\ConsoleConfigFiles  

In this case since we are going to test the webservice which inturn return the response, we will select the “SoapToSoapTwoWayLG.xml” – it’s for SOAP two-way transport. In the config file, the root element <LoadGenFramework> contains two key elements  

1. <CommonSection>
2. <Section>

<CommonSection> element configures the default settings common to all LoadGen tasks in a LoadGen scenario. Some of the key settings are  

<NumThreadsPerSection>: Number of concurrent threads that LoadGen will use to send messages.

<SleepInterval>:Interval between the subsequent threads.

<LotSizePerInterval>: Number of messages that will be sent as a lot in each thread.

<RetryInterval>: Interval time to reattempt in case of failure.

<OptimizeLimitFileSize>: If file size becomes greater than the specified limit (in bytes), LoadGen will perform streaming for optimization.  

<Transport>: Transport protocol to be used for the LoadGen tasks. In this case SOAP.  

<Section> element configures the setting specific to a LoadGen task.

          <SrcFilePath>: Path of the source file for a LoadGen task

<DstLocation> Element contains the configuration settings for the LoadGen transport component.

<URL>: Specifies the URL of the WebService.

<SOAPHeader>: Specifies the SOAP action header for the request message to deliver.

<SOAPPrefixEnv>: Specifies the start tags of the SOAP envelope and the SOAP body.

<SOAPPostfixEnv>: Specifies the end tags of the SOAP envelope and the SOAP body.

<ResponseMsgPath>: Specifies the directory location for the response message.  

Coming back to configuring our sample webservice in LoadGen configuration file, perform the following steps (We are going to refer the SOAP Request message):  

• Specify the input file path for the LoadGen in <SrcFilePath>

• Specify the WebService URL in <Section>\<DstLocation>\<Parameters>\<URL> element as

• Specify the SOAPHeader in the format of "SOAPAction:[value]".

• To specify the SOAPPrefixEnv, copy the start tags of SOAP envelope and the SOAP body in a text-editor from our example:

Replace the “<” with “&lt;” and “>” with “&gt;”, now we should have

Copy the above content and paste it in <SOAPPrefixEnv></SOAPPrefixEnv> element, like

 

To specify the <SOAPPostfixEnv>, perform the similar replacements with the end tags of the SOAP envelope and SOAP body. Then we should have

 

• Specify the directory location for the response message in <ResponseMsgPath>

Following figure could help in easily identifying the LoadGen sections from the SOAP message

<Click on the image to see the clearer view>

Create the Source file:

 While configuring the <SOAPPrefixEnv> and <SOAPPostfixEnv> elements in the LoadGen configuration, we have specified the start and end tags of the SOAP envelope and SOAP body. We have not used the actual body of the message. Now we will use this to create the source file for the LoadGen. Copy the SOAP body content and paste it in a text-editor with sample value in it. (I have saved the file as InputSrcFile.txt as specified in the <SrcFilePath>)

As per your test criteria, configure the <CommonSection> element.

I have configured it to 1 file in 1  thread per section to execute the try run

For the complete LoadGen Configuration file for this sample webservice (my skydrive folder location):

http://p9grea.bay.livefilestore.com/y1pVA4-qMhhNrBuLy7yUCv2AyVKjBHI8htB-3b-fwMopx11lrCuSDv9g91MP-3c9RiPJv5YIJmH1KodvcTNlSDwkQ/LGSOAPTwoway.xml

Execute the LoadGen:  

From the command prompt, navigate to the installation directory of the LoadGen and type “LoadGenConsole” with its only parameter, the name of the LoadGenConfiguration file  

[installation directory]\LoadGen\Bins>LoadGenConsole TryLGSOAPToSOAPTwoway.xml 

 If you want to enable tracing for your LoadGen (its very essential in my experience), Uncomment the follwing lines from your LoadGenConsole.exe.config file from “[Installation directory]\LoadGen\Bins\”  

After executing the LoadGen, check for the response in the directory specified in <ResponseMsgPath>

BizTalk Server 2009 beta documentation

BizTalk Server 2009 - beta documentation is now available for download at: http://www.microsoft.com/downloads/details.aspx?familyid=38923F29-167F-497D-80A5-899602FEF8D7&displaylang=en

Cheers
M.R.ASHWINPRABHU

Technorathi Tags: BizTalk Server, BizTalk Server 2009

BizTalk Server 2009 (Beta) for Public

Hi BizTalkers,

On the 8th of December 2008 - Microsoft announced the availability of BizTalk 2009 Beta for public.
https://connect.microsoft.com/site/sitehome.aspx?SiteID=218

Try your hands with the beta version of the new BizTalk Server, looking at other release cycles the actual production version is expected to release early next year. I have been waiting for this release since I am not a beta tester and now it’s available for public.

Along with the BizTalk Server 2009 beta, ESB Guidance 2.0 CTP is also available now http://msdn.microsoft.com/esb

Multiple MessageBox databases in a BizTalk Server group

My current client is one of the retail giants in UK. During the Christmas period, their numbers of business transactions are always very heavy. So, all of their IT applications will face heavy load during October - December period. As part of preparing for that period, the peak readiness activity will start in their initial part of the business year which starts from February. I have been employed to prepare their integration services applications to handle their projected high business transactions during Christmas. Many ideas were discussed for the integration applications which uses BizTalk server. One of them is, empowering the existing BizTalk servers inventory with more servers for scalability and reliability. The scalability of the BizTalk Server databases can be achieved by adding multiple Messagebox databases. The reliability for the BizTalk Server databases can be achieved by using clustered server mechanism. In this post, I am going to discuss about using multiple Messagebox databases in a BizTalk Server group.

First, we will see when we need to use multiple MessageBox databases.

We use the multiple MessageBox databases, when we have

• More database transactions which increases the Messagebox database size and results in lesser performance. For example, when the number of messages to the MessageBox is more.
• Heavy message interactions which is database-intensive and results in lesser performance. For example, if the messages interact with the MessageBox are heavy in size.

The MessageBox database performs three main functionalities. They are:

• Evaluate subscriptions.
• Message publication.
• Save the message state.

So when we decide to add multiple MessageBox databases, our intention should be to distribute the functionalities to multiple databases/servers rather than sharing these functionalities with multiple databases/servers. We will see more detail of this in the following section.

When we add the first MessageBox database to the BizTalk processing servers using the Configuration Wizard, we need to make this MessageBox database as Master MessageBox database. Master MessageBox database is one of the MessageBox databases in your BizTalk Server group which holds the master subscription information, performs the subscription evaluation functionality of the MessageBox and routes the messages to the appropriate MessageBox databases for active message publication. Yes, Master MessageBox database doesn't perform the message publication. It only performs the "Evaluate subscriptions". This is done by disabling the new message publication in the Master MessageBox database.

Figure 1: Disabling new message publication for Master MessageBox Database.

A Master Messagebox database can receive two types of messages:

• A new activation message.
• An active instance's message/Correlation message.

Master MessageBox database with new activation message:

When the new activation message received by the Master MessageBox, after evaluating the subscription the Master MessageBox database distributes the activation message to the next available MessageBox database(secondary). When you have more than one secondary MessageBox database, Master MessageBox database uses it own build-in logic for load balancing the message for the publication with secondary MessageBox database.

Note: Secondary MessageBox databases are the MessageBox databases which are other than Master MessageBox database in your BizTalk Server group. Though "Secondary" is not the official term, it is used to refer the non-Master MessageBox databases in the group.

Master MessageBox database with Correlation message:

When the correlation message is received by Master MessageBox database, it performs the lookup operation in the database for the secondary MessageBox databases that contains the state for the correlation message. And it distributes the message instance to the secondary Messagebox database which is waiting for this message. In this case, the Master MessageBox database doesn't perform round-robin pattern for message distribution. Rather it distributes to the specific secondary MessageBox database which is already expecting for this message from Master MessageBox database.

During dehydration, the Orchestration engine stores the in-memory representation of the Orchestration, its associated messages and state in the secondary MessageBox database which activated its instance.

Ok, we have decided to add multiple MessageBox databases in the BizTalk Server group. But how many is "multiple"? A minimum of 3 MessageBox databases (as recommended by Microsoft) for a BizTalk Server group. Out of these one should act as the Master MessageBox database (needs to disable the message publication on this MessageBox database) and other two should act as the secondary MessageBox databases. This recommendation is made because additional MessageBox databases incurs overhead by the Master MessageBox database for routing between the MessageBox databases. If you have only two MessageBox databases for a BizTalk Server group then most of the benefits gained by the additional MessageBox database is offset by the overhead consumed by the Master MessageBox for message routing. You have another option of using 2 MessageBox databases both performing the subscription evaluation and message publication with your own load balancing logic. In this case distributed transactional consistent with the MessageBoxes cannot be achieved, and hence this method should not be implemented.

So it is always

Master MessageBox database + n- Secondary MessageBox databases.

Another colleague of mine asked can we have multiple MessageBox databases in the same server. Yes, we can have this, but ensure that the existing database server has enough CPU and I/O resources and is bottlenecked only by the lock contention. In most of the enterprises situation this may not be the case. So by adding the secondary MessageBox database on the same physical server can cause server-intensive problems rather than solving the overhead problems involved in the high and heavy message transactions with the MessageBox database server. So it’s always advisable and better to use Messagebox databases in a separate server. For reliability use both the Master and Secondary MessageBox databases in the Clustered server environment.

Since we have started our peak readiness activity much before the Christmas peak time, we had a sufficient time to test the new BizTalk Server group with multiple MessageBox databases in terms of performance and scalability. So thorough testing of your updated server infrastructure is vital in achieving the desired result.

BizTalk Server 2006 - Performance tuning

Kelvin has shared his excellent collection of resources on performance tuning for BizTalk server 2006.

I have started to read through his entire resources. This shall be useful to many other like me.

Thanks Kelvi for sharing your collection.

http://intltechventures.blogspot.com/2008/11/2008-11-01-saturday-biztalk-2006-r2.html

BizTalk testing - Reference

Recently I have read a series of blogs on "BizTalk Testing Guidance".  It details the testing strategies and contains sample test harness code for schema, map, pipleline component, orchestration and .NET component in BizTalk Server. I felt it is interesting and useful, so I thought to refer it.

http://geekswithblogs.net/michaelstephenson/archive/2008/04/27/121687.aspx

BizTalk Server 2009 scheduled for release

The next version of the BizTalk server will be called "BizTalk Server 2009" not BizTalk Server R3. The newer version is scheduled for release on first half of 2009.

BizTalk Server 2009 will be full release of the product. Some of the updates on the newer version are:

• As one of the expected update, it supports the latest Microsoft platform technologies, including Windows Server 2008, Visual Studio 2008 SP1, SQL Server 2008 and the .NET Framework 3.5 SP1
• Hyper-V Virtualisation Support: Microsoft's official embracement of the Virtualisation of the servers. Microsoft was reluctant to provide support for its products on virtualised servers. In BizTalk Server R2, this feature was added in (http://msdn.microsoft.com/en-us/library/cc768518.aspx) and in the newer version it has been extended to take complete advantage of the latest Virtualization improvements included as part of Windows Server 2008 Hyper-V.
• BizTalk Server can be deployed in multi-site cluster scenarios, where cluster nodes could reside on separate IP subnets and avoid complicated VLANs.
• Two new adapters for Oracle E-Business Suites and SQL Server. And updates to the existing adapters.
• Expanding the BAM functionality with SQL Server 2008 Analysis Services.
• Enhanced support for EDI and AS2 protocols, updates for SWIFT support.
• New platform for RFID (Radio Frequency Identification).

Reference: http://www.microsoft.com/biztalk/en/us/roadmap.aspx