Loadrunner Controller issues

Below are some key facts to remember about controller. (Note: I am publishing some of these issues from my notes which I encountered at different times. I am just consolidating them here.)

The Maximum number of threads for driver. The number of threads per driver has been internally set for each specific protocol. 

Increasing Vuser Limit: 

You can increase the Vuser limit on Windows NT by modifying the load generator's Windows Registry as follows: 

Open the Windows Registry. 

Select Find>Data and search for the term "SharedSection". 

This search should return a path similar to this: 

MyComputer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ Session Manager\Sub Systems\Windows

Modify the second value to 4096. 

Run-time settings - 

When you modify a script's run-time settings in the Controller, the new settings are saved with the Controller scenario. If, you open the script in VuGen and modify the run-time settings, use the Refresh button in the Controller to see the new settings. 

Compiled Vusers - 

When the Vuser script is a compiled Vuser, the Controller doesn't send the dll to the remote machine. 
Workaround: Add the dll to the list of script files. In the Controller Scripts tab, right-click on your script name and select Details. In the Files tab, click Add and point to the dll. This will add your dll to the list of files to be transferred with the script. 

You may receive the following error message when launching the Controller: "Cannot install license information, probably access to system resource was denied." This indicates that you need to log in with administrator permission, since you installed the product with administrator permission. 

Workaround: Run setlicensepermissions.exe from the LoadRunner bin directory to change the registry permissions. 

Environment variables on load generator machines - If you change the value of an environment variable on a load generator machine and you configured the load generator to automatically run virtual users, restart the machine to use the new value of the environment variable. 

Output window - Keeping the Controller Output window open for long periods of time will affect the machine's memory usage.

VB Script VUser Protocol - Essbase Hyperion Application

Recently I had a challenge with Essbase Hyperion application. Application Characteristics: One should install Essbase Client on their machine and should have MS Excel 2003/2007. 

The communication to the DB happens via excel. Essbase client appears as Addin from Excel. I could not use Oracle 2 tier or ODBC protocols. The communication to the dB is not via SQL queries. 

Then I tried Winsock. I could able to develop a script but building scripts for mutliple drill downs that client requested became a pain. 

So adopted VB Script VUser Protocol and I could successfully able to run the tests in PC11. If you want to know how I could able to build scripts for this application please go through this document.

Analysis - Comparing data from LoadRunner Graphs with Raw Data

The below post includes what kind of questions a Performance Engineer may encounter during Analysis session for a detailed oriented and curious client.

Performance Engineer:
I provided the Transaction Response time summary Report and shown different Graphs to the customer

Client:
I would like to have the raw transaction times (number shown in "Pass" column). This means a Transaction passed for 50 times during the test, they want to see the response times of each of the 50 passed transactions.

Performance Engineer:
I provided the Response Times of the two transactions client requested with time stamps.

Client:
What is the difference between first Buffer Time and Over All Response time?

Performance Engineer:
The First Buffer Time is always less than the Overall Response time
what we see in the Response times graph is the TIME When Response Completed. It will be obviously more than that of First Buffer Time.
Please refer the below diagram.

Client:
Thank you for the raw data and the above explanation  Could you please verify that all data points were    included as I cannot match the numbers in the spreadsheet? On the spreadsheet it shows that there were 19 data points for the Transaction1 and 18 data points for Transaction2 yet only 17 data points are available for each in the Raw Data why?

In addition, I cannot see the corresponding entry that matches the value of the maximum number displayed in the summary spreadsheet for either transaction.

The only number that I can currently match is the minimum number. Please let me know if there are other data points that need to be included.

Performance Engineer:
I provided the raw data for the login the Graph. Now I am getting you the Real Raw data from data points.

There is a difference between these two raw data (Real Raw data and Graphed Raw Data)
For Example: In Real raw data, you see two values at 36 seconds. During that 36th second of the Test run two users took 26 sec and 23 seconds respectively.

The Graphed Raw data shows the average of these response times that happened at fraction of seconds. (Because our granularity is only 1 second We cannot go more granular than 1 second)
Average of 26.738 and 23.211 = 24.975. This is what you are seeing in the Graph and But in summary you are seeing 26.738. This value is based on Real Raw data.

Coming to the question, Why we are not able to map the max values to the Maximum data point in Summary? The Web Page component Break down graph is also Average of different users activity at different point. So if you try to map the Max value in Summary with Max value in Web Page Break down Graph we are not mapping same values. Because, they might not occurred at the same time.

What the Response Time Summary shows is pretty much clear. 90 percentile shows how many transactions are below that particulat data point. If the 90 percent column is exceeding your SLA, then we have to correlate it with the Standard Deviation. If deviation is high. Then we need to go to the specific Graph and observe the trend to see where the deviation occurred.

Difference in Response times with Different LoadGenrators. Why?

I recently encountered this issue and the below solution fixed this issue

Issue:
The response times of transactions for the users generated from a Load Generator on Windows 2003 are lower then the response times of the same transactions from Users generated from a Load generator on Windows 2008 R2

Solution:
Perform this on Load generator on Windows 2008 R2

1. Control Panel> User Accounts >Turn User Account control on or off> Uncheck the Check Box with Name: User Account Control Box On to protect your computer

2. Add SSL Certificate:

a. Go to Fire Fox that is in the Bin dir (if it is for Ajax True Client)
b. Open it> Tools > Options>Advance>View Certificate> Add Certificate
c. How will I get Certificate?
Play the application manually on this box. Then certificate may be downloaded in that location.

3. Turn Off Fire Wall

WINSOCKET PROTOCOL IN LOAD RUNNER

When your application is not supported by any Loadrunner protocols. Since most network protocols use Winsock an interface, we would be able to capture almost all application on Windows if we record at theWinsock layer. So we use Winsock when other protocols do not work.

The Below Document shows how to build a Winsock Script with Loadrunner. I also included some Tips and Tricks while using Winsock Protocol

Winsock Script Development

Tips and Tricks

1. Winsocket in multi-protocol mode does not support UDP.
2. Winsocket in single-protocol mode does not support asynchronous calls.
3. You may encounter problems recording in Winsocket, Winsocket/Web or Oracle NCA.
4. Workaround: Modify the registry key: HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\BrowseNewProcess and make sure that the string value BrowseNewProcess is set to yes. This causes IE to open a new process. If this workaround doesn't work for the Windows2000/2003 server, add the argument "-new" to the Program to Record. This causes IE to open a new process.

Function to write current system Time

This main use of this functions to return the current system time at any given
point of time while load runner script is running.This functiona can be used to
report transaction times , script starti time and end time.

long get_secs_since_midnight(void)
{

char * curr_hr; /* pointer to a parameter with current clock hr */
char * curr_min; /* pointer to a parameter with current clock min*/
char * curr_sec; /* pointer to a parameter with current clock sec */
long current_time, /* current number of seconds since midnight */
hr_secs, /* current hour converted to secs */
min_secs, /* current minutes converted to secs */
secs_secs; /* current number of seconds*/
curr_hr = lr_eval_string("{current_hr}>");
curr_min = lr_eval_string("{current_min}");
curr_sec = lr_eval_string("{current_sec}");
hr_secs = (atoi(curr_hr)) * 60 * 60;
min_secs = (atoi(curr_min)) * 60;
secs_secs = atoi(curr_sec);

current_time = hr_secs + min_secs + secs_secs;
return(current_time);

}

Then these two lines are enough for getting current time.

lr_save_datetime("Today is %H %M %S", DATE_NOW, "curr_time");
lr_output_message(lr_eval_string("{curr_time}"));

lr_save_datetime function in Loadrunner

lr_save_datetime function in Loadrunner

LoadRunner stores date/time parameters locally on the LoadRunner PC. This means that opening a script on another users PC, where the date/time parameter is not stored, can cause corruption to the date/time parameters in a script. You know that this has happened when a date/time parameter in this format [%Y%m%d%H%M%S] is sent to the server rather than the format that you intended.

Due to this problem, It is recommend that the lr_save_datetime function is used to save date or time values to a string. The string can then be used in the script and the script is portable between PCs because we are no longer relying on the locally stored parameters.

Sample code

lr_save_datetime("%d/%m/%y", DATE_NOW, "DDMMYY");
lr_output_message("Today's Date is %s",lr_eval_string("{DDMMYY}"));

lr_save_datetime("%d/%m/%Y", DATE_NOW, "DDMMYYYY");
lr_output_message("Today's Date is %s",lr_eval_string("{DDMMYYYY}"));

lr_save_datetime("%B %d %Y", DATE_NOW, "Today");
lr_output_message("Today is %s",lr_eval_string("{Today}"));

lr_save_datetime("%B %d %Y", DATE_NOW + ONE_DAY, "Tomorrow");
lr_output_message("Tomorrow is %s",lr_eval_string("{Tomorrow}"));

lr_save_datetime("%A", DATE_NOW - ONE_DAY, "Yesterday");
lr_output_message("Yesterday was %s",lr_eval_string("{Yesterday}"));

lr_save_datetime("%A", DATE_NOW, "Today");
lr_output_message("Today is %s",lr_eval_string("{Today}"));

lr_save_datetime("%A", DATE_NOW + ONE_DAY, "Tomorrow");
lr_output_message("Tomorrow is %s",lr_eval_string("{Tomorrow}"));

lr_save_datetime("%X", TIME_NOW , "Time");
lr_output_message("The time is %s (in this locale's date format)",lr_eval_string("{Time}"));

lr_save_datetime("%X", TIME_NOW + ONE_HOUR , "Time");
lr_output_message("In one hour it will be %s",lr_eval_string("{Time}"));

lr_save_datetime("%A", DATE_NOW + ONE_DAY, "Tomorrow");
lr_output_message("Tomorrow is %s",lr_eval_string("{Tomorrow}"));

lr_save_datetime("%j", DATE_NOW, "Today");
lr_output_message("Today is day %s in the year",lr_eval_string("{Today}"));

lr_save_datetime("%Z", DATE_NOW, "TimeZone");
lr_output_message("This machine is in the '%s' time zone",lr_eval_string("{TimeZone}"));

lr_save_datetime("%p", DATE_NOW, "AMPM");

if (strcmp (lr_eval_string("{AMPM}"),"PM")!=0)
{
lr_output_message("Good Morning");
}
else
lr_output_message("Good Afternoon");

Impact of Threadpool Architecture on Application Performance

During the performance Test, the web application is tested to respond in a timely and reliable manner to
simultaneous service requests. The response times of the transactions of these multi threaded applications depend up on thread pool architecture.

Creating and destroying a thread is expensive. It requires run-time memory allocation and deallocation.
These overheads may shutdown the system during high loads.

To minimize the overhead of thread creation and destruction, threads in the pool are reused during multiple requests and the creation and destruction of the thread is minimized to only once per thread not once per request.

Efficient thread management for a given system load depends on Thread pool size.

If thread pool is too large: there will be more unused threads --> idle threads increases then busy threads --> contextswitching increases --> processing and memory resources are wasted to maintain the thread pool

If the thread pool is too small:during peak loads, additional threads must be created and destroyed on the fly to handle new requests-->It requires run-time memory allocation and de allocation.

This causes overhead on the application and impacts the performance and may shutdown the system during high loads.

SAP BUSINESS OBJECTS [SAPBO] PERFORMANCE TESTING

This post explains how SAP BI performance testing is conducted to validate the performance of the the BO reports.

This post explains:
How to login to SAP Business Objects and create reports?
How to design a loadrunner script to simulate the above process for mutliple reports with multiple security groups in a Business Objects Universe.

Business Objects Login
1. Launch the SAP BI URL

2. Business Objects login screen will be displayed

3. Enter your user name & password in the boxes and select Windows AD from the drop down list of Authentication.
User name:
Password :
Athentication : Windows AD 

4. Click on Log on and you should be able to see the Business Objects Infoview page

5. Click on the Document List button to go to Navigation page.

6. Once you click on the Document list the navigation page will be opened

7. My Favorites folder in the navigation page is the user personal folder. Reports saved under this folder are accessible to user only and not visible to other users. Public Folders in the navigation page is the folder accessible to all the users. 8. To open the existing reports under Public Folders, navigate the folder under it and click on the folder name. All the reports available under it will be seen on the right side of the screen and double clicking on the Report name will open the report.

Creating a Business Objects Report 
1. In the Navigation page, select Web Intelligence Document under the drop down list of New
2. Depending on the user access, list of universes will be displayed. Select the universe by clicking on the Universe name to develop a report.
3. Once you click on the universe, query panel will be opened with all the universe dimensions
There will be three sections in the query panel.
Data tab: Left side of the panel is called data tab and shows the universe objects.
Result Objects: This section is used to include data in the report. To include data, select objects from the data tab and drag them here and click Run Query to return the data to the report.
Query Filters: This section is used to filer the query. To filter the query, drag objects here and then use the Filter Editor to define custom filters.
4. To develop a report, select objects and drag them to Result Objects section.
5.Click on the Run Query to return the data to the report
Report with selected data will be displayed
To add filter conditions to the report, go back to query panel by clicking on Edit Query button.
6.Drag and drop the object you want to apply filter on in Query Filters panel.
Select the filter condition from the drop down list
Enter the value OR select the Type of values from the list
7.Selecting Values from the list will display the values of the object to be selected.
select the value from the list and click on ‘ > ‘ button and click on ‘ OK’.
8. Click on the ‘Run Query’ button (available on top right corner of the page). This returns the data filtered by Node Description. 9. Reports can be saved to excel or pdf by selecting the option from save button(available on top left corner)

Requirements: The client wants to know
The response times of mutliple BO Reports for each security group.
Want to see response time of each report for every security group. The name of the security group should appear on each Transaction Name.
Quantity of the reports are in around 70-100 and security groups are around 6-9

Design:
In this case the script should be designed in a efficient way by considering following standards
Naming Convention.
Create a single script by consolidating mutliple scripts and writing functions. This makes script manitenance easy.
Apply inheritance - Write functions and use them in the script in multiple areas where ever required. In this way we can minimize script lenghth.
We see more dynamic data in these kind of applications identify and apply correlations in efficient way.
Precausion should be taked while creating Parameter file and while calling the parameters

Below example shows how a Load runner script is developped for SAP BO reports.

It concatinates the security group name(Utype) of the each user in the parameter file along with the transaction Name.




In SAP BI, There might be some prerequisite you should consider before running a script. below are some I had to do. 



The users are differentiated by passing a new parameter value, "User type"

The script executes 5 iterations. During each iteration it randomly picks one report and executes

How SAML -Open SSO works?

Example: A user logs into an application A and then clicks a link from Application A. Then He see information he is expecting from application B. when we simulated this transaction from load runner. The response time of the link is too high.

when I analyzed the communication of the click. I found the complete design behind the link.
and thought of implementing some component leve performance testing measure the latency of LDAP and SAML communication.

These two applications A and B can be B2B applications protected by firewalls of the respective clients. To pass information these two clients is a challenge. This issue is addresses using SAML and LDAP.

SAML is an XML standard for exchanging security-related information, called assertions, between business es over the web. The assertions can include attribute statements, authentication, decision statements, and authorization decision statements.

LDAP, the Lightweight Directory Access Protocol, is a protocol used to access a directory listing.

In the above example,

User Login to the Application A which is firewall protected by business A. 
Then he clicks a link. 
SAML Encrypted token is generated 
This is posted to a url of Business B Https://XXX.businessB.com in encrypted format 
The OIF which is associated with This URL Picks the encrypted SAML Token 
OIF does the following 
Decrypts this Token 
Verify Signature 
Finds SSN. 
It uses SSN to find Userid. To find the userID using SSN, LDAP is used. 
Identifies user associated with the SSN 

OAM comes in to Picture here. OAM Logs the user into the Application B and creates a new Session

as all these tasks are happening behind the scenes the response time is little higher than a normal clicks.

I will explain in the next post How I conducted performance Testing to find the response times of SAML communication and LDAP.