Tuesday

SilkPerformer - Opening stopping firewall on unix

For completely stopping the firewall, use these steps.
• Login as root
• On prompt give this command “service iptables stop”
• To check the status of firewall use command below
o service iptables status
o It normally gives output as follows “Firewall is stopped”

For opening specific ports run following commands as root
• Edit the iptables as follows (Note the port number decision may vary)
vi /etc/sysconfig/iptables
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 4155 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8443 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 9443 -j ACCEPT
• Now restart the iptables as follows
/etc/init.d/iptables restart

Monday

SilkPerformer - Linux - counters analysis

Linux - counters, symptoms and Resolution

Virtual Memory Statistics ( vmstat )
vmstat – vmstat reports virtual memory statistics of process, virtual memory, disk, trap, and CPU activity.

example% vmstat 5
procs memory page disk faults cpu
r b w swap free re mf pi p fr de sr s0 s1 s2 s3 in sy cs us sy id
0 0 0 11456 4120 1 41 19 1 3 0 2 0 4 0 0 48 112 130 4 14 82
0 0 1 10132 4280 0 4 44 0 0 0 0 0 23 0 0 211 230 144 3 35 62

Following columns has to be watched to determine if there is any cpu issue
1. Processes in the run queue (procs r)
2. User time (cpu us)
3. System time (cpu sy)
4. Idle time (cpu id)
procs cpu
r b w us sy id
0 0 0 4 14 82
0 0 1 3 35 62
0 0 1 3 33 64
0 0 1 1 21 78
Problem symptoms

A. CPU issues
1.) Number of processes in run queue
1.) If the number of processes in run queue (procs r) are consistently greater than the number of CPUs on the system it will slow down system as there are more processes then available CPUs .
2.) if this number is more than four times the number of available CPUs in the system then system is facing shortage of cpu power and will greatly slow down the processess on the system.
3.) If the idle time (cpu id) is consistently 0 and if the system time (cpu sy) is double the user time (cpu us) system is facing shortage of CPU resources.
Resolution
Resolution to these kind of issues involves tuning of application procedures to make efficient use of cpu and as a last resort increasing the cpu power or adding more cpu to the system.

B. Memory Issues
Memory bottlenecks are determined by the scan rate (sr) . The scan rate is the pages scanned by the clock algorithm per second. If the scan rate (sr) is continuously over 200 pages per second then there is a memory shortage.
Resolution
1. Tune the applications & servers to make efficient use of memory and cache.
2. Increase system memory .
3. Implement priority paging in s in pre solaris 8 versions by adding line “set priority paging=1″ in
/etc/system. Remove this line if upgrading from Solaris 7 to 8 & retaining old /etc/system file.

The top Command

The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel.
The top command monitors CPU utilization, process statistics, and memory utilization. The top section contains information related to overall system status - uptime, load average, process counts, CPU status, and utilization statistics for both memory and swap space.
[root@bigboy tmp]# top
3:04pm up 25 days, 23:23, 2 users, load average: 0.00, 0.02, 0.00
78 processes: 76 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 0.9% user, 0.5% system, 0.0% nice, 0.8% idle
Mem: 384716K av, 327180K used, 57536K free, 0K shrd, 101544K buff
Swap: 779112K av, 0K used, 779112K free 130776K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
27191 root 15 0 1012 1012 780 R 5.6 0.2 0:00 top
4545 root 16 0 5892 5888 4956 S 0.9 1.5 169:26 magicdev
1 root 15 0 476 476 432 S 0.0 0.1 0:05 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:00 keventd
5 root 15 0 0 0 0 SW 0.0 0.0 0:41 kswapd
6 root 25 0 0 0 0 SW 0.0 0.0 0:00 bdflush

Find out who is monopolizing or eating the CPUs

Finally, you need to determine which process is monopolizing or eating the CPUs. Following command will displays the top 10 CPU users on the Linux system.
# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
OR
# ps -eo pcpu,pid,user,args | sort -r -k1 | less
Output:
%CPU PID USER COMMAND
96 2148 vivek /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Ubuntu 64-bit/Ubuntu 64-bit.vmx -@ ""
0.7 3358 mysql /usr/libexec/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --socket=/var/lib/mysql/mysql.sock
0.4 29129 lighttpd /usr/bin/php
0.4 29126 lighttpd /usr/bin/php
0.2 2177 vivek [vmware-rtc]
0.0 9 root [kacpid]
0.0 8 root [khelper]
Now you know vmware-vmx process is eating up lots of CPU power. ps command displays every process (-e) with a user-defined format (-o pcpu). First field is pcpu (cpu utilization). It is sorted in reverse order to display top 10 CPU eating process.

Input Output statistics ( iostat )

iostat reports terminal and disk I/O activity and CPU utilization. The first line of output is for the time period since boot & each subsequent line is for the prior interval . Kernel maintains a number of counters to keep track of the values.

Basic synctax is iostat interval count
option – let you specify the device for which information is needed like disk , cpu or terminal. (-d , -c , -t or -tdc ) . x options gives the extended statistics .

To run iostat , sysstat package should be installed.
iostat -xtc 5 2
extended disk statistics tty cpu
disk r/s w/s Kr/s Kw/s wait actv svc_t %w %b tin tout us sy wt id
sd0 2.6 3.0 20.7 22.7 0.1 0.2 59.2 6 19 0 84 3 85 11 0
sd1 4.2 1.0 33.5 8.0 0.0 0.2 47.2 2 23
sd2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
The fields have the following meanings:
disk name of the disk
r/s reads per second
w/s writes per second
Kr/s kilobytes read per second
Kw/s kilobytes written per second
wait average number of transactions waiting for service (Q length)
actv average number of transactions actively being serviced
(removed from the queue but not yet completed)
%w percent of time there are transactions waiting
for service (queue non-empty)
%b percent of time the disk is busy (transactions
in progress)

The values to look from the iostat output are:
* Reads/writes per second (r/s , w/s)
* Percentage busy (%b)
* Service time (svc_t)
If a disk shows consistently high reads/writes along with , the percentage busy (%b) of the disks is greater than 5 percent, and the average service time (svc_t) is greater than 30 milliseconds, then one of the following action needs to be taken
1.) Tune the application to use disk i/o more efficiently by modifying the disk queries and using available cache facilities of application servers .
2.) Spread the file system of the disk on to two or more disk using disk striping feature of volume manager /disksuite etc.
3.) Increase the system parameter values for inode cache , ufs_ninode , which is Number of inodes to be held in memory. Inodes are cached globally (for UFS), not on a per-file system basis
4.) Move the file system to another faster disk /controller or replace existing disk/controller to a faster one.

Network Statistics (netstat)

netstat displays the contents of various network-related data structures in depending on the options selected.

Example 1:
Network Response
$ netstat -i
Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue
lo0 8232 loopback localhost 77814 0 77814 0 0 0
hme0 1500 server1 server1 10658 3 48325 0 279257 0

This option is used to diagnose the network problems when the connectivity is there but it is slow in response .
Values to look at:
1. Collisions (Collis)
2. Output packets (Opkts)
3. Input errors (Ierrs)
4. Input packets (Ipkts)

The above values will give information to workout
i. Network collision rate as follows :
Network collision rate = Output collision counts / Output packets
Network-wide collision rate greater than 10 percent will indicate
* Overloaded network,
* Poorly configured network,
* Hardware problems.
ii. Input packet error rate as follows :
Input Packet Error Rate = Ierrs / Ipkts.
If the input error rate is high (over 0.25 percent), the host is dropping packets. Hub/switch cables etc needs to be checked for potential problems.

Example 2:
#netstat -a
Local Address Remote Address Swind Send-Q Rwind Recv-Q State
*.* *.* 0 0 24576 0 IDLE
*.22 *.* 0 0 24576 0 LISTEN
if you see a lots of connections in FIN_WAIT state tcp/ip parameters have to be tuned because the
connections are not being closed and they gets accumulating . After some time system may run out of
resource . TCP parameter can be tuned to define a time out so that connections can be released and used by new connection.


sar command
The command is run by typing in:
sar -options int #samples
where valid options generally are:
-g or -p Paging
-q Average Q length
-u CPU Usage
-w Swapping and Paging
-y Terminal activity
-v State of kernel tables

The free Utility

The free utility can determine the amount of free RAM on your system. The output is easier to understand than vmstat's. Here's a sample.

[root@bigboy tmp]# free
total used free shared buffers cached
Mem: 126060 119096 6964 0 58972 40028
-/+ buffers/cache: 20096 105964
Swap: 522072 15496 506576
[root@bigboy tmp]#

You should generally try to make your system run with at least 20% free memory on average, which should allow it to handle moderate spikes in usage caused by running memory-intensive cron batch jobs or tape backups. If you cannot achieve this, consider running more efficient versions of programs, offloading some applications to servers with less load, and, of course, upgrading the capacity of your RAM.

Silk Performance Explorer - Server Monitor option for Linux gives NULL (zero) value for all the counters

Performance Explorer executes a set of commands against the REXEC daemon running on the Linux server, so the first thing to do would be to ensure the rexec daemon is running on the Linux box.

You will find the default measures under PRE-DEFINED DATA SOURCES | CUSTOM DATA | REXEC DATA
*********

QUESTION
----------------------------------
In SilkPerformer"s Server Analysis Module, how can I use the Rexec command to get performance data of a Unix Server ?

The easiest way to do this would be to obtain a MIB file for the device or server you wish to monitor.
Some vendors supply a MIB with their products.

If you do not have a MIB file then you can make use of the Rexec command.


ANSWER
----------------------------------
Open a command prompt from Windows and get information on the device - the command will be:
Rexec "netstat -i " (include the " marks)
You will need an admin level login and password

This will return values for all of the devices on the unix box, normally something like :-

Name MTU Net/Dest Address IPkts IErrors Opkts Oerrs Collis Queue
lo0 8232 loopback locahost 39112 0 22333 0 0 0
hme0 1500 293813 0 333332 0 0 0

The hme0 is the Network Card of the Unix machine which should provide us with the information we require.
Let"s say for example we are interested in the number of InPackets (InPkts) and OutPackets (Opkts) during the loadtest.

We now need to incorporate this Rexec command into SilkPerformer"s SAM, so that we can retrieve this information and either write it to a report or create a graphical representation of it.

Open SilkPerformer
Go to RESULTS | MONITOR SERVER
Performance Explorer will open.
Click MONITOR | ADD DATA SOURCE
Choose Custom Data from the list
Then choose Rexec Data from the Custom Data tree
Choose Next
Enter the Unix server name and the port you are connecting to.
Enter an administrator"s username and password

The Rexec Measures dialog opens.

1)Enter a Measure Type " My Unix"
2)Enter a Measure Name " In Pkts"
3)Enter a Command netstat - I hme0 // please ensure you use a capital letter I for this netstat command
4)Enter a Field Index (looking at the table above it is column 5 for Inpkts - so enter 5)
5)Enter a Line to Skip - 1 //this skips the table headers and choose the hme0 data only

Although the table above shows that you would need to skip 2 rows to get to hme0, however in our command (step 3) we are specifying the name of the device explicitly therefore this will be the only one listed

Click Add when done.
Now follow steps 1 - 5 to add Opkts to our custom measures, you will need to change the Field Index
(Step 4) to a value of 7

Click Add when done.

You should now be able to obtain these measures from your server.

Tuesday

SilkPerformer - How do I monitor a server in a remote location?

How do I monitor a server in a remote location?
Problem:

How do I monitor a server in a remote location?
Resolution:

SilkPerformer's Server Analysis Module (SAM) allows you to monitor data from remote servers in a straightforward manner, however in order to monitor any application through a firewall the correct ports must be opened. SAM can use one or more of the following methods / default ports to monitor applications and servers:


Windows - PerfMon - NetBIOS ports 137, 138, 139
UNIX - REXEC - Port 512
ALL - SNMP - 161

Should opening these ports pose an unacceptable security risk a workaround is to position an agent machine inside the firewall to do the monitoring (via monitoring scripts) and then access that machine in order to retrieve the monitor data. This can then be viewed alongside the test data on a machine outside the firewall.

Another possible workaround to this problem would be to have another MMC machine on the same side of the firewall of the machine that you are monitoring. You could then use SAM on this machine to monitor, the TSD file could be created and the data used in the normal way.

A further option which may be used for Windows is to use Perfmon to convert a binary PerfMon log to a CSV file and then convert/import CSV format to Time Series Data. For more information see 50150000000AFvv

Performance Explorer can also generate BDL monitoring scripts for server monitoring, which can be directly used by SilkVision and SilkPerformer. The advantages of monitoring scripts are:

*Monitors can be placed behind firewalls using a SilkPerformer remote agent
*Monitors can be directly integrated into load testing scripts
*Monitors scripts are the most flexible way to build sophisticated monitoring scenarios including event notifications, thresholds checks, or event-based monitoring.

How to generate monitoring scripts:
1. When Performance Explorer opens in "Server Monitoring" Mode select a data source to monitor.

2. Select the performance counter(s) to be monitored (with the "Data Source Wizard" it's easy to select the measurements you wish to have monitored).

3. Create a SilkPerformer BDL script for the active monitor graph BDL scripts can be generated at any time by either selecting
"FILE | GENERATE BDL SCRIPT..." from the menu bar or by selecting
"GENERATE BDL SCRIPT..." from the context menu in Monitor View's lower pane.

4. Specify a file name for the monitor script and save the file.

5. You will be asked to save Performance Explorer's workspace. Performance Explorer then closes and the script is added to your SilkPerformer project. The SilkPerformer window pops up, showing you the new monitor script.

6. Try Script - To verify the new monitor script, press the Try Script button on the workflow bar. The default update interval for collecting monitor data is 60 seconds.

7. You can adjust the monitoring interval by opening the Project Attribute Dialog
(SilkPerformer menu: PROJECT / PROJECT ATTRIBUTES) and changing the value of the "MeasureCollectionInterval" Attribute.

Note: Ensure that realistic values appear in the Virtual User Report File.

Some additional troubleshooting steps:
1 - When monitoring Windows machines SAM uses the same technology as Microsoft's Performance Monitor (perfmon), therefore if it is not possible to get data in perfmon it will not be possible to get values in SAM.

2 - If the server that you are attempting to connect to is behind a firewall. It may be the case that ports 137,138 and 139 need to be opened on this firewall. This may be something that you need to verify with the system administrator.

3 - In order to get Performance monitor data you need to have a valid network connection to the machine. To check this see if you can browse to the machine you are trying to monitor.

Borland ID: 5105 Author: Borland Created on: 37834 Product: Category:

Monday

SilkPerformer- How can I assign unique data from a CSV file, such as UserID and Password, to each virtual user in my loadtest?

Problem:

How can I assign unique data from a CSV file, such as UserID and Password, to each virtual user in my loadtest?
Resolution:

QUESTION
----------------------
How can I assign unique data from a CSV file, such as UserID and Password, to each virtual user in my loadtest ?

ANSWER
----------------------
The script below demonstrates how this can be achieved using the GetUserID() function which has been available since SilkPerformer V. This function returns a unique number for every Virtual User in the test and this can then be used to access an unique row in the CSV file..


NOTE: Before running this script the file login.csv must be in the Data folder as defined in SETTINGS | SYSTEM | DIRECTORIES tab


- User Data Files and it must be added to the project. This can be done via PROJECT pane | DATA FILES context menu (right click)
- Add Data Files

//----------------------------------------------------------------------
// How to access unique lines in a single SilkPerformer .csv file
//----------------------------------------------------------------------
benchmark SilkPerformerRecorder

use "WebAPI.bdh"

dcluser
user VUser

transactions
TInit : begin;
TLoadtest01 : 1;
TEnd : end;
var
hfile : number;
sUserName : string;
sPassword : string;

dcltrans
transaction TInit
begin
// Load CSV and set the delimiter to ","
FileCSVLoadGlobal(hfile, "Login.csv", ",");

// The GetUserID() function will retrieve a unique ID per
// virtual user. This ID is 1 for the first virtual user,
// 2 for the second and n for the n'th virtual user.

// Move to the row in the file using the unique ID obtained
// this means UserID 1 will use row 1, UserID 2 will use
// row 2 and so on...

FileGetRow(hfile, GetUserId());

// Set the username from the file at the specified
// row from the first column

sUserName:=FileGetCol (hfile, 1, 10);

// Set the password from the file at the specified row
// from the second column

sPassword:=FileGetCol (hfile, 2, 10);

end TInit;

transaction TLoadtest01
begin
//write to username and password to the output file for checking

writeln ("UserName: "+sUserName);
writeln ("Password: "+sPassword);

end TLoadtest01;

transaction TEnd
begin
//Unload the CSV File

FileUnload(hFile);

End TEnd;
Borland ID: 5965 Author: Borland Created on: