JMeterPluginsCMD Command Line Tool

There is small command-line utility for generating graphs out of JTL files. It behave just like right-click context menu on all graphs. The tool can generate PNG, CSV or both. Note you can use GraphsGeneratorListener if you want to bulk generate graphs after a test or for existing results.

Installation

The tool is placed inside distribution ZIP. The tool consists of CMDRunner.jar and shell wrappers. Wrappers just shorten call of "java -jar CMDRunner.jar --tool Reporter ...". The tool files must be placed inside lib/ext directory of JMeter installation. The tool requires JMeterPlugins.jar to be placed in lib/ext also. The tool requires JMeter 2.4 to function properly (2.3 will work only for PNG generation).

Available for download at:  http://jmeter-plugins.org/wiki/StandardSet/

Usage and Parameters

Generating PNG:

java -jar CMDRunner.jar --tool Reporter --generate-png test.png --input-jtl results.jtl \
  --plugin-type ResponseTimesOverTime --width 800 --height 600

Generating CSV:

java -jar CMDRunner.jar --tool Reporter --generate-csv test.csv \
  --input-jtl results.jtl --plugin-type ResponseTimesOverTime

Yoy may generate CSV and PNG in single tool run. --help will show you short help list on available parameters.

--generate-png <file>	generate PNG file containing graph
--generate-csv <file>	generate CSV file containing graph data
--input-jtl <file>	load data from specified JTL file
--plugin-type <class>	which type of graph use for results generation

You may customize graph drawing and data processing like GUI settings do with optional parameters:

--width <pixels>	for PNG only - width of the image, default is 800
--height <pixels>	for PNG only - height of the image, default is 600
--granulation <ms>	granulation time for samples
--relative-times <yes/no>	use relative X axis times, no will set absolute times
--aggregate-rows <yes/no>	aggregate all rows into one
--paint-gradient <yes/no>	paint gradient background
--paint-zeroing <yes/no>	paint zeroing lines
--paint-markers <yes/no>	paint markers on data points (since 1.1.3)
--prevent-outliers <yes/no>	prevent outliers on distribution graph
--limit-rows <num of points>	limit number of points in row
--force-y <limit>	force Y axis limit
--hide-low-counts <limit>	hide points with sample count below limit
--success-filter <true/false>	filter samples by success flag (since 0.5.6), possible values are true, false, if not set no filtering on success flag will occur
--include-labels <labels list>	include in report only samples with specified labels, comma-separated
--exclude-labels <labels list>	exclude from report samples with specified labels, comma-separated
--auto-scale <yes/no>	enable/disable auto-scale multipliers for perfmon/composite graph
--line-weight <num of pixels>	line thickness for graph rows
--extractor-regexps <regExps list>	list of keyRegExp and valRegExp pairs separated with {;}, only used by PageDataExtractorOverTime

Plugin Type Classes

Make note that you need to install corresponding plugins set to have Extras plugins available in CMD.
Most of class names are self-explanatory:

• AggregateReport = JMeter's native Aggregate Report, can be saved only as CSV
• ThreadsStateOverTime = Active Threads Over Time
• BytesThroughputOverTime
• HitsPerSecond
• LatenciesOverTime
• PerfMon = PerfMon Metrics Collector
• ResponseCodesPerSecond
• ResponseTimesDistribution
• ResponseTimesOverTime
• ResponseTimesPercentiles
• ThroughputVsThreads
• TimesVsThreads = Response Times VS Threads
• TransactionsPerSecond
• PageDataExtractorOverTime

Other Ways to Automate Reporting

You can use Loadosophia.org service to automate test results storage and reporting. See also: Loadosophia Uploader plugin.

Apache JMeter non-GUI commands

Prerequisites :

1. JRE & JDK installed in OS.
2. Environment variables paths are configured for JRE & JDK.
3. Downloaded Apace JMeter binary.
4. To run the JMeter Test Plan on non-GUI mode, the Test Plan must be made and configured on GUI mode.

 Steps:

To save the results for the <test plan>  (i.e summary report, aggregate report etc ..) specify the path in the GUI mode for file types (.jtl, .csv)

Go to the patch where Apache JMeter binary file is located:

 Command to start the test plan on non-GUI (command line) mode:

jmeter -n -t "D:\NSB\NSB.jmx" 

-n [This specifies JMeter is to run in non-gui mode]
-t  [name of JMX file that contains the Test Plan]
-l  [name of JTL file to log sample results to]
-j  [name of JMeter run log file].

JMeter has several other parameters that can be used for running in non-GUI mode.

-R [list of remote servers] Run the test in the specified remote servers
-H [proxy server hostname or ip address]
-P [proxy server port]

Above mentioned options are used for remote execution of JMeter tests and for using JMeter through a proxy server.

 

Performance Testing: Main Types, Top Tools (Infographics)

A software testing process usually includes many stages, methods and can have different goals. Among them performance testing is one the major components. In general, software performance testing is meant to identify how the system performs under some workload and measure its quality attributes such as speed, scalability, reliability and others. Performance testing is necessary to carry out before the system goes to market, otherwise you might unexpectedly face some cases when the system runs slowly.

There are plenty of useful tools for measuring the performance of web sites, web applications or any other objects. The list of the most popular performance testing tools is illustrated on the infographics below.

Performance testing includes several types of tests that aim to analyze different aspects.

1. Smoke testing
   This is a number of base tests that show if the system actually works or not. Minimal smoke tests can indicate whether all of the components function as expected, in a correct way.
2. Load Testing
   Load tests target to find out the normal performance metrics under the expected load. They are carried out, for instance, by setting some expected number of users making some transactions. Load testing refers to a non-functional type and reveals the response time for all the main procedures. When making changes in the code, load test can help to measure whether this improved the system performance or hurt it.
3. Stress Testing
   Stress tests aim to figure out the behavior of the system under the extreme load and identify the high end of the system’s capacity. Constantly increasing the load, you will see the limits of good performance under a heavy load. When you reach the threshold you can see what components of the system start lagging and fail first. In order to increase the system’s robustness beyond normal conditions you can make these components more efficient.
4. Soak Testing
   Soak tests (they are also called endurance tests) focus on long-time period. They determine whether the system can work under a continuous load. During soak tests it is important to pay attention to such an issue as the performance degradation which means that the response time after a long period would be worse than in the beginning.
5. Spike testing
   Spike tests are carried out to analyze the system behavior in conditions of sudden load increase. The goal is to figure out if such short-time huge changes can hurt the system or will be successfully handled.
6. Component Testing
   Component tests aim to check the performance of any separate component of the system such as a search function, a picture upload, online chat etc.
7. Configuration testing
   This is an essential test for the dynamic system. Changes made to the system configuration always need to be tested for understanding the effect they cause to the performance and behavior.

Accomplishing all types of tests does not automatically mean than your project is perfectly tested. A performance tester should always remember that testing is rather a continuous and repeating process, than a one-time event. Therefore, you should include testing into every stage of development.