Jul 31st, 2017
A load test, by definition, measures the performance of a system under an expected load. In contrast, a stress test overloads a system in order to find the breaking point.
A load test is performed in order to ensure that a website or web application is capable of handling specific numbers of users at once. Load Testing is often used in the process of capacity planning, to ensure that a system is able to handle growth to specified levels of simultaneous traffic.
A stress test is used to specifically push a system beyond its intended capacity to identify components that begin to slow down, identify bottlenecks in the system, and bring to light possible points of failure.
Example ab
testing for 100 requests with 10 concurrency (clients): We make 100 requests to [url] with 10 concurrency (we can think of it as 10 users making requests at the same time) as follows.
$ ab -k -n 100 -c 10 [url]
Example ab
testing for 10 clients 1000 times added header:
$ ab -k -n 1000 -c 10 -H “Authorization: Bearer <…token…>” http://api_url/
Usually what you get per service at least 4000 transactions per second. 4000-7000 per second per service.
What it means in practice in real world is that application service scalability is never the issue, the problem is always the database.
If the database is not scaled, the more application servers you will start upfront everyone will meet the backend and this usually probably causes problems.
If there is huge number of failed request issue, it is due to dynamic pages. It happens because the Content-Length
can vary between the requests. When using ab
with such pages you need to use the -l
option.
-l Accept variable document length (use this for dynamic pages)
example test can be done with
$ ab -n 10000 -c 16 -l https://domain.com:443/index.html
This runs 10.000 HTTP requests with a concurrency of 16.
Example test results:
Concurrency Level: 16
Time taken for tests: 403.432 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 131810000 bytes
HTML transferred: 129120000 bytes
Requests per second: 24.79 [#/sec] (mean)
Time per request: 645.492 [ms] (mean)
Time per request: 40.343 [ms] (mean, across all concurrent requests)
Transfer rate: 319.06 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 436 487 66.4 477 1854
Processing: 137 157 65.0 149 2022
Waiting: 136 150 34.4 146 1168
Total: 576 644 106.7 627 2822
Percentage of the requests served within a certain time (ms)
50% 627
66% 641
75% 652
80% 656
90% 669
95% 716
98% 841
99% 1012
100% 2822 (longest request)
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.10-SNAPSHOT</version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>default-check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<rules>
<!-- implementation is needed only for Maven 2 -->
<rule implementation="org.jacoco.maven.RuleConfiguration">
<element>BUNDLE</element>
<limits>
<!-- implementation is needed only for Maven 2 -->
<limit implementation="org.jacoco.report.check.Limit">
<counter>COMPLEXITY</counter>
<value>COVEREDRATIO</value>
<minimum>0.60</minimum>
</limit>
</limits>
</rule>
</rules>
</configuration>
</execution>
</executions>
</plugin>