Postfix Stress Test with smtp-source and top
Posted by mike Testing Load with smtp-source and top
In order to evaluate the load on your box you can run smtp-source and combine that with snapshots of top to evaluate the load on the server. Now open two terminals and in one run the smtp-source command and in the other snapshots for top.
Terminal #1
# time /usr/sbin/smtp-source -s 40 -l 10120 -m 500 -c -f test@example.com -t mike@example.com localhost:25
This example shows 40 parallel sessions (-s 40), almost 10KB sized messages (-l 10120), 500 messages sent (-m 500), counter display (-c), envelope sender and receiver (-f test@example.com -t mike@example.com) and connection on port 25 of the localhost (localhost:25).
Terminal #2
top -b -n10 -d7 > top.txt
This command with top will give you 10 snapshots (-n10) at 7 second intervals (-d7) and create a file called top.txt.
As you evaluate the sample data there are several fields to pay close attention to. The first is the Cpu wa or “amount of time the CPU has been waiting for I/O to complete.” This at any sustained level will dramatically decrease the speed or your mail server. Here is the wa from several snapshots taken and you can see that running at the load that is presented by the smtp-source is not sustainable. Spikes in wa are not a problem is just that your mail server will not be able to maintain anything over 10%, maybe even less.
17.5%wa – 21.5%wa – 20.7%wa – 28.6%wa – 47.9%wa
When you evaluate your I/O be sure that you also evaluate the additional resource load from scanning for virus and also Spam. When you add both of these on top of your mail server that whole process can slow down even more. This whole process is hard to nail down as a science but at least this kind of test will provide you with data that you could compare with multiple mail servers or that you can have a starting point for evaluation.
top – 06:57:35 up 25 min, 2 users, load average: 0.02, 0.21, 0.17
Tasks: 107 total, 1 running, 106 sleeping, 0 stopped, 0 zombie
Cpu(s): 5.6%us, 7.8%sy, 0.0%ni, 66.9%id, 17.5%wa, 0.1%hi, 2.0%si, 0.0%st
Mem: 254368k total, 199080k used, 55288k free, 18384k buffers
Swap: 761848k total, 0k used, 761848k free, 110884k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4354 syslog 20 0 1936 684 532 S 1.1 0.3 0:01.41 syslogd
4684 postfix 20 0 6300 2728 1468 S 0.9 1.1 0:02.41 qmgr
top – 06:57:42 up 25 min, 2 users, load average: 0.02, 0.20, 0.17
Tasks: 113 total, 2 running, 111 sleeping, 0 stopped, 0 zombie
Cpu(s): 9.0%us, 10.7%sy, 0.0%ni, 55.1%id, 21.5%wa, 0.4%hi, 3.3%si, 0.0%st
Mem: 254368k total, 204260k used, 50108k free, 19536k buffers
Swap: 761848k total, 0k used, 761848k free, 112776k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5485 postfix 20 0 5900 3056 2444 R 2.1 1.2 0:00.15 smtpd
4354 syslog 20 0 1936 684 532 S 1.6 0.3 0:01.52 syslogd
4684 postfix 20 0 6300 2728 1468 S 1.3 1.1 0:02.50 qmgr
5495 postfix 20 0 5476 1800 1460 S 1.3 0.7 0:00.09 cleanup
top – 06:57:49 up 26 min, 2 users, load average: 0.02, 0.20, 0.17
Tasks: 112 total, 1 running, 111 sleeping, 0 stopped, 0 zombie
Cpu(s): 8.3%us, 13.0%sy, 0.0%ni, 54.5%id, 20.7%wa, 0.3%hi, 3.3%si, 0.0%st
Mem: 254368k total, 207408k used, 46960k free, 20836k buffers
Swap: 761848k total, 0k used, 761848k free, 114900k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4354 syslog 20 0 1936 684 532 S 1.9 0.3 0:01.65 syslogd
4684 postfix 20 0 6420 2760 1468 S 1.4 1.1 0:02.60 qmgr
4677 root 20 0 5396 1736 1408 S 1.0 0.7 0:00.91 master
5500 postfix 20 0 5412 1688 1372 S 0.9 0.7 0:00.06 trivial-rewrite
top – 06:57:56 up 26 min, 2 users, load average: 1.62, 0.52, 0.27
Tasks: 155 total, 3 running, 152 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.8%us, 16.4%sy, 0.0%ni, 40.5%id, 28.6%wa, 0.4%hi, 1.3%si, 0.0%st
Mem: 254368k total, 228268k used, 26100k free, 21600k buffers
Swap: 761848k total, 0k used, 761848k free, 117700k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5501 postfix 20 0 5900 3060 2444 S 4.0 1.2 0:00.29 smtpd
4684 postfix 20 0 6800 3164 1468 R 3.9 1.2 0:02.87 qmgr
5510 postfix 20 0 5476 1792 1460 D 2.6 0.7 0:00.18 cleanup
4354 syslog 20 0 1936 684 532 S 1.6 0.3 0:01.76 syslogd
4677 root 20 0 5396 1736 1408 S 1.4 0.7 0:01.01 master
top – 06:58:03 up 26 min, 2 users, load average: 2.29, 0.68, 0.33
Tasks: 155 total, 1 running, 154 sleeping, 0 stopped, 0 zombie
Cpu(s): 9.7%us, 16.8%sy, 0.0%ni, 25.1%id, 47.9%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 254368k total, 233476k used, 20892k free, 22192k buffers
Swap: 761848k total, 0k used, 761848k free, 120992k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4684 postfix 20 0 6800 3164 1468 S 6.0 1.2 0:03.29 qmgr
4677 root 20 0 5396 1736 1408 S 1.8 0.7 0:01.14 master
4354 syslog 20 0 1936 684 532 S 1.3 0.3 0:01.85 syslogd
Solution for I/O Issues
Solutions may be increasing the CPU or number of CPUs available on the mail server and also increasing the ability to read/write on the disks. Increasing read/write can be done by choosing SCSI or using RAID or even LVM.
SCSI
If you choose to select SCSI you have the advantages of the server being able to read/write to multiple disks at the same time. When selecting SCSI be sure to select a brand with a large cache to enhance your speed and also select the fastest speeds you can afford. Once you move to SCSI you can use hardware RAID or software RAID and use RAID 0 to increase read and writes. The biggest problem with RAID 0 is that it does not provide redundancy.
LVM Striping
Striped logical volumes will lay down data on a number of drives, speeding up the I/O process. When laying down each stripe the size of the stripe cannot exceed the size of the extent. The striping will start with the first physical volume and each stripe or extent is then placed on the next physical volume. The size of the stripe is limited to the size of the extent.
The LVM cannot determine if there are multiple physical volumes on the drive, so if you use striping on one disk with several physical volumes it will actually slow down performance instead of enhancing performance.
You must be logged in to post a comment.