<div dir="ltr">I am continuing to do testing as time allows. Lat night I ran sysbench fileio tests, and again, the ploop CT yielded better performance then either then simfs CT or the vzhost. It wasn't as drastic a difference as the dbench results, but the difference was there. I'll continue in this vein with freshly created CTs. The machine was just built a few days ago, it's quiescent, it's doing nothing except hosting a few vanilla CTs.<div>
<br></div><div>As for the rules of thumb, I can tell you that the results are 100% repeatable. But explainable, ah, that's the thing. still working on that.<div><br></div><div>Regards,</div><div><br></div><div>J J<br>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, May 3, 2014 at 11:31 AM, Kir Kolyshkin <span dir="ltr"><<a href="mailto:kir@openvz.org" target="_blank">kir@openvz.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div class="">
<div>On 05/02/2014 04:38 PM, jjs -
mainphrame wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks Kir, the /dev/zero makes sense I suppose. I
tried with /dev/random but that blocks pretty quickly -
/dev/urandom is better, but still seems to be a bottleneck.</div>
</blockquote>
<br></div>
You can use a real file on tmpfs.<br>
<br>
Also, in general, there are very many factors that influence test
results. Starting from the cron jobs and other stuff (say, network
activity) that runs periodically or sporadically and spoils your
results, to the cache state (you need to use vm_drop_caches, or yet
better, reboot between tests), to the physical place on disk where
your data is placed (rotating hdds tend to be faster at the first
sectors compared to the last sectors, so ideally you need to do this
on a clean freshly formatted filesystem). There is much more to it,
can be some other factors, too. The rule of thumb is results need to
be reproducible and explainable. <br><span class="HOEnZb"><font color="#888888">
<br>
Kir.</font></span><div><div class="h5"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>As for the dbench results, I'd love to hear what results
others obtain from the same test, and/or any other testing
approaches that would give a more "acceptable" answer.</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>J J<br>
<div><br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Fri, May 2, 2014 at 4:01 PM, Kir
Kolyshkin <span dir="ltr"><<a href="mailto:kir@openvz.org" target="_blank">kir@openvz.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div>On 05/02/2014 03:00 PM, jjs - mainphrame wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Just for kicks, here are the data from
the tests. (these were run on a rather modest old
machine)
<div><br>
</div>
<div><img src="cid:part2.06030901.05090909@openvz.org" width="563" height="429.9113530326594"></div>
<div><br>
</div>
<div><br>
</div>
<div>Here are the raw dbench data:</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div><font face="courier new, monospace">#clients
vzhost simfs CT
ploop CT</font></div>
<div><font face="courier new, monospace">---------------------------------------------------------------------</font></div>
<div><font face="courier new, monospace">1
11.1297MB/sec 9.96657MB/sec
19.7214MB/sec</font></div>
<div><font face="courier new, monospace">2
12.2936MB/sec 14.3138MB/sec
23.5628MB/sec</font></div>
<div><font face="courier new, monospace">4
17.8909MB/sec 16.0859MB/sec
45.1936MB/sec</font></div>
<div><font face="courier new, monospace">8
25.8332MB/sec 22.9195MB/sec
84.2607MB/sec</font></div>
<div><font face="courier new, monospace">16
32.1436MB/sec 28.921MB/sec
155.207MB/sec</font></div>
<div><font face="courier new, monospace">32
35.5809MB/sec 32.1429MB/sec
206.571MB/sec</font></div>
<div><font face="courier new, monospace">64
34.3609MB/sec 29.9307MB/sec
221.119MB/sec</font></div>
</div>
</div>
</blockquote>
<br>
</div>
Well, I can't explain this, but there's probably something
wrong with the test.
<div>
<div><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div><br>
</div>
<div>Here is the script used to invoke dbench:</div>
<div><br>
</div>
<div>
<div><font face="courier new, monospace">HOST=`uname
-n`</font></div>
<div><font face="courier new, monospace">WD=/tmp</font></div>
<div> <font face="courier new, monospace">FILE=/usr/share/dbench/client.txt</font></div>
<div><font face="courier new, monospace"><br>
</font></div>
<div><font face="courier new, monospace">for i
in 1 2 4 8 16 32 64</font></div>
<div> <font face="courier new, monospace">do</font></div>
<div><font face="courier new, monospace">
dbench -D $WD -c $FILE $i
&>dbench-${HOST}-${i}</font></div>
<div><font face="courier new, monospace">done</font></div>
</div>
<div><br>
</div>
<div>Here are the dd commands and outputs:</div>
<div><br>
</div>
<div>
<div><font face="courier new, monospace">OPENVZ
HOST</font></div>
<div><font face="courier new, monospace">----------------</font></div>
<div><font face="courier new, monospace">[root@vzhost
~]# dd bs=1M count=512 if=/dev/zero
of=test conv=fdatasync</font></div>
<div><font face="courier new, monospace">512+0
records in</font></div>
<div><font face="courier new, monospace">512+0
records out</font></div>
<div><font face="courier new, monospace">536870912
bytes (537 MB) copied, 11.813 s, 45.4 MB/s</font></div>
<div><font face="courier new, monospace">[root@vzhost
~]# df -T</font></div>
<div><font face="courier new, monospace">Filesystem
Type 1K-blocks Used Available Use%
Mounted on</font></div>
<div><font face="courier new, monospace">/dev/sda2
ext4 20642428 2390620 17203232
13% /</font></div>
<div><font face="courier new, monospace">tmpfs
tmpfs 952008 0 952008
0% /dev/shm</font></div>
<div><font face="courier new, monospace">/dev/sda1
ext2 482922 68436 389552
15% /boot</font></div>
<div><font face="courier new, monospace">/dev/sda4
ext4 51633780 3631524 45379332
8% /local</font></div>
<div><font face="courier new, monospace">[root@vzhost
~]# </font></div>
<div><font face="courier new, monospace"><br>
</font></div>
<div><font face="courier new, monospace"><br>
</font></div>
<div><font face="courier new, monospace">PLOOP
CT</font></div>
<div><font face="courier new, monospace">----------------</font></div>
<div><font face="courier new, monospace">root@vz101:~#
dd bs=1M count=512 if=/dev/zero of=test
conv=fdatasync</font></div>
<div><font face="courier new, monospace">512+0
records in</font></div>
<div><font face="courier new, monospace">512+0
records out</font></div>
<div><font face="courier new, monospace">536870912
bytes (537 MB) copied, 2.50071 s, 215 MB/s</font></div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
This one I can explain :)<br>
<br>
This is caused by ploop optimization that was enabled in
the kernel recently.<br>
If data block is all zeroes, it is not written to the disk
(same thing as sparse files,<br>
just for ploop).<br>
<br>
So you need to test it with some real data (anything but
not all zeroes).<br>
I am not sure how fast is /dev/urandom but this is one of
the options.<br>
<br>
<blockquote type="cite">
<div>
<div>
<div dir="ltr">
<div>
<div>
<div><font face="courier new, monospace"><br>
</font></div>
<div><font face="courier new, monospace">root@vz101:~#
df -T</font></div>
<div><font face="courier new, monospace">Filesystem
Type 1K-blocks Used
Available Use% Mounted on</font></div>
<div><font face="courier new, monospace">/dev/ploop11054p1
ext4 4539600 1529316 2804928 36%
/</font></div>
<div><font face="courier new, monospace">none
devtmpfs 262144 4
262140 1% /dev</font></div>
<div><font face="courier new, monospace">none
tmpfs 52432 52
52380 1% /run</font></div>
<div><font face="courier new, monospace">none
tmpfs 5120 0
5120 0% /run/lock</font></div>
<div><font face="courier new, monospace">none
tmpfs 262144 0
262144 0% /run/shm</font></div>
<div><font face="courier new, monospace">root@vz101:~#</font></div>
<div><font face="courier new, monospace"><br>
</font></div>
<div><font face="courier new, monospace"><br>
</font></div>
<div><font face="courier new, monospace">SIMFS
CT</font></div>
<div><font face="courier new, monospace">----------------</font></div>
<div><font face="courier new, monospace">root@vz102:~#
dd bs=1M count=512 if=/dev/zero of=test
conv=fdatasync</font></div>
<div><font face="courier new, monospace">512+0
records in</font></div>
<div><font face="courier new, monospace">512+0
records out</font></div>
<div><font face="courier new, monospace">536870912
bytes (537 MB) copied, 12.6913 s, 42.3
MB/s</font></div>
<div><font face="courier new, monospace">root@vz102:~#
df -T</font></div>
<div><font face="courier new, monospace">Filesystem
Type 1K-blocks Used Available
Use% Mounted on</font></div>
<div><font face="courier new, monospace">/dev/simfs
simfs 4194304 1365500 2828804
33% /</font></div>
<div><font face="courier new, monospace">none
devtmpfs 262144 4
262140 1% /dev</font></div>
<div><font face="courier new, monospace">none
tmpfs 52432 52
52380 1% /run</font></div>
<div><font face="courier new, monospace">none
tmpfs 5120 0
5120 0% /run/lock</font></div>
<div><font face="courier new, monospace">none
tmpfs 262144 0
262144 0% /run/shm</font></div>
<div><font face="courier new, monospace">root@vz102:~# </font></div>
</div>
<div><font face="courier new, monospace"><br>
</font></div>
<div><font face="arial, helvetica, sans-serif">Regards,</font></div>
<div><font face="arial, helvetica, sans-serif"><br>
</font></div>
<div><font face="arial, helvetica, sans-serif">J
J</font></div>
<div><font face="arial, helvetica, sans-serif"><br>
</font></div>
<font face="arial, helvetica, sans-serif"></font><br>
</div>
</div>
<div class="gmail_extra"> <br>
<br>
<div class="gmail_quote">On Fri, May 2, 2014 at
2:10 PM, jjs - mainphrame <span dir="ltr"><<a href="mailto:jjs@mainphrame.com" target="_blank">jjs@mainphrame.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">You know the saying, "when
something seems too good to be true"...
<div><br>
</div>
<div>I just installed centos 6.5 and openvz
on an older machine, and when I built an
ubuntu 12.04 CT I noticed that ploop is
now the default layout. Cool. So I built
another ubuntu12.04 CT, identical in every
way except that I specified smifs, so I
could do a quick performance comparison.</div>
<div><br>
</div>
<div>First I did a quick timed dd run, then
I ran dbench with varying numbers of
clients.</div>
<div><br>
</div>
<div>The simfs CT showed performance roughly
similar to the host, which was not too
surprising.</div>
<div>What did surprise me was that the ploop
CT showed performance which was
significantly better than the host, in
both the dd test and the dbench tests.</div>
<div><br>
</div>
<div>I know someone will tell me "dbench is
a terrible benchmark" but it's also a
standard. Of course, if anyone knows a
"better" benchmark, I'd love to try it.</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>J J</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
Users mailing list
<a href="mailto:Users@openvz.org" target="_blank">Users@openvz.org</a>
<a href="https://lists.openvz.org/mailman/listinfo/users" target="_blank">https://lists.openvz.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@openvz.org" target="_blank">Users@openvz.org</a><br>
<a href="https://lists.openvz.org/mailman/listinfo/users" target="_blank">https://lists.openvz.org/mailman/listinfo/users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Users mailing list
<a href="mailto:Users@openvz.org" target="_blank">Users@openvz.org</a>
<a href="https://lists.openvz.org/mailman/listinfo/users" target="_blank">https://lists.openvz.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@openvz.org">Users@openvz.org</a><br>
<a href="https://lists.openvz.org/mailman/listinfo/users" target="_blank">https://lists.openvz.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div></div></div></div>