Smarty Forum Index Smarty
The discussions here are for Smarty, a template engine for the PHP programming language.

My speed test on a real site: smarty 3 vs 2

 
Post new topic   Reply to topic    Smarty Forum Index -> Smarty 3
View previous topic :: View next topic  
Author Message
Lemon Juice
Smarty Pro


Joined: 24 May 2006
Posts: 109

PostPosted: Mon Feb 08, 2010 12:29 pm    Post subject: My speed test on a real site: smarty 3 vs 2 Reply with quote

I was curious how the new version performs in comparison to Smarty 2 in a real-life situation so I did my own tests on a shop site. Below I will give URLs of the pages I tested along with times in seconds. While the URLs represent the real pages, I performed my tests on my local computer, which is a 5-year old Athlon XP 3000+ with 1.25GB RAM on Win XP SP3. I don't think I would be able to do it well on the live server which is some flavour of Unix. Here are some test parameters to be aware of:

1. I tested two sites and the only difference between one and the other was the version of Smarty used. One was ver. 2.6.19 and the other one ver. 3 revision 3467 from 2010-01-25. Today I ran some tests on the newest revision from 2010-02-07 but the results were almost identical.

2. PHP 5.2.4 as Apache module. The columns with "EA" refer to tests with eAccellerator enabled (ver. 0.9.5)

3. Time was measured from PHP itself from the beginning of a script (everything goes through a common index.php) to the very end of it. Time spent sending data to the browser was not taken into account because ob_start('gz_handler') was used, so the sending of data occurred after all page generation. "All page generation" include all scripts that generate a single HTML page together with database access. No SQL cache was used.

4. Caching was off and compile check was on. The only cached portion of the site was the left menu with links, the caching was done by an external utility - we can assume the whole menu was injected to smarty as plain html data from a cache file.

5. Each page was reloaded 10 times and the average is given below. Loading each page for the first time was not measured (it is definitely longer since files and other data are not yet cached by the system and PHP).

6. At the bottom I presented memory usage results (for the home page only). They seem to be consistent for each page request. I don't know how much we can rely on these PHP functions but I present them nevertheless. These functions were called at the end of the script.

Ok, so here are the results:
Code:

         http://www.gsm-support.net/

         Smarty 2     Smarty 3       Smarty 2 EA  Smarty 3 EA
         0.475        0.232          0.473        0.188
         0.466        0.232          0.432        0.185
         0.430        0.234          0.481        0.182
         0.352        0.219          0.558        0.181
         0.341        0.231          0.550        0.174
         0.416        0.230          0.523        0.184
         0.466        0.231          0.509        0.189
         0.471        0.230          0.503        0.180
         0.277        0.228          0.480        0.162
         0.406        0.229          0.583        0.186

average: 0.410        0.230          0.509        0.181

         ------------------------------------------
         http://www.gsm-support.net/en/offer

         Smarty 2     Smarty 3       Smarty 2 EA  Smarty 3 EA
         0.570        0.364          0.559        0.318
         0.504        0.358          0.633        0.318
         0.610        0.363          0.652        0.317
         0.516        0.388          0.569        0.295
         0.651        0.361          0.567        0.315
         0.571        0.360          0.546        0.321
         0.410        0.364          0.522        0.316
         0.674        0.363          0.656        0.325
         0.612        0.363          0.578        0.319
         0.518        0.366          0.858        0.311

average: 0.564        0.365          0.614        0.316

         ------------------------------------------
         http://www.gsm-support.net/en/mt-box-all-in-one-p1527

         Smarty 2     Smarty 3      Smarty 2 EA  Smarty 3 EA
         0.384        0.184         0.183        0.126
         0.406        0.162         0.326        0.126
         0.399        0.177         0.453        0.115
         0.331        0.178         0.389        0.134
         0.400        0.177         0.363        0.130
         0.296        0.177         0.292        0.127
         0.378        0.182         0.316        0.125
         0.275        0.179         0.290        0.130
         0.290        0.180         0.411        0.161
         0.340        0.180         0.256        0.128

average: 0.350        0.178         0.328        0.130

------------------------------------------
Memory usage: http://www.gsm-support.net/

                              Smarty 2     Smarty 3     Smarty 3 EA
memory_get_usage(true)        2816 KB      3328 KB      1536 KB
memory_get_peak_usage(true)   3072 KB      3328 KB      2048 KB


Conclusion:

It is clear that Smarty 3 is much faster than ver. 2. In spite of the fact that in Smarty 3 each variable is an object and there are some magic methods used (which are inherently slow) in the internals, the overall peformance is better.

The interesting this is that Smarty 2 didn't become really faster with eAccelerator and sometimes it was even slower! Smarty 3 clearly benefits from an opcode cache. Maybe a different opcode cache would yield different results.

Memory usage was marginally higher in Smarty 3 but I suppose it is well compensated by the speed increase. However, I don't know how much these numbers are accurate.

Overall Smarty 3 seems to be faster. Good work!
Back to top
View user's profile Send private message
mohrt
Administrator


Joined: 16 Apr 2003
Posts: 7366
Location: Lincoln Nebraska, USA

PostPosted: Mon Feb 08, 2010 2:09 pm    Post subject: Reply with quote

FYI the latest SVN code sweeps have eliminated much of the magic methods.
Back to top
View user's profile Send private message Visit poster's website
Lemon Juice
Smarty Pro


Joined: 24 May 2006
Posts: 109

PostPosted: Mon Feb 08, 2010 5:27 pm    Post subject: Reply with quote

Yes, I saw there fewer lazy loaded methods but the __call method itself became heavier instead. Anyway, it looks like this is negligible in a complex site like this because calling a magic method 2 or 3 times during the script execution is nothing compared to all the rest of the code needed to run the site, so I think we shouldn't be worried about performance here. But the other reason I am not fond of too much magic is because in IDE editors you can't quickly navigate to a given method because the method doesn't really exist, it's hidden behind some magic. It is useful to be able to quickly jump to a method to see it's implementation, parameters, docs in comments, etc. instead of having to search documentation.
Back to top
View user's profile Send private message
mohrt
Administrator


Joined: 16 Apr 2003
Posts: 7366
Location: Lincoln Nebraska, USA

PostPosted: Mon Feb 08, 2010 5:36 pm    Post subject: Reply with quote

At this point, the only thing __call() is used for is magic setter/getter methods and the Smarty 2 BC wrapper. I think using __call() for magic setters/getters is a fair compromise to hard coding all of these in the object. The overhead is negligible, and they all perform the same tasks.
Back to top
View user's profile Send private message Visit poster's website
falco
Smarty n00b


Joined: 17 Feb 2010
Posts: 3

PostPosted: Wed Feb 17, 2010 8:53 pm    Post subject: Reply with quote

For what it's worth, I dropped the latest Smarty3-b7 on my production server and ran some quick benchmarks. The server is already optimised for highest performance (services around 2-300 apache hits per second), so all caching is enabled.

Smarty cache was cleaned between version shifts and the target page was loaded once to rebuild the smarty cached version of that page. Therefore please note that this is a benchmark of fully cached performance, not compile times.

The target page was chosen as it is likely one of the most complex templates that is used on the site. Contains a few loops over datasets and conditionals inside those loops, as well as an included header and footer.

php: PHP 5.3.1-4 with Suhosin-Patch (cli) (built: Feb 11 2010 17:23:59)
EA: eAccelerator v0.9.6

ab -kc1 -t100 (1 thread running over 100 seconds)

Smarty 2.6.16
Code:
Requests per second:    66.81 [#/sec] (mean)
Time per request:       14.968 [ms] (mean)
Time per request:       14.968 [ms] (mean, across all concurrent requests)


Smarty 3b7
Code:
Requests per second:    63.46 [#/sec] (mean)
Time per request:       15.758 [ms] (mean)
Time per request:       15.758 [ms] (mean, across all concurrent requests)


This makes it look like there was a marginal drop in performance, but running a single-thread benchmark isn't very thorough (and the 0.8ms difference is almost not worth noting).

ab -kc5 -t100 (5 threads running over 100 seconds)

Smarty 2.6.16
Code:
Requests per second:    102.49 [#/sec] (mean)
Time per request:       48.786 [ms] (mean)
Time per request:       9.757 [ms] (mean, across all concurrent requests)


Smarty 3b7
Code:
Requests per second:    159.41 [#/sec] (mean)
Time per request:       31.365 [ms] (mean)
Time per request:       6.273 [ms] (mean, across all concurrent requests)


Okay, wow... what happened there? Because of the huge jump in results I thought it could be some cron'd task kicking in, but I ran this same benchmark 10 times on each version with results varying <5%.

I also reran the single-threaded approach and got very similar results. I am not sure of the reasoning behind this (some kind of file lock contention? better use of shared memory?), but these concurrent thread improvements are quite amazing.

If there are any questions or more specific tests you would like run, I will have both versions sitting around for the next while. Am tempted to give SVN a shot considering above conversation.
Back to top
View user's profile Send private message
Aryos
Smarty Rookie


Joined: 12 Jan 2010
Posts: 8

PostPosted: Thu Feb 18, 2010 2:29 pm    Post subject: Reply with quote

I can verify your results. After testing with complex code and tens of files, I have seen a decrease of about 35% up to 50% in memory usage on the latest SVN version, compared to the last beta (beta 7) released, which is is quite impressive.
Execution time benefits, compared to Smarty 2 range from 5% to 15% in the best case.

Do we have an estimated ETA for this release. After going through the code it seems mature enough now, perhaps there's still some space for tweaking for more speed and on security but I have not noticed any worth-mentioned bugs so far. An update from the developers would be nice to have.
Back to top
View user's profile Send private message
mohrt
Administrator


Joined: 16 Apr 2003
Posts: 7366
Location: Lincoln Nebraska, USA

PostPosted: Thu Feb 18, 2010 3:45 pm    Post subject: Reply with quote

No ETA yet, we are still doing some code optimization/organization changes. We figured its best to get this complete before making an official release.
Back to top
View user's profile Send private message Visit poster's website
chefuri
Smarty n00b


Joined: 09 May 2010
Posts: 3

PostPosted: Sun May 09, 2010 7:08 pm    Post subject: Reply with quote

Hello. In my case the results was bad.

ab -kc1 -t100 (1 thread running over 100 seconds)

Smarty 2.6.16
Code:
Requests per second: 73.85 [#/sec] (mean)
Time per request: 13.540 [ms] (mean)
Time per request: 13.540 [ms] (mean, across all concurrent requests)


Smarty 3 rc1
Code:
Requests per second: 49.09 [#/sec] (mean)
Time per request: 20.371 [ms] (mean)
Time per request: 20.371 [ms] (mean, across all concurrent requests)

The code is the same (only change is_cached by isCached). And with this options:

caching = 1
compile_check = false

I use this fuctions:

Code:

$this->tpl = new Smarty();
$this->tpl->display();
$this->tpl->isCached();
$this->tpl->assign();




If you want more tests or voluntee for another test, please contact me. Thanks
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Smarty Forum Index -> Smarty 3 All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP