Smarty

[moneysack]

Hi to all.

I having a problem with a massive performance impact on the initial template compiling with Smarty3 beta 6 trough Smarty3 beta 8 svn.
Other Smarty3 Versions not tested.

Maybe anyone here can help or give me a hint to find the problem.

First of all, a Smarty-Template is coded as following:
include header.tpl
- config_file
- include js_head.tpl
own template code
include footer.tpl
- include js_foot.tpl

The header-Template includes a config_file at the Beginning with some Language-Vars (round-about 240 KB / 5000 Rows). And both files include another Template (js_head.tpl in Header an js_foot.tpl in Footer).

On the initial compiling of the Templates the Apache/PHP Process need around 10 Seconds to Display the Result.

After the first call of the Template, the Page is lightning fast.

When i update the config-files and Smarty has to recompile it, it is slow again. When i comment out the Config-File in the Header.tpl, the initial compile-Time is a little bit faster (round-about 5 Seconds).

The Problem is not the compile-time itself directly, the problem is that i have a High-Load Website with approx. 4 Million Page-Impressions and when Smarty needs to recompile something, all Apache Processes locks each other because of File-Locking or something. In that case no Files are served for about 30 seconds up to 5 Minutes.

I don't use any special Smarty-Method - only the assign Method.

I have a productive site with mainly the same templates and Config-Files except the Smarty-Version (Version 2) and PHP-Version (5.2.x).

Here some other Details to the installed Software:

Freebsd 8.0-RELEASE-p2
Apache 2.2.15
PHP 5.3.2 with libxml, SPL, curl, memcache, eAccelerator, ADOdb-Ext, PCRE, mysql, xdebug (only for development)

Maybe i have forgotten some Extensions for use with Smarty3?

If any further details are needed, just say it.

Thanks for any help

[U.Tews]

Indeed compiling is in Smarty3 significant slower as in Smarty2 because of a lexer/parser which makes syntax definitions and maintance much more easier as in SMarty2.

As you said normally this does not really matter as compiling is a onetime only process after modifications.

I think what happens is that you are getting too many request for a modified page during the time until the first request has finished compiling. So the compiling gets triggered many times in parallel.

If your system is so havily loaded and the template are normally stable you could think about having $smarty->compile_check = false; to disable automatic compiling. This will also increase performance for all pages loads.

After you have modified your templates you could use the $smarty->untility-compileAllTemplates() method to get all modifcations compiled in one thread.

[mohrt]

With a very high traffic site, you are running into a race condition when template(s) need recompiled. You could have hundreds of threads (needlessly) trying to make the recompile happen. I would suggest one of the following:

1) bring offline, run $smarty->utility->compileAllTemplates(), bring online

2) copy new templates AND compiled templates from an identical development server so they won't need recompiled. Be sure file datestamps are preserved.

3) a custom code solution with a file lock. This would be a much more complicated approach, as it introduces possible deadlocks.

[mohrt]

I like U.Tews idea. disable compile_check, then manually compile files with compileAllTemplates, or have a special case that enables compile_check and loads the page.

[moneysack]

i think this is the way to go for the future.

Is there a method for compiling all Config-Files manually, or is this done by compileAllTemplates too?

Are the old compiled Templates deleted after compileAllTemplates has finished or before? And what is, if a Client requests a Template that is deleted because of the run of compileAllTemplates?
For Example the header.tpl, which is included in all other Templates.

Thanks for your replies.

[U.Tews]

Good point. The config files are currently not covered by compileAllTemplates(). I will extend the utility class for this in the near future.

compileAllTemplates() will replace the compiled template file whenever the compilation of a single template file is finished. But you can also set compile_dir to a different folder and copy all compiled templates at a later time.

[moneysack]

Allright...

I've done some Code-Engineering and found out that the Method compileAllTemplates() is checking all Templates, and if it find a modified template it begin to recompile.
But the recompiled template isn't saved because of using the Method Smarty_Internal_Template::getCompiledTemplate() called from line 85 in Smarty_Internal_Utilitys.

I think the Method Smarty_Internal_Template::compileTemplateSource() has to be called instead, to compile and save the Template.

Setting the $force_compile to true in Method-Call will build up all Templates, what isn't what i want.

Is this a failure, or is this wanted?

[b]EDIT:[/b]
On my System the Templates do not recompile. I have to set compile_check to true in the utility - Then all modified templates were recompiled and saved, but under another Filename. Why?!

[U.Tews]

The call of getCompiledTemplate() is correct. If force_compile is false it compiles and saves just the modified templates.

A call of compileTemplateSource() will always compile independing of the force_compile setting.

Note that the fullpath of the template source is encoded in the compiled filenames. This is required to handle templates with identical filenames but different folders. If you run compileAllTemplates() from a different source folder fullpath you can't copy them to the target system because different filenames.

[U.Tews]

I have changed the log output of compileAllTemplates() to display which templates have been found up to date and which have been compiled.

This is in the SVN now.

[U.Tews]

A new method utility->compileAllConfig() has been added to the SVN to compile all config files.

[rudder]

Sorry if this is offtopic, but what would be a few recommended scenarios in situations of load balanced servers each which have their independent templates_c folders?

In our situation we will have multiple servers that need templates compiled, and we push template changes multiple times per day.

It seem impractical to have to go into each server to run this utility, especially given our need to potentially push templates frequently in short bursts.

I wonder if the following is practical:

• push files to deployment server
  
• Run compileAllTemplate() utility
  
• rsync templates + templates_c to production servers
  

Previously our deployment method using Smarty2 is to:

• push files to deployment server
  
• rsync templates to production server, but EXCLUDE templates_c and let each serve recompile.
  

[moneysack]

@rudder:

With Smarty2 i used the same method as you.

With the complex Lexer/Parser of Smarty3, on a High-Load Website, this is not a good Idea.

I think the best is, to share the whole Web-Directory (with templates_c etc.) with NFS or something. Now you can run the Method compileAllTemplates() from any server.
The only Problem is that you need more Redundancy and maybe some sort of High-Availability Software.

@U.Tews:
I will checkout the SVN soon.
Whats with the fullpath-thing?

When no compiled Template is available and i request the Site as usual (example: index.php), then the fullpath is another as when i call another File to recompile all Templates (example: mgmt/recompileall.php)?

And thanks for programming the new utility Method

[Lemon Juice]

[mohrt]

[mohrt]