Smarty

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

I am making it so that smarty exceptions are caught, and it sends an e-mail to the admin user for our software application.

After doing this, we've gotten reports of people getting the error in their e-mail but when they go to the page in question it works fine:

mohrt · Posted: Fri Jan 13, 2012 1:59 am Post subject:

That is exactly what it sounds like, a race condition. At the time one thread tries to read a template file (or compile one), another thread has removed it in preparation of a recompile. If many threads are attempting to recompile, this can cause all sorts of erratic behavior (and unnecessary recompile processing.)

We have done a few things to help with this problem, but it is not completely avoidable. With very high traffic sites we can only suggest you do not do "live" template recompilation. The live site should do no compile checking, while an internal site can do manual compile check/recompile to avoid the racing -- even that can cause a race because the live site will still attempt a compile if something is missing, so the best solution is go offline to recompile... which isn't always a practical solution either.

I know there was some internal discussion about aggressive race condition handling, I'll dig up where that ended. I know it's not easy to solve every situation adequately.

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

If it is a race condition, one question I have is why would something need to recompile the template? I thought once something was compiled, as long as the Smarty template engine is not updated, it doesn't need to be re-compiled after that?

I ask because the templates this error tends to happen on, are ones that are used on every page like footer.tpl or header.tpl, templates that should have already been compiled from previous page loads. That's the part where I mentioned I don't know enough about how the Smarty compilation works anymore to be able to come up with the "best" or most elegant solution, since I'm not sure why it is needing to re-compile it in the first place.

mohrt · Posted: Fri Jan 13, 2012 4:26 am Post subject:

Oh, I assumed you meant you change templates on occasion, and that would be when an error would show up. Smarty does not have to recompile if the source templates don't change. Smarty does however, check if a template needs compiling on each invocation by default. You are best off disabling compile_check with heavy traffic sites, that will skip a somewhat expensive and unnecessary step. See if that helps with the errors too. I'm not exactly sure why those would pop up. For whatever reason, PHP is having trouble reading those files in some circumstance of your environment.

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

mohrt · Posted: Fri Jan 13, 2012 5:04 pm Post subject:

Smarty compares the file times of the source files with the compiled files, and recompiles if the source is newer. is anything touching the source files? is there any sort of error or fluctuation with your system time clock?

double check the setting of the template_dir, compile_dir, clear out all your compiled/cached files. make sure you are using the latest Smarty of the major version number.

mohrt · Posted: Fri Jan 13, 2012 5:13 pm Post subject:

Do you have compile_check disabled? (you should anyways)

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

Doh! Did a debug_backtrace() on the method that writes the compiled template, and it is because we are using $smarty->getTags().

Apparently getTags() will always compile the template no matter what, are the tags on a template only "known" if the compiled template is being compiled at that time? In other words is this intentional, is there no other way to get the tags used on a template other than re-compiling it every time?

I'll play around with it and see if I can figure out a way to get it to work without re-compiling the template every time.

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

OK so turns out like I said getTags will compile the template, and that is because it keeps track of what tags are used by adding to an array in the compileTag() method.

Think it would be possible to make it so that if $template->tags_used is populated, it sets that info in the properties of the compiled template? So if $smarty->get_used_tags is true, it keeps track of the tags used during compilation, then also saves those tags in the compiled template parameters that is part of the compiled template... Doing that should not affect performance, since it would only be done if $smarty->get_used_tags is true.

Although, I don't like that solution either. I really didn't like the solution I did come up with, parsing through the used tags (even if it did NOT result in re-compile, obviously it is not ideal as that is a lot of extra overhead). Does anyone have a way we can accomplish the following the "best" way?

end result:

This is super-simplifed for demonstration purposes...

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

Just wanted to update, I was able to find a solution to do what I needed without using getTags(). But my use of getTags only highlighted the problem originally posted, for high-traffic sites there is a "common" race condition that can happen when another page-load is re-writing the same compiled file at almost same time.

A simple search for the error will reveal that this is happening for sites. One scenario in particular, is when a search robot is indexing or re-indexing pages on your site, it could reproduce the issue if it happens to go to pages that are not used very often and not compiled yet, then it ends up with pages on the site indexed with the error (like all the pages that come up when you do that search).

If it was a complex problem to fix I would agree it may not be worth addressing, just accepting that race conditions will always be possible, but in this case the fix is easy: if the inclusion of the compiled file fails, sleep for a tenth of a second or so then try again, maybe try it up to 3 times or something before giving up. Sure it is a rare problem and can easily be avoided by compiling all templates on a production site, but I would argue that BECAUSE it is so rare, and is so hard to see the error, it would result in search engine indexing the error on pages on a site and the site owner would never know, since going to the page in question does not pull up the error "after the fact", that is more reason to fix it rather than tell people a work around. Most that have this problem wouldn't know they had it, so would not know they needed to possibly seek a work around...

U.Tews · Posted: Mon Jan 23, 2012 11:16 pm Post subject:

Smarty3 does prevent concurrent compilations of same template when a template was updated.

I your case concurrent compilations have been forced by getTags() calls in production scripts which was not intended when proviging this function.

I will revisit the code to check if we could run into race conditions when a compiled template did not exists.

jonyo · Smarty Rookie Joined: 14 Feb 2011 Posts: 16

U.Tews · Posted: Tue Jan 24, 2012 7:48 pm Post subject:

This condition is interlocked.

Anyway I will recheck the different conditions of concurrent compilation requests.

ikund · Smarty Rookie Joined: 22 Nov 2011 Posts: 9

Hi,

I'm getting the same exceptions on another high-traffic site. Exceptions pop up even if nobody is updating any templates, the timestamp for cache files is not updated (which would normally cause the recompile). The problem involves (mostly) templates that are used more (banner boxes etc).

However, I am not using getTags. Any ideas? Oh and the exceptions started appearing after Smarty upgrade from 3.1.5 to 3.1.10.

Thanks, I would really appreciate help with this matter.