View previous topic :: View next topic |
Author |
Message |
andre Smarty Pro
Joined: 23 Apr 2003 Posts: 164 Location: Karlsruhe, Germany
|
Posted: Wed Nov 05, 2003 4:16 pm Post subject: |
|
|
Short question: Anybody heard something about Turck MMCache and it's API functions mmcache_put() / mmcache_get() ?
Do you expect a performance gain if the cache files will be stored in shared memory (yes you heard right) instead of file system? If yes, the above "solution" for restricting the cache dir size could help so the shared memory doesn't run full. And if yes, I'll start to write my own custome cache handler function for using mmcache instead of writing to file system. The only two disadvantages I can think of in the moment are:
1) You need a hugh shared memory for a hugh cache dir
2) After restarting Apache server all cache files will be lost
Any oppinions? |
|
Back to top |
|
atu Smarty Regular
Joined: 18 Apr 2003 Posts: 51 Location: Luxembourg
|
Posted: Wed Nov 05, 2003 6:01 pm Post subject: |
|
|
Quote: |
messju wrote:
all this stuff (your code and my fragment above) is only a workaround of wrong usage of cache-ids. you shouldn't generate cache_ids based on user-input in the first place, because this leads to an unpredictable amount of cache-ids (and thus an unpredictable/infinite cache-size)
|
Yes thats true. This is only a workarround. The proper way is to check if the user input is what it seems to be. But I'm searching for a simple and fast solution that I have not yet found, which lead me to next suggestion about caching:
See:http://www.phpinsider.com/smarty-forum/viewtopic.php?t=1343
May someone have a workaround on what is mentioned there? |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Wed Nov 05, 2003 6:56 pm Post subject: |
|
|
andre wrote: | messju wrote: | a custom cache_handler_func would be the right place for tracking cache-size and clearing on demant, maybe |
Problem is that the clear_cache() function doesn't forward the expiration time as parameter to the custom cache handler function. After thinking some time about this it seems this is a bug / missing feature in Smarty. |
I'm not sure that is right. Last night I was thinking about this issue and it also occurred to me that the cache interface was the right place to implement this. I then looked at existing cache APIs like mmcache's and the ttl is almost universally passed at cache object creation time. That means that the cache handler is "taking care of business" and tracking object lifetime. I was looking at mmcache for this because 1) it can use ram and/or disk, 2) it is highly configurable, 3) supported on multiple platforms, 4) it is self-managing, 5) it looks like it is easily adaptable as a cache manager for Smarty -- I think someone may have done this already ?
EDIT: heh, looks like I wrote my post before seeing Andre's last message. Andre: let's do one!
1) I believe that you can spill to file storage as required with mmcache--have to check -- but that is okay anyhow if you have enough ram
2) yes. I was wondering if using the mmcache API, it would be possible to dump the current cache to disk on shutdown so that it could be warm started when it is brought back up.
EDIT: andre, after looking at how the caching is actually implemented, I think I _finally_ see what you mean about the ttl issue. hmmm. |
|
Back to top |
|
andre Smarty Pro
Joined: 23 Apr 2003 Posts: 164 Location: Karlsruhe, Germany
|
Posted: Thu Nov 06, 2003 10:31 am Post subject: |
|
|
Here's an idea of the mmcache handler function:
[php:1:f6aa71fb9d]<?php
/**
* Helper Function
*/
function _mmcache_clear_cache(&$hierarchie) {
foreach ($hierarchie as $key => $value) {
if (is_array($value)) {
_mmcache_clear_cache($value);
}
else {
mmcache_lock($value);
mmcache_rm($value);
mmcache_unlock($value);
}
}
}
function smarty_cache_mmcache($action, &$smarty, &$cache_content, $tpl_file=null, $cache_id=null, $compile_id=null, $exp_time=null)
{
if(!function_exists("mmcache")) {
$smarty->_trigger_error_msg("cache_handler: Turck MMCache not installed");
return false;
}
// create unique cache id
$_auto_id = $smarty->_get_auto_id($cache_id, $compile_id);
$_cache_file = substr($smarty->_get_auto_filename(".", $tpl_file, $_auto_id),2);
$mmcache_id = "smarty_mmcache|".$_cache_file;
// the index contains all stored cache ids and can be iterated
$mmcache_index_id = "smarty_mmcache_index";
switch ($action) {
case 'read':
// read cache from shared memory
$cache_content = mmcache_get($mmcache_id);
$return = true;
break;
case 'write':
// save cache to shared memory
$current_time = time();
if (is_null($exp_time) || $exp_time < $current_time)
$ttl = 0;
else
$ttl = $exp_time - time();
// First run garbage collection
mmcache_gc();
// Put content into cache
mmcache_lock($mmcache_id);
mmcache_put($mmcache_id, $cache_content, $ttl);
// Create an index association
mmcache_lock($mmcache_index_id);
$mmcache_index = mmcache_get($mmcache_index_id);
if (!is_array($mmcache_index))
$mmcache_index = array();
$indexes = explode(DIRECTORY_SEPARATOR, $_cache_file);
$_pointer =& $mmcache_index;
foreach ($indexes as $index) {
if (!isset($_pointer[$index]))
$_pointer[$index] = array();
$_pointer =& $_pointer[$index];
}
$_pointer = $mmcache_id;
mmcache_put($mmcache_index_id, $mmcache_index, 0);
mmcache_unlock($mmcache_index_id);
mmcache_unlock($mmcache_id);
break;
case 'clear':
// clear cache info
mmcache_lock($mmcache_index_id);
$mmcache_index = mmcache_get($mmcache_index_id);
if (is_array($mmcache_index)) {
if (empty($cache_id) && empty($compile_id) && empty($tpl_file)) {
// clear all cache
mmcache_lock($mmcache_id);
_mmcache_clear_cache($mmcache_index);
mmcache_unlock($mmcache_id);
$mmcache_index = array();
}
else {
// clear single file or cache group
$indexes = explode(DIRECTORY_SEPARATOR, $_cache_file);
if (is_null($tpl_file))
array_pop($indexes);
$_pointer =& $mmcache_index;
$_failed = false;
foreach ($indexes as $index) {
if (!isset($_pointer[$index])) {
$_failed = true;
break;
}
$_pointer =& $_pointer[$index];
}
if (!$_failed) {
if (is_array($_pointer)) {
// Clear cache group
_mmcache_clear_cache($_pointer);
}
else {
// Clear single file
mmcache_lock($_pointer);
mmcache_rm($_pointer);
mmcache_unlock($_pointer);
}
$_pointer = null;
}
}
}
mmcache_put($mmcache_index_id, $mmcache_index, 0);
mmcache_unlock($mmcache_index_id);
$return = true;
break;
default:
// error, unknown action
$smarty->_trigger_error_msg("cache_handler: unknown action \"$action\"");
$return = false;
break;
}
return $return;
}
?>[/php:1:f6aa71fb9d]
You will notice the $exp_time var at the end of the parameter list. It isn't used yet because Smarty doesn't support it yet.
The handler works for my developing system but isn't fully tested very well. Especially the cache group handling is a bit tricky. This is why I created an array to remember $cache_id / $compile_id / $tpl_name <-> $mmcache_id relationship. Otherwise it won't be possible to know which cached content belongs to which cache group.
Also I'm using Smarty::_get_auto_id() and Smarty::get_auto_filename() to be as much compatible as possible to the default cache handling.
Last edited by andre on Mon Nov 10, 2003 7:39 am; edited 4 times in total |
|
Back to top |
|
andre Smarty Pro
Joined: 23 Apr 2003 Posts: 164 Location: Karlsruhe, Germany
|
Posted: Thu Nov 06, 2003 4:03 pm Post subject: |
|
|
I've just updated the script so clearing cache groups now works correctly.
Ideas are welcome |
|
Back to top |
|
messju Administrator
Joined: 16 Apr 2003 Posts: 3336 Location: Oldenburg, Germany
|
Posted: Thu Nov 06, 2003 4:56 pm Post subject: |
|
|
a little off topic, sorry. FYI: i added the $exp_time-parameter to the calls to cache_handler_func() in the CVS-version of smarty. it should be fully backwards-compatible to existing usage of cache_handler_func. |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Fri Nov 07, 2003 7:25 am Post subject: |
|
|
andre: ahh, you are fast! I will have to try your mmcache based cache-handler and let you know Looks like you got the most interesting things in there, though. Have you tried timing? I would eventually want to test that against a dedicated content cache (squid??), but having programmable content caching from within the application itself is very nice
messju: ahh, very nice |
|
Back to top |
|
andre Smarty Pro
Joined: 23 Apr 2003 Posts: 164 Location: Karlsruhe, Germany
|
Posted: Fri Nov 07, 2003 8:21 am Post subject: |
|
|
@boots:
I didn't tested timing yet. But as I can see my pages are retrieved about 20% faster than before.
BTW: I have updated the script once again with some minor fixes. |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Sat Nov 08, 2003 4:55 am Post subject: |
|
|
Hi again andre. I did some initial playing with your code--very nice work! I tried it against messju's latest cvs update and it worked almost flawlessly (just some notices--see below).
- from the very limitied samples I tried, I did not see a marked speed improvement, if any at all--but I suspect that are certain conditions that could very much favour this solution. I need to test more
- I like that this quietly satisfies atu's observation of cache limits as mmcache allows those limits to be set . I also like that the content cache is programmable and can be fitted into your application's framework's overall caching model.
I realize this solution isn't for everyone--it is hard (and rare) to beat the filesystem, particularly when it only requires a single read (as is the case for a cached template read). After all, the filesystem also does some ram caching and in a pinch, a ramdrive usually performs very well in these situations, too. Also, for anyone on a shared host this is a non-starter altogether, IMO.
Never-the-less, from my point of view, I think it has a lot of merit and even more potential!
At this point I have only these minor code observations:
1) to get around the (deprecated) parameter passed by reference:
change _mmcache_clear_cache(&$_pointer);
to _mmcache_clear_cache($_pointer);
change function _mmcache_clear_cache($hierarchie)
to function _mmcache_clear_cache(&$hierarchie)
2) add $return = true; // in case 'read':
cheers!
Last edited by boots on Tue Nov 11, 2003 3:12 pm; edited 1 time in total |
|
Back to top |
|
andre Smarty Pro
Joined: 23 Apr 2003 Posts: 164 Location: Karlsruhe, Germany
|
Posted: Mon Nov 10, 2003 7:40 am Post subject: |
|
|
Yeah, you're right, boots. I've updated the code once again |
|
Back to top |
|
haste Smarty Rookie
Joined: 07 Jun 2003 Posts: 17
|
Posted: Mon Nov 10, 2003 9:36 am Post subject: |
|
|
Geez. A simple question I had has turned into quite a discussion. Great work boots and andre. |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Wed Nov 12, 2003 3:00 am Post subject: |
|
|
haste wrote: | Geez. A simple question I had has turned into quite a discussion. Great work boots and andre. |
I think the credit goes to andre on this one |
|
Back to top |
|
xces Smarty Regular
Joined: 09 Apr 2004 Posts: 77
|
Posted: Wed Jun 02, 2004 8:55 am Post subject: |
|
|
I like this implementation, this could also be used for the _templates_c directory if adjusted enough.. Is this code implemented in smarty to limit the size of the _cache or _templates_c directory to a specific size and autmomaticly delete the oldest files untill a certain buffer is reaced? |
|
Back to top |
|
|