Smarty Forum Index Smarty
WARNING: All discussion is moving to https://reddit.com/r/smarty, please go there! This forum will be closing soon.

Turck MMCache cache handler
Goto page 1, 2, 3  Next
 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Tips and Tricks
View previous topic :: View next topic  
Author Message
freestyler
Smarty n00b


Joined: 30 Nov 2003
Posts: 4
Location: Portugal

PostPosted: Mon Dec 01, 2003 1:24 am    Post subject: Turck MMCache cache handler Reply with quote

Attached is an experimental cache handler utilising Shared Memory as a store. Requires Turck MMCache ( http://turck-mmcache.sourceforge.net/ ) version 2.4.5 or later. In a scenario where a user interface ( read page wide template ) is split into different sections (to eliminate expensive DB calls for mostly stale data while still having some page sections as dynamic) in conjuction with a hacked {clipcache} block function http://www.phpinsider.com/smarty-forum/viewtopic.php?t=1343 , this yields a huge performance increase. By default MMCache caches compiled templates but somehow ignores the Smarty cache on disk. There is one incompatibility with this handler : At present I am not aware of a MMCache function or workaround to clear ALL keys other than clearing the WHOLE Shared Memory cache.

Our deployment setup : PHP 4.3.3, Smarty 2.6.0 RC3, Apache 1.3.28, Turck MMCache 2.4.5, Linux

[php:1:428226b5bc]<?php

/**
* Smarty Turck MMCache cache handler.
*
* File: mmcache_cache_handler.php
* Type: cache_handler
* Name: mmcache_cache_handler
* Date: Nov 30, 2003
* Purpose: Turck MMCache cache handler to minimize disk IO and excessive stat() calls.
* Requires the Turck MMCache ( http://turck-mmcache.sourceforge.net/ ) extension.
* @author Lourens Naudé <lourens_at_esouthafrica.com>
* @version 0.3
* @param string $action Cache operation to perform ( read | write | clear )
* @param mixed $smarty Reference to an instance of Smarty
* @param string $cache_content Reference to cached contents
* @param string $tpl_file Template file name
* @param string $cache_id Cache identifier
* @param string $compile_id Compile identifier
* @param integer $exp_time Expiration time
*/

function mmcache_cache_handler( $action, &$smarty, &$cache_content, $tpl_file = null, $cache_id = null, $compile_id = null, $exp_time = null ){

if( !function_exists( 'mmcache_get' ) ){
$smarty->trigger_error( 'cache_handler: Turck MMCache extension not installed !' );
}

$cache_id = md5( $tpl_file . $cache_id . $compile_id );

_mmcache_cache_handler_getcontents( $smarty );

$_cache_key_contents = $smarty->_mmcache_cache_handler_contents[$cache_id];

if( _mmcache_cache_handler_hasexpired( $_cache_key_contents ) == true ){

_mmcache_cache_handler_remove_key( $cache_id, $smarty );

}

switch ( $action ) {
case 'read' : $cache_content = $_cache_key_contents;
$return = true;

break;

case 'write' : $smarty->_mmcache_cache_handler_contents[$cache_id] = $cache_content;

$result = mmcache_put( '_smarty_cache', $smarty->_mmcache_cache_handler_contents, 0 );

if( !$result ){
$smarty->trigger_error( 'cache_handler: Cache write query failed.' );
}

$return = $result;

break;

case 'clear' : if( empty( $cache_id ) && empty( $compile_id ) && empty( $tpl_file ) ){
$result = mmcache_rm( '_smarty_cache' );
}else{

$result = _mmcache_cache_handler_remove_key( $cache_id, $smarty );

}

if( !$result ) {
$smarty->trigger_error( 'cache_handler: Clear cache query failed.' );
}

$return = $result;

break;

default : $smarty->trigger_error( 'cache_handler: Unknown cache action '. $action );
$return = false;

break;
}

return $return;

}

function _mmcache_cache_handler_getcontents( &$smarty ){

if( empty( $smarty->_mmcache_cache_handler_contents ) ){

$smarty->_mmcache_cache_handler_contents = mmcache_get( '_smarty_cache' );

}

if( $smarty->_mmcache_cache_handler_contents == null ){
$smarty->trigger_error( 'cache_handler: Cache Tree read query failed.' );
}

}

function _mmcache_cache_handler_remove_key( $cache_id, $smarty ){

unset( $smarty->_mmcache_cache_handler_contents[$cache_id] );

return mmcache_put( '_smarty_cache', $smarty->_mmcache_cache_handler_contents, 0 );

}

function _mmcache_cache_handler_hasexpired( &$contents ){

$split = explode( "\n", $contents, 2);
$header = $split[0];

$attributes = unserialize( $header );

if( $attributes['expires'] > 0 || time() > $attributes['expires'] ){
return true;
}else{
return false;
}

}

?>
[/php:1:428226b5bc]


Last edited by freestyler on Wed Dec 03, 2003 1:03 am; edited 3 times in total
Back to top
View user's profile Send private message MSN Messenger
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Mon Dec 01, 2003 4:24 am    Post subject: Reply with quote

Hi freestyler.

Nice job. You may be interested in this thread in which andre proposes a similar cache handler function.

I'm interested in the hacked {clipcache} and what modifications you might propose, if you don't mind me asking Smile.

Cheers!
Back to top
View user's profile Send private message
andre
Smarty Pro


Joined: 23 Apr 2003
Posts: 164
Location: Karlsruhe, Germany

PostPosted: Mon Dec 01, 2003 7:44 am    Post subject: Reply with quote

Nice work.

FYI: In my mmcache plugin I am using an index to handle correct cache clearing. So it works for me exactly like expected Wink
Back to top
View user's profile Send private message
freestyler
Smarty n00b


Joined: 30 Nov 2003
Posts: 4
Location: Portugal

PostPosted: Mon Dec 01, 2003 5:33 pm    Post subject: Alterations Reply with quote

Hi,

I slightly modified my post and the handler is now compatible with Smarty's garbage collection. I did not use an index, but rather stored the whole cache under a single key, _smarty_cache , as an array. To clear the entire cache, i just remove this key or unset individual array hashes to clear a specific cache identifier. Although not currently thread safe etc. , i tested it now on our deployment environment and it worked with no hickups. I will look at the internal smarty functions display, fetch and the cache one's later today to indentify why compiled templates are automatically cached by MMCache and a disk based cache, which essentially follows the same naming conventions, is not.
Back to top
View user's profile Send private message MSN Messenger
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Tue Dec 02, 2003 12:21 am    Post subject: Reply with quote

freestyler, I was initially going to say:
Quote:
If I'm not mistaken (and I may very well be), compiled files are directly included() by the engine while cached files need to be taken apart (there is a custom header). This likely precludes them from being directly included by the engine thus not given entry into the mmcache pool.

But looking at core.read_cache_file.php, it looks like the cached file is indeed included. Are you certain that cache files are not ending up in the mmcache (I mean with disk based Smarty caching on)?

Even though I think andre's handler is excellent, I'm glad to see an alternative implementation.

Two things: do you think that storing the entire cache-tree as a single key is too memory intensive? I don't think it is nice to have to load the entire tree in cases I just want a single leaf--and I do believe that to be the most frequent case.

Also, I notice you and andre both call mmcache_gc on every cache handler call but I wonder if it is necessary -- or perhaps too conservative Wink.

Finally, was there anything you can comment on re: {clipcache}?
Back to top
View user's profile Send private message
freestyler
Smarty n00b


Joined: 30 Nov 2003
Posts: 4
Location: Portugal

PostPosted: Tue Dec 02, 2003 1:14 am    Post subject: Reply with quote

boots,

I modified my listing once more - garbage collection works as expected. I resorted to borrowing the deserialization snippet used by the file based cache handler as Smarty 2.6.0 still calls the cache_handler_func hook with $exp_time = null. The custom header most probably interferes, considering that MMCache has the ability to serve up Gzipped content to compatible browsers (I have that enabled).

I agree with you on two statements :

- Yes, using a single key for the the whole cache tree would be counter productive in a big cache with many keys. I experimented with the handler initially to solve the problem of my application running mostly from shared memory ( double checked, disk based cache excluded by default ) but then still hitting the disk for otherwise resource intensive queries. In such a scenario the benefits of using a cache soon deminish. The handler works very well in my case as I'm mostly caching only small cache clips and at least in this project using a single shm key would do the job just fine. Would be interesting to eventually benchmark that against the array-looping overhead of using a cache index.

- Delegating garbage collection to the underlying Shared Memory cache , as in Andre's implementation, is probably best.

BTW: Busy experimenting with {clipcache} and may have something usefull going with that. Will post reply when have that sorted.
Back to top
View user's profile Send private message MSN Messenger
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Tue Dec 02, 2003 2:14 am    Post subject: Reply with quote

freestyler,

I see your point about small cache-trees. Perhaps there is room for something between andre's and your solution, something like a clustered index from SQL Server where the idea is to keep like items together. In this case, we'd still use caches trees like your implementation, but allow for separate keys based on a custom hash of cache_id and compile_id to keep overall tree small size and confined to items that are likely to be requested together anyhow.

Again, nice coding, its looking better all the time Smile
Back to top
View user's profile Send private message
andre
Smarty Pro


Joined: 23 Apr 2003
Posts: 164
Location: Karlsruhe, Germany

PostPosted: Tue Dec 02, 2003 12:17 pm    Post subject: Reply with quote

freestyler wrote:
Would be interesting to eventually benchmark that against the array-looping overhead of using a cache index.

I don't see the a real problem with using a cache index as long as you don't need to clear or write the cache. Normally I am using an expirery time of -1 and re-create the cache manually if needed. So the script normally doesn't iterate through the array at all. But benchmarking whould be interesting indeed.


boots wrote:
Finally, was there anything you can comment on re: {clipcache}?

Hey, clipcache works really quite nice for me (with mmcache enabled) Smile It's wonderful for enhancing my caching and works without flaws.
Back to top
View user's profile Send private message
andre
Smarty Pro


Joined: 23 Apr 2003
Posts: 164
Location: Karlsruhe, Germany

PostPosted: Tue Dec 02, 2003 12:24 pm    Post subject: Reply with quote

foods wrote:
Also, I notice you and andre both call mmcache_gc on every cache handler call but I wonder if it is necessary -- or perhaps too conservative .

I noticed a strange problem with my mmcache handler which seems to be solved by this mmcache_gc() call. But perhaps I have to inspect this a bit more. Wink It seemed to me mmcache reads a cached content even if it was already expired and sometimes it didn't overwrite the expired one on a new cache write. Very strange?! Confused
Back to top
View user's profile Send private message
andre
Smarty Pro


Joined: 23 Apr 2003
Posts: 164
Location: Karlsruhe, Germany

PostPosted: Tue Dec 02, 2003 1:25 pm    Post subject: Reply with quote

After looking in your source code and thinking about a few things I have recreated my mmcache plugin. You will notice alot of similarities. Hope you don't mind Wink

So here's the updated plugin from myself:
[php:1:2dc4e6e053]
<?php
/**
* Smarty Cache Handler<br>
* utilizing Turck MMCache extension (http://turck-mmcache.sourceforge.net/)<br>
*
* @package Smarty
* @subpackage plugins
*/

/**
* Helper function for smarty_cache_mmcache()
* Clears a whole hierarchy of cache entries.
*
* @access private
* @param array $hierarchy hierarchical array of cache ids
* @return void
*
* @see smarty_cache_mmcache()
*/
function _mmcache_clear_cache(&$hierarchy) {
foreach ($hierarchy as $key => $value) {
if (is_array($value)) {
_mmcache_clear_cache($value);
}
else {
mmcache_lock($value);
mmcache_rm($value);
mmcache_unlock($value);
}
}
}

/**
* Helper function for smarty_cache_mmcache()
* Checks whether a cached content has been expired by reading the content's header.
*
* @access private
* @param string $cache_content the cached content
* @return boolean TRUE if cache has been expired, FALSE otherwise
*
* @see smarty_cache_mmcache()
*/
function _mmcache_hasexpired(&$cache_content) {
$split = explode("\n", $cache_content, 2);
$attributes = unserialize($split[0]);

if ($attributes['expires'] > 0 && time() > $attributes['expires'])
return true;
else
return false;
}

/**
* Smarty Cache Handler<br>
* utilizing Turck MMCache extension (http://turck-mmcache.sourceforge.net/)<br>
*
* Name: smarty_cache_mmcache<br>
* Type: Cache Handler<br>
* Purpose: Replacement for the file based cache handling of Smarty. smarty_cache_mmcache() is
* using the Turck MMCache extension to minimize disk usage.
* File: cache.mmcache.php<br>
* Date: Dec 2, 2003<br>
*
* Usage Example<br>
* <pre>
* $smarty = new Smarty;
* $smarty->cache_handler_func = 'smarty_cache_mmcache';
* $smarty->caching = true;
* $smarty->display('index.tpl');
* </pre>
*
* @author André Rabold
* @version RC-1
*
* @param string $action Cache operation to perform ( read | write | clear )
* @param mixed $smarty Reference to an instance of Smarty
* @param string $cache_content Reference to cached contents
* @param string $tpl_file Template file name
* @param string $cache_id Cache identifier
* @param string $compile_id Compile identifier
* @param integer $exp_time Expiration time
* @return boolean TRUE on success, FALSE otherwise
*
* @link http://turck-mmcache.sourceforge.net/
* (Turck MMCache homepage)
* @link http://smarty.php.net/manual/en/section.template.cache.handler.func.php
* (Smarty online manual)
*/
function smarty_cache_mmcache($action, &$smarty, &$cache_content, $tpl_file=null, $cache_id=null, $compile_id=null, $exp_time=null)
{
if(!function_exists("mmcache")) {
$smarty->trigger_error("cache_handler: PHP Extension \"Turck MMCache\" (http://turck-mmcache.sourceforge.net/) not installed.");
return false;
}

// Create unique cache id:
// We are using smarty's internal functions here to be as compatible as possible.
$_auto_id = $smarty->_get_auto_id($cache_id, $compile_id);
$_cache_file = substr($smarty->_get_auto_filename(".", $tpl_file, $_auto_id),2);
$mmcache_id = "smarty_mmcache|".$_cache_file;

// The index contains all stored cache ids in a hierarchy and can be iterated later
$mmcache_index_id = "smarty_mmcache_index";

switch ($action) {

case 'read':
// read cache from shared memory
$cache_content = mmcache_get($mmcache_id);
if (!is_null($cache_content) && _mmcache_hasexpired($cache_content)) {
// Cache has been expired so we clear it now by calling ourself with another parameter Smile
$cache_content = null;
smarty_cache_mmcache('clear', $smarty, $cache_content, $tpl_file, $cache_id, $compile_id);
}

$return = true;
break;

case 'write':
// save cache to shared memory
$current_time = time();
if (is_null($exp_time) || $exp_time < $current_time)
$ttl = 0;
else
$ttl = $exp_time - time();

// First run garbage collection
mmcache_gc();

// Put content into cache
mmcache_lock($mmcache_id);
mmcache_put($mmcache_id, $cache_content, $ttl);

// Create an index association
mmcache_lock($mmcache_index_id);
$mmcache_index = mmcache_get($mmcache_index_id);
if (!is_array($mmcache_index))
$mmcache_index = array();
$indexes = explode(DIRECTORY_SEPARATOR, $_cache_file);
$_pointer =& $mmcache_index;
foreach ($indexes as $index) {
if (!isset($_pointer[$index]))
$_pointer[$index] = array();
$_pointer =& $_pointer[$index];
}
$_pointer = $mmcache_id;
mmcache_put($mmcache_index_id, $mmcache_index, 0);
mmcache_unlock($mmcache_index_id);

mmcache_unlock($mmcache_id);
break;

case 'clear':
// clear cache info
mmcache_lock($mmcache_index_id);
$mmcache_index = mmcache_get($mmcache_index_id);
if (is_array($mmcache_index)) {
if (empty($cache_id) && empty($compile_id) && empty($tpl_file)) {
// clear all cache
mmcache_lock($mmcache_id);
_mmcache_clear_cache($mmcache_index);
mmcache_unlock($mmcache_id);
$mmcache_index = array();
}
else {
// clear single file or cache group
$indexes = explode(DIRECTORY_SEPARATOR, $_cache_file);
if (is_null($tpl_file))
array_pop($indexes);

$_pointer =& $mmcache_index;
$_failed = false;
foreach ($indexes as $index) {
if (!isset($_pointer[$index])) {
$_failed = true;
break;
}
$_pointer =& $_pointer[$index];
}

if (!$_failed) {
if (is_array($_pointer)) {
// Clear cache group
_mmcache_clear_cache($_pointer);
}
else {
// Clear single file
mmcache_lock($_pointer);
mmcache_rm($_pointer);
mmcache_unlock($_pointer);
}
$_pointer = null;
}
}
}
mmcache_put($mmcache_index_id, $mmcache_index, 0);
mmcache_unlock($mmcache_index_id);

$return = true;
break;

default:
// error, unknown action
$smarty->trigger_error("cache_handler: unknown action \"$action\"");
$return = false;
break;
}
return $return;
}
?>
[/php:1:2dc4e6e053]

As freestyler pointed out it's a good thing to use the garbage collection of Smarty beneath the one MMCache offers - even if $exp_time parameter is 0 (zero) all the time. Now I am using both algorithms and I'm quite happy with it Smile See function _mmcache_hasexpired() which is stolen from freestyler Wink
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Dec 03, 2003 12:41 am    Post subject: Reply with quote

You guys are rocking on this one Smile
Back to top
View user's profile Send private message
freestyler
Smarty n00b


Joined: 30 Nov 2003
Posts: 4
Location: Portugal

PostPosted: Wed Dec 03, 2003 1:00 am    Post subject: Reply with quote

andre,

Good work - like the self reference with action 'clear' Wink

Yours is definately a better implementation for use with caching whole pages or when the cache gets rather big. A mmcache write call consumes x2 the amount of memory to store the actual data for a key and this could cause havoc on a multiple-vhost-smarty-powered box .

Just a small note regarding the use of mmcache_lock and mmcache_unlock - they are actually redundant :
Code:
if ((where == mmcache_shm_and_disk ||
         where == mmcache_shm ||
         where == mmcache_shm_only) && use_shm) {
      /* storing to shared memory */
      slot = q->hv & MM_USER_HASH_MAX;
      MMCACHE_LOCK_RW();
      mmcache_mm_instance->user_hash_cnt++;
      q->next = mmcache_mm_instance->user_hash[slot];
      mmcache_mm_instance->user_hash[slot] = q;
      p = q->next;
      while (p != NULL) {
        if ((p->hv == q->hv) && (strcmp(p->key, xkey) == 0)) {
          mmcache_mm_instance->user_hash_cnt--;
          q->next = p->next;
          mmcache_free_nolock(p);
          break;
        }
        q = p;
        p = p->next;
      }
      MMCACHE_UNLOCK_RW();
      MMCACHE_PROTECT();
      ret = 1;
    }


Modified mine to fetch the whole cache tree only once during the script lifetime - this should be adequate for our project setup as only {clipcache} segments are cached.
Back to top
View user's profile Send private message MSN Messenger
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Fri Jan 21, 2005 3:00 am    Post subject: Reply with quote

The Turck mmcache project is no longer maintained but the project has recently been forked as eaccelerator. Unlike mmcache, eaccelerator does work with current builds of PHP. As a result, I tool the liberty of updating andre's cache handler implementation and posted it at the wiki as CacheHandlerEaccelerator.
Back to top
View user's profile Send private message
OpenMacNews
Smarty Rookie


Joined: 03 Aug 2004
Posts: 34
Location: Floating on Io's Methane Seas ...

PostPosted: Thu May 04, 2006 6:02 am    Post subject: Reply with quote

boots wrote:
the project has recently been forked as eaccelerator. [/color][/url].


ok, i'll start by admitting that i haven't gotten my head completely around the interaction between this & {clipcache} ... yet ...

that said, a basic question:

other than saving the CacheHandlerEaccelerator as "/path/to/plugins/cache.eaccelerator.php", *what* do i need to do to enable this?

is this autoload-ed? or do i need to manually intervene?

thanks,

richard
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Thu May 04, 2006 7:38 am    Post subject: Reply with quote

Hi OpenMacNews,

no, it is not autoloaded. You load the function(s) manually and configure Smarty to use it as the cache handler as described in the comments of the code:
[php:1:6b647e7230]$smarty = new Smarty;
$smarty->cache_handler_func = 'smarty_cache_eaccelerator';
$smarty->caching = true;
$smarty->display('index.tpl');
[/php:1:6b647e7230]
http://smarty.php.net/manual/en/section.template.cache.handler.func.php
Back to top
View user's profile Send private message
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Tips and Tricks All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP