Smarty Forum Index Smarty
WARNING: All discussion is moving to https://reddit.com/r/smarty, please go there! This forum will be closing soon.

Caching on multiple servers
Goto page 1, 2, 3  Next
 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> General
View previous topic :: View next topic  
Author Message
rascal
Smarty Rookie


Joined: 19 Apr 2003
Posts: 13
Location: Switzerland

PostPosted: Wed Mar 16, 2005 6:41 pm    Post subject: Caching on multiple servers Reply with quote

Hello,

I use smarty for our homepage. We have may hits and now we need one more webserver for managing the heavy load. Now I have a question about the caching feature (I use it almost everywhere).
For example, in our forum application I generate cache. I have to delete it when there is a new post. How can I delete the cache of both machines at once? I've already thought about a extended Smarty class, in which I override the clear_cache* methods. My question is: Have you already done something like that and how do you solve this problem? Is there an elegant way to do this?

Thanks for your answers and regards from switzerland,
Pascal
Back to top
View user's profile Send private message Visit poster's website
mohrt
Administrator


Joined: 16 Apr 2003
Posts: 7368
Location: Lincoln Nebraska, USA

PostPosted: Wed Mar 16, 2005 7:42 pm    Post subject: Reply with quote

You could keep your cache files on a shared network drive, that would be the most painless.
Back to top
View user's profile Send private message Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Mar 16, 2005 8:56 pm    Post subject: Reply with quote

mohrt wrote:
You could keep your cache files on a shared network drive, that would be the most painless.


This is probably the easiest way to go about it but I tend to think it is the least promising. Network filesystem access can be painfully slow, Apache specifically recommends against network filesystems and concurrency is still an issue if you have multiple writers.

I don't know if it is really that important to maintain absolute synchronization (it depends on your overall architecure, I suppose) I would be tempted to let each server manage its own cache and write a daemon that accepted messages from the local server group to remove/create cache entries when you really need synchronization.

Then again, you needn't go through all that effort if you want a single caching mechanism; you might consider writing a memcached wrapper for the smarty cache. That way you don't have to worry about custom daemons and the rest and you can leverage a proven solution.

I believe that their are slides and presentations at php.net that cover some of the scale issues for yahoo, flickr and friendster. If not there, google for them. If nothing else, it is interesting to see the scaling experiences of these sites.

Just my 2c.
Back to top
View user's profile Send private message
mohrt
Administrator


Joined: 16 Apr 2003
Posts: 7368
Location: Lincoln Nebraska, USA

PostPosted: Wed Mar 16, 2005 9:00 pm    Post subject: Reply with quote

We have NFS v3 with a 30 sec file cache enabled and have had very good success with Smarty cache files in this setup, they are virtually local files.
Back to top
View user's profile Send private message Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Mar 16, 2005 9:23 pm    Post subject: Reply with quote

Sorry, I didn't mean that it isn't workable, but I don't think it is exceptionally scalable. Of course, my bias may be based on the fact that I don't do it that way.

YMMV Smile
Back to top
View user's profile Send private message
mohrt
Administrator


Joined: 16 Apr 2003
Posts: 7368
Location: Lincoln Nebraska, USA

PostPosted: Wed Mar 16, 2005 9:29 pm    Post subject: Reply with quote

In our case the advantages of maintaining one source of cache files outweighed the network latency involved (which is mostly eliminated by the NFS cache anyways.)
Back to top
View user's profile Send private message Visit poster's website
rascal
Smarty Rookie


Joined: 19 Apr 2003
Posts: 13
Location: Switzerland

PostPosted: Thu Mar 17, 2005 7:40 am    Post subject: Reply with quote

@mohrt: Because we have a second machine with a picture server, we already have some experiences with nfs. We store there the user pics, and on this server also runs a httpd which distributes only pictures. This is maybe a good idea to do this.

@boots: I think it is also a good idea to let each server manage it's own cache. What do you mean with "writing a memcached wrapper for the smarty cache"?
Back to top
View user's profile Send private message Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Thu Mar 17, 2005 3:16 pm    Post subject: Reply with quote

I meant using the following items:
http://www.danga.com/memcached/
http://pecl.php.net/package/memcache
http://smarty.php.net/manual/en/section.template.cache.handler.func.php

After some consideration I wonder if I am making things too hard Smile
Back to top
View user's profile Send private message
rascal
Smarty Rookie


Joined: 19 Apr 2003
Posts: 13
Location: Switzerland

PostPosted: Fri Mar 18, 2005 10:49 am    Post subject: Reply with quote

@boots: I read several articles about memcached after your post. This is a very clever solution I think. Do you have an idea, how to store cache groups in memcache, because it would be fine if I could delete groups.. I think, this could be a problem when writing the cache handler function.
Back to top
View user's profile Send private message Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Fri Mar 18, 2005 11:40 pm    Post subject: Reply with quote

Hi rascal. You might get some insights from this code http://www.phpinsider.com/smarty-forum/viewtopic.php?t=1507 (particluarly andre's solution) which implements a smarty cache handler using eaccelerator (was turck mmcache).

Although not a distributed solution, it is otherwise quite similar to using memcached and faces many of the same issues. To handle the cache groups, it stores an index directly in the caching engine as a plain item. The cache wrapper manages the index based on the items it enters or removes from the cache.

Let us know how you fare Smile
Back to top
View user's profile Send private message
rascal
Smarty Rookie


Joined: 19 Apr 2003
Posts: 13
Location: Switzerland

PostPosted: Sun Aug 21, 2005 5:15 pm    Post subject: Reply with quote

Since May, we are doing the whole thing with a nfs volume, but this is not a very good solution for us..
Now I had the time to analyse the code from the eAccelerator cachehandler. I think, it's impossible to port this piece of code to a memcached based cache handler. memcached doesn't support locking - so there is a problem of race conditions.. one thread can read the keys and analyse them to update the list, while the other thread updates the list while the other thread is still analyzing...

Do you have any other ideas, how I could solve this problem? I can't believe, that we are alone with our problem :-/
Back to top
View user's profile Send private message Visit poster's website
luxurylink
Smarty Rookie


Joined: 16 Nov 2005
Posts: 5

PostPosted: Fri Jan 06, 2006 9:05 pm    Post subject: Reply with quote

rascal wrote:
Since May, we are doing the whole thing with a nfs volume, but this is not a very good solution for us..
Now I had the time to analyse the code from the eAccelerator cachehandler. I think, it's impossible to port this piece of code to a memcached based cache handler. memcached doesn't support locking - so there is a problem of race conditions.. one thread can read the keys and analyse them to update the list, while the other thread updates the list while the other thread is still analyzing...

Do you have any other ideas, how I could solve this problem? I can't believe, that we are alone with our problem :-/


racal,

If you read some of the docs on memcached, it handles the "race condition" w/ the add, and set flags.

http://www.danga.com/memcached/

We are considering writing the smarty cache handler to use memcached..

Anyone out there already create an libary to do this?
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Fri Jan 06, 2006 9:28 pm    Post subject: Reply with quote

I wonder if anyone has yet created a cachehandler for Pear's SharedMemory package (which is now at the completed proposal stage: http://pear.php.net/pepr/pepr-proposal-show.php?id=294 ) This package is really just a consistent front-end API with pluggable backend support for various caching services. Also, more backends are already supported than listed in the proposal -- for example, I had requested the author to add support for sharedance and he promptly did.

If no one has, I may tackle this one.
Back to top
View user's profile Send private message
luxurylink
Smarty Rookie


Joined: 16 Nov 2005
Posts: 5

PostPosted: Sat Jan 07, 2006 12:24 am    Post subject: Reply with quote

boots wrote:
I wonder if anyone has yet created a cachehandler for Pear's SharedMemory package (which is now at the completed proposal stage: http://pear.php.net/pepr/pepr-proposal-show.php?id=294 ) This package is really just a consistent front-end API with pluggable backend support for various caching services. Also, more backends are already supported than listed in the proposal -- for example, I had requested the author to add support for sharedance and he promptly did.

If no one has, I may tackle this one.


I'm not aware of any packages out there... Would love to see one though, especially since it fits nicely into the smarty architecture.
Back to top
View user's profile Send private message
luxurylink
Smarty Rookie


Joined: 16 Nov 2005
Posts: 5

PostPosted: Mon Jan 09, 2006 4:02 pm    Post subject: Reply with quote

boots wrote:
I wonder if anyone has yet created a cachehandler for Pear's SharedMemory package (which is now at the completed proposal stage: http://pear.php.net/pepr/pepr-proposal-show.php?id=294 ) This package is really just a consistent front-end API with pluggable backend support for various caching services. Also, more backends are already supported than listed in the proposal -- for example, I had requested the author to add support for sharedance and he promptly did.

If no one has, I may tackle this one.



Here's one from Sweden... memcached as a smarty cache handler.
http://swag.dk/swag/kode/
Back to top
View user's profile Send private message
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> General All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP