View previous topic :: View next topic |
Author |
Message |
rascal Smarty Rookie
Joined: 19 Apr 2003 Posts: 13 Location: Switzerland
|
Posted: Wed Mar 16, 2005 6:41 pm Post subject: Caching on multiple servers |
|
|
Hello,
I use smarty for our homepage. We have may hits and now we need one more webserver for managing the heavy load. Now I have a question about the caching feature (I use it almost everywhere).
For example, in our forum application I generate cache. I have to delete it when there is a new post. How can I delete the cache of both machines at once? I've already thought about a extended Smarty class, in which I override the clear_cache* methods. My question is: Have you already done something like that and how do you solve this problem? Is there an elegant way to do this?
Thanks for your answers and regards from switzerland,
Pascal |
|
Back to top |
|
mohrt Administrator
Joined: 16 Apr 2003 Posts: 7368 Location: Lincoln Nebraska, USA
|
Posted: Wed Mar 16, 2005 7:42 pm Post subject: |
|
|
You could keep your cache files on a shared network drive, that would be the most painless. |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Wed Mar 16, 2005 8:56 pm Post subject: |
|
|
mohrt wrote: | You could keep your cache files on a shared network drive, that would be the most painless. |
This is probably the easiest way to go about it but I tend to think it is the least promising. Network filesystem access can be painfully slow, Apache specifically recommends against network filesystems and concurrency is still an issue if you have multiple writers.
I don't know if it is really that important to maintain absolute synchronization (it depends on your overall architecure, I suppose) I would be tempted to let each server manage its own cache and write a daemon that accepted messages from the local server group to remove/create cache entries when you really need synchronization.
Then again, you needn't go through all that effort if you want a single caching mechanism; you might consider writing a memcached wrapper for the smarty cache. That way you don't have to worry about custom daemons and the rest and you can leverage a proven solution.
I believe that their are slides and presentations at php.net that cover some of the scale issues for yahoo, flickr and friendster. If not there, google for them. If nothing else, it is interesting to see the scaling experiences of these sites.
Just my 2c. |
|
Back to top |
|
mohrt Administrator
Joined: 16 Apr 2003 Posts: 7368 Location: Lincoln Nebraska, USA
|
Posted: Wed Mar 16, 2005 9:00 pm Post subject: |
|
|
We have NFS v3 with a 30 sec file cache enabled and have had very good success with Smarty cache files in this setup, they are virtually local files. |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Wed Mar 16, 2005 9:23 pm Post subject: |
|
|
Sorry, I didn't mean that it isn't workable, but I don't think it is exceptionally scalable. Of course, my bias may be based on the fact that I don't do it that way.
YMMV |
|
Back to top |
|
mohrt Administrator
Joined: 16 Apr 2003 Posts: 7368 Location: Lincoln Nebraska, USA
|
Posted: Wed Mar 16, 2005 9:29 pm Post subject: |
|
|
In our case the advantages of maintaining one source of cache files outweighed the network latency involved (which is mostly eliminated by the NFS cache anyways.) |
|
Back to top |
|
rascal Smarty Rookie
Joined: 19 Apr 2003 Posts: 13 Location: Switzerland
|
Posted: Thu Mar 17, 2005 7:40 am Post subject: |
|
|
@mohrt: Because we have a second machine with a picture server, we already have some experiences with nfs. We store there the user pics, and on this server also runs a httpd which distributes only pictures. This is maybe a good idea to do this.
@boots: I think it is also a good idea to let each server manage it's own cache. What do you mean with "writing a memcached wrapper for the smarty cache"? |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
|
Back to top |
|
rascal Smarty Rookie
Joined: 19 Apr 2003 Posts: 13 Location: Switzerland
|
Posted: Fri Mar 18, 2005 10:49 am Post subject: |
|
|
@boots: I read several articles about memcached after your post. This is a very clever solution I think. Do you have an idea, how to store cache groups in memcache, because it would be fine if I could delete groups.. I think, this could be a problem when writing the cache handler function. |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Fri Mar 18, 2005 11:40 pm Post subject: |
|
|
Hi rascal. You might get some insights from this code http://www.phpinsider.com/smarty-forum/viewtopic.php?t=1507 (particluarly andre's solution) which implements a smarty cache handler using eaccelerator (was turck mmcache).
Although not a distributed solution, it is otherwise quite similar to using memcached and faces many of the same issues. To handle the cache groups, it stores an index directly in the caching engine as a plain item. The cache wrapper manages the index based on the items it enters or removes from the cache.
Let us know how you fare |
|
Back to top |
|
rascal Smarty Rookie
Joined: 19 Apr 2003 Posts: 13 Location: Switzerland
|
Posted: Sun Aug 21, 2005 5:15 pm Post subject: |
|
|
Since May, we are doing the whole thing with a nfs volume, but this is not a very good solution for us..
Now I had the time to analyse the code from the eAccelerator cachehandler. I think, it's impossible to port this piece of code to a memcached based cache handler. memcached doesn't support locking - so there is a problem of race conditions.. one thread can read the keys and analyse them to update the list, while the other thread updates the list while the other thread is still analyzing...
Do you have any other ideas, how I could solve this problem? I can't believe, that we are alone with our problem :-/ |
|
Back to top |
|
luxurylink Smarty Rookie
Joined: 16 Nov 2005 Posts: 5
|
Posted: Fri Jan 06, 2006 9:05 pm Post subject: |
|
|
rascal wrote: | Since May, we are doing the whole thing with a nfs volume, but this is not a very good solution for us..
Now I had the time to analyse the code from the eAccelerator cachehandler. I think, it's impossible to port this piece of code to a memcached based cache handler. memcached doesn't support locking - so there is a problem of race conditions.. one thread can read the keys and analyse them to update the list, while the other thread updates the list while the other thread is still analyzing...
Do you have any other ideas, how I could solve this problem? I can't believe, that we are alone with our problem :-/ |
racal,
If you read some of the docs on memcached, it handles the "race condition" w/ the add, and set flags.
http://www.danga.com/memcached/
We are considering writing the smarty cache handler to use memcached..
Anyone out there already create an libary to do this? |
|
Back to top |
|
boots Administrator
Joined: 16 Apr 2003 Posts: 5611 Location: Toronto, Canada
|
Posted: Fri Jan 06, 2006 9:28 pm Post subject: |
|
|
I wonder if anyone has yet created a cachehandler for Pear's SharedMemory package (which is now at the completed proposal stage: http://pear.php.net/pepr/pepr-proposal-show.php?id=294 ) This package is really just a consistent front-end API with pluggable backend support for various caching services. Also, more backends are already supported than listed in the proposal -- for example, I had requested the author to add support for sharedance and he promptly did.
If no one has, I may tackle this one. |
|
Back to top |
|
luxurylink Smarty Rookie
Joined: 16 Nov 2005 Posts: 5
|
Posted: Sat Jan 07, 2006 12:24 am Post subject: |
|
|
boots wrote: | I wonder if anyone has yet created a cachehandler for Pear's SharedMemory package (which is now at the completed proposal stage: http://pear.php.net/pepr/pepr-proposal-show.php?id=294 ) This package is really just a consistent front-end API with pluggable backend support for various caching services. Also, more backends are already supported than listed in the proposal -- for example, I had requested the author to add support for sharedance and he promptly did.
If no one has, I may tackle this one. |
I'm not aware of any packages out there... Would love to see one though, especially since it fits nicely into the smarty architecture. |
|
Back to top |
|
luxurylink Smarty Rookie
Joined: 16 Nov 2005 Posts: 5
|
Posted: Mon Jan 09, 2006 4:02 pm Post subject: |
|
|
boots wrote: | I wonder if anyone has yet created a cachehandler for Pear's SharedMemory package (which is now at the completed proposal stage: http://pear.php.net/pepr/pepr-proposal-show.php?id=294 ) This package is really just a consistent front-end API with pluggable backend support for various caching services. Also, more backends are already supported than listed in the proposal -- for example, I had requested the author to add support for sharedance and he promptly did.
If no one has, I may tackle this one. |
Here's one from Sweden... memcached as a smarty cache handler.
http://swag.dk/swag/kode/ |
|
Back to top |
|
|