View previous topic :: View next topic |
Author |
Message |
orrd Smarty Rookie
Joined: 29 Jun 2010 Posts: 11
|
Posted: Wed Jan 25, 2012 6:53 am Post subject: "unable to write file" cache error |
|
|
When I turn on use_sub_dirs, the site works fine for a few days until is starts getting "unable to write file" errors like this:
Code: | [24-Jan-2012 04:24:26] PHP Fatal error: Uncaught exception 'SmartyException' with message 'unable to write file /home/website/smarty/cache/28175/en/01/67/c4/wrt4f1e86daa3489' in /usr/local/lib/smarty3/libs/sysplugins/smarty_internal_write_file.php:44
|
It seems to start happening after there are a large number of subdirectories in the cache directory (over 32000 at least). I don't think the sheer number of directories would cause it to fail to create new ones (it's a Linux system). Once I clear all cache files, it's fine again for a few days.
My secondary question is why are there over 32000 subdirectories in the cache directory anyway? Isn't the point of use_sub_dirs to make it break the cache storage up into a reasonable number of subdirectories so that there isn't a huge number of files in any directory? |
|
Back to top |
|
mohrt Administrator
Joined: 16 Apr 2003 Posts: 7368 Location: Lincoln Nebraska, USA
|
Posted: Wed Jan 25, 2012 3:07 pm Post subject: |
|
|
What linux FS are you using? Some of them have limitations how many directories can be in one folder. See if you can manually create a new folder in the directory.
What version of Smarty? The cache folder should break the files into sub-dirs. it would take LOT of templates to fill with 32,000 directories. How are you caching, and do you use cache groups?
[edit] your path starts with 28175, that is not something Smarty generates (afaik). That must be a cache group, and you are somehow creating 32k of them (?) |
|
Back to top |
|
orrd Smarty Rookie
Joined: 29 Jun 2010 Posts: 11
|
Posted: Thu Jan 26, 2012 6:57 am Post subject: |
|
|
The filesystem is ext3. I am using cache groups for just some of the pages. It is a large database driven site with more than 100,000 pages when you include the foreign language translations, etc.
I noticed this topic is about the same thing:
www.smarty.net/forums/viewtopic.php?p=77561
But I believe I've fixed it. The problem is this line:
$_tmp_file = $_dirpath . DS . uniqid('wrt');
The function uniquid() basically just returns a code based on the current microsecond. I think the problem might actually be with multiple processes trying to create a temp file at the same microsecond (as unlikely as that seems, perhaps it isn't impossible on a high traffic site).
So I changed that line to this:
$_tmp_file = tempnam($_dirpath, 'wrt');
And ever since I changed that I haven't had any more errors so far. I believe tempnam() is the proper function to use for this purpose, not uniqud(). |
|
Back to top |
|
rodneyrehm Administrator
Joined: 30 Mar 2007 Posts: 674 Location: Germany, border to Switzerland
|
|
Back to top |
|
U.Tews Administrator
Joined: 22 Nov 2006 Posts: 5068 Location: Hamburg / Germany
|
Posted: Thu Jan 26, 2012 3:37 pm Post subject: |
|
|
The trunk version in the SVN does include already
$_tmp_file = $_dirpath . DS . uniqid('wrt', true);
To solve this problem, |
|
Back to top |
|
orrd Smarty Rookie
Joined: 29 Jun 2010 Posts: 11
|
Posted: Fri Jan 27, 2012 9:16 pm Post subject: |
|
|
Ok, hopefully uniqid('wrt', true) will provide enough entropy.
In any case, after a few more days it started to generate the errors again, and after some searching I found this: http://expert.mandriva.com/question/32318. So I was actually running into a limit with ext3 related to the number of directories (I don't know if this has changed in more recent versions of ext3 or Linux).
I also looked into why there are so many directories created in the root cache directory in the first place. I believe what is happening is Smarty divides the cache files into different directories (/0f/0d/2e/0f0d2ef2...) based on only the template name, but not the cache_id. Every cache ID just creates a new directory in the root cache path.
I don't understand the reasoning behind that. It's unlikely any site would have many many thousands of templates, but a typical database-driven site will have a few templates but many many thousands of cache_id values for each one. Shouldn't Smarty divide up the directories (/0f/0d/2e/0f0d2ef2...) based on the cache_id rather than the template name? |
|
Back to top |
|
U.Tews Administrator
Joined: 22 Nov 2006 Posts: 5068 Location: Hamburg / Germany
|
Posted: Fri Jan 27, 2012 9:52 pm Post subject: |
|
|
This has been a topic since Smarty2 which used also the items of of cache gruop and compile_id as directory name.
I think the reasion for it was quick execution of clearCache() calls by cache_id or partital cache groups.
There have been some thoughts about changing this, but all ideas ended up in very bad performance of clearCache() as it needs then to loop over all files checking for matching file names.
It is still on our todo list to look for a reasionable approach.
The only workaround I currently can think of is that you split your cache_id with some function in cache groups which will create some sort of distribution.
$cache_id = 'id_part1|id_part2|id_part3' |
|
Back to top |
|
orrd Smarty Rookie
Joined: 29 Jun 2010 Posts: 11
|
Posted: Fri Jan 27, 2012 10:48 pm Post subject: |
|
|
Thanks for the quick replies. It's great that questions actually get answered in this forum.
I can see how there would be some complications for clearCache() depending on how it's implemented.
Ok, so I need to break it up into subdirs myself using cache groups. The only other issue then is that the dividing (/0f/0d/2e/0f0d2ef2...) that it does based on the template name adds an additional several subdirectories that I think are unnecessary. Ideally I would like the only directories that are created be based on my own cache groups. Is there a flag or setting for that? |
|
Back to top |
|
U.Tews Administrator
Joined: 22 Nov 2006 Posts: 5068 Location: Hamburg / Germany
|
Posted: Sat Jan 28, 2012 7:18 am Post subject: |
|
|
There is no flag to disable the subdirectories by template name.
You could disable the code in lines 35 to 40 in file smarty_internal_cacheredource_files.php if you don't use clearCache().
I will look into general problem once again in the next couple of days. |
|
Back to top |
|
orrd Smarty Rookie
Joined: 29 Jun 2010 Posts: 11
|
Posted: Sat Jan 28, 2012 7:40 am Post subject: |
|
|
Thanks. |
|
Back to top |
|
|