Smarty Forum Index Smarty
WARNING: All discussion is moving to https://reddit.com/r/smarty, please go there! This forum will be closing soon.

"unable to write file" cache error

 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Smarty 3
View previous topic :: View next topic  
Author Message
orrd
Smarty Rookie


Joined: 29 Jun 2010
Posts: 11

PostPosted: Wed Jan 25, 2012 6:53 am    Post subject: "unable to write file" cache error Reply with quote

When I turn on use_sub_dirs, the site works fine for a few days until is starts getting "unable to write file" errors like this:

Code:
[24-Jan-2012 04:24:26] PHP Fatal error:  Uncaught exception 'SmartyException' with message 'unable to write file /home/website/smarty/cache/28175/en/01/67/c4/wrt4f1e86daa3489' in /usr/local/lib/smarty3/libs/sysplugins/smarty_internal_write_file.php:44


It seems to start happening after there are a large number of subdirectories in the cache directory (over 32000 at least). I don't think the sheer number of directories would cause it to fail to create new ones (it's a Linux system). Once I clear all cache files, it's fine again for a few days.

My secondary question is why are there over 32000 subdirectories in the cache directory anyway? Isn't the point of use_sub_dirs to make it break the cache storage up into a reasonable number of subdirectories so that there isn't a huge number of files in any directory?
Back to top
View user's profile Send private message
mohrt
Administrator


Joined: 16 Apr 2003
Posts: 7368
Location: Lincoln Nebraska, USA

PostPosted: Wed Jan 25, 2012 3:07 pm    Post subject: Reply with quote

What linux FS are you using? Some of them have limitations how many directories can be in one folder. See if you can manually create a new folder in the directory.

What version of Smarty? The cache folder should break the files into sub-dirs. it would take LOT of templates to fill with 32,000 directories. How are you caching, and do you use cache groups?

[edit] your path starts with 28175, that is not something Smarty generates (afaik). That must be a cache group, and you are somehow creating 32k of them (?)
Back to top
View user's profile Send private message Visit poster's website
orrd
Smarty Rookie


Joined: 29 Jun 2010
Posts: 11

PostPosted: Thu Jan 26, 2012 6:57 am    Post subject: Reply with quote

The filesystem is ext3. I am using cache groups for just some of the pages. It is a large database driven site with more than 100,000 pages when you include the foreign language translations, etc.

I noticed this topic is about the same thing:
www.smarty.net/forums/viewtopic.php?p=77561

But I believe I've fixed it. The problem is this line:

$_tmp_file = $_dirpath . DS . uniqid('wrt');

The function uniquid() basically just returns a code based on the current microsecond. I think the problem might actually be with multiple processes trying to create a temp file at the same microsecond (as unlikely as that seems, perhaps it isn't impossible on a high traffic site).

So I changed that line to this:
$_tmp_file = tempnam($_dirpath, 'wrt');

And ever since I changed that I haven't had any more errors so far. I believe tempnam() is the proper function to use for this purpose, not uniqud().
Back to top
View user's profile Send private message
rodneyrehm
Administrator


Joined: 30 Mar 2007
Posts: 674
Location: Germany, border to Switzerland

PostPosted: Thu Jan 26, 2012 2:36 pm    Post subject: Reply with quote

tempnam() has its issues. See http://www.php.net/manual/en/function.tempnam.php#97086 for an annoying one.
_________________
Twitter
Back to top
View user's profile Send private message Visit poster's website
U.Tews
Administrator


Joined: 22 Nov 2006
Posts: 5068
Location: Hamburg / Germany

PostPosted: Thu Jan 26, 2012 3:37 pm    Post subject: Reply with quote

The trunk version in the SVN does include already

$_tmp_file = $_dirpath . DS . uniqid('wrt', true);

To solve this problem,
Back to top
View user's profile Send private message
orrd
Smarty Rookie


Joined: 29 Jun 2010
Posts: 11

PostPosted: Fri Jan 27, 2012 9:16 pm    Post subject: Reply with quote

Ok, hopefully uniqid('wrt', true) will provide enough entropy.

In any case, after a few more days it started to generate the errors again, and after some searching I found this: http://expert.mandriva.com/question/32318. So I was actually running into a limit with ext3 related to the number of directories (I don't know if this has changed in more recent versions of ext3 or Linux).

I also looked into why there are so many directories created in the root cache directory in the first place. I believe what is happening is Smarty divides the cache files into different directories (/0f/0d/2e/0f0d2ef2...) based on only the template name, but not the cache_id. Every cache ID just creates a new directory in the root cache path.

I don't understand the reasoning behind that. It's unlikely any site would have many many thousands of templates, but a typical database-driven site will have a few templates but many many thousands of cache_id values for each one. Shouldn't Smarty divide up the directories (/0f/0d/2e/0f0d2ef2...) based on the cache_id rather than the template name?
Back to top
View user's profile Send private message
U.Tews
Administrator


Joined: 22 Nov 2006
Posts: 5068
Location: Hamburg / Germany

PostPosted: Fri Jan 27, 2012 9:52 pm    Post subject: Reply with quote

This has been a topic since Smarty2 which used also the items of of cache gruop and compile_id as directory name.

I think the reasion for it was quick execution of clearCache() calls by cache_id or partital cache groups.

There have been some thoughts about changing this, but all ideas ended up in very bad performance of clearCache() as it needs then to loop over all files checking for matching file names.

It is still on our todo list to look for a reasionable approach.

The only workaround I currently can think of is that you split your cache_id with some function in cache groups which will create some sort of distribution.

$cache_id = 'id_part1|id_part2|id_part3'
Back to top
View user's profile Send private message
orrd
Smarty Rookie


Joined: 29 Jun 2010
Posts: 11

PostPosted: Fri Jan 27, 2012 10:48 pm    Post subject: Reply with quote

Thanks for the quick replies. It's great that questions actually get answered in this forum.

I can see how there would be some complications for clearCache() depending on how it's implemented.

Ok, so I need to break it up into subdirs myself using cache groups. The only other issue then is that the dividing (/0f/0d/2e/0f0d2ef2...) that it does based on the template name adds an additional several subdirectories that I think are unnecessary. Ideally I would like the only directories that are created be based on my own cache groups. Is there a flag or setting for that?
Back to top
View user's profile Send private message
U.Tews
Administrator


Joined: 22 Nov 2006
Posts: 5068
Location: Hamburg / Germany

PostPosted: Sat Jan 28, 2012 7:18 am    Post subject: Reply with quote

There is no flag to disable the subdirectories by template name.
You could disable the code in lines 35 to 40 in file smarty_internal_cacheredource_files.php if you don't use clearCache().

I will look into general problem once again in the next couple of days.
Back to top
View user's profile Send private message
orrd
Smarty Rookie


Joined: 29 Jun 2010
Posts: 11

PostPosted: Sat Jan 28, 2012 7:40 am    Post subject: Reply with quote

Thanks.
Back to top
View user's profile Send private message
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Smarty 3 All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP