Smarty Forum Index Smarty
The discussions here are for Smarty, a template engine for the PHP programming language.

Banded Report Generator
Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next
 
Post new topic   Reply to topic    Smarty Forum Index -> Plugins
View previous topic :: View next topic  
Author Message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Wed Jun 22, 2005 2:49 am    Post subject: Reply with quote

boots wrote:
Skimming your code I wasn't sure if you are also handling the report header/footer and not just the group header/footer.


Yep, I am handling the report main summary with this rather clunky solution:

[php:1:a227c6b559]
// handle the main report header stats
$buffer = preg_replace("/##ALL##/", $params['header']['ALL'][0], $buffer,1);
[/php:1:a227c6b559]

It means that you just put ##ALL## where you want the report_header (without a group) to get the report_header_stats (without a group) output.
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Jun 22, 2005 3:18 am    Post subject: Reply with quote

sophistry wrote:
It means that you just put ##ALL## where you want the report_header (without a group) to get the report_header_stats (without a group) output.


Hmm. Now that I resorted to an associative array for my header buffers I should probably do something similar. It might make the code a little leaner...oh well, next time Smile
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Wed Jun 22, 2005 4:21 pm    Post subject: Note: Must Use HTML Entity for dollar sign Reply with quote

There is a problem using the $ (dollar sign) inside the header tags. Because of the multiple levels of processing that the summary data goes through in the new header stats system a dollar sign in the bufferred string or in a string_format modifier confuses the Smarty parser, or maybe it's the PHP parser - not sure which.

Try this. Add a dollar sign in front of the sales (number) data. It will do something weird. Then try to use str_format smarty modifier to change it to a decimal with a dollar sign. Neither works because Smarty (or PHP) thinks that you want to use a template variable (or a variable variable, maybe?).

To get a dollar sign, you have to use the HTML entity for dollar sign: [php:1:eada727383]$[/php:1:eada727383]

[php:1:eada727383]
{report recordset=$data record=rec groups="region" resort=true}

{report_header}
START OF REPORT<hr/>
{/report_header}

{report_header group="region"}

Simple DOLLAR SIGN outside variable<br />
SUMMARY FOR REGION '{$rec.region}'<br />
{$count.sales} items totalling ${$sum.sales} for an average of ${$avg.sales} per item<br />

STRING_FORMAT with DOLLAR SIGN<br>
SUMMARY FOR REGION '{$rec.region}'<br />
{$count.sales} items totalling {$sum.sales|string_format:"$%.2f"} for an average of {$avg.sales|string_format:"$%.2f"} per item<br />

HTML ENTITY <br>
SUMMARY FOR REGION '{$rec.region}'<br />
{$count.sales} items totalling {$sum.sales|string_format:"$%.2f"} for an average of {$avg.sales|string_format:"$%.2f"} per item<br />

{/report_header}

{/report}
[/php:1:eada727383][/code]
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Jun 22, 2005 5:36 pm    Post subject: Reply with quote

Sophistry, are you using a filter of some sort or doing multiple passes on your template output? I ask because I can't reproduce your claim -- results are expected with your examples on my my setup.
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Wed Jun 22, 2005 6:35 pm    Post subject: Reply with quote

The only filter I have on is:

[php:1:1397b64282]
$this->autoload_filters = array('output' => array('trimwhitespace'));
[/php:1:1397b64282]

Look closely at your results. I see no dollar sign on the summary data when i try to put a dollar sign on (except using HTML Entity).

Also, you may want to try the test after coercing your sales entires to strings. I do this:

[php:1:1397b64282]
$sales=rand(2000,20000) . "";
[/php:1:1397b64282]

I see really weird stuff when there is a string in the data that is being formatted with the dollar sign. It looks like it is removing the first two digits. Where it should say "516175", it just says "6175":

[php:1:1397b64282]
START OF REPORT
Simple DOLLAR SIGN outside variable
SUMMARY FOR REGION 'ca'
48 items totalling 6175 for an average of 753.6458333 per item
STRING_FORMAT with DOLLAR SIGN
SUMMARY FOR REGION 'ca'
48 items totalling 6175.00 for an average of 753.65 per item
HTML ENTITY
SUMMARY FOR REGION 'ca'
48 items totalling $516175.00 for an average of $10753.65 per item
Simple DOLLAR SIGN outside variable
SUMMARY FOR REGION 'us'
48 items totalling 0707 for an average of 264.7291667 per item
STRING_FORMAT with DOLLAR SIGN
SUMMARY FOR REGION 'us'
48 items totalling 0707.00 for an average of 264.73 per item
HTML ENTITY
SUMMARY FOR REGION 'us'
48 items totalling $540707.00 for an average of $11264.73 per item
[/php:1:1397b64282]

It is also hard to really see if the presentation is changing when the data is changing too. Why not remove the rand() functions or hard code arrays in the example?

Thanks for the clarifying questions!
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Jun 22, 2005 8:28 pm    Post subject: Reply with quote

bah and grrr.

Well, you encouraged me to check again and at first I was going to report that I still couldn't reproduce the issue. Then I noticed that I was only testing in a non-group header and that I can indeed reproduce your issue in a group header. In fact, in a single report that uses the same code for the report header and the group header, only the group header appears affected.

The only difference I could think of in terms of the implementation is that the report buffer is implemented as a string while the group buffers are implemented as an array of strings. Indeed, inspecting the buffers showed that the content was being stored properly. Replacement for the report header uses a str_replace while the group headers used a preg_replace and this is where it turns out the problem is -- the $ is somehow being interpreted as a backreference -- but I am fairly certain this is incorrect behaviour. If someone can verify that, it should be reported as a PHP bug, I think. I could understand something like ${1} being interpreted as a backreference but by the time the header contents are put into the buffer, something like ${$sum.sales} looks like $123456 and that should not be considered a back-reference, IMO.

Anyways, I've recoded the replacement as follows:
[php:1:3892bb5773]function smarty_block_report__return_content(&$smarty, &$repeat, &$params, $content)
{
// only return content generated stored in the output buffer by sub {report_*}
$buffer = str_replace('##SMARTY_BLOCK_REPORT_HEADER##', $params['header']['report']['buffer'], $params['buffer']);
foreach ($params['header']['group']['buffer'] as $group=>$headers) {
foreach ($headers as $group_buffer) {
$group_buffer = str_replace('$', '##DOLLAR_SIGN##', $group_buffer);
$buffer = preg_replace("/##SMARTY_BLOCK_GROUP_HEADER_{$group}##/", $group_buffer, $buffer, 1);
}
$buffer = str_replace('##DOLLAR_SIGN##', '$', $buffer);
}
return $buffer;
}[/php:1:3892bb5773]
Not great but it seems to do the trick.

BTW: I used rand() because I didn't want to cheat Smile


Last edited by boots on Thu Jun 23, 2005 4:09 pm; edited 1 time in total
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Wed Jun 22, 2005 8:59 pm    Post subject: Reply with quote

off the top of my head, shouldn't that be a generic replacement? if the problem is in the preg_replace backreference confusion, aren't there all kinds of things that can screw up preg_replace?

i don't know off the top of my head, but i suspect.

thanks for figuring out just where the problem was!
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Jun 22, 2005 9:28 pm    Post subject: Reply with quote

sophistry wrote:
off the top of my head, shouldn't that be a generic replacement? if the problem is in the preg_replace backreference confusion, aren't there all kinds of things that can screw up preg_replace?


That would be a replacement in a literal double quoted string but not in a preg_replace replacement coming from a variable, afaik. Besides, vars can't start with numbers in PHP. You are probably right though -- it at least seems that $nn (where nn is a 1 or 2 digit number) is considered a backref.

The only other thing that I can think of that can affect backrefs are the \\nn notation. I suppose I should code around those too. Triple hmmm.
Back to top
View user's profile Send private message
jaey
Smarty Regular


Joined: 25 May 2004
Posts: 36
Location: Louisville, Kentucky, USA

PostPosted: Thu Jun 23, 2005 1:51 pm    Post subject: Reply with quote

boots wrote:
...the $ is somehow being interpreted as a backreference -- but I am fairly certain this is incorrect behaviour. If someone can verify that, it should be reported as a PHP bug, I think. I could understand something like ${1} being interpreted as a backreference but by the time the header contents are put into the buffer, something like ${$sum.sales} looks like $123456 and that should not be considered a back-reference, IMO...


I don't believe this behaviour is a PHP bug. From the PHP manual page for "preg_replace":

Quote:
Replacement may contain references of the form \\n or (since PHP 4.0.4) $n, with the latter form being the preferred one. Every such reference will be replaced by the text captured by the n'th parenthesized pattern. n can be from 0 to 99...


Also, from a comment on the same page:

Quote:
if you have an unknown text as a replace string and you don't want any occurences of $ followed by a number act as a backreference make sure that all $ characters are escaped.

Otherwise your resulting string will be shorter when your searchstring doesn't contain any subpattern


The ${1} format for backreferences is used to avoid ambiguity when you want a literal digit to appear immediately after the backreference.
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Thu Jun 23, 2005 3:41 pm    Post subject: possible general solution to preg_replace problem Reply with quote

here are the characters that are special to preg_replace (according to the php.net manual http://www.php.net/manual/en/function.preg-replace.php )

[php:1:68e1aa9367]. \ + * ? [ ^ ] $ ( ) = ! < > | :[/php:1:68e1aa9367]

There currently are at least 3 comments there that talk about the dollar sign problem with reference to backreferences. None of them made me happy.

Here's what I came up with to solve the problem generally:

[php:1:68e1aa9367]function smarty_block_report__return_content(&$smarty, &$repeat, &$params, $content)
{
// only return content generated stored in the output buffer by sub {report_*}
$buffer = str_replace('##SMARTY_BLOCK_REPORT_HEADER##', $params['header']['report']['buffer'], $params['buffer']);
foreach ($params['header']['group']['buffer'] as $group=>$headers) {
foreach ($headers as $group_buffer) {
// 20050623 escape strings that will mess up preg_replace()
// include the common/default delimiter forward-slash "/"
$group_buffer = preg_quote($group_buffer, "/");
$buffer = preg_replace("/##SMARTY_BLOCK_GROUP_HEADER_{$group}##/", $group_buffer, $buffer, 1);
// remove the extra slashes
$buffer=stripslashes($buffer);
}
}
return $buffer;
}[/php:1:68e1aa9367]

Whaddya think? Surprised
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Thu Jun 23, 2005 4:05 pm    Post subject: Reply with quote

Hi soph.

Well, that set of characters are related to the actual regex search pattern syntax -- replacement has a slightly different syntax. From what I can tell only the $, \, { and } can play a part in the back reference syntax in replacement strings. The $nn issue is for real. I forget about it because I always think the "workaround" notation is ${nn} is superior and does not lead to stupid errors like the default behaviour. Of course, this is considered a "feature" and not a bug because that was how it was implemented so long ago.

Aside: Thanks for the info Jaey -- I looked it up and found the same thing, though I continue to disagree with the reasoning. It is rather non-intuitive that pcre should interpret $123 as ${12}3. Furthermore, the docs are confusing unless you do a real close read. It says: "Replacement may contain references of the form \\n or (since PHP 4.0.4) $n, with the latter form being the preferred one." then goes on to say that: "When working with a replacement pattern where a backreference is immediately followed by another number (i.e.: placing a literal number immediately after a matched pattern), you cannot use the familiar \\1 notation" which mentions the "non-prefered" method (\\1) but does not directly mention (but only backhandedly implies) that $1 suffers the same issue. The fact that it requires so much description and is still confusing, even to someone who has been using preg_replace must mean something. I may be the only person in the world who thinks so, but the implementation is not as good as it should be. Then again, that is a nit-pick and I should have known better Smile

As for the solution to our problem, I had originally considered preg_quote but it is really designed for the regex search pattern syntax. Using stripslashes only covers some of the ground that you would have to backout and it makes me queasy because it will likely affect other things in non-obvious ways. I think I will extend what I already did to use an array or replacements but I am rather unsatisfied. It would be ideal if str_replace supported a limit because we don't really need any of the regex features to begin with. Perhaps I will code the algorithm differently to avoid the preg_replace altogether.
Back to top
View user's profile Send private message
messju
Administrator


Joined: 16 Apr 2003
Posts: 3336
Location: Oldenburg, Germany

PostPosted: Thu Jun 23, 2005 4:21 pm    Post subject: Reply with quote

I don't know if that is of any help (sorry, I didn't read the whole thread), but smarty also had the need for a preg_quote-like quoting of the replace string and implements this in Smarty_Compiler.class.php:
[php:1:07aec3704c]
function _quote_replace($string)
{
return strtr($string, array('\\' => '\\\\', '$' => '\\$'));
}
[/php:1:07aec3704c]

this worked fine for ages Smile
Back to top
View user's profile Send private message Send e-mail Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Thu Jun 23, 2005 4:23 pm    Post subject: Reply with quote

@messju: ahh, thanks! I was just re-implementing about the exact same thing Smile ... but that is only in the compiler, huh? I guess I will have to re-implement it anyways...which sucks, but at least it is known to work well Wink

My other idea was to use preg_match_all to capture the positions of the tags and then work backwards through them use a normal str_replace...but that's overly complicated.

Here is the update:
[php:1:570e419e3c]function smarty_block_report__return_content(&$smarty, &$repeat, &$params, $content)
{
// only return content generated stored in the output buffer by sub {report_*}
$buffer = str_replace('##SMARTY_BLOCK_REPORT_HEADER##', $params['header']['report']['buffer'], $params['buffer']);
foreach ($params['header']['group']['buffer'] as $group=>$headers) {
foreach ($headers as $group_buffer) {
$buffer = preg_replace("/##SMARTY_BLOCK_GROUP_HEADER_{$group}##/", smarty_block_report__quote_replace($group_buffer), $buffer, 1);
}
}
return $buffer;
}

function smarty_block_report__quote_replace($string)
{
return strtr($string, array('\\' => '\\\\', '$' => '\\$'));
} [/php:1:570e419e3c]

Thanks messju!
Back to top
View user's profile Send private message
messju
Administrator


Joined: 16 Apr 2003
Posts: 3336
Location: Oldenburg, Germany

PostPosted: Thu Jun 23, 2005 4:34 pm    Post subject: Reply with quote

boots wrote:
My other idea was to use preg_match_all to capture the positions of the tags and then work backwards through them use a normal str_replace...but that's overly complicated.


again: I'm not deep enough into your problem+solution, to judge if it fits here, but as a rule of thumb: if preg_replace() gets overly complicated (and a tiny little performance can be sacrificed) in many many cases preg_replace_callback() is a good choice. Smile
Back to top
View user's profile Send private message Send e-mail Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Thu Jun 23, 2005 5:25 pm    Post subject: Updated to v0.1.6 Reply with quote

messju wrote:
as a rule of thumb: if preg_replace() gets overly complicated (and a tiny little performance can be sacrificed) in many many cases preg_replace_callback() is a good choice. Smile


heartily agreed -- I always love callbacks...but it is overkill here since we aren't at all interested in supporting captures or back-references here. All we would really need in this case is for str_replace to support a limit just as preg_replace does.

Anyways, I'm satisfied with this solution for now (again, thanks!) and I have updated the source code at the wiki:

v0.1.6 (June 23, 2005)
- fixed problem with replacements in grouped headers (boots, messju, sophistry)
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Smarty Forum Index -> Plugins All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next
Page 3 of 7

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP