Smarty Forum Index Smarty
WARNING: All discussion is moving to https://reddit.com/r/smarty, please go there! This forum will be closing soon.

Banded Report Generator
Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next
 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Plugins
View previous topic :: View next topic  
Author Message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Jun 15, 2005 7:14 pm    Post subject: Re: bug? with nested sort levels... Reply with quote

sophistry wrote:
It looks like the code determines if it is the last record in the group by testing if the curr is equal to the prev record. However, if the group one level up changes and the current group doesn't, this code thinks that the group has finished.


Maybe I am reading you wrong. If you are grouping on a,b and a changes then b must have ended as well. This is implied by sort ordering.

eg:

1999 OCT
1999 NOV <- end 'b' on change on 'b'
1999 DEC <- end 'b' on change on 'b'
2000 DEC <- end 'b' on change on 'a' (implied change on 'b')

I can't think of a reasonable way of dealing with this otherwise because 'b' is always assumed to be a subtotal of 'a'.

Is this not what happens? I am fairly sure I tested this back when. Perhaps I should test again Smile
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Wed Jun 15, 2005 7:31 pm    Post subject: nested sorting bug fixed Reply with quote

Here's the new code. It needed a check on the previous group's records too. It needed to check if the nesting level above had changed:

Code:

// process grouping levels
    foreach ($params['groups'] as $_group) {
        if (!is_null($record['curr'])) {
          // DEC 20050615
            if ($record['prev'][$_group] != $record['curr'][$_group] OR ($record['prev'][$prev_group] != $record['curr'][$prev_group])) {
                $group[$_group]['first'] = true;
                foreach ($record['curr'] as $field=>$value) {
                    $group[$_group]['stats']['sum'][$field] = $value;
                    $group[$_group]['stats']['count'][$field] = 1;
                    $group[$_group]['stats']['avg'][$field] = $value;
                }
            } else {
                $group[$_group]['first'] = false;
                foreach ($record['curr'] as $field=>$value) {
                    if (is_numeric($value)) {
                        $group[$_group]['stats']['sum'][$field] += $value;
                        ++$group[$_group]['stats']['count'][$field];
                        $group[$_group]['stats']['avg'][$field] = $group[$_group]['stats']['sum'][$field]/$group[$_group]['stats']['count'][$field];
                     }
                }
            }


            if ($record['last']) {
                $group[$_group]['last'] = true;
            // DEC 20050615
            } else if (($record['curr'][$_group] == $record['next'][$_group]) AND ($record['curr'][$prev_group] != $record['next'][$prev_group])) {
               $group[$_group]['last'] = true;
            } else if (($record['curr'][$_group] != $record['next'][$_group])) {
                $group[$_group]['last'] = true;
            } else {
                $group[$_group]['last'] = false;
            }
        } else {
            $group[$_group]['first'] = is_null($record['prev']);
            $group[$_group]['last'] = is_null($record['next']);
        }
       
        // DEC 20050615 to store previous group for nested comparisons
        $prev_group = $_group;
    }


Please check this to make sure I'm not seeing things... Very Happy
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Jun 15, 2005 8:19 pm    Post subject: Re: nested sorting bug fixed Reply with quote

sophistry wrote:
Here's the new code. It needed a check on the previous group's records too. It needed to check if the nesting level above had changed:

I haven't tested thoroughly but your code does indeed handle cases for where mine gave unexpected behaviour. Nicely done! This tells me that you got your head around the most difficult parts of the plugins now--first the structure and now the actual iteration logic when spanning multiple groups. That's a lot to digest so I am quite pleased that you were able to follow what I was doing and quite impressed at how quickly you picked it up.

I won't update the code I posted for now so anyone following this thread may want to manually replace the relevant portions to get this improved behaviour.

Thanks Sophistry!
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Wed Jun 15, 2005 8:27 pm    Post subject: Re: smarty architecture of nested plugins Reply with quote

boots wrote:
- exploit the buffering option that these plugins use. Since actual output is defered to the end of the report block, you can add additional logic to do replacements in the buffer itself before it is finally returned to Smarty. For example, you can have a new tag type (say {report_header_stats}) which is triggered using the same logic as {report_footer} (so it can have access to all of the stats that footer does) but writes to a separate buffer. You would still define a {report_header}, but that plugin would silently insert a placeholder or allow you to manually insert a placeholder (eg: to keep things simple, you might use something like ##HEADER_STATS## for now.) When the recordset is fully iterated and it came time to flush the output buffer (in smarty_block_report__close), you would first merge the output buffer with the special header stats buffer by replacing on the pre-inserted placeholders with the content collected in the header stat buffer. In this case, it would be easier to maintain the header stats buffer as an array while the normal output buffer would remain a string. None of the buffers would require associative arrays as far as I can see.


That's great!

Now I have a lucid description that I think I can piece together into working code.

Thanks!
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Thu Jun 16, 2005 10:08 pm    Post subject: buffering problems Reply with quote

Hi Boots,

Buffering / merging is not going well. I am having no trouble doing a simple string replace on the whole output buffer. For instance, I put the ##HEADER_STATS## manual tag into the {report_header} tag and str_replace() it in the block close function. That's fine. I can replace one string with another string. Whoopee.

But, I can't seem to get a grip on where I access the stats to stuff them into the replacement. I've created the duplicate header stats plugin so it behaves like footer and instead of appending content as a string it tries to stuff content into an array element.

I also added a named element (header_stats) to the assoc array $_params in the init function with the idea that it could hold the stats as they are gathered by the header_stats tag.

But, it's not working. I'm learning a lot about the way these plugins work, but not enough to solve this knot.

Regarding header_stats tag/plugin... Shouldn't it be possible to take what the footer has produced and str_replace it in the same way in the close block function? Why should I duplicate footer?

Further, isn't it possible using get_template_vars() function to access these data points? They are assigned into smarty template vars. The trouble is, I'm not sure how to address them while I'm inside the close function. Embarassed

Any ideas/direction?
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Thu Jun 16, 2005 11:01 pm    Post subject: Re: buffering problems Reply with quote

Hi sophistry.
sophistry wrote:
But, I can't seem to get a grip on where I access the stats to stuff them into the replacement. I've created the duplicate header stats plugin so it behaves like footer and instead of appending content as a string it tries to stuff content into an array element.

It should stuff each evaluated $content into a separate entry in the array.
sophistry wrote:
Regarding header_stats tag/plugin... Shouldn't it be possible to take what the footer has produced and str_replace it in the same way in the close block function? Why should I duplicate footer?

Because you want to allow the template author to control each of them separately -- they can and probably will have differing layouts.
sophistry wrote:
Further, isn't it possible using get_template_vars() function to access these data points? They are assigned into smarty template vars. The trouble is, I'm not sure how to address them while I'm inside the close function. Embarassed

You aren't interested in the values of the stats -- you are interested in the returned content which is by nature already processed and hence variable and code free.

Lets look at how it might work. Everytime a {report_header} plugin inserts a ##HEADER_STATS## placeholder (yes, the placeholders can all have the same name) the expectation is that a {report_header_stats} will follow and it will place a new element into the internal header stats array (ie: its processed content). So, as an example, you might have the {report_header_stats} block stuff the buffer as so:
[php:1:80281a8f40] $_parent_params['report']['header']['buffer'][] = $content;[/php:1:80281a8f40]
In theory, there are as many ##HEADER_STATS## placeholders in the $_parent_params['report']['buffer'] as there are entries in $_parent_params['report']['header']['buffer'] and they are correlated in the order that they appear in both. Your replacement logic could look something like:
[php:1:80281a8f40] $cnt = preg_match_all('/(##HEADER_STATS##)/', $_parent_params['report']['buffer'], $list);
if ($cnt) {
foreach ($list[1] as $placeholder) {
// replace each placeholder in turn
$replacement = array_shift($_parent_params['report']['header']['buffer']);
$_parent_params['report']['buffer'] = str_replace('##HEADER_STATS##', $replacement, $_parent_params['report']['buffer'], 1);
}
}
// remove any spurious entries
str_replace('##HEADER_STATS##', '', $_parent_params['report']['buffer']);[/php:1:80281a8f40]

Needless to say, that's untested and it can be easily broken if the placeholders are not co-ordinated with the stats but it approximates what I was suggesting and it should be possible to fashion something of that sort into a working implementation.

Let me know if I can be of further assistance.
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Fri Jun 17, 2005 3:57 pm    Post subject: trouble understanding scope Reply with quote

Hi Boots,

Thanks for the direction. I had most of the code you posted in place - the array appending, the replacement code.

But, I after trying my way and your way I am still unable to get this thing working.

I see that you used $_parent_params in your code in the previous post which implies that the code should be in the child plugin. However, I tried to put that code into the header_stats child plugin like this:

[php:1:d3270557cc]
function smarty_block_report_header_stats($params, $content, &$smarty, &$repeat)
{
$_parent_params =& smarty_get_parent_plugin_params($smarty, 'report');
if (is_null($content)) {
/* handle block open tag */
if (!array_key_exists('group', $params)) {
// report header stats
if (!$_parent_params['report']['record']['last']) {
$repeat = false;
return;
} else {
foreach ($_parent_params['report']['stats'] as $stat_type=>$stat) {
$smarty->assign($stat_type, $stat);
}
}
} else {
// group header stats
if (!in_array($params['group'], $_parent_params['report']['record']['fields'], true)) {
$smarty->trigger_error("{report_header_stats}: given group '{$params['group']}' does not have a corresponding record field in given recordset.", E_USER_ERROR);
$repeat = false;
return;
}
if (!$_parent_params['report']['group'][$params['group']]['last']) {
$repeat = false;
return;
} else {
foreach ($_parent_params['report']['group'][$params['group']]['stats'] as $stat_type=>$stat) {
$smarty->assign($stat_type, $stat);
}
}
}
return;
} else {
/* handle block close tag */
// DEC 20050616 store the stats in the parent buffer as an array
// so that on report block close array can be iterated to
// replace the proper ##HEADER_STATS## tags
$_parent_params['report']['header']['buffer'][] = $content;
//$_parent_params['report']['buffer'] .= $content;

// replace the string with the stats
// by reaching up to the parent tag buffer to get the string that's already written
$cnt = preg_match_all('/(##HEADER_STATS##)/', $_parent_params['report']['buffer'], $list);
// var_export($cnt);
if ($cnt>0) {
foreach ($list[1] as $placeholder) {
// replace each placeholder in turn
$replacement = array_shift($_parent_params['report']['header']['buffer']);
$_parent_params['report']['buffer'] = str_replace('##HEADER_STATS##', $replacement, $_parent_params['report']['buffer'], 1);
}
}
// remove any spurious entries
str_replace('##HEADER_STATS##', '', $_parent_params['report']['buffer']);


return;
}
}
[/php:1:d3270557cc]

This didn't work. It kept displaying NULL.

So, I moved the replacement code up into the main report plugin close code and changed the variables to get the proper scope and still it doesn't work.

Close function in report plugin. This creates warnings about arrays being missing.
[php:1:d3270557cc]
function smarty_block_report__close(&$smarty, &$repeat, &$params, $content)
{
// only content generated stored in the output buffer by sub {report_*}
// blocks is returned; the buffer is emptied for the next iteration

$buffer = $params['buffer'];

// replace the string with the stats
// by reaching up to the parent tag buffer to get the string that's already written
$cnt = preg_match_all('/(##HEADER_STATS##)/', $buffer, $list);
// var_export($cnt);
if ($cnt>0) {
foreach ($list[1] as $placeholder) {
// replace each placeholder in turn
$replacement = array_shift($params['header']['buffer']);
$buffer = str_replace('##HEADER_STATS##', $replacement, $buffer, 1);
}
}
// remove any spurious entries
str_replace('##HEADER_STATS##', '', $buffer);


//$buffer = $params['buffer'] . 'BUFFER_CLEAR';
//$buffer = $buffer . '<pre>' . var_export($params['group'],true) . '</pre>' . 'BUFFER_CLEAR';
$buffer = $buffer . '<pre>BUFFER_HEAD_START ' . var_export($params['header']['buffer'],true) . var_export($cnt,true) . ' BUFFER_HEAD_END</pre>';
if (!empty($buffer)) {

// DEC 20050616
// replace the ##HEADER_STATS## string with the results of the header_stats tag
//$buffer=str_replace('##HEADER_AVG##','I GOT REPLACED',$buffer);
$params['buffer'] = '';
return $buffer;
}
} [/php:1:d3270557cc]

And finally, I've added this array to the data structure at the init stage of report:
[php:1:d3270557cc]
$_params = array(
// setup the basic report data structure
'recordset' => (is_array($params['recordset'])) ? $params['recordset'] : array()
, 'buffer' => ''
, 'header' => array()
, 'group' => array()
, 'groups' => array()
, 'stats' => array()

// setup the record data sub-structure
, 'record' => array(
'name' => $params['record']
, 'count' => (is_array($params['recordset'])) ? count($params['recordset']) : 0
, 'first' => false
, 'last' => false
, 'iteration' => 0
, 'prev' => null
, 'curr' => null
, 'next' => null
, 'fields' => (is_array($params['recordset']) && count($params['recordset'])) ? array_keys($params['recordset'][0]) : array()
)
);
[/php:1:d3270557cc]


So, I have two questions:

1) What is the best way to debug in a architecture like this? The only thing I've been able to see/print/dump is at the report plugin buffer level where I've simply appended my debug statements to the buffer. But, sometimes I just want to see variable states lower down in the code.

2) What is going wrong here? I think it is a scoping problem and that I am not using the correct syntax to get at the variables, but I can't see how to fix it. Basically because I can't really visualize the 'handoff' points where a child plugin dumps it's load to the parent.

I know you said this was low priority for you, so I really appreciate you taking the time to set me straight on this!
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Fri Jun 17, 2005 6:32 pm    Post subject: Arrrrgh! I was editing the wrong file! Reply with quote

Embarassed Crying or Very sad

When i first made this new header_stats plugin file I had created a file with an underscorre instead of a dot.

block_report_header_stats.php

instead of the correct

block.report_header_stats.php

I corrected my error immediately by making the new properly named file. But, then I proceded to spend many, many (too many) hours editing a file that wasn't being called - it was the old, wrongly named file, and I didn't notice! Mad

Now, I am making a little bit of progress... and I'll let you know if I have any more questions.

This does highlight one danger in working at such a remove. I only discovered my error when I started putting in exit(); calls and when I didn't exit out of the header_stats file, it hit me like a hammer.

I think that must be a common error to name the files badly as it was one of the first corrections to this thread.

Beware the badly named file.
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Fri Jun 17, 2005 11:16 pm    Post subject: Reply with quote

Hi sophistry -- not much time to comment but I do want to say that I used $_parent_params merely as a result of a quick copy-and-paste. As I said, that wasn't tested -- it was merely typed directly into the forum editing box. I apologize if that lead to some confusion. The final buffer replacement ought to be done where I originally stated: in smarty_block_report__close().

As for using the wrong filenames -- well, it is very hard to protect ourselves from ourselves, yes? The wrong filename that was in the thread was a posting error, not a coding error but you are correct -- it is very easy to lose your place in an abstracted system. I have certainly fallen pray to it myself. What I usually do at that point is get some rest (!) and then when refreshed, come back and force myself to diagram out all of the related physical and logical items that I need to work on. I do that even though I typically already have a diagram to work from because I find that acting it out once again helps me better immerse myself in it.
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Sat Jun 18, 2005 9:15 pm    Post subject: Reply with quote

Hmm. Sophistry, I was thinking about some of your comments and it finally occurred to me that a separate {report_header_stats} was not needed and that (almost) all of the work could be done in {report_header} after all. The trick would be to have {report_header} trigger twice. The first time would be on the normal header trigger (ie: as-is) except that instead of returning $content, it would return the placeholder text (##FOO##) back to the regular buffer in the normal way. The second trigger would use the same logic as a {report_footer} tag and would append $content into an array as discussed previously. You would do the merge that was already discussed in smarty_block__close.
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Tue Jun 21, 2005 9:27 pm    Post subject: Reply with quote

Well, I'll eat some of my words. I decided to try to implement the last idea and it made me realize that smarty_block_report__close is called on *every* iteration of the report and that some more logic is needed to signal the buffer completion and to coordinate the buffer handling. I'm working on it but I can't promise anything timely. Just thought I'd mention it in-case someone else was trying to implement some of these ideas and running into problems (*cough* *cough*)
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Tue Jun 21, 2005 9:44 pm    Post subject: it works but it isn't pretty Reply with quote

so, i finally bent my head around the code and it broke (my head not the code!).

i got it working by forcing the last close tag to do the bulk replacement of header replacement tags.

the incremental replace code suggested earlier was breaking down because of unaccountable close tag triggers that were clearing the buffer before the replacement data was in place.

so, i made it just stuff the summary data into the report-header buffer array (i ended up using an associative array). Then, it loops through the groups and grabs the corrsepondingly named buffer array elements - but only on the last iteration of the report block.

i also added a "whole recordset" replacement line.

once i clean it up, i'll post the complete new relevant code bits tomorrow and see what you think.

but, for now, here is the core of the new report block:

[php:1:36cc7e7c74]
function smarty_block_report__close(&$smarty, &$repeat, &$params, $content)
{
static $hack_to_stop_double_buffer_dump_on_report_close=0;
$buffer = $params['buffer'];

if (!empty($buffer)) {
// 20050616
// replace the auto-named ##HEADER_STATS## string with the results of the header_stats tag
// 20050621 don't empty the buffer each time
// just wait until the last record
//$params['buffer'] = '';
if ($params['record']['last']==TRUE) {
if ($hack_to_stop_double_buffer_dump_on_report_close==1) {
//exit('<pre>'.var_export($params['header'],true).'</pre>');
// replace the string with the stats in each group
foreach ($params['groups'] as $groupname){
$cnt = preg_match_all("/(##$groupname##)/", $buffer, $list);
if ($cnt AND is_array($params['header'][$groupname])) {
foreach ($list[1] as $placeholder) {
// replace each placeholder in turn
$replacement = array_shift($params['header'][$groupname]);
// only relace the first one by using optional limit parameter
$buffer = preg_replace("/##$groupname##/", $replacement, $buffer,1);
}
}
}
// handle the main report header stats
$buffer = preg_replace("/##ALL##/", $params['header']['ALL'][0], $buffer,1);

return $buffer;
}else{
$hack_to_stop_double_buffer_dump_on_report_close++;
return;
}
}else{
return;
}
}
}
[/php:1:36cc7e7c74]

As you can see I couldn't really effectively control the 'last' record trigger so i made an ugly dumb hack. please let me know if you know why the 'last' test triggers twice.
Back to top
View user's profile Send private message
sophistry
Smarty Rookie


Joined: 31 Jan 2005
Posts: 33

PostPosted: Tue Jun 21, 2005 9:47 pm    Post subject: Reply with quote

oh, i see we were compsing messages at the same time.

curious to see what slick solution you come up with boots. Very Happy
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Tue Jun 21, 2005 11:05 pm    Post subject: Reply with quote

Hi again Sophistry.

I think I might have found a slightly better way of dealing with the __close issue. I too was putting more logic in __close to try and wait for the last iteration; as you note, it was feeling rather hacky. Instead, I added a single if condition in smarty_report_block to ensure that __close is only called when the report block tag is actually completed. Here is the revised smarty_block_report:
[php:1:ed4dea3299]function smarty_block_report($params, $content, &$smarty, &$repeat)
{
$_params =& smarty_get_current_plugin_params($smarty);

if (is_null($content)) {
$_params['report'] = smarty_block_report__init($smarty, $repeat, $params);
smarty_block_report__open($smarty, $repeat, $_params['report']);

} else {
smarty_block_report__open($smarty, $repeat, $_params['report']);
if (!$repeat)
return smarty_block_report__close($smarty, $repeat, $_params['report'], $content);
}
}
[/php:1:ed4dea3299]
Now __close only gets called if $content isn't null and $repeat is true.

Skimming your code I wasn't sure if you are also handling the report header/footer and not just the group header/footer.

Overall, I'm still not that happy with the overall implementation but I'm willing to live with it for now. Its quite fast enough for my current needs. I will update the code fairly soon--someone was kind enough to post the plugins at the wiki so I will do the update there and remove the code from the original post to avoid confusion.


Last edited by boots on Wed Jun 22, 2005 2:48 am; edited 1 time in total
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed Jun 22, 2005 2:47 am    Post subject: Updated to v0.1.5 Reply with quote

I've updated the source code at the wiki. Its about as much as I'm willing to do with it at the moment. It has passed my minimal tests -- it could stand some more rigorous testing.

Updates

v0.1.5 (June 21, 2005)
- general code cleanup (boots)
- headers now have access to grouping stats (boots)
- fixed grouping bug (sophistry)


Notes

The internal smarty_block_report__* functions have been renamed to smarty_block_report__initialize(), smarty_block_report__process_next() and smarty_block_report__return_content() to better reflect their purpose.

I didn't use a single indexed array for buffer headers after all as they would result in out-of-order headers. This is due to the complication that the header placeholder is inserted into the report buffer at a different time than the related content is collected and in the intervening period, an inner group can start (thereby inserting another placeholder) and then finish before the outer group does, thereby resulting in the out-of-order. So I added an associative dimension to the buffer but I still tried to keep the total number of buffers used and the replacement strategy minimal.

Additionally, I removed boiler-plated code and conditions in an effort to tidy things up and hopefully make some of the logic a bit clearer.

Thanks, sophistry, for suffering through the previous code iteration and for your comments, suggestions and fixes!


Last edited by boots on Wed Jun 22, 2005 3:25 am; edited 2 times in total
Back to top
View user's profile Send private message
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Plugins All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next
Page 2 of 7

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP