Smarty Forum Index Smarty
The discussions here are for Smarty, a template engine for the PHP programming language.
Dedicated server web hosting provided by Guru-host.eu.
use htmlentities() on every assigned variable
Goto page 1, 2  Next
 
Post new topic   Reply to topic    Smarty Forum Index -> General
View previous topic :: View next topic  
Author Message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Tue Apr 05, 2005 10:15 am    Post subject: use htmlentities() on every assigned variable Reply with quote

Hi!

To prevent XSS I would like to use htmlentities() on every variable, assigned to any template. The best would be if something like that does not need to modify all assigned values (arrays, objects...), but modify only all (smarty-) variables in the template, when compiling it. I don't think that this could be solved using post/output filters, because these can only work on the whole template, not only on the values inserted (AFAIK).

Do you have any ideas what is the best place to implement something like that, or do you know a (existing) solution?

best regards
Andreas


Last edited by andreas on Wed May 05, 2010 10:33 pm; edited 1 time in total
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5608
Location: Toronto, Canada

PostPosted: Tue Apr 05, 2005 5:02 pm    Post subject: Reply with quote

http://smarty.php.net/manual/en/variable.default.modifiers.php
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Tue Apr 05, 2005 5:29 pm    Post subject: Reply with quote

That's what I was looking for!

Thank you very much!


Last edited by andreas on Wed May 05, 2010 10:33 pm; edited 1 time in total
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Tue Apr 05, 2005 7:28 pm    Post subject: Reply with quote

OK, unfortunately it does not work for me Sad

I can brake it using the following code:

php:
Code:

$smarty->default_modifiers = array('escape:"htmlall"');
$smarty_test['a']['test'] = 1;
$smarty_test['b']['test'] = 2;
$smarty->assign('smarty_test', $smarty_test);


tpl:
Code:
{foreach from=$smarty_test item="t"}
  {$t.test}
{/foreach}


If I do this, I geht the following error:

Quote:
Warning: htmlentities() expects parameter 1 to be string, array given in /usr/lib/php/Smarty/plugins/modifier.escape.php on line 32


It took me some time to figure out what happends.

I put the following code into
Code:
function smarty_modifier_escape($string, $esc_type = 'html')
in modifier.escape.php:

Code:
if ($string != (string) $string) {
  var_dump(debug_backtrace());
  exit;
}


to get a backtrace to see where this is happening:

Code:
array
  0 =>
    array
      'file' => '/usr/lib/php/Smarty/Smarty.class.php'
      'line' => 1685
      'function' => 'smarty_modifier_escape'
      'args' =>
        array
          0 =>
            array
              'test' => 1
          1 => 'htmlall'
  1 =>
    array
      'file' => '/usr/lib/php/Smarty/Smarty.class.php'
      'line' => 1685
      'function' => 'call_user_func_array'
      'args' =>
        array
          0 => 'smarty_modifier_escape'
          1 =>
            array
              0 =>
                array
                  'test' => 1
              1 => 'htmlall'
  2 =>
    array
      'file' => '/var/www/test/templates_c/de-^%%01^01A^01AA9B3A%%header.tpl.php'
      'line' => 5
      'function' => '_run_mod_handler'
      'class' => 'Template'
      'type' => '->'
      'args' =>
        array
          0 => 'escape'
          1 => true
          2 =>
            array
              'a' =>
                array
                  'test' => 1
              'b' =>
                array
                  'test' => 2
          3 => 'htmlall'
  3 =>
    array
      'file' => '/usr/lib/php/Smarty/Smarty.class.php'
      'line' => 1865
      'args' =>
        array
          0 => '/var/www/test/templates_c/de-^%%01^01A^01AA9B3A%%header.tpl.php'
      'function' => 'include'



The code from the generated(compiled) template:
Code:
<?php /* Smarty version 2.6.8, created on 2005-04-05 21:07:37
         compiled from file:/var/www/test/templates/header.tpl */ ?>
<?php require_once(SMARTY_CORE_DIR . 'core.load_plugins.php');
smarty_core_load_plugins(array('plugins' => array(array('modifier', 'escape', 'file:/var/www/test/templates/header.tpl', 1, false),)), $this); ?>
<?php $_from = ((is_array($_tmp=$this->_tpl_vars['smarty_test'])) ? $this->_run_mod_handler('escape', true, $_tmp, 'htmlall') : smarty_modifier_escape($_tmp, 'htmlall')); if (!is_array($_from) && !is_object($_from)) { settype($_from, 'array'); }if (count($_from)):
    foreach ($_from as $this->_tpl_vars['t']):
 echo ((is_array($_tmp=$this->_tpl_vars['t']['test'])) ? $this->_run_mod_handler('escape', true, $_tmp, 'htmlall') : smarty_modifier_escape($_tmp, 'htmlall'));  endforeach; endif; unset($_from); ?>


I do not understand why the data is escaped twice, I think it is not necessary. First of all the whole array($smarty_test) is escaped:

Code:
is_array($_tmp=$this->_tpl_vars['smarty_test'])) ? $this->_run_mod_handler('escape', true, $_tmp, 'htmlall') : smarty_modifier_escape($_tmp, 'htmlall')


which actually does not work because it is a multidimensional array, so a recursive function would be necessary.

But every value (string) which gets displayed is escaped again:

Code:
    foreach ($_from as $this->_tpl_vars['t']):
echo ((is_array($_tmp=$this->_tpl_vars['t']['test'])) ? $this->_run_mod_handler('escape', true, $_tmp, 'htmlall') : smarty_modifier_escape($_tmp, 'htmlall'));  endforeach; endif; unset($_from);


The first part does not work, the second part works fine, and makes everything you need (as far as I can see). Couldn't the first part (escape whole array) be removed when compiling?

Perhaps I misunderstood something, but I never looked into smarty-code before. If this a bug please tell me where I should report it.


best regards,
Andreas


Last edited by andreas on Wed May 05, 2010 10:33 pm; edited 1 time in total
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5608
Location: Toronto, Canada

PostPosted: Tue Apr 05, 2005 7:40 pm    Post subject: Reply with quote

Searching the forums can save time:

http://www.phpinsider.com/smarty-forum/viewtopic.php?t=2179
http://www.phpinsider.com/smarty-forum/viewtopic.php?t=4011
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Tue Apr 05, 2005 9:42 pm    Post subject: Reply with quote

boots wrote:
Searching the forums can save time:

I searched before...

boots wrote:
http://www.phpinsider.com/smarty-forum/viewtopic.php?t=2179

and I found this one.

But:

1. I get a PHP-Warning when using a multidimensional array, so I think smarty_modifier_escape() or _run_mod_handler() from smarty 2.6.8 is broken (I'm using PHP 5.0.3).

2. I cannot find a posting which really solves the problem. There are some ideas, I think the idea of writing my own smarty_modifier_escape() function would be OK for the moment, but it is not an perfect solution.

What about the ideas of changing default_modifier or adding default_modifier_print?


Last edited by andreas on Wed May 05, 2010 10:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5608
Location: Toronto, Canada

PostPosted: Tue Apr 05, 2005 10:17 pm    Post subject: Reply with quote

Does this not work?

Code:
{foreach from=$smarty_test|smarty:nodefaults item="t"}
  {$t.test}
{/foreach}


In one of the threads I linked I made the comment that I don't really see the point of default modifiers. Its not hard to see why: in short, default modifiers suffer the same logical flaw as PHP's magic quotes, namely, they escape input. The reason that is flawed is due to the fact that input should be filtered while output should be escaped.

That's also why the code example above looks messy -- you have to essentially unescape input when you want to use it directly. Because default modifiers suffers from such a grave conceptual flaw I doubt they are fixable. Even if there was a "default_modifiers_print" how would it work? How can you define in advance what is printable or not without knowing what is in the template? The point: assigning vars to the template does not give us any clue as to how they will be used in the template. Therefore, you are generally best not using default_modifiers and instead ought manually escape data as necessary for output.

Greetings.


Last edited by boots on Tue Apr 05, 2005 10:53 pm; edited 2 times in total
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Tue Apr 05, 2005 10:48 pm    Post subject: Reply with quote

I have not tried because I had to change a lot of files. I think it would work, but I don't like to solve it this way.

I thought about using a default_modifiers function which does something like that:

Code:
if (!is_scalar($string)) {
  return $string;
}
else {
  return htmlspecialchars($string, ENT_QUOTES);
}


That works too, but the function is often called which could be avoided.

I have done the following changes (to 2.6.Cool, which (as far as I can see) solve my problem: I only apply default_modifiers if _parse_var_props is called from parsing a simple variable:

diff -u Smarty_Compiler.class.php_orig Smarty_Compiler.class.php:
Code:
--- Smarty_Compiler.class.php_orig  2005-04-06 00:21:09.000000000 +0200
+++ Smarty_Compiler.class.php  2005-04-06 00:22:44.000000000 +0200
@@ -442,7 +442,7 @@

         if (preg_match('~^' . $this->_num_const_regexp . '|' . $this->_obj_call_regexp . '|' . $this->_var_regexp . '$~', $tag_command)) {
             /* tag name is a variable or object */
-            $_return = $this->_parse_var_props($tag_command . $tag_modifier);
+            $_return = $this->_parse_var_props($tag_command . $tag_modifier, true);
             return "<?php echo $_return; ?>" . $this->_additional_newline;
         }

@@ -1591,7 +1591,7 @@
      * @param string $tag_attrs
      * @return string
      */
-    function _parse_var_props($val)
+    function _parse_var_props($val, $modify=false)
     {
         $val = trim($val);

@@ -1599,7 +1599,7 @@
             // $ variable or object
             $return = $this->_parse_var($match[1]);
             $modifiers = $match[2];
-            if (!empty($this->default_modifiers) && !preg_match('~(^|\|)smarty:nodefaults($|\|)~',$modifiers)) {
+            if ($modify===true && !empty($this->default_modifiers) && !preg_match('~(^|\|)smarty:nodefaults($|\|)~',$modifiers)) {
                 $_default_mod_string = implode('|',(array)$this->default_modifiers);
                 $modifiers = empty($modifiers) ? $_default_mod_string : $_default_mod_string . '|' . $modifiers;
             }


It probably breaks other code. After this change the default_modifier is only called for display-variables (if I understood the code correct). This is the way I would like such an encode-modifier to work. I'm not sure how such a behaviour could be implemented with BC, I think the best way is something like the suggested default_modifier_print or something like that.

best regards
Andreas


Last edited by andreas on Wed May 05, 2010 10:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5608
Location: Toronto, Canada

PostPosted: Tue Apr 05, 2005 11:01 pm    Post subject: Reply with quote

hmm. Actually, not too bad of a compromise...I'll have to test it but in the end I suspect it is still not suitable for the reasons I gave above.

Last edited by boots on Tue Apr 05, 2005 11:08 pm; edited 1 time in total
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Tue Apr 05, 2005 11:04 pm    Post subject: Reply with quote

boots wrote:
default modifiers suffers the same logical flaw as PHP's magic quotes. Namely, they escape input. The problem is that input should be filtered and output should be escaped.

Escaping output is what I try to do here.

boots wrote:
That's why the code example above looks messy -- you have to essentially unescape input when you want to use it directly.

I do not want to escape input. I only want to escape these strings, which will be desplayed in html and become directly visible to the browser.

boots wrote:
That's why default modifiers are usually not a good idea and also why there probably isn't a simple solution that can fix the situation.

I don't know smarty good enough to be sure, but what is the problem with my patch above? Isn't it possible to detect smarty-variables which will be displayed directly during parsing? These variables will allways contain scalar values, or not? OK, I think I missed things like HTML-Options, but theses issues are not difficult to solve I think. For me these re not important because there will no user-input get in here. What I'm looking for is a kind of policy - to make sure that no bad html/js will be displayed to the browser. Of course you should validate user-input before, and it would be nicer to encode every variable itself. But this is no policy, you have to to this everytime with every variable, no developer in a large project may forget it! Can you ensure this? Why do you hear nearly every day of xss-problems in software 1000s of people are using?

boots wrote:
Even if there was a "default_modifiers_print" how would it work?

As described above. But I must have missed something? Wink

boots wrote:
How can you define in advance what is printable or not without knowing what is in the template?

I'm not interested in "is printable", im interested in "is printed". And before printing it, I would like to escape it Wink


Last edited by andreas on Wed May 05, 2010 10:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Tue Apr 05, 2005 11:27 pm    Post subject: Reply with quote

boots wrote:
I'll have to test it but in the end I suspect it is still not suitable for the reasons I gave above.

OK, this could be a problem, but doesn't the same apply to the actual implementation of default_modifiers? The only difference is, that I do not apply this to all assigned variables, but only to these which get printed (that's what I intended to do, I think you can see better if this works or not).

What is missing is to apply something like my change to the html_* functions too. At the moment I think I do not need it, but I think its better to implement it too (perhaps optional).

If it is a problem with BC, I would implement it as default_modifier_output or so.


Last edited by andreas on Wed May 05, 2010 10:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5608
Location: Toronto, Canada

PostPosted: Tue Apr 05, 2005 11:46 pm    Post subject: Reply with quote

I understand why you are trying to solve this -- XSS is an important problem. Still, I think that your patch has the same general problem as default modifiers though I do think it is probably more usable. I'll try to illustrate my point by using a (contrived) example --

Say I had an assigned string:
[php:1:0227114206]$smarty->assign('sometext', 'love > science');[/php:1:0227114206]
Now I want to use a hypothetical custom plugin that takes text as input and generates a custom image for each character in the text using a different font styling. It then returns a string of html <img> tags that point to each of the generated images.
Code:
{colorize text=$sometext}

I haven't tested yet but I think your method breaks that (because of the '>') unless you use:
Code:
{colorize text=$sometext|smarty:nodefaults}

Again, the point is that default modifers really are the same as escaping input because anytime the variable is accessed, it is escaped. You're patch is probably an improvement because it limits the escaping behaviour to scalars but as my example tries to demonstrate, you can't be sure when a scalar should be escaped or not. Only the template writer knows for sure.

Note that even if you wanted to rule out escaping for parameter inputs, you still have modifiers to deal with. If the above plugin was a modifier instead of a function {$sometext|colorize} is still "broken".

That said, I do think your patch could be helpful.
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Wed Apr 06, 2005 7:17 am    Post subject: Reply with quote

I don't think this will break, because of

[php]if (preg_match('~^' . $this-&gt;_num_const_regexp . '|' . $this-&gt;_obj_call_regexp . '|' . $this-&gt;_var_regexp . '$~', $tag_command)) {
/* tag name is a variable or object */[/php]

but I'm not sure. Only if

Code:
{colorize text=$sometext}


does match this regex it will break.

The important point is to filter variables during parsing, which directly output content. As said I'm not sure if my solution is doing this exactly.

The only question is - if this task is solvable.

On the other hand I think such a global output-escape is by far more important (which as someone said not many people use at the moment, because of its limitations), than any complex functionality - which also not many people use. You do not need to use it! You have to activate it yourself! But I think if you activate this, you will prevent xss on the whole site, without a lot of effort. I think many people want to escape strings themselves, also for many applications it does not fit if they want to output html, or use modifiers which get strings with "&gt;"... but I think most people could use this without any difference and can enhance security of their web-applications.


Last edited by andreas on Wed May 05, 2010 10:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
andreas
Smarty Pro


Joined: 22 Apr 2003
Posts: 106

PostPosted: Wed Apr 06, 2005 8:09 am    Post subject: Reply with quote

Perhaps you could somehow mark modifiers/functions which will output html! I think this could solve the problem. Perhaps it would be useful to implement functionality so that these plugins could also use the escaping. HTML_* function could be implemented somewhere without changing these functions (I think), but you can't do this with other user-functions. Perhaps one could implement something these functions can use if they want to escape data if default_modifiers are enabled.

And yes, I see that this will break BC (but only for these people who use functions which output html AND use default_modifer for escaping ATM!), but wouldn't it be great to make it possible to generally escape output of variables in a convenient way? You don't need any nodefaults, you don't need own escape-functions...

I also see some limitations:

1. of someone who does not use html-escape before could get difficulties with inputfield-names, but that has nothing to do with my changes, this applies to the actual default_modifier too.

2. if someone wants to assign html-code to the templates, then he cannot use the default_modifier, or has to use nodefault. But he has to care for escaping himself, I don't think this is a good idea, but I'm sure there are situations where this is useful. But to say this again, nobody _must_ use default_modifiers. I only would like to finde an implementation which is more useful for most users.

3. if someone uses own functions/modifiers which output html or get special characters as input. I would try to solve this using some kind of "html_output" and/or "html_input" flag. If this is not set, the output should be escaped. Perhaps something like that.

I think XSS is one of the biggest problems in todays web-apps. In a template-engine as smarty I think you have the means to allow a convenient solution to the problem - at least for most people.

There have been ideas to change the behaviour of default_modifiers more than 1 year ago. Of course you cannot solve every problem, there will allways be situations when a global solution will not work. But as far as I can see it will be helpful for most applications. You have to weigh up which is more important - a convenient way to escape html-output, or break BC in user-functions working with html (but these are only affected if you activate default_modifier, and this is the same problem you have already today!).

The importance of XSS could be seen here:

* recent security-note from php-devs:
http://www.php.net/security-note.php
* recent foundation of http://phpsec.org/
* http://groups-beta.google.com/groups?q=php+xss&qt_s=Search+Groups finds more than 1000 results

(small extra-benefit: you will save some [useless] methode-calls with my solutions at runtime, if you use default_modifiers)


Last edited by andreas on Wed May 05, 2010 10:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
boots
Administrator


Joined: 16 Apr 2003
Posts: 5608
Location: Toronto, Canada

PostPosted: Sat Apr 09, 2005 4:45 pm    Post subject: Reply with quote

I really don't believe that security can be automated away. Never-the-less, I think that this patch has a better behaviour than the current implementation.

Monte, Messju, do either of you have an opinion on this?
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Smarty Forum Index -> General All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP