Smarty Forum Index Smarty
WARNING: All discussion is moving to https://reddit.com/r/smarty, please go there! This forum will be closing soon.

Implementing another compiler

 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Smarty 3
View previous topic :: View next topic  
Author Message
yankee
Smarty Rookie


Joined: 02 Mar 2009
Posts: 31

PostPosted: Sat Mar 07, 2009 12:05 pm    Post subject: Implementing another compiler Reply with quote

Hi @ll,

I wondered whether it is possible (and useful) to write an additional compiler for an xml-syntax.
I came to the conclusion that it would be helpful to change certain this about smarty to make it easier to exchange the parser.
Please consider these changes and tell me what you think of it.

At first I made the class Smarty_Internal_Base abstract. If I understood the design of smarty correctly, this class should be strict anyway:
Code:
Index: Smarty3Alpha/libs/sysplugins/internal.base.php
===================================================================
--- Smarty3Alpha/libs/sysplugins/internal.base.php      (revision 3024)
+++ Smarty3Alpha/libs/sysplugins/internal.base.php      (working copy)
@@ -11,7 +11,7 @@
 /**
 * Smarty Internal Base Class
 */
-class Smarty_Internal_Base {
+abstract class Smarty_Internal_Base {
     /**
     * Set up instance of Smarty object
     */

Next I introduced the class Smarty_Internal_Abstractcompiler which is an itermediate class between Smarty_Internal_Base and Smarty_Internal_Compiler. It implements most methods of the former class Smarty_Internal_Compiler:
Code:
<?php

/**
* Smarty Internal Plugin Abstract Compiler
*
*
* @package Smarty
* @subpackage Compiler
* @author Uwe Tews
*/
/**
* Main compiler class
*/
abstract class Smarty_Internal_Abstractcompiler extends Smarty_Internal_Base {
    // compile tag objects
    static $_tag_objects = array();
    // tag stack
    public $_tag_stack = array();
    // current template
    public $template = null;

    /**
    * Initialize compiler
    */
    public function __construct() {
        parent::__construct();
        // get required plugins
        if (!is_object($this->smarty->filter_handler) && (isset($this->smarty->autoload_filters['pre']) || isset($this->smarty->registered_filters['pre']) || isset($this->smarty->autoload_filters['post']) || isset($this->smarty->registered_filters['post']))) {
            $this->smarty->loadPlugin('Smarty_Internal_Run_Filter');
            $this->smarty->filter_handler = new Smarty_Internal_Run_Filter;
        }
    }

    /**
    * Methode to compile a Smarty template
    *
    * @param  $template template object to compile
    * @return bool true if compiling succeeded, false if it failed
    */
    public function compileTemplate($template)
    {
        /* here is where the compiling takes place. Smarty
       tags in the templates are replaces with PHP code,
       then written to compiled files. */
        if (!is_object($template->cacher_object)) {
            $this->smarty->loadPlugin($template->cacher_class);
            $template->cacher_object = new $template->cacher_class;
        }
        // flag for nochache sections
        $this->nocache = false;
        $this->tag_nocache = false;
        // assume successfull compiling
        $this->compile_error = false;
        // save template object in compiler class
        $this->template = $template;
        // get template source
        $_content = $template->getTemplateSource();
        // template header code
        $template_header = "<?php /* Smarty version " . Smarty::$_version . ", created on " . strftime("%Y-%m-%d %H:%M:%S") . "\n";
        $template_header .= "         compiled from \"" . $this->template->getTemplateFilepath() . "\" */ ?>\n";
        // run prefilter if required
        if (isset($this->smarty->autoload_filters['pre']) || isset($this->smarty->registered_filters['pre'])) {
            $_content = $this->smarty->filter_handler->execute('pre', $_content);
        }
        // on empty template just return header
        if ($_content == '') {
            $template->compiled_template = $template_header;
            return true;
        }
        // init cacher plugin
        $template->cacher_object->initCacher($this);
        // init the lexer/parser to compile the template
       
        $retvalue =$this->doCompile($_content);

        if (!$this->compile_error) {
            // close cacher and return compiled template
            $template->compiled_template = $template_header . $template->cacher_object->closeCacher($this, $retvalue);
            // run postfilter if required
            if (isset($this->smarty->autoload_filters['post']) || isset($this->smarty->registered_filters['post'])) {
                $template->compiled_template = $this->smarty->filter_handler->execute('post', $template->compiled_template);
            }
            return true;
        } else {
            // compilation error
            return false;
        }
    }
    abstract protected function doCompile($_content);

    /**
    * Compile Tag
    *
    *                                     This is a call back from the lexer/parser
    *                                     It executes the required compile plugin for the Smarty tag
    *
    * @param string $tag tag name
    * @param array $args array with tag attributes
    * @return string compiled code
    */
    public function compileTag($tag, $args)
    {
        // $args contains the attributes parsed and compiled by the lexer/parser
        // assume that tag does compile into code, but creates no HTML output
        $this->has_code = true;
        $this->has_output = false;
        // compile the smarty tag (required compile classes to compile the tag are autoloaded)
        if (!($_output = $this->$tag($args, $this)) === false) {
            // did we get compiled code
            if ($this->has_code) {
                // Does it create output?
                if ($this->has_output) {
                    $_output .= "\n";
                }
                // return compiled code
                return $_output;
            }
            // tag did not produce compiled code
            return '';
        } else {
            // not an internal compiler tag
            // check if tag is a registered object
            if (isset($this->smarty->registered_objects[$tag]) && isset($args['object_methode'])) {
                $methode = $args['object_methode'];
                unset ($args['object_methode']);
                if (!in_array($methode, $this->smarty->registered_objects[$tag][3]) &&
                        (empty($this->smarty->registered_objects[$tag][1]) || in_array($methode, $this->smarty->registered_objects[$tag][1]))) {
                    return $this->object_function($args, $tag, $methode, $this);
                } elseif (in_array($methode, $this->smarty->registered_objects[$tag][3])) {
                    return $this->object_block_function($args, $tag, $methode, $this);
                } else {
                    return $this->trigger_template_error ('unallowed methode "' . $methode . '" in registered object "' . $tag . '"');
                }
            }
            // check if tag is registered or is Smarty plugin
            $this->smarty->plugin_handler->loadSmartyPlugin($tag, $this->smarty->plugin_search_order);
            if (isset($this->smarty->registered_plugins[$tag])) {
                // check no cache
                if (!$this->smarty->registered_plugins[$tag][2]) {
                    $this->tag_nocache = true;
                }
                // if compiler function plugin call it now
                if ($this->smarty->registered_plugins[$tag][0] == 'compiler') {
                    return call_user_func_array($this->smarty->registered_plugins[$tag][1], array($args, $this));
                }
                // compile function or block plugin
                $plugin_type = $this->smarty->registered_plugins[$tag][0] . '_plugin';
                return $this->$plugin_type($args, $tag, $this);
            }
            // compile closing tag of block function
            if (strlen($tag) > 5 && substr_compare($tag, 'close', -5, 5) == 0) {
                $base_tag = substr($tag, 0, -5);
                // check if closing tag is a registered object
                if (isset($this->smarty->registered_objects[$base_tag]) && isset($args['object_methode'])) {
                    $methode = $args['object_methode'];
                    unset ($args['object_methode']);
                    if (in_array($methode, $this->smarty->registered_objects[$base_tag][3])) {
                        return $this->object_block_function($args, $tag, $methode, $this);
                    } else {
                        return $this->trigger_template_error ('unallowed closing tag methode "' . $methode . '" in registered object "' . $base_tag . '"');
                    }
                }
                // plugin ?
                if (isset($this->smarty->registered_plugins[$base_tag]) && $this->smarty->registered_plugins[$base_tag][0] == 'block') {
                    return $this->block_plugin($args, $tag, $this);
                }
            }
            $this->trigger_template_error ("unknown tag \"" . $tag . "\"");
        }
    }

    /**
    * lazy loads internal compile plugin for tag and calls the compile methode
    *
    * compile objects cached for reuse.
    * class name format:  Smarty_Internal_Compile_TagName
    * plugin filename format: internal.compile_tagname.php
    *
    * @param  $tag string tag name
    * @param  $args array with tag attributes
    * @return string compiled code
    */
    public function __call($name, $args)
    {
        // re-use object if already exists
        if (isset(self::$_tag_objects[$name])) {
            // compile this tag
            return call_user_func_array(array(self::$_tag_objects[$name], 'compile'), $args);
        }
        // lazy load internal compiler plugin
        $class_name = "Smarty_Internal_Compile_{$name}";
        if ($this->smarty->loadPlugin($class_name)) {
            // use plugin if found
            self::$_tag_objects[$name] = new $class_name;
            // compile this tag
            return call_user_func_array(array(self::$_tag_objects[$name], 'compile'), $args);
        }
        // no internal compile plugin for this tag
        return false;
    }

    /**
    * display compiler error messages without dying
    *
    * If parameter $args is empty it is a parser detected syntax error.
    * In this case the parser is called to obtain information about expected tokens.
    *
    * If parameter $args contains a string this is used as error message
    *
    * @todo output exact position of parse error in source line
    * @param  $args string individual error message or null
    */
    public function trigger_template_error($args = null)
    {
        $this->lex = Smarty_Internal_Templatelexer::instance();
        $this->parser = Smarty_Internal_Templateparser::instance();
        // get template source line which has error
        $line = $this->lex->line;
        if (isset($args)) {
            // $line--;
        }
        $match = preg_split("/\n/", $this->lex->data);
        $error_text = 'Syntax Error in template "' . $this->template->getTemplateFilepath() . '"  on line ' . $line . ' "' . $match[$line-1] . '" ';

        if (isset($args)) {
            // individual error message
            $error_text .= $args;
        } else {
            // expected token from parser
            foreach ($this->parser->yy_get_expected_tokens($yymajor) as $token) {
                $exp_token = $this->parser->yyTokenName[$token];
                if (isset($this->lex->smarty_token_names[$exp_token])) {
                    // token type from lexer
                    $expect[] = '"' . $this->lex->smarty_token_names[$exp_token] . '"';
                } else {
                    // otherwise internal token name
                    $expect[] = $this->parser->yyTokenName[$token];
                }
            }
            // output parser error message
            $error_text .= ' - Unexpected "' . $this->lex->value . '", expected one of: ' . implode(' , ', $expect);
        }
        throw new Exception($error_text);
        // set error flag
        $this->compile_error = true;
    }
}

?>

I load this abstract class here:
Code:
Index: Smarty3Alpha/libs/sysplugins/internal.template.php
===================================================================
--- Smarty3Alpha/libs/sysplugins/internal.template.php  (revision 3024)
+++ Smarty3Alpha/libs/sysplugins/internal.template.php  (working copy)
@@ -250,6 +250,7 @@
         if (!is_object($this->compiler_object)) {
             // load compiler
             $this->smarty->loadPlugin('Smarty_Internal_CompileBase');
+            $this->smarty->loadPlugin('Smarty_Internal_Abstractcompiler');
             $this->smarty->loadPlugin($this->compiler_class);
             $this->compiler_object = new $this->compiler_class;
         }

I changed the Class Smarty_Internal_Compiler to extend the previously mentioned abstract class instead of the Smarty_Internal_Base class. I removed all methods that are now implementet by its new superclass:
Code:
Index: Smarty3Alpha/libs/sysplugins/internal.compiler.php
===================================================================
--- Smarty3Alpha/libs/sysplugins/internal.compiler.php  (revision 3024)
+++ Smarty3Alpha/libs/sysplugins/internal.compiler.php  (working copy)
@@ -13,14 +13,8 @@
 /**
 * Main compiler class
 */
-class Smarty_Internal_Compiler extends Smarty_Internal_Base {
-    // compile tag objects
-    static $_tag_objects = array();
-    // tag stack
-    public $_tag_stack = array();
-    // current template
-    public $template = null;
-
+class Smarty_Internal_Compiler extends Smarty_Internal_Abstractcompiler {
+
     /**
     * Initialize compiler
     */
@@ -30,51 +24,10 @@
         // get required plugins
         $this->smarty->loadPlugin('Smarty_Internal_Templatelexer');
         $this->smarty->loadPlugin('Smarty_Internal_Templateparser');
-        if (!is_object($this->smarty->filter_handler) && (isset($this->smarty->autoload_filters['pre']) || isset($this->smarty->registered_filters['pre']) || isset($this->smarty->autoload_filters['post']) || isset($this->smarty->registered_filters['post']))) {
-            $this->smarty->loadPlugin('Smarty_Internal_Run_Filter');
-            $this->smarty->filter_handler = new Smarty_Internal_Run_Filter;
-        }
     }

-    /**
-    * Methode to compile a Smarty template
-    *
-    * @param  $template template object to compile
-    * @return bool true if compiling succeeded, false if it failed
-    */
-    public function compileTemplate($template)
+    protected function doCompile($_content)
     {
-        /* here is where the compiling takes place. Smarty
-       tags in the templates are replaces with PHP code,
-       then written to compiled files. */
-        if (!is_object($template->cacher_object)) {
-            $this->smarty->loadPlugin($template->cacher_class);
-            $template->cacher_object = new $template->cacher_class;
-        }
-        // flag for nochache sections
-        $this->nocache = false;
-        $this->tag_nocache = false;
-        // assume successfull compiling
-        $this->compile_error = false;
-        // save template object in compiler class
-        $this->template = $template;
-        // get template source
-        $_content = $template->getTemplateSource();
-        // template header code
-        $template_header = "<?php /* Smarty version " . Smarty::$_version . ", created on " . strftime("%Y-%m-%d %H:%M:%S") . "\n";
-        $template_header .= "         compiled from \"" . $this->template->getTemplateFilepath() . "\" */ ?>\n";
-        // run prefilter if required
-        if (isset($this->smarty->autoload_filters['pre']) || isset($this->smarty->registered_filters['pre'])) {
-            $_content = $this->smarty->filter_handler->execute('pre', $_content);
-        }
-        // on empty template just return header
-        if ($_content == '') {
-            $template->compiled_template = $template_header;
-            return true;
-        }
-        // init cacher plugin
-        $template->cacher_object->initCacher($this);
-        // init the lexer/parser to compile the template
         $lex = new Smarty_Internal_Templatelexer($_content);
         $parser = new Smarty_Internal_Templateparser($lex, $this);
         // $parser->PrintTrace();
@@ -91,177 +44,8 @@
             list($_open_tag, $_data) = array_pop($this->_tag_stack);
             $this->trigger_template_error("unclosed {" . $_open_tag . "} tag");
         }
-
-        if (!$this->compile_error) {
-            // close cacher and return compiled template
-            $template->compiled_template = $template_header . $template->cacher_object->closeCacher($this, $parser->retvalue);
-            // run postfilter if required
-            if (isset($this->smarty->autoload_filters['post']) || isset($this->smarty->registered_filters['post'])) {
-                $template->compiled_template = $this->smarty->filter_handler->execute('post', $template->compiled_template);
-            }
-            return true;
-        } else {
-            // compilation error
-            return false;
-        }
-    }
-
-    /**
-    * Compile Tag
-    *
-    *                                     This is a call back from the lexer/parser
-    *                                     It executes the required compile plugin for the Smarty tag
-    *
-    * @param string $tag tag name
-    * @param array $args array with tag attributes
-    * @return string compiled code
-    */
-    public function compileTag($tag, $args)
-    {
-        // $args contains the attributes parsed and compiled by the lexer/parser
-        // assume that tag does compile into code, but creates no HTML output
-        $this->has_code = true;
-        $this->has_output = false;
-        // compile the smarty tag (required compile classes to compile the tag are autoloaded)
-        if (!($_output = $this->$tag($args, $this)) === false) {
-            // did we get compiled code
-            if ($this->has_code) {
-                // Does it create output?
-                if ($this->has_output) {
-                    $_output .= "\n";
-                }
-                // return compiled code
-                return $_output;
-            }
-            // tag did not produce compiled code
-            return '';
-        } else {
-            // not an internal compiler tag
-            // check if tag is a registered object
-            if (isset($this->smarty->registered_objects[$tag]) && isset($args['object_methode'])) {
-                $methode = $args['object_methode'];
-                unset ($args['object_methode']);
-                if (!in_array($methode, $this->smarty->registered_objects[$tag][3]) &&
-                        (empty($this->smarty->registered_objects[$tag][1]) || in_array($methode, $this->smarty->registered_objects[$tag][1]))) {
-                    return $this->object_function($args, $tag, $methode, $this);
-                } elseif (in_array($methode, $this->smarty->registered_objects[$tag][3])) {
-                    return $this->object_block_function($args, $tag, $methode, $this);
-                } else {
-                    return $this->trigger_template_error ('unallowed methode "' . $methode . '" in registered object "' . $tag . '"');
-                }
-            }
-            // check if tag is registered or is Smarty plugin
-            $this->smarty->plugin_handler->loadSmartyPlugin($tag, $this->smarty->plugin_search_order);
-            if (isset($this->smarty->registered_plugins[$tag])) {
-                // check no cache
-                if (!$this->smarty->registered_plugins[$tag][2]) {
-                    $this->tag_nocache = true;
-                }
-                // if compiler function plugin call it now
-                if ($this->smarty->registered_plugins[$tag][0] == 'compiler') {
-                    return call_user_func_array($this->smarty->registered_plugins[$tag][1], array($args, $this));
-                }
-                // compile function or block plugin
-                $plugin_type = $this->smarty->registered_plugins[$tag][0] . '_plugin';
-                return $this->$plugin_type($args, $tag, $this);
-            }
-            // compile closing tag of block function
-            if (strlen($tag) > 5 && substr_compare($tag, 'close', -5, 5) == 0) {
-                $base_tag = substr($tag, 0, -5);
-                // check if closing tag is a registered object
-                if (isset($this->smarty->registered_objects[$base_tag]) && isset($args['object_methode'])) {
-                    $methode = $args['object_methode'];
-                    unset ($args['object_methode']);
-                    if (in_array($methode, $this->smarty->registered_objects[$base_tag][3])) {
-                        return $this->object_block_function($args, $tag, $methode, $this);
-                    } else {
-                        return $this->trigger_template_error ('unallowed closing tag methode "' . $methode . '" in registered object "' . $base_tag . '"');
-                    }
-                }
-                // plugin ?
-                if (isset($this->smarty->registered_plugins[$base_tag]) && $this->smarty->registered_plugins[$base_tag][0] == 'block') {
-                    return $this->block_plugin($args, $tag, $this);
-                }
-            }
-            $this->trigger_template_error ("unknown tag \"" . $tag . "\"");
-        }
-    }
-
-    /**
-    * lazy loads internal compile plugin for tag and calls the compile methode
-    *
-    * compile objects cached for reuse.
-    * class name format:  Smarty_Internal_Compile_TagName
-    * plugin filename format: internal.compile_tagname.php
-    *
-    * @param  $tag string tag name
-    * @param  $args array with tag attributes
-    * @return string compiled code
-    */
-    public function __call($name, $args)
-    {
-        // re-use object if already exists
-        if (isset(self::$_tag_objects[$name])) {
-            // compile this tag
-            return call_user_func_array(array(self::$_tag_objects[$name], 'compile'), $args);
-        }
-        // lazy load internal compiler plugin
-        $class_name = "Smarty_Internal_Compile_{$name}";
-        if ($this->smarty->loadPlugin($class_name)) {
-            // use plugin if found
-            self::$_tag_objects[$name] = new $class_name;
-            // compile this tag
-            return call_user_func_array(array(self::$_tag_objects[$name], 'compile'), $args);
-        }
-        // no internal compile plugin for this tag
-        return false;
-    }
-
-    /**
-    * display compiler error messages without dying
-    *
-    * If parameter $args is empty it is a parser detected syntax error.
-    * In this case the parser is called to obtain information about expected tokens.
-    *
-    * If parameter $args contains a string this is used as error message
-    *
-    * @todo output exact position of parse error in source line
-    * @param  $args string individual error message or null
-    */
-    public function trigger_template_error($args = null)
-    {
-        $this->lex = Smarty_Internal_Templatelexer::instance();
-        $this->parser = Smarty_Internal_Templateparser::instance();
-        // get template source line which has error
-        $line = $this->lex->line;
-        if (isset($args)) {
-            // $line--;
-        }
-        $match = preg_split("/\n/", $this->lex->data);
-        $error_text = 'Syntax Error in template "' . $this->template->getTemplateFilepath() . '"  on line ' . $line . ' "' . $match[$line-1] . '" ';
-
-        if (isset($args)) {
-            // individual error message
-            $error_text .= $args;
-        } else {
-            // expected token from parser
-            foreach ($this->parser->yy_get_expected_tokens($yymajor) as $token) {
-                $exp_token = $this->parser->yyTokenName[$token];
-                if (isset($this->lex->smarty_token_names[$exp_token])) {
-                    // token type from lexer
-                    $expect[] = '"' . $this->lex->smarty_token_names[$exp_token] . '"';
-                } else {
-                    // otherwise internal token name
-                    $expect[] = $this->parser->yyTokenName[$token];
-                }
-            }
-            // output parser error message
-            $error_text .= ' - Unexpected "' . $this->lex->value . '", expected one of: ' . implode(' , ', $expect);
-        }
-        throw new Exception($error_text);
-        // set error flag
-        $this->compile_error = true;
-    }
+        return $parser->retvalue;
+    }
 }

Now a compiler for another syntax can extend the Smarty_Internal_Abstractcompiler class and it does not need to do more than just overriding the abstract "doCompile"-Method.
The compiler that one wants to use can already be specified in Smarty's main class.

I think that smarty's build in syntax is best for most purposes. I don't want to criticize it, but I think it might be a benefit in certain situations...
Back to top
View user's profile Send private message
U.Tews
Administrator


Joined: 22 Nov 2006
Posts: 5068
Location: Hamburg / Germany

PostPosted: Sat Mar 07, 2009 1:11 pm    Post subject: Reply with quote

Thanks for you sugestion.

I will look into the details.

The template parser is already configurable.

The following properties of the Smarty class define which lexer / parser classes are being loaded.

public $template_lexer_class = 'Smarty_Internal_Templatelexer';
public $template_parser_class = 'Smarty_Internal_Templateparser';

Just create another set of lexer and parser for example for xml syntax.


Regards Uwe
Back to top
View user's profile Send private message
yankee
Smarty Rookie


Joined: 02 Mar 2009
Posts: 31

PostPosted: Sun Mar 08, 2009 10:23 am    Post subject: Reply with quote

I am working on the parser for xml, but I do not intend to use the parser generator that you used. Thus my parser will have a different layout. By just loading a different class as lexer/parser will easily result in code that's hard to read.
Back to top
View user's profile Send private message
U.Tews
Administrator


Joined: 22 Nov 2006
Posts: 5068
Location: Hamburg / Germany

PostPosted: Sun Mar 08, 2009 6:13 pm    Post subject: Reply with quote

I picked that up.

If you want to implement your own compiler class create a resource script. For example internal.resource_xml.php. In this case you could just copy internal.resource_file.php.

Modify

public $compiler_class = 'Smarty_Internal_SmartyTemplateCompiler';
public $template_lexer_class = 'Smarty_Internal_Templatelexer';
public $template_parser_class = 'Smarty_Internal_Templateparser';

to your needs.

If you do not use a template_lexer_class and template_lexer_class set it's values to null.

FYI There will be some code cleanup and changes at the resource scripts.

Regards
Uwe
Back to top
View user's profile Send private message
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    Smarty Forum Index -> Smarty 3 All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP