Smarty Forum Index Smarty
The discussions here are for Smarty, a template engine for the PHP programming language.

Reverse Transformation

 
Post new topic   Reply to topic    Smarty Forum Index -> Feature Requests
View previous topic :: View next topic  
Author Message
macinta
Smarty n00b


Joined: 07 May 2003
Posts: 3
Location: USA::MA::Boston

PostPosted: Wed May 07, 2003 2:51 am    Post subject: Reverse Transformation Reply with quote

Hello,

I've been charged with either finding or making a template system for a large-ish website that uses PHP and I have a question about a feature that doesn't appear to be in Smarty or any other template system that I know of. I'd like to know if anybody could give me some feedback on 1) what it would take to retro-fit Smarty with this feature; 2) whether somebody has suggestions that would achieve the same goals and would be simpler to retro-fit Smarty for; and/or 3) whether anybody has ever seen such a feature before in a template system?

Typically, a templating system works like this:
Code:


  Template1 ---
               \
  Template2 ------> Transform --> HTML
               /
  Template3 ---




The feature that I want is this:

Code:


                                --> Template1
                               /
  HTML --> Reverse-Transform -----> Template2
                               \
                                --> Template3




Before anybody tells me that this is insanely difficult, I will say that the "how" is coming below and seeing this idea (which wasn't my idea originally, incidentally) might make it worth your while you read through this very long message. First, the why...

I consider ease of use by designers to be extremely important in a templating system. Ideally, this would mean that there is seamless WYSIWYG support so that designers who use things like Dreamweaver can edit a *single* HTML file with *real* data on it and have the changes reflected in the templates that were used to build that page. I know there is a temptation to ask what's so hard about editing Template1, Template2, etc by hand, but for designers that only know Dreamweaver and don't know HTML, it's a moot point - regardless of how hard it is, it's not an option in a lot of cases.

I a huge fan of DOM manipulation templating solutions like XMLC and HTML_tree for this reason. They allow designers to work with valid HTML and realistic sample data which generates a fully realistic mockup. They don't, however, support a reverse transformation so there is still a degree of having to figure out which template to edit to effect which change.

Below is how this could be achieved in a system that would be implemented from scratch. The basic gist is that you can put the templating system in "debug" mode, which would then result in pages being generated with <span> tags wrapping template fragments and variables so as to identify them later. The designer would edit the single, exported page (which contains real data instead of variable names) in a WYSWIG editor and then upload the modified page to the template system. The template system would then use the <span> tags to figure out which templates created which pieces of the page and then modify those templates accordingly.

That's the Reader's Digest summary - see below for *much* more detail. My question again is whether something similar could be built into Smarty and what this would take? I know this would be extremely difficult to make work correctly with probably the majority of Smarty's features, but I'd still be happy even if getting a reverse transformation feature necessitated that I use a certain sub-set of Smarty features in my templates.

Enabling the reverse transformation and importation by utilizing <span> tags was originally Benjamin 'Quincy' Cabell V's idea, by the way, to give credit where credit is due. His summary of how he would design such a system from scratch is below.

Thanks,
- Tim Macinta


From: Quincy <TemplateX@besiex.org>
-----------------------------------

The features which I believe are critical to a good templating system
are the following:

- Multi-language
- Multi-style
- Caching
- Variable AND constant substitution
- Easy variable insertion (in template, no coding, per se)
- Significant separation of html and code
- Edit Once, Edit Everywhere

In my opinion all of the templating systems I have seen fail on these
last two points. On the issue of separating HTML and code, they succeed
only to the degree that they eradicate HTML from inside PHP files. They
separate HTML from code by spreading one page's HTML (final rendered
page the user will see) into dozens of template pieces which are then
merged together in a way that makes editing the GUI at a later time
difficult because of the need to a) read the code (or some a document
about the code/templates) to understand which template contains which
"piece" of the layout and b) modify each individual template (of dozens
or more) in a coordinated fashion to modify the design. These two
failures/weaknesses make templating systems much less effective than
they could be.

But, before talking about a solution to that, let me first cover the
other issues.

Multi-language and mult-style. This is really one feature, but looks
like two. The idea is that you have parallel template sets for style
and within those parallel template sets for language. So, each template
has a name, a style, and a language setting. To find the template you
would look in the database for the appropriate template of the
appropriate style, then the appropriate language. If you fail to find
the template of the appropriate language you fall back to the default
style with the appropriate language. If you fail to find that you fall
back to the appropriate style with the default langauge. If you fail to
find that you fall back to the default style and default language (which
will always exist). The setting of the style/language is optional
(otherwise default), and the setting of it could occur in the
initializer to the template class, or at any point, or specifically in a
template inclusion call.

Caching is somewhat obvious, somewhat not. The ability to store the
processed template (post variable/constant substitution) for X seconds
and have this get used instead of "creating" a new one. The bit that's
a bit awkward is that you are not caching to prevent the variable
substitution, which is pretty simple and efficient, you are caching to
prevent the PHP that would get the data ready to be laid out in a
template. So, the actual PHP code (business logic) would essentially
check with the template cache manager code to see if the content is in
cache (according to hash info and the rules of that template) and if it
is it returns it and bypasses the PHP code which would have done the SQL
calls/computations/etc. to create whatever data/templates the template
might have required. It's the generic object cache I had Tim Macinta
build, in the setting of a template manager.

Variable and constant substitution. The variable substitution part is
just that. It's the thing every template system does most
fundamentally. Replace a marker in the template with the value of some
variable. The "constant" substitution is a bit of a misnomer. It's
actually a style/language-specific constant substitution. Meaning that
you could define page_bg_color to be #0000cc in the "Cool Blue" style,
and #cc0000 in the "Hot Red" style. And, you could substitute other
things such as key strings on a language-by-language basis. So, the
constants are NOT constants in the true PHP sense (and they are not
defined as constants in the PHP code), but items which are small-ish
fragments of text you want to put in your pages, which will not change
based on page execution, and which get placed into templates. (This
variable/constant is functionally identical to vBulletin's template
system.)

Easy variable insertion is just an awkward way of saying that when
you're editing a template it shouldn't be painful, requiring code-like
things, require many more keystrokes than simply referring to the
variable/constant you're substituting by a name. Most systems do this
reasonably well.

Back to the thorny and significant problem of separation of html and
code. The other features are relatively common and relatively easy to
achieve (caching is a bit complicated, but it's a pretty "well
understood" problem). This item, though, is a a bit of a bitch. And I
don't think it has to be, and I think I know a way out of this maze.
Personally I've found templating systems hideous to use. Again, you
just can't edit things easily when you use them. You have to trace back
the building of the page, see all the templates used, discern from their
names or their order of execution what they do, edit the one or many
templates you need to (and the numbers can be truly staggering, see
vBulletin with its dozens and dozens of templates for one viewed page),
then hope to god you didn't miss a closing tag and have to journey into
a nightmarish world to find which template has one too many or too few
tags. On some level, this "cure" becomes worse than the original
"disease". So, what to do?

I'll present part of a possible solution. It's one I worked on a bit a
year or two ago, and started thinking again a bit when I was asking Tim
Macinta to look into templating systems for our site. From his initial
research it seems the state of templating systems still hadn't improved
(enough) to meet what I perceived as my/our needs. I'll present a
possible solution. It's one I worked on a bit a year or two ago, and
started thinking about again after the renewed look at templating
systems. If anyone reads this and responds, "Oh, I've seen that before,
it's in Foo Bar Inc.'s Fubaramatic Templating System." please forgive
me and let me know where it's to be found!

The goal is simply this:

Let's say I decide to clean up the look of our new "search system". I
open up a browser to my site, load up the search form, execute a search,
get a bunch of results, now that I'm looking at a page on the site that
contains all the content areas/items I want to edit I then I add a
"&templatex=1" to the URL (or set a cookie named templatex), the page
reloads, looks exactly like it did a moment ago (but has invisible
differences in hidden meta markup), I save the page with IE. This is a
"real" page, it has real results on it, not bogus sample data, but real
results I got from the real page. I open up Macromedia's Dreamweaver,
change the order of the cells in the result table, change the background
colors in the cells from alternating light and dark grey to alternating
light blue and light red, edit some copyright text, replace a few
graphics, and save. I go to the "TemplateX" management page, go the
"import" form, browse to and upload the HTML page I just edited (just
the HTML file, no graphics). I hit "submit". And the system
automagically makes the required changes to ALL the required templates
and constants (the style/language ones). No more trying to understand
how the templates fit together, or which has what, no more editing
multiple template files.

Now... I think it's possible to achieve all of that goal, relatively
easily, within certain limits. Here is some info on how such a system
could be achieved, and some of the limitations.

Achieving this goal goes a little like this. Every substitution (from a
variable or constant) is wrapped in a <span></span> wrapper. The
wrapper includes addition info in attributes about what did the
substitution (variable or constant, and the variable name). Each
template is wrapped in <span> telling with attributes which template it
is, and what language and style it is. And, for tag attributes, for
every "real" attribute where a substitution occurred, a paired fake
attribute exists which tells what variable/constant did the substitution
(could also do all this in one fake attribute per tag). There's a bit
more to how this would work, but these are the key points.

Now for some limitations. The templates cannot be "anything"; they must
be HTML fragments (text which can contain HTML tags, but no <table (with
no closing>) and attributes cannot be "partially" replaced by variables
(such as bgcolor=#cc$foo), an attribute may have a value of a variable,
or a constant (and perhaps even another template (which in turn could be
created by part of a variable) but maybe not I haven't thought this
out). But, a tag could not be like <table $border_attribute>, though it
could have <table border=$border_width>. If a template is included
multiple times only modifications to the first instance (presumably)
would be processed (the others would cause no harm, but the changes
there would be ignored, and would in fact be wrapped with span tags
indicating the data there was just sample data, not related to a
template). The page you edit contains the substituted variables, so it
renders as a real page, with real data. As these items are wrapped in
<span> tags with vital identifiers in the tags, the designer cannot
corrupt the wrapping when the move/alter the "text"
(<font><span>foo</span></font> would be very different than
<span><font>foo</font></span>). Similarly, as the HTML tags may contain
"false" attributes used to identify tag attribute substitutions, those
"false" attributes must also be unaltered by the designer.

The parser would read the document, identify the templates involved for
each region, reconstructing the variables/constants that were
substituted, and re-create the various template files (and alter
constants that had changed).

So, that's it. It should work, and I believe it would be a great
advancement over anything I've seen/used.

We could start with some core features, initially supporting no
"constants" and no tag attribute substitution. The parsing of the
document I do not believe should be too difficult, as the logic would be
pretty simple and the <span> tags we're using would be assumed to be
very exactly defined and should thus be easy to parse/locate via regex.
The true benefit of this "Edit Once, Edit Everywhere" design may not be
perfect. Dreamweaver or other applications may insist on putting <font>
tags on the inside of a <span> when you format text, and so true WYSIWYG
may not come. We may find we need to constuct our templates differently
than we might with other template systems, such as defining a <table>
header twice (with different attributes) rather than do variable
substitutions dynamically to add attributes in one template on the fly.
And so on. Nonetheless, this system would still be better and easier to
use than any current templating system out there. Assuming that it is
this major step forward in templating systems we could/should release it
GPL or LGPL to the open source community and allow others to develop it
further, which is likely to occur.



Import/Export Tool
------------------
In addition to the system mentioned above, where a designer can make
changes to one HTML file then automagically update all the appropriate
templates based on the changes made in that one file, another useful
method for doing large volumes of template changes, an import/export
tool would be useful. While the templates are stored in the database,
there would be great benefit in having a tool to import/export template
files to/from a file system for ease in making large scale template
changes/edits. In this situation, I would type in "templatex --export
/home/quincy" and it creates a directory structure roughly as I outlined
(with langauge directories, and style sub-directories), you can then
edit the files with Dreamweaver/TextPad32/Homesite/Emacs/etc. and then
run "templatex --import /home/quincy" and it imports them, creating
template entries for 'new' files and over-writing entries for edited
ones. My primary reason for believing this feature is needed is because
of my experience in doing lots of vBulletin changes, and feeling pretty
limited/frustrated by needing to do all edits within their limited
template editing system, instead of being able to use whatever tools I
like, leveraging their power, for syntax highlighting, HTML
preview/editing, doing global search and replaces, global finds, etc.


Automating Template Determination/Fetching
------------------------------------------
There are many lessons we can learn from experience with vBulletin and
seeing how they have implemented their system. Within each vBulletin
page they call a function which pre-fetches the templates that the page
expects it will need, saving on DB calls; and with my hack that caches
fetched templates to the filesystem bypassing the DB altogether. We
should (probably) try to create a system which automagically knows which
templates will be required for which scripts, based on previous runs of
the script (it would "learn"), as well as being able to manually control
which templates are pre-fetched. Of course we can start with manually
setting it, and later do it automagically. I just mention this feature
because with vB it gets very taxing to manually maintain the list, when
sometimes you've got 20 or 30 templates in the list, and you need to
remember which you've added/removed/etc. And if you forget a couple
then of course it needs to separately fetch them as needed, which is
(probably) less efficient than the overhead of automagically fetching
too many.


Parsing of Updated HTML File
----------------------------
The reverse transformation which occurs when the parser is asked to
process a final document that you modified (with a text editor or
Dreamweaver) back into the mods to the individual templates
is easy because it has been purposely "dumbed down" with only one
"branch" to follow, ever. That branch is the one that was used in the
actual page generation that you edit. Originally my goal when I started
thinking about this last year was to try to handle all the
cases/branches when you edit one page, but it's clearly a nightmare and
forces the templates or the editing of them to contain "code"
(code-suggestive markups, at least). So, I think the solution is to
bypass the issue entirely, and go with the simlpe one branch approach.
So, if I want to change the entire look of the search page, that does
mean I need to run through all of the possible variations of the search
page (showing 0 results, showing X results, in this mode, that mode,
etc.). And I don't think that's a bad thing, actually. Every time you
make a change that will show up in the other variations, it will show up
there when you check them, so it's not that you'll be doing extra work.
Now, the one danger is of course that you could forget about one of the
application's variations. But, I would say that risk is probably
acceptable, and with most applications, the sorts of changes you're
doing are few enough that it's unlikely you'd foget, and if the change
is big enough, you'd better be sure you do know all the variations.


Flexibility in Design through Proper Programming Decisions
----------------------------------------------------------
The page designer's/editor's flexibility is dictated by the coding
decisions the programmer makes. It is the programmer's responsibility
to build in this flexibility by breaking the pages up into appropriate
template fragments. The template manager has two primary template
processing functions, build_fragment() and show_fragment().
Build_fragment() processes the fragment but does not echo/print it,
instead it places it in a 'variable' in the template manager which will
then use that as a variable for substitution in other templates.
Show_fragment() is identical to build_fragment() except that it outputs
(echos/prints) the results of the template processing. So, here is one
example of how you might build a form processing page:

Code:

  build_fragment("header");
  if (count($errors) == 0) {
     build_fragment("abc_success");
  } else {
     build_fragment("abc_errors");
  }
  build_fragment("abc_form");
 
  build_document("abc_form_body");

  build_fragment("footer");

  show_fragment("standard_document");


In this design, abc_errors, abc_success, and abc_form are "positioned"
by abc_form_body. Header, footer, and abc_form_body are then positioned
by standard_document.

So, the designer could alter all these things, assuming of course that
the code is properly designed with this in mind, and (a very important
limitation) that no template changes its relationship with its parent;
technically they may not absolutely need to observe that but it would be
somewhat at their peril not to. In the above example, by editing the
output of a succesful form submission, they could move the placement of
the abc_success fragment directly into the standard_document and it
would "work" (that is, the rendering would all layout okay, since the
template abc_success is still being evaluated in the same place in the
code, it's just being told to appear in standard_document NOT in
abc_form_body. But, every other page that uses standard_document will
attempt to do the same (this may not cause problems since the variable
that abc_success fills would NOT be set for other pages, and thus
nothing would be output). But, the point is, it's a corruption of the
intended structure, and such things should be strongly guarded against
because they will cause problems.

Just to repeat for clarity, the idea of build_fragment is to build
variables like this from templates, as opposed to just setting template
variables from code (execution). One point to note is that in the above
example I don't make clear what the variable name should be for the
storage of the built template. The logic should probably be that it can
be "assumed" to be the template name, OR, it can be set, specifically.
In the above example, the abc_form_body should *NOT* use the default
name of the template as the variable name for the template manager,
because the standard_document would be using a generic name like
'body_variable' or something as a placeholder for the body, and that is
what should be holding the abc_form_body. So, the call to build
fragment for abc_form_body would need to specify this. In this way each
template has a clearly defined set of inputs.



How the Meta Markup Exists in the Document to be Edited
-------------------------------------------------------
The design of span tags used in this system should be something like:

Code:

<span id="template:en|default|standard_document">
 <span id="variable_from_template:en|default|header">Top of the
page</span>  <span id="variable:body_variable">
   <span
id="variable_from_template:default|default|abc_success">Success!</span>
   <span id="variable_from_template:default|default|abc_errors"></span>
   <span id="variable_from_template:default|default|abc_form_body">Form
goes here</span>  </span>  <span
id="variable_from_template:en|default|footer">Bottom of the page</span>
</span id="template:en|default|standard_document">


The 'id' I'm defining here is equal to something like a "type"
identifier (template: or variable: or constant: or
template_from_variable: (maybe) and maybe some other types) followed by
the language indicator|style indicator|template name. In the case of a
variable, all we (may) need is the variable name (but we may want/need
more to help us know where that variable is expected to be defined,
though contextually it would be clear, since it is inside another span).
For a constant: you would also need the language, style, and constant
name. Now, this variable_from_template: is something I just realized we
*might* want. It may be enough to just say template: and the parser
will "figure out" that it's imported via a variable because it's within
a variable:'s span. Again, it was just a thought I had now that we
*might* want to explicitly differentiate. So, we may want to
differentiate, we may not. I think maybe NOT, but I include it in case
I'm wrong. If we don't need it, then we'd just call these template: as
I'd originally thought. Also, there may be an issue about how to expose
it, as I have said, build_template needs to be able to store their
results under an alternate variable name to work within a generic
container template that will include it. In this case the template:
would need to have an additional element saying what its alias is.
Again, not entirely thought through, but a strong design suspicion.

I'm not at all set on the "template: something|something" formatting,
that's just for explanatory purposes, and it could be something
better/more compact. "" (empty) could be default, and template: could
be t: or t|, etc.

Also, we may want to add identical id info to the closing span tag (see
last /span), for easier parsing (I'm not sure if we want/need this
exactly, as part of the parsing, but if we did this would clearly be a
quick way to find the end of a span, versus tracking span counts, and
all that).

In addition to the <span> tags listed above, there would be another
type/modifier of an id tag, the "sample data" modifier. For example:

<span id="template:{sample data}en|default|standard_document">
<span id="variable_from_template:en|default|header">Top of the
page</span> </span>

Or some such representation (I don't like the one I made above), which
would modify the second (and nth) instances of a template, meaning that
the parser can disregard any changes within THAT exact level (which is
to say, if there was a variable/template WITHIN that sample data span,
that was NOT a 2nd plus instance, then its changes WOULD need to be
parsed. So, the sample data indicator does NOT mean everything within
the wrapped sample tags can be disregarded it only means that that
specific template can. Of course you could just let the parser figure
it out as it's running, and have no "sample data" modifier, and just let
the parser only modify where it sees changes and the "last change" (if
the user modified multiple instances of the same template) wins. Or do
it this way.

<span> tags are not sufficient to support all types of template
modifications. <span> tags can only be used to identify changes which
are OUTSIDE of tags. If you wanted a template to do something like
<table border=$border_width> this <span> system cannot work. The reason
is because whatever markup we use must allow valid HTML rendering while
preserving meta-information about what was dynamic. Thus, a different
system must be used to handle "inter-tag dynamicity" (sounds cool!)!

One proposal would be:

<table bgcolor="#cccccc" templatex__bgcolor="variable:foo">

The same span id structure would be used here, supporting templates,
variables, and constants

Obviously we could achieve *some* of the same end result with the
tag-level replacements, by having more duplication in the template
level, that is have one template variation for each variation we want to
achieve. But, this is pretty limited, since we couldn't have the color
set from a constant or anything, and if we were trying to vary more than
one attribute, we'd have an exponential growth in the number of tag
variations we'd need to store in templates.

One concern is that by this method you do replacements on the attribute
values. I do not know if in all cases those attributes in HTML which do
NOT have values (such as 'selected' and 'checked') would behave properly
in a browser if assigned values, like checked=0, selected=0?

Storing all the attribute info in one fake attribute may be a good
alternative to storing one fake attribute for each real one. I am not
satisfied with this attribute replacement scheme, it feels weak, since
you can't dynamically add new attributes/etc. You can't do stuff like
<table $border $width>, and that is annoying to feel limited like this.


Another alternative Tim proposed is something like:

Code:

<table
     templatex_span="variable:foo"
     bgcolor="#cccccc"
     templatex_span_end="variable:foo">


Then you could add a width attribute just as you would add
text between <span> tags:

Code:

   <table
     templatex_span="variable:foo"
     bgcolor="#cccccc"
     width="50%"
     templatex_span_end="variable:foo">


In pracice you would need to have the fake attributes which define the
span randomized such as templatex_span_89823 and
templatex_span_end_89823 to prevent attribute folding by editors which
wouldn't like duplicate attributes. Also, the arrangement of these fake
attributes is critical, and poses some risks as editors could (in
theory) re-arrange them.

One limitation of this design is that an attribute value cannot be as
easily modified, as you would need to specify both attribute and value
in the template/variable being used. One possible solution is by doing
something like:

Code:

   <table
     templatex_span="variable:foo|bgcolor"
     bgcolor="#cccccc"
     templatex_span_end="variable:foo|bgcolor">


Mentioning the variable in the variable: to aid the replacement is bad,
and breaks the concept of spanning the pure variable fill, since here
part is permanent (the attribute) and part is to be replaced by a
variable. So, this isn't ideal and should probably be replaced by
something better. I just feel like it would be good to have a mechanism
to optionally replace just the value of an attribute with a variable,
since that's very often how it would be used. And to make the user pass
in the attribute name and value would be a bit of duplication and once
again sort of shift some un-necessary HTML into the code or make a lot
of very tiny template segments to contain the attribute and the variable
value.

For safety with editors I prefer the first method, the
templatex__bgcolor="variable:foo".


Template Input
--------------
This indirectly brings up the issues of controlling the input to the
template. And for ease of use suggests a greater need for an ability to
pass in values to variables via the show_template/build_template calls.
So, it might look like:

$a = "#333333";

// where $template_a would be available in the template with the value
of $a
build_template("template_name",array('template_a'=>$a));

And perhaps there would be a default, assumed variable for templates
with only ONE variable, again for ease of use, so in this case you might
call it like:

// where '#cccccc' gets put in a variable called
'$template_default_variable' or something...
build_template("set_bgcolor_attribute",'#cccccc');

Again, the parameter list here are just examples, we also need the other
option to specify alternative name for the storage of this built
template in the template manager and so this parameter list order isn't
intended to be the real one.

Quincy
TemplateX@besiex.org
Back to top
View user's profile Send private message Send e-mail Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Wed May 07, 2003 3:18 am    Post subject: Reply with quote

First off, Tim, thanks for a very interesting post!

I have wondered about two-way systems myself, and I do have ideas on how Smarty can be made at least partially two-way.

My big problem, though, is that you are suggesting a 2-way HTML transform. I realize that designers use tools that create HTML, but AFAIK, HTML is completely unsuitable for 2-way tools and except for the tremendous effort that has gone into it, nearly unsuitable as a design language at all. If it were any good at it, we wouldn't see templating systems Wink

On the other-hand, the idea that some designers only know Dreamweaver but nothing about HTML makes me wonder about those designers. Its like a painter who knows paint colours but nothing of light or texture. Economics is a funny thing. Sometimes 10 guys working for $1 each can not outperform 3 guys working for $3 each. You may need Smarty-er designers Wink

IMHO, what is REALLY missing is an intermediate language that allows designers to express their ideas and ideals in an input/output neutral way. To be really useful, it would require tools that allowed for both visual and textual design.

As for your actual problem, your span idea is a good start. I haven't tried this, but if you change Smarty's delimiters to '<' and '>' and provide custom block functions for all HTML tags (except non-block tags, like IMG and BR) AND you made used judicious pre and post filtering (to strip and replace things like />'s) you can probably create a Smarty system that does the other side of the transformation--at least to some degree. I have no idea how you would gracefully unroll loops or do some of the more sophisticated features.

It strikes me, though, that you might be better off considering XHTML and an XML transform engine to do this job. I haven't used it myself, but I understand that the Pear project http://pear.php.net/package-info.php?pacid=37&release=0.8.1 allows you to bind PHP code to XML tags.

I find this an interesting topic. I hope to be able to contribute something to it in the next while. Please update us on your progress as you see fit!
Back to top
View user's profile Send private message
quinxy
Smarty n00b


Joined: 07 May 2003
Posts: 2

PostPosted: Wed May 07, 2003 5:15 am    Post subject: A bit of clarification... Reply with quote

Just to follow-up on Tim's post a bit, and respond to boots.

Tim's intro did a great job to summarize the idea/mission, but I didn't want anyone to misunderstand and think the focus/goal was to make it possible for designers with a WYSIWYG editor and no HTML skills to able to edit templated web sites, especially complex ones with large numbers of templates used to generate a page. That's not it. The goal is a bit like the goal of the WYSIWYG HTML editors themselves, they can be a tool to help you manage/visualize larger amounts of complicated HTML than you might with just a text editor. In no way does that mean someone with a copy of Dreamweaver and no HTML skills will have much success building a complicated site. I'm just trying to reduce the amount of thought I need to put into editing/managing my templates, by allowing this 'edit once, edit everywhere' approach. I've used a number of templating systems, and with all of them I've been very frustrated by the fact that as the site grows in complexity and you have dozens of templates involved in various ways in the building of any page, makign simple layout changes require quite a lot of work because you must remember exactly which templates are affected by those changes you want to make; and that requires looking at some document which describes the relationships all the templates have, or having a look at the code. How much better it would be if I could simply edit the page as it was rendered (in a text editor or Dreamweaver), and have the templating system work out how to apply those changes to the templates which generated the HTML that was changed. I would no longer really be "aware" of the templates (until I ran into the customizability limits of the template structure I originally created). So my goal isn't to help ignorant designers so much as it is just to help myself, a coder who often has to implement designs and design changes. The fact that it could be used by good designers with good HTML knowledge and an awareness of this tool's limitations, so much the better!

So, you are absolutely right, boots, about what you say about designers and it not being a good economic decision to use web designers who don't have a good knowledge of HTML and what you say about HTML being a poor language for 2-way tools. Neither of these problems are things we're hoping to solve here. This might go part of the way towards solving them, but that's more because the problems are related than because the problems are the same.

Thanks to all for the wonderful job with Smarty! I'm not sure if we'd be able to use it for what we're trying to accomplish, but I hope so. It would be great to have some or all of the features available that are in Smarty.

Quincy
Back to top
View user's profile Send private message
macinta
Smarty n00b


Joined: 07 May 2003
Posts: 3
Location: USA::MA::Boston

PostPosted: Wed May 07, 2003 3:04 pm    Post subject: A variation on the theme Reply with quote

Thanks for your response, Boots.

I took a look at the Pear XML transformation page - thanks. What benefits do you see XHTML with an XML transform engine providing over one that provides reverse transformations for Smarty?

To respond to your point about how you would handle more sophisticated features and loop unrolling, the short answer is it wouldn't necessarily handle those features at all. Using reverse transformations would necessarily restrict what features you could use in your templates (so it's not something everybody could then use since some people want/need the more advanced features). Loop unrolling in particular might be doable, though, if the output from each iteration is marked with an identifier that specifies what loop it was part of and then the importer would only pay attention to the last, changed element.

Here's an idea that's a simplification of the original and I'm going to toss it out there to see what people think - how about instead of having an "import" step, there is a "diff" step (like in CVS)? The process would work like this: you would put the template engine in debug mode, view and save two copies of the page that you want to change, make your changes, and then upload the original page and your modified page to Smarty's "diff" console. The diff console would list all of the templates that your changes affect and would provide a graphical display of which lines were different between the output from the template on the original page and the modified version of the output. The diff interface would work a lot like Gtkdiff ( http://home.catv.ne.jp/pp/ginoue/software/gtkdiff/index-e.html ). This would make it quite a bit easier to quickly see what changed in which templates. This also would work better than importing if you're using advanced features since you only need the human to be able to recognize the feature at that point. Of course, this probably wouldn't be automated enough to hide the HTML from designers who can't deal with it, but as that is really a peripheral goal, the diff console could still be of great use (in my opinion).

To follow up on your point about designers, I definitely agree with you that it is strongly preferable that the designer be comfortable working with raw HTML and that it is best to hire those that are, but for situations where you don't have that flexibility (e.g., the designer was hired before you), I would much prefer a solution that more fully exploits what the designer is experienced with (making the pages look pretty) otherwise the onus falls on the developer to act as a human template engine. And to be totally clear here, I'm not referring to my current project with Quincy when I talk about how to plan for the lowest common denominator - everybody I have worked with on this project has been very competent.

Thanks,
- Tim Macinta
Back to top
View user's profile Send private message Send e-mail Visit poster's website
boots
Administrator


Joined: 16 Apr 2003
Posts: 5611
Location: Toronto, Canada

PostPosted: Thu May 08, 2003 7:10 am    Post subject: Reply with quote

macinta:

XHTML + a transform engine vs Smarty ? Well, XML has more steam behind it to do that sort of thing. Here in the Smarty land, it is still being bandied about by a few souls like myself. When I do bring it up, I'm told to check out XML/XSLT -- but I assure you that that is not a lovely solution for anybody. Never-the-less, XML is already established and useful and if you can stay clear of XSLT you should be okay.

I think the idea of using diffs is very clever and can possibly play a role in the final process.


macinta, quinxy:

I'm still digesting all of the information you have both provided.

I was wondering what your comments are on what I posted here, especially in regards to the the two-way issue you are presenting. Basically, as I stated in my original reply on this thread, I think some form of a "semantic template" will be required to get the type of abilities you are both alluding to.

Can I ask how much research you have done on some common reversible transforms like bb<->html? Wikis seem to do a lot of that.

Also, I take it that the two of you are both engaged on a real-world project. Is the 2-way transform a research project for you or is there some sort of hard-deadline/product that you are working towards?

Nice to see you here again!
Back to top
View user's profile Send private message
quinxy
Smarty n00b


Joined: 07 May 2003
Posts: 2

PostPosted: Thu May 08, 2003 6:37 pm    Post subject: Reply with quote

I am pretty confident (as much as one can be without having actually written the code) that what I've proposed would work 2-way, within the limits I described. By avoiding the issues of looping and branching, the 2-way transformation should be pretty painless to code. What I don't know is how well any of this would map to the Smarty world. Smarty is very smart, and has tons of features (looping, conditions, etc.) which I specifically was avoiding as a design decision for this 2-way stuff. So, I think you're right about what you say about needing some sort of semantic template if that gap was to be bridged. Or doing the 2-way roughly as I outlined, with Smarty and basically throw errors if any of the smart Smarty features were found in any templates.

As for bb<->html, I'm not sure what bb is in the context of Wikis? I know what Wikis are, but am not sure what is meant by bb? Do you mean like phpBB? Like some common pseudo HTML markup that Wikis use? As for other transforms, I haven't done much research into it, I've used a number of templating systems which have used a number of transforms, and I've written a couple that did some. But, nothing entirely relevant in terms of the goal, too simple and reversible.

As for our goal/project. It is a real world project. And there are deadlines. I'm a contractor working for/on a large volume website (be glad to tell you which privately, though it's not so huge that it'd necessarily be known to you), and we're redesigning it. Having done a number of large commercial sites, and wrestled with templating systems through the years. And dealing in the future with internationalization, and supporting various branding, I was very interested in solving what to me has always been the main obstacle of templating systems, the pain of editing them. Now, maybe others haven't found this all that frustrating, because of how their sites are laid out, or because they are far more patient than I am Wink. The time has arrived for us to decide which templating system we'll go with for our redesign. And I'm leaning towards something like I've described. If we do it we'd do it LGPL/GPL. We can't expand it much beyond what we need for the site, hopefully others might. Obviously we'd like to use Smarty or some existing templating system as a base, so as to inherit the features and not re-invent the wheel, but I'm not sure how much work that would save. As for any deadlines we have, the sooner the better. Parts of the redesign are going live and other parts are waiting to be templated before they do.

I like Tim's clever 'diff' idea. I'm just concerned that while it might be go a long way toward aiding in the merging of changes back into templates, such a path would block the complete automation of that process, and require the users (designers or coders) to make too many decisions in merging those changes back in, which goes against the original intent. If diff was a solution which could blend the added Smarty feature set with partial automation of a 2 way transform, then clearly that might be worth it. I would just prefer a system where the avenue for a completely automated transform would remain open and available.

Q
Back to top
View user's profile Send private message
macinta
Smarty n00b


Joined: 07 May 2003
Posts: 3
Location: USA::MA::Boston

PostPosted: Thu May 08, 2003 6:56 pm    Post subject: Reply with quote

Boots,

Thanks again for your reply...

boots wrote:

Can I ask how much research you have done on some common reversible transforms like bb<->html? Wikis seem to do a lot of that.

Also, I take it that the two of you are both engaged on a real-world project. Is the 2-way transform a research project for you or is there some sort of hard-deadline/product that you are working towards?


Yes, this is for a real world project. The time is mostly constrained by the fact that templating is a peripheral piece of the project and as such we're searching for a solution that would be quick enough to complete that it wouldn't draw away substantial time from our core goals.

I have only been researching reversible textual transforms for the past few days. Quincy had this idea around a year ago, so he may or may not have looked into more than I have.

boots wrote:
macinta:
I was wondering what your comments are on what I posted here, especially in regards to the the two-way issue you are presenting. Basically, as I stated in my original reply on this thread, I think some form of a "semantic template" will be required to get the type of abilities you are both alluding to.


The first part of your post is almost exactly the same reason that the whole conversation started between myself and Quincy on whether to use a templating system or not and if so, which one. I was wondering if there was any substantial benefit to using a templating system with respect to achieving separation of design and logic when compared with simply keeping the PHP logic code in one file and including a PHP "template" file with <%=$varname%> to output the variables. This led to a whole discussion of what features would be necessary and/or useful in a templating system.

Having a semantic template system would certainly make the whole transformation process much easier. It's very ambitious, though, and it's consequently a bit outside the scope of what we're looking to do. We're looking for more of an incremental improvement that could be implemented quickly now with maximum backward compatibility so that designers don't need to be retrained, existing tools can be used without modification, etc. What you describe sounds a lot like what HTML was originally meant to do and what the W3C has tried to steer it back towards.

boots wrote:

I think the idea of using diffs is very clever and can possibly play a role in the final process.


Thanks. Even this could probably be simplified in the first iteration such that there is a client side diff tool which takes as input the original HTML and the modified HTML and outputs a list of templates with changes. Clicking on a template in the list would save the before and after fragments to separate files and then run a native 'diff' app for whatever platform you are using (Gtkdiff would work well in Linux). The next level of advancement over this would be to do a reverse transformation (or at least a partial reverse transformation) on the fragments before saving them and running the 'diff' program. As the reverse transformation becomes more thorough, it will eventually reach the point where using it as the basis for importing would be practical.

Thanks,
- Tim Macinta
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Smarty Forum Index -> Feature Requests All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group
Protected by Anti-Spam ACP