My new website uses WordPress and Smarty

Spurred on by the rather dull situation of my web dev portfolio being a single long page with pictures and writing on it (which simply will not do!), I spent the last week moving my blog (and my portfolio) to my own webspace with WordPress and a custom theme. This entailed re-learning PHP (ugh), learning more about WordPress, and pulling a lot of hair. And the whole thing is still a work in progress, but it’s here, and it’s better than my old blog. Yay!

WordPress has worked really well for me, so far. It has a really good post editor that doesn’t spew broken HTML all over my posts when I look at them wrong, the SQLite plugin works (for when I was testing locally), installing it was simple, and it has reasonably detailed documentation, most of the time.

But there is one place where WordPress and I simply cannot agree, and that is writing themes. My new site has a fancy header that picks up rows as you go from page to page. My attempt to create reasonable semantics for that under a typical WordPress theme was completely fruitless. The convention in WordPress is to write a theme in straight PHP, using stuff like the_content() to print the selected post’s content directly. Coming from Django, I was not a fan.

What I wanted was a good template engine, like Django’s. I wanted it to support inheritance, so I could have a base page and override specific parts of that base page in each subpage. I wanted the ability to use the same template for different back-ends without too much turmoil. I wanted to stick variables in page content without generating incomprehensible giberish, and I would prefer to separate logic from presentation at least a little. There is just a thing for PHP: Smarty! Now, some searching for Smarty support in WordPress led me to a lot of requests and no real implementations (or attempts), so that’s why I’m here. This site passes through Smarty for just about everything, and I probably didn’t do it right but at least for a small site this method seems to work. (And I’ll keep this post updated if it doesn’t). I’m going to write (or at least brain dump) some tips for those who are looking to do something similar. I am assuming, if you’re interested, that you have some idea how Smarty works and some idea how WordPress works, but not much of an idea is necessary.

This probably isn’t terribly efficient, though it’s worth noting that Smarty is normal PHP code once each template is run for the first time, so at least to me it intuitively felt more expensive than it ended up.

First, you need a custom WordPress theme that acts as an adapter to Smarty, because Smarty can’t talk to WordPress directly. (It can’t call functions). Unfortunately my WordPress theme was made in a big hurry so it isn’t reusable. (A clever person could do better – maybe something like the Sandbox theme). Coming from Django, I decided I would think of WordPress’s template files like Django’s views; the things that go between an HTTP request and the (Smarty) template system. Look through WordPress’s Template Files List. In each of those, instead of displaying something directly we’re going to gather data and send it over to Smarty.

You’ll need to add some stuff to functions.php. First, load the Smarty library (require('includes/Smarty.class.php')). Then we have a Smarty subclass that can be used for the project:

class Smarty_Base extends Smarty {
	function __construct() {
		parent::__construct();

		$basedir = get_theme_root().'/'.get_template();

		$this->setTemplateDir($basedir.'/smarty/templates');
		$this->setCompileDir($basedir.'/smarty/templates_c');
		$this->setCacheDir($basedir.'/smarty/cache');
		$this->setConfigDir($basedir.'/smarty/configs');

		//$this->caching = Smarty::CACHING_LIFETIME_CURRENT;

		//** un-comment the following line to show the debug console
		//$this->debugging = true;
	}
}

I added another subclass that would guarantee some variables be sent to every template. The most important ones here are wp_head and wp_footer, which are popular points for plugins to attach themselves.

class Smarty_Wordpress extends Smarty_Base {
	function __construct() {
		parent::__construct();

		$wptitle = wp_title(' | ', false, 'right');
		if ($wptitle) {
			$this->assign('page_title', sprintf("%s %s", $wptitle, get_bloginfo('name')));
		} else {
			$this->assign('page_title', sprintf("%s", get_bloginfo('name')));
		}

		$this->assign('resource', get_stylesheet_directory_uri().'/resource');

		/* get wp_head */
		ob_start();
		wp_head();
		$this->assign('wp_head', ob_get_contents());
		ob_end_clean();

		/* get wp_footer */
		ob_start();
		wp_footer();
		$this->assign('wp_footer', ob_get_contents());
		ob_end_clean();
	}
}

So, WordPress (bless its heart) doesn’t have a get_wp_head function; it will only print the head directly. We can work around that using PHP’s output buffering functions, which are those blocks of code with ob_start and ob_get_contents. Again, not pretty, but this solves our problem: now the template can choose when and where to stick wp_head and wp_footer. You’ll be using this a lot, actually.

From here, getting Smarty to render my pages has been as simple as making Smarty templates and running them (with the right values) from the appropriate WordPress template files. Because Smarty templates inherit from each other, don’t be afraid to add helper functions since you might have all of your pages sending the same variables to templates because they inherit from the same base.

For example, I added a function that is called for any template inside my blog section:

function prepare_smarty_for_blog($smarty) {
	$smarty->assign('page_number', get_query_var('paged'));
	$smarty->assign('next_posts_link', get_next_posts_link("Older posts"));
	$smarty->assign('previous_posts_link', get_previous_posts_link("Newer posts"));
	$smarty->assign('blog_sidebar_html', get_sidebar_html('blog-sidebar'));
}

By the way, get_sidebar_html is to do any collection of widgets. Just like wp_head and wp_footer, we need to use output buffering because widgets output their contents directly when asked. Otherwise, it’s a pretty ordinary sidebar that we can freely customize from the admin page:

function register_sidebars() {
	register_sidebar(array(
		'id' => 'blog-sidebar',
		'name' => 'Blog sidebar',
	));
}
add_action( 'widgets_init', 'register_sidebars' );
function get_sidebar_html($index) {
	$sidebar_html;
	/* get sidebar contents */
	ob_start();
	if ( dynamic_sidebar($index) ) {
		$sidebar_html = ob_get_contents();
	}
	ob_end_clean();
	return $sidebar_html;
}

In the end, the WordPress template for a post page (single.php) looks something like this:

<?php
/**
 * The Template for displaying all single posts.
 */

$smarty = new Smarty_Wordpress();

$post_id = get_the_id();
$post_full = new PostContent($post_id);
$smarty->assign('post', $post_full);

prepare_smarty_for_blog($smarty);

$smarty->display('blog-post.tpl');

?>

And here is the Smarty template blog-post.tpl:

{extends file="base-blog.tpl"}

{block name=main}
	<div id="post-{$post->id}" {$post->class_html}>
		{if $post->blog_import_url}
		<div class="post-message">This post was imported from my old blog at {$post->blog_import_url}</div>
		{/if}

		<div class="post-main writing expanded">
			{$post->content_html}
		</div>

		<div class="comments-wrapper">
			{$post->comments_html}
		</div>
	</div>
{/block}

Isn’t that tidy?

PostContent is just a little object that grabs bits of information for the selected post and stores it all in readily accessible variables intended for Smarty. Alternatively, you can just fill up an associative array or a dictionary and send that to Smarty. The object was useful for me because I like to needlessly add objects to things, and because I could reuse code between lists of posts and actual posts. It’s pretty simple (and pretty site-specific), but one thing that may jump at you using this approach is lots of WordPress functions know what “the_post” is based on a global variable — $post. Since we are doing so much stuff outside the ordinary flow of a WordPress theme, changing the global variable can cause some problems. So, I tend to do something like this:

global $post;
$old_post = $post;
$post = get_post($post_id);

/* do stuff with the post, like get_the_excerpt() */

$post = $old_post;

Whether you find yourself in that position seems to depend on who you are. I found my code reading post IDs and wanting to immediately delegate processing said IDs to something else. Somebody else might run get_post from a higher point, in which case that global variable silliness is completely unnecessary.

Another more common thing to keep in mind is that WordPress likes to generate strings with HTML for us right away, and sometimes we can’t change them very easily. (And usually that’s a good thing. It’s pretty cool how it will generate a comment form or a post password form and magically process it for us, and it generally just saves time and makes your theme more flexible). To make this a little tidier, I like to add _html to the end of variable names that point to HTML content, instead of fighting a losing battle and making WordPress less cool :)

And that’s pretty well it. There is still lots of boilerplate, but I find it much more readable than a lot of WordPress themes out there. Smarty’s template inheritance is really the main thing I wanted and it works beautifully. In short, you can use Smarty with WordPress, barring some limitations. If you (like me) have been convinced that dedicated template languages are the solution to everything, it could even be worthwhile!

June, 2013

I have been iterating on my theme, recently. I added some responsive design bits and pieces, I changed the layout for sidebars and footers and things, I set a maximum width for the content, and I made comments look a little prettier. This involved a lot of CSS, but it also involved a lot of changes to markup in order to support that CSS. I noticed something interesting: the last modified date for all of my PHP files except comments.php and functions.php is April 2012. I had to modify comments.php because I still haven’t gathered up the courage to sort out that mess and move that template to Smarty, and the only recent change for functions.php was registering a new sidebar. So, as a person who hates PHP, I think this was pretty cool.

I have a craving for keywords

Something interesting is going on with Gnome Shell and Unity: it is becoming very normal to search for applications. But search is more than just typing something’s name in a box: With the web, search has become synonymous with asking questions and getting answers. With really good searching, people can find what they need by saying what they want to do. So, I think searching for applications is something that should be addressed strongly to provide a great experience.

I have a hunch that one part of this puzzle is already mostly here. It just needs some glue. Almost every application is associated with at least one category, like BlocksGame and VectorGraphics. I’m hoping we can leverage that data to make searching more excellent.

Now, just in case I’ve lost anyone: there is cake involved!

I made the Category Keyword Collector to explore my theory.

If enough people enter enough data, I’m hoping we’ll see some interesting trends in how people describe applications. Ideally it will all line up and we can create a wonderful thing that expands each standard category into a bunch of searchable keywords. More likely we’ll see where the problems lie in my theory, and of course I’m hoping good things will emerge from that.

Tastier than it looks! (More so with a smaller mug and a crunchier cookie)

So, this is where the true heroes among us venture forth to the website.
Please play with it! It really enjoys the company, and it will only take a few seconds. I promise!

(Sorry about the bumped post. Blogger spat out uglier HTML than usual and I had to pick at it)

Harvest + GSoC week 12

Well, I managed my last-minute merge request for Summer of Code 2010, and with that done it is time for my last GSoC 2010 blog post!

I am really glad I did this. It’s taught me a lot about myself (having never done a project in this fashion before) and I’m happy with how it turned out. GSoC was a nice change of pace and I hope to keep this up for a while!

(Hamster still says I’m a slacker. I need to convince it otherwise)

So, here are the results of some recent tinkering:

First, of course: new font! It requests the delightful UbuntuBeta font first and borrows some conventions from the Ubuntu Web guidelines.
Daniel added help text to the opportunity list filters. He gave each one a tooltip with a nice explanation.
There’s now a permalink for every source package, a nice Edit button for each opportunity, and some other small visual tweaks. Just little stuff like that.

Previously, every opportunity in Harvest had a text box labelled “Comment” that saved to a single database field. The idea was someone could fill it in with useful information for other Harvest users. It felt sort of like Launchpad’s Whiteboard feature in Blueprints.

I decided that implementation would encourage very complex comments. That’s a problem because Harvest aggregates other stuff. There’s a good chance whatever it links to will have its own comment system. I wanted to differentiate Harvest’s comments so they would never be seen as alternatives to whatever is in a referenced bug tracker. Instead, these should be, from top to bottom, specifically for small pointers.

So, we played with it and I added Notes (which is apparently the least commonly implemented synonym of Comments). Notes are little messages you can add to opportunities. Each note is basically a text field limited to 250 characters. No special formatting and no newlines. When I write one, I think of IRC.

The feature isn’t as quick to find as I’d like (there’s an app bug report for that), but it’ll be good to see how people use Harvest before adding more stuff. There are probably all sorts of cool things that could be done, so I’d love to know in the comments here if you have an idea!

(…And I can’t believe I haven’t done this yet)
Thank you to Daniel for being a great mentor! I can confirm the rumours are true: he is an excellent guy. I’ve really enjoyed working with him.

I didn’t quite get to everything I wanted for GSoC (and I have to admit I got carried away with design trivialities this week), but now we’re just a few small pieces away from a spectacular tool for Ubuntu developers. After a bit of a coding / remembering last year’s math course break, I’ll be back helping with some more of those bits. It’s going to be fun!

Here’s a completely unrelated bit of good news: I learned that UDS-N is now short for UDS-Natty, and I miraculously have no exams anywhere near late October ;)
So, assuming I can find a narwhal and convince it I am indeed smart and fashionable, I’m totally hoping to get there, meet people, learn lots of stuff and wax lyrical about Harvest, slideshows and release notes (as I do). Should be fun!

Harvest + GSoC week 9!

The past week, unfortunately, hasn’t been very productive for me. Still, I have a few things to show!

I did a bunch of cleanup (again) and I toyed with animations for a while. There is now an animation (and a loading indicator) when new results are retrieved for the filters. One philosophy I have found myself following is that this UI that never, ever blocks on a task. So, no matter what is happening, you should always be able to click the same buttons you could before.
(On a related note, I’ve been very excited about Blender lately).

Turns out non-modal design has an interesting impact as far as loading indicators are concerned. I wanted this indicator to be totally unobtrusive, but also obvious. So, as soon as someone starts adjusting filters, they can see that Harvest is waiting, then loading results.

It’s a little bubble that appears from the top of the results and is always visible at the top of the screen as the user scrolls. Conveniently enough, the existing results are all pushed away to make room for it. That was an accident at first, but really it makes the slide in a bit more pronounced and it means it doesn’t overlap anything you may have been looking at. (And even if it does, you can fix that by scrolling up).

Pretty simple :)

I made a quick video to show how this is all working:

The other thing I tried doing was to have the Javascript interface record its state in the location so it can be bookmarked and navigated through in the browser history (with the Back and Forward buttons).

I learned a lot of terrifying things, since I’ve never done that before.

Of course, the first terrifying thing is that you can’t change any part of the URL except the part after #, which never gets sent to the server. Well, you can change the other parts, but when you do it redirects to the given URL. That’s a good thing, but it does make life a little difficult for me :)

So, what we need to do is change location.hash to describe the current state, just like we do with the query strings sent to the server. There also needs to be an event handler watching for that change. With those two bits in place, that event handler is called whenever the user presses the Back button or loads the page from a bookmark, and it can apply the given state.

There are piles of jQuery plugins for this job. Two stood out above the pack: Asual’s impressively professional looking jQuery Address, and Ben Alman’s deliciously silly sounding jQuery BBQ. After some pondering and licking my lips, I went with jQuery BBQ.

BBQ is very clever in its simplicity. It uses the existing jQuery.param stuff to generate a query string the usual way, then it puts that in the hash part of the url instead of the ? part. It adds a “deparam” function to deserialize those query strings, a browser-agnostic hashchange event, a few helper functions, a bunch of documentation, and that’s it.

Unfortunately, even with jQuery BBQ, my work got really loopy here. In short, there are a lot of variables to deal with already and adding more almost exploded my brain.
(That was no doubt strengthened by a brief interlude at the family cottage, where I regressed to glorious redneckism and finally unlearned PHP)

So, unfortunately, I’m going to hold off on that for now. I could have planned the Javascript stuff much better, and getting BBQ working smoothly is likely a rewrite away. It may be a bad idea anyway. If someone shares a link that has #querystring instead of ?querystring, that link won’t work for anyone who has Javascript disabled. Instead, I’ll probably go the Google Maps approach, offering a “Permalink” button but never editing the URL in operation.

Next up: nicely formatted package details, and editing opportunities!

Harvest stuff for GSoC: Week 7!

Time for my bi-weekly Harvest update! Everything this time went into the gsoc-client-stuff branch.

The first thing I learned (well, decided) is that YUI has incredibly dense, loopy and uncool documentation. I guess different people are compatible with different kinds of docs. As I read the YUI stuff I just couldn’t keep it all straight for some reason. Its landing page leads off in many directions: there’s an API reference that was written and designed to put me to sleep, an Examples section that doesn’t bother to link to the API reference (but is more pleasantly written), and a lot of extra listings in between.

All I could think about was how much I preferred JQuery’s docs. So, I dropped YUI for JQuery and nothing has exploded. In fact, it’s all gone wonderfully.

JQuery has a considerably simpler core than YUI, but there is lots of functionality in little self-contained plugins. Some of these are official parts of the project, others are external things linked from the plugin repository. (Granted, lots of redundancy there). It’s a different approach — YUI is richer from the start — but for me, JQuery wins by being so much easier to take in at a glance. That, and its documentation is a lot prettier. Instead of your boring automatically generated list of docstrings attached to function names, they have a beautifully presented web app. There are no frustrating stubs; everything I may think to use is there and explained in detail. And it’s a single destination. One page to learn everything there is to know about each bit of functionality in JQuery, including examples. Pretty cool.
JQuery, I promise I will always love you :)

I also learned about doing object-oriented Javascript. This one surprised me. I have used Javascript for lots of small jobs before, but I have never used it with anything big. I realized Javascript doesn’t have the class keyword I have come to love, but after some learning I am back to thinking it’s a pretty cool scripting language. That is mainly thanks to an excellent blog post by Stoyan Stefanov, all about doing classes in Javascript. Everything is an object, so of course we don’t need a class keyword! (Apparently we’re getting one some time this century, though, if everyone is nice to it).

Making something that feels like a class is a bit of a hack, but it works elegantly in the end. It’s really just a function, and we put other functions and things inside it for methods and properties. I’m doing them like this:function Filter (dom_node) { var filter = this; this.get_value_serialized = function () { return null; }}

To create an instance we use Javascript’s new keyword and write out the function as usual (including its parameters). The this keyword can’t be trusted if we are using callback functions, because each function is given its own version based on what object it is being called with. (If it’s via a reference to a function stored somewhere else, things get messy). On the other hand, anything inside Filter can see that filter variable we created at the top, as well as the dom_node parameter.

Those variables pointing at functions, of course, can be really easily reassigned to point at different functions. Lots of power here. For fancier stuff, including multiple inheritance (yes, it gets crazier), Mike Koss has an excellent article. In my case, I decided not to go into that. (Well, okay, I chickened out then called it a decision). Proper subclasses might make my code look smarter, but in this case that whole chunk barely needs to do anything anyway, so I’m fine how it is.

With that all out of the way, I worked on a really fun list of new stuff!

I have Harvest using XHR in a few different places now. XHR is a wonderful thing that lets us directly request new data for our page and handle it through Javascript. It means, if you want to select a bunch of filters, you can do it without flooding our server with each one.

When a filter’s value is changed, it posts the query parameter that change represents to the global Results object. (Some goodies here: if you change the value of a filter in a positive way, that filter is selected implicitly). Instead of instantly yapping at the server, the Results object stores the parameter it receives in an object (dictionary style) and starts (or restarts) its timer. When the timer finishes, it uses JQuery’s $.get function to request new results for the selected filters, passing the function its dictionary of new parameters. (JQuery magically turns that into a querystring for us, so if something weird happens and everyone decides to use something else, Harvest will still work).

The result surprised me. I’m still not doing much to limit the number of results, but even with a query that returns the most packages possible (around 3700 at once) the whole thing feels a lot quicker than it did. Funny…

The next one is expanding packages. When the user clicks a package, we send another http request asking for that package’s details. We get a snippet of HTML back from the server, throw it in the right element and reveal it with an animation. (Did I mention I love JQuery?).

By the way, I worked on the visual design for packages. Any comments? Thoughts on the arbitrary green highlight?

I don’t trust http requests going on unchecked. So, as is the convention, I added little “loading” indicators (from the very helpful and sickeningly popular ajaxload.info). It was really distracting to have these appear all the time, though, so I played with it and now the indicators will slowly fade in. A loading indicator should only become obvious if there is a long operation going on. If everything is normal and the operation is nearly instant, the indicator stays out of the way.

I went two whole days without an Internet connection this week (oh, the humanity!), so I was thinking about people with similar predicaments. It sucks when an application decides it wants to download something and keeps on trying and trying, oblivious to my repeated attempts to convince it I have no connection. JQuery’s $.get function returns an object to control the http request, so I store that using the $.data method:package_node.data('xhr', xhr);Later, if someone tries to collapse a package that was still loading its details before expanding, we can do something like:package_node.data('xhr').abort();We give $.get a callback function that is run when the request is finished — be it successful, an error from the server, or an abort. So, cleanup (like removing the loading indicator) can all be done there.

The Django debug-toolbar only kicks in when we load a whole new page inheriting from the base template, so I had to try something new to gauge performance. In this case I found a cool bit of middleware at DjangoSnippets.org. It adds a header for every page Django creates, saying how long it took to generate. It isn’t much information, but it helps! What I like here is it doesn’t edit the page at all and it’s a really small bit of code; it is as unobtrusive as possible. The data is always visible with a tool like Firebug or Webkit’s web inspector (or telnet, if you’re crazy). It is super easy to present this with Javascript, too:time_header = xhr.getResponseHeader('X-Django-Request-Time')if (time_header) { $('#requeststats').html('Results generated in '+parseFloat(time_header).toFixed(2) + ' seconds');}

And that is that! There is lots of polish left to do for next week, and a strange headache of a merge conflict to resolve. Assuming bzr doesn’t eat anyone’s work, things are really picking up!

Harvest GSoC project: week 5!

The last two weeks of my Harvest project have gone really well. It isn’t flashy and exciting and earth-shattering (yet), but I’m happy with it.

First of all, my branch now has Packages and Opportunities filters. I implemented a bunch of each, and they are resolved in order. First Harvest runs the package filters, then it filters the opportunities that belong to those packages, then it hides packages that have no visible opportunities after all that filtering.

After a long period of me obsessively poking things, Daniel talked me into making a merge request for my branch (to lp:harvest). He and James W gave it some really thorough code review, which has been a huge help! Now it all feels tidier and a little more justified, so I can use the code I wrote without that constant temptation to rewrite it.

At this point, performance is much improved. Details are only shown for one package at a time, so the incredibly long waits (and self-destructs) have gone away.

In addition, the filter system consistently does 4 SQL queries no matter what is being searched for. Of course, that doesn’t say it’s running as well as it can, but it does mean the system is more orderly. It only hits the database once for each type of data. (One query for package sets, one for opportunity lists, one for relevant source packages, one for relevant opportunities). This puts more thorough optimization within reach.

Rather than trying to optimizing things for eternity, I’m off to something completely different for now: fancy Javascript to load new content in line with the page! The idea is a big query will still have a visible wait (always will!), but a complete page won’t need to be created each time; just the specific results, with a nice spinner while they’re loaded. Less jumping around, quicker and more fun.

For the first step, I have to admit I got a little carried away and started redoing the base template from scratch. (It should save me some time, really. The Javascript will be attached to some kind of DOM tree and I don’t want to fiddle with that twice). Still an early WIP, but I think this looks quite pretty :)

I am starting with a prototype written in straight HTML; no Django template markup yet. It’s helping me straighten my thoughts for how the filters’ render() functions should link together. The interface is taller than I would like, so I will need to do something to collapse the Choice filters when they aren’t being used. Oh, and we don’t seem to have a logo. Still, the gist of it is there.

Next for me is figuring out YUI and making that filter interface on the left interactive. Really exciting! Javascript toolkits are amazingly fun to use.

Speaking of web design, I decided I didn’t like my nearly-stock Blogger template anymore so I spent way too long redesigning it (while cursing Blogger for being an awful platform to make templates with). What do you think?