What is a [web] widget

... a web widget is a portable chunk of code that can be installed and executed within any separate HTML-based web page by an end user without requiring additional compilation.
Web widget by Wikipedia

Put simply, we need to provide prospective site owners with a short piece of code (henceforth, the "embed code") that they can place on their sites and have the widget appear without any additional modifications to the site.

How would we go about creating an embeddable widget?

Cross-site scripting (XSS)

Cross-site scripting is more often mentioned as a web vulnerability which uses the same technique maliciously to manipulate the target site. In this case, Javascript will be used to create the widget on the host site and not to attack it (you should always beware of including scripts from untrusted sources).

This technique is based on embedding a script <tag> on the host site that will refer to a script residing on your service. The script should then dynamically create the widget on the host site by manipulating the DOM and adding stylesheets and script tags as needed. Dr. Nic gives a nice overview of this technique in this detailed tutorial.

As the tutorial mentions, your embeddable script should have a unique namespace to avoid colliding with local scripts on the host site. Don't be shy to use a long and verbose name for this purpose. Example:

;(function() {
	var techfounder_super_embed = window.techfounder_super_embed = {
		init: function() {
			alert('embedded!');
		}
	};
	techfounder_super_embed.init();
})();

Some quick notes:

• I used a self executing anonymous function. This allows me to declare additional variables and functions other than my main namespace ('techfounder_super_embed' in this case) without polluting the main scope or interferring with existing code on the site.
• In order for the main namespace to be available outside of the anonymous function scope, I attach it to the window object.
• I preceded the function with a semi-colon to close any potentially open Javascript statement that might have been loaded before it, which would have resulted in an error.

As opposed to the advice given in Dr. Nic's article, I usually opt to use an external library such as jQuery, in order to better account for cross-browser differences as well as leverage the power inherent in such libraries. jQuery in particular has the useful noConflict() method which allows it to work side-by-side with other libraries, and can be loaded from a google CDN so it does not add bandwidth load on your server.

As for retrieving data cross-site (from your service server to the host site), there are 3 paths you could take:

• Embedding extra script tags which include JSON data that is constructed dynamically (as shown in Dr. Nic's tutorial).
• Creating an Iframe element that accesses a remote resource (ie, on your server) to retrieve the data, and then passing the data back to the document.
• Using a JSONP call to simulate a cross-site AJAX request. I personally opt for this technique as I find it the most elegant. jQuery supports this natively in recent versions, making it almost too easy to implement.

An important note to keep in mind that styling the widget elements should be done very explicitly. As the widget is embedded directly into the host document it could be affected by any previously loaded stylesheets. A good start would be to use a CSS reset (such as this one by Eric Meyer) and apply it directly to a unique ID that identifies the base widget element.

Iframe embedding

An Iframe is the most direct approach to including content from a remote site. It's easy to pass arguments in the Iframe source URL and have a response dynamically built in return. However the Iframe approach suffers from some limitations:

• You have to specify the Iframe proportions in the embed code and it should match the content exactly, else the content can get truncated. This also means the embed code would need to change if the widget appearance changes (which comes up often for customizable widgets).
• There are some security limitations for Iframe scripting, so if you need to manipulate the host site page in any additional way,  you would need to include a script <tag> as well in the initial embed code to bypass the security restrictions.
• Iframes are invalid markup in XHTML Strict doctype. Depending on the host site doctype declaration, this could present a minor issue (if the host site owner is concerned about with valid markup).

A big advantage in using an Iframe over directly embedding the widget in the host page is that you don't have to be concerned with external styles affecting the widget.

Which should you use

Cross site scripting is useful when there is a need to dynamically manipulate the host site page. This would include placing the widget in positions different to where the embed code was placed at, offering additional user-interface interactions or communicating data back to your server (as Google analytics does, for example).

The Iframe approach is useful when you can tell ahead of time the exact size of the embedded widget, and you only want it to display content and not offer any Javascript interactions that occur outside of the Iframe.

As it might be expected, there are several techniques to optimize the delivery of web pages. The Exceptional Performance guide by Yahoo is a great resource for a multitude of optimizations practices, including specifically two techniques which I will address in this article - script minifcation and concatenation.

Reducing script size and the number of request

Modern web sites and web application serve increasingly heavier javascript and CSS files. The price for the increase in script payload is felt both on the client side, where loading time increase making pages feel slugish, and on the server side, where bandwidth and requests are at a premium.

Javascript files especially block the browser while downloading (no more than 2 components can be downloaded simultaneously as per HTML specifications), and are the type of scripts seeing the steepest increase in size in the last several years.

Minification and concatenation

Minifying is the process of removing all unnecessary characters from the source code without changing its functionality (removing whitespaces, line-breaks and comments, replacing variables names and more). Concatenation is the process of unifying two strings into one. Applying both to our scripts we reduce total script size and the amount of requests.

There are several public algorithms for minifying Javascript, and Dean Edwards gives a nice overview of those on his blog.

For PHP we have the handy Minify library, which handles both the minifcation and concatenation of scripts. Minify is quite robust, and comes with a set of standalone classes that can be used in other contexts as well. It also sets Etags and far-future headers for better client script caching.

Deploying Minify

Downloading and extracting the package available on google code creates several text files and two sub-directories. The directory that is of interest to us is '/min' which contains the actual source code, and we'll place it somewhere on our server about our document root (so it can be accessed by a regular http request).

The basic workflow of using Minify is to replace our <script> and <link> tags which load our javascript and CSS respectively with calls to the minify script. We need to tell the Minify script which scripts to process, and there are two main ways to accomplish that:

- Passing the scripts as parameters in the call to the minify script. We pass the path to the scripts separated by commas, translating the following:

<script type="text/javascript" src="/path/to/script_one.js"></script>
<script type="text/javascript" src="/path/to/script_two.js"></script>

Into:

<script type="text/javascript" src="/path/to/min/?f=/path/to/script_one.js,/path/to/script_two.js"></script>

There is actually a shorter way writing this, by using a common base path for both scripts:

<script type="text/javascript" src="/path/to/min/?b=path/to&f=script_one.js,script_two.js"></script>

Provided that all the paths are correct, we have successful concatenated two scripts into one minified script. This can repeated for as many scripts as we like, though there is an artificial limit (due to performance and memory limitations I assume) of 10 scripts per Minify request. This limit can be adjusted in the configuration file which exists at the base path of the library (config.php).

- Using pre-defined script arrays. This method is less flexible as it requires hardcoding the script names into an array, but is slightly more performant. Minify calls those array 'groups', which look something like:

 return array(
    'js' => array('//path/to/script_one.js', '//path/to/script_two.js')
);

You can find an example of those in the installation folder of Minify (groupsConfig.php). The request to the script is then made by specifying the group name:

<script type="text/javascript" src="/path/to/min/?g=js"></script>

There are several options when using groups, and you can read on those on the library's wiki.

Its important to set up the cache folder for Minify. Minifying the scripts can actually take a couple of seconds, which is a relatively long delay - however once generated once they will be cached until the scripts change (provided the cache folder is set-up properly).

To declare the cache folder, simply define a variable named $min_cachePath in the configuration file or the main app file before the main application start.

$min_cachePath = '/path/to/cache';

Make sure the folder exists and has write permissions by PHP.

Real world use

I've implemented Minify in all of my recent projects to great effect. An extreme case would be script deployment in my current startup, Octabox web platform. Being a heavy-duty web-application, it consumes much more javascript and styles than your average marketing site.

Without Minify

Javascript requests:

CSS requests:

With Minify

Javascript requests:

CSS requests:

51 total requests weighing 540kb reduced to 5 requests weighing 116kb.

1. OO code is less performant than procedural code

The number one argument against OO application design from procedural advocates. This argument is based more on intuition than fact. The usual examples pit short procedural code against equivalent OO code in which procedural code comes out triumphant as more terse and performant.

//Procedural, quick and to the point!
$string = 'hello world';
echo $string;
 
//OO, verbose and ineffecient...
class Speaker {
	public function say($string) {
		echo $string;
	}
}
 
$me = new Speaker();
$me -> say('hello world');

Actually, this example compares apples to oranges. Both approaches use 'echo' to output a string, only in the OO approach it is wrapped inside a class method. A more appropriate example for the procedural code would be:

function say($string) {
	echo $string;
}
 
say('hello string');

Which is very similar to the OO code written before. Strong application structure actually does not differ greatly between OO and procedural/functional approaches as my second example suggests, and it is mainly a function of the developer's skill and experience.

OO methodology however gives more language features and scope constructs to deal with code complexities and to handle code reuse, features that would require much more discipline and experience to implement using procedural programming only.

In reality, it is the efficiency of the language parser that will determine performance differences between procedural and OO code. There are no inherent noticeable performance differences between both methodologies for common usage in web applications. More typically in the web environment, IO operations such as server requests or database queries are the bottlenecks of performance / scalability.

Since it's impossible to know in advance what the bottlenecks will be (you can only make educated guesses), I try to follow the Rules Of Optimization:

1. First rule of optimization - Don't
2. Second rule of optimization - Don't ... yet
3. Third rule of optimization - Profile before optimizing

It is much more important to pick a methodology that will promote more maintainable and structured code, as OOP does, to possible minor performance gains. With a well structured design, profiling an application and refactoring slow-performing code is a relatively straightforward process.

2. The backend is the most important part of development

I often see web developers treat HTML, CSS and Javascript as second class languages. While HTML and CSS are not programming languages per-se, correct implementation of both in a cross-browser and semantic way requires much experience and skill that I often see lacking in otherwise proficient web developers.

It is not uncommon to relegate implementation of those two markup languages to web designers or "client-side developers" (which are usually regarded as lower level developers) or to use auto-generating tools for creating markup.

Both approaches are wrong - the client side is an important aspect of the web development process, especially because of the unique environment a web site or application runs in on the Internet - the browser, which has many different flavours each with its own set of problems and constraints.

Javascript (or DHTML) is another domain expertise which is undervalued and yet has become such a common requirement in today's rich Internet applications. Many web developers coming from a server-side language background have an intrinsic dislike for Javascript because of its weird OO syntax, its difficult debugging scheme and its un-uniform implementation across browsers . I would admit to once have thought the same way, however through necessity I came to like Javascript and now consider it an indispensable skill for a serious web developer.

Don't disrespect the client side languages. Sometimes the greatest challenges in web development are found there.

3. Graphical designers are good at user interface design

* This is more relevant for web applications as opposed to web sites

While there is a connection between graphical design and UI design, graphical designers do not naturally make for good UI designers. I can not emphasis this enough. User interface and user experience design is a separate expertise from graphical design. How a user will interact with a site / application is different from how he will view it. It is connected - but not the same.

Quoting wikipedia:

Where traditional graphic design seeks to make the object or application physically attractive, the goal of user interface design is to make the user's interaction as simple and efficient as possible

Graphical designers with no expertise in UI design tend to prefer aesthetics over usability, and this is only natural as it accentuates their strengths. On the flip side, (web) developers don't usually make for good UI designers either.

UI design is important for creating a usable application. Make sure the right people are creating it.

4. The existence of a superior programming language

While some programming languages are considered superior to others (higher level, more featured, better syntax etc.), no one language can be considered the superior language. Debating that one exists is more a statement of personal preferences than a declaration of actual merit.

All modern languages have roughly the same features and can perform most tasks equally well. Some languages might be more suited to specific problems than others, however no one language does everything better than the others.

Some promote specific languages because of highly acclaimed frameworks developed for them (I'm referring to Ruby on Rails, naturally). While this is a more relevant argument, currently all the major languages have at least one serious framework (Django for python, Zend Framework / CakePHP / Symphony for PHP).

Use the language you are comfortable with, but don't deride others based on your inexperience with them.

5. XML is more economic than a DB

Database are notorious for being the major bottleneck for many successful web sites / applications. It is this notoriety that prompts novice developers to seek out alternative persistence layers - and XML is the common alternative.

In general, parsing XML is slower and more resource intensive than querying a database. This is dependent on the complexity of the query, but holds true for 99% of the cases. Furthermore, databases have a much more complete language to fetch, order and manipulate result sets.

XML is a portable format for storing hierarchical data. It is useful for porting data between different databases or between databases and interfaces. However XML is not a database, so don't treat it as one.

If you have more to add to this list, I would love to hear about it.

Comet is an event driven communication scheme with between a web-browser and a web-server. In the normal flow of an http request, a web server can not initiate communications with the client (the web browser) - it can only respond to requests. Comet declares a reversal of roles, in which the server can notify the client whenever new data is available. This technique is very useful in a constantly changing environment such as stock prices or online messaging (chat).

The term was coined by Alex Russell of Dojo fame, in a blog piece a little more than two years ago. It has since gained minor traction in the development community, with several projects actively promoting it and educating on it. It never reached AJAX-levels in popularity though, mainly due to it being somewhat difficult to implement and use.

Enter HTML 5. The working draft of the HTML 5 specifications declares server-sent DOM events. This API allows for native implementation of Comet techniques without using elaborate client-server setups which will finally make Comet accessible enough to be considered mainstream. The specs go even further, and declare an interface for bidirectional communications between client and server and client and client (P2P). Such an interface would render current AJAX techniques absolute, as the XHR object would no longer be required to poll the server for data on demand. The peer-to-peer options offer an incredible opportunity to multi-user environment by removing the server as the middle-man (and the bottleneck).

Comet Daily gives an overview of this API and the WebSockets API in a recent piece. Michael Carter mentions the new network API and how it will affect future client-server communication in some detail and even shows some mock code of ridiculously simple setup for server-listening events.

So yeah, HTML 5 is definitely something to watch for. Hopefully the draft will be finalized as soon as possible so browser vendors could start pushing out implementations and make us developers very happy.

A design pattern is a general reusable solution to a recurring design problem in object-oriented systems. Design patterns are essentially blueprints that suggest how to solve a particular set of OO design problems while adhering to OO best good-practices (which I've recounted in my Object Oriented part of this series).

To explain by example, lets have a look at the Model-View-Controller pattern, a common pattern in use on the web and a source of much confusion amongst aspiring developers. The Model-View-Controller pattern (abbr. MVC) is a general solution for decoupling domain logic from the user interface, resulting in much better maintainability for both.

MVC suggests an application structure composed of three basic elements:

1. Models - which encapsulate domain logic (such as business rules and database access)
2. Views - which encapsulate presentation logic (such as HTML markup and recurring formatting decorators)
3. Controllers - which handle the application flow by handling requests and match models to views accordingly.

By separating those three concerns MVC aims to provide a reusable solution for improving maintainability, as each can theoretically be changed independently (in practice, things are never so simple). My personal experience has been that MVC is a huge improvement over common PHP development which promotes spaghetti code since PHP can be very easily integrated with markup (HTML).

The MVC patterns is often used with the Front-Controller pattern, which defines a single entry point to an application and allows for ever greater control of application flow (a technique also known as bootstrapping).

There are plenty of other useful patterns for the web-environment, such as Composite, Singleton (and Registry), Factory and Decorator among others.

Design patterns broke into the software mainstream with the release of the apt named book Design Patterns: Elements of Reusable Object-Oriented Software (which I think any serious developer should read). The WikiWikiWeb, which has the book authors among others as contributors, contains many articles on design patters - such as an overview of its history and a discussion on whether design patterns should be language features to name a few.

Design patterns are not limited to software design, and there are many patterns for design problems in all aspects of web development - such as graphical design and user interactions.

Extra reference material:
Patterns for PHP
User interface patterns for the web
Design patterns in web programming
Web patterns by Berkeley University
Ajax patterns

Previously: Testing, Next up: API

• Formal support for the CANVAS drawing API. Already implemented to various degrees in some browsers (excluding the usual suspect), the CANVAS drawing API is very useful for creating web graphics and interactions programmatically. (I recently wrote about a very interesting port of the Processing language using the Canvas element)
• Native drag-and-drop API (finally!)
• Formal API for editing (contentEditable/designMode). Gone are the days of 150kb+ WYSIWYG javascript libraries.
• An offline storage API
• Browser history API (for fixing broken behavior with modern techniques, ie AJAX)
• Server notifications API (standardizing a technique known as COMET).
• New attributes such as a global 'href' allowing every element to potentially be a hyperlink.

This partial list has enough improvements to make any sane web-developer happy. HTML is definitely heading in the right direction to retain its dominant position as the markup language for the web (its biggest advantage over XHTML being a much more user friendly error handling).

Now it remains to be seen how long it takes for this proposal to be finalized and implemented.

It's interesting to note that Dojo is not even featured on this list, though if you check the comments you'll see that it is used partially at the apple site (in the apple store). This makes me wonder even more regarding Zend's latest decision to integrate Dojo into their framework. As I commented in that release statement (comment #6 is mine), I feel that Dojo isn't appropriate as the default for the framework as it is more complex and much less documented than the other top frameworks.

As a long time jQuery developer I have no intentions of integrating a new Javascript library into my development environment, so I'm obviously biased. I still feel as though Zend missed an opportunity here to better cater to the needs of a broader user base and instead chose to prioritize its partners best interests.

I had believed I figured out most of the conventions used in the jQuery source code, however a recent addition has been bugging me and I could not find a reasonable explanation for it - I'm talking about the mysterious semi-colons appearing at the beginning of some of the source files in the library. What is its purpose? Does it make the closure invisible to giant robots from outer space? I had no leads to go on.

This blog post by the jQuery.rule team however, reveals the truth about the semi-colon debacle - apparently they're used for safe file concatenation (string join). Well that's a load off my chest. You learn something new every day.

In related news, jQuery UI 1.5 has been officially released, says the jQuery enquirer. jQuery 1.5 is an extensive UI oriented extension to jQuery, and version 1.5 bring forth many improvements such as a tighter API, an effects library called enchant, a skinning mechanism and plenty of bug fixes. I'm just excited they finally updated their documentation, as I've been using it for a while going only by source code.

Still, a nice tool for web designers wishing to test their HTML and CSS layouts against several generations of IE, without having to resort to hacking multiple installations of different versions (such as multipleIE).

IETester [via LifeHacker]

In addition, jQuery UI has recently reached release-candidate status, which means all non minor or trivial issues were solved and it's getting to the point that it will production ready shortly. I have been using its components for a while in anticipation of such release, and hopefully they will have their documentation up soon.

Good times to be jQuery developers