To overcome those issues, I wrote a utility class to automatically resolve Zend Framework dependencies by fetching them from the online SVN repository when they are needed (and then storing them locally). The class is available for download from Binpress under the MIT license.

How does it work:

• An autoload function is prepended to the spl autoload stack
• The auotload function catches requests for Zend Framework classes
• If the class is not available locally, the dependency manager downloads it from the SVN repository
• If the class has additional dependencies handled by internal class managers in the ZF (such as plugins and helpers), those are fetched as well
• All fetched class files are stored locally in a predefined location (such as /library/Zend as is the norm)
• The class is then loaded

Usage:

• Include the Zdm class (Zdm.php) at the beginning of the application (in the bootstrap, if you are using one)
• Run the static start() function
• The start() function accepts several optional parameters
  • 'libraryPath' - The local path to store fetched ZF classes. The default is the same directory as the Zdm class file (expecting it to be under /library)
  • 'zfVersion' - The Zend Framework version to fetch. Default is 1.10.8
  • 'prependStack' - By default the autoload function is prepended to the stack. If this interferes with any existing autoload functions, you can set this value to false and it will be appended to the stack instead.
  • 'repository' - Change the location of the repository to fetch ZF classes from. You can set this value to a local installation of the framework for better performance.
• The first time the application is run with the dependency manager, it might take some  time to download all the dependencies. To avoid the script timing out, you should set the time limit to '0' using set_time_limit()

Notes - you need to remove 'require' and 'require_once' statements that load Zend Framework classes. Those statements produce a fatal error when the file is not found and cannot be trapped by the autoload function. I tried to recover from the warning sent before the fatal error, however it could not prevent the fatal error from happening.

This class can be used to minimize the needed framework build for current projects as well. Rename your Zend Framework library directory, and allow the dependency manager to build the library from classes that are actually used in the application. This usually results in a much smaller framework footprint. I've successfully used this class in several active projects to recreate the needed framework library that runs the entire application.

It is likely there are some dependencies that are not mapped yet in the class. If you encounter such dependencies, please let me so I could add it to the class.

You need a software license when you are selling software

Seems obvious, right? however, many freelance developers do not use any license. Common misconception seems to be that when you develop custom code for a client, he basically owns the code when the job is done. That premise is usually false. While the license given to custom software solutions is usually liberal, allowing changes to source-code (with conditions) - there are many restrictions that apply on the buyer. For example, it's very rare to allow a client to resell your work as his - though without a license, he can certainly do that.

The purpose of a license is to protect your intellectual property (IP) and copyright. While you may intend to allow a client to modify the source code you provide to fit his needs, it does not necessarily mean he can sell it or include it in his other works.

Lets go over some decisions you have to make when writing a software license and explain the legal terms that describe it:

Do you want to allow clients to redistribute your software?

The most basic and critical question. If you answer YES, the license is called a Sublicensable license and it allows license holders (your clients) to resell your code with their own license, as long as that license does not contradict your original license.

Sometimes, an additional condition is added that the software cannot be sold as is and must be included in a greater work to be sublicensed. This condition prevents your client from becoming your competitor and basically selling the same product you are selling. A sublicesnable license should be used when you want to allow clients to sell solutions based on your work - for example, the client wants to sell CMS product that uses a plug-in developed by you.

The opposite of a sublicesnable license is a Personal license. A personal license prohibits license holders from reselling, renting or even allowing use of your code by 3rd parties. This type of license is common when you are selling premade (non-custom) code and want to limit use to the client only. The vast majority of desktop applications are sold under this license.

Do you want to allow clients to modify the source code?

Despite what it may seem, a sublicensable license by itself does not allow clients to modify your code. In order to allow that, the license must include the right to create derivative works. A derivative work is a modified version of your source code.

Having the right to create derivative works in a sublicensable license by itself does not give the client the right to sell modifications (only the original code). In order to allow the sale of a modified version of your source code (either as standalone or as a part of a larger work), the license must include the right to distribute derivative works.

How many sites and servers do you want to allow clients to use  your software on?

This web-oriented clause restricts the client's use of the software to pre-determined number of sites / servers. While more relevant if you are selling a non-custom solution, it can still apply to custom solutions as well. You can also provide the right to use the software on unlimited sites / servers. A common approach is to offer different pricing (mostly for non-custom solutions) depending on the extent of use granted.

Do you want to allow commercial use of your software?

While this condition is true for most custom solutions, there are cases - such as when you are giving a discount to a non-profit - where you want to prohibit commercial use. It is common for off-the-shelf solutions to have a pricing strategy in which the basic product is given for free and commercial versions or extensions are sold for a fee. In order to allow that to happen, commercial use must be covered by the license.

There are basically three main options - royalty free commercial use, non-commercial use and commercial use with conditions. The last one states that commercial use is allowed if certain criteria are met (criteria that you need to decide upon and must be specified in the license).

Do you want to allow clients to sell their license?

Self explanatory. In legal terms - a license that allows license holders to sell their license and their right to use the software to a 3rd party, is called an assignable license. A license that restricts sale of the license is called a non-assignable & non-transferable license.

Do you want to grant your clients an exclusive license?

An exclusive license allows you to sell your software to only one client (granting them exclusivity). This could be a requirement by a client, or your way to insure them that you will not be reselling the custom solution they paid you to develop. In most cases this only applies to custom solutions.

This is just a partial list - there are many more minor restrictions and rights that can be included in a software license. We cover some of those in our licensing guide on binpress, and there even more less common clauses that can be applied. If you want to create a legally termed and binding software license using those conditions, we provide a commercial license generator for registered developers in our marketplace.

If you have any questions regarding any of the topics I covered, I would be happy to elaborate or even pass it on to our attorney. The next time you will be talking to clients about a potential project, be sure to have a ready software license that covers their rights and restrictions. This can save you much anguish and provide you with legal protection in the case that things go south.

MySQL 5.0 added the index_merge select type, which allows the query optimizer to possibly select several indexes from a single table and merge them to improve query performance. I say possibly, since leaving such decisions to the optimizer is risky at best. In fact, as I will show next, you are sometimes left with no indexes selected out of several possible options, resulting in a full table scan.

Continuing from my last post, I'll use a real-world example to show the different paths the queries optimizer can take when preparing an execution plan for our queries.

I'll actually be working with the same query I profiled last time, with a minor change. The relevant table structure is as follows:

--
-- Table structure for table `tasks`
--
 
CREATE TABLE IF NOT EXISTS `tasks` (
  `id` int(13) NOT NULL AUTO_INCREMENT,
  `list_id` int(13) NOT NULL,
  `user_id` int(13) NOT NULL,
  `task` varchar(255) collate utf8_unicode_ci NOT NULL,
  `due` timestamp NULL DEFAULT NULL,
  `created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `checked` timestamp NULL DEFAULT NULL,
  `assigned` int(13) DEFAULT NULL,
  `done` enum('1') collate utf8_unicode_ci DEFAULT NULL,
  PRIMARY KEY  (`id`),
  KEY `user_id` (`user_id`,`due`),
  KEY `assigned` (`assigned`,`due`),
  KEY `due` (`due`)
) ENGINE=InnoDB

And the query itself:

SELECT `tasks`.`id`,
           `tasks`.`task`,
           UNIX_TIMESTAMP(due) AS `time`,
           `lists`.`name` AS `list_name`,
           `lists`.`id` AS `list_id`
FROM `tasks`
INNER JOIN `lists` ON lists.id=tasks.list_id
WHERE (tasks.user_id='1' OR assigned='1')
     AND (tasks.done IS NULL)
     AND (tasks.due IS NOT NULL)
ORDER BY `tasks`.`due` ASC

This query is almost the same as the one I worked with last time. The only difference being the first statement in the WHERE clause, involving an OR operator:

(tasks.user_id='1' OR assigned='1')

Just to understand what I'm doing here - The query selects from the tasks table with several filtering criteria. The last OR statement conveys the condition that the tasks selected either belong to specific user (i.e created by him) or assigned to him (those two possibly coincide).

For testing purposes I will be running the queries against a testing database I set up with plenty of mock data. The database is around 1Gb in total size, with the tasks table at about 1.8 million rows. It's not very large, but enough for significant data to be obtained while allowing relatively online tampering with schema (modifying keys takes only around 4 minutes to complete).

Running in original form computes as (average of 10 queries):

(304 total, Query took 6.84 sec)

6.84 sec is way too long for running a query in a typical web application. If you'd recall, I last got this query running at 0.0008 sec without the additional OR condition, meaning it is running at around 7,000 times slower (yikes). In such a case you would suspect the indexes are not selective enough, and sure enough an EXPLAIN reveals:

MySQL is performing a regular index select and not an index_merge as we'd like, and doing that it selects the worst possible index - the only one that doesn't filter the result set. Sure enough, all 1.8M rows are scanned and the query is underperforming badly. Trying to force the issue, I first add an IGNORE INDEX(due) to remove it from the equation:

(304 total, Query took 1.41 sec)

The query optimizer has decided not to use an index at all, despite several good candidates. However, query performance is improved since a full table scan with no index is performed straight up. It's still way too slow, and we'd like it to use a filtering index so it can avoid a full table scan. Trying to force the other indexes doesn't work so we're left with no choice but to try alternative query structures.

Our first candidate is replacing our OR condition with two UNION'ed select statements (the inspiration is from a couple of posts over at the MySQL performance blog). Breaking the original query into a UNION form results in:

(SELECT `tasks`.`id`,
            `tasks`.`task`,
            UNIX_TIMESTAMP(due) AS `time`,
            `lists`.`name` AS `list_name`,
            `lists`.`id` AS `list_id`
FROM `tasks`
INNER JOIN `lists` ON lists.id=tasks.list_id
WHERE (tasks.done IS NULL)
    AND (tasks.due IS NOT NULL)
    AND (tasks.user_id='1')
) UNION ALL (
 SELECT `tasks`.`id`,
            `tasks`.`task`,
            UNIX_TIMESTAMP(due) AS `time`,
            `lists`.`name` AS `list_name`,
            `lists`.`id` AS `list_id`
FROM `tasks`
INNER JOIN `lists` ON lists.id=tasks.list_id
WHERE (tasks.done IS NULL)
    AND (tasks.due IS NOT NULL)
    AND (assigned='1' AND tasks.user_id!='1')
) ORDER BY `time` ASC
 

This unsightly looking query looks much more complex, but actually gets the job done:

(304 total, Query took 0.0021 sec)

A big improvement to say the least (~3500 faster than the original query).
The explain shows why:

Only 304 rows are scanned (as opposed to the entire 1.8M table before). Despite the filesort, we are in the area of usability for this query.

Another approach would be to determine why the index_merging isn't take place and how can we enforce it. By reducing the WHERE clause to include only the columns covered by an index, index_merging kicks in resulting in a similar performance as the union, albeit with less selective filtering:

(368 total, Query took 0.0018 sec)

We could filter our result set in the application, but I'd like to keep it in the SQL where it belongs. In order to integrate the rest of the filtering statements, I use an IN condition on another index I have available - the primary key. This results in:

SELECT `tasks`.`id`,
           `tasks`.`task`,
           UNIX_TIMESTAMP(due) AS `time`,
           `lists`.`name` AS `list_name`,
           `lists`.`id` AS `list_id`
FROM `tasks`
INNER JOIN `lists` ON lists.id=tasks.list_id
WHERE (tasks.user_id='1' OR assigned='1')
   AND tasks.id IN
     (SELECT id
      FROM tasks
      WHERE (tasks.done IS NULL)
        AND (tasks.due IS NOT NULL)
     )
ORDER BY `tasks`.`due` ASC
 

Which brings us back to the performance levels of the UNION form:

(304 total, Query took 0.0021 sec)

And we can see in the EXPLAIN that it is indeed an index_merge operation:

So there you have it. From 6.83 seconds to 0.0021 seconds with some tweaking to the query structure. I am still not completely satisfied with the fact that it's using filesort, but I couldn't get it to use another index for the operation. Without the sorting the query is twice as fast:

(304 total, Query took 0.0010 sec)

So if it ever becomes an issue I could move sorting to the application code. Hopefully by then merge_index is better implemented in MySQL.

Regarding the query strucutre - personally I use the subquery format since it is more compact and maintainable. It is always good however to be familiar with all the alternatives.

The Performance Equation

Database performance is affected by many different variables - the running machine specs, OS, database engine and configuration, table schema and the queries running against it. Since I'm dealing mostly (only?) with MySQL, this article covers it mainly (though it is probably relevant to a large degree for other engines). As for OS and machine specs, I'll take them out of the equation as I'm interested in optimizing relative performance on the same machine.

Basically I'm interested in optimizing:

• The structure of my database tables (schema)
• The structure of my application-level queries (SELECT, UPDATE, INSERT, DELETE)

Profiling Database Performance

Blindly optimizing queries and database schema is counter-productive.  First we should know what we should be optimizing, and for that we need data. The act of gathering data for optimization is called profiling or performance analysis.

The first step I take when profiling database performance for a web app is to measure the running time of all the queries running in it. Absolute run time of a query is not necessarily a good measure of how optimized / performant it is, since some queries are naturally more complex or pull more data - It will give me a good idea however of where to start improving the response time of the application I'm optimizing. My main goal is not specific query performance, but overall system performance.

Measuring the run time of a query can be done with simple timers using microtime() calls (Check out this post for an abstraction), running it in a tool that automatically provides such statistics (phpMyAdmin for example) or using an integrated profiler.

I am using the Zend_Db_Profiler, which is convenient for me since I'm using the Zend Framework and all database access converges to a Zend_Db_Adapter connection. The profiler basically uses the microtime() approach but integrates it transparently into all of the queries without me having to wrap them one by one.

Usage is pretty simple. First you need to pass an extra parameter to your Zend_Db_Adapter instance to activate profiling:

$params = array(
    'host'     => 'localhost',
    'username' => 'dbusername',
    'password' => 'dbpassword',
    'dbname'   => 'dbname',
    'profiler' => true  // turn on profiler, disabled by default
);
 
$db = Zend_Db::factory('PDO_MYSQL', $params);

The Zend Framework manual shows some advanced usage (such as profiling directly into firebug, which is very convenient), however for our purposes we simply want to dump the queries and their run time.

$profiler = $db -> getProfiler();
 
$profile = '';
foreach($profiler -> getQueryProfiles() as $query) {
	$profile .= $query -> getQuery() . "\n"
	     . 'Time: ' . $query -> getElapsedSecs();
}
 
echo $profile;

This snippet goes of course after the view has been rendered and all queries executed.

Using a relevant dataset

It's important to know whether the data you are profiling against is relevant for when you believe you will start hitting performance issues. Granted, you can never really know when will that be, however it is always preferable to work with a dataset that resembles your (projected) production environment.

I will walk through a specific use-case I recently went through while preparing for the beta release of my startup, business platform Octabox (since defunct).

Running the profiler on one of the views in the application produced the following output on my development machine:

SELECT `history`.`id`, `history`.`type`, `history`.`action`, `tags`.`color`, `tags`.`id` AS `tag_id`
FROM `history`
LEFT JOIN `tags`
ON tags.id=history.tag_id AND tags.user_id=1
WHERE (history.user_id=1) AND (history.logged_on >= '2008-09-27 00:00:00')
ORDER BY `history`.`logged_on` DESC

Time: 0.00269

SELECT `tasks`.`id`, `tasks`.`task`,`lists`.`id` AS `list_id`
FROM `tasks`
INNER JOIN `lists` ON lists.id=tasks.list_id
WHERE (tasks.done IS NULL) AND (tasks.due IS NOT NULL) AND (tasks.user_id=1)
ORDER BY `tasks`.`due` ASC

Time: 0.000592

SELECT COUNT(*) AS `count`
FROM `dispatch`
WHERE (dispatch.user_id=1) AND (dispatch.STATUS=0)

Time: 0.00044

At first sight it would appear that the first query would be our immediate suspect for optimization, though the gains would not be great (completes in just over 0.002 seconds). Switching to a database I've prepared before hand with plenty of dummy data (~1.1Gb, some tables over 5M rows) the output looks very different:

SELECT `history`.`id`, `history`.`type`, `history`.`action`, `tags`.`color`, `tags`.`id` AS `tag_id`
FROM `history`
LEFT JOIN `tags`
ON tags.id=history.tag_id AND tags.user_id=1
WHERE (history.user_id=1) AND (history.logged_on >= '2008-09-27 00:00:00')
ORDER BY `history`.`logged_on` DESC

Time: 0.000426

SELECT `tasks`.`id`, `tasks`.`task`,`lists`.`id` AS `list_id`
FROM `tasks`
INNER JOIN `lists` ON lists.id=tasks.list_id
WHERE (tasks.done IS NULL) AND (tasks.due IS NOT NULL) AND (tasks.user_id=1)
ORDER BY `tasks`.`due` ASC

Time: 58.16 //Hmmm... problem?

SELECT COUNT(*) AS `count`
FROM `dispatch`
WHERE (dispatch.user_id=1) AND (dispatch.STATUS=0)

Time: 0.000322

A query that took just over 0.0005 seconds on a small database, took almost a minute to complete on a bigger one. Surprisingly enough the other queries remained blazingly fast (even faster than on my development machine with a small database, which is a credit to our server's power).

Optimizing The Errant Query

So I've found a very problematic query to say the least. Taking 58.1 seconds to complete is obviously not acceptable for any real time application. Running EXPLAIN against the query revealed the following execution plan:

The EXPLAIN results show the most immediate problem with the query - it was performing a join against every row in a 1.8M row table.

Examining the details of the query plan, it appears both selects are using an index, the first one limited by a WHERE (on the surface at least - since all rows were scanned). Since despite using an index the entire table was scanned very slowly, I tried to see the query will respond if I removed the index. Using IGNORE INDEX(user_id), I re-ran the query (this time in the Mysql command line):

304 rows in set (1.53 sec)

Wow, that's a big difference for not using an index. Running EXPLAIN on this reveals:

Despite using filesort to sort the query, it was running almost 30 times faster than before. By forcing it not to use an index which was not selective enough (i.e. not at all), the query ran a full table scan instead and completed much faster due to a better execution plan.

So we've improved greatly, but still not enough for my needs. Having one query taking over a second to complete when all the others are completing in sub 0.01 second times means it will be a bottleneck as the database grows (remember, the same query completed in just over 0.0005 seconds on a much smaller database).

Examining the structure of the table in question (tasks) revealed another interesting revelation - there is no index on the user_id column. So what happened? it appears there used to be an index on the user_id column, however it was removed just before all the dummy data was inserted into the database. Running ANALYZE TABLE tasks repaired the key information to the current state (no index on user_id).

I wanted to try the original query, this time with an actual index on the filtering column (user_id). After adding the index, I re-ran the query:

(304 total, Query took 0.0021 sec)

Much better! but I was not done. Running EXPLAIN yet again reveals the following execution plan:

So indeed, using an actual index was a big help, filtering the set to only 368 rows before processing the rest of the query. However, you'd notice it is running a filesort which we'd like to avoid. Preferably, I could set up an index combination that both filters and sorts using an index.

Creating an index composed of both the 'user_id' and 'due' columns got it right on the head, leading to:

(304 total, Query took 0.0008 sec)

Not as dramatic as previous improvements, but still three times better than the last iteration.

Final Words

The entire process took several hours of head scratching to arrive at all the results I've shown here. I managed to take a query that would cripple our server (58 seconds for one completion!) to an extremely fast one (0.0008 seconds against a 1.8M row table), through the use of some basic profiling and examining the execution plan.

Being an edge case (a non existent index used to filter against a 1.8M row table...), I gained valuable experience on potential pitfalls and on reading between the lines in the execution plan.

Some technical gibberish:

All queries were run with SQL_NO_CACHE to prevent caching from tampering with the results. I ran each query at least 5 times to make sure I was getting the right readings.

The specs of the server machine that ran the queries are as follows:

• Intel \ 2.4 GHz 1066FSB - Conroe \ Xeon 3060 (Dual Core)
• 2 x Generic \ 1024 MB \ DDR2 667 ECC
• 2 x Maxtor \ 146GB:SAS:10K RPM \ Atlas 10K - SAS
• CentOS Enterprise Linux - x86_64 - OS ES 5.0
• Running PHP 5.2.6 with MySQL 5.0.45 on Apache 2.2.4

A web browser built on the webkit engine (same as Safari), Chrome offers a simple UI and extensive support for web technologies. Having used it for a couple of days now, it is striking to me how obvious it is that Google is a web company - in bold contrast to another software giant currently pushing for their next-gen browser.

There is a lot to like about Chrome: It is open-source. The tab oriented UI is a great innovation. The simple UI allows for an impressively large work area in the browser. It comes with a built in DOM inspector and Javascript debugger (almost rivaling Firebug for functionality). The tab/process manager is an awesome feature (though a possible source of contention). And most importantly - aside from minor inconsistencies from the standards (shared by Safari), all web sites render perfectly under Chrome.

I was somehow surprised though it only reaches a score of 79 on the acid3 test, since it is based on an engine that already scored a perfect 100 (webkit). For comparison, Firefox 3.0.1 achieves a score of 71 (though with less graphical glitches), IE7 achieves a measly score of 13 in about triple the time, and IE8 beta 2 just barely beats that with a very low 21.

It is my personal hope that Chrome steals enough market share from Microsoft to help push out older IE versions, thus catapulting the web forward. This has been a fantastic move by Google, and the online world is watching to see how the market will respond. Is a Google OS next on the agenda?

1. Adding and encoding email headers
2. Protection against header injection
3. Multiple mail transports (SMTP is useful for sending multiple mails)
4. Creating and handling mime-compliant multipart messages
5. Composing HTML emails
6. Reading mail boxes

Sending mails

Using Zend_Mail is extremely straightforward: (Taken from the ZF documentation)

 require_once 'Zend/Mail.php';
$mail = new Zend_Mail();
$mail->setBodyText('This is the text of the mail.'); //Alternatively using setBodyHtml for HTML messages
$mail->setFrom('somebody@example.com', 'Some Sender'); //Adds a 'from' header
$mail->addTo('somebody_else@example.com', 'Some Recipient');
$mail->setSubject('TestSubject');
$mail->send();

Simple.

For sending multiple mails it is better to use the SMTP protocol for performance reasons. Doing this manually with PHP is quite elaborate however, but with Zend_Mail it is almost as simple as before:

 require_once 'Zend/Mail.php';
 
// Create transport
require_once 'Zend/Mail/Transport/Smtp.php';
$transport = new Zend_Mail_Transport_Smtp('localhost');
 
// Sending out multiple mails at once
for ($i = 0; $i > 5; $i++) {
    $mail = new Zend_Mail();
    $mail->addTo('studio@peptolab.com', 'Test');
    $mail->setFrom('studio@peptolab.com', 'Test');
    $mail->setSubject('Demonstration - Sending Multiple Mails per SMTP Connection');
    $mail->setBodyText('...Your message here...');
    $mail->send($transport); //Using the SMTP transport for sending the mail
}

* SMTP connections can be secured with SSL or TLS for sending sensitive mails.

This procedure reuses the SMTP connection for the lifetime of the entire process which is much better than sending out mails using the mail() function.

Reading mail boxes

Reading mail boxes is similarly simple. Zend_Mail supports POP3 and IMAP for remote connections, and Mbox and Maildir for local connections (mail boxes residing on the same server). Sample usage:

$mail = new Zend_Mail_Storage_Pop3( //Configuring with host and credentials
	array('host' => 'example.com',
	      'user' => 'test',
	      'password' => 'test'
	)
);
 
echo $mail->countMessages() . " messages found\n"; //Outputting message count in mail box
foreach ($mail as $message) { //Outputting message subjects
    echo "Mail from " . $message->from . ": " . $message->subject . "\n";
}

Using the different storage options is very similar. Messages returned are instances of the Zend_Mail_Message class, which provides an API for manipulating mail messages.

Advanced Usage

Working with multipart MIME messages is very tedious. The MIME specifications are rather complex and MIME messages aren't very readable to humans, and therefor we would like to be able to use Zend_Mail_Message API to reduce the pains of dealing with MIME messages. There are several general cases when working with MIME messages:

1. Working with messages which are stored as strings in a database
2. Working with messages which are read from files
3. Working with incoming messages directly (from an input stream)

Zend_Mail_Messsage can receive either a raw message string in its constructor, a file name or a file handle. Examples of instancing a Zend_Mail_Message object with all three:

//From a raw string
$message = new Zend_Mail_Message(array('raw' => $rawMessage)); 
 
//From a file name
$message = new Zend_Mail_Message(array('file' => '/path/to/mail/message'));
 
//From a file handle
$handle = fopen('/path/to/mail/message');
$message = new Zend_Mail_Message(array('file' => $handle));

The last one might seem a bit redundant. However it opens up creative ways to intercept mail directly using the input wrapper:

$handle = fopen("php://stdin", "r");
$message = new Zend_Mail_Message(array('file' => $handle));

The tricky part is directing the mail message stream into the script. You can read more about that in this nice tutorial from evolt.

Some minor bugs and fixes

Nothing is perfect however. Current version of Zend_Mail (shipped with the Zend_Framework ver. 1.5.2) has a buggy implementation of the headers encoding. This will become noticeable when working with UTF encoded mails. Fortunately there is a quick fix for that posted in the ZF issue tracker (and hopefully will be tested and integrated in the next release).

Direct usage statistics are hard to ascertain with great precision, but they revolve around 4%-6% of the total Internet population, which is not much. More feeds are being consumed indirectly by aggregation sites, such as my yahoo and iGoogle (as this Yahoo! paper shows), which shows that there is market ready to consume more feeds.

So why feed usage isn't more widespread?

1. Lack of awareness
Most people are simply unaware of the benefits of using feeds. When asked about what are feeds, only 12% of the Internet public were aware of its meaning. I will admit myself to be one of the ignorant up until not too long ago (about 6 months). For me that is mind boggling - I consider myself an extremely technical user and I love the Internet. How was I not aware of how useful this technology is?

2. Confusing terminology
What is a feed? syndication? aggregators? to someone hearing those terms for the first time it can be very discouraging. There is alternative terminology such as subscriptions and readers which is more similar to natural language but it isn't used enough to explain the technology.

3. Competing standards
Atom and RSS are competing standards for web syndication. Both are offering basically the same solution, so why haven't they merged yet? (actually there are reasons, but not very good ones). Most sites now offer both, which only add to the confusion as the names of the different formats are sometimes use interchangeably with the term Feed.

Content syndication is just waiting to erupt. Someone needs to take initiative and promote this technology to the masses:

1. Create awareness
The same way open standards such as OpenID made a splash some time ago, content syndication needs visibility. And in contrast to open standards, web syndication is useful now.

2. Improve description
Hand in hand in creating visibility, a low-tech explanation of the benefits of syndication should be drafted and used liberally. This will go a long way to break the barriers of adoption.

3. Unify standards
One to rule them all as they say. Since this is an open standard and certainly not for profit, the only real consideration would be backwards compatibility. Declare RSS officially as the standby for compatibility and Atom as the active standard for future development.

4. Do not call the technology by its standards name!
Stop using RSS or Atom feeds as the name for the subscription feature. People are much more likely to hit 'Subscribe' than 'Join my RSS feeds'...

P.S.
Syndication always reminds me of the PC classic 'Syndicate' in which rival clans fight for world domination in a futuristic setting using cyborg agents. Good times...

It's interesting to note that Dojo is not even featured on this list, though if you check the comments you'll see that it is used partially at the apple site (in the apple store). This makes me wonder even more regarding Zend's latest decision to integrate Dojo into their framework. As I commented in that release statement (comment #6 is mine), I feel that Dojo isn't appropriate as the default for the framework as it is more complex and much less documented than the other top frameworks.

As a long time jQuery developer I have no intentions of integrating a new Javascript library into my development environment, so I'm obviously biased. I still feel as though Zend missed an opportunity here to better cater to the needs of a broader user base and instead chose to prioritize its partners best interests.

I had believed I figured out most of the conventions used in the jQuery source code, however a recent addition has been bugging me and I could not find a reasonable explanation for it - I'm talking about the mysterious semi-colons appearing at the beginning of some of the source files in the library. What is its purpose? Does it make the closure invisible to giant robots from outer space? I had no leads to go on.

This blog post by the jQuery.rule team however, reveals the truth about the semi-colon debacle - apparently they're used for safe file concatenation (string join). Well that's a load off my chest. You learn something new every day.

In related news, jQuery UI 1.5 has been officially released, says the jQuery enquirer. jQuery 1.5 is an extensive UI oriented extension to jQuery, and version 1.5 bring forth many improvements such as a tighter API, an effects library called enchant, a skinning mechanism and plenty of bug fixes. I'm just excited they finally updated their documentation, as I've been using it for a while going only by source code.

Paginated queries typically look like this:

SELECT * FROM articles LIMIT 20, 10 //Selecting page 3 FROM TABLE articles (10 rows per page)

In order to generate page navigation, we would need to know the number of pages. We count the total rows that our query generates without the limit clause:

SELECT COUNT(*) FROM articles //Count ALL rows IN TABLE articles

And then we can perform simple arithmetics to calculate the number of pages.

It's relatively straightforward when dealing with simple queries, however for complicated queries (with multiple JOIN clauses) it can be quite a chore.

Lets switch to the Zend Framework and compose ourselves a somewhat elaborate query using Zend_Db_Select:

$select = new Zend_Db_Select();
$select -> from(array('pil' => 'project_item_list'),array('*'));
	-> join(array('pt' => 'project_tasks'),'pt.id=pil.task_id',array('task','ptid' => 'id'));
	-> join('projects','projects.id=pt.project_id',array());
	-> join(array('ptu' => 'projects_to_users'),'ptu.project_id=projects.id',array());
	-> join('users','users.id=pil.assigned_to',array('assignee' => 'name'));
	-> where("pt.project_id='" . $projectId . "'");
	-> where("ptu.user_id='" . $userId . "'");
	-> where("task_id='" . $taskId . "'");
	-> where("pil.assigned_to='" . $assignedTo . "'")
	-> limitPage(3,10);

Plenty of JOIN goodness. You'll notice the limitPage() call at the end, which is a nice Zend_Db_Select helper to add a limit clause by a page number (in this case it means limit by page 3, 10 rows per page).

So how do we count the number of rows? We'd want to have a COUNT() function returned from the base table (the one that appears at the from clause), however if we mix that with other columns, we'd have to use GROUP BY which would mess up our result-set.

There are two ways to handle this - generating a similar query sans the columns and the limit clauses, or using the SQL_CALC_FOUND_ROWS() function. The first approach results in two seperate queries while the second is just one query, however it is not that obvious which one is better for performance. For our case we'll use the first method, duplicating the original query while stripping out whats needed.

We could recompose the entire query using copy-paste methods, removing the columns and the limit clauses, but we would like a more general solution. I extend Zend_Db_Select and add a simple utility method:

class Techfounder_Db_Select extends Zend_Db_Select {
	public function prepareForCount($countColName = 'count') {
		$this -> reset(Zend_Db_Select::COLUMNS);
		$this -> reset(Zend_Db_Select::LIMIT_COUNT);
		$this -> reset(Zend_Db_Select::LIMIT_OFFSET);
		$table = key($this -> _parts['from']);
		$this -> _parts['columns'] = array(
			0 => array(
				0 => $table,
				1 => new Zend_Db_Expr('COUNT(*)'),
				2 => $countColName
			)
		);
		return $this;
	}
}

The prepareForCount() method modifies a select object to strip all previously set columns and LIMIT clauses, and to set a COUNT() function on the base table in the query (the row count is returned as a key named 'count'). We can now use this Select class to do the following:

$select = new Techfounder_Db_Select();
... //Multiple JOIN query, as composed previously
 
$countSelect = clone $select;
$countSelect -> prepareForCount();
$result = $db -> fetchRow($countSelect); //$db is an instance of Zend_Db_Adapter
$pageCount = ((int)(($result['count'] - 1) / $rowCount)) + 1; //Using the number of rows to calculate the number of pages.
//$rowCount is the number of rows per page

I go even a step further, and encapsulate this short process in a countPages() method in a custom wrapper for Zend_Db_Select (called Zend_Db_Gateway, for which I submitted a proposal and would talk about in my conclusion to my Models in the Zend Framework series).