I've recently completed a project where a very bizarre bug would occur on some machines, and I wasn't able to reproduce it myself on any test machine that I tried. The bug manifested in a calendar system we built for the project, and in certain situations form submissions would not pass the date validation through Zend_Validate_Date, despite using the exact same input that was passing on the test machines.

After bringing in one of the "affected" machines for testing, I used the process of elimination to determine the problem originates in the Zend_Date::isDate() method, which has different behavior on different machines.

Zend_Date tries to validate dates according to a given format (with a default fallback). The dangerous behavior is that it tries to convert the given format to a localized format using Zend_Locale. Zend_Locale attempts to detect automatically the locale of the requesting client, and it appears that on the machines that were exhibiting the bug, a different locale was determined than those I was testing it on.

Despite trying to validate a non-localized format (to be exact, the MySQL timestamp format - YYYY-MM-dd HH:mm:sss), it was being converted on the affected machine sdue to the auto-detection by Zend_Locale (on a side note - it was happening on Internet Explorer 8 on Windows 7). To avoid this issue, I set the locale for all requests manually to en_US, since I don't have any other use for Zend_Locale in the application. I put the following lines in the bootstrap -

$locale = new Zend_Locale('en_US');
Zend_Registry::set('Zend_Locale', $locale);

As recommended by the manual to achieve this effect.

In my opinion, the Locale should be not be auto-detected by default - it should be an opt-in feature if it affects other components behind the scenes.

SELECT product_images.src,products.name
FROM product_images
INNER JOIN products ON product_images.parent_id=products.id

Literally: Get the name of the parent product for each row in the products_images table.

Going the other direction is not as trivial. If we just want all the referenced rows in the child table, the query seen before would do the trick. But if we wanted a specific row to be found - for example, the latest inserted image, it becomes somewhat more complicated.

Lets explore a common scenario - fetching the latest comment for each row in a list of posts in a forum.

Latest posts from the devnet forums

Dealing with the following simplified schema for a forum:

CREATE TABLE `threads` (
   `id` INT NOT NULL  AUTO_INCREMENT,
   `title` VARCHAR( 250 ) NOT NULL ,
   `created` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ,
   PRIMARY KEY ( `id` )
) ENGINE = InnoDB;
 
CREATE TABLE `posts` (
   `id` INT NOT NULL  AUTO_INCREMENT,
   `thread_id` INT NOT NULL ,
   `content` TEXT NOT NULL ,
   `created` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ,
   PRIMARY KEY ( `id` ) ,
   INDEX ( `thread_id` )
) ENGINE = InnoDB;

A naive approach to fetching 10 threads with the latest post for each thread would be something like:

SELECT threads.*, posts.*
FROM threads
LEFT JOIN posts ON posts.thread_id=threads.id
GROUP BY threads.id
ORDER BY posts.created DESC
LIMIT 10

This query might appear to work as long as there is only one post per thread. Once there are more, you will start getting unexpected results (go ahead, try it). Lets try and understand why:

In the SQL standard, a GROUP BY modifier can return only columns that are either aggregated or the same for the entire group. MySQL extends this and allows returning non unique or aggregated columns from the group, instead of throwing an error.

However, values from those columns could come from any row in the group and we have no control over which row it will be.

... all rows in each group should have the same values for the columns that are ommitted from the GROUP BY part. The server is free to return any value from the group, so the results are indeterminate unless all values are the same.
The MySQL manual on GROUP BY modifiers

In addition, grouping occurs before the order clause is even applied. After the grouping, unpredictable values for the order column (posts.created) would exist for each group, making the sorting afterward somewhat meaningless (since we wanted the latest post for each thread).

A more experienced SQL user might think - why not just use an aggregate function instead? leading to the following query:

SELECT threads.*, posts.*, MAX(posts.created) AS maxcreated
FROM threads
LEFT JOIN posts ON posts.thread_id=threads.id
GROUP BY threads.id
ORDER BY maxcreated DESC
LIMIT 10

This is somewhat better, as it does retrieve the timestamp belonging to the latest post in each thread - however, surprisingly (or not, if you've been following closely) the rest of the non grouped columns from the threads table are still indeterminate and do not necessarily belong to the row with the latest timestamp.

The solution to this issue is not completely trivial, but it makes sense once you understand the limitations. The key is to use the GROUP BY statement to return the filtering criteria (latest timestamps from the posts table) but to avoid it affecting the result set directly.

First, the filtering criteria -

SELECT thread_id,MAX(created) AS created
FROM posts
GROUP BY posts.thread_id

Here we return only aggregate (latest timestamp) and unique (thread_id) data for each group. Now we have the timestamp of the latest post from each thread, along with the matching thread identifier.

We will move the filtering criteria into a subquery, and JOIN that against the threads and posts table respectively.

SELECT threads.*, posts.*
FROM threads
LEFT JOIN (
   SELECT thread_id,MAX(created) AS created FROM posts
   GROUP BY thread_id
) AS latest ON latest.thread_id = threads.id
LEFT JOIN posts ON latest.created=posts.created AND
latest.thread_id=posts.thread_id
ORDER BY posts.created DESC
LIMIT 10

Literally: Fetch rows from the threads table, (left) join that with the timestamp from the latest post in each thread, and then (left) join that with the posts table with the condition that the timestamp and thread_id match. This returns all the rows in the threads table along with the latest post from each thread.

People have a disposition for avoiding subqueries due to performance concerns, but in this case it is a derived subquery (also referred to as a derived table) which will compute once for the entire query as opposed to a correlated subquery which computes once per row ( = bad performance with many rows).

* Note - in real-world situations you would like to specify the columns to be returned in the result set

For more advanced usage, I suggest you read Baron Schwartz's article on selecting the first/least/max row per group in SQL

What is a [web] widget

... a web widget is a portable chunk of code that can be installed and executed within any separate HTML-based web page by an end user without requiring additional compilation.
Web widget by Wikipedia

Put simply, we need to provide prospective site owners with a short piece of code (henceforth, the "embed code") that they can place on their sites and have the widget appear without any additional modifications to the site.

How would we go about creating an embeddable widget?

Cross-site scripting (XSS)

Cross-site scripting is more often mentioned as a web vulnerability which uses the same technique maliciously to manipulate the target site. In this case, Javascript will be used to create the widget on the host site and not to attack it (you should always beware of including scripts from untrusted sources).

This technique is based on embedding a script <tag> on the host site that will refer to a script residing on your service. The script should then dynamically create the widget on the host site by manipulating the DOM and adding stylesheets and script tags as needed. Dr. Nic gives a nice overview of this technique in this detailed tutorial.

As the tutorial mentions, your embeddable script should have a unique namespace to avoid colliding with local scripts on the host site. Don't be shy to use a long and verbose name for this purpose. Example:

;(function() {
	var techfounder_super_embed = window.techfounder_super_embed = {
		init: function() {
			alert('embedded!');
		}
	};
	techfounder_super_embed.init();
})();

Some quick notes:

• I used a self executing anonymous function. This allows me to declare additional variables and functions other than my main namespace ('techfounder_super_embed' in this case) without polluting the main scope or interferring with existing code on the site.
• In order for the main namespace to be available outside of the anonymous function scope, I attach it to the window object.
• I preceded the function with a semi-colon to close any potentially open Javascript statement that might have been loaded before it, which would have resulted in an error.

As opposed to the advice given in Dr. Nic's article, I usually opt to use an external library such as jQuery, in order to better account for cross-browser differences as well as leverage the power inherent in such libraries. jQuery in particular has the useful noConflict() method which allows it to work side-by-side with other libraries, and can be loaded from a google CDN so it does not add bandwidth load on your server.

As for retrieving data cross-site (from your service server to the host site), there are 3 paths you could take:

• Embedding extra script tags which include JSON data that is constructed dynamically (as shown in Dr. Nic's tutorial).
• Creating an Iframe element that accesses a remote resource (ie, on your server) to retrieve the data, and then passing the data back to the document.
• Using a JSONP call to simulate a cross-site AJAX request. I personally opt for this technique as I find it the most elegant. jQuery supports this natively in recent versions, making it almost too easy to implement.

An important note to keep in mind that styling the widget elements should be done very explicitly. As the widget is embedded directly into the host document it could be affected by any previously loaded stylesheets. A good start would be to use a CSS reset (such as this one by Eric Meyer) and apply it directly to a unique ID that identifies the base widget element.

Iframe embedding

An Iframe is the most direct approach to including content from a remote site. It's easy to pass arguments in the Iframe source URL and have a response dynamically built in return. However the Iframe approach suffers from some limitations:

• You have to specify the Iframe proportions in the embed code and it should match the content exactly, else the content can get truncated. This also means the embed code would need to change if the widget appearance changes (which comes up often for customizable widgets).
• There are some security limitations for Iframe scripting, so if you need to manipulate the host site page in any additional way,  you would need to include a script <tag> as well in the initial embed code to bypass the security restrictions.
• Iframes are invalid markup in XHTML Strict doctype. Depending on the host site doctype declaration, this could present a minor issue (if the host site owner is concerned about with valid markup).

A big advantage in using an Iframe over directly embedding the widget in the host page is that you don't have to be concerned with external styles affecting the widget.

Which should you use

Cross site scripting is useful when there is a need to dynamically manipulate the host site page. This would include placing the widget in positions different to where the embed code was placed at, offering additional user-interface interactions or communicating data back to your server (as Google analytics does, for example).

The Iframe approach is useful when you can tell ahead of time the exact size of the embedded widget, and you only want it to display content and not offer any Javascript interactions that occur outside of the Iframe.

I'm looking forward to expand discussion on some subjects I've brought up in the past (especially models in the Zend Framework) as well write on some of things we've been doing lately. Hope you will find it useful / interesting.

The post links to a PDF document written by three Google researchers, and covers a subject I've been experimenting with a lot lately - the semantic extraction of data.

The document is a nice read though probably too technical for most, and it brings up the difficulty of implementation as one of the barriers for taking the next step towards structured data on the web. The argument is that most small content publishers do not posses the knowledge and expertise required to publish their content in a semantically meaningful way.

It seems to me that there's an easy solution for that - and that is to embed semantic awareness in the publishing tools themselves. Most people publish content through one of several high-profile content management systems (WordPress, Moveable type, Blogger, etc), meaning it is possible to reach a very large segment of content publishers from relatively few integration points.

Expecting people to learn about and implement web semantics is unreasonable, as the document suggests. Delegating that responsibility to the authoring tools by enhancing the backend logic and the interface, is very much doable. Need to add a calender event? allow the interface to add it in the proper microformat. Want to affect the styling of your content? allow the interface to give several semantically significant options (headers, paragraphs etc.). Most of those options are available today, yet they are not obvious enough that they are used in a consistent manner.

HTML 5 is around the corner, with more semantically relevant tags (such as header, footer, navigation, sections). Integrating support for it in those content management systems is a good first step towards more semantically accessible content.

Going even further with this, content management systems could provide more structured data on demand - or wait, they are already doing this! XML feeds, anyone? seems to me that if you poke a little beneath the surface you find that there is actually a lot of structure in modern online content. What's lacking is uniformity, but even so the number of dominant standards is not great.

Who will be "the next Google" who can examine this structure and extract meta-meaning that can provide value? you can be certain some bright minds are already on the case.

How much does it cost to build a website? what does building a website entails? there are no universal answers to those questions.

Client: "Enough talking, let's get down to business. What will 50$ get me?"
Brad: * looks at wrist watch *
Brad: "About 5 more minutes of my time."
Brad Colbow on Time

In fact, there is so much variation, that a recent client told us post-project completion that when he was shopping around for offers from various firms, he got offers between 2000$ and 60,000$. That's a factor of 30 between offers!

He picked us since we conveyed the best value (quality / price) of all offers, even though he had much cheaper offers (we were about midway in the range). All things being equal, all he had was his gut-feeling and our resume to guide him. That is, if we didn't engage in client development.

The client-provider relationship in web development is complicated. Most would-be clients are lay-man, possessing no insight on what goes into every feature and pixel you see on a website. They are also, most likely, unfamiliar with their prospective provider and how to evaluate him - is he a hack? what are the qualities that he brings to table? how can I quantify the value he gives me?

That's where the process we term "Client development" comes in. It involves several activities, most of which are not directly related to web development:

Matching expectations

This is a basic rule of engagement in most business operations, and web-development is not excluded - especially since it is such a misunderstood and technical field.

A lot of clients look at website features and think "that's easy! you just put the thing there and do that...". Well, in the real world things are usually more complicated. What might look simple on the surface could involve a lot of back-end work and a lot of custom integration with various services... and the user interface itself is probably more work than you'd think.

The first thing we try to do is lower expectations to some degree. Our stance is that if it takes any amount of time to design / develop / test, then it will be priced in the proposal. That might sound harsh - but it's the only way to properly evaluate and convey the value of the project.

As I've written in conveying value to clients, creating a transparent and fair price proposal can go a long way to matching up the client's expectations with yours. It should be accompanied by a well written specification document, composed interactively with the client and which is completely understood by him (or her).

Educating clients

To help clients really get a sense of what they are getting in return for their money, it's important to explain some basic terminology and elaborate on the process of web development.

Explaining terms such as usability, user experience, mockups, layouts, client-side / server-side, testing, maintainability, standards and the interactions between them can really shed light on the considerations and work involved in building a website.

This in turn allows clients to better understand the price-stamp put on features, and feel more secure overall in decisions they make since they better understand what is involved.

Knowing when to say "No"

In some cases, clients get over involved in day-to-day decisions, especially in something as tangible as design. The design of a website is very visible which makes it very susceptible to micro-management and over-feedback from clients as they watch it evolve into a finished product. Convincing clients to trust our experience and expertise is imperative.

Another aspect that is constantly under-pressure is scope. Scope-creep is a common yet undesirable side-effect of perfectionism inherent in most of us (including clients). This tendency to think how things could be improved beyond what was originally specified is something that needs to be properly monitored.

This often means just saying a strict "No" to client suggestions and ideas when it's appropriate to do so. If features and requirements change (and they will), they have to come at the expense of some other features deemed less important. The project scope must be maintained.

Selecting clients

Client development is a two way street. Consider for example the following project outline:

US Company looking to develop financial website

Requirements:
Budget 10,000$-25,000$
Maximum Billrate per Man Hour: $6/hour
Minimum Number of Men assigned to Project: 30

We require a well established firm that meets these basic requirements in addition to being able to delivery quality and accuracy in its work.

(This is an actual posting on Elance, in case you were wondering)

In short, this person wants a financial website, but gives no project requirements / specifications or feature list of any kind. Instead, the requirements are that the provider:
A) Has a minimum of 30 men (why gender biased?) assigned to the project, and
B) No one should be paid more than 6$ an hour (which is less than what a register clerk at McDonalds makes)

For this he wants quality and accuracy. Is his expectations unreasonable? might be (I'm voting no).

Even more common is the "work for minor shares" offer in what is probably "the next big thing". Somehow people believe that having an idea by itself is sufficient leverage, and should prompt serious firms into developing their products.

The fact that there's no market for startup ideas suggests there's no demand.
Which means, in the narrow sense of the word, that startup ideas are worthless.
Paul Graham on Ideas for startups

People in the industry seem to agree that execution is the main key for the success of a startup. We do execution (and also concept development, but that's for another article).

The point I'm making is that a good relationship with a client can only be achieved when both sides understand and respect each other. This is not just about money, it's about the value generated by both sides.

What does this mean? it means that in order to get the right clients, we need to be selective ourselves. Clients not only choose us, but we get to choose them as well. For this reason, we should never undercut our price-proposals. We need to believe they represent not only fair, but great value.

It's the client's privilege to agree or disagree, but we want to work with people who understand the value we represent. If they want to pay less, they need to drop features. It's give-and-take and there are no freebies - this is a business after all. Nobody gets rich from web development, the least they could do is get paid on par with the value they represent.

This feeling out process usually happens during the expectation matching phase. If both sides can not bring their expectations to the same level, it's best to part ways and wish each other the best than trying to hammer in a shaky relationship.

It's up to you

Client development is a process that a lot of small businesses and freelancers skip (and sometimes larger firms as well). Whether you put the time and effort to make prospective clients your partners instead of your employers is up to you.

It got me thinking though, on what are the attributes I consider useful in fellow programmers. So what makes a good programmer?

An analytical thinker

Programmers need to be problem solvers. The process of programming requires that we systematically break complicated problems down, plan and implement solutions and find / eliminate small inconsistencies in code (bugs).

Analytical thinking also manifests in the ability to follow complicated logic through disparate code segments and understand it. It allows us to grasp abstract concepts such as Object Oriented methodology and design patterns and implement it in practice.

Has his priorities straight

If I would ask you to rate the following according to priority, how would you order them?

• Security
• Maintainability
• Usability
• Performance
• LOC (lines-of-code) count

Take a moment to think about that, and then consider:

1. If you picked LOC count first, you failed big time in my book. In fact, LOC optimization can often go directly against the other metrics (such as maintainability). A lower LOC count should never be a goal, only a result of careful application of well factored architecture.
2. If you picked performance first, you are probably the guy who keeps writing articles about how you should use a while loop instead of a for loop since it came out a few milliseconds faster in your benchmarks. You might be inflicted with a case of premature optimization.
   

   We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.
   Donald Knuth

   Performance should only be good enough to satisfy the requirements of the application. Aside from caveats to well-known pitfalls (such as executing queries in each iteration of a long loop), performance optimizations should be deferred to the very last and even then should be used appropriately (profile ... profile ... profile ... optimize).

   The only exception to this is if you are primarily developing performance dependent applications (such as low-level system drivers).

3. Security is on somewhat of a middle ground. Depending on the application and distribution model it can be completely useless or mission critical. It's mostly somewhere in between, and thus can't be ranked as number one.
4. Maintainability is definitely one of the most important attributes of a software application. High maintainability allows you to improve other attributes (such as performance), when it is needed.

   Maintainability is the single most important factor for keeping productivity up and costs down. For a long time I strongly believed this to be the most important attribute of software design.
   However ...

5. The most important attribute is usability. In the end, the worth of your application is only as much value as it delivers to the end-user.

   We should always remember - software is not written to serve its developers or the systems they run on. They are written to solve problems. If those problems are not solved, then the project is a failure.

   I wrote usability here as a more general term than just UI/UX effectiveness. Even a command line application or a background service has its usability factor in the sense of how well it answers a specific need.

Gets things done

In principle, it’s simple. You’re looking for people who are

1. Smart, and
2. Get things done.
Joel Spolsky

Quite possibly the single most important trait in a developer. You can excel at all the previous attributes and still be a mediocre programmer if you just can't get things done. One average but productive developer could easily replace several highly talented but slowly moving developers, depending on his responsibilities.

At the end of the day you definitely want more highly-productive developers than those who are high on theory but not actual work.

Does more than "just enough"

Getting things done is important. Getting things done "the right way" is even more important.

Constantly paying off your technical debt is crucial - if you keep accruing debt by "hacking" quick fixes that work now but are not maintainable, you only create the appearance of progress. In reality, the cost of getting rid of a large technical debt could become prohibitive before you know it.

Taking the time to constantly refactor code into a more maintainable state is the best way to prevent the spiral into project oblivion.

Responsible

A person could be a very capable programmer on technical ability alone, however if he does not own up to his mistakes and does not respect deadlines he could become a liability very quickly.

Responsibility also means to know where to let go of your ego for the good of the project. We developer often high large egos as we consider ourselves experts on many things. Putting the project first is a sign of a good developer.

Good human relations

Another all-around useful trait, this one applies to programmers as well. There is some stereotype that programmers are reclusive, unsociable creatures - programmers are still people .

In order to be a part of a team or handle clients, a programmer must have above basic social skills. Rudeness, arrogance, short-temper - do not have a place in a professional work environment. All it takes is one bad apple to ruin the mood for everybody.

That's about it

If you answer to all of the above, you are probably a pretty good programmer (and you are welcome to apply with us ).

If you read the article I mentioned at the beginning, you might notice I didn't mention passion or technological diversity as qualifying traits. Simply put, I don't think they're very relevant to the quality of a programmer.

Passion is nice to have, however I've known many very professional and high-quality developers who were just content to go about their work professionally from 9 to 5 and then go home and have a meaningful and fulfilling family life. A programmer can definitely completely professional without being passionate about programming.

Technological diversity is another nice to have but not a prerequisite - as long as you are in command of the technologies you work with, a lack of diversity shouldn't affect you too much. Decision makers need to be well aware of all the options before starting a project, however nowadays the choice of technology simply is not that important.

You can achieve good results regardless of the programming language and database engine among other consideration. The biggest consideration should be the type of skills available to your personnel.

I hope to see more suggestions and thoughts in the comments on what you think make a good programmer.

Why spec?

Specifications encapsulate a project's goals, requirements, features and scope. This is very important for several reasons:

Understanding project requirements

Understanding what a project sets out to achieve might seem obvious, but in reality, when at least two people are involved - some difference in perception of the project goals and needs is inevitable. Putting in writing the goals and requirements goes a long way towards resolving such differences.

Writing specifications will often reveal requirements not thought of initially, as a result of detailing and thinking through the initial set of requirements. This is very important for determining project scope.

Developing a project scope

The scope of a project is probably the single most important attribute for determining project success. A well defined scope will determine the time-frame, costs and requirements with good precision.

In essence, a scope is the sum of a project features, requirements and deliverables or in other words - the sum of the work required for project completion.

Scope is the main element against which projects are priced at Lionite.

The more complicated and/or innovative a project is, the more likely it is that features will change during the project lifetime (sometimes even drastically).

We can allow that to happen by adhering to scope instead of individual features. With good specifications it is much easier to avoid scope-creep.

Estimating project time-frame and pricing

Accurate time and cost estimates are important are as important to project success as anything else, and specifications are what it should be based on.

The better the written specifications are, the more accurate those estimates would be.

Creating a workable reference for development and design

Working with a good specifications document can do wonders for productivity and focus in the design and development stages. At the very least it can serve as a task list for project completion.

Well thought through description of features and relevant mockups can solve a lot of potential dillemas for both designers and developers and save a lot of time.

The specifications process itself should be used for ironing out as many issues as possible, helping the implementation phase to be as productive as possible.

Having a standard to test against project delivery

Specifications outline the agreement between parties on what constitutes a successful project delivery. Contracts, if relevant, will point to the specifications as the legally binding document for this purpose.

A good specifications document can resolve many potential disputes simply by stating clearly what should be delivered.

How to create better specifications

Creating good specifications entails some ground work, there's no around it. Keep in mind the effort you put in will pay itself back as the project progresses.

Break everything down

Decompose every requirement and feature to a list of small tasks. Add relevant detail that might affect completion time. The level of detail and scope of each task should be enough so that it could be accurately estimated in work hours (and not days, and certainly not weeks).

Research what you are not sufficiently familiar with. This will greatly help avoid nasty surprises later, and will make your estimates that much more accurate.

Be thorough

Try to make sure there are no gaping holes in the specifications - are all requirements and features accounted for? are there any hidden prerequisites you might have over-looked? read everything twice and then give it to a colleague to read.

Don't forget to add more technical detail such as security concerns, coding standards, post-project technical support and supported browsers.

Compose interactively with clients

It is very important for the specifications to capture correctly the vision the client has for the project. Specifications writing can be used to correctly align client's expectations with your own.

Your interaction with the client can (and probably will) lead to more insights on project requirements and features. It is not at all uncommon for a project to change direction somewhat during specifications writing - and it is much better that it happens at that time than at mid-project.

Be clear and concise

Be clear and and use as much detail as necessary. Be concise, since overly descriptive texts are harder to consume and understand.

Don't skimp on rewording and reorganizing bulky / technical paragraphs and edit out unnecessary detail.

Read more articles like this one

Joel Spolsky on painless functional specifications - part 1, part 2
Allen Smith on functional specifications
Jason Fried of 37Signals on avoiding functional specifications (basically refuting everything I've written here. A good read for counter arguments.)

Too often I've seen such queries ran in long loops one at a time, which is very bad for performance (as I will show here) and sometimes equally bad for integrity (if the process is interrupted). So what are the alternatives?

Inserting multiple rows

Insertion of multiple rows comes about often in batch jobs, database migrations and handling table relationships. A naive approach, via PHP, would be to loop over the data to be inserted, inserting one row at a time. Suppose the data is already properly filtered and quoted:

foreach( $data as $row ) {
      $query = "INSERT INTO `test_table` (user_id,content)"
          . " VALUES ("
          . $row['user_id'] . ","
          . $row['content'] . ")";
      mysql_query($query);
}

Depending on the amount of rows to be inserted, this can be a costly process. A better approach would be to concatenate the values into one insert query and then execute it:

$values = array();
foreach( $data as $row ) {
    $values[] =  "(" . $row['user_id'] . "," . $row['content'] . ")";
}
if( !empty($values) ) {
    $query = "INSERT INTO `test_table` (user_id,content) VALUES "
             . implode(',',$values);
    mysql_query($query);
}

What's the difference? lets see some benchmarks:

A single query completes much faster than looping through multiple queries. At 2560 rows inserted, it took the loop ~36 seconds to complete, yet the single query took just 0.14 seconds.

Memory consumption shows a reverse trend however:

Since the values are concatenated to create the single query, it consumes more and more memory with more rows as it needs to hold a larger query string. At 2560 rows inserted, the single query approach consumed ~800kb in memory, while the loop consumed just ~60kb.

Since memory is much cheaper than CPU cycles and database connections, I'd usually opt for the single query approach. It's important though to be aware of the implications.

Inserting / Updating (multiple) values

Another use-case of multiple row operations is when we want to insert several rows that might exist already, and in case they exist we want to update existing values instead. Of course, we could check first which rows exist, update the ones that do and insert the ones that don't - for a total of at least three separate queries.

The update query is the major problem here - there is no way to update multiple rows with different values using one query (and it doesn't make much sense in most cases). However in this case it does make sense to do so, and fortunately MySQL provides a way to do just that, using INSERT ... ON DUPLICATE KEY UPDATE.

The KEY in this statement should be a unique key in the table you are inserting to (otherwise duplicates would be allowed and this operation would be moot). The syntax of this statement allows to pick which columns are updated in the case of matching rows. For example, suppose I'm adding translations for content pages. I have the following table schema:

 
pages
 - id
 - content
 - ...
 
pages_translations
 - page_id
 - lang_id
 - content
 

The pages_translations table has a primary (unique) key on ( page_id , lang_id ). When adding / updating a translation, the query would look something like:

INSERT INTO `pages_translations` (page_id,lang_id,content)
VALUES (5,2,'the brown fox ...')
ON DUPLICATE KEY UPDATE content=VALUES(content)
 

This will create a translation if one does not exist and update the content field if it does exist. We can combine this statement with the previous approach for multiple insertion to insert / update multiple rows with one statement.

Inserting / Updating with multiple tables

There are some cases where we would like to use data in one or several tables to update / insert into another table.

With insertion the syntax is relatively straightforward - replace the VALUES part of the statement with a SELECT statement.

INSERT INTO `my_table` (col1,col2,col3)
SELECT col4,col5,col6
FROM `another_table`

The ON DUPLICATE KEY condition can be used with this statement as well.

Updating rows cross-tables is done with a JOIN statement (of the less declarative type).

UPDATE `my_table`,`other_table`
   SET `my_table`.col1 = `other_table`.col2
WHERE `my_table`.parent_id = `other_table`.id

It's important to try the INSERT / UPDATE select statements separately to determine how many rows they will select and how fast will they complete. INSERT / UPDATE operations are relatively costly and can be a severe bottleneck for the database if not managed correctly.

Despite this, the application of those business models in the web arena is still very much experimental, and it's often hard for web businesses to find and implement a model that works well for them.

Got ads?

The most basic misconception about web business models is that you can never go wrong choosing an advertising-based model. If it was really that easy, then everybody would be making mad cash - however, advertising based revenue is actually hard to come by.

In fact, those who can actually make it stick are few and far between (Mike Speiser shows the numbers don't lie). The amount of traffic you need to generate in order to make substantial revenue from online advertising is out of reach for the vast majority of websites.

So when is an advertising model appropriate?

• When generating traffic is the main purpose of your web service. Such sites include all flavors of traditional content sites, social networks, user-generated content sites and the like.
• When your service has the ability to deliver targeted advertising. Targeting advertising at user activity, search criteria and the like, greatly increase the relevancy of ads and thus conversion rates. A little-known company has made targeted advertising its main business to devastating effect.
• Advertising is incorporated as a part of a hybrid model, to subsidize free parts of the service (see below for more details).

The important attribute here is that advertising must target a large enough audience with sufficient relevancy in order to generate significant revenue.

Subscription is king

Subscription business models are nothing new but have been gaining popularity on the web, especially with the proliferation of on-demand online services in which subscription payments often make the most sense.

A subscription-based model is often appropriate when the service / content offered is unique and hard to find elsewhere. For site visitors to shell out even a few dollars, they need to be convinced that they are getting something they can't get elsewhere (since much of the web is available for free). This is doubly true since the process of handing out payments details online is often a barrier by itself.

Whether it's entertainment or productivity, when the value to users is higher than or equal to the monetary cost subscription becomes a very viable model. Niche sites and web services often fall in this category when they provide something of value.

I'd note here that by subscription model I don't necessarily mean just recurring annual subscriptions - there are also milestone-based subscriptions (pay-as-you-go) and feature-based subscriptions (pay-per-feature). Those three types can be mixed together to create many different hybrids, with the common denominator being the "subscribed user" - a person willing to spend money to use the service.

The freemium model

Freemium is a trendy new name for a familiar model - offer basic or limited services for free, and charge for advanced (premium) features. The freemium model is often a derivative of the subscription model, with the basic subscription plan as the free part of it.

Using a freemium model can be very useful for businesses who are counting on long-term users instead of one-time purchases. By getting limited services for free, a user can have a taste of what the web service has to offer thus lowering the barrier of entry. The balance of free to premium is important in creating the right incentive for paying for the premium services (example - basecamp).

One common application of the freemium model is combining the subscription and advertising models, by offering free services that display ads and premium services that are ad-free - this basically allows the site user to decide how he will support the service.

When we are conducting business development with clients I Personally lean mostly to subscription based freemium model, as

The brokers

Brokers are services that bring together buyers and sellers thus creating a marketplace. Revenue is generated by charging usage of the marketplace in a variety of ways:

• Sellers can be charged for being listed by the broker (one time / per listing)
• Buyers can be charged for gaining access to the listing of sellers
• Fee / commission on successful transactions

The best known example of the broker model is Ebay, a giant marketplace that succeeds on the strength of its reputation system and internal policing. A couple of good examples for creative application of the broker model are threadless and crowdspring (which I described in a previous post on web business models).

The partnership, the exit

Partnerships can be the basis of viable business model in certain circumstances - such as affiliates and sponsorships, or can be another part of the whole for others. It is often used as a hybrid with other models (such as advertising or subscription) being the driving force behind revenue.

An extreme case of this model is the outright takeover of the business by another company, often called "an exit". Though some businesses set out with this strategy in mind, its very unpredictable nature makes it very risky without an adequate backup model. I call this strategy the "no-model" model, with businesses basically saying that they'll figure out as they go along, hoping that some giant company will swoop in and end all their problems.

Picking the right one

Deciding on a web business model can be tough for early stage start-ups and traditional offline businesses transitioning to the web. Each model has its pros and cons and the fear of making a mistake can be paralyzing. Sometimes the perfect business model is very obvious but more often than not it's a journey of self discovery for the business.

When we engage in business development with our clients, we usually ask them the following questions:

• What is your target audience?
• How much would you pay for your service?
• Who are your competitors? What is their business model?
• When (or how fast) do you expect web revenue to cover operating expenses?

Those are mostly introductory questions, aimed at helping the business orient itself on what it needs to know to determine its business model.

Analyze, improve, rinse, repeat

Choosing a business model is only the beginning. Constant collection and analysis of related data - such as conversion and churn rates, usage data and statistics - is essential for tweaking and evolving the business model and increasing its chances of success. The process of testing how much customers are willing to pay for services is sometimes called bounding-box pricing.

There are no strict rules for choosing a business model, so don't be afraid to experiment and look for business opportunities in unexpected places. You never know when or where your big break will happen, but if you believe in your product and your team your business model will reveal itself eventually - be prepared for it when it happens.