Come the industrial revolution however, there was suddenly an issue of scale introduced into the equation.  Suddenly instead of a handful of customers, an enterprise might have hundreds, thousands or, in the modern world, millions and billions.  One of the first and most fundamental actions of the industrial revolution was to cut personalization out at its heart.  That’s because personalization didn’t scale.  The factory needed to get thousands of units manufactured on their assembly lines, and couldn’t stop to deal with pesky user requests like “colors-other-than-black” on their Model Ts.

Then cometh the Internet.  With its oh-so-seductive lack of physicality and supposed promise of infinite supply (and the associated supposed-zero-incremental-customization cost) there was suddenly a burst of activity around creating customized experiences for individuals on the web.  Personalization was back.  All sales leads would convert and prosper.

Wait a second, though.  Go look for some books on personalization.  You might notice something interesting, as I did when I went looking.  Nothing has really been published about personalization since about 2002 or so.  That’s kind of funny.  Wonder why that is?  Almost like…something went wrong…

Black Boxes

New technology always debuts as a black box that promises to solve all of our problems.  Generally speaking, the people signing the checks don’t care to understand how it works, only that it’s going to bequeath competitive advantage on them (in the case of early adopters) or because all of their peers are signing on to it (in the case of the majority).  Also, sadly, the role of technology has been treated in a very arms-length manner within much of the business world.  That’s too bad, because it has some of the greatest impact on business.

In the case of personalization, engagements typically went like this: you fed all of your data into it (products, articles etc) and let it perform some sort of proprietary analysis on that data.  Then you dropped some recommendation widgets onto your site, clicked your heels together three times, and suddenly you were looking at theoretically relevant content and/or personalized product recommendations.

Over time, however there have been concerns by people who have deployed such systems.  In a nutshell those concerns are:

• How do we know our recommendations are working?
• How do make sure the black box personalization system doesn’t do something weird or inappropriate?
• How can we use personalization to distinguish ourselves from our competition?

In the world of black box personalization, there haven’t really been good answers to these questions, because you’re dealing with a black box.  You don’t know how the system is really working, there aren’t any controls or visible workings that you can have mastery over and it’s difficult to gain insight on how things work, and how your business strategy could be incorporated into the personalization system’s execution.

I believe these combined factors took a lot of the sheen off of the face of personalization.  Business operators, unable to really get any idea about how these systems worked, decided they weren’t that happy about delegating such a fundamental aspect of their marketing to an automated system that provided no means of understanding its workings.  From there the only practical choice was to go back to manual curation.  And that, as we know, doesn’t scale.

This problem is, I believe, at the heart of why I’ve had trouble finding any writing about personalization that has been published within the last 5 years.  I have a hard time believing that the technology behind personalization has come to a halt, but the business application of it seems to have hit a brick wall.

Splitting the Dilemna

So faced between a choice between the scaling problems of manual curation, and the unpredictability and the unknown factor that comes with black box personalization, business hasn’t been left with a great choice.  Generally speaking, it seems like businesses have mostly come down on the side of manual curation, and have put personalization into a ghetto on their website (a recommendations sidebar or such).  Again, that’s too bad, because there’s so much being left on the table that personalization could address, but without more control and insight into the process, business are unable to trust it.

What would be nice is a mechanism for businesses to be able to define the principles, or rules, under which they want to extend personalized service, and then from those rules deploy automation to execute that service.  A kind of  ”white box” personalization, if you will.  This is where Conversational Architecture comes in.

I propose that we treat personalization a little differently than we have in the past.  Using Conversational Architectural terms, business can define how their personalization system should respond to particular cases that are important to the business, in the way the business prefers.  Because the business is essentially defining rules, however, and not individual cases, the system can scale through automation.

Tuning the Content Mix

Conversational Architecture thinks of product recommendations, or related or personalized content, as all part of the same problem.  We call these “content policy” objects, and treat them as components that can be included in various contexts.  A content policy object is a placeholder for a set of business rules that describes how a component should be populated with content (content meaning articles, products, profiles or whatever).  The point, however, is that the mix of content within these policy objects can be tuned by the website operator.

Here’s what kind of content you can get in a content policy object:

• Personalized:  Based on the user’s expressed affinities, personalized content tends to be related (via metadata) to content that the user has endorsed in some way.  For example, if we’re talking about products, then personalized products would be related, (via metadata, tagging or collaborative filtering style aggregation), to products in which the user has expressed an interest in some way, such as by buying them, adding them to a wishlist or so forth.
• Related:  Related content is similar to personalized content, but instead of user affinities, the starting point is something on the page that serves as a base content.  In the case of e-commerce, the starting point could be the product featured on a product page, or perhaps a particular metadata or tag value represented by a product category page.
• Curated:  Sometimes you just want some specific content to show up in a particular place.  This is curation, meaning the individual content objects are specified directly.
• Random: With all this content targeting, it’s occasionally nice to dial in a little bit of noise.  Random content could be anything, and allows the policy editor to back off a little from complete content targeting, allowing for the possibility of serendipitous discovery by users.  You never know.

How Content is Related

Another important idea with regards to personalized and related content is the idea of vectors of relevancy.   This has to do with the specific metadata that is used to relate content to each other, and the relative importance of similar values within that metadata.  Not all characteristics of content are equally important when tuning a content policy.

For example when we built Whiskey Engine (whiskey-engine.com), we consulted with an expert who explained to us the elements of the taste of whiskey.  He broke the taste of whiskey into elements like “herbs/spice”, “flowers” and “smoothness”.  Based on his tastings, we accumulated data about the taste profiles of each whiskey in his collection, with a value between 1 and 5 (1 meaning the least, 5 the most) for each taste characteristic.

However when we spoke with him further, it turned out that people tended to place a premium on the “smoothness” factor above all else.  In other words, once people decide how smooth they like their whiskey, they tend to follow that vector when looking for new whiskies.  People place far less concern on other whiskey taste values such as “herbs/spice”.

For a different example, think about music: if you like the artist who recorded one track, there are reasonably good odds that you’ll like another track by the same artist.  On the other hand, the record company that release the recording is much less significant.  People usually don’t care (and don’t know) which record company released a particular recording.

Buy carefully defining the metadata for content as well as specifying its importance as a vector of relevancy, recommendations and related content can be improved within policy objects.

Gaining Confidence: Measuring Content Policy Performance

So how can you know if your personalization strategy is working?  By treating the content policy object as the head end of a conversion funnel, and measuring how well it performs (we call this “Confidence“).  At its simplest, you should simply be able to see how well each content policy object is performing.  If you’re not satisfied with the results, then go tweak your content policy mix, or try altering the relevancy vectors of the content metadata to increase the quality of the personalized and related content in the policy.  Specific personalization deployments should never be treated as “set and forget”.  You need to work on them, adapting them to your specific audience.

Of course the ultimate version of responding to confidence measurements would be for the content policy object to reconfigure dynamically based on performance.  Conversational architecture has a name for this too – we call it “conductivity“.  Essentially we measure the performance of recommendations and weight the effectiveness of the current vectors of relevancy.  Over time we’ll start to adjust the vectors (“oh look, it turns out that herbs/spice is a bigger deal than we thought”) to increase the performance of the content policy objects.

What about Modality?

In the original conversational architecture article I talked a fair amount about modal websites.  How does modality apply to personalization then?  I think that first one should realize that while you don’t have to use modality to build a personalized website using conversational architecture concepts, modality can provide you with a way of describe some special features that could take your personalized website over the top.

The basic guideline is: when moving away from ideas that can be expressed easily in terms of content policy objects – modality can be very helpful.  For example, imagine an e-commerce website that provided discounts for people who frequently shared products from the site through social networking channels like Facebook and Twitter.  In Conversational Architecture terms, you could identify this kind of person as a Persona (call them “social sharer” or something) and trigger a mode from that persona that caused a discount to appear in their shopping cart.  This is an easily expressed business rule that could then be implemented through automation on the site.

Wrap Up

So we introduced five new concepts that Conversational Architecture provides that allow businesses to more clearly articulate what they want from personalization: content policy objects that provide customized mixes of content, products etc., vectors of relevancy that determine the relative importance of content metadata when calculating recommendations, confidence which expresses the performance of content policy objects in terms of conversion, conductivity which expresses the evolution of vectors of relevancy over time, and finally how modality can be applied to a personalization problem.

This is obviously a lot of new ideas, however it’s entirely possible to just try them out one at a time and look for incremental improvement in your personalization implementation.  By using these concepts, you have acquired a language in which to discuss personalization, which will allow businesses to more closely align personalization with their specific approaches to operating their business.

On one hand, the hackers/programmers were acutely aware they were sitting on top of a bunch of potentially useful technology for the publishing world. Semantic web, collective intelligence, mobile apps, location-aware, social network integration and so forth. There are now many, many tools in the swiss army knife of technology with which one could build a new-generation media empire.

On the other side of the fence, we found the business people we encountered quite open and interested in the new technologies. Everywhere we’ve been, we’ve encountered attitudes ranging from “oh yeah, I’ve heard of that – I should really learn more about that” to “yes it’s awesome. I’m really interested in getting on that train!”.

What we have not found, however, is a bridge between these two worlds.  When we’ve started to speak with user experience, visual design and product development people, they seem at a loss as how to incorporate these technologies into their products and leverage them to their advantage.  There is no common design language or methodology for making sense of the whole collective intelligence world in a comprehensive manner.

Before we go any further though, let’s go camping.

The Idea

Let’s pretend for a moment we have a clothing retailer whom we’ll call Rugged Clothes.  They want a complete coordinated digital marketing strategy.  Here’s what it might look like:

The Smith family goes camping.  During the camping trip, the kids wear some new clothing their parents have bought from Rugged.  The parents take pictures of their kids with the Rugged iPhone app, uploading the pictures.

When they get home, the mother opens up the Rugged app on her iPad.  Because they’ve recently uploaded the photos, the app automatically opens up in photo editing mode.  Mom goes through the photos, picking out the best ones, adding comments to them.  Then she elects to “publish” them.

The photos are published to the Smith family page on Facebook via the Rugged Facebook app.  Friends and family can access them.  The photos are automatically tagged with the articles of clothing that appear in them.  Clicking on them will take users to a special category page on the Rugged website based on the Smith family.  From their the individual pieces of clothing can be purchased.

This story tracks the experience of users across four different media (iPhone, iPad, Facebook and the website) but describes a single, coherent experience that is aware of the current state of it’s users.   This is the kind of story that catches the attention of the more visionary business people these days.  It’s the promise of the collective intelligence technologies, and of the integration of mobile, social and web services.

The Page: A Bootstrapping Metaphor

We’re in a period of time with collective intelligence technologies analogous to the early days of film, a century ago.  When motion picture technology first came on the scene, people simply leveraged the methodology of a previous medium, theater, and filmed it.  It was several years before they started to realize they had a completely separate medium on their hand, and started to experiment with film editing (montage) and moving the camera during a shot (tracking and panning).

Similarly, in the web world, we’ve had the page.  The concept of the web page swiftly became an incredibly convenient metaphor for designers in the early days of the commercial web.  It allowed people who had a background in print design to make the jump into web, because they already knew how to lay out a page.  (Put aside the endless problems created by designers who assume web pages work like print, it was actually a net advantage: it bootstrapped web design).  By framing design decisions in the metaphor of a page, and a website in the terms of a “collection of pages”, we had the foundation to structure the question of how to build in this new medium.

Unfortunately, all that is breaking down now.  The page metaphor is becoming increasingly strained and less relevant in our modern world.  Consider the following:

• Mobile applications on multiple platforms
• Highly dynamic AJAX/DHTML/HTML5-style websites
• Social Networking Platforms
• Location aware services
• Collective Intelligence / Semantic Web technology

None of this stuff has much to do with pages.  But without a design language or metaphor to fall back on, a chasm emerges between business people who can see the potential of these technologies and are willing to fund the right projects, and the technology folks who stand ready to build this stuff if only someone could let them know what, exactly, they should be building.  There is no way to capture this stuff simply by discussing “pages”.   It’s time to put that wireframe down, and step back.

Brave New World

Let’s refrain, for a moment, from discussing the specifics of any individual new medium (web, mobile, social etc) for a minute and try and consider the big picture organizational communications.  There are three basic characteristics to which we should aspire in our communications strategy:

• Multi-Channel: As is largely conventional wisdom now, just having a web page, or just having a Twitter account etc., is usually not sufficient.  Different media have different mechanics and areas in which they are effective, and the best approach would be a comprehensive communications strategy that takes advantage of the strengths of each platform and leverages them in such a way that makes the most sense for the organization.
• Multi-Modal: This is an important concept: people’s interactions with organizations are modal.  Often they are driven by some purpose, specific or not, held by the individual.  One of the biggest design challenges on the Internet has been to try and present what are essentially modeless designs (e.g. “good for everybody, all the time”) that are actually used in very specific, modal ways.  A great communications approach would be multi-modal, rather than a “one size fits all” mode.
• Multi-Directional: Communications is really a two-way street.  While it’s one thing to have a touchy-feely marketing message in which you claim to listen to your customers, actually implementing it in a quality way at scale is extremely challenging.  Nonetheless, an organization that can actually respond to feedback and requests from individuals is at an incredible advantage.

So what metaphor can we use to pull together these qualities?  What exists in the natural world that’s a good fit for this?

Enter The Conversation

Conversations have obviously been around forever, which conveniently means that most people have something of an intuitive grasp of what they’re about.  Looking  at our criteria above, we can see that this metaphor maps nicely onto what we’re trying to accomplish.  Conversations can traverse various media (multi-channel) can shift modes depending on various action of the partipants (multi-modal) and involve two or more parties both listening and speaking (multi-directional).

Conversations should be used as the foundational design metaphor, at the point after when the initial concept for online communications has been proposed, but before specific user interfaces are designed.

This let’s you know what to build.  By modeling the conversation (or conversations) your organization is having with the outside world, you will be able to shape your online communications strategy in a way that is targeted towards specific audiences, over specific channels.  More targeted communication means less noise, and the more fruitful the conversation will be.

Conversation Design: How to Do It

So how exactly does one design a conversation?  Let’s break it down into steps:

Conversation Mapping

The first step is conversation mapping, or essentially to determine what conversations exist between participants.  This is a high level, strategic activity, that creates some shape to the universe into which specific design thought can be injected.   The first step is to identify the participants.  Here’s an example from my company, Saaspire:

Saaspire itself sits in the middle as the reactor.  This isn’t meant to indicate that it’s passive in any sense, but because it’s the participant that we actually control, any automation we build will live there, so the term “reactor” is accurate as far as describing the process.  All around are the main constituencies with which Saaspire communicates: customers, investors, developers and press.  Those are the “actors”.

Now we need to establish the conversations we’re having with each participant.

Basically what we’re saying is that Saaspire is having four different conversations with different participants.  For customers our communications tend to be about product support and education – similar to this is the developer conversation in which our communications are more technical and platform oriented, but again about documentation and support.

To investors we speak about the value and potential of the company itself, and finally to the press we try and make an attractive “next big thing” story.

Identify User Contexts

So once the existence of a conversation is established, how do we gain some insight on how it functions?  First we need to look at the driving forces of conversation modality, which I call context.  Context is an aggregation of the various factors about a participant that, in combination, drives the conversation from one mode to another.

Context consists of four factors:

• Personas are behavioral aggregates of participants.  They represent any long-lived group of user behavior that’s worth addressing en masse.  Traditionally in web design, personas have been expressed in demographic and (high level) motivational terms (“Cindy is 36 and wants to get things done fast.”).  While we’re less concerned with the demographic aspect of personas in this case, it can still useful to think of them in motivational terms: How do they think?  What do they want?
• Affinity represents the stuff that people like.  It might refer to content or advertising on a media site, or it might instead refer to products on an e-commerce site.
• A state or goal represents a temporary condition in which a participant exists.  For example, a user that has started a checkout process on an e-commerce site could be said to be in a specific state that will conclude with finalizing checkout.  The main difference between states and personas is the temporary nature of states.
• Environment is a catch-all meaning the circumstances under which the conversation takes place.  In digital terms, it tends to refer to the browser, the operating system and the form factor of the device used for access, but can also be broadened to include concepts like location.

This leads up to a very important concept in conversation design:  contexts trigger modes.

Mode Mapping

Once you’ve identified the possible personas, affinities, states/goals and environments that you’re going to support, the next step in conversation design is to determine your response.  This is done by having specific combinations, or contexts, trigger modes.  Modes are your response to that context.  For example, within our customer support/education conversation, we might identify the following mode:

In this particular case, we are looking at participants that we’ve tagged in two specific ways: first, they are considered part of the “qualified customer” persona.  This might have been established in any number of ways, such as requiring them to log in or otherwise establishing that they hold at least once license for one of our products, or it may just be some much softer form of self-identification on their part.  Secondly, they have exhibited behavior (perhaps a search on our site, or in inbound link from a specific Google search) that let’s us know that they have the immediate goal of seeking information.

Given these two criteria (and we don’t care about their affinity or environment in this case) we trigger “customer support mode”.  Within customer support mode, we might provide facilities on our website that are slightly (or substantially) different from what other users might see.

So What’s A Mode, Exactly?

A mode is a building block for your web service.  They are the states that your web service passes through for individual users as they are triggered by those users’ contexts.  In different modes, your web service might contain different functional components, variations in user interface, different content and so forth.  The question of what to vary between modes will be one of the foundational skills in this approach to design.

For example, you could use different modes to optimize for the way the user likes to interact with the site.  Perhaps certain users tend to search for information within your website whereas others are more browsers (they use the navigation system).  The site could adjust modes to emphasize the elements of the UI (search vs navigation) that most suit those users.

Modal design could emphasize the state in which a user exists at a given point.  For example, consider the “offline buying decision” in e-commerce.  A visitor goes to an e-commerce website and browses around, looking at various wares.  Then he leaves the site, and while he’s away from the site, decides to purchases something he’s seen.  At that point he goes back to the website and immediately purchases the item.

Most websites don’t know what to do with this behavior.  They see the first session as a conversion failure, and the second session as a success without any explanation.  But a modal site would recognize this as a state shift for the same profile, and the site could optimize for the appropriate state (browsing vs buying).

If the site suspected it had an offline purchase decision maker (a Persona), it could switch between “browse mode” and “buy mode”, based on the inferred State/Goal of the individual user.  In browse mode it would be always showing more options to the user, up-selling, suggesting more items and generally just extending the engagement between the user and the site.

If the person came back to the site, and immediately added the item to their shopping cart (something they had looked at before) the site would switch modes, into “buy” mode.  In this mode, the objective of the site is to let the customer check out as fast as possible.  No more distractions, no upsell, no additional options.  The user is now in buy mode – just let them buy.

More to Come

So that’s the quick introduction to Conversational Architecture.  I’ll be drilling down into more of this in future posts.  Please let me know what you think – I’d love to discuss this.

Having spend the last 4 years or so working in web agencies, I’ve seen the inner workings of the IT department in many, many companies – from mom and pop operations to Fortune 500 outfits. More often than not, the IT department is really, really broken.

First, let’s have a look at what broken means:

Symptoms

Lack of Prioritization

Faced with conflicting demands from different business units, the IT group is unable to plan where to spend its energy, and so lives constantly in a reactive state, servicing whatever the squeakiest wheel happens to be at any given moment. Essentially it’s locked into a state of putting out fires, and is unable to put energy into actually making things better.

Often this can be a result of lack of “informed power” regarding the IT department: senior management doesn’t really have the understanding to view IT as an allocatable asset, and no one in IT has the power to push back on the immediacy of the next business stakeholder’s fire – regardless of whether it’s important to the business or not.

Lack of Bandwidth

Related to the prioritization issue, but slightly different, is the fact that most IT organizations simply don’t have enough people to do the volume of IT work demanded by the organization. At the same time, however, there is resistance to hiring in IT, because IT is viewed as a pure cost center in most organizations. In other words, it is not viewed as a vehicle through which the organization can make money, but rather just a place where spending occurs. In an environment of cutbacks, it will be trimmed.

Communication Problems

Often business stakeholders and IT just can’t understand what the hell each other is saying. Business users generally have a sense that certain features are vastly more important than others, but don’t always communicate this prioritization if they’re not asked. Additionally they may jump immediately to a specific feature implementation (that may be taken as a requirement by IT) that isn’t really what is best for them. Business stakeholders are not going to have any sense of feasibility, however: they don’t know how hard it’s going to be to do things.

IT comes from the opposite end. They understand feasibility, but not business importance. Unfortunately IT has the habit of taking all “requirements” from business as of equal importance, and just plugging away, looking at things purely from a feasibility perspective. When combined with the business stakeholder’s habit of proscribing features instead of just stating needs, it can lead to a lot of wasted effort.

Fortress Mentality

When things go wrong with technology, who get’s blamed? IT. When things go right, who get’s credit? No one.

In a sane business decision, one weighs the upside against the risk. There are many cases where it’s worth it to take a risk in order to achieve a big payoff. But IT is essentially balancing risk against an upside of zero. Naturally, that means they will seek risk avoidance in every case. This minimizes the chances of things going wrong, but it also minimizes benefits that might be had from technology.

Consequences

These symptoms turn an organization’s interactions with their own IT group into a big, dysfunctional dance. Business stakeholders feel that IT is unresponsive, slow to complete projects, incomprehensible and too restrictive. They start to come to the blanket judgment that IT is not going to be a solution for their problems.

IT for their part, feels under-appreciated, constantly putting out fires, under-staffed and forever dealing with clueless users. They have ideas for how to make things better, but never have time to implement them. On the projects that they do get to work on, they’re shocked to find out that features they’ve worked so hard to implement are being unused by business stakeholders.

This dysfunction has even led to a culture of circumventing IT. Business stakeholders (as much as they might like to) realize that they simply cannot live without having certain technology services performed, but feel that IT will be unable to help them. At this point they may look for outside help – either from product vendors that claim to “do it all”, to agencies that usually make the same claim.

The product vendors or agencies may be just fine, but that doesn’t change the fact that the organization has chosen to outsource a part of its business, not on the basis that it isn’t core to their enterprise, but rather on the basis that they are incompetent to execute. Since IT work is often the automation of a core business process, outsourcing it can lead to a weaker overall organization.

So there’s a picture that’s probably familiar to many. How do we fix it though?

How to Fix IT

Fix the Accounting

If technology has benefit to an organization, then that benefit needs to be reflected in the way the organization accounts for itself. From a financial perspective, a business will look at the profitability of its various components (at various levels of formality, depending on the size of the business) and make decisions about where to invest.

Businesses need to stop looking at IT as pure cost centers. That means that the credit for adding value to business units needs to find it’s way back to IT somehow, and be measured against IT costs. Here’s two possible ways to do that:

Charge the Business Users

If business stakeholders use IT services, they have to pay for them. That way the “cost of IT” lands squarely on the business stakeholders that use IT services. Of course, the side effect of this is that the quality of what the business users are paying for will immediately fall under scrutiny, but that’s okay. The point is to make the organization stronger.

Credit IT with Results

Let’s say IT builds a system for a business group, and as a result there is a reduction of cost for that business group (or even better, a measurable increase in revenue). Some of that savings/earnings increase should again be credited back to the IT department, and shown as income (or something equivalent). Again the political side of this action will be determining how much credit to kick back to IT. Too much would additionally have the negative side effect of making the business groups less interested in such activities, as they aren’t being credited with the results.

Hire Technology Leadership

Particularly aimed at the communication / prioritization issues, the solution is to hire a senior person with a technology background who:

Can Translate: Someone who can speak the feasibility-heavy language of developers, and the priority-heavy language of business stakeholders. Someone who knows when to ask context-breaking questions, such as identifying the real business issues instead of just accepting prematurely prescribed solutions dictated by business stakeholders.

Can Design Solutions: Someone who can take the real business requirements and come up with a technology-based solution, then take that solution and articulate it as technical requirements for the IT Staff. This is an individual who understands both technology and business.

Has Authority: This person needs to be able to push back. It is inevitable that forces within the organization will attempt to mis-use IT, either pre-selecting the wrong solution, or by short-circuiting the priorities of the IT group. The Tech Lead has to be able to say no to those forces, and find a real solution that balances the real needs of the business stakeholders within the larger context of the organization.

Decompose IT

The most radical step is to send IT staff out into the wilds of business stakeholders. Instead of a common pool of IT resources, an organization could choose to send, for example, a technical PM, a couple of developers and a sysadmin out to exclusively work on the problems of a particular business group.

This is an approach that probably makes the most sense when the needs of a business group grow beyond a certain point. At that point using centralized IT resources becomes less effective than maintaining their own technical staff. The obvious concern is that efficiencies of scale are lost when technical resources are distributed amongst business groups, but this is a good choice when the effectiveness of those resources within the individual business groups makes up for any loss from decentralization.

The upside to decentralization is that the accounting problems just go away. The efforts and results of these distributed staff are just measured within the individual business group, who will either be able to justify the expense or not, on the same kind of merits that they use for everything else.

It is important in these cases that technology leadership/management is included in the distribution, or the business units can find themselves saddled with a lot of the communication issues mentioned above. Because of this, this is a significant and potentially expensive step for a business group.

Conclusion

IT in most organizations is broken – it’s not providing the kind of support and advantages to business users that it could, and is the source of frustration and feelings of helplessness for many employees, both business and tech. By re-structuring the way IT fits into organizations, it can deliver on it promise at last.

Jerry: Oh… You mean… shrinkage.

George: Yes. Significant shrinkage!

So how do you enter a mature, well understood market?  The naive answer is: “well if we can just get 3% of this multi-billion dollar market then we’ll be doing okay”.  In practice, that doesn’t work – the market will eat you.  The rules of an established market are always stacked against newcomers, and in favor of the incumbents.

Well, how about destroying the market?  Shrinking it until the market leaders die?

Sounds harsh.  But this can be one of the more effective tools for a start-up trying to break in.  Instead of trying to carve out a space by following the existing rules of the marketplace, the start-up just lets all the air out of the balloon.  The incumbent giants asphyxiate from lack of oxygen (meaning money) and the startup is the only one left standing.  (Did I get enough imagery in there?)

Classic example: Craigslist.  Apparently there once was this profitable part of the newspaper business called “classifieds”.  Ostensibly, (and I know this may be hard to believe), people paid actual money to place their posts, er, ads, into the newspaper for other people to see.

Craig Newmark made short work of that.  Craigslist monetizes only a tiny portion of their posting traffic – recruitment ads primarily – posted by companies, not individuals.  Everything else is free.  It’s hard to compete with free (as the music industry found out), so most of the air has gone out of the classified balloon.  Craigslist snatched up the tiny amount of air that was left.  By shrinking the market, Craigslist successfully competed against the big guys.

If you think about it, it’s a brilliant application of lateral thinking to the business problem.  Problem: the market is too big for me to compete.  Solution: shrink the market down until it’s a size I can handle.

The Web 1.0 shrinkage story was Ebay, who let the wind out of every auction house in the world.  More recently there’s stubhub, who’s wiping out the lucrative event ticket aftermarket (also known as scalpers).

The ingredients of effective market shrinkage seem to be:

1. Identify a cash-rich market built on inefficiencies – artificial scarcity, a massive imbalance of power etc.
2. Enter the market, replacing the basic business transaction.
3. Give 90%+ of the service away for free.
4. Watch the big guys asphyxiate.
5. Build your business on the remaining 10%- left over.

Presto – a market where you’re now the leader, at a managable size for you.  Mind the pile of bodies.

The idea of maintaining a single technology stack in a development shop is quite appealing.  If everyone knows the same programming language (the thinking goes), the same web framework, the same RDBMS and so forth, then life would be better.  Business really likes this kind of thinking.  It promises efficiencies of scale, reduced training costs, and easily replaceable employees.  This kind of “one true technology” approach is the foundational philosophy of corporate darlings Java and .NET.    Once you have the organization standardized on the tech, it is promised, then every part will play nicely with every other part, critical dependencies (such as employees) will be reduced, and the whole “what are we doing about technology” question just goes away.

Now, of course I support operational efficiencies, reducing training costs (well, to an extent: I view training as an investment), and so forth, but I think that the idea of “one true stack”, especially in service-oriented tech shops like web agencies, is a pipe dream.  The advantages of standardizing an entire company on a single tech stack are outweighed by the disadvantages.

Before I get into that, however, I want to introduce the idea of a “jealous” technology.  A jealous technology tries to take over as many aspects of the stack as possible; mandating a specific component at each level.  Some technologies are more “jealous” than others.  Microsoft is notoriously jealous:  the Microsoft web framework, ASP.NET, really wants to be deployed on the Microsoft web server: IIS, on the Microsoft operating system: Windows, backed by the Microsoft RDBMS: SQL Server.  Other combinations (yes, I know about Mono) simply don’t work as well.  And, on the other hand, Windows is something of a second-class deployment platform for most other web technologies (ever try to deploy Rails on Windows…with IIS?  Yuck…)

Speaking of Ruby on Rails, they’ve become a bit of a jealous technology themselves.  Rails projects now prefer a particular JavaScript framework (Prototype), version control system (Git), editor (TextMate), build system (Capistrano) and development operating system (OS X).  (As well as, obviously, programming language: Ruby). While it’s certainly not as bad as the Microsoft scenario,  I do have to ask the question as to why the web framework has any influence over what version control system I use.  Do you have to use all the technologies that I’ve listed above with Rails?  No, of course not.  But they are preferred.  Things go better with Coke Capistrano.

The point is that some technologies are better suited towards certain problems than others, and a blanket organizational standardization on a single tech stack leads to a tremendous amount of wasted energy spent on driving square pegs into round holes.  Ruby is not a great foundation for an enterprise-class messaging framework.  Java is not a good choice for a lightweight CMS.  And anything other than .NET is a bad choice for an app destined to be maintained by a bunch of MSDN types.

In fact, I’d argue that the cost of operation efficiencies lost by maintaining multiple tech stacks is exceeded by the cost of jamming technologies into inappropriate use cases.  So, as messy as it sounds, any technical organization that is project or service based (as opposed to being exclusively product based) should maintain a strategic “quiver” of different technologies, so it can deploy the best choice when faced with a business problem.  Otherwise a fragile monoculture is created, wasting a tremendous amount of developer effort, and adding artificial risk and expense to projects.