Danilo Gurovich

The Role of Technology in eCommerce
ECommerce enterprises are not “technology” or “software” companies. While more modern technologies can and do offer massive benefits to them, the business as a whole does not exist for any specific technology, good or mediocre, as long as it can further its mission and goals.
The technology departments in these organizations exist to implement the business requirements of various business units, resulting in revenues to perpetuate and grow the business further. This argument can become quite heated depending upon with whom the conversational partner is (especially engineers) and their group affiliation within these businesses. Let’s assume the following statements:

• The eCommerce enterprise as a whole exists to get people into their “store” to buy something.
• If this were a traditional “brick and mortar” company, the technology division would be the building’s contractors, janitors, errand runners and parking attendants.
• The enterprise doesn’t really care about their code base from a business standpoint, as long as it is “running as usual.”
• Improving on something that is working diverts resources from doing things that make money.
• Engineers in nearly all eCommerce enterprises think their code base sucks.

“The eCommerce enterprise exists to entice users into their “store” to buy something.”
The eCommerce enterprise’s “store” is an application. Most, but not all of the applications involve websites, but other applications as well may generate an exchange of money for goods and services. There are “border conditions” within the scope of this model that may include sales over telephone or even storefronts, but for the purposes of this treatise one will assume that an eCommerce Enterprise is one that “depends upon some type of internet connection to generate all, or nearly all, of its revenue.”
The object of eCommerce is to make money by getting people to “press the submit button” and generate some type of revenue generating transaction. The various channels within the store have responsibilities to find new ways of making money from the existing user base – increasing conversion, advertising revenue, secondary transactions, etc.
Anything that affects revenue generation and growth in a dispositive manner is a proportional threat to the business. Assuming that all changes to a system are correspondingly risky based on their complexity, weight and frequency, it is necessary for any change to have a revenue impact that maps to this profit/threat matrix.
“Change for the better” is defined as any change that furthers the goals or revenue for the company in proportion to the risk involved. Any change falling outside of this envelope cannot be tolerated – all changes must have a clear and well-defined business case for a company to survive. Change for the better can occur with new, tested, functional changes, and also cleaner, efficiency-related changes on the “server-side” that will deliver more resources away from channels like maintenance and QE. The best-of-breed applications are aware of these and are able to accurately predict short-and-long-term revenue impact before anything moves away from the planning stages.
Change in eCommerce is necessary due to outside and internal pressure. As new competition arises, the established eCommerce operation must rise to meet this competition. As resources supporting legacy technologies diminish, one must address how to migrate to more modern, maintainable and mutable platforms.
Is there room for the slickest new thing? Is there room for cool new techie, “under the covers” stuff? Is there room for stuff that really has no direct revenue attachment? What more conservative changes are smarter? All changes are a gamble. This gamble has many variables:

• Opportunity Cost — Is the change being made at the risk of something else that could be done to increase the position of the risk/reward value? Is this the most profitable move possible with the resources available?
• Efficient Competition – Is your competition being smarter? Is this change being made to improve something that, while it is being done, the competition is building and extending applications that will take customers away or put them far into the lead?
• Enterprise Requirements – Is this change to increase some performance metric of an application that already meets the needs of the business?
• Dependency – Is this change dependent upon other deltas? Does it require future development that may or may not have revenue impact?
• Political – Does this change have no sponsorship in one or more of the common stakeholders for the channel? Do some stakeholders resent it? Does everyone have “skin in the game?”

Gambling is a fast way to the poor house if there is no real reward attached to it. Expert Technologists that specialize in eCommerce cannot and will not do anything that has a negative effect on revenue. Often this means with living in a code base that ranges from “less than optimal” to “downright scary”. The amazing thing is, if it is a going, successful concern, this scary code base has somehow survived and made quite a bit of money. This is because it works well enough, is extendable enough, and maintainable enough for the business to reach its goals.
The bad news it makes Engineers sick to their stomachs at best, and often makes them dread coming to work in the morning wondering what changes will trigger some insanely weird output in their IDEs, or worse, a customer-facing disaster.
There is good news here, too; evil, weird and application-crashing changes to this mess of code rarely if ever make it to production. Why do we know this? Because the business, code, staff and processes to create it are still there. Proof of this exists in the fact that the business is growing and flourishing at some level. Big disasters can cause serious and often unrecoverable revenue hits. Any internal or external changes to this stasis that cause enough problems will trigger a situation where too much risk will exist for this code base to continue in its current state. This is the crux of the problem – understanding the right time to make changes, and defining just how these changes should take place.
We will explore ways of ameliorating this problem after we set all of the elements involved on the stage in the following sections.

Technology is no more or less important than any other business stakeholder.

Technology doesn’t drive the eCommerce Enterprise no more than any other department. While they might actually create the application environment for the business to achieve it’s goals, Tremendous amounts of Business Development, Analysis, Marketing, UI Design, Project and Product Management must be involved to truly find success.
Technology is the conduit for the execution of the combined efforts of an eCommerce Enterprise, and often this particular gate in the general process can use its weight to implement changes in the code base that are not necessarily in-line with the Corporate Goals. These changes can have drastic revenue impact.
If one walks a mile in the shoes of the average Engineer in a group, a completely different opinion would emerge!

• If it weren’t for us, all these “marketing and business types” would be surfing Monster for their next gig (of course it might not occur to think that this would go the other way….).
• We hate our code base more than Osama bin Laden, and think that it needs to be overhauled from end-to-end and brought up-to-date because we are literally MINUTES away from crashing.
• Business doesn’t have its priorities straight. We need better requirements and fewer changes mid-way through a project.
• We’re working our butts off here and nobody gives us any credit.
• We’re holding this ship together, etc. etc.

Great Engineering Staff sticks to the fundamentals and gets off the soapbox. They know that coding to a specific implementation is bad. Loosely-coupled code is the way to go. They’ve learned over time that most of the latest technologies are drastic improvements over old legacy stuff, but implementation can be difficult and risky, especially as an “early adopter”. Finally, many of Engineers are used to long development cycles that are isolated, coordinated and last for months using a (mostly) waterfall approach.
There are literally thousands of books, articles, white papers, and other new media showing “best practices” and “industry standard” protocols, formats, processes and anything else that involves getting an idea into deployed bytes of code. The one thing for sure is, they work for someone very well, but the more specific they become the less they will work for a broad spectrum of teams across a matrix of projects and channels. Problems around process and projects of this nature generally point to Engineering Management issues.
Processes and Projects in an eCommerce environment are quite fluid and often hectic. There are constant maintenance deployments (that’s when code is branched, stabilized, QE’d and deployed, often weekly and even more frequently). Business may have several projects running concurrently that will have different delivery periods with dependencies on existing code. This makes dramatic, “fork-lifted” changes to an application nearly impossible (i.e. “Yeah? Well, YOU go tell business that they can’t do anything for three months.”). Re-architecture vs. refactoring as a question surfaces constantly, with an obvious answer on nearly every occasion.

“The enterprise doesn’t really care about their code base from a business standpoint, as long as it is “running as usual.”

They just don’t. If something is working well enough to meet or exceed the goals of the business, nobody outside of engineering cares what’s under the hood. How often has an engineer been faced with the “GUI doesn’t work right” question, when there’s something deep down in the code that actually is causing the problem? Business should have to care about what’s under the hood as long as it’s working. Business only cares that what they want gets done and the goals of the business are met.
Assuming that the engineering staff is a competent, capable and hard-working group, the code that emanates from this group will meet or exceed industry standards and “work” properly for long periods of time. Business cares about the competency of it’s entire staff; they expect that the underlying work is accurate and working properly to support business goals.
Engineering must line up behind this. Fix what can be fixed, but don’t let it get in the way of making money. “Perfect Code” belongs in Publishing, School Projects, Personal ideas and Open Source Committees. It often is missing in these genres, why would one expect it to exist in an operation with tremendous pressure, deadlines and years of legacy development?

“Improving on something that is working diverts resources from doing things that make money.”

“That man will fight us every day and every hour till the end of the war.
— Lt. General James Longstreet speaking of Lt. General Ulysses S. Grant.

 

The point of the above quote is that pressure in the form of fierce competition will be kept upon any Enterprise offering services and goods, especially if success is achieved. Anytime that this pressure is diverted, one must instantly assume that the competition is either gaining ground, or worse, stretching out a lead. The resources of a company are exactly like the soldiers in an army. They are limited in number and specialty, and they must be brought to bear to inflict the heaviest force in the places where they will be most effective and efficient.
Diversion from the goals of a business towards re-architecture projects is in almost all cases folly. A complex, highly fungible eCommerce application cannot be both maintained and extended in one area and simultaneously re-architected for later deployment. There is absolutely no way to manage change, improvements, maintenance and QE of an existing application while ensuring that the new application will meet functionality requirements that are only determined at time of deployment. The cracks that things will fall through can be chasms, and recovery from this may cost more than the original project.

“All Engineers probably think their enterprise code base sucks.”

/**
* Always returns true. Please excuse multiple returns. That sucks too.
*/
public boolean isCodeSucky(sucks){
if (sucks){
return sucks;
}else{
return !sucks;
}

Engineers are problem solvers. Most are just wired that way. They want the best for their projects and are often if not always forced to make compromises that constraint within the project dictates. In the ServerSide Article, “Does your code suck?” They state a few cardinal gates that your code must pass to be considered professional:

• Your code sucks if it doesn’t work.
• Your code sucks if it isn’t testable.
• Your code sucks if it’s hard to read.
• Your code sucks if it’s not understandable.
• Your code sucks if it has duplication.

Source: ServerSide.com :: Humor :: Why your code sucks.

Author’s note: “I will not be the one to cast the first stone here. Looking through the code bases of the last eCommerce and software companies with and without one’s contributions, I would venture there would be nobody on the mound for the first pitch. It is possible to do your best work in a professional manner, but could it possibly exceed the business requirements of the application being built?”

The “Evolving” Code Base

So what if an application has JSP 1.1, Java 1.3 and runs on Tomcat 4.x? What if it only has twenty percent Unit Test coverage? Woefully incomplete documentation?

• Is it stable?
• Does it perform in a reasonable manner?
• Are all of the business requirements met in a reasonable time frame?

Chances are that the basic kernel of any ongoing eCommerce application is quite old. It’s implemented with proven technologies that allow changes to be made in a reasonable time frame with reasonable resource cost. It’s stable enough to generate the money needed to keep the lights on and the salaries paid.
It’s also a reasonable assumption that the entire bank of developers, specialists and technical staff are able to develop, maintain and scale the application within reasonable expectations. If all of these factors are in place, an “evolved” or ‘Darwinian” code base exists.
“Darwinian” code bases are usually implementation-specific and fairly tightly coupled. Time and other constraints have manifested themselves into fixes, patches and other compromises. These compromises have created the need for other “fixes”, creating a “Big Ball of Mud” that is about as portable as the Eiffel Tower. The thing is, it works, and nobody in the engineering-side of an organization can fully explain why.
Types of evolving code

Just as life on Earth evolves, so does any code. Evolution is driven by need, in the case of life, these needs could be competition from within or without the species, gradual and inevitable change of the environment, and/or possible stronger “shocks” to the system ranging from a few hard drought years to a well-placed comet-strike.
Code bases too, must evolve. From the new release of the OS Kernel that the systems run on, to the ever-changing needs of the consumer and tastes, slow, gradual change is inevitable. Shocks to the system are also part of life – massive system failures, changes in platform requirements and radical new technologies implemented by the competition are all major shocks.
The minor changes described above may be described as incremental. These changes may be “planned” for and “expected”. While the stronger “shocks” are also expected from time-to-time, they are rarely welcome and can be costly and disruptive. These major, “forced” evolution can best be described as adaptive.
Incremental Evolution

Incremental Evolution is just that. Changing the code base of an application through small changes driven by business requirements, maintenance issues and small changes to the languages used. These changes are usually expected; they have incremental revenue impact, risk or time-to-market.
Adaptive Evolution

Code bases undergoing Adaptive Evolutionary changes are the result of shocks to the system. While some of these shocks may be due to disasters within the environment, most major shocks are borne out of externally driven issues – Denial of Service Attacks, Introduction of new frameworks and more efficient/fierce competition for customers are a few examples. These changes are expected but not anticipated; they have high revenue impact, risk and often can take quite a long time to get to market.

How does a “Darwinian Code Base” handle this?

Darwinian Code handles shocks and gradual changes evenly and with results and risks that are both predictable and manageable. While they may not be “best of breed” and actually be less efficient than ideal implementations, the overall abilities of the code base to work within constraints and absorb and adapt to various inputs make it a valuable asset to an enterprise. When risks, revenue, and level of effort are all factored in, the Darwinian Code Base is too tough to rewrite; the rewards for moving away from it are simply not evident.
What makes a code base strong and “to tough to rewrite” is actually quite simple. The code must be very stable, enough so that it has a highly diminished risk of any disasters causing serious revenue spikes. It must be mutable, in that any changes have diminished risk due to excellent processes, extended tribal knowledge, ease of maintenance, or great design. Finally, it must be workable, in that the existing resources of the Enterprise may impart changes or maintenance upon it with relative and straightforward ease.

So Darwinian code is great!

Stable, Mutable and Workable code sounds great. Now let’s look at some challenges. Heroic efforts sometimes have to come to bear to “save” a deployment, and often there is a very real danger of intellectual capital departing the company and taking critical knowledge with them. As newer and better ways of making the application work become available, the existing Big Ball of Mud code base cannot take advantage of them because they are tied to their implementations and tightly coupled to their dependent technologies.
So the “Good Enough” code base can actually become so “good” that it can never get “better.” Another problem becomes turnover, because developers really aren’t all that enthused about working on really old stuff. The Darwinian Code Base is rarely Great, often Good Enough, and usually On the Cusp of Relevance.

The developer vs. the “fighter jock”

Nobody ever joins the Air Force to become a transport pilot. Most think bombers are slow and dumb as well. Everyone wants to fly the latest, greatest, fastest plane that dumps all kinds of cool bombs and missiles on bad guys.
Developers are pretty much the same; once a developer has been doing their job for a period of time, they get bored and want to do the “exciting”, “interesting” and “new” stuff. Everyone is thinking about their career and worries about getting pigeonholed into old technology and not being able to code up the latest and greatest stuff when they get the call.

This is a really big problem with Darwinian code:

Your best talent will leave you because it’s not fun or interesting to extend and maintain five-year-old-crap!

So is there an answer?

Manage well – Change slowly.

Evolution vs. Revolution

With all the engineers in a company breathing down your neck to DO SOMETHING, it’s very convenient them on Revolution. Just like any country where the politics are a mess, it is easy to promise the followers that everything will be perfect when the revolution is over. History really doesn’t support this in the real world. Revolutions are bloody and fraught with risk and hardship, and that’s AFTER the fighting stops. Changing one’s code base from the ground up while trying to maintain a business is nearly impossible.
Remember that the existing code base helped the company arrive at its current point. More often than not is a successful, but possibly critical, juncture. Revolution doesn’t offer the chance of using what is right with the old code. That paradigm requires a complete rebuild and forklift of new code in place, effectively “throwing the baby out with the bath-water.”
An Evolutionary development style stresses refactoring over re-architecture. Find the low-hanging fruit, clean that area up. Find more. Create maintainable interfaces. Like the instructions on the back of a shampoo bottle; “Lather, Rinse, Repeat”. This allows for the application to grow and flourish while gradually making the changes that need to be made. Finishing is less important than the ability to maintain momentum – one may continuously alter your roadmap to take advantage of the tuition you’ve paid along the way. It becomes a “Zen” thing.

The “end” is not important, only the “journey”.

This becomes easy to sell to your staff and will allow you to manage change and growth effectively. Sell the “Zen” thing:

• Create teams to tackle specific problems and convene meetings to discuss the technologies to use. These teams can develop strategies and methods to migrate an application in a positive and maintainable direction, and still keep good people because they are doing the work that keeps them “on the edge”.
• Interfaces are your friend. Well-defined APIs will allow various parts of an application to be componentized. A team can then “divide and conquer” the application, lowering risk, times to market, and allowing for the “good stuff” to stay. Good interfaces also allow various parts of an application to be “black boxed,” which is highly convenient if different groups are involved, since a group doesn’t have to “own” the risk of changes to code that is not currently under a their purveyance.
• Create “battle plans” to anticipate shocks and adapt to changing environments. When languages face major revisions, it is often published years in advance of the actual release, and backward-compatibility is most likely a very obvious, addressable issue that can be tackled with good planning.
• Constantly involve the team in finding “low hanging fruit” to clean up and extend the existing frameworks. Is the documentation complete? When an engineer opens up a file, how hard is it to check the documentation and append/complete it before check in? Can a custom or open source Tag Library replace the existing scripted code? Is there repeated code in the file that can be added to a utility class? These are only examples, but incremental “clean up” is like compound interest; it takes very little development time, boost morale and standards, and finally keeps the existing code base up-to-date for very little risk.
• Evolve into a code base that is continuously upgraded with “service packs”, and business gets a continuously operational application that allows them to implement their goals.

It doesn’t have to be boring, but in the eCommerce world, it is vital to do “the right thing at the right time.” A successful eCommerce enterprise is almost never alone in its space. Once success is identified, copycats abound as the barriers to entry drop and the product is commoditized. One must assume that any deviation from a stated goal is wasted time, as any efficient competition will NOT deviate. Gambling for cool technology over revenue is a sucker bet.
It’s not the perfect solution for engineers that want to create the next killer app, but remember, in eCommerce, “we’re not a technology or software company.”

* Thanks to Rod Morrison and Josh Lucas for peer review and suggestions.
The Darwinian CodeBase — Dowload as Word Doc.