Danilo Gurovich

Building a Consistent, Predictable and Efficient Environment for Enterprise eCommerce Applications.
 

Predictable Schedules Create Outstanding Teams

When many eCommerce shops first set up, there are usually a few developers wearing many hats; they have access to everything, develop within specific areas and channels, and check in code for deployment as it becomes ready. While this type of development will work as a business starts, it will soon become unmanageable and will result undesired consequences as changes are not integrated, and business priorities are not addressed, or worse.

Predictability and easily enforced processes set all stakeholders within an application free to do their jobs and collaborate at the highest levels possible. With a strong process, planned changes will be scheduled for deployment with a high degree of accuracy. Engineers enjoy complete and accurate requirements, and know what is expected of them throughout each cycle. Quality Assurance will have time to create plans and fully test changes, and Releases to Production will no longer be a nail-biting, all-hands-on-deck stress-fest.

The following proposed process details the steps necessary for day-to-day development, maintenance and enhancements for the IHO/Offer Channels applications. They are exclusive of longer-term enhancements and development requiring multiple-week pulls. Only those issues that may be completed in short periods will applies here,. This is significant because with most mature applications, nearly 75% of all issues encompass this type of development.

Tracking the status of Issues

An Issue Tracking System is a “blog-like” application that allows issues to be tracked throughout the development/release cycle. Its main features include fields to track:

• Who the Business Stakeholder is for the issue (i.e. owner)
• Who has responsibility for the Issue’s forward progress at a point in time
• The Scheduled Release Date
• An identifier for the issue so that communication may occur around it
• A field to delinieate the Issue’s status
• Attachments to the Issue for requirements or any information needed to proceed in the process
• A “Comments” section that allows all stakeholders to create a chronological and permanent record of what happened to the issue over its lifecycle

Other features of a robust issue tracking system are excellent aggregation and reporting tools, integration with Wiki pages, email notifications and IDE integration. The tool chosen by many, and arguably a “best of breed” is “JIRA” from Atlassian Software.

Assumptions

Many, if not most, of the web applications built in an enterprise are ongoing products that require maintenance, small tweaks and enhancements on an ongoing basis. While larger changes to the system occur on a much more infrequent basis, it is essential that these be separated out into a different schedule and allow for the ongoing implementations to occur on a regular schedule that has no dependencies upon larger, more complex deployments.

There are several assumptions that must be made to completely embrace the Overlapping Schedule Model fully:

1. There will always be more issues and tweaks needed than resources available
2. A method for communicating issues between all stakeholders exists
3. There is a need for predictability and pacing
4. While not perfect, the application has some stability

The Overlapping Schedule Model

An Overlapping Schedule is a very efficient model to keep the all business process stakeholders fully engaged, collaborative and running at peak efficiency. It uses tight structure to coordinate activities in three separate areas, Business, Development, and Release/QA. While Business Stakeholders are planning future releases, the Engineering Team is developing the next release, and the QA and Release Management is readying/deploying the current release for production.

Figure 1 shows the basic development cycle and it’s overlapping steps, creating a never-ending, continuous process that keeps all resources utilized at maximum efficiency:

 

The Details of the Process

Collaboration exists at various points between all stakeholder groups to allow for seamless transitions and predictable behaviors. Issues planned for release will be vetted between Business and Engineering for resourcing purposes, Developers will interface with Business to demonstrate functionality, and Engineers, Business and QA will collaborate with Release Management in the “Go-No-Go” process gate as final release is set (other collaborations are spelled out in the details below).

An Issue planned in one week will be released on the second Thursday following. Metrics can be attached to every step in the process. Everyone knows what to do and when to do it, and communication between the disparate groups are formalized and clarified. The entire process becomes engrained within the fabric of the company. The team implementing it becomes responsible for the continuous and uninterrupted revenue operations of the application. This “Profit Team” process is outlined:

 

The Steps

Starting at the End – Release

Time: Thursday, 2pm

Release is scheduled for Thursday at 2pm. This is no arbitrary time. Releases should never occur on Fridays or Mondays because there is no time for Go-No-Go meetings or mitigation in the event of emergencies. Tuesdays and Wednesdays are also out because of the need to QA all issues thoroughly, and this has to happen through unbroken week and not broken up by a weekend. (Mon-Fri).

Thursday becomes the release day by elimination.  It is impossible to release first thing in the morning (the Go-No-Go meeting must occur first), and time must be made for a “burn in” period after release to production. This means that after lunch is the best time, but since preparation must occur immediately before the release, so it can’t be done before lunch  (e.g.Thursday, 2pm).”

The Release of the current, tested and accepted code signifies the very end of the process and the beginning of a new one, so this point is a perfect place to point out the criteria for starting the next cycle — this occurs when new enhancements have been identified and a decision has been made to move them forward into production.

These identified issues are only in their basic state. They have no prioritization. All of their requirements may not be complete, and the majority certainly haven’t been resourced. They are probably known only to the business stakeholders and have no release date.

Adding New Enhancements/Maintenance to the Issue Tracker

Time: Thursday Morning – Friday Morning

Identified Issues are entered into the Issue Tracking System with their status marked as “Prioritization”. Once all the issues are added to the Tracking system, the process can move forward. At this point, the business stakeholders are still the only group actively involved in the process.

Add Supporting Docs

Time: Thursday Morning – Friday Morning
With all the issues in the Tracking System, any supporting documentation that is available at this point is added to the corresponding issue, along with any commentary that will give the prioritization committee enough information to put the issues in the right order for development and eventual release.

Prioritization – The First Big Step

Time: Friday Noon
Lunch is served as Business Stakeholders engage to prioritize the issues in the system for eventual deployment. Still only a Business Stakeholder exercise, the process is now setting up to collaborate with the rest of the stakeholders. Issues are prioritized and prepared to meet the Development Managers for mapping to resources.

Meet the Engineers – Dev/Resource Alignment Meeting

Time: Friday Afternoon
The Program Coordinator for the business generates a report containing all issues prioritized for the next release. Their status at this time should be marked as “Requirements” in the Issue Tracker. The Coordinator and Development Manager(s) should be prepared meet and map each issue to Engineering Resources for Implementation. The Program Coordinator will go through each issue with the Manager(s), and the Managers will assign a resource to each issue. At this point a “contract” is being made between the Business and Development groups, spelling out just what can and will be done for the next release cycle.

It is the Development Manager’s responsibility to carefully look at each Issue and ensure that most, if not all requirements are complete and formatted for maximum understanding with respect to the assigned engineer. All gaps must be pointed out and recorded against each issue for its corresponding business owner’s follow-up.

It is also possible that the prioritized issues may lack the development resources necessary to for completion within the next release cycle (developers may be unavailable, an emergency may have supplanted resources, etc.). When this occurs, they will be re-prioritized and sent back into the queue for the next week. A team can only do so much, and creating situations requiring extraordinary effort will “raise the bar” and eventually burn out the team. Consistency is critical, and this step is the decision gate to ensure that all follow-on processes go smoothly.

Once all issues have been mapped to resources and the Business-to-Development hand-off is complete, all issues that are to be worked on are marked “Ready for Development”.

Finalize Requirements

Time: Friday Afternoon – Monday Morning
At this point, there is an “Engineering Perspective” to the issues. The Program Coordinator has the opportunity to work with the stakeholders of the individual issues and update any requirements and answer any questions that have arisen from the Dev/Resource Alignment meeting. The stakeholders can have some informal contact with their assigned engineer, and set up stakeholder – engineer/developer meeting, also known as a JAD session.

Dev/Biz JAD session – Is This What You Want?

Time: Monday 11am

As soon as possible on the Monday morning of the Development week, the Business Stakeholder and any other interested parties should meet with the assigned developer of the Issue to have an overview of what is to be built and positively identify every unclear detail surrounding it’s development. This is called a Joint Application Design/Developemnt (JAD) session.

The JAD process does for computer systems development what Henry Ford did for the manufacture of automobiles (a method of organizing machinery, materials, and labor so that a car could be put together much faster and cheaper than ever before – the assembly line). The goal in systems development is to identify what the users really need and then set up a system or process that will provide it. Traditional methods have several built-in delay factors that get worse as more people become involved. JAD cures this by addressing four simple assumptions:

• People who actually do a job have the best understanding of that job.
• People who are trained in information technology have the best understanding of the possibilities of that technology.
• Information systems and business processes rarely exist in isolation — they transcend the confines of any single system or office and affect work in related departments. People working in these related areas have valuable insight on the role of a system within a larger community.
• The best information systems are designed when all of these groups work together on a project as equal partners.

The outcome of the JAD session should show in evidence:
   

• Definition of the Issue to the satisfaction of both parties.
• Understanding and translation of all requirements
• Obtained approval from all parties with respect to the forward development of the issue.
• Surety that all decisions and changes are reflected in the Issue Tracker.
• An Agreed upon time for demonstration before the code is checked in for QA

With the completion of this step, the Issue is marked “In Development”.

For more information on a JAD session, consult http://www.umsl.edu/~sauterv/analysis/JAD.html.

Development – Where the Rubber Meets the Road

Time: Monday 11am – Friday 12pm

The Developer will now build what has been set forth in the Issue Tracker and based upon the learnings from the JAD session. If any problems or roadblocks occur, they should log the problem clearly in the Issue Tracker, mark the Issue “Development Problem”, and notify their Manager and the Stakeholder for the Issue.

Development on the issue can continue until the Code Freeze/Check-in time of Friday, 12pm. Any Issues being worked on after that point are at risk for being pushed off to another cycle, and will need to be addressed by stakeholders.

As necessary, a demonstration should take place based upon a time set up in the JAD session. This demonstration should answer the question “is this what you asked me to build?” It is important to have some time left in the Development Week to make any last minute corrections to the implementation.

During the Development Cycle, the QA team will be pulling the Issues that are to be tested from the Reporting System of the Issue Tracker, and setting up their tests. They may be asking any Stakeholders questions based upon the testing needs of the particular issue.

Ready For QA Code Check-in

Time: Friday 12pm

All code is checked into the Source Control System at the Friday deadline. The Release manager will merge the newly developed code into QA Branch. The QA team will begin reviewing issues to set up tests and begin next week’s three-day testing cycle.

Developers will prepare next for the next Development Week. All checked in Issues status is set <em>“Ready for QA”</em> at time of check in.

It is important to check development code in when finished and not wait till the last minute to prevent collisions, over-writing of code and any last minute problems that can surround more than one person trying to check in the same file(s). Check in soon and often should be a mantra.

Sandbox Demo to Business

Time: Friday Afternoon

Developers demonstrate changes to Business Stakeholders on an “as needed basis” . All Business Stakeholders sign off on their particular issues and agree that QA can take over and start testing. This is the Final gate check for Development. Issue Tracker Status is marked <em>“QA”</em> by QA personnel.

 

QA Complete, Final Business Check

Time: Wednesday afternoon
The business stakeholder will view the corresponding implementations to their issues served from the QA machines. Any final defects may be addressed, or the Issue may be delayed for another cycle.
If all is ok, the Issue Tracker Status for QA complete is “Ready for Production”. Business has final say. 

Move Code to Head/Staging

Time: Wednesday 4pm
Release manager will take all issues marked “Ready for Production” and merge to the Production Staging Branch. If Staging exists, it is set to staging for final demonstrations, review and any business collaborations on an as-needed basis (sometimes there are outside dependencies that may need to have a “production-level” deployment to expose any final issues, or be observed by external stakeholders. This is the place to do this.

Last Minute Gut Check

Time: Thursday Morning
Final Business reviews. Go-No-Go meeting. Any last minute issues are addressed before launch. Afternoon begins with ITOPS/Release Coordination for Production and launch.

Release – OK, Let’s start at the top!

Time: Thursday, 2pm
Code is launched to production. The deployment must be watched throughout the afternoon for any production/emergency issues. Business stakeholders should review their changes on production sites. Any outside communication with external stakeholders is handled. Issues marked “Production” in Issue Tracker. Issues from previous week marked “Production” are now marked “Closed”.

The Three P Method: P1, P2, P3.

Below is an industry-standard practice of prioritizing issues with respect to their urgency. They are categorized as P1 – P3, the most urgent being P1. Each has impact to issues that are “already” in the queue for development, which must be taken under consideration.

• P1 — when a fix is needed to affect a serious break that will affect the company legally, morally, or for revenue impact over a set value (usually 50k or so). Everything is dropped and a release to production is done ASAP, outside of the schedule. Depending upon the impact, an upcoming release may be delayed for a week or certain issues may not be completed for the next scheduled release. A post-mortem is required to decide what the effect to the normal schedule is.A P1 fix usually requires a VP for “go-forward” approval. Often a meeting of stakeholders must occur to gather information with respect to impact and fixes.
• P2 — when a fix is needed to affect a break within the system. In this scenario, the break is fixed for the very next scheduled release — other issues being worked on may be affected, but no special release is done. Only in the rarest of cases is a Post-Mortem needed. Needs Director-level approval.
• P3 — anything within the normal schedule. Planned for one week, developed for a week, QA’d and released the following week.

The Overlapping Schedule Model Process within the Issue Tracker

The JIRA issue tracking system aligns workflow with Issues to allow for clean and simple end-to-end tracking of issues aligned closely to physical business processes. It employs a “status-transition” method to follow the process. Each “status” may have various “transitions” that allow for different scenarios. By clicking on a particular transition, the workflow will change to a corresponding status, and the path chosen will be reflected in the next available transitions available to the JIRA User.

The Diagram below shows the workflow available to the Team using JIRA and implementing the Overlapping Schedule Model. The workflow has been mapped out and attached to the Schemes available to the IHO. This model can be used or extended for other development channels. Note various paths that do not take a particular Issue into production, and other paths that have various conditions that must be satisfied to move forward. The “straightest” path to production is indicated by bold blue lines, showing the transitions and their corresponding status indicators.

Tags: Development Processes, eCommerce Development Processes, eCommerce Process, JIRA, Overlapping Schedule Model, workflow