Cubicles are the worst invention in the history of business. I’m still trying to think about how/why somebody thought this was a good idea. The only thing I can come up with is a secret society of Janitors, maintenance people and office furniture salespersons have formed some kind of plutocracy that actually control how we work.

Cubes make sense if you just take them in a two-dimensional plane. They are configurable in thousands of ways. You get a desk. You get a chair. You get some drawers and overhead storage. It locks. When you’re not there, you can put everything away and it’s all pretty and clean. And beige, or gray, or some kind of color that a tribe in Peru may call “Llama Vomit”. They were made for another place and time.

I have sat in cubes for the last 15 years except on a few wonderful occasions. They all pretty much looked the same. High walls that caused you to “gopher” when any kind of sound or personal communication was needed. Cramped and created so you face away from your door, so collaborating with anyone or even coming into another’s cube is an unexpected interruption at best, downright startling at worst. If you’ve been sentenced to a cube, how many of you have “rear view mirrors” or your monitor is positioned in such a way as you can see shadows behind you?

Of course, if you’re some kind of “middle manager”, you might have one of the “bigger” cubes that have a “desk” that protrudes through the middle and allows others to sit at it facing you. You get to be the “boss”, because it’s “your” cube, “your” desk, and others are there to seek your patronage. Now think about it. You’re really not going to do any “serious” business and your superiors sitting in their offices are showing that they have given you no “real” power. If you raise your voice above a whisper, you’ll be heard by persons 20 feet away. Nothing important or sensitive can be discussed there, so a “meeting room” must be booked or squatted in. Worthless.

Cubes have fabric walls and fixed furniture. Want to do some design? Where do you put the whiteboards? How do you collaborate? There’s drawers everywhere, and cubes are designed to stick your monitor in the corner (designed when monitors were 17″ wide and 20″ deep, too) and anyone working with you at your desk will be constantly banging their knees in the crappy “guest” chair that he’s borrowed from the cube around the wall.

Then there’s the cube farm. Row upon row of cubes in a giant room. 6 feet high, 50-75 feet wide and 150 feet long. The rows have hastily-printed pieces of paper in fourteen-point type that tells you who “lives” down that row. It’s like some kind of weird Stalinist housing project gone horribly wrong.

But it’s really easy to keep “looking” clean (whether it’s actually “sanitary” is debatable). I’t easy for your boss to walk down the row and quickly see who’s there and if everyone’s working. Everyone has their place. Everyone knows their job. Everyone is pounding away, waiting for 5 o’clock or a meeting to come up on their dilapidated, ancient version of Outlook, or the biannual review.

It’s also a Petri Dish of the germs that kill a company. People don’t collaborate. People have to book meeting rooms just to have any kind of communication, and that communication is delayed until the meeting room can be booked or until the “weekly” standing meeting because nobody actually talks or can collaborate in a free flowing environment. If someone has a question, they must get up, find the target person’s cube and interrupt whatever they are doing, simply because they have no idea if they are busy or not and have wasted time walking many yards across a room just to get a quick answer, and as soon as they “peek in”, the unwanted distraction has occurred. Politics and backstabbing occur in a cubicle environment. People “go back” to their cubes to stew, and only communicate by email or instant messenger. How many times have you read a message or mail and misread the intention of the sender? How often has a sender sent you an email and beat the thing to your cube so he/she can interrupt you anyway, just to ensure that you’ve been “papered”? How many times have you heard gossip wafting over your cube?

Cubes are STUPID-expensive and?inefficient. A single cubicle can cost upwards of $2000. Each cube takes up large amounts of empty space. It is expensive to move, very heavy and incredibly bulky to store. They have all the character of a minivan. An area that has 5 cubicles can support 10 persons sitting around a large, comfortable table with the best chairs, open lighting, a possible view out the window and… wait for it… face-to-face interaction!

What I’ve found out as I’ve sat around a table with 6 colleagues is:

• You can’t lie to anyone when you stare them in the face 8 hours a day, 200 days a year
• Nobody gossips, ever
• There is no disciplinary action — your peers keep you on the straight and narrow — you have to face your colleagues and pull your weight
• If you have a question, you either ask the person you’re working with, or you wait until they have a free moment
• Meetings start and end on time, and most of the time, nobody gets up. We schedule meetings not only by looking at calendars, we actually go around the room and make sure that the time is good for everyone.
• We more often than not eat lunch together at our table
• There’s no heroes, no chumps, no wallflowers and everyone carries their own weight
• If there is a knowledge gap that needs to be filled, the group will find a solution through pair programming, mentoring or pointing out the best places to find the information
• There’s just not a lot of paper, ever
• People TALK instead of EMAIL; the amount of time I’ve spent reading my email has dropped dramatically. I don’t even leave my email program up in plain site, anticipating some communication.
• The room is bigger — with no walls, the air flows freely around the room, light comes in from the windows and shadows are cast by human beings
• There are no “cold cubes” or “hot cubes” because of ceiling vent positioning
• We’re all happy to come to work. Stupid-happy. We all get along. We know things about our families and care about making sure that everyone is lifted to make the team, and the company a better place
• It’s better to “run with the Buffalo” instead of standing in your own poop with like a steer

And the biggest thing I’ve learned? We innovate. We invent. We create great code because we’ve combined our best practices. There’s no real “leader”, no “tail end charlie”. We write code with few bugs.

We get things done.

I’m doing the best work of my career. I have the best working relationships of my career. I have the best bosses in my career, I know exactly what is expected of me, have confidence in my place in my group, and feel empowered and sometimes humbled by the abilities of my colleagues. I feel respected and willingly give respect. I am happy.

The cube has become a Dinosaur Feed Pen. It’s long past it’s sell-by date in the modern corporation. In fact, it might be the single-largest factor in an unhappy and unproductive environment. Let me tell you, the happier your workers are, the better off you are. Would you like happy or sad people to build your car? How about unhappy airline mechanics? Do you want people that write the code, the intellectual capital of your eCommerce or IT department to feel like people are watching them all the time, or that they will be constantly interrupted, or wasting time to go somewhere to “get some air”? Companies face the necessity of cutting budgets and squeezing dimes to make a dollar. Simply sitting around a table with a great wireless connection, a projector, screen, places to put cards and some whiteboard space, will yield huge dividends.

Alignment is everything

For many years, managers in the software industry submitted budgets that were unrealistic to Business Unit managers that had no clue as to what to question, what to approve or where to cut. As business has grown up and increased its expectations of IT departments, Development and Infrastructure has also been forced to mature and face the fact that they aren’t an “overhead” cost. This is especially true in eCommerce.

The Best is the enemy of the Good

I’ve never met an Engineer or Manager of Engineers that didn’t see an infrastructure project as vital to the organization’s ongoing health and stability. I’ve seen (and been) that Manager before. They’ve actually been wrong more than they’ve been right. The reason for this is the lack of a complete picture of the goals of the organization as a whole, the priorities for various Development/Infrastructure (DEV-INF) projects, and the fact that budgeting exists because resources are finite. Sometimes an eCommerce site can limp along for years on a code base that is far from the ideal architecturally, but it may cost far less to maintain and extend it than to upgrade it to the “Next Great Killer Platform” at the cost of revenue-driven projects.

The Holistic view of a Budget

While all projects will have some value, it’s important to prioritize projects in alignment with the overall organization’s strategy and objectives. There is no use in a complete re-architecture of the site when current demands of Marketing, Consumer Experience and other various Business Units cannot be met. The goal in eCommerce is simple. Keep the business growing, keep the site in “five nines”, and refine the user experience to increase conversion. DEV-INF budgets must play in this ballpark or, as a development leader, all credibility with Business Units will be lost; ultimately will leding to a breakdown in communications between Engineering and Business, which is a crisis in an eCommerce workplace.

Metrics are the key

Once projects are aligned with Corporate Objectives, it’s vital to show all cost drivers for these expenses. Many times the cost of ongoing maintenance is not factored in, but can be as much as 65% of an overall budget. This is also true for time budgeted for QE/QA. While the individual Engineer writing code may only write 250 lines that finally make it into production in a three-month long project, how many times he/she had to write it, how much time was eaten up managing and checking this code, and finally what documentation, knowledge transfer and future extendibility these 250 lines will have also affect costs. All these things must be taken into consideration.

Budgeting isn’t just all planning. Once the planning is complete, the spending begins. Vital in this aspect is accountability and assessment of performance. Hard, industry-standard metrics exist to gauge the performance of DEV-INF budget items. In eCommerce I’ve found many easily measured values:

1. Actual vs. Proposed Development times.
2. Actual vs. Proposed Resources.
3. Pre-release bugs.
4. Post-release bugs.
5. Effects upon site stability and performance.
6. Did the Project “Do” what the owners said it would do?
7. Were the expectations of Business Units met, and are these stakeholders satisfied with the results?
8. Did any problems that come up get handled in appropriate manners?
9. Were in-place processes broken?

Approaches

Approaches to DEV-INF budgeting should be aligned with the current culture and expectations of the Business. They must have significant buy-in by various business stakeholders. If business drivers have “skin in the game” as to line-items in a DEV-INF budget, this automatically give credibility to it and also act as a check to make sure that I’ve done my homework. Many successful organizations use the Consolidated Projects Approach to budgeting and allocation for DEV-INF projects with outstanding success. In a nutshell:

Consolidated Projects List basic points:

• IT budget planning process starts before the organization’s process.
• IT managers submit summary project data for consolidation into a single spreadsheet
• These projects are then sorted and prioritized by DEV-INF managers before submission to the overall business Prioritization Committee.
• The Prioritization Committee decides how these projects are to be aligned, budgeted and implemented within the entire Organization’s overall Business needs and strategies.

This approach provides Senior Executives with a complete picture of all proposed projects, their costs, and priority ranking; these units will have a “30,000 foot” picture to make a more informed decision. Total project requirements may now be viewed as a single picture to senior management; priorities become a decision factor. No single group can drive pet projects or non-revenue generating development without prioritization or fully understanding what the business impact or opportunity costs involved are.

This process may be extended with Gate Checks to assure relevance to the goals of the company as time changes. This is especially true with larger organizations that may be widely distributed. Projects can often times have huge scope (ah — to de-scope, another topic!) and can take months of development time.? The needs of a business can change over this period, and any project that is underway should have enough flexibility to move with the times or the business stakeholders or group as a whole should have the gumption to kill or suspend development indefinitely or until such time as solid metrics warrant completion.

Predictable Schedules Create Outstanding Teams

When many eCommerce shops first set up, there are usually a few developers wearing many hats; they have access to everything, develop within specific areas and channels, and check in code for deployment as it becomes ready. While this type of development will work as a business starts, it will soon become unmanageable and will result undesired consequences as changes are not integrated, and business priorities are not addressed, or worse.

Predictability and easily enforced processes set all stakeholders within an application free to do their jobs and collaborate at the highest levels possible. With a strong process, planned changes will be scheduled for deployment with a high degree of accuracy. Engineers enjoy complete and accurate requirements, and know what is expected of them throughout each cycle. Quality Assurance will have time to create plans and fully test changes, and Releases to Production will no longer be a nail-biting, all-hands-on-deck stress-fest.

The following proposed process details the steps necessary for day-to-day development, maintenance and enhancements for the IHO/Offer Channels applications. They are exclusive of longer-term enhancements and development requiring multiple-week pulls. Only those issues that may be completed in short periods will applies here,. This is significant because with most mature applications, nearly 75% of all issues encompass this type of development.

Tracking the status of Issues

An Issue Tracking System is a “blog-like” application that allows issues to be tracked throughout the development/release cycle. Its main features include fields to track:

• Who the Business Stakeholder is for the issue (i.e. owner)
• Who has responsibility for the Issue’s forward progress at a point in time
• The Scheduled Release Date
• An identifier for the issue so that communication may occur around it
• A field to delinieate the Issue’s status
• Attachments to the Issue for requirements or any information needed to proceed in the process
• A “Comments” section that allows all stakeholders to create a chronological and permanent record of what happened to the issue over its lifecycle

Other features of a robust issue tracking system are excellent aggregation and reporting tools, integration with Wiki pages, email notifications and IDE integration. The tool chosen by many, and arguably a “best of breed” is “JIRA” from Atlassian Software.

Assumptions

Many, if not most, of the web applications built in an enterprise are ongoing products that require maintenance, small tweaks and enhancements on an ongoing basis. While larger changes to the system occur on a much more infrequent basis, it is essential that these be separated out into a different schedule and allow for the ongoing implementations to occur on a regular schedule that has no dependencies upon larger, more complex deployments.

There are several assumptions that must be made to completely embrace the Overlapping Schedule Model fully:

1. There will always be more issues and tweaks needed than resources available
2. A method for communicating issues between all stakeholders exists
3. There is a need for predictability and pacing
4. While not perfect, the application has some stability

The Overlapping Schedule Model

An Overlapping Schedule is a very efficient model to keep the all business process stakeholders fully engaged, collaborative and running at peak efficiency. It uses tight structure to coordinate activities in three separate areas, Business, Development, and Release/QA. While Business Stakeholders are planning future releases, the Engineering Team is developing the next release, and the QA and Release Management is readying/deploying the current release for production.

Figure 1 shows the basic development cycle and it’s overlapping steps, creating a never-ending, continuous process that keeps all resources utilized at maximum efficiency:

Main stakeholder activities in the Overlapping Schedule Model.

The Details of the Process

Collaboration exists at various points between all stakeholder groups to allow for seamless transitions and predictable behaviors. Issues planned for release will be vetted between Business and Engineering for resourcing purposes, Developers will interface with Business to demonstrate functionality, and Engineers, Business and QA will collaborate with Release Management in the “Go-No-Go” process gate as final release is set (other collaborations are spelled out in the details below).

An Issue planned in one week will be released on the second Thursday following. Metrics can be attached to every step in the process. Everyone knows what to do and when to do it, and communication between the disparate groups are formalized and clarified. The entire process becomes engrained within the fabric of the company. The team implementing it becomes responsible for the continuous and uninterrupted revenue operations of the application. This “Profit Team” process is outlined:

Single Cycle Process Flow for Overlapping Schedule Model

The Steps

Starting at the End – Release

Time: Thursday, 2pm

Release is scheduled for Thursday at 2pm. This is no arbitrary time. Releases should never occur on Fridays or Mondays because there is no time for Go-No-Go meetings or mitigation in the event of emergencies. Tuesdays and Wednesdays are also out because of the need to QA all issues thoroughly, and this has to happen through unbroken week and not broken up by a weekend. (Mon-Fri).

Thursday becomes the release day by elimination.  It is impossible to release first thing in the morning (the Go-No-Go meeting must occur first), and time must be made for a “burn in” period after release to production. This means that after lunch is the best time, but since preparation must occur immediately before the release, so it can’t be done before lunch  (e.g.Thursday, 2pm).”

The Release of the current, tested and accepted code signifies the very end of the process and the beginning of a new one, so this point is a perfect place to point out the criteria for starting the next cycle — this occurs when new enhancements have been identified and a decision has been made to move them forward into production.

These identified issues are only in their basic state. They have no prioritization. All of their requirements may not be complete, and the majority certainly haven’t been resourced. They are probably known only to the business stakeholders and have no release date.

Adding New Enhancements/Maintenance to the Issue Tracker

Time: Thursday Morning – Friday Morning

Identified Issues are entered into the Issue Tracking System with their status marked as “Prioritization”. Once all the issues are added to the Tracking system, the process can move forward. At this point, the business stakeholders are still the only group actively involved in the process.

Add Supporting Docs

Time: Thursday Morning – Friday Morning
With all the issues in the Tracking System, any supporting documentation that is available at this point is added to the corresponding issue, along with any commentary that will give the prioritization committee enough information to put the issues in the right order for development and eventual release.

Prioritization – The First Big Step

Time: Friday Noon
Lunch is served as Business Stakeholders engage to prioritize the issues in the system for eventual deployment. Still only a Business Stakeholder exercise, the process is now setting up to collaborate with the rest of the stakeholders. Issues are prioritized and prepared to meet the Development Managers for mapping to resources.

Meet the Engineers – Dev/Resource Alignment Meeting

Time: Friday Afternoon
The Program Coordinator for the business generates a report containing all issues prioritized for the next release. Their status at this time should be marked as “Requirements” in the Issue Tracker. The Coordinator and Development Manager(s) should be prepared meet and map each issue to Engineering Resources for Implementation. The Program Coordinator will go through each issue with the Manager(s), and the Managers will assign a resource to each issue. At this point a “contract” is being made between the Business and Development groups, spelling out just what can and will be done for the next release cycle.

It is the Development Manager’s responsibility to carefully look at each Issue and ensure that most, if not all requirements are complete and formatted for maximum understanding with respect to the assigned engineer. All gaps must be pointed out and recorded against each issue for its corresponding business owner’s follow-up.

It is also possible that the prioritized issues may lack the development resources necessary to for completion within the next release cycle (developers may be unavailable, an emergency may have supplanted resources, etc.). When this occurs, they will be re-prioritized and sent back into the queue for the next week. A team can only do so much, and creating situations requiring extraordinary effort will “raise the bar” and eventually burn out the team. Consistency is critical, and this step is the decision gate to ensure that all follow-on processes go smoothly.

Once all issues have been mapped to resources and the Business-to-Development hand-off is complete, all issues that are to be worked on are marked “Ready for Development”.

Finalize Requirements

Time: Friday Afternoon – Monday Morning
At this point, there is an “Engineering Perspective” to the issues. The Program Coordinator has the opportunity to work with the stakeholders of the individual issues and update any requirements and answer any questions that have arisen from the Dev/Resource Alignment meeting. The stakeholders can have some informal contact with their assigned engineer, and set up stakeholder – engineer/developer meeting, also known as a JAD session.

Dev/Biz JAD session – Is This What You Want?

Time: Monday 11am

As soon as possible on the Monday morning of the Development week, the Business Stakeholder and any other interested parties should meet with the assigned developer of the Issue to have an overview of what is to be built and positively identify every unclear detail surrounding it’s development. This is called a Joint Application Design/Developemnt (JAD) session.

The JAD process does for computer systems development what Henry Ford did for the manufacture of automobiles (a method of organizing machinery, materials, and labor so that a car could be put together much faster and cheaper than ever before – the assembly line). The goal in systems development is to identify what the users really need and then set up a system or process that will provide it. Traditional methods have several built-in delay factors that get worse as more people become involved. JAD cures this by addressing four simple assumptions:

• People who actually do a job have the best understanding of that job.
• People who are trained in information technology have the best understanding of the possibilities of that technology.
• Information systems and business processes rarely exist in isolation — they transcend the confines of any single system or office and affect work in related departments. People working in these related areas have valuable insight on the role of a system within a larger community.
• The best information systems are designed when all of these groups work together on a project as equal partners.

The outcome of the JAD session should show in evidence:
   

• Definition of the Issue to the satisfaction of both parties.
• Understanding and translation of all requirements
• Obtained approval from all parties with respect to the forward development of the issue.
• Surety that all decisions and changes are reflected in the Issue Tracker.
• An Agreed upon time for demonstration before the code is checked in for QA

With the completion of this step, the Issue is marked “In Development”.

For more information on a JAD session, consult http://www.umsl.edu/~sauterv/analysis/JAD.html.

Development – Where the Rubber Meets the Road

Time: Monday 11am – Friday 12pm

The Developer will now build what has been set forth in the Issue Tracker and based upon the learnings from the JAD session. If any problems or roadblocks occur, they should log the problem clearly in the Issue Tracker, mark the Issue “Development Problem”, and notify their Manager and the Stakeholder for the Issue.

Development on the issue can continue until the Code Freeze/Check-in time of Friday, 12pm. Any Issues being worked on after that point are at risk for being pushed off to another cycle, and will need to be addressed by stakeholders.

As necessary, a demonstration should take place based upon a time set up in the JAD session. This demonstration should answer the question “is this what you asked me to build?” It is important to have some time left in the Development Week to make any last minute corrections to the implementation.

During the Development Cycle, the QA team will be pulling the Issues that are to be tested from the Reporting System of the Issue Tracker, and setting up their tests. They may be asking any Stakeholders questions based upon the testing needs of the particular issue.

Ready For QA Code Check-in

Time: Friday 12pm

All code is checked into the Source Control System at the Friday deadline. The Release manager will merge the newly developed code into QA Branch. The QA team will begin reviewing issues to set up tests and begin next week’s three-day testing cycle.

Developers will prepare next for the next Development Week. All checked in Issues status is set <em>“Ready for QA”</em> at time of check in.

It is important to check development code in when finished and not wait till the last minute to prevent collisions, over-writing of code and any last minute problems that can surround more than one person trying to check in the same file(s). Check in soon and often should be a mantra.

Sandbox Demo to Business

Time: Friday Afternoon

Developers demonstrate changes to Business Stakeholders on an “as needed basis” . All Business Stakeholders sign off on their particular issues and agree that QA can take over and start testing. This is the Final gate check for Development. Issue Tracker Status is marked <em>“QA”</em> by QA personnel.

Dev/QA process "swim lanes" - blended colors represent collaborations. click to enlarge.

QA Complete, Final Business Check

Time: Wednesday afternoon
The business stakeholder will view the corresponding implementations to their issues served from the QA machines. Any final defects may be addressed, or the Issue may be delayed for another cycle.
If all is ok, the Issue Tracker Status for QA complete is “Ready for Production”. Business has final say. 

Move Code to Head/Staging

Time: Wednesday 4pm
Release manager will take all issues marked “Ready for Production” and merge to the Production Staging Branch. If Staging exists, it is set to staging for final demonstrations, review and any business collaborations on an as-needed basis (sometimes there are outside dependencies that may need to have a “production-level” deployment to expose any final issues, or be observed by external stakeholders. This is the place to do this.

Last Minute Gut Check

Time: Thursday Morning
Final Business reviews. Go-No-Go meeting. Any last minute issues are addressed before launch. Afternoon begins with ITOPS/Release Coordination for Production and launch.

Release – OK, Let’s start at the top!

Time: Thursday, 2pm
Code is launched to production. The deployment must be watched throughout the afternoon for any production/emergency issues. Business stakeholders should review their changes on production sites. Any outside communication with external stakeholders is handled. Issues marked “Production” in Issue Tracker. Issues from previous week marked “Production” are now marked “Closed”.

The Three P Method: P1, P2, P3.

Below is an industry-standard practice of prioritizing issues with respect to their urgency. They are categorized as P1 – P3, the most urgent being P1. Each has impact to issues that are “already” in the queue for development, which must be taken under consideration.

• P1 — when a fix is needed to affect a serious break that will affect the company legally, morally, or for revenue impact over a set value (usually 50k or so). Everything is dropped and a release to production is done ASAP, outside of the schedule. Depending upon the impact, an upcoming release may be delayed for a week or certain issues may not be completed for the next scheduled release. A post-mortem is required to decide what the effect to the normal schedule is.A P1 fix usually requires a VP for “go-forward” approval. Often a meeting of stakeholders must occur to gather information with respect to impact and fixes.
• P2 — when a fix is needed to affect a break within the system. In this scenario, the break is fixed for the very next scheduled release — other issues being worked on may be affected, but no special release is done. Only in the rarest of cases is a Post-Mortem needed. Needs Director-level approval.
• P3 — anything within the normal schedule. Planned for one week, developed for a week, QA’d and released the following week.

The Overlapping Schedule Model Process within the Issue Tracker

The JIRA issue tracking system aligns workflow with Issues to allow for clean and simple end-to-end tracking of issues aligned closely to physical business processes. It employs a “status-transition” method to follow the process. Each “status” may have various “transitions” that allow for different scenarios. By clicking on a particular transition, the workflow will change to a corresponding status, and the path chosen will be reflected in the next available transitions available to the JIRA User.

The Diagram below shows the workflow available to the Team using JIRA and implementing the Overlapping Schedule Model. The workflow has been mapped out and attached to the Schemes available to the IHO. This model can be used or extended for other development channels. Note various paths that do not take a particular Issue into production, and other paths that have various conditions that must be satisfied to move forward. The “straightest” path to production is indicated by bold blue lines, showing the transitions and their corresponding status indicators.

Overlapping Schedule Model Workflow. click to enlarge

– Abraham Lincoln, upon appointing Grant to overall command of the Union Army

Constraints are things that we live with in our daily lives, accept them, and move on. With software engineering groups that are well-established and have strong leadership, constraints can get legs of their own and become excuses for not delivering projects on time, not building something correctly, or even stepping outside Enterprise Goals because “they do not fit within our design.” Many times the inability for an engineering team to confront a constraint will go so far as to create blame, cast dispersion and create a poison atmosphere to anyone that “gets in their way”.

If engineers faced no constraints, the Golden Gate Bridge would have been built out of Titanium, painted in gold leaf and had the busts of every designer on the pilasters of the side rails. Constraints in a Software Engineering environment can be obvious, like the language for the code to be written in, the database that will be connected, legacy systems integration points and the office environment that everyone writes their code. Less obvious (some would debate the “less” here!) constraints are deployment systems and hardware, delivery dates, Quality Engineering/Assurance windows, customer demographics, Marketing and Advertising purchase windows, integration dependencies from other teams, long-term maintenance of hardware and software, etc…

Consider a “Software Team” that is responsible for building high-transaction software for a large enterprise. This enterprise has instituted the standardization of all hardware within an infrastructure to allow for better purchasing, streamlined maintenance and quicker deployments. The benefits from a shared infrastructure environment are obvious from a cost-savings standpoint, and when the ITOPS group engages the teams building software (and vice-versa), hardware that meets the enterprise goals for both teams will be integrated into the final implementation. This software team did not participate, and further, believes that the ITOPS group has very little competence.

The “Software Team” purchases all of its hardware outside of the system and any products that need to integrate must do so on their terms. Often the kernels of the OS are different, requiring the ITOPS team to have additional resources and skills to maintain this completely separate branch of hardware and deployments. Since the “Software Team” is politically powerful, it gets what it wants. The company suffers because the arrogance of a Software Team and their inability to live within constraints causes other teams to pick up their slack.

Another constraint is to provide services that are consumable by all clients. If a team is Java-centric and only builds services that can be consumed by Java clients, this causes a great deal of difficulty for other teams within an organization that do not develop within a JVM. Soap, REST and other XML-RPC interfaces should always be considered first, along with other OS/Language independent methods. Groups that build products for themselves and do not collaborate with their client teams are not facing up to constraints, and will always deliver inadequate products that are late, incomplete and full of excuses.

Excuses. Excuses abound within a team that cannot face up to constraints. “Wha-happa-wuz” will be the first things heard by management, along with a list of “things out of their control” that affected why their product was late, broken, inadequate or some combination of all three.

Constraints are things that affect your product but are “out of your control”.

Just because you can’t build your bridge out of Titanium, or your clients don’t use Java, or your ITOPS team can’t buy you the latest T-2000 running Debian Linux, means that you have an excuse to deliver less than you promised. Careful planning, collaboration with internal and external stakeholders, and holding yourself responsible for what you say and do is the professional way of doing things. Here is a checklist of often-ignored constraints:

• Is the software being delivered in-line with the business stakeholders vision?
• Are you trying to force external resources into a particular technology?
• Can your product/application be supported on the existing/implemented hardware platforms?
• Has the schedule been set to allow for the usual delays in deployment, testing and configuration?
• Have all consumers or clients of your product/application been consulted and had a chance to fully comment?
• Is your team really the proper group to be building the software?
• Is your schedule realistic?
• Are non-engineering stakeholders like marketing and sales properly informed of all risks and deliverables?

This article is intended more as a “gut check” to an engineering team. If, as a team, products are consistently late, stuck in testing, not meeting expectations or hard to maintain, it is “gut check time”. It’s very easy to blame others; demand “impossiblities” of management and other groups and cast dispersions. It is a harder, but more successful approach, to look at what impedes the group, accept them as constraints and work within them.

Projects in general are date-driven, that is the business goals are to deploy something in a specified time-frame and with necessary features with reasonable quality. Planning around this is difficult but not impossible — often QA takes a back seat to development with respect to time-weight concerns, causing any delays in development to be thrown on the backs of QA teams, demanding that they complete the same amount of testing with less time. As development is delayed, the reluctance to accurately modify the schedule to a reasonable period decreases — stakeholders become more and more optimistic with their times, and consequently, any shock to the system will result in blown schedules and scrambling to “make the date”. This scrambling can bring on Involuntary Prototyping — when this happens, be prepared for a bear to eat your schedule and poop it off a cliff.

The Role of QA

The role of Quality Assurance (QA) in an enterprise is to use a planned and systematic approach to the evaluation of the quality of software product/applications, and their adherence to standards, processes, and procedures. QA warrants that standards and procedures are established and followed throughout a product/application life cycle.
Software development and control processes must include highly defined and published approval points, where a QA evaluation of the product may be accomplished in line with applicable standards. This involves collaboration with Development Groups through all stages of a product’s life cycle. Acceptance Testing provides a quantifiable gate for formally moving a product/application from the Development Phase to the QA Phase.
QA predicts their expected testing time with the assumption that they will get a Functionally Complete build from development that can pass a full Acceptance Tests (generally a build with all Designed Functionality included with zero or very few blocked areas of code). As projects get delayed because builds are not ready, resources become less and less available, and additional, serialized projects either have to wait or be taken on by smaller, ad hoc teams and offshore resources. Keeping offshore resources aligned with the onsite project teams requires complete documentation, plans and tight management. With at-risk projects, this is almost never the case.

Increased Project Risk Results in Process Failure

QA takes all projects at face value and goes to work. At-risk, large projects often force QA to take builds before they are complete in order to give them a fighting chance to make their release dates. These functionally incomplete projects soon become overwhelmed, and anti-processes form; reulting project delays become inevitable.

By testing for designed features and issuing, tracking and closing bugs that should have been caught before QA actual, mission-critical problems that must be found are deferred at best or just neglected, as the time for deployment draws near and the Product/Development teams’ launch pressures increase.
Delays in the QA process from a product with more-than-expected issues and compressed dates in guarantee a lapse into an “in for a penny, in for a pound” mentality. In the end the project will face delay, and all projects serialized behind it will be correspondingly backlogged in dramatic fashion, creating a potentially exponential expansion in delays as projects, dependencies and development resources are repurposed in a haphazard manner.

This inefficiency rolls up quickly. Initial show-stopping bugs trigger new builds. Builds are often weekly, causing resources to cycle between fully utilized to dormant depending upon their roles in the process. Risks also increase with respect to changes in configuration and regression.

Prototyping and its Pitfalls

If an incomplete product is sent through a Formal QA Testing Cycle, the result will be continuous builds, large numbers of bugs and issues, and further development/test cycles before a Release Candidate may be obtained. This process is known as “Prototyping”.

“A prototype is an easily modified and extensible model (representation, simulation or demonstration) of a planned software system, likely including its interface and input/output functionality. It is not a production-ready, fully-functional product ready for QA testing and deployment”

While prototyping is excellent for functional gap analysis and continuous improvement of an application under development, inefficiencies exist when a centralized QA team must test products from many different channels with limited resources. These resources will become overwhelmed when shocks to the system occur. These system shocks are identified as:

• Loss of resources or rapid turnover
• Increased production from development teams
• Increase in the number of products to deploy
• Geographical and cultural issues from off shoring
• Political issues and pet projects disrupting the schedule

As these shocks manifest themselves singularly, management must move to allow for scheduling issues, level resources and argue against artificial schedule changes. Delays are inevitable as resources are shuffled, and the output of the QA Team will degrade.

If these shocks come in as a group, management may become overwhelmed at the number of changes needed to be made and QA may have to place projects on “hold”, create an ad hoc triage process to test applications that may not be in alignment with business goals, and possibly lose all ability to test products for indefinite periods of time. Multiple shocks will also affect team morale and possibly create attrition issues and collisions of will between them and various Development/Business groups.
Prototyping most large organization is not deliberate. When a product is released to QA, the expectation of the Development/Business group is that Acceptance Candidacy has occurred and the product is to be tested and sent to production. When Designed Functionality is not met and major roadblocks are found, the product devolves into an Involuntary Prototype and the peculiar, often repeated and destructive dance of negotiation, multiple builds, huge bug backlogs and long delays commence.

Figure 1 — The Involuntary Prototype anti-process showing relaxed QE Acceptance standards and the cyclical nature of bug fixes and testing.

A Demonstrably Complete Product must be the Standard for Acceptance

To prevent Involuntary Prototyping during an application lifecycle, all products entering the QE environment must meet a minimum acceptable standard for initial testing:

• All Designed Functionality for an expected release must be met in an environment that is identical to, or certifiably mimics, the environment that QE will test the application in
• All Use Cases for the application must be documented, published to, and completely understood by QE stakeholders before initial Acceptance Testing.
• QA Environments must be understood, deployable and created.
• Product Managers and Development Leads must certify in writing that the product/application meets all Designed Functionality, with a checklist of all Use Cases tested.
• The expectations surrounding the schedule for QA testing and later deployment phases.
• All documentation, in a coherent and meaningful format, must be published to the QA group in advance of any Acceptance Testing so that QA may formulate its test plans.

This Acceptance Candidate, as defined above, is a functionally complete, verifiable, documented and demonstrable build that represents what the development group certifies as a potential Release Candidate.
QA must Reject Products that do not meet their Acceptance Threshold

QA must perform a full Acceptance Test before a Formal QA Testing Cycle may begin. It should be institutionalized throughout the entire organization that the mission of QA is to maintain corporate standards, and in a testing environment concentrate wholly on identifying problems with edge cases and border conditions, performance and difficult to find, non-obvious bugs.

A professional QA group is NOT in the business of identifying and logging Designed Functionality issues and sloppy development such as spelling errors, layout mistakes, non-functioning log-ins, network and connectivity issues. The responsibility for identifying and fixing the aforementioned issues must lie solely and completely with the Product/Development group that is responsible for the engineering of said product/application. Any product/application submitted for QA testing should be regarded as complete and demonstrably tested from the aspect of all Functional Design attributes.
Current development processes/groups may have various and disparate methods of engaging QA, and also have widely different views with respect to the role of QA within an organization. The engagement process and role QA changes in the an organization’s Development Doctrine and Executive Championing. The changes to the process should allow for most, if not all organizations to continue using successful, ingrained strategies, only needing to add QA touch points and additional demonstrations to ensure that all product/application stakeholders have met their responsibilities for delivering an application that is Functionally Complete and passing the required Acceptance Candidate Test.
The following items would be the expected minimum requirements for a product/application to be accepted for a full, Formal QA Testing Cycle.

• The Acceptance Threshold must be determined at project kick off and revisited before delivery to QA
• Full documentation of Use Cases and Deployment Diagrams must be available to QA resources for testing and configuration.
• Before Acceptance Testing begins, the Product Manager will verify the functionality of the product is complete through a Acceptance Candidate Test.
• Products not meeting the Acceptance Test will be rejected from QA and possibly lose their “window”, at the very least they will need to be re-entered into the testing queue.
• Disputes over Acceptance Testing, Rejecting and Product Acceptance will be mediated by the senior staff for the product and QA departments, with minutes and notes from the meetings sent to the VP-level staff for review.

These bullets should be easily integrated within most current development processes, and indeed many groups meet most, if not all, of the above responsibilities. The object is to formalize the process into the “DNA” of the Product/Development groups and give the QA team a “gate” and set of rules that allow for responsible and sensible pushback that allows for the entire organization to benefit, and reduce the bottleneck around QA entry and exit points.

The Complete Process as Integrated within the Development Cycle

Involvement with QA should begin as early as possible, and frequent touchbacks are a method of guaranteeing success.
As the Development Process is kicked off, documentation and expectations are transmitted to the QA group to communicate an understanding of the product/application and set a baseline for acceptance criteria. This documentation should include, but is not limited to, all design documentation, Use Cases, Product Requirements, and schedule expectations. As development continues, this document repository should be updated and extended as needs arise.
During the development phases, Product Managers and development stakeholders should get together on a pre-determined basis with QA resources as optional observers to demonstrate the features that have been added to the product and self-police any limitations or errors that have arisen at that point. Issues around any shortcomings in Designed Functionality should be created and addressed before the final Acceptance Candidate Test.
Once the development phase has reached conclusion, the Acceptance Candidate Test should be conducted, ensuring that all Designed Functionality, configuration, layout, spelling and craftsmanship are complete and up to industry standards. The Acceptance Candidate Test should guarantee that the product/application would pass the QA Acceptance Test either through the publishing of results or by direct observation from a responsible QA stakeholder. At this point the Product Manager and Development Lead verifies and guarantees in writing that Designed Functionality is complete.
Upon passing the Acceptance Candidate Test, the product is Ready for the QA Acceptance test. If the Acceptance Candidate Test has been properly conducted and communicated with QA, this test may be nothing more than a required formality. It also will allow QA to familiarize itself with the product/application, receive any training needed to test it correctly, finalize test plans and communicate any forward-looking concerns. If for any reason the QA Acceptance test fails, QA will reject the application wholly and the product will be sent back to development accompanied with a full report as to why the rejection occurred. This report shall be communicated to all stakeholders, managers and interested executives within the Product/Development group.
Figure 2 represents the integration of Development and QA processes to prevent Involuntary Prototyping and ensure that all Designed Functionality has been met before a Formal QA Testing Cycle begins.

Figure 2 — QE & Development Process Integration to prevent Involuntary Prototyping

Once the Acceptance Test passes, the Formal QA Testing Cycle begins. This cycle is similar, if not identical to, what the current cycle entails, except the burden of finding and logging Designed Functionality problems will be virtually non-existent, and issues that will cause full cycles to stall should occur with very low frequencies. Bugs will be logged and if required, new builds and testing cycles will occur as needed. Once this Testing Cycle ends, the product will be cleared for deployment.