Standards Best Practices Blog

So, your technical committee or work group has just completed development work on a specification. How do you go about approving the work and releasing it into the wild? Obviously you need some sort of approval process, and obviously that would include approval by the people who developed the specification. But is that enough?

Most standards organizations have multiple, usually two, levels of approval within the organization, There’s a good reason for this: the two levels of approval combined, within the committee that developed the work and then at the organizational level, provide a good level of oversight and allows all members to participate in the development and/or approval at whatever level interests them.

Some smaller, less formal organizations may not see the need for multiple levels of approval. Isn’t the approval within the developing committee sufficient? Perhaps, but there’s a few problems with doing this. First and foremost, any group of people who develops something will always approve their work once they’ve completed it. (Have you ever heard of a group putting a lot of time and effort into something then, when it is completed, deciding that it’s not any good?) A second opinion is needed.

The second opinion also brings the opportunity to look at a work from a different perspective. The developing committee should have been working from a specific set of technical requirements and developed a technical solution to that problem. But the wider organizational membership may have a different and broader set of requirements or criteria, perhaps more related to market needs, adoptability, political issues, etc.

Furthermore, if the specification is to carry the organization’s name, calling it e.g. “An XYZ Standard,” shouldn’t the entire organization have a say in its approval? And while the membership of the entire organization may have had the opportunity to participate in the developing committee, perhaps they weren’t interested or competent enough to do so.

Having multiple levels of approval also provides the opportunity for different criteria for voting eligibility. For example, voting in the development committee would generally be one vote per person, as it is individuals who are doing the development work regardless of who they are working for. But at the organizational level voting would be one vote per member company.

For purposes of ANSI requirements – or for any other organization that seeks to maintain balance in its approval process – membership in the consensus body, i.e. the body giving the final approval, must be balanced by interest category, e.g. manufacturer, vendor, implementer, user, or academic. It is sometimes difficult to get balance among participants who are interested in developing a specification; these are usually technical people from manufacturer or vendor members. So having two levels of approval allows the development committee to be dominated by technical people, while the final level of approval is given by a membership balanced among all types.

An issue that needs to be considered in bi-level voting, however, is when not enough of the entire organizational membership is interested or competent enough in a particular area to vote one way or the other. The organization may work on a variety of topics that, while somehow related, are not related enough to be of interest to everyone; any particular organizational member may be interested in some but not all of the specifications being created.

There are two ways to handle this situation. The first is to have a low threshold for approval. For example, the organization may only require that one quarter or one third of the membership approve the specification, as long as there are more affirmatives than negatives. Another is to define voting groups, where members declare their interest in a topic at the beginning of the work, and only the votes of those members are used to approve the work at the end.

With either of these solutions there is going to be the problem that it is only some small portion of the membership speaking on behalf of the entire organization, but the alternative is to require that members vote on something that they have no interest or competence in.

Several days ago I made some suggestions for improvement of the processes used for advancement of standards to international approval. In various blog articles I’ve been critical of the practice of stacking, where membership in a committee or other voting body suddenly increases just prior to an approval vote, and have suggested restricting the right to vote to those who have been involved in the committee for some period of time.

But on the other hand, I’ve also been a very vocal proponent of the principle of openness, defined in the U.S. Standards Strategy as “Participation is open to all affected interests,” and consensus, defined as “Decisions are reached through consensus among those affected.”

Am I contradicting myself? Is it a contradiction to suggest that all interested parties should be allowed to participate, yet participants should not be allowed to join at the last minute? What if they are interested in the topic and have something to say?

Let me suggest some reasons why this is not a contradiction, and why allowing last-minute voting eligibility is not a good idea.

First, if a person or group is truly interested in the topic or the work being done they probably would not have waited until the last minute to join. For committees at the development level, once the specification is completed and ready for vote it’s too late to be making new contributions. For example, a vote against a specification because it does not fill a particular requirement would have been prevented by that requirement having been communicated at the start of the development process. And for committees or bodies at higher approval levels, some amount of background or familiarity with the topic is required in order to make an educated decision.

In many cases the specification or other work under consideration may be of some length or technical complexity; it takes time to read and understand the specification, and having participated in the development of the work will help a participant understand how and why the specification defines things as it does. For levels of approval post development, a potential voter should have been involved long enough to be acquainted with the specification during its development and/or reviews. A considerable amount of work and discussion goes into the creation of most specifications; if the new participant isn’t acquainted with this discussion they may not have knowledge of all the issues involved.

At the very least, granting the same voting rights to someone arriving at the last minute as those given to someone who has been working on the topic for years devalues the contributions of the latter. In some respects a participant should have to prove his interest through the investment of effort over a period of time, either by participating in the development of the work or in taking the time to read and study the specification, participate in reviews of the work, or becoming acquainted with the issues related to the work.

The organization should be aware of the possibility that a late-joining voters have some hidden motivation. Are they, for example, participating in a scheme to “buy” votes? Granted, all participants have motives for participating in standards work; they participate because they seek to gain something from that standard, but generally that is accomplished by steering the direction of the work through participation in its development or reviews. A party that has no interest in a standard’s development but only in its final approval or disapproval may be suspect.

Finally, late joiners can cause problems within the committee going forward, even outside the scope of the standard being voted on. Most committees operate under a process that requires quorum before doing any sort of business or making any decisions, and if large numbers of new members join simply for a specific vote then drop out of sight after the vote, their continued presence on the membership roster of the committee will make it difficult or impossible for the committee to achieve quorum.

So I’ll suggest again that voting rights should be reserved for those who (in addition to other eligibility requirements the organization may have) have been members of the committee or group long enough to be acquainted with the work in question, and have been involved in enough of the discussions to be aware of the issues related to the work.

I wrote in January about ITU’s experiment in allowing free downloads of their standards in hopes of promoting adoption. Yesterday ITU announced that the program has been deemed a success.

Not only has the program resulted in a greater awareness of ITU work, but the no-charge policy is helping promote adoption in developing countries. Malcolm Johnson, Director of ITU’s Telecommunication Standardization Bureau (TSB) reported that “Last year exactly 500 ITU-T Recommendations had been sold to developing countries; this year, after allowing free access, they have downloaded some 300,000.”

ITU is to be congratulated for taking this step. For an organization that has gained a fair amount of revenue from the sales of standards, making these freely available was taking a big risk. But ITU has shown that the benefits of broader adoption, especially in markets that may have been discouraged by the price of purchasing the standards, is worth the change.

I read an interesting editorial at Wired.com by Bruce Schneier discussing single points of failure, using the recent NBA referee betting scandal as an example. Schneier suggests that a small number of players and a reliance on judgment calls by trusted individuals rather than quantitative measures put the system at risk. Vulnerabilities can exist in any system where a person has authority or access to privileged information, including referees, police officers, judges, TSA employees, stock brokers, or even computer repairmen.

There have been articles in the press lately suggesting that the greatest threat to enterprise security is from within; employees have access to company data and physical property, and can cause much greater damage than someone from outside. Think of the damage that could be inflicted on a company by a disgruntled IT sysadmin. (This is not new; military conquests going back to the dawn of recorded history have used traitor insiders to grant them access to a city when a siege had turned to a stalemate. And when a traitor cannot be found, a Trojan Horse can always be introduced.)

What has this got to do with standards? Plenty.

Standards activities are based on and assume a desire by all parties involved to work for the common good. While the participants each have their own motives – usually involving increased revenues for their company – they recognize that this is best achieved by working together to grow the size the pie through developing a standard around which they can all build products.

A great deal of trust is placed in small number of people in positions of authority, including committee chairs and process administrators. Participants should be able to trust other participants to play by the rules, make fair IPR disclosures, do their share of the work, and work for the good of the project. Participants should not need to be monitored closely, nor should anyone have to worry about watching their backs.

But we can’t always count on such a Pollyanna-ish situation; a rule book and a referee are required before a sandlot game can turn into the major leagues. Here’s a few things that should be done:

• Select committee chairs that are professional, fair, and technically competent, that can be relied on to provide leadership for the committee and follow the rules. The chair should have the ability to take off his or her company hat and act as a neutral party for the benefit of the committee’s work and the standards organization.
• The same goes for the organization’s standards administrator, only more so. This person monitors the activities of the organization’s committees to make sure that the rules are being followed. The administrator must know the rules and how to enforce them. Judgment calls must occasionally be made, so the administrator must be capable of making these in an impartial and consistent manner.
• Of course none of the above is possible if the organization does not have a good set of rules, the committee process document. It’s fine to work under the assumption that everyone will play nice together, but what happens when they don’t, or when some controversy arises? The standards administrator cannot enforce rules that don’t exist, and making it up as you go is not a good policy.
• The rules need to include an appeals process. What happens when a participant disagrees with something that the chair did, or the standards administrator? Who can she appeal to, and how? Who settles the dispute, and how?
• The organization needs to be very careful about fully documenting the standards development and approval process including archiving all email discussions, versions of documents, and ballots, and taking and retaining meeting minutes. If a question arises about how or whether something was done, whether as part of an appeal or audit or simply for a participant’s curiosity, historical records can save the day.

Doing the above will save the standards organization from the points of failure inherent in a system based on trust. Having competent leadership and a good set of rules will go a long ways towards ensuring the success of the standards activity.

If you’ve been following the progress of OOXML through Ecma and the fast track approval process at ISO you know that the approval vote over the weekend failed. There’s been a large number of bloggers tracking and commenting on the issue, so I don’t need to go into the details. Over the past few months I’ve contributed a few entries related to process in keeping with this blog’s theme of best practices; in this entry I’d like to wrap it up a bit.

The potential for ISO approval of OOXML isn’t over quite yet; there is still the possibility that the hundreds of submitted comments can be resolved at the February ballot resolution meeting and sufficient numbers of “No With Comment” votes be changed to “Yes,” but it sounds unlikely at this point.

So now that the game is (almost) over, it’s time to grab the popcorn and start the post-game analysis. Specifically, how did all this controversy happen, and what can or should be done to improve the situation the next time around? I suppose that depends a bit on whose camp you’re in; some may say that the process should be loosened to allow a submitted work to sail through the ISO approval process without so much fuss, but I am of the opinion that doing so would cheapen the value of ISO standards and the idea of standards in general.

The fast track process at ISO allows work created outside of ISO committees to be submitted to ISO for approval under its process to become an International Standard. This process recognizes that not all “worthy” work is being done at ISO but can come from any of a number of sources including consortia and other industry organizations. This is a good thing, in my opinion. I’ve been involved in doing this myself; in my work at OASIS we took advantage of this process and submitted OASIS-created and approved work to ISO and ITU for approval.

However, ISO has allowed a loophole in that, while work created under its own process must be created using the accepted principles of openness, balance, due process, transparency, etc., work it accepts from elsewhere has no such requirement. This is proved by the fact that a specification, in this case OOXML, created by a single vendor under a closed process without input from any other interested parties, was able to be accepted by and then submitted to ISO by an organization, in this case Ecma, which does not require an open process.

After submission of the specification to JTC1 it is then sent to the national bodies for six months for their review and vote. The first month is allowed for submission of “contradictions” by national bodies, but this term isn’t well defined. In the case of a contentious issue such as OOXML this caused a great deal of confusion with the various camps, pro and con, trying to use the undefined term to their advantage.

The following five months are allowed for the national bodies to comment on the specification and determine how they were going to vote. The national bodies themselves are allowed to determine which comments they will submit and how they will vote, but in the case of OOXML there were numerous allegations of vote stacking, disenfranchisement, and mismanagement of the voting process. While this is really the national bodies’ business, ISO and JTC1 should be concerned about how the vote is achieved.

If and when ISO and its JTC1 committee decide to re-evaluate their fast track process, and I hope that they do, here’s a few of my suggestions for them to consider:

• Specifications submitted for consideration for fast-track approval, must at their source have been developed under a process that follows the same principles of openness, balance, due process, transparency, etc. that the ISO process is based upon. And further, the organization submitting a specification to ISO must show that its approval and advancement of the work was done following those same principles with sufficient input and further development to show that the specification is an open work – in other words, no rubber stamping.
• Terminology used in the fast track process must be well defined.
• As part of the fast track process ISO and JTC1 should establish rules regarding how national bodies decide their vote, including for example that voting eligibility requires membership from the beginning of the comment period or for some significant period of time before the vote.
• To avoid the last-minute stacking of JTC1 itself, last-minute upgrades of memberships from non-voting to voting should not be allowed; only countries who were voting members at the time of the submission and during the entire six-month review period should be allowed to vote.
• Allegations of mismanagement or disenfranchisement will be investigated and if found true could result in loss of eligibility for some future period of time.

A voluntary consensus standards process works quite well when all parties are interested in cooperatively developing and approving work for everyone’s benefit. Many organizations have less-than-robust processes because they operate under the belief that everyone will work together for the common good. In most cases this is true. But even well thought out processes will have weaknesses and loopholes that are only discovered under extreme situations. In the few cases where the work becomes contentious, when market position and large amounts of money are at stake, it is wise for the organization to have a robust process that will provide a defense against parties trying to game the system.