Some clarifications on the OOXML Ballot Resolution Meeting
I have been one of the two Greek delegates to the OOXML Ballot Resolution Meeting (BRM). Lots of things have already been written about the meeting. I will not repeat them here, but will only make a few clarifications on things that I think are not well understood. If you want a general account, I think that Tim Bray's is the best so far.
The opinion of the BRM
First, it is important to clarify that the BRM did not say either that the specification is OK, or that it is not OK, because it is not within its competence to say such a thing. There was good co-operation, and, to a large extent, good will, from all sides, because the BRM has a single purpose: make the specification better. Let me repeat that: the BRM has the single purpose of making the specification better, so that if it is accepted, it is the best possible. Or, the same thing viewed from another viewpoint, the BRM attempts to make it better, to maximize its chances of going through. Well, this is in theory, of course; in practice some want to maximize its chances of not going through, and I'm certain that it's not the first standard in which this happens.
The BRM, therefore, has made no statement that the fast-track process was appropriate, nor that it was inappropriate. No claim that there was enough time to address the issues, nor that there was not. No recommendation to approve the outcome, or to not approve the outcome. The BRM only provides an outcome, implying nothing more than the fact that these are the improvements that we were able to make within the given time frame.
You don't need to take my word for that: you can read ISO's description of the objective of the BRM and Alex Brown's clarification on this issue (Alex Brown was the chairman of the meeting).
The paper ballot
Lots of things have been written about the paper ballot. Andy's blog entry, which has been the centre of attention, was one of the posts that discusses the paper ballot very much. I told Andy that, in my opinion, he overinterprets the paper ballot, which is not such a significant issue. Let me explain from the start, however, because it is quite complicated.
After the national bodies submitted their comments in September 2007, Ecma studied and proposed changes to the original six-thousand-page proposal in order to address these comments. Ecma's proposed changes, called Disposition of Comments, was submitted to the national bodies on 14 January 2008. It is a 2300-page document containing about one thousand proposed changes. However, Ecma does not have the power to make changes to ISO DIS 29500; only the BRM has such power. Therefore, Ecma's changes would not make it in unless approved by the BRM.
It became clear soon enough that there was no time to discuss all one thousand of Ecma's proposed changes, but at the same time it was thought that, if they constitute improvements to the original document, it would be a pity to disapprove them for lack of time. Therefore, the BRM attempted to find a solution to the problem. Several options were discussed, and the option that gained most support was for each national body to fill in a form like the following:
For all ECMA dispositions which have not been explicitly accepted or rejected
in the meeting, we want to
Approve [ ] Disapprove [ ] Abstain [ ]
Except for the following:
Response ... Approve [ ] Disapprove [ ] Abstain [ ]
Response ... Approve [ ] Disapprove [ ] Abstain [ ]
Response ... Approve [ ] Disapprove [ ] Abstain [ ]
Response ... Approve [ ] Disapprove [ ] Abstain [ ]
Response ... Approve [ ] Disapprove [ ] Abstain [ ]
Initially it was thought that, e.g. 1 vote in favour for a specific response, and 31 abstentions, should probably mean that it is somewhat extreme to consider as in favour. But later it was pointed out that if, for example, a specific response affects one country only, that country's opinion should weigh heavily. Examples that were discussed include right-to-left writing, which none in the room understood besides Israel; and some measurement units, which few people besides UK and Australia had a grasp on.
After considering several options, such as distinguishing between "abstain" and "no position", which all proved to be infeasible, it was decided, on Wednesday afternoon, with an overwhelming majority of 29 votes in favour, that the paper ballot would be conducted, and that a simple majority of Approvals, without counting Abstentions, would mean approval. The paper ballot affected only Ecma Responses that had not explicitly been addressed in other ways during the meeting.
So, from the 80 or so responses that Greece had studied, we specified Approve on something like 70 of them, Abstain on the others, and Abstain on the 900 or so others that we had not studied. I believe that most countries voted similarly to Greece.
(The "default" option, besides Approve, Disapprove and Abstain also had "we do not wish to record any position", which for practical purposes is the same as Abstain.)
The way the paper ballot was conducted, especially that 1 approval and 31 abstentions counted as approval, was an issue of course, but national bodies were aware of it. There was no solution that did not have problems, and the national bodies chose to follow this one. Canada attempted to work around the problem by posting a short list of responses which they considered worse than the original text, encouraging countries to vote "no" to those instead of "abstain". Greece, however, did not manage to study Canada's list.
The fact that 98% of Ecma responses were adopted means that the BRM believes that they improve the original text. It does not mean that the BRM believes that the resulting text is OK. My opinion, for example, and many delegates agree with me, is that the Ecma responses make the text slightly better, but though slightly better it is still abysmal. But since they were an improvement, we adopted them.
The success or failure of the BRM
Brian Jones and Jason Matusow of Microsoft have said that the BRM was a success because it fulfilled its purpose, which was to make changes to the text. Although this is technically correct, if the original text got 1 out of 10 and the BRM managed to improve it to 1.1, it is somewhat misleading to call it a success. Brian Jones says that there was consensus in the changes. This is also true, but the reason there was consensus was that we quickly became disillusioned, lowered our standards, and only discussed modifications which we knew could pass within the given constraints. Let me give an example.
One of the issues brought up by the Greek delegation was that of Office Open Math ML (OOMML). Given that the purpose of OOXML is to faithfully represent the existing corpus of legacy documents, and OOMML is an entirely new language that has nothing to do with such backwards compatibility, we were at a loss as to why it was designed from scratch rather than being based on the existing standard of MathML. Ecma's reply was that OOMML has features that MathML does not.
It took almost an entire week of discussions in the corridors and in emails to sufficiently clarify the issue. During our investigation we learned that Office 2003 documents converted to OOMML by Office 2007 retain their equation fields as equation fields, and equation editor objects as embedded binary objects; you can use OOMML only in new equations. After some discussion with Rick Jelliffe, Australian delegate, on the extensibility of MathML, we figured out that you can extend MathML, although the resulting extended language will not be MathML anymore. There were reservations by Canada because extending MathML might break existing MathML accessibility tools, and there was the argument, on my side, that it is easier to fix the accessibility tools for an extended MathML than to fix them for an entirely new language.
The purpose of the BRM was not just to discuss things in theory, but also to propose specific resolutions. What resolution could be proposed on this issue? "Redesign OOMML on top of MathML" was not realistic, not only during the BRM, but also for the time left for Ecma to make the final changes in March. Therefore, the only realistic resolution could be "drop OOMML" entirely, since the old ways of writing equations (equation fields and equation editor) are unaffected, and MathML had also been added (although without extensions). Would that, however, gain enough support?
While the chairman was reviewing remaining issues on late Friday, he asked Greece about it. Greece replied that we are still uncomfortable about OOMML, but that there is no time to resolve the issue. Therefore, we removed it from the agenda. The time was 15:45.
Another issue that came in late was that ISO 29500 should be backwards compatible with Ecma 376. I first heard this by Brian Jones on Thursday, during lunch, and on Friday morning it was made clear to the BRM that this was an issue. This, of course, was an entirely new argument, which affected everything. The BRM simply did not have time to discuss whether we want this compatibility or not. It was around 16:30 on Friday when the BRM decided to add the clarification "Microsoft Office 97 to 2008" to the Scope, where it says about the existing corpus of MS Office documents. Now although the "97" was undisputable, it was not clear that the BRM wanted "2008" rather than "2003". It was clear to everyone, however, that there was absolutely no time to discuss about that, so when Greece proposed the "97 to 2007" it went to vote after minimal discussion and only the technical correction of replacing 2007 with 2008 (the Mac OS X version of MS Office 2007).
Incidentally, while researching the equations, an issue of contention was the phrase "MathML renderers are allowed, but not required, to accept non-standard elements and attributes," in the MathML specification. This did not make much sense, and it took considerable effort and several email exchanges, before David Carlisle, MathML editor, clarified in an email to Rick Jelliffe that this phrase appears to be an error (although the rest of the section (2.4.2 in MathML 3.0, 7.3.2 in MathML 2.0) appears to be ok). The reason I'm mentioning this is that it shows how a slight carelessness in a minor detail of a standard can create trouble, and all of us who have used standards know that all too well, because no standard is perfect.
The contrast with OOXML is sharp, and this brings us to another issue of contention. The Greek workgroup on OOXML had been handed only the Ecma Responses for Greece. It was at the BRM when we found out that we should have studied all responses, not only those for Greece. It is not clear if this is an error by Ecma or by the Greek NB, but, in both cases, we did not have the time to study one thousand responses, so there would have been no difference. In fact, even the 80 responses that Greece studied, we did not study at the level of scrutiny that is required when you inspect a standard. There was no time for that. What we did was glance through, and make fast decisions based on what seems right at a quick glance.
Dates
The date issue highlights several other problems. The original text proposes to store and manipulate dates by using two different representations, i.e. number of days from either 1900 or 1904, which contain a deliberate error for backwards compatibility: 1900 is considered to be a leap year. Other problems that were pointed out were that specifying dates before 1900 is not possible, that there is no reason to have two representations when one is enough, and that new ways of storing dates should not be invented since we already have ISO 8601. In the Disposition of Comments, Ecma proposed to add two additional ways of representation, which solved some of the problems but complicated the problem too much.
Before the BRM, I had prepared an alternative proposal on dates, which is much cleaner. There are three issues that demonstrate how the whole process was affected by the time constraints.
The first issue is that my proposal for dates, despite the fact that I had written it with care in the course of a few weeks, and that it had been reviewed by several people, did have shortcomings. By comparing my work to the work by Ecma and Germany I found out that I had failed to notice that data types were described in two places, 3.17.2.6 and 3.18.12, and I had only noticed 3.17.2.6. And while thinking about the whole issue I saw that the way I had proposed for handling durations probably did not work (although now I'm having second thoughts - but I'll need very careful studying of ISO 8601 to arrive at a good conclusion). Both problems were easy to fix, and I submitted an updated proposal on Friday morning. However, this shows that it was very hard to write error-free proposals when the magnitude of changes were so large.
The second issue is related, but more important. When I discussed my proposal with Brian Jones on Thursday morning, he pointed out that it would be difficult for Ecma to accept it, because they did not have the time to verify that it actually works in all cases. Now this was a very valid concern. My proposal was more than 30 pages. Even if it were well thought and error free, Ecma had no way of knowing that. Therefore, the BRM was essentially confined to making changes that only scratched the surface of the problems.
The third issue is that, while writing my proposal, I and my reviewers found 13 additional errors in the original specification. However, national bodies were not allowed to submit new comments (and rightly so, otherwise there would have been total chaos). Therefore, there was no way to submit and correct them.
The changes that actually passed in Friday morning did add the possibility to store dates in ISO 8601 format, but they also keep the old ways, and in addition they add all ways proposed in the Disposition of Comments. Therefore we now have five different ways of representing the same thing.