OASIS Open Document Format for Office Applications (OpenDocument) TC
  1. OASIS Open Document Format for Office Applications (OpenDocument) TC
  2. OFFICE-3740

4.8.14.2 ODF 1.2 Requiring <manifest:manifest> manifest:version breaks downlevel and early 1.2 implementations

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: ODF 1.2
    • Fix Version/s: ODF 1.2 Errata 01
    • Component/s: None
    • Labels:
      None
    • Environment:
      This defect applies to ODF 1.2 Part 3 since Committee Specification 01.
    • Proposal:
      Hide
      In the manifest schema, make the manfiest:version attribute optional for <manifest:manifest>.

      In 1.2-3 4.8.14.2 Change "The manifest:version attribute ..." to begin "The optional manifest:version attribute ... ".
      Show
      In the manifest schema, make the manfiest:version attribute optional for <manifest:manifest>. In 1.2-3 4.8.14.2 Change "The manifest:version attribute ..." to begin "The optional manifest:version attribute ... ".

      Description

      In ODF 1.2-3 4.8.14.2, the manifest:version="1.2" attribute is mandatory on <manifest:manifest> elements. This attribute provision was introduced in ODF 1.2. There were no manifest:version attributes for the <manifest:manifest> attribute in ODF 1.0 and ODF 1.1.

      The presence of this attribute prevents ODF 1.1 and earlier implementations that expect strict honoring of older <manifest:manifest> schemas from accepting ODF 1.2 documents for potential down-level acceptability.

      In addition, documents identified as ODF 1.2 documents produced before the provision was added to the ODF 1.2 specification will now be declared as non-conforming by document validators.

      The Catch 22 consists of the fact that expecting the attribute will invalidate previous documents that were identified as ODF 1.2 documents and that producing the attribute will cause error messages (at least) in down-level use of documents that may well have no specific dependency on material ODF 1.2 provisions whatsoever.

      The provision is too brittle and causes more problems without solving very many.

        Activity

        Hide
        Andreas Guelzow added a comment -
        Regarding the PPS in the last comment: So the documents were created with a buggy implementation .Are you suggesting that there is any special weight be given to the way that implementation interprets or uses the standard?
        Show
        Andreas Guelzow added a comment - Regarding the PPS in the last comment: So the documents were created with a buggy implementation .Are you suggesting that there is any special weight be given to the way that implementation interprets or uses the standard?
        Hide
        Dennis Hamilton added a comment - - edited
        @Andreas,

        It wasn't a bug at the time the producer used was released.

        Or put differently, the bug was to claim that ODF 1.2 was being produced, when ODF 1.2 had not even achieved Committee Specification. That is not exactly a bug.

        Furthermore, the users who produced the documents did the usual thing of using recent versions of the producer implementation, and did so in some sort of happy disregard for ODF 1.2 (extended!) being the default output option when ODF 1.2 was not baked yet.

        This is a real-world situation. I am simply pointing out that it arose here. There are now many such documents in the world. There was a time when validators would report that those documents are fine and that the appearance of the <manifest:manifest> manifest:version was in error. Now the later validators make the opposite determination. Clearly, validators are easier to adjust and to have them make warnings about such small deviations, rather than grand statements about the entire document based on this one string of markup.

        Similarly, the implementation of the ODF 1.1 consumer in Office 2007 in its performance of strict validation on documents did not encounter any difficulties accepting documents from these other producers until the manifest::version started showing up. It appears there are several cases in the world:

         1. There are 1.2-identified producers that do not produce it and whose consumers are indifferent to its presence. How those consumers respond to material differences in the manifest.xml is determined by other means. (It appears that Office 2013 Preview is in this category. Older versions of OpenOffice-lineage producers are also.)

         2. There are 1.2-identified producers that produce it and whose consumers are apparently tolerant of its absence. These will respond to other ODF 1.2-introduced manifest provisions depending on whether they are supported in that consumer or not. (Relatively-recent OpenOffice-lineage 1.2 producers appear to be in this category. Their consumers can still fail badly on provisions of the manifest that are not supported in all but the latest versions, e.g., in failing to recognize non-default encryption options.)

         3. There have been ODF 1.2-identified validators that accept and others that reject the attribuite, but most now require the attribute. (These are the easiest to adjust, it seems.)

         4. There are downlevel consumers that are indifferent to its presence and will succeed or not depending on the presence of other features..

         5. There are downlevel consumers (OpenOffice 2007 and 2010 being notable) that require manifest.xml to satisfy the ODF 1.1 manifest schema. That is not a bug.

         6. In terms of *documents*, there are produced-as-ODF-1.2 document in the world that have the attribute and others that don't have the attribute and presence or absence of the attribute apparently tell us nothing about whether or not there is anythng in the manifest that requires ODF 1.2 provisions to be consumed properly.
        Show
        Dennis Hamilton added a comment - - edited @Andreas, It wasn't a bug at the time the producer used was released. Or put differently, the bug was to claim that ODF 1.2 was being produced, when ODF 1.2 had not even achieved Committee Specification. That is not exactly a bug. Furthermore, the users who produced the documents did the usual thing of using recent versions of the producer implementation, and did so in some sort of happy disregard for ODF 1.2 (extended!) being the default output option when ODF 1.2 was not baked yet. This is a real-world situation. I am simply pointing out that it arose here. There are now many such documents in the world. There was a time when validators would report that those documents are fine and that the appearance of the <manifest:manifest> manifest:version was in error. Now the later validators make the opposite determination. Clearly, validators are easier to adjust and to have them make warnings about such small deviations, rather than grand statements about the entire document based on this one string of markup. Similarly, the implementation of the ODF 1.1 consumer in Office 2007 in its performance of strict validation on documents did not encounter any difficulties accepting documents from these other producers until the manifest::version started showing up. It appears there are several cases in the world:  1. There are 1.2-identified producers that do not produce it and whose consumers are indifferent to its presence. How those consumers respond to material differences in the manifest.xml is determined by other means. (It appears that Office 2013 Preview is in this category. Older versions of OpenOffice-lineage producers are also.)  2. There are 1.2-identified producers that produce it and whose consumers are apparently tolerant of its absence. These will respond to other ODF 1.2-introduced manifest provisions depending on whether they are supported in that consumer or not. (Relatively-recent OpenOffice-lineage 1.2 producers appear to be in this category. Their consumers can still fail badly on provisions of the manifest that are not supported in all but the latest versions, e.g., in failing to recognize non-default encryption options.)  3. There have been ODF 1.2-identified validators that accept and others that reject the attribuite, but most now require the attribute. (These are the easiest to adjust, it seems.)  4. There are downlevel consumers that are indifferent to its presence and will succeed or not depending on the presence of other features..  5. There are downlevel consumers (OpenOffice 2007 and 2010 being notable) that require manifest.xml to satisfy the ODF 1.1 manifest schema. That is not a bug.  6. In terms of *documents*, there are produced-as-ODF-1.2 document in the world that have the attribute and others that don't have the attribute and presence or absence of the attribute apparently tell us nothing about whether or not there is anythng in the manifest that requires ODF 1.2 provisions to be consumed properly.
        Hide
        Dennis Hamilton added a comment -
        After the call, I dug further into my collection of document-forensics samples.

         1. With regard to ODF 1.2 producers that do not provide <manifest:manifest> manifest:version, but whose consumers evidently ignore the presence of such an attribute, that includes OpenOffice.org 3.3.0 and IBM Lotus Symphony 3.0.1. These are the last stable releases of those products. OpenOffice.org 3.3.0 was used to produce the ODF 1.2 Committee Specifications and the final OS authoritative documents.

         2. With regard to ODF 1.2 producers that do produce it but succesfully consume documents where it is absent, I can confirm that OpenOffice.org ooo-dev 3.4 (a "beta" that did not achieve stable release) produces <manifest:manifest> manifest:version="1.2" and consequently, so do all Apache OpenOffice releases so far. I know LibreOffice 3.3.2 is in this category (3.6.2 is latest) and I suspect nearly all LibreOffice releases have been in this category.

        NOTE 1: This is different than the occurence of <manifest:file-entry> manifest:version="1.2". More products produce that *optional* attribute, especially on the file entry for content.xml, but apparently no one is bothered by its absence.

        NOTE 2: With all of this fuss, there is still no explicit warning when a package is an extended package (trivia question: what makes an ODF 1.2 package extended?) or when a document is an extended document.

        NOTE 3: I think it is important to keep in mind that <manifest:manifest> manifest:version is about the manifest (i.e., those things specific to packages), and should not be treated the same as office:version, which is about the document. In that respect, <manifest:file-entry> manifest:version seems to be misnamed. It also seems more appropriate to have it on the file-entry for "/" rather than content.xml.
        Show
        Dennis Hamilton added a comment - After the call, I dug further into my collection of document-forensics samples.  1. With regard to ODF 1.2 producers that do not provide <manifest:manifest> manifest:version, but whose consumers evidently ignore the presence of such an attribute, that includes OpenOffice.org 3.3.0 and IBM Lotus Symphony 3.0.1. These are the last stable releases of those products. OpenOffice.org 3.3.0 was used to produce the ODF 1.2 Committee Specifications and the final OS authoritative documents.  2. With regard to ODF 1.2 producers that do produce it but succesfully consume documents where it is absent, I can confirm that OpenOffice.org ooo-dev 3.4 (a "beta" that did not achieve stable release) produces <manifest:manifest> manifest:version="1.2" and consequently, so do all Apache OpenOffice releases so far. I know LibreOffice 3.3.2 is in this category (3.6.2 is latest) and I suspect nearly all LibreOffice releases have been in this category. NOTE 1: This is different than the occurence of <manifest:file-entry> manifest:version="1.2". More products produce that *optional* attribute, especially on the file entry for content.xml, but apparently no one is bothered by its absence. NOTE 2: With all of this fuss, there is still no explicit warning when a package is an extended package (trivia question: what makes an ODF 1.2 package extended?) or when a document is an extended document. NOTE 3: I think it is important to keep in mind that <manifest:manifest> manifest:version is about the manifest (i.e., those things specific to packages), and should not be treated the same as office:version, which is about the document. In that respect, <manifest:file-entry> manifest:version seems to be misnamed. It also seems more appropriate to have it on the file-entry for "/" rather than content.xml.
        Hide
        Andreas Guelzow added a comment -
        I think we had part of this discussion before: If an implementer chooses to implement a draft committee specification and to ship that implementation (rather than to simulate the new features using a different name-space) of course there are problems arising if the draft changes before it becomes a standard.
        Show
        Andreas Guelzow added a comment - I think we had part of this discussion before: If an implementer chooses to implement a draft committee specification and to ship that implementation (rather than to simulate the new features using a different name-space) of course there are problems arising if the draft changes before it becomes a standard.
        Hide
        Dennis Hamilton added a comment -
        @Andreas,

        I agree that speculative upgrade to ODF 1.2 before the specification was baked was dangerous.

        Although that accounts for there being all of these variations in the wild, they are now in the wild and the documents that originated with them are there too. I don't think it is fair to penalize users for this.

        Also, I think it is important to understand that the breaking case is with regard to down-level consumers that were not implemented based on any speculation whatsoever. That is the focus of my attention with regard to this specific issue. This is the oversight that was made when <manifest:manifest> manifest:version was introduced and made mandatory. It just didn't receive the analysis that has since arisen out of hind-sight and recognition of what all the cases in the wild have turned out to be. Instead of fostering interoperability, the outcome has been a reduction of interoperability. It appears, in fact, that the attribute is consistently ignored on consumption by those processors that produce it and also many that don't produce it.

        More generally, I think the discussion on the list concerning ways to have provisional implementations (not using ODF-official namespaces) that are for features with reserved definitions in future releases of ODF is important. This would have made the handling of implementation-specific digital signature extensions in ODF 1.1 documents more transparent and it will help with anticipated features for ODF 1.3 that are related to document privacy and integrity.

        Downlevel disruption, even among implementations of the same ODF version, will still happen, of course. (One case now in existence is the change of what encryption is used by default, causing older implementations to fail. In this case, <manifest:manifest> manifest:version="1.2" has been no help because it was being produced either way. Although that ends up relying on published standards, the result is not necessarily more secure. The down-level behavior is quite marvelous.)
        Show
        Dennis Hamilton added a comment - @Andreas, I agree that speculative upgrade to ODF 1.2 before the specification was baked was dangerous. Although that accounts for there being all of these variations in the wild, they are now in the wild and the documents that originated with them are there too. I don't think it is fair to penalize users for this. Also, I think it is important to understand that the breaking case is with regard to down-level consumers that were not implemented based on any speculation whatsoever. That is the focus of my attention with regard to this specific issue. This is the oversight that was made when <manifest:manifest> manifest:version was introduced and made mandatory. It just didn't receive the analysis that has since arisen out of hind-sight and recognition of what all the cases in the wild have turned out to be. Instead of fostering interoperability, the outcome has been a reduction of interoperability. It appears, in fact, that the attribute is consistently ignored on consumption by those processors that produce it and also many that don't produce it. More generally, I think the discussion on the list concerning ways to have provisional implementations (not using ODF-official namespaces) that are for features with reserved definitions in future releases of ODF is important. This would have made the handling of implementation-specific digital signature extensions in ODF 1.1 documents more transparent and it will help with anticipated features for ODF 1.3 that are related to document privacy and integrity. Downlevel disruption, even among implementations of the same ODF version, will still happen, of course. (One case now in existence is the change of what encryption is used by default, causing older implementations to fail. In this case, <manifest:manifest> manifest:version="1.2" has been no help because it was being produced either way. Although that ends up relying on published standards, the result is not necessarily more secure. The down-level behavior is quite marvelous.)

          People

          • Assignee:
            Patrick Durusau
            Reporter:
            Dennis Hamilton
          • Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated: