1 Introduction
The inevitable evolution of languages by adding, deleting, or changing
syntax or semantics is called versioning. Making versioning work in practice
is one of the most difficult problems in computing. Arguably, the Web rose
dramatically in popularity because evolution and versioning were built into
HTML and HTTP. Both systems provide explicit extensibility points and rules
for understanding extensions that enable their decentralized extension and
versioning.
This finding describes general problems and techniques in evolving systems
in compatible and incompatible ways. These techniques are designed to allow
compatible changes with or without schema propagation. A number of questions,
design patterns and rules are discussed with a focus towards enabling
versioning in XML vocabularies, making use of XML Namespaces and XML Schema
constructs. This includes not only general rules, but also rules for working
with languages that provide an extensible container model, notably SOAP.
1.1 Terminology
The terminology for describing languages, producers, consumers,
information, constraints, syntax, evolvability etc. follows. Let us consider
an example. Two or more systems need to exchange name information. Names may
not be the perfect choice of example because of internationalization reasons,
but it resonates strongly with a very large audience. The Name Language is
created to be exchanged. [Definition: A producer is an agent that creates
text. ] Continuing our example, Fred is a producer of Name Language text. [Definition: An
Act of Production is the creation of text.]. A producer produces text
for the intent of conveying information. When Fred does the actual creation
of the text, that is an act of production. [Definition: A consumer is an
agent that consumes text.] We will use Barney and Wilma as consumers of text.
[Definition: An Act of Consumption is the
processing of text of a language.] Wilma and Barney consume the text
separately from each other, each of these being a consumption event. A
consumer is impacted by the instance that it consumes. That is, it interprets
that instance and bases future processing, in part, on the information that
it believes was present in that instance. Text can be consumed many times, by
many consumers, and have many different impacts.
[Definition: A
Language consists of a set of text, any syntactic constraints on the
text, a set of information, any semantic constraints on the information, and
the mapping between texts and information. ][Definition: Text is a specific, discrete sequence of
characters]. Given that there are constraints on a language, any particular
text may or may not have membership in a language. Indeed, a particular
string of characters may be a member of many languages, and there may be many
different strings of characters that are members of a given language. The
text of the language are the units of exchange. Documents are texts of a
language. The Name Language consists of text set that have 3 terms and
specifies syntactic constraints: that a name consists of a given and a
family. [Definition: A language has a set of
constraints that apply to the set of strings in the language. ] These
constraints can be defined in machine processable syntactic constraint
languages such as XML Schema, microformats, human readable textual
descriptions such as HTML descriptions, or are embodied in software.
Languages may or may not be defined by a schema in any particular schema
language. The constraints on a language determine the strings that qualify
for membership in the language. Vocabulary terms contribute to the set of
strings, but they are not the only source of characters to the set of strings
in a given language. The language strings may include characters outside of
terms, such as punctuation. One reason for additional characters is to
distinguish or separate terms, such as whitespace and markup.
<name>
<given>Dave</given>
<family>Orchard</family>
</name>
name="Dave Orchard"
<span class="fn">Dave Orchard</span>
urn:nameschem:given:Dave:family:Orchard
The set of information in a language almost always has semantics. In the
Name Language, given and family have the semantics of given and family names
of people. The language also has the binding from the items in the
information set to the text set. Any potential act of interpretation, that is
any consumption or production, conveys information from text according to the
language's binding. The language is designed for acts of interpretation, that
being the purpose of languages. In our example, this mapping is obvious and
trivial, but many languages it is not. Two languages may have the exact same
strings but different meanings for them. In general, the intended meaning of
a vocabulary term is scoped by the language in which the term is found.
However, there is some expectation that terms drawn from a given vocabulary
will have a consistent meaning across all languages in which they are used.
Confusion often arises when terms have inconsistent meaning across language.
The Name terms might be used in other languages, but it is generally expected
that they will still be "the same" in some meaningful sense.
These terms and their relationships are shown below
We say that Fred engages in an Act of Production that results a Name
Instance with respect to Name Language V1. The Name Instance is in the set of
Name V1 Strings, that is the set of strings in the Name Language V1. The
production of the Name Instance has the intent of conveying Information,
which we call Information 1. This is shown below:
We say that Barney engages in an Act of Consumption of a Name Instance
with respect to Name Language V1. The consumption of the Name Instance has
the impact of conveying Information 1. This is shown below:
Versioning is an issue that effects almost all applications eventually.
Whether it's a processor styling documents in batch to produce PDF files, Web
services engaged in financial transactions, HTML browsers, the language and
instances will likely change over time. The versioning policies for a
language, particularly whether the language is mutable or immutable, should
be specified by the language owner. Versioning is closely related to
extensibility as extensible languages may allow different versions of
instances than those known by the language designer. Applications may receive
versions of a language that they aren't expecting.
If a Name Language V2 exists, with its set of strings and Information set,
Wilma may consume the same Name Instance but with respect to the Name
Language V2 and have impact of Information 2. Name Language V2 relates to V1
by relationship r2, which is forwards compatible comparing language V1 to V2
instances, and backwards compatible comparing language V2 to V1 instances.
Similarly, Information 2 - as conveyed by Consumption 2 - relates to
Information 1 - as conveyed by Consumption 1 - by relationship r1.
Extensibility is a property that enables evolvability of software. It is
perhaps the biggest contributor to loose coupling in systems as it enables
the independent and potentially compatible evolution of languages. Languages
are defined to be [Definition: Extensible if the syntax of a
language allows information that is not defined in the current version of the
language.]. The Name Language is extensible if it can include terms that
aren't defined in the language, like a new middle term.
1.1.1 Compatibility
As languages evolve, it becomes possible to speak of both backwards and
forwards compatibility. We base our definitions of backwards and forwards
compatibility on the [FOLDOC] definitions. There are
two aspects of compatibility that are called out: software compatibility and
schema compatibility. While it is often the case that they are directly
related, sometimes they are not.
[Definition: A language change is
backwards compatible if newer processors can process all instances of
the old language. ] Backwards compatibility means that a newer version of a
consumer can be rolled out in a way that does not break existing producers. A
producer can send an older version of a message to a consumer that
understands the new version and still have the message successfully
processed. A software example is a word processor at version 5 being able to
read and process version 4 documents. A schema example is a schema at version
5 being able to validate version 4 documents. This means that a producer can
send an old version of a message to a consumer that understands the new
version and still have the message successfully processed. In the case of Web
services, this means that new Web services consumers, ones designed for the
new version, will be able to process all instances of the old language.
[Definition: A language change is forwards
compatible if older processors can process all instances of the newer
language.] Forwards compatibility means that a newer version of a producer
can be deployed in a way that does not break existing consumers. Of course
the older consumer will not implement any new behavior, but a producer can
send a newer version of an instance and still have the instance successfully
processed. An example is a word processing software at version 4 being able
to read and process version 5 documents. A schema example is a schema at
version 4 being able to validate version 5 documents. This means that a
producer can send a newer version of a message to an existing consumer and
still have the message successfully processed. In the case of Web services,
this means that existing Web service consumers, designed for a previous
version of the language, will be able to process all instances of the new
language.
In general, backwards compatibility means that existing documents can be
used by updated consumers, and forwards compatibility means that newer
documents can be used by existing consumers. Another way of thinking of this
is in terms of message exchanges. Backwards compatibility is where the
consumer is updated and forwards compatibility is where the producer is
updated, as shown below:
In broad terms, backwards compatibility means that newer producers can
continue to use existing services, and forwards compatibility means that
existing producers can use newer services
One form of compatibility is with respect to the strings, that is whether
all the strings in one language are also strings in another language. Another
form of compatibility is with respect to the information conveyed, that is
whether the information conveyed by the strings in one language is conveyed
by the same strings in another language. The strings could be compatible but
the information conveyed is not compatible.
Compatibility is defined for the producer and consumer of an individual
document instance. Most messaging specifications, such as Web Services,
provide inputs and outputs. Using these definitions of compatibility, a Web
service that updates its output message is considered a newer producer
because it is sending a newer version of the message. Conversely, updating
the input message makes the service a newer consumer because it is consuming
a newer version of the message. All systems of inputs and outputs must
consider both when making changes and determining compatibility. For full
compatibility, any output messages changes must be forwards compatible (for
the older receivers aka consumers) and any input message changes must be
backwards compatible (for the older senders aka producers).
Compatibility can be defined in terms of the information and syntax sets.
Any document that contains only V1 information set in the V1 syntax will be
processable by V2, so V2 is backwards compatible with V1. Note the constraint
that it is the V1 information set items in V1 syntax. If there was an
extension to a language then that language is no longer V1. The compatibility
guarantees are only for V1 and V2, not for other versions. More formally:
principle
Language V2 is backwards compatible with Language V1 if
Language V2 Information set > (superset) Language V1 Information set.
Because V1 syntax set is larger than (superset of) V2 syntax, a V1
processor can process all V2 documents, so we say that V2 is forwards
compatible with V1. More formally:
principle
Language V2 is forwards compatible with Language V1 if
Language V1 Syntax > (superset) Language V2 Syntax.
These 2 principles can be combined together into:
principle
Language V2 is compatible with Language V1 if Language
V1 Syntax > Language V2 Syntax > Language V2 Information Set >
Language V1 Information set.
We have shown that forwards and backwards compatibility is only achievable
through extensibility, and the process of compatible versioning is gradually
narrowing the extensibility gap between the information set and the syntax
set.
1.1.1.1 Partial
Understanding
We have defined compatibility for all possible expressions of the
language, that is full compatibility. There are many scenarios where a
consumer will consume only part of the information set. Partial understanding
affects both information and syntax set. Partial understanding usually
results in a subset of the information set (because only part of the
information set is understand). Interestingly, partial understanding usually
increases or supersets the syntax. This is usually because the process of
extracting the subset needed means that extra content, even those illegal
under V1 syntax, is ignored.
We can imagine an application that only looks at given names and
everything else is ignored. My favourite example of this is a _Baby Name_
Wizard. The application might use a simple XPath expression to extract the
given name from inside the name. This is a different version of the Name
Language, which we will call the Given Name Language. The syntax set for the
Given Name Language is anything, given, anything. The information set for the
Given Name language is given names. Because the Given Name Language syntax
set is more relaxed that the Name Language V1, an addition of the middle name
between the given and family is a compatible change for the Given Name
Language.
Our principles with respect to compatibiity and language versioning need
no change to deal with partial understanding.
principle
Partial Understanding a language is the creation of a
language VPU such that Language VPU Syntax > Language V1 Syntax >
Language V1 Information Set > Language VPU Information set.
Interestingly, partially understanding a language is creating a language
VPU, such that the V1 language is a compatible change with the VPU. There may
be many different versions that are all partial understandings of a language.
We call these related languages "flavours" (note: dialects doesn't seem
right, subsets also doesn't seem right). It may be very difficult for a
language designer to know many different language flavours are in existance.
However, a language designer can sometimes use the different flavours to
their advantage in designing for a mixture of compatible and incompatible
changes. Some changes could be compatible with some flavours but not other.
It may be very useful to have some changes be compatible with some flavours,
that is those consumers do not need to be updated or changed.
It is crucial to point out that the partially understood versions of the
language are NOT producers of the language. They have relaxed the
restrictions on the consuming side, but not on the production side of the
language. If a flavour of a language was also used for consumption, it would
have create an instance that is valid according to the Language V1 rules.
Perhaps the only exceptions are if they are guaranteed that they will be
producing for compatible flavours. Typically this is not the case and hard to
determine, so the safest course is to produce according to the Language V1
rules.
We have shown how relaxing the constraints on a language when consuming
instances of it can turn an otherwise incompatible change into a compatible
change. We have also shown that abiding by the language constraints when
producing instances is the safest course. Said more eloquently is the
internet robustness principle, "be conservative in what you do, be liberal in
what you accept from others" from [tcp].
This leads us to two versioning practices:
Good Practice
Use Least Partial Languages: Consumers should use a
flavour of a language that has the least amount of understanding.
The least amount of understanding will be the most liberal or have the
largest syntax set possible.
Good Practice
Produce no partial languages: Producuers should use the
complete version of a language and no partial flavours of a language.
This guarantees interoperability with all the consumers and is the most
conservative production possible.
EdNote: I think related to principle of least power. The least powerful
the language, the easier to have partial understanding?
1.1.2 Open or Closed
systems
The cost of changes that are not backward or forward compatible is often
very high. All the software that uses the language must be updated to the
newer version. The magnitude of that cost is directly related to whether the
system in question is open or closed.
[Definition: A closed system is one in which
all of the producers and consumers are more-or-less tightly connected and
under the control of a single organization.] Closed systems can often provide
integrity constraints across the entire system. A traditional database is a
good example of a closed system: all of the database schemas are known at
once, all of the tables are known to conform to the appropriate schema, and
all of the elements in the each row are known to be valid for the schema to
which the table conforms.
From a versioning perspective, it might be practical in a closed system to
say that a new version of a particular language is being introduced into the
system at such and such a time and all of the data that conforms to the
previous version of the schema will be migrated to the new schema.
[Definition: An open system is one in which
some producers and consumers are loosely connected or are not controlled by
the same organization. The internet is a good example of an open system.]
In an open system, it's simply not practical to handle language evolution
with universal, simultaneous, atomic upgrades to all of the affected software
components. Existing producers and receivers outside the immediate control of
the organization that has publishing a changed language will continue to use
the previous version for some (possibly long) period of time.
Finally, it's important to remember that systems evolve over time and have
different requirements at different stages in their life cycle. During
development, when the first version of a language is under active
development, it may be valuable to persue a much more aggressive, draconian
versioning strategy. After a system is in production and there is an
expectation of stability in the language, it may be necessary to proceed with
more caution. Being prepared to move forward in a backwards and forwards
compatible manner is the strongest argument for worrying about versioning at
the very beginning of a project.
1.2 XML Terminology
There are many different systems for exchanging texts in languages, such
as SQL, Java, XML, ECMAScript, C#. We will briefly describe some key
refinements to our lexicon for XML. An XML language has a vocabulary that may
use terms from one or more XML Namespaces (or none), each of which has a
namespace name. [Definition: An XML language is an identifiable
set of vocabulary terms with defined XML syntactic and semantic constraints.
] By XML language, we mean the set of elements and attributes, or instances,
used by a particular application. The Name Language - consisting of name,
given, family terms - has a namespace for the terms. We use the prefix
"namens" to refer to that namespace. The Name Language could consist of terms
from other vocabularies, such as Dublin Core or UBL. These terms each have
their own namespaces, illustrating that a language can comprise vocabularies
from multiple namespaces. An XML Namespace is a convenient container for
collecting terms that are intended to be used together within a language or
across languages. It provides a mechanism for creating globally unique
names.
We shall use the term instance
when speaking of sequences of characters (aka text) in XML. [Definition: An
instance is a specific, discrete Text in XML format.] Documents are
instances of a language. In XML, they must have a root element. A name
document might have a name element as the root element. Alternatively, the
name vocabulary may be used by a language such as purchase orders. The
purchase order documents may contain name elements. Thus instances of a
language are always part of a document and also may be the entire document.
XML instances (and all other instances of markup languages) consist of markup
and content. In the name example, the given and family elements including the
end markers are the markup. The values between the start and end markers are
the content. An instance has an information model. There are a variety of
data models within and without the W3C, and the one standardized by the W3C
is the XML infoset.
The XML related terms and their relationships are shown below
A stylesheet processor is a consumer of the XML document that it is
processing (the producer isn't mentioned); in the Web services context the
roles of producer and consumer alternate as messages are passed back and
forth.Note that most Web service specifications provide definitions of inputs
and outputs. By our definitions, a Web service that updates its output schema
is considered a new producer. A service that updates its input schema is a
new consumer.
1.3 Why Worry About Extensibility
and Versioning?
As documents, or messages, are exchanged between applications, they are
processed. Most applications are designed to discriminate between valid and
invalid inputs. In order to have any sort of interoperability, a language
must be defined or described in some normative way so that the terms
"invalid" and "valid" have meaning.
There are a variety of tools that might be employed for this purpose
(DTDs, W3C XML Schema, RELAX NG, Schematron, etc.). These tools might be
augmented with normative prose documentation or even some
application-specific validation logic. In many cases, the schema language is
the only validation logic that is available.
It is almost unheard of for a single version of a language to be deployed
without requiring some kind of augmentation. Invariably, the original
language designer did not include certain terms and constraints. In fact,
good designers should not try to define all the possible terms and
constraints. This is sometimes called "boiling the ocean". Knowing that a
language will not be all things to all people, a language designer can allow
parties to extend instances of the language or the language itself. Typically
the tools will allow the language designer to specify where extensions in the
instance and extensions in the language are allowed. Of note, we do not call
extending an instances of a language a new version. This limits our
discussion of versioning to changes in a language, not changes to
instances.
Whether you've deployed ten services, or a hundred, or a million, if you
change a language in such a way that all those services will consider
instances of the new language invalid, you've introduced a versioning problem
with real costs.
Once a language is used outside of its development environment, there will
be some cost associated with changing it: software, user expectations, and
documentation may have to be updated to accomodate the change. Once a
language is used in environments outside of a single realm of control, any
changes made will introduce multiple versions of the language.
1.4 Why Do Languages Change?
There are many reasons why a different version of a language may be
needed. A few of them include:
Bugs may need to be fixed. Production use may reveal defects or
oversights that need to be fixed. This may involve changes to components
of the language or changes to the semantics of existing components.
Changing requirements may motivate changes in the schema design. For
example, a callback may be added to a service that performs some
processing so that it is able to notify the caller when processing has
completed.
Different flavors of a schema may be desirable. For example, the
XHTML 1.0 Recommendation defines strict, transitional, and frameset
schemas. All three of those schemas purport to define the same namespace,
but they describe very different languages.
And additional schemas may be defined by other specifications, such as
the XHTML Basic Recommendation.
Whatever the cause, over time, different versions of the language exist
and designing applications to deal with this change in a predictable, useful
way requires a versioning strategy.
1.5 Why Extend languages?
The primary motivation to allow instances of a language to be extended is
to decentralize the task of designing, maintaining, and implementing
extensions. It allows producers to change the instances without going through
a centralized authority. It means that changes can occur at the producer or
consumer without the language owner approving of them. Consider the effort
that the HTML Working Group put into modularity of HTML. Without some
decentralized process for extension, every single variant of HTML would have
to be called something else or the HTML Working Group would have to
agree to include it in the next revision of HTML.
1.6 How Do Languages Change?
At the most basic level, languages can change in only a few ways:
Content: The allowable content can evolve through addition
or deletion. In XML, this becomes
ElementsNew elements can be added, existing elements
can be removed, or the acceptable number of occurences of an element
can change. In addition, the content of an element could change from
element only content to mixed content, or vice versa.
For elements with simple content, the type or range of values that
are acceptable can change.
Attributes: New attributes can be added, existing
attributes can be removed, or the type or range of values that are
acceptable can change.
Semantics: The meaning of a an existing term can change.
Of course, the difference between two versions of a language can be an
arbitrary number of these changes.
One of the most important aspects of a change is whether or not it is
backwards or forwards compatible.
Some typical backwards- and forwards-compatible changes:
adding optional components (elements and/or attributes)
adding optional content, for example extending an enumeration
Some typical forwards-compatible changes:
Some typical backwards-compatible changes:
Some typical incompatible changes:
changing the meaning or semantics of existing components
adding required components
removing required components
restricting a components content model, such as changing a choice to
a sequence
1.7 Kinds of Languages
Ultimately, there are different kinds of languages. The versioning
approaches and strategies that are appropriate for one kind of language may
not be appropriate for another. Among the various kinds of vocabulares, we
find:
Just Names: some languages don't actually identify elements
or attributes, they're just lists of names. Using QNames to identify
words in the WordNet database, for example, or the names of functions and
operators in XPath2 are examples of "just name" languages.
Standalone: languages designed to be used more-or-less by
themselves, for example XHTML, DocBook, or The TEI.
Containers: languages designed to be used as a wrapper or
framework for some other language or payload, for example SOAP or
WSDL.
Container Extensions: languages designed to extend or
augment a particular class of container. Specifications that extend SOAP
by defining SOAP header blocks, for example, to provide security,
asynchrony or reliable messaging are examples of container extension
languages.
There are a couple types of XML extension languages, element extension
and attribute extension.
Element Extension. Languages that are elements. SOAP,
etc. are element extensions.
Attribute or type Extensions. Langages that are types
or attributes. These languages must exist in the context of an
element. Sometimes called "parasite" languages as they require a
"host" element. XLink is an example.
Mixtures: languages designed for, or often used for,
encapsulating some semantics inside another language. For example, MathML
might be mixed inside of another language.
This is by no means an exhaustive list. Nor are these categories
completely clear cut. MathML can certainly be used standalone, for example,
and languages like SVG are a combination of standalone, containers, and
mixtures.
2 Language Questions
Given the definitions of compatibility described above, what questions
must the language designer consider?
2.1 Can 3rd parties extend
the language?
It is rarely desirable to prevent 3rd parties from extending languages,
but it does happen. An example may be a tightly constrained security
environment where distributed authoring is considered a _bug_ rather than a
feature.
2.2 Can 3rd parties extend
the language in a compatible way?
If so, a substitution mechanism is required for forwards compatibility. If
an older consumer has no mechanism for dealing with new content, then
forwards-compatible evolution isn't possible. One simple substitution
mechanism is simply ignoring the unrecognized components.
2.3 Can 3rd parties extend
the language in an incompatible way?
If so, and if compatible extensions are also possible, then it must be
possible to identify incompatible changes so that they can override the
substitution mechanism used for extensible changes.
In environments where unrecognized components are ignored, a _must
understand_ component can be added to identify incompatible changes.
If compatible changes are not possible, then incompatible changes simply
become the default. For example, WS-Security mandates that 3rd parties can
only provide incompatible extensions. Unlike most languages, a security
language has unique requirements where the consequences of ignored data can
be severe. WS-Security accomplished this by specifying that all extensions
are required to be understood and there is no substitution mechanism.
2.4 Can the designer extend
the language in a compatible way?
As with 3rd parties compatible extensions, a substitution mechanism for
the designer_s extensions is required for forwards compatibility.
2.5 Can the designer extend
the language in an incompatible way?
In XML, the designer can always do this by using new namespace names,
element names or version numbers. In other languages, this may not be
possible because there is no mechanism for indicating the incompatible
change.
2.6 Is the vocabulary a
stand-alone language or an extension of another vocabulary?
A part of this question is whether the language depends on another
language. That determines which, if any, facilities are provided for the
containing language and what must be provided by a contained language.
SOAP is an example of a container language. The SOAP processing model
applies uniformly to all headers, which may employ
soap:mustUnderstand to identify incompatible changes, even
though the contents of the SOAP headers are languages independent from
SOAP.
2.7 What language form
Languages can be expressed in text, comma separted values, XML, SGML,
binary, source code, and almost any kind of form. See the Architecture of the
World-Wide Web section on data formats for more information -
http://www.w3.org/TR/2004/REC-webarch-20041215/#formats.
2.8 What Schema
language(s)?
Choosing a schema language or languages guides the language design in many
ways. Some features, particularly extensibility, must be anticipated in the
first version of a language in order to take advantage of the features of
some schema languages.
In addition, various features may be incompatible across different
languages. For example, writing a V2 compatible schema in W3C XML Schema
requires special design, which is not required in a schema language such as
RELAX NG. Some of the language design choices mandated by W3C XML Schema are
discussed in other sections of this Finding.
2.9 Should extensions or
versions be expressible in the Schema language?
The ability to write a schema for extensions or versions is directly
affected by the schema design and the compatibility desires.
3 Language Decisions
Upon answering these questions, there are some key decisions that a
language developer makes, whether they are consciously made or not.
3.1 Schema language design
choices or constraints.
If the language can be extended in a compatible way, then a few specific
schema design choices must be followed.
Wildcards are used to provide extensibility in XML Schema. If revisions to
the Schema are to support substitution, specific schema designs must be used
in conjunction with the wildcard. The main choices are: provide wildcards,
provide Extension elements, or provide delimiter elements. Extension and
delimiter elements are described in the new components in existing or new
namespaces section. If Extension/delimiter elements are not provided, then
a compatible V2 Schema cannot be written.
3.2 Substitution
Mechanism.
Forwards compatibility can only be achieved by providing a substitution
mechanism for Version 2 instances or Version 1 extensions to V1 without
knowledge of V2. A V1 consumer must be able to transform any instances,
such as V1 + extensions, to a V1 instance in order to process the instance.
The _Must Ignore unknown_ rule is a simple substitution mechanism. This rule
says that any extensions are _ignored_. Using it, a V1 + extensions document
is transform into a V1 document by ignoring the extensions. Others
substitution mechanisms exist, such as the fallback model in XSLT.
3.3 Component
identification
The identification of components into language versions or extensions has
a variety of general mechanisms related to namespaces. These are detailed in
the Versioning section.
3.4 Identification of
incompatible extensions
The identification of versions is covered by language identification, but
3rd parties cannot arbitrarily change versions or change namespaces. They may
need a mechanism to indicate that an extension is an incompatible change. A
couple of mechanisms are a _Must Understand_ identifier (such as a flag or
list of required namespaces) or requiring that extensions are in substitution
groups.
4 Identifying and Extending
Languages
Designing extensibility into languages typically results in systems that
are more loosely coupled. Extensibility allows authors to change instances
without going through a centralized authority, and may allow the centralized
authority greater opportunities for versioning. The common characteristic of
a compatible change is the use of extensibility.
A supreme example of the benefits of extensibility is HTML. The first
version of HTML was designed for extensibility; it said that _unknown markup_
may be encountered. An example of this in action is the addition of the IMG
tag by the Mosaic browser team.
The first rule introduced in this Finding relating to extensibility is:
Good Practice
Allow Extensibility rule: Languages SHOULD be designed
for extensibility.
A fundamental requirement for extensibility is to be able to determine the
language of elements and attributes. XML Namespaces [13]
provide a mechanism for associating a URI with an XML element or attribute
name, thus specifying the language of the name. This also serves to prevent
name collisions.
HTML did not have the ability to distinguish between the languages of
extensions. This meant that authors could produce the same element name but
with different interpretations, and software would have no way of determining
which interpretation was applicable. This is a great part of the motivation
to move from HTML to the XML vocabulary of HTML, XHTML.
W3C XML Schema [14] provides a mechanism called a
wildcard, <xs:any>, for controlling where elements from certain
namespaces are allowed. The wildcard indicates that elements in specified
namespaces are allowed in instance documents where the wildcard occurs. This
allows extension in a well-defined manner. A consumer of extended documents
can identify and, depending upon its processing model, safely ignore the
extensions it doesn't understand.
<xs:any> uses the namespace attribute to control what namespaces
extension elements can come from. The most interesting values for this
attribute are: ##any, which means one can extend the schema using an element
from any possible namespace; ##other, which only allows extension elements
from namespaces other than the target namespace of the schema; and
##targetnamespace, which only allows extension elements from the target
namespace of the schema.
<xs:any> uses the processContents attribute to control how a XML
parser validates extended elements. Permissible methods include _lax_ -
validate any elements from supported namespaces but ignore all other
elements, _strict__validate all elements, and _skip__validate no elements.
This Finding recommends _lax_ validation, as it is the most flexible and is
the typical choice for Web services specifications.
RDF/OWL and RelaxNG are 2 other popular technologies for schema design.
They have different mechanisms for allowing and controlling schema
evolution.
5 Example
Suppose that you have designed a language for handling personal
information consisting of a single _Name_ element. The first version of the
Name contains a _given_ and a _family_ element. There are a variety of
strategies for extensibility and versioning, detailed later. This example
will simply show the "new components in new namespace" strategy.
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://www.example.org/name/1"
xmlns:name="http://www.example.org/name/1">
<xs:complexType name="name">
<xs:sequence>
<xs:element name="given" type="xs:string"/>
<xs:element name="family" type="xs:string"/>
<xs:element name="middle" type="xs:string" minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
<xs:anyAttribute/>
</xs:complexType>
</xs:schema>
The language designer and 3rd parties can now use different namespaces for
their versions. The language designer makes a variety of choices,
particularly the schema language, that affect their strategy for namespaces.
Thus the first rule for namespaces covers all language choices, but is open
to extension based upon other design choices.First the rule for
namespaces:
Good Practice
Allow Extensions in Other Namespace rule: The
extensibility point SHOULD at least allow for extension in other
namespaces.
The rule for allowing extensibility:
Good Practice
Full Extensibility rule: All XML Elements SHOULD allow
for element extensibility after element definitions, and allow any
attributes.
In general, an extension can be defined by a new specification that makes
a normative reference to the earlier specification and then defines the new
element. No permission should be needed from the authors of the specification
to make such an extension. In fact, the major design point of XML namespaces
is to allow decentralized extensions. The corollary is that permission is
required for extensions in the same namespace. A namespace has an owner;
non-owners changing the meaning of something can be harmful.
Attribute extensions do not have non-determinism issues because the
attributes are always unordered and the model group for attributes uses a
different mechanism for associating attributes with schema types than the
model group for elements.
6 Understanding
Extensions
Ideally, producers should be able to extend existing XML documents with
new elements without consumers having to change existing implementations.
Extensibility is one step towards this goal, but achieving compatibility also
requires a processing model for the extensions. The behavior of software when
it encounters an extension should be clear. For this, we introduce the next
rule:
Good Practice
Provide Processing Model Rule: Languages SHOULD specify a
processing model for dealing with extensions.
Achieving forwards-compatible evolution requires that the processing model
must be a substitution mechanism. The instance containing the extension,
which isn't known by the consumer, must be transformed into an instance which
is of a type known by the consumer.
Good Practice
Provide Substitution model: Languages MUST provide a
substitution model for forwards-compatible evolution.
The simplest substitution model that enables compatible changes is to
ignore content that is not understood. This rule is:
Good Practice
Must Ignore Rule: Document consumers MUST ignore any XML
attributes or elements in a valid XML document that they do not recognize.
This rule does not require that the elements be physically removed; only
ignored for processing purposes. There is a great deal of historic usage of
the Must Ignore rule. HTML 1, 2 and 3.2 follow the Must Ignore rule as they
specify that any unknown start tags or end tags are mapped to nothing during
tokenization. HTTP 1.1 [7] specifies that a consumer
should ignore any headers it doesn't understand: "Unrecognized header fields
SHOULD be ignored by the recipient and MUST be forwarded by transparent
proxies." The Must Ignore rule for XML was first standardized in the WebDAV
specification RFC 2518 [6] section 14 and later
separately published as the Flexible XML Processing Profile [3].
There are two broad types of Must Ignore rules for dealing with
extensions, either ignoring the entire tree or just the unknown element. The
rule for ignoring the entire tree is:
Good Practice
Must Ignore All Rule: The Must Ignore rule applies to
unrecognized elements and their descendents in data-oriented formats.
For example, if a message is received with unrecognized elements in a SOAP
header block, they must be ignored unless marked as _Must Understand_ (see
Rule 10 below). Note that this rule is not broken if the unrecognized
elements are written to a log file. That is, _ignored_ doesn_t mean that
unrecognized extensions can_t be processed; only that they can_t be the
grounds for failure to process.
Other applications may need a different rule as the application will
typically want to retain the content of an unknown element, perhaps for
display purposes. The rule for ignoring the element only is:
Good Practice
Must Ignore Container Rule: The Must Ignore rule applies
only to unrecognized elements in presentation-oriented formats.
This retains the element descendents in the processing model so that they
can still affect interpretation of the document, such as for display
purposes.
Ignoring content is a simple solution to the problem of substitution. In
order to achieve a compatible evolution, the newer instances of a language
must be transformable (or substitutable) into older instances. Object systems
typically call this _polymorphism_, where a new type can behave as the old
type.
Other substitution models have been successfully deployed. One such model
is a fallback model, where alternate elements are provided if the consumer
does not understand the extension. XSLT 2.0 provides such a model. Another
model is that a transform from the new type to the old type is made
available, either by value or reference.
As desirable as compatible evolution often is, sometimes a language may
not want to allow it. In this model, a consumer will generate a fault if it
finds a component it doesn_t understand. An example might be a security
specification where a consumer must understand each and every extension. This
suffers from the significant drawback that it does not allow compatible
changes to occur in the language, as any changes require both consumer and
producer to change.
7 Versioning
A language designer decides how new versions of their language, as well as
extensions, are related to previous versions. They decide how to use
namespace names, component names for their language, as well as possibly
introducing versioning-specific components such as version identifiers and
incompatible extension identifiers. When a new version of a language is
required, the author must make a decision about the namespace name for names
in the new language.
Version identification has traditionally been done with a decimal
separating the major versions from the minor versions, ie _8.1_, _1.0_. Often
the definition of a _major_ change is that it is incompatible, and the
definition of a _minor_ change is that it is forwards- and/or backwards -
compatible. Usually the first broadly available version starts at _1.0_. A
compatible version change from 1.0 might be identified as _1.1_ and an
incompatible change as _2.0_. It should be noted that this is idealistic as
there abundant cases where this system does not hold. New major version
identifiers are often aligned with product releases, or incompatible changes
identified as a _minor_ change. A good example of an incompatible changed
identified as a minor change is XML 1.1. XML 1.0 processors cannot process
all XML 1.1 documents because XML 1.1 extended XML 1.0 where XML 1.0 does not
allow such extension.
7.1 Versioning Strategies
Versioning is a broad and complex issue. Different communities have
different notions about what constitutes a version, what constitutes a
reasonable policy, and what the appropriate behavior is in the face of
deviations from that policy. Historically, it has always proved more
complicated in practice than in theory.
In broad terms, the approaches to versioning fall into a number of classes
ranging from "none" to a "big bang":
None. No distinction is made between versions of the
language. Applications are either expected not to care, or they are
expected to cope with any version they encounter.
Compatible. Designers are expected to limit changes to
those that are either backwards or forwards compatible, or both.
Backwards compatibility. Applications are expected to
behave properly if they receive an instance document of the "older"
version of a language. Backwards compatible changes allow
applications to behave properly if they receive an "older" version of
the language.
Forwards compatibility. Applications are expected to
behave properly if they receive an instance document of the "newer"
version of a language. Forwards compatible changes allow existing
applications to behave properly if they receive a "newer" version of
the language.
Flavors. Applications are expected to behave properly if
they receive one of a set of flavors of the document type.
Big bang. Applications are expected to abort if they see an
unexpected version.
There's no single approach that's always correct. Different application
domains will choose different approaches. But by the same token, the
approaches that are available depend on other choices, especially with
respect to namespaces. This dependency makes it imperative to plan for
versioning from the start. If you don't plan for versioning from the start,
when you do decide to adopt a plan for versioning, you may be constrained in
the available approaches by decisions that you've already made.
A language commonly goes through a lifecycle of iterative development
followed by deployment followed by deployment of new versions. The point in
the lifecycle will affect the selection of the versioning strategy for the
language
Just as there are a number of approaches, there are a number of strategies
for implementing an approach. The internet - including MIME, markup
languages, and XML languages have succesfully used various strategies, either
singly or in combination. Summaries of strategies and requirements have been
produced for earlier technologies and guided XML Namespaces and Schema, such
as [Web Architecture: Extensible Languages].
For any given approach, some strategies may be more appropriate than
others. Among the strategies we find:
Must Understand. consumers must understand all of the
elements and attributes received and are expected to abort processing if
they do not. SOAP processors must understand headers that are explicitly
identified to be mandatory.
Must Ignore Unknown. consumers must ignore elements or
attributes that they do not understand. Sometimes the must understand and
must ignore approaches can be combined for more selective use. SOAP
processors must ignore headers they do not recognize unless the header
explicitly identifies itself as one that must be understood.
There are 2 variations of the Must Ignore strategy:
Must Ignore All This variation on must ignore
requires the consumer to ignore an element or attribute it does not
understand and, in the case of elements, all of the descendents of
that element. Most data applications, such as Web services that use
SOAP header blocks or WSDL extensions, adopt this approach to dealing
with unexpected markup. For XML, the Must Ignore all rule was first
standardized in the WebDAV specification RFC 2518 [WebDAV] section 14 and later separately
published as the [FlexXMLP].
Must Ignore Container. This variation on must
ignore requires the consumer to ignore an element or attribute
that it does not understand, but in the case of elements, to process
the children of that element. The Must Ignore Container practice was
described in [HTML 2.0]
Explicit Fallback. A language can provide mechanisms for
explicit fallback if the extension is not supported. [MIME] provides multipart/alternative for equivalent,
and hence fallback, representations of content. [HTML
4.0] uses this approach in the NOFRAMES element. In XML, the XML
Inclusions specification [XInclude] provides a
fallback element to handle the case where the putatively included
resource cannot be retreived. There are many variations on where the
fallback content can be found. For example, a schema language could
specify that fallback content is found in an instance document, in a
schema document, or even in the schema for the schema language.
Explicit Testing. A language can provide a mechanism for
explicit testing. The XSLT Specification provides a conditional logic
element and a function to test for the existence of extension functions.
This allows designers of stylesheets to deal with different consumer
capabilities in an explicit fashion.
Languages can choose a mixture of approaches. For example, XSLT provides
both an explicit fallback mechanism for some conditions and explicit testing
for others. The SOAP specification, another example, specifies Must Ignore as
the default strategy and the ability to dynamically mark components as being
in the Must Understand strategy.
7.2 Why Have a Strategy?
Different kinds of languages and different versioning strategies expose
different problems. If you don't have a strategy at all, you are effectively
choosing the "no versioning" strategy.
It's probably obvious that attempting to deploy a system that provides no
versioning mechanism is frought with peril. Putting the burden of version
"discovery" on consumers is probably impractical in anything except a closed
system.
At the other end of the spectrum is the "big bang" approach which is also
problematic.
"Big bang" is a very coarse-grained approach to versioning. It establishes
a single version identifier, either a version number or namespace name, for
an entire document.
The semantics of the "big bang" are that applications decide on the basis
of the document version whether or not they know how to process that
document. If the version isn't recognized, the entire document is rejected..
Typically, when introducing a new version using the big bang
approach, all of the software that produces or consumes the instance
documents is updated in a sweeping overhaul in which the entire system is
brought down, the new software deployed and the system is restarted. This
big bang approach to versioning is practical only in circumstances
where there is a single controlling authority, and even in that case, it
carries with it all manner of problems. The process can take a considerable
amount of time, leaving the system out of commission for hours if not days.
This can result in significant losses if the system is a key component of a
revenue generating business process and the cost of coordinating the system
overhaul can also be quite costly as well.
The "big bang" approach is appropriate when the new version is radically
different from its predecessor. But in many cases, the changes are
incremental and often a consumer could, in practice, cope with the new
version. For example, it might be that there are many messages that don't use
any features of the new version or perhaps it is appropriate to simply ignore
elements that are not recognized.
For example, consider two services exchanging messages. Imagine that some
future version of the language that they are using defines a new "priority"
element. Because producers and consumers are distributed, it may happen that
an old consumer, one unprepared for a priority element, encounters a message
sent by a newer producer.
If big bang versioning is used, old systems will reject the new message.
However, if the versioning strategy allowed the old consumer to simply ignore
unrecognized content, it's quite possible that other components of the system
could simply adapt to the previous behavior. In effect, the old system would
ignore the priority element and its descendents so it would "see" a message
that looks just like the old format it is expecting.
For the producer, the result would be that the request is fulfilled,
though perhaps in a more or less timely fashion than expected. In many cases
this may be better behavior than receiving an error. In particular, producers
using the new format can be written to cope with the possibility that they
will be speaking to old consumers.
If the new system needs to make sure that priority is respected, then it
can change the purchase order's name or namespace to indicate that the new
behavior is not considered backwards compatible.
What is needed is some sort of middle ground solution. An evolving system
should be designed with backwards and forwards compatibility in mind.
One approach to mitigate against a complete overhaul is to run multiple
versions of the system. One variant is offering both versions of the system,
for example by using different URIs for the old and new resources. The
request to one resource gets mapped to the other resource behind the scenes
using a proxy or gateway. This "alternative" approach works when the
intermediary can completely handle or generate the new information (for
backwards compatibility) or ignore the new information (for forwards
compatibility). For example, adding SSL security to a resource changes the
URI but a Web server can typically handle mapping the https: URI to the older
http: URI. If both URIs are maintained, then the addition is a compatible
change. Another example is where new information is required, such as the
priority, and the intermediary can apply a default value to provide the
required priority. However, this too has its costs as multiple versions of
the software must be supported and maintained over time and there is the
added cost of developing the proxy or gateway between the two environments.
Further, this does not work in scenarios where the intermediary cannot
generate the new required content. For example, if a middle name is required
in V2, a middle cannot be generated from just a family and a given name.
Having multiple versions naturally leads to identifying versions. One
approach to version identification is to use version numbers, with a goal of
using "major version" changes for incompatible changes and "minor version"
changes for compatible ones. The version numbers can be contained in the
instance documents, in the protocol messages containing in the instance
document, or the address for the protocol messages (ie
http://example.com/foo/v2).
Unfortunately, version numbers often wind up looking very similar to the
big bang approach. In many approaches, each language is given a version
identifier, almost always a number, that's incremented each time the language
changes. Although it's possible to design a system with version numbers that
enables both backward and forward compatibility - for example XSLT -
typically a version change is treated as if that the new language is not
backwards compatible with the old language.
Some efforts, such as HTTP, try to have the best of both worlds by
allowing for extensibility (in HTTP's case, via headers) as well as version
numbers that explicitly identify when a new version is backwards compatible
with an old version.
One argument in favor of version numbers is that they allow one to
determine what is a 'new version' and what is an 'old version'. But in
practice this is not necessarily true. For example, RSS has 0.9x, 1.x, and
2.x versions, all being actively developed in parallel. In effect the version
numbers, even though they appear to be ordered, are simply opaque
identifiers. Using version numbers does not gaurantee that version 1+x has
any particular relationship to version 1.
The self-describing and extensible nature of XML markup, and the addition
of XML Namespaces, provide a much better framework for developing languages
that can evolve.
8 Version Identification Strategies
There are a large variety of version identification designs. They range
from many namespaces to only 1 namespace for all versions of a language. A
few of the most common are listed below and described in more detail
later.
all components in new namespace(s) for each version
ie version 1 consists of namespaces a + b, version 1.1 consists of
namespaces c + d; or version 1 consists of namespace a, version 1.1
consists of namespace b.
all new components in new namespace(s) for each compatible
version
ie version 1 consists of namespaces a + b; version 1.1 consists of
namespaces a + b + c; version 2.0 consists of namespaces d + e.
all new components in existing or new namespace(s) for each
compatible version
ie version 1 consists of namespace a, version 1.1 consists of
namespace a, version 2 consists of namespace b; or version 1 consists of
namespace a, version 1.1 consists of namespace a + b.
all new components in existing or new namespace(s) for each version
and a version identifier
ie version 1 consists of namespace a + b + version attribute _1_,
version 2 consists of namespace c + d + version attribute _2_.
all components in existing namespace(s) for each version (compatible
and incompatible) and a version identifier
ie version 1 consists of namespace a + version attribute _1.0_,
version 1.1 consists of namespace a + version attribute _1.1_, version
2.0 consists of namespace a + version attribute "2.0".
Whatever the design chosen, the language designer must decide the
component name, namespace name, and any version identifier for new and all
existing components. The trade-offs between the decisions relate to the
importance of:
Supporting Compatible evolution.
namespaces for identifying compatible components. Changing namespace
names is typically a very invasive change
A complete Schema for the language. We will see how some designs
preclude full Schema description
Use of generic XML and namespace only (precluding vocabulary
specific versions) tools. This itself is a trade-off because some generic
XML tools (like XPath) are more difficult to use with multiple namespaces
containing the same "thing", like XHTML's P element.
Elaborating on these designs is illustrative.
8.1 Version Strategy: all
components in new namespace(s) for each version (#1)
The following names would be valid:
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
</name>
<name xmlns="http://www.example.org/name/2">
<given>Dave</given>
<family>Orchard</family>
<middle>Bryce</middle>
</name>
<name xmlns="http://www.example.org/name/3">
<given>Dave</given>
<family>Orchard</family>
<mid:middle xmlns:mid="http://www.example.org/name/3/mid/1">Bryce</mid:middle>
</name>
<name xmlns="http://www.example.org/name/3">
<given>Dave</given>
<family>Orchard</family>
<middiffdomain:middle xmlns:middiffdomain="http://www.example.com/mid/1">Bryce</middiffdomain:middle>
</name>
The 2nd example shows all the components in the same new
namespace. The 3rd and 4th example show an additional middle
element in 2 different namespace names. The 3rd example comes from a
namespace name that is in the same domain as the name element_s new namespace
name. One reason for 2 namespaces is to modularize the language. The
4th example shows a namespace name from a different domain for the
middle. It is probable that the mid:middle was created by the name author,
and the middiffdomain:middle was created by a 3rd party.
8.2 Version Strategy: all
new components in new namespace(s) for each compatible version (#2)
In this strategy, the following names would be valid:
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
</name>
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
<mid:middle xmlns:mid="http://www.example.org/name/mid/1">Bryce</mid:middle>
</name>
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
<middiffdomain:middle xmlns:middiffdomain="http://www.example.com/mid/1">Bryce</middiffdomain:middle>
</name>
The 2nd and 3rd example show an additional middle
element in 2 different namespace names. The first middle, the 2nd
example, comes from a namespace name that is in the same domain as the name
element_s namespace name. The 3rd example shows a complete
different namespace name for the middle. It is probable that the mid:middle
was created by the name author, and the middiffdomain:middle was created by a
3rd party.
8.3 Version Strategy: all
new components in new or existing namespace(s) for each compatible version
(#3)
In this strategy, the following names would be valid:
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
</name>
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
<middle>Bryce</middle>
</name>
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
<mid:middle xmlns:mid="http://www.example.org/name/mid/1">Bryce</mid:middle>
</name>
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
<middiffdomain:middle xmlns:middiffdomain="http://www.example.com/mid/1">Bryce</middiffdomain:middle>
</name>
The 2nd example shows the use of the optional middle name in
the name namespace. The 3rd and 4th example show an
additional middle element in 2 different namespace names. The first middle,
the 3rd example, comes from a namespace name that is in the same domain as
the name element_s namespace name. The 4th example shows a
complete different namespace name for the middle. It is probable that the
mid:middle was created by the name author, and the middiffdomain:middle was
created by a 3rd party.
8.4 Version Strategy: all
new components in existing or new namespace(s) for each version and a version
identifier(#4)
Using a version identifier, the name instances would change to show the
version of the name they use, such as:
<name xmlns="http://www.example.org/name/1" version="1.0">
<given>Dave</given>
<family>Orchard</family>
</name>
<name xmlns="http://www.example.org/name/1" version="1.1">
<given>Dave</given>
<family>Orchard</family>
<middle>Bryce</middle>
</name>
<name xmlns="http://www.example.org/name/1" version="1.1">
<given>Dave</given>
<family>Orchard</family>
<mid:middle xmlns:mid="http://www.example.org/name/mid/1">Bryce</mid:middle>
</name>
<name xmlns="http://www.example.org/name/1" version="1.0">
<given>Dave</given>
<family>Orchard</family>
<mid:middle xmlns:mid="http://www.example.org/name/mid/1">Bryce</mid:middle>
</name>
<name xmlns="http://www.example.org/name/1" version="2.0">
<given>Dave</given>
<family>Orchard</family>
<mid:middle xmlns:mid="http://www.example.org/name/mid/1">Bryce</mid:middle>
</name>
<name xmlns="http://www.example.org/name/2" version="2.0">
<given>Dave</given>
<family>Orchard</family>
<middle>Bryce</middle>
</name>
The last two examples show that the middle is now a mandatory part of the
name. This is indicated by just the version number or a new namespace plus
version number.
A significant downside with using version identifiers is that software
that supports both versions of the name must perform special processing on
top of XML and namespaces. For example, many components _bind_ XML types into
particular programming language types. Custom software must process the
version attribute before using any of the _binding_ software. In Web
services, toolkits often take SOAP body content, parse it into types and
invoke methods on the types. There are rarely _hooks_ for the custom code to
intercept processing between the _SOAP_ processing and the _name_ processing.
Further, if version attributes are used by any 3rd party extensions_say
mid:middle has a version_then the schema cannot refer to the correct
middle.
8.5 Version Strategy: all
components in existing namespace(s) for each version and a version
identifier(#5)
Using a version identifier, the name instances would change to show the
version of the name they use, such as:
<name xmlns="http://www.example.org/name/1" version="1.0">
<given>Dave</given>
<family>Orchard</family>
</name>
<name xmlns="http://www.example.org/name/1" version="1.1">
<given>Dave</given>
<family>Orchard</family>
<middle>Bryce</middle>
</name>
<name xmlns="http://www.example.org/name/1" version="2.0">
<given>Dave</given>
<family>Orchard</family>
<middle>Bryce</middle>
</name>
The 2nd example shows that the middle is an optional part of
the name. The last example shows that the middle is a mandatory part of the
name.
A downside with using new namespace names is that some tools, like XPath,
can be harder to use in the face of new namespace names. Software that
extracts the given and family name based upon the expanded name will often
break if a new namespace name is used.
9 Indicating Incompatible
changes
Given adoption of the Must Ignore rule, it is often the case that the
creator of an extension or a new version wants to require that the consumer
understand the extension, overriding the Must Ignore rule. The previous
section showed how a version author could use new namespace names, element
names, or version numbers to indicate an incompatible change. An extension
author does not have these mechanisms available for indicating an
incompatible or mandatory extension. A language provider that wants to allow
extension authors to indicate incompatible extension must provide a mechanism
for indicating that consumers must understand the extension.
Good Practice
Provide Must Understand Rule: Container languages SHOULD
provide a _Must Understand_ model for indicating extensions that override a
default Must Ignore Rule.
This rule and the Must Ignore rule work together to provide a stable and
flexible processing model for extensions.
Must Understand flag
Arguably the simplest and most flexible over-ride technique is a Must
Understand flag that indicates whether the item must be understood. The SOAP
[8], WSDL [9], and WS-Policy [10] attributes and values for specifying understand are
respectively: soap:mustUnderstand=_1_, wsdl:required=_1_,
wsp:Usage=_wsp:Required_. SOAP is probably the most common case of a
container that provides a Must Understand model. The default value is 0,
which is effectively the Must Ignore rule.
A language designer can re-use an existing Must Understand model by
constraining their language to an existing Must Understand model. A number of
Web services specifications have done this by specifying that the components
are SOAP header blocks, which explicitly brings in the SOAP Must Understand
model.
A language designer can design a Must Understand model into their
language. A Must Understand flag allows the producer to insert extensions
into the container and use the Must Understand attribute to over-ride the
must Ignore rule. This allows producers to extend instances without changing
the extension element_s parent_s namespace, retaining backwards
compatibility. Obviously the consumer must be extended to handle new
extensions, but there is now a loose coupling between the language_s
processing model and the extension_s processing model. A Must Understand flag
is provided below:
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://www.example.org/name/1"
xmlns:name="http://www.example.org/name/1">
<xs:complexType name="name">
<xs:sequence>
<xs:element name="given" type="xs:string"/>
<xs:element name="family" type="xs:string"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
<xs:attribute ref="name:mustUnderstand"/>
<xs:anyAttribute/>
</xs:complexType>
<xs:attribute name="mustUnderstand" type="xs:boolean"/>
</xs:schema>
An example of an instance of a 3rd party indicating that a middle
component is an incompatible change:
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
<mid:middle xmlns:mid="http://www.example.org/name/mid/1"
name:mustUnderstand="true">
Bryce
</mid:middle>
</name>
Specification of a Must Understand flag must be treated carefully as it
can be computationally expensive. Typically a processor will either: perform
a scan for Must Understand components to ensure it can process the entire
document, or incrementally process the instance and is prepared to rollback
or undo any processing if an not understood Must Understand is found.
There are other refinements related to Must Understand. One example is
providing an element that indicates which extension namespaces must be
understood, which avoids the scan of the instance for Must Understand
flags.
It is also possible to re-use the SOAP processing model with it's
mustUnderstand.
<soap:envelope>
<soap:body>
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
</name>
</soap:body>
</soap:envelope>
<soap:envelope>
<soap:header>
<mid:middle xmlns:mid="http://www.example.org/name/mid/1"
soap:mustUnderstand="true">
Bryce
</mid:middle>
</soap:header>
<soap:body>
<name xmlns="http://www.example.org/name/1">
<given>Dave</given>
<family>Orchard</family>
</name>
</soap:body>
</soap:envelope>
9.1 Type changes
The various schema languages have a variety of defined mechanisms to
indicate extensions that are incompatible. For example, XML Schema provides
Type Extension and Substitution Groups. These mechanisms are described in
Part 2.
10 Extension versus
Versioning
The usage of namespace names for identifying components has led to the
interesting situation where the distinction between an extension and a
version can be quite blurred, depending upon the language designer_s
choices.
One rough way of thinking of these two concepts is that extension is
typically the addition of components over space; that is, designers other
than the language_s creator are adding components. Versioning is typically
the addition of components over time, under the designer_s explicit control.
In either case, a change to the language may be done in a compatible or an
incompatible way. The simple cases of extensions are compatible decentralized
additions and versions are compatible or incompatible centralized changes are
how we typically distinguish the terms. But these break down depending upon
how the language is designed.
There are a couple of scenarios that illustrate the ambiguity in these
terms. Imagine that version 1.0 of a Name consists of _First_ and _Last_
elements. A 3rd party author extends the Name with a _middle_ element in a
new namespace which they control.
In scenario 1, the Name author decides to formally incorporate the middle
name as an optional (and hence compatible) addition to the name, producing
version 1.1 of the Name type. They do this by referring to the third party_s
definition and namespace for middle names. This is typically considered a new
_version_ of the Name and would probably result in a new schema definition.
If the Name author re-uses namespace names for compatible revisions, there
will be no difference in an instance document containing middle that is of
Version 1.0 or Version 1.1 type. The instance documents are the same, and
thus the distinction between a _version_ and an _extension_ is meaningless
for an individual document.
In scenario 2, the middle author decides that the middle name is a
mandatory part of the Name type. They were provided a mechanism for
indicating an incompatible change and they use it. Now an instance of Name
with the middle is incompatible with version 1.0 of the Name. What _version_
of the Name is this middle, and is the middle an _extension_ or a _version_?
It isn_t 1.0. It_s probably more accurately thought of as a version defined
by the 3rd party. Again, the presence of the _extension_ is actually an
incompatible change.
These two examples_a 3rd party extension being added into a compatible
version and a 3rd party extension resulting in an incompatible version_show
the ability to specify (in)compatibility has blurred the distinction between
these two terms.
11 Conclusion
This Finding is intended to motivate language designers to plan for
versioning and extensibility in the languages from the very first version. It
details the downsides of ignoring versioniong. To help the language designer
provide versioning in their language, the finding describes a number of
questions, decisions and rules for using XML, XML Namespaces and schema
languages in XML language construction and extension. The main goal of the
set of rules is to allow language designers to know their options for
language design, and ideally make backwards- and forwards-compatible changes
to their languages to achieve loose coupling between systems.
12 References
- FOLDOC
- Free Online
Dictionary of Computing. (See
http://wombat.doc.ic.ac.uk/foldoc/.)
- FlexXMLP
- Flexible
XML Processing Profile. (See
http://www.upnp.org/download/draft-goland-fxpp-01.txt.)
- tcp
- RFC 793,
TCP (See http://www.ietf.org/rfc/rfc793.txt.)
- MIME
- RFC 1521,
MIME. (See http://www.ietf.org/rfc/rfc1521.txt.)
- HTML 2.0
- RFC 1866, HTML
2.0. (See http://www.ietf.org/rfc/rfc1866.txt.)
- WebDAV XMLIgnore post
- Yaron
GolandXML Ignore proposed for WebDAV (See
http://lists.w3.org/Archives/Public/w3c-dist-auth/1997AprJun/0190.html.)
- WebDAV
- RFC 2518,
WebDAV (See http://www.ietf.org/rfc/rfc2518.txt.)
- HTML 4.0
- HTML
4.0. (See
http://www.w3.org/TR/1998/REC-html40-19980424/.)
- TBL Mandatory
Extensions
- Berners-Lee.
Web Architecture: Mandatory extensions. (See
http://www.w3.org/DesignIssues/Mandatory.html.)
- TBL
Extensible languages
- Berners-Lee.
Web Architecture: Extensible languages. (See
http://www.w3.org/DesignIssues/Extensible.html.)
- TBL
Evolution
- Berners-Lee.
Web Architecture: Evolvability. (See
http://www.w3.org/DesignIssues/Evolution.html.)
- Web
Architecture: Extensible Languages
- Berners-Lee
and Connolly, ed. Web Architecture: Extensible Languages
World Wide Web Consortium, 1998. (See
http://www.w3.org/TR/1998/NOTE-webarch-extlang-19980210.)
- HTML Document
types
- Connolly, ed.
HTML Document dialects World Wide Web Consortium,
1996. (See http://www.w3.org/MarkUp/WD-doctypes.)
- SOAP 1.2
- W3C Recommendation, SOAP
1.2 Part 1: Messaging Framework (See
http://www.w3.org/TR/SOAP/.)
- WSDL 1.1
- W3C Note, WSDL
1.1 (See http://www.w3.org/TR/WSDL/.)
- XML 1.0
- W3C Recommendation, XML
1.0 (See http://www.w3.org/TR/REC-xml.)
- XInclude
- W3C Working Draft, XML
Inclusions (See http://www.w3.org/TR-Xinclude.)
- XML
Namespaces
- W3C
Recommendation, XML Namespaces (See
http://www.w3.org/TR/REC-xml-names.)
- XML
Schema Part 2
- W3C Recommendation,
XML Schema, Part 2 (See
http://www.w3.org/TR/xmlschema-2.)
- XML Schema Wildcard Test
Collection
- XML
Schema Wildcard Test collection (See
http://www.w3.org/XML/2001/05/xmlschema-test-collection/result-ms-wildcards.htm.)
- XFront Schema Best Practices
- XFront
Schema Best Practices (See
http://www.xfront.com/BestPracticesHomepage.html.)
- XML.com Schema Design Patterns
- Dare
ObasanjoXML.com Schema design patterns (See
http://www.xml.com/pub/a/2002/07/03/schema_design.html.)
- Dave
Orchard writings on Extensibility and Versioning
- Dave
Orchard writings on extensibility and versioning (See
http://www.pacificspirit.com/Authoring/Compatibility.)
13 Acknowledgements
The author thanks the many reviewers that have contributed to the article,
particularly David Bau, William Cox, Ed Dumbill, Chris Ferris, Yaron Goland,
Hal Lockhart, Mark Nottingham, Jeffrey Schlimmer, Cliff Schmidt, and Norman
Walsh.
