Open Source is not Interoperability

Posted by Cheng Soon Ong on November 26, 2008

Trying to prepare some thoughts about interoperability to be discussed at the NIPS workshop, I came across a bunch of websites roughly in the following order:

• a rather negative article about the state of open source software and how they interoperate at eweek.com.

• a very positive blog at ugotrade which talks about OpenSim and how it will be the next hot thing.

• a post about why open source and interoperability are really two different things.

Quoting the third author:

1. Interop is not open source.
2. Interop does not require open source implementations
3. Open source does not guarantee Interop

While one thinks that somehow it is natural for open source developers to make use of other bits of (open source) software, it usually doesn't happen. For me, interoperability can occur in two ways: the first being having a common set of protocols (as argued for by the third post above), and/or the second which is integrating another software library or method. In some sense, the "integration" idea also requires a set of protocols or APIs. It may be that I'm just being pedantic about trying to semantically differentiate between protocols and APIs. But the main idea remains: We need software that talks to other bits of software.

However, if both pieces of software are open source, we can do more than just have software that talks to other bits of software (which is why OpenSim is raising so much interest). In the process of having to push together two software projects, we may be able to come up with better interfaces between them. This is especially true in the research area (which in some sense practices carpentry) where it is not that clear from the start how programs should interact. For supervised machine learning, datasets are a good place to start. It seems "obvious" that this is one place where different machine learning algorithms can interface with each other. Even in this "simple" interface, there is a multitude of data formats and standards. Another quite fruitful area is in convex optimization, where there are several projects (even here on mloss.org) which easily link to different back ends, or several solvers which are used by various front ends. Interestingly, here the interfaces are actually dictated by the mathematics, and the software implementations are just mirroring these forms. I think it is within our reach to have these kinds of interoperability for many other areas of machine learning.

As for the long term goal of software systems being well integrated in the application specific fashion, I think we still have a way to go yet...