Today I'd like to talk about Opticks and other programming languages. The next release of Opticks will contain an IDL (the ITT programming language) scripting extension and I've begun work in COAN on a Python scripting extension.
Opticks is written in C++ and Opticks extensions are also written in C++. C++ is very much a tragic hero in the Greek sense. For all it's worth and praise, it has a tragic flaw; C++ is hard and awkward to program. Skilled programmers find it difficult or impossible to rapidly prototype in C++. Less skilled programmers and non-programmers find it quite impossible to develop non-trivial extensions. The efficiency and flexibility afforded by C++ is not always needed in an extension and it would be a boon to provide alternate languages for development. In addition to ease of use, interpreted scripting languages often provide interactive capabilities. This can be leveraged in Opticks for experimentation and algorithm development.
In 2008, a number of Ball developers explored this problem. The project was a more complete extension of exploratory work done previously by multiple parties. In short, the problem is difficult. Opticks uses many advanced features of C++ including templates and multiple inheritance. These concepts do not always translate to other languages. In order to completely map the Opticks plug-in API to other languages, we need to rewrite the API using a common set of features. The proposed solution is to rewrite the API using the XPIDL interface specification. It's a minor modification of the CORBA interface description language (IDL). The implementations of these interfaces can be rewritten (or wrapped) as XPCOM components. Once this is done, most of the API is available to any language with XPCOM bindings including C++, JavaScript, and Python. This requires significantly more work from the core developers and a mental shift to the XPCOM programming model for plug-in developers. The change can be done in a source compatible (and possible binary compatible) way but this increases maintenance burden even more.
Ok, so it's very difficult to share the full Opticks API with other programming languages, but do we really need the ability to write full-fledged Opticks extensions? Not if we are primarily interested in interactive and scripting capability. A scientist or developer who wants to rapidly prototype a new algorithm does not need to efficiently access all the data structures in Opticks and create seamless GUIs, she only needs to get at key pieces or data, adjust some view properties, and perhaps display a basic GUI or two. This turns out to be a much easier problem to solve and I'll describe the key pieces of needed functionality in a moment.
First, I'd like to talk briefly about standardization. We can't, and shouldn't, directly map the C++ objects and methods into a target language. Different languages provide features which often set them apart from other languages and we should utilize these as appropriate. This doesn't mean we should not provide some consistency in the interfaces. This makes it easy to switch from one language to another without needing to learn a completely new API. This is a concept embraced by the Qt toolkit (which Opticks uses extensively). Basically, functions that perform similar tasks should have similar names, similar calling conventions, and similar behaviors. When it makes sense to change behavior, etc. that's ok but there should be a good reason for the change.
There are a few key capabilities which are needed in a scripting environment. The most important capability for most scripting scenarios is access to raster element data. This is the primary data element in Opticks and most scripting sessions will need read and possibly write access to existing data. Ideally, this would avoid duplication of data for memory efficiency but this is not always possible. The ability to copy portions of a data cube into the scripting environment is a reasonable compromise. Transparent write-back support is not always possible in a target language, so the ability to push data back to Opticks is a useful alternative to writable data access.
AOIs are frequently used to delineate areas to process or as general bi-level raster data containers. If an AOI is viewed as a bi-level raster element, a similar interface to the one described in the previous paragraph is appropriate. The other may data element input in Opticks is a signature. Signatures are collections of vector data and can also be viewed as specialized cases of raster elements.
All this data access is not very useful without some basic view manipulation capability. New views and layers need to be created to display results and this capability can be integrated into the data access functions as additional parameters. Some basic layer property access should also be made available so threshold levels, pseudocolor classes, etc. can be controlled.
One of Opticks' more recent features, and a big selling point for many is the animation capability. Creation and manipulation of animations are nice features but by far the most interesting capability is an animation callback mechanism. The ability to execute scripting code in response to frame changes allows prototyping of video algorithms such as background suppression, object finding, and tracking. Annotations are useful for displaying the results of these sorts of algorithms so creation and modification of annotation elements would be a good addition to our evolving API.
I've presented a few key areas which need to be available to scripting languages. Even though the implementations and concrete APIs will differ from language to language, there are definitely some common procedures which could be added to a library of scripting language utility routines. Also, some of these "soft" interfaces could be documented and discussed on a wiki page.