New Preprint: Active-Code Replacement in the OODIDA Data Analytics Platform

A preprint of our paper “Active-Code Replacement in the OODIDA Data Analytics Platform” is now available on arXiv. It describes a key feature of OODIDA, which is a distributed system for data analytics for the automotive industry, targeting a fleet of reference vehicles. With active-code reloading, it is possible to replace code for custom computations without taking any component of the system down. The abstract is reproduced below:

Active-Code Replacement in the OODIDA Data Analytics Platform
Gregor Ulm, Emil Gustavsson, Mats Jirstrand

OODIDA (On-board/Off-board Distributed Data Analytics) is a
platform for distributing and executing concurrent data analysis
tasks. It targets a fleet of reference vehicles in the
automotive industry and has a particular focus on rapid
prototyping. Its underlying message-passing infrastructure has
been implemented in Erlang/OTP, but the external applications
for user interaction and carrying out data analysis tasks use
a language-independent JSON interface. These applications are
primarily implemented in Python. A data analyst interacting with
OODIDA uses a Python library. The bulk of the data analytics
tasks are performed by clients (on-board), while a central server
performs supplementary tasks (off-board). OODIDA can be
automatically packaged and deployed, which necessitates restarting
parts of the system, or all of it. This is potentially disruptive.
To address this issue, we added the ability to execute
user-defined Python modules on both the client and the server,
which can be replaced without restarting any part of the system.
Modules can even be swapped between iterations of an ongoing
assignment. This facilitates use cases such as iterative A/B
testing of machine learning algorithms or deploying experimental
algorithms on-the-fly. Active-code replacement is a key feature
of our system as well as an example of interoperability between
a functional and a non-functional programming language.

Leave a Reply

Your email address will not be published. Required fields are marked *

Spammer prevention; the answer is an integer: * Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.