Remarkable Progress of WP5 : The technical implementation of the framework

The work package 5, “Adaptive experiment-driven complex analytics framework”, with a focus on providing the core services of the framework of ExtremeXP, has continued to make strides in achieving its defined objectives since the third month of the start of the project. The project partners VRIJE University of Amsterdam, ICCS, Activeeon, Charles University, Delft University of Technology, Intracom Telecom, and BitSparkles are making major contributions towards the goals of the work package, the first results of which will be published in deliverable D5.2 in May 2024.  The contributions are made through the tasks described in the following.

In the first task, led by VRIJE University of Amsterdam (VUA), the Experimentation Engine is under development, whose task is to create and manage experiments. It also continuously determines which complex analytics variant of the workflow should be running. The experimentation engine selects the best analytics variant based on several parameters including the user intent, the constraints and goals defined by the user, etc. The selected complex analytics variants are constantly sent to the Executionware to be executed.

To support the adaptive experiment planning of the Experimentation Engine, the data abstraction layer is developed by Charles University (CUNI), providing a robust monitoring service. This data abstraction layer handles all the data including metadata of analytics processes, input and output data and metrics of the experiments, the status of running experiments, etc. The implementation of the data layer and monitoring services is based on the IVIS platform which is also provided by Charles University (CUNI).

Complex Workflow Scheduling, led by Activeeon (AE), enhances the execution of complex analytics across diverse environments. Leveraging AE’s ProActive Workflows and Scheduling technology, particularly the Scheduling Abstraction Layer (SAL), this task streamlines workflow deployment on heterogeneous infrastructures. SAL facilitates seamless integration with monitoring frameworks and control access tools, alongside external frameworks like Big Data technologies. Extending ProActive Workflow capabilities enables experiment creation alignment and paves the way for Analytics-OPS, offering sophisticated analytics and Analytics as a Service.        

ICCS with participation with Delft University of Technology (TUD) led the task on implementing context-aware access control for the ExtremeXP framework. Using attribute-based access control (ABAC), it defines and enforces data authorization based on contextual information. This involves modeling access control policies, handling context, and enforcing access control. Leveraging the Meta Data Schema (MDS89) and WSO2 Balana-based XACML authorization engine, the project ensures flexible and adaptive policies for protecting extreme data and analytics results. The task ensures flexible and runtime adaptive authorization policies for extreme data protection and experimentation driven analytics results within the ExtremeXP framework. 

ICCS with participation with TUDelft University of Technology (TUD), also leads another task to the secure and distributed management of datasets, knowledge assets, and experimentation-based learning outcomes within the ExtremeXP framework. The task addresses the challenge of managing data and knowledge assets securely and efficiently, considering their decentralized nature across different administrative domains and entities. The focus is on designing, developing, and integrating mechanisms to safeguard these assets, enabling distributed data sharing, learning, and experimentation driven analytics. This enhances ExtremeXP metadata editing capabilities, managing knowledge bases, and implementing data sharing lifecycle management.

All in all, the contribution of the partners under work package 5, provides the technical implementation of the software components of the ExtremeXP framework for planning, optimizing, and running the experiments, as well as semantics regarding data models, secure access control, and knowledge management. Lastly, the interfaces developed in this work package provide efficient interaction with the modules being simultaneously implemented under work packages 3 and 4. The requirements of the candidate users of the framework and the needed attributes of the system are constantly being managed and evaluated through interaction with the 5 use case partners of the project. The artifact of WP5, the adaptive experiment-driven complex analytics framework, provides a novel experience in efficient, accurate, and fit-for-purpose experiment-driven analysis.