Challenge 1: Core functionalities of the ExtremeXP framework
In this challenge, you will decompose a machine learning workflow into tasks, define it using the DSL, configure an experiment with multiple model variants and hyperparameters, execute it on ProActive, and visualize the results.
Estimated Time : 60–90 minutes
Difficulty Level : Intermediate (intermediate DSL knowledge, some Python understanding)
Every experiment in ExtremeXP begins with a model. You can create this model in one of two ways:
- Using the DSL Framework (the method we will use in this tutorial).
- Using the Graphical Editors
A third option, the Intent-based Editor, can also generate experiments, but it is not covered in this tutorial.
Once your experiment model is defined, it is passed to the Experimentation Engine, a Python module that serves as the core interpreter. The Experimentation Engine reads your experiment description and decides which workflows need to be executed.
To run the workflows, the Experimentation Engine communicates with the Execution Engine. This component is built on top of the Proactive Scheduling and Abstraction Layer, although other workflow tools such as Kubeflow could also be used. After the workflows are complete, their results are sent back to the Experimentation Engine.
The Experimentation Engine then updates two key components:
- Data Abstraction Layer (DAL):
Stores and retrieves executed experiments, workflows, and their associated metrics. You can think of DAL as the system’s experiment memory.
- Decentralized Data Management (DDM):
Handles dataset storage and access across the framework. DDM includes a graphical interface for uploading and tagging datasets, as well as an API used by other components to load or save data.
Finally, results become available through two visualization tools:
- Experiment Visualization:
Allows users to explore detailed results of individual experiments.
- Experiment Cards:
Provides a high-level overview of all past experiments, helping analysts identify patterns, extract best practices, and plan future studies.
Prerequisites
-
Format
CSV with mixed numeric and categorical features
-
Size
1,500 rows, 14 features
-
Target
Binary column churn (Yes/No)
-
Features include
⦁ Numeric : tenure (months), monthly_charges, total_charges, number_support_calls, pages_consulted_per_month
⦁ Categorical : internet_service_type, contract_type, online_security, tech_support, paperless_billing
Download the provided Python scripts from the shared folder
Read through each script
⦁ Task 2 - Preprocess Data
⦁ Task 3 - Train Model
⦁ Task 4 - Evaluate Model
⦁ Task 5 - Export Artifacts (Optional)
Understand the input and output of each script
Identify the parameters each script expects and where they come from
-
Create a file named churn_workflow.xxp that defines the workflow structure, task ordering, and data flow
-
Define all 5 tasks with correct implementation paths, inputs, outputs, and parameters
-
Establish the data dependencies and execution order
-
Ensure parameter names match those in the Python scripts
-
Create a file named churn_experiment.xxp that defines the experiment space to explore
-
Declare the experiment parameters and their value ranges
-
Bind experiment parameters to workflow task parameters
-
Define output metrics
You will be working with the “”
- The second task should have a parameter TODO
- Maybe also a metric?
- Maybe also input/output datasets (local)?
Success criteria
Workflow DSL Checklist
- All five tasks are defined
- Each task specifies its implementation script path
- Input/output artifacts are correctly declared
- Task parameters match the script's expected parameters
- Data flow is explicit (outputs of one task feed into inputs of the next)
Experiment DSL Checklist
- Experiment references the correct workflow
- All experiment parameters are declared
- Bindings correctly connect experiment parameters to workflow task parameters
- Output metrics are specified
Provide screenshots that show
