... | ... | @@ -2,6 +2,7 @@ |
|
|
title: How to set up MLflow
|
|
|
---
|
|
|
|
|
|
|
|
|
# How to set up MLFlow on the cluster (with VSCode)
|
|
|
|
|
|
*Work in progress*
|
... | ... | @@ -20,6 +21,8 @@ module load anaconda # to load conda |
|
|
conda create -n mlflow-env python=3.10 # name your environment as you like. I chose "mlflow-env"
|
|
|
source activate mlflow-env # activate your environment
|
|
|
```
|
|
|
> **Note:** Instead of anaconda and pip, you could also use uv, to set up and manage your environment. Click [here](https://gitlab.pik-potsdam.de/sdmc/cluster-wiki/-/wikis/Tutorials/UV-for-Python), to find out more.
|
|
|
|
|
|
- Install MLFlow, ipykernel, and other Python packages you need: <br>
|
|
|
* `pip install mlflow` Note: This will also install `numpy, pandas, scikit-learn` and others <br>
|
|
|
* `pip install ipykernel` To run jupyter notebooks in VSCode <br>
|
... | ... | @@ -56,9 +59,23 @@ If this is not set within your notebook or runtime environment, the runs will be |
|
|
|
|
|
3. Run your ML run (e.g. the quickstart notebook) and look at your logged data in the mlflow UI: <br>
|
|
|
|
|
|
## Forward a port from the cluster to your local browser to view the MLFlow UI
|
|
|
## Open the MLflow UI
|
|
|
|
|
|
You should be able to start the mlflow ui via the command
|
|
|
|
|
|
```bash
|
|
|
mlflow ui
|
|
|
```
|
|
|
|
|
|
or `uv run mlflow ui` (if you use uv). By default this will run the ui on the local web server [http://127.0.0.1:5000](http://127.0.0.1:5000). If the connection is already in use, you need to manually set a different port, e.g. 5001 instead of 5000:
|
|
|
|
|
|
To view the UI, you must create a secure tunnel from your local localhost:YourPortNumber (on your laptop) to the remote 127.0.0.1:YourPortNumber (on the cluster). In VScode:
|
|
|
```bash
|
|
|
mlflow ui --host 127.0.0.1 --port 5001
|
|
|
```
|
|
|
|
|
|
If you did not set a server, new runs or file changes in mlruns/ may not appear immediately. To refresh, you may need to restart the mlflow ui process.
|
|
|
|
|
|
If there are any other problems with `mlflow ui` on the cluster, you can alternatively create a secure tunnel from your local localhost:YourPortNumber (on your laptop) to the remote 127.0.0.1:YourPortNumber (on the cluster). In VScode:
|
|
|
* Open the Command Palette: Ctrl + Shift + P (or Cmd + Shift + P on Mac)
|
|
|
* "Forward a Port": When prompted, enter: YourPortNumber e.g. 8081
|
|
|
* VS Code will now show a forwarded port in the “Ports” panel at the bottom. Click on the globe symbol to see the UI in your local browser.
|
... | ... | @@ -68,13 +85,6 @@ To view the UI, you must create a secure tunnel from your local localhost:YourPo |
|
|
|
|
|
{width=1438 height=523}
|
|
|
|
|
|
If you did not set up a tracking server, you can access the MLFlow UI simply via this command:
|
|
|
|
|
|
`mlflow ui --port 8081` (this is equivalent to `mlflow ui --host 127.0.0.1 --port 8081`)
|
|
|
Then forward the prompt to your local computer (see the steps above).
|
|
|
|
|
|
Note: The UI in this mode is read-only, and new runs or file changes in mlruns/ may not appear immediately. To refresh, you may need to restart the mlflow ui process.
|
|
|
|
|
|
|
|
|
|
|
|
|