Skip to main content

How to Log a Folder of Files to MLflow with log_artifacts

· 5 min read
Jagdish Kumawat
Founder @ Dewiride

When a step of your code writes several files, such as a set of reports or charts, logging them one by one gets tedious. mlflow.log_artifacts() uploads a whole folder to the run in one call, subfolders included. One detail decides where the files end up: artifact_path. This guide logs a folder of two reports, with real output from MLflow 3.18.0.

Syntax​

Syntax
mlflow.log_artifacts("<folder on disk>", artifact_path="<folder in the run>")
PartWhat it means
"<folder on disk>"The folder whose contents to upload: "Log all the contents of a local directory as artifacts of the run"
artifact_path=The folder in the run to put them in. Without it, the contents go to the top level of the run's artifacts, and the folder's own name isn't kept
The current runMLflow logs to the active run, and starts one if none is active

Subfolders keep their structure: reports/charts/c.txt on disk becomes reports/charts/c.txt in the run.

Prerequisites​

Step 1: Log a Folder​

The example writes two small text files into a folder called reports, then logs the folder to a run, keeping its name. Create log_artifacts.py:

The log_artifacts.py script in a macOS-style code window: it creates a reports folder with pathlib, writes summary.txt and columns.txt into it, and inside a run named log-a-folder calls mlflow.log_artifacts on reports with artifact_path set to reports.

log_artifacts.py
from pathlib import Path

import mlflow

mlflow.set_tracking_uri("http://127.0.0.1:8080")
mlflow.set_experiment("artifacts-demo")

reports = Path("reports")
reports.mkdir(exist_ok=True)
(reports / "summary.txt").write_text("5 houses, 3 columns\n")
(reports / "columns.txt").write_text("rooms\nsize_m2\nprice\n")

with mlflow.start_run(run_name="log-a-folder"):
mlflow.log_artifacts("reports", artifact_path="reports")
LineWhat it does
reports.mkdir(exist_ok=True)Creates the folder, or leaves it if it already exists
The two write_text linesWrite a file each into it. How to Build File Paths in Python with pathlib explains the /
mlflow.log_artifacts("reports", artifact_path="reports")Uploads everything in reports into a folder of the same name in the run
Terminal
uv run python log_artifacts.py
Output
🏃 View run log-a-folder at: http://127.0.0.1:8080/#/experiments/9/runs/e1347de3a1b44fe2bbce7bdca3f5272d
🧪 View experiment at: http://127.0.0.1:8080/#/experiments/9

Step 2: See the Folder in the UI​

Open the run, click Artifacts, expand reports and click summary.txt:

The Artifacts tab of the log-a-folder run: a reports folder holding columns.txt and summary.txt, with summary.txt selected and its 20-byte content shown, 5 houses, 3 columns.

Both files are inside reports, as on disk.

Without artifact_path​

mlflow.log_artifacts("reports"), without artifact_path, uploads the same two files to the top level of the run: summary.txt and columns.txt sit next to any other artifact, and no reports folder appears. That's how MLflow's reference describes it, as the folder's contents rather than the folder. Pass artifact_path whenever the folder's name matters, or when a run logs more than one folder.

mlflow.log_artifact("reports"), the single-file function given a folder, behaves the other way round: it keeps the name and creates reports/ in the run.

Common Mistakes​

MistakeWhat happensDo this instead
Leaving out artifact_pathThe files land at the top level, mixed with the run's other artifactsPass artifact_path="<name>"
A misspelled folder nameNo error, and nothing is logged: a folder that doesn't exist has no contents to uploadCheck that the folder exists, for example with Path("reports").is_dir()
Logging a folder that also holds large or private filesEverything in it is uploadedWrite only what belongs to the run into the folder
Logging the same folder twice in one runFiles with the same names are replaced; others are addedLog each folder once, after all its files are written

Troubleshooting​

SymptomLikely causeFix
The run's Artifacts tab is emptyThe folder path was wrong or the folder was emptyPrint the folder's contents before logging
Files appear at the top level instead of in a folderartifact_path was left outAdd it, as above

FAQ​

Should I log a temporary folder?​

Often, yes: write the files into a tempfile.TemporaryDirectory() and log it before the with block ends, so nothing is left on disk: How to Create a Temporary Folder in Python with tempfile.

Can I log a folder into a subfolder of the run?​

Yes: artifact_path="reports/daily" creates both levels.

What's the difference from log_artifact?​

log_artifact uploads one file, or a folder under its own name; log_artifacts uploads a folder's contents to the place artifact_path names: How to Log a File to MLflow with log_artifact.

Conclusion​

mlflow.log_artifacts("reports", artifact_path="reports") uploaded a folder of two files, keeping its name and structure in the run. Without artifact_path the same files land at the top level, and a misspelled folder logs nothing without an error, so check the folder and name its place in the run.

Additional Resources​