[ENH] Add save/load serialization for all skpro objects (#1072)

[patelchaitany]

Reference Issues/PRs

Fixes #1072

What does this implement/fix? Explain your changes.

• Add save and load methods to BaseObject (pickle and joblib backends)
• Standalone load() function in skpro.base
• capability:serializable tag for opt-out
• Round-trip tests for in-memory and file-based persistence

Does your contribution introduce a new dependency? If yes, which one?

What should a reviewer concentrate their feedback on?

Did you add any tests for the change?

Any other comments?

PR checklist

For all contributions

• I've added myself to the list of contributors with any new badges I've earned :-)
  How to: add yourself to the all-contributors file in the skpro root directory (not the CONTRIBUTORS.md). Common badges: code - fixing a bug, or adding code logic. doc - writing or improving documentation or docstrings. bug - reporting or diagnosing a bug (get this plus code if you also fixed the bug in the PR).maintenance - CI, test framework, release.
  See here for full badge reference
• The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.

For new estimators

• I've added the estimator to the API reference - in docs/source/api_reference/taskname.rst, follow the pattern.
• I've added one or more illustrative usage examples to the docstring, in a pydocstyle compliant Examples section.
• If the estimator relies on a soft dependency, I've set the python_dependencies tag and ensured
  dependency isolation, see the estimator dependencies guide.

[patelchaitany]

The save() method recursively traverses the object tree. Each sub-component (e.g., each estimator in a VotingProbaRegressor) gets saved to its own subfolder inside the zip. A manifest.json at the root acts as a blueprint - lists every component, its class, its location in the zip, and the topological load order (leaves first, root last). On load, read the manifest, load in topological order, reassemble the tree.

Recursion & stop condition:

Each class that has save/load capability (skpro BaseObject descendants) is responsible for its own serialization - save() is called recursively down the tree. The recursion stops when:

1. A component implements its own save/load but has no sub-components (leaf node) -saved as a structured entry in its own subfolder.
2. A component does not implement save/load (e.g., sklearn estimators, third-party objects) - falls back to pickle as a .pkl blob and is marked as "serialized_as": "pickle_fallback" in the manifest so it's transparent to the user.
3. If an object is not even picklable - raise an error, since there's no way to serialize it.

This avoids requiring every class to implement custom save/load logic (too much to ask), while keeping the manifest honest about what used structured save vs raw pickle fallback.

File structure (example: fitted VotingProbaRegressor with 2 sub-estimators):

voter_model.zip
├── manifest.json
├── root/
│   ├── _metadata
│   └── _obj
└── components/
    ├── estimators__0__1/    # r1 (unfitted)
    ├── estimators__1__1/    # r2 (unfitted)
    ├── estimators___0__1/   # r1 (fitted)
    └── estimators___1__1/   # r2 (fitted)

Inspired by the artifacts.yaml manifest pattern from the pytorch-forecasting EP - same core idea of manifest as single source of truth for what was saved, where, and how to reconstruct it.

[patelchaitany]

@fkiraly What do you think?

[fkiraly]

Very nice! I think this is great!

• may I kindly ask to document the file format somewhere? I think the save docstring should either link, or contain, the description of what is being saved exactly where.
  • in sktime, we will document the format in the docs (next to the ts format) and reference it there
• I would also kindly like a review by @geetu040 and @phoeenniixx to ensure we are working towards a consistent format in all three packages
  • in particular, if we have composites that use objects from two or three of then, the recursive format should match

[phoeenniixx]

Sorry, I have a silly question: Why dont we save the copy of the class instance itself in a var and return that? Why do we need use pickle here? What I am able to understand is we are storing the class inside serialized (inside RAM) and if the session is complete, it would be lost anyway?

Or is it to provide data sharing over a network or something?

[patelchaitany]

the serialized model is only saved when the path is not provided so the user can use that for cashing as well as the network transfer as i know it is not full saving but it is build for this purpose in mind only.

[phoeenniixx]

what is a v1 flat format?

[patelchaitany]

It is like for the simpler format if any one used pickle to save them, then current version of load can load them successfully.

[phoeenniixx]

Nice!
I think the superficial design is similar to ptf (using a manifest, saving the artifacts in a nested structure). I have a few doubts (see above) so that I can better understand the requirements you have thought of - would be really helpful for me in ptf!

[geetu040]

Reviewing in context of sktime/sktime#10582

Is there a design discussion or document that I am missing on, would be nice to share if there is any. For now I am relying on serialization.rst in docs/ to understand the pattern.

Nice implementation. I think the recursive handling of composite is good enough. We might need to consider the overall design and format for saving/loading.

Current sktime layout

I have documented the current design in the PR description for sktime/sktime#10453, but I'll add some for context here.

For an ordinary estimator, sktime keeps _metadata and _obj at the root:

estimator.zip
├── _metadata
└── _obj

When the estimator owns native artifacts, those artifacts are stored alongside the regular object state:

estimator.zip
├── _metadata
├── _obj
└── _artifacts/
    ├── index.json
    └── model_/
        ├── config.json
        └── model.safetensors

Another native backend might produce:

estimator.zip
├── _metadata
├── _obj
└── _artifacts/
    ├── index.json
    └── network_/
        └── state_dict.pt

The important property is that a serialized estimator is a self-contained node:

node
├── _metadata
├── _obj
└── _artifacts/       # optional

_obj contains the regular estimator state, while _artifacts contains attributes that must be saved using framework-native serialization.

Current skpro layout

The recursive layout in this PR is different:

estimator.zip
├── manifest.json
├── _format
├── _version
├── root/
│   ├── _metadata
│   └── _obj
└── components/
    ├── estimators__0__1/
    │   ├── _metadata
    │   └── _obj
    └── estimators__1__1/
        ├── _metadata
        └── _obj

I do not think the additional root/ directory is needed. The archive root already represents the root estimator, so _metadata and _obj can remain there. This also preserves the existing sktime layout and makes a non-composite archive a natural subset of a composite archive.

Proposed recursive layout

I suggest treating every estimator as the same self-contained serialization node and applying that node structure recursively:

estimator.zip
├── _metadata
├── _obj
├── _artifacts/                   # optional native artifacts of root
│   ├── index.json
│   └── model_/
│       └── model.safetensors
└── _components/                  # optional child estimators
    ├── index.json
    ├── forecaster_/
    │   ├── _metadata
    │   ├── _obj
    │   ├── _artifacts/           # optional native artifacts of child
    │   │   ├── index.json
    │   │   └── network_/
    │   │       └── state_dict.pt
    │   └── _components/          # optional grandchildren
    │       ├── index.json
    │       └── transformer_/
    │           ├── _metadata
    │           └── _obj
    └── transformer_/
        ├── _metadata
        └── _obj

_components/index.json could describe how child nodes are connected to the parent:

{
  "forecaster_": {
    "path": "forecaster_",
    "attribute": "forecaster_",
    "class": "sktime.forecasting.some_module.SomeForecaster"
  },
  "transformer_": {
    "path": "transformer_",
    "attribute": "transformer_",
    "class": "sktime.transformations.some_module.SomeTransformer"
  }
}

The parent _obj would contain placeholders or references for extracted child estimators. Loading would then:

1. Unpickle the parent _obj.
2. Restore the parent's native artifacts.
3. Read _components/index.json.
4. Recursively load each child using the same node contract.
5. Reattach each child at the recorded location.

This makes the format recursive without introducing a different serialization format for the root.

[geetu040]

I think skpro should preserve the established two-element API:

That is:

(metadata, serialized_bytes)

And all estimator-related info can go in metadata:

metadata = {
    "class": type(self),
    "serialization_format": "pickle",
    "format_version": 2,
}

[geetu040]

There is also an important difference in reconstruction.

The current skpro recursive loader reconstructs composite nodes approximately as:

obj = cls.__new__(cls)
obj.__dict__.update(state_dict)

By contrast, sktime first unpickles _obj normally and then restores native artifacts.

I am not against this design, I just though this should be mentioned and if preffered, we could have the same thing for sktime.