Skip to content

fix(storage): remove VerboseModule to make zarr.storage picklable (gh-4029)#4030

Open
NIK-TIGER-BILL wants to merge 4 commits into
zarr-developers:mainfrom
NIK-TIGER-BILL:fix/remove-verbosemodule-gh4029
Open

fix(storage): remove VerboseModule to make zarr.storage picklable (gh-4029)#4030
NIK-TIGER-BILL wants to merge 4 commits into
zarr-developers:mainfrom
NIK-TIGER-BILL:fix/remove-verbosemodule-gh4029

Conversation

@NIK-TIGER-BILL

Copy link
Copy Markdown
Contributor

Description of PR

Fixes #4029.

The VerboseModule subclass prevented zarr.storage from being pickled by cloudpickle (and therefore by Dask). The only purpose of the subclass was to emit a deprecation warning when setting zarr.storage.default_compressor, which has been deprecated for a long time.

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.md
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

The VerboseModule subclass prevented  from being
serialized by cloudpickle (and therefore Dask). The only purpose of
the subclass was to emit a deprecation warning when setting
, which has been deprecated for a
long time.

Closes zarr-developers#4029

Signed-off-by: NIK-TIGER-BILL <nik.tiger.bill@github.com>
@github-actions github-actions Bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jun 3, 2026
@maxrjones

Copy link
Copy Markdown
Member

We likely want to also remove

# these keys were removed from the config as part of the 3.1.0 release.
# these deprecations should be removed in 3.1.1 or thereabouts.
deprecations = {
"array.v2_default_compressor.numeric": None,
"array.v2_default_compressor.string": None,
"array.v2_default_compressor.bytes": None,
"array.v2_default_filters.string": None,
"array.v2_default_filters.bytes": None,
"array.v3_default_filters.numeric": None,
"array.v3_default_filters.raw": None,
"array.v3_default_filters.bytes": None,
"array.v3_default_serializer.numeric": None,
"array.v3_default_serializer.string": None,
"array.v3_default_serializer.bytes": None,
"array.v3_default_compressors.string": None,
"array.v3_default_compressors.bytes": None,
"array.v3_default_compressors": None,
}
and the tests in
@pytest.mark.parametrize(
"key",
[
"array.v2_default_compressor.numeric",
"array.v2_default_compressor.string",
"array.v2_default_compressor.bytes",
"array.v2_default_filters.string",
"array.v2_default_filters.bytes",
"array.v3_default_filters.numeric",
"array.v3_default_filters.raw",
"array.v3_default_filters.bytes",
"array.v3_default_serializer.numeric",
"array.v3_default_serializer.string",
"array.v3_default_serializer.bytes",
"array.v3_default_compressors.string",
"array.v3_default_compressors.bytes",
"array.v3_default_compressors",
],
)
def test_deprecated_config(key: str) -> None:
"""
Test that a valuerror is raised when setting the default chunk encoding for a given
data type category
"""
with pytest.raises(ValueError):
with zarr.config.set({key: "foo"}):
pass
to complete the deprecation cycle. What do you think @d-v-b @chuckwondo?

@d-v-b

d-v-b commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

What do you think @d-v-b @chuckwondo?

Sounds good to me! And we should consider building up a bit more tooling around deprecations. It's not great that we flag stuff as deprecated and the fragments linger for a long time in the codebase.

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.25%. Comparing base (036ede7) to head (6ddae03).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4030      +/-   ##
==========================================
- Coverage   93.50%   93.25%   -0.25%     
==========================================
  Files          90       90              
  Lines       11981    11970      -11     
==========================================
- Hits        11203    11163      -40     
- Misses        778      807      +29     
Files with missing lines Coverage Δ
src/zarr/storage/__init__.py 100.00% <ø> (+5.00%) ⬆️

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@maxrjones

Copy link
Copy Markdown
Member

What do you think @d-v-b @chuckwondo?

Sounds good to me! And we should consider building up a bit more tooling around deprecations. It's not great that we flag stuff as deprecated and the fragments linger for a long time in the codebase.

Sounds good, thanks for confirming. I agree with the tooling directly, let's handle that separately. @NIK-TIGER-BILL are you up for removing the components mentioned in #4030 (comment) as part of this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs release notes Automatically applied to PRs which haven't added release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VerboseModule cannot be pickled

3 participants