Skip to content

[Bug] OBS filesystem loader should not require access and secret key #8957

Description

@thswlsqls

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version
master @ 526b91f (2.1-SNAPSHOT)

Compute Engine
Engine-agnostic (core FileIO); surfaces in any Flink/Spark/Hive catalog on obs://.

Minimal reproduce step

  1. Create a catalog with warehouse = 'obs://<bucket>/warehouse', fs.obs.endpoint and fs.obs.security.provider, without static access/secret keys.
  2. Any table operation fails with UnsupportedSchemeException: Could not find a file io implementation for scheme 'obs' in the classpath.

OBSLoader.requiredOptions() declares fs.obs.access.key and fs.obs.secret.key. Required options are a selection gate (FileIOLoader.requiredOptions() Javadoc): when one is absent, FileIO.get() discards the loader and falls back to HadoopFileIO, which cannot resolve OBSFileSystem — it lives in the plugin-only classloader.

What doesn't meet your expectations?
hadoop-huaweicloud authenticates without static keys, through fs.obs.security.provider (IObsCredentialsProvider, e.g. ECS agency) or a Hadoop credential provider. Those configurations should route through the bundled OBS plugin instead of being gated out. fs.obs.endpoint should stay required: DefaultOBSClientFactory reads it without a default.

Anything else?
The same defect was reported and fixed for S3 in #8674 / #8679.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions