Skip to content

fix(storage): authorize MGF1 digest to prevent INCOMPATIBLE_DEVICE on…4.x - #1049

Open
utkrishtsahu wants to merge 1 commit into
mainfrom
repro/mgf1-incompatible-device-keystore2
Open

fix(storage): authorize MGF1 digest to prevent INCOMPATIBLE_DEVICE on…4.x#1049
utkrishtsahu wants to merge 1 commit into
mainfrom
repro/mgf1-incompatible-device-keystore2

Conversation

@utkrishtsahu

@utkrishtsahu utkrishtsahu commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Changes

What & why: On newer Android Keystore2 hardware, SecureCredentialsManager can fail immediately after a successful login with CredentialsManagerException.INCOMPATIBLE_DEVICE ("This device is not compatible with the ... class"), blocking token persistence even though authentication succeeded. Reported on Pixel 10 / Galaxy S26 (Android 16+).

Root cause: CryptoUtil encrypts stored credentials with an RSA-OAEP key whose cipher uses the MGF1/SHA-1 digest, but the key was generated without explicitly authorizing an MGF1 digest. Older KeyMint defaulted the MGF1 digest to SHA-1, so it matched. Newer Keystore2 firmware strictly enforces the authorized MGF1 set and its default is vendor-dependent (some default to SHA-256), so the private-key decrypt is rejected with INCOMPATIBLE_MGF_DIGEST ("Incompatible padding mode") and surfaces as INCOMPATIBLE_DEVICE.

Changes (both in CryptoUtil.java, no public API change):

  • getRSAKeyEntry() — when generating a new RSA key, explicitly authorize the MGF1 digests (SHA-1 and SHA-256) via KeyGenParameterSpec.Builder#setMgf1Digests, guarded to Build.VERSION_CODES.VANILLA_ICE_CREAM (API 35+, where the API exists). The key no longer depends on the vendor's MGF1 default.
  • RSADecrypt()InvalidKeyException / InvalidAlgorithmParameterException (the MGF1-authorization rejection on an existing, pre-fix key) are now treated as recoverable: delete the stale RSA + AES keys and throw CryptoException so the caller regenerates a correctly-authorized key, instead of the terminal IncompatibleDeviceException. NoSuchAlgorithmException / NoSuchPaddingException remain terminal (genuine device incompatibility). This self-heals users already stuck (one re-login).

No endpoints, classes, or public methods added/removed/deprecated. CryptoUtil is internal; behavior change only.

Testing

CryptoUtil cannot exercise the real Android Keystore under unit tests (Robolectric mocks it), so the hardware behavior was validated on physical devices and the exception-handling logic via unit tests.

  • Updated the existing RSADecrypt InvalidKeyException test to assert the new recovery behavior (throws CryptoException, deletes RSA + AES keys) and added an equivalent test for InvalidAlgorithmParameterException.

  • :auth0:testReleaseUnitTest (CryptoUtilTest) and :auth0:lintRelease pass (JDK 17).

  • On-device verification (Pixel 10 / Android 17 and Galaxy S26 Ultra / Android 16): confirmed an MGF1 digest mismatch reproduces the exact INCOMPATIBLE_MGF_DIGEST failure, and that generating the key with setMgf1Digests(SHA-1, SHA-256) + the SDK's MGF1/SHA-1 cipher round-trips successfully.

  • Not reproduced end-to-end on hardware: a device whose KeyMint defaults MGF1 to SHA-256 (the failing-on-default case) — our test devices default to SHA-1, so the failure was reproduced by forcing the mismatch. The fix removes reliance on the vendor default in all cases.

  • This change adds unit test coverage

  • This change adds integration test coverage

  • This change has been tested on the latest version of the platform/language or why not

Checklist

Summary by CodeRabbit

  • Bug Fixes
    • Improved RSA encryption compatibility on newer Android versions.
    • Automatically recovers from invalid or incompatible encryption keys by securely recreating them.
    • Distinguishes recoverable key issues from devices that lack required encryption algorithms.

@utkrishtsahu
utkrishtsahu requested a review from a team as a code owner August 12, 2026 12:19
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

RSA cryptography updates

Layer / File(s) Summary
API-specific RSA key generation
auth0/src/main/java/com/auth0/android/authentication/storage/CryptoUtil.java, auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java
RSA key generation authorizes SHA-1 and SHA-256 MGF1 digests on API 35+. API 30 tests verify the previous configuration remains unchanged.
RSA decryption recovery
auth0/src/main/java/com/auth0/android/authentication/storage/CryptoUtil.java, auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java
Invalid RSA keys and OAEP parameters delete RSA and AES key data and raise CryptoException. Unavailable algorithms continue to raise IncompatibleDeviceException. Tests cover both recovery cases.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: amitsingh05667

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the primary change: authorizing the MGF1 digest to prevent an INCOMPATIBLE_DEVICE error.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch repro/mgf1-incompatible-device-keystore2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java (1)

746-779: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert the CryptoException message explicitly.

Assert.assertThrows(String, Class, ThrowingRunnable) uses the first argument as the assertion failure message. Capture the returned CryptoException and assert getMessage() in both tests.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java`
around lines 746 - 779, Update both RSADecrypt tests,
shouldRecreateKeysAndThrowCryptoExceptionOnInvalidKeyExceptionWhenTryingToRSADecrypt
and
shouldRecreateKeysAndThrowCryptoExceptionOnInvalidAlgorithmParameterExceptionWhenTryingToRSADecrypt,
to capture the CryptoException returned by Assert.assertThrows instead of
passing the expected text as its assertion message. Explicitly assert that the
captured exception’s getMessage() equals the documented message in each test.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java`:
- Around line 251-285: Rename
shouldNotAuthorizeMgf1DigestWhenCreatingRSAKeyPairBelowAPI33 to reflect the API
35 boundary, and retain it as the pre-API-35 coverage. Add a separate
`@Config`(sdk = 35) test around cryptoUtil.getRSAKeyEntry that captures the
KeyGenParameterSpec and asserts getMgf1Digests contains SHA-1 and SHA-256.
Ensure the new coverage exercises both successful authorization and the existing
failure/error path for MGF1 configuration.

---

Outside diff comments:
In
`@auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java`:
- Around line 746-779: Update both RSADecrypt tests,
shouldRecreateKeysAndThrowCryptoExceptionOnInvalidKeyExceptionWhenTryingToRSADecrypt
and
shouldRecreateKeysAndThrowCryptoExceptionOnInvalidAlgorithmParameterExceptionWhenTryingToRSADecrypt,
to capture the CryptoException returned by Assert.assertThrows instead of
passing the expected text as its assertion message. Explicitly assert that the
captured exception’s getMessage() equals the documented message in each test.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4823915d-18cf-429e-92c9-96c872497bfe

📥 Commits

Reviewing files that changed from the base of the PR and between 45a6c06 and edf70a1.

📒 Files selected for processing (2)
  • auth0/src/main/java/com/auth0/android/authentication/storage/CryptoUtil.java
  • auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java

Comment on lines +251 to +285
@Test
@Config(sdk = 30)
public void shouldNotAuthorizeMgf1DigestWhenCreatingRSAKeyPairBelowAPI33() throws Exception {
// Below API 33 setMgf1Digests does not exist; the key must be generated without it (the
// pre-existing behavior), relying on the SHA-1 default that older Keystore uses.
Mockito.when(keyStore.containsAlias(KEY_ALIAS)).thenReturn(false);
KeyStore.PrivateKeyEntry expectedEntry = Mockito.mock(KeyStore.PrivateKeyEntry.class);
Mockito.when(keyStore.getEntry(KEY_ALIAS, null)).thenReturn(expectedEntry);

ArgumentCaptor<AlgorithmParameterSpec> specCaptor = ArgumentCaptor.forClass(AlgorithmParameterSpec.class);

cryptoUtil.getRSAKeyEntry();

Mockito.verify(keyPairGenerator).initialize(specCaptor.capture());
Mockito.verify(keyPairGenerator).generateKeyPair();
KeyGenParameterSpec spec = (KeyGenParameterSpec) specCaptor.getValue();

// The rest of the spec is unchanged from the pre-fix behavior.
assertThat(spec.getEncryptionPaddings(), is(new String[]{KeyProperties.ENCRYPTION_PADDING_RSA_OAEP}));
assertThat(spec.getDigests(), is(new String[]{KeyProperties.DIGEST_SHA1, KeyProperties.DIGEST_SHA256}));

String[] mgf1Digests = readMgf1Digests(spec);
// No MGF1 digest was authorized (null/empty), matching the pre-fix key spec.
assertThat(mgf1Digests == null || mgf1Digests.length == 0, is(true));
}


private static String[] readMgf1Digests(KeyGenParameterSpec spec) {
try {
java.lang.reflect.Method m = spec.getClass().getMethod("getMgf1Digests");
Object result = m.invoke(spec);
return (String[]) result;
} catch (Throwable t) {
return null;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add API 35 coverage for MGF1 authorization.

This test only verifies the API 30 path. It does not execute setMgf1Digests. Add an @Config(sdk = 35) test that asserts SHA-1 and SHA-256 are authorized. Rename this test because the production boundary is API 35, not API 33.

As per coding guidelines, test both success and failure paths for behavior changes.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@auth0/src/test/java/com/auth0/android/authentication/storage/CryptoUtilTest.java`
around lines 251 - 285, Rename
shouldNotAuthorizeMgf1DigestWhenCreatingRSAKeyPairBelowAPI33 to reflect the API
35 boundary, and retain it as the pre-API-35 coverage. Add a separate
`@Config`(sdk = 35) test around cryptoUtil.getRSAKeyEntry that captures the
KeyGenParameterSpec and asserts getMgf1Digests contains SHA-1 and SHA-256.
Ensure the new coverage exercises both successful authorization and the existing
failure/error path for MGF1 configuration.

Source: Coding guidelines

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant