Contacts whose name starts with an accented letter sort after Z in the contact list

[theabhishekchandra]

Guidelines

• I have searched open and closed issues for duplicates
• I am submitting a bug report for existing functionality that does not work as intended
• This isn't a feature request or a discussion topic

Bug description

Contacts whose display name begins with an accented letter (Österreich, Ángel, Émile, Ünal, Øyvind) sort below every unaccented name in the contact list — after Z. Names where the accent is not the first character (e.g. Desirée) are unaffected, which I think is why this gets reported inconsistently.

This is the same class of bug as #14487, which was fixed for group member lists in 3f990327d8, and for the message details list in bef261e9dd. The contact list itself still has it.

Root cause

RecipientTable orders contact lists by SORT_NAME with no COLLATE clause, so SQLite falls back to BINARY (code point) ordering, and every accented initial is above z in code point order:

// RecipientTable.kt
return getSignalContacts(includeSelfMode, "$SORT_NAME, $SYSTEM_JOINED_NAME, $SEARCH_PROFILE_NAME, $USERNAME, $E164")

This looks like a regression from 582028f2c2 ("Search contacts via the RecipientDatabase.", 2019-09-24), which moved contact search off ContactsDatabase and in doing so replaced two DISPLAY_NAME + " COLLATE LOCALIZED ASC" sorts with a bare orderBy = SORT_NAME + ", " + ... string. That string is still in place today, and COLLATE LOCALIZED has not appeared anywhere in the tree since.

Affected queries, all in RecipientTable: getSignalContacts, querySignalContacts, querySignalContactLetterHeaders, queryRecipientsForMentions, and the four that go through orderByPreferringAlphaOverNumeric() (getNonGroupContacts, getGroupMemberContacts, queryGroupMemberContacts, queryGroupMemberContactsForGroup). These feed ContactRepository → ContactSearchPagedDataSource, so the ordering reaches new chat, the forward/share pickers, story recipient selection and distribution lists.

Reproduction — instrumented test against a build of current main, inserting contacts named Albert, Zoe, Österreich, Ángel, Émile, Hermann and reading back getSignalContacts(IncludeSelfMode.Exclude):

actual:   albert, hermann, zoe, Ángel, Émile, Österreich
expected: albert, ángel, émile, hermann, österreich, zoe

One note for whoever picks this up: COLLATE LOCALIZED does not fix it

I tried the obvious one-line fix first. It has no effect, because the app runs on net.zetetic:sqlcipher-android, which registers LOCALIZED as a stub. Probing the actual signalWritableDatabase on device:

lower('Österreich') = Österreich      <- SQLite LOWER()/UPPER() are ASCII-only
upper('österreich') = öSTERREICH
(none)            -> [Albert, Zoe, anna, Ángel, Émile, Österreich]
COLLATE BINARY    -> [Albert, Zoe, anna, Ángel, Émile, Österreich]
COLLATE NOCASE    -> [Albert, anna, Zoe, Ángel, Émile, Österreich]   <- ordering does change, so COLLATE is honored
COLLATE LOCALIZED -> [Albert, Zoe, anna, Ángel, Émile, Österreich]   <- identical to BINARY: silent no-op
COLLATE UNICODE   -> ERROR: no such collation sequence: UNICODE

SQLiteDatabase.setLocale() exists on the wrapper but is never called; calling it first (locale en_US) does not change the LOCALIZED result either.

Two consequences worth noting: because LOWER() is ASCII-only, sort_name never lowercases accented names, so they keep their capital initial and sort even further out; and UPPER(SUBSTR($SORT_NAME, 0, 2)) in querySignalContactLetterHeaders cannot uppercase an accented initial, so a name stored lowercase gets a lowercase section header.

So a real fix seems to need a stored, normalized sort key — computed in Java with Normalizer/Collator, maintained on write and ordered by — similar to sort_key/phonebook_label in AOSP Contacts. That means a schema migration on recipient, which felt like your call rather than something to submit unasked.

Happy to put together a PR with tests if you can say which direction you'd accept.

Device

Vivo V2036

Android version

13

Signal version

8.25.2 (built from main)

Link to debug log

No response

[theabhishekchandra]

Reproduced in the UI. Seeded a build of main with six plain names (Albert, Bruno, Hermann, Nils, Oscar, Zoe) and four with a leading accent (Ángel, Émile, Ömer, Øyvind), then opened New message:

The list runs N → Nils, O → Oscar, Z → Zoe, and then Á → Ángel, É → Émile, Ö → Ömer, Ø → Øyvind. Every accented initial sorts after Z, each stranded in its own trailing section header.

The section headers are the second symptom mentioned above: UPPER(SUBSTR($SORT_NAME, 0, 2)) can't fold these into the A/E/O groups they belong in.

Device: Vivo V2036, Android 13.