Race conditions in opening multiple files simaltanously.

[capfredf]

I have been experiecing this issue for quite a while. Now I finally had time to look into it.

I have .rhm associated racket-hash-lang-mode and racket-xp-mode. I have some sort of session restoration enabled so that when I start Emacs I can pick up where I left off.

But racket-hash-lang-mode always reports instantiate-linket errors when ".rhm" buffers are restored.
I need to restart the racket-backend server and reopen those files.

Since Emacs just repoens those files during session restoration, this issue apparently is due to some limitatations of how racket-mode handles opening multiple files rapidly.

Here is a recipe to reproduce:

1. create a few .rhm files.

 #lang rhombus

def value = 1 

and name them 01.rhm, 02.rhm and etc.

2. create the elisp program open-all.el in the same folder:

(require 'package)
(package-initialize)
(require 'racket-hash-lang)
(require 'racket-xp)

(add-to-list 'auto-mode-alist '("\\.rhm\\'" . racket-hash-lang-mode))
(add-hook 'racket-hash-lang-mode-hook #'racket-xp-mode)

(let* ((directory (file-name-directory (or load-file-name buffer-file-name)))
       (files (directory-files directory t "\\`[0-9][0-9]\\.rhm\\'")))
  (dolist (file files)
    (message "Opening %s" (file-name-nondirectory file))
    (find-file file))
  (message "Opened %d .rhm files without waiting" (length files)))

;; Batch Emacs would otherwise exit before the asynchronous backend replies.
(when noninteractive
  (let ((deadline (+ (float-time) 30)))
    (while (< (float-time) deadline)
      (accept-process-output nil 0.1)))
  (dolist (buffer (buffer-list))
    (with-current-buffer buffer
      (when (string-suffix-p ".rhm" (or buffer-file-name ""))
        (message "%s: mode=%S id=%S read-only=%S"
                 (buffer-name) major-mode
                 (bound-and-true-p racket--hash-lang-id)
                 buffer-read-only)))))

3. run the above elisp program: emacs -Q --batch -l open-all.el

Please copy all of the following lines and paste them into your bug report
at https://github.com/greghendershott/racket-mode/issues/.

Details

Package

metadata

(#s(package-desc racket-mode (20260726 2002) "Racket editing, REPL, and more."
                                           ((emacs (25 1)) (compat (30 0 2 0))) nil nil
                                           "/home/capfredf/.emacs.d/elpa/racket-mode-20260726.2002"
                                           ((:authors
                                             ("Greg Hendershott"
                                              . "racket-mode-author@greghendershott.com"))
                                            (:revdesc . "f92a33dcc3b6")
                                            (:commit . "f92a33dcc3b604f53ef23a538e26e1f25c4fea47")
                                            (:url . "https://www.racket-mode.com/"))
                                           nil))

package-archives

(("gnu" . "https://elpa.gnu.org/packages/")
                                   ("nongnu" . "https://elpa.nongnu.org/nongnu/")
                                   ("melpa" . "https://melpa.org/packages/"))

racket--el-source-dir

"/home/capfredf/.emacs.d/elpa/racket-mode-20260726.2002/"

racket--rkt-source-dir

"/home/capfredf/.emacs.d/elpa/racket-mode-20260726.2002/racket/"

System values

emacs-version

"31.1"

major-mode

racket-hash-lang-mode

system-type

gnu/linux

x-gtk-use-system-tooltips

t

display-graphic-p

t

Buffer values

after-change-functions

(jit-lock-after-change t ws-butler-after-change racket--xp-after-change-hook
                                                               racket--hash-lang-after-change)

before-change-functions

(t syntax-ppss-flush-cache)

completion-at-point-functions

(tempel-expand racket-xp-complete-at-point)

eldoc-documentation-function

eldoc-documentation-default

eldoc-documentation-strategy

eldoc-documentation-default

eldoc-documentation-functions

(racket-xp-eldoc-point-or-sexp-head t)

font-lock-defaults

(nil t)

pre-command-hook

(eldoc-pre-command-refresh-echo-area t)

post-command-hook

(jinx--reschedule eldoc-schedule-timer yas--post-command-handler t)

post-self-insert-hook

(electric-pair-post-self-insert-function
                                        electric-pair-open-newline-between-pairs-psif
                                        electric-indent-post-self-insert-function blink-paren-post-self-insert-function)

xref-backend-functions

(racket-xp-xref-backend-function t)

Racket Mode values

racket--cmd-open-p

t

racket-after-run-hook

nil

racket-back-end-configurations

((:directory "/" :racket-program nil :remote-source-dir nil
                                                             :restart-watch-directories nil :windows nil))

racket-before-run-hook

(racket-ansi-color-context-reset)

racket-browse-url-function

browse-url-default-browser

racket-command-timeout

10

racket-debug-break-expression-string

"●"

racket-debuggable-files

racket-same-directory-files

racket-doc-index-directory

"~/.emacs.d/racket-mode/"

racket-doc-index-predicate-function

always

racket-documentation-search-location

"https://docs.racket-lang.org/search/index.html?q=%s"

racket-error-context

medium

racket-expand-hiding

standard

racket-hash-lang-mode-hook

(racket-xp-mode my/racket-hash-lang-disable-jinx
                                                            my/racket-hash-lang-fontify-directive)

racket-hash-lang-module-language-hook

(my/racket-hash-lang-module-lang-hook)

racket-hash-lang-token-face-alist

((builtin . font-lock-builtin-face) (constant . font-lock-constant-face)
                                                    (error . error) (operator . font-lock-operator-face)
                                                    (other . font-lock-doc-face) (keyword . font-lock-keyword-face)
                                                    (hash-colon-keyword . racket-keyword-argument-face) (at . font-lock-doc-face))

racket-history-filter-regexp

"\\`\\s *\\'"

racket-imagemagick-props

nil

racket-images-do-not-use-svg

nil

racket-images-inline

t

racket-images-keep-last

100

racket-images-system-viewer

"display"

racket-indent-curly-as-sequence

t

racket-indent-sequence-depth

0

racket-input-prefix

"\\"

racket-input-translations

(("All" "∀") ("Union" "U") ("Intersection" "∩") ("test-->>E" "test-->>∃")
                                            ("vdash" "⊢") ("gg" "≫") ("succ" "≻") ("times" "×") ("Uparrow" "⇑")
                                            ("Downarrow" "⇓") ("Leftarrow" "⇐") ("Rightarrow" "⇒") ("leftarrow" "←")
                                            ("rightarrow" "→") ("nwarrow" "↖") ("nearrow" "↗") ("uparrow" "↑")
                                            ("downarrow" "↓") ("searrow" "↘") ("swarrow" "↙") ("leftrightarrow" "↔")
                                            ("updownarrow" "⇕") ("aleph" "א") ("emptyset" "∅") ("nabla" "∇") ("surd" "√")
                                            ("negation" "¬") ("infinity" "∞") ("prod" "∏") ("coprod" "∐") ("integrate" "∫")
                                            ("oint" "∮") ("vee" "∨") ("wedge" "∧") ("follows" "∘") ("setin" "∈") ("ni" "∋")
                                            ("notin" "∉") ("sqsubset" "⊏") ("sqsupset" "⊐") ("sqsubseteq" "⊑")
                                            ("sqsupseteq" "⊒") ("subset" "⊂") ("superset" "⊃") ("subseteq" "⊆")
                                            ("supseteq" "⊇") ("approx" "≈") ("cong" "≌") ("neq" "≠") ("sqcap" "⊓")
                                            ("sqcup" "⊔") ("models" "⊧") ("alpha" "α") ("Alpha" "Α") ("beta" "β")
                                            ("Beta" "Β") ("gamma" "γ") ("Gamma" "Γ") ("delta" "δ") ("Delta" "Δ")
                                            ("epsilon" "ε") ("Epsilon" "Ε") ("zeta" "ζ") ("Zeta" "Ζ") ("eta" "η")
                                            ("Eta" "Η") ("theta" "θ") ("Theta" "Θ") ("iota" "ι") ("Iota" "Ι") ("kappa" "κ")
                                            ("Kappa" "Κ") ("lambda" "λ") ("Lambda" "Λ") ("lamda" "λ") ("Lamda" "Λ")
                                            ("mu" "μ") ("Mu" "Μ") ("nu" "ν") ("Nu" "Ν") ("xi" "ξ") ("Xi" "Ξ")
                                            ("omicron" "ο") ("Omicron" "Ο") ("pi" "π") ("Pi" "Π") ("rho" "ρ") ("Rho" "Ρ")
                                            ("sigma" "σ") ("Sigma" "Σ") ("tau" "τ") ("Tau" "Τ") ("upsilon" "υ")
                                            ("Upsilon" "Υ") ("phi" "φ") ("Phi" "Φ") ("chi" "χ") ("Chi" "Χ") ("psi" "ψ")
                                            ("Psi" "Ψ") ("omega" "ω") ("Omega" "Ω") ("digamma" "ϝ") ("Digamma" "Ϝ")
                                            ("san" "ϻ") ("San" "Ϻ") ("qoppa" "ϙ") ("Qoppa" "Ϙ") ("sampi" "ϡ") ("Sampi" "Ϡ")
                                            ("stigma" "ϛ") ("Stigma" "Ϛ") ("heta" "ͱ") ("Heta" "Ͱ") ("sho" "ϸ") ("Sho" "Ϸ")
                                            ("|A|" "𝔸") ("|B|" "𝔹") ("|C|" "ℂ") ("|D|" "𝔻") ("|E|" "𝔼") ("|F|" "𝔽")
                                            ("|G|" "𝔾") ("|H|" "ℍ") ("|I|" "𝕀") ("|J|" "𝕁") ("|K|" "𝕂") ("|L|" "𝕃")
                                            ("|M|" "𝕄") ("|N|" "ℕ") ("|O|" "𝕆") ("|P|" "ℙ") ("|Q|" "ℚ") ("|R|" "ℝ")
                                            ("|S|" "𝕊") ("|T|" "𝕋") ("|U|" "𝕌") ("|V|" "𝕍") ("|W|" "𝕎") ("|X|" "𝕏")
                                            ("|Y|" "𝕐") ("|Z|" "ℤ") ("|gamma|" "ℽ") ("|Gamma|" "ℾ") ("|pi|" "ℼ")
                                            ("|Pi|" "ℿ") ("forall" "∀") ("exists" "∃") ("_0" "₀") ("_1" "₁") ("_2" "₂")
                                            ("_3" "₃") ("_4" "₄") ("_5" "₅") ("_6" "₆") ("_7" "₇") ("_8" "₈") ("_9" "₉")
                                            ("^0" "⁰") ("^1" "¹") ("^2" "²") ("^3" "³") ("^4" "⁴") ("^5" "⁵") ("^6" "⁶")
                                            ("^7" "⁷") ("^8" "⁸") ("^9" "⁹") ("gl" "«") ("gr" "»"))

racket-logger-config

((cm-accomplice . warning) (GC . info) (module-prefetch . warning)
                                       (optimizer . info) (racket/contract . error) (racket-mode-debugger . info)
                                       (sequence-specialization . info) (* . fatal))

racket-memory-limit

2048

racket-mode-hook

(racket-xp-mode enable-paredit-mode #[nil ((highlight-parentheses-mode t)) (t)])

racket-module-forms

"\\s(\\(?:module[*+]?\\|library\\)"

racket-pretty-lambda

nil

racket-pretty-print

t

racket-program

"racket"

=>

"/home/capfredf/.racket/bin/racket"

racket-repl-buffer-name-function

racket-repl-buffer-name-project

racket-repl-command-file

"/home/capfredf/.emacs.d/racket-mode/repl.rkt"

racket-repl-echo-sent-expressions

t

racket-repl-history-directory

"~/.emacs.d/racket-mode/"

racket-repl-mode-hook

nil

racket-sexp-comment-fade

0.5

racket-shell-or-terminal-function

racket-shell

racket-show-functions

(racket-show-pseudo-tooltip)

racket-smart-open-bracket-enable

nil

racket-submodules-to-run

((test) (main))

racket-use-repl-submit-predicate

nil

racket-xp-add-binding-faces

t

racket-xp-after-change-refresh-delay

1

racket-xp-eldoc-level

summary

racket-xp-highlight-unused-regexp

"^[^_]"

racket-xp-mode-lighter

(:eval (racket--xp-mode-lighter))

Minor modes

enabled

((auto-composition-mode) (auto-compression-mode) (auto-encryption-mode)
                          (blink-cursor-mode) (codex-ide-track-active-buffer-mode) (column-number-mode)
                          (delete-selection-mode) (denote-menu-bar-mode) (eldoc-mode)
                          (electric-pair-mode) (file-name-shadow-mode) (font-lock-mode)
                          (ghostel-compile-global-mode) (global-corfu-mode) (global-eldoc-mode)
                          (global-font-lock-mode) (global-git-commit-mode) (global-jinx-mode) (jinx-mode)
                          (line-number-mode) (magit-auto-revert-mode) (marginalia-mode) (meow-esc-mode)
                          (meow-global-mode) (meow-mode) (meow-normal-mode) (minibuffer-nonselected-mode)
                          (minibuffer-regexp-mode) (mouse-wheel-mode) (override-global-mode)
                          (racket-xp-mode) (repeat-mode) (savehist-mode) (semantic-minor-modes-format)
                          (server-mode) (show-paren-mode) (tab-bar-mode) (tabspaces-mode) (tooltip-mode)
                          (transient-mark-mode) (vertico-mode) (windmove-mode) (winner-mode)
                          (ws-butler-mode) (yas-minor-mode))

Disabled minor modes

disabled

((abbrev-mode) (auto-fill-function) (auto-fill-mode) (auto-revert-mode)
                           (auto-revert-tail-mode) (auto-save-mode) (auto-save-visited-mode)
                           (buffer-face-mode) (buffer-read-only) (bug-reference-mode)
                           (bug-reference-prog-mode) (button-mode) (center-line-mode)
                           (cl-old-struct-compat-mode) (codex-ide-session-approval-minor-mode)
                           (codex-ide-session-mention-minor-mode) (codex-ide-session-prompt-minor-mode)
                           (codex-ide-session-slash-command-minor-mode) (comint-fontify-input-mode)
                           (compilation-minor-mode) (compilation-shell-minor-mode)
                           (completion-in-region-mode) (cond-let-fontify-mode) (context-menu-mode)
                           (corfu-mode) (cursor-face-highlight-mode) (cursor-intangible-mode)
                           (cursor-sensor-mode) (dash-fontify-mode) (defining-kbd-macro)
                           (delete-trailing-whitespace-mode) (denote-dired-mode)
                           (denote-fontify-links-mode) (denote-rename-buffer-mode) (desktop-save-mode)
                           (diff-auto-refine-mode) (diff-hl-dir-mode) (diff-hl-mode) (diff-minor-mode)
                           (dired-click-to-select-mode) (dired-hide-details-mode)
                           (dired-isearch-filenames-mode) (dired-omit-mode) (electric-indent-mode)
                           (electric-layout-mode) (electric-quote-mode) (find-function-mode)
                           (flymake-mode) (ghostel-compile-toggle-mode) (git-commit-mode)
                           (global-auto-revert-mode) (global-dash-fontify-mode) (global-diff-hl-mode)
                           (global-goto-address-mode) (global-highlight-parentheses-mode)
                           (global-hl-line-mode) (global-prettify-symbols-mode)
                           (global-semantic-highlight-edits-mode) (global-semantic-highlight-func-mode)
                           (global-semantic-show-parser-state-mode)
                           (global-semantic-show-unmatched-syntax-mode) (global-semantic-stickyfunc-mode)
                           (global-visual-line-mode) (global-xref-mouse-mode) (goto-address-mode)
                           (goto-address-prog-mode) (highlight-parentheses-mode) (hl-line-mode)
                           (horizontal-scroll-bar-mode) (hs-indentation-mode) (hs-minor-mode)
                           (indent-tabs-mode) (isearch-fold-quotes-mode) (isearch-mode)
                           (ispell-minor-mode) (jit-lock-debug-mode) (llama-fontify-mode) (lock-file-mode)
                           (lost-selection-mode) (magit-blame-mode) (magit-blame-read-only-mode)
                           (magit-blob-mode) (magit-wip-mode) (markdown-live-preview-mode) (menu-bar-mode)
                           (meow-beacon-mode) (meow-insert-mode) (meow-keypad-mode) (meow-motion-mode)
                           (meow-paren-mode) (mode-line-invisible-mode) (modifier-bar-mode)
                           (mouse-shift-adjust-mode) (next-error-follow-minor-mode) (outline-minor-mode)
                           (overwrite-mode) (package-autosuggest-mode) (paragraph-indent-minor-mode)
                           (paredit-mode) (prettify-special-glyphs-mode) (prettify-symbols-mode)
                           (racket-debug-mode) (racket-hash-lang-repl-mode) (racket-input-mode)
                           (racket-smart-open-bracket-mode) (read-extended-command-mode)
                           (read-passwd-mode) (semantic-highlight-edits-mode)
                           (semantic-highlight-func-mode) (semantic-mode)
                           (semantic-show-parser-state-mode) (semantic-show-unmatched-syntax-mode)
                           (semantic-stickyfunc-mode) (sh-electric-here-document-mode)
                           (shell-command-with-editor-mode) (shell-dirtrack-mode)
                           (shell-highlight-undef-mode) (size-indication-mode) (smerge-mode)
                           (tab-bar-history-mode) (temp-buffer-resize-mode) (text-scale-mode)
                           (tool-bar-mode) (transient-resume-mode) (treesit-explore-mode)
                           (treesit-inspect-mode) (undelete-frame-mode) (url-handler-mode)
                           (use-hard-newlines) (vc-auto-revert-mode) (vc-dir-git-mode) (vc-parent-buffer)
                           (view-mode) (visible-mode) (visual-line-mode) (which-function-mode)
                           (window-divider-mode) (with-editor-mode) (ws-butler-global-mode)
                           (xref-etags-mode) (xref-mouse-mode) (yas-global-mode))

Steps to reproduce:

[greghendershott]

Although I haven't been situated to work on this, for a few days, I can start tomorrow.

It is a little unusual (I think) for an Emacs major mode to require an outside-Emacs process to be fully initialized, before the mode can do even basic things (like syntax highlighting, indent, and structural navigation).

I didn't find any proven examples how to handle, both internally and UX, that intermediate semi-initialized state.

I think the history here is that I started with something super safe, like initializing in read-only prog-mode, but @samth found it understandably too awkward... so I tried to make it more flexible, but maybe that opened the door to the problem you're reporting. Or maybe this problem has been here all along. I'll dig in....

[capfredf]

It is a little unusual (I think) for an Emacs major mode to require an outside-Emacs process to be fully initialized, before the mode can do even basic things (like syntax highlighting, indent, and structural navigation).

I think it is less unusual now (maybe just a bit). LSP now supports semantic highlighting and other stuff. To use them in a supported LSP client, a LSP server needs to be fired up first. For other languages, they also have "static" major modes. For hash langs, however, the information is fully provided by an external server.

[capfredf]

FWIW, my current workaround is that before restoring .rhm buffers, remove hash-lang-mode from file assoc list for ".rhm". By doing so, the buffers restored are in the fundamental mode. Then I just need to reopen those buffers manually again.

[greghendershott]

I think it is less unusual now (maybe just a bit). LSP now supports semantic highlighting and other stuff. To use them in a supported LSP client, a LSP server needs to be fired up first. For other languages, they also have "static" major modes. For hash langs, however, the information is fully provided by an external server.

Good point. In addition to minor modes like eglot or lsp-mode that work with external LSP server processes, there are also the newer treesitter major mode flavors (a foo-ts-mode alternative to "classic" foo-mode).

I think these are implemented with treesitter running as an Emacs in-process dynamic module as opposed to an external process. But either way, there might be enough startup delay that the ts major modes need to handle. I could see how they handle it for examples/ideas.

[greghendershott]

I started on this yesterday. I'm able to reproduce using your example files -- thanks!

So far I'm pretty confused what's happening -- especially with the symptom being "instantiate-linklet: mismatch" messages. I don't understand why those are happening, at all. Much less happening only under the multi file open scenario -- which seems orthogonal, i.e., is the Racket module registry really race- or concurrency-vulnerable, and if so, how/why?

I'm looking at bit at how DrRacket is doing some "insulated read-language" machinations. Although I thought the motivation was to make DrRacket resilient in the face of buggy lang-info implementations, maybe the aspect where it's parameterizing current-namespace is relevant here??

I will keep marinading myself in the problem and trying a few educational experiments, for now ...

[capfredf]

Yeah, I feel the same. Opening multi files rapidly is just a way to expose the issue in the Racket module registry.

[greghendershott]

Trying to follow the example of what DrR does, in part, I experimented with parameterizing a special namespace with certain modules attached, for the dynamic extent of the call to read-language, as well as calls to procedure values obtained from that "get-info" procedure.

"It worked", but I didn't understand why.

Next I tried something simpler -- guarding the use of read-language with a sempahore. See e.g. commit 07365a5.

"It works". I still don't understand why. I'm also not sure if the concurrency was exposing a bug in my code, really, or in the rhombus color-lexer implementation, or what exactly.