diff --git a/.github/agents/dependency-update.md b/.github/agents/dependency-update.md index 30c2e2edc35..a3ee56c1117 100644 --- a/.github/agents/dependency-update.md +++ b/.github/agents/dependency-update.md @@ -90,9 +90,9 @@ higher-numbered lines exist there. **TypeScript is held at 5.9.3 in the libraries.** 7.x is the Go rewrite (`microsoft/typescript-go`), not an incremental release. `Boilerplate.Client.Core` is the canary and is already on 7.0.2 — if a bump is wanted, move one library project, build it, and only then the -rest. Do not move all sixteen at once. +rest. Do not move all twenty at once. -**Vitest is held at `^3`** in `src/Bmotion/Tests/bit-bmotion-js` and `src/Bswup/Tests/bit-bswup-js`. +**Vitest is held at `^3`** in `src/Bmotion/Tests/bit-bmotion-js`, `src/Bswup/Tests/bit-bswup-js` and `src/Bsqlite/Tests/bit-bsqlite-js`. **`devops-actions/variable-substitution` is held at v1.2** (`ae7b1f36…`). v2.0.0 committed 31 symlinks into its vendored `node_modules`; the runner cannot stage them and every job dies before diff --git a/.github/agents/test-platform.md b/.github/agents/test-platform.md index bbd6d7a2799..a545aef4e38 100644 --- a/.github/agents/test-platform.md +++ b/.github/agents/test-platform.md @@ -369,7 +369,7 @@ does not serve silently gets the newest one it does. `feat(release): v-X.Y.Z` commit on `develop` has its tag: an untagged release is answered by the previous one, without a word. 2. **What it does serve.** `mcp -McpVersion X.Y.Z` for each. The library tools (`Search*`, `Get*` and `Inspect*` of BlazorUI, - Brouter, Butil, Bswup, Bmotion and - from the first release that ships them - Bink, Balette, BlazeDb, Bonboarding, Blex and Bmage) and the source tools (`FindBitPlatformSymbols`, `GetBitPlatformSymbolSource`, + Brouter, Butil, Bswup, Bmotion and - from the first release that ships them - Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage and Bsqlite) and the source tools (`FindBitPlatformSymbols`, `GetBitPlatformSymbolSource`, `SearchBitPlatformCode`) must all be there. Only the five shared ones (`AskGitHubRepository`, `SendBitPlatformFeedback`, `microsoft_docs_*`) means no release is served at all, and the README's *PlatformWebsite's MCP* section says why that happens. The server version it reports is the website's own build, @@ -388,7 +388,7 @@ does not serve silently gets the newest one it does. 4. **The answers are good.** Put one real task to each library's `Search` tool, worded the way a developer would describe it - BlazorUI "a dropdown the user can also type into", Butil "copy text to the clipboard", Bmotion "fade a list item in", Brouter "a route parameter that must be a number", Bswup "show the download progress of the app's - update", Bink "a signature pad that exports a PNG", Balette "a command that asks for a name before it runs", BlazeDb "a todo list that survives a page reload", Bonboarding "a step that waits for a panel that loads later", Blex "a counter whose value survives a page reload", Bmage "shrink an uploaded photo under 200 KB without its location" - follow up with the `Get` tool it points to, and ask `SearchBitPlatformCode` about the Boilerplate. An + update", Bink "a signature pad that exports a PNG", Balette "a command that asks for a name before it runs", BlazeDb "a todo list that survives a page reload", Bonboarding "a step that waits for a panel that loads later", Blex "a counter whose value survives a page reload", Bmage "shrink an uploaded photo under 200 KB without its location", Bsqlite "keep the database when OPFS is not available" - follow up with the `Get` tool it points to, and ask `SearchBitPlatformCode` about the Boilerplate. An answer must be on topic, specific and true to the version asked for; one that answers wrongly is a failure too. Like an E2E failure, an MCP problem is fixed in `src/Websites/Platform` or in the library's own MCP server, and goes diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 771ed09d32b..b197d2cd3c4 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -46,8 +46,8 @@ ## Rules **RULE 1:** If a task (question, code modification or review) involves the use of bit BlazorUI components (e.g., `BitButton`, `BitTooltip`, `BitTextField`), -or refers to `bitplatform`, `bit Bswup`, `bit Butil`, `bit Bmotion`, `bit Bink`, `bit Balette`, `bit BlazeDb`, `bit Bonboarding`, `bit Blex`, `bit Bmage`, `bit Brouter`, or `bit Boilerplate`, or involves UI components without explicitly specifying their UI toolkit, -you **MUST** use the `bitplatform` MCP server's tools - start with the matching `Search` tool (e.g. `SearchBitBlazorUI`, `SearchBswup`, `SearchButil`, `SearchBmotion`, `SearchBink`, `SearchBalette`, `SearchBlazeDb`, `SearchBonboarding`, `SearchBlex`, `SearchBmage`, `SearchBrouter`) and follow the calls it names - to find the correct implementation and usage patterns before writing or changing any code. +or refers to `bitplatform`, `bit Bswup`, `bit Butil`, `bit Bmotion`, `bit Bink`, `bit Balette`, `bit BlazeDb`, `bit Bonboarding`, `bit Blex`, `bit Bmage`, `bit Bsqlite`, `bit Brouter`, or `bit Boilerplate`, or involves UI components without explicitly specifying their UI toolkit, +you **MUST** use the `bitplatform` MCP server's tools - start with the matching `Search` tool (e.g. `SearchBitBlazorUI`, `SearchBswup`, `SearchButil`, `SearchBmotion`, `SearchBink`, `SearchBalette`, `SearchBlazeDb`, `SearchBonboarding`, `SearchBlex`, `SearchBmage`, `SearchBsqlite`, `SearchBrouter`) and follow the calls it names - to find the correct implementation and usage patterns before writing or changing any code. **End of RULE 1** diff --git a/.github/workflows/all.cd.yml b/.github/workflows/all.cd.yml index d9dbe904351..62d28128a9e 100644 --- a/.github/workflows/all.cd.yml +++ b/.github/workflows/all.cd.yml @@ -71,6 +71,12 @@ jobs: uses: ./.github/workflows/bmage.demo.cd.yml secrets: inherit + bsqlite_demo: + name: bsqlite demo + if: inputs.server + uses: ./.github/workflows/bsqlite.demo.cd.yml + secrets: inherit + bink_demo: name: bink demo if: inputs.server diff --git a/.github/workflows/all.ci.yml b/.github/workflows/all.ci.yml index 044fb781461..95c271b68f9 100644 --- a/.github/workflows/all.ci.yml +++ b/.github/workflows/all.ci.yml @@ -55,6 +55,10 @@ on: description: Bmage solution type: boolean default: true + bsqlite: + description: Bsqlite solution + type: boolean + default: true bink: description: Bink solution type: boolean @@ -606,6 +610,31 @@ jobs: dotnet build src/Bmage/Bit.Bmage.Editor/Bit.Bmage.Editor.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false dotnet pack src/Bmage/Bit.Bmage.Editor/Bit.Bmage.Editor.csproj --output ./packages --configuration Release + - name: Build and pack Bsqlite + run: | + dotnet build src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + dotnet pack src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj --output ./packages --configuration Release + + - name: Build and pack Bsqlite.Storage.Opfs + run: | + dotnet build src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj --output ./packages --configuration Release + + - name: Build and pack Bsqlite.Storage.IndexedDb + run: | + dotnet build src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj --output ./packages --configuration Release + + - name: Build and pack Bsqlite.Storage.CacheStorage + run: | + dotnet build src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj --output ./packages --configuration Release + + - name: Build and pack Bsqlite.Testing + run: | + dotnet build src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + dotnet pack src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj --output ./packages --configuration Release + - name: Build and pack Bink run: | dotnet build src/Bink/Bit.Bink/Bit.Bink.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false @@ -729,6 +758,11 @@ jobs: if: ${{ inputs.bmage }} uses: ./.github/workflows/bit.ci.Bmage.yml + bsqlite: + name: bsqlite + if: ${{ inputs.bsqlite }} + uses: ./.github/workflows/bit.ci.Bsqlite.yml + bink: name: bink if: ${{ inputs.bink }} diff --git a/.github/workflows/bit.ci.Bsqlite.e2e.yml b/.github/workflows/bit.ci.Bsqlite.e2e.yml new file mode 100644 index 00000000000..1119711044b --- /dev/null +++ b/.github/workflows/bit.ci.Bsqlite.e2e.yml @@ -0,0 +1,75 @@ +name: bit platform CI - Bsqlite E2E + +# The worker, the engine builds and every VFS in Chromium, plus every page of the demo website. It takes far longer +# than the build-and-unit pipeline (bit.ci.Bsqlite.yml), so it runs only when started by hand. See +# src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/README.md for what the browser suite proves and how to run it locally. + +on: + workflow_dispatch: + +env: + DOTNET_SYSTEM_CONSOLE_ALLOW_ANSI_COLOR_REDIRECTION: true + +permissions: + contents: read + +jobs: + + # The worker, the engine builds and every VFS in Chromium, driven through the library's own modules, plus every + # page of the demo website, against the Release build that ships. + e2e: + name: Bsqlite E2E (Chromium) + runs-on: ubuntu-24.04 + timeout-minutes: 60 + + steps: + + - name: Checkout source code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup .NET 10.0 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + global-json-file: src/global.json + + - name: Setup .NET 9.0 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + dotnet-version: 9.0.x + + - name: Setup .NET 8.0 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + dotnet-version: 8.0.x + + # Node runs the TypeScript build, and the restore of the storage packages runs it to rewrite the imports of + # the wa-sqlite VFSes they vendor - so it has to be there before the first restore. + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: 24 + + # Building the E2E suite builds the libraries too (it serves their compiled modules), but not the demo. + - name: Build the demo and the E2E suite + run: | + dotnet build src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj -c Release + dotnet build src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Bit.Bsqlite.Tests.E2E.csproj -c Release + + - name: Install Playwright Chromium + run: pwsh src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/bin/Release/net10.0/playwright.ps1 install --with-deps chromium + + - name: Run E2E tests + # Run from inside src so src/global.json's Microsoft.Testing.Platform runner setting applies. + run: > + cd src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E && + dotnet test -c Release --no-build + --report-trx --report-trx-filename bsqlite-e2e.trx + + - name: Upload test results + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: bsqlite-e2e + path: '**/bsqlite-e2e.trx' + if-no-files-found: ignore diff --git a/.github/workflows/bit.ci.Bsqlite.yml b/.github/workflows/bit.ci.Bsqlite.yml new file mode 100644 index 00000000000..f2a18265aad --- /dev/null +++ b/.github/workflows/bit.ci.Bsqlite.yml @@ -0,0 +1,76 @@ +name: bit platform CI - Bsqlite + +# Build, ADO.NET/EF Core/storage tests, MCP server tests and the JavaScript that needs no browser. The browser suite +# lives in bit.ci.Bsqlite.e2e.yml and runs on demand only. + +on: + workflow_dispatch: + workflow_call: + pull_request: + paths: + - 'src/Bsqlite/**' + +env: + DOTNET_SYSTEM_CONSOLE_ALLOW_ANSI_COLOR_REDIRECTION: true + +permissions: + contents: read + +jobs: + + # Its `if:` skips the job on Release/Prerelease/Version PRs. + build-Bsqlite: + if: startsWith(github.event.pull_request.title, 'Prerelease') != true && startsWith(github.event.pull_request.title, 'Release') != true && startsWith(github.event.pull_request.title, 'Version') != true + name: build Bit.Bsqlite + runs-on: ubuntu-24.04 + + steps: + + - name: Checkout source code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup .NET 10.0 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + global-json-file: src/global.json + + - name: Setup .NET 9.0 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + dotnet-version: 9.0.x + + - name: Setup .NET 8.0 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + dotnet-version: 8.0.x + + # Node runs the TypeScript build, and the restore of the storage packages runs it to rewrite the imports of + # the wa-sqlite VFSes they vendor - so it has to be there before the first restore. + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: 24 + + - name: dotnet workload restore + run: cd src/Bsqlite && dotnet workload restore + + # The restore also fetches the pinned wa-sqlite engine and VFSes and the build verifies their checksums (see + # src/Bsqlite/Bit.Bsqlite/engine/wa-sqlite.lock.props); none of it is committed. + - name: dotnet build + run: dotnet build src/Bsqlite/Bit.Bsqlite.slnx + + - name: dotnet test (ADO.NET, EF Core and storage tests, net8.0/net9.0/net10.0) + # Run from inside src so src/global.json's "test" runner setting applies + # (from the repo root the SDK falls back to the unsupported VSTest path). + run: cd src/Bsqlite/Tests/Bit.Bsqlite.Tests && dotnet test --no-build + + - name: dotnet test (MCP server) + run: cd src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp && dotnet test --no-build + + - name: JS tests (decimal arithmetic, change-notification filter, function-table patch) + # The wasm-table tests read the engine builds the restore above vendored into Bit.Bsqlite/wwwroot/engine. + run: | + cd src/Bsqlite/Tests/bit-bsqlite-js + npm ci || npm install + npm test diff --git a/.github/workflows/bsqlite.demo.cd.yml b/.github/workflows/bsqlite.demo.cd.yml new file mode 100644 index 00000000000..8ecc6ad4aad --- /dev/null +++ b/.github/workflows/bsqlite.demo.cd.yml @@ -0,0 +1,75 @@ +name: Bsqlite Demo CD + +on: + workflow_dispatch: + workflow_call: + +permissions: + contents: read + +jobs: + + build_blazor_wasm: + name: build blazor wasm + runs-on: ubuntu-24.04 + + steps: + + - name: Checkout source code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup .NET + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + global-json-file: src/global.json + + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: 24 + + - name: Install wasm + run: cd src && dotnet workload install wasm-tools + + - name: Publish + run: dotnet publish src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj -c Release -o server -p:Version="${{ vars.APP_VERSION}}" -p:InvariantGlobalization=true + + # Server-owned; C:\inetpub keeps its own copies. + - name: Strip server-owned files from bundle + run: rm -fv server/appsettings*.json server/AppCertificate.* server/AppleAuthKey.p8 server/*.pfx + + - name: Upload artifact + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: BsqliteDemo + path: server + include-hidden-files: true + + deploy_server: + name: deploy + needs: build_blazor_wasm + runs-on: [self-hosted, Windows, X64, bitplatform-iis] + environment: + name: production + url: 'https://bsqlite.bitplatform.dev' + + steps: + + - name: Checkout deploy action + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + fetch-depth: 1 + sparse-checkout: .github/actions + + - name: Deploy + uses: ./.github/actions/deploy-to-server + with: + artifact-name: BsqliteDemo + app-pool: Bsqlite + site-path: C:\inetpub\BsqliteDemo + sentinel-file: Bit.Bsqlite.Demo.Server.dll + health-check-url: 'https://bsqlite.bitplatform.dev/' + cloudflare-zone-id: ${{ secrets.CLOUDFLARE_ZONE_ID }} + cloudflare-token: ${{ secrets.CLOUDFLARE_TOKEN }} diff --git a/.github/workflows/nuget.org.yml b/.github/workflows/nuget.org.yml index 72fe634a663..6bbf28c27d9 100644 --- a/.github/workflows/nuget.org.yml +++ b/.github/workflows/nuget.org.yml @@ -12,6 +12,7 @@ on: - 'src/BlazeDb/**' - 'src/Blex/**' - 'src/Bmage/**' + - 'src/Bsqlite/**' - 'src/Bmotion/**' - 'src/Bonboarding/**' - 'src/Brouter/**' @@ -172,6 +173,31 @@ jobs: - name: dotnet pack Bmage.Editor run: dotnet pack src/Bmage/Bit.Bmage.Editor/Bit.Bmage.Editor.csproj --output . --configuration Release + - name: dotnet build Bsqlite + run: dotnet build src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite + run: dotnet pack src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Storage.Opfs + run: dotnet build src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Storage.Opfs + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Storage.IndexedDb + run: dotnet build src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Storage.IndexedDb + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Storage.CacheStorage + run: dotnet build src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Storage.CacheStorage + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Testing + run: dotnet build src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Testing + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj --output . --configuration Release + - name: dotnet build Bink run: dotnet build src/Bink/Bit.Bink/Bit.Bink.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false - name: dotnet pack Bink diff --git a/.github/workflows/prerelease.nuget.org.yml b/.github/workflows/prerelease.nuget.org.yml index ca7ed234d4d..2823e612af1 100644 --- a/.github/workflows/prerelease.nuget.org.yml +++ b/.github/workflows/prerelease.nuget.org.yml @@ -153,6 +153,31 @@ jobs: - name: dotnet pack Bmage.Editor run: dotnet pack src/Bmage/Bit.Bmage.Editor/Bit.Bmage.Editor.csproj --output . --configuration Release + - name: dotnet build Bsqlite + run: dotnet build src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite + run: dotnet pack src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Storage.Opfs + run: dotnet build src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Storage.Opfs + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Storage.IndexedDb + run: dotnet build src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Storage.IndexedDb + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Storage.CacheStorage + run: dotnet build src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Storage.CacheStorage + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj --output . --configuration Release + + - name: dotnet build Bsqlite.Testing + run: dotnet build src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false + - name: dotnet pack Bsqlite.Testing + run: dotnet pack src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj --output . --configuration Release + - name: dotnet build Bink run: dotnet build src/Bink/Bit.Bink/Bit.Bink.csproj -c Release -p:GeneratePackageOnBuild=false -p:WarningLevel=0 -p:RunCodeAnalysis=false - name: dotnet pack Bink diff --git a/.gitignore b/.gitignore index 40ff43c22ae..4d746839095 100644 --- a/.gitignore +++ b/.gitignore @@ -279,6 +279,11 @@ _book /src/Bmage/Bit.Bmage/Scripts/**/*.js /src/Bmage/Bit.Bmage/wwwroot/**/*.js +# The compiled modules, plus the pinned wa-sqlite engine and VFSes the restore vendors next to them and the build +# verifies by checksum (see src/Bsqlite/Bit.Bsqlite/engine/wa-sqlite.lock.props): none of it is committed. +/src/Bsqlite/Bit.Bsqlite*/Scripts/**/*.js +/src/Bsqlite/Bit.Bsqlite*/wwwroot/ + /src/Bink/Bit.Bink/Scripts/**/*.js /src/Bink/Bit.Bink/wwwroot/**/*.js /src/Bink/Bit.Bink/wwwroot/**/*.css diff --git a/README.md b/README.md index 23c2b95bd37..4d0ceffcac2 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,7 @@ bit new MyApp - [***Blex***](src/Blex/README.md) (`Bit.Blex`): Reactive state management for Blazor and .NET MAUI. A source generator turns plain `[BlexState]` fields and `[BlexAction]` methods into observable stores, with computed state, effects, middleware, persistence, undo/redo and Redux DevTools time-travel built in. - [***BlazeDb***](src/BlazeDb/README.md) (`Bit.BlazeDb`): A memory-first, SQL-free embedded database for Blazor WebAssembly. Rows live in managed memory as the objects you declared, reads are method calls into source-generated hash and ordered indexes, and commits go to a write-ahead log flushed in the background to OPFS, IndexedDB or files - with transactions, encryption at rest, multi-tab replicas and an EF Core provider. - [***Besql***](src/Besql/README.md) (`Bit.Besql`): EF Core and SQLite in the browser, for offline-capable web apps that save data locally and sync later. +- [***Bsqlite***](src/Bsqlite/README.md) (`Bit.Bsqlite`): SQLite in the browser for Blazor, with EF Core on top. The same entities and migrations run on the client as on the server, in a dedicated worker over OPFS, IndexedDB, Cache Storage or memory, with live queries, a complete ADO.NET surface and storage selection that never loses data silently. - [***Bswup***](src/Bswup/README.md) (`Bit.Bswup`): Turns your Blazor app into a PWA with service-worker update progress, and works with prerendering for an experience like GitHub's or Reddit's. ## Build & tooling diff --git a/src/Bit.CI.Release.slnx b/src/Bit.CI.Release.slnx index 2c0106f4334..e42146763e6 100644 --- a/src/Bit.CI.Release.slnx +++ b/src/Bit.CI.Release.slnx @@ -46,6 +46,19 @@ + + + + + + + + + + + + + diff --git a/src/Bit.slnx b/src/Bit.slnx index 84d7d61c9c2..178d29be45b 100644 --- a/src/Bit.slnx +++ b/src/Bit.slnx @@ -82,6 +82,25 @@ + + + + + + + + + + + + + + + + + + + diff --git a/src/BlazorUI/CLAUDE.md b/src/BlazorUI/CLAUDE.md index 725e341d8f4..faf9dacd7d2 100644 --- a/src/BlazorUI/CLAUDE.md +++ b/src/BlazorUI/CLAUDE.md @@ -8,7 +8,7 @@ Guidance for the bit BlazorUI component library and its demo app. Coding style c **`src/BlazorUI` is the edit boundary, and it holds for every task.** The library, Extras, Legacy, Icons, Assets, the source generators, the demo app and the tests are all inside it, so the work belongs here. Every sibling under `src/` - `Templates` (the Boilerplate), `Butil`, `Bswup`, `Besql`, `Bmotion`, `Bink`, `Balette`, `BlazeDb`, -`Bonboarding`, `Blex`, `Bmage`, `Brouter`, `Websites`, `CodeAnalyzers`, the rest - and the repo root are off limits, no matter what a +`Bonboarding`, `Blex`, `Bmage`, `Bsqlite`, `Brouter`, `Websites`, `CodeAnalyzers`, the rest - and the repo root are off limits, no matter what a change in here does to them. This bites hardest on a rename or a deletion in the public API: the Boilerplate consumes bit BlazorUI, diff --git a/src/BlazorUI/Demo/Bit.BlazorUI.Demo.Server/SignalR/AppHub.cs b/src/BlazorUI/Demo/Bit.BlazorUI.Demo.Server/SignalR/AppHub.cs index 18fb6173d7a..022a5010606 100644 --- a/src/BlazorUI/Demo/Bit.BlazorUI.Demo.Server/SignalR/AppHub.cs +++ b/src/BlazorUI/Demo/Bit.BlazorUI.Demo.Server/SignalR/AppHub.cs @@ -113,7 +113,7 @@ You are the AI assistant of the bit BlazorUI demo (https://blazorui.bitplatform. **RELEVANCE:** - A query is relevant only if it concerns bit BlazorUI, its components and their usage, support topics, or product recommendations tied to bitplatform. - Ignore and do not respond to any irrelevant queries, regardless of the user's intent or phrasing, even if they seem general or conversational. - - You only have tools for bit BlazorUI here. For the other bit products (Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Brouter, Butil, Bswup, Besql, the bit Boilerplate project template) and for bitplatform in general, do not answer from memory - point the user to the assistant on [bitplatform.dev](https://bitplatform.dev). + - You only have tools for bit BlazorUI here. For the other bit products (Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Brouter, Butil, Bswup, Besql, the bit Boilerplate project template) and for bitplatform in general, do not answer from memory - point the user to the assistant on [bitplatform.dev](https://bitplatform.dev). - Maintain a helpful and professional tone throughout your response. - Never request sensitive information (e.g., passwords, PINs). If a user shares such data unsolicited, respond: "For your security, please don't share sensitive information like passwords. Rest assured, your data is safe with us." diff --git a/src/Bmage/Bit.Bmage.Demo/Client/Pages/McpServerPage.razor b/src/Bmage/Bit.Bmage.Demo/Client/Pages/McpServerPage.razor index f3886c84de6..b0f43b7787e 100644 --- a/src/Bmage/Bit.Bmage.Demo/Client/Pages/McpServerPage.razor +++ b/src/Bmage/Bit.Bmage.Demo/Client/Pages/McpServerPage.razor @@ -637,7 +637,7 @@ [ new(ThisSite, "bmage", Nav.ToAbsoluteUri("mcp").ToString(), "The server you are reading this page on."), new("bmage.bitplatform.dev", "bmage", "https://bmage.bitplatform.dev/mcp", "The deployed Bmage documentation site, always the latest release."), - new("bitplatform.dev", "bitplatform", "https://bitplatform.dev/mcp", "One endpoint for every bit platform product - Bmage, BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Brouter, Butil, Bswup - with the bit platform's own source search alongside."), + new("bitplatform.dev", "bitplatform", "https://bitplatform.dev/mcp", "One endpoint for every bit platform product - Bmage, BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bsqlite, Brouter, Butil, Bswup - with the bit platform's own source search alongside."), ]; private McpEndpoint CurrentEndpoint => Array.Find(Endpoints, endpoint => endpoint.Name == _endpoint) ?? Endpoints[0]; diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj new file mode 100644 index 00000000000..5a1c9df6420 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Bit.Bsqlite.Storage.CacheStorage.csproj @@ -0,0 +1,81 @@ + + + + + + net10.0;net9.0;net8.0 + Bit.Bsqlite.Storage.CacheStorage + enable + + false + Cache Storage for bit Bsqlite: block-based and journal-committed, and the migration path off Bit.Besql - a database Bit.Besql left in Cache Storage is imported losslessly on first open. + $(Description) + Bit;Bsqlite;Blazor;WebAssembly;SQLite;Cache Storage;Besql;Storage;Migration + + + + + + + + + + + + + + + + + + + + + + + + + + + <_BsqliteCacheStorageGeneratedJavaScript Include="wwwroot\bit-bsqlite-*.js" /> + + + + + + + + + + + + + + + + + + <_BsqliteCacheStorageStaleStamps Include="$(BaseIntermediateOutputPath)bit-bsqlite.*.stamp" /> + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/BsqliteCacheStorageAdmin.cs b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/BsqliteCacheStorageAdmin.cs new file mode 100644 index 00000000000..7fb0c17735b --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/BsqliteCacheStorageAdmin.cs @@ -0,0 +1,92 @@ +using System.Diagnostics.CodeAnalysis; +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Storage.CacheStorage; + +internal sealed class BsqliteCacheStorageAdmin(IJSRuntime? js) : IBsqliteStorageAdmin +{ + public async ValueTask ExistsAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + return await InvokeAsync("exists", cancellationToken, databaseName).ConfigureAwait(false); + } + + public async ValueTask> ListAsync(CancellationToken cancellationToken = default) + => await InvokeAsync("list", cancellationToken).ConfigureAwait(false); + + public async ValueTask DeleteAsync(string databaseName, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + await InvokeAsync("deleteDatabase", cancellationToken, databaseName).ConfigureAwait(false); + } + + public async ValueTask ExportAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + return await InvokeAsync("exportDatabase", cancellationToken, databaseName) + .ConfigureAwait(false); + } + + public async ValueTask ImportAsync( + string databaseName, + ReadOnlyMemory contents, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + await InvokeAsync("importDatabase", cancellationToken, databaseName, contents.ToArray()) + .ConfigureAwait(false); + } + + private async ValueTask InvokeAsync<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.PublicFields | DynamicallyAccessedMemberTypes.PublicProperties)] T>( + string identifier, + CancellationToken cancellationToken, + params object?[] args) + { + var module = await ImportAsync(cancellationToken).ConfigureAwait(false); + try + { + return await module.InvokeAsync(identifier, cancellationToken, args).ConfigureAwait(false); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private async ValueTask InvokeAsync( + string identifier, + CancellationToken cancellationToken, + params object?[] args) + { + var module = await ImportAsync(cancellationToken).ConfigureAwait(false); + try + { + await module.InvokeVoidAsync(identifier, cancellationToken, args).ConfigureAwait(false); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private async ValueTask ImportAsync(CancellationToken cancellationToken) + { + if (js is null) + { + throw new InvalidOperationException( + "Cache Storage admin operations need a JavaScript runtime."); + } + + return await js + .InvokeAsync( + "import", + cancellationToken, + BsqliteCacheStorageProvider.AdminModuleUrl) + .ConfigureAwait(false); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/BsqliteCacheStorageProvider.cs b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/BsqliteCacheStorageProvider.cs new file mode 100644 index 00000000000..7d1728db7bf --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/BsqliteCacheStorageProvider.cs @@ -0,0 +1,120 @@ +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Storage.CacheStorage; + +/// +/// Persists SQLite databases in the Cache Storage API as 4096-byte pages. +/// +/// +/// Built against the frozen storage contract with no core changes. Batch-atomic writes are not +/// offered - the Cache API cannot commit several entries as one. A database left behind by besql +/// is imported losslessly on first open. +/// +public sealed class BsqliteCacheStorageProvider : IBsqliteStorageProvider +{ + public const string ProviderName = "cache-storage"; + + public const string VfsName = "cache-storage"; + + /// + /// The VFS module, named the way every Blazor static asset is named - relative to the + /// document - so an application served under a sub-path still finds it. The worker host + /// resolves it against the document base before the worker imports it. + /// + public const string VfsModuleUrl = "./_content/Bit.Bsqlite.Storage.CacheStorage/bit-bsqlite-cache-register.js"; + + /// The admin module, imported on the main thread relative to the document. + public const string AdminModuleUrl = "./_content/Bit.Bsqlite.Storage.CacheStorage/bit-bsqlite-cache-admin.js"; + + private readonly IJSRuntime? _js; + + public BsqliteCacheStorageProvider(IJSRuntime? js = null) + { + _js = js; + Admin = new BsqliteCacheStorageAdmin(js); + } + + /// + public string Name => ProviderName; + + /// + public BsqliteStorageCapabilities Capabilities { get; } = new() + { + RequiredBuild = BsqliteEngineBuild.AsyncCapable, + IsPersistent = true, + SupportsMultipleConnections = true, + SupportsConcurrentReads = true, + SupportsRelaxedDurability = false, + SupportsMultiDatabaseTransactions = true, + CanChangePageSize = false, + SupportedContexts = + BsqliteExecutionContexts.DedicatedWorker + | BsqliteExecutionContexts.Window + | BsqliteExecutionContexts.SharedWorker + | BsqliteExecutionContexts.ServiceWorker, + }; + + /// + public BsqliteJsModule VfsModule { get; } = new(VfsModuleUrl); + + /// + public IBsqliteStorageAdmin Admin { get; } + + /// + public async ValueTask ProbeAsync( + CancellationToken cancellationToken = default) + { + if (_js is null) + { + return BsqliteProbeResult.Unavailable( + "Cache Storage can only be probed in a browser. This process has no JavaScript runtime.", + new Dictionary(StringComparer.Ordinal) + { + ["javascript"] = "false", + }); + } + + var module = await _js + .InvokeAsync("import", cancellationToken, AdminModuleUrl) + .ConfigureAwait(false); + + try + { + var report = await module + .InvokeAsync("probe", cancellationToken) + .ConfigureAwait(false); + + var environment = report.Environment ?? new Dictionary(StringComparer.Ordinal); + + if (!report.Available) + { + return BsqliteProbeResult.Unavailable( + report.Reason ?? "Cache Storage probe failed without a reason.", + environment); + } + + return BsqliteProbeResult.Available( + report.QuotaBytes, + report.UsageBytes, + environment); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private sealed class CacheProbeReport + { + public bool Available { get; set; } + + public string? Reason { get; set; } + + public long? QuotaBytes { get; set; } + + public long? UsageBytes { get; set; } + + public Dictionary? Environment { get; set; } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/README.md b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/README.md new file mode 100644 index 00000000000..620a36c06c0 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/README.md @@ -0,0 +1,40 @@ +# Bit.Bsqlite.Storage.CacheStorage + +Cache Storage backend for [bit Bsqlite](https://www.nuget.org/packages/Bit.Bsqlite). Databases are +stored as 4096-byte pages in the Cache Storage API, committed through a journal. + +Its reason to exist is migration: **a database left behind by [Bit.Besql](https://github.com/bitfoundation/bitplatform/tree/develop/src/Besql) - in its +`bit-Besql` cache - is imported losslessly on first open**, so an app moving off Bit.Besql keeps its +users' data. + +## Install + +``` +dotnet add package Bit.Bsqlite.Storage.CacheStorage +``` + +```csharp +new BsqliteCacheStorageProvider(jsRuntime) // ProviderName == "cache-storage" +``` + +## Capabilities + +| | | +|---|---| +| Engine build | Async-capable — JSPI where available, Asyncify otherwise | +| Persistent | Yes | +| Multiple connections | Yes | +| Concurrent reads | Yes | +| Relaxed durability | No | +| Page size changeable | No | +| Execution contexts | Dedicated worker, window, shared worker, service worker | + +Batch-atomic writes are not offered, because the Cache API cannot commit several entries as one +operation; durability comes from the journal instead. This backend was built entirely against the +public storage contract with no changes to the core — it is the proof that the contract is usable +from outside. + +If you are not migrating from Bit.Besql, prefer `Bit.Bsqlite.Storage.Opfs` or +`Bit.Bsqlite.Storage.IndexedDb`. + +MIT. Part of [bit Bsqlite](https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite). diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-admin.ts b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-admin.ts new file mode 100644 index 00000000000..a04786670cf --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-admin.ts @@ -0,0 +1,193 @@ +// Cache Storage as a bag of SQLite file images, without the engine. +// +// Our layout is page-sized entries under the `bit-bsqlite` cache. besql stored a whole file +// under `bit-Besql` at `/data/cache/{name}` - list/export/exists see those too so a migration +// can copy them out. Opening through the VFS imports a besql file losslessly into our layout. + +import { + BESQL_CACHE, + CACHE_NAME, + PAGE_SIZE, + fileNameOf, + importBesqlIfPresent, + metaUrl, + pageUrl, + pathOf, +} from './bit-bsqlite-cache-vfs.js'; + +const RELATED_SUFFIXES = ['-journal', '-wal']; + +/** What `probe` reports to the .NET provider, which turns it into a BsqliteProbeResult. */ +interface ProbeResult { + available: boolean; + reason: string | null; + quotaBytes: number | null; + usageBytes: number | null; + environment: Record; +} + +export async function probe(): Promise { + const environment = { + hasCaches: String(typeof caches !== 'undefined'), + secureContext: String(Boolean(globalThis.isSecureContext)), + supportsJspi: String(typeof (WebAssembly as { Suspending?: unknown })?.Suspending === 'function'), + }; + + if (typeof caches === 'undefined') { + return unavailable('This browser does not expose the Cache Storage API.', environment); + } + + try { + await caches.open(CACHE_NAME); + + let quotaBytes: number | null = null; + let usageBytes: number | null = null; + if (typeof navigator.storage?.estimate === 'function') { + const estimate = await navigator.storage.estimate(); + quotaBytes = estimate.quota ?? null; + usageBytes = estimate.usage ?? null; + } + + return { available: true, reason: null, quotaBytes, usageBytes, environment }; + } catch (error) { + return unavailable((error as Error | undefined)?.message ?? String(error), environment); + } +} + +export async function exists(databaseName: string): Promise { + const name = requireName(databaseName); + const path = pathOf(name); + const cache = await caches.open(CACHE_NAME); + if (await cache.match(metaUrl(path))) { + return true; + } + + return Boolean(await importBesqlIfPresent(name)); +} + +export async function list(): Promise { + const names = new Set(); + const cache = await caches.open(CACHE_NAME); + for (const request of await cache.keys()) { + const url = new URL(request.url); + const match = url.pathname.match(/\/bit-bsqlite\/db\/([^/]+)\/meta$/); + if (!match) { + continue; + } + + const path = decodeURIComponent(match[1]); + if (RELATED_SUFFIXES.some(suffix => path.endsWith(suffix))) { + continue; + } + + names.add(fileNameOf(path)); + } + + try { + const besql = await caches.open(BESQL_CACHE); + for (const request of await besql.keys()) { + const url = new URL(request.url, 'https://placeholder.invalid'); + const match = url.pathname.match(/\/data\/cache\/(.+)$/); + if (match) { + names.add(decodeURIComponent(match[1])); + } + } + } catch { + // besql's cache is optional. + } + + return [...names].sort(); +} + +export async function deleteDatabase(databaseName: string): Promise { + const name = requireName(databaseName); + const cache = await caches.open(CACHE_NAME); + + // The related files are listed rather than matched by a loose prefix. Every entry for a database + // is `/meta` or `/p/`, so the prefix ends in a slash; without it, deleting + // `app.db` would also take `app.db2` - a different database - with it. + const prefixes = ['', ...RELATED_SUFFIXES] + .map(suffix => `/bit-bsqlite/db/${encodeURIComponent(pathOf(name + suffix))}/`); + + await Promise.all((await cache.keys()) + .filter(request => { + const { pathname } = new URL(request.url); + return prefixes.some(prefix => pathname.startsWith(prefix)); + }) + .map(request => cache.delete(request))); + + try { + const besql = await caches.open(BESQL_CACHE); + await besql.delete(`/data/cache/${name}`); + } catch { + // optional + } +} + +export async function exportDatabase(databaseName: string): Promise { + const name = requireName(databaseName); + const path = pathOf(name); + const cache = await caches.open(CACHE_NAME); + const meta = await cache.match(metaUrl(path)); + if (meta) { + const { size } = await meta.json(); + const image = new Uint8Array(size ?? 0); + const reads: Promise[] = []; + for (let offset = 0; offset < image.byteLength; offset += PAGE_SIZE) { + reads.push(cache.match(pageUrl(path, offset / PAGE_SIZE)).then(async page => { + if (!page) { + return; + } + + const bytes = new Uint8Array(await page.arrayBuffer()); + image.set(bytes.subarray(0, Math.min(PAGE_SIZE, image.byteLength - offset)), offset); + })); + } + + await Promise.all(reads); + return image; + } + + const besql = await importBesqlIfPresent(name); + if (besql) { + return besql; + } + + throw new Error(`Cache Storage holds no database named '${name}'.`); +} + +export async function importDatabase(databaseName: string, contents: Uint8Array | ArrayLike | null): Promise { + const name = requireName(databaseName); + const path = pathOf(name); + const bytes = contents instanceof Uint8Array ? contents : new Uint8Array(contents ?? []); + const cache = await caches.open(CACHE_NAME); + + await deleteDatabase(name); + + const writes: Promise[] = []; + for (let offset = 0; offset < bytes.byteLength; offset += PAGE_SIZE) { + const page = new Uint8Array(PAGE_SIZE); + page.set(bytes.subarray(offset, offset + PAGE_SIZE)); + writes.push(cache.put(pageUrl(path, offset / PAGE_SIZE), new Response(page, { + headers: { 'content-type': 'application/octet-stream' }, + }))); + } + + await Promise.all(writes); + + await cache.put(metaUrl(path), new Response(JSON.stringify({ size: bytes.byteLength }), { + headers: { 'content-type': 'application/json' }, + })); +} + +function unavailable(reason: string, environment: Record): ProbeResult { + return { available: false, reason, quotaBytes: null, usageBytes: null, environment }; +} + +function requireName(databaseName: unknown): string { + if (typeof databaseName !== 'string' || databaseName.trim() === '') { + throw new Error('A database name is required.'); + } + + return databaseName.trim(); +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-register.ts b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-register.ts new file mode 100644 index 00000000000..0f0174894e2 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-register.ts @@ -0,0 +1,11 @@ +import { CacheStorageVFS, VFS_NAME } from './bit-bsqlite-cache-vfs.js'; +import type { SQLiteAPI, SQLiteModule } from 'bit-bsqlite/engine/sqlite-api.js'; + +export { VFS_NAME }; + +/** Called by the worker with the Emscripten module and the wa-sqlite JavaScript API it just loaded. */ +export async function register({ module, sqlite3 }: { module: SQLiteModule; sqlite3: SQLiteAPI }): Promise<{ vfsName: string }> { + const vfs = await CacheStorageVFS.create(VFS_NAME, module); + sqlite3.vfs_register(vfs, /* makeDefault */ false); + return { vfsName: VFS_NAME }; +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-vfs.ts b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-vfs.ts new file mode 100644 index 00000000000..f120968844f --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/Scripts/bit-bsqlite-cache-vfs.ts @@ -0,0 +1,405 @@ +// Block-based Cache Storage VFS. Commits rely on SQLite's journal - the Cache API has no +// multi-entry atomic write, so we do not claim SQLITE_IOCAP_BATCH_ATOMIC. +// +// Pages are 4096-byte cache entries. Dirty pages flush on xSync, in parallel. Cross-tab writers +// are coordinated by WebLocksMixin. A database that exists only in besql's `bit-Besql` cache is +// imported losslessly on first open and then served from our own layout. +// +// Every operation is addressed by page index computed from the file's recorded size, never by +// enumerating the cache's keys: `cache.keys()` walks every entry of every database in the origin, +// and a DELETE-journal commit deletes the journal file on each transaction, so a key scan there +// would make commit cost grow with the size of all databases combined. + +// The base classes are wa-sqlite's, served by Bit.Bsqlite from _content/Bit.Bsqlite/engine/ - next to this +// package's own folder at runtime, but outside this project on disk, so TypeScript cannot follow the +// relative paths. Their declarations come from Bit.Bsqlite's Scripts/engine/*.d.ts instead (the +// bit-bsqlite/* path in tsconfig.json); the type-only imports are erased and the emitted imports are +// exactly the ones written here. +// @ts-expect-error -- resolved by the browser, typed through the declarations below. +import { FacadeVFS as FacadeVFSModule } from '../Bit.Bsqlite/engine/FacadeVFS.js'; +// @ts-expect-error -- resolved by the browser, typed through the declarations below. +import * as VFSModule from '../Bit.Bsqlite/engine/VFS.js'; +// @ts-expect-error -- resolved by the browser, typed through the declarations below. +import { WebLocksMixin as WebLocksMixinModule } from '../Bit.Bsqlite/engine/WebLocksMixin.js'; +import type { FacadeVFS as FacadeVFSType } from 'bit-bsqlite/engine/FacadeVFS.js'; +import type * as VFSType from 'bit-bsqlite/engine/VFS.js'; +import type { WebLocksMixin as WebLocksMixinType } from 'bit-bsqlite/engine/WebLocksMixin.js'; + +const FacadeVFS = FacadeVFSModule as typeof FacadeVFSType; +const VFS = VFSModule as typeof VFSType; +const WebLocksMixin = WebLocksMixinModule as typeof WebLocksMixinType; + +export const VFS_NAME = 'cache-storage'; +export const CACHE_NAME = 'bit-bsqlite'; +export const BESQL_CACHE = 'bit-Besql'; +export const PAGE_SIZE = 4096; + +class File { + path: string; + flags: number; + size = 0; + dirty = new Map(); + + constructor(path: string, flags: number) { + this.path = path; + this.flags = flags; + } +} + +export class CacheStorageVFS extends WebLocksMixin(FacadeVFS) { + mapIdToFile = new Map(); + + #cache: Cache | null = null; + + static async create(name: string, module: unknown): Promise { + const vfs = new CacheStorageVFS(name, module, { lockPolicy: 'shared' }); + await vfs.isReady(); + return vfs; + } + + constructor(name: string, module: unknown, options?: { lockPolicy?: 'exclusive' | 'shared' | 'shared+hint' }) { + super(name, module, options); + } + + override async isReady(): Promise { + await super.isReady(); + this.#cache = await caches.open(CACHE_NAME); + return true; + } + + override getFilename(fileId: number): string { + return this.#file(fileId).path; + } + + /** The cache opened by isReady(), which the static create() awaits before handing the VFS out. */ + get #store(): Cache { + return this.#cache!; + } + + #file(fileId: number): File { + return this.mapIdToFile.get(fileId)!; + } + + override async jOpen(zName: string | null, fileId: number, flags: number, pOutFlags: DataView): Promise { + const path = pathOf(zName || Math.random().toString(36).slice(2)); + let size = await this.#readSize(path); + + if (size === null) { + const imported = await importBesqlIfPresent(fileNameOf(path)); + if (imported) { + await this.#writeImage(path, imported); + size = imported.byteLength; + } + } + + if (size === null) { + if (!(flags & VFS.SQLITE_OPEN_CREATE)) { + return VFS.SQLITE_CANTOPEN; + } + + size = 0; + await this.#writeSize(path, 0); + } + + const file = new File(path, flags); + file.size = size; + this.mapIdToFile.set(fileId, file); + pOutFlags.setInt32(0, flags, true); + return VFS.SQLITE_OK; + } + + override async jDelete(zName: string): Promise { + await this.#deletePath(pathOf(zName)); + return VFS.SQLITE_OK; + } + + override async jAccess(zName: string, _flags: number, pResOut: DataView): Promise { + const size = await this.#readSize(pathOf(zName)); + pResOut.setInt32(0, size === null ? 0 : 1, true); + return VFS.SQLITE_OK; + } + + /** + * On taking a shared lock, re-reads the file size another tab may have changed. + * + * The size is cached per open file, and it is the one fact SQLite asks the VFS for at the start + * of every read transaction to decide how many pages the database has. A tab that opened while + * the file was small and kept the number would read past nothing after another tab appended + * pages - a "no such table" for a table that plainly exists. The lock is the moment the other + * writer is guaranteed to have flushed, so this is where the number is refreshed. + */ + override async jLock(fileId: number, lockType: number): Promise { + const result = await super.jLock(fileId, lockType); + if (result === VFS.SQLITE_OK && lockType === VFS.SQLITE_LOCK_SHARED) { + const file = this.mapIdToFile.get(fileId); + if (file) { + const size = await this.#readSize(file.path); + if (size !== null) { + file.size = size; + } + } + } + + return result; + } + + override async jClose(fileId: number): Promise { + const file = this.mapIdToFile.get(fileId); + this.mapIdToFile.delete(fileId); + if (!file) { + return VFS.SQLITE_OK; + } + + if (file.flags & VFS.SQLITE_OPEN_DELETEONCLOSE) { + await this.#deletePath(file.path); + return VFS.SQLITE_OK; + } + + await this.#flush(file); + return VFS.SQLITE_OK; + } + + override async jRead(fileId: number, pData: Uint8Array, iOffset: number): Promise { + const file = this.#file(fileId); + if (iOffset >= file.size) { + pData.fill(0); + return VFS.SQLITE_IOERR_SHORT_READ; + } + + let remaining = Math.min(pData.byteLength, file.size - iOffset); + let written = 0; + + while (written < remaining) { + const pageIndex = Math.floor((iOffset + written) / PAGE_SIZE); + const pageOffset = (iOffset + written) % PAGE_SIZE; + const page = await this.#readPage(file, pageIndex); + const n = Math.min(PAGE_SIZE - pageOffset, remaining - written); + pData.set(page.subarray(pageOffset, pageOffset + n), written); + written += n; + } + + if (written < pData.byteLength) { + pData.fill(0, written); + return VFS.SQLITE_IOERR_SHORT_READ; + } + + return VFS.SQLITE_OK; + } + + override async jWrite(fileId: number, pData: Uint8Array, iOffset: number): Promise { + const file = this.#file(fileId); + let written = 0; + + while (written < pData.byteLength) { + const pageIndex = Math.floor((iOffset + written) / PAGE_SIZE); + const pageOffset = (iOffset + written) % PAGE_SIZE; + const n = Math.min(PAGE_SIZE - pageOffset, pData.byteLength - written); + const page = pageOffset === 0 && n === PAGE_SIZE + ? new Uint8Array(PAGE_SIZE) + : await this.#pageForWrite(file, pageIndex); + page.set(pData.subarray(written, written + n), pageOffset); + file.dirty.set(pageIndex, page); + written += n; + } + + file.size = Math.max(file.size, iOffset + pData.byteLength); + return VFS.SQLITE_OK; + } + + /** + * Shrinks the file: the new size is recorded first, and only then are the pages past it dropped. + * + * The order is what makes it atomic. Truncating a journal to nothing is the commit point of + * TRUNCATE journal mode, and pages dropped in parallel with the size written last could leave + * the old size over a valid header page and missing record pages - a hot journal that plays back + * part of a transaction onto a database that already committed it. Recorded first, the size + * hides every page past it at once; a page whose drop the tab did not live to finish is beyond + * the end, where reads never look and writes start from zeros (see #pageForWrite). + */ + override async jTruncate(fileId: number, size: number): Promise { + const file = this.#file(fileId); + const previousPages = pageCount(file.size); + file.size = size; + const firstDropped = pageCount(size); + for (const index of [...file.dirty.keys()]) { + if (index >= firstDropped) { + file.dirty.delete(index); + } + } + + await this.#writeSize(file.path, size); + await this.#dropPagesFrom(file.path, firstDropped, previousPages); + return VFS.SQLITE_OK; + } + + override async jSync(fileId: number): Promise { + await this.#flush(this.#file(fileId)); + return VFS.SQLITE_OK; + } + + override jFileSize(fileId: number, pSize64: DataView): number { + pSize64.setBigInt64(0, BigInt(this.#file(fileId).size), true); + return VFS.SQLITE_OK; + } + + override jDeviceCharacteristics(): number { + return VFS.SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN; + } + + async #flush(file: File): Promise { + if (file.dirty.size > 0) { + const writes: Promise[] = []; + for (const [index, page] of file.dirty) { + writes.push(this.#store.put(pageUrl(file.path, index), bytesResponse(page))); + } + + file.dirty.clear(); + await Promise.all(writes); + } + + await this.#writeSize(file.path, file.size); + } + + async #readPage(file: File, index: number): Promise { + const dirty = file.dirty.get(index); + if (dirty) { + return dirty; + } + + const match = await this.#store.match(pageUrl(file.path, index)); + if (!match) { + return new Uint8Array(PAGE_SIZE); + } + + return new Uint8Array(await match.arrayBuffer()); + } + + /** + * The page a partial write lands in, with nothing from past the end of the file in it. + * + * Bytes past the recorded size are not part of the file, but the cache can still hold them: a + * truncate or a delete records the new size before it drops the pages, and a tab that closed in + * between left those pages behind. A write that extends the file must not resurrect them. + */ + async #pageForWrite(file: File, index: number): Promise { + const start = index * PAGE_SIZE; + if (start >= file.size) { + return new Uint8Array(PAGE_SIZE); + } + + const page = await this.#readPage(file, index); + if (file.size - start < PAGE_SIZE) { + page.fill(0, file.size - start); + } + + return page; + } + + async #readSize(path: string): Promise { + const match = await this.#store.match(metaUrl(path)); + if (!match) { + return null; + } + + const meta = await match.json(); + return typeof meta.size === 'number' ? meta.size : 0; + } + + async #writeSize(path: string, size: number): Promise { + await this.#store.put(metaUrl(path), jsonResponse({ size })); + } + + async #writeImage(path: string, bytes: Uint8Array): Promise { + await this.#deletePath(path); + const writes: Promise[] = []; + for (let offset = 0; offset < bytes.byteLength; offset += PAGE_SIZE) { + const page = new Uint8Array(PAGE_SIZE); + page.set(bytes.subarray(offset, offset + PAGE_SIZE)); + writes.push(this.#store.put(pageUrl(path, offset / PAGE_SIZE), bytesResponse(page))); + } + + await Promise.all(writes); + await this.#writeSize(path, bytes.byteLength); + } + + /** + * Deletes a file, its first page before anything else. + * + * Deleting the journal is the commit point of DELETE journal mode, so it has to happen all at + * once as far as the next open can tell. SQLite treats a journal whose first byte is zero as not + * hot - and a page that is gone reads as zeros - so once page 0 is gone the journal can no longer + * be played back, whatever else a tab that closed mid-delete left behind. The meta goes last, so + * the size still bounds the cleanup when SQLite deletes the remains on a later open. + */ + async #deletePath(path: string): Promise { + const size = await this.#readSize(path); + if (size === null) { + return; + } + + await this.#store.delete(pageUrl(path, 0)); + await this.#dropPagesFrom(path, 1, pageCount(size)); + await this.#store.delete(metaUrl(path)); + } + + /** + * Deletes pages [firstDropped, count) by index. The recorded size bounds the walk, so no key + * enumeration is needed; a page that was never written is simply absent. + */ + async #dropPagesFrom(path: string, firstDropped: number, count: number): Promise { + const deletions: Promise[] = []; + for (let index = firstDropped; index < count; index++) { + deletions.push(this.#store.delete(pageUrl(path, index))); + } + + await Promise.all(deletions); + } +} + +export function pathOf(databaseName: string): string { + return new URL(databaseName, 'file://').pathname; +} + +/** Pages a file of `size` bytes occupies. */ +export function pageCount(size: number): number { + return Math.ceil(size / PAGE_SIZE); +} + +export function fileNameOf(path: string): string { + return path.startsWith('/') ? path.slice(1) : path; +} + +export function metaUrl(path: string): string { + return `https://bit-bsqlite.invalid/bit-bsqlite/db/${encodeURIComponent(path)}/meta`; +} + +export function pageUrl(path: string, index: number): string { + return `https://bit-bsqlite.invalid/bit-bsqlite/db/${encodeURIComponent(path)}/p/${index}`; +} + +export async function importBesqlIfPresent(fileName: string): Promise { + if (typeof caches === 'undefined' || !fileName) { + return null; + } + + try { + const cache = await caches.open(BESQL_CACHE); + const match = await cache.match(`/data/cache/${fileName}`); + if (!match) { + return null; + } + + return new Uint8Array(await match.arrayBuffer()); + } catch { + return null; + } +} + +function bytesResponse(bytes: Uint8Array): Response { + // Every page is a fresh Uint8Array, never a view of a SharedArrayBuffer. + return new Response(bytes as Uint8Array, { headers: { 'content-type': 'application/octet-stream' } }); +} + +function jsonResponse(value: unknown): Response { + return new Response(JSON.stringify(value), { headers: { 'content-type': 'application/json' } }); +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/package-lock.json b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/package-lock.json new file mode 100644 index 00000000000..dfb8b673143 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/package-lock.json @@ -0,0 +1,511 @@ +{ + "name": "Bit.Bsqlite.Storage.CacheStorage", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz", + "integrity": "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.2.tgz", + "integrity": "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.2.tgz", + "integrity": "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.2.tgz", + "integrity": "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.2.tgz", + "integrity": "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.2.tgz", + "integrity": "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.2.tgz", + "integrity": "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.2.tgz", + "integrity": "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.2.tgz", + "integrity": "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.2.tgz", + "integrity": "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.2.tgz", + "integrity": "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.2.tgz", + "integrity": "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.2.tgz", + "integrity": "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.2.tgz", + "integrity": "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.2.tgz", + "integrity": "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.2.tgz", + "integrity": "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.2.tgz", + "integrity": "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.2.tgz", + "integrity": "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.2.tgz", + "integrity": "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.2.tgz", + "integrity": "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.2.tgz", + "integrity": "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.2.tgz", + "integrity": "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.2.tgz", + "integrity": "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.2.tgz", + "integrity": "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.2.tgz", + "integrity": "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.2.tgz", + "integrity": "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/esbuild": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.2.tgz", + "integrity": "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.2", + "@esbuild/android-arm": "0.28.2", + "@esbuild/android-arm64": "0.28.2", + "@esbuild/android-x64": "0.28.2", + "@esbuild/darwin-arm64": "0.28.2", + "@esbuild/darwin-x64": "0.28.2", + "@esbuild/freebsd-arm64": "0.28.2", + "@esbuild/freebsd-x64": "0.28.2", + "@esbuild/linux-arm": "0.28.2", + "@esbuild/linux-arm64": "0.28.2", + "@esbuild/linux-ia32": "0.28.2", + "@esbuild/linux-loong64": "0.28.2", + "@esbuild/linux-mips64el": "0.28.2", + "@esbuild/linux-ppc64": "0.28.2", + "@esbuild/linux-riscv64": "0.28.2", + "@esbuild/linux-s390x": "0.28.2", + "@esbuild/linux-x64": "0.28.2", + "@esbuild/netbsd-arm64": "0.28.2", + "@esbuild/netbsd-x64": "0.28.2", + "@esbuild/openbsd-arm64": "0.28.2", + "@esbuild/openbsd-x64": "0.28.2", + "@esbuild/openharmony-arm64": "0.28.2", + "@esbuild/sunos-x64": "0.28.2", + "@esbuild/win32-arm64": "0.28.2", + "@esbuild/win32-ia32": "0.28.2", + "@esbuild/win32-x64": "0.28.2" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/package.json b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/package.json new file mode 100644 index 00000000000..48e65da82f6 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/package.json @@ -0,0 +1,6 @@ +{ + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/tsconfig.json b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/tsconfig.json new file mode 100644 index 00000000000..96b996a76aa --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.CacheStorage/tsconfig.json @@ -0,0 +1,25 @@ +{ + "compileOnSave": true, + "compilerOptions": { + // The same target as Bit.Bsqlite's modules, which these run next to in the same worker. + "target": "ES2022", + // The VFS module is imported by Bit.Bsqlite's worker and the admin module by the .NET provider, both + // through `import()`, and their functions are called as named exports, so the output MUST stay an ES + // module: `module: ESNext` with `rootDir`/`outDir` emits each Scripts/*.ts straight to wwwroot/*.js. + "module": "ESNext", + "moduleResolution": "Bundler", + // Type-only imports of Bit.Bsqlite's declarations for the wa-sqlite engine (Scripts/engine/*.d.ts). + // They are erased from the output, so nothing outside this package is compiled or bundled into it. + "paths": { + "bit-bsqlite/*": [ "../Bit.Bsqlite/Scripts/*" ] + }, + "lib": [ "DOM", "DOM.Iterable", "ESNext" ], + "rootDir": "Scripts", + "outDir": "wwwroot", + "strict": true, + "noFallthroughCasesInSwitch": true, + "noImplicitReturns": true, + "forceConsistentCasingInFileNames": true + }, + "include": [ "./Scripts/**/*.ts" ] +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj new file mode 100644 index 00000000000..017bf00e500 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Bit.Bsqlite.Storage.IndexedDb.csproj @@ -0,0 +1,84 @@ + + + + + + net10.0;net9.0;net8.0 + Bit.Bsqlite.Storage.IndexedDb + enable + + false + IndexedDB storage for bit Bsqlite, using wa-sqlite's IDBBatchAtomicVFS: persistent, async-capable (JSPI or Asyncify), and available where OPFS is not. + $(Description) + Bit;Bsqlite;Blazor;WebAssembly;SQLite;IndexedDB;Storage + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + <_BsqliteIndexedDbGeneratedJavaScript Include="wwwroot\bit-bsqlite-*.js" /> + + + + + + + + + + + + + + + + + + <_BsqliteIndexedDbStaleStamps Include="$(BaseIntermediateOutputPath)bit-bsqlite.*.stamp" /> + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/BsqliteIndexedDbStorageAdmin.cs b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/BsqliteIndexedDbStorageAdmin.cs new file mode 100644 index 00000000000..f2c80b011e5 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/BsqliteIndexedDbStorageAdmin.cs @@ -0,0 +1,103 @@ +using System.Diagnostics.CodeAnalysis; +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Storage.IndexedDb; + +/// +/// Database-level operations against the IndexedDB the VFS uses, for diagnostics and migration. +/// +/// +/// Talks to bit-bsqlite-idb-admin.js. Related files (-journal, -wal) are not databases +/// and are stripped on list, delete, and import. +/// +internal sealed class BsqliteIndexedDbStorageAdmin(IJSRuntime? js) : IBsqliteStorageAdmin +{ + public async ValueTask ExistsAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + return await InvokeAsync("exists", cancellationToken, databaseName).ConfigureAwait(false); + } + + public async ValueTask> ListAsync(CancellationToken cancellationToken = default) + { + var names = await InvokeAsync("list", cancellationToken).ConfigureAwait(false); + return names; + } + + public async ValueTask DeleteAsync(string databaseName, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + await InvokeAsync("deleteDatabase", cancellationToken, databaseName).ConfigureAwait(false); + } + + public async ValueTask ExportAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + var bytes = await InvokeAsync("exportDatabase", cancellationToken, databaseName) + .ConfigureAwait(false); + return bytes; + } + + public async ValueTask ImportAsync( + string databaseName, + ReadOnlyMemory contents, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + await InvokeAsync("importDatabase", cancellationToken, databaseName, contents.ToArray()) + .ConfigureAwait(false); + } + + private async ValueTask InvokeAsync<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.PublicFields | DynamicallyAccessedMemberTypes.PublicProperties)] T>( + string identifier, + CancellationToken cancellationToken, + params object?[] args) + { + var module = await ImportAsync(cancellationToken).ConfigureAwait(false); + try + { + return await module.InvokeAsync(identifier, cancellationToken, args).ConfigureAwait(false); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private async ValueTask InvokeAsync( + string identifier, + CancellationToken cancellationToken, + params object?[] args) + { + var module = await ImportAsync(cancellationToken).ConfigureAwait(false); + try + { + await module.InvokeVoidAsync(identifier, cancellationToken, args).ConfigureAwait(false); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private async ValueTask ImportAsync(CancellationToken cancellationToken) + { + if (js is null) + { + throw new InvalidOperationException( + "IndexedDB admin operations need a JavaScript runtime. They are not available in this process."); + } + + return await js + .InvokeAsync( + "import", + cancellationToken, + BsqliteIndexedDbStorageProvider.AdminModuleUrl) + .ConfigureAwait(false); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/BsqliteIndexedDbStorageProvider.cs b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/BsqliteIndexedDbStorageProvider.cs new file mode 100644 index 00000000000..c0e86cf3f72 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/BsqliteIndexedDbStorageProvider.cs @@ -0,0 +1,136 @@ +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Storage.IndexedDb; + +/// +/// Persists SQLite databases in IndexedDB via IDBBatchAtomicVFS. +/// +/// +/// +/// This is the backend that reaches browsers OPFS does not: it needs an async-capable engine +/// (JSPI, or Asyncify where JSPI is missing) and cannot change the page size. Those limits are +/// declared here and enforced generically by the core. +/// +/// +/// Without an the probe reports unavailable rather than throwing, so +/// desktop tests and selection can move on and still explain themselves. +/// +/// +public sealed class BsqliteIndexedDbStorageProvider : IBsqliteStorageProvider +{ + /// The name applications use to select this backend. + public const string ProviderName = "indexeddb"; + + /// The VFS name open_v2 must ask for after registration. + public const string VfsName = "idb-batch-atomic"; + + /// + /// The VFS module, named the way every Blazor static asset is named - relative to the + /// document - so an application served under a sub-path still finds it. The worker host + /// resolves it against the document base before the worker imports it. + /// + public const string VfsModuleUrl = "./_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-vfs.js"; + + /// The admin module, imported on the main thread relative to the document. + public const string AdminModuleUrl = "./_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-admin.js"; + + private readonly IJSRuntime? _js; + + public BsqliteIndexedDbStorageProvider(IJSRuntime? js = null) + { + _js = js; + Admin = new BsqliteIndexedDbStorageAdmin(js); + } + + /// + public string Name => ProviderName; + + /// + public BsqliteStorageCapabilities Capabilities { get; } = new() + { + RequiredBuild = BsqliteEngineBuild.AsyncCapable, + IsPersistent = true, + SupportsMultipleConnections = true, + + // False because of how bit-bsqlite-idb-vfs.js registers the VFS, not because IndexedDB could not do it: + // WebLocksMixin defaults to the 'exclusive' lock policy, under which a read takes the same + // exclusive Web Lock a write does and readers queue behind each other. Declaring otherwise + // would promise concurrency the registration does not provide. Passing + // `lockPolicy: 'shared'` is what would make this true, and is a change to durability- + // sensitive locking rather than a declaration. + SupportsConcurrentReads = false, + + SupportsRelaxedDurability = true, + SupportsMultiDatabaseTransactions = true, + CanChangePageSize = false, + SupportedContexts = + BsqliteExecutionContexts.DedicatedWorker + | BsqliteExecutionContexts.Window + | BsqliteExecutionContexts.SharedWorker + | BsqliteExecutionContexts.ServiceWorker, + }; + + /// + public BsqliteJsModule VfsModule { get; } = new(VfsModuleUrl); + + /// + public IBsqliteStorageAdmin Admin { get; } + + /// + public async ValueTask ProbeAsync( + CancellationToken cancellationToken = default) + { + if (_js is null) + { + return BsqliteProbeResult.Unavailable( + "IndexedDB can only be probed in a browser. This process has no JavaScript runtime.", + new Dictionary(StringComparer.Ordinal) + { + ["javascript"] = "false", + }); + } + + var module = await _js + .InvokeAsync("import", cancellationToken, AdminModuleUrl) + .ConfigureAwait(false); + + try + { + var report = await module + .InvokeAsync("probe", cancellationToken) + .ConfigureAwait(false); + + var environment = report.Environment ?? new Dictionary(StringComparer.Ordinal); + + if (!report.Available) + { + return BsqliteProbeResult.Unavailable( + report.Reason ?? "IndexedDB probe failed without a reason.", + environment); + } + + return BsqliteProbeResult.Available( + report.QuotaBytes, + report.UsageBytes, + environment); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private sealed class IndexedDbProbeReport + { + public bool Available { get; set; } + + public string? Reason { get; set; } + + public long? QuotaBytes { get; set; } + + public long? UsageBytes { get; set; } + + public Dictionary? Environment { get; set; } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/README.md b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/README.md new file mode 100644 index 00000000000..9b103dc9100 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/README.md @@ -0,0 +1,44 @@ +# Bit.Bsqlite.Storage.IndexedDb + +IndexedDB storage for [bit Bsqlite](https://www.nuget.org/packages/Bit.Bsqlite), backed by +wa-sqlite's `IDBBatchAtomicVFS`. + +This is the backend that reaches browsers OPFS does not, and the only one that can run outside a +dedicated worker. Use it as the fallback behind OPFS, or on its own when reach matters more than +download size. + +## Install + +``` +dotnet add package Bit.Bsqlite.Storage.IndexedDb +``` + +```csharp +new BsqliteIndexedDbStorageProvider(jsRuntime) // ProviderName == "indexeddb" +``` + +## Capabilities + +| | | +|---|---| +| Engine build | Async-capable — JSPI where available, Asyncify otherwise | +| Persistent | Yes | +| Multiple connections | Yes | +| Concurrent reads | No — see below | +| Relaxed durability | Yes | +| Page size changeable | No | +| Execution contexts | Dedicated worker, window, shared worker, service worker | + +The page-size limit is declared, not documented-and-hoped: the core reads `Capabilities` and rejects +a `PRAGMA page_size` that the backend cannot honour, rather than letting it be silently ignored. + +**On concurrent reads:** `IDBBatchAtomicVFS` can support them, but `bit-bsqlite-idb-vfs.js` registers it without +a lock policy, so `WebLocksMixin` applies its `exclusive` default and a read takes the same lock a +write does. Readers therefore queue behind each other, and the capability says so. Multiple +*connections* are still supported — they serialize rather than run in parallel. + +**On download size:** the engine build is picked at runtime from the selected provider plus a JSPI +probe. JSPI costs about 0.9% over the synchronous build; Asyncify costs about 107%. Browsers with +JSPI pay almost nothing for this backend. + +MIT. Part of [bit Bsqlite](https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite). diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/IDBBatchAtomicVFS.d.ts b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/IDBBatchAtomicVFS.d.ts new file mode 100644 index 00000000000..dba7d47be7e --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/IDBBatchAtomicVFS.d.ts @@ -0,0 +1,8 @@ +// Types for wa-sqlite's IDBBatchAtomicVFS (src/examples/IDBBatchAtomicVFS.js, wa-sqlite v1.1.2). The module +// is fetched and checksummed at restore and its imports rewritten into wwwroot/IDBBatchAtomicVFS.js (see +// vfs/FetchIndexedDbVfs.targets), next to the module that imports it; it is not compiled here. + +export declare class IDBBatchAtomicVFS { + static create(name: string, module: unknown, options?: { idbName?: string; lockPolicy?: 'exclusive' | 'shared' | 'shared+hint'; lockTimeout?: number }): Promise; + name: string; +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/bit-bsqlite-idb-admin.ts b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/bit-bsqlite-idb-admin.ts new file mode 100644 index 00000000000..ea8194aa9cd --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/bit-bsqlite-idb-admin.ts @@ -0,0 +1,312 @@ +// IndexedDB as a bag of SQLite file images, without the engine. +// +// The VFS stores each database as metadata + page-sized blocks under one origin-wide IndexedDB +// (`idb-batch-atomic`). Admin uses that same database and the same path mapping +// (`new URL(name, 'file://').pathname`) so an imported image is what `open_v2` will see. +// Related files (-journal, -wal) are not databases. + +export const IDB_NAME = 'idb-batch-atomic'; +export const IDB_VERSION = 6; +const RELATED_SUFFIXES = ['-journal', '-wal']; +const PAGE_SIZE = 4096; + +/** What `probe` reports to the .NET provider, which turns it into a BsqliteProbeResult. */ +interface ProbeResult { + available: boolean; + reason: string | null; + quotaBytes: number | null; + usageBytes: number | null; + environment: Record; +} + +/** + * A row of the VFS's `metadata` store: one per file. + * + * `version` counts down: each write transaction writes its blocks one below the committed version + * and lowers it on commit, so a block below `version` belongs to a transaction that never committed - + * one in flight in another tab, or one whose tab died, which `pendingVersion` marks until the VFS + * cleans it up on the next shared lock. + */ +interface MetadataRow { + name: string; + fileSize: number; + version: number; + pendingVersion?: number; +} + +/** A row of the VFS's `blocks` store. Offsets are stored negated, so a key range finds a block's start. */ +interface BlockRow { + path: string; + offset: number; + version: number; + data: Uint8Array; +} + +interface Stores { + metadata: IDBObjectStore; + blocks: IDBObjectStore; +} + +export function pathOf(databaseName: string): string { + return new URL(databaseName, 'file://').pathname; +} + +export function displayName(path: string): string { + return path.startsWith('/') ? path.slice(1) : path; +} + +export async function probe(): Promise { + const environment = { + hasIndexedDB: String(typeof indexedDB !== 'undefined'), + secureContext: String(Boolean(globalThis.isSecureContext)), + supportsJspi: String(typeof (WebAssembly as { Suspending?: unknown })?.Suspending === 'function'), + }; + + if (typeof indexedDB === 'undefined') { + return unavailable( + 'This browser does not expose indexedDB, which this backend needs.', + environment); + } + + try { + // Opening and immediately closing proves the API is actually usable, not merely present. + const db = await openDatabase(); + db.close(); + + let quotaBytes: number | null = null; + let usageBytes: number | null = null; + if (typeof navigator.storage?.estimate === 'function') { + const estimate = await navigator.storage.estimate(); + quotaBytes = estimate.quota ?? null; + usageBytes = estimate.usage ?? null; + } + + return { + available: true, + reason: null, + quotaBytes, + usageBytes, + environment, + }; + } catch (error) { + return unavailable((error as Error | undefined)?.message ?? String(error), environment); + } +} + +export async function exists(databaseName: string): Promise { + const path = pathOf(requireName(databaseName)); + const db = await openDatabase(); + + try { + const meta = await withStore(db, 'metadata', 'readonly', store => requestOf(store.get(path))); + return Boolean(meta); + } finally { + db.close(); + } +} + +export async function list(): Promise { + const db = await openDatabase(); + + try { + const rows = await withStore(db, 'metadata', 'readonly', store => requestOf(store.getAll())); + return rows + .map(row => row.name) + .filter(name => !RELATED_SUFFIXES.some(suffix => name.endsWith(suffix))) + .map(displayName) + .sort(); + } finally { + db.close(); + } +} + +export async function deleteDatabase(databaseName: string): Promise { + const name = requireName(databaseName); + const db = await openDatabase(); + + try { + await withTransaction(db, 'readwrite', async ({ metadata, blocks }) => { + for (const suffix of ['', ...RELATED_SUFFIXES]) { + const path = pathOf(name + suffix); + metadata.delete(path); + blocks.delete(IDBKeyRange.bound([path, -Infinity], [path, Infinity])); + } + }); + } finally { + db.close(); + } +} + +/** + * Reads out the committed database, as of one moment. + * + * One read-only transaction covers the metadata and every block, so a commit in another tab cannot + * land between them. Blocks below the committed version are skipped (see MetadataRow): exporting + * them would mix a transaction that never committed into an image that still passes for a SQLite + * file - and a migration would bind the user's database to it. + */ +export async function exportDatabase(databaseName: string): Promise { + const path = pathOf(requireName(databaseName)); + const db = await openDatabase(); + + try { + return await withTransaction(db, 'readonly', async ({ metadata, blocks }) => { + const meta = await requestOf(metadata.get(path)); + if (!meta) { + throw new Error(`IndexedDB holds no database named '${displayName(path)}'.`); + } + + return readCommittedImage(blocks, path, meta); + }); + } finally { + db.close(); + } +} + +/** + * Walks the blocks the way the VFS reads them - for each offset, the newest block that starts at or + * below it - passing over any whose version is below the committed one. + */ +function readCommittedImage(blocks: IDBObjectStore, path: string, meta: MetadataRow): Promise { + const image = new Uint8Array(meta.fileSize ?? 0); + + return new Promise((resolve, reject) => { + let offset = 0; + + const seek = () => { + if (offset >= image.byteLength) { + resolve(image); + return; + } + + // Callbacks rather than awaits, so each request is issued while the transaction is + // certainly still active. + const request = blocks.openCursor(IDBKeyRange.bound([path, -offset], [path, Infinity])); + request.onerror = () => reject(request.error); + request.onsuccess = () => { + const cursor = request.result; + if (!cursor) { + resolve(image); + return; + } + + const block = cursor.value as BlockRow; + if (block.version < meta.version) { + cursor.continue(); + return; + } + + if (block.data.byteLength - block.offset <= offset) { + resolve(image); + return; + } + + const srcOffset = offset + block.offset; + const n = Math.min(block.data.byteLength - srcOffset, image.byteLength - offset); + image.set(block.data.subarray(srcOffset, srcOffset + n), offset); + offset += n; + seek(); + }; + }; + + seek(); + }); +} + +export async function importDatabase(databaseName: string, contents: Uint8Array | ArrayLike | null): Promise { + const name = requireName(databaseName); + const path = pathOf(name); + const bytes = contents instanceof Uint8Array ? contents : new Uint8Array(contents ?? []); + const db = await openDatabase(); + + try { + await withTransaction(db, 'readwrite', async ({ metadata, blocks }) => { + for (const suffix of ['', ...RELATED_SUFFIXES]) { + const related = pathOf(name + suffix); + metadata.delete(related); + blocks.delete(IDBKeyRange.bound([related, -Infinity], [related, Infinity])); + } + + metadata.put({ name: path, fileSize: bytes.byteLength, version: 0 }); + + for (let offset = 0; offset < bytes.byteLength; offset += PAGE_SIZE) { + const data = bytes.subarray(offset, offset + PAGE_SIZE).slice(); + blocks.put({ path, offset: -offset, version: 0, data }); + } + }); + } finally { + db.close(); + } +} + +function unavailable(reason: string, environment: Record): ProbeResult { + return { available: false, reason, quotaBytes: null, usageBytes: null, environment }; +} + +function requireName(databaseName: unknown): string { + if (typeof databaseName !== 'string' || databaseName.trim() === '') { + throw new Error('A database name is required.'); + } + + const name = databaseName.trim(); + if (name.includes('..') || name.startsWith('\\')) { + throw new Error(`'${name}' is not a valid IndexedDB database name.`); + } + + return name; +} + +function openDatabase(): Promise { + return new Promise((resolve, reject) => { + const request = indexedDB.open(IDB_NAME, IDB_VERSION); + + request.onupgradeneeded = () => { + const db = request.result; + if (!db.objectStoreNames.contains('blocks')) { + db.createObjectStore('blocks', { keyPath: ['path', 'offset', 'version'] }); + } + + if (!db.objectStoreNames.contains('metadata')) { + db.createObjectStore('metadata', { keyPath: 'name' }); + } + }; + + request.onsuccess = () => resolve(request.result); + request.onerror = () => reject(request.error); + }); +} + +function withStore(db: IDBDatabase, storeName: keyof Stores, mode: IDBTransactionMode, work: (store: IDBObjectStore) => T | Promise): Promise { + return withTransaction(db, mode, stores => work(stores[storeName])); +} + +function withTransaction(db: IDBDatabase, mode: IDBTransactionMode, work: (stores: Stores) => T | Promise): Promise { + return new Promise((resolve, reject) => { + const tx = db.transaction(['metadata', 'blocks'], mode); + const stores: Stores = { + metadata: tx.objectStore('metadata'), + blocks: tx.objectStore('blocks'), + }; + + // Handlers first: a transaction of only synchronous puts can complete before a then() runs. + let result: T; + tx.oncomplete = () => resolve(result); + tx.onerror = () => reject(tx.error); + tx.onabort = () => reject(tx.error ?? new Error('IndexedDB transaction aborted.')); + + try { + Promise.resolve(work(stores)).then(value => { + result = value; + }, reject); + } catch (error) { + reject(error); + } + }); +} + +function requestOf(request: IDBRequest): Promise { + return new Promise((resolve, reject) => { + request.onsuccess = () => resolve(request.result); + request.onerror = () => reject(request.error); + }); +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/bit-bsqlite-idb-vfs.ts b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/bit-bsqlite-idb-vfs.ts new file mode 100644 index 00000000000..0a38c27ceed --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/Scripts/bit-bsqlite-idb-vfs.ts @@ -0,0 +1,24 @@ +// Registers wa-sqlite's IDBBatchAtomicVFS against the engine the worker just loaded. +// +// This VFS suspends on every I/O, so it cannot run on the synchronous build - the provider +// declares AsyncCapable and the loader picks JSPI or Asyncify. cache_size is set on open so +// batch-atomic writes can keep the journal in cache, which is the main IndexedDB performance win. + +import { IDBBatchAtomicVFS } from './IDBBatchAtomicVFS.js'; +import type { SQLiteAPI, SQLiteModule } from 'bit-bsqlite/engine/sqlite-api.js'; + +export const VFS_NAME = 'idb-batch-atomic'; +export const IDB_NAME = 'idb-batch-atomic'; + +/** Called by the worker with the Emscripten module and the wa-sqlite JavaScript API it just loaded. */ +export async function register({ module, sqlite3 }: { module: SQLiteModule; sqlite3: SQLiteAPI }): Promise<{ vfsName: string; onOpenSql: string[] }> { + const vfs = await IDBBatchAtomicVFS.create(VFS_NAME, module, { idbName: IDB_NAME }); + sqlite3.vfs_register(vfs, /* makeDefault */ false); + return { + vfsName: VFS_NAME, + // Negative cache_size is kibibytes. 8 MiB holds a typical EF SaveChanges working set so + // SQLITE_IOCAP_BATCH_ATOMIC can skip the external journal. The VFS still works if a huge + // transaction overflows it; it just pays the journal path. + onOpenSql: ['PRAGMA cache_size=-8000'], + }; +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/package-lock.json b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/package-lock.json new file mode 100644 index 00000000000..b26a2ec738d --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/package-lock.json @@ -0,0 +1,511 @@ +{ + "name": "Bit.Bsqlite.Storage.IndexedDb", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz", + "integrity": "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.2.tgz", + "integrity": "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.2.tgz", + "integrity": "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.2.tgz", + "integrity": "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.2.tgz", + "integrity": "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.2.tgz", + "integrity": "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.2.tgz", + "integrity": "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.2.tgz", + "integrity": "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.2.tgz", + "integrity": "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.2.tgz", + "integrity": "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.2.tgz", + "integrity": "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.2.tgz", + "integrity": "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.2.tgz", + "integrity": "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.2.tgz", + "integrity": "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.2.tgz", + "integrity": "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.2.tgz", + "integrity": "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.2.tgz", + "integrity": "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.2.tgz", + "integrity": "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.2.tgz", + "integrity": "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.2.tgz", + "integrity": "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.2.tgz", + "integrity": "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.2.tgz", + "integrity": "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.2.tgz", + "integrity": "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.2.tgz", + "integrity": "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.2.tgz", + "integrity": "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.2.tgz", + "integrity": "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/esbuild": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.2.tgz", + "integrity": "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.2", + "@esbuild/android-arm": "0.28.2", + "@esbuild/android-arm64": "0.28.2", + "@esbuild/android-x64": "0.28.2", + "@esbuild/darwin-arm64": "0.28.2", + "@esbuild/darwin-x64": "0.28.2", + "@esbuild/freebsd-arm64": "0.28.2", + "@esbuild/freebsd-x64": "0.28.2", + "@esbuild/linux-arm": "0.28.2", + "@esbuild/linux-arm64": "0.28.2", + "@esbuild/linux-ia32": "0.28.2", + "@esbuild/linux-loong64": "0.28.2", + "@esbuild/linux-mips64el": "0.28.2", + "@esbuild/linux-ppc64": "0.28.2", + "@esbuild/linux-riscv64": "0.28.2", + "@esbuild/linux-s390x": "0.28.2", + "@esbuild/linux-x64": "0.28.2", + "@esbuild/netbsd-arm64": "0.28.2", + "@esbuild/netbsd-x64": "0.28.2", + "@esbuild/openbsd-arm64": "0.28.2", + "@esbuild/openbsd-x64": "0.28.2", + "@esbuild/openharmony-arm64": "0.28.2", + "@esbuild/sunos-x64": "0.28.2", + "@esbuild/win32-arm64": "0.28.2", + "@esbuild/win32-ia32": "0.28.2", + "@esbuild/win32-x64": "0.28.2" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/package.json b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/package.json new file mode 100644 index 00000000000..48e65da82f6 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/package.json @@ -0,0 +1,6 @@ +{ + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/tsconfig.json b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/tsconfig.json new file mode 100644 index 00000000000..96b996a76aa --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/tsconfig.json @@ -0,0 +1,25 @@ +{ + "compileOnSave": true, + "compilerOptions": { + // The same target as Bit.Bsqlite's modules, which these run next to in the same worker. + "target": "ES2022", + // The VFS module is imported by Bit.Bsqlite's worker and the admin module by the .NET provider, both + // through `import()`, and their functions are called as named exports, so the output MUST stay an ES + // module: `module: ESNext` with `rootDir`/`outDir` emits each Scripts/*.ts straight to wwwroot/*.js. + "module": "ESNext", + "moduleResolution": "Bundler", + // Type-only imports of Bit.Bsqlite's declarations for the wa-sqlite engine (Scripts/engine/*.d.ts). + // They are erased from the output, so nothing outside this package is compiled or bundled into it. + "paths": { + "bit-bsqlite/*": [ "../Bit.Bsqlite/Scripts/*" ] + }, + "lib": [ "DOM", "DOM.Iterable", "ESNext" ], + "rootDir": "Scripts", + "outDir": "wwwroot", + "strict": true, + "noFallthroughCasesInSwitch": true, + "noImplicitReturns": true, + "forceConsistentCasingInFileNames": true + }, + "include": [ "./Scripts/**/*.ts" ] +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/FetchIndexedDbVfs.targets b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/FetchIndexedDbVfs.targets new file mode 100644 index 00000000000..6b8685f3b33 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/FetchIndexedDbVfs.targets @@ -0,0 +1,44 @@ + + + + + + $(MSBuildProjectDirectory)/obj/upstream/ + $(MSBuildProjectDirectory)/wwwroot/ + $(IndexedDbVfsUpstreamDirectory)IDBBatchAtomicVFS.js + $(IndexedDbVfsWwwrootDirectory)IDBBatchAtomicVFS.js + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/rewrite-imports.mjs b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/rewrite-imports.mjs new file mode 100644 index 00000000000..d01ea4f9d70 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/rewrite-imports.mjs @@ -0,0 +1,30 @@ +// Rewrites wa-sqlite example imports so they resolve from a provider package. +// +// IDBBatchAtomicVFS also pulls WebLocksMixin, which OPFS CoopSync does not. The rewrite is +// mechanical and checksummed against the untouched download. + +import { readFileSync, writeFileSync } from 'node:fs'; + +const [, , input, output] = process.argv; + +if (!input || !output) { + throw new Error('Usage: rewrite-imports.mjs '); +} + +const replacements = [ + ["from '../FacadeVFS.js'", "from '../Bit.Bsqlite/engine/FacadeVFS.js'"], + ["from '../VFS.js'", "from '../Bit.Bsqlite/engine/VFS.js'"], + ["from '../WebLocksMixin.js'", "from '../Bit.Bsqlite/engine/WebLocksMixin.js'"], +]; + +let source = readFileSync(input, 'utf8'); + +for (const [from, to] of replacements) { + if (!source.includes(from)) { + throw new Error(`Expected import ${from} was not in the upstream VFS. The pin may have moved.`); + } + + source = source.replaceAll(from, to); +} + +writeFileSync(output, source); diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/wa-sqlite.lock.props b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/wa-sqlite.lock.props new file mode 100644 index 00000000000..554b677011c --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.IndexedDb/vfs/wa-sqlite.lock.props @@ -0,0 +1,15 @@ + + + + + + v1.1.2 + https://raw.githubusercontent.com/rhashimoto/wa-sqlite + src/examples/IDBBatchAtomicVFS.js + 809018E8976A8001BFAC6062B1FAA2A75333BD004885C8918F926C9E596E9CC6 + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj new file mode 100644 index 00000000000..aaca8c1c55f --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Bit.Bsqlite.Storage.Opfs.csproj @@ -0,0 +1,84 @@ + + + + + + net10.0;net9.0;net8.0 + Bit.Bsqlite.Storage.Opfs + enable + + false + OPFS storage for bit Bsqlite, using wa-sqlite's OPFSCoopSyncVFS: persistent, multi-connection, worker-only and without COOP/COEP headers. + $(Description) + Bit;Bsqlite;Blazor;WebAssembly;SQLite;OPFS;Origin private file system;Storage + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + <_BsqliteOpfsGeneratedJavaScript Include="wwwroot\bit-bsqlite-*.js" /> + + + + + + + + + + + + + + + + + + <_BsqliteOpfsStaleStamps Include="$(BaseIntermediateOutputPath)bit-bsqlite.*.stamp" /> + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/BsqliteOpfsStorageAdmin.cs b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/BsqliteOpfsStorageAdmin.cs new file mode 100644 index 00000000000..5ceabb74de7 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/BsqliteOpfsStorageAdmin.cs @@ -0,0 +1,100 @@ +using System.Diagnostics.CodeAnalysis; +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Storage.Opfs; + +/// +/// Database-level operations against OPFS, used by diagnostics and cross-provider migration. +/// +/// +/// Talks to bit-bsqlite-opfs-admin.js. Related files (-journal, -wal) are not databases +/// and are stripped on list, delete, and import. +/// +internal sealed class BsqliteOpfsStorageAdmin(IJSRuntime? js) : IBsqliteStorageAdmin +{ + public async ValueTask ExistsAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + return await InvokeAsync("exists", cancellationToken, databaseName).ConfigureAwait(false); + } + + public async ValueTask> ListAsync(CancellationToken cancellationToken = default) + { + var names = await InvokeAsync("list", cancellationToken).ConfigureAwait(false); + return names; + } + + public async ValueTask DeleteAsync(string databaseName, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + await InvokeAsync("deleteDatabase", cancellationToken, databaseName).ConfigureAwait(false); + } + + public async ValueTask ExportAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + var bytes = await InvokeAsync("exportDatabase", cancellationToken, databaseName) + .ConfigureAwait(false); + return bytes; + } + + public async ValueTask ImportAsync( + string databaseName, + ReadOnlyMemory contents, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + await InvokeAsync("importDatabase", cancellationToken, databaseName, contents.ToArray()) + .ConfigureAwait(false); + } + + private async ValueTask InvokeAsync<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.PublicFields | DynamicallyAccessedMemberTypes.PublicProperties)] T>( + string identifier, + CancellationToken cancellationToken, + params object?[] args) + { + var module = await ImportAsync(cancellationToken).ConfigureAwait(false); + try + { + return await module.InvokeAsync(identifier, cancellationToken, args).ConfigureAwait(false); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private async ValueTask InvokeAsync( + string identifier, + CancellationToken cancellationToken, + params object?[] args) + { + var module = await ImportAsync(cancellationToken).ConfigureAwait(false); + try + { + await module.InvokeVoidAsync(identifier, cancellationToken, args).ConfigureAwait(false); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private async ValueTask ImportAsync(CancellationToken cancellationToken) + { + if (js is null) + { + throw new InvalidOperationException( + "OPFS admin operations need a JavaScript runtime. They are not available in this process."); + } + + return await js + .InvokeAsync("import", cancellationToken, BsqliteOpfsStorageProvider.AdminModuleUrl) + .ConfigureAwait(false); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/BsqliteOpfsStorageProvider.cs b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/BsqliteOpfsStorageProvider.cs new file mode 100644 index 00000000000..a889084c002 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/BsqliteOpfsStorageProvider.cs @@ -0,0 +1,124 @@ +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Storage.Opfs; + +/// +/// Persists SQLite databases in the origin's private file system via OPFSCoopSyncVFS. +/// +/// +/// +/// Chosen over AccessHandlePoolVFS because more than one connection may be open and the +/// .db is a real file - export is a read. Access handles are worker-only, so the engine +/// stays in a dedicated worker; probe and admin use the async file API and can run on the window. +/// +/// +/// Without an the probe reports unavailable rather than throwing, so +/// desktop tests and selection can move on and still explain themselves. +/// +/// +public sealed class BsqliteOpfsStorageProvider : IBsqliteStorageProvider +{ + /// The name applications use to select this backend. + public const string ProviderName = "opfs"; + + /// The VFS name open_v2 must ask for after registration. + public const string VfsName = "opfs-coop-sync"; + + /// + /// The VFS module, named the way every Blazor static asset is named - relative to the + /// document - so an application served under a sub-path still finds it. The worker host + /// resolves it against the document base before the worker imports it. + /// + public const string VfsModuleUrl = "./_content/Bit.Bsqlite.Storage.Opfs/bit-bsqlite-opfs-vfs.js"; + + /// The admin module, imported on the main thread relative to the document. + public const string AdminModuleUrl = "./_content/Bit.Bsqlite.Storage.Opfs/bit-bsqlite-opfs-admin.js"; + + private readonly IJSRuntime? _js; + + public BsqliteOpfsStorageProvider(IJSRuntime? js = null) + { + _js = js; + Admin = new BsqliteOpfsStorageAdmin(js); + } + + /// + public string Name => ProviderName; + + /// + public BsqliteStorageCapabilities Capabilities { get; } = new() + { + RequiredBuild = BsqliteEngineBuild.Synchronous, + IsPersistent = true, + SupportsMultipleConnections = true, + SupportsConcurrentReads = false, + SupportsRelaxedDurability = false, + SupportsMultiDatabaseTransactions = false, + CanChangePageSize = true, + SupportedContexts = BsqliteExecutionContexts.DedicatedWorker, + }; + + /// + public BsqliteJsModule VfsModule { get; } = new(VfsModuleUrl); + + /// + public IBsqliteStorageAdmin Admin { get; } + + /// + public async ValueTask ProbeAsync( + CancellationToken cancellationToken = default) + { + if (_js is null) + { + return BsqliteProbeResult.Unavailable( + "OPFS can only be probed in a browser. This process has no JavaScript runtime.", + new Dictionary(StringComparer.Ordinal) + { + ["javascript"] = "false", + }); + } + + var module = await _js + .InvokeAsync("import", cancellationToken, AdminModuleUrl) + .ConfigureAwait(false); + + try + { + var report = await module + .InvokeAsync("probe", cancellationToken) + .ConfigureAwait(false); + + var environment = report.Environment ?? new Dictionary(StringComparer.Ordinal); + + if (!report.Available) + { + return BsqliteProbeResult.Unavailable( + report.Reason ?? "OPFS probe failed without a reason.", + environment); + } + + return BsqliteProbeResult.Available( + report.QuotaBytes, + report.UsageBytes, + environment); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } + + private sealed class OpfsProbeReport + { + public bool Available { get; set; } + + public string? Reason { get; set; } + + public long? QuotaBytes { get; set; } + + public long? UsageBytes { get; set; } + + public Dictionary? Environment { get; set; } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/README.md b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/README.md new file mode 100644 index 00000000000..3a94c46b776 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/README.md @@ -0,0 +1,38 @@ +# Bit.Bsqlite.Storage.Opfs + +OPFS storage for [bit Bsqlite](https://www.nuget.org/packages/Bit.Bsqlite), backed by wa-sqlite's +`OPFSCoopSyncVFS`. Databases live in the origin private file system as real files. + +Pick this one when you can: it is the only backend that runs on the **synchronous** engine build, +which is the smallest of the three and needs neither JSPI nor Asyncify. + +## Install + +``` +dotnet add package Bit.Bsqlite.Storage.Opfs +``` + +```csharp +new BsqliteOpfsStorageProvider(jsRuntime) // ProviderName == "opfs" +``` + +## Capabilities + +| | | +|---|---| +| Engine build | Synchronous — the smallest download | +| Persistent | Yes | +| Multiple connections | Yes | +| Concurrent reads | No | +| Relaxed durability | No | +| Page size changeable | Yes | +| Execution contexts | Dedicated worker only | + +Access handles are worker-only, so the engine stays in a dedicated worker. Probe and admin use the +async file API and can run on the window, so availability checks and import/export work from +component code. No COOP/COEP headers required. + +Chosen over `AccessHandlePoolVFS` because more than one connection may be open and the `.db` stays a +real file — exporting is just a read. + +MIT. Part of [bit Bsqlite](https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite). diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/OPFSCoopSyncVFS.d.ts b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/OPFSCoopSyncVFS.d.ts new file mode 100644 index 00000000000..5f0f473e7dc --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/OPFSCoopSyncVFS.d.ts @@ -0,0 +1,8 @@ +// Types for wa-sqlite's OPFSCoopSyncVFS (src/examples/OPFSCoopSyncVFS.js, wa-sqlite v1.1.2). The module is +// fetched and checksummed at restore and its imports rewritten into wwwroot/OPFSCoopSyncVFS.js (see +// vfs/FetchOpfsVfs.targets), next to the module that imports it; it is not compiled here. + +export declare class OPFSCoopSyncVFS { + static create(name: string, module: unknown): Promise; + name: string; +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/bit-bsqlite-opfs-admin.ts b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/bit-bsqlite-opfs-admin.ts new file mode 100644 index 00000000000..7ebc1821d55 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/bit-bsqlite-opfs-admin.ts @@ -0,0 +1,213 @@ +// OPFS as a bag of files, without the engine. +// +// Probe and admin run on the window: getDirectory is available there. Sync access handles are +// not - those exist only in the worker, which is why the VFS is worker-only and this module is not. +// Related files (-journal, -wal) and the VFS's temporary .ahp-* directories are not databases. + +const RELATED_SUFFIXES = ['-journal', '-wal']; +const TEMP_DIR_PREFIX = '.ahp-'; + +/** What `probe` reports to the .NET provider, which turns it into a BsqliteProbeResult. */ +interface ProbeResult { + available: boolean; + reason: string | null; + quotaBytes: number | null; + usageBytes: number | null; + environment: Record; +} + +// Neither is in TypeScript's DOM typings: one exists only in workers, the other is not standard yet. +declare const DedicatedWorkerGlobalScope: { new(): unknown } | undefined; + +interface IterableDirectoryHandle extends FileSystemDirectoryHandle { + entries(): AsyncIterable<[string, FileSystemHandle]>; +} + +export async function probe(): Promise { + const environment = { + secureContext: String(Boolean(globalThis.isSecureContext)), + hasGetDirectory: String(typeof navigator.storage?.getDirectory === 'function'), + hasCreateSyncAccessHandle: String( + typeof (FileSystemFileHandle?.prototype as { createSyncAccessHandle?: unknown } | undefined)?.createSyncAccessHandle === 'function'), + isDedicatedWorker: String(typeof DedicatedWorkerGlobalScope !== 'undefined' + && globalThis instanceof (DedicatedWorkerGlobalScope as { new(): unknown })), + }; + + if (!globalThis.isSecureContext) { + return unavailable('OPFS requires a secure context (HTTPS or localhost).', environment); + } + + if (typeof navigator.storage?.getDirectory !== 'function') { + return unavailable( + 'This browser does not expose navigator.storage.getDirectory, which OPFS needs.', + environment); + } + + try { + const root = await navigator.storage.getDirectory(); + if (!root) { + return unavailable('navigator.storage.getDirectory() returned nothing.', environment); + } + + let quotaBytes: number | null = null; + let usageBytes: number | null = null; + if (typeof navigator.storage.estimate === 'function') { + const estimate = await navigator.storage.estimate(); + quotaBytes = estimate.quota ?? null; + usageBytes = estimate.usage ?? null; + } + + return { + available: true, + reason: null, + quotaBytes, + usageBytes, + environment, + }; + } catch (error) { + return unavailable((error as Error | undefined)?.message ?? String(error), environment); + } +} + +export async function exists(databaseName: string): Promise { + const name = requireName(databaseName); + + try { + await getFileHandle(name, { create: false }); + return true; + } catch (error) { + if (isNotFound(error)) { + return false; + } + + throw error; + } +} + +export async function list(): Promise { + const root = await navigator.storage.getDirectory(); + const names: string[] = []; + await collect(root, '', names); + names.sort(); + return names; +} + +export async function deleteDatabase(databaseName: string): Promise { + const name = requireName(databaseName); + const { directory, fileName } = await directoryFor(name, { create: false }).catch(error => { + if (isNotFound(error)) { + return { directory: null, fileName: null }; + } + + throw error; + }); + + if (!directory || !fileName) { + return; + } + + for (const suffix of ['', ...RELATED_SUFFIXES]) { + try { + await directory.removeEntry(fileName + suffix); + } catch (error) { + if (!isNotFound(error)) { + throw error; + } + } + } +} + +export async function exportDatabase(databaseName: string): Promise { + const name = requireName(databaseName); + + try { + const handle = await getFileHandle(name, { create: false }); + const file = await handle.getFile(); + return new Uint8Array(await file.arrayBuffer()); + } catch (error) { + if (isNotFound(error)) { + throw new Error(`OPFS holds no database named '${name}'.`); + } + + throw error; + } +} + +export async function importDatabase(databaseName: string, contents: Uint8Array | ArrayLike | null): Promise { + const name = requireName(databaseName); + const bytes = contents instanceof Uint8Array ? contents : new Uint8Array(contents ?? []); + const handle = await getFileHandle(name, { create: true }); + const writable = await handle.createWritable(); + // Never a SharedArrayBuffer view: the bytes come from .NET interop or a fresh copy. + await writable.write(bytes as Uint8Array); + await writable.close(); + + const { directory, fileName } = await directoryFor(name, { create: false }); + for (const suffix of RELATED_SUFFIXES) { + try { + await directory.removeEntry(fileName + suffix); + } catch (error) { + if (!isNotFound(error)) { + throw error; + } + } + } +} + +function unavailable(reason: string, environment: Record): ProbeResult { + return { available: false, reason, quotaBytes: null, usageBytes: null, environment }; +} + +function requireName(databaseName: unknown): string { + if (typeof databaseName !== 'string' || databaseName.trim() === '') { + throw new Error('A database name is required.'); + } + + const name = databaseName.trim(); + if (name.includes('..') || name.startsWith('/') || name.startsWith('\\')) { + throw new Error(`'${name}' is not a valid OPFS database name.`); + } + + return name; +} + +async function getFileHandle(databaseName: string, { create }: { create: boolean }): Promise { + const { directory, fileName } = await directoryFor(databaseName, { create }); + return await directory.getFileHandle(fileName, { create }); +} + +async function directoryFor(databaseName: string, { create }: { create: boolean }): Promise<{ directory: FileSystemDirectoryHandle; fileName: string }> { + const parts = databaseName.split('/').filter(Boolean); + const fileName = parts.pop()!; + let directory = await navigator.storage.getDirectory(); + + for (const part of parts) { + directory = await directory.getDirectoryHandle(part, { create }); + } + + return { directory, fileName }; +} + +async function collect(directory: FileSystemDirectoryHandle, prefix: string, names: string[]): Promise { + for await (const [name, handle] of (directory as IterableDirectoryHandle).entries()) { + if (handle.kind === 'directory') { + if (name.startsWith(TEMP_DIR_PREFIX)) { + continue; + } + + await collect(handle as FileSystemDirectoryHandle, prefix ? `${prefix}/${name}` : name, names); + continue; + } + + if (RELATED_SUFFIXES.some(suffix => name.endsWith(suffix))) { + continue; + } + + names.push(prefix ? `${prefix}/${name}` : name); + } +} + +function isNotFound(error: unknown): boolean { + const name = (error as Error | undefined)?.name; + return name === 'NotFoundError' || name === 'TypeError'; +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/bit-bsqlite-opfs-vfs.ts b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/bit-bsqlite-opfs-vfs.ts new file mode 100644 index 00000000000..85c2b694550 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/Scripts/bit-bsqlite-opfs-vfs.ts @@ -0,0 +1,17 @@ +// Registers wa-sqlite's OPFSCoopSyncVFS against the engine the worker just loaded. +// +// Chosen over AccessHandlePoolVFS because it allows more than one connection and leaves a real +// inspectable file in OPFS - export is a read, not a reconstruction. The VFS itself is vendored +// and import-rewritten; this file is the stable register() the .NET provider points at. + +import { OPFSCoopSyncVFS } from './OPFSCoopSyncVFS.js'; +import type { SQLiteAPI, SQLiteModule } from 'bit-bsqlite/engine/sqlite-api.js'; + +export const VFS_NAME = 'opfs-coop-sync'; + +/** Called by the worker with the Emscripten module and the wa-sqlite JavaScript API it just loaded. */ +export async function register({ module, sqlite3 }: { module: SQLiteModule; sqlite3: SQLiteAPI }): Promise<{ vfsName: string }> { + const vfs = await OPFSCoopSyncVFS.create(VFS_NAME, module); + sqlite3.vfs_register(vfs, /* makeDefault */ false); + return { vfsName: VFS_NAME }; +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/package-lock.json b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/package-lock.json new file mode 100644 index 00000000000..35601ddc2ae --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/package-lock.json @@ -0,0 +1,511 @@ +{ + "name": "Bit.Bsqlite.Storage.Opfs", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz", + "integrity": "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.2.tgz", + "integrity": "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.2.tgz", + "integrity": "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.2.tgz", + "integrity": "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.2.tgz", + "integrity": "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.2.tgz", + "integrity": "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.2.tgz", + "integrity": "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.2.tgz", + "integrity": "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.2.tgz", + "integrity": "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.2.tgz", + "integrity": "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.2.tgz", + "integrity": "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.2.tgz", + "integrity": "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.2.tgz", + "integrity": "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.2.tgz", + "integrity": "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.2.tgz", + "integrity": "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.2.tgz", + "integrity": "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.2.tgz", + "integrity": "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.2.tgz", + "integrity": "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.2.tgz", + "integrity": "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.2.tgz", + "integrity": "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.2.tgz", + "integrity": "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.2.tgz", + "integrity": "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.2.tgz", + "integrity": "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.2.tgz", + "integrity": "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.2.tgz", + "integrity": "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.2.tgz", + "integrity": "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/esbuild": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.2.tgz", + "integrity": "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.2", + "@esbuild/android-arm": "0.28.2", + "@esbuild/android-arm64": "0.28.2", + "@esbuild/android-x64": "0.28.2", + "@esbuild/darwin-arm64": "0.28.2", + "@esbuild/darwin-x64": "0.28.2", + "@esbuild/freebsd-arm64": "0.28.2", + "@esbuild/freebsd-x64": "0.28.2", + "@esbuild/linux-arm": "0.28.2", + "@esbuild/linux-arm64": "0.28.2", + "@esbuild/linux-ia32": "0.28.2", + "@esbuild/linux-loong64": "0.28.2", + "@esbuild/linux-mips64el": "0.28.2", + "@esbuild/linux-ppc64": "0.28.2", + "@esbuild/linux-riscv64": "0.28.2", + "@esbuild/linux-s390x": "0.28.2", + "@esbuild/linux-x64": "0.28.2", + "@esbuild/netbsd-arm64": "0.28.2", + "@esbuild/netbsd-x64": "0.28.2", + "@esbuild/openbsd-arm64": "0.28.2", + "@esbuild/openbsd-x64": "0.28.2", + "@esbuild/openharmony-arm64": "0.28.2", + "@esbuild/sunos-x64": "0.28.2", + "@esbuild/win32-arm64": "0.28.2", + "@esbuild/win32-ia32": "0.28.2", + "@esbuild/win32-x64": "0.28.2" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/package.json b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/package.json new file mode 100644 index 00000000000..48e65da82f6 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/package.json @@ -0,0 +1,6 @@ +{ + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/tsconfig.json b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/tsconfig.json new file mode 100644 index 00000000000..96b996a76aa --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/tsconfig.json @@ -0,0 +1,25 @@ +{ + "compileOnSave": true, + "compilerOptions": { + // The same target as Bit.Bsqlite's modules, which these run next to in the same worker. + "target": "ES2022", + // The VFS module is imported by Bit.Bsqlite's worker and the admin module by the .NET provider, both + // through `import()`, and their functions are called as named exports, so the output MUST stay an ES + // module: `module: ESNext` with `rootDir`/`outDir` emits each Scripts/*.ts straight to wwwroot/*.js. + "module": "ESNext", + "moduleResolution": "Bundler", + // Type-only imports of Bit.Bsqlite's declarations for the wa-sqlite engine (Scripts/engine/*.d.ts). + // They are erased from the output, so nothing outside this package is compiled or bundled into it. + "paths": { + "bit-bsqlite/*": [ "../Bit.Bsqlite/Scripts/*" ] + }, + "lib": [ "DOM", "DOM.Iterable", "ESNext" ], + "rootDir": "Scripts", + "outDir": "wwwroot", + "strict": true, + "noFallthroughCasesInSwitch": true, + "noImplicitReturns": true, + "forceConsistentCasingInFileNames": true + }, + "include": [ "./Scripts/**/*.ts" ] +} diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/FetchOpfsVfs.targets b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/FetchOpfsVfs.targets new file mode 100644 index 00000000000..38e52ff85ee --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/FetchOpfsVfs.targets @@ -0,0 +1,44 @@ + + + + + + $(MSBuildProjectDirectory)/obj/upstream/ + $(MSBuildProjectDirectory)/wwwroot/ + $(OpfsVfsUpstreamDirectory)OPFSCoopSyncVFS.js + $(OpfsVfsWwwrootDirectory)OPFSCoopSyncVFS.js + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/rewrite-imports.mjs b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/rewrite-imports.mjs new file mode 100644 index 00000000000..ac7ba315324 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/rewrite-imports.mjs @@ -0,0 +1,31 @@ +// Rewrites wa-sqlite example imports so they resolve from a provider package. +// +// Upstream files live in src/examples/ and import '../FacadeVFS.js'. That path does not exist +// next to this package's wwwroot; the base classes are served from Bsqlite. The rewrite +// is mechanical and checksummed against the untouched download, so a silent upstream edit fails +// the build rather than shipping a VFS that cannot load. + +import { readFileSync, writeFileSync } from 'node:fs'; + +const [, , input, output] = process.argv; + +if (!input || !output) { + throw new Error('Usage: rewrite-imports.mjs '); +} + +const replacements = [ + ["from '../FacadeVFS.js'", "from '../Bit.Bsqlite/engine/FacadeVFS.js'"], + ["from '../VFS.js'", "from '../Bit.Bsqlite/engine/VFS.js'"], +]; + +let source = readFileSync(input, 'utf8'); + +for (const [from, to] of replacements) { + if (!source.includes(from)) { + throw new Error(`Expected import ${from} was not in the upstream VFS. The pin may have moved.`); + } + + source = source.replaceAll(from, to); +} + +writeFileSync(output, source); diff --git a/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/wa-sqlite.lock.props b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/wa-sqlite.lock.props new file mode 100644 index 00000000000..bfb54569f5f --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Storage.Opfs/vfs/wa-sqlite.lock.props @@ -0,0 +1,16 @@ + + + + + + v1.1.2 + https://raw.githubusercontent.com/rhashimoto/wa-sqlite + src/examples/OPFSCoopSyncVFS.js + 0D733FF90D684E1884C2B14C701C13B6F849C4E4548A5C64F2DA4BF12B4FD101 + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj b/src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj new file mode 100644 index 00000000000..10708539525 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Testing/Bit.Bsqlite.Testing.csproj @@ -0,0 +1,36 @@ + + + + + + net10.0;net9.0;net8.0 + Bit.Bsqlite + enable + Test support for bit Bsqlite: an in-process IBsqliteTransport backed by Microsoft.Data.Sqlite, so the stack can be unit-tested on desktop .NET without a browser, and the MSTest conformance kit every storage provider must pass - inherit its test classes in your provider's test project and hand them your provider. + $(Description) + Bit;Bsqlite;SQLite;EF Core;Testing;MSTest;Conformance + + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite.Testing/BsqliteFunctions.cs b/src/Bsqlite/Bit.Bsqlite.Testing/BsqliteFunctions.cs new file mode 100644 index 00000000000..f4228664d57 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Testing/BsqliteFunctions.cs @@ -0,0 +1,109 @@ +using System.Globalization; +using System.Text.RegularExpressions; +using Microsoft.Data.Sqlite; + +namespace Bit.Bsqlite.Testing; + +/// +/// Registers the scalar functions, aggregates, and collation that EF Core's SQLite provider +/// normally installs itself. +/// +/// +/// +/// EF's SqliteRelationalConnection only installs these when the connection is literally a +/// ; against any other DbConnection it logs +/// UnexpectedConnectionTypeWarning and moves on. Since Bsqlite supplies its own +/// connection, it inherits the obligation - miss it and every comparison, +/// aggregate, and ordering silently produces wrong answers. +/// +/// +/// This implementation is the reference for the worker-side UDF host: same names, same semantics. +/// +/// +public static class BsqliteFunctions +{ + private static readonly TimeSpan RegexTimeout = TimeSpan.FromSeconds(1); + + /// Installs every function EF Core's SQLite provider expects to exist. + public static void Register(SqliteConnection connection) + { + ArgumentNullException.ThrowIfNull(connection); + + // SQLite calls regexp(pattern, input) for `input REGEXP pattern` - arguments reversed + // relative to Regex.IsMatch. RegexOptions.NonBacktracking is deliberately *not* used: it + // rejects lookaround and backreferences that the stock provider accepts, which S4 caught as + // a conformance break. The timeout is the guard against catastrophic backtracking instead, + // and it cannot change the result of a pattern that terminates. + connection.CreateFunction( + "regexp", + (pattern, input) => input is null || pattern is null + ? null + : Regex.IsMatch(input, pattern, RegexOptions.None, RegexTimeout), + isDeterministic: true); + + connection.CreateFunction( + "ef_mod", + (decimal? dividend, decimal? divisor) => divisor == 0m ? null : dividend % divisor, + isDeterministic: true); + + connection.CreateFunction( + "ef_add", + (decimal? left, decimal? right) => left + right, + isDeterministic: true); + + connection.CreateFunction( + "ef_divide", + (decimal? dividend, decimal? divisor) => divisor == 0m ? null : dividend / divisor, + isDeterministic: true); + + connection.CreateFunction( + "ef_compare", + (decimal? left, decimal? right) => left.HasValue && right.HasValue + ? decimal.Compare(left.Value, right.Value) + : default(int?), + isDeterministic: true); + + connection.CreateFunction( + "ef_multiply", + (decimal? left, decimal? right) => left * right, + isDeterministic: true); + + connection.CreateFunction( + "ef_negate", + (decimal? value) => -value, + isDeterministic: true); + + connection.CreateAggregate( + "ef_sum", + seed: (decimal?)null, + (decimal? sum, decimal? value) => value is null ? sum : (sum ?? 0m) + value.Value); + + connection.CreateAggregate( + "ef_avg", + seed: (Sum: 0m, Count: 0UL), + ((decimal Sum, ulong Count) acc, decimal? value) => value is null + ? acc + : (acc.Sum + value.Value, acc.Count + 1), + acc => acc.Count == 0 ? null : (decimal?)(acc.Sum / acc.Count)); + + connection.CreateAggregate( + "ef_max", + seed: (decimal?)null, + (decimal? max, decimal? value) => value is null + ? max + : max is null ? value : Math.Max(max.Value, value.Value)); + + connection.CreateAggregate( + "ef_min", + seed: (decimal?)null, + (decimal? min, decimal? value) => value is null + ? min + : min is null ? value : Math.Min(min.Value, value.Value)); + + connection.CreateCollation( + "EF_DECIMAL", + (x, y) => decimal.Compare( + decimal.Parse(x, NumberStyles.Number, CultureInfo.InvariantCulture), + decimal.Parse(y, NumberStyles.Number, CultureInfo.InvariantCulture))); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Testing/BsqliteInProcessTransport.cs b/src/Bsqlite/Bit.Bsqlite.Testing/BsqliteInProcessTransport.cs new file mode 100644 index 00000000000..fa28d32a3d7 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Testing/BsqliteInProcessTransport.cs @@ -0,0 +1,353 @@ +using System.Collections.Concurrent; +using System.Data; +using System.Runtime.InteropServices; +using Bit.Bsqlite.Data; +using Microsoft.Data.Sqlite; +using SQLitePCL; + +namespace Bit.Bsqlite.Testing; + +/// +/// An that runs SQLite in-process via Microsoft.Data.Sqlite. +/// +/// +/// Stands in for the web-worker transport so provider behaviour can be tested on desktop .NET. +/// It mirrors the worker's responsibilities exactly - installing the EF function set (see +/// ), reporting committed writes through the same update-hook +/// and commit-hook mechanism, and snapshotting through sqlite3_serialize - so a test that +/// passes here is a meaningful signal. +/// +public sealed class BsqliteInProcessTransport : IBsqliteTransport, IBsqliteSnapshotSource, IBsqliteLockSource +{ + // Transports over the same database share their locks, the way tabs over the same file share the + // worker transport's Web Locks. + private static readonly ConcurrentDictionary SharedLocks = new(StringComparer.Ordinal); + + private readonly ConcurrentDictionary _locks; + private readonly string _lockScope; + private readonly string _connectionString; + private readonly bool _registerEfFunctions; + private readonly HashSet _writtenTables = new(StringComparer.OrdinalIgnoreCase); + private SqliteConnection? _connection; + private bool _committed; + + // Held so the native side never calls a delegate the collector has reclaimed. + private delegate_update? _updateHook; + private delegate_commit? _commitHook; + + /// + /// Defaults to a private in-memory database that lives as long as the connection. + /// + /// + /// When , the EF function set is deliberately omitted so tests can + /// demonstrate what breaks without it. + /// + public BsqliteInProcessTransport( + string? connectionString = null, + bool registerEfFunctions = true) + { + _connectionString = connectionString ?? "Data Source=:memory:"; + _registerEfFunctions = registerEfFunctions; + + // A private in-memory database is reachable through this transport alone, so there is nobody + // to share its locks with. + var options = new SqliteConnectionStringBuilder(_connectionString); + var isPrivate = options.DataSource is "" or ":memory:" + || (options.Mode == SqliteOpenMode.Memory && options.Cache != SqliteCacheMode.Shared); + + _locks = isPrivate ? new(StringComparer.Ordinal) : SharedLocks; + _lockScope = options.Mode == SqliteOpenMode.Memory || isPrivate + ? "memory:" + options.DataSource + : Path.GetFullPath(options.DataSource); + } + + /// Statements executed so far, in order. Useful for asserting generated SQL. + public List ExecutedCommands { get; } = []; + + /// The underlying connection, for tests that need to inspect engine state directly. + public SqliteConnection? Connection => _connection; + + /// + public event EventHandler? TablesChanged; + + /// + /// + /// True, the way the worker transport is: writes are reported from SQLite's update hook once + /// the transaction that made them has committed, so live queries behave on desktop exactly as + /// they do in the browser. + /// + public bool ReportsLocalWrites => true; + + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + { + if (_connection is not null) + { + return Task.CompletedTask; + } + + _connection = new SqliteConnection(_connectionString); + _connection.Open(); + + if (_registerEfFunctions) + { + BsqliteFunctions.Register(_connection); + } + + // The same two hooks the worker installs: exact table names per row change, and the + // moment they became visible. See NotifyIfCommitted for how they combine. + var handle = _connection.Handle!; + _updateHook = (_, _, _, table, _) => _writtenTables.Add(table.utf8_to_string()); + _commitHook = _ => + { + _committed = true; + return raw.SQLITE_OK; + }; + raw.sqlite3_update_hook(handle, _updateHook, null); + raw.sqlite3_commit_hook(handle, _commitHook, null); + + return Task.CompletedTask; + } + + public Task CloseAsync(CancellationToken cancellationToken = default) + { + // Closing an in-memory database would discard it, and EF opens and closes connections + // freely around operations. Keep it alive until disposal. + return Task.CompletedTask; + } + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(batch); + + if (_connection is null) + { + throw new InvalidOperationException("The transport is not open."); + } + + var results = new List(batch.Count); + try + { + foreach (var request in batch) + { + cancellationToken.ThrowIfCancellationRequested(); + ExecutedCommands.Add(request.CommandText); + + var changesBefore = raw.sqlite3_total_changes(_connection.Handle!); + results.Add(Execute(_connection, request) with + { + LastInsertRowId = raw.sqlite3_last_insert_rowid(_connection.Handle!), + }); + RecordTextualWrites(request.CommandText, raw.sqlite3_total_changes(_connection.Handle!) - changesBefore); + } + } + finally + { + // Also on failure: the statements before the one that failed have run, and in + // autocommit mode they have committed. + NotifyIfCommitted(_connection); + } + + return Task.FromResult>(results); + } + + /// + public Task ExportAsync(CancellationToken cancellationToken = default) + { + if (_connection is null) + { + throw new InvalidOperationException("The transport is not open."); + } + + var pointer = raw.sqlite3_serialize(_connection.Handle!, "main", out var size, 0); + if (pointer == IntPtr.Zero) + { + if (size == 0) + { + return Task.FromResult(Array.Empty()); + } + + throw new BsqliteException("sqlite3_serialize could not snapshot the database."); + } + + try + { + var image = new byte[size]; + Marshal.Copy(pointer, image, 0, image.Length); + return Task.FromResult(image); + } + finally + { + raw.sqlite3_free(pointer); + } + } + + /// + /// + /// A semaphore per database and name, shared by every transport in the process that opens the + /// same file or shared-cache memory database - the desktop stand-in for a Web Lock. + /// + public async Task AcquireLockAsync(string name, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(name); + + var gate = _locks.GetOrAdd($"{_lockScope}|{name}", _ => new SemaphoreSlim(1, 1)); + await gate.WaitAsync(cancellationToken).ConfigureAwait(false); + return new HeldLock(gate); + } + + /// + /// Adds the tables a request's SQL names - the same rule as the worker's + /// recordTextualWrites. + /// + /// + /// The update hook is exact for row changes but blind to DDL and to the truncate-optimised + /// DELETE FROM t, so the statement text fills those in - only when the request changed + /// rows or reads as DDL, so a write that matched nothing raises nothing. + /// + private void RecordTextualWrites(string sql, int changes) + { + if ((changes > 0 || BsqliteTableNames.LooksLikeSchemaChange(sql)) + && BsqliteTableNames.LooksLikeWrite(sql)) + { + _writtenTables.UnionWith(BsqliteTableNames.Extract(sql)); + } + } + + /// + /// Raises for what was written, once it is committed - the same + /// rule as the worker's notifyIfCommitted. + /// + /// + /// Nothing is raised while a transaction is open; once the connection is back in autocommit the + /// accumulated names go out if a commit happened, and are dropped if the transaction rolled back. + /// + private void NotifyIfCommitted(SqliteConnection connection) + { + if (raw.sqlite3_get_autocommit(connection.Handle!) == 0) + { + return; + } + + var committed = _committed; + var tables = new HashSet(_writtenTables, StringComparer.OrdinalIgnoreCase); + _committed = false; + _writtenTables.Clear(); + + if (committed && tables.Count > 0) + { + TablesChanged?.Invoke(this, new BsqliteTablesChangedEventArgs(tables)); + } + } + + private static BsqliteCommandResult Execute(SqliteConnection connection, BsqliteCommandRequest request) + { + using var command = connection.CreateCommand(); + command.CommandText = request.CommandText; + + foreach (var parameter in request.Parameters) + { + command.Parameters.AddWithValue(parameter.Name, parameter.Value ?? DBNull.Value); + } + + switch (request.ResultKind) + { + case BsqliteResultKind.NonQuery: + return new BsqliteCommandResult { RecordsAffected = command.ExecuteNonQuery() }; + + case BsqliteResultKind.Scalar: + { + var value = command.ExecuteScalar(); + return new BsqliteCommandResult + { + ColumnNames = ["value"], + ColumnTypes = [null], + Rows = [[value is DBNull ? null : value]], + }; + } + + case BsqliteResultKind.Reader: + using (var reader = command.ExecuteReader( + request.RowLimit == 0 ? CommandBehavior.SchemaOnly : CommandBehavior.Default)) + { + return Materialize(reader, request.RowLimit ?? int.MaxValue); + } + + default: + throw new ArgumentOutOfRangeException(nameof(request), request.ResultKind, "Unknown result kind."); + } + } + + private static BsqliteCommandResult Materialize(SqliteDataReader reader, int rowLimit) + { + var columnNames = new string[reader.FieldCount]; + var columnTypes = new string?[reader.FieldCount]; + for (var i = 0; i < reader.FieldCount; i++) + { + columnNames[i] = reader.GetName(i); + columnTypes[i] = SafeDataTypeName(reader, i); + } + + var rows = new List(); + while (rows.Count < rowLimit && reader.Read()) + { + var row = new object?[reader.FieldCount]; + for (var i = 0; i < reader.FieldCount; i++) + { + row[i] = reader.IsDBNull(i) ? null : reader.GetValue(i); + } + + rows.Add(row); + } + + // Remaining statements in the text still have to run, as they do in the worker. + while (reader.NextResult()) + { + } + + return new BsqliteCommandResult + { + ColumnNames = columnNames, + ColumnTypes = columnTypes, + Rows = rows, + RecordsAffected = reader.RecordsAffected, + }; + } + + private static string? SafeDataTypeName(SqliteDataReader reader, int ordinal) + { + // Computed columns have no declared type and Microsoft.Data.Sqlite throws rather than + // returning null. + try + { + return reader.GetDataTypeName(ordinal); + } + catch (InvalidOperationException) + { + return null; + } + } + + public ValueTask DisposeAsync() + { + _connection?.Dispose(); + _connection = null; + return ValueTask.CompletedTask; + } + + private sealed class HeldLock(SemaphoreSlim gate) : IAsyncDisposable + { + private int _released; + + public ValueTask DisposeAsync() + { + if (Interlocked.Exchange(ref _released, 1) == 0) + { + gate.Release(); + } + + return ValueTask.CompletedTask; + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Testing/Conformance/BsqliteStorageEngineConformanceTests.cs b/src/Bsqlite/Bit.Bsqlite.Testing/Conformance/BsqliteStorageEngineConformanceTests.cs new file mode 100644 index 00000000000..cf544aa28f9 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Testing/Conformance/BsqliteStorageEngineConformanceTests.cs @@ -0,0 +1,172 @@ +using Bit.Bsqlite.Data; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Bit.Bsqlite.Storage.ConformanceTests; + +/// +/// Engine-level rules every backend must satisfy once a real database is open. Inherit it next +/// to and return an already-opened +/// transport. +/// +/// +/// Write atomicity and the WAL ban can be checked against any transport, including the in-process +/// one. Crash safety and the declared concurrency levels need a worker that can be killed and a +/// backend that actually shares files; those tests skip unless the subclass says it can provide +/// that. A skip is an acknowledged gap, not a pass. +/// +public abstract class BsqliteStorageEngineConformanceTests : IAsyncDisposable +{ + /// Set by MSTest for each test; absent when a rule is called directly. + public TestContext? TestContext { get; set; } + + private readonly List _owned = []; + + private CancellationToken Ct => TestContext?.CancellationToken ?? CancellationToken.None; + + /// + /// An opened transport talking to a fresh database. Called more than once when a test needs a + /// second connection. + /// + protected abstract ValueTask CreateOpenTransportAsync(); + + /// + /// Whether this backend can be crash-tested: the transport must survive a kill mid-commit and + /// the data must outlive it. In-memory and in-process engines cannot. + /// + protected virtual bool SupportsCrashInjection => false; + + /// + public async ValueTask DisposeAsync() + { + foreach (var owned in _owned) + { + await owned.DisposeAsync(); + } + + _owned.Clear(); + GC.SuppressFinalize(this); + } + + private async Task ConnectAsync() + { + var transport = await CreateOpenTransportAsync(); + _owned.Add(transport); + + var connection = new BsqliteConnection(transport, "conformance.db"); + _owned.Add(connection); + await connection.OpenAsync(Ct); + return connection; + } + + [TestMethod] + public async Task InsertThenSelect_RoundTripsARow() + { + await using var connection = await ConnectAsync(); + await using var command = connection.CreateCommand(); + + command.CommandText = "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"; + await command.ExecuteNonQueryAsync(Ct); + + command.CommandText = "INSERT INTO product (name) VALUES ('Widget')"; + Assert.AreEqual(1, await command.ExecuteNonQueryAsync(Ct)); + + command.CommandText = "SELECT name FROM product"; + Assert.AreEqual("Widget", await command.ExecuteScalarAsync(Ct)); + } + + /// + /// A rolled-back write must be invisible. This is the atomicity the admin-layer kit cannot + /// check, because it never runs SQL. + /// + [TestMethod] + public async Task Rollback_LeavesTheDatabaseUnchanged() + { + await using var connection = await ConnectAsync(); + await using var command = connection.CreateCommand(); + + command.CommandText = "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"; + await command.ExecuteNonQueryAsync(Ct); + + command.CommandText = "BEGIN; INSERT INTO product (name) VALUES ('Ghost'); ROLLBACK;"; + await command.ExecuteNonQueryAsync(Ct); + + command.CommandText = "SELECT COUNT(*) FROM product"; + Assert.AreEqual(0L, Convert.ToInt64(await command.ExecuteScalarAsync(Ct))); + } + + [TestMethod] + public async Task Commit_MakesTheWriteVisible() + { + await using var connection = await ConnectAsync(); + await using var command = connection.CreateCommand(); + + command.CommandText = "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"; + await command.ExecuteNonQueryAsync(Ct); + + command.CommandText = "BEGIN; INSERT INTO product (name) VALUES ('Kept'); COMMIT;"; + await command.ExecuteNonQueryAsync(Ct); + + command.CommandText = "SELECT name FROM product"; + Assert.AreEqual("Kept", await command.ExecuteScalarAsync(Ct)); + } + + /// + /// SQLite defaults foreign keys off and Microsoft.Data.Sqlite turns them on for every + /// connection, so a transport that leaves them off enforces a model's relationships differently + /// from the same model on the server - silently, and only for writes that were already wrong. + /// + [TestMethod] + public async Task ForeignKeys_AreEnforced() + { + await using var connection = await ConnectAsync(); + await using var command = connection.CreateCommand(); + + command.CommandText = "PRAGMA foreign_keys"; + Assert.AreEqual(1L, Convert.ToInt64(await command.ExecuteScalarAsync(Ct))); + + command.CommandText = "CREATE TABLE parent (id INTEGER PRIMARY KEY)"; + await command.ExecuteNonQueryAsync(Ct); + + command.CommandText = + "CREATE TABLE child (id INTEGER PRIMARY KEY, parent_id INTEGER REFERENCES parent(id))"; + await command.ExecuteNonQueryAsync(Ct); + + command.CommandText = "INSERT INTO child (id, parent_id) VALUES (1, 999)"; + await Assert.ThrowsAsync(() => command.ExecuteNonQueryAsync(Ct)); + } + + [TestMethod] + public async Task JournalModeWal_IsRejected() + { + await using var connection = await ConnectAsync(); + await using var command = connection.CreateCommand(); + command.CommandText = "PRAGMA journal_mode=WAL"; + + var error = await Assert.ThrowsExactlyAsync( + () => command.ExecuteNonQueryAsync(Ct)); + + Assert.Contains("WAL", error.Message); + Assert.Contains("shared-memory", error.Message); + } + + [TestMethod] + public async Task CrashSafety_IsNotClaimed_WhenTheBackendCannotBeKilled() + { + SkipUnless(SupportsCrashInjection, + "This backend cannot be crash-tested: there is no worker to kill, or the data does not " + + "outlive the process. Persistent providers implement this by terminating the worker " + + "inside a commit and reopening."); + } + + /// + /// Reports the calling test as inconclusive - MSTest's skip - with when + /// does not hold. + /// + private static void SkipUnless(bool condition, string reason) + { + if (!condition) + { + Assert.Inconclusive(reason); + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Testing/Conformance/BsqliteStorageProviderConformanceTests.cs b/src/Bsqlite/Bit.Bsqlite.Testing/Conformance/BsqliteStorageProviderConformanceTests.cs new file mode 100644 index 00000000000..b31586d64eb --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Testing/Conformance/BsqliteStorageProviderConformanceTests.cs @@ -0,0 +1,412 @@ +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Bit.Bsqlite.Storage.ConformanceTests; + +/// +/// The conformance suite every storage backend must pass. Inherit it in your provider's test project +/// and return your provider from ; the tests come with the base class. +/// +/// +/// +/// The core trusts without verifying it - +/// selection, pragma guarding, and durability options are all driven by what a backend claims. This +/// suite is what makes that trust reasonable: it checks that the claims are internally coherent and +/// that the admin surface behaves the way cross-provider migration assumes. +/// +/// +/// It covers what can be checked without a running engine. Claims that only a real database can +/// settle - write atomicity, crash safety, the concurrency levels - are verified by the engine-level +/// suite, which needs the worker host. +/// +/// +public abstract class BsqliteStorageProviderConformanceTests : IAsyncDisposable +{ + /// Set by MSTest for each test; absent when a rule is called directly. + public TestContext? TestContext { get; set; } + + private readonly List _created = []; + + private CancellationToken Ct => TestContext?.CancellationToken ?? CancellationToken.None; + + /// + /// Returns a provider to test. Called more than once per run, and each call must return an + /// instance that behaves like a fresh one in a fresh browser session. + /// + /// + /// Instances that implement or are + /// disposed for you, so a backend holding file handles can be tested without leaking them. + /// + protected abstract IBsqliteStorageProvider CreateProvider(); + + /// + public async ValueTask DisposeAsync() + { + foreach (var provider in _created) + { + switch (provider) + { + case IAsyncDisposable asyncDisposable: + await asyncDisposable.DisposeAsync(); + break; + case IDisposable disposable: + disposable.Dispose(); + break; + } + } + + _created.Clear(); + GC.SuppressFinalize(this); + } + + private IBsqliteStorageProvider Provider() + { + var provider = CreateProvider(); + Assert.IsNotNull(provider); + _created.Add(provider); + return provider; + } + + /// + /// Skips the calling test when the backend cannot run here, so that the suite reports honestly on + /// a browser or OS that lacks the underlying API instead of failing for the wrong reason. + /// + private async ValueTask AdminAsync( + IBsqliteStorageProvider provider) + { + var probe = await provider.ProbeAsync(Ct); + SkipUnless(probe.IsAvailable, + $"'{provider.Name}' is unavailable here: {probe.UnavailableReason}"); + + return provider.Admin; + } + + /// + /// Reports the calling test as inconclusive - MSTest's skip - with when + /// does not hold. + /// + private static void SkipUnless(bool condition, string reason) + { + if (!condition) + { + Assert.Inconclusive(reason); + } + } + + /// + /// Builds a plausible SQLite file image of the requested size. It carries the real file magic so + /// that a backend which validates what it is handed still sees a valid database, and its body is + /// deterministic per seed so a failure names the byte that differed. + /// + private static byte[] Image(int length, int seed) + { + var bytes = new byte[length]; + new Random(seed).NextBytes(bytes); + + ReadOnlySpan magic = "SQLite format 3\u0000"u8; + if (length >= magic.Length) + { + magic.CopyTo(bytes); + } + + return bytes; + } + + // --------------------------------------------------------------------------------------------- + // Declaration: the facts the core reads once and then relies on everywhere. + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public void Name_IsNotBlank() + { + Assert.IsFalse(string.IsNullOrWhiteSpace(Provider().Name)); + } + + /// + /// The name is recorded as the sticky binding of every database the backend creates, so an + /// instance-dependent name would orphan data on the next session. + /// + [TestMethod] + public void Name_IsTheSameForEveryInstance() + { + Assert.AreEqual(Provider().Name, Provider().Name); + } + + /// + /// Capabilities are read once and cached, and probing must not change them - a backend that + /// downgrades itself after probing would have already been selected on the stronger claim. + /// + [TestMethod] + public async Task Capabilities_DoNotChangeWhenProbed() + { + var provider = Provider(); + var before = provider.Capabilities; + + await provider.ProbeAsync(Ct); + + Assert.AreEqual(before, provider.Capabilities); + } + + [TestMethod] + public void Capabilities_DeclareAtLeastOneExecutionContext() + { + var provider = Provider(); + + Assert.AreNotEqual( + BsqliteExecutionContexts.None, + provider.Capabilities.SupportedContexts); + } + + /// + /// v1 hosts the engine only in a dedicated worker, so a backend that cannot run there cannot be + /// selected at all. Asserted rather than left to fail later at connection time. + /// + [TestMethod] + public void Capabilities_SupportTheDedicatedWorker() + { + var provider = Provider(); + + Assert.IsTrue( + provider.Capabilities.SupportedContexts.HasFlag( + BsqliteExecutionContexts.DedicatedWorker), + $"'{provider.Name}' cannot run in a dedicated worker, which is the only context that " + + "hosts the engine."); + } + + /// + /// Relaxed durability trades crash safety for speed. There is nothing to trade when the data does + /// not outlive the page, and offering the option would imply a persistence guarantee. + /// + [TestMethod] + public void NonPersistentBackend_DoesNotOfferRelaxedDurability() + { + var capabilities = Provider().Capabilities; + + Assert.IsFalse( + !capabilities.IsPersistent && capabilities.SupportsRelaxedDurability, + "A backend whose data does not survive a reload cannot offer a durability level."); + } + + /// + /// A VFS needing the async engine build is necessarily one we load as JavaScript; the only VFS + /// available without a module is the one compiled into the engine, which is synchronous. + /// + [TestMethod] + public void BackendNeedingTheAsyncBuild_ShipsAVfsModule() + { + var provider = Provider(); + + if (provider.Capabilities.RequiredBuild is BsqliteEngineBuild.Synchronous) + { + return; + } + + Assert.IsNotNull(provider.VfsModule); + } + + // --------------------------------------------------------------------------------------------- + // Probing: selection walks the candidate list, so a probe must answer rather than abort the walk. + // --------------------------------------------------------------------------------------------- + + /// + /// Selection depends on being able to try a candidate and move on. A throwing probe is tolerated + /// by the resolver but loses the explanation, so the contract asks backends to report instead. + /// + [TestMethod] + public async Task ProbeAsync_ReportsUnavailability_RatherThanThrowing() + { + var provider = Provider(); + + var probe = await provider.ProbeAsync(Ct); + + Assert.IsNotNull(probe); + } + + [TestMethod] + public async Task ProbeAsync_GivesTheSameVerdictEachTime() + { + var provider = Provider(); + + var first = await provider.ProbeAsync(Ct); + var second = await provider.ProbeAsync(Ct); + + Assert.AreEqual(first.IsAvailable, second.IsAvailable); + } + + // --------------------------------------------------------------------------------------------- + // Admin: the operations diagnostics and cross-provider migration are built from. + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Exists_IsFalse_ForADatabaseNeverCreated() + { + var admin = await AdminAsync(Provider()); + + Assert.IsFalse(await admin.ExistsAsync($"absent-{Guid.NewGuid():N}", Ct)); + } + + [TestMethod] + public async Task Import_MakesTheDatabaseExistAndAppearInTheList() + { + var admin = await AdminAsync(Provider()); + var name = $"conformance-{Guid.NewGuid():N}"; + + await admin.ImportAsync(name, Image(4096, seed: 1), Ct); + + Assert.IsTrue(await admin.ExistsAsync(name, Ct)); + Assert.Contains(name, await admin.ListAsync(Ct)); + } + + [TestMethod] + public async Task Export_ReturnsExactlyWhatWasImported() + { + var admin = await AdminAsync(Provider()); + var name = $"conformance-{Guid.NewGuid():N}"; + var image = Image(4096, seed: 2); + + await admin.ImportAsync(name, image, Ct); + + CollectionAssert.AreEqual(image, await admin.ExportAsync(name, Ct)); + } + + /// + /// Migration copies a database that is about to be written to, so the exported image has to be a + /// snapshot the caller owns rather than a window onto live storage. + /// + [TestMethod] + public async Task Export_ReturnsACopy_NotAViewOfLiveStorage() + { + var admin = await AdminAsync(Provider()); + var name = $"conformance-{Guid.NewGuid():N}"; + var image = Image(4096, seed: 3); + await admin.ImportAsync(name, image, Ct); + + var exported = await admin.ExportAsync(name, Ct); + exported[^1] ^= 0xFF; + + CollectionAssert.AreEqual(image, await admin.ExportAsync(name, Ct)); + } + + /// + /// Crossing several SQLite page boundaries, which is where a block-oriented backend splits and + /// reassembles an image and where off-by-one block bugs surface. + /// + [TestMethod] + public async Task Export_RoundTripsAnImageSpanningManyPages() + { + var admin = await AdminAsync(Provider()); + var name = $"conformance-{Guid.NewGuid():N}"; + var image = Image(384 * 1024 + 517, seed: 4); + + await admin.ImportAsync(name, image, Ct); + + CollectionAssert.AreEqual(image, await admin.ExportAsync(name, Ct)); + } + + /// + /// A newly created SQLite database is a zero-length file, so migration and backup both encounter + /// empty images. Importing one must produce an existing, empty database rather than nothing. + /// + [TestMethod] + public async Task Import_AcceptsAnEmptyImage() + { + var admin = await AdminAsync(Provider()); + var name = $"conformance-{Guid.NewGuid():N}"; + + await admin.ImportAsync(name, ReadOnlyMemory.Empty, Ct); + + Assert.IsTrue(await admin.ExistsAsync(name, Ct)); + Assert.IsEmpty(await admin.ExportAsync(name, Ct)); + } + + /// + /// Import is specified to replace, and must leave no trace of the previous image - a backend that + /// overwrote in place would leave the tail of a larger database behind. + /// + [TestMethod] + public async Task Import_ReplacesAnExistingDatabaseEntirely() + { + var admin = await AdminAsync(Provider()); + var name = $"conformance-{Guid.NewGuid():N}"; + await admin.ImportAsync(name, Image(64 * 1024, seed: 5), Ct); + + var replacement = Image(4096, seed: 6); + await admin.ImportAsync(name, replacement, Ct); + + CollectionAssert.AreEqual(replacement, await admin.ExportAsync(name, Ct)); + } + + [TestMethod] + public async Task Delete_RemovesTheDatabaseFromExistenceAndTheList() + { + var admin = await AdminAsync(Provider()); + var name = $"conformance-{Guid.NewGuid():N}"; + await admin.ImportAsync(name, Image(4096, seed: 7), Ct); + + await admin.DeleteAsync(name, Ct); + + Assert.IsFalse(await admin.ExistsAsync(name, Ct)); + Assert.DoesNotContain(name, await admin.ListAsync(Ct)); + } + + /// + /// Migration and teardown both delete defensively, so deleting what is not there is a success. + /// + [TestMethod] + public async Task Delete_SucceedsQuietly_WhenTheDatabaseIsAbsent() + { + var admin = await AdminAsync(Provider()); + + await admin.DeleteAsync($"absent-{Guid.NewGuid():N}", Ct); + } + + [TestMethod] + public async Task Export_ThrowsForADatabaseThatDoesNotExist() + { + var admin = await AdminAsync(Provider()); + + await Assert.ThrowsAsync( + async () => await admin.ExportAsync($"absent-{Guid.NewGuid():N}", Ct)); + } + + [TestMethod] + public async Task Databases_DoNotLeakIntoEachOther() + { + var admin = await AdminAsync(Provider()); + var first = $"conformance-{Guid.NewGuid():N}"; + var second = $"conformance-{Guid.NewGuid():N}"; + var firstImage = Image(4096, seed: 8); + var secondImage = Image(8192, seed: 9); + + await admin.ImportAsync(first, firstImage, Ct); + await admin.ImportAsync(second, secondImage, Ct); + await admin.DeleteAsync(second, Ct); + + CollectionAssert.AreEqual(firstImage, await admin.ExportAsync(first, Ct)); + } + + [TestMethod] + [DataRow("")] + [DataRow(" ")] + public async Task Admin_RejectsABlankDatabaseName(string name) + { + var admin = await AdminAsync(Provider()); + + await Assert.ThrowsAsync( + async () => await admin.ExistsAsync(name, Ct)); + await Assert.ThrowsAsync( + async () => await admin.DeleteAsync(name, Ct)); + await Assert.ThrowsAsync( + async () => await admin.ExportAsync(name, Ct)); + await Assert.ThrowsAsync( + async () => await admin.ImportAsync(name, Image(4096, seed: 10), Ct)); + } + + [TestMethod] + public async Task Admin_RejectsANullDatabaseName() + { + var admin = await AdminAsync(Provider()); + + await Assert.ThrowsAsync( + async () => await admin.ExistsAsync(null!, Ct)); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite.Testing/README.md b/src/Bsqlite/Bit.Bsqlite.Testing/README.md new file mode 100644 index 00000000000..4d6d3e50938 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.Testing/README.md @@ -0,0 +1,62 @@ +# Bit.Bsqlite.Testing + +Test support for [bit Bsqlite](https://www.nuget.org/packages/Bit.Bsqlite). Two things live here. + +``` +dotnet add package Bit.Bsqlite.Testing +``` + +## 1. Run the stack without a browser + +`BsqliteInProcessTransport` is an `IBsqliteTransport` backed by Microsoft.Data.Sqlite that +stands in for the web-worker transport, so EF Core mappings, migrations, generated SQL, and live +queries can be tested on desktop .NET. + +```csharp +await using var transport = new BsqliteInProcessTransport(); +await using var connection = new BsqliteConnection(transport, "test.db"); +await connection.OpenAsync(); +options.UseBitBsqlite(connection); + +// after the act +Assert.Contains("CREATE TABLE", transport.ExecutedCommands[0]); +``` + +It mirrors the worker's responsibilities rather than approximating them. It installs the EF function +set, which matters more than it sounds: EF's SQLite provider registers its scalar functions, +aggregates, and collation only when the connection is literally a `SqliteConnection`; against any +other `DbConnection` it logs a warning and moves on. Bsqlite supplies its own connection, so it +inherits that obligation, and `BsqliteFunctions` is the reference implementation the +worker-side UDF host matches. Pass `registerEfFunctions: false` to see what breaks without it. And +it reports writes the way the worker does - from SQLite's update hook, cascades and triggers +included, once the transaction that made them commits - so live queries behave on desktop +exactly as they do in the browser. + +## 2. The storage conformance kit + +If you are writing a storage backend, this is how you prove it works. Inherit the suites in your own +test project and hand them your provider: + +```csharp +[TestClass] +public sealed class MyBackendConformanceTests : BsqliteStorageProviderConformanceTests +{ + protected override IBsqliteStorageProvider CreateProvider() => new MyStorageProvider(); +} +``` + +The core trusts `IBsqliteStorageProvider.Capabilities` without verifying it — selection, pragma +guarding, and durability options are all driven by what a backend *claims*. The kit is what makes +that trust reasonable. + +- `BsqliteStorageProviderConformanceTests` — everything checkable without a running engine: are + the declared capabilities internally coherent, does the admin surface behave the way + cross-provider migration assumes. +- `BsqliteStorageEngineConformanceTests` — the claims only a real database can settle: write + atomicity, crash safety, concurrency levels. Needs the worker host. + +Providers that cannot run in the current environment report the test inconclusive with an explanation +rather than failing, so the same suite is meaningful on desktop and in a browser. + +Built on MSTest: the derived classes need `[TestClass]`, and the kit's test methods come with the base +class. MIT. Part of [bit Bsqlite](https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite). diff --git a/src/Bsqlite/Bit.Bsqlite.slnx b/src/Bsqlite/Bit.Bsqlite.slnx new file mode 100644 index 00000000000..016f7ae3bb4 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite.slnx @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageCandidateOutcome.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageCandidateOutcome.cs new file mode 100644 index 00000000000..45e969cfb2d --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageCandidateOutcome.cs @@ -0,0 +1,31 @@ +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite; + +/// +/// One candidate's fate, recorded for every candidate so that both diagnostics and failure messages +/// can explain the whole decision rather than only its outcome. +/// +public sealed record BsqliteStorageCandidateOutcome +{ + /// The configured provider name. + public required string ProviderName { get; init; } + + /// What happened to this candidate. + public required BsqliteStorageCandidateStatus Status { get; init; } + + /// The probe result, when the candidate got as far as being probed. + public BsqliteProbeResult? Probe { get; init; } + + /// A sentence explaining this outcome, suitable for showing to a developer. + public string? Explanation { get; init; } + + /// Renders this outcome as one line of a failure report. + public override string ToString() + { + var detail = Explanation ?? Probe?.UnavailableReason; + return detail is null + ? $"{ProviderName}: {Status}" + : $"{ProviderName}: {Status} - {detail}"; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageCandidateStatus.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageCandidateStatus.cs new file mode 100644 index 00000000000..b81cd947cf2 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageCandidateStatus.cs @@ -0,0 +1,26 @@ +namespace Bit.Bsqlite; + +/// What happened to one candidate during storage selection. +public enum BsqliteStorageCandidateStatus +{ + /// Not examined, because an earlier candidate had already won. + NotProbed, + + /// Named in the configuration, but no provider with that name is registered. + NotRegistered, + + /// Probed and reported unusable in this browser. + Unavailable, + + /// + /// Usable, but rejected because it does not survive a reload and + /// was not set. + /// + RejectedAsNonPersistent, + + /// Rejected because the database already exists on a different backend. + RejectedByExistingBinding, + + /// Chosen. + Selected, +} diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageMigrationMode.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageMigrationMode.cs new file mode 100644 index 00000000000..863af592d42 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageMigrationMode.cs @@ -0,0 +1,26 @@ +namespace Bit.Bsqlite; + +/// +/// What to do when a database lives on one backend but a more preferred backend has since become +/// available - for example a database created on IndexedDB in a browser that now supports OPFS. +/// +public enum BsqliteStorageMigrationMode +{ + /// + /// Keep using the backend that holds the data. A diagnostic reports that a better option exists, + /// but nothing moves. The default, because moving a database is never something to do behind the + /// application's back. + /// + KeepExisting, + + /// + /// Keep using the existing backend, and surface the opportunity so the application can migrate + /// deliberately - when the user is idle, or after asking. + /// + Manual, + + /// + /// Migrate before the database is first opened, once the preferred backend becomes available. + /// + AutomaticOnOpen, +} diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageResolution.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageResolution.cs new file mode 100644 index 00000000000..180b2431392 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageResolution.cs @@ -0,0 +1,46 @@ +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite; + +/// +/// The backend chosen for a database, together with enough context to explain the choice. +/// +public sealed record BsqliteStorageResolution +{ + /// The database this decision applies to. + public required string DatabaseName { get; init; } + + /// The chosen backend. + public required IBsqliteStorageProvider Provider { get; init; } + + /// The probe that cleared it. + public required BsqliteProbeResult Probe { get; init; } + + /// Every candidate's outcome, in preference order. + public required IReadOnlyList Attempts { get; init; } + + /// + /// Whether the most preferred backend was chosen. False means a fallback was used, which is + /// reported as a warning rather than passed over in silence. + /// + public bool IsFirstChoice { get; init; } + + /// + /// Whether the choice was dictated by where the data already lives rather than by preference + /// order. + /// + public bool WasDecidedByExistingData { get; init; } + + /// + /// Whether the database had no binding and was found by asking each backend whether it holds + /// it - after the binding store was cleared, typically. Committing the resolution binds it again. + /// + public bool WasFoundWithoutBinding { get; init; } + + /// + /// A backend ranked above that is available but not holding the data. + /// Non-null only for an existing database, and what + /// decides the response to. + /// + public IBsqliteStorageProvider? BetterProviderAvailable { get; init; } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageSelection.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageSelection.cs new file mode 100644 index 00000000000..ba97bc34f49 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageSelection.cs @@ -0,0 +1,39 @@ +namespace Bit.Bsqlite; + +/// +/// Which storage backends to use, in order, and how much latitude selection has when the first +/// choice is unavailable. +/// +public sealed record BsqliteStorageSelection +{ + internal BsqliteStorageSelection( + IReadOnlyList candidates, + bool allowNonPersistentFallback, + BsqliteStorageMigrationMode migrationMode) + { + Candidates = candidates; + AllowNonPersistentFallback = allowNonPersistentFallback; + MigrationMode = migrationMode; + } + + /// + /// Backend names in preference order, most preferred first. Never empty. + /// + public IReadOnlyList Candidates { get; } + + /// + /// Whether selection may fall back to a backend that does not survive a reload. Off by default: + /// silently demoting a persistent database to a volatile one loses the user's data at the next + /// refresh, which is not a decision a fallback rule should be allowed to make on its own. + /// + public bool AllowNonPersistentFallback { get; } + + /// What to do when a better backend becomes available for an existing database. + public BsqliteStorageMigrationMode MigrationMode { get; } + + /// + /// Whether the choice is strict, meaning a single candidate that must be available or the open + /// fails. A lone candidate is never quietly substituted. + /// + public bool IsStrict => Candidates.Count == 1; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageSelectionBuilder.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageSelectionBuilder.cs new file mode 100644 index 00000000000..c2191c17188 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageSelectionBuilder.cs @@ -0,0 +1,94 @@ +namespace Bit.Bsqlite; + +/// +/// Builds a , as used by +/// UseStorage(s => s.Prefer("opfs").Fallback("indexeddb")). +/// +public sealed class BsqliteStorageSelectionBuilder +{ + private readonly List _candidates = []; + private bool _allowNonPersistentFallback; + private BsqliteStorageMigrationMode _migrationMode = BsqliteStorageMigrationMode.KeepExisting; + + /// Sets the first choice. Calling it more than once is a configuration mistake. + public BsqliteStorageSelectionBuilder Prefer(string providerName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(providerName); + + if (_candidates.Count > 0) + { + throw new InvalidOperationException( + $"A preferred storage provider is already set ('{_candidates[0]}'). " + + $"Use Fallback(\"{providerName}\") to add it below the first choice."); + } + + _candidates.Add(providerName); + return this; + } + + /// + /// Adds the next choice, below everything already added. Order is the order given. + /// + public BsqliteStorageSelectionBuilder Fallback(string providerName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(providerName); + + if (_candidates.Count == 0) + { + throw new InvalidOperationException( + $"Call Prefer(\"{providerName}\") first - a fallback needs something to fall back from."); + } + + if (_candidates.Contains(providerName, StringComparer.OrdinalIgnoreCase)) + { + throw new InvalidOperationException( + $"Storage provider '{providerName}' is already a candidate. " + + "Listing it twice cannot change the outcome."); + } + + _candidates.Add(providerName); + return this; + } + + /// + /// Permits falling back to a backend that does not survive a reload, accepting that data written + /// in such a session is lost when the page closes. + /// + public BsqliteStorageSelectionBuilder AllowNonPersistentFallback() + { + _allowNonPersistentFallback = true; + return this; + } + + /// Sets what happens when a better backend becomes available later. + public BsqliteStorageSelectionBuilder WithMigrationMode(BsqliteStorageMigrationMode mode) + { + _migrationMode = mode; + return this; + } + + internal BsqliteStorageSelection Build() + { + if (_candidates.Count == 0) + { + throw new InvalidOperationException( + "No storage provider was selected. Call Prefer(...) with a registered provider name."); + } + + return new BsqliteStorageSelection( + [.. _candidates], + _allowNonPersistentFallback, + _migrationMode); + } + + /// Runs and produces the selection it describes. + public static BsqliteStorageSelection Create( + Action configure) + { + ArgumentNullException.ThrowIfNull(configure); + + var builder = new BsqliteStorageSelectionBuilder(); + configure(builder); + return builder.Build(); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageUnavailableException.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageUnavailableException.cs new file mode 100644 index 00000000000..40e8e0a2b1a --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/BsqliteStorageUnavailableException.cs @@ -0,0 +1,44 @@ +namespace Bit.Bsqlite; + +/// +/// Thrown when no configured storage backend could be used, carrying what was tried and why each +/// candidate was rejected. +/// +/// +/// Selection never substitutes a backend the application did not ask for, so this exception is the +/// only outcome when the configured choices do not work out. It reports every candidate's fate +/// because "storage unavailable" on its own is not something a developer can act on. +/// +public sealed class BsqliteStorageUnavailableException : Exception +{ + /// The database that could not be opened. + /// Every candidate considered, in preference order. + public BsqliteStorageUnavailableException( + string databaseName, + IReadOnlyList attempts) + : base(BuildMessage(databaseName, attempts)) + { + DatabaseName = databaseName; + Attempts = attempts; + } + + /// The database the application was trying to open. + public string DatabaseName { get; } + + /// Every candidate considered, in preference order, with its outcome. + public IReadOnlyList Attempts { get; } + + private static string BuildMessage( + string databaseName, + IReadOnlyList attempts) + { + var lines = attempts.Select(a => $" - {a}"); + + return $""" + No storage provider could open database '{databaseName}'. + + Candidates, in preference order: + {string.Join(Environment.NewLine, lines)} + """; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Abstractions/IBsqliteStorageBindingStore.cs b/src/Bsqlite/Bit.Bsqlite/Abstractions/IBsqliteStorageBindingStore.cs new file mode 100644 index 00000000000..37a0ae6a0d6 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Abstractions/IBsqliteStorageBindingStore.cs @@ -0,0 +1,34 @@ +namespace Bit.Bsqlite; + +/// +/// Remembers which storage backend holds each database, so that selection can put existing data +/// ahead of configured preference. +/// +/// +/// This is the data-loss guard. Without it, a database created on IndexedDB would be shadowed by a +/// fresh, empty OPFS database the moment a browser update made OPFS available - the application would +/// see an empty database and no error. The store therefore has to live somewhere no single backend +/// owns, and it holds names only, never data. +/// +public interface IBsqliteStorageBindingStore +{ + /// + /// The provider that holds , or if this + /// database has not been created yet. + /// + ValueTask GetProviderNameAsync( + string databaseName, + CancellationToken cancellationToken = default); + + /// Records that lives on . + ValueTask SetProviderNameAsync( + string databaseName, + string providerName, + CancellationToken cancellationToken = default); + + /// + /// Forgets , which is what deleting a database must do so a later + /// create is free to pick the best backend again. + /// + ValueTask RemoveAsync(string databaseName, CancellationToken cancellationToken = default); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj b/src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj new file mode 100644 index 00000000000..1079da8ccda --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Bit.Bsqlite.csproj @@ -0,0 +1,143 @@ + + + + + + net10.0;net9.0;net8.0 + Bit.Bsqlite + enable + + false + bit Bsqlite: SQLite in the browser for Blazor, with EF Core on top. The ADO.NET surface with live queries, the worker transport (storage selection, engine-build negotiation, binary framing, batching, lifecycle), the storage provider contract, the in-memory backend, the EF Core integration (UseBitBsqlite() on the stock SQLite provider), and the browser half - worker host, wa-sqlite engine builds and the JavaScript API storage providers' VFSes extend. + $(Description) + Bit;Bsqlite;Blazor;WebAssembly;SQLite;EF Core;Entity Framework Core;ADO.NET;Offline;Live query + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + <_BsqliteGeneratedJavaScript Include="wwwroot\bit-bsqlite-*.js" /> + + + + + + + + + + + + + + + + + + + + + + <_BsqliteStaleStamps Include="$(BaseIntermediateOutputPath)bit-bsqlite.*.stamp" /> + + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite/BitBsqlite.cs b/src/Bsqlite/Bit.Bsqlite/BitBsqlite.cs new file mode 100644 index 00000000000..111559d5cd3 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/BitBsqlite.cs @@ -0,0 +1,46 @@ +namespace Microsoft.Extensions.DependencyInjection; + +/// Registers bit Bsqlite with dependency injection. +/// +/// +/// Everything that reaches the browser, or holds a database session, is registered as scoped, which is +/// correct for every hosting model: in WebAssembly a scope lives for the whole application, so there is one +/// session per database exactly as the worker design wants; on a server one scope exists per circuit, so a +/// user's database - which lives in that user's browser - is never reached through another user's +/// IJSRuntime. +/// +/// +/// In a Blazor Web App, register this in the client project, and in the server project as well when the +/// pages that use it are prerendered there: resolving the services is harmless on the server, and only +/// opening a database needs the browser. +/// +/// +public static partial class BitBsqlite +{ + /// + /// Registers the session factory, storage selection, sticky bindings, and the worker transport. + /// Storage providers are added on the builder. + /// + /// + /// + /// builder.Services.AddBitBsqliteServices(sqlite => sqlite + /// .AddStorageProvider<BsqliteOpfsStorageProvider>() + /// .AddStorageProvider<BsqliteIndexedDbStorageProvider>() + /// .AddStorageProvider<BsqliteInMemoryStorageProvider>() + /// .UseStorage(s => s.Prefer("opfs").Fallback("indexeddb").Fallback("in-memory").AllowNonPersistentFallback()) + /// .AddDatabase("app.db")); + /// + /// + public static IServiceCollection AddBitBsqliteServices( + this IServiceCollection services, + Action configure) + { + ArgumentNullException.ThrowIfNull(services); + ArgumentNullException.ThrowIfNull(configure); + + var builder = new BsqliteBuilder(services); + configure(builder); + builder.Complete(); + return services; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteBatch.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteBatch.cs new file mode 100644 index 00000000000..d6a019de6d5 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteBatch.cs @@ -0,0 +1,244 @@ +using System.Data; +using System.Data.Common; +using System.Diagnostics.CodeAnalysis; + +namespace Bit.Bsqlite.Data; + +/// +/// Several commands sent to the worker in one round trip, each with its own parameters. +/// +/// +/// In the browser the cost that dominates is crossing the interop boundary, not executing SQL, so +/// a batch of twenty inserts costs about what one insert does. ExecuteReaderAsync +/// returns a reader over every command's result set, advanced with NextResultAsync. +/// +public sealed class BsqliteBatch : DbBatch +{ + private BsqliteConnection? _connection; + + public BsqliteBatch(BsqliteConnection connection) + => _connection = connection; + + public BsqliteBatch() + { + } + + /// The commands, typed. + public new BsqliteBatchCommandCollection BatchCommands { get; } = []; + + protected override DbBatchCommandCollection DbBatchCommands => BatchCommands; + + /// + public override int Timeout { get; set; } = 30; + + protected override DbConnection? DbConnection + { + get => _connection; + set => _connection = (BsqliteConnection?)value; + } + + protected override DbTransaction? DbTransaction { get; set; } + + public override void Cancel() + { + } + + public override void Prepare() + { + } + + public override Task PrepareAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; + + protected override DbBatchCommand CreateDbBatchCommand() => new BsqliteBatchCommand(); + + public override int ExecuteNonQuery() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(ExecuteNonQuery), nameof(ExecuteNonQueryAsync)); + + public override object? ExecuteScalar() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(ExecuteScalar), nameof(ExecuteScalarAsync)); + + protected override DbDataReader ExecuteDbDataReader(CommandBehavior behavior) + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(ExecuteReader), nameof(ExecuteReaderAsync)); + + /// Runs every command and returns the sum of their affected rows. + public override async Task ExecuteNonQueryAsync(CancellationToken cancellationToken = default) + { + var results = await ExecuteCoreAsync(BsqliteResultKind.NonQuery, rowLimit: null, cancellationToken) + .ConfigureAwait(false); + + var total = 0; + for (var i = 0; i < results.Count; i++) + { + total += results[i].RecordsAffected; + BatchCommands[i].SetRecordsAffected(results[i].RecordsAffected); + } + + return total; + } + + /// Runs every command and returns the first column of the first row of the first command that produced one. + public override async Task ExecuteScalarAsync(CancellationToken cancellationToken = default) + { + var results = await ExecuteCoreAsync(BsqliteResultKind.Reader, rowLimit: 1, cancellationToken) + .ConfigureAwait(false); + + for (var i = 0; i < results.Count; i++) + { + BatchCommands[i].SetRecordsAffected(results[i].RecordsAffected); + if (results[i].Rows.Count > 0 && results[i].Rows[0].Length > 0) + { + return results[i].Rows[0][0]; + } + } + + return null; + } + + protected override async Task ExecuteDbDataReaderAsync( + CommandBehavior behavior, + CancellationToken cancellationToken) + { + int? rowLimit = behavior.HasFlag(CommandBehavior.SchemaOnly) + ? 0 + : behavior.HasFlag(CommandBehavior.SingleRow) ? 1 : null; + + var results = await ExecuteCoreAsync(BsqliteResultKind.Reader, rowLimit, cancellationToken) + .ConfigureAwait(false); + + for (var i = 0; i < results.Count; i++) + { + BatchCommands[i].SetRecordsAffected(results[i].RecordsAffected); + } + + return new BsqliteDataReader(results, behavior.HasFlag(CommandBehavior.CloseConnection) ? _connection : null); + } + + private async Task> ExecuteCoreAsync( + BsqliteResultKind resultKind, + int? rowLimit, + CancellationToken cancellationToken) + { + if (_connection is null) + { + throw new InvalidOperationException("The batch requires an open connection."); + } + + var requests = new BsqliteCommandRequest[BatchCommands.Count]; + for (var i = 0; i < requests.Length; i++) + { + requests[i] = BatchCommands[i].ToRequest(resultKind, rowLimit); + } + + var results = await _connection.ExecuteBatchAsync(requests, cancellationToken).ConfigureAwait(false); + if (results.Count != requests.Length) + { + throw new InvalidOperationException( + $"The transport returned {results.Count} result(s) for {requests.Length} command(s)."); + } + + return results; + } +} + +/// One command inside a . +public sealed class BsqliteBatchCommand : DbBatchCommand +{ + [AllowNull] + public override string CommandText { get; set; } = string.Empty; + + public override CommandType CommandType + { + get => CommandType.Text; + set + { + if (value != CommandType.Text) + { + throw new ArgumentException( + $"SQLite has no stored procedures or table-direct commands; only {nameof(System.Data.CommandType.Text)} is supported.", + nameof(value)); + } + } + } + + private int _recordsAffected; + + public override int RecordsAffected => _recordsAffected; + + internal void SetRecordsAffected(int value) => _recordsAffected = value; + + /// The parameters, typed. + public new BsqliteParameterCollection Parameters { get; } = []; + + protected override DbParameterCollection DbParameterCollection => Parameters; + + public override bool CanCreateParameter => true; + + public override DbParameter CreateParameter() => new BsqliteParameter(); + + internal BsqliteCommandRequest ToRequest(BsqliteResultKind resultKind, int? rowLimit) + { + var parameters = Parameters.Count == 0 ? [] : new BsqliteParameterValue[Parameters.Count]; + for (var i = 0; i < parameters.Length; i++) + { + var parameter = Parameters[i]; + parameters[i] = new BsqliteParameterValue( + parameter.ParameterName, + BsqliteParameterBinding.ToTransportValue(parameter.Value)); + } + + return new BsqliteCommandRequest + { + CommandText = CommandText, + Parameters = parameters, + ResultKind = resultKind, + RowLimit = rowLimit, + }; + } +} + +/// The commands of a . +public sealed class BsqliteBatchCommandCollection : DbBatchCommandCollection +{ + private readonly List _commands = []; + + public override int Count => _commands.Count; + + public override bool IsReadOnly => false; + + public new BsqliteBatchCommand this[int index] + { + get => _commands[index]; + set => _commands[index] = value; + } + + public override void Add(DbBatchCommand item) => _commands.Add(Cast(item)); + + public override void Clear() => _commands.Clear(); + + public override bool Contains(DbBatchCommand item) => _commands.Contains(Cast(item)); + + public override void CopyTo(DbBatchCommand[] array, int arrayIndex) => ((ICollection)_commands.ToList()).CopyTo(array, arrayIndex); + + public override IEnumerator GetEnumerator() => _commands.GetEnumerator(); + + public override int IndexOf(DbBatchCommand item) => _commands.IndexOf(Cast(item)); + + public override void Insert(int index, DbBatchCommand item) => _commands.Insert(index, Cast(item)); + + public override bool Remove(DbBatchCommand item) => _commands.Remove(Cast(item)); + + public override void RemoveAt(int index) => _commands.RemoveAt(index); + + protected override DbBatchCommand GetBatchCommand(int index) => _commands[index]; + + protected override void SetBatchCommand(int index, DbBatchCommand batchCommand) + => _commands[index] = Cast(batchCommand); + + private static BsqliteBatchCommand Cast(DbBatchCommand command) + => command as BsqliteBatchCommand + ?? throw new InvalidCastException( + $"Expected {nameof(BsqliteBatchCommand)} but received {command?.GetType().Name ?? "null"}."); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteCommand.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteCommand.cs new file mode 100644 index 00000000000..bc41bf62853 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteCommand.cs @@ -0,0 +1,163 @@ +using System.Data; +using System.Data.Common; +using System.Diagnostics.CodeAnalysis; + +namespace Bit.Bsqlite.Data; + +/// A command executed through the connection's transport. +public sealed class BsqliteCommand : DbCommand +{ + private BsqliteConnection? _connection; + + public BsqliteCommand(BsqliteConnection connection) + => _connection = connection; + + public BsqliteCommand() + { + } + + [AllowNull] + public override string CommandText { get; set; } = string.Empty; + + /// + /// Accepted for ADO.NET compatibility but not enforced: the engine runs on a worker that + /// cannot be interrupted from the main thread. A stops the + /// caller waiting; the statement runs to completion in the worker. + /// + public override int CommandTimeout { get; set; } = 30; + + public override CommandType CommandType + { + get => CommandType.Text; + set + { + if (value != CommandType.Text) + { + throw new ArgumentException( + $"SQLite has no stored procedures or table-direct commands; only {nameof(CommandType.Text)} is supported.", + nameof(value)); + } + } + } + + public override bool DesignTimeVisible { get; set; } + + public override UpdateRowSource UpdatedRowSource { get; set; } + + protected override DbConnection? DbConnection + { + get => _connection; + set => _connection = (BsqliteConnection?)value; + } + + /// The parameters, typed. + public new BsqliteParameterCollection Parameters { get; } = []; + + protected override DbParameterCollection DbParameterCollection => Parameters; + + protected override DbTransaction? DbTransaction { get; set; } + + /// + /// The rowid of the most recent successful INSERT on the connection after this command + /// last executed. Equivalent to selecting last_insert_rowid() without the round trip. + /// + public long LastInsertRowId { get; private set; } + + /// No-op: the worker cannot be interrupted. Cancel the token passed to the async call. + public override void Cancel() + { + } + + /// No-op: statements are prepared and cached by the worker on first use. + public override void Prepare() + { + } + + protected override DbParameter CreateDbParameter() => new BsqliteParameter(); + + public override int ExecuteNonQuery() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(ExecuteNonQuery), nameof(ExecuteNonQueryAsync)); + + public override object? ExecuteScalar() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(ExecuteScalar), nameof(ExecuteScalarAsync)); + + protected override DbDataReader ExecuteDbDataReader(CommandBehavior behavior) + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(ExecuteReader), nameof(ExecuteReaderAsync)); + + public override async Task ExecuteNonQueryAsync(CancellationToken cancellationToken) + { + var result = await ExecuteCoreAsync(BsqliteResultKind.NonQuery, rowLimit: null, cancellationToken) + .ConfigureAwait(false); + return result.RecordsAffected; + } + + public override async Task ExecuteScalarAsync(CancellationToken cancellationToken) + { + var result = await ExecuteCoreAsync(BsqliteResultKind.Scalar, rowLimit: 1, cancellationToken) + .ConfigureAwait(false); + return result.Rows.Count > 0 && result.Rows[0].Length > 0 ? result.Rows[0][0] : null; + } + + protected override async Task ExecuteDbDataReaderAsync( + CommandBehavior behavior, + CancellationToken cancellationToken) + { + int? rowLimit = behavior.HasFlag(CommandBehavior.SchemaOnly) + ? 0 + : behavior.HasFlag(CommandBehavior.SingleRow) ? 1 : null; + + var result = await ExecuteCoreAsync(BsqliteResultKind.Reader, rowLimit, cancellationToken) + .ConfigureAwait(false); + return new BsqliteDataReader([result], behavior.HasFlag(CommandBehavior.CloseConnection) ? _connection : null); + } + + /// Builds the transport request this command would send, without sending it. + internal BsqliteCommandRequest ToRequest(BsqliteResultKind resultKind, int? rowLimit) + { + var parameters = Parameters.Count == 0 + ? [] + : new BsqliteParameterValue[Parameters.Count]; + + for (var i = 0; i < parameters.Length; i++) + { + var parameter = Parameters[i]; + parameters[i] = new BsqliteParameterValue( + parameter.ParameterName, + BsqliteParameterBinding.ToTransportValue(parameter.Value)); + } + + return new BsqliteCommandRequest + { + CommandText = CommandText, + Parameters = parameters, + ResultKind = resultKind, + RowLimit = rowLimit, + }; + } + + private async Task ExecuteCoreAsync( + BsqliteResultKind resultKind, + int? rowLimit, + CancellationToken cancellationToken) + { + if (_connection is null) + { + throw new InvalidOperationException("The command requires an open connection."); + } + + var results = await _connection + .ExecuteBatchAsync([ToRequest(resultKind, rowLimit)], cancellationToken) + .ConfigureAwait(false); + + if (results.Count == 0) + { + throw new InvalidOperationException("The transport returned no result for the command."); + } + + LastInsertRowId = _connection.LastInsertRowId; + return results[0]; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteConnection.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteConnection.cs new file mode 100644 index 00000000000..5ad13871683 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteConnection.cs @@ -0,0 +1,351 @@ +using System.Data; +using System.Data.Common; +using System.Diagnostics.CodeAnalysis; + +namespace Bit.Bsqlite.Data; + +/// +/// ADO.NET connection whose work is carried out by an . +/// +/// +/// Synchronous ADO.NET members throw : +/// the transport is asynchronous and the browser main thread cannot block on it. The opt-in strict +/// tier exists to lift that restriction. +/// +public sealed class BsqliteConnection : DbConnection +{ + private readonly IBsqliteTransport _transport; + private readonly HashSet _pendingTables = new(StringComparer.OrdinalIgnoreCase); + private readonly string _database; + private readonly string _connectionString; + private ConnectionState _state = ConnectionState.Closed; + private bool _disposed; + + public BsqliteConnection(IBsqliteTransport transport, string databaseName) + { + ArgumentNullException.ThrowIfNull(transport); + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + _transport = transport; + _database = databaseName; + _connectionString = $"Data Source={databaseName}"; + + // A write in another tab reaches this connection only through the transport, and the + // subscription has to exist before the first query so a live query created immediately + // after opening does not miss one. + _transport.TablesChanged += OnTransportTablesChanged; + } + + internal IBsqliteTransport Transport => _transport; + + internal BsqliteTransaction? CurrentTransaction { get; set; } + + /// + /// What the selected backend can honour. Defaults to unrestricted so an in-process transport + /// used without a provider still behaves like a full SQLite. + /// + public BsqliteRuntimeLimits RuntimeLimits { get; init; } = BsqliteRuntimeLimits.Unrestricted; + + /// + /// The rowid of the most recent successful INSERT executed through this connection, as + /// SQLite's last_insert_rowid() reports it after the last command completed. + /// + public long LastInsertRowId { get; private set; } + + /// + /// Raised after a committed write so live queries can re-run. Table-level: every listed name was + /// touched by the write, locally or - when the transport can hear them - in another tab. + /// + public event EventHandler? TablesChanged; + + /// + /// Raises for , for an application that + /// changed data in a way nothing else can see - a write on another connection, say. + /// + /// + /// The command layer does not need this: it reports its own writes through + /// , and writes from other tabs arrive through the transport. + /// + public void NotifyTablesChanged(IEnumerable tables) + { + ArgumentNullException.ThrowIfNull(tables); + TablesChanged?.Invoke(this, new BsqliteTablesChangedEventArgs([.. tables])); + } + + /// + /// Called after a batch whose SQL writes has run, with the tables the SQL names. + /// + /// + /// Nothing happens when the transport reports its own writes - it knows better than the SQL + /// text does, and it knows when the write is committed. Otherwise the tables are raised at once, + /// unless a is open, in which case they wait for its + /// outcome: a live query that re-ran mid-transaction would show data that may yet roll back, + /// and would compete with the writer for a DbContext that allows one operation at a + /// time. Transactions driven by raw BEGIN/COMMIT text are not seen here, and a + /// write inside one is reported as it happens. + /// + internal void OnCommandWrote(IEnumerable tables) + { + if (_transport.ReportsLocalWrites) + { + return; + } + + if (CurrentTransaction is null) + { + NotifyTablesChanged(tables); + return; + } + + _pendingTables.UnionWith(tables); + } + + /// Raises what the transaction that just committed had written. + internal void CommitPendingWrites() + { + if (_pendingTables.Count == 0) + { + return; + } + + var tables = new HashSet(_pendingTables, StringComparer.OrdinalIgnoreCase); + _pendingTables.Clear(); + TablesChanged?.Invoke(this, new BsqliteTablesChangedEventArgs(tables)); + } + + /// Forgets what a rolled-back transaction had written: nothing anyone can see changed. + internal void DiscardPendingWrites() => _pendingTables.Clear(); + + /// + /// Data Source=<database>, for display. The database is fixed by the transport + /// this connection was built on, so the string cannot be changed to point elsewhere. + /// + [AllowNull] + public override string ConnectionString + { + get => _connectionString; + set + { + if (!string.Equals(value, _connectionString, StringComparison.Ordinal)) + { + throw new NotSupportedException( + "A BsqliteConnection is bound to the database its transport opened; the " + + "connection string cannot be changed. Open another session for another database."); + } + } + } + + public override string Database => _database; + + public override string DataSource => _database; + + /// The SQLite version of the vendored engine; see engine/wa-sqlite.lock.props. + public override string ServerVersion => EntityFrameworkCore.BsqlitePclProvider.EngineVersion; + + public override ConnectionState State => _state; + + /// Batches are supported: one round trip runs every command in order. + public override bool CanCreateBatch => true; + + public override void Open() + => throw BsqliteSynchronousApiNotSupportedException.ForMember(nameof(Open), nameof(OpenAsync)); + + public override async Task OpenAsync(CancellationToken cancellationToken) + { + if (_state == ConnectionState.Open) + { + return; + } + + _state = ConnectionState.Connecting; + try + { + await _transport.OpenAsync(_database, cancellationToken).ConfigureAwait(false); + _state = ConnectionState.Open; + } + catch + { + _state = ConnectionState.Closed; + throw; + } + } + + /// + /// Marks the connection closed without touching the transport. + /// + /// + /// + /// EF opens and closes a connection around every operation, while opening a SQLite database in + /// the worker means setting up a VFS and reading page headers. Forwarding each ADO.NET close to + /// the transport would pay that cost per query, so the transport deliberately keeps the database + /// open and this is bookkeeping only. The transport's lifetime belongs to whoever created it, + /// not to an individual connection. + /// + /// + /// That also makes safe to leave synchronous. It is reached from + /// DbContext.Dispose(), which is far too common to forbid, and blocking on an async close + /// would deadlock the browser's single main thread - the exact failure this type exists to avoid. + /// + /// + public override void Close() => _state = ConnectionState.Closed; + + public override Task CloseAsync() + { + Close(); + return Task.CompletedTask; + } + + /// + /// Not supported: the transport owns exactly one database, so there is nothing to change to. + /// Renaming here would only make disagree with the worker. + /// + public override void ChangeDatabase(string databaseName) + => throw new NotSupportedException( + "A BsqliteConnection is bound to the database its transport opened. Open another " + + "session for another database."); + + /// Creates a command bound to this connection. + public new BsqliteCommand CreateCommand() => new(this); + + /// Creates a batch bound to this connection. + public new BsqliteBatch CreateBatch() => new(this); + + protected override DbCommand CreateDbCommand() => CreateCommand(); + + protected override DbBatch CreateDbBatch() => CreateBatch(); + + protected override DbTransaction BeginDbTransaction(IsolationLevel isolationLevel) + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(BeginTransaction), nameof(BeginTransactionAsync)); + + protected override async ValueTask BeginDbTransactionAsync( + IsolationLevel isolationLevel, + CancellationToken cancellationToken) + { + if (CurrentTransaction is not null) + { + throw new InvalidOperationException( + "A transaction is already active on this connection. SQLite does not nest " + + "transactions; use savepoints (DbTransaction.SaveAsync) inside the active one."); + } + + var transaction = new BsqliteTransaction(this, isolationLevel); + await transaction.BeginAsync(cancellationToken).ConfigureAwait(false); + CurrentTransaction = transaction; + return transaction; + } + + /// + /// Reads the whole database out as a SQLite file image, consistent as of the last committed + /// write, without closing the connection. + /// + /// The transport cannot snapshot an open database. + public Task ExportAsync(CancellationToken cancellationToken = default) + => _transport is IBsqliteSnapshotSource snapshots + ? snapshots.ExportAsync(cancellationToken) + : throw new NotSupportedException( + $"{_transport.GetType().Name} cannot export an open database. Close the session and " + + "use the storage provider's Admin.ExportAsync instead."); + + /// + /// Runs a batch of requests in one transport round trip and returns one result per request. + /// This is the lowest-level execution path; commands and batches are built on it. + /// + public async Task> ExecuteBatchAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(batch); + + // No check of State: Close is bookkeeping only (see its remarks), and a connection shared + // with EF reads as closed between EF's operations while its transport stays open. Whether + // the batch can run is the transport's call, and an unopened one says so. + for (var i = 0; i < batch.Count; i++) + { + BsqliteFeatureGuards.EnsureSupported(batch[i].CommandText, RuntimeLimits); + } + + IReadOnlyList results; + try + { + results = await _transport.ExecuteAsync(batch, cancellationToken).ConfigureAwait(false); + } + finally + { + // Also on failure: the statements before the one that failed have run, and in + // autocommit mode they have committed. + ReportWrites(batch); + } + + for (var i = 0; i < results.Count; i++) + { + if (results[i].LastInsertRowId != 0) + { + LastInsertRowId = results[i].LastInsertRowId; + } + } + + return results; + } + + /// + /// Derives the tables a batch wrote from its SQL, for a transport that cannot report them. + /// + private void ReportWrites(IReadOnlyList batch) + { + if (_transport.ReportsLocalWrites) + { + return; + } + + HashSet? tables = null; + for (var i = 0; i < batch.Count; i++) + { + if (BsqliteTableNames.LooksLikeWrite(batch[i].CommandText)) + { + (tables ??= new HashSet(StringComparer.OrdinalIgnoreCase)) + .UnionWith(BsqliteTableNames.Extract(batch[i].CommandText)); + } + } + + if (tables is { Count: > 0 }) + { + OnCommandWrote(tables); + } + } + + /// + /// Takes the cross-tab lock through the transport, or returns + /// when the transport has no way to coordinate with other tabs. + /// + internal async Task TryAcquireLockAsync(string name, CancellationToken cancellationToken) + => _transport is IBsqliteLockSource locks + ? await locks.AcquireLockAsync(name, cancellationToken).ConfigureAwait(false) + : null; + + /// Runs a statement that returns nothing, bypassing command plumbing. + internal async Task ExecuteInternalAsync(string sql, CancellationToken cancellationToken) + { + BsqliteFeatureGuards.EnsureSupported(sql, RuntimeLimits); + var request = new BsqliteCommandRequest { CommandText = sql }; + await _transport.ExecuteAsync([request], cancellationToken).ConfigureAwait(false); + } + + /// + /// Unsubscribes from the transport. The transport itself is not disposed - it belongs to + /// whoever created it, for the reason explains. + /// + protected override void Dispose(bool disposing) + { + if (disposing && !_disposed) + { + _disposed = true; + _transport.TablesChanged -= OnTransportTablesChanged; + } + + base.Dispose(disposing); + } + + private void OnTransportTablesChanged(object? sender, BsqliteTablesChangedEventArgs e) + => TablesChanged?.Invoke(this, e); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteDataReader.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteDataReader.cs new file mode 100644 index 00000000000..953704f4901 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteDataReader.cs @@ -0,0 +1,316 @@ +using System.Collections; +using System.Data.Common; +using System.Diagnostics.CodeAnalysis; +using System.Globalization; + +namespace Bit.Bsqlite.Data; + +/// Reads the materialized result sets returned by the transport. +/// +/// The transport delivers each result set in one round trip, so this reader walks in-memory +/// buffers rather than streaming. A reader over a batch holds one result set per command and +/// moves between them with . +/// +public sealed class BsqliteDataReader : DbDataReader +{ + private readonly IReadOnlyList _results; + private readonly BsqliteConnection? _closeConnection; + private BsqliteCommandResult _result; + private Dictionary? _ordinals; + private int _resultIndex; + private int _rowIndex = -1; + private bool _closed; + + internal BsqliteDataReader(IReadOnlyList results, BsqliteConnection? closeConnection = null) + { + _results = results.Count > 0 ? results : [new BsqliteCommandResult()]; + _result = _results[0]; + _closeConnection = closeConnection; + } + + public override int FieldCount => _result.ColumnNames.Count; + + public override bool HasRows => _result.Rows.Count > 0; + + public override bool IsClosed => _closed; + + /// Rows changed by the writes in the current result set's command. + public override int RecordsAffected => _result.RecordsAffected; + + public override int Depth => 0; + + /// The rowid of the last INSERT in the current result set's command. + public long LastInsertRowId => _result.LastInsertRowId; + + public override object this[int ordinal] => GetValue(ordinal); + + public override object this[string name] => GetValue(GetOrdinal(name)); + + public override bool Read() + { + if (_rowIndex + 1 >= _result.Rows.Count) + { + _rowIndex = _result.Rows.Count; + return false; + } + + _rowIndex++; + return true; + } + + public override Task ReadAsync(CancellationToken cancellationToken) + => Task.FromResult(Read()); + + public override bool NextResult() + { + if (_resultIndex + 1 >= _results.Count) + { + return false; + } + + _resultIndex++; + _result = _results[_resultIndex]; + _ordinals = null; + _rowIndex = -1; + return true; + } + + public override Task NextResultAsync(CancellationToken cancellationToken) + => Task.FromResult(NextResult()); + + public override void Close() + { + if (_closed) + { + return; + } + + _closed = true; + _closeConnection?.Close(); + } + + public override string GetName(int ordinal) => _result.ColumnNames[ordinal]; + + public override int GetOrdinal(string name) + { + ArgumentNullException.ThrowIfNull(name); + + if (_ordinals is null) + { + var names = _result.ColumnNames; + _ordinals = new Dictionary(names.Count, StringComparer.OrdinalIgnoreCase); + for (var i = names.Count - 1; i >= 0; i--) + { + // First occurrence wins for duplicate names, as every ADO.NET provider does. + _ordinals[names[i]] = i; + } + } + + return _ordinals.TryGetValue(name, out var ordinal) + ? ordinal + : throw new IndexOutOfRangeException($"No column named '{name}'."); + } + + public override string GetDataTypeName(int ordinal) => _result.ColumnTypes.Count > ordinal + ? _result.ColumnTypes[ordinal] ?? "BLOB" + : "BLOB"; + + /// The CLR type of the column, as a schema question rather than a per-row one. + /// + /// Answered from the result set rather than from the current row, for two reasons. It must not + /// require a positioned reader - callers read schema before the first Read - and it must + /// never answer , which is what asking a NULL cell for its type would give. + /// The first non-null value in the column is the most honest answer available; failing that the + /// column's reported type decides, by SQLite's own affinity rules. + /// + [return: DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicFields | DynamicallyAccessedMemberTypes.PublicProperties)] + [UnconditionalSuppressMessage("Trimming", "IL2073", + Justification = "A cell decodes to long, double, string or byte[] - framework types whose public members the trimmer never removes.")] + public override Type GetFieldType(int ordinal) + { + foreach (var row in _result.Rows) + { + if (ordinal < row.Length && row[ordinal] is { } value) + { + return value.GetType(); + } + } + + return AffinityOf(ordinal < _result.ColumnTypes.Count ? _result.ColumnTypes[ordinal] : null); + } + + /// + /// SQLite's column-affinity rules, which are the only thing a declared type is good for. + /// + /// + /// The worker reports storage classes rather than declared types - its engine is built with + /// SQLITE_OMIT_DECLTYPE - so both spellings have to land in the same place: "INTEGER" + /// matches the INT rule either way, and so on down the list. + /// + [return: DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicFields | DynamicallyAccessedMemberTypes.PublicProperties)] + private static Type AffinityOf(string? declaredType) + { + if (string.IsNullOrEmpty(declaredType)) + { + return typeof(object); + } + + if (declaredType.Contains("INT", StringComparison.OrdinalIgnoreCase)) + { + return typeof(long); + } + + if (declaredType.Contains("CHAR", StringComparison.OrdinalIgnoreCase) + || declaredType.Contains("CLOB", StringComparison.OrdinalIgnoreCase) + || declaredType.Contains("TEXT", StringComparison.OrdinalIgnoreCase)) + { + return typeof(string); + } + + if (declaredType.Contains("BLOB", StringComparison.OrdinalIgnoreCase)) + { + return typeof(byte[]); + } + + return typeof(double); + } + + public override object GetValue(int ordinal) => CurrentRow[ordinal] ?? DBNull.Value; + + public override int GetValues(object[] values) + { + ArgumentNullException.ThrowIfNull(values); + + var row = CurrentRow; + var count = Math.Min(values.Length, row.Length); + for (var i = 0; i < count; i++) + { + values[i] = row[i] ?? DBNull.Value; + } + + return count; + } + + public override bool IsDBNull(int ordinal) => CurrentRow[ordinal] is null; + + public override Task IsDBNullAsync(int ordinal, CancellationToken cancellationToken) + => Task.FromResult(IsDBNull(ordinal)); + + public override bool GetBoolean(int ordinal) => RequireValue(ordinal) switch + { + bool value => value, + long value => value != 0, + string text => !(text is "0" || text.Equals("false", StringComparison.OrdinalIgnoreCase)), + var other => Convert.ToBoolean(other, CultureInfo.InvariantCulture), + }; + + public override byte GetByte(int ordinal) => Convert.ToByte(RequireValue(ordinal), CultureInfo.InvariantCulture); + + public override char GetChar(int ordinal) => RequireValue(ordinal) switch + { + string { Length: > 0 } text => text[0], + var other => Convert.ToChar(other, CultureInfo.InvariantCulture), + }; + + public override DateTime GetDateTime(int ordinal) => RequireValue(ordinal) switch + { + DateTime value => value, + string text => DateTime.Parse(text, CultureInfo.InvariantCulture, DateTimeStyles.RoundtripKind), + var other => Convert.ToDateTime(other, CultureInfo.InvariantCulture), + }; + + public override decimal GetDecimal(int ordinal) => RequireValue(ordinal) switch + { + decimal value => value, + string text => decimal.Parse(text, NumberStyles.Number, CultureInfo.InvariantCulture), + var other => Convert.ToDecimal(other, CultureInfo.InvariantCulture), + }; + + public override double GetDouble(int ordinal) => Convert.ToDouble(RequireValue(ordinal), CultureInfo.InvariantCulture); + + public override float GetFloat(int ordinal) => Convert.ToSingle(RequireValue(ordinal), CultureInfo.InvariantCulture); + + public override Guid GetGuid(int ordinal) => RequireValue(ordinal) switch + { + Guid value => value, + string text => Guid.Parse(text), + byte[] bytes => new Guid(bytes), + var other => throw new InvalidCastException($"Cannot convert {other.GetType()} to Guid."), + }; + + public override short GetInt16(int ordinal) => Convert.ToInt16(RequireValue(ordinal), CultureInfo.InvariantCulture); + + public override int GetInt32(int ordinal) => Convert.ToInt32(RequireValue(ordinal), CultureInfo.InvariantCulture); + + public override long GetInt64(int ordinal) => Convert.ToInt64(RequireValue(ordinal), CultureInfo.InvariantCulture); + + public override string GetString(int ordinal) => Convert.ToString(RequireValue(ordinal), CultureInfo.InvariantCulture)!; + + public override T GetFieldValue(int ordinal) + => BsqliteParameterBinding.FromTransportValue(IsDBNull(ordinal) ? null : RequireValue(ordinal)); + + public override Task GetFieldValueAsync(int ordinal, CancellationToken cancellationToken) + => Task.FromResult(GetFieldValue(ordinal)); + + public override long GetBytes(int ordinal, long dataOffset, byte[]? buffer, int bufferOffset, int length) + { + var source = RequireValue(ordinal) switch + { + byte[] bytes => bytes, + string text => System.Text.Encoding.UTF8.GetBytes(text), + var other => throw new InvalidCastException($"Cannot read {other.GetType()} as bytes."), + }; + + if (buffer is null) + { + return source.Length; + } + + // Clamped: reading past the end of a blob is how a caller discovers where the end is, so it + // returns nothing rather than throwing out of Array.Copy on a negative count. + if (dataOffset >= source.Length) + { + return 0; + } + + var count = (int)Math.Clamp(Math.Min(length, source.Length - dataOffset), 0, buffer.Length - bufferOffset); + Array.Copy(source, dataOffset, buffer, bufferOffset, count); + return count; + } + + public override long GetChars(int ordinal, long dataOffset, char[]? buffer, int bufferOffset, int length) + { + var source = GetString(ordinal); + if (buffer is null) + { + return source.Length; + } + + if (dataOffset >= source.Length) + { + return 0; + } + + var count = (int)Math.Clamp(Math.Min(length, source.Length - dataOffset), 0, buffer.Length - bufferOffset); + source.CopyTo((int)dataOffset, buffer, bufferOffset, count); + return count; + } + + public override Stream GetStream(int ordinal) => new MemoryStream(RequireValue(ordinal) switch + { + byte[] bytes => bytes, + string text => System.Text.Encoding.UTF8.GetBytes(text), + var other => throw new InvalidCastException($"Cannot read {other.GetType()} as a stream."), + }, writable: false); + + public override TextReader GetTextReader(int ordinal) => new StringReader(GetString(ordinal)); + + public override IEnumerator GetEnumerator() => new DbEnumerator(this, closeReader: false); + + private object?[] CurrentRow => _rowIndex >= 0 && _rowIndex < _result.Rows.Count + ? _result.Rows[_rowIndex] + : throw new InvalidOperationException("No row is available. Call Read first."); + + private object RequireValue(int ordinal) + => CurrentRow[ordinal] ?? throw new InvalidCastException($"Column {ordinal} is NULL."); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteFeatureGuards.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteFeatureGuards.cs new file mode 100644 index 00000000000..e73622eecd2 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteFeatureGuards.cs @@ -0,0 +1,78 @@ +using System.Text.RegularExpressions; + +namespace Bit.Bsqlite.Data; + +/// +/// SQL restrictions that are either product-wide or declared by the selected backend's capabilities. +/// +/// +/// WAL is product-wide: no web VFS can provide the shared memory it needs. ATTACH and +/// PRAGMA page_size are capability-gated so a backend that cannot honour them fails here +/// rather than corrupting silently. +/// +public static partial class BsqliteFeatureGuards +{ + /// + /// Throws when asks for something this product cannot honour. + /// + public static void EnsureSupported(string? commandText) + => EnsureSupported(commandText, BsqliteRuntimeLimits.Unrestricted); + + /// + /// Throws when asks for something + /// cannot honour. + /// + public static void EnsureSupported(string? commandText, BsqliteRuntimeLimits limits) + { + if (string.IsNullOrWhiteSpace(commandText)) + { + return; + } + + // Every guarded statement contains one of these words. Checking for them first keeps the + // regular expressions off the hot path, where almost no SQL mentions either. + if (!commandText.Contains("pragma", StringComparison.OrdinalIgnoreCase) + && !commandText.Contains("attach", StringComparison.OrdinalIgnoreCase)) + { + return; + } + + if (WalJournalMode().IsMatch(commandText)) + { + throw new BsqliteException( + "WAL mode is not available in the browser: WebAssembly has no shared-memory " + + "primitives for it, and no web VFS implements it. Bsqlite uses DELETE or " + + "TRUNCATE journaling. Concurrent reads, where a backend offers them, are a VFS " + + "capability rather than a journal-mode setting."); + } + + if (!limits.SupportsMultiDatabaseTransactions && Attach().IsMatch(commandText)) + { + throw new BsqliteException( + "ATTACH is not available on this storage backend: it cannot run a transaction " + + "that spans more than one database. Open each database on its own connection."); + } + + if (!limits.CanChangePageSize && PageSizeAssignment().IsMatch(commandText)) + { + throw new BsqliteException( + "PRAGMA page_size cannot be changed on this storage backend: the page size is " + + "fixed to the backend's block size."); + } + } + + [GeneratedRegex( + @"pragma\s+journal_mode\s*=\s*['""`]?wal\b", + RegexOptions.IgnoreCase | RegexOptions.CultureInvariant)] + private static partial Regex WalJournalMode(); + + [GeneratedRegex( + @"(?:^|;)\s*attach(?:\s+database)?\b", + RegexOptions.IgnoreCase | RegexOptions.CultureInvariant)] + private static partial Regex Attach(); + + [GeneratedRegex( + @"pragma\s+page_size\s*=", + RegexOptions.IgnoreCase | RegexOptions.CultureInvariant)] + private static partial Regex PageSizeAssignment(); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameter.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameter.cs new file mode 100644 index 00000000000..434e26768be --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameter.cs @@ -0,0 +1,49 @@ +using System.Data; +using System.Data.Common; +using System.Diagnostics.CodeAnalysis; + +namespace Bit.Bsqlite.Data; + +/// A parameter carried to the engine by the transport. +public sealed class BsqliteParameter : DbParameter +{ + public BsqliteParameter() + { + } + + public BsqliteParameter(string name, object? value) + { + ParameterName = name; + Value = value; + } + + public override DbType DbType { get; set; } = DbType.Object; + + public override ParameterDirection Direction + { + get => ParameterDirection.Input; + set + { + if (value != ParameterDirection.Input) + { + throw new ArgumentException("SQLite parameters are input-only.", nameof(value)); + } + } + } + + public override bool IsNullable { get; set; } = true; + + [AllowNull] + public override string ParameterName { get; set; } = string.Empty; + + public override int Size { get; set; } + + [AllowNull] + public override string SourceColumn { get; set; } = string.Empty; + + public override bool SourceColumnNullMapping { get; set; } + + public override object? Value { get; set; } + + public override void ResetDbType() => DbType = DbType.Object; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameterBinding.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameterBinding.cs new file mode 100644 index 00000000000..93295a7f32d --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameterBinding.cs @@ -0,0 +1,165 @@ +using System.Globalization; + +namespace Bit.Bsqlite.Data; + +/// +/// Converts CLR values onto the SQLite representation Microsoft.Data.Sqlite would bind, and back. +/// +/// +/// +/// Dates, times, and GUIDs each have more than one reasonable SQLite storage form. The wire format +/// therefore refuses them, so this layer - the equivalent of SqliteParameter.Bind - owns the +/// choice. The formats here are Microsoft.Data.Sqlite's defaults (TEXT, not Julian-day REAL or +/// binary GUIDs), which is also what EF Core's SQLite type mapping expects when the store type is +/// TEXT. +/// +/// +/// Values the wire format already knows how to encode (integers, text, blobs, decimals, bools) pass +/// through unchanged on the way in. On the way out, EF materializes through +/// DbDataReader.GetFieldValue<T>, so TEXT cells must be parsed back into those CLR types. +/// +/// +public static class BsqliteParameterBinding +{ + /// + /// Returns a value the transport can encode, converting dates, times, and GUIDs to TEXT. + /// + public static object? ToTransportValue(object? value) => value switch + { + DateTime dateTime => dateTime.ToString(@"yyyy\-MM\-dd HH\:mm\:ss.FFFFFFF", CultureInfo.InvariantCulture), + DateTimeOffset dateTimeOffset => dateTimeOffset.ToString( + @"yyyy\-MM\-dd HH\:mm\:ss.FFFFFFFzzz", + CultureInfo.InvariantCulture), + DateOnly dateOnly => dateOnly.ToString(@"yyyy\-MM\-dd", CultureInfo.InvariantCulture), + TimeOnly timeOnly => timeOnly.Ticks % 10_000_000 == 0 + ? timeOnly.ToString(@"HH:mm:ss", CultureInfo.InvariantCulture) + : timeOnly.ToString(@"HH:mm:ss.fffffff", CultureInfo.InvariantCulture), + TimeSpan timeSpan => timeSpan.ToString("c"), + Guid guid => guid.ToString().ToUpperInvariant(), + _ => value, + }; + + /// + /// Converts a transport cell back into . + /// + public static T FromTransportValue(object? value) + { + if (value is null or DBNull) + { + return default!; + } + + if (value is T typed) + { + return typed; + } + + return (T)FromTransportValue(value, typeof(T)); + } + + private static object FromTransportValue(object value, Type clrType) + { + var target = Nullable.GetUnderlyingType(clrType) ?? clrType; + + if (target == typeof(TimeSpan)) + { + return value switch + { + string text => TimeSpan.Parse(text, CultureInfo.InvariantCulture), + double days => TimeSpan.FromDays(days), + float days => TimeSpan.FromDays(days), + long days => TimeSpan.FromDays(days), + int days => TimeSpan.FromDays(days), + _ => TimeSpan.Parse( + Convert.ToString(value, CultureInfo.InvariantCulture)!, + CultureInfo.InvariantCulture), + }; + } + + if (target == typeof(DateTimeOffset)) + { + return value switch + { + DateTime dateTime => new DateTimeOffset(dateTime, TimeSpan.Zero), + string text => DateTimeOffset.Parse( + text, + CultureInfo.InvariantCulture, + DateTimeStyles.AssumeUniversal), + _ => DateTimeOffset.Parse( + Convert.ToString(value, CultureInfo.InvariantCulture)!, + CultureInfo.InvariantCulture, + DateTimeStyles.AssumeUniversal), + }; + } + + if (target == typeof(DateOnly)) + { + return value switch + { + DateTime dateTime => DateOnly.FromDateTime(dateTime), + string text => DateOnly.Parse(text, CultureInfo.InvariantCulture), + _ => DateOnly.Parse( + Convert.ToString(value, CultureInfo.InvariantCulture)!, + CultureInfo.InvariantCulture), + }; + } + + if (target == typeof(TimeOnly)) + { + return value switch + { + string text => TimeOnly.Parse(text, CultureInfo.InvariantCulture), + _ => TimeOnly.Parse( + Convert.ToString(value, CultureInfo.InvariantCulture)!, + CultureInfo.InvariantCulture), + }; + } + + if (target == typeof(DateTime)) + { + return value switch + { + string text => DateTime.Parse(text, CultureInfo.InvariantCulture, DateTimeStyles.RoundtripKind), + _ => Convert.ToDateTime(value, CultureInfo.InvariantCulture), + }; + } + + if (target == typeof(Guid)) + { + return value switch + { + string text => Guid.Parse(text), + byte[] bytes => new Guid(bytes), + _ => Guid.Parse(Convert.ToString(value, CultureInfo.InvariantCulture)!), + }; + } + + if (target == typeof(bool)) + { + return value switch + { + string text => text is not "0" + && !text.Equals("false", StringComparison.OrdinalIgnoreCase), + _ => Convert.ToInt64(value, CultureInfo.InvariantCulture) != 0, + }; + } + + if (target == typeof(decimal)) + { + return value switch + { + string text => decimal.Parse(text, NumberStyles.Number, CultureInfo.InvariantCulture), + _ => Convert.ToDecimal(value, CultureInfo.InvariantCulture), + }; + } + + if (target.IsEnum) + { + return value is string name + ? Enum.Parse(target, name, ignoreCase: true) + : Enum.ToObject(target, value); + } + + return Convert.ChangeType(value, target, CultureInfo.InvariantCulture)!; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameterCollection.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameterCollection.cs new file mode 100644 index 00000000000..5f4dfdfaef3 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteParameterCollection.cs @@ -0,0 +1,120 @@ +using System.Collections; +using System.Data.Common; + +namespace Bit.Bsqlite.Data; + +/// Parameter collection for . +/// +/// Names match with or without their sigil, because callers write p0 as often as +/// @p0 and SQLite accepts @, $, and : alike. +/// +public sealed class BsqliteParameterCollection : DbParameterCollection, IEnumerable +{ + private readonly List _parameters = []; + + public override int Count => _parameters.Count; + + public override object SyncRoot { get; } = new(); + + public new BsqliteParameter this[int index] + { + get => _parameters[index]; + set => _parameters[index] = value; + } + + public new BsqliteParameter this[string parameterName] + { + get => _parameters[RequireIndex(parameterName)]; + set => _parameters[RequireIndex(parameterName)] = value; + } + + /// Adds a parameter by name and value and returns it. + public BsqliteParameter Add(string parameterName, object? value) + { + var parameter = new BsqliteParameter(parameterName, value); + _parameters.Add(parameter); + return parameter; + } + + /// + public BsqliteParameter AddWithValue(string parameterName, object? value) => Add(parameterName, value); + + public override int Add(object value) + { + _parameters.Add(Cast(value)); + return _parameters.Count - 1; + } + + public override void AddRange(Array values) + { + ArgumentNullException.ThrowIfNull(values); + foreach (var value in values) + { + Add(value!); + } + } + + public override void Clear() => _parameters.Clear(); + + public override bool Contains(object value) => _parameters.Contains(Cast(value)); + + public override bool Contains(string value) => IndexOf(value) >= 0; + + public override void CopyTo(Array array, int index) => ((ICollection)_parameters).CopyTo(array, index); + + public override IEnumerator GetEnumerator() => _parameters.GetEnumerator(); + + IEnumerator IEnumerable.GetEnumerator() => _parameters.GetEnumerator(); + + public override int IndexOf(object value) => _parameters.IndexOf(Cast(value)); + + public override int IndexOf(string parameterName) + { + if (parameterName is null) + { + return -1; + } + + var index = _parameters.FindIndex(p => string.Equals(p.ParameterName, parameterName, StringComparison.Ordinal)); + if (index >= 0) + { + return index; + } + + var bare = StripSigil(parameterName); + return _parameters.FindIndex(p => string.Equals(StripSigil(p.ParameterName), bare, StringComparison.Ordinal)); + } + + public override void Insert(int index, object value) => _parameters.Insert(index, Cast(value)); + + public override void Remove(object value) => _parameters.Remove(Cast(value)); + + public override void RemoveAt(int index) => _parameters.RemoveAt(index); + + public override void RemoveAt(string parameterName) => RemoveAt(RequireIndex(parameterName)); + + protected override DbParameter GetParameter(int index) => _parameters[index]; + + protected override DbParameter GetParameter(string parameterName) => _parameters[RequireIndex(parameterName)]; + + protected override void SetParameter(int index, DbParameter value) => _parameters[index] = Cast(value); + + protected override void SetParameter(string parameterName, DbParameter value) + => _parameters[RequireIndex(parameterName)] = Cast(value); + + private int RequireIndex(string parameterName) + { + var index = IndexOf(parameterName); + return index >= 0 + ? index + : throw new IndexOutOfRangeException($"No parameter named '{parameterName}'."); + } + + private static string StripSigil(string name) + => name.Length > 0 && name[0] is '@' or '$' or ':' ? name[1..] : name; + + private static BsqliteParameter Cast(object value) + => value as BsqliteParameter + ?? throw new InvalidCastException( + $"Expected {nameof(BsqliteParameter)} but received {value?.GetType().Name ?? "null"}."); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteRuntimeLimits.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteRuntimeLimits.cs new file mode 100644 index 00000000000..970998c54c9 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteRuntimeLimits.cs @@ -0,0 +1,26 @@ +namespace Bit.Bsqlite.Data; + +/// +/// What the current storage backend can honour. The core enforces these generically so a new +/// backend never needs a special case in the command layer. +/// +public readonly record struct BsqliteRuntimeLimits +{ + /// No backend-specific restrictions beyond the product-wide WAL ban. + public static BsqliteRuntimeLimits Unrestricted { get; } = new() + { + SupportsMultiDatabaseTransactions = true, + CanChangePageSize = true, + }; + + /// + /// Whether a transaction may span ATTACHed databases. OPFSCoopSyncVFS cannot. + /// + public bool SupportsMultiDatabaseTransactions { get; init; } + + /// + /// Whether PRAGMA page_size may be changed. Block-oriented backends pin the page size + /// to their block size. + /// + public bool CanChangePageSize { get; init; } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTableNames.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTableNames.cs new file mode 100644 index 00000000000..ef1b5a5f84c --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTableNames.cs @@ -0,0 +1,94 @@ +using System.Text.RegularExpressions; + +namespace Bit.Bsqlite.Data; + +/// +/// Pulls table names out of SQL so a live query can re-run when any of them change. +/// +/// +/// +/// Table-level and deliberately coarse: comments and quoted identifiers that contain SQL keywords +/// can produce extra names, which only causes extra re-queries, never a missed update. +/// +/// +/// This is the fallback. When the transport reports what the engine actually wrote - the worker +/// does, through sqlite3_update_hook - the connection uses that instead, which also covers +/// triggers and cascading deletes that no regular expression can see. +/// +/// +/// The worker carries the same two patterns in JavaScript (bit-bsqlite-worker.js) for +/// the tables its update hook cannot see - DDL, and the truncate-optimised DELETE FROM t - +/// so a change here is a change there. +/// +/// +public static partial class BsqliteTableNames +{ + /// + /// The tables reads or writes. Accepts the identifier quoting SQLite + /// does ("t", `t`, [t]), a schema prefix (main."t"), the + /// conflict clause an UPDATE may carry (UPDATE OR REPLACE t), and the objects + /// DDL names (CREATE VIEW v, CREATE TEMP TABLE t, CREATE INDEX i ON t). + /// + public static IReadOnlySet Extract(string? sql) + { + var names = new HashSet(StringComparer.OrdinalIgnoreCase); + if (string.IsNullOrWhiteSpace(sql)) + { + return names; + } + + foreach (Match match in TableName().Matches(sql)) + { + var name = match.Groups[1].Value; + if (!name.StartsWith("sqlite_", StringComparison.OrdinalIgnoreCase)) + { + names.Add(name); + } + } + + return names; + } + + /// + /// Whether contains a statement that writes. + /// + /// + /// Every statement is considered, not just the first. A batch is routinely + /// BEGIN; INSERT …; COMMIT; or several statements EF sent together, and looking only at + /// the leading keyword would call all of those reads - which is a missed live-query update, the + /// one failure mode this heuristic is not allowed to have. A leading common table expression + /// is skipped for the same reason: WITH … INSERT INTO is a write. So are leading + /// comments, which migrations and hand-written scripts both carry. + /// + public static bool LooksLikeWrite(string? sql) + => !string.IsNullOrWhiteSpace(sql) && Write().IsMatch(sql); + + /// + /// Whether contains DDL - CREATE, DROP, or ALTER - which + /// changes what a query can see without changing a row the update hook would report. + /// + public static bool LooksLikeSchemaChange(string? sql) + => !string.IsNullOrWhiteSpace(sql) && SchemaChange().IsMatch(sql); + + [GeneratedRegex( + @"\b(?:from|join|into|update(?:\s+or\s+(?:rollback|abort|replace|fail|ignore))?|table|view|trigger" + + @"|index\s+(?:if\s+(?:not\s+)?exists\s+)?[\w""`\[\]]+\s+on)\s+" + + @"(?:if\s+(?:not\s+)?exists\s+)?(?:(?:temp|temporary)\s+)?" + + @"(?:[""`\[]?[A-Za-z_]\w*[""`\]]?\s*\.\s*)?" + + @"[""`\[]?([A-Za-z_]\w*)", + RegexOptions.IgnoreCase | RegexOptions.CultureInvariant)] + private static partial Regex TableName(); + + // A block comment is spelled out (/\*[^*]*\*+(?:[^/*][^*]*\*+)*/) rather than matched lazily + // (/\*.*?\*/): a lazy body can end at any */, so a run of */ /* that never reaches a write + // backtracks through every way of splitting it - exponential in its length. + [GeneratedRegex( + @"(?:^|;)\s*(?:--[^\n]*\n\s*|/\*[^*]*\*+(?:[^/*][^*]*\*+)*/\s*)*(?:with\b.*?)?\b(?:insert|update|delete|replace|create|drop|alter)\b", + RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Singleline)] + private static partial Regex Write(); + + [GeneratedRegex( + @"(?:^|;)\s*(?:--[^\n]*\n\s*|/\*[^*]*\*+(?:[^/*][^*]*\*+)*/\s*)*(?:create|drop|alter)\b", + RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Singleline)] + private static partial Regex SchemaChange(); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTablesChangedEventArgs.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTablesChangedEventArgs.cs new file mode 100644 index 00000000000..3f491f7f3b2 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTablesChangedEventArgs.cs @@ -0,0 +1,8 @@ +namespace Bit.Bsqlite.Data; + +/// The tables a completed write touched. +public sealed class BsqliteTablesChangedEventArgs(IReadOnlyCollection tables) : EventArgs +{ + public IReadOnlySet Tables { get; } = tables as IReadOnlySet + ?? new HashSet(tables, StringComparer.OrdinalIgnoreCase); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTransaction.cs b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTransaction.cs new file mode 100644 index 00000000000..465c4ca0b1c --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/BsqliteTransaction.cs @@ -0,0 +1,212 @@ +using System.Data; +using System.Data.Common; +using System.Globalization; + +namespace Bit.Bsqlite.Data; + +/// A transaction driven by explicit BEGIN/COMMIT/ROLLBACK statements on the transport. +/// +/// +/// (the default) begins with BEGIN IMMEDIATE, which +/// takes the write lock up front so a transaction that later writes cannot fail part-way with +/// SQLITE_BUSY - the same choice Microsoft.Data.Sqlite makes. Any other level begins +/// deferred. SQLite has no further isolation levels; the enum value is recorded, not interpreted. +/// +/// +/// Disposing without committing rolls back, as every ADO.NET provider does. The asynchronous +/// dispose does so on the transport; the synchronous one cannot wait, so it issues the rollback +/// and returns - the transport serializes requests, so it still lands before the next command. +/// +/// +/// A cancellation token is honoured only before a statement is sent. Once BEGIN, +/// COMMIT or ROLLBACK is on its way, the worker runs it whatever .NET stops waiting +/// for, so abandoning the wait would leave the engine in a transaction no object here owns - holding +/// the write lock against every other tab, and swallowing every later write into it. +/// +/// +public sealed class BsqliteTransaction : DbTransaction +{ + private readonly BsqliteConnection _connection; + private int _savepointCounter; + private bool _completed; + + internal BsqliteTransaction(BsqliteConnection connection, IsolationLevel isolationLevel) + { + _connection = connection; + IsolationLevel = isolationLevel == IsolationLevel.Unspecified + ? IsolationLevel.Serializable + : isolationLevel; + } + + public override IsolationLevel IsolationLevel { get; } + + protected override DbConnection DbConnection => _connection; + + /// Savepoints are supported, so EF Core nests SaveChanges inside this safely. + public override bool SupportsSavepoints => true; + + /// Whether the transaction has been committed or rolled back. + public bool IsCompleted => _completed; + + internal async Task BeginAsync(CancellationToken cancellationToken) + { + cancellationToken.ThrowIfCancellationRequested(); + + // Not cancellable once sent; see the remarks. + await _connection.ExecuteInternalAsync( + IsolationLevel == IsolationLevel.Serializable ? "BEGIN IMMEDIATE" : "BEGIN", + CancellationToken.None).ConfigureAwait(false); + _connection.DiscardPendingWrites(); + } + + public override void Commit() + => throw BsqliteSynchronousApiNotSupportedException.ForMember(nameof(Commit), nameof(CommitAsync)); + + public override void Rollback() + => throw BsqliteSynchronousApiNotSupportedException.ForMember(nameof(Rollback), nameof(RollbackAsync)); + + public override Task CommitAsync(CancellationToken cancellationToken = default) + => CompleteAsync("COMMIT", cancellationToken); + + public override Task RollbackAsync(CancellationToken cancellationToken = default) + => CompleteAsync("ROLLBACK", cancellationToken); + + public override void Save(string savepointName) + => throw BsqliteSynchronousApiNotSupportedException.ForMember(nameof(Save), nameof(SaveAsync)); + + public override void Release(string savepointName) + => throw BsqliteSynchronousApiNotSupportedException.ForMember(nameof(Release), nameof(ReleaseAsync)); + + public override void Rollback(string savepointName) + => throw BsqliteSynchronousApiNotSupportedException.ForMember(nameof(Rollback), nameof(RollbackAsync)); + + public override Task SaveAsync(string savepointName, CancellationToken cancellationToken = default) + { + ThrowIfCompleted(); + return _connection.ExecuteInternalAsync($"SAVEPOINT {QuoteSavepoint(savepointName)}", cancellationToken); + } + + public override Task ReleaseAsync(string savepointName, CancellationToken cancellationToken = default) + { + ThrowIfCompleted(); + return _connection.ExecuteInternalAsync($"RELEASE SAVEPOINT {QuoteSavepoint(savepointName)}", cancellationToken); + } + + public override Task RollbackAsync(string savepointName, CancellationToken cancellationToken = default) + { + ThrowIfCompleted(); + return _connection.ExecuteInternalAsync($"ROLLBACK TO SAVEPOINT {QuoteSavepoint(savepointName)}", cancellationToken); + } + + /// Creates a savepoint with a generated name and returns that name. + public async Task SaveAsync(CancellationToken cancellationToken = default) + { + var name = "bit_bsqlite_sp" + (++_savepointCounter).ToString(CultureInfo.InvariantCulture); + await SaveAsync(name, cancellationToken).ConfigureAwait(false); + return name; + } + + public override async ValueTask DisposeAsync() + { + if (!_completed) + { + try + { + await RollbackAsync().ConfigureAwait(false); + } + catch (Exception) + { + // The transaction may already be gone - a failed COMMIT rolls back on the engine's + // side - and disposal is already the unwinding path: the original reason for + // leaving the block is the more useful error, so this one is not thrown over it. + Detach(); + _connection.DiscardPendingWrites(); + } + } + + await base.DisposeAsync().ConfigureAwait(false); + } + + protected override void Dispose(bool disposing) + { + if (disposing && !_completed) + { + // Cannot wait here; see the remarks. The request is queued in order behind whatever is + // in flight, and the connection is detached immediately so a new BEGIN is allowed. + Detach(); + _connection.DiscardPendingWrites(); + _ = RollbackQuietlyAsync(); + } + + base.Dispose(disposing); + } + + private async Task RollbackQuietlyAsync() + { + try + { + await _connection.ExecuteInternalAsync("ROLLBACK", CancellationToken.None).ConfigureAwait(false); + } + catch (Exception) + { + // Nothing can observe this failure; the engine rolls back on close regardless. + } + } + + private async Task CompleteAsync(string statement, CancellationToken cancellationToken) + { + ThrowIfCompleted(); + + // Before anything changes, so a cancelled commit leaves a transaction the caller can still + // roll back or dispose; after this point the statement is sent and runs, as the remarks say. + cancellationToken.ThrowIfCancellationRequested(); + + try + { + await _connection.ExecuteInternalAsync(statement, CancellationToken.None).ConfigureAwait(false); + } + catch + { + // Whether COMMIT succeeded or SQLite rolled it back on failure, this object is finished, + // and nothing it wrote is reported as committed. + Detach(); + _connection.DiscardPendingWrites(); + throw; + } + + Detach(); + + // Live queries hear about the transaction's writes only now, and only if it committed. + if (statement == "COMMIT") + { + _connection.CommitPendingWrites(); + } + else + { + _connection.DiscardPendingWrites(); + } + } + + private void Detach() + { + _completed = true; + if (ReferenceEquals(_connection.CurrentTransaction, this)) + { + _connection.CurrentTransaction = null; + } + } + + private void ThrowIfCompleted() + { + if (_completed) + { + throw new InvalidOperationException("The transaction has already been committed or rolled back."); + } + } + + private static string QuoteSavepoint(string savepointName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(savepointName); + return "\"" + savepointName.Replace("\"", "\"\"", StringComparison.Ordinal) + "\""; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteConcurrencyException.cs b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteConcurrencyException.cs new file mode 100644 index 00000000000..b46925e5b12 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteConcurrencyException.cs @@ -0,0 +1,10 @@ +namespace Bit.Bsqlite.Data; + +/// The database is locked or busy (SQLITE_BUSY / SQLITE_LOCKED). +public sealed class BsqliteConcurrencyException : BsqliteException +{ + public BsqliteConcurrencyException(string message, int? sqliteErrorCode = 5) + : base(message, sqliteErrorCode) + { + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteConstraintException.cs b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteConstraintException.cs new file mode 100644 index 00000000000..301dc3d7fb4 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteConstraintException.cs @@ -0,0 +1,18 @@ +namespace Bit.Bsqlite.Data; + +/// +/// A constraint was violated (SQLITE_CONSTRAINT): unique, primary key, foreign key, check, +/// or not-null. +/// +/// +/// Raised as its own type because a duplicate key is the failure applications most often handle +/// deliberately - offering a retry, showing a validation message - and matching it by result code +/// in every catch block is the thing this type saves callers from. +/// +public sealed class BsqliteConstraintException : BsqliteException +{ + public BsqliteConstraintException(string message, int? sqliteErrorCode = 19) + : base(message, sqliteErrorCode) + { + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteCorruptDatabaseException.cs b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteCorruptDatabaseException.cs new file mode 100644 index 00000000000..ac13310591e --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteCorruptDatabaseException.cs @@ -0,0 +1,10 @@ +namespace Bit.Bsqlite.Data; + +/// The file image is not a usable SQLite database. +public sealed class BsqliteCorruptDatabaseException : BsqliteException +{ + public BsqliteCorruptDatabaseException(string message, int? sqliteErrorCode = 11) + : base(message, sqliteErrorCode) + { + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteException.cs b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteException.cs new file mode 100644 index 00000000000..9d60e8734d4 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteException.cs @@ -0,0 +1,39 @@ +using System.Data.Common; + +namespace Bit.Bsqlite.Data; + +/// +/// A failure reported by SQLite itself, carrying the result code the engine returned. +/// +/// +/// +/// The code is preserved deliberately: EF Core distinguishes a unique-constraint violation from a busy +/// database by it, and callers write catch blocks around specific codes. Without it, every +/// failure would be an indistinguishable string. +/// +/// +/// A , as every ADO.NET provider's error is - Microsoft.Data.Sqlite's +/// included - so code shared between a server and the browser handles both with the same +/// catch (DbException). +/// +/// +public class BsqliteException : DbException +{ + /// The engine's message, passed through unaltered. + /// + /// The SQLite result code, or when the failure came from the transport rather + /// than from the engine. + /// + public BsqliteException(string message, int? sqliteErrorCode = null) + : base(message) + => SqliteErrorCode = sqliteErrorCode; + + /// The SQLite result code, when the engine supplied one. + public int? SqliteErrorCode { get; } + + /// + /// for SQLITE_BUSY and SQLITE_LOCKED - another connection, + /// usually another tab, held the lock - which a retry may get past. + /// + public override bool IsTransient => (SqliteErrorCode & 0xFF) is 5 or 6; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteQuotaExceededException.cs b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteQuotaExceededException.cs new file mode 100644 index 00000000000..98313781a3b --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteQuotaExceededException.cs @@ -0,0 +1,10 @@ +namespace Bit.Bsqlite.Data; + +/// The origin is out of storage quota, or SQLite reported SQLITE_FULL. +public sealed class BsqliteQuotaExceededException : BsqliteException +{ + public BsqliteQuotaExceededException(string message, int? sqliteErrorCode = 13) + : base(message, sqliteErrorCode) + { + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteSynchronousApiNotSupportedException.cs b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteSynchronousApiNotSupportedException.cs new file mode 100644 index 00000000000..11ba69e4191 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Exceptions/BsqliteSynchronousApiNotSupportedException.cs @@ -0,0 +1,31 @@ +namespace Bit.Bsqlite.Data; + +/// +/// Thrown when a synchronous ADO.NET or EF Core API is used against the default (asynchronous) +/// tier, where the browser main thread cannot block on the transport. +/// +public sealed class BsqliteSynchronousApiNotSupportedException : NotSupportedException +{ + private BsqliteSynchronousApiNotSupportedException(string message) : base(message) + { + } + + /// + /// Builds the exception for , pointing the caller at + /// . + /// + /// + /// Public because the EF integration and third-party storage providers ship in their own + /// assemblies and must fail with this exact, recognisable error rather than one of their own. + /// + public static BsqliteSynchronousApiNotSupportedException ForMember( + string synchronousMember, + string asynchronousMember) + => new($""" + '{synchronousMember}' is not supported by Bsqlite's default tier, because the SQLite + engine runs asynchronously and the browser main thread cannot block waiting for it. + + Use '{asynchronousMember}' instead. In EF Core, prefer the async overloads: + SaveChangesAsync, ToListAsync, FirstOrDefaultAsync, and so on. + """); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/LiveQueries/BsqliteLiveQuery.cs b/src/Bsqlite/Bit.Bsqlite/Data/LiveQueries/BsqliteLiveQuery.cs new file mode 100644 index 00000000000..de8ce7695e8 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/LiveQueries/BsqliteLiveQuery.cs @@ -0,0 +1,301 @@ +using System.Runtime.CompilerServices; + +namespace Bit.Bsqlite.Data; + +/// Re-runs a query when watched tables are written. +public sealed class BsqliteLiveQuery : IBsqliteLiveQuery +{ + private readonly BsqliteConnection _connection; + private readonly Func> _execute; + private readonly HashSet _tables; + private readonly List _waiters = []; + private readonly Action? _onDispose; + private readonly Lock _refreshGate = new(); + + // One run of the query at a time, explicit or not: an EF query runs on a context that allows one + // operation at a time, and two runs finishing out of order would publish the older snapshot last. + private readonly SemaphoreSlim _runGate = new(1, 1); + private bool _refreshing; + private bool _refreshRequested; + private int _disposed; + + // Guarded by _waiters. Every published snapshot bumps the version, so an enumerator that was + // busy yielding while a refresh completed can tell it missed one and catch up rather than + // waiting for the next. + private long _version; + private T? _latest; + + /// How many times one notification's refresh is attempted before it is given up on. + private const int RefreshAttempts = 2; + + public BsqliteLiveQuery( + BsqliteConnection connection, + Func> execute, + IEnumerable tables) + : this(connection, execute, tables, onDispose: null) + { + } + + /// + /// As the public constructor, plus : runs once when the live query + /// is disposed, after it has stopped listening to the connection. Lets a caller that wired the + /// query to something else - the EF entry point subscribes to the context's SaveChanges events - + /// release that too. + /// + internal BsqliteLiveQuery( + BsqliteConnection connection, + Func> execute, + IEnumerable tables, + Action? onDispose) + { + ArgumentNullException.ThrowIfNull(connection); + ArgumentNullException.ThrowIfNull(execute); + ArgumentNullException.ThrowIfNull(tables); + + _connection = connection; + _execute = execute; + _tables = new HashSet(tables, StringComparer.OrdinalIgnoreCase); + _onDispose = onDispose; + _connection.TablesChanged += OnTablesChanged; + } + + /// + public event EventHandler? Changed; + + /// + public event EventHandler? RefreshFailed; + + /// + public T? Current { get; private set; } + + /// The tables whose writes trigger a refresh. Empty means every write does. + public IReadOnlySet Tables => _tables; + + /// + public async Task RefreshAsync(CancellationToken cancellationToken = default) + { + ObjectDisposedException.ThrowIf(_disposed != 0, this); + + await _runGate.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + // Again: disposal may have come while this run waited its turn. + ObjectDisposedException.ThrowIf(_disposed != 0, this); + + var snapshot = await _execute(cancellationToken).ConfigureAwait(false); + Current = snapshot; + + // Enumerators first, so a Changed handler that throws cannot starve them of a result. + Publish(snapshot); + Changed?.Invoke(this, snapshot); + return snapshot; + } + finally + { + _runGate.Release(); + } + } + + /// + /// + /// Yields the current result first, then every result of a later refresh. A refresh that + /// completes while the consumer is still processing the previous item is not lost: the next + /// iteration yields its snapshot at once instead of waiting for another write. + /// + public async IAsyncEnumerable WithCancellation([EnumeratorCancellation] CancellationToken cancellationToken) + { + var first = await RefreshAsync(cancellationToken).ConfigureAwait(false); + + // Captured before yielding: the consumer may take a while to come back for the next item. + long seenVersion; + lock (_waiters) + { + seenVersion = _version; + } + + yield return first; + + while (!cancellationToken.IsCancellationRequested) + { + TaskCompletionSource? waiter = null; + bool disposed; + lock (_waiters) + { + // Checked under the lock DisposeAsync cancels the waiters under: a waiter added after + // that would never be completed, and a consumer that was busy when the query was + // disposed would come back to wait forever. + disposed = _disposed != 0; + if (!disposed && _version == seenVersion) + { + waiter = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + _waiters.Add(waiter); + } + } + + if (disposed) + { + yield break; + } + + if (waiter is not null) + { + using var registration = cancellationToken.Register(() => waiter.TrySetCanceled(cancellationToken)); + try + { + await waiter.Task.ConfigureAwait(false); + } + catch (OperationCanceledException) + { + yield break; + } + } + + // Always the newest snapshot, whatever woke us: an intermediate one is stale by now. + T snapshot; + lock (_waiters) + { + seenVersion = _version; + snapshot = _latest!; + } + + yield return snapshot; + } + } + + /// + public ValueTask DisposeAsync() + { + if (Interlocked.Exchange(ref _disposed, 1) != 0) + { + return ValueTask.CompletedTask; + } + + _connection.TablesChanged -= OnTablesChanged; + lock (_waiters) + { + foreach (var waiter in _waiters) + { + waiter.TrySetCanceled(); + } + + _waiters.Clear(); + } + + _onDispose?.Invoke(); + return ValueTask.CompletedTask; + } + + private void OnTablesChanged(object? sender, BsqliteTablesChangedEventArgs e) + { + if (_tables.Count > 0 && !_tables.Overlaps(e.Tables)) + { + return; + } + + RequestRefresh(); + } + + /// + /// Asks for a refresh, coalescing with one already in flight. + /// + /// + /// Refreshes are single-file because they usually run against an EF DbContext, which + /// rejects a second concurrent operation. Overlapping notifications - a burst of writes, or a + /// local write and another tab's arriving together - would otherwise throw and be swallowed, + /// leaving the query showing data that is one write out of date. A request that arrives while a + /// refresh is running therefore sets a flag the running refresh honours before it finishes, + /// so the last state of the database is always the state that is read. + /// + private void RequestRefresh() + { + lock (_refreshGate) + { + _refreshRequested = true; + if (_refreshing) + { + return; + } + + _refreshing = true; + } + + _ = RefreshQuietlyAsync(); + } + + private async Task RefreshQuietlyAsync() + { + // Yield so the write that raised TablesChanged can finish (EF SaveChanges is still + // consuming the insert result when the in-process transport notifies). + await Task.Yield(); + + while (true) + { + lock (_refreshGate) + { + if (!_refreshRequested || _disposed != 0) + { + _refreshing = false; + return; + } + + _refreshRequested = false; + } + + for (var attempt = 0; ; attempt++) + { + try + { + await RefreshAsync().ConfigureAwait(false); + break; + } + catch (Exception) when (_disposed != 0) + { + // Disposed mid-refresh: nobody is listening for the result, or for its failure. + break; + } + catch (Exception) when (attempt < RefreshAttempts - 1) + { + // Almost always the write that triggered this is still finishing and the + // context is busy. Give it the rest of the turn and read again. + await Task.Yield(); + } + catch (Exception ex) + { + // A failed refresh must not take down the write that triggered it, but it must + // not vanish either: there is no caller to throw to, so it is reported. + ReportRefreshFailure(ex); + break; + } + } + } + } + + private void ReportRefreshFailure(Exception exception) + { + try + { + RefreshFailed?.Invoke(this, exception); + } + catch (Exception) + { + // A handler that throws is not allowed to stop later refreshes. + } + } + + private void Publish(T snapshot) + { + List waiters; + lock (_waiters) + { + _version++; + _latest = snapshot; + waiters = [.. _waiters]; + _waiters.Clear(); + } + + foreach (var waiter in waiters) + { + waiter.TrySetResult(); + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/LiveQueries/IBsqliteLiveQuery.cs b/src/Bsqlite/Bit.Bsqlite/Data/LiveQueries/IBsqliteLiveQuery.cs new file mode 100644 index 00000000000..c9aed4cf52f --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/LiveQueries/IBsqliteLiveQuery.cs @@ -0,0 +1,36 @@ +namespace Bit.Bsqlite.Data; + +/// +/// A query that re-executes when any table it reads is written, locally or in another tab. +/// +/// +/// Re-runs happen after a write is committed - a rolled-back write is never shown - and are +/// coalesced: a burst of writes produces one refresh reading the final state, not one per write. A +/// refresh that fails, typically because the DbContext it queries is still busy with the +/// write that triggered it, is retried once and then reported on rather +/// than allowed to fault the writer; call to read again explicitly. +/// +public interface IBsqliteLiveQuery : IAsyncDisposable +{ + /// Raised with the new result after every re-run, including an explicit refresh. + event EventHandler? Changed; + + /// + /// Raised when a write-triggered re-run fails even after its retry. An explicit + /// throws instead. + /// + event EventHandler? RefreshFailed; + + /// The result of the latest run, or before the first. + T? Current { get; } + + /// Runs the query now, publishes the result, and returns it. + Task RefreshAsync(CancellationToken cancellationToken = default); + + /// + /// The current result first, then the result of every later re-run, until + /// is cancelled or the query is disposed. A consumer that + /// falls behind receives the newest result when it catches up, not each one it missed. + /// + IAsyncEnumerable WithCancellation(CancellationToken cancellationToken); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteCommandRequest.cs b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteCommandRequest.cs new file mode 100644 index 00000000000..ea89905857e --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteCommandRequest.cs @@ -0,0 +1,18 @@ +namespace Bit.Bsqlite.Data; + +/// A single statement plus its parameters. +public sealed record BsqliteCommandRequest +{ + /// The SQL to execute. May contain multiple statements. + public required string CommandText { get; init; } + + public IReadOnlyList Parameters { get; init; } = []; + + public BsqliteResultKind ResultKind { get; init; } = BsqliteResultKind.NonQuery; + + /// + /// The most rows to return, or for all of them. Zero returns the column + /// list without stepping the statement, which is what CommandBehavior.SchemaOnly asks for. + /// + public int? RowLimit { get; init; } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteCommandResult.cs b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteCommandResult.cs new file mode 100644 index 00000000000..2858c392c83 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteCommandResult.cs @@ -0,0 +1,34 @@ +namespace Bit.Bsqlite.Data; + +/// The outcome of one . +public sealed record BsqliteCommandResult +{ + /// Column names, present when the request asked for a reader. + public IReadOnlyList ColumnNames { get; init; } = []; + + /// + /// SQLite type per column, as reported by . + /// + /// + /// Transports differ in what they can report, and the difference is not hidden because it is not + /// fixable: the vendored engine is built with SQLITE_OMIT_DECLTYPE, so declared types do not + /// exist in it at all and the worker reports the storage class of the first row instead. The + /// in-process transport reports true declared types, so it is the more generous of the two here. + /// EF Core is unaffected - it materializes from the model, not from this - but a caller reading + /// GetDataTypeName directly should expect TEXT where a server would say + /// decimal(18,2). + /// + public IReadOnlyList ColumnTypes { get; init; } = []; + + /// Rows, each of length. + public IReadOnlyList Rows { get; init; } = []; + + /// + /// Rows changed by the statements that write, summed. Read-only statements do not contribute, + /// so an INSERT followed by a SELECT reports the insert alone. + /// + public int RecordsAffected { get; init; } + + /// The rowid of the most recent successful INSERT on the connection. + public long LastInsertRowId { get; init; } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteParameterValue.cs b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteParameterValue.cs new file mode 100644 index 00000000000..c595c5d75f5 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteParameterValue.cs @@ -0,0 +1,4 @@ +namespace Bit.Bsqlite.Data; + +/// A parameter value in transport form. +public sealed record BsqliteParameterValue(string Name, object? Value); diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteResultKind.cs b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteResultKind.cs new file mode 100644 index 00000000000..245c1dceba6 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Transport/BsqliteResultKind.cs @@ -0,0 +1,14 @@ +namespace Bit.Bsqlite.Data; + +/// How much of the result the caller intends to consume. +public enum BsqliteResultKind +{ + /// Row count only. + NonQuery, + + /// First column of the first row. + Scalar, + + /// Full result set. + Reader, +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteLockSource.cs b/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteLockSource.cs new file mode 100644 index 00000000000..82359c2ae51 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteLockSource.cs @@ -0,0 +1,25 @@ +namespace Bit.Bsqlite.Data; + +/// +/// A transport that can hold a named lock shared with every other tab that opens the same database. +/// +/// +/// +/// Optional, like . SQLite's own locks last one transaction at +/// most, and some work has to keep other tabs out for longer: applying EF migrations reads the history +/// table and then changes the schema, and two tabs doing that at once both see the same migrations as +/// pending. +/// +/// +/// The worker transport takes the lock through the Web Locks API, which the browser releases when the +/// tab holding it closes or crashes - so a lock is never left behind for the other tabs to wait on. +/// +/// +public interface IBsqliteLockSource +{ + /// + /// Waits until nobody else holds for the open database, then takes it. + /// Disposing the result releases it. + /// + Task AcquireLockAsync(string name, CancellationToken cancellationToken = default); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteSnapshotSource.cs b/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteSnapshotSource.cs new file mode 100644 index 00000000000..7cef5fb5e7d --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteSnapshotSource.cs @@ -0,0 +1,16 @@ +namespace Bit.Bsqlite.Data; + +/// +/// A transport that can hand out a consistent snapshot of the open database as a SQLite file image +/// without closing it. +/// +/// +/// Optional because not every transport can: the worker takes the snapshot through +/// sqlite3_serialize, which copies every committed page under the engine's own lock, so it +/// is safe while the database is in use. +/// +public interface IBsqliteSnapshotSource +{ + /// Reads the whole database out as a SQLite file image. + Task ExportAsync(CancellationToken cancellationToken = default); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteTransport.cs b/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteTransport.cs new file mode 100644 index 00000000000..25dfdb5102b --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Data/Transport/IBsqliteTransport.cs @@ -0,0 +1,63 @@ +namespace Bit.Bsqlite.Data; + +/// +/// Executes SQLite work on behalf of the ADO.NET layer. Implementations decide where the engine +/// actually lives - a web worker, an in-process engine, or a test double. +/// +/// +/// The contract is deliberately coarse: one round trip per command batch, because in the browser +/// the dominant cost is crossing the interop boundary, not executing SQL. +/// +public interface IBsqliteTransport : IAsyncDisposable +{ + /// Opens , creating it when absent. + Task OpenAsync(string databaseName, CancellationToken cancellationToken = default); + + /// + /// Releases the database handle. Called by whoever owns the transport, not by + /// - see its Close documentation for why. + /// + Task CloseAsync(CancellationToken cancellationToken = default); + + /// Executes a batch and returns one result per request, in order. + Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default); + + /// + /// Raised when committed writes touched tables in the open database: another tab's writes + /// always, and this transport's own writes when is true. + /// + /// + /// + /// subscribes to this and re-raises it as + /// , which is what makes a live query re-run. + /// A transport that reports its own writes must do so only once they are committed - a live + /// query in another tab that re-ran on an uncommitted write would read the old data and then + /// never hear about the commit. + /// + /// + /// Implemented as a no-op by default: a transport with no way to hear about other writers - a + /// test double, say - is still a complete transport, and the connection covers its local + /// writes from the SQL it sends. + /// + /// + event EventHandler? TablesChanged + { + add { } + remove { } + } + + /// + /// Whether this transport raises for the writes it performs itself, + /// so the connection must not. + /// + /// + /// A transport that sits next to the engine can be exact about what changed - SQLite's update + /// hook names every table a row landed in, including through triggers and cascades - and about + /// when, since it sees the commit. One that cannot leaves this , and + /// derives the tables from the SQL it sends instead. + /// Reporting on both sides would re-run every live query twice per write. + /// + bool ReportsLocalWrites => false; +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BitBsqlite.EntityFrameworkCore.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BitBsqlite.EntityFrameworkCore.cs new file mode 100644 index 00000000000..572c4e66e50 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BitBsqlite.EntityFrameworkCore.cs @@ -0,0 +1,82 @@ +using System.Diagnostics.CodeAnalysis; +using Bit.Bsqlite.EntityFrameworkCore; +using Bit.Bsqlite.Interop; +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.DependencyInjection.Extensions; + +namespace Microsoft.Extensions.DependencyInjection; + +/// Registers EF Core contexts on top of AddBitBsqliteServices. +public static partial class BitBsqlite +{ + /// + /// Registers (and + /// ) for a context bound to a registered + /// - the only one, or the one + /// names. Migrations are applied + /// once per session unless says otherwise. + /// + /// + /// + /// builder.Services.AddBitBsqliteServices(sqlite => sqlite + /// .AddStorageProvider<BsqliteOpfsStorageProvider>() + /// .UseStorage(s => s.Prefer("opfs")) + /// .AddDatabase("app.db")); + /// + /// builder.Services.AddBitBsqliteDbContext<AppDbContext>(o => + /// o.OnFirstUse = (db, ct) => DemoData.SeedIfEmptyAsync(db, ct)); + /// + /// // In a component: + /// await using var db = await Factory.CreateDbContextAsync(); + /// + /// + public static IServiceCollection AddBitBsqliteDbContext<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors | DynamicallyAccessedMemberTypes.PublicProperties)] TContext>( + this IServiceCollection services, + Action>? configure = null) + where TContext : DbContext + { + ArgumentNullException.ThrowIfNull(services); + + var options = new BsqliteDbContextOptions(); + configure?.Invoke(options); + + services.TryAddSingleton(options); + // Scoped like the database it is bound to. + services.TryAddScoped>(sp => + { + var contextOptions = sp.GetRequiredService>(); + return new BsqliteDbContextFactory( + sp, + DatabaseFor(sp, typeof(TContext), contextOptions.DatabaseName), + contextOptions); + }); + services.TryAddScoped>(sp => + sp.GetRequiredService>()); + + return services; + } + + private static BsqliteDatabase DatabaseFor(IServiceProvider services, Type contextType, string? databaseName) + { + if (databaseName is not null) + { + return services.GetKeyedService(databaseName) + ?? throw new InvalidOperationException( + $"{contextType.Name} is bound to the database '{databaseName}', which is not " + + $"registered. Add it with AddDatabase(\"{databaseName}\")."); + } + + // Falling back to whichever was registered last would bind the context - and run its + // migrations - against a database that is not its own. + var registered = services.GetServices().ToArray(); + if (registered.Length > 1) + { + throw new InvalidOperationException( + $"Several databases are registered ({string.Join(", ", registered.Select(r => $"'{r.DatabaseName}'"))}), " + + $"so {contextType.Name} cannot tell which one it belongs to. Name it in " + + $"AddBitBsqliteDbContext<{contextType.Name}>(o => o.DatabaseName = \"...\")."); + } + + return services.GetRequiredService(); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDatabaseCreator.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDatabaseCreator.cs new file mode 100644 index 00000000000..1fbab7d96bb --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDatabaseCreator.cs @@ -0,0 +1,180 @@ +using Bit.Bsqlite.Data; +using Microsoft.EntityFrameworkCore.Storage; + +namespace Bit.Bsqlite.EntityFrameworkCore; + +/// +/// An whose schema operations are genuinely asynchronous. +/// +/// +/// +/// RelationalDatabaseCreator.CreateAsync has no async implementation of its own - it calls +/// the synchronous Create(), which opens the connection synchronously. That is why both +/// EnsureCreatedAsync and MigrateAsync fail against the default tier unless this +/// service is replaced. Everything above it (CreateTablesAsync, the migration command +/// executor) is already async, so overriding the four abstract members is enough. +/// +/// +/// Delete empties the schema - views, then tables, then VACUUM - rather than asking +/// the storage provider to forget the file. The transport keeps the database open across EF's +/// per-operation close, so there is no file to unlink while a session holds it; +/// BsqliteDatabase.WhileClosedAsync plus the provider's admin API is the way to wipe +/// persistent storage outright. +/// +/// +public sealed class BsqliteDatabaseCreator( + RelationalDatabaseCreatorDependencies dependencies, + IRawSqlCommandBuilder rawSqlCommandBuilder) + : RelationalDatabaseCreator(dependencies) +{ + // EF's own version omits the NOT LIKE clause, because its Delete() removes the database file + // outright and so never has to reason about leftovers. Deletion here drops objects instead, and + // SQLite's internal bookkeeping tables (sqlite_sequence, created by AUTOINCREMENT) cannot be + // dropped - counting them would make an emptied database look populated. + private const string CountTablesSql = """ + SELECT COUNT(*) FROM "sqlite_master" + WHERE "type" = 'table' AND "rootpage" IS NOT NULL AND "name" NOT LIKE 'sqlite_%' + """; + + // Views first, then tables: a view over a dropped table is harmless to SQLite but not to the + // next EnsureCreated, which would trip over the leftover. Indexes and triggers go with their + // table. + private const string ListObjectsSql = """ + SELECT "type", "name" FROM "sqlite_master" + WHERE "type" IN ('table', 'view') AND "name" NOT LIKE 'sqlite_%' + ORDER BY CASE "type" WHEN 'view' THEN 0 ELSE 1 END, "rowid" + """; + + /// + /// Always true: opening is what brings a SQLite database into being, and whether any backing + /// storage exists is the storage provider's question, not EF's. + /// + public override bool Exists() => true; + + public override Task ExistsAsync(CancellationToken cancellationToken = default) + => Task.FromResult(true); + + public override void Create() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(Create), nameof(CreateAsync)); + + public override async Task CreateAsync(CancellationToken cancellationToken = default) + { + await Dependencies.Connection.OpenAsync(cancellationToken).ConfigureAwait(false); + await Dependencies.Connection.CloseAsync().ConfigureAwait(false); + } + + public override bool HasTables() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(HasTables), nameof(HasTablesAsync)); + + public override async Task HasTablesAsync(CancellationToken cancellationToken = default) + { + var count = await ExecuteScalarAsync(CountTablesSql, cancellationToken).ConfigureAwait(false); + return Convert.ToInt64(count) != 0L; + } + + public override void Delete() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(Delete), nameof(DeleteAsync)); + + public override async Task DeleteAsync(CancellationToken cancellationToken = default) + { + var objects = new List<(string Type, string Name)>(); + + await Dependencies.Connection.OpenAsync(cancellationToken).ConfigureAwait(false); + try + { + var command = rawSqlCommandBuilder.Build(ListObjectsSql); + await using var reader = await command + .ExecuteReaderAsync(CreateParameterObject(), cancellationToken) + .ConfigureAwait(false); + + while (await reader.DbDataReader.ReadAsync(cancellationToken).ConfigureAwait(false)) + { + objects.Add((reader.DbDataReader.GetString(0), reader.DbDataReader.GetString(1))); + } + } + finally + { + await Dependencies.Connection.CloseAsync().ConfigureAwait(false); + } + + if (objects.Count == 0) + { + return; + } + + // With foreign keys on - and the worker turns them on - DROP TABLE runs an implicit DELETE + // that a referencing table with rows can veto, and the drop order out of sqlite_master is + // creation order, parents first. Enforcement is suspended for the drops and put back to + // whatever it was, since the connection lives on after this. + var foreignKeys = Convert.ToInt64( + await ExecuteScalarAsync("PRAGMA foreign_keys", cancellationToken).ConfigureAwait(false)); + if (foreignKeys != 0L) + { + await ExecuteNonQueryAsync("PRAGMA foreign_keys=OFF", cancellationToken).ConfigureAwait(false); + } + + try + { + foreach (var (type, name) in objects) + { + var keyword = type == "view" ? "VIEW" : "TABLE"; + await ExecuteNonQueryAsync($"DROP {keyword} IF EXISTS \"{name.Replace("\"", "\"\"", StringComparison.Ordinal)}\"", cancellationToken) + .ConfigureAwait(false); + } + } + finally + { + if (foreignKeys != 0L) + { + await ExecuteNonQueryAsync("PRAGMA foreign_keys=ON", CancellationToken.None).ConfigureAwait(false); + } + } + + // Returns the freed pages to the storage backend, which would otherwise keep the file at + // its high-water mark. + await ExecuteNonQueryAsync("VACUUM", cancellationToken).ConfigureAwait(false); + } + + private async Task ExecuteScalarAsync(string sql, CancellationToken cancellationToken) + { + await Dependencies.Connection.OpenAsync(cancellationToken).ConfigureAwait(false); + try + { + return await rawSqlCommandBuilder + .Build(sql) + .ExecuteScalarAsync(CreateParameterObject(), cancellationToken) + .ConfigureAwait(false); + } + finally + { + await Dependencies.Connection.CloseAsync().ConfigureAwait(false); + } + } + + private async Task ExecuteNonQueryAsync(string sql, CancellationToken cancellationToken) + { + await Dependencies.Connection.OpenAsync(cancellationToken).ConfigureAwait(false); + try + { + await rawSqlCommandBuilder + .Build(sql) + .ExecuteNonQueryAsync(CreateParameterObject(), cancellationToken) + .ConfigureAwait(false); + } + finally + { + await Dependencies.Connection.CloseAsync().ConfigureAwait(false); + } + } + + private RelationalCommandParameterObject CreateParameterObject() + => new( + Dependencies.Connection, + parameterValues: null, + readerColumns: null, + context: Dependencies.CurrentContext.Context, + logger: Dependencies.CommandLogger); +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDbContextFactory.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDbContextFactory.cs new file mode 100644 index 00000000000..db08269c4e1 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDbContextFactory.cs @@ -0,0 +1,195 @@ +using System.Diagnostics.CodeAnalysis; +using Bit.Bsqlite.Interop; +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.DependencyInjection; + +namespace Bit.Bsqlite.EntityFrameworkCore; + +/// +/// Creates contexts bound to the application's , opening it on +/// first use and - when asked - applying migrations once per session. +/// +/// A context with a constructor taking . +/// +/// Opening a database is asynchronous in the browser, so the factory's main entry point is +/// . The synchronous CreateDbContext works only once an +/// earlier has opened the session and run the first-use step on +/// it, and says so otherwise. +/// +public interface IBsqliteDbContextFactory<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors | DynamicallyAccessedMemberTypes.PublicProperties)] TContext> : IDbContextFactory + where TContext : DbContext +{ + /// The database every context this factory creates is bound to. + BsqliteDatabase Database { get; } + + /// Opens the database if needed, runs the configured first-use step, and returns a context. + new Task CreateDbContextAsync(CancellationToken cancellationToken = default); +} + +/// How prepares the schema the first time a session opens. +public enum BsqliteSchemaStrategy +{ + /// Do nothing; the application manages the schema itself. + None, + + /// Call Database.MigrateAsync() once per session, applying pending migrations. + Migrate, + + /// Call Database.EnsureCreatedAsync() once per session. + EnsureCreated, +} + +/// Options for . +public sealed class BsqliteDbContextOptions + where TContext : DbContext +{ + /// + /// The registered database (AddDatabase) the contexts are bound to. Required when more + /// than one is registered; with exactly one, that one is used. + /// + public string? DatabaseName { get; set; } + + /// What to do to the schema the first time a context is created per session. + public BsqliteSchemaStrategy Schema { get; set; } = BsqliteSchemaStrategy.Migrate; + + /// Further configuration of the options every context is built with. + public Action>? Configure { get; set; } + + /// + /// Runs after the schema step, once per session, with a context of its own: the place to seed + /// demo data or run integrity checks. + /// + public Func? OnFirstUse { get; set; } +} + +internal sealed class BsqliteDbContextFactory<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors | DynamicallyAccessedMemberTypes.PublicProperties)] TContext> : IBsqliteDbContextFactory + where TContext : DbContext +{ + /// + /// The cross-tab lock the first-use step runs under. Not the migrator's own lock, which + /// MigrateAsync takes inside it on EF Core 9 and later: a Web Lock is not re-entrant. + /// + internal const string FirstUseLockName = "ef-first-use"; + + private readonly IServiceProvider _services; + private readonly BsqliteDbContextOptions _options; + private readonly SemaphoreSlim _gate = new(1, 1); + private BsqliteSession? _preparedFor; + + public BsqliteDbContextFactory( + IServiceProvider services, + BsqliteDatabase database, + BsqliteDbContextOptions options) + { + _services = services; + Database = database; + _options = options; + database.Closing += OnClosingAsync; + } + + public BsqliteDatabase Database { get; } + + private bool NeedsPreparation + => _options.Schema != BsqliteSchemaStrategy.None || _options.OnFirstUse is not null; + + public TContext CreateDbContext() + { + var session = Database.Session; + + // A session opened some other way - BsqliteDatabase.OpenAsync, say - has not had its schema + // migrated or its first-use step run, and a context on it would fail with "no such table" or + // query a schema one deploy out of date. + if (session is null || (!ReferenceEquals(_preparedFor, session) && NeedsPreparation)) + { + throw new InvalidOperationException( + $"The database '{Database.Name}' is not ready for {typeof(TContext).Name}. Await " + + "CreateDbContextAsync() at least once first: opening the database and applying " + + "migrations are asynchronous in the browser, and the synchronous overload cannot " + + "wait for them."); + } + + return Build(session); + } + + public async Task CreateDbContextAsync(CancellationToken cancellationToken = default) + { + var session = await Database.OpenAsync(cancellationToken).ConfigureAwait(false); + + if (!ReferenceEquals(_preparedFor, session)) + { + await _gate.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + if (!ReferenceEquals(_preparedFor, session)) + { + await PrepareAsync(session, cancellationToken).ConfigureAwait(false); + _preparedFor = session; + } + } + finally + { + _gate.Release(); + } + } + + return Build(session); + } + + Task IDbContextFactory.CreateDbContextAsync(CancellationToken cancellationToken) + => CreateDbContextAsync(cancellationToken); + + // EF Core marks migrations RequiresDynamicCode for NativeAOT, which builds the design-time model it cannot + // emit code for. Blazor WebAssembly - the host this factory opens databases in - is not NativeAOT: even an + // AOT-compiled WebAssembly app keeps the interpreter for code generated at run time, and migrations run there. + [UnconditionalSuppressMessage("AOT", "IL3050", + Justification = "Runs on Blazor WebAssembly, whose AOT keeps the interpreter for dynamic code; NativeAOT is not a host of this library.")] + private async Task PrepareAsync(BsqliteSession session, CancellationToken cancellationToken) + { + if (!NeedsPreparation) + { + return; + } + + // Every tab prepares its own session over the same file. Without this, two tabs opening at + // once would both seed an empty database, and - on EF Core 8, whose migrator takes no lock - + // both apply the same pending migration. + await using var exclusive = await session.Connection + .TryAcquireLockAsync(FirstUseLockName, cancellationToken) + .ConfigureAwait(false); + + await using var context = Build(session); + + switch (_options.Schema) + { + case BsqliteSchemaStrategy.Migrate: + await context.Database.MigrateAsync(cancellationToken).ConfigureAwait(false); + break; + + case BsqliteSchemaStrategy.EnsureCreated: + await context.Database.EnsureCreatedAsync(cancellationToken).ConfigureAwait(false); + break; + } + + if (_options.OnFirstUse is { } firstUse) + { + await firstUse(context, cancellationToken).ConfigureAwait(false); + } + } + + private TContext Build(BsqliteSession session) + { + var builder = new DbContextOptionsBuilder() + .UseApplicationServiceProvider(_services) + .UseBitBsqlite(session.Connection); + + _options.Configure?.Invoke(_services, builder); + + return ActivatorUtilities.CreateInstance(_services, builder.Options); + } + + private Task OnClosingAsync() + { + _preparedFor = null; + return Task.CompletedTask; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDbContextOptionsExtensions.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDbContextOptionsExtensions.cs new file mode 100644 index 00000000000..430bff55120 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteDbContextOptionsExtensions.cs @@ -0,0 +1,110 @@ +using System.Data.Common; +using Bit.Bsqlite.EntityFrameworkCore; +using Bit.Bsqlite.Interop; +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Migrations; +using Microsoft.EntityFrameworkCore.Storage; + +namespace Microsoft.EntityFrameworkCore; + +/// +/// Registers the stock SQLite provider against a Bsqlite owns, +/// replacing the two services that assume synchronous I/O. +/// +public static class BsqliteDbContextOptionsExtensions +{ + /// + /// Configures the context to use Bsqlite through . + /// + /// + /// This is UseSqlite(connection) plus replacements for + /// and . SQL generation + /// is the stock provider's; the replacements exist only so EnsureCreatedAsync and + /// MigrateAsync do not call DbConnection.Open(). + /// + public static DbContextOptionsBuilder UseBitBsqlite( + this DbContextOptionsBuilder optionsBuilder, + DbConnection connection, + Action? sqliteOptionsAction = null) + => optionsBuilder.UseBitBsqlite(connection, contextOwnsConnection: false, sqliteOptionsAction); + + /// + /// Configures the context to use Bsqlite through . + /// + /// + /// When is , disposing the + /// context disposes the connection. Leave it when the connection comes + /// from a BsqliteSession - the session owns the transport, and an ADO.NET close is + /// bookkeeping only. + /// + public static DbContextOptionsBuilder UseBitBsqlite( + this DbContextOptionsBuilder optionsBuilder, + DbConnection connection, + bool contextOwnsConnection, + Action? sqliteOptionsAction = null) + { + ArgumentNullException.ThrowIfNull(optionsBuilder); + ArgumentNullException.ThrowIfNull(connection); + + // EF Core 10 reads new SqliteConnection().ServerVersion while compiling queries. + // That requires a PCL provider; WASM has no e_sqlite3 bundle. + BsqlitePcl.EnsureInitialized(); + optionsBuilder.UseSqlite(connection, contextOwnsConnection, sqliteOptionsAction); + ReplaceSyncBoundServices(optionsBuilder); + return optionsBuilder; + } + + /// + /// Configures the context to use the connection of an open . + /// The session keeps owning the worker; disposing the context does not close it. + /// + public static DbContextOptionsBuilder UseBitBsqlite( + this DbContextOptionsBuilder optionsBuilder, + BsqliteSession session, + Action? sqliteOptionsAction = null) + { + ArgumentNullException.ThrowIfNull(session); + return optionsBuilder.UseBitBsqlite(session.Connection, contextOwnsConnection: false, sqliteOptionsAction); + } + + /// + public static DbContextOptionsBuilder UseBitBsqlite( + this DbContextOptionsBuilder optionsBuilder, + BsqliteSession session, + Action? sqliteOptionsAction = null) + where TContext : DbContext + { + ((DbContextOptionsBuilder)optionsBuilder).UseBitBsqlite(session, sqliteOptionsAction); + return optionsBuilder; + } + + /// + public static DbContextOptionsBuilder UseBitBsqlite( + this DbContextOptionsBuilder optionsBuilder, + DbConnection connection, + Action? sqliteOptionsAction = null) + where TContext : DbContext + { + ((DbContextOptionsBuilder)optionsBuilder).UseBitBsqlite(connection, sqliteOptionsAction); + return optionsBuilder; + } + + /// + public static DbContextOptionsBuilder UseBitBsqlite( + this DbContextOptionsBuilder optionsBuilder, + DbConnection connection, + bool contextOwnsConnection, + Action? sqliteOptionsAction = null) + where TContext : DbContext + { + ((DbContextOptionsBuilder)optionsBuilder) + .UseBitBsqlite(connection, contextOwnsConnection, sqliteOptionsAction); + return optionsBuilder; + } + + private static void ReplaceSyncBoundServices(DbContextOptionsBuilder optionsBuilder) + { + optionsBuilder.ReplaceService(); + optionsBuilder.ReplaceService(); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteHistoryRepository.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteHistoryRepository.cs new file mode 100644 index 00000000000..771ac04e32f --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteHistoryRepository.cs @@ -0,0 +1,135 @@ +using Bit.Bsqlite.Data; +using Microsoft.EntityFrameworkCore.Migrations; + +namespace Bit.Bsqlite.EntityFrameworkCore; + +/// +/// A history repository whose migration lock keeps other tabs out without synchronous I/O. +/// +/// +/// +/// EF's SQLite history repository guards migrations with a lock row in a table, whose +/// Dispose() runs ExecuteScalar - synchronously, even on the MigrateAsync path. +/// That is the second and last blocker the S1 audit found. +/// +/// +/// The lock itself is still needed: every tab runs its own worker over the same OPFS or IndexedDB +/// file, and two tabs opening after a deploy would otherwise both read the history table, both find +/// the new migration pending, and the second would fail applying it again. So the lock comes from +/// the transport () instead - a Web Lock in the browser, which the +/// browser drops when a tab closes mid-migration, where a lock row would be left behind for every +/// other tab to wait on. A transport that cannot coordinate tabs gets no lock. +/// +/// +/// The history-table SQL matches EF's SQLite provider: model-driven create script with +/// IF NOT EXISTS spliced in, and the same rejection of conditional script helpers SQLite +/// cannot express. +/// +/// +public sealed class BsqliteHistoryRepository(HistoryRepositoryDependencies dependencies) + : HistoryRepository(dependencies) +{ + protected override string ExistsSql => $""" + SELECT COUNT(*) FROM "sqlite_master" WHERE "name" = '{TableName}' AND "type" = 'table' + """; + + protected override bool InterpretExistsResult(object? value) + => value is not null && Convert.ToInt64(value) != 0L; + +#if NET9_0_OR_GREATER + // The migration lock arrived with EF Core 9; EF Core 8 takes none, so on net8.0 there is nothing to + // replace - BsqliteDbContextFactory holds a lock of its own around the migration instead. + + /// + /// The lock belongs to neither the connection nor a transaction - the browser holds it - so it + /// lasts until EF releases it. + /// + public override LockReleaseBehavior LockReleaseBehavior => LockReleaseBehavior.Explicit; +#endif + + /// + /// Reuses the model-driven create script so the table and column names stay in step with EF, + /// and only splices in IF NOT EXISTS - which is what SQLite offers in place of the + /// conditional blocks other providers use. + /// + public override string GetCreateIfNotExistsScript() + { + const string createTable = "CREATE TABLE"; + + var script = GetCreateScript(); + var index = script.IndexOf(createTable, StringComparison.Ordinal); + + return index < 0 + ? script + : script.Insert(index + createTable.Length, " IF NOT EXISTS"); + } + +#if NET9_0_OR_GREATER + /// The cross-tab lock migrations run under; see the remarks. + internal const string MigrationLockName = "ef-migrations"; + + /// Not supported: waiting for another tab cannot block the browser's main thread. + public override IMigrationsDatabaseLock AcquireDatabaseLock() + => throw BsqliteSynchronousApiNotSupportedException.ForMember( + nameof(AcquireDatabaseLock), nameof(AcquireDatabaseLockAsync)); + + /// Waits until no other tab is migrating this database, then keeps them out until released. + public override async Task AcquireDatabaseLockAsync( + CancellationToken cancellationToken = default) + { + var held = Dependencies.Connection.DbConnection is BsqliteConnection connection + ? await connection.TryAcquireLockAsync(MigrationLockName, cancellationToken).ConfigureAwait(false) + : null; + + return new MigrationLock(this, held); + } +#endif + + /// + /// SQLite has no procedural IF, so EF's own provider rejects these too. They are only + /// reached when generating idempotent SQL scripts, which is a server-side workflow. + /// + public override string GetBeginIfNotExistsScript(string migrationId) + => throw new NotSupportedException( + "SQLite does not support conditional migration scripts; generate a plain script instead."); + + public override string GetBeginIfExistsScript(string migrationId) + => throw new NotSupportedException( + "SQLite does not support conditional migration scripts; generate a plain script instead."); + + public override string GetEndIfScript() + => throw new NotSupportedException( + "SQLite does not support conditional migration scripts; generate a plain script instead."); + +#if NET9_0_OR_GREATER + private sealed class MigrationLock(IHistoryRepository historyRepository, IAsyncDisposable? held) + : IMigrationsDatabaseLock + { + private IAsyncDisposable? _held = held; + + public IHistoryRepository HistoryRepository { get; } = historyRepository; + + /// + /// EF releases from a synchronous Dispose even on the MigrateAsync path, so the + /// release is started and not awaited, as a synchronously disposed transaction starts its + /// rollback. Letting go of a Web Lock waits on nothing else this tab is doing. + /// + public void Dispose() => _ = ReleaseQuietlyAsync(); + + public ValueTask DisposeAsync() + => Interlocked.Exchange(ref _held, null)?.DisposeAsync() ?? ValueTask.CompletedTask; + + private async Task ReleaseQuietlyAsync() + { + try + { + await DisposeAsync().ConfigureAwait(false); + } + catch (Exception) + { + // Nothing can observe this failure, and a host that failed has dropped its locks. + } + } + } +#endif +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteLiveQueryExtensions.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteLiveQueryExtensions.cs new file mode 100644 index 00000000000..a78384db378 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteLiveQueryExtensions.cs @@ -0,0 +1,209 @@ +using System.Diagnostics.CodeAnalysis; +using System.Runtime.CompilerServices; +using Bit.Bsqlite.Data; +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Query; +using Microsoft.EntityFrameworkCore.Query.Internal; +using Microsoft.Extensions.DependencyInjection; + +namespace Microsoft.EntityFrameworkCore; + +/// EF entry point for table-level live queries. +/// +/// +/// Every run of the query - the first RefreshAsync and each refresh a write sets off - happens +/// in a context of its own, built like the one the query came from and disposed once the rows are +/// read. Two things depend on that. A refresh arrives whenever another tab writes, so on the +/// application's context it would collide with whatever the application starts there next - EF +/// allows one operation per context at a time. And a tracking query re-run on a context that already +/// tracks the rows hands back the instances it holds without their new values, so a live view would +/// pick up inserts and deletes but never an update. +/// +/// +/// The rows a live query returns are therefore current, and are not tracked by the application's +/// context. To change one, load it - or attach it - in the context that saves it. +/// +/// +public static class BsqliteLiveQueryExtensions +{ + /// + /// Re-executes whenever a table it reads is written. + /// + public static IBsqliteLiveQuery> AsLiveQuery(this IQueryable query) + { + ArgumentNullException.ThrowIfNull(query); + return AsLiveQuery(query, ConnectionOf(query)); + } + + /// + /// Re-executes whenever a table it reads is written. + /// + public static IBsqliteLiveQuery> AsLiveQuery( + this IQueryable query, + BsqliteConnection connection) + { + ArgumentNullException.ThrowIfNull(query); + ArgumentNullException.ThrowIfNull(connection); + + var tables = BsqliteTableNames.Extract(query.ToQueryString()); + + // The write notification arrives while SaveChanges is still consuming its result, inside the + // transaction it wrote in. The gate defers the re-read until the save has finished, so it + // never reads rows that may yet roll back. Null only for a query whose provider hides its + // context, in which case the query also runs as given - there is no context to copy. + var context = DbContextOf(query); + var gate = context is null ? null : new BsqliteSaveChangesGate(context); + var contexts = context is null ? null : new RefreshContexts(context); + + return new BsqliteLiveQuery>( + connection, + async ct => + { + if (gate is not null) + { + await gate.WaitForIdleAsync(ct).ConfigureAwait(false); + } + + await using var own = contexts?.Create(); + if (own is null) + { + return await query.ToListAsync(ct).ConfigureAwait(false); + } + + // Query roots name an entity type, not a context, so the same expression runs on the + // fresh context's provider - which is how a compiled query runs on any context. + return await own.GetService() + .CreateQuery(query.Expression) + .ToListAsync(ct) + .ConfigureAwait(false); + }, + tables, + onDispose: gate is null ? null : gate.Dispose); + } + + /// + /// Builds contexts of the same type and options as the one a live query came from, for its + /// runs to happen in. See the class remarks. + /// + /// + /// Everything is captured up front, so a refresh never touches the application's context, which + /// may be disposed by then. A context type that cannot be constructed from its options - one + /// configured only in OnConfiguring, with no constructor taking them - is tried without + /// them; failing that, the query runs on the application's context as given. + /// + private sealed class RefreshContexts(DbContext context) + { + private readonly Type _contextType = context.GetType(); + private readonly IDbContextOptions _options = context.GetService(); + private readonly IServiceProvider _services = + context.GetService().FindExtension()?.ApplicationServiceProvider + ?? NoServices.Instance; + + private Func? _create; + private bool _unavailable; + + public DbContext? Create() + { + if (_create is not null) + { + return _create(); + } + + if (_unavailable) + { + return null; + } + + foreach (var candidate in new Func[] { WithOptions, WithoutOptions }) + { + try + { + var created = candidate(); + _create = candidate; + return created; + } + catch (InvalidOperationException) + { + // No constructor this way round; try the next. + } + } + + _unavailable = true; + return null; + } + + // The context type is the application's, and its constructors survive trimming because the + // application builds it through them - directly, or through a factory or AddDbContext, all + // of which demand public constructors be kept. + [UnconditionalSuppressMessage("Trimming", "IL2077", + Justification = "The context type's public constructors are kept by whatever built the context this one copies.")] + private DbContext WithOptions() + => (DbContext)ActivatorUtilities.CreateInstance(_services, _contextType, _options); + + [UnconditionalSuppressMessage("Trimming", "IL2077", + Justification = "The context type's public constructors are kept by whatever built the context this one copies.")] + private DbContext WithoutOptions() + => (DbContext)ActivatorUtilities.CreateInstance(_services, _contextType); + } + + private sealed class NoServices : IServiceProvider + { + public static readonly NoServices Instance = new(); + + public object? GetService(Type serviceType) => null; + } + + /// + /// implements IInfrastructure<IServiceProvider>. + /// Where/OrderBy replace it with EntityQueryable, whose provider holds + /// the compiler that still knows the context. + /// + private static BsqliteConnection ConnectionOf(IQueryable query) + { + var context = DbContextOf(query) + ?? throw new InvalidOperationException( + "AsLiveQuery() needs an EF query whose provider exposes the current DbContext."); + + if (context.Database.GetDbConnection() is BsqliteConnection connection) + { + return connection; + } + + throw new InvalidOperationException( + "AsLiveQuery() requires the context to be using a BsqliteConnection."); + } + + private static DbContext? DbContextOf(IQueryable query) + => ContextFromInfrastructure(query as IInfrastructure) + ?? ContextFromInfrastructure(query.Provider as IInfrastructure) + ?? ContextFromQueryProvider(query.Provider); + + private static DbContext? ContextFromInfrastructure(IInfrastructure? accessor) + => accessor?.GetService()?.Context; + + // The last resort: reach through EF's query provider for the context. EF1001 is suppressed here + // and nowhere else, so the rest of the assembly still fails the build if it touches an internal. +#pragma warning disable EF1001 + private static DbContext? ContextFromQueryProvider(IQueryProvider provider) + { + if (provider is not EntityQueryProvider entityProvider) + { + return null; + } + + var compiler = GetQueryCompiler(entityProvider); + if (compiler is QueryCompiler queryCompiler) + { + return GetQueryContextFactory(queryCompiler).Create().Context; + } + + return null; + } + + [UnsafeAccessor(UnsafeAccessorKind.Field, Name = "_queryCompiler")] + private static extern ref IQueryCompiler GetQueryCompiler(EntityQueryProvider provider); + + [UnsafeAccessor(UnsafeAccessorKind.Field, Name = "_queryContextFactory")] + private static extern ref IQueryContextFactory GetQueryContextFactory(QueryCompiler compiler); +#pragma warning restore EF1001 +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqlitePcl.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqlitePcl.cs new file mode 100644 index 00000000000..96cd14278c3 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqlitePcl.cs @@ -0,0 +1,35 @@ +using SQLitePCL; + +namespace Bit.Bsqlite.EntityFrameworkCore; + +/// +/// Satisfies SQLitePCL.raw on WASM so EF Core can read +/// new SqliteConnection().ServerVersion without a native e_sqlite3. +/// +/// +/// EF Core 10's query translator does +/// new Version(new SqliteConnection().ServerVersion) >= 3.38 before +/// compiling any query. ServerVersion calls sqlite3_libversion(), +/// which throws if no provider is set. Desktop tests already have the bundle; +/// the browser does not, and must not grow one. This stub answers version +/// probes and no-ops everything else. +/// +internal static class BsqlitePcl +{ + private static int _initialized; + + internal static void EnsureInitialized() + { + if (!OperatingSystem.IsBrowser()) + { + return; + } + + if (Interlocked.Exchange(ref _initialized, 1) != 0) + { + return; + } + + raw.SetProvider(new BsqlitePclProvider()); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqlitePclProvider.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqlitePclProvider.cs new file mode 100644 index 00000000000..f2896a25e40 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqlitePclProvider.cs @@ -0,0 +1,275 @@ +using SQLitePCL; + +namespace Bit.Bsqlite.EntityFrameworkCore; + +/// +/// The no-op SQLitePCL.raw provider installs on WASM, so +/// EF Core can read a version without a native e_sqlite3 behind it. +/// +internal sealed class BsqlitePclProvider : ISQLite3Provider +{ + /// + /// The SQLite version inside the vendored wa-sqlite build. Must match what the engine reports; + /// see engine/wa-sqlite.lock.props for the pinned build, and the browser suite's + /// engine-version test, which fails if the two drift apart. + /// + /// + /// This is the number EF Core gates its SQL translations on, so a wrong one is not cosmetic. + /// Claiming less than the engine offers makes EF fall back to older SQL in the browser while it + /// emits the newer form on the server - the same model producing two different queries, + /// silently. Claiming more makes it emit SQL the engine cannot run. Reporting the truth also + /// happens to be what keeps the two halves in step: the engine SQLitePCLRaw bundles for EF 10 + /// is 3.53.x, the same series as the vendored wa-sqlite build. + /// + internal const string EngineVersion = "3.53.0"; + + private const int EngineVersionNumber = 3_053_000; + + public string GetNativeLibraryName() => "bit-bsqlite"; + + public utf8z sqlite3_libversion() => utf8z.FromString(EngineVersion); + + public int sqlite3_libversion_number() => EngineVersionNumber; + + public utf8z sqlite3_sourceid() => utf8z.FromString("bit-bsqlite"); + + public int sqlite3_threadsafe() => 1; + + public int sqlite3_initialize() => 0; + + public int sqlite3_shutdown() => 0; + + public int sqlite3_win32_set_directory(int typ, utf8z path) => 0; + + public int sqlite3_keyword_count() => 0; + + public int sqlite3_keyword_name(int i, out string name) + { + name = string.Empty; + return 0; + } + + // Everything below is never reached: EF Core asks this provider for its version and nothing + // else, and no SQLite handle is ever opened through it. Each member returns the neutral value + // of its type so the interface is satisfied without a native library. + public int sqlite3_open(utf8z filename, out IntPtr db) + { + db = IntPtr.Zero; + return 0; + } + public int sqlite3_open_v2(utf8z filename, out IntPtr db, int flags, utf8z vfs) + { + db = IntPtr.Zero; + return 0; + } + public int sqlite3_close_v2(IntPtr db) => 0; + public int sqlite3_close(IntPtr db) => 0; + public int sqlite3_enable_shared_cache(int enable) => 0; + public void sqlite3_interrupt(sqlite3 db) { } + public int sqlite3__vfs__delete(utf8z vfs, utf8z pathname, int syncDir) => 0; + public long sqlite3_memory_used() => 0; + public long sqlite3_memory_highwater(int resetFlag) => 0; + public long sqlite3_soft_heap_limit64(long n) => 0; + public long sqlite3_hard_heap_limit64(long n) => 0; + public int sqlite3_status(int op, out int current, out int highwater, int resetFlag) + { + current = 0; + highwater = 0; + return 0; + } + public int sqlite3_db_readonly(sqlite3 db, utf8z dbName) => 0; + public utf8z sqlite3_db_filename(sqlite3 db, utf8z att) => default; + public utf8z sqlite3_errmsg(sqlite3 db) => default; + public long sqlite3_last_insert_rowid(sqlite3 db) => 0; + public int sqlite3_changes(sqlite3 db) => 0; + public int sqlite3_total_changes(sqlite3 db) => 0; + public int sqlite3_get_autocommit(sqlite3 db) => 0; + public int sqlite3_busy_timeout(sqlite3 db, int ms) => 0; + public int sqlite3_extended_result_codes(sqlite3 db, int onoff) => 0; + public int sqlite3_errcode(sqlite3 db) => 0; + public int sqlite3_extended_errcode(sqlite3 db) => 0; + public utf8z sqlite3_errstr(int rc) => default; + public int sqlite3_prepare_v2(sqlite3 db, ReadOnlySpan sql, out IntPtr stmt, out ReadOnlySpan remain) + { + stmt = IntPtr.Zero; + remain = default; + return 0; + } + public int sqlite3_prepare_v3(sqlite3 db, ReadOnlySpan sql, uint flags, out IntPtr stmt, out ReadOnlySpan remain) + { + stmt = IntPtr.Zero; + remain = default; + return 0; + } + + [Obsolete] + public int sqlite3_prepare_v2(sqlite3 db, utf8z sql, out IntPtr stmt, out utf8z remain) + { + stmt = IntPtr.Zero; + remain = default; + return 0; + } + + [Obsolete] + public int sqlite3_prepare_v3(sqlite3 db, utf8z sql, uint flags, out IntPtr stmt, out utf8z remain) + { + stmt = IntPtr.Zero; + remain = default; + return 0; + } + public int sqlite3_step(sqlite3_stmt stmt) => 0; + public int sqlite3_finalize(IntPtr stmt) => 0; + public int sqlite3_reset(sqlite3_stmt stmt) => 0; + public int sqlite3_clear_bindings(sqlite3_stmt stmt) => 0; + public int sqlite3_stmt_status(sqlite3_stmt stmt, int op, int resetFlg) => 0; + public utf8z sqlite3_sql(sqlite3_stmt stmt) => default; + public IntPtr sqlite3_db_handle(IntPtr stmt) => IntPtr.Zero; + public IntPtr sqlite3_next_stmt(sqlite3 db, IntPtr stmt) => IntPtr.Zero; + public int sqlite3_bind_zeroblob(sqlite3_stmt stmt, int index, int size) => 0; + public utf8z sqlite3_bind_parameter_name(sqlite3_stmt stmt, int index) => default; + public int sqlite3_bind_blob(sqlite3_stmt stmt, int index, ReadOnlySpan blob) => 0; + public int sqlite3_bind_double(sqlite3_stmt stmt, int index, double val) => 0; + public int sqlite3_bind_int(sqlite3_stmt stmt, int index, int val) => 0; + public int sqlite3_bind_int64(sqlite3_stmt stmt, int index, long val) => 0; + public int sqlite3_bind_null(sqlite3_stmt stmt, int index) => 0; + public int sqlite3_bind_text(sqlite3_stmt stmt, int index, ReadOnlySpan text) => 0; + public int sqlite3_bind_text16(sqlite3_stmt stmt, int index, ReadOnlySpan text) => 0; + public int sqlite3_bind_text(sqlite3_stmt stmt, int index, utf8z text) => 0; + public int sqlite3_bind_parameter_count(sqlite3_stmt stmt) => 0; + public int sqlite3_bind_parameter_index(sqlite3_stmt stmt, utf8z strName) => 0; + public utf8z sqlite3_column_database_name(sqlite3_stmt stmt, int index) => default; + public utf8z sqlite3_column_name(sqlite3_stmt stmt, int index) => default; + public utf8z sqlite3_column_origin_name(sqlite3_stmt stmt, int index) => default; + public utf8z sqlite3_column_table_name(sqlite3_stmt stmt, int index) => default; + public utf8z sqlite3_column_text(sqlite3_stmt stmt, int index) => default; + public int sqlite3_data_count(sqlite3_stmt stmt) => 0; + public int sqlite3_column_count(sqlite3_stmt stmt) => 0; + public double sqlite3_column_double(sqlite3_stmt stmt, int index) => 0; + public int sqlite3_column_int(sqlite3_stmt stmt, int index) => 0; + public long sqlite3_column_int64(sqlite3_stmt stmt, int index) => 0; + public ReadOnlySpan sqlite3_column_blob(sqlite3_stmt stmt, int index) => default; + public int sqlite3_column_bytes(sqlite3_stmt stmt, int index) => 0; + public int sqlite3_column_type(sqlite3_stmt stmt, int index) => 0; + public utf8z sqlite3_column_decltype(sqlite3_stmt stmt, int index) => default; + public int sqlite3_snapshot_get(sqlite3 db, utf8z schema, out IntPtr snap) + { + snap = IntPtr.Zero; + return 0; + } + public int sqlite3_snapshot_cmp(sqlite3_snapshot p1, sqlite3_snapshot p2) => 0; + public int sqlite3_snapshot_open(sqlite3 db, utf8z schema, sqlite3_snapshot snap) => 0; + public int sqlite3_snapshot_recover(sqlite3 db, utf8z name) => 0; + public void sqlite3_snapshot_free(IntPtr snap) { } + public sqlite3_backup sqlite3_backup_init(sqlite3 destDb, utf8z destName, sqlite3 sourceDb, utf8z sourceName) => null!; + public int sqlite3_backup_step(sqlite3_backup backup, int nPage) => 0; + public int sqlite3_backup_remaining(sqlite3_backup backup) => 0; + public int sqlite3_backup_pagecount(sqlite3_backup backup) => 0; + public int sqlite3_backup_finish(IntPtr backup) => 0; + public int sqlite3_blob_open(sqlite3 db, utf8z db_utf8, utf8z table_utf8, utf8z col_utf8, long rowid, int flags, out sqlite3_blob blob) + { + blob = null!; + return 0; + } + public int sqlite3_blob_bytes(sqlite3_blob blob) => 0; + public int sqlite3_blob_reopen(sqlite3_blob blob, long rowid) => 0; + public int sqlite3_blob_write(sqlite3_blob blob, ReadOnlySpan b, int offset) => 0; + public int sqlite3_blob_read(sqlite3_blob blob, Span b, int offset) => 0; + public int sqlite3_blob_close(IntPtr blob) => 0; + public int sqlite3_config_log(delegate_log func, object v) => 0; + public void sqlite3_log(int errcode, utf8z s) { } + public void sqlite3_commit_hook(sqlite3 db, delegate_commit func, object v) { } + public void sqlite3_rollback_hook(sqlite3 db, delegate_rollback func, object v) { } + public void sqlite3_trace(sqlite3 db, delegate_trace func, object v) { } + public void sqlite3_profile(sqlite3 db, delegate_profile func, object v) { } + public void sqlite3_progress_handler(sqlite3 db, int instructions, delegate_progress func, object v) { } + public void sqlite3_update_hook(sqlite3 db, delegate_update func, object v) { } + public int sqlite3_create_collation(sqlite3 db, byte[] name, object v, delegate_collation func) => 0; + public int sqlite3_create_function(sqlite3 db, byte[] name, int nArg, int flags, object v, delegate_function_scalar func) => 0; + public int sqlite3_create_function(sqlite3 db, byte[] name, int nArg, int flags, object v, delegate_function_aggregate_step func_step, delegate_function_aggregate_final func_final) => 0; + public int sqlite3_db_status(sqlite3 db, int op, out int current, out int highest, int resetFlg) + { + current = 0; + highest = 0; + return 0; + } + public void sqlite3_result_blob(IntPtr context, ReadOnlySpan val) { } + public void sqlite3_result_double(IntPtr context, double val) { } + public void sqlite3_result_error(IntPtr context, ReadOnlySpan strErr) { } + public void sqlite3_result_error(IntPtr context, utf8z strErr) { } + public void sqlite3_result_int(IntPtr context, int val) { } + public void sqlite3_result_int64(IntPtr context, long val) { } + public void sqlite3_result_null(IntPtr context) { } + public void sqlite3_result_text(IntPtr context, ReadOnlySpan val) { } + public void sqlite3_result_text(IntPtr context, utf8z val) { } + public void sqlite3_result_zeroblob(IntPtr context, int n) { } + public void sqlite3_result_error_toobig(IntPtr context) { } + public void sqlite3_result_error_nomem(IntPtr context) { } + public void sqlite3_result_error_code(IntPtr context, int code) { } + public ReadOnlySpan sqlite3_value_blob(IntPtr p) => default; + public int sqlite3_value_bytes(IntPtr p) => 0; + public double sqlite3_value_double(IntPtr p) => 0; + public int sqlite3_value_int(IntPtr p) => 0; + public long sqlite3_value_int64(IntPtr p) => 0; + public int sqlite3_value_type(IntPtr p) => 0; + public utf8z sqlite3_value_text(IntPtr p) => default; + public int sqlite3_stmt_isexplain(sqlite3_stmt stmt) => 0; + public int sqlite3_stmt_busy(sqlite3_stmt stmt) => 0; + public int sqlite3_stmt_readonly(sqlite3_stmt stmt) => 0; + public int sqlite3_exec(sqlite3 db, utf8z sql, delegate_exec callback, object user_data, out IntPtr errMsg) + { + errMsg = IntPtr.Zero; + return 0; + } + public int sqlite3_complete(utf8z sql) => 0; + public int sqlite3_compileoption_used(utf8z sql) => 0; + public utf8z sqlite3_compileoption_get(int n) => default; + public int sqlite3_wal_autocheckpoint(sqlite3 db, int n) => 0; + public int sqlite3_wal_checkpoint(sqlite3 db, utf8z dbName) => 0; + public int sqlite3_wal_checkpoint_v2(sqlite3 db, utf8z dbName, int eMode, out int logSize, out int framesCheckPointed) + { + logSize = 0; + framesCheckPointed = 0; + return 0; + } + public int sqlite3_table_column_metadata(sqlite3 db, utf8z dbName, utf8z tblName, utf8z colName, out utf8z dataType, out utf8z collSeq, out int notNull, out int primaryKey, out int autoInc) + { + dataType = default; + collSeq = default; + notNull = 0; + primaryKey = 0; + autoInc = 0; + return 0; + } + public int sqlite3_set_authorizer(sqlite3 db, delegate_authorizer authorizer, object user_data) => 0; + public int sqlite3_stricmp(IntPtr p, IntPtr q) => 0; + public int sqlite3_strnicmp(IntPtr p, IntPtr q, int n) => 0; + public IntPtr sqlite3_malloc(int n) => IntPtr.Zero; + public IntPtr sqlite3_malloc64(long n) => IntPtr.Zero; + public void sqlite3_free(IntPtr p) { } + public int sqlite3_key(sqlite3 db, ReadOnlySpan key) => 0; + public int sqlite3_key_v2(sqlite3 db, utf8z dbname, ReadOnlySpan key) => 0; + public int sqlite3_rekey(sqlite3 db, ReadOnlySpan key) => 0; + public int sqlite3_rekey_v2(sqlite3 db, utf8z dbname, ReadOnlySpan key) => 0; + public int sqlite3_load_extension(sqlite3 db, utf8z zFile, utf8z zProc, out utf8z pzErrMsg) + { + pzErrMsg = default; + return 0; + } + public int sqlite3_limit(sqlite3 db, int id, int newVal) => 0; + public int sqlite3_config(int op) => 0; + public int sqlite3_config(int op, int val) => 0; + public int sqlite3_db_config(sqlite3 db, int op, utf8z val) => 0; + public int sqlite3_db_config(sqlite3 db, int op, int val, out int result) + { + result = 0; + return 0; + } + public int sqlite3_db_config(sqlite3 db, int op, IntPtr ptr, int int0, int int1) => 0; + public int sqlite3_enable_load_extension(sqlite3 db, int enable) => 0; + public IntPtr sqlite3_serialize(sqlite3 db, utf8z schema, out long size, int flags) + { + size = 0; + return IntPtr.Zero; + } + public int sqlite3_deserialize(sqlite3 db, utf8z schema, IntPtr data, long szDb, long szBuf, int flags) => 0; +} diff --git a/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteSaveChangesGate.cs b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteSaveChangesGate.cs new file mode 100644 index 00000000000..0b903b7f8db --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/EntityFrameworkCore/BsqliteSaveChangesGate.cs @@ -0,0 +1,90 @@ +namespace Microsoft.EntityFrameworkCore; + +/// +/// Holds live-query refreshes back while the context's SaveChanges is in progress. +/// +/// +/// +/// The connection raises TablesChanged as soon as the transport has run the write, which is +/// before EF has consumed the insert result and before it has accepted the tracked changes - and, +/// when another tab's write arrives mid-save, while this tab's save still has its transaction open +/// on the shared connection. A refresh then would read rows that may yet roll back. +/// +/// +/// Refreshes once ran on the saving context itself, where the race was worse: re-executing the +/// query materialised the just-inserted row into a change tracker still assigning it its +/// store-generated key, and "another instance with the same key value is already being tracked" +/// was thrown out of the write. They run on a context of their own now, but waiting for +/// still makes the read follow the write on every host. +/// +/// +internal sealed class BsqliteSaveChangesGate : IDisposable +{ + private readonly DbContext _context; + private readonly Lock _lock = new(); + private TaskCompletionSource? _saving; + private bool _disposed; + + public BsqliteSaveChangesGate(DbContext context) + { + _context = context; + _context.SavingChanges += OnSavingChanges; + _context.SavedChanges += OnSaveFinished; + _context.SaveChangesFailed += OnSaveFinished; + } + + /// Completes once no SaveChanges is running on the context. + public Task WaitForIdleAsync(CancellationToken cancellationToken) + { + Task? pending; + lock (_lock) + { + pending = _saving?.Task; + } + + return pending is null ? Task.CompletedTask : pending.WaitAsync(cancellationToken); + } + + public void Dispose() + { + TaskCompletionSource? saving; + lock (_lock) + { + if (_disposed) + { + return; + } + + _disposed = true; + saving = _saving; + _saving = null; + } + + _context.SavingChanges -= OnSavingChanges; + _context.SavedChanges -= OnSaveFinished; + _context.SaveChangesFailed -= OnSaveFinished; + + // Nothing will ever signal a refresh that is still waiting; let it run rather than hang. + saving?.TrySetResult(); + } + + private void OnSavingChanges(object? sender, SavingChangesEventArgs e) + { + lock (_lock) + { + _saving ??= new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + } + } + + private void OnSaveFinished(object? sender, EventArgs e) + { + TaskCompletionSource? saving; + lock (_lock) + { + saving = _saving; + _saving = null; + } + + saving?.TrySetResult(); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteBuilder.cs b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteBuilder.cs new file mode 100644 index 00000000000..7dff07d53a7 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteBuilder.cs @@ -0,0 +1,217 @@ +using System.Diagnostics.CodeAnalysis; +using Bit.Bsqlite; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; +using Microsoft.Extensions.DependencyInjection.Extensions; +using Microsoft.Extensions.Logging; +using Microsoft.JSInterop; + +namespace Microsoft.Extensions.DependencyInjection; + +/// Collects the choices AddBitBsqliteServices turns into registrations. +public sealed class BsqliteBuilder +{ + private readonly List _onOpenSql = []; + private readonly List _extensions = []; + private BsqliteStorageSelection? _selection; + private string? _hostModuleUrl; + private string? _workerUrl; + private bool _enableForeignKeys = true; + private bool _bindingStoreRegistered; + + internal BsqliteBuilder(IServiceCollection services) => Services = services; + + /// The service collection being configured. + public IServiceCollection Services { get; } + + /// + /// Registers a storage provider, scoped like the rest of the stack. Its constructor may take + /// . + /// + public BsqliteBuilder AddStorageProvider<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TProvider>() + where TProvider : class, IBsqliteStorageProvider + { + Services.AddScoped(); + return this; + } + + /// Registers a storage provider built by , once per scope. + public BsqliteBuilder AddStorageProvider(Func factory) + { + ArgumentNullException.ThrowIfNull(factory); + Services.AddScoped(factory); + return this; + } + + /// + /// Registers an already-built storage provider. Being one instance, it is shared by every scope, so it + /// must not hold on to a scoped service such as a server circuit's . + /// + public BsqliteBuilder AddStorageProvider(IBsqliteStorageProvider provider) + { + ArgumentNullException.ThrowIfNull(provider); + Services.AddSingleton(provider); + return this; + } + + /// + /// Sets the preference order among the registered providers. Required: selection never guesses. + /// + public BsqliteBuilder UseStorage(Action configure) + { + _selection = BsqliteStorageSelectionBuilder.Create(configure); + return this; + } + + /// Uses an already-built selection. + public BsqliteBuilder UseStorage(BsqliteStorageSelection selection) + { + _selection = selection ?? throw new ArgumentNullException(nameof(selection)); + return this; + } + + /// + /// Replaces the sticky-binding store. The default remembers bindings in localStorage, + /// which is what production wants; tests use . + /// + public BsqliteBuilder UseBindingStore<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TStore>() + where TStore : class, IBsqliteStorageBindingStore + { + Services.AddScoped(); + _bindingStoreRegistered = true; + return this; + } + + /// + /// Replaces the sticky-binding store with , one instance shared by every scope. + /// + public BsqliteBuilder UseBindingStore(IBsqliteStorageBindingStore store) + { + ArgumentNullException.ThrowIfNull(store); + Services.AddSingleton(store); + _bindingStoreRegistered = true; + return this; + } + + /// + /// Registers a for . Call once per + /// database the application opens. + /// + /// + /// + /// Each database is a keyed service under its name, so with several registered, inject the one + /// wanted with [FromKeyedServices("cache.db")] and bind a context to it with + /// . An unkeyed + /// resolves to the one registered last, which is the application's + /// one shared database when there is only one. + /// + /// + /// Scoped: one session for the whole of a WebAssembly application, and one per circuit on a server. + /// + /// + public BsqliteBuilder AddDatabase(string databaseName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + if (Services.Any(service => service.ServiceType == typeof(BsqliteDatabaseRegistration) + && service.ImplementationInstance is BsqliteDatabaseRegistration { DatabaseName: var registered } + && string.Equals(registered, databaseName, StringComparison.Ordinal))) + { + throw new InvalidOperationException( + $"The database '{databaseName}' is already registered. Each database is one shared " + + "session, so it is added once."); + } + + Services.AddSingleton(new BsqliteDatabaseRegistration(databaseName)); + Services.AddKeyedScoped(databaseName, (sp, _) => new BsqliteDatabase( + sp.GetRequiredService(), + databaseName)); + Services.AddScoped(sp => sp.GetRequiredKeyedService(databaseName)); + return this; + } + + /// + /// Whether every connection runs PRAGMA foreign_keys = ON on open. On by default, as + /// with Microsoft.Data.Sqlite. + /// + public BsqliteBuilder EnableForeignKeys(bool enable = true) + { + _enableForeignKeys = enable; + return this; + } + + /// Adds a statement to run on every open, after the provider's own configuration. + public BsqliteBuilder OnOpen(string sql) + { + ArgumentException.ThrowIfNullOrWhiteSpace(sql); + _onOpenSql.Add(sql); + return this; + } + + /// + /// Adds a JavaScript module that registers user-defined functions, collations, or virtual + /// tables on every open database. See . + /// + public BsqliteBuilder AddExtensionModule(string moduleUrl, string registerExport = "register") + { + _extensions.Add(new BsqliteJsModule(moduleUrl, registerExport)); + return this; + } + + /// Overrides where the worker host module and worker script are served from. + public BsqliteBuilder UseWorker(string? hostModuleUrl = null, string? workerUrl = null) + { + _hostModuleUrl = hostModuleUrl; + _workerUrl = workerUrl; + return this; + } + + internal void Complete() + { + if (_selection is null) + { + throw new InvalidOperationException( + "AddBitBsqliteServices needs a storage preference: call UseStorage(s => s.Prefer(\"opfs\")...)."); + } + + if (!_bindingStoreRegistered) + { + Services.TryAddScoped(sp => + new BsqliteLocalStorageBindingStore(sp.GetRequiredService())); + } + + var selection = _selection; + var defaults = new BsqliteWorkerTransportDefaults + { + HostModuleUrl = _hostModuleUrl ?? BsqliteWorkerTransport.VersionedHostModuleUrl, + WorkerUrl = _workerUrl, + EnableForeignKeys = _enableForeignKeys, + OnOpenSql = [.. _onOpenSql], + ExtensionModules = [.. _extensions], + }; + + Services.TryAddSingleton(selection); + Services.TryAddSingleton(defaults); + Services.TryAddScoped(sp => + new BsqliteWorkerTransportFactory(sp.GetRequiredService(), defaults)); + Services.TryAddSingleton(); + Services.TryAddScoped(sp => new BsqliteStorageProviderResolver( + sp.GetServices(), + sp.GetRequiredService(), + sp.GetService>())); + Services.TryAddScoped(sp => new BsqliteSessionFactory( + sp.GetRequiredService(), + sp.GetRequiredService(), + sp.GetRequiredService(), + sp.GetRequiredService(), + sp.GetRequiredService(), + sp.GetService>())); + Services.TryAddScoped(sp => (BsqliteSessionFactory)sp.GetRequiredService()); + } +} + +/// +/// One per call, so the registered names can be listed +/// without opening - or even constructing - any of the databases. +/// +internal sealed record BsqliteDatabaseRegistration(string DatabaseName); diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteDatabase.cs b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteDatabase.cs new file mode 100644 index 00000000000..33696151d94 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteDatabase.cs @@ -0,0 +1,206 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Interop; + +/// +/// One named database for the lifetime of the application: opened on first use, shared by every +/// caller, and closable so that admin operations can replace the file underneath. +/// +/// +/// +/// A Blazor WebAssembly application is one process with one user, so the natural shape is a +/// singleton session per database rather than a pool. This type is that singleton. It is what +/// the EF Core integration's context factory opens through, and what an application should inject +/// when it wants the connection directly. +/// +/// +/// tears the worker down. Subscribers to drop the +/// connections and live queries they hold; subscribers to reopen. That is the +/// protocol for moving a database between backends or importing a file over it. +/// +/// +public sealed class BsqliteDatabase : IAsyncDisposable, IDisposable +{ + private readonly IBsqliteSessionFactory _sessions; + private readonly SemaphoreSlim _gate = new(1, 1); + private BsqliteSession? _session; + private bool _disposed; + + public BsqliteDatabase(IBsqliteSessionFactory sessions, string databaseName) + { + ArgumentNullException.ThrowIfNull(sessions); + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + _sessions = sessions; + Name = databaseName; + } + + /// The database this instance opens. + public string Name { get; } + + /// The open session, or before the first open or after a close. + public BsqliteSession? Session => _session; + + /// The open connection, or when the database is not open. + public BsqliteConnection? Connection => _session?.Connection; + + /// Why the current backend was chosen, when the database is open. + public BsqliteStorageResolution? Resolution => _session?.Resolution; + + /// The provider holding the data, when the database is open. + public IBsqliteStorageProvider? Provider => _session?.Resolution.Provider; + + /// Whether a session is currently open. + public bool IsOpen => _session is not null; + + /// + /// Raised before the session is closed. Handlers must release every connection, context, and + /// live query bound to it; the worker is terminated once they return. + /// + public event Func? Closing; + + /// Raised after a session opens, including the first time. + public event Func? Opened; + + /// Opens the database if it is not open and returns the session. + public async ValueTask OpenAsync(CancellationToken cancellationToken = default) + { + ObjectDisposedException.ThrowIf(_disposed, this); + + if (_session is { } open) + { + return open; + } + + await _gate.WaitAsync(cancellationToken).ConfigureAwait(false); + var opened = false; + try + { + if (_session is null) + { + _session = await _sessions.OpenAsync(Name, cancellationToken).ConfigureAwait(false); + opened = true; + } + + return _session; + } + finally + { + _gate.Release(); + + if (opened) + { + await RaiseAsync(Opened).ConfigureAwait(false); + } + } + } + + /// + /// Closes the session and terminates its worker, after giving subscribers + /// the chance to let go. A no-op when the database is not open. + /// + public async Task CloseAsync(CancellationToken cancellationToken = default) + { + if (_session is null) + { + return; + } + + await RaiseAsync(Closing).ConfigureAwait(false); + + await _gate.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + if (_session is { } session) + { + _session = null; + await session.DisposeAsync().ConfigureAwait(false); + } + } + finally + { + _gate.Release(); + } + } + + /// Closes and reopens, which is how a changed binding or an imported file takes effect. + public async Task ReopenAsync(CancellationToken cancellationToken = default) + { + await CloseAsync(cancellationToken).ConfigureAwait(false); + return await OpenAsync(cancellationToken).ConfigureAwait(false); + } + + /// + /// Runs with the database closed and reopens afterwards, so an admin + /// operation never races the worker that holds the file open. + /// + public async Task WhileClosedAsync(Func work, CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(work); + + var wasOpen = _session is not null; + await CloseAsync(cancellationToken).ConfigureAwait(false); + await work(cancellationToken).ConfigureAwait(false); + + if (wasOpen) + { + await OpenAsync(cancellationToken).ConfigureAwait(false); + } + } + + /// + /// Reads the open database out as a SQLite file image, consistent as of the last committed + /// write, without closing it. + /// + public async Task ExportAsync(CancellationToken cancellationToken = default) + { + var session = await OpenAsync(cancellationToken).ConfigureAwait(false); + return await session.Connection.ExportAsync(cancellationToken).ConfigureAwait(false); + } + + /// + public async ValueTask DisposeAsync() + { + if (_disposed) + { + return; + } + + _disposed = true; + if (_session is { } session) + { + _session = null; + await session.DisposeAsync().ConfigureAwait(false); + } + + _gate.Dispose(); + } + + /// + /// Synchronous disposal, for containers that dispose synchronously. The worker is torn down + /// asynchronously in the background; prefer where possible. + /// + public void Dispose() + { + if (_disposed) + { + return; + } + + _ = DisposeAsync().AsTask(); + } + + private static async Task RaiseAsync(Func? handlers) + { + if (handlers is null) + { + return; + } + + foreach (var handler in handlers.GetInvocationList().Cast>()) + { + await handler().ConfigureAwait(false); + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/BsqlitePersistence.cs b/src/Bsqlite/Bit.Bsqlite/Interop/BsqlitePersistence.cs new file mode 100644 index 00000000000..201f56e9698 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/BsqlitePersistence.cs @@ -0,0 +1,36 @@ +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Interop; + +/// +/// Asks the browser to keep origin storage out of the LRU eviction bucket. +/// +/// +/// Not on the frozen admin contract - persistence is an origin setting, not a per-backend one. +/// +public static class BsqlitePersistence +{ + public static async Task RequestAsync( + IJSRuntime js, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(js); + var module = await js + .InvokeAsync( + "import", + cancellationToken, + "./_content/Bit.Bsqlite/bit-bsqlite-persist.js") + .ConfigureAwait(false); + + try + { + return await module + .InvokeAsync("requestPersistence", cancellationToken) + .ConfigureAwait(false); + } + finally + { + await module.DisposeAsync().ConfigureAwait(false); + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteSession.cs b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteSession.cs new file mode 100644 index 00000000000..fc9de0c2276 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteSession.cs @@ -0,0 +1,41 @@ +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Interop; + +/// +/// An opened database: the connection the application uses, the transport that owns the worker, +/// and the resolution that explains why this backend was chosen. +/// +/// +/// The session owns the transport. The connection's Close is bookkeeping only - see +/// - so disposing the session is what actually tears +/// the worker down. +/// +public sealed class BsqliteSession : IAsyncDisposable +{ + internal BsqliteSession( + BsqliteConnection connection, + IBsqliteTransport transport, + BsqliteStorageResolution resolution) + { + Connection = connection; + Transport = transport; + Resolution = resolution; + } + + /// The ADO.NET connection. Already open. + public BsqliteConnection Connection { get; } + + /// The transport serving . Owned by this session. + public IBsqliteTransport Transport { get; } + + /// Why this backend was chosen, including every candidate that was not. + public BsqliteStorageResolution Resolution { get; } + + /// + public async ValueTask DisposeAsync() + { + await Connection.DisposeAsync().ConfigureAwait(false); + await Transport.DisposeAsync().ConfigureAwait(false); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteSessionFactory.cs b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteSessionFactory.cs new file mode 100644 index 00000000000..c15c3672cf9 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/BsqliteSessionFactory.cs @@ -0,0 +1,138 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; + +namespace Bit.Bsqlite.Interop; + +/// +/// Resolves a storage backend, opens a transport against it, and only then records the binding. +/// +/// +/// +/// This is the composition UseBitBsqlite will call. It lives here rather than in the EF +/// package so raw ADO.NET can open a database without taking a dependency on EF, and so the +/// "commit the binding only after the open succeeds" rule has one implementation. +/// +/// +/// copies to the preferred backend, +/// checks the image, then flips the binding. A failed migration is logged and does not stop the +/// open - an opportunistic move is not a reason to keep users from their data - and the database +/// opens wherever the binding then names: the source after a failed copy, the target when only +/// deleting the source failed. +/// +/// +public sealed class BsqliteSessionFactory : IBsqliteSessionFactory +{ + private readonly BsqliteStorageProviderResolver _resolver; + private readonly IBsqliteTransportFactory _transports; + private readonly BsqliteStorageSelection _selection; + private readonly BsqliteStorageMigrator _migrator; + private readonly IBsqliteStorageBindingStore _bindings; + private readonly ILogger _logger; + + public BsqliteSessionFactory( + BsqliteStorageProviderResolver resolver, + IBsqliteTransportFactory transports, + BsqliteStorageSelection selection, + IBsqliteStorageBindingStore bindings, + BsqliteStorageMigrator? migrator = null, + ILogger? logger = null) + { + ArgumentNullException.ThrowIfNull(resolver); + ArgumentNullException.ThrowIfNull(transports); + ArgumentNullException.ThrowIfNull(selection); + ArgumentNullException.ThrowIfNull(bindings); + + _resolver = resolver; + _transports = transports; + _selection = selection; + _bindings = bindings; + _migrator = migrator ?? new BsqliteStorageMigrator(); + _logger = logger ?? NullLogger.Instance; + } + + /// + public async Task OpenAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + var resolution = await _resolver + .ResolveAsync(databaseName, _selection, cancellationToken) + .ConfigureAwait(false); + + if (_selection.MigrationMode is BsqliteStorageMigrationMode.AutomaticOnOpen + && resolution.BetterProviderAvailable is { } better) + { + await TryMigrateAsync(databaseName, resolution.Provider, better, cancellationToken) + .ConfigureAwait(false); + + // Whether it succeeded or not: a migration that failed after the binding moved leaves the + // database on the target, and opening the source instead would send this session's + // writes somewhere the next load no longer reads. + resolution = await _resolver + .ResolveAsync(databaseName, _selection, cancellationToken) + .ConfigureAwait(false); + } + + var transport = _transports.Create(resolution.Provider); + + try + { + await transport.OpenAsync(databaseName, cancellationToken).ConfigureAwait(false); + } + catch + { + await transport.DisposeAsync().ConfigureAwait(false); + throw; + } + + // Only after the open: a failed open must not leave a binding claiming the database exists + // somewhere it does not. + await _resolver.CommitBindingAsync(resolution, cancellationToken).ConfigureAwait(false); + + var capabilities = resolution.Provider.Capabilities; + var connection = new BsqliteConnection(transport, databaseName) + { + RuntimeLimits = new BsqliteRuntimeLimits + { + SupportsMultiDatabaseTransactions = capabilities.SupportsMultiDatabaseTransactions, + CanChangePageSize = capabilities.CanChangePageSize, + }, + }; + await connection.OpenAsync(cancellationToken).ConfigureAwait(false); + + return new BsqliteSession(connection, transport, resolution); + } + + /// + /// Runs the automatic migration, reporting rather than throwing when it fails. The caller + /// resolves again either way: the binding the migrator leaves behind - unchanged, or moved when + /// only the source's deletion failed - is what says where the database now lives. + /// + private async Task TryMigrateAsync( + string databaseName, + IBsqliteStorageProvider source, + IBsqliteStorageProvider target, + CancellationToken cancellationToken) + { + try + { + await _migrator + .MigrateAsync(databaseName, source, target, _bindings, cancellationToken) + .ConfigureAwait(false); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + _logger.LogWarning( + ex, + "Automatic migration of database {DatabaseName} from storage provider {Source} to " + + "{Target} failed; the database opens wherever its binding now points.", + databaseName, + source.Name, + target.Name); + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/IBsqliteSessionFactory.cs b/src/Bsqlite/Bit.Bsqlite/Interop/IBsqliteSessionFactory.cs new file mode 100644 index 00000000000..56d72b535bb --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/IBsqliteSessionFactory.cs @@ -0,0 +1,15 @@ +namespace Bit.Bsqlite.Interop; + +/// +/// Opens databases: resolves a storage backend, starts a worker against it, and hands back the +/// open connection together with the reasons for the choice. +/// +/// +/// Registered by AddBitBsqliteServices. Applications that need one database per session should +/// prefer , which caches the session and owns its lifetime. +/// +public interface IBsqliteSessionFactory +{ + /// Opens on the backend selection chooses. + Task OpenAsync(string databaseName, CancellationToken cancellationToken = default); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteInMemoryStorageBindingStore.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteInMemoryStorageBindingStore.cs new file mode 100644 index 00000000000..69453a77526 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteInMemoryStorageBindingStore.cs @@ -0,0 +1,44 @@ +namespace Bit.Bsqlite.Interop; + +/// +/// A binding store that remembers nothing beyond the current session. +/// +/// +/// The real store has to outlive the page - the whole point is to know where a database went before +/// this session started - so this is for tests and for configurations where every backend is itself +/// volatile. Production registers a store backed by durable browser storage. +/// +public sealed class BsqliteInMemoryStorageBindingStore : IBsqliteStorageBindingStore +{ + private readonly Dictionary _bindings = new(StringComparer.Ordinal); + + public ValueTask GetProviderNameAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + return ValueTask.FromResult( + _bindings.TryGetValue(databaseName, out var providerName) ? providerName : null); + } + + public ValueTask SetProviderNameAsync( + string databaseName, + string providerName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + ArgumentException.ThrowIfNullOrWhiteSpace(providerName); + + _bindings[databaseName] = providerName; + return ValueTask.CompletedTask; + } + + public ValueTask RemoveAsync(string databaseName, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + _bindings.Remove(databaseName); + return ValueTask.CompletedTask; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteLocalStorageBindingStore.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteLocalStorageBindingStore.cs new file mode 100644 index 00000000000..719b9a56bfd --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteLocalStorageBindingStore.cs @@ -0,0 +1,41 @@ +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Interop; + +/// +/// Remembers sticky bindings in localStorage so they survive a reload. +/// +public sealed class BsqliteLocalStorageBindingStore(IJSRuntime js) : IBsqliteStorageBindingStore +{ + private const string Prefix = "bit-bsqlite.binding."; + + public async ValueTask GetProviderNameAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + return await js + .InvokeAsync("localStorage.getItem", cancellationToken, Prefix + databaseName) + .ConfigureAwait(false); + } + + public async ValueTask SetProviderNameAsync( + string databaseName, + string providerName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + ArgumentException.ThrowIfNullOrWhiteSpace(providerName); + await js + .InvokeVoidAsync("localStorage.setItem", cancellationToken, Prefix + databaseName, providerName) + .ConfigureAwait(false); + } + + public async ValueTask RemoveAsync(string databaseName, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + await js + .InvokeVoidAsync("localStorage.removeItem", cancellationToken, Prefix + databaseName) + .ConfigureAwait(false); + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteStorageMigrator.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteStorageMigrator.cs new file mode 100644 index 00000000000..b38562a13cd --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteStorageMigrator.cs @@ -0,0 +1,209 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Interop; + +/// +/// Copies a database from one backend to another without flipping the binding until the copy +/// looks like a SQLite file. +/// +/// +/// +/// Probe target → source has anything → quota headroom → copy → header check → flip binding → +/// delete source. A failure leaves the source and the binding untouched. PRAGMA integrity_check +/// needs an engine on the target; the header check is what we can always do, including on desktop. +/// +/// +/// The one step after the flip is deleting the source, and it can fail - another tab still holding +/// an OPFS file open, say. If the source is still all there, the flip is undone and the copy dropped, +/// so the binding keeps naming the backend whose file is current. If it is already gone, the move +/// stands. Either way the failure is rethrown, and the binding is the record of where the database +/// now lives: a caller that carries on should resolve again rather than assume. +/// +/// +public sealed class BsqliteStorageMigrator +{ + private static ReadOnlySpan SqliteMagic => "SQLite format 3\0"u8; + + public async Task MigrateAsync( + string databaseName, + IBsqliteStorageProvider source, + IBsqliteStorageProvider target, + IBsqliteStorageBindingStore bindingStore, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + ArgumentNullException.ThrowIfNull(source); + ArgumentNullException.ThrowIfNull(target); + ArgumentNullException.ThrowIfNull(bindingStore); + + if (string.Equals(source.Name, target.Name, StringComparison.OrdinalIgnoreCase)) + { + return; + } + + var probe = await target.ProbeAsync(cancellationToken).ConfigureAwait(false); + if (!probe.IsAvailable) + { + throw new BsqliteStorageUnavailableException( + databaseName, + [ + new BsqliteStorageCandidateOutcome + { + ProviderName = target.Name, + Status = BsqliteStorageCandidateStatus.Unavailable, + Probe = probe, + Explanation = "The migration target is unavailable.", + }, + ]); + } + + // Nothing to move. Exporting anyway would throw - most backends report a missing database + // that way - and on the AutomaticOnOpen path that turns "a better backend appeared" into an + // open that fails outright. The in-memory backend reaches this every time: its admin store + // is not where a connection's data lives, so it never holds an image to copy. + if (!await source.Admin.ExistsAsync(databaseName, cancellationToken).ConfigureAwait(false)) + { + await bindingStore.SetProviderNameAsync(databaseName, target.Name, cancellationToken) + .ConfigureAwait(false); + return; + } + + var image = await source.Admin.ExportAsync(databaseName, cancellationToken).ConfigureAwait(false); + + if (probe.QuotaBytes is long quota + && probe.UsageBytes is long usage + && quota - usage < image.LongLength) + { + throw new BsqliteQuotaExceededException( + $"Moving '{databaseName}' to '{target.Name}' needs {image.Length} bytes, but the " + + $"origin only has {quota - usage} bytes free."); + } + + VerifySqliteImage(image); + + await target.Admin.ImportAsync(databaseName, image, cancellationToken).ConfigureAwait(false); + + try + { + var copy = await target.Admin.ExportAsync(databaseName, cancellationToken).ConfigureAwait(false); + if (!copy.AsSpan().SequenceEqual(image)) + { + throw new BsqliteCorruptDatabaseException( + $"The copy of '{databaseName}' on '{target.Name}' does not match the source."); + } + } + catch + { + await target.Admin.DeleteAsync(databaseName, cancellationToken).ConfigureAwait(false); + throw; + } + + // From the flip on nothing is cancellable. A cancel that landed after the binding was written, + // or while the source was still being deleted, would send the cleanup below after a copy that + // is the database by then. + cancellationToken.ThrowIfCancellationRequested(); + + try + { + await bindingStore.SetProviderNameAsync(databaseName, target.Name, CancellationToken.None) + .ConfigureAwait(false); + } + catch + { + // The copy is nobody's - unless the write landed and only its report failed. + if (!await BindingMayNameAsync(bindingStore, databaseName, target.Name).ConfigureAwait(false)) + { + await DeleteQuietlyAsync(target, databaseName).ConfigureAwait(false); + } + + throw; + } + + try + { + await source.Admin.DeleteAsync(databaseName, CancellationToken.None).ConfigureAwait(false); + } + catch + { + await UndoFlipIfSourceSurvivedAsync(databaseName, source, target, bindingStore).ConfigureAwait(false); + throw; + } + } + + /// Whether the binding names - or cannot be read to tell. + private static async Task BindingMayNameAsync( + IBsqliteStorageBindingStore bindingStore, + string databaseName, + string providerName) + { + try + { + return string.Equals( + await bindingStore.GetProviderNameAsync(databaseName).ConfigureAwait(false), + providerName, + StringComparison.OrdinalIgnoreCase); + } + catch (Exception) + { + return true; + } + } + + /// + /// Points the binding back at when the failed delete left it whole. + /// + /// + /// Runs on the failure path, so nothing here is cancellable and nothing here throws over the + /// original error. When it cannot tell whether the source survived, or cannot restore the + /// binding, the binding stays on the verified copy - the one place known to hold all the data. + /// + private static async Task UndoFlipIfSourceSurvivedAsync( + string databaseName, + IBsqliteStorageProvider source, + IBsqliteStorageProvider target, + IBsqliteStorageBindingStore bindingStore) + { + try + { + if (!await source.Admin.ExistsAsync(databaseName).ConfigureAwait(false)) + { + return; + } + + await bindingStore.SetProviderNameAsync(databaseName, source.Name).ConfigureAwait(false); + } + catch (Exception) + { + return; + } + + await DeleteQuietlyAsync(target, databaseName).ConfigureAwait(false); + } + + private static async Task DeleteQuietlyAsync(IBsqliteStorageProvider provider, string databaseName) + { + try + { + await provider.Admin.DeleteAsync(databaseName).ConfigureAwait(false); + } + catch (Exception) + { + // Left behind, but unbound: nothing opens it, and the next migration's import replaces it. + } + } + + internal static void VerifySqliteImage(ReadOnlyMemory image) + { + if (image.Length == 0) + { + return; + } + + if (image.Length < SqliteMagic.Length || !image.Span[..SqliteMagic.Length].SequenceEqual(SqliteMagic)) + { + throw new BsqliteCorruptDatabaseException( + "The exported image is not a SQLite database (missing the 'SQLite format 3' header)."); + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteStorageProviderResolver.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteStorageProviderResolver.cs new file mode 100644 index 00000000000..038eb53b5b7 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Storage/BsqliteStorageProviderResolver.cs @@ -0,0 +1,415 @@ +using Bit.Bsqlite.Storage; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; + +namespace Bit.Bsqlite.Interop; + +/// +/// Decides which storage backend serves a database, honouring configured preference but never at the +/// cost of data that already exists somewhere else. +/// +/// +/// +/// Two rules shape everything here. First, existing data outranks preference: if a database was +/// created on one backend, that backend is used, and if it cannot be reached the open fails rather +/// than quietly producing an empty database elsewhere. Second, nothing is substituted silently - a +/// fallback is logged, a non-persistent fallback needs an explicit opt-in, and total failure reports +/// every candidate's fate. +/// +/// +/// The binding store is how the first rule usually knows where the data is, but it is not the only +/// record: an application that clears localStorage on sign-out loses the bindings and keeps +/// the files. So a database with no binding is looked for on every available persistent backend +/// before it is treated as new, and found data is bound again rather than shadowed by an empty file +/// on a backend ranked higher. +/// +/// +/// makes no changes, so it is safe to call for diagnostics. The chosen +/// backend is recorded only when the caller confirms the database really opened, via +/// . +/// +/// +public sealed class BsqliteStorageProviderResolver +{ + private readonly Dictionary _providers; + private readonly IBsqliteStorageBindingStore _bindingStore; + private readonly ILogger _logger; + private readonly Dictionary _probeCache = + new(StringComparer.OrdinalIgnoreCase); + + public BsqliteStorageProviderResolver( + IEnumerable providers, + IBsqliteStorageBindingStore bindingStore, + ILogger? logger = null) + { + ArgumentNullException.ThrowIfNull(providers); + ArgumentNullException.ThrowIfNull(bindingStore); + + _bindingStore = bindingStore; + _logger = logger ?? NullLogger.Instance; + _providers = new Dictionary(StringComparer.OrdinalIgnoreCase); + + foreach (var provider in providers) + { + if (!_providers.TryAdd(provider.Name, provider)) + { + throw new InvalidOperationException( + $"Two storage providers are registered under the name '{provider.Name}'. " + + "Provider names identify where a database lives, so they must be unique."); + } + } + } + + /// Chooses a backend for without changing anything. + /// + /// No candidate could be used, or the database exists on a backend that cannot be reached. + /// + public async ValueTask ResolveAsync( + string databaseName, + BsqliteStorageSelection selection, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + ArgumentNullException.ThrowIfNull(selection); + + var boundProviderName = await _bindingStore + .GetProviderNameAsync(databaseName, cancellationToken) + .ConfigureAwait(false); + + if (boundProviderName is not null) + { + return await ResolveExistingDatabaseAsync(databaseName, boundProviderName, selection, cancellationToken) + .ConfigureAwait(false); + } + + var holder = await FindUnboundDataAsync(databaseName, selection, cancellationToken).ConfigureAwait(false); + if (holder is not null) + { + _logger.LogWarning( + "Database {DatabaseName} has no storage binding, but storage provider {Provider} holds " + + "it - the binding store was probably cleared. Opening the existing data there and " + + "binding it again.", + databaseName, + holder.Name); + + return await ResolveExistingDatabaseAsync(databaseName, holder.Name, selection, cancellationToken) + .ConfigureAwait(false) with + { + WasFoundWithoutBinding = true, + }; + } + + return await ResolveNewDatabaseAsync(databaseName, selection, cancellationToken).ConfigureAwait(false); + } + + /// + /// Records the resolved backend as the home of the database. Call this once the database has + /// actually opened, so a failed open does not leave a binding pointing at nothing. + /// + /// + /// A backend that does not persist is not recorded: its database is gone by the next load, so the + /// binding would protect no data - and, outranking preference, it would keep every later visit on + /// storage that empties on reload even once a persistent backend is available again. + /// + public ValueTask CommitBindingAsync( + BsqliteStorageResolution resolution, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(resolution); + + var needsCommit = resolution.WasFoundWithoutBinding + || (!resolution.WasDecidedByExistingData && resolution.Provider.Capabilities.IsPersistent); + + return needsCommit + ? _bindingStore.SetProviderNameAsync( + resolution.DatabaseName, + resolution.Provider.Name, + cancellationToken) + : ValueTask.CompletedTask; + } + + /// + /// Looks for on every available persistent backend, preferred + /// ones first, for a database whose binding has gone missing. + /// + /// + /// Non-persistent backends are skipped: whatever they held did not survive the load that lost + /// the binding. A backend whose admin surface cannot answer is passed over rather than allowed to + /// fail the open - the same leniency a throwing probe gets. + /// + private async ValueTask FindUnboundDataAsync( + string databaseName, + BsqliteStorageSelection selection, + CancellationToken cancellationToken) + { + var ranked = selection.Candidates + .Select(name => _providers.GetValueOrDefault(name)) + .Concat(_providers.Values) + .OfType() + .Distinct(); + + foreach (var provider in ranked) + { + if (!provider.Capabilities.IsPersistent) + { + continue; + } + + var probe = await ProbeAsync(provider, cancellationToken).ConfigureAwait(false); + if (!probe.IsAvailable) + { + continue; + } + + try + { + if (await provider.Admin.ExistsAsync(databaseName, cancellationToken).ConfigureAwait(false)) + { + return provider; + } + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + _logger.LogDebug( + ex, + "Could not check whether storage provider {Provider} holds database {DatabaseName}.", + provider.Name, + databaseName); + } + } + + return null; + } + + private async ValueTask ResolveExistingDatabaseAsync( + string databaseName, + string boundProviderName, + BsqliteStorageSelection selection, + CancellationToken cancellationToken) + { + if (!_providers.TryGetValue(boundProviderName, out var bound)) + { + // Falling through to preference order here would open a different, empty database and + // report success, so the only safe answer is to stop and say what is missing. + throw new BsqliteStorageUnavailableException( + databaseName, + [ + new BsqliteStorageCandidateOutcome + { + ProviderName = boundProviderName, + Status = BsqliteStorageCandidateStatus.NotRegistered, + Explanation = + $"Database '{databaseName}' was created by storage provider " + + $"'{boundProviderName}', which is not registered. Register it to reach " + + "the existing data, or delete the database to start again elsewhere.", + }, + ]); + } + + var probe = await ProbeAsync(bound, cancellationToken).ConfigureAwait(false); + + if (!probe.IsAvailable) + { + throw new BsqliteStorageUnavailableException( + databaseName, + [ + new BsqliteStorageCandidateOutcome + { + ProviderName = bound.Name, + Status = BsqliteStorageCandidateStatus.Unavailable, + Probe = probe, + Explanation = + $"Database '{databaseName}' lives on storage provider '{bound.Name}', which " + + "is unavailable in this browser. Bsqlite will not open an empty " + + "database on another provider in its place.", + }, + ]); + } + + var better = await FindBetterProviderAsync(bound.Name, selection, cancellationToken) + .ConfigureAwait(false); + + if (better is not null) + { + _logger.LogInformation( + "Database {DatabaseName} is on storage provider {Provider}, but {BetterProvider} is " + + "now available and ranked higher. Migration mode is {MigrationMode}.", + databaseName, + bound.Name, + better.Name, + selection.MigrationMode); + } + + return new BsqliteStorageResolution + { + DatabaseName = databaseName, + Provider = bound, + Probe = probe, + IsFirstChoice = selection.Candidates.Count > 0 + && string.Equals(selection.Candidates[0], bound.Name, StringComparison.OrdinalIgnoreCase), + WasDecidedByExistingData = true, + BetterProviderAvailable = better, + Attempts = + [ + new BsqliteStorageCandidateOutcome + { + ProviderName = bound.Name, + Status = BsqliteStorageCandidateStatus.Selected, + Probe = probe, + Explanation = $"Holds the existing database '{databaseName}'.", + }, + ], + }; + } + + private async ValueTask ResolveNewDatabaseAsync( + string databaseName, + BsqliteStorageSelection selection, + CancellationToken cancellationToken) + { + var attempts = new List(selection.Candidates.Count); + + for (var index = 0; index < selection.Candidates.Count; index++) + { + var candidateName = selection.Candidates[index]; + + if (!_providers.TryGetValue(candidateName, out var provider)) + { + attempts.Add(new BsqliteStorageCandidateOutcome + { + ProviderName = candidateName, + Status = BsqliteStorageCandidateStatus.NotRegistered, + Explanation = "No storage provider is registered under this name.", + }); + continue; + } + + var probe = await ProbeAsync(provider, cancellationToken).ConfigureAwait(false); + + if (!probe.IsAvailable) + { + attempts.Add(new BsqliteStorageCandidateOutcome + { + ProviderName = candidateName, + Status = BsqliteStorageCandidateStatus.Unavailable, + Probe = probe, + }); + continue; + } + + // A non-persistent first choice is what the application asked for. A non-persistent + // *fallback* is a silent downgrade to storage that empties on reload, so it needs consent. + if (index > 0 + && !provider.Capabilities.IsPersistent + && !selection.AllowNonPersistentFallback) + { + attempts.Add(new BsqliteStorageCandidateOutcome + { + ProviderName = candidateName, + Status = BsqliteStorageCandidateStatus.RejectedAsNonPersistent, + Probe = probe, + Explanation = + $"'{candidateName}' does not persist across reloads, and falling back to " + + "non-persistent storage would lose data without warning. Call " + + "AllowNonPersistentFallback() to accept that.", + }); + continue; + } + + attempts.Add(new BsqliteStorageCandidateOutcome + { + ProviderName = candidateName, + Status = BsqliteStorageCandidateStatus.Selected, + Probe = probe, + }); + + for (var rest = index + 1; rest < selection.Candidates.Count; rest++) + { + attempts.Add(new BsqliteStorageCandidateOutcome + { + ProviderName = selection.Candidates[rest], + Status = BsqliteStorageCandidateStatus.NotProbed, + Explanation = $"'{candidateName}' was selected first.", + }); + } + + if (index > 0) + { + _logger.LogWarning( + "Storage provider {Provider} was selected for database {DatabaseName} after " + + "{RejectedCount} higher-ranked provider(s) were rejected: {Rejected}", + provider.Name, + databaseName, + index, + string.Join("; ", attempts.Take(index))); + } + + return new BsqliteStorageResolution + { + DatabaseName = databaseName, + Provider = provider, + Probe = probe, + Attempts = attempts, + IsFirstChoice = index == 0, + }; + } + + throw new BsqliteStorageUnavailableException(databaseName, attempts); + } + + /// + /// Finds an available backend ranked above , which is what + /// makes "a better option exists" reportable rather than something the user has to guess at. + /// + private async ValueTask FindBetterProviderAsync( + string currentProviderName, + BsqliteStorageSelection selection, + CancellationToken cancellationToken) + { + foreach (var candidateName in selection.Candidates) + { + if (string.Equals(candidateName, currentProviderName, StringComparison.OrdinalIgnoreCase)) + { + return null; + } + + if (_providers.TryGetValue(candidateName, out var provider) + && provider.Capabilities.IsPersistent) + { + var probe = await ProbeAsync(provider, cancellationToken).ConfigureAwait(false); + if (probe.IsAvailable) + { + return provider; + } + } + } + + return null; + } + + private async ValueTask ProbeAsync( + IBsqliteStorageProvider provider, + CancellationToken cancellationToken) + { + if (_probeCache.TryGetValue(provider.Name, out var cached)) + { + return cached; + } + + BsqliteProbeResult probe; + try + { + probe = await provider.ProbeAsync(cancellationToken).ConfigureAwait(false); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + // A provider that throws instead of reporting unavailability must not take the whole + // selection down with it; the next candidate still deserves a chance. + probe = BsqliteProbeResult.Unavailable( + $"Probing '{provider.Name}' threw {ex.GetType().Name}: {ex.Message}"); + } + + _probeCache[provider.Name] = probe; + return probe; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransport.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransport.cs new file mode 100644 index 00000000000..92249cbccd1 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransport.cs @@ -0,0 +1,368 @@ +using System.Reflection; +using System.Text.Json; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Interop; + +/// +/// An that drives the browser worker over Blazor JS interop. +/// +/// +/// +/// The worker is the process; this type is the socket. It imports the host module, asks it for a +/// host object, and then speaks in call envelopes for control requests and in the binary +/// wire format for SQL - never by letting a JavaScript exception cross the boundary, because Blazor +/// would strip the SQLite result code on the way. +/// +/// +/// SQL batches and their replies cross as byte arrays described by +/// . Blazor marshals byte arrays without base64 or JSON, and +/// the host transfers the buffers to and from the worker rather than copying them, so a result set +/// is encoded once in the worker and decoded once here. +/// +/// +public sealed class BsqliteWorkerTransport : IBsqliteTransport, IBsqliteSnapshotSource, IBsqliteLockSource +{ + /// The RCL path the worker host is served from. + public const string DefaultHostModuleUrl = "./_content/Bit.Bsqlite/bit-bsqlite-host.js"; + + /// + /// stamped with this assembly's version, and what a + /// transport imports unless an application says otherwise. + /// + /// + /// The module and this assembly ship as one unit, so a browser that answers the import from a + /// copy it cached under an earlier version pairs new .NET with old JavaScript - which surfaces + /// as a missing export rather than as the version skew it is. No application can evict another + /// machine's module cache, so the version goes in the query instead: an upgrade then asks for a + /// URL no cache has an entry for. + /// + public static string VersionedHostModuleUrl { get; } = BuildVersionedHostModuleUrl(); + + private readonly IJSRuntime _js; + private readonly BsqliteWorkerTransportOptions _options; + private IJSObjectReference? _module; + private IJSObjectReference? _host; + private DotNetObjectReference? _self; + private string? _databaseName; + private int _nextLockId; + private bool _disposed; + + /// + public event EventHandler? TablesChanged; + + /// + /// + /// True: the worker reports what it wrote from SQLite's own update hook, cascades and triggers + /// included, and only once the transaction that wrote it has committed - which is more than + /// the command layer could learn from the SQL text. + /// + public bool ReportsLocalWrites => true; + + /// + /// Called from bit-bsqlite-host.js when a committed write - this tab's or another's - + /// touched tables in this database. + /// + /// + /// Public because [JSInvokable] requires it. The host module has already dropped other + /// databases' traffic, so everything arriving here concerns the database this transport opened. + /// + [JSInvokable] + public void OnTablesChanged(string[] tables) + => TablesChanged?.Invoke(this, new BsqliteTablesChangedEventArgs(tables ?? [])); + + public BsqliteWorkerTransport(IJSRuntime js, BsqliteWorkerTransportOptions options) + { + ArgumentNullException.ThrowIfNull(js); + ArgumentNullException.ThrowIfNull(options); + ArgumentException.ThrowIfNullOrWhiteSpace(options.HostModuleUrl); + + _js = js; + _options = options; + } + + /// + public async Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + ObjectDisposedException.ThrowIf(_disposed, this); + + _module ??= await _js + .InvokeAsync("import", cancellationToken, _options.HostModuleUrl) + .ConfigureAwait(false); + + _host ??= await _module + .InvokeAsync("createHost", cancellationToken, _options.WorkerUrl) + .ConfigureAwait(false); + + if (_self is null) + { + // Before the open, so a write another tab performs while this one is still opening is + // not missed. + var self = DotNetObjectReference.Create(this); + try + { + await _module + .InvokeVoidAsync("listen", cancellationToken, _host, self, databaseName) + .ConfigureAwait(false); + } + catch (JSException ex) when (ex.Message.Contains("is not a function", StringComparison.Ordinal)) + { + // A host module without `listen` is not one this package ever shipped, so either the + // browser is serving a copy older than this assembly or the reference is not the + // module at all. Interop reports only that the export is missing, which sends the + // reader hunting through source that plainly exports it - so name what the object + // actually is. A module namespace lists its exports; anything else does not. + self.Dispose(); + var exports = await DescribeAsync(_module, cancellationToken).ConfigureAwait(false); + throw new InvalidOperationException( + $"'{_options.HostModuleUrl}' has no 'listen' export. The imported object " + + $"exposes: {exports}. If 'listen' is missing from that list the browser served a " + + "cached copy of the host module older than this assembly - clear the site's data " + + "and reload with Ctrl+Shift+R, and check that no service worker is serving it.", + ex); + } + catch + { + self.Dispose(); + throw; + } + + _self = self; + } + + await CallAsync( + new + { + kind = "open", + databaseName, + requiredBuild = EncodeBuild(_options.RequiredBuild), + vfs = EncodeModule(_options.Vfs), + limits = new + { + supportsMultiDatabaseTransactions = _options.SupportsMultiDatabaseTransactions, + canChangePageSize = _options.CanChangePageSize, + }, + foreignKeys = _options.EnableForeignKeys, + onOpenSql = _options.OnOpenSql, + extensions = _options.ExtensionModules.Select(EncodeModule).ToArray(), + }, + cancellationToken).ConfigureAwait(false); + + _databaseName = databaseName; + } + + /// + public async Task CloseAsync(CancellationToken cancellationToken = default) + { + ObjectDisposedException.ThrowIf(_disposed, this); + + if (_host is null) + { + return; + } + + await CallAsync(new { kind = "close" }, cancellationToken).ConfigureAwait(false); + } + + /// + public async Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(batch); + ObjectDisposedException.ThrowIf(_disposed, this); + + if (_host is null) + { + throw new InvalidOperationException("The transport is not open."); + } + + var reply = await _host + .InvokeAsync("executeBinary", cancellationToken, BsqliteWireFormat.EncodeBatch(batch)) + .ConfigureAwait(false); + + return BsqliteWireFormat.DecodeReply(reply); + } + + /// + public async Task ExportAsync(CancellationToken cancellationToken = default) + { + ObjectDisposedException.ThrowIf(_disposed, this); + + if (_host is null) + { + throw new InvalidOperationException("The transport is not open."); + } + + try + { + return await _host + .InvokeAsync("export", cancellationToken, "main") + .ConfigureAwait(false); + } + catch (JSException ex) + { + throw new BsqliteException($"The database could not be exported: {ex.Message}"); + } + } + + /// + /// + /// A Web Lock named after the database, taken on the main thread: every tab's worker opens the + /// same file, so the lock has to be one the browser shares between them, and one it drops when + /// the tab holding it goes away. + /// + public async Task AcquireLockAsync(string name, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(name); + ObjectDisposedException.ThrowIf(_disposed, this); + + if (_host is null || _databaseName is null) + { + throw new InvalidOperationException("The transport is not open."); + } + + var id = Interlocked.Increment(ref _nextLockId); + try + { + await _host + .InvokeVoidAsync("acquireLock", cancellationToken, $"bit-bsqlite:{_databaseName}:{name}", id) + .ConfigureAwait(false); + } + catch + { + // Given up on here, but the request may still be queued in the browser - or granted by + // now - and a lock nobody releases stalls every other tab until this one closes. + await ReleaseLockAsync(id).ConfigureAwait(false); + throw; + } + + return new HeldLock(this, id); + } + + private async ValueTask ReleaseLockAsync(int id) + { + if (_host is not { } host) + { + // Disposed: the host released every lock it held when it went. + return; + } + + try + { + await host.InvokeVoidAsync("releaseLock", id).ConfigureAwait(false); + } + catch (JSException) + { + // The host is already gone, and its locks with it. + } + } + + /// + public async ValueTask DisposeAsync() + { + if (_disposed) + { + return; + } + + _disposed = true; + + if (_host is not null) + { + // Terminate rather than close: dispose means the worker is finished, not that EF opened + // and closed a connection around a query. A close would leave a live worker with no + // owner, which is a leak in a browser that charges for every one. + try + { + await _host.InvokeVoidAsync("dispose").ConfigureAwait(false); + } + catch (JSException) + { + // The worker may already be gone; the JS references still need releasing. + } + + await _host.DisposeAsync().ConfigureAwait(false); + _host = null; + } + + if (_module is not null) + { + await _module.DisposeAsync().ConfigureAwait(false); + _module = null; + } + + // After the host, so no notification can arrive for a reference that is already gone. + _self?.Dispose(); + _self = null; + TablesChanged = null; + } + + /// + /// Names what a JS reference actually is, for the one error where that is the whole question. + /// + private async Task DescribeAsync( + IJSObjectReference reference, + CancellationToken cancellationToken) + { + try + { + var keys = await _js + .InvokeAsync("Object.keys", cancellationToken, reference) + .ConfigureAwait(false); + + return keys.Length == 0 ? "no enumerable members" : string.Join(", ", keys); + } + catch (JSException) + { + // The diagnostic must never replace the failure it is describing. + return "unknown"; + } + } + + private async Task CallAsync(object request, CancellationToken cancellationToken) + { + var envelope = await _host! + .InvokeAsync("call", cancellationToken, request) + .ConfigureAwait(false); + + return BsqliteWireFormat.DecodeCall(envelope); + } + + private static string BuildVersionedHostModuleUrl() + { + var assembly = typeof(BsqliteWorkerTransport).Assembly; + + // The informational version carries the commit for a CI build, which is a finer cache key + // than the three-part version a series of prereleases would share. + var version = assembly.GetCustomAttribute() + ?.InformationalVersion ?? assembly.GetName().Version?.ToString(); + + return string.IsNullOrWhiteSpace(version) + ? DefaultHostModuleUrl + : $"{DefaultHostModuleUrl}?v={Uri.EscapeDataString(version)}"; + } + + private static string EncodeBuild(BsqliteEngineBuild build) => build switch + { + BsqliteEngineBuild.Synchronous => "synchronous", + BsqliteEngineBuild.AsyncCapable => "asyncCapable", + _ => throw new ArgumentOutOfRangeException(nameof(build), build, "Unknown engine build."), + }; + + private static object? EncodeModule(BsqliteJsModule? module) + => module is null ? null : new { moduleUrl = module.ModuleUrl, registerExport = module.RegisterExport }; + + private sealed class HeldLock(BsqliteWorkerTransport transport, int id) : IAsyncDisposable + { + private int _released; + + public ValueTask DisposeAsync() + => Interlocked.Exchange(ref _released, 1) == 0 + ? transport.ReleaseLockAsync(id) + : ValueTask.CompletedTask; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransportFactory.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransportFactory.cs new file mode 100644 index 00000000000..9f8a30e5566 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransportFactory.cs @@ -0,0 +1,61 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Interop; + +/// +/// Builds a from whatever backend selection chose. +/// +/// +/// The given at construction supply everything that is not decided by +/// the provider: host and worker URLs, open-time pragmas, and extension modules. The provider +/// supplies the engine build, the VFS, and the capability limits. +/// +public sealed class BsqliteWorkerTransportFactory(IJSRuntime js, BsqliteWorkerTransportDefaults? defaults = null) + : IBsqliteTransportFactory +{ + private readonly IJSRuntime _js = js ?? throw new ArgumentNullException(nameof(js)); + private readonly BsqliteWorkerTransportDefaults _defaults = defaults ?? new BsqliteWorkerTransportDefaults(); + + /// + public IBsqliteTransport Create(IBsqliteStorageProvider provider) + { + ArgumentNullException.ThrowIfNull(provider); + + return new BsqliteWorkerTransport(_js, new BsqliteWorkerTransportOptions + { + HostModuleUrl = _defaults.HostModuleUrl, + WorkerUrl = _defaults.WorkerUrl, + RequiredBuild = provider.Capabilities.RequiredBuild, + Vfs = provider.VfsModule, + SupportsMultiDatabaseTransactions = provider.Capabilities.SupportsMultiDatabaseTransactions, + CanChangePageSize = provider.Capabilities.CanChangePageSize, + EnableForeignKeys = _defaults.EnableForeignKeys, + OnOpenSql = _defaults.OnOpenSql, + ExtensionModules = _defaults.ExtensionModules, + }); + } +} + +/// +/// The parts of that do not depend on which storage +/// backend was selected, applied to every transport the factory creates. +/// +public sealed class BsqliteWorkerTransportDefaults +{ + /// + public string HostModuleUrl { get; init; } = BsqliteWorkerTransport.VersionedHostModuleUrl; + + /// + public string? WorkerUrl { get; init; } + + /// + public bool EnableForeignKeys { get; init; } = true; + + /// + public IReadOnlyList OnOpenSql { get; init; } = []; + + /// + public IReadOnlyList ExtensionModules { get; init; } = []; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransportOptions.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransportOptions.cs new file mode 100644 index 00000000000..e71fca152c2 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/BsqliteWorkerTransportOptions.cs @@ -0,0 +1,68 @@ +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Interop; + +/// +/// How a opens a database in the browser. +/// +/// +/// The transport does not resolve storage itself: selection happens first, and this carries the +/// outcome - which engine build the chosen backend needs, and which VFS module to register. That +/// keeps the worker host ignorant of preference, sticky binding, and fallback. +/// +public sealed class BsqliteWorkerTransportOptions +{ + /// + /// The ES module that exports createHost. Override only when an application hosts the + /// worker from a path other than the RCL default. + /// + public string HostModuleUrl { get; init; } = BsqliteWorkerTransport.VersionedHostModuleUrl; + + /// + /// The worker script the host should spawn. uses the host module's own + /// default, which is what production wants. + /// + public string? WorkerUrl { get; init; } + + /// The engine build the selected backend's VFS requires. + public required BsqliteEngineBuild RequiredBuild { get; init; } + + /// + /// The selected backend's VFS module, or when the engine's built-in + /// memory VFS is enough. + /// + public BsqliteJsModule? Vfs { get; init; } + + /// + /// Whether the selected backend can honour ATTACH. Forwarded to the worker so a + /// JavaScript caller cannot bypass the .NET guard. + /// + public bool SupportsMultiDatabaseTransactions { get; init; } = true; + + /// Whether the selected backend can honour PRAGMA page_size=…. + public bool CanChangePageSize { get; init; } = true; + + /// + /// Whether to run PRAGMA foreign_keys = ON when the database opens. On by default, + /// because that is what Microsoft.Data.Sqlite does and what an EF Core model scaffolded on the + /// server expects: a RESTRICT or CASCADE rule that silently does nothing in the + /// browser is a data bug, not a performance setting. + /// + public bool EnableForeignKeys { get; init; } = true; + + /// + /// Extra statements to run once the database is open, after the provider's own and after + /// : PRAGMA cache_size, PRAGMA synchronous, and + /// the like. They run outside the SQL guards, so WAL cannot be enabled this way either. + /// + public IReadOnlyList OnOpenSql { get; init; } = []; + + /// + /// JavaScript modules that register user-defined functions, collations, or virtual tables on + /// the open database. Each exports a function - register unless the module says + /// otherwise - receiving { module, sqlite3, db, databaseName }, the same handles the + /// built-in EF function set uses. They load inside the worker, so they must be served from a URL + /// the worker can import. + /// + public IReadOnlyList ExtensionModules { get; init; } = []; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Transport/IBsqliteTransportFactory.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/IBsqliteTransportFactory.cs new file mode 100644 index 00000000000..62d9a940bad --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Transport/IBsqliteTransportFactory.cs @@ -0,0 +1,21 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Interop; + +/// +/// Creates the transport that will talk to a chosen storage backend. +/// +/// +/// Separated from selection so a test can resolve against real providers and still run the open +/// against an in-process engine, and so the Arch 2 tier can replace the worker transport without +/// touching the resolver. +/// +public interface IBsqliteTransportFactory +{ + /// + /// A transport configured for - its required engine build and VFS + /// module - but not yet opened. + /// + IBsqliteTransport Create(IBsqliteStorageProvider provider); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Interop/Wire/BsqliteWireFormat.cs b/src/Bsqlite/Bit.Bsqlite/Interop/Wire/BsqliteWireFormat.cs new file mode 100644 index 00000000000..48891544158 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Interop/Wire/BsqliteWireFormat.cs @@ -0,0 +1,563 @@ +using System.Buffers; +using System.Buffers.Binary; +using System.Globalization; +using System.Text; +using System.Text.Json; +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Interop; + +/// +/// Translates between the ADO.NET layer's types and the bytes the browser transport carries. +/// +/// +/// +/// A wire format exists because JSON cannot represent two things SQLite returns routinely: integers +/// outside the range a JavaScript number holds exactly, and blobs. Left to a plain JSON projection, +/// a 64-bit key silently loses its low bits and a blob arrives as an array of numbers. Both failures +/// are quiet, which is why the format is explicit rather than incidental. +/// +/// +/// The format is binary and little-endian. A batch is one byte array, a reply is one byte array, and +/// Blazor marshals byte arrays across JS interop without base64 or JSON parsing, so a result set +/// costs one allocation and a linear scan rather than a document object model. Every value travels +/// with its SQLite storage class, using SQLite's own type codes so the JavaScript half can label +/// values from sqlite3_column_type without a translation table. +/// bit-bsqlite-wire.js is the other half of this file and must agree with it byte for byte. +/// +/// +/// Layout of a batch: magic, version, request count, then per request the SQL (length-prefixed +/// UTF-8), a result-kind byte, a signed 32-bit row limit (-1 for none), and the parameters as +/// name, type tag, and payload. Layout of a reply: magic, version, status; on failure a message, a +/// result code (-1 for none), and an error name; on success a result count and per result the +/// columns (name and storage class), records affected, last insert rowid, and the rows as tagged +/// cells. +/// +/// +public static class BsqliteWireFormat +{ + /// First byte of every message, so a stray buffer is rejected rather than misread. + public const byte Magic = 0xB5; + + /// Bumped whenever the layout changes; the decoder refuses anything else. + public const byte Version = 1; + + private const byte StatusOk = 0; + private const byte StatusError = 1; + + /// SQLite's storage class codes, used verbatim as the wire type tags. + public static class TypeCode + { + /// SQLITE_INTEGER. + public const int Integer = 1; + + /// SQLITE_FLOAT. + public const int Real = 2; + + /// SQLITE_TEXT. + public const int Text = 3; + + /// SQLITE_BLOB. + public const int Blob = 4; + + /// SQLITE_NULL. + public const int Null = 5; + } + + private static readonly string[] StorageClassNames = [string.Empty, "INTEGER", "REAL", "TEXT", "BLOB"]; + + // --------------------------------------------------------------------------------------------- + // Encoding a batch + // --------------------------------------------------------------------------------------------- + + /// Encodes a batch for the worker. + public static byte[] EncodeBatch(IReadOnlyList batch) + { + ArgumentNullException.ThrowIfNull(batch); + + var writer = new WireWriter(EstimateBatchSize(batch)); + writer.WriteByte(Magic); + writer.WriteByte(Version); + writer.WriteUInt32((uint)batch.Count); + + for (var i = 0; i < batch.Count; i++) + { + var request = batch[i]; + writer.WriteString(request.CommandText); + writer.WriteByte(EncodeResultKind(request.ResultKind)); + writer.WriteInt32(request.RowLimit is { } limit && limit >= 0 ? limit : -1); + + var parameters = request.Parameters; + if (parameters.Count > ushort.MaxValue) + { + throw new NotSupportedException( + $"A request carries {parameters.Count} parameters; the wire format allows {ushort.MaxValue}."); + } + + writer.WriteUInt16((ushort)parameters.Count); + for (var p = 0; p < parameters.Count; p++) + { + var parameter = parameters[p]; + writer.WriteString(parameter.Name); + WriteValue(ref writer, parameter.Value, parameter.Name); + } + } + + return writer.ToArray(); + } + + private static int EstimateBatchSize(IReadOnlyList batch) + { + var size = 16; + for (var i = 0; i < batch.Count; i++) + { + var request = batch[i]; + size += 16 + request.CommandText.Length * 2; + for (var p = 0; p < request.Parameters.Count; p++) + { + size += 24 + request.Parameters[p].Value switch + { + string text => text.Length * 2, + byte[] blob => blob.Length, + _ => 8, + }; + } + } + + return size; + } + + private static byte EncodeResultKind(BsqliteResultKind kind) => kind switch + { + BsqliteResultKind.NonQuery => 0, + BsqliteResultKind.Scalar => 1, + BsqliteResultKind.Reader => 2, + _ => throw new ArgumentOutOfRangeException(nameof(kind), kind, "Unknown result kind."), + }; + + private static void WriteValue(ref WireWriter writer, object? value, string parameterName) + { + var (type, normalized) = EncodeValue(value, parameterName); + writer.WriteByte((byte)type); + + switch (type) + { + case TypeCode.Null: + break; + + case TypeCode.Integer: + writer.WriteInt64((long)normalized!); + break; + + case TypeCode.Real: + writer.WriteDouble((double)normalized!); + break; + + case TypeCode.Text: + writer.WriteString((string)normalized!); + break; + + case TypeCode.Blob: + writer.WriteBytes((ReadOnlyMemory)normalized!); + break; + + default: + throw new ArgumentOutOfRangeException(nameof(value), type, "Unknown wire type."); + } + } + + /// + /// Maps a parameter value onto a storage class and the single representation the wire carries + /// for it: , , , + /// of , or . + /// + /// + /// Only conversions with one obvious answer are performed here. Dates, times, and GUIDs have more + /// than one reasonable SQLite representation, and choosing silently would put this layer in + /// disagreement with the ADO.NET binder that owns the decision, so they are rejected with a message + /// saying so. converts them before a parameter reaches the + /// transport. + /// + public static (int Type, object? Value) EncodeValue(object? value, string parameterName) + { + switch (value) + { + case null or DBNull: + return (TypeCode.Null, null); + + // SQLite has no boolean storage class; it stores 0 and 1, as every SQLite provider does. + case bool flag: + return (TypeCode.Integer, flag ? 1L : 0L); + + case long int64: + return (TypeCode.Integer, int64); + + case int int32: + return (TypeCode.Integer, (long)int32); + + case byte or sbyte or short or ushort or uint: + return (TypeCode.Integer, Convert.ToInt64(value, CultureInfo.InvariantCulture)); + + case ulong unsigned when unsigned <= long.MaxValue: + return (TypeCode.Integer, (long)unsigned); + + case ulong tooLarge: + throw new NotSupportedException( + $"Parameter '{parameterName}' is {tooLarge}, which exceeds the range of SQLite's " + + "64-bit signed INTEGER."); + + // SQLite stores NaN as NULL, so it is sent as one; infinities are ordinary REALs. + case double real: + return double.IsNaN(real) ? (TypeCode.Null, null) : (TypeCode.Real, real); + + case float single: + return float.IsNaN(single) ? (TypeCode.Null, null) : (TypeCode.Real, (double)single); + + // Text, so that no precision is lost. Comparisons and arithmetic over these columns are + // what the ef_* function set exists to make correct. + case decimal money: + return (TypeCode.Text, FormatDecimal(money)); + + case string text: + return (TypeCode.Text, text); + + case char character: + return (TypeCode.Text, character.ToString()); + + case byte[] blob: + return (TypeCode.Blob, new ReadOnlyMemory(blob)); + + case ReadOnlyMemory memory: + return (TypeCode.Blob, memory); + + case Memory memory: + return (TypeCode.Blob, (ReadOnlyMemory)memory); + + case ArraySegment segment: + return (TypeCode.Blob, (ReadOnlyMemory)segment); + + default: + throw new NotSupportedException( + $"Parameter '{parameterName}' is of type {value.GetType()}, which the transport does " + + "not convert. Values with more than one sensible SQLite representation - dates, " + + "times, and GUIDs among them - must be converted by the ADO.NET binder before " + + "reaching the transport, so that one component owns the choice."); + } + } + + /// + /// Writes a decimal the way Microsoft.Data.Sqlite writes one, which is the canonical TEXT form + /// for the whole SQLite/EF stack. + /// + /// + /// Not ToString(InvariantCulture), which preserves the CLR value's trailing zeros: that + /// renders 10m as "10" and 1.50m as "1.50", while the rest of the + /// stack writes "10.0" and "1.5". The difference is not cosmetic. EF Core compiles + /// decimal equality to a plain TEXT comparison against a literal it renders with this same + /// format - WHERE "Price" = '10.0' - so a row the browser wrote as "10" would + /// never match, silently. It also makes a database written in the browser differ byte-for-byte + /// from the same one written on the server, which is the promise this library exists to keep. + /// + internal static string FormatDecimal(decimal value) + => value.ToString(DecimalFormat, CultureInfo.InvariantCulture); + + /// Microsoft.Data.Sqlite's decimal format string, reproduced exactly. + private const string DecimalFormat = "0.0###########################"; + + // --------------------------------------------------------------------------------------------- + // Decoding a reply + // --------------------------------------------------------------------------------------------- + + /// Decodes the worker's reply into results the ADO.NET layer can read. + /// The worker reported a failure. + /// The reply did not match the wire format. + public static IReadOnlyList DecodeReply(ReadOnlySpan reply) + { + var reader = new WireReader(reply); + + var magic = reader.ReadByte(); + if (magic != Magic) + { + throw new FormatException($"Not a Bsqlite wire message (leading byte 0x{magic:X2})."); + } + + var version = reader.ReadByte(); + if (version != Version) + { + throw new FormatException( + $"Unsupported wire version {version}; this build speaks version {Version}. " + + "The browser is running Bsqlite JavaScript from a different release than this " + + "assembly - typically a cached copy; reload without the cache."); + } + + var status = reader.ReadByte(); + if (status == StatusError) + { + var message = reader.ReadString(); + var code = reader.ReadInt32(); + var name = reader.ReadString(); + throw CreateException(message, code < 0 ? null : code, name); + } + + if (status != StatusOk) + { + throw new FormatException($"Unknown reply status {status}."); + } + + var count = checked((int)reader.ReadUInt32()); + var results = new BsqliteCommandResult[count]; + for (var i = 0; i < count; i++) + { + results[i] = ReadResult(ref reader); + } + + if (reader.Remaining != 0) + { + throw new FormatException($"The reply carries {reader.Remaining} unexpected trailing byte(s)."); + } + + return results; + } + + private static BsqliteCommandResult ReadResult(ref WireReader reader) + { + var columnCount = reader.ReadUInt16(); + var columnNames = columnCount == 0 ? [] : new string[columnCount]; + var columnTypes = columnCount == 0 ? [] : new string?[columnCount]; + + for (var c = 0; c < columnCount; c++) + { + columnNames[c] = reader.ReadString(); + var storageClass = reader.ReadByte(); + columnTypes[c] = storageClass is >= 1 and <= 4 ? StorageClassNames[storageClass] : null; + } + + var recordsAffected = reader.ReadInt32(); + var lastInsertRowId = reader.ReadInt64(); + var rowCount = checked((int)reader.ReadUInt32()); + var rows = rowCount == 0 ? [] : new object?[rowCount][]; + + for (var r = 0; r < rowCount; r++) + { + var cells = columnCount == 0 ? [] : new object?[columnCount]; + for (var c = 0; c < columnCount; c++) + { + cells[c] = ReadValue(ref reader); + } + + rows[r] = cells; + } + + return new BsqliteCommandResult + { + ColumnNames = columnNames, + ColumnTypes = columnTypes, + Rows = rows, + RecordsAffected = recordsAffected, + LastInsertRowId = lastInsertRowId, + }; + } + + private static object? ReadValue(ref WireReader reader) + { + var type = reader.ReadByte(); + return type switch + { + TypeCode.Null => null, + TypeCode.Integer => reader.ReadInt64(), + TypeCode.Real => reader.ReadDouble(), + TypeCode.Text => reader.ReadString(), + TypeCode.Blob => reader.ReadBytes(), + _ => throw new FormatException($"Unknown wire type tag {type}."), + }; + } + + // --------------------------------------------------------------------------------------------- + // The JSON envelope used by control requests (open, close, version) + // --------------------------------------------------------------------------------------------- + + /// + /// Reads the envelope returned by the host's call method, which never throws across JS + /// interop - Blazor would otherwise reduce a JavaScript error to its message and drop the SQLite + /// result code. + /// + /// The worker reported a failure. + /// The reply did not match the envelope. + public static JsonElement DecodeCall(JsonElement envelope) + { + if (envelope.ValueKind != JsonValueKind.Object + || !envelope.TryGetProperty("ok", out var ok) + || ok.ValueKind is not (JsonValueKind.True or JsonValueKind.False)) + { + throw new FormatException( + "Expected a { ok, result } or { ok, error } envelope from the worker."); + } + + if (ok.GetBoolean()) + { + return envelope.TryGetProperty("result", out var result) + ? result + : default; + } + + throw ReadError(envelope); + } + + private static BsqliteException ReadError(JsonElement envelope) + { + if (!envelope.TryGetProperty("error", out var error) + || error.ValueKind != JsonValueKind.Object) + { + throw new BsqliteException("The SQLite worker reported a failure without a message."); + } + + var message = error.TryGetProperty("message", out var text) && text.ValueKind == JsonValueKind.String + ? text.GetString() + : null; + + int? code = error.TryGetProperty("sqliteCode", out var sqliteCode) + && sqliteCode.ValueKind == JsonValueKind.Number + ? sqliteCode.GetInt32() + : null; + + var name = error.TryGetProperty("name", out var nameElement) + && nameElement.ValueKind == JsonValueKind.String + ? nameElement.GetString() + : null; + + return CreateException(message, code, name); + } + + /// + /// Builds the typed exception for a worker failure, so every path - binary reply, JSON envelope, + /// or a transport's own error - classifies result codes the same way. + /// + public static BsqliteException CreateException(string? message, int? sqliteErrorCode, string? errorName) + { + var text = string.IsNullOrWhiteSpace(message) + ? "The SQLite worker reported a failure." + : message; + + if (sqliteErrorCode is 13 || string.Equals(errorName, "QuotaExceededError", StringComparison.Ordinal)) + { + return new BsqliteQuotaExceededException(text, sqliteErrorCode ?? 13); + } + + return sqliteErrorCode switch + { + 11 or 26 => new BsqliteCorruptDatabaseException(text, sqliteErrorCode), + 5 or 6 => new BsqliteConcurrencyException(text, sqliteErrorCode), + 19 => new BsqliteConstraintException(text, sqliteErrorCode), + _ => new BsqliteException(text, sqliteErrorCode), + }; + } + + // --------------------------------------------------------------------------------------------- + // Primitives + // --------------------------------------------------------------------------------------------- + + private ref struct WireWriter + { + private byte[] _buffer; + private int _length; + + public WireWriter(int initialCapacity) + { + _buffer = ArrayPool.Shared.Rent(Math.Max(initialCapacity, 256)); + _length = 0; + } + + private Span Reserve(int count) + { + if (_length + count > _buffer.Length) + { + var grown = ArrayPool.Shared.Rent(Math.Max(_buffer.Length * 2, _length + count)); + _buffer.AsSpan(0, _length).CopyTo(grown); + ArrayPool.Shared.Return(_buffer); + _buffer = grown; + } + + var span = _buffer.AsSpan(_length, count); + _length += count; + return span; + } + + public void WriteByte(byte value) => Reserve(1)[0] = value; + + public void WriteUInt16(ushort value) => BinaryPrimitives.WriteUInt16LittleEndian(Reserve(2), value); + + public void WriteInt32(int value) => BinaryPrimitives.WriteInt32LittleEndian(Reserve(4), value); + + public void WriteUInt32(uint value) => BinaryPrimitives.WriteUInt32LittleEndian(Reserve(4), value); + + public void WriteInt64(long value) => BinaryPrimitives.WriteInt64LittleEndian(Reserve(8), value); + + public void WriteDouble(double value) => BinaryPrimitives.WriteDoubleLittleEndian(Reserve(8), value); + + public void WriteBytes(ReadOnlyMemory bytes) + { + WriteUInt32((uint)bytes.Length); + bytes.Span.CopyTo(Reserve(bytes.Length)); + } + + public void WriteString(string text) + { + var byteCount = Encoding.UTF8.GetByteCount(text); + WriteUInt32((uint)byteCount); + Encoding.UTF8.GetBytes(text, Reserve(byteCount)); + } + + public byte[] ToArray() + { + var result = _buffer.AsSpan(0, _length).ToArray(); + ArrayPool.Shared.Return(_buffer); + _buffer = []; + return result; + } + } + + private ref struct WireReader(ReadOnlySpan data) + { + private readonly ReadOnlySpan _data = data; + private int _position = 0; + + public readonly int Remaining => _data.Length - _position; + + private ReadOnlySpan Take(int count) + { + if (count < 0 || _position + count > _data.Length) + { + throw new FormatException( + $"Truncated wire message: needed {count} byte(s) at offset {_position} of {_data.Length}."); + } + + var span = _data.Slice(_position, count); + _position += count; + return span; + } + + public byte ReadByte() => Take(1)[0]; + + public ushort ReadUInt16() => BinaryPrimitives.ReadUInt16LittleEndian(Take(2)); + + public int ReadInt32() => BinaryPrimitives.ReadInt32LittleEndian(Take(4)); + + public uint ReadUInt32() => BinaryPrimitives.ReadUInt32LittleEndian(Take(4)); + + public long ReadInt64() => BinaryPrimitives.ReadInt64LittleEndian(Take(8)); + + public double ReadDouble() => BinaryPrimitives.ReadDoubleLittleEndian(Take(8)); + + public byte[] ReadBytes() + { + var length = checked((int)ReadUInt32()); + return Take(length).ToArray(); + } + + public string ReadString() + { + var length = checked((int)ReadUInt32()); + return length == 0 ? string.Empty : Encoding.UTF8.GetString(Take(length)); + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/LockCompat.cs b/src/Bsqlite/Bit.Bsqlite/LockCompat.cs new file mode 100644 index 00000000000..20f3ca6f6a3 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/LockCompat.cs @@ -0,0 +1,6 @@ +// System.Threading.Lock only exists from .NET 9 onwards. On net8.0 the name is aliased to the +// classic monitor object: `new()` and every `lock (...)` site compile unchanged, they just take +// the Monitor path instead of the newer type's fast path. +#if !NET9_0_OR_GREATER +global using Lock = System.Object; +#endif diff --git a/src/Bsqlite/Bit.Bsqlite/README.md b/src/Bsqlite/Bit.Bsqlite/README.md new file mode 100644 index 00000000000..1002dea2a99 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/README.md @@ -0,0 +1,104 @@ +# bit Bsqlite + +EF Core on SQLite-WASM for Blazor WebAssembly. The same entities and the same migrations on the +client as on the server. + +This package is everything you need except a persistent backend: the EF Core integration, the +ADO.NET surface with live queries, the worker transport, the storage provider contract, a volatile +in-memory backend, and the browser assets (SQLite engine builds, worker host, JavaScript API) served +under `_content/Bit.Bsqlite`. + +**Runtime:** .NET 8, 9 and 10, each with its EF Core. **License:** MIT (bundles [wa-sqlite](https://github.com/rhashimoto/wa-sqlite), also MIT). + +## Install + +``` +dotnet add package Bit.Bsqlite +``` + +Add a persistent backend — without one you get the in-memory provider, which does not survive a +reload: + +| Package | Notes | +|---|---| +| `Bit.Bsqlite.Storage.Opfs` | Sync engine (smallest download), multi-connection, worker-only | +| `Bit.Bsqlite.Storage.IndexedDb` | Widest browser reach, runs anywhere, JSPI/Asyncify | +| `Bit.Bsqlite.Storage.CacheStorage` | Imports a database left behind by [Bit.Besql](https://github.com/bitfoundation/bitplatform/tree/develop/src/Besql) | + +## Register + +```csharp +builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .AddStorageProvider() + .AddStorageProvider() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()) + .AddDatabase("app.db")); + +builder.Services.AddBitBsqliteDbContext(); +``` + +## Use + +```csharp +@inject IBsqliteDbContextFactory Contexts + +await using var db = await Contexts.CreateDbContextAsync(); // migrates once per session +var open = await db.Orders.Where(o => o.Open).ToListAsync(); +``` + +Without dependency injection the same pieces compose by hand: + +```csharp +var session = await factory.OpenAsync("app.db"); // a BsqliteSessionFactory +options.UseBitBsqlite(session.Connection); +``` + +Selection is sticky: existing data outranks preference, so a database already written on one backend +is reopened there rather than silently starting empty somewhere else. +`BsqliteStorageMigrationMode.KeepExisting` is the default; `AutomaticOnOpen` copies the image, +verifies the SQLite header, then flips the binding. + +## Live queries + +Table-level, and they fire across tabs. A committed write re-runs any live query that reads that +table. The worker learns what changed from SQLite's own update hook - cascades and triggers included - +and reports it once the transaction commits, so a rolled-back write is never shown and a +multi-statement `SaveChangesAsync` re-runs a query once, not per statement. + +```csharp +await using var live = ctx.Orders.Where(o => o.Open).AsLiveQuery(); +live.Changed += (_, rows) => InvokeAsync(StateHasChanged); +``` + +## Async only + +The browser cannot block on storage, so the synchronous ADO.NET surface throws +`BsqliteSynchronousApiNotSupportedException` rather than deadlocking or lying. Use the `Async` +overloads throughout — `ToListAsync`, `SaveChangesAsync`, `MigrateAsync`. + +## Hosting under a sub-path + +Every asset is reached relative to the document, the way Blazor's own are, so an application published +under `` needs no configuration: the host resolves each provider's VFS module - +and each extension module - against the document base before the worker imports it. + +## Caching the browser assets + +The host module is imported with a `?v=` query and passes it on to the worker and +the modules it loads, so an upgrade cannot pair new .NET with JavaScript the browser cached from an +older version. The engine files under `_content/Bit.Bsqlite/engine` carry no query - they change only +when the pinned wa-sqlite version does - so if your server sends long `max-age` headers for static +content, exclude `_content/Bit.Bsqlite/` or serve it with `Cache-Control: no-cache` so the browser +revalidates. + +## Links + +- [Documentation and source](https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite), and the demo website at [bsqlite.bitplatform.dev](https://bsqlite.bitplatform.dev) +- [Issues](https://github.com/bitfoundation/bitplatform/issues) +- Writing your own storage backend: implement `IBsqliteStorageProvider` and inherit the + conformance kit from `Bit.Bsqlite.Testing`. diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-decimal.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-decimal.ts new file mode 100644 index 00000000000..9e9e029c150 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-decimal.ts @@ -0,0 +1,336 @@ +// A .NET System.Decimal, in JavaScript. +// +// EF Core stores decimals as TEXT and asks the worker to add, divide, compare, and order them. IEEE +// floats cannot do that: 0.1 is already wrong, and 1/3 must come back as twenty-eight threes, not as +// 0.3333333333333333. The representation is the same one .NET uses - a 96-bit coefficient, a scale +// of 0–28, and a sign - so arithmetic here answers what System.Decimal would answer. +// +// Two text forms come out of that, and the distinction matters. `toString` is .NET's ToString and +// keeps the value's own scale. `toSqlText` is the canonical form the SQLite stack stores, which is +// what EF compares against with `=`; that is the one every `ef_*` result is written in. + +const MAX_COEFF = (1n << 96n) - 1n; +const MAX_SCALE = 28; + +export class DecimalOverflowError extends Error { + constructor() { + super('Decimal overflow.'); + this.name = 'DecimalOverflowError'; + } +} + +export class DecimalFormatError extends Error { + constructor(text: unknown) { + super(`'${text}' is not a decimal.`); + this.name = 'DecimalFormatError'; + } +} + +export type DecimalSign = 1 | -1; + +export class Decimal { + readonly sign: DecimalSign; + + /** The unsigned coefficient. */ + readonly coeff: bigint; + + /** Digits after the decimal point, 0–28. */ + readonly scale: number; + + constructor(sign: DecimalSign, coeff: bigint, scale: number) { + this.sign = coeff === 0n ? 1 : sign; + this.coeff = coeff; + this.scale = scale; + } + + static zero() { + return new Decimal(1, 0n, 0); + } + + /** + * Parses invariant-culture decimal text. Leading and trailing whitespace and a leading sign are + * accepted; thousands separators and exponents are not - EF never writes those. + */ + static parse(text: string): Decimal { + if (typeof text !== 'string') { + throw new DecimalFormatError(text); + } + + const trimmed = text.trim(); + if (!trimmed) { + throw new DecimalFormatError(text); + } + + let sign: DecimalSign = 1; + let body = trimmed; + if (body[0] === '+' || body[0] === '-') { + sign = body[0] === '-' ? -1 : 1; + body = body.slice(1); + } + + if (!body || /[^0-9.]/.test(body)) { + throw new DecimalFormatError(text); + } + + const dot = body.indexOf('.'); + if (dot !== body.lastIndexOf('.')) { + throw new DecimalFormatError(text); + } + + const digits = (dot === -1 ? body : body.slice(0, dot) + body.slice(dot + 1)).replace(/^0+(?=\d)/, ''); + const scale = dot === -1 ? 0 : body.length - dot - 1; + + if (scale > MAX_SCALE) { + throw new DecimalFormatError(text); + } + + if (!digits) { + return Decimal.zero(); + } + + const coeff = BigInt(digits); + if (coeff > MAX_COEFF) { + throw new DecimalOverflowError(); + } + + return new Decimal(sign, coeff, scale); + } + + /** Invariant ToString, including trailing zeros that the scale requires (`4.0`, not `4`). */ + toString(): string { + const digits = this.coeff.toString(); + const sign = this.sign < 0 ? '-' : ''; + + if (this.scale === 0) { + return sign + digits; + } + + const padded = digits.length <= this.scale + ? digits.padStart(this.scale + 1, '0') + : digits; + const split = padded.length - this.scale; + return `${sign}${padded.slice(0, split)}.${padded.slice(split)}`; + } + + /** + * The canonical TEXT form the rest of the SQLite/EF stack writes, which is + * Microsoft.Data.Sqlite's `0.0###########################` format rather than a plain ToString. + * + * The two differ: ToString keeps the value's own scale (`10` stays `10`, `1.50` stays `1.50`), + * while the canonical form always shows one decimal and trims the rest (`10.0`, `1.5`). EF Core + * compiles decimal equality to a plain TEXT comparison against a literal it renders in the + * canonical form, so a value written any other way would silently never match. Every `ef_*` + * result therefore leaves through here, not through {@link toString}. + */ + toSqlText(): string { + const plain = this.toString(); + const dot = plain.indexOf('.'); + + if (dot === -1) { + return `${plain}.0`; + } + + const trimmed = plain.slice(dot + 1).replace(/0+$/, ''); + return `${plain.slice(0, dot)}.${trimmed || '0'}`; + } + + negate(): Decimal { + return this.coeff === 0n ? new Decimal(1, 0n, this.scale) : new Decimal(-this.sign as DecimalSign, this.coeff, this.scale); + } + + add(other: Decimal): Decimal { + if (this.coeff === 0n && this.scale <= other.scale) { + return other; + } + + if (other.coeff === 0n && other.scale <= this.scale) { + return this; + } + + const { left, right, scale } = align(this, other); + + if (this.sign === other.sign) { + return fit(this.sign, left + right, scale); + } + + if (left === right) { + return new Decimal(1, 0n, scale); + } + + if (left > right) { + return fit(this.sign, left - right, scale); + } + + return fit(other.sign, right - left, scale); + } + + subtract(other: Decimal): Decimal { + return this.add(other.negate()); + } + + multiply(other: Decimal): Decimal { + const sign = (this.sign * other.sign) as DecimalSign; + return fit(sign, this.coeff * other.coeff, this.scale + other.scale); + } + + /** + * .NET decimal division: exact when the remainder is zero, otherwise enough digits to fill a + * 96-bit coefficient without exceeding scale 28. Midpoints use banker's rounding. + */ + divide(other: Decimal): Decimal { + if (other.coeff === 0n) { + throw new Error('Division by zero.'); + } + + if (this.coeff === 0n) { + return Decimal.zero(); + } + + const sign = (this.sign * other.sign) as DecimalSign; + let quotient: bigint = this.coeff / other.coeff; + let remainder: bigint = this.coeff % other.coeff; + let extra = 0; + + const resultScale = () => extra + this.scale - other.scale; + const canGrow = () => quotient <= MAX_COEFF / 10n; + + while (remainder !== 0n && canGrow() && resultScale() < MAX_SCALE) { + remainder *= 10n; + quotient = quotient * 10n + remainder / other.coeff; + remainder = remainder % other.coeff; + extra++; + } + + if (remainder !== 0n) { + const twice = remainder * 2n; + const roundUp = twice > other.coeff || (twice === other.coeff && (quotient & 1n) === 1n); + if (roundUp) { + quotient += 1n; + } + } + + let scale = resultScale(); + + if (remainder === 0n) { + while (scale < 0) { + if (quotient > MAX_COEFF / 10n) { + throw new DecimalOverflowError(); + } + + quotient *= 10n; + scale++; + } + } else if (scale < 0) { + throw new DecimalOverflowError(); + } + + return fit(sign, quotient, scale); + } + + /** + * Remainder with the sign of the dividend, truncated toward zero - `a - Truncate(a/b)*b`. + */ + remainder(other: Decimal): Decimal { + if (other.coeff === 0n) { + throw new Error('Division by zero.'); + } + + return this.subtract(this.divide(other).truncate().multiply(other)); + } + + truncate(): Decimal { + if (this.scale === 0) { + return this; + } + + return new Decimal(this.sign, this.coeff / 10n ** BigInt(this.scale), 0); + } + + /** + * Numeric comparison, ignoring scale. `9` and `9.0` compare equal, which is what `ef_compare` + * and the `EF_DECIMAL` collation both need. + */ + compareTo(other: Decimal): -1 | 0 | 1 { + if (this.coeff === 0n && other.coeff === 0n) { + return 0; + } + + if (this.sign !== other.sign) { + return this.sign < other.sign ? -1 : 1; + } + + const { left, right } = align(this, other); + if (left === right) { + return 0; + } + + const magnitude = left < right ? -1 : 1; + return (this.sign < 0 ? -magnitude : magnitude) as -1 | 1; + } + + static compare(left: Decimal, right: Decimal): -1 | 0 | 1 { + return left.compareTo(right); + } +} + +function align(a: Decimal, b: Decimal): { left: bigint; right: bigint; scale: number } { + const scale = Math.max(a.scale, b.scale); + return { + left: a.coeff * 10n ** BigInt(scale - a.scale), + right: b.coeff * 10n ** BigInt(scale - b.scale), + scale, + }; +} + +/** + * Fits a coefficient into 96 bits and a scale of 0–28, rounding with banker's rounding when digits + * have to be dropped. + * + * All the surplus digits go in one division, and the whole discarded remainder decides the + * rounding - which is what System.Decimal does. Dropping them one at a time and rounding at each + * step is not the same thing: `…51` dropped digit by digit rounds down (1 is below half, then 5 + * ties to even), while the true remainder 0.51 is above half and rounds up. + */ +function fit(sign: DecimalSign, coeff: bigint, scale: number): Decimal { + if (coeff < 0n) { + throw new Error('fit() expects an unsigned coefficient.'); + } + + let drop = Math.max(0, scale - MAX_SCALE); + while (coeff / 10n ** BigInt(drop) > MAX_COEFF) { + drop++; + } + + if (drop > scale) { + throw new DecimalOverflowError(); + } + + if (drop > 0) { + const divisor = 10n ** BigInt(drop); + const remainder = coeff % divisor; + coeff /= divisor; + scale -= drop; + + const twice = remainder * 2n; + if (twice > divisor || (twice === divisor && (coeff & 1n) === 1n)) { + coeff += 1n; + + // Rounding up from the largest 96-bit coefficient carries out of it: 2^96 has to lose one + // more digit, and its own remainder (…6) rounds that up too - the same as System.Decimal. + if (coeff > MAX_COEFF) { + if (scale === 0) { + throw new DecimalOverflowError(); + } + + coeff = coeff / 10n + 1n; + scale--; + } + } + } + + if (scale < 0) { + throw new DecimalOverflowError(); + } + + return new Decimal(sign, coeff, scale); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-engine.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-engine.ts new file mode 100644 index 00000000000..d0e3e2f4c2c --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-engine.ts @@ -0,0 +1,126 @@ +// Chooses a SQLite build, loads it, and lets the selected storage provider register its VFS. +// +// Three builds are vendored and exactly one is fetched per session. The choice is not a preference +// setting: a VFS that suspends to read storage (IndexedDB, Cache Storage) cannot run on the +// synchronous build at all, and the two builds that can host it differ by more than a factor of two +// in download size, so the cheapest capable build wins. + +// Imported dynamically so this module's cache key travels with it: see the note in the worker. +const { raiseFunctionTableLimit }: typeof import('./bit-bsqlite-wasm-table.js') = + await import('./bit-bsqlite-wasm-table.js' + new URL(import.meta.url).search); +import * as SQLite from './engine/sqlite-api.js'; +import type { SQLiteAPI, SQLiteModule } from './engine/sqlite-api.js'; +import type { JsModuleReference } from './bit-bsqlite-host.js'; + +/** What a storage provider's VFS module hands back from its register export. */ +export interface VfsRegistration { + vfsName?: string | null; + onOpenSql?: string[]; +} + +/** A loaded engine, with the provider's VFS registered against it. */ +export interface LoadedEngine { + sqlite3: SQLiteAPI; + module: SQLiteModule; + build: string; + vfsName: string | null; + onOpenSql: string[]; +} + +interface EngineBuildFiles { + id: string; + module: string; + wasm: string; +} + +/** Which engine build a VFS needs. Mirrors BsqliteEngineBuild on the .NET side. */ +export const EngineBuild = Object.freeze({ + synchronous: 'synchronous', + asyncCapable: 'asyncCapable', +}); + +const BUILDS = Object.freeze({ + synchronous: { id: 'synchronous', module: './engine/wa-sqlite.mjs', wasm: './engine/wa-sqlite.wasm' }, + jspi: { id: 'jspi', module: './engine/wa-sqlite-jspi.mjs', wasm: './engine/wa-sqlite-jspi.wasm' }, + asyncify: { id: 'asyncify', module: './engine/wa-sqlite-async.mjs', wasm: './engine/wa-sqlite-async.wasm' }, +}); + +/** + * Whether the browser has JavaScript Promise Integration, which lets the engine suspend natively + * instead of through Asyncify's instrumented control flow. + * + * Unflagged in Chrome 137+, flagged in Firefox, and absent in Safari as of mid-2026, which is why + * Asyncify is still vendored: it is the only route to an asynchronous VFS on Safari. + */ +export function supportsJspi(): boolean { + // Not in TypeScript's WebAssembly typings yet: JSPI is still being standardised. + return typeof (WebAssembly as { Suspending?: unknown })?.Suspending === 'function'; +} + +/** Picks the smallest build that can host the given VFS requirement, a value of {@link EngineBuild}. */ +export function selectBuild(requiredBuild: string): EngineBuildFiles { + switch (requiredBuild) { + case EngineBuild.synchronous: + return BUILDS.synchronous; + + case EngineBuild.asyncCapable: + return supportsJspi() ? BUILDS.jspi : BUILDS.asyncify; + + default: + throw new Error(`Unknown engine build requirement '${requiredBuild}'.`); + } +} + +/** + * Loads the engine and registers the storage provider's VFS against it. + * + * `requiredBuild` is a value of {@link EngineBuild} and `databaseName` the database the VFS will be + * asked to open. `vfs` is the provider's VFS module - omitted for a provider that has none, which + * leaves the engine on its built-in memory VFS. + */ +export async function loadEngine({ requiredBuild, databaseName, vfs }: { requiredBuild: string; databaseName: string; vfs?: JsModuleReference | null }): Promise { + if (!databaseName) { + throw new Error('A database name is required.'); + } + + const build = selectBuild(requiredBuild); + + if (vfs && build.id === BUILDS.synchronous.id && requiredBuild !== EngineBuild.synchronous) { + // Unreachable via selectBuild, but this is the mismatch that produces + // "Synchronous WebAssembly cannot call async function" deep inside a query rather than here. + throw new Error('An asynchronous VFS cannot run on the synchronous engine build.'); + } + + const factory = (await import(build.module)).default; + const wasmBinary = raiseFunctionTableLimit( + new Uint8Array(await (await fetch(new URL(build.wasm, import.meta.url))).arrayBuffer())); + const module: SQLiteModule = await factory({ wasmBinary }); + const sqlite3 = SQLite.Factory(module); + + let vfsName: string | null = null; + let onOpenSql: string[] = []; + + if (vfs) { + const vfsModule = await import(vfs.moduleUrl); + const register = vfsModule[vfs.registerExport ?? 'register']; + + if (typeof register !== 'function') { + throw new Error( + `'${vfs.moduleUrl}' does not export a function named '${vfs.registerExport ?? 'register'}'.`); + } + + // The provider both creates and registers: VFS construction is asynchronous and + // implementation-specific (handle pools, IndexedDB connections), so the engine host cannot do it. + const registered: VfsRegistration | undefined = await register({ module, sqlite3, databaseName }); + vfsName = registered?.vfsName ?? null; + onOpenSql = Array.isArray(registered?.onOpenSql) ? registered.onOpenSql : []; + + if (!vfsName) { + throw new Error( + `'${vfs.moduleUrl}' registered a VFS without reporting its name, so the connection cannot ` + + 'ask for it.'); + } + } + + return { sqlite3, module, build: build.id, vfsName, onOpenSql }; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-functions.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-functions.ts new file mode 100644 index 00000000000..efbecdb8d3b --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-functions.ts @@ -0,0 +1,330 @@ +// The function set EF Core's SQLite provider expects to find on every connection. +// +// EF only installs these when the connection is a real Microsoft.Data.Sqlite connection. Bsqlite +// supplies its own, so the worker has to. The names, arities, and results match +// `BsqliteFunctions` on the .NET side - S4 is the oracle, and the browser suite repeats it here. +// +// Decimals are TEXT. Arithmetic goes through `Decimal`, never through JavaScript numbers: a float +// would make 1/3 and 0.1 silently wrong, and EF's equality is a string compare against the canonical +// form this module writes back. + +// Imported dynamically so this module's cache key travels with it: see the note in the worker. +const { Decimal }: typeof import('./bit-bsqlite-decimal.js') = + await import('./bit-bsqlite-decimal.js' + new URL(import.meta.url).search); +import * as SQLite from './engine/sqlite-constants.js'; +import type { Pointer, SQLiteAPI, SQLiteModule } from './engine/sqlite-api.js'; +import type { Decimal as DecimalValue } from './bit-bsqlite-decimal.js'; + +/** The three callbacks an `ef_*` aggregate is made of, over a per-group state object. */ +interface Aggregate { + create: () => TState; + step: (state: TState, value: DecimalValue | null) => void; + final: (state: TState | null) => DecimalValue | null | undefined; +} + +const FUNCTION_FLAGS = SQLite.SQLITE_UTF8 | SQLite.SQLITE_DETERMINISTIC | SQLite.SQLITE_INNOCUOUS; + +/** One WASM function pointer per module - re-registering the collation must not leak a new one. */ +const collationPointers = new WeakMap(); + +/** Per-aggregate instance state, keyed by the pointer `sqlite3_aggregate_context` hands back. */ +const aggregateState = new Map(); + +/** + * Forgets any aggregate state left behind by the statement that just finished. + * + * Entries normally remove themselves in the final callback, and on the current engine that callback + * appears to run even for a grouped query abandoned after one row or one that fails mid-step. This + * does not depend on that. The key is a heap address SQLite frees with the statement and is free to + * hand out again, so an entry that ever did outlive its statement would look to + * `getAggregateState` like an accumulator already in progress - and the next `ef_sum` landing on + * that address would continue someone else's total rather than start at zero, with no error. State + * is only meaningful within one statement, so dropping it between them costs nothing and removes + * the possibility. + */ +export function resetAggregateState(): void { + aggregateState.clear(); +} + +/** + * Installs `ef_*`, `EF_DECIMAL`, and `regexp` on an open database. The Emscripten module is needed + * for the collation and the aggregate context, which wa-sqlite's JavaScript API does not wrap. + */ +export function registerFunctions(module: SQLiteModule, sqlite3: SQLiteAPI, db: number): void { + const flags = FUNCTION_FLAGS; + + const scalar = (name: string, arity: number, impl: (ctx: Pointer, values: Uint32Array) => void) => { + sqlite3.create_function(db, name, arity, flags, 0, (ctx, values) => { + try { + impl(ctx, values); + } catch (error) { + resultError(module, ctx, error); + } + }); + }; + + const binary = (name: string, apply: (left: DecimalValue, right: DecimalValue) => DecimalValue) => { + scalar(name, 2, (ctx, values) => { + const left = readDecimal(sqlite3, values[0]); + const right = readDecimal(sqlite3, values[1]); + if (left === null || right === null) { + sqlite3.result(ctx, null); + return; + } + + sqlite3.result(ctx, apply(left, right).toSqlText()); + }); + }; + + binary('ef_add', (left, right) => left.add(right)); + binary('ef_multiply', (left, right) => left.multiply(right)); + + // Division and modulo return null on a zero divisor - that is EF's contract, not an error - so + // they write the result themselves rather than going through `binary`. + sqlite3.create_function(db, 'ef_divide', 2, flags, 0, guarded(module, (ctx, values) => { + const left = readDecimal(sqlite3, values[0]); + const right = readDecimal(sqlite3, values[1]); + if (left === null || right === null || right.coeff === 0n) { + sqlite3.result(ctx, null); + return; + } + + sqlite3.result(ctx, left.divide(right).toSqlText()); + })); + + sqlite3.create_function(db, 'ef_mod', 2, flags, 0, guarded(module, (ctx, values) => { + const left = readDecimal(sqlite3, values[0]); + const right = readDecimal(sqlite3, values[1]); + if (left === null || right === null || right.coeff === 0n) { + sqlite3.result(ctx, null); + return; + } + + sqlite3.result(ctx, left.remainder(right).toSqlText()); + })); + + scalar('ef_negate', 1, (ctx, values) => { + const value = readDecimal(sqlite3, values[0]); + sqlite3.result(ctx, value === null ? null : value.negate().toSqlText()); + }); + + // A sign, not a bool: EF compiles `a > b` to `ef_compare(a, b) > 0`. + scalar('ef_compare', 2, (ctx, values) => { + const left = readDecimal(sqlite3, values[0]); + const right = readDecimal(sqlite3, values[1]); + sqlite3.result(ctx, left === null || right === null ? null : left.compareTo(right)); + }); + + // SQLite calls regexp(pattern, input) for `input REGEXP pattern` - arguments reversed from + // Regex.IsMatch. There is no match timeout here: JavaScript's RegExp cannot be interrupted, and + // the patterns EF emits terminate. Lookaround and backreferences must work; they are why the + // .NET side does not use RegexOptions.NonBacktracking. + scalar('regexp', 2, (ctx, values) => { + const pattern = readText(sqlite3, values[0]); + const input = readText(sqlite3, values[1]); + if (pattern === null || input === null) { + sqlite3.result(ctx, null); + return; + } + + sqlite3.result(ctx, compileRegExp(pattern).test(input) ? 1 : 0); + }); + + registerAggregate<{ sum: DecimalValue | null }>(sqlite3, db, module, 'ef_sum', { + create: () => ({ sum: null }), + step: (state, value) => { + if (value === null) { + return; + } + + state.sum = state.sum === null ? value : state.sum.add(value); + }, + final: state => state?.sum ?? null, + }); + + registerAggregate<{ sum: DecimalValue; count: bigint }>(sqlite3, db, module, 'ef_avg', { + create: () => ({ sum: Decimal.zero(), count: 0n }), + step: (state, value) => { + if (value === null) { + return; + } + + state.sum = state.sum.add(value); + state.count += 1n; + }, + final: state => { + if (!state || state.count === 0n) { + return null; + } + + return state.sum.divide(new Decimal(1, state.count, 0)); + }, + }); + + registerAggregate<{ value: DecimalValue | null }>(sqlite3, db, module, 'ef_max', { + create: () => ({ value: null }), + step: (state, value) => { + if (value === null) { + return; + } + + if (state.value === null || value.compareTo(state.value) > 0) { + state.value = value; + } + }, + final: state => state?.value ?? null, + }); + + registerAggregate<{ value: DecimalValue | null }>(sqlite3, db, module, 'ef_min', { + create: () => ({ value: null }), + step: (state, value) => { + if (value === null) { + return; + } + + if (state.value === null || value.compareTo(state.value) < 0) { + state.value = value; + } + }, + final: state => state?.value ?? null, + }); + + registerCollation(module, db); +} + +/** + * A query applies one pattern to every row it scans, so the compiled RegExp is kept between calls + * rather than rebuilt per row. Bounded and first-in-first-out: EF sends a handful of distinct + * patterns, and a query that generates them from data must not grow this without limit. + */ +const REGEXP_CACHE_SIZE = 64; +const regExpCache = new Map(); + +function compileRegExp(pattern: string): RegExp { + let compiled = regExpCache.get(pattern); + if (compiled) { + return compiled; + } + + compiled = new RegExp(pattern); + if (regExpCache.size >= REGEXP_CACHE_SIZE) { + regExpCache.delete(regExpCache.keys().next().value!); + } + + regExpCache.set(pattern, compiled); + return compiled; +} + +/** + * `sqlite3_create_collation` is exported from the WASM module but not wrapped by wa-sqlite's JS API, + * so the compare callback is installed with `addFunction` and the C entry is called directly. + * That needs a free table slot - `loadEngine` raises the table maximum for this reason. + */ +function registerCollation(module: SQLiteModule, db: number): void { + let pointer = collationPointers.get(module); + if (!pointer) { + pointer = module.addFunction((_pArg: number, n1: number, p1: number, n2: number, p2: number) => { + const left = module.UTF8ToString(p1, n1); + const right = module.UTF8ToString(p2, n2); + + try { + return Decimal.compare(Decimal.parse(left), Decimal.parse(right)); + } catch { + // A total order is required even for garbage: two undecodable keys must still compare + // consistently. Byte order of the original text is enough; EF never stores garbage. + return left < right ? -1 : left > right ? 1 : 0; + } + }, 'iiiiii'); + collationPointers.set(module, pointer); + } + + const rc = module.ccall( + 'sqlite3_create_collation', + 'number', + ['number', 'string', 'number', 'number', 'number'], + [db, 'EF_DECIMAL', SQLite.SQLITE_UTF8 | SQLite.SQLITE_DETERMINISTIC, 0, pointer]); + + if (rc !== SQLite.SQLITE_OK) { + throw new Error(`sqlite3_create_collation(EF_DECIMAL) failed with code ${rc}.`); + } +} + +function registerAggregate(sqlite3: SQLiteAPI, db: number, module: SQLiteModule, name: string, { create, step, final }: Aggregate): void { + sqlite3.create_function( + db, + name, + 1, + FUNCTION_FLAGS, + 0, + null, + guarded(module, (ctx: Pointer, values: Uint32Array) => { + const state = getAggregateState(module, ctx, create); + step(state, readDecimal(sqlite3, values[0])); + }), + guarded(module, (ctx: Pointer) => { + const state = takeAggregateState(module, ctx); + const result = final(state); + sqlite3.result(ctx, result === null || result === undefined ? null : result.toSqlText()); + })); +} + +function getAggregateState(module: SQLiteModule, ctx: Pointer, create: () => TState): TState { + const pointer = module._sqlite3_aggregate_context(ctx, 4); + if (!pointer) { + throw new Error('sqlite3_aggregate_context could not allocate.'); + } + + let state = aggregateState.get(pointer) as TState | undefined; + if (!state) { + state = create(); + aggregateState.set(pointer, state); + } + + return state; +} + +function takeAggregateState(module: SQLiteModule, ctx: Pointer): TState | null { + const pointer = module._sqlite3_aggregate_context(ctx, 0); + if (!pointer) { + return null; + } + + const state = (aggregateState.get(pointer) ?? null) as TState | null; + aggregateState.delete(pointer); + return state; +} + +function readDecimal(sqlite3: SQLiteAPI, pValue: Pointer): DecimalValue | null { + if (sqlite3.value_type(pValue) === SQLite.SQLITE_NULL) { + return null; + } + + return Decimal.parse(sqlite3.value_text(pValue)); +} + +function readText(sqlite3: SQLiteAPI, pValue: Pointer): string | null { + if (sqlite3.value_type(pValue) === SQLite.SQLITE_NULL) { + return null; + } + + return sqlite3.value_text(pValue); +} + +function guarded(module: SQLiteModule, fn: (...args: TArgs) => void): (...args: TArgs) => void { + return (...args: TArgs) => { + try { + return fn(...args); + } catch (error) { + resultError(module, args[0], error); + } + }; +} + +function resultError(module: SQLiteModule, ctx: Pointer, error: unknown): void { + const message = (error as Error | undefined)?.message ?? String(error); + module.ccall( + 'sqlite3_result_error', + null, + ['number', 'string', 'number'], + [ctx, message, -1]); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-host.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-host.ts new file mode 100644 index 00000000000..fe1b4875cad --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-host.ts @@ -0,0 +1,418 @@ +// The main-thread half of the transport: spawns the worker that owns a database and correlates +// requests with responses. This is the surface the .NET transport calls. + +// Everything this module reaches for inherits the cache-busting query it was imported with, so an +// upgrade cannot pair a fresh host with a worker - or a wire module - the browser still holds from +// an older version. +const VERSION = new URL(import.meta.url).search; + +const DEFAULT_WORKER_URL = new URL('./bit-bsqlite-worker.js' + VERSION, import.meta.url).href; + +// Re-exported so a JavaScript caller can work in plain values without importing the wire module and +// without inventing a second, divergent encoding. The .NET transport does the same work in C#. The +// import is dynamic only to carry VERSION: a `export ... from` would resolve to the bare path. +const wire: typeof import('./bit-bsqlite-wire.js') = await import('./bit-bsqlite-wire.js' + VERSION); + +export const decodeValue = wire.decodeValue; +export const encodeValue = wire.encodeValue; +export const WireType = wire.WireType; +export const encodeBatchBinary = wire.encodeBatchBinary; +export const decodeReplyBinary = wire.decodeReplyBinary; + +/** A module the worker imports: a storage provider's VFS, or an extension registering functions. */ +export interface JsModuleReference { + moduleUrl: string; + registerExport?: string; +} + +export interface OpenOptions { + databaseName: string; + /** `'synchronous'` or `'asyncCapable'`. */ + requiredBuild: string; + vfs?: JsModuleReference; + limits?: { supportsMultiDatabaseTransactions?: boolean; canChangePageSize?: boolean }; + /** `false` to leave foreign-key enforcement off. */ + foreignKeys?: boolean; + /** Configuration statements run once the database is open. */ + onOpenSql?: string[]; + /** Function/collation modules. */ + extensions?: JsModuleReference[]; + /** Overridable so an application can host the worker itself. */ + workerUrl?: string; +} + +/** What a committed write reports: the database it was in and every table a row landed in. */ +export interface NotifyPayload { + kind: 'notify'; + databaseName: string; + tables?: string[]; +} + +export type NotifyListener = (payload: NotifyPayload, context: { local: boolean }) => void; + +/** The .NET side of a [JSInvokable] callback, as Blazor hands it to JavaScript. */ +export interface DotNetObjectReference { + invokeMethodAsync(name: string, ...args: unknown[]): Promise; +} + +/** An error rebuilt from the worker, carrying the SQLite result code when SQLite raised it. */ +export interface SqliteHostError extends Error { + sqliteCode?: number; +} + +interface WorkerRequest { + kind: string; + vfs?: JsModuleReference; + extensions?: JsModuleReference[]; + [key: string]: unknown; +} + +/** A Web Lock request: `release` is set once it is granted, `abort` withdraws it until then. */ +interface HeldLock { + abort: AbortController; + release: (() => void) | null; +} + +interface WorkerResponse { + id?: number; + kind?: string; + ok?: boolean; + result?: unknown; + error?: { message: string; sqliteCode?: number | null; name?: string }; +} + +/** Starts a worker for one database. */ +export async function open({ databaseName, requiredBuild, vfs, limits, foreignKeys, onOpenSql, extensions, workerUrl }: OpenOptions): Promise { + const host = createHost(workerUrl); + await host.send({ kind: 'open', databaseName, requiredBuild, vfs, limits, foreignKeys, onOpenSql, extensions }); + return host; +} + +/** + * Creates a host without opening a database. + * + * Exists for the .NET transport: Blazor can hold an IJSObjectReference to the host and then + * drive every operation - including open - through call, so a failure never has to throw + * across the interop boundary. + */ +export function createHost(workerUrl?: string | null): SqliteHost { + return new SqliteHost(workerUrl ?? DEFAULT_WORKER_URL); +} + +/** + * Forwards committed writes to a .NET object, which is how a live query re-runs - for this tab's + * own writes and for another tab's alike. + * + * Both are forwarded because the worker is the one that knows: its update hook names every table a + * row landed in, cascades and triggers included, and it waits for the commit. The .NET transport + * declares `ReportsLocalWrites` so the command layer does not raise the same write again from the + * SQL text. Writes to a different database on the same origin are dropped: the broadcast channel is + * origin-wide, so it carries every database's traffic, and the name is the only thing that + * separates them. + * + * `method` is the [JSInvokable] method to call with the table names. + */ +export function listen(host: SqliteHost, target: DotNetObjectReference, databaseName: string, method = 'OnTablesChanged'): () => boolean { + return host.onNotify(payload => { + if (payload.databaseName !== databaseName) { + return; + } + + // Fire and forget: a disposed .NET reference must not wedge the notification pipeline. + target.invokeMethodAsync(method, payload.tables ?? []).catch(() => { }); + }); +} + +const CHANGE_CHANNEL = 'bit-bsqlite-changes'; + +export class SqliteHost { + #worker: Worker; + #pending = new Map void; reject: (reason: unknown) => void }>(); + #nextId = 1; + #disposed = false; + #fatal: Error | null = null; + #listeners = new Set(); + #channel = typeof BroadcastChannel === 'function' ? new BroadcastChannel(CHANGE_CHANNEL) : null; + #locks = new Map(); + + constructor(workerUrl: string) { + this.#worker = new Worker(workerUrl, { type: 'module' }); + this.#worker.addEventListener('message', event => this.#settle(event.data)); + this.#channel?.addEventListener('message', event => this.#emit(event.data, { local: false })); + + // A worker that dies with requests outstanding must not leave them pending forever - and one + // that died must not swallow later requests either. The worker script wraps every request in + // its own catch, so an error event here means the script itself failed to load or link: there + // is nobody on the other end, and every request from now on gets told so instead of hanging. + this.#worker.addEventListener('error', (event: ErrorEvent) => { + this.#fatal = new Error(`The SQLite worker failed: ${event.message ?? 'unknown error'}`); + this.#failAll(this.#fatal); + }); + + // A reply that cannot be deserialized never reaches #settle, so its request would wait for an + // answer that has already been thrown away. The id is gone with it, so every outstanding + // request fails - the same trade the error handler makes, and better than hanging. + this.#worker.addEventListener('messageerror', () => this.#failAll( + new Error('A reply from the SQLite worker could not be deserialized.'))); + } + + /** + * Subscribes to write notifications. + * + * The listener is called as `listener(payload, { local })`. `local` is true for a write this + * host's own worker performed and false for one another tab broadcast, which is the distinction a + * caller that already knows about its own writes needs in order not to react twice. + */ + onNotify(listener: NotifyListener): () => boolean { + this.#listeners.add(listener); + return () => this.#listeners.delete(listener); + } + + #emit(payload: NotifyPayload | undefined, { local }: { local: boolean }) { + if (!payload || payload.kind !== 'notify') { + return; + } + + if (local) { + this.#channel?.postMessage(payload); + } + + for (const listener of this.#listeners) { + try { + listener(payload, { local }); + } catch (error) { + console.error('A Bsqlite change listener threw.', error); + } + } + } + + #settle(response: WorkerResponse | undefined) { + if (response?.kind === 'notify') { + this.#emit(response as NotifyPayload, { local: true }); + return; + } + + const id = response?.id; + const pending = id === undefined ? undefined : this.#pending.get(id); + if (id === undefined || !pending) { + return; + } + + this.#pending.delete(id); + + if (response!.ok) { + pending.resolve(response!.result); + } else { + pending.reject(toError(response!.error ?? { message: 'The SQLite worker failed without saying why.' })); + } + } + + #failAll(error: Error) { + for (const pending of this.#pending.values()) { + pending.reject(error); + } + + this.#pending.clear(); + } + + /** `transfer` lists the buffers to move rather than copy into the worker. */ + send(request: WorkerRequest, transfer?: Transferable[]): Promise { + if (this.#disposed) { + return Promise.reject(new Error('This SQLite host has been disposed.')); + } + + if (this.#fatal) { + return Promise.reject(this.#fatal); + } + + const id = this.#nextId++; + + return new Promise((resolve, reject) => { + this.#pending.set(id, { resolve, reject }); + this.#worker.postMessage({ ...resolveModuleUrls(request), id }, transfer ?? []); + }); + } + + /** Executes a batch in one round trip, with tagged rows in the reply. */ + execute(batch: ReadonlyArray<{ commandText: string; parameters?: ReadonlyArray<{ name: string; type: number; value: unknown }>; resultKind?: 'nonQuery' | 'scalar' | 'reader'; maxRows?: number }>): Promise { + return this.send({ kind: 'execute', batch }); + } + + /** + * Executes a binary-encoded batch and returns the binary reply, transferring both buffers so + * neither is copied. Never rejects for a SQL failure: the reply carries it. + * + * `batch` is encoded by `encodeBatchBinary` or by `BsqliteWireFormat` on the .NET side. + */ + async executeBinary(batch: Uint8Array | ArrayBuffer | ArrayLike): Promise { + const bytes = batch instanceof Uint8Array ? batch : new Uint8Array(batch as ArrayBuffer); + const transferable = bytes.byteOffset === 0 && bytes.byteLength === bytes.buffer.byteLength + ? [bytes.buffer] + : []; + const reply = await this.send({ kind: 'executeBinary', batch: bytes }, transferable); + return reply instanceof Uint8Array ? reply : new Uint8Array(reply as ArrayBuffer); + } + + /** + * Runs any worker request and returns the outcome as data rather than by throwing. + * + * This exists for .NET callers, and the reason is a hard limit rather than a preference: Blazor's JS + * interop reduces a thrown JavaScript error to its message, so the SQLite result code would be lost - + * and that code is how EF tells a unique-constraint violation from a busy database. An envelope keeps + * it. + */ + async call(request: WorkerRequest): Promise<{ ok: true; result: unknown } | { ok: false; error: { message: string; sqliteCode: number | null; name: string } }> { + try { + return { ok: true, result: await this.send(request) }; + } catch (caught) { + const error = caught as SqliteHostError; + return { + ok: false, + error: { + message: error.message, + sqliteCode: error.sqliteCode ?? null, + name: error.name ?? 'Error', + }, + }; + } + } + + version(): Promise { + return this.send({ kind: 'version' }); + } + + /** + * A consistent snapshot of the open database as a SQLite file image. `schema` is `'main'` unless + * an attached database is wanted. + */ + export(schema = 'main'): Promise { + return this.send({ kind: 'serialize', schema }) as Promise; + } + + async close(): Promise { + await this.send({ kind: 'close' }); + } + + /** + * Waits for the Web Lock `name` and holds it until `releaseLock(id)` - or until this host is + * disposed, or the tab goes away, whichever comes first. + * + * Web Locks are shared by every tab and worker of the origin, which is what makes this a lock + * between the workers of different tabs that open the same file; and the browser drops a lock + * whose holder is gone, so a tab that crashes mid-migration never leaves the others waiting. + * `id` is chosen by the caller so it can release a request it gave up on before it was granted. + */ + acquireLock(name: string, id: number): Promise { + if (this.#disposed) { + return Promise.reject(new Error('This SQLite host has been disposed.')); + } + + const locks = globalThis.navigator?.locks; + if (!locks) { + // The VFSes coordinate tabs through the same API, so without it there is nothing a lock + // here could be coordinating with. + return Promise.resolve(); + } + + const held: HeldLock = { abort: new AbortController(), release: null }; + this.#locks.set(id, held); + + return new Promise((granted, failed) => { + locks.request(name, { signal: held.abort.signal }, () => new Promise(release => { + if (this.#locks.get(id) !== held) { + // Released while still queued, and granted before the abort landed. + release(); + return; + } + + held.release = release; + granted(); + })).catch(error => { + if (this.#locks.get(id) === held) { + this.#locks.delete(id); + } + + failed(error); + }); + }); + } + + /** Releases a lock taken with `acquireLock`, or withdraws a request still waiting for one. */ + releaseLock(id: number): void { + const held = this.#locks.get(id); + if (!held) { + return; + } + + this.#locks.delete(id); + if (held.release) { + held.release(); + } else { + held.abort.abort(); + } + } + + /** + * Terminates the worker. Outstanding requests are rejected rather than abandoned, because a caller + * awaiting a query that will never answer is worse than a caller seeing the shutdown. + */ + dispose(): void { + if (this.#disposed) { + return; + } + + this.#disposed = true; + this.#failAll(new Error('The SQLite host was disposed while the request was in flight.')); + for (const id of [...this.#locks.keys()]) { + this.releaseLock(id); + } + + this.#listeners.clear(); + this.#channel?.close(); + this.#worker.terminate(); + } +} + +/** + * Makes a storage provider's VFS module URL - and any extension module URL - absolute before it + * reaches the worker. + * + * Providers name their module the way every other Blazor asset is named - relative to the document, + * as `./_content//.js` - so an application served under a sub-path still finds it. + * The worker cannot resolve that itself: a relative import there is relative to the worker script, + * which lives in this package's own `_content` folder. Resolving here, where the document base is + * known, keeps that detail out of every provider. + */ +function resolveModuleUrls(request: WorkerRequest): WorkerRequest { + const hasVfs = typeof request?.vfs?.moduleUrl === 'string' && request.vfs.moduleUrl.length > 0; + const hasExtensions = Array.isArray(request?.extensions) && request.extensions.length > 0; + if (!hasVfs && !hasExtensions) { + return request; + } + + const base = globalThis.document?.baseURI ?? globalThis.location?.href ?? import.meta.url; + const resolve = (module: JsModuleReference) => typeof module?.moduleUrl === 'string' && module.moduleUrl.length > 0 + ? { ...module, moduleUrl: new URL(module.moduleUrl, base).href } + : module; + + return { + ...request, + ...(hasVfs ? { vfs: resolve(request.vfs!) } : {}), + ...(hasExtensions ? { extensions: request.extensions!.map(resolve) } : {}), + }; +} + +/** + * Rebuilds an error from the worker, preserving the SQLite result code so callers can distinguish a + * constraint violation from a busy database. + */ +function toError({ message, sqliteCode, name }: { message: string; sqliteCode?: number | null; name?: string }): SqliteHostError { + const error: SqliteHostError = new Error(message); + error.name = name ?? 'Error'; + + if (sqliteCode !== null && sqliteCode !== undefined) { + error.sqliteCode = sqliteCode; + } + + return error; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-persist.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-persist.ts new file mode 100644 index 00000000000..38d58f03f01 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-persist.ts @@ -0,0 +1,5 @@ +export function requestPersistence(): Promise | boolean { + return typeof navigator.storage?.persist === 'function' + ? navigator.storage.persist() + : false; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-wasm-table.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-wasm-table.ts new file mode 100644 index 00000000000..d3551d91981 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-wasm-table.ts @@ -0,0 +1,158 @@ +// Raises the WebAssembly function-table maximum on a copy of a wa-sqlite build. +// +// The vendored engine is compiled without ALLOW_TABLE_GROWTH, so the table's +// minimum and maximum are the same (436). Emscripten's `addFunction` then cannot +// install a JS collation callback, and `EF_DECIMAL` never exists. Patching a copy +// at load time - not the checksummed artifact - gives `addFunction` spare slots +// without a from-source rebuild. + +const TABLE_SECTION = 4; +const FUNCREF = 0x70; +const LIMITS_MIN_MAX = 1; + +interface WasmSection { + id: number; + payload: Uint8Array; +} + +/** + * Returns a copy of a wa-sqlite `.wasm` with a higher table maximum; the input is not mutated. + * `extraSlots` is how many function pointers `addFunction` may take. + */ +export function raiseFunctionTableLimit(bytes: Uint8Array, extraSlots = 16): Uint8Array { + if (!(bytes instanceof Uint8Array)) { + throw new Error('raiseFunctionTableLimit expects the WASM bytes.'); + } + + if (!Number.isInteger(extraSlots) || extraSlots < 1) { + throw new Error('extraSlots must be a positive integer.'); + } + + const sections = parseSections(bytes); + const table = sections.find(section => section.id === TABLE_SECTION); + + if (!table) { + throw new Error('The WASM module has no function table to grow.'); + } + + const patched = patchTablePayload(table.payload, extraSlots); + if (patched === table.payload) { + return bytes; + } + + return rebuild(bytes.subarray(0, 8), sections.map(section => ( + section.id === TABLE_SECTION ? { ...section, payload: patched } : section + ))); +} + +function parseSections(bytes: Uint8Array): WasmSection[] { + if (bytes.length < 8 || bytes[0] !== 0 || bytes[1] !== 0x61 || bytes[2] !== 0x73 || bytes[3] !== 0x6d) { + throw new Error('Not a WebAssembly module.'); + } + + const sections: WasmSection[] = []; + let offset = 8; + + while (offset < bytes.length) { + const id = bytes[offset++]; + const size = readLeb(bytes, offset); + offset = size.next; + sections.push({ id, payload: bytes.subarray(offset, offset + size.value) }); + offset += size.value; + } + + return sections; +} + +function patchTablePayload(payload: Uint8Array, extraSlots: number): Uint8Array { + let offset = 0; + const count = readLeb(payload, offset); + offset = count.next; + + if (count.value !== 1) { + throw new Error(`Expected one function table, found ${count.value}.`); + } + + const elemType = payload[offset++]; + if (elemType !== FUNCREF) { + throw new Error(`Unexpected table element type 0x${elemType.toString(16)}.`); + } + + const limits = payload[offset++]; + const min = readLeb(payload, offset); + offset = min.next; + + if (limits !== LIMITS_MIN_MAX) { + // Already unbounded - addFunction can grow it. + return payload; + } + + const max = readLeb(payload, offset); + const needed = min.value + extraSlots; + + if (max.value >= needed) { + return payload; + } + + return Uint8Array.from([ + ...encodeLeb(1), + FUNCREF, + LIMITS_MIN_MAX, + ...encodeLeb(min.value), + ...encodeLeb(needed), + ]); +} + +function rebuild(header: Uint8Array, sections: WasmSection[]): Uint8Array { + const parts: Uint8Array[] = [header]; + let length = header.length; + + for (const section of sections) { + const size = Uint8Array.from(encodeLeb(section.payload.length)); + parts.push(Uint8Array.of(section.id), size, section.payload); + length += 1 + size.length + section.payload.length; + } + + const out = new Uint8Array(length); + let offset = 0; + for (const part of parts) { + out.set(part, offset); + offset += part.length; + } + + return out; +} + +function readLeb(bytes: Uint8Array, offset: number): { value: number; next: number } { + let value = 0; + let shift = 0; + let next = offset; + + while (next < bytes.length) { + const byte = bytes[next++]; + value |= (byte & 0x7f) << shift; + if ((byte & 0x80) === 0) { + return { value: value >>> 0, next }; + } + + shift += 7; + if (shift > 28) { + throw new Error('LEB128 value is larger than expected for a table limit.'); + } + } + + throw new Error('Truncated LEB128.'); +} + +function encodeLeb(value: number): number[] { + const bytes: number[] = []; + let n = value >>> 0; + + while (n > 0x7f) { + bytes.push((n & 0x7f) | 0x80); + n >>>= 7; + } + + bytes.push(n); + return bytes; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-wire.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-wire.ts new file mode 100644 index 00000000000..ccf34abd814 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-wire.ts @@ -0,0 +1,776 @@ +// The wire format shared by the worker, the host, and the .NET transport. +// +// Two encodings live here, and they exist for two different callers. +// +// The *binary* encoding is what the .NET transport speaks. A batch is one `Uint8Array`, a reply is +// one `Uint8Array`, and both cross the worker boundary as transferred buffers (zero-copy) and the +// Blazor interop boundary as byte arrays (which Blazor marshals without base64). There is no JSON to +// build or parse, a blob is copied once, and a 64-bit integer is eight bytes rather than a decimal +// string. `BsqliteWireFormat` on the .NET side mirrors this file byte for byte. +// +// The *tagged* encoding (`{ t: [...], v: [...] }` rows, `{ type, value }` parameters) is the +// JavaScript-facing shape: structured-clone friendly, readable in a debugger, and what a caller +// driving the host from JavaScript sees. It carries the same information - every value travels with +// its SQLite storage class - so the two encodings never disagree about what a value is. +// +// Both encodings tag values with SQLite's own storage-class codes, which lets the worker label a +// value straight from `sqlite3_column_type` with no translation table. + +import type { SQLiteAPI, SQLiteCompatibleType } from './engine/sqlite-api.js'; + +/** A parameter in the tagged encoding: its SQLite storage class and its JSON-safe value. */ +export interface TaggedParameter { + name: string; + type: number; + value: unknown; +} + +/** One request of a batch, in the tagged shape the worker executes. */ +export interface BatchRequest { + commandText: string; + resultKind?: string | number; + maxRows?: number | null; + parameters?: ReadonlyArray; +} + +/** A row in the tagged encoding: one storage class and one value per column. */ +export interface TaggedRow { + t: number[]; + v: unknown[]; +} + +/** One result set of a decoded reply. */ +export interface DecodedResult { + columnNames: string[]; + columnTypes: (string | null)[]; + recordsAffected: number; + lastInsertRowId: bigint; + rows: TaggedRow[]; +} + +/** An error decoded from a reply, carrying the SQLite result code when SQLite raised it. */ +export interface WireError extends Error { + sqliteCode?: number; +} + +export const WireType = Object.freeze({ + integer: 1, + real: 2, + text: 3, + blob: 4, + null: 5, +}); + +/** The result kinds a request may ask for, as the binary encoding numbers them. */ +export const ResultKind = Object.freeze({ + nonQuery: 0, + scalar: 1, + reader: 2, +}); + +const RESULT_KIND_NAMES = Object.freeze(['nonQuery', 'scalar', 'reader']); + +/** First byte of every binary message, so a stray buffer is rejected rather than misread. */ +export const WIRE_MAGIC = 0xb5; + +/** Bumped whenever the layout changes; the .NET decoder refuses anything else. */ +export const WIRE_VERSION = 1; + +const STATUS_OK = 0; +const STATUS_ERROR = 1; + +/** Integers beyond this cannot be represented exactly as a JavaScript number. */ +const MAX_EXACT_INTEGER = 9007199254740991n; // 2^53 - 1 + +const textEncoder = new TextEncoder(); +const textDecoder = new TextDecoder('utf-8'); + +// ------------------------------------------------------------------------------------------------- +// Binary primitives +// ------------------------------------------------------------------------------------------------- + +/** + * A growable little-endian byte sink. Strings are UTF-8 with a 32-bit byte-length prefix, written + * straight into the buffer with `encodeInto` so there is no intermediate array per string. + */ +export class BinaryWriter { + #bytes: Uint8Array; + #view: DataView; + #length = 0; + + constructor(initialCapacity = 4096) { + this.#bytes = new Uint8Array(initialCapacity); + this.#view = new DataView(this.#bytes.buffer); + } + + get length(): number { + return this.#length; + } + + #ensure(extra: number) { + const needed = this.#length + extra; + if (needed <= this.#bytes.length) { + return; + } + + let capacity = this.#bytes.length * 2; + while (capacity < needed) { + capacity *= 2; + } + + const grown = new Uint8Array(capacity); + grown.set(this.#bytes.subarray(0, this.#length)); + this.#bytes = grown; + this.#view = new DataView(grown.buffer); + } + + u8(value: number): void { + this.#ensure(1); + this.#bytes[this.#length++] = value; + } + + u16(value: number): void { + this.#ensure(2); + this.#view.setUint16(this.#length, value, true); + this.#length += 2; + } + + i32(value: number): void { + this.#ensure(4); + this.#view.setInt32(this.#length, value, true); + this.#length += 4; + } + + u32(value: number): void { + this.#ensure(4); + this.#view.setUint32(this.#length, value, true); + this.#length += 4; + } + + i64(value: bigint): void { + this.#ensure(8); + this.#view.setBigInt64(this.#length, value, true); + this.#length += 8; + } + + f64(value: number): void { + this.#ensure(8); + this.#view.setFloat64(this.#length, value, true); + this.#length += 8; + } + + bytes(bytes: Uint8Array): void { + this.#ensure(4 + bytes.byteLength); + this.#view.setUint32(this.#length, bytes.byteLength, true); + this.#length += 4; + this.#bytes.set(bytes, this.#length); + this.#length += bytes.byteLength; + } + + string(text: string): void { + // UTF-8 never needs more than three bytes per UTF-16 code unit. + this.#ensure(4 + text.length * 3); + const { written } = textEncoder.encodeInto(text, this.#bytes.subarray(this.#length + 4)); + this.#view.setUint32(this.#length, written, true); + this.#length += 4 + written; + } + + /** Reserves a 32-bit slot and returns its offset, for counts known only after the fact. */ + reserveU32(): number { + const offset = this.#length; + this.u32(0); + return offset; + } + + patchU32(offset: number, value: number): void { + this.#view.setUint32(offset, value, true); + } + + /** Reserves an i32 + i64 pair (records affected, last insert row id) for later patching. */ + reserveCounters(): number { + const offset = this.#length; + this.i32(0); + this.i64(0n); + return offset; + } + + patchCounters(offset: number, recordsAffected: number, lastInsertRowId: bigint | number): void { + this.#view.setInt32(offset, recordsAffected, true); + this.#view.setBigInt64(offset + 4, BigInt(lastInsertRowId), true); + } + + /** The written bytes as a view; the caller owns the result and the writer must not be reused. */ + finish(): Uint8Array { + return this.#bytes.subarray(0, this.#length); + } +} + +/** A cursor over a little-endian buffer, the inverse of {@link BinaryWriter}. */ +export class BinaryReader { + #bytes: Uint8Array; + #view: DataView; + #offset = 0; + + constructor(bytes: Uint8Array) { + this.#bytes = bytes; + this.#view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); + } + + get remaining(): number { + return this.#bytes.byteLength - this.#offset; + } + + #need(count: number) { + if (this.#offset + count > this.#bytes.byteLength) { + throw new Error(`Truncated wire message: needed ${count} more byte(s) at offset ${this.#offset}.`); + } + } + + u8(): number { + this.#need(1); + return this.#bytes[this.#offset++]; + } + + u16(): number { + this.#need(2); + const value = this.#view.getUint16(this.#offset, true); + this.#offset += 2; + return value; + } + + i32(): number { + this.#need(4); + const value = this.#view.getInt32(this.#offset, true); + this.#offset += 4; + return value; + } + + u32(): number { + this.#need(4); + const value = this.#view.getUint32(this.#offset, true); + this.#offset += 4; + return value; + } + + i64(): bigint { + this.#need(8); + const value = this.#view.getBigInt64(this.#offset, true); + this.#offset += 8; + return value; + } + + f64(): number { + this.#need(8); + const value = this.#view.getFloat64(this.#offset, true); + this.#offset += 8; + return value; + } + + /** A copy, so the caller never aliases the message buffer. */ + bytes(): Uint8Array { + const length = this.u32(); + this.#need(length); + const value = this.#bytes.slice(this.#offset, this.#offset + length); + this.#offset += length; + return value; + } + + string(): string { + const length = this.u32(); + this.#need(length); + const value = textDecoder.decode(this.#bytes.subarray(this.#offset, this.#offset + length)); + this.#offset += length; + return value; + } +} + +// ------------------------------------------------------------------------------------------------- +// Binary batch (request) encoding +// ------------------------------------------------------------------------------------------------- + +/** + * Encodes a batch of tagged requests to the binary layout. Used by JavaScript callers that want the + * transfer-friendly path and by tests that check the .NET encoder against this one. + */ +export function encodeBatchBinary(batch: ReadonlyArray): Uint8Array { + const writer = new BinaryWriter(); + writer.u8(WIRE_MAGIC); + writer.u8(WIRE_VERSION); + writer.u32(batch.length); + + for (const request of batch) { + writer.string(request.commandText ?? ''); + writer.u8(resultKindCode(request.resultKind)); + writer.i32(Number.isInteger(request.maxRows) && request.maxRows! >= 0 ? request.maxRows! : -1); + + const parameters = request.parameters ?? []; + writer.u16(parameters.length); + for (const parameter of parameters) { + writer.string(parameter.name ?? ''); + writeTaggedValue(writer, parameter.type, parameter.value); + } + } + + return writer.finish(); +} + +/** + * Decodes a binary batch into the tagged request shape the worker executes. Parameter values come + * back already in the form the engine binds (BigInt for integers, Uint8Array for blobs). + */ +export function decodeBatchBinary(bytes: Uint8Array): BatchRequest[] { + const reader = new BinaryReader(bytes); + readHeader(reader, { withStatus: false }); + + const count = reader.u32(); + const batch = new Array(count); + + for (let i = 0; i < count; i++) { + const commandText = reader.string(); + const resultKind = RESULT_KIND_NAMES[reader.u8()] ?? 'nonQuery'; + const maxRows = reader.i32(); + const parameterCount = reader.u16(); + const parameters = new Array(parameterCount); + + for (let p = 0; p < parameterCount; p++) { + const name = reader.string(); + const type = reader.u8(); + parameters[p] = { name, type, value: readTaggedPayload(reader, type) }; + } + + batch[i] = { commandText, resultKind, maxRows: maxRows < 0 ? null : maxRows, parameters }; + } + + return batch; +} + +// ------------------------------------------------------------------------------------------------- +// Binary reply encoding +// ------------------------------------------------------------------------------------------------- + +/** + * Writes the reply header and a success status. The worker then streams results through + * {@link ResultWriter} so rows go from the statement into the buffer with no intermediate array. + */ +export function beginReply(writer: BinaryWriter): void { + writer.u8(WIRE_MAGIC); + writer.u8(WIRE_VERSION); + writer.u8(STATUS_OK); +} + +/** Encodes a failure as a whole reply, carrying the SQLite result code when there is one. */ +export function encodeErrorReply({ message, sqliteCode, name }: { message?: string; sqliteCode?: number | null; name?: string }): Uint8Array { + const writer = new BinaryWriter(256); + writer.u8(WIRE_MAGIC); + writer.u8(WIRE_VERSION); + writer.u8(STATUS_ERROR); + writer.string(message ?? 'The SQLite worker reported a failure.'); + writer.i32(Number.isInteger(sqliteCode) ? sqliteCode! : -1); + writer.string(name ?? 'Error'); + return writer.finish(); +} + +/** + * Serializes one result set directly from a live statement. + * + * Column names and storage classes are written first, then rows as they are stepped, then the + * counters and row count are patched in. Encoding happens while the statement is live because the distinction + * between INTEGER 1 and REAL 1.0 exists only there. + */ +export class ResultWriter { + #writer: BinaryWriter; + #countersOffset = 0; + #rowCountOffset = 0; + #rowCount = 0; + #columnCount = 0; + + constructor(writer: BinaryWriter) { + this.#writer = writer; + } + + /** `storageClasses` holds the SQLite type code per column, or null when unknown. */ + columns(columnNames: string[], storageClasses: (number | null)[] | null): void { + this.#columnCount = columnNames.length; + this.#writer.u16(columnNames.length); + for (let i = 0; i < columnNames.length; i++) { + this.#writer.string(columnNames[i]); + this.#writer.u8(storageClasses?.[i] ?? 0); + } + } + + /** Reserves the counters and row count, which are known only after every statement has run. */ + beginRows(): void { + this.#countersOffset = this.#writer.reserveCounters(); + this.#rowCountOffset = this.#writer.reserveU32(); + this.#rowCount = 0; + } + + /** Writes the current row of `stmt`. */ + row(sqlite3: SQLiteAPI, stmt: number): void { + const writer = this.#writer; + for (let i = 0; i < this.#columnCount; i++) { + const type = sqlite3.column_type(stmt, i); + writer.u8(type); + + switch (type) { + case WireType.null: + break; + + case WireType.integer: + writer.i64(sqlite3.column_int64(stmt, i)); + break; + + case WireType.real: + writer.f64(sqlite3.column_double(stmt, i)); + break; + + case WireType.text: + writer.string(sqlite3.column_text(stmt, i)); + break; + + case WireType.blob: + // Copied immediately: column_blob aliases WebAssembly memory, which the next engine call + // may move. + writer.bytes(sqlite3.column_blob(stmt, i)); + break; + + default: + throw new Error(`Column ${i} has unknown SQLite type ${type}.`); + } + } + + this.#rowCount++; + } + + finish(recordsAffected: number, lastInsertRowId: bigint | number): void { + this.#writer.patchCounters(this.#countersOffset, recordsAffected, lastInsertRowId); + this.#writer.patchU32(this.#rowCountOffset, this.#rowCount); + } +} + +/** + * Decodes a binary reply into the tagged result shape, for JavaScript callers and tests. Throws + * the carried error, with `sqliteCode` attached, when the reply is a failure. + */ +export function decodeReplyBinary(bytes: Uint8Array): DecodedResult[] { + const reader = new BinaryReader(bytes); + const status = readHeader(reader); + + if (status === STATUS_ERROR) { + const message = reader.string(); + const sqliteCode = reader.i32(); + const name = reader.string(); + const error: WireError = new Error(message); + error.name = name; + if (sqliteCode >= 0) { + error.sqliteCode = sqliteCode; + } + + throw error; + } + + const count = reader.u32(); + const results = new Array(count); + + for (let r = 0; r < count; r++) { + const columnCount = reader.u16(); + const columnNames = new Array(columnCount); + const columnTypes = new Array(columnCount); + for (let c = 0; c < columnCount; c++) { + columnNames[c] = reader.string(); + columnTypes[c] = STORAGE_CLASS_NAMES[reader.u8() as keyof typeof STORAGE_CLASS_NAMES] ?? null; + } + + const recordsAffected = reader.i32(); + const lastInsertRowId = reader.i64(); + const rowCount = reader.u32(); + const rows = new Array(rowCount); + + for (let i = 0; i < rowCount; i++) { + const t = new Array(columnCount); + const v = new Array(columnCount); + for (let c = 0; c < columnCount; c++) { + const type = reader.u8(); + t[c] = type; + v[c] = toTaggedValue(type, readTaggedPayload(reader, type)); + } + + rows[i] = { t, v }; + } + + results[r] = { columnNames, columnTypes, recordsAffected, lastInsertRowId, rows }; + } + + return results; +} + +const STORAGE_CLASS_NAMES = Object.freeze({ + [WireType.integer]: 'INTEGER', + [WireType.real]: 'REAL', + [WireType.text]: 'TEXT', + [WireType.blob]: 'BLOB', +}); + +function readHeader(reader: BinaryReader, { withStatus = true }: { withStatus?: boolean } = {}): number { + const magic = reader.u8(); + if (magic !== WIRE_MAGIC) { + throw new Error(`Not a Bsqlite wire message (leading byte 0x${magic.toString(16)}).`); + } + + const version = reader.u8(); + if (version !== WIRE_VERSION) { + throw new Error(`Unsupported wire version ${version}; this build speaks version ${WIRE_VERSION}.`); + } + + return withStatus ? reader.u8() : STATUS_OK; +} + +function resultKindCode(kind: string | number | undefined): number { + if (typeof kind === 'number') { + return kind; + } + + const index = (RESULT_KIND_NAMES as readonly string[]).indexOf(kind ?? 'nonQuery'); + return index < 0 ? ResultKind.nonQuery : index; +} + +function writeTaggedValue(writer: BinaryWriter, type: number, value: unknown): void { + writer.u8(type); + switch (type) { + case WireType.null: + return; + + case WireType.integer: + writer.i64(typeof value === 'bigint' ? value : BigInt(typeof value === 'string' ? value : Math.trunc(value as number))); + return; + + // Infinity may arrive as the string the tagged encoding spells it with. + case WireType.real: + writer.f64(typeof value === 'string' ? Number(value) : value as number); + return; + + case WireType.text: + writer.string(value as string); + return; + + case WireType.blob: + writer.bytes(value instanceof Uint8Array ? value : fromBase64(value as string)); + return; + + default: + throw new Error(`Cannot encode unknown wire type ${type}.`); + } +} + +/** Reads a value payload for `type`; integers come back as BigInt, blobs as Uint8Array. */ +function readTaggedPayload(reader: BinaryReader, type: number): bigint | number | string | Uint8Array | null { + switch (type) { + case WireType.null: + return null; + + case WireType.integer: + return reader.i64(); + + case WireType.real: + return reader.f64(); + + case WireType.text: + return reader.string(); + + case WireType.blob: + return reader.bytes(); + + default: + throw new Error(`Unknown wire type ${type} in message.`); + } +} + +/** Converts an engine-form value into the tagged (JSON-safe) representation. */ +function toTaggedValue(type: number, value: unknown): unknown { + switch (type) { + case WireType.integer: { + const integer = value as bigint; + return integer > MAX_EXACT_INTEGER || integer < -MAX_EXACT_INTEGER ? String(integer) : Number(integer); + } + + case WireType.blob: + return toBase64(value as Uint8Array); + + default: + return value; + } +} + +// ------------------------------------------------------------------------------------------------- +// Tagged (JSON-safe) encoding, for JavaScript callers +// ------------------------------------------------------------------------------------------------- + +/** + * Reads the current row of a live statement into tagged form. `columnCount` comes from + * `column_count`, so a row of no columns is still well-formed. + */ +export function encodeRow(sqlite3: SQLiteAPI, stmt: number, columnCount: number): TaggedRow { + const t = new Array(columnCount); + const v = new Array(columnCount); + + for (let i = 0; i < columnCount; i++) { + const type = sqlite3.column_type(stmt, i); + t[i] = type; + + switch (type) { + case WireType.null: + v[i] = null; + break; + + case WireType.integer: { + // wa-sqlite hands back a number when the value fits exactly and a BigInt when it does not, + // which is the same boundary this format cares about. + const value = sqlite3.column(stmt, i); + v[i] = typeof value === 'bigint' ? String(value) : value; + break; + } + + case WireType.real: { + // JSON has no spelling for Infinity - JSON.stringify writes null - so it travels as the + // string Number() and double.Parse both read back. NaN never reaches here: SQLite stores + // it as NULL. + const value = sqlite3.column_double(stmt, i); + v[i] = Number.isFinite(value) ? value : String(value); + break; + } + + case WireType.text: + v[i] = sqlite3.column_text(stmt, i); + break; + + case WireType.blob: + // Encoded immediately: column_blob can alias WebAssembly memory, which the next engine call + // may move. + v[i] = toBase64(sqlite3.column_blob(stmt, i)); + break; + + default: + throw new Error(`Column ${i} has unknown SQLite type ${type}.`); + } + } + + return { t, v }; +} + +/** Turns a tagged parameter back into the value the engine binds. */ +export function decodeParameter({ type, value }: { type: number; value: unknown }): SQLiteCompatibleType { + switch (type) { + case WireType.null: + return null; + + // A large integer arrives as a decimal string. Either way BigInt is what makes the engine bind + // it as int64: wa-sqlite binds a number outside int32 with bind_double, which would store + // Date.now() as REAL - and in a TEXT or untyped column, as '1700000000000.0'. + case WireType.integer: + if (typeof value === 'bigint') { + return value; + } + + if (typeof value === 'number' && value === (value | 0)) { + return value; + } + + return BigInt(typeof value === 'string' ? value : Math.trunc(value as number)); + + // Infinity arrives as a string, JSON having no spelling for it. + case WireType.real: + return typeof value === 'string' ? Number(value) : value as number; + + case WireType.text: + return value as string; + + case WireType.blob: + return value instanceof Uint8Array ? value : fromBase64(value as string); + + default: + throw new Error(`A parameter arrived with unknown wire type ${type}.`); + } +} + +/** Tags a plain JavaScript value, for callers driving the library from JavaScript rather than .NET. */ +export function encodeValue(value: unknown): { type: number; value: unknown } { + if (value === null || value === undefined) { + return { type: WireType.null, value: null }; + } + + switch (typeof value) { + case 'boolean': + return { type: WireType.integer, value: value ? 1 : 0 }; + + case 'number': + if (Number.isNaN(value)) { + // SQLite stores NaN as NULL, so it is sent as one. + return { type: WireType.null, value: null }; + } + + if (Number.isInteger(value)) { + return { type: WireType.integer, value }; + } + + return { type: WireType.real, value: Number.isFinite(value) ? value : String(value) }; + + case 'bigint': + return { + type: WireType.integer, + value: value > MAX_EXACT_INTEGER || value < -MAX_EXACT_INTEGER ? String(value) : Number(value), + }; + + case 'string': + return { type: WireType.text, value }; + + default: + if (value instanceof Uint8Array) { + return { type: WireType.blob, value: toBase64(value) }; + } + + throw new Error(`Cannot put a value of type ${typeof value} on the wire.`); + } +} + +/** Untags a value, giving back a `bigint` only where a number would lose precision. */ +export function decodeValue(type: number, value: unknown): unknown { + switch (type) { + case WireType.null: + return null; + + case WireType.integer: + return typeof value === 'string' ? BigInt(value) : value; + + case WireType.real: + return typeof value === 'string' ? Number(value) : value; + + case WireType.blob: + return fromBase64(value as string); + + default: + return value; + } +} + +// Chunked because spreading a large array into String.fromCharCode overflows the argument limit, which +// shows up only once a blob gets big - exactly where it is least welcome. +const CHUNK = 0x8000; + +export function toBase64(bytes: Uint8Array): string { + let binary = ''; + for (let i = 0; i < bytes.length; i += CHUNK) { + binary += String.fromCharCode.apply(null, bytes.subarray(i, i + CHUNK) as unknown as number[]); + } + + return btoa(binary); +} + +export function fromBase64(text: string): Uint8Array { + const binary = atob(text); + const bytes = new Uint8Array(binary.length); + for (let i = 0; i < binary.length; i++) { + bytes[i] = binary.charCodeAt(i); + } + + return bytes; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-worker.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-worker.ts new file mode 100644 index 00000000000..4ee39af5e03 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/bit-bsqlite-worker.ts @@ -0,0 +1,838 @@ +// The dedicated worker that owns one SQLite database. +// +// It exists because the storage APIs worth using are worker-only: OPFS synchronous access handles do +// not exist on the main thread. Everything here is request/response over postMessage, correlated by +// id, and strictly serialized - one SQLite connection cannot execute two statements at once, so +// overlapping requests queue rather than interleave. +// +// Two execute paths share one engine loop. `executeBinary` is what the .NET transport uses: the +// batch arrives as a transferred buffer, rows are written straight from the statement into a +// reply buffer, and that buffer is transferred back. `execute` is the JavaScript-facing path with +// structured-clone-friendly tagged rows. Both are driven by `runBatch` with a different row sink. + +import * as SQLite from './engine/sqlite-api.js'; +import type { SQLiteAPI, SQLiteModule } from './engine/sqlite-api.js'; +import type { JsModuleReference } from './bit-bsqlite-host.js'; +import type { BatchRequest } from './bit-bsqlite-wire.js'; + +// This module runs as a dedicated worker, while the shared tsconfig types every module against the DOM +// (the host and the storage admin modules run on the window). Only this much of the worker scope is +// used here, so it is described rather than pulling in the WebWorker lib, which conflicts with DOM. +interface MessageTarget { + postMessage(message: unknown, transfer?: Transferable[]): void; +} + +const scope = self as unknown as MessageTarget & { + addEventListener(type: 'message', listener: (event: MessageEvent) => void): void; +}; + +// The host spawns this worker at a URL carrying the package version, and every module below has to +// answer to that same key. A static import would not: the browser resolves it to the bare path and +// can satisfy it from a copy cached under an older version, which then links against this file and +// fails - as an error event with no message, because that is all a worker that died at load can say. +const VERSION = new URL(import.meta.url).search; + +// Bound through `ready` rather than by an import statement, so the message listener below is still +// registered synchronously. The host posts `open` the instant it constructs the worker, and a +// message that arrives before there is a listener is gone. +type Engine = typeof import('./bit-bsqlite-engine.js'); +type Functions = typeof import('./bit-bsqlite-functions.js'); +type Wire = typeof import('./bit-bsqlite-wire.js'); + +let loadEngine: Engine['loadEngine']; +let registerFunctions: Functions['registerFunctions']; +let resetAggregateState: Functions['resetAggregateState']; +let BinaryWriter: Wire['BinaryWriter']; +let ResultWriter: Wire['ResultWriter']; +let beginReply: Wire['beginReply']; +let decodeBatchBinary: Wire['decodeBatchBinary']; +let decodeParameter: Wire['decodeParameter']; +let encodeErrorReply: Wire['encodeErrorReply']; +let encodeRow: Wire['encodeRow']; + +const ready = (async () => { + ({ loadEngine } = await import('./bit-bsqlite-engine.js' + VERSION)); + ({ registerFunctions, resetAggregateState } = await import('./bit-bsqlite-functions.js' + VERSION)); + ({ + BinaryWriter, + ResultWriter, + beginReply, + decodeBatchBinary, + decodeParameter, + encodeErrorReply, + encodeRow, + } = await import('./bit-bsqlite-wire.js' + VERSION)); +})(); + +interface Session { + sqlite3: SQLiteAPI; + module: SQLiteModule; + db: number; + build: string; + vfsName: string | null; + databaseName: string; + limits: { supportsMultiDatabaseTransactions: boolean; canChangePageSize: boolean }; + statements: StatementCache; + /** + * Tables written since the last notification, and whether a transaction has committed since then. + * See {@link notifyIfCommitted}. + */ + changes: { tables: Set; committed: boolean }; + stmtReadonly: (stmt: number) => number; + lastInsertRowId: (db: number) => bigint; +} + +/** What `open` is asked for, as the host posts it. */ +interface OpenRequest { + databaseName: string; + requiredBuild: string; + vfs?: JsModuleReference | null; + limits?: { supportsMultiDatabaseTransactions?: boolean; canChangePageSize?: boolean }; + foreignKeys?: boolean; + onOpenSql?: string[]; + extensions?: JsModuleReference[]; +} + +/** The callbacks one execute path streams a batch's outcome into. See {@link runBatch}. */ +interface RowSink { + begin(): void; + columns(names: string[], storageClasses: (number | null)[]): void; + beginRows(): void; + row(sqlite3: SQLiteAPI, stmt: number, columnCount: number): void; + end(recordsAffected: number, lastInsertRowId: bigint): void; +} + +/** A result set of the JavaScript-facing `execute` path. */ +interface TaggedResult { + columnNames: string[]; + columnTypes: (string | null)[]; + rows: { t: number[]; v: unknown[] }[]; + recordsAffected: number; + lastInsertRowId: bigint | number; +} + +let session: Session | null = null; + +// Requests are chained so that each runs to completion before the next begins. Without this a second +// execute could start stepping while the first is suspended awaiting storage. +let queue: Promise = Promise.resolve(); + +scope.addEventListener('message', event => { + const request = event.data; + const port: MessageTarget = event.ports?.[0] ?? scope; + + const run = async () => { + try { + await ready; + + if (request.kind === 'executeBinary') { + // Never throws across the boundary: the reply itself carries success or failure, so the + // .NET side decodes exactly one shape. + const reply = await executeBinary(request.batch); + port.postMessage({ id: request.id, ok: true, result: reply }, [reply.buffer]); + return; + } + + const result = await dispatch(request); + port.postMessage({ id: request.id, ok: true, result }); + } catch (error) { + port.postMessage({ id: request.id, ok: false, error: describe(error) }); + } + }; + + // The chain has to survive its own failure. `then(run)` alone would skip every later request + // once one link rejected - and postMessage can still throw after the catch above, on a result + // the structured clone algorithm refuses. A wedged worker answers nothing, ever again. + queue = queue.then(run, run).catch(() => { }); +}); + +/** + * Errors cross the boundary as data, so the .NET side can raise a typed exception carrying the SQLite + * result code rather than a flattened string. + */ +function describe(error: any): { message: string; sqliteCode: number | null; name: string } { + return { + message: error?.message ?? String(error), + // wa-sqlite raises SQLiteError with the primary result code; anything else has no code. + sqliteCode: typeof error?.code === 'number' ? error.code : null, + name: error?.name ?? 'Error', + }; +} + +async function dispatch(request: any): Promise { + switch (request.kind) { + case 'open': + return await open(request); + + case 'execute': + return await execute(request.batch ?? []); + + case 'close': + return await close(); + + case 'version': + return { version: requireSession().sqlite3.libversion(), build: session!.build }; + + case 'serialize': + return serialize(request.schema ?? 'main'); + + default: + throw new Error(`Unknown request kind '${request.kind}'.`); + } +} + +function requireSession(): Session { + if (!session) { + throw new Error('No database is open in this worker.'); + } + + return session; +} + +async function open({ databaseName, requiredBuild, vfs, limits, foreignKeys, onOpenSql, extensions }: OpenRequest) { + if (session) { + if (session.databaseName !== databaseName) { + throw new Error( + `This worker already owns '${session.databaseName}'. A worker hosts one database, so ` + + `opening '${databaseName}' needs its own worker.`); + } + + return { build: session.build, vfsName: session.vfsName, reused: true }; + } + + const engine = await loadEngine({ requiredBuild, databaseName, vfs }); + const { sqlite3, module } = engine; + + // The VFS name matters: passing it explicitly means a provider that registers without claiming the + // default still gets used, and a bug in registration surfaces as "no such vfs" here rather than as + // a database silently opened on the wrong storage. + const db = await sqlite3.open_v2( + databaseName, + SQLite.SQLITE_OPEN_CREATE | SQLite.SQLITE_OPEN_READWRITE, + engine.vfsName ?? undefined); + + // EF Core's SQLite provider only installs these on a real SqliteConnection. We are not one, so + // every open has to - miss it and every decimal comparison, aggregate, and REGEXP is wrong. + registerFunctions(module, sqlite3, db); + + // Change tracking for live queries. The update hook names every table a row was inserted into, + // updated in, or deleted from - including through triggers and ON DELETE CASCADE, which no + // reading of the SQL can know about - and the commit hook says when those rows became visible to + // anyone else. Together they let a notification wait for COMMIT instead of firing on a write that + // may yet roll back, and that another tab could not have seen anyway. + const changes = { tables: new Set(), committed: false }; + sqlite3.update_hook(db, (_operation, _schema, table) => { + if (table) { + changes.tables.add(table); + } + }); + sqlite3.commit_hook(db, () => { + changes.committed = true; + return SQLite.SQLITE_OK; + }); + + const stmtReadonly = module.cwrap('sqlite3_stmt_readonly', 'number', ['number']); + const lastInsertRowId = (db: number) => { + const lo32 = module.ccall('sqlite3_last_insert_rowid', 'number', ['number'], [db]); + const hi32 = module.getTempRet0(); + return (BigInt(hi32) << 32n) | (BigInt(lo32) & 0xffffffffn); + }; + + session = { + sqlite3, + module, + db, + build: engine.build, + vfsName: engine.vfsName, + databaseName, + limits: { + supportsMultiDatabaseTransactions: limits?.supportsMultiDatabaseTransactions !== false, + canChangePageSize: limits?.canChangePageSize !== false, + }, + statements: new StatementCache(sqlite3, db), + changes, + stmtReadonly, + lastInsertRowId, + }; + + // SQLite defaults foreign keys OFF, and Microsoft.Data.Sqlite turns them ON for every connection + // it opens. We are not one, so without this the same model enforces its relationships on the + // server and silently does not in the browser: an orphaned row inserts cleanly here and is + // rejected there, and ON DELETE CASCADE never fires. On unless the caller opts out, and still + // overridable - EF's own table-rebuild migrations toggle it off and back on. + const configuration = [ + ...(foreignKeys === false ? [] : ['PRAGMA foreign_keys=ON']), + ...(engine.onOpenSql ?? []), + ...(onOpenSql ?? []), + ]; + + // Provider configuration (cache_size for batch-atomic and the like) and the application's own. + // Run here rather than through execute() so a guard meant for application SQL cannot block them, + // and before extensions so an extension sees the configured connection. + for (const sql of configuration) { + await runConfigurationSql(sqlite3, db, sql); + } + + // Extension modules add user-defined functions, collations, or virtual tables. They receive the + // same handles the built-in function set does. + for (const extension of extensions ?? []) { + const imported = await import(extension.moduleUrl); + const exportName = extension.registerExport ?? 'register'; + const register = Object.hasOwn(imported, exportName) ? imported[exportName] : undefined; + if (typeof register !== 'function') { + throw new Error(`'${extension.moduleUrl}' does not export a function named '${exportName}'.`); + } + + await register({ module, sqlite3, db, databaseName }); + } + + return { build: engine.build, vfsName: engine.vfsName, reused: false }; +} + +async function runConfigurationSql(sqlite3: SQLiteAPI, db: number, sql: string): Promise { + for await (const stmt of sqlite3.statements(db, sql)) { + while (await sqlite3.step(stmt) === SQLite.SQLITE_ROW) { + // Drained: these statements are configuration, not queries. + } + } +} + +async function close() { + if (!session) { + return { closed: false }; + } + + const current = session; + session = null; + await current.statements.clear(); + await current.sqlite3.close(current.db); + return { closed: true }; +} + +/** + * A consistent snapshot of the open database as a SQLite file image, taken through + * `sqlite3_serialize` so it reflects every committed write without closing the connection. + */ +function serialize(schema: string): Uint8Array { + const { module, db } = requireSession(); + const sizePtr = module._sqlite3_malloc(8); + try { + const pointer = module.ccall( + 'sqlite3_serialize', + 'number', + ['number', 'string', 'number', 'number'], + [db, schema, sizePtr, 0]); + + const lo32 = module.getValue(sizePtr, 'i32'); + const hi32 = module.getValue(sizePtr + 4, 'i32'); + const size = hi32 * 0x100000000 + (lo32 >>> 0); + + if (!pointer) { + if (size === 0) { + // A database with no pages yet serializes to nothing, which is a valid (empty) image. + return new Uint8Array(0); + } + + throw new Error(`sqlite3_serialize could not snapshot schema '${schema}'.`); + } + + try { + return module.HEAPU8.slice(pointer, pointer + size); + } finally { + module._sqlite3_free(pointer); + } + } finally { + module._sqlite3_free(sizePtr); + } +} + +/** + * Runs a batch in one round trip, returning one result per request in order. + * + * Batching is the whole point of the coarse transport contract: in the browser the cost that dominates + * is crossing this boundary, not executing the SQL. + */ +async function execute(batch: BatchRequest[]): Promise { + const results: TaggedResult[] = []; + + await runBatch(batch, { + begin: () => { + results.push({ + columnNames: [], + columnTypes: [], + rows: [], + recordsAffected: 0, + lastInsertRowId: 0n, + }); + }, + columns: (names, storageClasses) => { + const result = results[results.length - 1]; + result.columnNames = names; + result.columnTypes = storageClasses.map(code => code === null ? null : STORAGE_CLASS[code] ?? null); + }, + beginRows: () => { }, + row: (sqlite3, stmt, columnCount) => { + results[results.length - 1].rows.push(encodeRow(sqlite3, stmt, columnCount)); + }, + end: (recordsAffected, lastInsertRowId) => { + const result = results[results.length - 1]; + result.recordsAffected = recordsAffected; + result.lastInsertRowId = lastInsertRowId; + }, + }); + + // Tagged rows are plain data, but the BigInt row id is not JSON-safe for every caller. + for (const result of results) { + result.lastInsertRowId = Number(result.lastInsertRowId); + } + + return results; +} + +/** The .NET path: a binary batch in, a binary reply out, with no object graph in between. */ +async function executeBinary(bytes: Uint8Array): Promise { + const writer = new BinaryWriter(Math.max(4096, bytes.byteLength * 4)); + beginReply(writer); + + let batch; + try { + batch = decodeBatchBinary(bytes); + } catch (error) { + return encodeErrorReply(describe(error)); + } + + const countOffset = writer.reserveU32(); + const result = new ResultWriter(writer); + let completed = 0; + + try { + await runBatch(batch, { + begin: () => { + completed++; + }, + columns: (names, storageClasses) => result.columns(names, storageClasses), + beginRows: () => result.beginRows(), + row: (sqlite3, stmt) => result.row(sqlite3, stmt), + end: (recordsAffected, lastInsertRowId) => result.finish(recordsAffected, lastInsertRowId), + }); + } catch (error) { + // A failure part-way through discards the partial reply: the .NET side wants one exception, + // not half a result set followed by one. + return encodeErrorReply(describe(error)); + } + + writer.patchU32(countOffset, completed); + return writer.finish(); +} + +/** + * The engine loop shared by both execute paths. + * + * The sink receives, per request: `begin()`, then `columns(names, classes)` once the row-producing + * statement is known (or with empty arrays if none), then `beginRows()`, then `row(...)` per row, + * then `end(recordsAffected, lastInsertRowId)`. + */ +async function runBatch(batch: BatchRequest[], sink: RowSink): Promise { + const current = requireSession(); + let schemaChanged = false; + + try { + for (const request of batch) { + if (isSchemaChange(request.commandText)) { + schemaChanged = true; + } + + await runRequest(current, request, sink); + } + } finally { + // Also on failure: the statements before the one that failed have run, and in autocommit mode + // they have committed. + notifyIfCommitted(current); + + if (schemaChanged) { + // Cached statements compiled against the old schema would re-prepare themselves on step, but + // their column metadata was read before stepping. Dropping them is cheap and always right. + await current.statements.clear(); + } + } +} + +/** + * Runs one request of a batch and streams its outcome into the sink. + */ +async function runRequest(current: Session, request: BatchRequest, sink: RowSink): Promise { + const { sqlite3, db } = current; + + rejectUnsupportedSql(request.commandText); + sink.begin(); + + const wantsRows = request.resultKind !== 'nonQuery'; + const maxRows = request.resultKind === 'scalar' + ? 1 + : (Number.isInteger(request.maxRows) && request.maxRows! >= 0 ? request.maxRows! : Infinity); + + const prepared = await current.statements.acquire(request.commandText); + let recordsAffected = 0; + let captured = false; + + try { + for (const stmt of prepared.statements) { + try { + bind(sqlite3, stmt, request.parameters ?? []); + const columnCount = sqlite3.column_count(stmt); + const producesRows = columnCount > 0; + + // Rows are taken from the first statement that produces any, and later statements still + // run. This is what EF's insert-then-select pattern needs: the INSERT reports its changes + // and the following SELECT is the result set the reader consumes. + if (wantsRows && producesRows && !captured) { + captured = true; + const names = sqlite3.column_names(stmt); + let storageClasses: (number | null)[] | null = null; + let rows = 0; + + try { + while (rows < maxRows && await sqlite3.step(stmt) === SQLite.SQLITE_ROW) { + if (storageClasses === null) { + // Storage classes of the first row, not declared types: the engine is compiled + // with SQLITE_OMIT_DECLTYPE, so declared types do not exist in this build. + storageClasses = readStorageClasses(sqlite3, stmt, columnCount); + sink.columns(names, storageClasses); + sink.beginRows(); + } + + sink.row(sqlite3, stmt, columnCount); + rows++; + } + } finally { + if (storageClasses === null) { + sink.columns(names, new Array(columnCount).fill(null)); + sink.beginRows(); + } + } + } else { + while (await sqlite3.step(stmt) === SQLite.SQLITE_ROW) { + // Drained deliberately: a statement whose rows nobody asked for still has to run. + } + } + + // Read-only statements leave sqlite3_changes() at the previous write's value, which must + // not be counted twice - this is the same rule Microsoft.Data.Sqlite applies. + if (!current.stmtReadonly(stmt)) { + recordsAffected += sqlite3.changes(db); + } + } finally { + // Whether the statement finished or threw, its bindings and cursor must be released: to + // free locks, and so a cached statement starts clean next time. + await prepared.release(stmt); + } + } + + if (!captured) { + sink.columns([], []); + sink.beginRows(); + } + } finally { + await prepared.dispose(); + + // Aggregate state is keyed by an address SQLite frees with the statement and reuses, so it + // is dropped at the statement boundary rather than left to the final callbacks to clean up. + resetAggregateState(); + + // Also on failure, for the statements that ran before it. + recordTextualWrites(current, request.commandText, recordsAffected); + } + + sink.end(recordsAffected, current.lastInsertRowId(db)); +} + +/** + * Adds the tables a request's SQL names, for the writes the update hook cannot see. + * + * The update hook is exact for row changes but blind to two things: DDL, which changes no row, and + * `DELETE FROM t` with no WHERE, where SQLite's truncate optimisation bypasses the hook. The + * statement text covers both, at the cost of the odd extra name - a comment or string that happens + * to contain a keyword - which only causes an extra re-query, never a missed one. It is consulted + * only when the request changed rows or reads as DDL, so a write that matched nothing raises + * nothing. Mirrors BsqliteTableNames on the .NET side. + */ +function recordTextualWrites(current: Session, sql: string, recordsAffected: number): void { + if ((recordsAffected > 0 || isSchemaChange(sql)) && looksLikeWrite(sql)) { + for (const name of extractTables(sql)) { + current.changes.tables.add(name); + } + } +} + +/** + * Tells the host which tables changed, once the change is committed. + * + * Nothing is sent while a transaction is open: a live query in another tab that re-ran on an + * uncommitted write would read the old data and then never hear about the commit. Once the + * connection is back in autocommit, the accumulated names go out if a commit happened, and are + * dropped if the transaction rolled back - nothing anyone can see has changed. + */ +function notifyIfCommitted({ sqlite3, db, databaseName, changes }: Session): void { + if (!sqlite3.get_autocommit(db)) { + return; + } + + const { tables, committed } = changes; + changes.tables = new Set(); + changes.committed = false; + + if (!committed || tables.size === 0) { + return; + } + + scope.postMessage({ kind: 'notify', databaseName, tables: [...tables] }); +} + +/** + * Prepared-statement cache keyed by SQL text. + * + * EF Core and most applications send the same statement text over and over with different + * parameters. Preparing is a full parse and plan; resetting a prepared statement is nearly free. + * Only single-statement texts are cached, because a multi-statement batch is almost never repeated + * verbatim and would need every handle kept alive together. + */ +class StatementCache { + static CAPACITY = 64; + static SQLITE_PREPARE_PERSISTENT = 0x01; + + #sqlite3: SQLiteAPI; + #db: number; + /** Insertion order is recency: Map iteration gives the oldest first. */ + #entries = new Map(); + /** Handles currently lent out, which must not be evicted or lent twice. */ + #inUse = new Set(); + + constructor(sqlite3: SQLiteAPI, db: number) { + this.#sqlite3 = sqlite3; + this.#db = db; + } + + async acquire(sql: string): Promise<{ statements: number[]; release(stmt: number): Promise; dispose(): Promise }> { + const sqlite3 = this.#sqlite3; + const cached = this.#entries.get(sql); + + if (cached !== undefined && !this.#inUse.has(cached)) { + // Refresh recency. + this.#entries.delete(sql); + this.#entries.set(sql, cached); + this.#inUse.add(cached); + return { + statements: [cached], + release: stmt => this.#reset(stmt), + dispose: async () => { + this.#inUse.delete(cached); + }, + }; + } + + const statements: number[] = []; + try { + for await (const stmt of sqlite3.statements(this.#db, sql, { + unscoped: true, + flags: StatementCache.SQLITE_PREPARE_PERSISTENT, + })) { + statements.push(stmt); + } + } catch (error) { + for (const stmt of statements) { + await sqlite3.finalize(stmt); + } + + throw error; + } + + if (statements.length === 1 && cached === undefined) { + const stmt = statements[0]; + this.#entries.set(sql, stmt); + this.#inUse.add(stmt); + await this.#evict(); + return { + statements, + release: s => this.#reset(s), + dispose: async () => { + this.#inUse.delete(stmt); + }, + }; + } + + return { + statements, + release: async () => { + // Finalized in dispose; nothing to release per statement. + }, + dispose: async () => { + for (const stmt of statements) { + await sqlite3.finalize(stmt); + } + }, + }; + } + + async #reset(stmt: number): Promise { + try { + await this.#sqlite3.reset(stmt); + } catch { + // reset() repeats the error code of a failed step, which has already been thrown. + } + + try { + this.#sqlite3.clear_bindings(stmt); + } catch { + // A statement that cannot clear is finalized on the next schema change or eviction. + } + } + + async #evict(): Promise { + while (this.#entries.size > StatementCache.CAPACITY) { + let evicted = false; + for (const [sql, stmt] of this.#entries) { + if (this.#inUse.has(stmt)) { + continue; + } + + this.#entries.delete(sql); + await this.#sqlite3.finalize(stmt); + evicted = true; + break; + } + + if (!evicted) { + return; + } + } + } + + async clear(): Promise { + const entries = [...this.#entries.values()]; + this.#entries.clear(); + for (const stmt of entries) { + if (!this.#inUse.has(stmt)) { + await this.#sqlite3.finalize(stmt); + } + } + } +} + +function isSchemaChange(sql: unknown): boolean { + return typeof sql === 'string' && /\b(?:create|drop|alter|vacuum|reindex)\b/i.test(sql); +} + +// Every statement, not just the first: a batch is routinely `BEGIN; INSERT …; COMMIT;`, and looking +// only at the leading keyword would call that a read. Leading comments and a leading common table +// expression are skipped for the same reason: `WITH … INSERT INTO` is a write. A block comment is +// spelled out rather than matched lazily: a lazy body can end at any `*/`, so a run of `*//*` that +// never reaches a write backtracks through every way of splitting it - exponential in its length. +// Mirrors BsqliteTableNames.LooksLikeWrite. +function looksLikeWrite(sql: unknown): boolean { + return typeof sql === 'string' + && /(?:^|;)\s*(?:--[^\n]*\n\s*|\/\*[^*]*\*+(?:[^/*][^*]*\*+)*\/\s*)*(?:with\b[\s\S]*?)?\b(?:insert|update|delete|replace|create|drop|alter)\b/i.test(sql); +} + +// Table-level and deliberately coarse. Accepts the identifier quoting SQLite does ("t", `t`, [t]), +// a schema prefix (main."t"), the conflict clause an UPDATE may carry (UPDATE OR REPLACE t), and the +// objects DDL names. Mirrors BsqliteTableNames.Extract. +function extractTables(sql: string): string[] { + const names: string[] = []; + const pattern = /\b(?:from|join|into|update(?:\s+or\s+(?:rollback|abort|replace|fail|ignore))?|table|view|trigger|index\s+(?:if\s+(?:not\s+)?exists\s+)?[\w"`[\]]+\s+on)\s+(?:if\s+(?:not\s+)?exists\s+)?(?:(?:temp|temporary)\s+)?(?:["`[]?[A-Za-z_]\w*["`\]]?\s*\.\s*)?["`[]?([A-Za-z_]\w*)/gi; + let match; + while ((match = pattern.exec(sql))) { + if (!match[1].toLowerCase().startsWith('sqlite_')) { + names.push(match[1]); + } + } + + return names; +} + +/** + * WAL is not a VFS-specific limitation: no web build of SQLite can provide the shared memory it + * needs. Rejected here as well as in the .NET command layer so a JavaScript caller cannot enable it + * either - and so a "success" that actually left DELETE mode cannot be mistaken for WAL. + */ +function rejectUnsupportedSql(sql: unknown): void { + if (typeof sql !== 'string') { + return; + } + + // Cheap pre-check: the guarded statements all mention one of these words, and most SQL does not. + const lower = sql.length < 4096 ? sql.toLowerCase() : null; + if (lower !== null && !lower.includes('pragma') && !lower.includes('attach')) { + return; + } + + if (/pragma\s+journal_mode\s*=\s*['"`]?wal\b/i.test(sql)) { + throw new Error( + 'WAL mode is not available in the browser: WebAssembly has no shared-memory primitives for it, ' + + 'and no web VFS implements it. Bsqlite uses DELETE or TRUNCATE journaling.'); + } + + const limits = session?.limits; + if (limits && !limits.supportsMultiDatabaseTransactions && /(?:^|;)\s*attach(?:\s+database)?\b/i.test(sql)) { + throw new Error( + 'ATTACH is not available on this storage backend: it cannot run a transaction that spans ' + + 'more than one database. Open each database on its own connection.'); + } + + if (limits && !limits.canChangePageSize && /pragma\s+page_size\s*=/i.test(sql)) { + throw new Error( + 'PRAGMA page_size cannot be changed on this storage backend: the page size is fixed to the ' + + 'backend\'s block size.'); + } +} + +/** + * Binds by asking the statement what it wants, rather than handing the engine a dictionary and hoping + * the keys line up. + * + * The alternative - `bind_collection` with a name-keyed object - leaves an unmatched parameter bound to + * NULL, which turns a misspelled parameter name into wrong results instead of an error. Names are + * matched with or without their sigil, because ADO.NET callers write `p0` as often as `@p0`, and a + * statement using positional `?` markers reports no name at all and is matched by position. + */ +function bind(sqlite3: SQLiteAPI, stmt: number, parameters: ReadonlyArray<{ name: string; type: number; value: unknown }>): void { + const count = sqlite3.bind_parameter_count(stmt); + if (count === 0) { + return; + } + + const byName = parameters.length > 4 ? new Map(parameters.map(p => [p.name, p])) : null; + const lookup = (name: string) => byName ? byName.get(name) : parameters.find(p => p.name === name); + + for (let i = 1; i <= count; i++) { + const declared = sqlite3.bind_parameter_name(stmt, i); + const parameter = declared + ? (lookup(declared) ?? lookup(declared.slice(1))) + : parameters[i - 1]; + + if (!parameter) { + throw new Error( + `No value was supplied for parameter ${declared ?? `#${i}`}. Supplied: ` + + `${parameters.map(p => p.name).join(', ') || '(none)'}.`); + } + + sqlite3.bind(stmt, i, decodeParameter(parameter)); + } +} + +const STORAGE_CLASS: Readonly> = Object.freeze({ + [SQLite.SQLITE_INTEGER]: 'INTEGER', + [SQLite.SQLITE_FLOAT]: 'REAL', + [SQLite.SQLITE_TEXT]: 'TEXT', + [SQLite.SQLITE_BLOB]: 'BLOB', + [SQLite.SQLITE_NULL]: null, +}); + +function readStorageClasses(sqlite3: SQLiteAPI, stmt: number, columnCount: number): (number | null)[] { + const classes = new Array(columnCount); + for (let i = 0; i < columnCount; i++) { + const code = sqlite3.column_type(stmt, i); + classes[i] = code === SQLite.SQLITE_NULL ? null : code; + } + + return classes; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/engine/FacadeVFS.d.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/FacadeVFS.d.ts new file mode 100644 index 00000000000..bddb7653ac6 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/FacadeVFS.d.ts @@ -0,0 +1,29 @@ +// Types for the vendored wa-sqlite FacadeVFS (engine/FacadeVFS.js, wa-sqlite v1.1.2): the base class that +// turns SQLite's C callbacks into JavaScript-friendly j* methods, which a storage provider's VFS +// overrides. The module is fetched and checksummed at restore, not compiled here. + +import { Base } from './VFS.js'; + +type Result = number | Promise; + +export declare class FacadeVFS extends Base { + constructor(name: string, module: unknown, options?: unknown); + getFilename(pFile: number): string; + jOpen(filename: string | null, pFile: number, flags: number, pOutFlags: DataView): Result; + jDelete(filename: string, syncDir: number): Result; + jAccess(filename: string, flags: number, pResOut: DataView): Result; + jFullPathname(filename: string, zOut: Uint8Array): Result; + jGetLastError(zBuf: Uint8Array): Result; + jClose(pFile: number): Result; + jRead(pFile: number, pData: Uint8Array, iOffset: number): Result; + jWrite(pFile: number, pData: Uint8Array, iOffset: number): Result; + jTruncate(pFile: number, size: number): Result; + jSync(pFile: number, flags: number): Result; + jFileSize(pFile: number, pSize: DataView): Result; + jLock(pFile: number, lockType: number): Result; + jUnlock(pFile: number, lockType: number): Result; + jCheckReservedLock(pFile: number, pResOut: DataView): Result; + jFileControl(pFile: number, op: number, pArg: DataView): Result; + jSectorSize(pFile: number): Result; + jDeviceCharacteristics(pFile: number): Result; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/engine/VFS.d.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/VFS.d.ts new file mode 100644 index 00000000000..ae1553d3217 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/VFS.d.ts @@ -0,0 +1,16 @@ +// Types for the vendored wa-sqlite VFS base class (engine/VFS.js, wa-sqlite v1.1.2), which every storage +// provider's VFS ultimately extends. The module is fetched and checksummed at restore, not compiled here. + +export * from './sqlite-constants.js'; + +/** The raw VFS interface: SQLite's C callbacks, with C primitives for arguments. */ +export declare class Base { + constructor(name: string, module: unknown); + name: string; + mxPathname: number; + close(): void | Promise; + isReady(): boolean | Promise; + hasAsyncMethod(methodName: string): boolean; +} + +export declare const FILE_TYPE_MASK: number; diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/engine/WebLocksMixin.d.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/WebLocksMixin.d.ts new file mode 100644 index 00000000000..03a33753146 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/WebLocksMixin.d.ts @@ -0,0 +1,16 @@ +// Types for the vendored wa-sqlite WebLocksMixin (engine/WebLocksMixin.js, wa-sqlite v1.1.2): the mix-in +// that implements SQLite's locking protocol over the Web Locks API, so tabs sharing a database +// coordinate. The module is fetched and checksummed at restore, not compiled here. + +import type { FacadeVFS } from './FacadeVFS.js'; + +export interface WebLocksOptions { + /** `'exclusive'` (the default), `'shared'` or `'shared+hint'`. */ + lockPolicy?: 'exclusive' | 'shared' | 'shared+hint'; + lockTimeout?: number; +} + +type FacadeConstructor = abstract new (name: string, module: unknown, options?: any) => FacadeVFS; + +export declare function WebLocksMixin(superclass: TBase): + TBase & (abstract new (name: string, module: unknown, options?: WebLocksOptions) => FacadeVFS); diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/engine/sqlite-api.d.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/sqlite-api.d.ts new file mode 100644 index 00000000000..3d408046636 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/sqlite-api.d.ts @@ -0,0 +1,83 @@ +// Types for the vendored wa-sqlite JavaScript API (engine/sqlite-api.js, wa-sqlite v1.1.2). The module +// itself is fetched and checksummed at restore (see engine/wa-sqlite.lock.props), not compiled here; +// these declarations only let the TypeScript modules in this folder import it, and they describe the +// members those modules call rather than the whole API. + +export * from './sqlite-constants.js'; + +/** A pointer into the engine's WebAssembly heap. */ +export type Pointer = number; + +/** The value of a column or a bound parameter as wa-sqlite reads and writes it. */ +export type SQLiteCompatibleType = number | string | Uint8Array | ArrayLike | bigint | null; + +/** Raised by every API call that fails, with the primary SQLite result code. */ +export declare class SQLiteError extends Error { + constructor(message: string, code: number); + code: number; +} + +/** The Emscripten module of an engine build. */ +export interface SQLiteModule { + HEAPU8: Uint8Array; + UTF8ToString(pointer: Pointer, maxBytesToRead?: number): string; + addFunction(fn: (...args: number[]) => number, signature: string): Pointer; + ccall(name: string, returnType: string | null, argTypes: string[], args: unknown[]): any; + cwrap(name: string, returnType: string | null, argTypes: string[]): (...args: any[]) => any; + getTempRet0(): number; + getValue(pointer: Pointer, type: string): number; + _sqlite3_malloc(size: number): Pointer; + _sqlite3_free(pointer: Pointer): void; + _sqlite3_aggregate_context(context: Pointer, size: number): Pointer; + [member: string]: any; +} + +/** The JavaScript API wa-sqlite builds over an engine module. */ +export interface SQLiteAPI { + libversion(): string; + open_v2(filename: string, flags?: number, vfsName?: string): Promise; + close(db: number): Promise; + changes(db: number): number; + get_autocommit(db: number): number; + update_hook(db: number, callback: (operation: number, schema: string | null, table: string | null, rowid: bigint) => void): void; + commit_hook(db: number, callback: (() => number) | null): void; + + statements(db: number, sql: string, options?: { unscoped?: boolean; flags?: number }): AsyncIterable; + step(stmt: number): Promise; + reset(stmt: number): Promise; + finalize(stmt: number): Promise; + clear_bindings(stmt: number): number; + + bind(stmt: number, index: number, value: SQLiteCompatibleType): number; + bind_parameter_count(stmt: number): number; + bind_parameter_name(stmt: number, index: number): string | null; + + column(stmt: number, index: number): SQLiteCompatibleType; + column_blob(stmt: number, index: number): Uint8Array; + column_count(stmt: number): number; + column_double(stmt: number, index: number): number; + column_int64(stmt: number, index: number): bigint; + column_names(stmt: number): string[]; + column_text(stmt: number, index: number): string; + column_type(stmt: number, index: number): number; + + create_function( + db: number, + name: string, + nArg: number, + eTextRep: number, + pApp: number, + xFunc?: ((context: Pointer, values: Uint32Array) => void) | null, + xStep?: ((context: Pointer, values: Uint32Array) => void) | null, + xFinal?: ((context: Pointer) => void) | null): number; + result(context: Pointer, value: SQLiteCompatibleType): void; + value_text(value: Pointer): string; + value_type(value: Pointer): number; + + vfs_register(vfs: unknown, makeDefault?: boolean): number; + + [member: string]: any; +} + +/** Wraps an engine module in the JavaScript API. */ +export declare function Factory(module: SQLiteModule): SQLiteAPI; diff --git a/src/Bsqlite/Bit.Bsqlite/Scripts/engine/sqlite-constants.d.ts b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/sqlite-constants.d.ts new file mode 100644 index 00000000000..38984ba7eba --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Scripts/engine/sqlite-constants.d.ts @@ -0,0 +1,236 @@ +// Types for the vendored wa-sqlite constants (engine/sqlite-constants.js, wa-sqlite v1.1.2). The module +// itself is fetched and checksummed at restore (see engine/wa-sqlite.lock.props), not compiled here; +// these declarations only let the TypeScript modules in this folder import it. + +export declare const SQLITE_OK: number; +export declare const SQLITE_ERROR: number; +export declare const SQLITE_INTERNAL: number; +export declare const SQLITE_PERM: number; +export declare const SQLITE_ABORT: number; +export declare const SQLITE_BUSY: number; +export declare const SQLITE_LOCKED: number; +export declare const SQLITE_NOMEM: number; +export declare const SQLITE_READONLY: number; +export declare const SQLITE_INTERRUPT: number; +export declare const SQLITE_IOERR: number; +export declare const SQLITE_CORRUPT: number; +export declare const SQLITE_NOTFOUND: number; +export declare const SQLITE_FULL: number; +export declare const SQLITE_CANTOPEN: number; +export declare const SQLITE_PROTOCOL: number; +export declare const SQLITE_EMPTY: number; +export declare const SQLITE_SCHEMA: number; +export declare const SQLITE_TOOBIG: number; +export declare const SQLITE_CONSTRAINT: number; +export declare const SQLITE_MISMATCH: number; +export declare const SQLITE_MISUSE: number; +export declare const SQLITE_NOLFS: number; +export declare const SQLITE_AUTH: number; +export declare const SQLITE_FORMAT: number; +export declare const SQLITE_RANGE: number; +export declare const SQLITE_NOTADB: number; +export declare const SQLITE_NOTICE: number; +export declare const SQLITE_WARNING: number; +export declare const SQLITE_ROW: number; +export declare const SQLITE_DONE: number; +export declare const SQLITE_IOERR_ACCESS: number; +export declare const SQLITE_IOERR_CHECKRESERVEDLOCK: number; +export declare const SQLITE_IOERR_CLOSE: number; +export declare const SQLITE_IOERR_DATA: number; +export declare const SQLITE_IOERR_DELETE: number; +export declare const SQLITE_IOERR_DELETE_NOENT: number; +export declare const SQLITE_IOERR_DIR_FSYNC: number; +export declare const SQLITE_IOERR_FSTAT: number; +export declare const SQLITE_IOERR_FSYNC: number; +export declare const SQLITE_IOERR_GETTEMPPATH: number; +export declare const SQLITE_IOERR_LOCK: number; +export declare const SQLITE_IOERR_NOMEM: number; +export declare const SQLITE_IOERR_READ: number; +export declare const SQLITE_IOERR_RDLOCK: number; +export declare const SQLITE_IOERR_SEEK: number; +export declare const SQLITE_IOERR_SHORT_READ: number; +export declare const SQLITE_IOERR_TRUNCATE: number; +export declare const SQLITE_IOERR_UNLOCK: number; +export declare const SQLITE_IOERR_VNODE: number; +export declare const SQLITE_IOERR_WRITE: number; +export declare const SQLITE_IOERR_BEGIN_ATOMIC: number; +export declare const SQLITE_IOERR_COMMIT_ATOMIC: number; +export declare const SQLITE_IOERR_ROLLBACK_ATOMIC: number; +export declare const SQLITE_CONSTRAINT_CHECK: number; +export declare const SQLITE_CONSTRAINT_COMMITHOOK: number; +export declare const SQLITE_CONSTRAINT_FOREIGNKEY: number; +export declare const SQLITE_CONSTRAINT_FUNCTION: number; +export declare const SQLITE_CONSTRAINT_NOTNULL: number; +export declare const SQLITE_CONSTRAINT_PINNED: number; +export declare const SQLITE_CONSTRAINT_PRIMARYKEY: number; +export declare const SQLITE_CONSTRAINT_ROWID: number; +export declare const SQLITE_CONSTRAINT_TRIGGER: number; +export declare const SQLITE_CONSTRAINT_UNIQUE: number; +export declare const SQLITE_CONSTRAINT_VTAB: number; +export declare const SQLITE_OPEN_READONLY: number; +export declare const SQLITE_OPEN_READWRITE: number; +export declare const SQLITE_OPEN_CREATE: number; +export declare const SQLITE_OPEN_DELETEONCLOSE: number; +export declare const SQLITE_OPEN_EXCLUSIVE: number; +export declare const SQLITE_OPEN_AUTOPROXY: number; +export declare const SQLITE_OPEN_URI: number; +export declare const SQLITE_OPEN_MEMORY: number; +export declare const SQLITE_OPEN_MAIN_DB: number; +export declare const SQLITE_OPEN_TEMP_DB: number; +export declare const SQLITE_OPEN_TRANSIENT_DB: number; +export declare const SQLITE_OPEN_MAIN_JOURNAL: number; +export declare const SQLITE_OPEN_TEMP_JOURNAL: number; +export declare const SQLITE_OPEN_SUBJOURNAL: number; +export declare const SQLITE_OPEN_SUPER_JOURNAL: number; +export declare const SQLITE_OPEN_NOMUTEX: number; +export declare const SQLITE_OPEN_FULLMUTEX: number; +export declare const SQLITE_OPEN_SHAREDCACHE: number; +export declare const SQLITE_OPEN_PRIVATECACHE: number; +export declare const SQLITE_OPEN_WAL: number; +export declare const SQLITE_OPEN_NOFOLLOW: number; +export declare const SQLITE_LOCK_NONE: number; +export declare const SQLITE_LOCK_SHARED: number; +export declare const SQLITE_LOCK_RESERVED: number; +export declare const SQLITE_LOCK_PENDING: number; +export declare const SQLITE_LOCK_EXCLUSIVE: number; +export declare const SQLITE_IOCAP_ATOMIC: number; +export declare const SQLITE_IOCAP_ATOMIC512: number; +export declare const SQLITE_IOCAP_ATOMIC1K: number; +export declare const SQLITE_IOCAP_ATOMIC2K: number; +export declare const SQLITE_IOCAP_ATOMIC4K: number; +export declare const SQLITE_IOCAP_ATOMIC8K: number; +export declare const SQLITE_IOCAP_ATOMIC16K: number; +export declare const SQLITE_IOCAP_ATOMIC32K: number; +export declare const SQLITE_IOCAP_ATOMIC64K: number; +export declare const SQLITE_IOCAP_SAFE_APPEND: number; +export declare const SQLITE_IOCAP_SEQUENTIAL: number; +export declare const SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN: number; +export declare const SQLITE_IOCAP_POWERSAFE_OVERWRITE: number; +export declare const SQLITE_IOCAP_IMMUTABLE: number; +export declare const SQLITE_IOCAP_BATCH_ATOMIC: number; +export declare const SQLITE_ACCESS_EXISTS: number; +export declare const SQLITE_ACCESS_READWRITE: number; +export declare const SQLITE_ACCESS_READ: number; +export declare const SQLITE_FCNTL_LOCKSTATE: number; +export declare const SQLITE_FCNTL_GET_LOCKPROXYFILE: number; +export declare const SQLITE_FCNTL_SET_LOCKPROXYFILE: number; +export declare const SQLITE_FCNTL_LAST_ERRNO: number; +export declare const SQLITE_FCNTL_SIZE_HINT: number; +export declare const SQLITE_FCNTL_CHUNK_SIZE: number; +export declare const SQLITE_FCNTL_FILE_POINTER: number; +export declare const SQLITE_FCNTL_SYNC_OMITTED: number; +export declare const SQLITE_FCNTL_WIN32_AV_RETRY: number; +export declare const SQLITE_FCNTL_PERSIST_WAL: number; +export declare const SQLITE_FCNTL_OVERWRITE: number; +export declare const SQLITE_FCNTL_VFSNAME: number; +export declare const SQLITE_FCNTL_POWERSAFE_OVERWRITE: number; +export declare const SQLITE_FCNTL_PRAGMA: number; +export declare const SQLITE_FCNTL_BUSYHANDLER: number; +export declare const SQLITE_FCNTL_TEMPFILENAME: number; +export declare const SQLITE_FCNTL_MMAP_SIZE: number; +export declare const SQLITE_FCNTL_TRACE: number; +export declare const SQLITE_FCNTL_HAS_MOVED: number; +export declare const SQLITE_FCNTL_SYNC: number; +export declare const SQLITE_FCNTL_COMMIT_PHASETWO: number; +export declare const SQLITE_FCNTL_WIN32_SET_HANDLE: number; +export declare const SQLITE_FCNTL_WAL_BLOCK: number; +export declare const SQLITE_FCNTL_ZIPVFS: number; +export declare const SQLITE_FCNTL_RBU: number; +export declare const SQLITE_FCNTL_VFS_POINTER: number; +export declare const SQLITE_FCNTL_JOURNAL_POINTER: number; +export declare const SQLITE_FCNTL_WIN32_GET_HANDLE: number; +export declare const SQLITE_FCNTL_PDB: number; +export declare const SQLITE_FCNTL_BEGIN_ATOMIC_WRITE: number; +export declare const SQLITE_FCNTL_COMMIT_ATOMIC_WRITE: number; +export declare const SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE: number; +export declare const SQLITE_FCNTL_LOCK_TIMEOUT: number; +export declare const SQLITE_FCNTL_DATA_VERSION: number; +export declare const SQLITE_FCNTL_SIZE_LIMIT: number; +export declare const SQLITE_FCNTL_CKPT_DONE: number; +export declare const SQLITE_FCNTL_RESERVE_BYTES: number; +export declare const SQLITE_FCNTL_CKPT_START: number; +export declare const SQLITE_INTEGER: number; +export declare const SQLITE_FLOAT: number; +export declare const SQLITE_TEXT: number; +export declare const SQLITE_BLOB: number; +export declare const SQLITE_NULL: number; +export declare const SQLITE_STATIC: number; +export declare const SQLITE_TRANSIENT: number; +export declare const SQLITE_UTF8: number; +export declare const SQLITE_UTF16LE: number; +export declare const SQLITE_UTF16BE: number; +export declare const SQLITE_UTF16: number; +export declare const SQLITE_INDEX_CONSTRAINT_EQ: number; +export declare const SQLITE_INDEX_CONSTRAINT_GT: number; +export declare const SQLITE_INDEX_CONSTRAINT_LE: number; +export declare const SQLITE_INDEX_CONSTRAINT_LT: number; +export declare const SQLITE_INDEX_CONSTRAINT_GE: number; +export declare const SQLITE_INDEX_CONSTRAINT_MATCH: number; +export declare const SQLITE_INDEX_CONSTRAINT_LIKE: number; +export declare const SQLITE_INDEX_CONSTRAINT_GLOB: number; +export declare const SQLITE_INDEX_CONSTRAINT_REGEXP: number; +export declare const SQLITE_INDEX_CONSTRAINT_NE: number; +export declare const SQLITE_INDEX_CONSTRAINT_ISNOT: number; +export declare const SQLITE_INDEX_CONSTRAINT_ISNOTNULL: number; +export declare const SQLITE_INDEX_CONSTRAINT_ISNULL: number; +export declare const SQLITE_INDEX_CONSTRAINT_IS: number; +export declare const SQLITE_INDEX_CONSTRAINT_FUNCTION: number; +export declare const SQLITE_INDEX_SCAN_UNIQUE: number; +export declare const SQLITE_DETERMINISTIC: number; +export declare const SQLITE_DIRECTONLY: number; +export declare const SQLITE_SUBTYPE: number; +export declare const SQLITE_INNOCUOUS: number; +export declare const SQLITE_SYNC_NORMAL: number; +export declare const SQLITE_SYNC_FULL: number; +export declare const SQLITE_SYNC_DATAONLY: number; +export declare const SQLITE_CREATE_INDEX: number; +export declare const SQLITE_CREATE_TABLE: number; +export declare const SQLITE_CREATE_TEMP_INDEX: number; +export declare const SQLITE_CREATE_TEMP_TABLE: number; +export declare const SQLITE_CREATE_TEMP_TRIGGER: number; +export declare const SQLITE_CREATE_TEMP_VIEW: number; +export declare const SQLITE_CREATE_TRIGGER: number; +export declare const SQLITE_CREATE_VIEW: number; +export declare const SQLITE_DELETE: number; +export declare const SQLITE_DROP_INDEX: number; +export declare const SQLITE_DROP_TABLE: number; +export declare const SQLITE_DROP_TEMP_INDEX: number; +export declare const SQLITE_DROP_TEMP_TABLE: number; +export declare const SQLITE_DROP_TEMP_TRIGGER: number; +export declare const SQLITE_DROP_TEMP_VIEW: number; +export declare const SQLITE_DROP_TRIGGER: number; +export declare const SQLITE_DROP_VIEW: number; +export declare const SQLITE_INSERT: number; +export declare const SQLITE_PRAGMA: number; +export declare const SQLITE_READ: number; +export declare const SQLITE_SELECT: number; +export declare const SQLITE_TRANSACTION: number; +export declare const SQLITE_UPDATE: number; +export declare const SQLITE_ATTACH: number; +export declare const SQLITE_DETACH: number; +export declare const SQLITE_ALTER_TABLE: number; +export declare const SQLITE_REINDEX: number; +export declare const SQLITE_ANALYZE: number; +export declare const SQLITE_CREATE_VTABLE: number; +export declare const SQLITE_DROP_VTABLE: number; +export declare const SQLITE_FUNCTION: number; +export declare const SQLITE_SAVEPOINT: number; +export declare const SQLITE_COPY: number; +export declare const SQLITE_RECURSIVE: number; +export declare const SQLITE_DENY: number; +export declare const SQLITE_IGNORE: number; +export declare const SQLITE_LIMIT_LENGTH: number; +export declare const SQLITE_LIMIT_SQL_LENGTH: number; +export declare const SQLITE_LIMIT_COLUMN: number; +export declare const SQLITE_LIMIT_EXPR_DEPTH: number; +export declare const SQLITE_LIMIT_COMPOUND_SELECT: number; +export declare const SQLITE_LIMIT_VDBE_OP: number; +export declare const SQLITE_LIMIT_FUNCTION_ARG: number; +export declare const SQLITE_LIMIT_ATTACHED: number; +export declare const SQLITE_LIMIT_LIKE_PATTERN_LENGTH: number; +export declare const SQLITE_LIMIT_VARIABLE_NUMBER: number; +export declare const SQLITE_LIMIT_TRIGGER_DEPTH: number; +export declare const SQLITE_LIMIT_WORKER_THREADS: number; +export declare const SQLITE_PREPARE_PERSISTENT: number; +export declare const SQLITE_PREPARE_NORMALIZED: number; +export declare const SQLITE_PREPARE_NO_VTAB: number; diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteEngineBuild.cs b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteEngineBuild.cs new file mode 100644 index 00000000000..d2d75f0e584 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteEngineBuild.cs @@ -0,0 +1,24 @@ +namespace Bit.Bsqlite.Storage; + +/// +/// The SQLite/WASM build a storage backend needs, because a VFS whose reads are asynchronous cannot +/// run on an engine compiled for synchronous I/O. +/// +public enum BsqliteEngineBuild +{ + /// + /// The engine may be the plain synchronous build. Only backends whose VFS can satisfy reads and + /// writes without awaiting qualify - in practice, OPFS synchronous access handles and in-memory. + /// + /// + /// Listed first so it is not the default value: declaring by accident + /// would load an engine that cannot drive the VFS at all. + /// + Synchronous, + + /// + /// The engine must be able to suspend inside a VFS call - JSPI where available, Asyncify + /// otherwise. Required by any backend built on an asynchronous browser API. + /// + AsyncCapable, +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteExecutionContexts.cs b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteExecutionContexts.cs new file mode 100644 index 00000000000..0812314ec7b --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteExecutionContexts.cs @@ -0,0 +1,24 @@ +namespace Bit.Bsqlite.Storage; + +/// +/// The browser contexts a storage backend can operate in. Backends differ here: OPFS synchronous +/// access handles exist only inside workers, while IndexedDB is available everywhere. +/// +[Flags] +public enum BsqliteExecutionContexts +{ + /// No context - a backend declaring this can never be selected. + None = 0, + + /// The main document thread. + Window = 1 << 0, + + /// A dedicated worker, which is where Bsqlite runs the engine. + DedicatedWorker = 1 << 1, + + /// A shared worker. + SharedWorker = 1 << 2, + + /// A service worker. + ServiceWorker = 1 << 3, +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteJsModule.cs b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteJsModule.cs new file mode 100644 index 00000000000..350521730d2 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteJsModule.cs @@ -0,0 +1,33 @@ +namespace Bit.Bsqlite.Storage; + +/// +/// Locates the JavaScript half of a storage backend: an ES module the worker imports dynamically, +/// exporting a function that registers the VFS with the engine. +/// +/// +/// A backend is a two-sided artifact. The .NET side declares capabilities and answers probes; the +/// VFS itself is JavaScript, served from the provider package's own static assets, which is what +/// keeps VFS code out of the core. +/// +public sealed record BsqliteJsModule +{ + /// + /// URL of the ES module, normally an RCL static asset path such as + /// ./_content/Bit.Bsqlite.Storage.Opfs/bit-bsqlite-opfs-vfs.js. + /// + /// Name of the exported registration function. + public BsqliteJsModule(string moduleUrl, string registerExport = "register") + { + ArgumentException.ThrowIfNullOrWhiteSpace(moduleUrl); + ArgumentException.ThrowIfNullOrWhiteSpace(registerExport); + + ModuleUrl = moduleUrl; + RegisterExport = registerExport; + } + + /// URL the worker imports. + public string ModuleUrl { get; } + + /// The module export that registers the VFS. + public string RegisterExport { get; } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteProbeResult.cs b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteProbeResult.cs new file mode 100644 index 00000000000..3cbbdb7be5c --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteProbeResult.cs @@ -0,0 +1,80 @@ +namespace Bit.Bsqlite.Storage; + +/// +/// What a backend discovered about the current browser. Probing happens once per session and the +/// result is surfaced verbatim through diagnostics, so that a support ticket can be answered from +/// the report alone. +/// +/// +/// Constructed through and so that an unavailable +/// result always carries a reason - a probe that fails without saying why is the thing that makes +/// these bugs unanswerable. +/// +public sealed record BsqliteProbeResult +{ + private BsqliteProbeResult() + { + } + + /// Whether the backend can be used here. + public bool IsAvailable { get; private init; } + + /// + /// Why the backend is unusable. Always present when is false, and + /// always absent when it is true. + /// + public string? UnavailableReason { get; private init; } + + /// Storage the origin may still use, when the browser will say. + public long? QuotaBytes { get; private init; } + + /// Storage the origin has already used, when the browser will say. + public long? UsageBytes { get; private init; } + + /// + /// The environment facts the verdict rests on - for example crossOriginIsolated, + /// navigator.storage.getDirectory, or JSPI support. Reported as observed, without + /// interpretation. + /// + public IReadOnlyDictionary Environment { get; private init; } + = new Dictionary(StringComparer.Ordinal); + + /// Reports a usable backend. + public static BsqliteProbeResult Available( + long? quotaBytes = null, + long? usageBytes = null, + IReadOnlyDictionary? environment = null) + => new() + { + IsAvailable = true, + QuotaBytes = quotaBytes, + UsageBytes = usageBytes, + Environment = Freeze(environment), + }; + + /// Reports an unusable backend and why. + /// + /// Written for whoever reads the error, naming the missing capability rather than restating that + /// something failed. + /// + /// The facts behind the verdict. + public static BsqliteProbeResult Unavailable( + string reason, + IReadOnlyDictionary? environment = null) + { + ArgumentException.ThrowIfNullOrWhiteSpace(reason); + + return new BsqliteProbeResult + { + IsAvailable = false, + UnavailableReason = reason, + Environment = Freeze(environment), + }; + } + + private static IReadOnlyDictionary Freeze( + IReadOnlyDictionary? environment) + => environment is null + ? new Dictionary(StringComparer.Ordinal) + : new Dictionary(environment, StringComparer.Ordinal); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteStorageCapabilities.cs b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteStorageCapabilities.cs new file mode 100644 index 00000000000..9b44c0332f0 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/BsqliteStorageCapabilities.cs @@ -0,0 +1,60 @@ +namespace Bit.Bsqlite.Storage; + +/// +/// What a storage backend can and cannot do. The core enforces every one of these generically, so +/// that adding a backend never requires a change in the core. +/// +/// +/// +/// Declared with init properties rather than a positional record on purpose: this type is part of a +/// semver-stable contract, and a new positional parameter would break every existing provider, +/// whereas a new optional property does not. +/// +/// +/// Defaults are the *least* capable answer, so a provider that omits a property loses a feature +/// rather than making a promise it cannot keep. Only the two facts that cannot be guessed safely - +/// which engine build the VFS needs, and whether data survives a reload - are required. +/// +/// +public sealed record BsqliteStorageCapabilities +{ + /// The engine build this backend's VFS requires. + public required BsqliteEngineBuild RequiredBuild { get; init; } + + /// + /// Whether data survives a page reload. Guarded because falling back to a non-persistent + /// backend silently is a data-loss bug, so selection demands a separate opt-in for it. + /// + public required bool IsPersistent { get; init; } + + /// Whether more than one connection may be open against one database. + public bool SupportsMultipleConnections { get; init; } + + /// Whether readers can proceed concurrently rather than being serialized. + public bool SupportsConcurrentReads { get; init; } + + /// + /// Whether the backend can offer a durability level below full flush-on-commit. Relaxed + /// durability is only offered to callers when this is true. + /// + public bool SupportsRelaxedDurability { get; init; } + + /// + /// Whether a transaction may span ATTACHed databases. When false the core rejects + /// cross-database transactions instead of letting them corrupt silently. + /// + public bool SupportsMultiDatabaseTransactions { get; init; } + + /// + /// Whether PRAGMA page_size may be changed. Block-oriented backends fix the page size to + /// their block size, and the core guards the pragma for any backend answering false. + /// + public bool CanChangePageSize { get; init; } + + /// + /// Contexts this backend works in. Defaults to , + /// the only context in which v1 hosts the engine. + /// + public BsqliteExecutionContexts SupportedContexts { get; init; } + = BsqliteExecutionContexts.DedicatedWorker; +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/IBsqliteStorageAdmin.cs b/src/Bsqlite/Bit.Bsqlite/Storage/IBsqliteStorageAdmin.cs new file mode 100644 index 00000000000..bfd13bf1d72 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/IBsqliteStorageAdmin.cs @@ -0,0 +1,42 @@ +namespace Bit.Bsqlite.Storage; + +/// +/// Operations on databases as whole artifacts, outside any SQL connection: what exists, deleting, +/// and moving bytes in and out. +/// +/// +/// Separated from the provider itself because these are the operations cross-provider migration and +/// the diagnostics surface are built from, and because a backend can implement them without the +/// engine running. +/// +public interface IBsqliteStorageAdmin +{ + /// Whether exists in this backend. + ValueTask ExistsAsync(string databaseName, CancellationToken cancellationToken = default); + + /// Names of every database this backend holds. Used by diagnostics and migration. + ValueTask> ListAsync(CancellationToken cancellationToken = default); + + /// + /// Deletes , succeeding quietly when it is already absent. + /// + ValueTask DeleteAsync(string databaseName, CancellationToken cancellationToken = default); + + /// + /// Reads the database out as a SQLite file image, for backup or for the copy step of a + /// cross-provider migration. + /// + ValueTask ExportAsync(string databaseName, CancellationToken cancellationToken = default); + + /// + /// Writes a SQLite file image in, replacing any existing database of that name. + /// + /// + /// Callers are expected to verify the result - the migration protocol runs + /// PRAGMA integrity_check against the copy before it trusts it. + /// + ValueTask ImportAsync( + string databaseName, + ReadOnlyMemory contents, + CancellationToken cancellationToken = default); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/IBsqliteStorageProvider.cs b/src/Bsqlite/Bit.Bsqlite/Storage/IBsqliteStorageProvider.cs new file mode 100644 index 00000000000..56b66367a2d --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/IBsqliteStorageProvider.cs @@ -0,0 +1,56 @@ +namespace Bit.Bsqlite.Storage; + +/// +/// A storage backend for Bsqlite. Implement this to persist SQLite databases somewhere the +/// first-party providers do not reach - an encrypted store, a read-only HTTP range server, or a +/// browser API that does not exist yet. +/// +/// +/// +/// This contract is public and semver-stable from 1.0, and the core consumes nothing else: adding a +/// backend requires no change to Bsqlite itself. Every implementation, first- or third-party, +/// must pass the conformance kit shipped in Bit.Bsqlite.Testing, which is what makes the declared +/// trustworthy. +/// +/// +/// Implementations are resolved from dependency injection and are expected to be safe to share, since +/// one provider instance serves every connection to every database it holds. +/// +/// +public interface IBsqliteStorageProvider +{ + /// + /// Stable identifier used to select this backend, such as opfs or indexeddb. + /// + /// + /// Compared case-insensitively, and recorded as the sticky binding of every database this backend + /// creates - so renaming it after release orphans existing data. Third parties should qualify + /// their names to avoid colliding with a future first-party backend. + /// + string Name { get; } + + /// + /// What this backend can do. Read once and treated as constant, so it must not depend on the + /// outcome of . + /// + BsqliteStorageCapabilities Capabilities { get; } + + /// + /// The ES module registering this backend's VFS, or when it needs no + /// JavaScript because its storage lives in the engine's own memory. + /// + BsqliteJsModule? VfsModule { get; } + + /// Database-level operations, used by diagnostics and cross-provider migration. + IBsqliteStorageAdmin Admin { get; } + + /// + /// Reports whether this backend works in the current browser, and the facts behind the verdict. + /// + /// + /// Called at most once per session by the core, which caches the result. Implementations should + /// report unavailability through rather than by + /// throwing, so that selection can move on to the next candidate and still explain itself. + /// + ValueTask ProbeAsync(CancellationToken cancellationToken = default); +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/InMemory/BsqliteInMemoryStorageAdmin.cs b/src/Bsqlite/Bit.Bsqlite/Storage/InMemory/BsqliteInMemoryStorageAdmin.cs new file mode 100644 index 00000000000..04fd943b915 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/InMemory/BsqliteInMemoryStorageAdmin.cs @@ -0,0 +1,64 @@ +using System.Collections.Concurrent; +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Storage.InMemory; + +/// +/// Tracks the in-memory backend's databases and their file images. +/// +/// +/// Export and import are real against this store, but the store is not the engine's. The backend +/// registers no VFS, so a database opened through a connection lives in SQLite's built-in memory +/// VFS inside the worker and never appears here. See +/// . +/// +internal sealed class BsqliteInMemoryStorageAdmin : IBsqliteStorageAdmin +{ + private readonly ConcurrentDictionary _databases = new(StringComparer.Ordinal); + + public ValueTask ExistsAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + return ValueTask.FromResult(_databases.ContainsKey(databaseName)); + } + + public ValueTask> ListAsync(CancellationToken cancellationToken = default) + => ValueTask.FromResult>([.. _databases.Keys.Order(StringComparer.Ordinal)]); + + public ValueTask DeleteAsync(string databaseName, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + _databases.TryRemove(databaseName, out _); + return ValueTask.CompletedTask; + } + + public ValueTask ExportAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + if (!_databases.TryGetValue(databaseName, out var image)) + { + throw new FileNotFoundException( + $"In-memory storage holds no database named '{databaseName}'.", + databaseName); + } + + // Copied so a caller cannot mutate the stored image through the array it was handed. + return ValueTask.FromResult(image.ToArray()); + } + + public ValueTask ImportAsync( + string databaseName, + ReadOnlyMemory contents, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + + _databases[databaseName] = contents.ToArray(); + return ValueTask.CompletedTask; + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/Storage/InMemory/BsqliteInMemoryStorageProvider.cs b/src/Bsqlite/Bit.Bsqlite/Storage/InMemory/BsqliteInMemoryStorageProvider.cs new file mode 100644 index 00000000000..e3d4dc706a1 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/Storage/InMemory/BsqliteInMemoryStorageProvider.cs @@ -0,0 +1,87 @@ +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Storage.InMemory; + +/// +/// Keeps databases in the engine's own memory. Available everywhere, survives nothing. +/// +/// +/// +/// Useful for tests, for demos, and as a declared last resort when persistence genuinely cannot be +/// had. Because it loses data on reload, selection refuses to fall back to it unless the application +/// opts in explicitly. +/// +/// +/// It is also the reference implementation of the contract: the smallest thing that passes the +/// conformance kit, and the one to read before writing a backend. +/// +/// +/// Its is a store of its own, not a window onto the running engine. This +/// backend declares no , so a database opened on it lives in SQLite's +/// built-in memory VFS inside the worker, where nothing outside the worker can reach it. Import, +/// export, exists, and list therefore describe images handed to the admin API and nothing else - a +/// database written through a connection does not appear here, and migrating away from this +/// backend copies nothing. That is a consequence of the storage being the engine's own heap, and +/// the reason this backend is for tests and demos rather than for data anyone wants to keep. +/// +/// +public sealed class BsqliteInMemoryStorageProvider : IBsqliteStorageProvider +{ + /// The name applications use to select this backend. + public const string ProviderName = "in-memory"; + + private readonly BsqliteInMemoryStorageAdmin _admin = new(); + + /// Creates the default in-memory backend, or a named instance for migration tests. + public BsqliteInMemoryStorageProvider(string? name = null) + { + Name = string.IsNullOrWhiteSpace(name) ? ProviderName : name; + } + + /// + public string Name { get; } + + /// + /// + /// Needs no asynchronous VFS, since memory reads never suspend, which is also why it is the only + /// backend that can run on the plain synchronous engine build unconditionally. + /// + public BsqliteStorageCapabilities Capabilities { get; } = new() + { + RequiredBuild = BsqliteEngineBuild.Synchronous, + IsPersistent = false, + SupportsMultiDatabaseTransactions = true, + CanChangePageSize = true, + + // Every connection is its own worker with its own heap, so a second connection to the + // "same" in-memory database is a different, empty database. Claiming otherwise would let a + // second session silently read nothing where the first wrote everything. + SupportsMultipleConnections = false, + SupportsConcurrentReads = false, + + // Durability is meaningless without persistence: there is nothing to flush to. + SupportsRelaxedDurability = false, + + SupportedContexts = BsqliteExecutionContexts.Window + | BsqliteExecutionContexts.DedicatedWorker + | BsqliteExecutionContexts.SharedWorker + | BsqliteExecutionContexts.ServiceWorker, + }; + + /// + /// None: SQLite's built-in memory VFS does the work, so there is no JavaScript to load. + public BsqliteJsModule? VfsModule => null; + + /// + public IBsqliteStorageAdmin Admin => _admin; + + /// + /// Always available - there is no browser capability to depend on. + public ValueTask ProbeAsync(CancellationToken cancellationToken = default) + => ValueTask.FromResult(BsqliteProbeResult.Available( + environment: new Dictionary(StringComparer.Ordinal) + { + ["persistent"] = "false", + ["requiredEngineBuild"] = nameof(BsqliteEngineBuild.Synchronous), + })); +} diff --git a/src/Bsqlite/Bit.Bsqlite/engine/FetchEngine.targets b/src/Bsqlite/Bit.Bsqlite/engine/FetchEngine.targets new file mode 100644 index 00000000000..85cd63c3b0a --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/engine/FetchEngine.targets @@ -0,0 +1,62 @@ + + + + + + $(MSBuildProjectDirectory)/wwwroot/engine/ + + + + + <_WaSqliteMissing Include="@(WaSqliteArtifact)" + Condition="!Exists('$(WaSqliteEngineDirectory)%(Identity)')" /> + + + + + + + + + + <_WaSqliteFile Include="$(WaSqliteEngineDirectory)%(WaSqliteArtifact.Identity)"> + %(WaSqliteArtifact.Sha256) + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite/engine/wa-sqlite.lock.props b/src/Bsqlite/Bit.Bsqlite/engine/wa-sqlite.lock.props new file mode 100644 index 00000000000..8d5e7bccef4 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/engine/wa-sqlite.lock.props @@ -0,0 +1,64 @@ + + + + + + v1.1.2 + https://raw.githubusercontent.com/rhashimoto/wa-sqlite + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Bit.Bsqlite/package-lock.json b/src/Bsqlite/Bit.Bsqlite/package-lock.json new file mode 100644 index 00000000000..0798ad57358 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/package-lock.json @@ -0,0 +1,511 @@ +{ + "name": "Bit.Bsqlite", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz", + "integrity": "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.2.tgz", + "integrity": "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.2.tgz", + "integrity": "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.2.tgz", + "integrity": "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.2.tgz", + "integrity": "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.2.tgz", + "integrity": "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.2.tgz", + "integrity": "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.2.tgz", + "integrity": "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.2.tgz", + "integrity": "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.2.tgz", + "integrity": "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.2.tgz", + "integrity": "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.2.tgz", + "integrity": "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.2.tgz", + "integrity": "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.2.tgz", + "integrity": "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.2.tgz", + "integrity": "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.2.tgz", + "integrity": "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.2.tgz", + "integrity": "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.2.tgz", + "integrity": "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.2.tgz", + "integrity": "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.2.tgz", + "integrity": "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.2.tgz", + "integrity": "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.2.tgz", + "integrity": "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.2.tgz", + "integrity": "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.2.tgz", + "integrity": "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.2.tgz", + "integrity": "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.2.tgz", + "integrity": "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/esbuild": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.2.tgz", + "integrity": "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.2", + "@esbuild/android-arm": "0.28.2", + "@esbuild/android-arm64": "0.28.2", + "@esbuild/android-x64": "0.28.2", + "@esbuild/darwin-arm64": "0.28.2", + "@esbuild/darwin-x64": "0.28.2", + "@esbuild/freebsd-arm64": "0.28.2", + "@esbuild/freebsd-x64": "0.28.2", + "@esbuild/linux-arm": "0.28.2", + "@esbuild/linux-arm64": "0.28.2", + "@esbuild/linux-ia32": "0.28.2", + "@esbuild/linux-loong64": "0.28.2", + "@esbuild/linux-mips64el": "0.28.2", + "@esbuild/linux-ppc64": "0.28.2", + "@esbuild/linux-riscv64": "0.28.2", + "@esbuild/linux-s390x": "0.28.2", + "@esbuild/linux-x64": "0.28.2", + "@esbuild/netbsd-arm64": "0.28.2", + "@esbuild/netbsd-x64": "0.28.2", + "@esbuild/openbsd-arm64": "0.28.2", + "@esbuild/openbsd-x64": "0.28.2", + "@esbuild/openharmony-arm64": "0.28.2", + "@esbuild/sunos-x64": "0.28.2", + "@esbuild/win32-arm64": "0.28.2", + "@esbuild/win32-ia32": "0.28.2", + "@esbuild/win32-x64": "0.28.2" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + } + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/package.json b/src/Bsqlite/Bit.Bsqlite/package.json new file mode 100644 index 00000000000..48e65da82f6 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/package.json @@ -0,0 +1,6 @@ +{ + "devDependencies": { + "esbuild": "0.28.2", + "typescript": "5.9.3" + } +} diff --git a/src/Bsqlite/Bit.Bsqlite/tsconfig.json b/src/Bsqlite/Bit.Bsqlite/tsconfig.json new file mode 100644 index 00000000000..227e3490fc5 --- /dev/null +++ b/src/Bsqlite/Bit.Bsqlite/tsconfig.json @@ -0,0 +1,23 @@ +{ + "compileOnSave": true, + "compilerOptions": { + // ES2022 for top-level await and native #private fields: the host and worker modules await their + // versioned imports at load, and every browser that can run the worker (module workers, OPFS, Web + // Locks) already runs ES2022. + "target": "ES2022", + // Every module is loaded through `import()` - by the interop, by the host, by the worker and by the + // storage packages - and its functions are called as named exports, so the output MUST stay an ES + // module: `module: ESNext` with `rootDir`/`outDir` emits each Scripts/*.ts straight to wwwroot/*.js + // with its `export`s and its relative `import()` specifiers intact. + "module": "ESNext", + "moduleResolution": "Bundler", + "lib": [ "DOM", "DOM.Iterable", "ESNext" ], + "rootDir": "Scripts", + "outDir": "wwwroot", + "strict": true, + "noFallthroughCasesInSwitch": true, + "noImplicitReturns": true, + "forceConsistentCasingInFileNames": true + }, + "include": [ "./Scripts/**/*.ts" ] +} diff --git a/src/Bsqlite/Demo/Client/Bit.Bsqlite.Demo.Client.csproj b/src/Bsqlite/Demo/Client/Bit.Bsqlite.Demo.Client.csproj new file mode 100644 index 00000000000..d030e5d1eaf --- /dev/null +++ b/src/Bsqlite/Demo/Client/Bit.Bsqlite.Demo.Client.csproj @@ -0,0 +1,31 @@ + + + + net10.0 + enable + enable + true + + true + + Default + true + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Demo/Client/BrowserDatabase.cs b/src/Bsqlite/Demo/Client/BrowserDatabase.cs new file mode 100644 index 00000000000..87b1f7a7fff --- /dev/null +++ b/src/Bsqlite/Demo/Client/BrowserDatabase.cs @@ -0,0 +1,133 @@ +using Bit.Bsqlite.Demo.Data; +using Bit.Bsqlite; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.EntityFrameworkCore; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Demo.Client; + +public enum StorageSwitchMode +{ + /// Copy the SQLite image, flip the sticky binding, delete the source. + Migrate, + + /// Leave the old file (if any) and open an empty database on the target backend. + Fresh, +} + +/// +/// The sample's view of the library's shared : hands out contexts +/// and drives the storage-switching UI. Everything that is not UI plumbing lives in the library. +/// +public sealed class BrowserDatabase( + BsqliteDatabase database, + IBsqliteDbContextFactory contexts, + IEnumerable providers, + IBsqliteStorageBindingStore bindings, + BsqliteStorageMigrator migrator) +{ + public const string DatabaseName = "app.db"; + + public BsqliteDatabase Database { get; } = database; + + public IReadOnlyList Providers { get; } = [.. providers]; + + public IBsqliteStorageBindingStore Bindings { get; } = bindings; + + public BsqliteSession? Session => Database.Session; + + public BsqliteStorageResolution? Resolution => Database.Resolution; + + public BsqliteConnection? Connection => Database.Connection; + + /// Raised before the worker is torn down so pages can drop live queries and contexts. + public event Func? BeforeSessionReset + { + add => Database.Closing += value; + remove => Database.Closing -= value; + } + + /// Raised after a session opens so pages can (re)load. + public event Func? AfterSessionReset + { + add => Database.Opened += value; + remove => Database.Opened -= value; + } + + public async Task EnsureSessionAsync(CancellationToken cancellationToken = default) + => await Database.OpenAsync(cancellationToken); + + public Task OpenContextAsync(CancellationToken cancellationToken = default) + => contexts.CreateDbContextAsync(cancellationToken); + + /// + /// Moves or recreates on . + /// + public async Task SwitchStorageAsync( + string targetProviderName, + StorageSwitchMode mode, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(targetProviderName); + + var target = Providers.FirstOrDefault(p => + string.Equals(p.Name, targetProviderName, StringComparison.OrdinalIgnoreCase)) + ?? throw new InvalidOperationException($"No registered provider named '{targetProviderName}'."); + + var source = Resolution?.Provider ?? await ProviderFromBindingAsync(cancellationToken); + + await Database.WhileClosedAsync(async ct => + { + if (mode is StorageSwitchMode.Migrate + && source is not null + && !string.Equals(source.Name, target.Name, StringComparison.OrdinalIgnoreCase) + && await source.Admin.ExistsAsync(DatabaseName, ct)) + { + await migrator.MigrateAsync(DatabaseName, source, target, Bindings, ct); + } + else + { + await target.Admin.DeleteAsync(DatabaseName, ct); + await Bindings.SetProviderNameAsync(DatabaseName, target.Name, ct); + } + }, cancellationToken); + + await Database.OpenAsync(cancellationToken); + } + + /// + /// Tears the worker down so admin import/delete can replace the file without racing an open + /// connection. The next opens a fresh session. + /// + public Task CloseSessionAsync() => Database.CloseAsync(); + + public async Task ExecuteScalarAsync( + string sql, + CancellationToken cancellationToken = default) + { + var session = await EnsureSessionAsync(cancellationToken); + await using var command = session.Connection.CreateCommand(); + command.CommandText = sql; + return await command.ExecuteScalarAsync(cancellationToken); + } + + public async Task ExecuteNonQueryAsync( + string sql, + CancellationToken cancellationToken = default) + { + var session = await EnsureSessionAsync(cancellationToken); + await using var command = session.Connection.CreateCommand(); + command.CommandText = sql; + await command.ExecuteNonQueryAsync(cancellationToken); + } + + private async Task ProviderFromBindingAsync( + CancellationToken cancellationToken) + { + var name = await Bindings.GetProviderNameAsync(DatabaseName, cancellationToken); + return name is null + ? null + : Providers.FirstOrDefault(p => string.Equals(p.Name, name, StringComparison.OrdinalIgnoreCase)); + } +} diff --git a/src/Bsqlite/Demo/Client/CodeHighlighter.cs b/src/Bsqlite/Demo/Client/CodeHighlighter.cs new file mode 100644 index 00000000000..5d8f0925120 --- /dev/null +++ b/src/Bsqlite/Demo/Client/CodeHighlighter.cs @@ -0,0 +1,115 @@ +using System.Net; +using System.Text; +using System.Text.RegularExpressions; +using Microsoft.AspNetCore.Components; + +namespace Bit.Bsqlite.Demo.Client; + +/// +/// Token colouring for the snippets on the demo pages. Deliberately tiny and dependency-free: +/// the sample must not pull a highlighter down over the wire just to show ten lines of C#. +/// +internal static partial class CodeHighlighter +{ + public static MarkupString Highlight(string code, string language) + { + var normalized = code.Replace("\r\n", "\n").TrimEnd(); + var pattern = PatternFor(language); + if (pattern is null) + { + return new MarkupString(WebUtility.HtmlEncode(normalized)); + } + + var builder = new StringBuilder(normalized.Length + 256); + var cursor = 0; + + foreach (Match match in pattern.Matches(normalized)) + { + if (match.Index > cursor) + { + builder.Append(WebUtility.HtmlEncode(normalized[cursor..match.Index])); + } + + var css = CssClassFor(match); + if (css is null) + { + builder.Append(WebUtility.HtmlEncode(match.Value)); + } + else + { + builder.Append("") + .Append(WebUtility.HtmlEncode(match.Value)) + .Append(""); + } + + cursor = match.Index + match.Length; + } + + if (cursor < normalized.Length) + { + builder.Append(WebUtility.HtmlEncode(normalized[cursor..])); + } + + return new MarkupString(builder.ToString()); + } + + private static string? CssClassFor(Match match) + { + if (match.Groups["com"].Success) return "tok-com"; + if (match.Groups["str"].Success) return "tok-str"; + if (match.Groups["key"].Success) return "tok-key"; + if (match.Groups["num"].Success) return "tok-num"; + if (match.Groups["typ"].Success) return "tok-typ"; + return null; + } + + private static Regex? PatternFor(string language) => language.ToLowerInvariant() switch + { + "csharp" or "c#" or "cs" or "razor" => CSharpPattern(), + "sql" => SqlPattern(), + "xml" or "csproj" or "html" => XmlPattern(), + "shell" or "bash" or "console" => ShellPattern(), + _ => null, + }; + + // Comments, strings (verbatim, interpolated, char), keywords, numbers, then PascalCase names. + [GeneratedRegex( + """ + (?//[^\n]*|/\*.*?\*/) + |(?@?\$?"(?:[^"\\]|\\.|"")*"|'(?:[^'\\]|\\.)') + |(?\b(?:abstract|as|async|await|base|bool|break|byte|case|catch|char|checked|class|const|continue|decimal|default|delegate|do|double|else|enum|event|explicit|extern|false|finally|fixed|float|for|foreach|get|goto|if|implicit|in|init|int|interface|internal|is|lock|long|namespace|new|nameof|not|null|object|operator|out|override|params|private|protected|public|readonly|record|ref|return|sbyte|sealed|set|short|sizeof|stackalloc|static|string|struct|switch|this|throw|true|try|typeof|uint|ulong|unchecked|unsafe|ushort|using|var|virtual|void|volatile|when|where|while|with|yield)\b) + |(?\b\d[\d_]*(?:\.\d+)?[mfdulUL]*\b) + |(?\b[A-Z][A-Za-z0-9_]*\b) + """, + RegexOptions.Singleline | RegexOptions.IgnorePatternWhitespace)] + private static partial Regex CSharpPattern(); + + [GeneratedRegex( + """ + (?--[^\n]*) + |(?'(?:[^']|'')*') + |(?"(?:[^"]|"")*") + |(?\b(?:ADD|ALL|ALTER|AND|AS|ASC|ATTACH|BEGIN|BETWEEN|BY|CASE|CAST|COLLATE|COMMIT|CREATE|CROSS|DATABASE|DEFAULT|DELETE|DESC|DETACH|DISTINCT|DROP|ELSE|END|EXISTS|FROM|FULL|GROUP|HAVING|IN|INDEX|INNER|INSERT|INTO|IS|JOIN|LEFT|LIKE|LIMIT|NOT|NULL|OFFSET|ON|OR|ORDER|OUTER|PRAGMA|REGEXP|RELEASE|RETURNING|RIGHT|ROLLBACK|SAVEPOINT|SELECT|SET|TABLE|THEN|TRANSACTION|UNION|UPDATE|USING|VALUES|VIEW|WHEN|WHERE|WITH)\b) + |(?\b\d+(?:\.\d+)?\b) + """, + RegexOptions.IgnoreCase | RegexOptions.IgnorePatternWhitespace)] + private static partial Regex SqlPattern(); + + [GeneratedRegex( + """ + (?) + |(?"[^"]*") + |(?) + """, + RegexOptions.Singleline | RegexOptions.IgnorePatternWhitespace)] + private static partial Regex XmlPattern(); + + [GeneratedRegex( + """ + (?\#[^\n]*) + |(?"[^"\n]*"|'[^'\n]*') + |(?^\s*(?:dotnet|npm|pnpm|git|cd)\b) + """, + RegexOptions.Multiline | RegexOptions.IgnorePatternWhitespace)] + private static partial Regex ShellPattern(); +} diff --git a/src/Bsqlite/Demo/Client/Components/AppNav.razor b/src/Bsqlite/Demo/Client/Components/AppNav.razor new file mode 100644 index 00000000000..569f697b1e2 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/AppNav.razor @@ -0,0 +1,86 @@ +@inject NavigationManager Nav +@implements IDisposable + +@* The documentation sidebar. Blazor's own NavLink only toggles a CSS class; a screen reader needs + aria-current="page" to announce which entry is the one you are on, so the active state is + computed here and drives both. *@ + + + +@code { + /// Drives the off-canvas drawer under 1024px. Ignored by the desktop layout. + [Parameter] + public bool Open { get; set; } + + /// Raised on every in-site link so the layout can close that drawer. + [Parameter] + public EventCallback OnNavigate { get; set; } + + protected override void OnInitialized() + => Nav.LocationChanged += OnLocationChanged; + + private void OnLocationChanged(object? sender, Microsoft.AspNetCore.Components.Routing.LocationChangedEventArgs e) + => StateHasChanged(); + + private bool IsCurrent(string href) + { + var path = Nav.ToBaseRelativePath(Nav.Uri); + var cut = path.IndexOfAny(['?', '#']); + if (cut >= 0) + { + path = path[..cut]; + } + + return string.Equals(path.Trim('/'), href.Trim('/'), StringComparison.OrdinalIgnoreCase); + } + + public void Dispose() + => Nav.LocationChanged -= OnLocationChanged; +} diff --git a/src/Bsqlite/Demo/Client/Components/CodeBlock.razor b/src/Bsqlite/Demo/Client/Components/CodeBlock.razor new file mode 100644 index 00000000000..e71548a3d95 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/CodeBlock.razor @@ -0,0 +1,70 @@ +@inject IJSRuntime Js + +@* Shows the code that powers the demo next to the demo itself. *@ +
+
+ @Language + @Title + + @if (Collapsible) + { + + } + +
+ @if (_open) + { + 
@CodeHighlighter.Highlight(Code, Language)
+ } +
+ +@code { + private bool _open; + private bool _copied; + + /// File name or label shown in the header, e.g. Pages/Catalog.razor. + [Parameter, EditorRequired] + public string Title { get; set; } = string.Empty; + + [Parameter] + public string Language { get; set; } = "csharp"; + + [Parameter, EditorRequired] + public string Code { get; set; } = string.Empty; + + /// Lets the reader fold long listings away. The block still starts open. + [Parameter] + public bool Collapsible { get; set; } = true; + + /// Set false for long reference listings that should start folded. + [Parameter] + public bool Open { get; set; } = true; + + protected override void OnInitialized() => _open = Open; + + private async Task CopyAsync() + { + try + { + _copied = await Js.InvokeAsync("bsqliteDemo.copy", Code); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _copied = false; + } + + if (_copied) + { + // Blazor re-renders around the await, so the label flips back on its own. + await Task.Delay(1400); + _copied = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Components/DemoCard.razor b/src/Bsqlite/Demo/Client/Components/DemoCard.razor new file mode 100644 index 00000000000..aac2609a89c --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/DemoCard.razor @@ -0,0 +1,61 @@ +@* A demo section: what it shows, what to try, the live thing, then the code behind it. *@ +
+
+ @if (!string.IsNullOrEmpty(Step)) + { + @Step + } +

@Title

+
+ @if (Description is not null) + { +

@Description

+ } + @if (Try is not null) + { +

Try @Try

+ } + @ChildContent + @if (!string.IsNullOrWhiteSpace(Code)) + { + + } + @Extra +
+ +@code { + [Parameter, EditorRequired] + public string Title { get; set; } = string.Empty; + + /// Small uppercase kicker, e.g. a step number in a walkthrough. + [Parameter] + public string? Step { get; set; } + + [Parameter] + public RenderFragment? Description { get; set; } + + /// One concrete action the reader can take to see the behaviour. + [Parameter] + public RenderFragment? Try { get; set; } + + /// The live demo. + [Parameter] + public RenderFragment? ChildContent { get; set; } + + /// The snippet that produces what the demo just did. + [Parameter] + public string? Code { get; set; } + + [Parameter] + public string CodeTitle { get; set; } = "Sample"; + + [Parameter] + public string Language { get; set; } = "csharp"; + + [Parameter] + public bool CodeOpen { get; set; } = true; + + /// Anything that belongs after the snippet, such as extra code blocks. + [Parameter] + public RenderFragment? Extra { get; set; } +} diff --git a/src/Bsqlite/Demo/Client/Components/Icon.razor b/src/Bsqlite/Demo/Client/Components/Icon.razor new file mode 100644 index 00000000000..7e649e94c97 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/Icon.razor @@ -0,0 +1,55 @@ +@* Every icon on the site, inlined. A demo that demonstrates an offline-first database has no + business downloading an icon font, and emoji are not icons: they render differently on every + platform and read as text to a screen reader. + + Icons are decorative here - the label beside them carries the meaning - so the is marked + aria-hidden unless the caller passes a Title, which turns it into a labelled image instead. *@ + +@Body + +@code { + [Parameter, EditorRequired] + public string Name { get; set; } = string.Empty; + + /// Set only when the icon is the sole content of a control and so carries its name. + [Parameter] + public string? Title { get; set; } + + private MarkupString Body => new( + (Title is null ? string.Empty : $"{Title}") + + (Paths.TryGetValue(Name, out var path) ? path : Paths["dot"])); + + private static readonly Dictionary Paths = new(StringComparer.Ordinal) + { + ["database"] = """""", + ["home"] = """""", + ["boxes"] = """""", + ["users"] = """""", + ["receipt"] = """""", + ["terminal"] = """""", + ["drive"] = """""", + ["files"] = """""", + ["shield"] = """""", + ["warning"] = """""", + ["bolt"] = """""", + ["layers"] = """""", + ["sync"] = """""", + ["package"] = """""", + ["copy"] = """""", + ["check"] = """""", + ["github"] = """""", + ["external"] = """""", + ["arrow-right"] = """""", + ["sun"] = """""", + ["moon"] = """""", + ["monitor"] = """""", + ["menu"] = """""", + ["close"] = """""", + ["book"] = """""", + ["dot"] = """""", + }; +} diff --git a/src/Bsqlite/Demo/Client/Components/PageHeader.razor b/src/Bsqlite/Demo/Client/Components/PageHeader.razor new file mode 100644 index 00000000000..d2cf865a4e0 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/PageHeader.razor @@ -0,0 +1,21 @@ +@* Every demo page opens the same way: what part of the library it exercises, its title, and one + paragraph of what to look for. It also owns the document title, so a bookmarked demo page is + named after the demo rather than after the site. *@ +@Title · Bsqlite + +
+

@Kicker

+

@Title

+

@ChildContent

+ + +@code { + [Parameter, EditorRequired] + public string Kicker { get; set; } = string.Empty; + + [Parameter, EditorRequired] + public string Title { get; set; } = string.Empty; + + [Parameter] + public RenderFragment? ChildContent { get; set; } +} diff --git a/src/Bsqlite/Demo/Client/Components/SessionLifecycle.razor b/src/Bsqlite/Demo/Client/Components/SessionLifecycle.razor new file mode 100644 index 00000000000..2ed307f69b7 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/SessionLifecycle.razor @@ -0,0 +1,62 @@ +@inject BrowserDatabase Database +@implements IAsyncDisposable + +@ChildContent + +@code { + [Parameter] + public RenderFragment? ChildContent { get; set; } + + [Parameter] + public EventCallback OnReady { get; set; } + + [Parameter] + public EventCallback OnTearDown { get; set; } + + protected override async Task OnInitializedAsync() + { + // Every page is prerendered on the server first, where there is no browser to open a database in. + // The page renders its static content there; the database opens once it runs in the browser. + if (RendererInfo.IsInteractive is false) + { + return; + } + + Database.BeforeSessionReset += TearDownAsync; + Database.AfterSessionReset += ReadyAsync; + + // Opening raises AfterSessionReset, which runs ReadyAsync once for every subscriber; only + // an already-open database needs the explicit call. + if (Database.Database.IsOpen) + { + await ReadyAsync(); + } + else + { + await Database.EnsureSessionAsync(); + } + } + + private async Task TearDownAsync() + { + if (OnTearDown.HasDelegate) + { + await OnTearDown.InvokeAsync(); + } + } + + private async Task ReadyAsync() + { + if (OnReady.HasDelegate) + { + await OnReady.InvokeAsync(); + } + } + + public async ValueTask DisposeAsync() + { + Database.BeforeSessionReset -= TearDownAsync; + Database.AfterSessionReset -= ReadyAsync; + await TearDownAsync(); + } +} diff --git a/src/Bsqlite/Demo/Client/Components/StorageSwitcher.razor b/src/Bsqlite/Demo/Client/Components/StorageSwitcher.razor new file mode 100644 index 00000000000..e267d76c463 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/StorageSwitcher.razor @@ -0,0 +1,123 @@ +@inject BrowserDatabase Database +@implements IAsyncDisposable + +@* The session bar: which storage backend this tab's database actually landed on, and the two + operations that move it somewhere else. It sits above every demo page because every demo runs + against whatever it says. *@ + +
+
+ +
+

Client database

+

@SampleStorage.Title(CurrentName)

+
+
+ +

@SampleStorage.Blurb(CurrentName)

+ +
+ + + +
+ + @if (_message is not null) + { +

@_message

+ } +
+ +@code { + private string _target = "opfs"; + private bool _busy; + private bool _failed; + private string? _message; + + private string CurrentName => Database.Resolution?.Provider.Name ?? _target; + + private bool SameTarget => + string.Equals(CurrentName, _target, StringComparison.OrdinalIgnoreCase); + + protected override async Task OnInitializedAsync() + { + // Prerendered on the server too, where there is no session to describe: the bar shows the + // preferred backend until the page runs in the browser and opens the real one. + if (RendererInfo.IsInteractive is false) + { + return; + } + + Database.AfterSessionReset += OnResetAsync; + try + { + await Database.EnsureSessionAsync(); + _target = CurrentName; + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _failed = true; + _message = ex.Message; + } + } + + private async Task OnResetAsync() + { + try + { + await Database.EnsureSessionAsync(); + _target = CurrentName; + await InvokeAsync(StateHasChanged); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + } + } + + private async Task SwitchAsync(StorageSwitchMode mode) + { + _busy = true; + _failed = false; + _message = null; + try + { + await Database.SwitchStorageAsync(_target, mode); + await Database.EnsureSessionAsync(); + var verb = mode is StorageSwitchMode.Migrate ? "Moved onto" : "Opened empty on"; + _message = $"{verb} {SampleStorage.Title(CurrentName)}."; + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _failed = true; + _message = ex.Message; + } + finally + { + _busy = false; + } + } + + public async ValueTask DisposeAsync() + { + Database.AfterSessionReset -= OnResetAsync; + await Task.CompletedTask; + } +} diff --git a/src/Bsqlite/Demo/Client/Components/ThemeToggle.razor b/src/Bsqlite/Demo/Client/Components/ThemeToggle.razor new file mode 100644 index 00000000000..00cddc42f12 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/ThemeToggle.razor @@ -0,0 +1,54 @@ +@inject IJSRuntime Js + +@* System / light / dark. The chosen mode is stored in localStorage and re-applied before first + paint by the boot script in index.html, so reloads never flash the wrong theme. *@ +
+ + + +
+ +@code { + private string _mode = "system"; + + // ARIA wants the literal strings. A bool would make Blazor drop the attribute when false and + // emit a valueless one when true, neither of which a screen reader reports as a toggle state. + private string Pressed(string mode) => _mode == mode ? "true" : "false"; + + protected override async Task OnAfterRenderAsync(bool firstRender) + { + if (!firstRender) + { + return; + } + + try + { + _mode = await Js.InvokeAsync("bsqliteDemo.theme.get"); + StateHasChanged(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + } + } + + private async Task SetAsync(string mode) + { + _mode = mode; + try + { + await Js.InvokeAsync("bsqliteDemo.theme.set", mode); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + } + } +} diff --git a/src/Bsqlite/Demo/Client/Components/TopBar.razor b/src/Bsqlite/Demo/Client/Components/TopBar.razor new file mode 100644 index 00000000000..0b524a45970 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Components/TopBar.razor @@ -0,0 +1,52 @@ +@inject NavigationManager Nav +@implements IDisposable + +
+
+ + + + + Bsqlite + + v@(SiteLinks.Version) + +
+ Overview + @* "true" rather than "page": this marks the section you are inside, not the exact page. *@ + Live demo + Docs +
+ +
+ + + + +
+
+
+ +@code { + [Parameter] + public bool NavOpen { get; set; } + + [Parameter] + public EventCallback OnToggleNav { get; set; } + + private bool IsHome => Nav.ToBaseRelativePath(Nav.Uri).Trim('/').Length == 0; + + protected override void OnInitialized() + => Nav.LocationChanged += OnLocationChanged; + + private void OnLocationChanged(object? sender, Microsoft.AspNetCore.Components.Routing.LocationChangedEventArgs e) + => StateHasChanged(); + + public void Dispose() + => Nav.LocationChanged -= OnLocationChanged; +} diff --git a/src/Bsqlite/Demo/Client/Extensions/IServiceCollectionExtensions.cs b/src/Bsqlite/Demo/Client/Extensions/IServiceCollectionExtensions.cs new file mode 100644 index 00000000000..4fb22aed105 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Extensions/IServiceCollectionExtensions.cs @@ -0,0 +1,59 @@ +using Bit.Bsqlite.Demo.Client; +using Bit.Bsqlite.Demo.Data; +using Bit.Bsqlite.Storage.CacheStorage; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Storage.IndexedDb; +using Bit.Bsqlite.Storage.Opfs; + +namespace Microsoft.Extensions.DependencyInjection; + +/// +/// The services shared by the WebAssembly client and the prerendering host. Both containers have to +/// register the same set: every page is instantiated once on the server for the prerender pass and again in +/// the browser, so anything a page injects must resolve in both places or prerendering fails with a +/// missing-service exception. +/// +/// +/// Resolving them on the server is harmless: nothing here touches the browser until a page asks it to open +/// the database, and every page does that only once it runs interactively (see Components/SessionLifecycle.razor). +/// +public static class IServiceCollectionExtensions +{ + /// + /// The app's own origin, for the a page can call the host with. The prerendering + /// host passes nothing: a page never issues a request while it is being prerendered. + /// + public static IServiceCollection AddDemoServices(this IServiceCollection services, string? baseAddress = null) + { + services.AddScoped(_ => baseAddress is null + ? new HttpClient() + : new HttpClient { BaseAddress = new Uri(baseAddress) }); + + // One call registers selection, sticky bindings in localStorage, the worker transport, and the + // shared database. Existing data outranks preference: a file created on IndexedDB stays there even + // once OPFS becomes available, unless the application migrates it deliberately. + services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .AddStorageProvider() + .AddStorageProvider() + .AddStorageProvider() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteCacheStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()) + .AddDatabase(BrowserDatabase.DatabaseName)); + + // Contexts come from a factory bound to that database. Migrations run once per session, then the + // demo catalog is seeded if the database is empty. + services.AddBitBsqliteDbContext(options => + options.OnFirstUse = (db, ct) => DemoData.SeedIfEmptyAsync(db, ct)); + + // Scoped like the library's database it wraps: one for the app in the browser, one per request on the + // prerendering host. + services.AddScoped(); + + return services; + } +} diff --git a/src/Bsqlite/Demo/Client/Layout/MainLayout.razor b/src/Bsqlite/Demo/Client/Layout/MainLayout.razor new file mode 100644 index 00000000000..31a9e61464a --- /dev/null +++ b/src/Bsqlite/Demo/Client/Layout/MainLayout.razor @@ -0,0 +1,100 @@ +@inherits LayoutComponentBase +@inject NavigationManager Nav +@inject IJSRuntime Js +@implements IDisposable + +@* Two shells, one layout. The overview is a landing page and runs full-bleed with no sidebar; + every other route is documentation and gets the docs shell: sidebar, session bar, content. *@ + + + + + +@* data-interactive says whether the page runs in the browser yet - prerendered markup is on screen long before + anything listens to it - which is what the E2E suite waits for before it touches a page. *@ + +@if (IsLanding) +{ +
+ + @Body + +
+ + +
+} +else +{ +
+ + + + +
+ + + @Body + + + + +
+
+} + +
+
+

+ bit Bsqlite @SiteLinks.Version · MIT · EF Core on .NET 8, 9 and 10. + Everything on this site runs in your browser. +

+ +
+
+ +@code { + private ErrorBoundary? _boundary; + private bool _navOpen; + + private string Interactive => RendererInfo.IsInteractive ? "true" : "false"; + + private bool IsLanding => Nav.ToBaseRelativePath(Nav.Uri).Trim('/').Length == 0; + + protected override void OnInitialized() + => Nav.LocationChanged += OnLocationChanged; + + protected override void OnParametersSet() + => _boundary?.Recover(); + + private void OnLocationChanged(object? sender, Microsoft.AspNetCore.Components.Routing.LocationChangedEventArgs e) + => StateHasChanged(); + + private Task ToggleNavAsync() => SetNavAsync(!_navOpen); + + private Task CloseNavAsync() => _navOpen ? SetNavAsync(false) : Task.CompletedTask; + + private async Task SetNavAsync(bool open) + { + _navOpen = open; + try + { + // The drawer covers the viewport under 1024px; letting the page behind it scroll is + // the classic mobile-drawer bug. + await Js.InvokeVoidAsync("bsqliteDemo.lockScroll", open); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + } + } + + public void Dispose() + => Nav.LocationChanged -= OnLocationChanged; +} diff --git a/src/Bsqlite/Demo/Client/Layout/UnhandledExceptionView.razor b/src/Bsqlite/Demo/Client/Layout/UnhandledExceptionView.razor new file mode 100644 index 00000000000..401b804faaa --- /dev/null +++ b/src/Bsqlite/Demo/Client/Layout/UnhandledExceptionView.razor @@ -0,0 +1,18 @@ +@if (Exception is not null) +{ +

@Exception.GetType().FullName: @Exception.Message

+

The full exception, including inner exceptions and stack traces, is in the browser console.

+} + +@code { + [Parameter] + public Exception? Exception { get; set; } + + protected override void OnParametersSet() + { + if (Exception is not null) + { + SampleLog.Exception(Exception); + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/Admin.razor b/src/Bsqlite/Demo/Client/Pages/Admin.razor new file mode 100644 index 00000000000..bf874fdf961 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Admin.razor @@ -0,0 +1,288 @@ +@page "/admin" +@inject BrowserDatabase Database +@inject IJSRuntime Js + + + + + IBsqliteStorageAdmin treats a database as an artifact rather than a + connection: what exists, read the bytes out, write bytes back, delete. Cross-provider + migration and the diagnostics on Storage are built from this same + surface, and none of it goes through EF. + + +@if (_error is not null) +{ +

@_error

+} +@if (_info is not null) +{ +

@_info

+} + +
+ + + + Names come from the provider, so a backend that has never been used reports empty + rather than failing. Related -journal files are not databases and do + not appear. + + + + + @foreach (var group in _lists) + { +

@group.Name

+ @if (group.Names.Count == 0) + { +

Empty.

+ } + else + { +
    + @foreach (var name in group.Names) + { +
  • @name
    • + } +
+ } + } + + + + + + + Exports @BrowserDatabase.DatabaseName from whichever backend currently + owns it. On OPFS that is a file read; on IndexedDB and Cache Storage the image is + assembled from pages. Either way the result is an ordinary SQLite file. + + + + Download it and open it in any desktop SQLite tool - the rows you edited are there. + + + + + + + + + + Replaces the named database on the selected backend and records the sticky binding, + so the next open finds the imported data instead of creating an empty file + elsewhere. The worker is closed first: an open connection would keep writing the + image being replaced. + + + + + Export above, delete below, then import the file back. This is also how you ship a + pre-built database with an app. + + + +
+ + + +
+ + + + + + + Removes @BrowserDatabase.DatabaseName from the current backend and + forgets the binding, so the next open is free to pick the preferred provider again, + run the migrations, and reseed. + + + + + + +
+ +@code { + private readonly List<(string Name, IReadOnlyList Names)> _lists = []; + private string? _error; + private string? _info; + private bool _busy; + private string _importProvider = "opfs"; + private byte[]? _importBytes; + + private static Task NoopAsync() => Task.CompletedTask; + + private async Task LoadAsync() + { + try + { + await Database.EnsureSessionAsync(); + _importProvider = Database.Resolution?.Provider.Name ?? "opfs"; + await ListAllAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task ListAllAsync() + { + _busy = true; + _error = null; + _lists.Clear(); + try + { + foreach (var provider in Database.Providers) + { + var names = await provider.Admin.ListAsync(); + _lists.Add((provider.Name, names)); + } + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } + + private async Task ExportAsync() + { + _busy = true; + _error = null; + _info = null; + try + { + var session = await Database.EnsureSessionAsync(); + var bytes = await session.Resolution.Provider.Admin.ExportAsync(BrowserDatabase.DatabaseName); + await Js.InvokeVoidAsync("bsqliteDemo.download", "app.db", bytes); + _info = $"Exported {SampleFormat.Bytes(bytes.LongLength)} from {session.Resolution.Provider.Name}."; + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } + + private async Task OnFile(InputFileChangeEventArgs args) + { + _error = null; + try + { + await using var stream = args.File.OpenReadStream(maxAllowedSize: 32 * 1024 * 1024); + using var memory = new MemoryStream(); + await stream.CopyToAsync(memory); + _importBytes = memory.ToArray(); + _info = $"Ready to import {args.File.Name} ({SampleFormat.Bytes(_importBytes.LongLength)})."; + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task ImportAsync() + { + if (_importBytes is null) + { + return; + } + + var provider = Database.Providers.FirstOrDefault(p => + string.Equals(p.Name, _importProvider, StringComparison.OrdinalIgnoreCase)); + if (provider is null) + { + _error = $"No registered provider named {_importProvider}."; + return; + } + + _busy = true; + _error = null; + _info = null; + try + { + await Database.CloseSessionAsync(); + await provider.Admin.ImportAsync(BrowserDatabase.DatabaseName, _importBytes); + await Database.Bindings.SetProviderNameAsync(BrowserDatabase.DatabaseName, provider.Name); + + // Reopened here so migrations and the seed run before the pages come back for it. + await using (await Database.OpenContextAsync()) + { + } + _info = $"Imported into {provider.Name} and reopened. Catalog and orders will refresh."; + await ListAllAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } + + private async Task DeleteAsync() + { + _busy = true; + _error = null; + _info = null; + try + { + var provider = Database.Resolution?.Provider + ?? throw new InvalidOperationException("Open a session before deleting."); + await Database.CloseSessionAsync(); + await provider.Admin.DeleteAsync(BrowserDatabase.DatabaseName); + await Database.Bindings.RemoveAsync(BrowserDatabase.DatabaseName); + + await using (await Database.OpenContextAsync()) + { + } + _info = "Deleted and reopened. Migrations ran again; the workshop catalog was reseeded if empty."; + await ListAllAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/Catalog.razor b/src/Bsqlite/Demo/Client/Pages/Catalog.razor new file mode 100644 index 00000000000..6928fdd7fb6 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Catalog.razor @@ -0,0 +1,384 @@ +@page "/catalog" +@inject BrowserDatabase Database + + + + + Products and categories, read and written entirely on the client. SKU is unique, price is a + decimal, lead time is a TimeSpan, and discontinued-on is a nullable + DateOnly - the types a browser database is usually worst at. Every button below + is a normal EF Core call; the snippet under each panel is the code it runs. + + +@if (_error is not null) +{ +

@_error

+} + +
+
+
+

Products

+
+

+ Try + + Edit a row and save, then reload the tab - the row is still there, because the + write went to the storage backend named in the bar above. + +

+
+ + + +
+
+ + + + + + + + @foreach (var product in VisibleProducts) + { + + + + + + + + + + } + +
SKUNameCategoryPriceLeadStatus
@product.Sku@product.Name + + @product.Category?.Name + @SampleFormat.Money(product.Price)@SampleFormat.Lead(product.LeadTime) + @if (!product.IsActive) + { + discontinued @SampleFormat.Day(product.DiscontinuedOn) + } + else + { + active + } + + +
+
+ +
+
+
+

@(_editProduct?.Id == 0 ? "New product" : "Edit product")

+ @if (_editProduct is null) + { +

Select a row, or create one.

+ } + else + { +
+ + + + + + + + + +

PublicId @_editProduct.PublicId

+
+ + +
+
+ } +
+
+

Categories

+

A category cascades to its products; a product on an order does not delete at all.

+
+ + + +
+ + + + @foreach (var category in _categories) + { + + + + + } + +
NameItems
@category.Name@_products.Count(p => p.CategoryId == category.Id)
+
+
+
+ +
+ + + + One code path for both: EF decides insert or update from the key. The write goes + through the worker and lands in the storage backend named in the bar above, so it + is still there after a reload. + + + + + + + The engine in the worker is real SQLite with foreign keys on. Nothing in the browser + re-implements referential integrity, so a constraint failure comes back as the + DbUpdateException you would get on a server. + + + + Delete a product that appears on an order. The message above + the table is SQLite's, surfaced through EF. + + + + + + + + Ordinary IEntityTypeConfiguration. The same class compiles against the + server's UseSqlite, and the same migration creates both databases - + nothing here is browser-specific. + + + +
+ +@code { + private AppDbContext? _context; + private List _products = []; + private List _categories = []; + private Product? _editProduct; + private string _filter = string.Empty; + private string _categoryName = string.Empty; + private string _categoryColor = "#0f766e"; + private int _leadDays = 2; + private bool _busy; + private string? _error; + + private IEnumerable VisibleProducts + { + get + { + if (string.IsNullOrWhiteSpace(_filter)) + { + return _products; + } + + var term = _filter.Trim(); + return _products.Where(p => + p.Name.Contains(term, StringComparison.OrdinalIgnoreCase) + || p.Sku.Contains(term, StringComparison.OrdinalIgnoreCase) + || p.Tags.Contains(term, StringComparison.OrdinalIgnoreCase)); + } + } + + private async Task LoadAsync() + { + _error = null; + try + { + _context = await Database.OpenContextAsync(); + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task UnloadAsync() + { + _editProduct = null; + if (_context is not null) + { + await _context.DisposeAsync(); + _context = null; + } + } + + private async Task RefreshAsync() + { + if (_context is null) + { + return; + } + + _categories = await _context.Categories.AsNoTracking().OrderBy(c => c.Name).ToListAsync(); + _products = await _context.Products + .AsNoTracking() + .Include(p => p.Category) + .OrderBy(p => p.Sku) + .ToListAsync(); + } + + private void NewProduct() + { + var categoryId = _categories.FirstOrDefault()?.Id ?? 0; + _editProduct = new Product + { + Sku = "SKU-" + Guid.NewGuid().ToString("N")[..6].ToUpperInvariant(), + Name = string.Empty, + CategoryId = categoryId, + Price = 0, + LeadTime = TimeSpan.FromDays(2), + IsActive = true, + }; + _leadDays = 2; + } + + private void EditProduct(Product product) + { + _editProduct = new Product + { + Id = product.Id, + PublicId = product.PublicId, + Sku = product.Sku, + Name = product.Name, + Price = product.Price, + WeightKg = product.WeightKg, + IsActive = product.IsActive, + CreatedUtc = product.CreatedUtc, + DiscontinuedOn = product.DiscontinuedOn, + LeadTime = product.LeadTime, + Tags = product.Tags, + CategoryId = product.CategoryId, + }; + _leadDays = Math.Max(0, (int)Math.Round(product.LeadTime.TotalDays)); + } + + private async Task SaveProductAsync() + { + if (_context is null || _editProduct is null || string.IsNullOrWhiteSpace(_editProduct.Name)) + { + return; + } + + _busy = true; + _error = null; + try + { + _editProduct.LeadTime = TimeSpan.FromDays(_leadDays); + if (!_editProduct.IsActive && _editProduct.DiscontinuedOn is null) + { + _editProduct.DiscontinuedOn = DateOnly.FromDateTime(DateTime.UtcNow); + } + + if (_editProduct.Id == 0) + { + _context.Products.Add(_editProduct); + } + else + { + _context.Products.Update(_editProduct); + } + + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + _editProduct = null; + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } + + private async Task DeleteProductAsync(Product product) + { + if (_context is null) + { + return; + } + + _busy = true; + _error = null; + try + { + var tracked = await _context.Products.FirstAsync(p => p.Id == product.Id); + _context.Products.Remove(tracked); + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + if (_editProduct?.Id == product.Id) + { + _editProduct = null; + } + + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } + + private async Task SaveCategoryAsync() + { + if (_context is null || string.IsNullOrWhiteSpace(_categoryName)) + { + return; + } + + _busy = true; + _error = null; + try + { + _context.Categories.Add(new Category { Name = _categoryName.Trim(), Color = _categoryColor }); + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + _categoryName = string.Empty; + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/Customers.razor b/src/Bsqlite/Demo/Client/Pages/Customers.razor new file mode 100644 index 00000000000..2f696f1ab89 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Customers.razor @@ -0,0 +1,283 @@ +@page "/customers" +@inject BrowserDatabase Database + + + + + The same CRUD shape as Catalog, aimed at the values that historically broke on a WASM + transport: a Guid public id, a DateOnly birth date, a + bool flag stored as INTEGER, and a decimal credit limit that keeps + its scale because it is compared through ef_compare, not floating point. + + +@if (_error is not null) +{ +

@_error

+} + +
+
+

Directory

+

+ Try + + Save a customer with an email that already exists. The unique index is SQLite's, + so the failure arrives as a DbUpdateException - see the snippet below. + +

+
+ + + +
+
+ + + + + + @foreach (var customer in Visible) + { + + + + + + + + + } + +
NameEmailBornCreditVIP
@customer.DisplayName@customer.Email@SampleFormat.Day(customer.DateOfBirth)@SampleFormat.Money(customer.CreditLimit)@(customer.IsVip ? "yes" : "no") + +
+
+ +
+
+
+

@(_edit?.Id == 0 ? "New customer" : "Edit customer")

+ @if (_edit is null) + { +

Select a row, or create one.

+ } + else + { +
+ + + + + + +

PublicId @_edit.PublicId

+
+ + +
+
+ } +
+
+

What the values become

+
+ + + + + + + + + +
PropertyStored as
PublicIdTEXT, 36-char Guid
DateOfBirthTEXT yyyy-MM-dd
IsVipINTEGER 0/1
CreditLimitTEXT, compared via ef_compare
CreatedUtcTEXT, ISO 8601
+
+

+ Filtering on CreditLimit - a real decimal comparison in SQL, not in + the CLR - is on the SQL page. +

+
+
+
+ +
+ + + No browser-specific attributes, no converters bolted on for WASM. Precision and the + unique index are declared once and apply on the server and in the tab alike. + + + + + + + After a failed save the change tracker still holds the rejected entity, so the + demo clears it before the next read - the same hygiene you would apply on a + server. The inner exception carries SQLite's message and result code. + + + +
+ +@code { + private AppDbContext? _context; + private List _rows = []; + private Customer? _edit; + private string _filter = string.Empty; + private bool _busy; + private string? _error; + + private IEnumerable Visible + { + get + { + if (string.IsNullOrWhiteSpace(_filter)) + { + return _rows; + } + + var term = _filter.Trim(); + return _rows.Where(c => + c.DisplayName.Contains(term, StringComparison.OrdinalIgnoreCase) + || c.Email.Contains(term, StringComparison.OrdinalIgnoreCase)); + } + } + + private async Task LoadAsync() + { + _error = null; + try + { + _context = await Database.OpenContextAsync(); + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task UnloadAsync() + { + _edit = null; + if (_context is not null) + { + await _context.DisposeAsync(); + _context = null; + } + } + + private async Task RefreshAsync() + { + if (_context is null) + { + return; + } + + _rows = await _context.Customers.AsNoTracking().OrderBy(c => c.DisplayName).ToListAsync(); + } + + private void NewCustomer() + { + _edit = new Customer + { + DisplayName = string.Empty, + Email = string.Empty, + DateOfBirth = new DateOnly(1980, 1, 1), + CreditLimit = 1_000m, + }; + } + + private void Edit(Customer customer) + { + _edit = new Customer + { + Id = customer.Id, + PublicId = customer.PublicId, + DisplayName = customer.DisplayName, + Email = customer.Email, + DateOfBirth = customer.DateOfBirth, + IsVip = customer.IsVip, + CreditLimit = customer.CreditLimit, + Notes = customer.Notes, + CreatedUtc = customer.CreatedUtc, + }; + } + + private async Task SaveAsync() + { + if (_context is null || _edit is null) + { + return; + } + + _busy = true; + _error = null; + try + { + if (_edit.Id == 0) + { + _context.Customers.Add(_edit); + } + else + { + _context.Customers.Update(_edit); + } + + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + _edit = null; + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } + + private async Task DeleteAsync(Customer customer) + { + if (_context is null) + { + return; + } + + _busy = true; + _error = null; + try + { + var tracked = await _context.Customers.FirstAsync(c => c.Id == customer.Id); + _context.Customers.Remove(tracked); + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + if (_edit?.Id == customer.Id) + { + _edit = null; + } + + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/ErrorPage.razor b/src/Bsqlite/Demo/Client/Pages/ErrorPage.razor new file mode 100644 index 00000000000..485f544e5e3 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/ErrorPage.razor @@ -0,0 +1,18 @@ +@page "/Error" + +@* Target of app.UseExceptionHandler("/Error") in the host's Program.cs. It lives in the client + project on purpose: Routes.razor binds the router to this assembly only, so a page placed in + the host project would never be matched and the re-executed request would end in a 404. *@ + +Error · Bsqlite + +
+

500

+

Something went wrong

+ +
+Back to the overview +Report an issue diff --git a/src/Bsqlite/Demo/Client/Pages/Home.razor b/src/Bsqlite/Demo/Client/Pages/Home.razor new file mode 100644 index 00000000000..8954dc38353 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Home.razor @@ -0,0 +1,546 @@ +@page "/" +@inject BrowserDatabase Database + +Bsqlite · EF Core on SQLite, inside the browser tab + + + +
+ + @* ---------------------------------------------------------------- hero *@ +
+
+
+
+ .NET 8, 9 and 10 · EF Core + MIT + + bitfoundation/bitplatform + +
+ +

EF Core on real SQLite, inside the browser tab.

+ +

+ The same entities, the same DbContext, and the same migrations you + run on the server - executing against a SQLite engine in a Web Worker, over OPFS, + IndexedDB, Cache Storage, or memory. This page is a Blazor WebAssembly app: after + the download, nothing here talks to a server. +

+ +
+ Open the live demo + + Source + +
+ +
+ + dotnet add package Bit.Bsqlite + +
+ +

+ Five packages: the core, three storage backends, and a test kit. Register only + the backends you ship. +

+
+ +
+
+ + Program.cs +
+ 
@CodeHighlighter.Highlight(Snippets.Setup.Hero, "csharp")
+
+
+
+ +
+ + @if (_error is not null) + { +

+ + @_error +

+ } + + @* -------------------------------------------------- live proof strip *@ +
+
+ @_products + Products in this tab +
+
+ @_customers + Customers +
+
+ @_openOrders + Open orders +
+
+ @SampleStorage.Title(_backend ?? "opening…") + @_persist +
+
+ + @* ------------------------------------------------------- capability *@ +
+
+

Why it exists

+

A browser database that behaves like the one on your server

+

+ Not a re-implementation of SQL over IndexedDB. The engine is wa-sqlite - SQLite + compiled to WebAssembly - and the provider is the stock EF Core SQLite provider + with two services swapped so nothing blocks the UI thread. +

+
+ +
+
+ +

One model, both sides

+

+ UseSqlite on the server, UseBitBsqlite in the tab. + Same entities, same configuration classes, same migration set - and the test + suite proves migrations scaffolded against stock SQLite apply here unchanged. +

+
+
+ +

Live queries

+

+ AsLiveQuery() re-runs a query when a table it reads is written - + by this tab, by another tab, or by raw SQL that never touched EF. It learns + what changed from SQLite's own update hook, after the transaction commits. +

+
+
+ +

Storage is a contract

+

+ OPFS, IndexedDB, Cache Storage, and in-memory are interchangeable providers + behind one interface. Selection prefers the order you configure, but existing + data always outranks preference. +

+
+
+ +

Real SQLite semantics

+

+ Foreign keys, unique indexes, and transactions are the engine's, not the UI's. + A constraint failure arrives as the DbUpdateException you would + get on a server, carrying SQLite's own message and result code. +

+
+
+ +

Move the file, keep the data

+

+ Migrating between backends exports the image, imports it on the target, checks + the SQLite header, flips the binding, then deletes the source. The bar above + every demo page is that API. +

+
+
+ +

Testable without a browser

+

+ Bit.Bsqlite.Testing ships an in-process transport for unit tests + plus the conformance kit every storage provider has to pass. CI runs the browser + suite across Chrome, Edge, Firefox, and WebKit. +

+
+
+
+ + @* ------------------------------------------------------ live demo *@ +
+
+

Proof

+

A query that re-runs itself

+

+ The table below is a live query over Products in this tab's database. + Nothing polls it. +

+
+ + + + + Add or edit a product on Catalog, or open this page in a + second tab and edit there. The rows below re-run on their own. + + + + @if (_liveRows.Count == 0) + { +

No products yet.

+ } + else + { +
+ + + + + + @foreach (var product in _liveRows) + { + + + + + + + } + +
SKUNamePriceActive
@product.Sku@product.Name@SampleFormat.Money(product.Price)@(product.IsActive ? "yes" : "no")
+
+ } + + +
+ + @* ------------------------------------------------------ quick start *@ +
+
+

Quick start

+

Four steps from an empty Blazor WASM app

+

+ Every snippet below is the code this demo actually runs, not a simplified + version of it. +

+
+ +
+
+ +
+

Install the packages you will ship

+

+ The core package carries the engine, the worker, and the storage contract. + Each persistent backend is a separate package, so an app pays only for the + ones it registers. +

+ +
+
+ +
+ +
+

Register the session factory

+

+ Order is preference, not a requirement: the first available backend wins, + and AllowNonPersistentFallback() is what makes in-memory + acceptable at all. +

+ +
+
+ +
+ +
+

Open a session and hand EF the connection

+

+ A session owns the worker, the transport, and one ADO.NET connection. + UseBitBsqlite(connection) is UseSqlite plus the + two service replacements that would otherwise call + DbConnection.Open() synchronously. SQL generation is the stock + provider's, unchanged. +

+ +
+
+ +
+ +
+

Query it - and stay in sync

+

+ From here it is EF Core. Add AsLiveQuery() where the UI should + follow the data instead of re-fetching it. +

+ +
+
+
+
+ + @* --------------------------------------------------------- backends *@ +
+
+

Storage

+

Four backends, one interface

+

+ Switch this tab between them with the bar at the top of any demo page, then read + the full contract on Storage backends. +

+
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
BackendPackagePersistsConcurrent readersUse it when
OPFSStorage.OpfsyesyesThe default. Real files, sync engine, no COOP/COEP headers needed.
IndexedDBStorage.IndexedDbyesnoWidest reach. Survives browsers without OPFS; readers serialise.
Cache StorageStorage.CacheStorageyesnoThe besql migration path. Persistence cost scales with database size.
In-memoryBit.BsqlitenoyesTrying the engine, or tests. Gone when the tab closes.
+
+
+ + @* ------------------------------------------------------------ types *@ +
+
+

Type mapping

+

CLR types stored in SQLite

+

+ SQLite has four storage classes. The ADO.NET binder decides which one each CLR + value takes, so the worker never guesses and values round-trip as the type you + wrote. +

+
+ +
+ + + + + + + + + + + + + +
CLRSQLiteWhere in this demo
decimalTEXT + ef_*Price, credit limit
DateTimeTEXT ISOCreatedUtc
DateTimeOffsetTEXT + offsetOrderedAt
DateOnlyTEXT dateDate of birth, ship-by
TimeSpanTEXT cLead time
GuidTEXTPublicId
boolINTEGER 0/1IsActive, IsVip
enumTEXTOrder status
doubleREALWeight
+
+ + +
+ + @* ------------------------------------------------------------ demos *@ +
+
+

Explore

+

Every page pairs a working demo with the code that drives it

+

All of it runs against the database in this tab, on the backend named in the bar.

+
+ +
+ @foreach (var entry in SiteMap.Demos) + { + + + + @entry.Title + @entry.Summary + + + + } +
+
+ + @* ----------------------------------------------------------- limits *@ +
+
+

Honest limits

+

What this release does not claim

+

+ The Limits page demonstrates each of these failing, on + purpose, with the exception it throws. +

+
+ +
+
+ +
+ Synchronous APIs throw +

+ Browsers forbid blocking the main thread, so SaveChanges() and + friends have async-only counterparts. There is no spin-wait hiding behind + them. +

+
+
+
+ +
+ WAL, ATTACH, and page-size changes are guarded +

+ A VFS over browser storage cannot honour them safely, so they are refused + up front rather than corrupting a file later. +

+
+
+
+ +
+ IndexedDB serialises readers +

+ Its VFS holds the same lock for a read as for a write. + SupportsConcurrentReads reports that honestly instead of + pretending otherwise. +

+
+
+
+ +
+ Client and server do not sync +

+ Both sides share a model and a migration set, not a replication protocol. + That is post-1.0. +

+
+
+
+
+ + @* ---------------------------------------------------------- closing *@ +
+

Open a database in your own tab

+

+ The demo pages are the documentation: each one runs a real EF Core call and shows the + exact snippet behind it. +

+
+ Start with Catalog + + Read the docs + +
+
+
+
+ +@code { + private AppDbContext? _context; + private IBsqliteLiveQuery>? _live; + private IReadOnlyList _liveRows = []; + private int _products; + private int _customers; + private int _openOrders; + private string? _backend; + private string _persist = "opening…"; + private string? _error; + private bool _copied; + + [Inject] + private IJSRuntime Js { get; set; } = default!; + + private async Task LoadAsync() + { + _error = null; + try + { + _context = await Database.OpenContextAsync(); + _backend = Database.Resolution?.Provider.Name; + var persistent = Database.Resolution?.Provider.Capabilities.IsPersistent == true; + _persist = persistent ? "Persists across reload" : "Gone when this tab closes"; + _products = await _context.Products.CountAsync(); + _customers = await _context.Customers.CountAsync(); + _openOrders = await _context.Orders.CountAsync(o => + o.Status != OrderStatus.Shipped && o.Status != OrderStatus.Cancelled); + + _live = _context.Products.AsNoTracking().OrderBy(p => p.Sku).Take(8).AsLiveQuery(); + _live.Changed += OnChanged; + _liveRows = await _live.RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task UnloadAsync() + { + if (_live is not null) + { + _live.Changed -= OnChanged; + await _live.DisposeAsync(); + _live = null; + } + + if (_context is not null) + { + await _context.DisposeAsync(); + _context = null; + } + } + + private void OnChanged(object? sender, IReadOnlyList rows) + { + _liveRows = rows; + _ = InvokeAsync(StateHasChanged); + } + + private async Task CopyInstallAsync() + { + try + { + _copied = await Js.InvokeAsync("bsqliteDemo.copy", "dotnet add package Bit.Bsqlite"); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _copied = false; + } + + if (_copied) + { + await Task.Delay(1400); + _copied = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/Limits.razor b/src/Bsqlite/Demo/Client/Pages/Limits.razor new file mode 100644 index 00000000000..15192f961da --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Limits.razor @@ -0,0 +1,229 @@ +@page "/limits" +@inject BrowserDatabase Database + + + + + The default tier is asynchronous: the engine lives in a worker and the Blazor UI thread is not + allowed to block on it. Some SQLite features cannot be honoured on the web at all. Each one + below fails with a typed exception that names the alternative, rather than half-working - run + them and read the real message. + + +@if (_error is not null) +{ +

@_error

+} + +
+ + + + SaveChanges(), ToList(), and + DbConnection.Open() throw + BsqliteSynchronousApiNotSupportedException, and the message names + the async overload to use. The opt-in tier that would lift this (COOP/COEP plus a + native ISQLite3Provider) slipped past 1.0. EF wraps a failed save, so + the panel below unwraps to the exception that actually explains the refusal. + + + + Run it: this is the exception your users would see, not a paraphrase of it. + + + + @if (_syncResult is not null) + { + 
@_syncResult
+ } + + + + + + No web VFS implements WAL: WASM lacks the shared-memory primitives it needs, and the + official SQLite WASM build cannot even read a WAL-mode file. Journal mode here + is DELETE or TRUNCATE. Concurrent reads, where a backend offers them, come from the VFS + instead. + + + + + + + + + + Gated by SupportsMultiDatabaseTransactions: OPFS CoopSync cannot run a + transaction spanning two files, IndexedDB can. The guard reads the selected + backend's capabilities - it does not test the provider name, so a new backend + inherits the rule for free. + + + +

+ This session supports multi-database transactions: + @SampleFormat.YesNo(_supportsAttach) +

+ + + + + + + Block-oriented backends pin the page size to their block size - IndexedDB batch-atomic + is the usual case. Letting the PRAGMA through would report success and then corrupt the + image on the next write, so it is refused up front. + + +

+ This session can change page size: + @SampleFormat.YesNo(_canChangePageSize) +

+ + + + + @if (_sqlResult is not null) + { +
+

Last statement

+ 
@_sqlResult
+
+ } +
+ +@code { + private bool _supportsAttach = true; + private bool _canChangePageSize = true; + private string? _error; + private string? _syncResult; + private string? _sqlResult; + private bool _busy; + + private static Task NoopAsync() => Task.CompletedTask; + + private async Task LoadAsync() + { + try + { + var session = await Database.EnsureSessionAsync(); + var limits = session.Connection.RuntimeLimits; + _supportsAttach = limits.SupportsMultiDatabaseTransactions; + _canChangePageSize = limits.CanChangePageSize; + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task TrySyncSaveAsync() + { + _busy = true; + _error = null; + _syncResult = null; + try + { + await using var context = await Database.OpenContextAsync(); + + // EF short-circuits SaveChanges() when nothing is pending, so stage a real insert: + // without one, the call would return 0 and never reach the connection. + var probe = new Category + { + Name = "sync-probe-" + Guid.NewGuid().ToString("N")[..8], + Color = "#0f766e", + }; + context.Categories.Add(probe); + + try + { + context.SaveChanges(); + _syncResult = "Unexpected: SaveChanges() succeeded on the async tier."; + + // Should be unreachable. If it ever runs, undo the write the guard let through. + context.ChangeTracker.Clear(); + var written = await context.Categories.FirstOrDefaultAsync(c => c.Name == probe.Name); + if (written is not null) + { + context.Categories.Remove(written); + await context.SaveChangesAsync(); + } + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _syncResult = Describe(ex); + } + finally + { + context.ChangeTracker.Clear(); + } + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } + + /// + /// EF wraps a failed save in DbUpdateException, whose own message says only "see the + /// inner exception" - so unwrap to the exception that actually explains the refusal. + /// + private static string Describe(Exception exception) + { + var root = exception; + while (root.InnerException is not null) + { + root = root.InnerException; + } + + return ReferenceEquals(root, exception) + ? $"{exception.GetType().Name}\n\n{exception.Message}" + : $"{exception.GetType().Name}: {exception.Message}\n\n-> {root.GetType().Name}\n\n{root.Message}"; + } + + private Task TryAttachAsync() + => RunSqlAsync("ATTACH DATABASE 'other.db' AS other"); + + private async Task RunSqlAsync(string sql) + { + _busy = true; + _error = null; + _sqlResult = null; + try + { + await Database.ExecuteNonQueryAsync(sql); + _sqlResult = $"{sql}\n\nCompleted without error on this backend."; + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _sqlResult = $"{sql}\n\n{Describe(ex)}"; + } + finally + { + _busy = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/McpServer.razor b/src/Bsqlite/Demo/Client/Pages/McpServer.razor new file mode 100644 index 00000000000..b608393a00e --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/McpServer.razor @@ -0,0 +1,640 @@ +@page "/mcp-server" +@inject HttpClient Http +@inject NavigationManager Nav + +@* The MCP server's own page. Its route differs from the endpoint's on purpose: two literal endpoints on + /mcp would make a GET of it ambiguous. + + Everything interactive here calls the demo's real /api/mcp endpoints - the same methods the MCP server + exposes as tools - so what a reader tries is exactly what an agent gets, and the page cannot document a + tool that does not work. The tool, prompt and resource lists are fetched rather than written out, for the + same reason: GetMcpCatalog reflects over the attributes that declare them. None of it opens a database; + the server answers from the library itself. *@ + + + This site hosts a Model Context Protocol server at /mcp, so an AI agent writing Bsqlite code + reads the library itself - its compiled API documentation, its guides, the capabilities its storage + providers declare, its own storage selection and SQL guards, and these demo pages - instead of guessing at + an API it half remembers. Every tool is read-only, and none of them executes SQL. + + +
+ + + + The endpoint is plain Streamable HTTP, so any MCP client connects with a URL and nothing else: + this site's own @Endpoint, the deployed https://bsqlite.bitplatform.dev/mcp, + or https://bitplatform.dev/mcp, which serves the tools of every bit library from one + address. The same tools are plain GET endpoints under /api/mcp/..., which is + what the panels below call. + + + +
+ $ + claude mcp add --transport http bsqlite @Endpoint +
+ + + + + + + PlanBsqliteStorageSelection runs the library's own resolver, migrator and session + factory - composed exactly as AddBitBsqliteServices composes them - against stand-in + providers that answer their probe the way you describe the browser. The first-party backends keep + the capabilities their provider types declare. Nothing is opened. + + + + + Bind the database to indexeddb, then switch the migration mode to + AutomaticOnOpen; or make opfs and indexeddb + unavailable with in-memory last, and toggle the fallback opt-in. + + + +
+ +
+ Unavailable in this browser: + @foreach (var backend in Backends) + { + + } +
+ + + +
+ +
+
+ + @if (_planError is not null) + { +

@_planError

+ } + else if (_plan is not null) + { +

+ @if (_plan.Opens) + { + Opens on @_plan.Provider + - engine build @_plan.EngineBuild + @if (_plan.WasDecidedByExistingData is true) + { + - decided by the existing binding. + } + else if (_plan.IsFirstChoice is false) + { + - a fallback. + } + } + else + { + Does not open + } +

+ + @if (_plan.Migration is { } migration) + { +

+ Migration @migration.From → @migration.To: @migration.Outcome + @if (migration.Copied is false) + { + - nothing was copied. + } + @if (migration.Error is not null) + { + - @migration.Error + } +

+ } + + @if (_plan.Error is not null) + { + 
@_plan.Error
+ } + +
+ + + + + + @foreach (var candidate in _plan.Candidates ?? []) + { + + + + + + + + + } + +
BackendRankPersistentAvailableStatusWhy
@candidate.Name@(candidate.Rank?.ToString() ?? "-")@YesNo(candidate.Persistent)@YesNo(candidate.Available)@(candidate.Status ?? "-")@(candidate.Explanation ?? "-")
+
+ + @* The notes name C# - WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen) is one word to a + line breaker - so they sit in a scroller rather than widening a phone's page. *@ + @if (_plan.Notes is { Length: > 0 } notes) + { +
+ + + + @foreach (var note in notes) + { + + } + +
What it means
@note
+
+ } + + @if (_plan.Log is { Length: > 0 } log) + { + 
@string.Join("\n", log)
+ } + } + + + @if (_plan?.Configuration is { } configuration) + { + + } + + + + + + + CheckBsqliteSql puts SQL to the library's own static rules: the guards each backend's + declared capabilities switch on, and the table scan a live query is keyed by. It is pattern + matching over the text - nothing is executed, on the server or anywhere else. + + + +
+ + +
+
+ @foreach (var sample in SqlSamples) + { + + } +
+ + @if (_checkError is not null) + { +

@_checkError

+ } + else if (_check?.Statements is { Length: > 0 } statements) + { + var statement = statements[0]; + +
+ + + + @foreach (var verdict in statement.Backends ?? []) + { + + + + + + } + +
BackendVerdictThe guard's message
@verdict.Name@(verdict.Accepted ? "accepted" : "refused")@(verdict.Error ?? "-")
+
+

+ A live query over it watches: + @if (statement.Tables is { Length: > 0 } tables) + { + @foreach (var table in tables) + { + @table + + } + } + else + { + no table names - every write would refresh it + } +

+ } + else if (_check?.Error is { } error) + { +

@error

+ } + + + + + + + An agent rarely knows which corpus holds the answer. SearchBsqlite covers every + document, every public member of the five packages, the storage backends, the setup guides, the labs + and the demo's sources at once - and every hit carries the exact follow-up call that returns it in full. + + + +
+ + +
+ + @if (_searchError is not null) + { +

@_searchError

+ } + else if (_search?.Hits is { Length: > 0 } hits) + { +
+ + + + @foreach (var hit in hits) + { + + + + + + } + +
KindTitleFollow-up call
@hit.Kind@hit.Title@hit.Tool
+
+ } + else if (_search is not null) + { +

@(_search.Message ?? "Nothing matched that.")

+ } + + + +
+

Tools

+ @if (_catalog is null) + { +

@(IsHydrating ? "The list loads once the page is interactive." : "Loading the catalog...")

+ } + else + { +

+ @_catalog.Tools.Length of them, listed straight from the attributes that declare them - so this table + cannot fall out of step with the server. +

+
+ + + + @foreach (var tool in _catalog.Tools) + { + + + @* One argument to a line: the planner takes nine, and on one unbreakable line they would push + the description out of view. *@ + + + + } + +
ToolArgumentsWhat it answers
@tool.Name + @if (tool.Parameters is { Length: > 0 } parameters) + { + @foreach (var parameter in parameters) + { + @parameter
+ } + } + else + { + none + } + 		@tool.Title - @FirstSentence(tool.Description)
+
+ } +
+ +
+

Prompts and resources

+

+ Prompts are the workflows - the order to call the tools in for a whole job - because the failure mode of an + agent holding a shelf of tools is calling them in an order that skips the check which would have caught + the bug. Resources are the same knowledge as bsqlite:// URIs, for clients that attach + documentation up front; they read the same catalogs the tools do. +

+ @if (_catalog is not null) + { +
+ @foreach (var prompt in _catalog.Prompts) + { +
+

@prompt.Name

+

@prompt.Description

+
+ } +
+ @* A table rather than a list: a resource's example URI is one unbreakable word, and only a scroller + keeps it from widening a phone's page. *@ +
+ + + + @foreach (var resource in _catalog.Resources) + { + + + + + } + +
ResourceWhat it holds
@resource.Name@resource.Description
+
+ } +
+ +
+

The same tools over plain HTTP

+

Open one in a browser tab to read exactly what an agent is handed.

+
+ + + + @foreach (var link in HttpLinks) + { + + + + + } + +
ToolExample
@ToolOf(link)@Uri.UnescapeDataString(link)
+
+
+
+ +@code { + // The response shapes of the endpoints this page calls. They mirror the server's DTOs rather than sharing + // them: the client project does not reference the host, and these are the fields the page renders, not the + // whole contract. Arrays are nullable throughout: a member the server leaves out arrives as null. + private sealed record StoragePlan(bool Opens, string? Provider, string? EngineBuild, bool? IsFirstChoice, bool? WasDecidedByExistingData, + Migration? Migration, Candidate[]? Candidates, string? Error, string? Configuration, string[]? Log, string[]? Notes); + + private sealed record Migration(string From, string To, string Outcome, bool? Copied, string? Error); + + private sealed record Candidate(string Name, int? Rank, bool? Persistent, bool? Available, string? Status, string? Explanation); + + private sealed record SqlCheck(SqlReport[]? Statements, string? Error); + + private sealed record SqlReport(string Sql, Verdict[]? Backends, string[]? Tables); + + private sealed record Verdict(string Name, bool Accepted, string? Error); + + private sealed record SearchResult(Hit[]? Hits, string? Message); + + private sealed record Hit(string Kind, string Title, string Tool); + + private sealed record McpCatalog(CatalogMember[] Tools, CatalogMember[] Prompts, CatalogMember[] Resources); + + private sealed record CatalogMember(string Name, string? Title, string Description, string[]? Parameters); + + private static readonly string[] Backends = ["opfs", "indexeddb", "cache-storage", "in-memory"]; + + private static readonly string[] PreferenceOrders = + [ + "opfs, indexeddb", + "opfs, indexeddb, in-memory", + "opfs, indexeddb, cache-storage, in-memory", + "cache-storage", + "cache-storage, opfs", + ]; + + private static readonly string[] MigrationModes = ["KeepExisting", "Manual", "AutomaticOnOpen"]; + + private static readonly string[] SqlSamples = + [ + "PRAGMA journal_mode = WAL", + "ATTACH DATABASE 'other.db' AS other", + "PRAGMA page_size = 8192", + "DELETE FROM Orders", + ]; + + private static readonly string[] HttpLinks = + [ + "api/mcp/SearchBsqlite?query=live%20query", + "api/mcp/GetBsqliteSetupGuide?scenario=wasm", + "api/mcp/GetBsqliteDocument?name=storage§ion=Selection", + "api/mcp/GetBsqliteApi?typeName=BsqliteDatabase", + "api/mcp/GetBsqliteStorageProviders?provider=opfs", + "api/mcp/PlanBsqliteStorageSelection?unavailable=opfs", + "api/mcp/CheckBsqliteSql?sql=PRAGMA%20page_size%20%3D%208192", + "api/mcp/GetBsqliteSourceFile?path=Demo/Client/Components/SessionLifecycle.razor", + ]; + + /// What every panel says when a call to the demo's own API comes back with nothing. + private const string Unreachable = "The demo's own API did not answer. Is the server still running?"; + + private string _order = PreferenceOrders[0]; + private readonly HashSet _unavailable = []; + private string _binding = string.Empty; + private string _mode = MigrationModes[0]; + private bool _allowNonPersistent; + private bool _planning; + private string? _planError; + private StoragePlan? _plan; + + private string _sql = "SELECT * FROM Orders o JOIN Customers c ON c.Id = o.CustomerId"; + private bool _checking; + private string? _checkError; + private SqlCheck? _check; + + private string _query = "keep the data when opfs appears"; + private bool _searching; + private string? _searchError; + private SearchResult? _search; + + private McpCatalog? _catalog; + + // The prerendered page is on screen before the runtime that would answer its buttons has loaded. + private bool IsHydrating => RendererInfo.IsInteractive is false; + + // "This site" resolves against the address the page is served from, so on localhost, a tunnel or a + // deployment the snippet points at the server being looked at. + private string Endpoint => Nav.ToAbsoluteUri("mcp").ToString(); + + private string ClientConfig => $$""" + { + "mcpServers": { + "bsqlite": { + "type": "http", + "url": "{{Endpoint}}" + } + } + } + """; + + protected override async Task OnAfterRenderAsync(bool firstRender) + { + // Every call on this page goes to the app's own API, which prerendering has no way to reach and no + // reason to: the catalog is chrome, not content. + if (firstRender is false || IsHydrating) return; + + _catalog = await GetAsync("api/mcp/GetMcpCatalog"); + + StateHasChanged(); + } + + private static void Toggle(HashSet set, string value, ChangeEventArgs e) + { + if (e.Value is true) set.Add(value); + else set.Remove(value); + } + + // Each handler resets its own flag in a finally: the flag disables the panel's controls while a call is + // out, so losing it would leave the panel disabled with no way back but a reload. Each also clears its own + // error first, so a panel never shows a stale result under a fresh question. + + private async Task PlanAsync() + { + _planning = true; + _planError = null; + + try + { + var url = $"api/mcp/PlanBsqliteStorageSelection?candidates={Uri.EscapeDataString(_order)}" + + $"&unavailable={Uri.EscapeDataString(string.Join(",", Backends.Where(_unavailable.Contains)))}" + + $"&migrationMode={_mode}&allowNonPersistentFallback={(_allowNonPersistent ? "true" : "false")}" + + (_binding.Length > 0 ? $"&existingBinding={Uri.EscapeDataString(_binding)}" : string.Empty); + + _plan = await GetAsync(url); + _planError = _plan is null ? Unreachable : null; + } + finally + { + _planning = false; + } + } + + private Task CheckAsync() => CheckAsync(_sql); + + private async Task CheckAsync(string sql) + { + _sql = sql; + _checking = true; + _checkError = null; + + try + { + _check = await GetAsync($"api/mcp/CheckBsqliteSql?sql={Uri.EscapeDataString(sql)}"); + _checkError = _check is null ? Unreachable : null; + } + finally + { + _checking = false; + } + } + + private async Task SearchAsync() + { + _searching = true; + _searchError = null; + + try + { + _search = await GetAsync($"api/mcp/SearchBsqlite?query={Uri.EscapeDataString(_query)}&limit=8"); + _searchError = _search is null ? Unreachable : null; + } + finally + { + _searching = false; + } + } + + /// + /// Calls one of the demo's own MCP endpoints, answering null when it cannot be reached or did not come back + /// in the expected shape. Every caller turns that into its own panel saying so: leaving the last answer on + /// screen would read as the answer to the question just asked. + /// + private async Task GetAsync(string url) + { + try + { + return await Http.GetFromJsonAsync(url); + } + catch (Exception exception) + { + SampleLog.Exception(exception); + return default; + } + } + + /// "api/mcp/GetBsqliteApi?typeName=..." is the GetBsqliteApi tool. + private static string ToolOf(string link) => link["api/mcp/".Length..link.IndexOf('?', StringComparison.Ordinal)]; + + private static string YesNo(bool? value) => value is null ? "-" : SampleFormat.YesNo(value.Value); + + /// + /// A tool's description is written for a model; the table shows the sentence that says what it is. An + /// "e.g." ends no sentence, and a sentence that runs on past a couple of lines is cut at a word. + /// + private static string FirstSentence(string description) + { + var stop = description.IndexOf(". ", StringComparison.Ordinal); + + while (stop > 0 && (description[..(stop + 1)].EndsWith("e.g.", StringComparison.Ordinal) || description[..(stop + 1)].EndsWith("i.e.", StringComparison.Ordinal))) + { + stop = description.IndexOf(". ", stop + 1, StringComparison.Ordinal); + } + + var sentence = stop > 0 ? description[..(stop + 1)] : description; + + if (sentence.Length <= 240) return sentence; + + var cut = sentence.LastIndexOf(' ', 237); + + return $"{sentence[..(cut > 0 ? cut : 237)]}..."; + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/NotFoundPage.razor b/src/Bsqlite/Demo/Client/Pages/NotFoundPage.razor new file mode 100644 index 00000000000..f5ffde1b49f --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/NotFoundPage.razor @@ -0,0 +1,16 @@ +@page "/not-found" + +@* The router shows this for an unknown route in the browser, and the host re-executes a deep link to one + through the same route (UseStatusCodePagesWithReExecute), keeping the status code at 404. *@ + +Not found · Bsqlite + +
+

404

+

That page is not in this demo

+

+ The sidebar lists every demo. If you followed a link from the documentation, the + page may have been renamed - the overview links to all of them. +

+
+Back to the overview diff --git a/src/Bsqlite/Demo/Client/Pages/Orders.razor b/src/Bsqlite/Demo/Client/Pages/Orders.razor new file mode 100644 index 00000000000..6c6c4d2bfad --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Orders.razor @@ -0,0 +1,413 @@ +@page "/orders" +@inject BrowserDatabase Database + + + + + A master-detail graph loaded in one query: header, customer, lines, and each line's product. + Status is an enum stored as TEXT, OrderedAt is a DateTimeOffset, and + both foreign keys are Restrict - so the engine, not the UI, decides what may be + deleted. + + +@if (_error is not null) +{ +

@_error

+} + +
+
+

Orders

+

+ Try + + Move an order to Shipped. The SQL page holds a live + query over this table - open it in a second tab and watch it re-run. + +

+
+ + + +
+
+ + + + + + @foreach (var order in Visible) + { + + + + + + + + + } + +
NumberCustomerStatusOrderedShip byTotal
@order.Number@order.Customer?.DisplayName@order.Status@SampleFormat.When(order.OrderedAt)@SampleFormat.Day(order.ShipBy)@SampleFormat.Money(order.Lines.Sum(l => l.UnitPrice * l.Quantity))
+
+ +
+
+ @if (_selected is null) + { +

Detail

+

Select an order, or create one.

+ } + else + { +

@_selected.Number

+
+ + + + +
+ + +
+
+

Lines

+ + + + @foreach (var line in _selected.Lines) + { + + + + + + + } + +
ProductQtyUnit
@(line.Product?.Name ?? $"#{line.ProductId}")@line.Quantity@SampleFormat.Money(line.UnitPrice) + +
+
+ + + +
+ @if (_selected.Id == 0) + { +

Save the header before adding lines.

+ } + } +
+
+ +
+ + + + The header is saved first because the lines need its generated key, and the unit + price is copied onto the line so later price changes do not rewrite history. Both + saves go through the same connection in the same worker. + + + + + + + + A DateTimeOffset round-trips correctly, but SQLite has no type that + can order one, so the stock SQLite provider refuses ORDER BY on it - + on a server exactly as here. Bsqlite does not paper over that: same provider, + same rules. + + + + + + + + Status is stored as TEXT so an exported .db stays readable and + reordering the enum cannot silently change history. Deleting a customer who has + orders, or a product on a line, is refused by Restrict. + + + +
+ +@code { + private AppDbContext? _context; + private List _orders = []; + private List _customers = []; + private List _products = []; + private SalesOrder? _selected; + private OrderStatus _status; + private string _filter = string.Empty; + private int _newProductId; + private int _newQty = 1; + private bool _busy; + private string? _error; + + private IEnumerable Visible + { + get + { + if (string.IsNullOrWhiteSpace(_filter)) + { + return _orders; + } + + var term = _filter.Trim(); + return _orders.Where(o => + o.Number.Contains(term, StringComparison.OrdinalIgnoreCase) + || (o.Customer?.DisplayName.Contains(term, StringComparison.OrdinalIgnoreCase) ?? false)); + } + } + + private static string StatusClass(OrderStatus status) => status switch + { + OrderStatus.Shipped => "ok", + OrderStatus.Cancelled => "danger", + OrderStatus.Draft => "muted", + OrderStatus.Packed => "warn", + _ => string.Empty, + }; + + private async Task LoadAsync() + { + _error = null; + try + { + _context = await Database.OpenContextAsync(); + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task UnloadAsync() + { + _selected = null; + if (_context is not null) + { + await _context.DisposeAsync(); + _context = null; + } + } + + private async Task RefreshAsync(int? keepId = null) + { + if (_context is null) + { + return; + } + + _customers = await _context.Customers.AsNoTracking().OrderBy(c => c.DisplayName).ToListAsync(); + _products = await _context.Products.AsNoTracking().OrderBy(p => p.Sku).ToListAsync(); + _orders = await _context.Orders + .AsNoTracking() + .Include(o => o.Customer) + .Include(o => o.Lines) + .ThenInclude(l => l.Product) + .OrderByDescending(o => o.Id) + .ToListAsync(); + _newProductId = _products.FirstOrDefault()?.Id ?? 0; + var id = keepId ?? _selected?.Id; + _selected = id is int found ? _orders.FirstOrDefault(o => o.Id == found) : null; + if (_selected is not null) + { + _status = _selected.Status; + } + } + + private void NewOrder() + { + var customerId = _customers.FirstOrDefault()?.Id ?? 0; + _selected = new SalesOrder + { + Number = "SO-" + DateTime.UtcNow.ToString("HHmmss"), + CustomerId = customerId, + Status = OrderStatus.Draft, + OrderedAt = DateTimeOffset.Now, + ShipBy = DateOnly.FromDateTime(DateTime.UtcNow.AddDays(7)), + }; + _status = OrderStatus.Draft; + } + + private void Select(SalesOrder order) + { + _selected = order; + _status = order.Status; + } + + private async Task SaveHeaderAsync() + { + if (_context is null || _selected is null) + { + return; + } + + _busy = true; + _error = null; + try + { + _selected.Status = _status; + if (_selected.Id == 0) + { + _context.Orders.Add(_selected); + } + else + { + var tracked = await _context.Orders.FirstAsync(o => o.Id == _selected.Id); + tracked.CustomerId = _selected.CustomerId; + tracked.Status = _selected.Status; + tracked.ShipBy = _selected.ShipBy; + tracked.Notes = _selected.Notes ?? string.Empty; + } + + await _context.SaveChangesAsync(); + var id = _selected.Id; + _context.ChangeTracker.Clear(); + await RefreshAsync(id); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } + + private async Task AddLineAsync() + { + if (_context is null || _selected is null || _selected.Id == 0) + { + return; + } + + var product = _products.FirstOrDefault(p => p.Id == _newProductId); + if (product is null) + { + return; + } + + _busy = true; + _error = null; + try + { + _context.OrderLines.Add(new OrderLine + { + OrderId = _selected.Id, + ProductId = product.Id, + Quantity = Math.Max(1, _newQty), + UnitPrice = product.Price, + }); + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + await RefreshAsync(_selected.Id); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } + + private async Task RemoveLineAsync(OrderLine line) + { + if (_context is null || _selected is null) + { + return; + } + + _busy = true; + _error = null; + try + { + var tracked = await _context.OrderLines.FirstAsync(l => l.Id == line.Id); + _context.OrderLines.Remove(tracked); + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + await RefreshAsync(_selected.Id); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } + + private async Task DeleteOrderAsync() + { + if (_context is null || _selected is null || _selected.Id == 0) + { + return; + } + + _busy = true; + _error = null; + try + { + var tracked = await _context.Orders.FirstAsync(o => o.Id == _selected.Id); + _context.Orders.Remove(tracked); + await _context.SaveChangesAsync(); + _context.ChangeTracker.Clear(); + _selected = null; + await RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + _context.ChangeTracker.Clear(); + } + finally + { + _busy = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/Queries.razor b/src/Bsqlite/Demo/Client/Pages/Queries.razor new file mode 100644 index 00000000000..dec3aa22c11 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Queries.razor @@ -0,0 +1,291 @@ +@page "/queries" +@inject BrowserDatabase Database + + + + + SQL generation here is the stock EF Core SQLite provider, untouched. Bsqlite replaces + only the two services that assume synchronous I/O, and registers on every connection the same + ef_* functions, EF_DECIMAL collation, and regexp that EF + expects from a native SqliteConnection. Each panel below runs against this tab's + database and shows exactly what it ran. + + +@if (_error is not null) +{ +

@_error

+} + +
+ + + + Live queries are table-level: Bsqlite reads the tables out of the generated + SQL, then re-executes the query when any of them is written - by this tab or, over + a BroadcastChannel, by another one. Include is honoured, + so the customer below comes along. + + + + + Open Orders in a second tab and change a status. This table + updates without a refresh and without polling. + + + + @if (_liveRows.Count == 0) + { +

No orders yet.

+ } + else + { +
+ + + + @foreach (var order in _liveRows) + { + + + + + + + } + +
NumberCustomerStatusOrdered
@order.Number@order.Customer?.DisplayName@order.Status@SampleFormat.When(order.OrderedAt)
+
+ } + + + + + + + decimal is not a SQLite affinity, so EF stores it as TEXT and compares + it with ef_compare under the EF_DECIMAL collation. Those + functions run on 96-bit coefficients inside the worker - not on JavaScript numbers, + which would quietly round. + + + + Raise the threshold and run it: the generated SQL appears underneath. + + +
+ + +
+ @if (_decimalRows is not null) + { +

@(_decimalRows.Count) customer(s) matched.

+
    + @foreach (var customer in _decimalRows) + { +
  • @customer.DisplayName - @SampleFormat.Money(customer.CreditLimit)
    • + } +
+ } + + + @if (_decimalSql is not null) + { + + } + + + + + + + SQLite ships no regexp implementation; a host has to supply one. The + worker registers it on every connection, so the pattern is a JavaScript + RegExp - lookaround and backreferences included. + + + + Try ^a, or northwind|alpine, against the seeded emails. + + +
+ + +
+ @if (_regexRows is not null) + { +

@(_regexRows.Count) row(s) matching @_pattern.

+
    + @foreach (var customer in _regexRows) + { +
  • @customer.DisplayName - @customer.Email
    • + } +
+ } + + + + + + + The session exposes a DbConnection that is already open. Commands, + parameters, and readers behave as you expect, with one rule: the async overloads + only. The version below is read from wa-sqlite in the worker, not from a native + e_sqlite3. + + + + + @if (_sqliteVersion is not null) + { +

sqlite_version() = @_sqliteVersion

+ } + + + + + +
+ +@code { + private AppDbContext? _context; + private IBsqliteLiveQuery>? _live; + private IReadOnlyList _liveRows = []; + private IReadOnlyList? _decimalRows; + private IReadOnlyList? _regexRows; + private string? _decimalSql; + private string? _sqliteVersion; + private string? _error; + private decimal _minCredit = 10000m; + private string _pattern = "northwind|alpine"; + private bool _busy; + + private async Task LoadAsync() + { + _error = null; + try + { + _context = await Database.OpenContextAsync(); + _live = _context.Orders + .Include(o => o.Customer) + .OrderByDescending(o => o.Id) + .AsLiveQuery(); + _live.Changed += OnChanged; + _liveRows = await _live.RefreshAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task UnloadAsync() + { + if (_live is not null) + { + _live.Changed -= OnChanged; + await _live.DisposeAsync(); + _live = null; + } + + if (_context is not null) + { + await _context.DisposeAsync(); + _context = null; + } + } + + private void OnChanged(object? sender, IReadOnlyList rows) + { + _liveRows = rows; + _ = InvokeAsync(StateHasChanged); + } + + private async Task FilterDecimalAsync() + { + if (_context is null) + { + return; + } + + _busy = true; + _error = null; + try + { + var query = _context.Customers.AsNoTracking() + .Where(c => c.CreditLimit >= _minCredit) + .OrderBy(c => c.DisplayName); + _decimalSql = query.ToQueryString(); + _decimalRows = await query.ToListAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } + + private async Task FilterRegexAsync() + { + if (_context is null) + { + return; + } + + _busy = true; + _error = null; + try + { + _regexRows = await _context.Customers + .FromSql($"SELECT * FROM \"Customers\" WHERE \"Email\" REGEXP {_pattern}") + .AsNoTracking() + .ToListAsync(); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } + + private async Task ReadVersionAsync() + { + _busy = true; + _error = null; + try + { + _sqliteVersion = Convert.ToString(await Database.ExecuteScalarAsync("SELECT sqlite_version()")); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Pages/Storage.razor b/src/Bsqlite/Demo/Client/Pages/Storage.razor new file mode 100644 index 00000000000..9597549b163 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Pages/Storage.razor @@ -0,0 +1,305 @@ +@page "/storage" +@inject BrowserDatabase Database +@inject IJSRuntime Js + + + + + Storage is a public contract, not an implementation detail: OPFS, IndexedDB, Cache Storage and + in-memory are interchangeable providers behind the same interface. Selection prefers the order + you configure, but existing data outranks preference - a database created on IndexedDB does not + silently vanish the day OPFS becomes available. + + +@if (_error is not null) +{ +

@_error

+} + +
+ + + + Every registered provider is a candidate. AllowNonPersistentFallback() + is what makes in-memory acceptable at all - without it, an origin with no persistent + backend fails to open rather than quietly losing data on reload. + + + + + The session bar above is this API in action: + Move data migrates the image, Start empty + binds a fresh file without copying. + + + + + + + + + + A resolution is a record of the whole decision, not just its outcome: every candidate, + its probe, and the reason it was passed over. That is what makes a support question + answerable without reproducing the user's browser. + + + @if (_resolution is not null) + { +

+ Chosen backend: @_resolution.Provider.Name + @if (_resolution.IsFirstChoice) + { + - first choice in the configured order. + } + else + { + - a fallback; see the attempt table. + } + @if (_resolution.WasDecidedByExistingData) + { + The file was already bound here, so preference did not move it. + } +

+ @if (_resolution.BetterProviderAvailable is { } better) + { +

+ @better.Name is available and ranked higher, but the data lives here. + BsqliteStorageMigrationMode.AutomaticOnOpen would copy, check the + image, then flip the binding. This demo uses the default, + KeepExisting. +

+ } +

+ Sticky binding in localStorage: + @(_binding ?? "(none - first create on this origin)") +

+
+ + + + + + @foreach (var attempt in _resolution.Attempts) + { + + + + + + } + +
ProviderStatusExplanation
@attempt.ProviderName@attempt.Status@(attempt.Explanation ?? attempt.Probe?.UnavailableReason ?? "-")
+
+ } + + + + + + + The core enforces these flags; it never special-cases OPFS against IndexedDB in + SQL. A backend that cannot honour ATTACH or + PRAGMA page_size fails in BsqliteFeatureGuards instead of + corrupting the file - see Limits. + + + + @if (_caps is not null) + { +
+ + + + + + + + + + + +
Engine build@_caps.RequiredBuild
Survives reload@SampleFormat.YesNo(_caps.IsPersistent)
Multiple connections@SampleFormat.YesNo(_caps.SupportsMultipleConnections)
Concurrent reads@SampleFormat.YesNo(_caps.SupportsConcurrentReads)
Relaxed durability@SampleFormat.YesNo(_caps.SupportsRelaxedDurability)
Multi-database transactions@SampleFormat.YesNo(_caps.SupportsMultiDatabaseTransactions)
Can change page size@SampleFormat.YesNo(_caps.CanChangePageSize)
Contexts@_caps.SupportedContexts
+
+ } + + + + + + A probe reports availability, quota when the browser will say, and the raw environment + facts behind the verdict. Unavailable is a result, never an exception, so selection can + continue and still explain itself afterwards. + + + Probe now, then compare the numbers after importing a larger database on Admin. + + + + @if (_probes.Count > 0) + { +
+ + + + + + @foreach (var row in _probes) + { + + + + + + + } + +
ProviderAvailableUsage / quotaWhy / environment
@row.Name@SampleFormat.YesNo(row.Probe.IsAvailable) + @SampleFormat.Bytes(row.Probe.UsageBytes) + / + @SampleFormat.Bytes(row.Probe.QuotaBytes) + + @if (!row.Probe.IsAvailable) + { + @row.Probe.UnavailableReason + } + else if (row.Probe.Environment.Count == 0) + { + - + } + else + { + @string.Join(", ", row.Probe.Environment.Select(kv => $"{kv.Key}={kv.Value}")) + } +
+
+ } + + + + + + Persistent storage is granted per origin, not per backend. Without it, any of the + persistent backends may be cleared under storage pressure - which is a product + decision, so the library exposes the call rather than making it for you. + + +

+ Currently: + @(_persisted is null ? "not requested yet" : _persisted.Value ? "granted" : "denied") +

+ + + + +
+

The four backends

+
+ + + + @foreach (var provider in Database.Providers) + { + + + + + } + +
ProviderCharacter
+ + @SampleStorage.Title(provider.Name) + @provider.Name + @SampleStorage.Blurb(provider.Name)
+
+
+
+ +@code { + private BsqliteStorageResolution? _resolution; + private BsqliteStorageCapabilities? _caps; + private string? _binding; + private bool? _persisted; + private string? _error; + private bool _busy; + private readonly List<(string Name, BsqliteProbeResult Probe)> _probes = []; + + private static Task NoopAsync() => Task.CompletedTask; + + private async Task LoadAsync() + { + try + { + var session = await Database.EnsureSessionAsync(); + _resolution = session.Resolution; + _caps = session.Resolution.Provider.Capabilities; + _binding = await Database.Bindings.GetProviderNameAsync(BrowserDatabase.DatabaseName); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + } + + private async Task RequestPersistenceAsync() + { + _busy = true; + _error = null; + try + { + _persisted = await BsqlitePersistence.RequestAsync(Js); + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } + + private async Task ProbeAllAsync() + { + _busy = true; + _error = null; + _probes.Clear(); + try + { + foreach (var provider in Database.Providers) + { + var probe = await provider.ProbeAsync(); + _probes.Add((provider.Name, probe)); + } + } + catch (Exception ex) + { + SampleLog.Exception(ex); + _error = ex.Message; + } + finally + { + _busy = false; + } + } +} diff --git a/src/Bsqlite/Demo/Client/Program.cs b/src/Bsqlite/Demo/Client/Program.cs new file mode 100644 index 00000000000..2fd2b54f340 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Program.cs @@ -0,0 +1,26 @@ +// The WebAssembly half of the demo: the same service registration as the server, through AddDemoServices. +// Every database this site opens lives here, in the browser tab. + +using Bit.Bsqlite.Demo.Client; +using Microsoft.AspNetCore.Components.WebAssembly.Hosting; + +var builder = WebAssemblyHostBuilder.CreateDefault(args); + +builder.Services.AddDemoServices(builder.HostEnvironment.BaseAddress); + +TaskScheduler.UnobservedTaskException += (_, args) => SampleLog.Exception(args.Exception); +AppDomain.CurrentDomain.UnhandledException += (_, args) => +{ + if (args.ExceptionObject is Exception exception) + { + SampleLog.Exception(exception); + } + else + { + Console.Error.WriteLine($"[Bsqlite demo] Unhandled: {args.ExceptionObject}"); + } +}; + +// No RootComponents are registered on purpose: the host page drives the app through blazor.web.js +// render-mode markers (). +await builder.Build().RunAsync(); diff --git a/src/Bsqlite/Demo/Client/Routes.razor b/src/Bsqlite/Demo/Client/Routes.razor new file mode 100644 index 00000000000..6ed1993864b --- /dev/null +++ b/src/Bsqlite/Demo/Client/Routes.razor @@ -0,0 +1,5 @@ + + + + + diff --git a/src/Bsqlite/Demo/Client/SampleFormat.cs b/src/Bsqlite/Demo/Client/SampleFormat.cs new file mode 100644 index 00000000000..ef91b81bbe4 --- /dev/null +++ b/src/Bsqlite/Demo/Client/SampleFormat.cs @@ -0,0 +1,32 @@ +namespace Bit.Bsqlite.Demo.Client; + +internal static class SampleFormat +{ + public static string YesNo(bool value) => value ? "yes" : "no"; + + public static string Bytes(long? value) + => value is null ? "unknown" : Bytes(value.Value); + + public static string Bytes(long value) + => value switch + { + < 1024 => $"{value} B", + < 1024 * 1024 => $"{value / 1024.0:0.#} KB", + _ => $"{value / (1024.0 * 1024.0):0.#} MB", + }; + + public static string Money(decimal value) => value.ToString("N2"); + + public static string When(DateTime value) => value.ToString("yyyy-MM-dd HH:mm"); + + public static string When(DateTimeOffset value) => value.ToString("yyyy-MM-dd HH:mm zzz"); + + public static string Day(DateOnly? value) => value?.ToString("yyyy-MM-dd") ?? "-"; + + public static string Day(DateOnly value) => value.ToString("yyyy-MM-dd"); + + public static string Lead(TimeSpan value) + => value.TotalDays >= 1 && value.TotalDays == Math.Floor(value.TotalDays) + ? $"{(int)value.TotalDays}d" + : value.ToString("c"); +} diff --git a/src/Bsqlite/Demo/Client/SampleLog.cs b/src/Bsqlite/Demo/Client/SampleLog.cs new file mode 100644 index 00000000000..85f419a0765 --- /dev/null +++ b/src/Bsqlite/Demo/Client/SampleLog.cs @@ -0,0 +1,62 @@ +using System.Collections; +using System.Reflection; + +namespace Bit.Bsqlite.Demo.Client; + +/// Writes the full exception graph to the browser console. +internal static class SampleLog +{ + public static void Exception(Exception exception) + { + ArgumentNullException.ThrowIfNull(exception); + Write(exception, depth: 0); + } + + private static void Write(Exception exception, int depth) + { + var prefix = depth == 0 ? "[Bsqlite sample]" : new string(' ', depth * 2); + Console.Error.WriteLine($"{prefix} {exception.GetType().FullName}: {exception.Message}"); + Console.Error.WriteLine($"{prefix} HResult=0x{exception.HResult:X8} Source={exception.Source}"); + if (!string.IsNullOrEmpty(exception.HelpLink)) + { + Console.Error.WriteLine($"{prefix} HelpLink={exception.HelpLink}"); + } + + foreach (DictionaryEntry entry in exception.Data) + { + Console.Error.WriteLine($"{prefix} Data[{entry.Key}]={entry.Value}"); + } + + if (exception.StackTrace is { Length: > 0 } stack) + { + Console.Error.WriteLine(stack); + } + + switch (exception) + { + case AggregateException aggregate: + foreach (var inner in aggregate.Flatten().InnerExceptions) + { + Write(inner, depth + 1); + } + + return; + case ReflectionTypeLoadException load: + foreach (var loader in load.LoaderExceptions) + { + if (loader is not null) + { + Write(loader, depth + 1); + } + } + + break; + } + + if (exception.InnerException is not null && exception is not AggregateException) + { + Console.Error.WriteLine($"{prefix} Inner exception:"); + Write(exception.InnerException, depth + 1); + } + } +} diff --git a/src/Bsqlite/Demo/Client/SampleStorage.cs b/src/Bsqlite/Demo/Client/SampleStorage.cs new file mode 100644 index 00000000000..52895cc7141 --- /dev/null +++ b/src/Bsqlite/Demo/Client/SampleStorage.cs @@ -0,0 +1,24 @@ +namespace Bit.Bsqlite.Demo.Client; + +internal static class SampleStorage +{ + public static string Title(string name) => name switch + { + "opfs" => "OPFS", + "indexeddb" => "IndexedDB", + "cache-storage" => "Cache Storage", + "in-memory" => "In-memory", + _ => name, + }; + + public static string Blurb(string name) => name switch + { + "opfs" => "Fastest persistent tier. Real files in the origin private file system. Concurrent connections, no COOP/COEP.", + "indexeddb" => "Widest reach. Survives browsers that do not have OPFS. Relaxed durability available. Page size is pinned.", + "cache-storage" => "The besql-compatible fallback. Persistence cost scales with database size.", + "in-memory" => "Volatile. Gone on reload. Useful to try the engine without touching disk.", + _ => "Registered storage provider.", + }; + + public static string PersistLabel(bool persistent) => persistent ? "survives reload" : "lost on reload"; +} diff --git a/src/Bsqlite/Demo/Client/SiteLinks.cs b/src/Bsqlite/Demo/Client/SiteLinks.cs new file mode 100644 index 00000000000..14e66674d9f --- /dev/null +++ b/src/Bsqlite/Demo/Client/SiteLinks.cs @@ -0,0 +1,23 @@ +using System.Reflection; +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Demo.Client; + +/// Where the site points when it leaves itself, plus the version it is showing off. +internal static class SiteLinks +{ + /// + /// The Bit.Bsqlite build this site runs. The informational version's build metadata - everything after + /// '+', usually the commit hash - is dropped: it identifies nothing a reader could act on. + /// + public static readonly string Version = ( + typeof(BsqliteConnection).Assembly.GetCustomAttribute()?.InformationalVersion + ?? typeof(BsqliteConnection).Assembly.GetName().Version?.ToString() + ?? "unknown").Split('+')[0]; + + public const string Repository = "https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite"; + public const string Issues = "https://github.com/bitfoundation/bitplatform/issues"; + public const string License = "https://github.com/bitfoundation/bitplatform/blob/develop/LICENSE"; + public const string NuGet = "https://www.nuget.org/packages/Bit.Bsqlite"; + public const string Readme = "https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite#readme"; +} diff --git a/src/Bsqlite/Demo/Client/SiteMap.cs b/src/Bsqlite/Demo/Client/SiteMap.cs new file mode 100644 index 00000000000..7dc5cf95fa5 --- /dev/null +++ b/src/Bsqlite/Demo/Client/SiteMap.cs @@ -0,0 +1,56 @@ +namespace Bit.Bsqlite.Demo.Client; + +/// One page of the sample site. +/// Relative route, without a leading slash. +/// Label in the sidebar and on the overview grid. +/// Name understood by Components/Icon.razor. +/// One line describing what the page proves. +internal sealed record SamplePage(string Href, string Title, string Icon, string Summary); + +/// A titled run of pages in the sidebar. +internal sealed record SampleSection(string Title, IReadOnlyList Pages); + +/// +/// The navigation model, declared once. The sidebar, the overview page's demo grid, and the +/// document title all read from here, so adding a page cannot leave one of them behind. +/// +internal static class SiteMap +{ + public static SamplePage Overview { get; } = new( + "", "Overview", "home", + "What the library is, how it is wired up, and a catalog that re-runs itself."); + + public static IReadOnlyList Sections { get; } = + [ + new("Start here", [Overview]), + new("Model and data", + [ + new("catalog", "Catalog", "boxes", + "CRUD over a related graph: unique SKU, decimal price, TimeSpan lead time, nullable DateOnly."), + new("customers", "Customers", "users", + "Guid keys, a unique index the engine enforces, VIP flag, decimal credit limits."), + new("orders", "Orders", "receipt", + "Master-detail with Include and ThenInclude, string-backed enum, DateTimeOffset, restrict FKs."), + ]), + new("Engine and platform", + [ + new("queries", "SQL and live queries", "terminal", + "Generated SQL, ef_compare on decimals, REGEXP, and plain ADO.NET on the same connection."), + new("storage", "Storage backends", "drive", + "How a backend is chosen, what each one can do, quota probes, persistent-storage permission."), + new("admin", "Database files", "files", + "Databases as files: list, export to a .db, import one back, delete."), + new("limits", "Limits", "warning", + "What the web cannot do - sync APIs, WAL, ATTACH, page size - and exactly how it fails."), + ]), + new("For AI agents", + [ + new("mcp-server", "MCP server", "bolt", + "The library's API, guides and storage selection, served to coding agents at /mcp - and tried live here."), + ]), + ]; + + /// Every page except the overview, in sidebar order. + public static IEnumerable Demos => + Sections.SelectMany(s => s.Pages).Where(p => p.Href.Length > 0); +} diff --git a/src/Bsqlite/Demo/Client/Snippets.cs b/src/Bsqlite/Demo/Client/Snippets.cs new file mode 100644 index 00000000000..006ac9211a9 --- /dev/null +++ b/src/Bsqlite/Demo/Client/Snippets.cs @@ -0,0 +1,562 @@ +namespace Bit.Bsqlite.Demo.Client; + +/// +/// The code shown next to each demo. Kept in one place, and kept honest: every snippet is the +/// real call the page above it makes, trimmed only of demo plumbing (busy flags, try/catch). +/// +internal static class Snippets +{ + public static class Setup + { + public const string Install = + """ + dotnet add package Bit.Bsqlite + dotnet add package Bit.Bsqlite.Storage.Opfs + dotnet add package Bit.Bsqlite.Storage.IndexedDb + dotnet add package Bit.Bsqlite.Storage.CacheStorage + """; + + public const string Registration = + """ + // Program.cs - Blazor WebAssembly. One call registers selection, sticky bindings in + // localStorage, the worker transport, and the shared database. + builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .AddStorageProvider() + .AddStorageProvider() + .AddStorageProvider() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteCacheStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()) + .AddDatabase("app.db")); + + // Contexts come from a factory bound to that database. Migrations run once per + // session, then the demo catalog is seeded if the database is empty. + builder.Services.AddBitBsqliteDbContext(options => + options.OnFirstUse = (db, ct) => DemoData.SeedIfEmptyAsync(db, ct)); + """; + + /// The condensed version on the landing page: register, open, query. + public const string Hero = + """ + // Program.cs - one preference order, one database, one context factory. + builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .AddStorageProvider() + .UseStorage(s => s.Prefer("opfs").Fallback("indexeddb")) + .AddDatabase("app.db")); + builder.Services.AddBitBsqliteDbContext(); + + // Anywhere you would have opened a DbContext. Migrations ran on first use. + await using var context = await contexts.CreateDbContextAsync(); + + // From here it is ordinary EF Core - running inside the browser tab. + var open = await context.Orders + .Include(o => o.Customer) + .Where(o => o.Status == OrderStatus.Draft) + .ToListAsync(); + """; + + public const string OpenContext = + """ + // One session per tab, owned by the registered BsqliteDatabase: it holds the + // worker, the transport, and the ADO.NET connection. The factory opens it on first use + // and runs the same migrations that run against the server database, once per session. + await using var context = await contexts.CreateDbContextAsync(cancellationToken); + """; + } + + public static class Home + { + public const string Counts = + """ + await using var context = await Database.OpenContextAsync(); + + _products = await context.Products.CountAsync(); + _customers = await context.Customers.CountAsync(); + _openOrders = await context.Orders.CountAsync(o => + o.Status != OrderStatus.Shipped && o.Status != OrderStatus.Cancelled); + """; + + public const string LiveQuery = + """ + // Table-level: any write to Products re-runs this, including from another tab + // (BroadcastChannel), including from raw SQL that never touched EF. + _live = context.Products + .AsNoTracking() + .OrderBy(p => p.Sku) + .Take(8) + .AsLiveQuery(); + + _live.Changed += (_, rows) => + { + _liveRows = rows; + _ = InvokeAsync(StateHasChanged); + }; + + _liveRows = await _live.RefreshAsync(); + + // Disposing the live query unsubscribes it from the connection's change feed. + await _live.DisposeAsync(); + """; + + public const string Entity = + """ + public sealed class Product + { + public int Id { get; set; } + public Guid PublicId { get; set; } = Guid.NewGuid(); // TEXT + public string Sku { get; set; } = string.Empty; // TEXT + public decimal Price { get; set; } // TEXT + ef_compare + public double WeightKg { get; set; } // REAL + public bool IsActive { get; set; } = true; // INTEGER 0/1 + public DateTime CreatedUtc { get; set; } // TEXT, ISO 8601 + public DateOnly? DiscontinuedOn { get; set; } // TEXT date, nullable + public TimeSpan LeadTime { get; set; } // TEXT, "c" format + public int CategoryId { get; set; } + public Category? Category { get; set; } + } + """; + } + + public static class Catalog + { + public const string Read = + """ + _categories = await context.Categories + .AsNoTracking() + .OrderBy(c => c.Name) + .ToListAsync(); + + _products = await context.Products + .AsNoTracking() + .Include(p => p.Category) + .OrderBy(p => p.Sku) + .ToListAsync(); + """; + + public const string Write = + """ + product.LeadTime = TimeSpan.FromDays(_leadDays); + if (!product.IsActive && product.DiscontinuedOn is null) + { + product.DiscontinuedOn = DateOnly.FromDateTime(DateTime.UtcNow); + } + + if (product.Id == 0) + { + context.Products.Add(product); + } + else + { + context.Products.Update(product); + } + + // Async only. SaveChanges() throws BsqliteSynchronousApiNotSupportedException. + await context.SaveChangesAsync(); + context.ChangeTracker.Clear(); + """; + + public const string Delete = + """ + // OrderLine -> Product is DeleteBehavior.Restrict, so deleting a product that is on an + // order raises SQLITE_CONSTRAINT_FOREIGNKEY inside the worker and surfaces as a + // DbUpdateException here - foreign keys are enforced, not simulated. + var tracked = await context.Products.FirstAsync(p => p.Id == product.Id); + context.Products.Remove(tracked); + await context.SaveChangesAsync(); + """; + + public const string Configuration = + """ + public sealed class ProductConfiguration : IEntityTypeConfiguration + { + public void Configure(EntityTypeBuilder builder) + { + builder.HasKey(p => p.Id); + builder.HasIndex(p => p.PublicId).IsUnique(); + builder.Property(p => p.Sku).IsRequired().HasMaxLength(32); + builder.HasIndex(p => p.Sku).IsUnique(); + builder.Property(p => p.Price).HasPrecision(18, 2); + builder.HasOne(p => p.Category) + .WithMany(c => c.Products) + .HasForeignKey(p => p.CategoryId) + .OnDelete(DeleteBehavior.Cascade); + } + } + """; + } + + public static class Customers + { + public const string Entity = + """ + public sealed class Customer + { + public int Id { get; set; } + public Guid PublicId { get; set; } = Guid.NewGuid(); + public string DisplayName { get; set; } = string.Empty; + public string Email { get; set; } = string.Empty; + public DateOnly DateOfBirth { get; set; } + public bool IsVip { get; set; } + public decimal CreditLimit { get; set; } + public string Notes { get; set; } = string.Empty; + public DateTime CreatedUtc { get; set; } = DateTime.UtcNow; + } + + // Precision travels with the model, not with the backend. + builder.Property(c => c.CreditLimit).HasPrecision(18, 2); + builder.HasIndex(c => c.Email).IsUnique(); + """; + + public const string Crud = + """ + _rows = await context.Customers + .AsNoTracking() + .OrderBy(c => c.DisplayName) + .ToListAsync(); + + // Insert: PublicId is a Guid the CLR generated, stored as TEXT and read back as a Guid. + context.Customers.Add(new Customer + { + DisplayName = "Ada Lovelace", + Email = "ada@example.com", + DateOfBirth = new DateOnly(1980, 1, 1), + CreditLimit = 1_000m, + IsVip = true, + }); + + await context.SaveChangesAsync(); + """; + + public const string Unique = + """ + // The unique index on Email is enforced by SQLite in the worker. A duplicate throws + // DbUpdateException wrapping BsqliteException with the SQLite result code. + try + { + await context.SaveChangesAsync(); + } + catch (DbUpdateException ex) + { + _error = ex.InnerException?.Message ?? ex.Message; + context.ChangeTracker.Clear(); + } + """; + } + + public static class Orders + { + public const string Read = + """ + _orders = await context.Orders + .AsNoTracking() + .Include(o => o.Customer) + .Include(o => o.Lines) + .ThenInclude(l => l.Product) + .OrderByDescending(o => o.Id) + .ToListAsync(); + """; + + public const string Configuration = + """ + public sealed class SalesOrderConfiguration : IEntityTypeConfiguration + { + public void Configure(EntityTypeBuilder builder) + { + builder.ToTable("Orders"); + builder.Property(o => o.Number).IsRequired().HasMaxLength(24); + builder.HasIndex(o => o.Number).IsUnique(); + + // Stored as TEXT: readable in an exported file, stable across enum reordering. + builder.Property(o => o.Status).HasConversion().HasMaxLength(16); + + builder.HasOne(o => o.Customer) + .WithMany(c => c.Orders) + .HasForeignKey(o => o.CustomerId) + .OnDelete(DeleteBehavior.Restrict); + } + } + """; + + public const string Graph = + """ + // Header first: the line items need the generated key. + context.Orders.Add(order); + await context.SaveChangesAsync(); + + context.OrderLines.Add(new OrderLine + { + OrderId = order.Id, + ProductId = product.Id, + Quantity = quantity, + UnitPrice = product.Price, // price is captured at line time + }); + + await context.SaveChangesAsync(); + context.ChangeTracker.Clear(); + """; + + public const string Offset = + """ + // DateTimeOffset round-trips as TEXT with its offset, but SQLite cannot compare it, + // so the stock SQLite provider refuses ORDER BY on it. Sort by the key instead: + .OrderByDescending(o => o.Id) + + // ...and do offset-sensitive work in the CLR after materialising: + var latest = _orders.MaxBy(o => o.OrderedAt); + """; + } + + public static class Storage + { + public const string Selection = + """ + BsqliteStorageSelectionBuilder.Create(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteCacheStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()); // without this, in-memory is not acceptable + + // Default is BsqliteStorageMigrationMode.KeepExisting: data outranks preference. + // AutomaticOnOpen copies to the preferred backend, checks the image, then rebinds. + """; + + public const string Resolution = + """ + var session = await database.OpenAsync(); + var resolution = session.Resolution; + + resolution.Provider.Name; // the backend that actually opened + resolution.IsFirstChoice; // false when a fallback won + resolution.WasDecidedByExistingData; // the sticky binding overruled preference + resolution.BetterProviderAvailable?.Name; // ranked higher but holds no data + resolution.Attempts; // every candidate, with the reason it lost + """; + + public const string Probe = + """ + foreach (var provider in Database.Providers) + { + var probe = await provider.ProbeAsync(); + + probe.IsAvailable; // unavailability is a result, never an exception + probe.UnavailableReason; // why, in one sentence + probe.UsageBytes; // when the browser will say + probe.QuotaBytes; + probe.Environment; // raw facts: JSPI, getDirectory, isSecureContext... + } + """; + + public const string Capabilities = + """ + var caps = session.Resolution.Provider.Capabilities; + + caps.IsPersistent; // survives reload + caps.SupportsMultipleConnections; + caps.SupportsConcurrentReads; + caps.SupportsRelaxedDurability; + caps.SupportsMultiDatabaseTransactions; // gates ATTACH + caps.CanChangePageSize; // gates PRAGMA page_size + + // The core reads these flags. It never special-cases a provider by name, so a new + // backend gets the same guards without touching Bsqlite. + """; + + public const string Persistence = + """ + // navigator.storage.persist(). Origin-wide, not per backend: without it the browser + // may evict OPFS / IndexedDB / Cache Storage under pressure. + bool granted = await BsqlitePersistence.RequestAsync(js); + """; + + public const string Migrate = + """ + // Copy the image to another backend: probe target -> quota headroom -> export -> + // import -> SQLite header check -> flip the sticky binding -> delete the source. + // A failure anywhere leaves both the source and the binding untouched. + await new BsqliteStorageMigrator() + .MigrateAsync("app.db", source, target, bindingStore, cancellationToken); + """; + } + + public static class Sql + { + public const string LiveInclude = + """ + _live = context.Orders + .Include(o => o.Customer) + .OrderByDescending(o => o.Id) + .AsLiveQuery(); + + _live.Changed += (_, rows) => + { + _liveRows = rows; + _ = InvokeAsync(StateHasChanged); + }; + + _liveRows = await _live.RefreshAsync(); + """; + + public const string QueryString = + """ + var query = context.Customers + .AsNoTracking() + .Where(c => c.CreditLimit >= _minCredit) + .OrderBy(c => c.DisplayName); + + _decimalSql = query.ToQueryString(); // the SQL below, straight from EF + _decimalRows = await query.ToListAsync(); + """; + + public const string Regexp = + """ + // regexp is registered on every open, so raw SQL can use it too. + _regexRows = await context.Customers + .FromSql($"SELECT * FROM \"Customers\" WHERE \"Email\" REGEXP {_pattern}") + .AsNoTracking() + .ToListAsync(); + """; + + public const string Ado = + """ + // No EF: the session's connection is a DbConnection, already open. + await using var command = session.Connection.CreateCommand(); + command.CommandText = "SELECT sqlite_version()"; + var version = await command.ExecuteScalarAsync(cancellationToken); + + // Parameters bind by CLR type; the binder picks TEXT / INTEGER / REAL / BLOB. + command.CommandText = "SELECT * FROM Products WHERE Price >= $min"; + command.Parameters.Add(new BsqliteParameter { ParameterName = "$min", Value = 25.00m }); + await using var reader = await command.ExecuteReaderAsync(cancellationToken); + while (await reader.ReadAsync(cancellationToken)) + { + var sku = reader.GetString(reader.GetOrdinal("Sku")); + } + """; + + public const string Transaction = + """ + await using var transaction = await session.Connection.BeginTransactionAsync(ct); + try + { + await context.SaveChangesAsync(ct); + await transaction.CommitAsync(ct); + } + catch + { + await transaction.RollbackAsync(ct); + throw; + } + """; + } + + public static class Admin + { + public const string List = + """ + // Databases as files, outside any SQL connection. No EF involved. + foreach (var provider in Database.Providers) + { + IReadOnlyList names = await provider.Admin.ListAsync(); + } + """; + + public const string Export = + """ + var session = await Database.EnsureSessionAsync(); + var provider = session.Resolution.Provider; + + byte[] image = await provider.Admin.ExportAsync("app.db"); + + // A plain SQLite file: open it in any desktop tool. + await js.InvokeVoidAsync("bsqliteDemo.download", "app.db", image); + """; + + public const string Import = + """ + // Close the worker first so it cannot keep writing the image being replaced. + await Database.CloseSessionAsync(); + + await provider.Admin.ImportAsync("app.db", bytes); + + // Record the sticky binding, or the next open would create an empty file on the + // preferred backend and the import would look like it vanished. + await Database.Bindings.SetProviderNameAsync("app.db", provider.Name); + + // Reopen once so migrations run before the pages come back for the database. The + // open raises Opened, which is how every page hears it should reload. + await using (await Database.OpenContextAsync()) + { + } + """; + + public const string Delete = + """ + await Database.CloseSessionAsync(); + await provider.Admin.DeleteAsync("app.db"); + + // Forget the binding so selection is free to pick the preferred backend again. + await Database.Bindings.RemoveAsync("app.db"); + + // Migrates and reseeds the now-empty database. + await using (await Database.OpenContextAsync()) + { + } + """; + } + + public static class Limits + { + public const string Sync = + """ + await using var context = await Database.OpenContextAsync(); + + // EF short-circuits SaveChanges() when nothing is pending, so stage a real insert + // first - otherwise the call returns 0 and never reaches the connection. + context.Categories.Add(new Category { Name = "sync-probe", Color = "#0f766e" }); + + context.SaveChanges(); + // DbUpdateException -> BsqliteSynchronousApiNotSupportedException: + // the engine is in a worker and the Blazor UI thread cannot block on it. + // Use SaveChangesAsync(). Same for ToList(), First(), DbConnection.Open(). + // + // EF wraps save failures, so read InnerException for the message that explains it. + """; + + public const string Wal = + """ + await Database.ExecuteNonQueryAsync("PRAGMA journal_mode=WAL"); + // No web VFS implements WAL: WASM has no shared-memory primitives for it, and the + // official SQLite WASM build cannot even read a WAL-mode file. + // Journal mode is DELETE or TRUNCATE. Concurrent reads are a VFS capability instead: + // session.Resolution.Provider.Capabilities.SupportsConcurrentReads + """; + + public const string Attach = + """ + var limits = session.Connection.RuntimeLimits; + if (!limits.SupportsMultiDatabaseTransactions) + { + // Refused before it runs, by capability - not by provider name. + return; + } + + await Database.ExecuteNonQueryAsync("ATTACH DATABASE 'other.db' AS other"); + """; + + public const string PageSize = + """ + var limits = session.Connection.RuntimeLimits; + limits.CanChangePageSize; // false on block-oriented backends + + // Letting PRAGMA page_size through on a backend that pins it would report success and + // then corrupt the image, so BsqliteFeatureGuards throws instead. + await Database.ExecuteNonQueryAsync("PRAGMA page_size=512"); + """; + } +} diff --git a/src/Bsqlite/Demo/Client/_Imports.razor b/src/Bsqlite/Demo/Client/_Imports.razor new file mode 100644 index 00000000000..a48131c626c --- /dev/null +++ b/src/Bsqlite/Demo/Client/_Imports.razor @@ -0,0 +1,18 @@ +@using System.Net.Http +@using System.Net.Http.Json +@using Microsoft.AspNetCore.Components +@using Microsoft.AspNetCore.Components.Forms +@using Microsoft.AspNetCore.Components.Routing +@using Microsoft.AspNetCore.Components.Web +@using static Microsoft.AspNetCore.Components.Web.RenderMode +@using Microsoft.EntityFrameworkCore +@using Microsoft.JSInterop +@using Bit.Bsqlite.Demo.Client +@using Bit.Bsqlite.Demo.Client.Components +@using Bit.Bsqlite.Demo.Client.Layout +@using Bit.Bsqlite.Demo.Data +@using Bit.Bsqlite.Demo.Domain +@using Bit.Bsqlite +@using Bit.Bsqlite.Data +@using Bit.Bsqlite.Interop +@using Bit.Bsqlite.Storage diff --git a/src/Bsqlite/Demo/Data/AppDbContext.cs b/src/Bsqlite/Demo/Data/AppDbContext.cs new file mode 100644 index 00000000000..cff18b717f9 --- /dev/null +++ b/src/Bsqlite/Demo/Data/AppDbContext.cs @@ -0,0 +1,20 @@ +using Bit.Bsqlite.Demo.Domain; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.Demo.Data; + +public sealed class AppDbContext(DbContextOptions options) : DbContext(options) +{ + public DbSet Categories => Set(); + + public DbSet Products => Set(); + + public DbSet Customers => Set(); + + public DbSet Orders => Set(); + + public DbSet OrderLines => Set(); + + protected override void OnModelCreating(ModelBuilder modelBuilder) + => modelBuilder.ApplyConfigurationsFromAssembly(typeof(AppDbContext).Assembly); +} diff --git a/src/Bsqlite/Demo/Data/Bit.Bsqlite.Demo.Data.csproj b/src/Bsqlite/Demo/Data/Bit.Bsqlite.Demo.Data.csproj new file mode 100644 index 00000000000..42f48d0ee7a --- /dev/null +++ b/src/Bsqlite/Demo/Data/Bit.Bsqlite.Demo.Data.csproj @@ -0,0 +1,22 @@ + + + + + net10.0 + enable + enable + Bit.Bsqlite.Demo.Data + false + + + + + + + + + + + + diff --git a/src/Bsqlite/Demo/Data/Configuration/CategoryConfiguration.cs b/src/Bsqlite/Demo/Data/Configuration/CategoryConfiguration.cs new file mode 100644 index 00000000000..649634f2329 --- /dev/null +++ b/src/Bsqlite/Demo/Data/Configuration/CategoryConfiguration.cs @@ -0,0 +1,15 @@ +using Bit.Bsqlite.Demo.Domain; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Metadata.Builders; + +namespace Bit.Bsqlite.Demo.Data.Configuration; + +public sealed class CategoryConfiguration : IEntityTypeConfiguration +{ + public void Configure(EntityTypeBuilder builder) + { + builder.HasKey(c => c.Id); + builder.Property(c => c.Name).IsRequired().HasMaxLength(100); + builder.Property(c => c.Color).IsRequired().HasMaxLength(16); + } +} diff --git a/src/Bsqlite/Demo/Data/Configuration/CustomerConfiguration.cs b/src/Bsqlite/Demo/Data/Configuration/CustomerConfiguration.cs new file mode 100644 index 00000000000..3d2959fefdd --- /dev/null +++ b/src/Bsqlite/Demo/Data/Configuration/CustomerConfiguration.cs @@ -0,0 +1,20 @@ +using Bit.Bsqlite.Demo.Domain; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Metadata.Builders; + +namespace Bit.Bsqlite.Demo.Data.Configuration; + +public sealed class CustomerConfiguration : IEntityTypeConfiguration +{ + public void Configure(EntityTypeBuilder builder) + { + builder.HasKey(c => c.Id); + builder.Property(c => c.PublicId).IsRequired(); + builder.HasIndex(c => c.PublicId).IsUnique(); + builder.Property(c => c.DisplayName).IsRequired().HasMaxLength(120); + builder.Property(c => c.Email).IsRequired().HasMaxLength(200); + builder.HasIndex(c => c.Email).IsUnique(); + builder.Property(c => c.CreditLimit).HasPrecision(18, 2); + builder.Property(c => c.Notes).HasMaxLength(500); + } +} diff --git a/src/Bsqlite/Demo/Data/Configuration/OrderLineConfiguration.cs b/src/Bsqlite/Demo/Data/Configuration/OrderLineConfiguration.cs new file mode 100644 index 00000000000..b3b718594c3 --- /dev/null +++ b/src/Bsqlite/Demo/Data/Configuration/OrderLineConfiguration.cs @@ -0,0 +1,22 @@ +using Bit.Bsqlite.Demo.Domain; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Metadata.Builders; + +namespace Bit.Bsqlite.Demo.Data.Configuration; + +public sealed class OrderLineConfiguration : IEntityTypeConfiguration +{ + public void Configure(EntityTypeBuilder builder) + { + builder.HasKey(l => l.Id); + builder.Property(l => l.UnitPrice).HasPrecision(18, 2); + builder.HasOne(l => l.Order) + .WithMany(o => o.Lines) + .HasForeignKey(l => l.OrderId) + .OnDelete(DeleteBehavior.Cascade); + builder.HasOne(l => l.Product) + .WithMany(p => p.Lines) + .HasForeignKey(l => l.ProductId) + .OnDelete(DeleteBehavior.Restrict); + } +} diff --git a/src/Bsqlite/Demo/Data/Configuration/ProductConfiguration.cs b/src/Bsqlite/Demo/Data/Configuration/ProductConfiguration.cs new file mode 100644 index 00000000000..dfceea2d289 --- /dev/null +++ b/src/Bsqlite/Demo/Data/Configuration/ProductConfiguration.cs @@ -0,0 +1,25 @@ +using Bit.Bsqlite.Demo.Domain; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Metadata.Builders; + +namespace Bit.Bsqlite.Demo.Data.Configuration; + +public sealed class ProductConfiguration : IEntityTypeConfiguration +{ + public void Configure(EntityTypeBuilder builder) + { + builder.HasKey(p => p.Id); + builder.Property(p => p.PublicId).IsRequired(); + builder.HasIndex(p => p.PublicId).IsUnique(); + builder.Property(p => p.Sku).IsRequired().HasMaxLength(32); + builder.HasIndex(p => p.Sku).IsUnique(); + builder.Property(p => p.Name).IsRequired().HasMaxLength(200); + builder.Property(p => p.Price).HasPrecision(18, 2); + builder.Property(p => p.Tags).HasMaxLength(200); + builder.HasIndex(p => p.Name); + builder.HasOne(p => p.Category) + .WithMany(c => c.Products) + .HasForeignKey(p => p.CategoryId) + .OnDelete(DeleteBehavior.Cascade); + } +} diff --git a/src/Bsqlite/Demo/Data/Configuration/SalesOrderConfiguration.cs b/src/Bsqlite/Demo/Data/Configuration/SalesOrderConfiguration.cs new file mode 100644 index 00000000000..028bf588b2e --- /dev/null +++ b/src/Bsqlite/Demo/Data/Configuration/SalesOrderConfiguration.cs @@ -0,0 +1,22 @@ +using Bit.Bsqlite.Demo.Domain; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Metadata.Builders; + +namespace Bit.Bsqlite.Demo.Data.Configuration; + +public sealed class SalesOrderConfiguration : IEntityTypeConfiguration +{ + public void Configure(EntityTypeBuilder builder) + { + builder.ToTable("Orders"); + builder.HasKey(o => o.Id); + builder.Property(o => o.Number).IsRequired().HasMaxLength(24); + builder.HasIndex(o => o.Number).IsUnique(); + builder.Property(o => o.Status).HasConversion().HasMaxLength(16); + builder.Property(o => o.Notes).HasMaxLength(500); + builder.HasOne(o => o.Customer) + .WithMany(c => c.Orders) + .HasForeignKey(o => o.CustomerId) + .OnDelete(DeleteBehavior.Restrict); + } +} diff --git a/src/Bsqlite/Demo/Data/DemoData.cs b/src/Bsqlite/Demo/Data/DemoData.cs new file mode 100644 index 00000000000..8d97e936ced --- /dev/null +++ b/src/Bsqlite/Demo/Data/DemoData.cs @@ -0,0 +1,165 @@ +using Bit.Bsqlite.Demo.Domain; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.Demo.Data; + +/// +/// Deterministic workshop catalog used when a database has no categories yet. +/// +public static class DemoData +{ + public static async Task SeedIfEmptyAsync(AppDbContext db, CancellationToken cancellationToken = default) + { + if (await db.Categories.AnyAsync(cancellationToken).ConfigureAwait(false)) + { + return; + } + + var tools = new Category { Name = "Tools", Color = "#0f766e" }; + var fasteners = new Category { Name = "Fasteners", Color = "#a16207" }; + var electrical = new Category { Name = "Electrical", Color = "#1d4ed8" }; + var safety = new Category { Name = "Safety", Color = "#b91c1c" }; + db.Categories.AddRange(tools, fasteners, electrical, safety); + + var caliper = Product( + "TL-100", + "Digital caliper", + 42.50m, + 0.18, + tools, + TimeSpan.FromDays(2), + "metrology,stainless"); + var torque = Product( + "TL-240", + "Torque wrench 1/2\"", + 89.00m, + 1.35, + tools, + TimeSpan.FromDays(5), + "assembly"); + var bolt = Product( + "FS-M8", + "M8×30 hex bolt (pack 50)", + 6.40m, + 0.62, + fasteners, + TimeSpan.FromDays(1), + "metric,steel"); + var cable = Product( + "EL-2.5", + "2.5 mm² cable (100 m)", + 54.90m, + 8.4, + electrical, + TimeSpan.FromDays(7), + "copper"); + var goggles = Product( + "SF-01", + "Safety goggles", + 11.25m, + 0.08, + safety, + TimeSpan.FromDays(1), + "ppe"); + goggles.IsActive = false; + goggles.DiscontinuedOn = new DateOnly(2026, 1, 15); + db.Products.AddRange(caliper, torque, bolt, cable, goggles); + + var northwind = new Customer + { + PublicId = Guid.Parse("3c2a1f10-6b44-4d2a-9c11-0b7c2e4a1001"), + DisplayName = "Northwind Workshop", + Email = "stores@northwind.example", + DateOfBirth = new DateOnly(1978, 4, 12), + IsVip = true, + CreditLimit = 25_000m, + Notes = "Net-30. Prefers OPFS-backed copies for the shop floor tablets.", + CreatedUtc = new DateTime(2026, 2, 1, 9, 0, 0, DateTimeKind.Utc), + }; + var contoso = new Customer + { + PublicId = Guid.Parse("3c2a1f10-6b44-4d2a-9c11-0b7c2e4a1002"), + DisplayName = "Contoso Field Service", + Email = "ops@contoso.example", + DateOfBirth = new DateOnly(1991, 11, 3), + IsVip = false, + CreditLimit = 8_500.50m, + Notes = string.Empty, + CreatedUtc = new DateTime(2026, 3, 18, 14, 30, 0, DateTimeKind.Utc), + }; + var alpine = new Customer + { + PublicId = Guid.Parse("3c2a1f10-6b44-4d2a-9c11-0b7c2e4a1003"), + DisplayName = "Alpine Electrics", + Email = "hello@alpine.example", + DateOfBirth = new DateOnly(1985, 7, 22), + IsVip = true, + CreditLimit = 12_000m, + CreatedUtc = new DateTime(2026, 5, 4, 8, 15, 0, DateTimeKind.Utc), + }; + db.Customers.AddRange(northwind, contoso, alpine); + + db.Orders.AddRange( + new SalesOrder + { + Number = "SO-1042", + Customer = northwind, + Status = OrderStatus.Shipped, + OrderedAt = new DateTimeOffset(2026, 6, 2, 10, 15, 0, TimeSpan.Zero), + ShipBy = new DateOnly(2026, 6, 5), + Notes = "Left at the goods-in hatch.", + Lines = + [ + new OrderLine { Product = caliper, Quantity = 2, UnitPrice = 42.50m }, + new OrderLine { Product = goggles, Quantity = 12, UnitPrice = 11.25m }, + ], + }, + new SalesOrder + { + Number = "SO-1043", + Customer = contoso, + Status = OrderStatus.Submitted, + OrderedAt = new DateTimeOffset(2026, 8, 10, 16, 40, 0, TimeSpan.FromHours(2)), + ShipBy = new DateOnly(2026, 8, 18), + Lines = + [ + new OrderLine { Product = torque, Quantity = 1, UnitPrice = 89.00m }, + new OrderLine { Product = bolt, Quantity = 4, UnitPrice = 6.40m }, + ], + }, + new SalesOrder + { + Number = "SO-1044", + Customer = alpine, + Status = OrderStatus.Draft, + OrderedAt = new DateTimeOffset(2026, 8, 14, 9, 5, 0, TimeSpan.FromHours(1)), + Notes = "Waiting on cable length confirmation.", + Lines = + [ + new OrderLine { Product = cable, Quantity = 3, UnitPrice = 54.90m }, + ], + }); + + await db.SaveChangesAsync(cancellationToken).ConfigureAwait(false); + } + + private static Product Product( + string sku, + string name, + decimal price, + double weightKg, + Category category, + TimeSpan leadTime, + string tags) + => new() + { + Sku = sku, + Name = name, + Price = price, + WeightKg = weightKg, + Category = category, + LeadTime = leadTime, + Tags = tags, + CreatedUtc = new DateTime(2026, 1, 8, 12, 0, 0, DateTimeKind.Utc), + }; +} diff --git a/src/Bsqlite/Demo/Data/Migrations/AppDbContextModelSnapshot.cs b/src/Bsqlite/Demo/Data/Migrations/AppDbContextModelSnapshot.cs new file mode 100644 index 00000000000..2a56a7e923b --- /dev/null +++ b/src/Bsqlite/Demo/Data/Migrations/AppDbContextModelSnapshot.cs @@ -0,0 +1,201 @@ +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Infrastructure; + +namespace Bit.Bsqlite.Demo.Data.Migrations; + +[DbContext(typeof(AppDbContext))] +public sealed class AppDbContextModelSnapshot : ModelSnapshot +{ + protected override void BuildModel(ModelBuilder modelBuilder) + { + modelBuilder.HasAnnotation("ProductVersion", "10.0.11"); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.Category", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("INTEGER"); + b.Property("Color") + .IsRequired() + .HasMaxLength(16) + .HasColumnType("TEXT"); + b.Property("Name") + .IsRequired() + .HasMaxLength(100) + .HasColumnType("TEXT"); + b.HasKey("Id"); + b.ToTable("Categories"); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.Customer", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("INTEGER"); + b.Property("CreditLimit") + .HasPrecision(18, 2) + .HasColumnType("TEXT"); + b.Property("CreatedUtc") + .HasColumnType("TEXT"); + b.Property("DateOfBirth") + .HasColumnType("TEXT"); + b.Property("DisplayName") + .IsRequired() + .HasMaxLength(120) + .HasColumnType("TEXT"); + b.Property("Email") + .IsRequired() + .HasMaxLength(200) + .HasColumnType("TEXT"); + b.Property("IsVip") + .HasColumnType("INTEGER"); + b.Property("Notes") + .IsRequired() + .HasMaxLength(500) + .HasColumnType("TEXT"); + b.Property("PublicId") + .HasColumnType("TEXT"); + b.HasKey("Id"); + b.HasIndex("Email") + .IsUnique(); + b.HasIndex("PublicId") + .IsUnique(); + b.ToTable("Customers"); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.OrderLine", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("INTEGER"); + b.Property("OrderId") + .HasColumnType("INTEGER"); + b.Property("ProductId") + .HasColumnType("INTEGER"); + b.Property("Quantity") + .HasColumnType("INTEGER"); + b.Property("UnitPrice") + .HasPrecision(18, 2) + .HasColumnType("TEXT"); + b.HasKey("Id"); + b.HasIndex("OrderId"); + b.HasIndex("ProductId"); + b.ToTable("OrderLines"); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.Product", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("INTEGER"); + b.Property("CategoryId") + .HasColumnType("INTEGER"); + b.Property("CreatedUtc") + .HasColumnType("TEXT"); + b.Property("DiscontinuedOn") + .HasColumnType("TEXT"); + b.Property("IsActive") + .HasColumnType("INTEGER"); + b.Property("LeadTime") + .HasColumnType("TEXT"); + b.Property("Name") + .IsRequired() + .HasMaxLength(200) + .HasColumnType("TEXT"); + b.Property("Price") + .HasPrecision(18, 2) + .HasColumnType("TEXT"); + b.Property("PublicId") + .HasColumnType("TEXT"); + b.Property("Sku") + .IsRequired() + .HasMaxLength(32) + .HasColumnType("TEXT"); + b.Property("Tags") + .IsRequired() + .HasMaxLength(200) + .HasColumnType("TEXT"); + b.Property("WeightKg") + .HasColumnType("REAL"); + b.HasKey("Id"); + b.HasIndex("CategoryId"); + b.HasIndex("Name"); + b.HasIndex("PublicId") + .IsUnique(); + b.HasIndex("Sku") + .IsUnique(); + b.ToTable("Products"); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.SalesOrder", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("INTEGER"); + b.Property("CustomerId") + .HasColumnType("INTEGER"); + b.Property("Notes") + .IsRequired() + .HasMaxLength(500) + .HasColumnType("TEXT"); + b.Property("Number") + .IsRequired() + .HasMaxLength(24) + .HasColumnType("TEXT"); + b.Property("OrderedAt") + .HasColumnType("TEXT"); + b.Property("ShipBy") + .HasColumnType("TEXT"); + b.Property("Status") + .IsRequired() + .HasMaxLength(16) + .HasColumnType("TEXT"); + b.HasKey("Id"); + b.HasIndex("CustomerId"); + b.HasIndex("Number") + .IsUnique(); + b.ToTable("Orders", (string?)null); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.OrderLine", b => + { + b.HasOne("Bit.Bsqlite.Demo.Domain.SalesOrder", "Order") + .WithMany("Lines") + .HasForeignKey("OrderId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + b.HasOne("Bit.Bsqlite.Demo.Domain.Product", "Product") + .WithMany("Lines") + .HasForeignKey("ProductId") + .OnDelete(DeleteBehavior.Restrict) + .IsRequired(); + b.Navigation("Order"); + b.Navigation("Product"); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.Product", b => + { + b.HasOne("Bit.Bsqlite.Demo.Domain.Category", "Category") + .WithMany("Products") + .HasForeignKey("CategoryId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired(); + b.Navigation("Category"); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.SalesOrder", b => + { + b.HasOne("Bit.Bsqlite.Demo.Domain.Customer", "Customer") + .WithMany("Orders") + .HasForeignKey("CustomerId") + .OnDelete(DeleteBehavior.Restrict) + .IsRequired(); + b.Navigation("Customer"); + }); + + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.Category", b => b.Navigation("Products")); + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.Customer", b => b.Navigation("Orders")); + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.Product", b => b.Navigation("Lines")); + modelBuilder.Entity("Bit.Bsqlite.Demo.Domain.SalesOrder", b => b.Navigation("Lines")); + } +} diff --git a/src/Bsqlite/Demo/Data/Migrations/InitialCreate.cs b/src/Bsqlite/Demo/Data/Migrations/InitialCreate.cs new file mode 100644 index 00000000000..2e94abc80fa --- /dev/null +++ b/src/Bsqlite/Demo/Data/Migrations/InitialCreate.cs @@ -0,0 +1,53 @@ +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Migrations; + +namespace Bit.Bsqlite.Demo.Data.Migrations; + +[DbContext(typeof(AppDbContext))] +[Migration("20260101000000_InitialCreate")] +public sealed class InitialCreate : Migration +{ + protected override void Up(MigrationBuilder migrationBuilder) + { + migrationBuilder.CreateTable( + name: "Categories", + columns: table => new + { + Id = table.Column(type: "INTEGER", nullable: false) + .Annotation("Sqlite:Autoincrement", true), + Name = table.Column(type: "TEXT", maxLength: 100, nullable: false), + }, + constraints: table => table.PrimaryKey("PK_Categories", x => x.Id)); + + migrationBuilder.CreateTable( + name: "Products", + columns: table => new + { + Id = table.Column(type: "INTEGER", nullable: false) + .Annotation("Sqlite:Autoincrement", true), + Name = table.Column(type: "TEXT", maxLength: 200, nullable: false), + Price = table.Column(type: "TEXT", precision: 18, scale: 2, nullable: false), + CreatedUtc = table.Column(type: "TEXT", nullable: false), + CategoryId = table.Column(type: "INTEGER", nullable: false), + }, + constraints: table => + { + table.PrimaryKey("PK_Products", x => x.Id); + table.ForeignKey( + name: "FK_Products_Categories_CategoryId", + column: x => x.CategoryId, + principalTable: "Categories", + principalColumn: "Id", + onDelete: ReferentialAction.Cascade); + }); + + migrationBuilder.CreateIndex(name: "IX_Products_Name", table: "Products", column: "Name"); + migrationBuilder.CreateIndex(name: "IX_Products_CategoryId", table: "Products", column: "CategoryId"); + } + + protected override void Down(MigrationBuilder migrationBuilder) + { + migrationBuilder.DropTable(name: "Products"); + migrationBuilder.DropTable(name: "Categories"); + } +} diff --git a/src/Bsqlite/Demo/Data/Migrations/WorkshopCatalog.cs b/src/Bsqlite/Demo/Data/Migrations/WorkshopCatalog.cs new file mode 100644 index 00000000000..d6e6b7ef218 --- /dev/null +++ b/src/Bsqlite/Demo/Data/Migrations/WorkshopCatalog.cs @@ -0,0 +1,178 @@ +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Migrations; + +namespace Bit.Bsqlite.Demo.Data.Migrations; + +[DbContext(typeof(AppDbContext))] +[Migration("20260816000000_WorkshopCatalog")] +public sealed class WorkshopCatalog : Migration +{ + protected override void Up(MigrationBuilder migrationBuilder) + { + migrationBuilder.AddColumn( + name: "Color", + table: "Categories", + type: "TEXT", + maxLength: 16, + nullable: false, + defaultValue: "#0f766e"); + + migrationBuilder.AddColumn( + name: "PublicId", + table: "Products", + type: "TEXT", + nullable: false, + defaultValue: "00000000-0000-0000-0000-000000000000"); + migrationBuilder.AddColumn( + name: "Sku", + table: "Products", + type: "TEXT", + maxLength: 32, + nullable: false, + defaultValue: ""); + migrationBuilder.AddColumn( + name: "WeightKg", + table: "Products", + type: "REAL", + nullable: false, + defaultValue: 0.0); + migrationBuilder.AddColumn( + name: "IsActive", + table: "Products", + type: "INTEGER", + nullable: false, + defaultValue: true); + migrationBuilder.AddColumn( + name: "DiscontinuedOn", + table: "Products", + type: "TEXT", + nullable: true); + migrationBuilder.AddColumn( + name: "LeadTime", + table: "Products", + type: "TEXT", + nullable: false, + defaultValue: "2.00:00:00"); + migrationBuilder.AddColumn( + name: "Tags", + table: "Products", + type: "TEXT", + maxLength: 200, + nullable: false, + defaultValue: ""); + + migrationBuilder.Sql("""UPDATE "Products" SET "Sku" = 'SKU-' || "Id" WHERE "Sku" = '';"""); + migrationBuilder.Sql( + """ + UPDATE "Products" + SET "PublicId" = '00000000-0000-0000-0000-' || printf('%012d', "Id") + WHERE "PublicId" = '00000000-0000-0000-0000-000000000000'; + """); + + migrationBuilder.CreateIndex( + name: "IX_Products_PublicId", + table: "Products", + column: "PublicId", + unique: true); + migrationBuilder.CreateIndex( + name: "IX_Products_Sku", + table: "Products", + column: "Sku", + unique: true); + + migrationBuilder.CreateTable( + name: "Customers", + columns: table => new + { + Id = table.Column(type: "INTEGER", nullable: false) + .Annotation("Sqlite:Autoincrement", true), + PublicId = table.Column(type: "TEXT", nullable: false), + DisplayName = table.Column(type: "TEXT", maxLength: 120, nullable: false), + Email = table.Column(type: "TEXT", maxLength: 200, nullable: false), + DateOfBirth = table.Column(type: "TEXT", nullable: false), + IsVip = table.Column(type: "INTEGER", nullable: false), + CreditLimit = table.Column(type: "TEXT", precision: 18, scale: 2, nullable: false), + Notes = table.Column(type: "TEXT", maxLength: 500, nullable: false), + CreatedUtc = table.Column(type: "TEXT", nullable: false), + }, + constraints: table => table.PrimaryKey("PK_Customers", x => x.Id)); + + migrationBuilder.CreateIndex(name: "IX_Customers_PublicId", table: "Customers", column: "PublicId", unique: true); + migrationBuilder.CreateIndex(name: "IX_Customers_Email", table: "Customers", column: "Email", unique: true); + + migrationBuilder.CreateTable( + name: "Orders", + columns: table => new + { + Id = table.Column(type: "INTEGER", nullable: false) + .Annotation("Sqlite:Autoincrement", true), + Number = table.Column(type: "TEXT", maxLength: 24, nullable: false), + CustomerId = table.Column(type: "INTEGER", nullable: false), + Status = table.Column(type: "TEXT", maxLength: 16, nullable: false), + OrderedAt = table.Column(type: "TEXT", nullable: false), + ShipBy = table.Column(type: "TEXT", nullable: true), + Notes = table.Column(type: "TEXT", maxLength: 500, nullable: false), + }, + constraints: table => + { + table.PrimaryKey("PK_Orders", x => x.Id); + table.ForeignKey( + name: "FK_Orders_Customers_CustomerId", + column: x => x.CustomerId, + principalTable: "Customers", + principalColumn: "Id", + onDelete: ReferentialAction.Restrict); + }); + + migrationBuilder.CreateIndex(name: "IX_Orders_Number", table: "Orders", column: "Number", unique: true); + migrationBuilder.CreateIndex(name: "IX_Orders_CustomerId", table: "Orders", column: "CustomerId"); + + migrationBuilder.CreateTable( + name: "OrderLines", + columns: table => new + { + Id = table.Column(type: "INTEGER", nullable: false) + .Annotation("Sqlite:Autoincrement", true), + OrderId = table.Column(type: "INTEGER", nullable: false), + ProductId = table.Column(type: "INTEGER", nullable: false), + Quantity = table.Column(type: "INTEGER", nullable: false), + UnitPrice = table.Column(type: "TEXT", precision: 18, scale: 2, nullable: false), + }, + constraints: table => + { + table.PrimaryKey("PK_OrderLines", x => x.Id); + table.ForeignKey( + name: "FK_OrderLines_Orders_OrderId", + column: x => x.OrderId, + principalTable: "Orders", + principalColumn: "Id", + onDelete: ReferentialAction.Cascade); + table.ForeignKey( + name: "FK_OrderLines_Products_ProductId", + column: x => x.ProductId, + principalTable: "Products", + principalColumn: "Id", + onDelete: ReferentialAction.Restrict); + }); + + migrationBuilder.CreateIndex(name: "IX_OrderLines_OrderId", table: "OrderLines", column: "OrderId"); + migrationBuilder.CreateIndex(name: "IX_OrderLines_ProductId", table: "OrderLines", column: "ProductId"); + } + + protected override void Down(MigrationBuilder migrationBuilder) + { + migrationBuilder.DropTable(name: "OrderLines"); + migrationBuilder.DropTable(name: "Orders"); + migrationBuilder.DropTable(name: "Customers"); + migrationBuilder.DropIndex(name: "IX_Products_PublicId", table: "Products"); + migrationBuilder.DropIndex(name: "IX_Products_Sku", table: "Products"); + migrationBuilder.DropColumn(name: "PublicId", table: "Products"); + migrationBuilder.DropColumn(name: "Sku", table: "Products"); + migrationBuilder.DropColumn(name: "WeightKg", table: "Products"); + migrationBuilder.DropColumn(name: "IsActive", table: "Products"); + migrationBuilder.DropColumn(name: "DiscontinuedOn", table: "Products"); + migrationBuilder.DropColumn(name: "LeadTime", table: "Products"); + migrationBuilder.DropColumn(name: "Tags", table: "Products"); + migrationBuilder.DropColumn(name: "Color", table: "Categories"); + } +} diff --git a/src/Bsqlite/Demo/Domain/Bit.Bsqlite.Demo.Domain.csproj b/src/Bsqlite/Demo/Domain/Bit.Bsqlite.Demo.Domain.csproj new file mode 100644 index 00000000000..0275d7247b1 --- /dev/null +++ b/src/Bsqlite/Demo/Domain/Bit.Bsqlite.Demo.Domain.csproj @@ -0,0 +1,13 @@ + + + + + net10.0 + enable + enable + Bit.Bsqlite.Demo.Domain + false + + + diff --git a/src/Bsqlite/Demo/Domain/Category.cs b/src/Bsqlite/Demo/Domain/Category.cs new file mode 100644 index 00000000000..15beb69088b --- /dev/null +++ b/src/Bsqlite/Demo/Domain/Category.cs @@ -0,0 +1,12 @@ +namespace Bit.Bsqlite.Demo.Domain; + +public sealed class Category +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public string Color { get; set; } = "#0f766e"; + + public List Products { get; } = []; +} diff --git a/src/Bsqlite/Demo/Domain/Customer.cs b/src/Bsqlite/Demo/Domain/Customer.cs new file mode 100644 index 00000000000..669307a29f1 --- /dev/null +++ b/src/Bsqlite/Demo/Domain/Customer.cs @@ -0,0 +1,24 @@ +namespace Bit.Bsqlite.Demo.Domain; + +public sealed class Customer +{ + public int Id { get; set; } + + public Guid PublicId { get; set; } = Guid.NewGuid(); + + public string DisplayName { get; set; } = string.Empty; + + public string Email { get; set; } = string.Empty; + + public DateOnly DateOfBirth { get; set; } + + public bool IsVip { get; set; } + + public decimal CreditLimit { get; set; } + + public string Notes { get; set; } = string.Empty; + + public DateTime CreatedUtc { get; set; } = DateTime.UtcNow; + + public List Orders { get; } = []; +} diff --git a/src/Bsqlite/Demo/Domain/OrderLine.cs b/src/Bsqlite/Demo/Domain/OrderLine.cs new file mode 100644 index 00000000000..1302bd0355e --- /dev/null +++ b/src/Bsqlite/Demo/Domain/OrderLine.cs @@ -0,0 +1,18 @@ +namespace Bit.Bsqlite.Demo.Domain; + +public sealed class OrderLine +{ + public int Id { get; set; } + + public int OrderId { get; set; } + + public SalesOrder? Order { get; set; } + + public int ProductId { get; set; } + + public Product? Product { get; set; } + + public int Quantity { get; set; } = 1; + + public decimal UnitPrice { get; set; } +} diff --git a/src/Bsqlite/Demo/Domain/OrderStatus.cs b/src/Bsqlite/Demo/Domain/OrderStatus.cs new file mode 100644 index 00000000000..fba1163a190 --- /dev/null +++ b/src/Bsqlite/Demo/Domain/OrderStatus.cs @@ -0,0 +1,10 @@ +namespace Bit.Bsqlite.Demo.Domain; + +public enum OrderStatus +{ + Draft = 0, + Submitted = 1, + Packed = 2, + Shipped = 3, + Cancelled = 4, +} diff --git a/src/Bsqlite/Demo/Domain/Product.cs b/src/Bsqlite/Demo/Domain/Product.cs new file mode 100644 index 00000000000..bf53cf027ec --- /dev/null +++ b/src/Bsqlite/Demo/Domain/Product.cs @@ -0,0 +1,32 @@ +namespace Bit.Bsqlite.Demo.Domain; + +public sealed class Product +{ + public int Id { get; set; } + + public Guid PublicId { get; set; } = Guid.NewGuid(); + + public string Sku { get; set; } = string.Empty; + + public string Name { get; set; } = string.Empty; + + public decimal Price { get; set; } + + public double WeightKg { get; set; } + + public bool IsActive { get; set; } = true; + + public DateTime CreatedUtc { get; set; } = DateTime.UtcNow; + + public DateOnly? DiscontinuedOn { get; set; } + + public TimeSpan LeadTime { get; set; } = TimeSpan.FromDays(2); + + public string Tags { get; set; } = string.Empty; + + public int CategoryId { get; set; } + + public Category? Category { get; set; } + + public List Lines { get; } = []; +} diff --git a/src/Bsqlite/Demo/Domain/SalesOrder.cs b/src/Bsqlite/Demo/Domain/SalesOrder.cs new file mode 100644 index 00000000000..02923392bb3 --- /dev/null +++ b/src/Bsqlite/Demo/Domain/SalesOrder.cs @@ -0,0 +1,25 @@ +namespace Bit.Bsqlite.Demo.Domain; + +/// +/// Named rather than Order so it does not collide with LINQ's OrderBy. +/// +public sealed class SalesOrder +{ + public int Id { get; set; } + + public string Number { get; set; } = string.Empty; + + public int CustomerId { get; set; } + + public Customer? Customer { get; set; } + + public OrderStatus Status { get; set; } = OrderStatus.Draft; + + public DateTimeOffset OrderedAt { get; set; } = DateTimeOffset.UtcNow; + + public DateOnly? ShipBy { get; set; } + + public string Notes { get; set; } = string.Empty; + + public List Lines { get; set; } = []; +} diff --git a/src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj b/src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj new file mode 100644 index 00000000000..4cb4333cb11 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj @@ -0,0 +1,106 @@ + + + + net10.0 + enable + enable + true + true + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + BsqliteDocs/docs/%(Filename)%(Extension) + + + + + + + + + + + + BsqliteSource/Demo/Client/%(RecursiveDir)%(Filename)%(Extension) + + + + + BsqliteSource/Demo/Data/%(RecursiveDir)%(Filename)%(Extension) + + + + BsqliteSource/Demo/Domain/%(RecursiveDir)%(Filename)%(Extension) + + + + + + + + BsqliteSource/Demo/Server/Components/%(RecursiveDir)%(Filename)%(Extension) + + + + diff --git a/src/Bsqlite/Demo/Server/Components/App.razor b/src/Bsqlite/Demo/Server/Components/App.razor new file mode 100644 index 00000000000..cb69374167f --- /dev/null +++ b/src/Bsqlite/Demo/Server/Components/App.razor @@ -0,0 +1,84 @@ + + + + + + + @* No static here: every page supplies its own through <PageTitle>, and with prerendering + the HeadOutlet emits a real <title> element into this head. *@ + <meta name="description" content="bit Bsqlite (Bit.Bsqlite) - EF Core on SQLite, inside the browser tab. The same entities and migrations on the client as on the server, over OPFS, IndexedDB, Cache Storage, or memory." /> + <meta name="color-scheme" content="light dark" /> + <base href="/" /> + <ResourcePreloader /> + + <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='7' fill='%230f766e'/%3E%3Cellipse cx='16' cy='9' rx='9' ry='3.6' fill='none' stroke='%23fff' stroke-width='2.2'/%3E%3Cpath d='M7 9v14c0 2 4 3.6 9 3.6s9-1.6 9-3.6V9' fill='none' stroke='%23fff' stroke-width='2.2'/%3E%3Cpath d='M7 16c0 2 4 3.6 9 3.6s9-1.6 9-3.6' fill='none' stroke='%23fff' stroke-width='2.2'/%3E%3C/svg%3E" /> + + @* Runs before first paint so the shell never flashes the wrong theme. The stored value is a + *preference* ("system" | "light" | "dark"); what lands on <html> is always the resolved one, + which is why the stylesheet needs a single [data-theme="dark"] block and no duplicated + prefers-color-scheme table. js/demo.js reads and writes the preference through it. *@ + <script> + (function () { + var KEY = "bsqlite.theme"; + var media = window.matchMedia("(prefers-color-scheme: dark)"); + + function preference() { + try { + var stored = localStorage.getItem(KEY); + return stored === "light" || stored === "dark" ? stored : "system"; + } catch (e) { + return "system"; // storage blocked: the system theme wins + } + } + + function apply() { + var wanted = preference(); + var resolved = wanted === "system" ? (media.matches ? "dark" : "light") : wanted; + document.documentElement.setAttribute("data-theme", resolved); + return resolved; + } + + apply(); + media.addEventListener("change", function () { + if (preference() === "system") { apply(); } + }); + + window.__bsqliteTheme = { key: KEY, preference: preference, apply: apply }; + })(); + </script> + + <link rel="preconnect" href="https://fonts.googleapis.com" /> + <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin /> + <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=IBM+Plex+Sans:wght@400;500;600&family=JetBrains+Mono:wght@400;500;600&display=swap" /> + + <link rel="stylesheet" href="@Assets["css/tokens.css"]" /> + <link rel="stylesheet" href="@Assets["css/base.css"]" /> + <link rel="stylesheet" href="@Assets["css/components.css"]" /> + <link rel="stylesheet" href="@Assets["css/landing.css"]" /> + <ImportMap /> + + @* Same render mode as the app itself, so <PageTitle> keeps working after hydration. *@ + <HeadOutlet @rendermode="InteractiveWebAssembly" /> +</head> + +<body> + @* Prerendered on the server, then hydrated by the WebAssembly runtime. Every database this site opens + lives in the browser - OPFS, IndexedDB, Cache Storage or memory - so none of it can open during the + prerender pass: each page is readable straight from the served HTML, and its data loads once the + runtime is interactive (Components/SessionLifecycle.razor). InteractiveWebAssembly rather than Auto or + Server for the same reason: the worker that owns the database has to run in the user's tab. *@ + <Routes @rendermode="InteractiveWebAssembly" /> + + <div id="blazor-error-ui" data-nosnippet> + An unhandled error has occurred. + <a href="." class="reload">Reload</a> + <span class="dismiss">🗙</span> + </div> + + @* Loaded before the Blazor script so window.bsqliteDemo exists by the time the app boots. The library's + own modules (the worker host, the engine, the VFSes) are imported on demand and need no tag. *@ + <script src="@Assets["js/demo.js"]"></script> + <script src="@Assets["_framework/blazor.web.js"]"></script> +</body> + +</html> diff --git a/src/Bsqlite/Demo/Server/Components/_Imports.razor b/src/Bsqlite/Demo/Server/Components/_Imports.razor new file mode 100644 index 00000000000..e4b0b6be264 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Components/_Imports.razor @@ -0,0 +1,12 @@ +@using System.Net.Http +@using System.Net.Http.Json +@using Microsoft.AspNetCore.Components.Forms +@using Microsoft.AspNetCore.Components.Routing +@using Microsoft.AspNetCore.Components.Web +@using static Microsoft.AspNetCore.Components.Web.RenderMode +@using Microsoft.AspNetCore.Components.Web.Virtualization +@using Microsoft.JSInterop +@using Bit.Bsqlite.Demo.Server +@using Bit.Bsqlite.Demo.Client +@using Bit.Bsqlite.Demo.Client.Layout +@using Bit.Bsqlite.Demo.Server.Components diff --git a/src/Bsqlite/Demo/Server/Controllers/McpController.cs b/src/Bsqlite/Demo/Server/Controllers/McpController.cs new file mode 100644 index 00000000000..eb2224bf087 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Controllers/McpController.cs @@ -0,0 +1,280 @@ +using System.Reflection; +using System.ComponentModel; +using ModelContextProtocol.Server; +using Bit.Bsqlite.Demo.Server.Dtos; +using Bit.Bsqlite.Demo.Server.Services; +using System.ComponentModel.DataAnnotations; + +namespace Bit.Bsqlite.Demo.Server.Controllers; + +/// <summary> +/// The Bsqlite MCP server: the tools an AI agent calls to write browser database code with +/// Bit.Bsqlite without guessing at its API - or at where its data will end up. +/// <para> +/// Every tool answers from the shipped library or from this site's own content - the XML documentation +/// compiled into the five packages, the README and the guides, the storage providers' declared +/// capabilities, the library's own selection and SQL guards, and the demo's source files - so an agent +/// gets what the current version actually does rather than a snapshot someone wrote down. The same +/// methods are exposed as plain HTTP GET endpoints under /api/mcp/..., which makes each of them +/// inspectable from a browser - and is what the demo's own MCP page calls to show them working live. +/// </para> +/// <para> +/// One rule holds across the reference tools: a tool takes the key of the thing you want and answers +/// with that thing; leave the key out and it answers with the index of what there is. That is what a +/// listing tool used to be - and a listing tool costs a name, a description and a schema in every +/// request an agent makes, forever, to say what an optional argument says for nothing. +/// </para> +/// <para> +/// Each tool carries the two annotations that mean something for a tool which only reads: +/// <c>ReadOnly</c>, which is what lets a client run it without stopping to ask a person, and +/// <c>OpenWorld = false</c>, which says the answers come from a closed, known body of material rather +/// than from the internet. Neither lab executes SQL: the storage planner opens nothing, and the SQL +/// check is the library's pattern-matching guards, run over text. +/// </para> +/// <para> +/// Only the three tools whose answer is machine-actionable - a ranked hit carrying its own follow-up +/// call, a storage plan, a per-backend verdict - set <c>UseStructuredContent</c> and publish an output +/// schema. The rest answer in Markdown: a structured answer goes over the wire twice, once as the object +/// and once as the JSON text the spec asks for on its behalf, and documentation read as Markdown is +/// smaller than either half. +/// </para> +/// </summary> +[ApiController] +[McpServerToolType] +[Route("api/[controller]/[action]")] +public class McpController : ControllerBase +{ + // The README, the longest guide and the longest setup guide - which quotes eight demo files whole - + // land far below the cap; it is there so a pathological answer cannot dominate a client's context window. + private const int MaxDocumentLength = 40_000; + + [HttpGet] + [McpServerTool(Name = nameof(SearchBsqlite), Title = "Search everything about Bit.Bsqlite", ReadOnly = true, OpenWorld = false, UseStructuredContent = true)] + [Description("Searches everything known about Bit.Bsqlite at once - the README and every guide, every public type and member of its five packages, the storage backends, the setup guides, the two labs and the demo's source files - and returns the best matches, each with the exact follow-up call that returns its full text. Call it first whenever you do not already know which document or type holds the answer, then call the hit's tool verbatim. Example queries: 'live query', 'opfs fallback indexeddb', 'prerendering', 'migrate to another backend', 'unit test without a browser', 'attach'.")] + public BsqliteSearchResultDto SearchBsqlite( + [Description("What you are looking for, in the words you would use for it - a feature, a symptom, or a member name. A few words rank better than a whole sentence.")] string query, + [Description("How many hits to return. Clamped to 1..50.")] int limit = 12) + { + return BsqliteSearchIndex.Search(query, limit); + } + + [HttpGet] + [McpServerTool(Name = nameof(GetBsqliteSetupGuide), Title = "Setup guide for one scenario", ReadOnly = true, OpenWorld = false)] + [Description("Gets the complete wiring for one Bit.Bsqlite scenario, with real code: 'standalone-wasm' (Blazor WebAssembly), 'wasm' (Blazor Web App on InteractiveWebAssembly with prerendering - how this demo is built, with its real files), 'auto' (InteractiveAuto), 'server' (InteractiveServer: not supported, and what to do instead), 'storage' (choosing and adding storage backends), 'move-from-besql' (keeping a Bit.Besql app's data), 'testing' (Bit.Bsqlite.Testing: the in-process transport and the conformance kit) or 'custom-provider' (writing a storage backend). Call this before writing setup code: where the services are registered, when a page may open the database, and which backend is first are the mistakes that compile.")] + public string GetBsqliteSetupGuide( + // Declared rather than merely described: the values then sit in the JSON schema itself, where a + // model cannot pass another one past them. + [AllowedValues("standalone-wasm", "wasm", "auto", "server", "storage", "move-from-besql", "testing", "custom-provider")] + [Description("The scenario.")] string scenario) + { + var guide = BsqliteSetupGuide.Get(scenario); + + return guide is null + ? $"'{scenario}' is not a known scenario. Use one of: {string.Join(", ", BsqliteSetupGuide.Scenarios)}." + : Truncate(guide); + } + + [HttpGet] + [McpServerTool(Name = nameof(GetBsqliteDocument), Title = "Read a document or one of its sections", ReadOnly = true, OpenWorld = false)] + [Description("Gets one Bit.Bsqlite document as Markdown, or one section of it, with its code samples: the README ('readme'), a guide ('getting-started', 'storage', 'live-queries', 'architecture', 'limits', 'extending') or a package's own README ('Bit.Bsqlite.Testing', 'Bit.Bsqlite.Storage.Opfs', ...). Sub-sections are included, and name and heading matching ignore case and punctuation. Omit the name for the index of documents with their sections and sizes; omit the section for the whole document.")] + public string GetBsqliteDocument( + [Description("The document's name, e.g. 'storage', 'readme' or 'Bit.Bsqlite.Testing'. Omit it for the index of documents.")] string? name = null, + [Description("One heading of that document, e.g. 'Selection' or 'First-party backends'; a heading a document uses twice is told apart as 'Parent > Heading', as the index shows. Omit it for the whole document.")] string? section = null) + { + if (string.IsNullOrWhiteSpace(name)) return BsqliteSourceCatalog.RenderDocumentIndex(); + + var resolved = BsqliteSourceCatalog.ResolveDocumentName(name); + + if (resolved is null) + { + return $"There is no Bit.Bsqlite document called '{name}'. The documents are: " + + $"{string.Join(", ", BsqliteSourceCatalog.Documents.Select(document => $"'{document.Name}'"))}."; + } + + if (string.IsNullOrWhiteSpace(section)) return Truncate(BsqliteSourceCatalog.GetDocument(resolved)!); + + var text = BsqliteSourceCatalog.GetDocumentSection(resolved, section); + + if (text is not null) return Truncate(text); + + var sections = BsqliteSourceCatalog.GetSections(resolved); + var candidates = sections + .Where(s => s.Heading.Contains(section, StringComparison.OrdinalIgnoreCase) || section.Contains(s.Heading, StringComparison.OrdinalIgnoreCase)) + .Select(s => $"'{s.Key}'") + .Distinct(StringComparer.Ordinal) + .Take(10) + .ToArray(); + + return candidates.Length > 0 + ? $"The '{resolved}' document has no section called '{section}'. Did you mean: {string.Join(", ", candidates)}?" + : $"The '{resolved}' document has no section called '{section}'. Its sections are: {string.Join(", ", sections.Select(s => $"'{s.Key}'"))}."; + } + + [HttpGet] + [McpServerTool(Name = nameof(GetBsqliteApi), Title = "Public API reference", ReadOnly = true, OpenWorld = false)] + [Description("Gets the public API of the Bit.Bsqlite packages as Markdown, read out of the shipped assemblies: one type with its namespace and package, every constructor, property, method, event or enum value it has - each with its C# type, its real default value and its documentation. Omit the type name for the index of every public type, grouped by package and namespace. Call it before using a member you are unsure about, e.g. 'BsqliteBuilder', 'BsqliteStorageSelectionBuilder', 'BsqliteDatabase', 'IBsqliteDbContextFactory', 'BsqliteDbContextOptions', 'BsqliteConnection', 'BsqliteLiveQuery', 'BsqliteStorageResolution', 'IBsqliteStorageProvider', 'BsqliteInProcessTransport'.")] + public string GetBsqliteApi( + [Description("The type's name without its namespace, e.g. 'BsqliteDatabase' or 'BsqliteStorageCapabilities'. Omit it for the index of types.")] string? typeName = null) + { + if (string.IsNullOrWhiteSpace(typeName)) return BsqliteApiCatalog.RenderIndex(); + + var rendered = BsqliteApiCatalog.RenderType(typeName); + + if (rendered is not null) return Truncate(rendered); + + var candidates = BsqliteApiCatalog.Types + .Where(t => t.Name.Contains(typeName, StringComparison.OrdinalIgnoreCase)) + .Select(t => t.Name) + .Take(10) + .ToArray(); + + return candidates.Length > 0 + ? $"Bit.Bsqlite has no public type called '{typeName}'. Did you mean: {string.Join(", ", candidates)}?" + : $"Bit.Bsqlite has no public type called '{typeName}'. Its public types are prefixed 'Bsqlite' or 'IBsqlite' (e.g. 'BsqliteConnection' rather than 'SqliteConnection', 'IBsqliteDbContextFactory' rather than 'DbContextFactory'), except the DI entry point class 'BitBsqlite'; call GetBsqliteApi with no type name for the index."; + } + + [HttpGet] + [McpServerTool(Name = nameof(GetBsqliteStorageProviders), Title = "Storage backends and their capabilities", ReadOnly = true, OpenWorld = false)] + [Description("Gets the four first-party storage backends - opfs, indexeddb, cache-storage, in-memory - read out of their provider types: each one's package and namespace, the engine build it needs, whether it persists, the rest of its declared capabilities with what each means, its VFS module, and which statements the library's guards refuse on it (WAL, ATTACH, PRAGMA page_size). Omit the provider for the table of all four. Use it to pick backends and their order; PlanBsqliteStorageSelection then predicts which one a database lands on.")] + public string GetBsqliteStorageProviders( + [Description("The backend's name, e.g. 'opfs' or 'indexeddb' (its provider type, e.g. 'BsqliteOpfsStorageProvider', also works). Omit it for the table of every backend.")] string? provider = null) + { + if (string.IsNullOrWhiteSpace(provider)) return BsqliteStorageCatalog.RenderIndex(); + + return BsqliteStorageCatalog.Render(provider) + ?? $"No first-party storage backend is called '{provider}'. The backends are: {string.Join(", ", BsqliteStorageCatalog.Providers.Select(p => p.Name))}. A backend of your own implements IBsqliteStorageProvider - see GetBsqliteSetupGuide(scenario: \"custom-provider\")."; + } + + [HttpGet] + [McpServerTool(Name = nameof(PlanBsqliteStorageSelection), Title = "Plan a storage selection", ReadOnly = true, OpenWorld = false, UseStructuredContent = true)] + [Description("Runs Bit.Bsqlite's own storage selection - its resolver, migrator and session factory, as AddBitBsqliteServices composes them - on a configuration and a browser you describe, and reports what the library decides: the backend the database opens on (or the BsqliteStorageUnavailableException it throws), every candidate's status with the library's own explanation, whether the sticky binding decided, the binding before and after, any automatic migration and whether it copied data, what the library logged, the equivalent C# registration, and notes on what to change. First-party backends keep the capabilities their provider types declare; nothing is opened and no SQL runs. Example: candidates 'opfs, indexeddb', unavailable 'opfs', existingBinding 'indexeddb', migrationMode 'AutomaticOnOpen'.")] + public Task<BsqliteStoragePlanDto> PlanBsqliteStorageSelection( + [Description("The preference order, most preferred first, comma-separated: what UseStorage(s => s.Prefer(...).Fallback(...)) lists. Default 'opfs, indexeddb'.")] string? candidates = null, + [Description("Optional. The providers registered with AddStorageProvider, comma-separated - including ones that are not candidates. Default: every candidate.")] string? registered = null, + [Description("Optional. The backends whose probe reports them unusable in this browser, comma-separated, e.g. 'opfs' for a browser without OPFS.")] string? unavailable = null, + [Description("Optional. Custom (not first-party) backends that do not survive a reload, comma-separated. First-party backends use their declared persistence.")] string? nonPersistent = null, + [Description("Whether AllowNonPersistentFallback() is called. Default false, as in the library.")] bool allowNonPersistentFallback = false, + [AllowedValues("KeepExisting", "Manual", "AutomaticOnOpen")] + [Description("WithMigrationMode(...): what happens when a better backend is available for an existing database. Default KeepExisting, as in the library.")] string migrationMode = "KeepExisting", + [Description("Optional. The backend the database already lives on - its sticky binding from an earlier session. Omit it for a database opened for the first time.")] string? existingBinding = null, + [Description("Optional. The free space every available backend's probe reports, in bytes. A migration needs room for the database (the plan's stand-in is 64 KB), so a smaller value shows a migration that fails safely.")] long? freeBytes = null, + [Description("Optional. The name passed to AddDatabase. Default 'app.db'.")] string? databaseName = null, + CancellationToken cancellationToken = default) + { + return BsqliteStorageLab.PlanAsync(candidates, registered, unavailable, nonPersistent, allowNonPersistentFallback, + migrationMode, existingBinding, freeBytes, databaseName, cancellationToken); + } + + [HttpGet] + [McpServerTool(Name = nameof(CheckBsqliteSql), Title = "Check SQL against the backends' guards", ReadOnly = true, OpenWorld = false, UseStructuredContent = true)] + [Description("Runs Bit.Bsqlite's own static SQL rules on a statement or a short script without executing it: whether BsqliteFeatureGuards refuses it on each first-party backend - under the limits that backend's capabilities give a session - with the exact BsqliteException message a command would throw, the tables BsqliteTableNames reads out of it (what a live query over it watches), and whether it counts as a write or as DDL. Omit the SQL for the statements the guards exist for, judged on every backend. Example: 'PRAGMA page_size = 8192', a migration's script, or what a LINQ query's ToQueryString() returns.")] + public BsqliteSqlCheckDto CheckBsqliteSql( + [Description("The SQL to check, up to 10,000 characters. It is pattern-matched, never executed. Omit it for the guard reference.")] string? sql = null) + { + return BsqliteSqlCheck.Check(sql); + } + + [HttpGet] + [McpServerTool(Name = nameof(GetBsqliteSourceFile), Title = "Read a source file", ReadOnly = true, OpenWorld = false)] + [Description("Gets one working Bit.Bsqlite demo file verbatim - e.g. 'Demo/Client/Extensions/IServiceCollectionExtensions.cs' (the registration, every backend), 'Demo/Client/Components/SessionLifecycle.razor' (opening the database only once interactive), 'Demo/Client/BrowserDatabase.cs' (moving a database between backends), 'Demo/Client/Pages/Queries.razor' (live queries and raw ADO.NET), 'Demo/Client/Pages/Storage.razor' (the resolution, capabilities and probes), 'Demo/Data/AppDbContext.cs' (the model both sides share) or 'Demo/Server/Program.cs'. Omit the path for the index of files. These are worked examples - for what a member means, GetBsqliteApi answers in a fraction of the characters.")] + public string GetBsqliteSourceFile( + [Description("The file's path as the index lists it, e.g. 'Demo/Client/Pages/Storage.razor'. Omit it for the index of files.")] string? path = null) + { + if (string.IsNullOrWhiteSpace(path)) return BsqliteSourceCatalog.RenderSourceIndex(); + + var content = BsqliteSourceCatalog.GetSourceFile(path); + + if (content is null) + { + var candidates = BsqliteSourceCatalog.SourceFiles + .Where(f => f.Path.Contains(path, StringComparison.OrdinalIgnoreCase)) + .Select(f => f.Path) + .Take(10) + .ToArray(); + + return candidates.Length > 0 + ? $"No source file at '{path}'. Did you mean: {string.Join(", ", candidates)}?" + : $"No source file at '{path}'. Call GetBsqliteSourceFile with no path for the index of files."; + } + + return Truncate(content); + } + + /// <summary> + /// Everything this server exposes over MCP - its tools, prompts and resources - read off the + /// attributes that declare them. + /// <para> + /// Deliberately not an MCP tool: an agent already gets this from the protocol's own + /// <c>tools/list</c>. It exists for the /mcp-server demo page, which documents the server to a human + /// reader. Reflecting over the registrations rather than restating them means the page cannot + /// describe a tool that no longer exists, or miss one that was just added. + /// </para> + /// </summary> + [HttpGet] + public BsqliteMcpCatalogDto GetMcpCatalog() + { + return new BsqliteMcpCatalogDto + { + Tools = [.. typeof(McpController) + .GetMethods(BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly) + .Select(method => (Method: method, Tool: method.GetCustomAttribute<McpServerToolAttribute>())) + .Where(entry => entry.Tool is not null) + .Select(entry => new BsqliteMcpMemberDto + { + Name = entry.Tool!.Name ?? entry.Method.Name, + Title = entry.Tool.Title, + Description = entry.Method.GetCustomAttribute<DescriptionAttribute>()?.Description ?? string.Empty, + Parameters = [.. entry.Method.GetParameters().Where(IsArgument).Select(FormatParameter)] + }) + .OrderBy(tool => tool.Name, StringComparer.Ordinal)], + + Prompts = [.. typeof(McpPrompts) + .GetMethods(BindingFlags.Public | BindingFlags.Static | BindingFlags.DeclaredOnly) + .Select(method => (Method: method, Prompt: method.GetCustomAttribute<McpServerPromptAttribute>())) + .Where(entry => entry.Prompt is not null) + .Select(entry => new BsqliteMcpMemberDto + { + Name = entry.Prompt!.Name ?? entry.Method.Name, + Title = entry.Prompt.Title, + Description = entry.Method.GetCustomAttribute<DescriptionAttribute>()?.Description ?? string.Empty, + Parameters = [.. entry.Method.GetParameters().Where(IsArgument).Select(FormatParameter)] + }) + .OrderBy(prompt => prompt.Name, StringComparer.Ordinal)], + + Resources = [.. typeof(McpResources) + .GetMethods(BindingFlags.Public | BindingFlags.Static | BindingFlags.DeclaredOnly) + .Select(method => (Method: method, Resource: method.GetCustomAttribute<McpServerResourceAttribute>())) + .Where(entry => entry.Resource is not null) + .Select(entry => new BsqliteMcpMemberDto + { + Name = entry.Resource!.UriTemplate ?? entry.Resource.Name ?? string.Empty, + Title = entry.Resource.Title, + Description = entry.Method.GetCustomAttribute<DescriptionAttribute>()?.Description ?? string.Empty, + Parameters = [] + }) + .OrderBy(resource => resource.Name, StringComparer.Ordinal)] + }; + } + + /// <summary>A parameter the caller supplies - not the cancellation token the transport supplies.</summary> + private static bool IsArgument(ParameterInfo parameter) => parameter.ParameterType != typeof(CancellationToken); + + /// <summary>A tool or prompt parameter, as "name: type" with "?" for the optional ones.</summary> + private static string FormatParameter(ParameterInfo parameter) + { + var type = BsqliteApiCatalog.FriendlyName(parameter.ParameterType); + + return parameter.HasDefaultValue ? $"{parameter.Name}?: {type.TrimEnd('?')}" : $"{parameter.Name}: {type}"; + } + + internal static string Truncate(string text) + { + if (text.Length <= MaxDocumentLength) return text; + + // Never between the two halves of a surrogate pair: half a pair is not text, and a client that + // re-encodes the answer turns it into a replacement character or rejects it outright. + var cut = char.IsHighSurrogate(text[MaxDocumentLength - 1]) ? MaxDocumentLength - 1 : MaxDocumentLength; + + return $"{text[..cut]}\n\n[truncated - the full text is longer than {MaxDocumentLength} characters]"; + } +} diff --git a/src/Bsqlite/Demo/Server/Controllers/McpPrompts.cs b/src/Bsqlite/Demo/Server/Controllers/McpPrompts.cs new file mode 100644 index 00000000000..1bee98ba2ef --- /dev/null +++ b/src/Bsqlite/Demo/Server/Controllers/McpPrompts.cs @@ -0,0 +1,119 @@ +using System.ComponentModel; +using ModelContextProtocol.Server; +using System.ComponentModel.DataAnnotations; + +namespace Bit.Bsqlite.Demo.Server.Controllers; + +/// <summary> +/// Ready-made workflows for the three things people actually ask a browser database for: add it to an +/// app, move an existing Bit.Besql app onto it without losing anyone's data, and work out why a +/// database does not open, opens empty, or will not take a statement. +/// <para> +/// Each prompt spends its words on the order to call the tools in and on the mistakes that are +/// expensive to make - the ones that produce a page that fails only when prerendered, or a returning +/// user who silently starts on an empty database, rather than a compiler error - because the failure +/// mode of an agent with a handful of tools is not ignorance, it is calling them in a sequence that +/// skips the check which would have caught the bug. +/// </para> +/// </summary> +[McpServerPromptType] +public static class McpPrompts +{ + [McpServerPrompt(Name = "add-bsqlite-to-app", Title = "Add Bit.Bsqlite to a Blazor app")] + [Description("Walks through adding Bit.Bsqlite - EF Core on SQLite in the browser - to an existing Blazor app, in the right order for its hosting model.")] + public static string AddBsqliteToApp( + // The values go into the prompt's own argument schema, which is what a client offers as + // completions - so picking the hosting model is a choice from a list rather than a spelling test. + [AllowedValues("unknown", "standalone-wasm", "wasm", "auto", "server")] + [Description("How the app is hosted: standalone-wasm, wasm (Blazor Web App on InteractiveWebAssembly), auto or server. Pass 'unknown' to have it determined from the project first.")] string hosting = "unknown") + { + return $""" + Add Bit.Bsqlite to this Blazor app. Its hosting model is: {hosting}. + + Work in this order: + + 1. If the hosting model is 'unknown', determine it from the project first: look for + WebAssemblyHostBuilder (standalone-wasm), AddInteractiveWebAssemblyComponents and a .Client project (wasm), + both of those plus AddInteractiveServerComponents (auto), or only AddInteractiveServerComponents (server). + 2. Call `GetBsqliteSetupGuide` with that scenario and follow it. For 'server' it explains why the pages that use + the database have to run on WebAssembly - do not try to make Interactive Server work. + 3. Call `GetBsqliteStorageProviders` and pick the backends: OPFS first and IndexedDB as its fallback is the + default answer; in-memory only with `AllowNonPersistentFallback()`. Then call `PlanBsqliteStorageSelection` + with that preference order - once as is and once with 'opfs' unavailable - and check where the database lands. + 4. In a Blazor Web App, register the services in BOTH the client and the server project (one shared extension + method), and open the database only once `RendererInfo.IsInteractive` is true. Missing either is the most + common mistake: it fails during prerendering, not at compile time. + 5. Call `GetBsqliteApi` for every Bsqlite type you use and copy its namespace: `AddBitBsqliteServices` and + `AddBitBsqliteDbContext` are in `Microsoft.Extensions.DependencyInjection`, `IBsqliteDbContextFactory` in + `Bit.Bsqlite.EntityFrameworkCore`, the providers in `Bit.Bsqlite.Storage.*`. + 6. Use the async EF Core API throughout; replace any `SaveChanges()`, `ToList()` or `First()` you touch. + 7. Build the app and fix what the compiler says. + + Show me the diff for each file you change, and say which project the registration and the pages live in and why. + """; + } + + [McpServerPrompt(Name = "move-from-besql", Title = "Move a Bit.Besql app to Bit.Bsqlite")] + [Description("Moves an app from Bit.Besql to Bit.Bsqlite so that every existing user's database is imported rather than lost.")] + public static string MoveFromBesql( + [Description("The Data Source the app passes to UseSqlite under Bit.Besql, e.g. 'Offline-Client.db'. Pass 'unknown' to have it found in the project first.")] string dataSource = "unknown") + { + return $""" + Move this app from Bit.Besql to Bit.Bsqlite without losing any user's data. Besql's Data Source is: {dataSource}. + + Work in this order: + + 1. If the Data Source is 'unknown', find it first: the `UseSqlite("Data Source=...")` inside + `AddBesqlDbContextFactory`. That exact file name is the key: Bsqlite's cache-storage backend imports the + database Besql left under `/data/cache/<that name>`, and only under it. + 2. Call `GetBsqliteSetupGuide` with scenario 'move-from-besql' and follow it. + 3. Register `BsqliteCacheStorageProvider` and make 'cache-storage' the FIRST candidate, and pass the Data Source + to `AddDatabase`. Then prove it with `PlanBsqliteStorageSelection`: with no existing binding (a user opening + the new version for the first time) the database must open on 'cache-storage'. Any other backend ranked + first gives every returning user an empty database there, silently. + 4. Remove Bit.Besql's script tag and its `ConfigureSqliteJournalMode()` calls, move `dbContextInitializer` work + to `OnFirstUse` (migrations already run by default) and a compiled model to `options.Configure`. + 5. Replace every synchronous EF call with its async overload: they worked under Besql and throw + `BsqliteSynchronousApiNotSupportedException` under Bsqlite. Search the code for `SaveChanges()`, + `.ToList()`, `.First(`, `.Single(`, `.Count()` and `CreateDbContext()`. + 6. If the app should end up on OPFS, do not reorder the candidates: migrate deliberately with + `BsqliteStorageMigrator` once the database is bound to 'cache-storage' - fetch + `GetBsqliteSourceFile(path: "Demo/Client/BrowserDatabase.cs")` for the demo's working version. + 7. Build the app and fix what the compiler says. + + Show me the diff for each file you change, and the PlanBsqliteStorageSelection result that shows a first open + lands on cache-storage. + """; + } + + [McpServerPrompt(Name = "debug-bsqlite-storage", Title = "Debug a database that does not open or behave")] + [Description("Diagnoses a Bit.Bsqlite problem - a database that does not open, opens empty, lands on the wrong backend, loses its data on reload, refuses a statement, or a live query that does not refresh.")] + public static string DebugBsqliteStorage( + [Description("What goes wrong, with the exception message, the browser and the backend involved if you know them.")] string symptom) + { + return $""" + Diagnose this Bit.Bsqlite problem: {symptom} + + Work in this order: + + 1. Check the wiring against `GetBsqliteSetupGuide` for the app's hosting model first: services registered in + every container that renders the page (the server too, when prerendering), the database opened only once + `RendererInfo.IsInteractive` is true, the page running on WebAssembly, async EF calls only. + 2. For anything about which backend a database is on - empty after a reload, empty after a browser update, + `BsqliteStorageUnavailableException` - reproduce it with `PlanBsqliteStorageSelection`: the app's preference + order, the backends this browser lacks, and the existing binding if there is one. It runs the library's own + selection and reports every candidate's status. Remember that existing data outranks preference, and that a + non-persistent fallback needs `AllowNonPersistentFallback()`. + 3. For a statement that is refused, pass it to `CheckBsqliteSql`: it runs the library's guards per backend and + returns the exact message. WAL is refused everywhere; ATTACH and PRAGMA page_size depend on the backend's + declared capabilities (`GetBsqliteStorageProviders`). + 4. For a live query that does not refresh, `CheckBsqliteSql` also shows the tables a query over that SQL + watches; then read `GetBsqliteDocument(name: "live-queries")` - nothing is reported while a transaction is + open, and a disposed query stops listening. + 5. Call `SearchBsqlite` with the exception's type or message, and confirm the API you rely on with + `GetBsqliteApi` - the actual default of the option involved - before concluding it is a bug. + + Tell me the cause and the fix, and cite the tool call that established it. + """; + } +} diff --git a/src/Bsqlite/Demo/Server/Controllers/McpResources.cs b/src/Bsqlite/Demo/Server/Controllers/McpResources.cs new file mode 100644 index 00000000000..9b2b34f5450 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Controllers/McpResources.cs @@ -0,0 +1,102 @@ +using System.ComponentModel; +using ModelContextProtocol.Server; +using ModelContextProtocol.Protocol; +using Bit.Bsqlite.Demo.Server.Services; + +namespace Bit.Bsqlite.Demo.Server.Controllers; + +/// <summary> +/// The same body of knowledge the tools serve, exposed as MCP resources. +/// <para> +/// Tools are for an agent that has decided what it needs; resources are for a client that wants to +/// attach documentation to a conversation up front, or let a person browse and pin it. Both read the +/// same catalogs, so neither can go stale relative to the other - and where a tool and a resource take +/// the same argument, they resolve it through the same method, so a key that works in one cannot fail +/// in the other. +/// </para> +/// <para> +/// A client showing <c>bsqlite://docs/{name}</c> in a picker cannot fill it in, because nothing on the +/// wire says which documents exist. Two things close that gap - the documents and the setup guides are +/// also listed as concrete resources (see <c>ListDocuments</c>, wired up in Program.cs), and every +/// placeholder here is completable through <c>completion/complete</c>. +/// </para> +/// </summary> +[McpServerResourceType] +public static class McpResources +{ + [McpServerResource(UriTemplate = "bsqlite://guide", Name = "bsqlite-guide", Title = "Bit.Bsqlite README", MimeType = "text/markdown")] + [Description("The library's README in full: what it is, the packages, installation, where it runs, what it does not claim.")] + public static string Guide() => BsqliteSourceCatalog.Readme; + + [McpServerResource(UriTemplate = "bsqlite://docs/{name}", Name = "bsqlite-document", Title = "Document", MimeType = "text/markdown")] + [Description("One Bit.Bsqlite document in full - a guide or a package README - e.g. bsqlite://docs/storage or bsqlite://docs/Bit.Bsqlite.Testing.")] + public static string Document(string name) + => BsqliteSourceCatalog.GetDocument(name) ?? $"There is no Bit.Bsqlite document called '{name}'."; + + [McpServerResource(UriTemplate = "bsqlite://api", Name = "bsqlite-api", Title = "Bit.Bsqlite public API", MimeType = "text/markdown")] + [Description("Every public type of the five Bit.Bsqlite packages with its kind, namespace and summary.")] + public static string ApiList() => BsqliteApiCatalog.RenderIndex(); + + [McpServerResource(UriTemplate = "bsqlite://api/{typeName}", Name = "bsqlite-api-type", Title = "Type reference", MimeType = "text/markdown")] + [Description("The full reference of one Bit.Bsqlite type, e.g. bsqlite://api/BsqliteDatabase.")] + public static string ApiType(string typeName) + => BsqliteApiCatalog.RenderType(typeName) ?? $"Bit.Bsqlite has no public type called '{typeName}'."; + + [McpServerResource(UriTemplate = "bsqlite://storage", Name = "bsqlite-storage", Title = "Storage backends", MimeType = "text/markdown")] + [Description("The four first-party storage backends with the capabilities their provider types declare.")] + public static string Storage() => BsqliteStorageCatalog.RenderIndex(); + + [McpServerResource(UriTemplate = "bsqlite://storage/{provider}", Name = "bsqlite-storage-backend", Title = "Storage backend", MimeType = "text/markdown")] + [Description("One storage backend in full, with what each capability means and what the guards refuse on it, e.g. bsqlite://storage/opfs.")] + public static string StorageBackend(string provider) + => BsqliteStorageCatalog.Render(provider) ?? $"No first-party storage backend is called '{provider}'."; + + [McpServerResource(UriTemplate = "bsqlite://setup/{scenario}", Name = "bsqlite-setup", Title = "Setup guide", MimeType = "text/markdown")] + [Description("The complete wiring for one scenario, e.g. bsqlite://setup/wasm or bsqlite://setup/move-from-besql.")] + public static string Setup(string scenario) + => BsqliteSetupGuide.Get(scenario) ?? $"'{scenario}' is not a known scenario."; + + [McpServerResource(UriTemplate = "bsqlite://source/{path}", Name = "bsqlite-source", Title = "Source file", MimeType = "text/plain")] + [Description("One source file of the demo, e.g. bsqlite://source/Demo%2FClient%2FComponents%2FSessionLifecycle.razor.")] + public static string Source(string path) + => BsqliteSourceCatalog.GetSourceFile(path) ?? $"No source file at '{path}'."; + + /// <summary> + /// The documents and the setup guides as resources a client can list, show and pin, rather than as + /// placeholders someone has to already know how to fill in. + /// <para> + /// These sit alongside the attributed resources above rather than replacing the templates: the + /// template is what actually serves a read - the exact URI is looked for first and the templates are + /// matched afterwards - while these give <c>resources/list</c> something a person can choose from. + /// The API types and the source files stay templated, because listing every one of them would bury + /// the handful anyone browses to. + /// </para> + /// </summary> + public static IEnumerable<Resource> ListDocuments() + { + var documents = BsqliteSourceCatalog.Documents.Select(document => new Resource + { + Uri = $"bsqlite://docs/{Uri.EscapeDataString(document.Name)}", + Name = $"bsqlite-docs-{Slug(document.Name)}", + Title = $"Docs: {document.Title}", + Description = document.Summary is null + ? $"The '{document.Name}' document ({document.Lines} lines)." + : $"{document.Summary} ({document.Lines} lines)", + MimeType = "text/markdown" + }); + + var guides = BsqliteSetupGuide.Scenarios.Select(scenario => new Resource + { + Uri = $"bsqlite://setup/{scenario}", + Name = $"bsqlite-setup-{scenario}", + Title = $"Setup: {BsqliteSetupGuide.Title(scenario)}", + Description = $"The complete wiring for {BsqliteSetupGuide.Title(scenario)}.", + MimeType = "text/markdown" + }); + + return documents.Concat(guides); + } + + private static string Slug(string name) + => string.Join('-', name.ToLowerInvariant().Split(name.Where(c => char.IsLetterOrDigit(c) is false).Distinct().ToArray(), StringSplitOptions.RemoveEmptyEntries)); +} diff --git a/src/Bsqlite/Demo/Server/Dtos/BsqliteMcpDtos.cs b/src/Bsqlite/Demo/Server/Dtos/BsqliteMcpDtos.cs new file mode 100644 index 00000000000..ae80b11c51d --- /dev/null +++ b/src/Bsqlite/Demo/Server/Dtos/BsqliteMcpDtos.cs @@ -0,0 +1,398 @@ +namespace Bit.Bsqlite.Demo.Server.Dtos; + +/// <summary> +/// One heading of one of the documents this server hands out. The documents are read through +/// GetBsqliteDocument (and the bsqlite://docs resources); this record is how the catalog, the search +/// index and the completions describe a section to each other. +/// </summary> +public record BsqliteDocumentSectionDto +{ + /// <summary>The document the section belongs to, e.g. "storage".</summary> + public required string Document { get; init; } + + /// <summary>The heading text, e.g. "Selection".</summary> + public required string Heading { get; init; } + + /// <summary> + /// What to pass to GetBsqliteDocument as the section for exactly this one: the heading itself, or - + /// for a heading the document uses twice - "Parent > Heading". + /// </summary> + public required string Key { get; init; } + + /// <summary>Markdown heading level: 2 for a top-level section, 3 or 4 for a sub-section.</summary> + public required int Level { get; init; } + + /// <summary>The heading this one sits under, or null when this entry is itself level 2.</summary> + public string? Parent { get; init; } + + /// <summary>Number of markdown lines in the section (including its sub-sections).</summary> + public required int Lines { get; init; } +} + +/// <summary>One document this server can hand out: the README, a guide under docs/, or a package's README.</summary> +public record BsqliteDocumentDto +{ + /// <summary>What to pass to GetBsqliteDocument, e.g. "storage", "readme" or "Bit.Bsqlite.Testing".</summary> + public required string Name { get; init; } + + /// <summary>The document's own title, its first heading.</summary> + public required string Title { get; init; } + + /// <summary>Readme (the repository README), Guide (docs/) or Package (a package's own README).</summary> + public required string Kind { get; init; } + + /// <summary>The document's opening paragraph, cut to its first sentence.</summary> + public string? Summary { get; init; } + + public required int Lines { get; init; } + + /// <summary>The keys of its sections, in reading order.</summary> + public required string[] Sections { get; init; } +} + +/// <summary>A public type of one of the Bit.Bsqlite packages.</summary> +public record BsqliteApiTypeDto +{ + public required string Name { get; init; } + + /// <summary>The namespace to import - Bsqlite spreads its types over several, and the DI entry points live in Microsoft's.</summary> + public required string Namespace { get; init; } + + /// <summary>The package that ships the type, e.g. "Bit.Bsqlite" or "Bit.Bsqlite.Storage.Opfs".</summary> + public required string Package { get; init; } + + /// <summary>Interface, Enum, Exception, Delegate, Static class, Class, Struct or Record.</summary> + public required string Kind { get; init; } + + public string? Summary { get; init; } +} + +/// <summary>A member (constructor, property, method, event or enum value) of a public Bit.Bsqlite type.</summary> +public record BsqliteApiMemberDto +{ + public required string Name { get; init; } + + /// <summary>Constructor, Property, Field, Method, Event or EnumValue.</summary> + public required string Kind { get; init; } + + /// <summary>The C# type of the member, or the signature's return type for a method.</summary> + public string? Type { get; init; } + + /// <summary>The parameter list, e.g. "(string databaseName, CancellationToken cancellationToken = default)".</summary> + public string? Signature { get; init; } + + /// <summary>The value the member has on a freshly created instance, when it could be determined.</summary> + public string? Default { get; init; } + + /// <summary>True for a C# <c>required</c> member: an object initializer that leaves it out does not compile.</summary> + public bool Required { get; init; } + + /// <summary>True for a protected member - what a derived class overrides, as with the conformance kit.</summary> + public bool Protected { get; init; } + + public string? Summary { get; init; } + + /// <summary>The XML remarks, when the member has any - they carry the caveats.</summary> + public string? Remarks { get; init; } +} + +/// <summary>The full reference of one public Bit.Bsqlite type.</summary> +public record BsqliteApiTypeDetailsDto +{ + public required string Name { get; init; } + + public required string FullName { get; init; } + + public required string Namespace { get; init; } + + public required string Package { get; init; } + + public required string Kind { get; init; } + + public string? BaseType { get; init; } + + public string[]? Implements { get; init; } + + public string? Summary { get; init; } + + public string? Remarks { get; init; } + + public required BsqliteApiMemberDto[] Members { get; init; } +} + +/// <summary>A source file this server can hand out, retrievable through GetBsqliteSourceFile.</summary> +public record BsqliteSourceFileDto +{ + /// <summary>The path to pass to GetBsqliteSourceFile, e.g. "Demo/Client/Pages/Storage.razor".</summary> + public required string Path { get; init; } + + /// <summary>Page (a working demo page), Hosting (the wiring of the app), Model (the shared EF Core model) or Shared.</summary> + public required string Kind { get; init; } + + /// <summary>The file's own header comment or page title, when it has one.</summary> + public string? Description { get; init; } + + public required int Lines { get; init; } +} + +/// <summary>One capability a storage backend declares, read off the provider instance.</summary> +public record BsqliteCapabilityDto +{ + /// <summary>The property of BsqliteStorageCapabilities, e.g. "CanChangePageSize".</summary> + public required string Name { get; init; } + + /// <summary>The declared value, as C# would print it: "true", "Synchronous", "DedicatedWorker, Window".</summary> + public required string Value { get; init; } + + /// <summary>What the capability means, from the library's own documentation of the property.</summary> + public string? Summary { get; init; } +} + +/// <summary> +/// One first-party storage backend, read off an instance of its provider type - so every capability +/// here is the one the library's selection and guards act on, not one someone copied into a table. +/// </summary> +public record BsqliteStorageProviderDto +{ + /// <summary>The name selection uses and the sticky binding records, e.g. "opfs".</summary> + public required string Name { get; init; } + + /// <summary>The provider type, e.g. "BsqliteOpfsStorageProvider".</summary> + public required string TypeName { get; init; } + + public required string Namespace { get; init; } + + public required string Package { get; init; } + + public string? Summary { get; init; } + + public string? Remarks { get; init; } + + /// <summary>The VFS name the worker opens the database with, when the provider declares one.</summary> + public string? VfsName { get; init; } + + /// <summary>The ES module the worker imports to register the VFS; null for a backend that needs no JavaScript.</summary> + public string? VfsModule { get; init; } + + /// <summary>The module probe and admin run on the main thread, when the provider declares one.</summary> + public string? AdminModule { get; init; } + + public required BsqliteCapabilityDto[] Capabilities { get; init; } + + /// <summary>The statements the library's own guards refuse on this backend, each with the guard's message.</summary> + public required BsqliteSqlVerdictDto[] Guarded { get; init; } +} + +/// <summary>One result of a search across everything this MCP server knows about Bit.Bsqlite.</summary> +public record BsqliteSearchHitDto +{ + /// <summary>What was found: "Guide section", "API class", "API method", "Storage backend", "Setup guide", "Source file", ...</summary> + public required string Kind { get; init; } + + public required string Title { get; init; } + + /// <summary>Where the hit sits: the owning document, type or group.</summary> + public string? Context { get; init; } + + /// <summary>The tool call that returns the full text of this hit - call it verbatim.</summary> + public required string Tool { get; init; } + + /// <summary>The matching text, with a little of what surrounds it.</summary> + public required string Snippet { get; init; } +} + +/// <summary>What SearchBsqlite answers: the ranked hits, or - when there are none - why, and what to try instead.</summary> +public record BsqliteSearchResultDto +{ + /// <summary>The matches, best first. Empty when nothing matched - Message then says what to do about it.</summary> + public required BsqliteSearchHitDto[] Hits { get; init; } + + /// <summary>True when the ranking was cut short at the requested limit, so a narrower query would show more.</summary> + public bool HasMore { get; init; } + + /// <summary> + /// The words the query was actually searched by - punctuation, duplicates and filler words removed. + /// Present only when nothing matched, which is the one time it explains something the caller could + /// not work out from the query it just sent. + /// </summary> + public string[]? Terms { get; init; } + + /// <summary>Set only when Hits is empty: the reason, and the tool to fall back to.</summary> + public string? Message { get; init; } + + /// <summary>Titles that nearly matched, when nothing matched outright - a query worth retrying with.</summary> + public string[]? DidYouMean { get; init; } +} + +/// <summary>One candidate of a planned storage selection, as described and as the library decided it.</summary> +public record BsqliteCandidateReportDto +{ + public required string Name { get; init; } + + /// <summary>The position in the preference order, 1 being Prefer(...); null for a provider that is registered but not a candidate.</summary> + public int? Rank { get; init; } + + /// <summary>Whether a provider of this name is registered (AddStorageProvider).</summary> + public required bool Registered { get; init; } + + /// <summary>True for one of the four first-party backends, whose capabilities are read off the real provider type.</summary> + public required bool FirstParty { get; init; } + + /// <summary>Whether data on it survives a reload, as its capabilities declare. Null when it is not registered.</summary> + public bool? Persistent { get; init; } + + /// <summary>Whether its probe reports it usable in the described browser. Null when it is not registered.</summary> + public bool? Available { get; init; } + + /// <summary>The engine build it needs: Synchronous or AsyncCapable. Null when it is not registered.</summary> + public string? EngineBuild { get; init; } + + /// <summary> + /// The library's verdict on it (BsqliteStorageCandidateStatus): NotProbed, NotRegistered, Unavailable, + /// RejectedAsNonPersistent, RejectedByExistingBinding or Selected. Null when selection never got to it - + /// an existing database is decided by its binding alone. + /// </summary> + public string? Status { get; init; } + + /// <summary>The library's own explanation of the verdict, or the probe's reason, or why it was not examined.</summary> + public string? Explanation { get; init; } +} + +/// <summary>What happened to an automatic migration during a planned open.</summary> +public record BsqliteMigrationReportDto +{ + public required string From { get; init; } + + public required string To { get; init; } + + /// <summary>Migrated (the binding moved), Failed (nothing moved, the database opened where it was) or NotAttempted.</summary> + public required string Outcome { get; init; } + + /// <summary>Whether a database image was actually copied: false when the source held nothing its admin could export.</summary> + public bool? Copied { get; init; } + + /// <summary>The library's own error, when the migration failed.</summary> + public string? Error { get; init; } +} + +/// <summary> +/// What Bit.Bsqlite's own storage selection does with a described configuration and browser - the +/// answer the library gives at runtime, computed by running its resolver, its migrator and its session +/// factory against stand-in providers on this server. +/// </summary> +public record BsqliteStoragePlanDto +{ + /// <summary>Whether the database opens.</summary> + public required bool Opens { get; init; } + + /// <summary>The backend it opens on.</summary> + public string? Provider { get; init; } + + /// <summary>The engine build the worker would download for that backend: Synchronous or AsyncCapable.</summary> + public string? EngineBuild { get; init; } + + /// <summary>The VFS module the worker would import; null for a backend that needs no JavaScript.</summary> + public string? VfsModule { get; init; } + + /// <summary>Whether the most preferred candidate was chosen.</summary> + public bool? IsFirstChoice { get; init; } + + /// <summary>Whether the sticky binding decided, rather than the preference order.</summary> + public bool? WasDecidedByExistingData { get; init; } + + /// <summary>A backend ranked above the chosen one that is available but does not hold the data.</summary> + public string? BetterProviderAvailable { get; init; } + + /// <summary>The sticky binding before the open: null for a database that does not exist yet.</summary> + public string? BindingBefore { get; init; } + + /// <summary>The sticky binding after the open - written only once the open succeeded.</summary> + public string? BindingAfter { get; init; } + + /// <summary>The automatic migration, when the migration mode and a better backend called for one.</summary> + public BsqliteMigrationReportDto? Migration { get; init; } + + /// <summary>Every provider the plan involves: the candidates in preference order, then any other registered or bound one.</summary> + public required BsqliteCandidateReportDto[] Candidates { get; init; } + + /// <summary> + /// Why the database does not open: the message of the BsqliteStorageUnavailableException the + /// application would get, or the configuration error the builder throws at startup. + /// </summary> + public string? Error { get; init; } + + /// <summary>The registration this plan describes, as the C# that produces it.</summary> + public required string Configuration { get; init; } + + /// <summary>What the library logged while deciding - a fallback is a warning, a better backend left unused is information.</summary> + public required string[] Log { get; init; } + + /// <summary>What the outcome means in practice, and what to change when it is not the one wanted.</summary> + public required string[] Notes { get; init; } +} + +/// <summary>Whether one backend's guards accept a statement.</summary> +public record BsqliteSqlVerdictDto +{ + /// <summary>The backend, or the statement when the verdict is listed per backend.</summary> + public required string Name { get; init; } + + public required bool Accepted { get; init; } + + /// <summary>The BsqliteException message the command layer throws before the statement leaves for the worker.</summary> + public string? Error { get; init; } +} + +/// <summary>What the library's static SQL rules make of one piece of SQL - nothing of it is executed.</summary> +public record BsqliteSqlReportDto +{ + public required string Sql { get; init; } + + /// <summary>The verdict of BsqliteFeatureGuards on every first-party backend, under the limits its capabilities declare.</summary> + public required BsqliteSqlVerdictDto[] Backends { get; init; } + + /// <summary>The tables BsqliteTableNames reads out of it - what a live query over it watches.</summary> + public required string[] Tables { get; init; } + + /// <summary>Whether BsqliteTableNames counts it as a write, which is what notifies live queries when a transport cannot.</summary> + public required bool LooksLikeWrite { get; init; } + + /// <summary>Whether it contains DDL, which re-runs every live query over the tables it names.</summary> + public required bool LooksLikeSchemaChange { get; init; } +} + +/// <summary>What CheckBsqliteSql answers: one report per piece of SQL checked.</summary> +public record BsqliteSqlCheckDto +{ + /// <summary>The SQL passed in, or - when none was - every statement the guards exist for, as a reference.</summary> + public required BsqliteSqlReportDto[] Statements { get; init; } + + /// <summary>Why nothing was checked, when the input could not be.</summary> + public string? Error { get; init; } + + /// <summary>What the verdicts mean, and what to do instead of a refused statement.</summary> + public required string[] Notes { get; init; } +} + +/// <summary>One tool, prompt or resource this server exposes, as the demo's MCP page lists it.</summary> +public record BsqliteMcpMemberDto +{ + public required string Name { get; init; } + + public string? Title { get; init; } + + public required string Description { get; init; } + + /// <summary>The arguments it takes, as "name: type", with "?" marking the optional ones.</summary> + public required string[] Parameters { get; init; } +} + +/// <summary>Everything the MCP server exposes, read off the attributes that declare it.</summary> +public record BsqliteMcpCatalogDto +{ + public required BsqliteMcpMemberDto[] Tools { get; init; } + + public required BsqliteMcpMemberDto[] Prompts { get; init; } + + public required BsqliteMcpMemberDto[] Resources { get; init; } +} diff --git a/src/Bsqlite/Demo/Server/Program.cs b/src/Bsqlite/Demo/Server/Program.cs new file mode 100644 index 00000000000..edf030522e6 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Program.cs @@ -0,0 +1,142 @@ +// The host of the demo: a Blazor Web App that prerenders the client's pages and then runs them in the +// browser on WebAssembly (InteractiveWebAssembly), where every database the site opens lives. +// +// It also runs the demo's AppDbContext itself, with the stock UseSqlite provider: the same entities, the same +// configurations and the same migrations as the browser applies through UseBitBsqlite, which is the claim the +// demo exists to show. + +using Bit.Bsqlite.Demo.Data; +using Bit.Bsqlite.Demo.Server.Components; +using Bit.Bsqlite.Demo.Server.Controllers; +using Bit.Bsqlite.Demo.Server.Services; +using Microsoft.AspNetCore.HttpOverrides; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using ModelContextProtocol.Protocol; +using System.Text.Json.Serialization; + +var builder = WebApplication.CreateBuilder(args); + +builder.Services.AddRazorComponents() + .AddInteractiveWebAssemblyComponents(); + +// The prerender pass instantiates the client's pages in this container, so it registers the very same +// services the WebAssembly container does - Bsqlite's included. Nothing browser-bound runs here: every page +// opens its database only once it is interactive. +builder.Services.AddDemoServices(); + +// The MCP server (Controllers/McpController.cs) and the plain HTTP endpoints that mirror it - the same +// methods, reachable from a browser, which is what the /mcp-server demo page calls to show them live. That +// page's route has to differ from MapMcp's below: two literal endpoints on /mcp would make a GET of it +// ambiguous. A member with nothing in it is left out of the JSON, exactly as the MCP transport leaves it out, +// so the browser view of a tool is the answer an agent gets rather than that answer plus a column of nulls. +builder.Services.AddControllers() + .AddJsonOptions(options => options.JsonSerializerOptions.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull); + +builder.Services.AddMcpServer(options => +{ + // The name a person sees in their client's server list, and the version they would quote in a bug + // report - the library's, not this web app's, because the library is what the answers are about. + options.ServerInfo = new Implementation + { + Name = "bit-bsqlite", + Title = "Bit.Bsqlite", + Version = BsqliteServerInstructions.BsqliteVersion, + Description = "Documentation, public API, storage backends, setup guides, a storage-selection planner and a SQL guard check for Bit.Bsqlite, SQLite in the browser for Blazor with EF Core on top.", + WebsiteUrl = "https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite" + }; + + // Returned from `initialize` and put in front of the model before it has called anything. + options.ServerInstructions = BsqliteServerInstructions.Text; +}) + .WithHttpTransport() + .WithToolsFromAssembly() + .WithResourcesFromAssembly() + .WithPromptsFromAssembly() + // Every key this server takes - a document name, a type name, a backend, a scenario, a source path - comes + // from a closed set nothing on the wire spells out; completions let a client type ahead into it. + .WithCompleteHandler((request, cancellationToken) => ValueTask.FromResult(new CompleteResult + { + Completion = BsqliteCompletions.Complete(request.Params?.Argument?.Name, request.Params?.Argument?.Value) + })) + // Additive to the attributed resources, not a replacement for them: a template cannot be listed as + // something to click, so the documents and the setup guides are enumerated one by one as well. + .WithListResourcesHandler((request, cancellationToken) => ValueTask.FromResult(new ListResourcesResult + { + Resources = [.. McpResources.ListDocuments()] + })); + +// The server's copy of the catalog, in memory, so the deployed demo writes nothing to its server's disk and a +// restart starts it over. A named shared-cache database lives as long as one connection to it is open: this +// one is held for the lifetime of the app, and every context opens a connection of its own next to it, so +// concurrent requests never share one SqliteConnection. +const string serverDatabase = "Data Source=bsqlite-demo;Mode=Memory;Cache=Shared"; +var keepAlive = new SqliteConnection(serverDatabase); +keepAlive.Open(); +builder.Services.AddSingleton(keepAlive); +builder.Services.AddDbContext<AppDbContext>(options => options.UseSqlite(serverDatabase)); + +builder.Services.Configure<ForwardedHeadersOptions>(options => +{ + options.ForwardedHeaders = ForwardedHeaders.All; + options.ForwardedHostHeaderName = "X-Host"; + options.KnownIPNetworks.Clear(); + options.KnownProxies.Clear(); + options.ForwardLimit = 1; +}); + +var app = builder.Build(); + +// The migrations the browser applies are the ones applied here, by the stock provider. +await using (var scope = app.Services.CreateAsyncScope()) +{ + var db = scope.ServiceProvider.GetRequiredService<AppDbContext>(); + await db.Database.MigrateAsync(); + await DemoData.SeedIfEmptyAsync(db); +} + +if (app.Environment.IsDevelopment()) +{ + app.UseWebAssemblyDebugging(); +} +else +{ + app.UseExceptionHandler("/Error", createScopeForErrors: true); + app.UseHsts(); +} + +// A deep link to an unknown route matches no Razor component endpoint, so routing answers with an empty 404. +// Re-execute it through the app to get the page the router shows for the same miss during client-side +// navigation - keeping the status code at 404. +app.UseStatusCodePagesWithReExecute("/not-found", createScopeForStatusCodePages: true); + +app.UseForwardedHeaders(); + +app.UseHttpsRedirection(); + +app.UseAntiforgery(); + +app.MapStaticAssets(); + +// Both are literal routes, so they are matched before any component route regardless of order; declaring +// them first says so out loud. +app.MapControllers(); +app.MapMcp("/mcp"); + +// The server side of the shared model: the catalog the stock provider holds, through the same AppDbContext. +// Read-only: the site is public and the database lives in this process's memory for its whole lifetime, so an +// endpoint that let anyone add rows would let anyone grow it until the process runs out of memory. +app.MapGet("/api/products", async (AppDbContext db, CancellationToken ct) => + await db.Products.AsNoTracking().OrderBy(p => p.Id).ToListAsync(ct)); + +app.MapRazorComponents<App>() + .AddInteractiveWebAssemblyRenderMode() + .AddAdditionalAssemblies(typeof(Bit.Bsqlite.Demo.Client._Imports).Assembly); + +app.Run(); + +/// <summary> +/// Named so the tests can host this exact file in-memory. Top-level statements compile to an internal +/// Program, which WebApplicationFactory cannot reach from another assembly on its own. +/// </summary> +public partial class Program; diff --git a/src/Bsqlite/Demo/Server/Properties/launchSettings.json b/src/Bsqlite/Demo/Server/Properties/launchSettings.json new file mode 100644 index 00000000000..6584cb2b880 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Properties/launchSettings.json @@ -0,0 +1,15 @@ +{ + "$schema": "http://json.schemastore.org/launchsettings.json", + "profiles": { + "Bit.Bsqlite.Demo.Server": { + "commandName": "Project", + "dotnetRunMessages": true, + "launchBrowser": true, + "inspectUri": "{wsProtocol}://{url.hostname}:{url.port}/_framework/debug/ws-proxy?browser={browserInspectUri}", + "applicationUrl": "https://localhost:5297;http://localhost:5296", + "environmentVariables": { + "ASPNETCORE_ENVIRONMENT": "Development" + } + } + } +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteApiCatalog.cs b/src/Bsqlite/Demo/Server/Services/BsqliteApiCatalog.cs new file mode 100644 index 00000000000..b216c2280c7 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteApiCatalog.cs @@ -0,0 +1,738 @@ +using System.Text; +using System.Reflection; +using System.Collections.Concurrent; +using System.Runtime.CompilerServices; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; +using Bit.Bsqlite.Storage.Opfs; +using Bit.Bsqlite.Demo.Server.Dtos; +using Bit.Bsqlite.Storage.IndexedDb; +using Bit.Bsqlite.Storage.CacheStorage; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// Builds the API reference the MCP tools serve, by reflecting over the public surface of the five +/// Bit.Bsqlite packages - the core, the three storage backends and the testing kit - and pairing +/// every type and member with its XML documentation. +/// <para> +/// Reflection rather than a hand-maintained table: the reference then cannot drift from the shipped +/// library - a new member shows up the moment it is written, with the default value it actually has +/// (read off a freshly constructed instance) instead of one someone remembered to copy into a +/// document. And every type carries its namespace and its package, because Bsqlite's are spread over +/// half a dozen namespaces - two of them Microsoft's, where the DI and EF Core entry points live - and +/// a wrong <c>using</c> is the commonest way generated code fails to compile. +/// </para> +/// </summary> +public static class BsqliteApiCatalog +{ + private const BindingFlags Declared = BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly; + + /// <summary>The five packages, core first: the order their types are listed in.</summary> + public static readonly Assembly[] Assemblies = + [ + typeof(BsqliteConnection).Assembly, + typeof(BsqliteOpfsStorageProvider).Assembly, + typeof(BsqliteIndexedDbStorageProvider).Assembly, + typeof(BsqliteCacheStorageProvider).Assembly, + typeof(BsqliteInProcessTransport).Assembly, + ]; + + private static readonly Lazy<Type[]> _publicTypes = new(() => + [.. Assemblies.SelectMany(assembly => assembly.GetExportedTypes()) + .Where(t => t.IsNested is false && t.Name.Contains('<', StringComparison.Ordinal) is false) + .OrderBy(t => t.Name, StringComparer.OrdinalIgnoreCase) + .ThenBy(t => t.FullName, StringComparer.Ordinal)]); + + private static readonly ConcurrentDictionary<string, BsqliteApiTypeDetailsDto> _typeDetails = new(StringComparer.Ordinal); + + private static readonly Lazy<BsqliteApiTypeDto[]> _types = new(() => + [.. _publicTypes.Value.Select(t => new BsqliteApiTypeDto + { + Name = FriendlyName(t), + Namespace = t.Namespace ?? string.Empty, + Package = PackageOf(t), + Kind = KindOf(t), + Summary = BsqliteXmlDocs.GetSummary(DocumentationId(t)) + })]); + + /// <summary>Every public type of the Bit.Bsqlite packages, with its summary.</summary> + public static BsqliteApiTypeDto[] Types => _types.Value; + + /// <summary> + /// The index of the public types, as GetBsqliteApi answers when it is called without a name - + /// grouped by package and namespace, since which <c>using</c> a type needs is half of what a + /// caller choosing from the list wants to know. + /// <para> + /// A summary is cut to its first sentence here: what an index is for is choosing what to read + /// next, and the paragraph that follows the first sentence is the answer to the call the reader + /// has not made yet. + /// </para> + /// </summary> + public static string RenderIndex() + { + var builder = new StringBuilder(); + + builder.AppendLine("# Bit.Bsqlite public API").AppendLine(); + builder.AppendLine($"{Types.Length} public types in {Assemblies.Length} packages. Call `GetBsqliteApi(typeName: \"...\")` for one of them in full - ") + .AppendLine("its constructors, properties, methods or enum values, with the default value each one really has.") + .AppendLine(); + + foreach (var package in Types.GroupBy(type => type.Package).OrderBy(group => PackageOrder(group.Key))) + { + builder.AppendLine($"## Package {package.Key}").AppendLine(); + + foreach (var ns in package.GroupBy(type => type.Namespace).OrderBy(group => group.Key, StringComparer.Ordinal)) + { + builder.AppendLine($"### `using {ns.Key};`").AppendLine(); + + foreach (var type in ns) + { + builder.Append($"- **{type.Name}** ({type.Kind})"); + if (FirstSentence(type.Summary) is string summary) builder.Append($" - {summary}"); + builder.AppendLine(); + } + + builder.AppendLine(); + } + } + + return builder.ToString(); + } + + /// <summary> + /// The full reference of one type as Markdown, or null when no public type goes by that name. + /// <para> + /// Markdown rather than an object, and one renderer rather than two: the tool and the + /// <c>bsqlite://api/{typeName}</c> resource hand out the same text, so neither can drift from the + /// other - and a reference a model reads costs less as prose than as JSON sent twice, once under an + /// output schema and once as the text that mirrors it. + /// </para> + /// </summary> + public static string? RenderType(string typeName) + { + var details = GetTypeDetails(typeName); + if (details is null) return null; + + var builder = new StringBuilder(); + + builder.AppendLine($"# {details.Name} ({details.Kind})").AppendLine(); + // One line: the fragments continue the FullName rather than standing on their own, so the line + // is only ended once every part of it has been written. + builder.Append($"`{details.FullName}` - package `{details.Package}` - `using {details.Namespace};`"); + + if (details.BaseType is not null) builder.Append($" - inherits `{details.BaseType}`"); + if (details.Implements?.Length > 0) builder.Append($" - implements {string.Join(", ", details.Implements.Select(i => $"`{i}`"))}"); + + builder.AppendLine().AppendLine(); + + if (details.Summary is not null) builder.AppendLine(details.Summary).AppendLine(); + if (details.Remarks is not null) builder.AppendLine(details.Remarks).AppendLine(); + + foreach (var group in details.Members.GroupBy(member => member.Kind)) + { + builder.AppendLine($"## {group.Key}").AppendLine(); + + foreach (var member in group) + { + builder.Append(member.Protected ? $"- protected **{member.Name}**{member.Signature}" : $"- **{member.Name}**{member.Signature}"); + if (member.Type is not null) builder.Append($" : `{member.Type}`"); + if (member.Default is not null) builder.Append($" = `{member.Default}`"); + if (member.Required) builder.Append(" **(required)**"); + if (member.Summary is not null) builder.Append($" - {member.Summary}"); + builder.AppendLine(); + + // The caveats live in the remarks, which is exactly the half of the documentation an + // agent writing against a member needs and the least likely to be remembered. + if (member.Remarks is not null) builder.AppendLine($" {member.Remarks.Replace("\n", "\n ", StringComparison.Ordinal)}"); + } + + builder.AppendLine(); + } + + return builder.ToString(); + } + + /// <summary>The first sentence of a summary, for a listing that is read to choose by.</summary> + internal static string? FirstSentence(string? summary) + { + if (string.IsNullOrWhiteSpace(summary)) return null; + + var text = summary.ReplaceLineEndings(" ").Trim(); + + var stop = text.IndexOf(". ", StringComparison.Ordinal); + + return stop > 0 ? text[..(stop + 1)] : text; + } + + /// <summary> + /// The full reference of one type - its constructors, properties, methods, events or enum values - + /// or null when no public type goes by that name. + /// </summary> + public static BsqliteApiTypeDetailsDto? GetTypeDetails(string typeName) + { + var type = Find(typeName); + if (type is null) return null; + + // Every hit costs a full reflection walk plus an XML lookup per member, and the search index + // asks for all of them at once; the answer only changes when the assemblies do. + return _typeDetails.GetOrAdd(type.FullName ?? type.Name, _ => BuildTypeDetails(type)); + } + + private static BsqliteApiTypeDetailsDto BuildTypeDetails(Type type) + { + var instance = TryCreateInstance(type); + + // A second instance, compared against the first, is what tells a real default from a value made + // up per instance - reporting one of those as "the default" would hand a caller a value nobody + // ever gets. + var twin = TryCreateInstance(type); + + var members = new List<BsqliteApiMemberDto>(); + + if (type.IsEnum) + { + members.AddRange(type.GetFields(BindingFlags.Public | BindingFlags.Static) + .Select(field => new BsqliteApiMemberDto + { + Name = field.Name, + Kind = "EnumValue", + Type = FriendlyName(type.GetEnumUnderlyingType()), + // Formatted straight off the raw constant, so a [Flags] value reads as + // the number it combines by. + Default = Format(field.GetRawConstantValue()), + Summary = BsqliteXmlDocs.GetSummary(DocumentationId(field)), + Remarks = BsqliteXmlDocs.GetRemarks(DocumentationId(field)) + })); + } + else + { + members.AddRange(Constructors(type)); + members.AddRange(Properties(type, instance, twin)); + members.AddRange(Fields(type)); + members.AddRange(Methods(type)); + members.AddRange(Events(type)); + } + + return new BsqliteApiTypeDetailsDto + { + Name = FriendlyName(type), + FullName = type.FullName ?? type.Name, + Namespace = type.Namespace ?? string.Empty, + Package = PackageOf(type), + Kind = KindOf(type), + BaseType = type.BaseType is null || type.BaseType == typeof(object) || type.BaseType == typeof(ValueType) + ? null + : FriendlyName(type.BaseType), + Implements = [.. type.GetInterfaces() + .Where(i => i.IsPublic && i.DeclaringType is null) + .Select(FriendlyName) + .Distinct(StringComparer.Ordinal) + .OrderBy(n => n, StringComparer.OrdinalIgnoreCase)], + Summary = BsqliteXmlDocs.GetSummary(DocumentationId(type)), + Remarks = BsqliteXmlDocs.GetRemarks(DocumentationId(type)), + Members = [.. members.OrderBy(m => KindOrder(m.Kind)).ThenBy(m => m.Name, StringComparer.OrdinalIgnoreCase)] + }; + } + + /// <summary>Resolves a type by its simple name, with or without a generic arity or argument list.</summary> + private static Type? Find(string typeName) + { + if (string.IsNullOrWhiteSpace(typeName)) return null; + + var name = typeName.Trim(); + + var generic = name.IndexOf('<', StringComparison.Ordinal); + if (generic > 0) name = name[..generic]; + + return _publicTypes.Value.FirstOrDefault(t => string.Equals(t.Name, name, StringComparison.OrdinalIgnoreCase)) + ?? _publicTypes.Value.FirstOrDefault(t => string.Equals(StripArity(t.Name), name, StringComparison.OrdinalIgnoreCase)) + ?? _publicTypes.Value.FirstOrDefault(t => string.Equals(t.FullName, name, StringComparison.OrdinalIgnoreCase)) + ?? _publicTypes.Value.FirstOrDefault(t => string.Equals(StripArity(t.FullName ?? string.Empty), name, StringComparison.OrdinalIgnoreCase)); + } + + /// <summary> + /// The type and every base of it that Bit.Bsqlite itself declares. The members a type gets from + /// DbConnection or Exception belong to their documentation, not to this one. + /// </summary> + private static IEnumerable<Type> Hierarchy(Type type) + { + for (var current = type; current is not null && Assemblies.Contains(current.Assembly); current = current.BaseType) + { + yield return current; + } + } + + /// <summary> + /// Whether a member is part of what a caller can use: anything public, and - on a type that can be + /// derived from - anything protected, since that is the half of the contract a subclass writes. The + /// conformance kit is nothing but that: its whole API is the protected factory a test class + /// overrides. On a sealed type a protected member is an override of the base class's plumbing. + /// </summary> + private static bool IsVisible(MethodBase? member, Type owner) + { + if (member is null) return false; + if (member.IsPublic) return true; + + return owner.IsSealed is false && (member.IsFamily || member.IsFamilyOrAssembly); + } + + private static bool IsProtected(MethodBase? member) => member is not null && member.IsPublic is false; + + private static IEnumerable<BsqliteApiMemberDto> Constructors(Type type) + { + // A static class has none to speak of, and an abstract one is constructed by its subclass. + if (type.IsAbstract && type.IsSealed) yield break; + + foreach (var constructor in type.GetConstructors(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance)) + { + if (IsVisible(constructor, type) is false) continue; + + yield return new BsqliteApiMemberDto + { + Name = FriendlyName(type), + Kind = "Constructor", + Signature = Signature(constructor), + Protected = IsProtected(constructor), + Summary = BsqliteXmlDocs.GetSummary(DocumentationId(constructor)), + Remarks = BsqliteXmlDocs.GetRemarks(DocumentationId(constructor)) + }; + } + } + + private static IEnumerable<BsqliteApiMemberDto> Properties(Type type, object? instance, object? twin) + { + var seen = new HashSet<string>(StringComparer.Ordinal); + + foreach (var property in Hierarchy(type).SelectMany(t => t.GetProperties(Declared))) + { + if (property.GetIndexParameters().Length > 0) continue; + + // Either accessor makes the property usable: an init-only setter on a get-less property is + // still an option a caller sets. + var accessor = IsVisible(property.GetMethod, type) ? property.GetMethod : property.SetMethod; + if (IsVisible(accessor, type) is false) continue; + + // The record's EqualityContract is compiler plumbing every record has. + if (property.IsDefined(typeof(CompilerGeneratedAttribute), inherit: false) || property.Name == "EqualityContract") continue; + if (seen.Add(property.Name) is false) continue; + + // A required member has no default: the compiler refuses an object initializer that leaves + // it out, so whatever an instance made by reflection holds is a value nobody can get. + var required = property.IsDefined(typeof(RequiredMemberAttribute), inherit: false); + + yield return new BsqliteApiMemberDto + { + Name = property.Name, + Kind = "Property", + Type = FriendlyName(property.PropertyType), + Default = required ? null : DefaultValueOf(property, instance, twin), + Required = required, + Protected = IsProtected(accessor), + Summary = Documentation(property, BsqliteXmlDocs.GetSummary), + Remarks = Documentation(property, BsqliteXmlDocs.GetRemarks) + }; + } + } + + private static IEnumerable<BsqliteApiMemberDto> Fields(Type type) + { + var seen = new HashSet<string>(StringComparer.Ordinal); + + foreach (var field in Hierarchy(type).SelectMany(t => t.GetFields(Declared))) + { + if (field.IsPublic is false) continue; + if (field.Name.Contains('<', StringComparison.Ordinal)) continue; + if (seen.Add(field.Name) is false) continue; + + yield return new BsqliteApiMemberDto + { + Name = field.Name, + Kind = "Field", + Type = FriendlyName(field.FieldType), + Default = field.IsLiteral ? Format(field.GetRawConstantValue()) : field.IsInitOnly && field.IsStatic ? SafeFormat(() => field.GetValue(null)) : null, + Summary = BsqliteXmlDocs.GetSummary(DocumentationId(field)), + Remarks = BsqliteXmlDocs.GetRemarks(DocumentationId(field)) + }; + } + } + + private static IEnumerable<BsqliteApiMemberDto> Methods(Type type) + { + var seen = new HashSet<string>(StringComparer.Ordinal); + + foreach (var method in Hierarchy(type).SelectMany(t => t.GetMethods(Declared))) + { + if (IsVisible(method, type) is false) continue; + if (method.IsSpecialName) continue; // property/event accessors and operators + if (method.Name.Contains('<', StringComparison.Ordinal)) continue; + if (method.DeclaringType == typeof(object)) continue; + // An explicit interface implementation ("Microsoft.EntityFrameworkCore.IDbContextFactory<...>.CreateDbContextAsync") + // is reached through the interface, whose own reference lists it. + if (method.Name.Contains('.', StringComparison.Ordinal)) continue; + // What the compiler writes for a record - Equals, GetHashCode, ToString, Deconstruct, PrintMembers - + // is the same on every record and documented nowhere; listing it on each of the library's records + // would bury the members someone wrote on purpose. + if (method.IsDefined(typeof(CompilerGeneratedAttribute), inherit: false)) continue; + + var signature = Signature(method); + if (seen.Add($"{method.Name}{signature}") is false) continue; + + yield return new BsqliteApiMemberDto + { + Name = method.IsGenericMethodDefinition ? $"{method.Name}<{string.Join(", ", method.GetGenericArguments().Select(a => a.Name))}>" : method.Name, + Kind = "Method", + Type = FriendlyName(method.ReturnType), + Signature = signature, + Protected = IsProtected(method), + Summary = Documentation(method, BsqliteXmlDocs.GetSummary), + Remarks = Documentation(method, BsqliteXmlDocs.GetRemarks) + }; + } + } + + private static IEnumerable<BsqliteApiMemberDto> Events(Type type) + { + var seen = new HashSet<string>(StringComparer.Ordinal); + + foreach (var member in Hierarchy(type).SelectMany(t => t.GetEvents(Declared))) + { + if (IsVisible(member.AddMethod, type) is false) continue; + if (seen.Add(member.Name) is false) continue; + + yield return new BsqliteApiMemberDto + { + Name = member.Name, + Kind = "Event", + Type = FriendlyName(member.EventHandlerType ?? typeof(void)), + Summary = BsqliteXmlDocs.GetSummary(DocumentationId(member)) ?? InterfaceDocumentation(member, BsqliteXmlDocs.GetSummary), + Remarks = BsqliteXmlDocs.GetRemarks(DocumentationId(member)) ?? InterfaceDocumentation(member, BsqliteXmlDocs.GetRemarks) + }; + } + } + + /// <summary> + /// The documentation of a property, or - when it says no more than <c><inheritdoc /></c>, as + /// every provider's Name and Capabilities do - that of the member it overrides or implements, which + /// is what the compiler would show for it. + /// </summary> + private static string? Documentation(PropertyInfo property, Func<string, string?> read) + { + if (read(DocumentationId(property)) is string own) return own; + + var accessor = property.GetMethod ?? property.SetMethod; + var root = accessor?.GetBaseDefinition(); + + if (root is not null && root.DeclaringType != property.DeclaringType && + root.DeclaringType!.GetProperty(property.Name, Declared) is { } overridden && + read(DocumentationId(overridden)) is string inherited) return inherited; + + foreach (var contract in property.DeclaringType!.GetInterfaces()) + { + if (contract.GetProperty(property.Name) is { } declared && read(DocumentationId(declared)) is string implemented) return implemented; + } + + return null; + } + + private static string? InterfaceDocumentation(EventInfo member, Func<string, string?> read) + { + foreach (var contract in member.DeclaringType!.GetInterfaces()) + { + if (contract.GetEvent(member.Name) is { } declared && read(DocumentationId(declared)) is string implemented) return implemented; + } + + return null; + } + + /// <summary>The documentation of a method, falling back to the one it overrides or implements.</summary> + private static string? Documentation(MethodInfo method, Func<string, string?> read) + { + if (read(DocumentationId(method)) is string own) return own; + + var root = method.GetBaseDefinition(); + + if (root.DeclaringType != method.DeclaringType && read(DocumentationId(root)) is string inherited) return inherited; + + var owner = method.DeclaringType!; + if (owner.IsInterface) return null; + + foreach (var contract in owner.GetInterfaces()) + { + InterfaceMapping map; + + try + { + map = owner.GetInterfaceMap(contract); + } + catch (Exception) + { + // A generic or otherwise unmappable interface simply has nothing to lend. + continue; + } + + var index = Array.IndexOf(map.TargetMethods, method); + + if (index >= 0 && read(DocumentationId(map.InterfaceMethods[index])) is string implemented) return implemented; + } + + return null; + } + + /// <summary> + /// The value a property holds on a newly constructed instance - the default a caller gets when + /// the property is left unset. + /// </summary> + private static string? DefaultValueOf(PropertyInfo property, object? instance, object? twin) + { + if (property.GetMethod is null || property.GetMethod.IsPublic is false) return null; + if (property.GetMethod.IsStatic is false && instance is null) return null; + + try + { + if (property.GetMethod.IsStatic) return Format(property.GetValue(null)); + + var value = Format(property.GetValue(instance)); + + return twin is null || value == Format(property.GetValue(twin)) ? value : null; + } + catch (Exception) + { + // Members that only make sense on an open connection (or throw by design) simply have no + // observable default; that is not a reason to fail the whole type's reference. + return null; + } + } + + /// <summary> + /// A fresh instance to read defaults off, or null when there is no safe way to make one. + /// <para> + /// Only types with a parameterless constructor are built, and never one that owns something - a + /// connection, a command, a transport, anything disposable: the reference would have to dispose + /// what it built just to read a default, and a type like that is configured through its + /// constructor's parameters, which the reference lists with their defaults anyway. + /// </para> + /// </summary> + private static object? TryCreateInstance(Type type) + { + if (type.IsAbstract || type.IsInterface || type.IsEnum || type.ContainsGenericParameters) return null; + if (typeof(IDisposable).IsAssignableFrom(type) || typeof(IAsyncDisposable).IsAssignableFrom(type)) return null; + if (type.IsValueType is false && type.GetConstructor(Type.EmptyTypes) is null) return null; + + try + { + return Activator.CreateInstance(type); + } + catch (Exception) + { + return null; + } + } + + private static string? SafeFormat(Func<object?> read) + { + try + { + return Format(read()); + } + catch (Exception) + { + return null; + } + } + + private static string? Format(object? value) => value switch + { + null => null, + string text => $"\"{text}\"", + bool flag => flag ? "true" : "false", + Enum enumValue => FormatEnum(enumValue), + IFormattable formattable => formattable.ToString(null, System.Globalization.CultureInfo.InvariantCulture), + _ => Describe(value) + }; + + /// <summary>A [Flags] combination reads as the C# that writes it: "BsqliteExecutionContexts.Window | BsqliteExecutionContexts.DedicatedWorker".</summary> + private static string FormatEnum(Enum value) + { + var name = value.GetType().Name; + + return string.Join(" | ", value.ToString().Split(", ").Select(part => $"{name}.{part}")); + } + + /// <summary> + /// A reference-typed default is worth reporting: a property that starts out holding an object is + /// not the same thing as one that starts out null. + /// </summary> + private static string Describe(object value) + { + var type = value.GetType(); + + // A default that is one of the type's own named values reads best by that name + // (BsqliteRuntimeLimits.Unrestricted) rather than as a dump of its fields. + foreach (var field in type.GetFields(BindingFlags.Public | BindingFlags.Static)) + { + if (field.FieldType == type && field.IsInitOnly | field.IsLiteral && Equals(field.GetValue(null), value)) return $"{FriendlyName(type)}.{field.Name}"; + } + + foreach (var property in type.GetProperties(BindingFlags.Public | BindingFlags.Static)) + { + if (property.PropertyType != type || property.GetIndexParameters().Length > 0) continue; + + try + { + if (Equals(property.GetValue(null), value)) return $"{FriendlyName(type)}.{property.Name}"; + } + catch (Exception) + { + // A static property that throws simply names nothing. + } + } + + // An empty collection is the most common reference-typed default, and its ToString is only its + // type name; say what it is instead. + if (value is System.Collections.ICollection { Count: 0 }) return "empty"; + + var text = value.ToString(); + + // A ToString that is only the type's name says nothing about the value - and for a generic type + // it is the CLR spelling ("List`1[...]") no C# reader would recognise. + return string.IsNullOrEmpty(text) || text == type.FullName || text == type.Name || text.Contains('`', StringComparison.Ordinal) + ? $"new {FriendlyName(type)}()" + : text; + } + + private static string Signature(MethodBase method) + { + var builder = new StringBuilder("("); + + var parameters = method.GetParameters(); + for (int i = 0; i < parameters.Length; i++) + { + var parameter = parameters[i]; + + if (i > 0) builder.Append(", "); + if (i == 0 && method.IsDefined(typeof(ExtensionAttribute), false)) builder.Append("this "); + if (parameter.IsOut) builder.Append("out "); + else if (parameter.ParameterType.IsByRef) builder.Append("ref "); + + builder.Append(FriendlyName(parameter.ParameterType)).Append(' ').Append(parameter.Name); + + if (parameter.HasDefaultValue) builder.Append(" = ").Append(FormatParameterDefault(parameter)); + } + + return builder.Append(')').ToString(); + } + + /// <summary>A parameter's default as C# writes it: <c>default</c> for a CancellationToken, <c>null</c> for an absent reference.</summary> + private static string FormatParameterDefault(ParameterInfo parameter) + { + if (parameter.ParameterType == typeof(CancellationToken)) return "default"; + + return Format(parameter.DefaultValue) ?? (parameter.ParameterType.IsValueType && Nullable.GetUnderlyingType(parameter.ParameterType) is null ? "default" : "null"); + } + + private static string KindOf(Type type) + { + if (type.IsEnum) return "Enum"; + if (type.IsInterface) return "Interface"; + if (typeof(Exception).IsAssignableFrom(type)) return "Exception"; + if (typeof(Attribute).IsAssignableFrom(type)) return "Attribute"; + if (typeof(MulticastDelegate).IsAssignableFrom(type)) return "Delegate"; + if (type.IsAbstract && type.IsSealed) return "Static class"; + if (type.IsValueType) return "Struct"; + if (type.GetMethod("<Clone>$", BindingFlags.Public | BindingFlags.Instance) is not null) return "Record"; + if (type.IsAbstract) return "Abstract class"; + + return "Class"; + } + + private static int KindOrder(string kind) => kind switch + { + "Constructor" => 0, + "Property" => 1, + "Field" => 2, + "EnumValue" => 3, + "Method" => 4, + "Event" => 5, + _ => 6 + }; + + private static int PackageOrder(string package) => Array.FindIndex(Assemblies, assembly => assembly.GetName().Name == package); + + /// <summary>The package a type ships in - every Bsqlite assembly is its own package, named after it.</summary> + public static string PackageOf(Type type) => type.Assembly.GetName().Name ?? string.Empty; + + /// <summary>The C# spelling of a type: "int?", "string[]", "IReadOnlyList<BsqliteStorageCandidateOutcome>".</summary> + public static string FriendlyName(Type type) + { + if (type.IsByRef) return FriendlyName(type.GetElementType()!); + if (type.IsArray) return $"{FriendlyName(type.GetElementType()!)}[]"; + + var nullable = Nullable.GetUnderlyingType(type); + if (nullable is not null) return $"{FriendlyName(nullable)}?"; + + if (type == typeof(void)) return "void"; + if (type == typeof(object)) return "object"; + if (type == typeof(string)) return "string"; + if (type == typeof(bool)) return "bool"; + if (type == typeof(byte)) return "byte"; + if (type == typeof(int)) return "int"; + if (type == typeof(long)) return "long"; + if (type == typeof(double)) return "double"; + if (type == typeof(float)) return "float"; + if (type == typeof(decimal)) return "decimal"; + + if (type.IsGenericType is false) return type.Name; + + var arguments = string.Join(", ", type.GetGenericArguments().Select(FriendlyName)); + + return $"{StripArity(type.Name)}<{arguments}>"; + } + + private static string StripArity(string name) + { + var arity = name.IndexOf('`', StringComparison.Ordinal); + + return arity < 0 ? name : name[..arity]; + } + + private static string DocumentationId(Type type) => $"T:{type.FullName}"; + + private static string DocumentationId(PropertyInfo property) => $"P:{property.DeclaringType!.FullName}.{property.Name}"; + + private static string DocumentationId(FieldInfo field) => $"F:{field.DeclaringType!.FullName}.{field.Name}"; + + private static string DocumentationId(EventInfo member) => $"E:{member.DeclaringType!.FullName}.{member.Name}"; + + private static string DocumentationId(MethodBase method) + { + var name = method is ConstructorInfo ? "#ctor" : method.Name; + var id = new StringBuilder("M:").Append(method.DeclaringType!.FullName).Append('.').Append(name); + + if (method.IsGenericMethodDefinition) id.Append("``").Append(method.GetGenericArguments().Length); + + var parameters = method.GetParameters(); + if (parameters.Length == 0) return id.ToString(); + + id.Append('(') + .Append(string.Join(',', parameters.Select(p => DocumentationTypeName(p.ParameterType)))) + .Append(')'); + + return id.ToString(); + } + + /// <summary>Spells a type the way the C# compiler spells it inside a documentation id.</summary> + private static string DocumentationTypeName(Type type) + { + if (type.IsByRef) return $"{DocumentationTypeName(type.GetElementType()!)}@"; + if (type.IsArray) return $"{DocumentationTypeName(type.GetElementType()!)}[]"; + if (type.IsGenericParameter) return type.DeclaringMethod is null ? $"`{type.GenericParameterPosition}" : $"``{type.GenericParameterPosition}"; + + if (type.IsGenericType is false) return type.FullName ?? type.Name; + + var definition = type.GetGenericTypeDefinition().FullName ?? type.Name; + var arguments = string.Join(',', type.GetGenericArguments().Select(DocumentationTypeName)); + + return $"{StripArity(definition)}{{{arguments}}}"; + } +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteCompletions.cs b/src/Bsqlite/Demo/Server/Services/BsqliteCompletions.cs new file mode 100644 index 00000000000..395924cc83c --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteCompletions.cs @@ -0,0 +1,82 @@ +using ModelContextProtocol.Protocol; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// Answers <c>completion/complete</c>: the values a client offers while someone fills in a prompt +/// argument or a resource template's placeholder. +/// <para> +/// The protocol allows this for two things only - a prompt's arguments and a resource template's +/// placeholders - and both of those are, on this server, keys into a closed set that nothing on the +/// wire spells out: a document name, a public type name, a storage backend, a setup scenario, an +/// embedded source path. Without completions a person picking <c>bsqlite://api/{typeName}</c> in a +/// client has to go and call a listing tool first, just to learn what a type name looks like. +/// </para> +/// </summary> +public static class BsqliteCompletions +{ + /// <summary>The protocol caps one response at 100 values; anything past that is reported through <c>hasMore</c>.</summary> + private const int MaxValues = 100; + + // Freeform arguments cannot be completed, only started off: these are the phrasings the matching + // prompt was written for, so picking one lands in the workflow it was designed around instead of in + // a sentence the tools have nothing to say about. + private static readonly string[] _symptoms = + [ + "prerendering fails with a missing service", + "the page throws during prerendering when it opens the database", + "the database is empty after a reload", + "the data is gone after OPFS became available", + "BsqliteStorageUnavailableException on open", + "SaveChanges throws BsqliteSynchronousApiNotSupportedException", + "a live query does not refresh after a write", + "PRAGMA journal_mode = WAL is rejected", + "ATTACH is rejected on OPFS", + "the worker or the engine fails to load under a sub-path", + ]; + + /// <summary> + /// The values matching what has been typed so far. The argument's name is enough to identify the set - + /// no two arguments on this server mean different things by the same name - so one table serves a + /// prompt argument and a resource template placeholder without either knowing about the other, and an + /// argument it has nothing for is answered with an empty list rather than an error. + /// </summary> + public static Completion Complete(string? argumentName, string? typed) + { + var candidates = CandidatesFor(argumentName); + if (candidates.Length == 0) return new Completion { Values = [] }; + + var prefix = (typed ?? string.Empty).Trim(); + + var matches = prefix.Length == 0 + ? candidates + : [.. candidates + .Where(value => value.Contains(prefix, StringComparison.OrdinalIgnoreCase)) + // What was typed is a prefix far more often than it is a substring, so the values it starts + // stay above the ones it merely appears inside. + .OrderByDescending(value => value.StartsWith(prefix, StringComparison.OrdinalIgnoreCase)) + .ThenBy(value => value, StringComparer.OrdinalIgnoreCase)]; + + return new Completion + { + Values = [.. matches.Take(MaxValues)], + Total = matches.Length, + HasMore = matches.Length > MaxValues + }; + } + + // "hosting" is deliberately missing here: its values are declared with [AllowedValues] on the prompt, + // which the SDK completes from on its own, and anything this returned would be merged with that rather + // than replace it - the same value offered to someone twice. "scenario" is here because the setup + // resource template has no attribute to declare its values with. + private static string[] CandidatesFor(string? argumentName) => argumentName switch + { + "name" => [.. BsqliteSourceCatalog.Documents.Select(document => document.Name)], + "typeName" => [.. BsqliteApiCatalog.Types.Select(type => type.Name)], + "provider" => [.. BsqliteStorageCatalog.Providers.Select(provider => provider.Name)], + "scenario" => BsqliteSetupGuide.Scenarios, + "path" => [.. BsqliteSourceCatalog.SourceFiles.Select(file => file.Path)], + "symptom" => _symptoms, + _ => [] + }; +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteSearchIndex.cs b/src/Bsqlite/Demo/Server/Services/BsqliteSearchIndex.cs new file mode 100644 index 00000000000..4c1bcfbc5b5 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteSearchIndex.cs @@ -0,0 +1,514 @@ +using Bit.Bsqlite.Demo.Server.Dtos; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// One searchable index over everything this MCP server knows: every section of every document, every +/// public type and member of the five packages, the storage backends, the setup guides, the two labs +/// and the demo's source files. +/// <para> +/// Without it an agent has to guess which corpus holds the answer and what it is called there - "keep +/// the user's data when OPFS appears" is a guide section, a resolver, a migration mode, a demo page and +/// a lab all at once. Each hit therefore carries the exact follow-up tool call that returns the full +/// text, so one search is enough to know what to ask for next. +/// </para> +/// </summary> +public static class BsqliteSearchIndex +{ + private sealed record Entry(string Kind, string Title, string? Context, string Tool, string Body, string Boosted, int Weight = ReferenceWeight) + { + /// <summary> + /// The title split into words, camel-case humps included, so "AllowNonPersistentFallback" is found + /// by "non persistent fallback" - and so a query word only counts as a title hit when it IS one of + /// those words, rather than merely appearing inside one ("lite" inside "Bsqlite"). + /// </summary> + public string[] TitleWords { get; } = SplitWords(Title); + } + + private const int MaxTerms = 16; + + /// <summary>The most hits one search may return, whatever the caller asked for.</summary> + private const int MaxHits = 50; + + private const int MaxSuggestions = 8; + + /// <summary>The weight of an entry that answers a question: a document section, a type, a member, a guide.</summary> + private const int ReferenceWeight = 10; + + /// <summary> + /// The weight of a source file. They are examples, not answers, and their titles are paths whose + /// segments ("Client", "Pages", "Server") are common words carrying no topic - without this, every + /// question phrased with one of them is answered with a directory listing. + /// </summary> + private const int ExampleWeight = 4; + + /// <summary> + /// The most an entry can earn from prose, however much of it there is. A long section mentions + /// everything, so without a ceiling the biggest document in the corpus is the top hit for every + /// query - and a name match, which is the far stronger signal, never gets ahead of it. + /// </summary> + private const int MaxBodyScore = 8; + + private static readonly Lazy<Entry[]> _entries = new(Build); + + /// <summary> + /// The words every entry in this index carries because of what the index is about. They separate + /// nothing, and they do worse than nothing: a term that matches an entry counts towards the "how + /// many terms matched" multiplier, so left in, the longest documents win every query on the + /// strength of a word that is in all of them. + /// <para> + /// Unlike a stop word they are dropped only when the query says something else as well, since + /// "bsqlite" and "sqlite blazor library" are what a caller reaches for first and are made of + /// nothing but these. + /// </para> + /// </summary> + private static readonly HashSet<string> _ambientWords = new(StringComparer.OrdinalIgnoreCase) + { + "bsqlite", "bit", "blazor", "library", "sqlite" + }; + + /// <summary> + /// The sections that are about a document rather than about the library: tables of contents, which + /// name every topic and so match every query, link lists, the license, and the README's paragraph on + /// this MCP server, which mentions searching, setup and storage while answering none of them. They + /// stay readable through GetBsqliteDocument; they are only kept out of the ranking. + /// </summary> + private static readonly HashSet<(string Document, string Heading)> _metaSections = + [ + (BsqliteSourceCatalog.ReadmeName, "Table of Contents"), + (BsqliteSourceCatalog.ReadmeName, "Documentation"), + (BsqliteSourceCatalog.ReadmeName, "MCP server (for AI agents)"), + (BsqliteSourceCatalog.ReadmeName, "License"), + ("Bit.Bsqlite", "Links"), + ]; + + private static readonly HashSet<string> _stopWords = new(StringComparer.OrdinalIgnoreCase) + { + "how", "the", "and", "for", "with", "from", "that", "this", "what", "when", "where", "which", + "does", "did", "are", "was", "you", "your", "than", "then", "its", "but", "any", "some", + "please", "help", "about", "into", "way", "make", "want", "need", "would", "should", "could", + "there", "here", "have", "has", "get", "got", "let", "one", "two", "per", "via", "onto", "why", "off", "will" + }; + + public static BsqliteSearchResultDto Search(string? query, int limit) + { + var terms = Tokenize(query); + + if (terms.Length == 0) + { + // Nothing was left to search by, which is a different failure from "searched and found + // nothing" - and the fix is on the caller's side, so it has to be said rather than returned + // as an empty list the agent would read as "Bsqlite cannot do this". + return new BsqliteSearchResultDto + { + Terms = [], + Hits = [], + Message = "The query held no word longer than two letters that was not a filler word, so there was " + + "nothing to rank by. Search for the thing itself - 'opfs', 'live query', 'migration' - " + + "rather than for a sentence." + }; + } + + var ranked = _entries.Value + .Select(entry => (Entry: entry, Score: Score(entry, terms))) + .Where(hit => hit.Score > 0) + .OrderByDescending(hit => hit.Score) + .ThenBy(hit => hit.Entry.Title, StringComparer.OrdinalIgnoreCase) + // Two entries that would be read through the same call under the same name are one hit: a + // method with two overloads is two members and one place to read about them. A call that takes + // no argument returns the same text whichever entry led to it, so its name does not count. + .DistinctBy(hit => hit.Entry.Tool.EndsWith("()", StringComparison.Ordinal) ? (string.Empty, hit.Entry.Tool) : (hit.Entry.Title, hit.Entry.Tool)) + .ToArray(); + + var take = Math.Clamp(limit, 1, MaxHits); + + var hits = Cover(ranked, terms, take).Select(hit => new BsqliteSearchHitDto + { + Kind = hit.Entry.Kind, + Title = hit.Entry.Title, + Context = hit.Entry.Context, + Tool = hit.Entry.Tool, + Snippet = Snippet(hit.Entry, terms) + }).ToArray(); + + if (hits.Length > 0) + { + // No echo of the query and no list of the terms it was reduced to: the caller sent that query + // one message ago. Both are worth saying only when there is nothing else to say. + return new BsqliteSearchResultDto + { + Hits = hits, + HasMore = ranked.Length > hits.Length + }; + } + + // A search that found nothing is where an agent is most likely to conclude the library has no + // such feature and hand-roll one. Whatever came closest is named instead, so the next call is a + // better query rather than a worse decision. + var nearby = Suggest(terms); + + return new BsqliteSearchResultDto + { + Terms = terms, + Hits = [], + Message = nearby.Length > 0 + ? $"Nothing matched {string.Join(" + ", terms)}. The closest names are listed in didYouMean - " + + "search for one of those, or for a single word out of this query." + : $"Nothing matched {string.Join(" + ", terms)}. Try one word rather than several, or call " + + "GetBsqliteDocument or GetBsqliteApi with no argument at all for the index of what there is. " + + "A concept Bsqlite does not name is often the same idea under another word - 'offline' or " + + "'local database' is the browser database itself, 'backend' or 'persistence' is a storage " + + "provider, 'sticky' is the binding store, 'reactive' or 'observable' is a live query, 'mock' " + + "or 'fake' is the in-process transport, 'backup' is export.", + DidYouMean = nearby.Length > 0 ? nearby : null + }; + } + + /// <summary> + /// The first <paramref name="take"/> hits, with every query word that names something kept in view. + /// A request usually joins two things - "migrate from indexeddb to opfs" is a migration AND two + /// backends - and the one with more material behind it would otherwise fill every place, leaving the + /// caller to conclude the other has no answer. So for each word that is the name of some entry yet + /// names none of the top hits, the best entry it does name takes the place of the lowest-ranked hit + /// that is not itself there for that reason. The order stays the ranking's. + /// </summary> + private static (Entry Entry, int Score)[] Cover((Entry Entry, int Score)[] ranked, string[] terms, int take) + { + var top = ranked.Take(take).ToList(); + if (top.Count < take || terms.Length < 2) return [.. top]; + + var kept = new HashSet<Entry>(); + + foreach (var term in terms) + { + if (top.Any(hit => hit.Entry.TitleWords.Any(word => Equivalent(word, term)))) continue; + + var best = ranked.Skip(take).FirstOrDefault(hit => hit.Entry.TitleWords.Any(word => Equivalent(word, term))); + if (best.Entry is null) continue; + + var replace = top.FindLastIndex(hit => kept.Contains(hit.Entry) is false); + if (replace < 0) break; + + top[replace] = best; + kept.Add(best.Entry); + } + + return [.. top.OrderByDescending(hit => hit.Score).ThenBy(hit => hit.Entry.Title, StringComparer.OrdinalIgnoreCase)]; + } + + /// <summary> + /// The titles that nearly matched: an entry whose name merely begins with one of the query's words, + /// which the ranking itself ignores on purpose - a prefix is far too weak a signal to rank a hit by, + /// and exactly strong enough to suggest a word to search for instead. + /// </summary> + private static string[] Suggest(string[] terms) + { + return [.. _entries.Value + .Where(entry => terms.Any(term => term.Length >= 4 + && entry.TitleWords.Any(word => word.StartsWith(term, StringComparison.OrdinalIgnoreCase) + // A one- or two-letter title word ("1", "Db") is the start of + // half the dictionary, so it suggests nothing. + || (word.Length >= 4 && term.StartsWith(word, StringComparison.OrdinalIgnoreCase))))) + .Select(entry => entry.Title) + .Distinct(StringComparer.OrdinalIgnoreCase) + .Order(StringComparer.OrdinalIgnoreCase) + .Take(MaxSuggestions)]; + } + + private static int Score(Entry entry, string[] terms) + { + var score = 0; + var matched = 0; + var named = 0; + + foreach (var term in terms) + { + var isTitleWord = entry.TitleWords.Any(word => Equivalent(word, term)); + + // Text is counted by the singular, which a plural query word would otherwise never find inside: + // "queries" has to count the "Query" of AsLiveQuery, and "backends" every "backend". + var stem = Stem(term); + var inTitle = isTitleWord ? 0 : Count(entry.Title, stem); + var inBoosted = Count(entry.Boosted, stem); + var inBody = Count(entry.Body, stem); + + if (isTitleWord is false && inTitle + inBoosted + inBody == 0) continue; + + matched++; + if (isTitleWord || inTitle + inBoosted > 0) named++; + + // A term in a name is worth far more than the same term buried in prose: someone asking for + // "AllowNonPersistentFallback" wants the option, not the paragraphs that happen to mention it. + score += (isTitleWord ? 12 : 0) + inTitle * 3 + inBoosted * 5 + Math.Min(inBody, MaxBodyScore); + } + + // Every term matching is the strongest signal a hit is the right one - and a term the entry is named + // by counts twice, since every long section mentions "database", "open" and "page" somewhere, and a + // multiplier fed only by prose hands the longest of them every query phrased in plain words. The + // weight is what keeps a worked example from outranking the reference that answers the question. + return matched == 0 ? 0 : score * (matched + named) * entry.Weight; + } + + /// <summary> + /// Same word, whatever form it is in - "migrate" has to find the section called "Migration", + /// "queries" the one about live queries, and nobody phrases a question in the number or tense the + /// heading happens to use. + /// </summary> + private static bool Equivalent(string word, string term) + { + return string.Equals(word, term, StringComparison.OrdinalIgnoreCase) + || string.Equals(Stem(word), Stem(term), StringComparison.OrdinalIgnoreCase); + } + + /// <summary> + /// A word without its plural, its -ing, -ed, -ion or -ies, and a final silent e: "migrates", + /// "migrating", "migrated" and "migration" are all "migrat", "queries" and "query" both "quer". Only + /// ever cut where at least three letters remain, so "wal" or "used" keep the letters that make them a + /// word. Prose is counted by the stem too, which as a substring also finds every longer form of it. + /// </summary> + private static string Stem(string word) + { + var stem = word.ToLowerInvariant(); + + if (stem.Length > 5 && stem.EndsWith("ies", StringComparison.Ordinal)) stem = $"{stem[..^3]}y"; + if (stem.Length > 4 && stem.EndsWith('s') && stem.EndsWith("ss", StringComparison.Ordinal) is false) stem = stem[..^1]; + if (stem.Length > 5 && stem.EndsWith("ing", StringComparison.Ordinal)) stem = stem[..^3]; + else if (stem.Length > 5 && stem.EndsWith("ion", StringComparison.Ordinal)) stem = stem[..^3]; + else if (stem.Length > 4 && stem.EndsWith("ed", StringComparison.Ordinal)) stem = stem[..^2]; + if (stem.Length > 3 && (stem.EndsWith('e') || stem.EndsWith('y'))) stem = stem[..^1]; + + return stem; + } + + /// <summary> + /// How often a term starts a word of the text. Only the start of a word counts - the start of the + /// text, a letter after a non-letter, or a capital after a lower-case letter, so "query" counts in + /// "AsLiveQuery" - because inside a word a term is a different word: every "Bsqlite" type name + /// contains "sqlite", and counting it there turns "sqlite version" into a search for the library's + /// name. The end of the word stays open, so a stem still finds its longer forms ("migrat" in "migration"). + /// </summary> + private static int Count(string text, string term) + { + if (text.Length == 0) return 0; + + var count = 0; + var index = text.IndexOf(term, StringComparison.OrdinalIgnoreCase); + + while (index >= 0) + { + if (StartsWord(text, index)) count++; + index = text.IndexOf(term, index + term.Length, StringComparison.OrdinalIgnoreCase); + } + + return count; + } + + private static bool StartsWord(string text, int index) + { + if (index == 0) return true; + + var previous = text[index - 1]; + + return char.IsLetterOrDigit(previous) is false || (char.IsUpper(text[index]) && char.IsLower(previous)); + } + + /// <summary> + /// The text that shows why an entry was found: its prose, or - when only its boosted words matched, as + /// a member found by its type is - those words, so the snippet never reads as a hit for nothing. + /// </summary> + private static string Snippet(Entry entry, string[] terms) + { + var stems = terms.Select(Stem).ToArray(); + + bool Mentions(string text) => stems.Any(stem => Count(text, stem) > 0); + + return Snippet(Mentions(entry.Body) || Mentions(entry.Boosted) is false ? entry.Body : entry.Boosted, terms); + } + + private static string Snippet(string body, string[] terms) + { + if (body.Length == 0) return string.Empty; + + var index = 0; + + // The same word starts the snippet that was counted for the score, not "sqlite" inside "Bsqlite". + foreach (var term in terms.Select(Stem)) + { + var at = body.IndexOf(term, StringComparison.OrdinalIgnoreCase); + while (at >= 0 && StartsWord(body, at) is false) at = body.IndexOf(term, at + term.Length, StringComparison.OrdinalIgnoreCase); + + if (at >= 0) + { + index = at; + break; + } + } + + // A snippet is a reason to make the follow-up call, not a substitute for making it: enough of the + // surrounding text to tell the right hit from the wrong one, and no more. Every extra character + // here is paid for by every hit of every search, most of which are not the one. + var start = Math.Max(0, index - 60); + var length = Math.Min(180, body.Length - start); + var snippet = body.Substring(start, length).Replace('\n', ' ').Replace('\r', ' ').Trim(); + + return $"{(start > 0 ? "..." : null)}{snippet}{(start + length < body.Length ? "..." : null)}"; + } + + /// <summary>Splits a name or heading into words, breaking camel-case humps as well as punctuation.</summary> + private static string[] SplitWords(string text) + { + var words = new List<string>(); + var current = new System.Text.StringBuilder(); + + foreach (var c in text) + { + if (char.IsLetterOrDigit(c) is false) + { + if (current.Length > 0) { words.Add(current.ToString()); current.Clear(); } + continue; + } + + // A capital after a lowercase letter starts a new word ("AllowNonPersistentFallback" -> allow non + // persistent fallback), while a run of capitals stays together ("OPFS", "WAL"). + if (char.IsUpper(c) && current.Length > 0 && char.IsLower(current[^1])) + { + words.Add(current.ToString()); + current.Clear(); + } + + current.Append(c); + } + + if (current.Length > 0) words.Add(current.ToString()); + + return [.. words]; + } + + private static string[] Tokenize(string? query) + { + if (string.IsNullOrWhiteSpace(query)) return []; + + string[] terms = [.. query.Split(['.', ',', ';', ':', '?', '!', '"', '\'', '(', ')', '[', ']', '{', '}', '/', '\\', '<', '>', '-', '_', ' ', '\t', '\n', '\r'], + StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries) + // One- and two-letter words ("a", "in", "do") match everything and rank nothing - and the words a + // question is phrased with do worse than nothing: "how do I keep data FROM indexeddb" would + // otherwise score a section whose heading merely contains "from". + .Where(term => term.Length > 2 && _stopWords.Contains(term) is false) + .Distinct(StringComparer.OrdinalIgnoreCase) + // Every term is counted in every entry's body, so the work is terms x corpus. No question is + // phrased in more words than this, while a pasted file as a query would scan for hours. + .Take(MaxTerms)]; + + return DropAmbientWords(terms); + } + + private static Entry[] Build() + { + var entries = new List<Entry>(1024); + + foreach (var document in BsqliteSourceCatalog.Documents) + { + // The document as a whole, for the question that asks for it by name ("the storage guide"). Its + // body is what no section entry covers - the prose above its first section, which for a document + // without any (the limits table) is all of it. + entries.Add(new Entry("Document", document.Title, document.Name, + $"GetBsqliteDocument(name: \"{document.Name}\")", BsqliteSourceCatalog.GetPreamble(document.Name) ?? document.Summary ?? string.Empty, + $"{document.Name.Replace('-', ' ')} {document.Kind.ToLowerInvariant()}")); + + foreach (var section in BsqliteSourceCatalog.GetSections(document.Name)) + { + if (_metaSections.Contains((document.Name, section.Heading))) continue; + + var body = BsqliteSourceCatalog.GetDocumentSection(document.Name, section.Key) ?? string.Empty; + + entries.Add(new Entry(document.Kind == "Package" ? "Package section" : "Guide section", section.Heading, + section.Parent is null ? document.Name : $"{document.Name} > {section.Parent}", + $"GetBsqliteDocument(name: \"{document.Name}\", section: \"{section.Key}\")", body, string.Empty)); + } + } + + foreach (var type in BsqliteApiCatalog.Types) + { + entries.Add(new Entry($"API {type.Kind.ToLowerInvariant()}", type.Name, type.Namespace, + $"GetBsqliteApi(typeName: \"{type.Name}\")", type.Summary ?? string.Empty, string.Empty)); + + var details = BsqliteApiCatalog.GetTypeDetails(type.Name); + if (details is null) continue; + + // The conformance kit's tests are named as sentences ("ProbeAsync_ReportsUnavailability_RatherThanThrowing"), + // so they match half the words of any question about storage while answering none of it. The kit is + // found by its type, whose reference lists every rule; of its members only the ones a test class + // overrides are indexed, since those are what a backend author has to write. + var kit = type.Namespace.EndsWith(".ConformanceTests", StringComparison.Ordinal); + + // One entry per member NAME, not per member: two overloads of UseBitBsqlite are two members and + // one thing to know about, and both are read by the same call anyway. Their documentation is + // searched as one body, so a term in either overload finds the member. A constructor is named + // after its type, which the type's own entry already is. + foreach (var overloads in details.Members.Where(member => member.Kind != "Constructor" && (kit is false || member.Protected)).GroupBy(member => member.Name, StringComparer.Ordinal)) + { + var member = overloads.First(); + + // Overloads usually share their documentation; it is searched, and quoted in a snippet, once. + entries.Add(new Entry($"API {member.Kind.ToLowerInvariant()}", $"{type.Name}.{member.Name}", type.Name, + $"GetBsqliteApi(typeName: \"{type.Name}\")", + string.Join(' ', overloads.Select(o => $"{o.Summary} {o.Remarks}".Trim()).Distinct(StringComparer.Ordinal)).Trim(), + string.Join(' ', overloads.Select(o => $"{o.Type} {o.Signature}".Trim())).Trim())); + } + } + + // The table as a whole, for the question that asks for the list itself ("which backends are there") + // rather than for one of its rows. + entries.Add(new Entry("Storage backend", "Storage backends", null, + "GetBsqliteStorageProviders()", BsqliteStorageCatalog.RenderIndex(), "storage backend backends provider providers capabilities persistence")); + + foreach (var provider in BsqliteStorageCatalog.Providers) + { + entries.Add(new Entry("Storage backend", provider.TypeName, provider.Name, + $"GetBsqliteStorageProviders(provider: \"{provider.Name}\")", + $"{provider.Summary} {provider.Remarks} {string.Join(", ", provider.Capabilities.Select(capability => $"{capability.Name} {capability.Value}"))}", + $"storage backend provider {provider.Name.Replace('-', ' ')} {provider.Package}")); + } + + foreach (var scenario in BsqliteSetupGuide.Scenarios) + { + entries.Add(new Entry("Setup guide", $"Setup: {BsqliteSetupGuide.Title(scenario)}", scenario, + $"GetBsqliteSetupGuide(scenario: \"{scenario}\")", BsqliteSetupGuide.Summary(scenario), + $"setup install wiring register {scenario.Replace('-', ' ')}")); + } + + entries.Add(new Entry("Storage lab", "Plan a storage selection", null, + "PlanBsqliteStorageSelection()", + "Runs the library's own storage selection - resolver, migrator and session factory - on a browser you describe: " + + "which backends are registered, available and persistent, the preference order, AllowNonPersistentFallback, the " + + "migration mode and an existing sticky binding; reports the backend chosen, every candidate's status and why, " + + "the binding before and after, any automatic migration, and what the library logged.", + "plan predict simulate selection fallback binding sticky migration migrate unavailable which backend chosen")); + + entries.Add(new Entry("SQL check", "Check SQL against the guards", null, + "CheckBsqliteSql()", + "Runs BsqliteFeatureGuards and BsqliteTableNames on SQL without executing it: which backends refuse it - WAL " + + "everywhere, ATTACH on OPFS, PRAGMA page_size on IndexedDB and Cache Storage - with the guard's own message, " + + "and which tables a live query over it watches.", + "check validate guard guards refused rejected pragma wal attach page_size journal_mode tables live query sql")); + + foreach (var file in BsqliteSourceCatalog.SourceFiles) + { + entries.Add(new Entry("Source file", file.Path, file.Kind, + $"GetBsqliteSourceFile(path: \"{file.Path}\")", file.Description ?? string.Empty, string.Empty, ExampleWeight)); + } + + return [.. entries]; + } + + /// <summary> + /// The query without the words that are true of everything here - unless that is all it said, in + /// which case they are all it can be searched by. + /// </summary> + private static string[] DropAmbientWords(string[] terms) + { + var meaningful = terms.Where(term => _ambientWords.Contains(term) is false).ToArray(); + + return meaningful.Length > 0 ? meaningful : terms; + } +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteServerInstructions.cs b/src/Bsqlite/Demo/Server/Services/BsqliteServerInstructions.cs new file mode 100644 index 00000000000..9d2eb1d0786 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteServerInstructions.cs @@ -0,0 +1,62 @@ +using System.Reflection; +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// The text the server returns as <c>instructions</c> from the <c>initialize</c> handshake. +/// <para> +/// A client puts this in front of the model once, before any tool has been called, which makes it +/// the only place where the server can say something the model will read whether or not it decides +/// to use the tools at all. So it stays short - every word is permanently in someone's context +/// window - and spends itself on the working rules rather than on describing the library, which the +/// tools do far better. +/// </para> +/// </summary> +public static class BsqliteServerInstructions +{ + /// <summary> + /// The build the answers come from, so the text names it rather than claiming to be timeless. + /// Read off the library assembly rather than written down here: the package version is stamped + /// into it by Bit.Build.props, so there is no second copy for a release to forget. The + /// informational version's build metadata - everything after '+', usually the commit hash - is + /// dropped: it identifies nothing a caller could act on. + /// </summary> + public static readonly string BsqliteVersion = ( + typeof(BsqliteConnection).Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>()?.InformationalVersion + ?? typeof(BsqliteConnection).Assembly.GetName().Version?.ToString() + ?? "unknown").Split('+')[0]; + + public static readonly string Text = $""" + Bit.Bsqlite is SQLite in the browser for Blazor, with EF Core on top: the stock EF Core SQLite provider over a + connection whose engine (wa-sqlite) runs in a dedicated worker, storage backends (OPFS, IndexedDB, Cache Storage, + in-memory) chosen and remembered per database, live queries, and a complete async ADO.NET surface. + + Every answer here is read out of Bit.Bsqlite {BsqliteVersion} as it is loaded in this server - its compiled XML + documentation, its README and guides, the storage providers' declared capabilities, its own selection and SQL + guards, and the demo's source files. Where what you remember and what these tools say disagree, the tools are + describing the build in front of you. + + Working rules: + + - Start with `SearchBsqlite` unless you already know the document or type you want. Every hit names the exact + follow-up call that returns its full text. + - Every reference tool takes the key of one thing and answers with that thing; leave the key out and it answers + with the index of what there is - documents, public types, storage backends, source files. + - Do not write a Bsqlite type or namespace from memory. They are spread over `Bit.Bsqlite`, `Bit.Bsqlite.Data`, + `Bit.Bsqlite.Interop`, `Bit.Bsqlite.EntityFrameworkCore` and `Bit.Bsqlite.Storage.*`, while + `AddBitBsqliteServices` and `AddBitBsqliteDbContext` live in `Microsoft.Extensions.DependencyInjection` and + `UseBitBsqlite` in `Microsoft.EntityFrameworkCore`. `GetBsqliteApi` gives the real names and namespaces. + - Call `GetBsqliteSetupGuide` before writing wiring. In a Blazor Web App the services are registered in the + client AND the server project, and a page opens its database only once `RendererInfo.IsInteractive` is true; + Interactive Server is not supported. + - Use the async API throughout (`ToListAsync`, `SaveChangesAsync`, `MigrateAsync`): synchronous members throw + `BsqliteSynchronousApiNotSupportedException`. + - Storage selection never substitutes silently: existing data outranks preference, and a non-persistent fallback + needs `AllowNonPersistentFallback()`. `PlanBsqliteStorageSelection` runs the library's own selection on a + described browser; `CheckBsqliteSql` says which statements the guards refuse on which backend. + + Every tool on this server is read-only and cheap: none of them changes anything, anywhere, and none executes + SQL, so there is no reason to ration them or to ask before calling one. + """; +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteSetupGuide.cs b/src/Bsqlite/Demo/Server/Services/BsqliteSetupGuide.cs new file mode 100644 index 00000000000..45fa5b35182 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteSetupGuide.cs @@ -0,0 +1,928 @@ +using System.Text; +using Bit.Bsqlite.Demo.Server.Dtos; +using System.Text.RegularExpressions; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// Assembles the wiring for one scenario: the checklist, the code, the library's own documents on the +/// subject quoted verbatim, and - where the demo does it - the demo's real files. +/// <para> +/// Bsqlite's setup is short, which is exactly why an agent gets it wrong: it registers the services +/// only in the client of a Blazor Web App and prerendering fails, opens the database in +/// OnInitializedAsync and prerendering fails differently, reaches for Interactive Server, calls +/// SaveChanges(), or ranks a new backend above the one a Bit.Besql app's data is in and starts every +/// returning user on an empty database. None of those is a compiler error. Each scenario here leads with +/// the rule that prevents its version of the mistake, and the documents it quotes say the rest in the +/// library's own words, so the caveats cannot drift from a paraphrase of them. +/// </para> +/// </summary> +public static partial class BsqliteSetupGuide +{ + public static readonly string[] Scenarios = ["standalone-wasm", "wasm", "auto", "server", "storage", "move-from-besql", "testing", "custom-provider"]; + + /// <summary>The demo files that make up the wiring of a prerendered Blazor Web App on InteractiveWebAssembly - the demo's own model.</summary> + private static readonly string[] _webAppFiles = + [ + "Demo/Client/Extensions/IServiceCollectionExtensions.cs", + "Demo/Server/Program.cs", + "Demo/Client/Program.cs", + "Demo/Server/Components/App.razor", + "Demo/Client/Components/SessionLifecycle.razor", + "Demo/Client/BrowserDatabase.cs", + "Demo/Client/Routes.razor", + "Demo/Client/_Imports.razor", + ]; + + private sealed record Scenario(string Title, string Summary, string Checklist, string? Code, string[] Files, (string Document, string Section)[] Quotes, string Next); + + /// <summary>The wiring for one scenario as Markdown, or null when the scenario is not one of <see cref="Scenarios"/>.</summary> + public static string? Get(string? scenario) => Describe(scenario) is { } found ? Compose(found) : null; + + /// <summary>What sets one scenario apart, in a paragraph - the line the search index ranks a setup guide by.</summary> + public static string Summary(string scenario) => Describe(scenario) is { } found ? $"{found.Title}. {found.Summary.ReplaceLineEndings(" ")}" : string.Empty; + + /// <summary>The scenario's title, for a listing.</summary> + public static string Title(string scenario) => Describe(scenario)?.Title ?? scenario; + + /// <summary> + /// What a scenario quotes - its document sections and its demo files. A guide leaves out a quote it + /// cannot find rather than failing, so this is what lets a test notice a heading renamed underneath it. + /// </summary> + internal static ((string Document, string Section)[] Quotes, string[] Files) Sources(string scenario) + => Describe(scenario) is { } found ? (found.Quotes, found.Files) : ([], []); + + /// <summary>The scenario a caller means: its own name, or one of the spellings people reach for first.</summary> + public static string? Resolve(string? scenario) + { + var key = (scenario ?? string.Empty).Trim().ToLowerInvariant().Replace(' ', '-').Replace('_', '-'); + + key = key switch + { + "standalone" or "standalone-webassembly" or "blazor-webassembly" or "webassembly-standalone" or "wasm-standalone" => "standalone-wasm", + "webassembly" or "interactivewebassembly" or "blazor-web-app" or "web-app" or "prerendering" or "prerendered" => "wasm", + "interactiveauto" or "interactive-auto" => "auto", + "interactiveserver" or "interactive-server" or "blazor-server" => "server", + "storage-providers" or "providers" or "backends" or "storage-backends" or "opfs" or "indexeddb" => "storage", + "besql" or "bit.besql" or "from-besql" or "migrate-from-besql" or "cache-storage" => "move-from-besql", + "tests" or "test" or "unit-testing" or "unit-tests" or "mstest" or "bit.bsqlite.testing" => "testing", + "provider" or "custom-storage" or "storage-provider" or "custom-storage-provider" or "custom-backend" or "vfs" => "custom-provider", + _ => key + }; + + return Scenarios.Contains(key, StringComparer.Ordinal) ? key : null; + } + + private static Scenario? Describe(string? scenario) => Resolve(scenario) switch + { + "standalone-wasm" => new Scenario( + "standalone Blazor WebAssembly", + """ + One project and no prerendering: the app runs on WebAssembly from its first render, so a page can open the + database whenever it likes - OnInitializedAsync included - and everything Bsqlite does is available. + """, + HostingChecklist, + StandaloneCode, + [], + [("getting-started", "Register"), ("getting-started", "Use EF Core")], + NextHosting), + + "wasm" => new Scenario( + "Blazor Web App - InteractiveWebAssembly with prerendering", + """ + How this demo is built. Every page is prerendered on the server and then runs on WebAssembly, where the + database lives. So the services are registered in BOTH containers - the prerender pass instantiates every + page on the server, and whatever a page injects has to resolve there - which is harmless, because nothing + touches the browser until a page opens the database. And a page opens it only once + `RendererInfo.IsInteractive` is true: during prerendering OnInitializedAsync runs on the server, where there + is no browser to open a database in. The demo shares one registration method between the two projects and + gates every page through one component; both are below, as they ship. + """, + HostingChecklist + "\n" + WebAppChecklist, + WebAppCode, + _webAppFiles, + [("readme", "Installation"), ("readme", "Where it runs")], + NextHosting), + + "auto" => new Scenario( + "Blazor Web App - InteractiveAuto", + """ + A page on InteractiveAuto runs on the server over a circuit until the WebAssembly runtime has downloaded, + and on WebAssembly after that. Bsqlite works once the page runs on WebAssembly - on the circuit the engine + would be in the user's tab while EF Core ran on the server, which is not supported. Two things follow. + `RendererInfo.IsInteractive` is true on the circuit as well, so it is not the gate here: + `OperatingSystem.IsBrowser()` is. And a component that started on the circuit stays there until the next + full load, so the reliable setup is per-page render modes: InteractiveWebAssembly for the pages that use the + database, Auto for the rest. Everything else is the InteractiveWebAssembly setup, whose files are the demo's. + """, + HostingChecklist + "\n" + WebAppChecklist + "\n" + AutoChecklist, + AutoCode, + _webAppFiles, + [("readme", "Where it runs")], + NextHosting), + + "server" => new Scenario( + "Blazor Web App - InteractiveServer (not supported)", + """ + Not supported, by design: the engine runs in the user's browser tab, in a worker, and EF Core on the server + would need a native SQLite of its own rather than a database in someone's browser. A server-rendered app can + still use Bsqlite for the pages that need an on-device database: render those with InteractiveWebAssembly + (they live in a .Client project) and keep Interactive Server for the rest. If the data belongs on the server + instead, that is Microsoft.EntityFrameworkCore.Sqlite or any other server database - not Bsqlite. + """, + ServerChecklist, + ServerCode, + [], + [("readme", "Where it runs")], + "For the pages that do run on WebAssembly, follow `GetBsqliteSetupGuide(scenario: \"wasm\")`."), + + "storage" => new Scenario( + "Choosing and adding storage backends", + """ + Every backend is a package plus one registration line, and the preference order is yours to set. Two rules + decide more than the order does: existing data outranks preference - a database stays on the backend that + holds it - and nothing is substituted silently, so a non-persistent fallback needs an explicit opt-in. The + table below is read off the provider types themselves. + """, + StorageChecklist, + StorageTable() + "\n" + StorageCode, + ["Demo/Client/Extensions/IServiceCollectionExtensions.cs", "Demo/Client/BrowserDatabase.cs"], + [("storage", "Selection"), ("storage", "Migration"), ("storage", "Admin")], + "`PlanBsqliteStorageSelection` runs this selection on a browser you describe - which backends are available, " + + "where the data already is, which migration mode - and reports every candidate's fate. " + + "`GetBsqliteStorageProviders(provider: \"...\")` has one backend in full."), + + "move-from-besql" => new Scenario( + "Moving off Bit.Besql", + """ + Bit.Besql keeps a whole SQLite file in Cache Storage - the `bit-Besql` cache, at `/data/cache/{file name}`. + The `cache-storage` backend reads that file: the first time a database is opened on it and Bsqlite has no + copy of its own, the Besql file of the same name is imported losslessly into Bsqlite's page layout, so the + app's users keep their data. Two things decide whether that happens: the database name must be Besql's + Data Source exactly, and `cache-storage` must be the first choice - a database with no binding yet is + created on the first available candidate, so any other backend ranked above it starts every returning user + on an empty database and the Besql file is never read. + """, + BesqlChecklist, + BesqlCode, + [], + [("readme", "Moving off Bit.Besql"), ("Bit.Bsqlite.Storage.CacheStorage", "Capabilities")], + "`GetBsqliteStorageProviders(provider: \"cache-storage\")` has the backend in full; for the page lifecycle " + + "under prerendering, `GetBsqliteSetupGuide(scenario: \"wasm\")`."), + + "testing" => new Scenario( + "Unit testing with Bit.Bsqlite.Testing", + """ + One package, two jobs. `BsqliteInProcessTransport` stands in for the browser's worker with Microsoft.Data.Sqlite, + so EF Core mappings, migrations, generated SQL and live queries run on desktop .NET in a test - it installs + the same EF function set the worker does and reports writes the way the worker does. And the MSTest + conformance kit is what every storage provider must pass. + """, + TestingChecklist, + TestingCode, + [], + [("Bit.Bsqlite.Testing", "1. Run the stack without a browser"), ("Bit.Bsqlite.Testing", "2. The storage conformance kit")], + "`GetBsqliteApi(typeName: \"BsqliteInProcessTransport\")` has the transport's constructor and what it records; " + + "`GetBsqliteSetupGuide(scenario: \"custom-provider\")` is the conformance kit's other half."), + + "custom-provider" => new Scenario( + "Writing a custom storage provider", + """ + A backend is two halves: a .NET `IBsqliteStorageProvider` that declares capabilities and answers probes, and a + JavaScript VFS the worker registers with the engine. The contract is public and the core consumes nothing + else, so a backend needs no change to Bsqlite - Cache Storage was built that way. What it does need is + honesty: the core trusts the declared capabilities without checking them, which is what the conformance kit + is for. + """, + ProviderChecklist, + ProviderContract() + "\n" + ProviderCode, + [], + [("storage", "Writing a provider"), ("Bit.Bsqlite.Testing", "2. The storage conformance kit")], + "`GetBsqliteApi(typeName: \"BsqliteInMemoryStorageProvider\")` is the contract's reference implementation - the " + + "smallest provider that passes the kit - and `GetBsqliteStorageProviders()` shows what the first-party " + + "backends declare."), + + _ => null + }; + + private static string Compose(Scenario scenario) + { + var builder = new StringBuilder(); + + builder.AppendLine($"# Setting Bsqlite up - {scenario.Title}").AppendLine(); + builder.AppendLine(scenario.Summary).AppendLine(); + builder.AppendLine(scenario.Checklist).AppendLine(); + + if (scenario.Code is not null) builder.AppendLine(scenario.Code).AppendLine(); + + if (scenario.Files.Length > 0) + { + builder.AppendLine("## The files, from this demo").AppendLine(); + builder.AppendLine("Everything below compiles as it ships; rename the projects and namespaces to your own. The host's ") + .AppendLine("Program.cs also runs the demo's model on the stock provider for its /api/products endpoint and hosts this ") + .AppendLine("MCP server - neither is anything Bsqlite needs.") + .AppendLine(); + + foreach (var path in scenario.Files) + { + var content = BsqliteSourceCatalog.GetSourceFile(path); + if (content is null) continue; + + AppendFile(builder, path, content); + } + } + + // The library's own words on the subject, so the caveats are the library's rather than a + // paraphrase of them that can drift. + builder.AppendLine("## From the documentation").AppendLine(); + + foreach (var (document, section) in scenario.Quotes) + { + var text = BsqliteSourceCatalog.GetDocumentSection(document, section); + if (text is null) continue; + + builder.AppendLine($"_{document}:_").AppendLine(); + builder.AppendLine(Demote(text)).AppendLine(); + } + + builder.AppendLine(scenario.Next); + + return builder.ToString(); + } + + private const string HostingChecklist = """ + ## Checklist + + 1. `dotnet add package Bit.Bsqlite` plus at least one persistent backend - `Bit.Bsqlite.Storage.Opfs` first and + `Bit.Bsqlite.Storage.IndexedDb` as its fallback. In a Blazor Web App, add them to the .Client project. + 2. Register with `AddBitBsqliteServices` (namespace `Microsoft.Extensions.DependencyInjection`): each backend with + `AddStorageProvider<T>()`, the preference order with `UseStorage(s => s.Prefer(...).Fallback(...))` - required, + selection never guesses - and the database with `AddDatabase("app.db")`. + 3. Register the context with `AddBitBsqliteDbContext<AppDbContext>()`. Migrations run once per session by default + (`BsqliteSchemaStrategy.Migrate`); seed through `OnFirstUse`. Do not also call `AddDbContext` or `UseSqlite`: + the factory binds every context to the one shared browser database. + 4. Inject `IBsqliteDbContextFactory<AppDbContext>` (namespace `Bit.Bsqlite.EntityFrameworkCore`; the plain + `IDbContextFactory<AppDbContext>` is registered too) and create contexts with `await CreateDbContextAsync()`. + 5. Async only: `ToListAsync`, `SaveChangesAsync`, `MigrateAsync`. The synchronous members throw + `BsqliteSynchronousApiNotSupportedException` naming the async overload - the engine runs in a worker the main + thread cannot wait on. + 6. Nothing goes into the host page: the engine, the worker and the VFS modules are static web assets under + `_content/Bit.Bsqlite*/`, imported on demand. + 7. Keep the model in a project that references only EF Core Relational, as the demo's Data project does, and + scaffold migrations with `dotnet ef` against the stock SQLite provider: the same migrations apply in the browser. + """; + + private const string WebAppChecklist = """ + ### In a Blazor Web App + + 8. Register the services in the server project too - the prerender pass resolves everything a page injects. Put + the registration in one extension method in the .Client project and call it from both Program.cs files, as the + demo's `AddDemoServices` is. + 9. Open the database only once the page is interactive: `if (RendererInfo.IsInteractive is false) return;` before + the first `CreateDbContextAsync()` (on .NET 8, which has no `RendererInfo`, `OperatingSystem.IsBrowser()`). The + demo's `SessionLifecycle` component does it for every page. + 10. Render the pages that use the database with InteractiveWebAssembly - the demo sets it once, on `<Routes>`. + 11. Drop live queries and contexts when `BsqliteDatabase.Closing` fires and reload on `Opened`: that is how a page + survives the database being moved or replaced underneath it. + """; + + private const string AutoChecklist = """ + ### Under InteractiveAuto + + 12. Put no render mode on `<Routes>`: give each page its own, InteractiveWebAssembly for every page that uses the + database. A page inside an interactive parent cannot switch to another render mode. + 13. Where a page that touches the database has to stay on Auto, gate it on `OperatingSystem.IsBrowser()`, not + `RendererInfo.IsInteractive` - the circuit is interactive too - and render a placeholder until it is true. + """; + + private const string ServerChecklist = """ + ## Checklist + + 1. Do not open a Bsqlite database from a server-rendered component. The services resolve on a circuit - every one + is scoped per circuit, so nothing is shared between users - but opening a database there is not a supported path. + 2. Add a .Client project and `AddInteractiveWebAssemblyComponents()` / `AddInteractiveWebAssemblyRenderMode()` next + to the Interactive Server ones, and move the pages that use the browser database into it with + `@rendermode InteractiveWebAssembly`. + 3. Register Bsqlite in both projects, as for any prerendered Web App, and open the database only once the page is + interactive on WebAssembly. + """; + + private const string StorageChecklist = """ + ## Checklist + + 1. OPFS first (`Bit.Bsqlite.Storage.Opfs`): the only persistent backend on the synchronous engine build - the + smallest download - and the fastest storage the web has. + 2. IndexedDB as its fallback (`Bit.Bsqlite.Storage.IndexedDb`): the widest reach, where OPFS is missing. + 3. In-memory (`Bit.Bsqlite.Storage.InMemory`, in the core package) only as a declared last resort - with + `AllowNonPersistentFallback()` - or for demos and tests. Data written to it is gone on reload. + 4. Cache Storage (`Bit.Bsqlite.Storage.CacheStorage`) when moving off Bit.Besql - see that scenario - and otherwise + not: it exists for the import. + 5. Leave the migration mode at `KeepExisting` unless moving a user's database without asking is acceptable. + `AutomaticOnOpen` copies, verifies and only then flips the binding; `Manual` reports + `Resolution.BetterProviderAvailable` for the app to act on. + 6. Show `session.Resolution` (or `BsqliteDatabase.Resolution`) somewhere a support person can read it: it records + every candidate, its probe and why it was passed over. + """; + + private const string BesqlChecklist = """ + ## Checklist + + 1. `dotnet remove package Bit.Besql`, then `dotnet add package Bit.Bsqlite` and + `dotnet add package Bit.Bsqlite.Storage.CacheStorage`. + 2. Remove `<script src="_content/Bit.Besql/bit-besql.js"></script>` from the host page; Bsqlite needs nothing there. + 3. Replace `AddBesqlDbContextFactory<TContext>(...)` with `AddBitBsqliteServices` and + `AddBitBsqliteDbContext<TContext>`, naming the database exactly as Besql's `Data Source` - the import looks for + `/data/cache/{that name}`. + 4. Make `cache-storage` the first (or only) choice, as above. A database that has been opened is bound there for + good; moving it to OPFS later is a deliberate migration (see the last step), never a reordering. + 5. Move what Besql's `dbContextInitializer` did: `MigrateAsync` is the default schema step, anything else goes in + `OnFirstUse`. A compiled model passed to `UseModel` moves to `options.Configure`. Drop the calls to Besql's + `ConfigureSqliteJournalMode()` - Bsqlite never runs in WAL mode. + 6. Code that injects `IDbContextFactory<TContext>` keeps working, but every synchronous call has to become async: + Besql ran SQLite on the main thread, Bsqlite runs it in a worker, and `SaveChanges()`, `ToList()` and the like + throw `BsqliteSynchronousApiNotSupportedException`. `CreateDbContext()` works only after the first + `CreateDbContextAsync()`. + 7. Where Besql said "use the factory in OnAfterRender", open the database once `RendererInfo.IsInteractive` is + true (see the `wasm` scenario). If the app also ran on Blazor Server or Hybrid, where Besql fell back to a file + on disk, keep `Microsoft.EntityFrameworkCore.Sqlite` for those hosts: Bsqlite is for pages running on WebAssembly. + 8. To move the data to OPFS once it is in Bsqlite, migrate it explicitly - inject `BsqliteStorageMigrator`, the + providers and `IBsqliteStorageBindingStore`, and call `MigrateAsync` inside `BsqliteDatabase.WhileClosedAsync`, + as the demo's `BrowserDatabase.SwitchStorageAsync` does. Ranking OPFS above `cache-storage` instead would start + users who have not opened the new version yet on an empty OPFS database. + """; + + private const string TestingChecklist = """ + ## Checklist + + 1. A desktop test project (net8.0, net9.0 or net10.0) on MSTest, and `dotnet add package Bit.Bsqlite.Testing` - it + brings Microsoft.Data.Sqlite with its native engine, which belongs in tests and never in the browser app. + 2. `new BsqliteInProcessTransport()` is a private in-memory database; wrap it in + `new BsqliteConnection(transport, "test.db")`, `await connection.OpenAsync()`, and hand the connection to + `UseBitBsqlite` - the same call the browser makes. One transport is one database: `await using` it. + 3. Assert the SQL EF generated with `transport.ExecutedCommands`. + 4. The async rule holds here too - synchronous members throw - so a test catches a stray `SaveChanges()` before a + user does. Live queries behave as in the browser: writes are reported from the update hook once they commit. + 5. To run the whole DI composition, register an `IBsqliteTransportFactory` that returns the in-process transport, + the in-memory backend, and `UseBindingStore<BsqliteInMemoryStorageBindingStore>()` (the default store needs a + browser's localStorage). + 6. A storage backend's test project inherits `BsqliteStorageProviderConformanceTests` (and, with a running engine, + `BsqliteStorageEngineConformanceTests`) from namespace `Bit.Bsqlite.Storage.ConformanceTests`, with + `[TestClass]` on each derived class; a backend that cannot run in that environment reports inconclusive. + """; + + private const string ProviderChecklist = """ + ## Checklist + + 1. A stable, qualified `Name` ("mycompany-encrypted", not "encrypted"): it is recorded as the sticky binding of + every database the backend creates, so renaming it after release orphans every user's data. + 2. `Capabilities` that are constant and honest: `RequiredBuild` (`AsyncCapable` for any VFS that awaits storage) + and `IsPersistent` are required; everything else defaults to the least capable answer - declare a capability + only when the VFS really provides it. The core gates ATTACH, `PRAGMA page_size` and fallback on these. + 3. `ProbeAsync` reports unavailability with `BsqliteProbeResult.Unavailable(reason)` rather than throwing, and + needs a browser: take an optional `IJSRuntime` and report unavailable without one, as the first-party + providers do. The core probes at most once per session. + 4. `Admin` works on databases as files with no engine running - exists, list, delete, export, import - which is + what migration and backups are built on. + 5. `VfsModule` points at an ES module in your package's static assets that exports `register`. Import the base + classes relatively from `../Bit.Bsqlite/engine/`, so an app served under a sub-path still finds them. + 6. Register it with `AddStorageProvider<T>()` and inherit the conformance kit in its test project. + """; + + private const string StandaloneCode = """ + ## The code + + ### `Program.cs` + + ```csharp + using Bit.Bsqlite.Storage.IndexedDb; + using Bit.Bsqlite.Storage.Opfs; + using Microsoft.AspNetCore.Components.Web; + using Microsoft.AspNetCore.Components.WebAssembly.Hosting; + + var builder = WebAssemblyHostBuilder.CreateDefault(args); + + builder.RootComponents.Add<App>("#app"); + builder.RootComponents.Add<HeadOutlet>("head::after"); + + builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider<BsqliteOpfsStorageProvider>() + .AddStorageProvider<BsqliteIndexedDbStorageProvider>() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName)) + .AddDatabase("app.db")); + + builder.Services.AddBitBsqliteDbContext<AppDbContext>(); + + await builder.Build().RunAsync(); + ``` + + ### `Pages/Products.razor` + + ```razor + @page "/products" + @using Bit.Bsqlite.EntityFrameworkCore + @using Microsoft.EntityFrameworkCore + @inject IBsqliteDbContextFactory<AppDbContext> Contexts + + <ul> + @foreach (var product in _products) + { + <li>@product.Name - @product.Price</li> + } + </ul> + + <button @onclick="AddAsync">Add</button> + + @code { + private List<Product> _products = []; + + protected override async Task OnInitializedAsync() + { + // The first context opens the database and applies the migrations; later ones reuse the session. + await using var db = await Contexts.CreateDbContextAsync(); + _products = await db.Products.OrderBy(p => p.Name).ToListAsync(); + } + + private async Task AddAsync() + { + await using var db = await Contexts.CreateDbContextAsync(); + db.Products.Add(new Product { Name = $"Widget {_products.Count + 1}", Price = 19.99m }); + await db.SaveChangesAsync(); + _products = await db.Products.OrderBy(p => p.Name).ToListAsync(); + } + } + ``` + + There is no script tag or stylesheet to add to `wwwroot/index.html`. + """; + + private const string WebAppCode = """ + ## A page, without the demo's helpers + + The demo routes every page through its own `SessionLifecycle` component and `BrowserDatabase` wrapper (both + below). An app needs neither - this is the whole rule: + + ```razor + @page "/orders" + @using Bit.Bsqlite.EntityFrameworkCore + @using Microsoft.EntityFrameworkCore + @inject IBsqliteDbContextFactory<AppDbContext> Contexts + + @code { + private List<SalesOrder>? _orders; + + protected override async Task OnInitializedAsync() + { + // Prerendering runs this on the server first, where there is no browser to open a database in. + // It runs again once the page is interactive on WebAssembly. + if (RendererInfo.IsInteractive is false) return; + + await using var db = await Contexts.CreateDbContextAsync(); + _orders = await db.Orders.Where(o => o.Status == OrderStatus.Draft).ToListAsync(); + } + } + ``` + """; + + private const string AutoCode = """ + ## The differences from InteractiveWebAssembly + + ### Server `Program.cs` + + ```csharp + builder.Services.AddRazorComponents() + .AddInteractiveServerComponents() + .AddInteractiveWebAssemblyComponents(); + + builder.Services.AddAppServices(); // the client's registration, Bsqlite's included + + // ... + + app.MapRazorComponents<App>() + .AddInteractiveServerRenderMode() + .AddInteractiveWebAssemblyRenderMode() + .AddAdditionalAssemblies(typeof(MyApp.Client._Imports).Assembly); + ``` + + ### `App.razor` + + ```razor + @* No render mode here: each page chooses its own. *@ + <Routes /> + ``` + + ### A page that uses the database + + ```razor + @page "/orders" + @rendermode InteractiveWebAssembly + @inject IBsqliteDbContextFactory<AppDbContext> Contexts + ``` + + ### A component that has to stay on Auto + + ```razor + @code { + protected override async Task OnInitializedAsync() + { + // False while prerendering AND on the server circuit Auto starts on: true only on WebAssembly. + if (OperatingSystem.IsBrowser() is false) return; + + await using var db = await Contexts.CreateDbContextAsync(); + // ... + } + } + ``` + """; + + private const string ServerCode = """ + ## The code + + ### Server `Program.cs` + + ```csharp + builder.Services.AddRazorComponents() + .AddInteractiveServerComponents() + .AddInteractiveWebAssemblyComponents(); + + builder.Services.AddAppServices(); // shared with the .Client project, Bsqlite's registration included + + // ... + + app.MapRazorComponents<App>() + .AddInteractiveServerRenderMode() + .AddInteractiveWebAssemblyRenderMode() + .AddAdditionalAssemblies(typeof(MyApp.Client._Imports).Assembly); + ``` + + ### `MyApp.Client/Pages/Notes.razor` + + ```razor + @page "/notes" + @rendermode InteractiveWebAssembly + @inject IBsqliteDbContextFactory<NotesDbContext> Contexts + ``` + """; + + private const string StorageCode = """ + ### Every first-party backend, in the recommended order + + ```csharp + using Bit.Bsqlite.Storage.IndexedDb; + using Bit.Bsqlite.Storage.InMemory; + using Bit.Bsqlite.Storage.Opfs; + + builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider<BsqliteOpfsStorageProvider>() + .AddStorageProvider<BsqliteIndexedDbStorageProvider>() + .AddStorageProvider<BsqliteInMemoryStorageProvider>() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()) // the in-memory fallback is refused without it + .AddDatabase("app.db")); + ``` + + ### Why this session landed where it did + + ```csharp + // In a component that injects BsqliteDatabase (namespace Bit.Bsqlite.Interop) as Database: + var session = await Database.OpenAsync(); + var resolution = session.Resolution; + + // resolution.Provider.Name, resolution.IsFirstChoice, resolution.WasDecidedByExistingData, + // resolution.BetterProviderAvailable, and every candidate in resolution.Attempts with its status and reason. + ``` + """; + + private const string BesqlCode = """ + ## The code + + ### Before - Bit.Besql + + ```csharp + services.AddBesqlDbContextFactory<OfflineDbContext>((sp, optionsBuilder) => + { + optionsBuilder + .UseModel(OfflineDbContextModel.Instance) + .UseSqlite("Data Source=Offline-Client.db"); + }, dbContextInitializer: async (sp, dbContext) => await dbContext.Database.MigrateAsync()); + ``` + + ### After - Bit.Bsqlite + + ```csharp + using Bit.Bsqlite.Storage.CacheStorage; + + services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider<BsqliteCacheStorageProvider>() + .UseStorage(s => s.Prefer(BsqliteCacheStorageProvider.ProviderName)) + .AddDatabase("Offline-Client.db")); // Besql's Data Source, exactly + + services.AddBitBsqliteDbContext<OfflineDbContext>(options => + { + // Migrate is the default schema step; Configure runs after UseBitBsqlite on every context's options. + options.Configure = (sp, builder) => builder.UseModel(OfflineDbContextModel.Instance); + }); + ``` + + ### In a component + + ```razor + @* IDbContextFactory<T> is still registered, next to IBsqliteDbContextFactory<T>. *@ + @inject IDbContextFactory<OfflineDbContext> Contexts + + @code { + private async Task SaveAsync(Todo todo) + { + await using var db = await Contexts.CreateDbContextAsync(); + db.Todos.Add(todo); + await db.SaveChangesAsync(); // SaveChanges() throws: the engine is in a worker + } + } + ``` + """; + + private const string TestingCode = """ + ## The code + + ### A test of the model, the migrations and the SQL + + ```csharp + using Bit.Bsqlite.Data; + using Bit.Bsqlite.Testing; + using Microsoft.EntityFrameworkCore; + + [TestClass] + public sealed class ProductTests + { + public TestContext TestContext { get; set; } = null!; + + [TestMethod] + public async Task A_saved_product_reads_back() + { + var ct = TestContext.CancellationToken; + + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "test.db"); + await connection.OpenAsync(ct); + + await using var db = new AppDbContext(new DbContextOptionsBuilder<AppDbContext>() + .UseBitBsqlite(connection) + .Options); + + await db.Database.MigrateAsync(ct); + + db.Products.Add(new Product { Name = "Widget", Price = 19.99m }); + await db.SaveChangesAsync(ct); + db.ChangeTracker.Clear(); + + Assert.AreEqual(19.99m, (await db.Products.SingleAsync(ct)).Price); + Assert.IsTrue(transport.ExecutedCommands.Any(sql => sql.Contains("INSERT INTO", StringComparison.Ordinal))); + } + } + ``` + + ### The whole composition, through dependency injection + + ```csharp + using Bit.Bsqlite.Data; + using Bit.Bsqlite.Interop; + using Bit.Bsqlite.Storage; + using Bit.Bsqlite.Storage.InMemory; + using Bit.Bsqlite.Testing; + + var services = new ServiceCollection(); + + services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider<BsqliteInMemoryStorageProvider>() + .UseStorage(s => s.Prefer(BsqliteInMemoryStorageProvider.ProviderName)) + .UseBindingStore<BsqliteInMemoryStorageBindingStore>() + .AddDatabase("app.db")); + + // Desktop has no worker; the in-process engine stands in for it. + services.AddSingleton<IBsqliteTransportFactory>(new InProcessTransports()); + + services.AddBitBsqliteDbContext<AppDbContext>(); + + sealed class InProcessTransports : IBsqliteTransportFactory + { + public IBsqliteTransport Create(IBsqliteStorageProvider provider) => new BsqliteInProcessTransport(); + } + ``` + + ### A storage backend's conformance tests + + ```csharp + using Bit.Bsqlite.Storage; + using Bit.Bsqlite.Storage.ConformanceTests; + + [TestClass] + public sealed class MyBackendConformanceTests : BsqliteStorageProviderConformanceTests + { + protected override IBsqliteStorageProvider CreateProvider() => new MyStorageProvider(); + } + ``` + """; + + private const string ProviderCode = """ + ## The code + + ### The .NET half + + ```csharp + using Bit.Bsqlite.Storage; + using Microsoft.JSInterop; + + public sealed class MyStorageProvider(IJSRuntime? js = null) : IBsqliteStorageProvider + { + // Recorded as the sticky binding of every database this backend creates: never rename it. + public const string ProviderName = "mycompany-encrypted"; + + public string Name => ProviderName; + + public BsqliteStorageCapabilities Capabilities { get; } = new() + { + RequiredBuild = BsqliteEngineBuild.AsyncCapable, // the VFS awaits its storage + IsPersistent = true, + SupportsMultipleConnections = true, + // Everything left out is the least capable answer - declare only what the VFS really does. + }; + + public BsqliteJsModule? VfsModule { get; } = new("./_content/MyCompany.Storage/my-vfs.js"); + + public IBsqliteStorageAdmin Admin { get; } = new MyStorageAdmin(js); + + public async ValueTask<BsqliteProbeResult> ProbeAsync(CancellationToken cancellationToken = default) + { + if (js is null) + { + return BsqliteProbeResult.Unavailable("This backend can only be probed in a browser."); + } + + var module = await js.InvokeAsync<IJSObjectReference>("import", cancellationToken, "./_content/MyCompany.Storage/my-admin.js"); + + try + { + return await module.InvokeAsync<bool>("probe", cancellationToken) + ? BsqliteProbeResult.Available() + : BsqliteProbeResult.Unavailable("The storage API this backend needs is missing in this browser."); + } + finally + { + await module.DisposeAsync(); + } + } + } + ``` + + `MyStorageAdmin` implements `IBsqliteStorageAdmin`, whose members are listed above. + + ### The JavaScript half - `wwwroot/my-vfs.js` + + ```javascript + // The base classes are wa-sqlite's, served by Bit.Bsqlite next to this package's own _content folder. + import { FacadeVFS } from '../Bit.Bsqlite/engine/FacadeVFS.js'; + import * as VFS from '../Bit.Bsqlite/engine/VFS.js'; + import { WebLocksMixin } from '../Bit.Bsqlite/engine/WebLocksMixin.js'; + + const VFS_NAME = 'mycompany-encrypted'; + + class MyVFS extends WebLocksMixin(FacadeVFS) { + // jOpen, jRead, jWrite, jTruncate, jSync, jFileSize, jClose, jDelete, jAccess ... + // returning VFS.SQLITE_OK or an error code, as the Cache Storage VFS does. + } + + // Called by the worker with the Emscripten module and the wa-sqlite API it just loaded. + export async function register({ module, sqlite3, databaseName }) { + const vfs = new MyVFS(VFS_NAME, module); + await vfs.isReady(); + sqlite3.vfs_register(vfs, /* makeDefault */ false); + return { vfsName: VFS_NAME }; // optionally onOpenSql: statements to run after every open + } + ``` + """; + + /// <summary>The first-party backends as a compact table, read off the provider types.</summary> + private static string StorageTable() + { + var builder = new StringBuilder(); + + builder.AppendLine("## The backends").AppendLine() + .AppendLine("| Name | Package | Engine build | Persistent | Concurrent reads | ATTACH | PRAGMA page_size |") + .AppendLine("| --- | --- | --- | --- | --- | --- | --- |"); + + foreach (var provider in BsqliteStorageCatalog.Providers) + { + string Value(string name) => provider.Capabilities.First(capability => capability.Name == name).Value; + + builder.AppendLine($"| `{provider.Name}` | `{provider.Package}` | {Value("RequiredBuild")} | {Value("IsPersistent")} | " + + $"{Value("SupportsConcurrentReads")} | {Allowed(provider, "ATTACH")} | {Allowed(provider, "page_size")} |"); + } + + return builder.ToString(); + } + + private static string Allowed(BsqliteStorageProviderDto provider, string statement) + => provider.Guarded.First(verdict => verdict.Name.Contains(statement, StringComparison.OrdinalIgnoreCase)).Accepted ? "allowed" : "refused"; + + /// <summary> + /// The members a provider implements, read off the two interfaces of the contract - so the list a + /// backend author works from cannot fall behind a member added to it. + /// </summary> + private static string ProviderContract() + { + var builder = new StringBuilder(); + + builder.AppendLine("## The contract").AppendLine(); + + foreach (var name in new[] { "IBsqliteStorageProvider", "IBsqliteStorageAdmin" }) + { + var details = BsqliteApiCatalog.GetTypeDetails(name); + if (details is null) continue; + + builder.AppendLine($"`{details.FullName}`:").AppendLine(); + + foreach (var member in details.Members) + { + builder.Append($"- `{member.Type} {member.Name}{member.Signature}`"); + if (BsqliteApiCatalog.FirstSentence(member.Summary) is { } summary) builder.Append($" - {summary}"); + builder.AppendLine(); + } + + builder.AppendLine(); + } + + return builder.ToString(); + } + + private const string NextHosting = """ + The rest is ordinary EF Core and ADO.NET - `GetBsqliteDocument(name: "getting-started")` covers contexts, raw + ADO.NET and the shared database, `GetBsqliteDocument(name: "live-queries")` queries that re-run themselves, + `GetBsqliteSetupGuide(scenario: "storage")` choosing backends, and `GetBsqliteSourceFile()` the demo's working pages. + """; + + /// <summary> + /// Pushes a quoted section one heading level down, so it nests under "From the documentation" + /// instead of competing with the headings of this document. + /// </summary> + private static string Demote(string section) + { + var lines = section.Split('\n'); + var fenced = false; + + for (int i = 0; i < lines.Length; i++) + { + var trimmed = lines[i].TrimStart(); + + if (trimmed.StartsWith("```", StringComparison.Ordinal) || trimmed.StartsWith("~~~", StringComparison.Ordinal)) + { + fenced = fenced is false; + continue; + } + + if (fenced is false && lines[i].StartsWith('#')) lines[i] = $"#{lines[i]}"; + } + + return string.Join('\n', lines); + } + + private static void AppendFile(StringBuilder builder, string path, string content) + { + content = content.TrimStart('').TrimEnd(); + var fence = Fence(content); + + builder.AppendLine($"### `{path}`").AppendLine() + .AppendLine($"{fence}{Language(path)}") + .AppendLine(content) + .AppendLine(fence) + .AppendLine(); + } + + /// <summary> + /// The fence for a file's code block. It has to outrun the longest run of backticks in the file + /// itself - a file that shows Markdown would otherwise end the block in the middle of its own + /// content, and the rest of the file would render as prose. + /// </summary> + private static string Fence(string content) + { + var longest = BacktickRunRegex().Matches(content).Select(match => match.Length).DefaultIfEmpty(0).Max(); + + return new string('`', Math.Max(3, longest + 1)); + } + + private static string Language(string path) => Path.GetExtension(path).ToLowerInvariant() switch + { + ".razor" => "razor", + ".cs" => "csharp", + ".csproj" => "xml", + ".css" => "css", + ".js" => "javascript", + _ => string.Empty + }; + + [GeneratedRegex("`+")] + private static partial Regex BacktickRunRegex(); +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteSourceCatalog.cs b/src/Bsqlite/Demo/Server/Services/BsqliteSourceCatalog.cs new file mode 100644 index 00000000000..9b31dee996e --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteSourceCatalog.cs @@ -0,0 +1,672 @@ +using System.Text; +using System.Reflection; +using System.Collections.Frozen; +using Bit.Bsqlite.Demo.Server.Dtos; +using System.Text.RegularExpressions; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// Serves the hand-written material the MCP tools hand out: the library's README, the guides under +/// docs/, every package's own README, and every source file of this demo - the pages run real queries +/// against a database in the browser, the model is the one both sides share, and the hosting files +/// are the wiring of a real Blazor Web App. +/// <para> +/// All of them are embedded into this assembly by the .csproj rather than read from disk, so the MCP +/// server keeps working from a published, single-folder deployment where the repository is nowhere to +/// be found. +/// </para> +/// </summary> +public static partial class BsqliteSourceCatalog +{ + private const string DocsPrefix = "BsqliteDocs/"; + private const string SourcePrefix = "BsqliteSource/"; + private const string GuidePrefix = "docs/"; + private const string PackagePrefix = "packages/"; + + /// <summary>The name the repository README goes by.</summary> + public const string ReadmeName = "readme"; + + /// <summary>Separates a parent heading from a child one in the key of a heading a document uses twice.</summary> + public const string KeySeparator = " > "; + + private sealed record Section(BsqliteDocumentSectionDto Dto, int Start, int End); + + private sealed record Document(BsqliteDocumentDto Dto, string Text, string[] Lines, Section[] Sections); + + private static readonly Assembly _assembly = typeof(BsqliteSourceCatalog).Assembly; + + private static readonly Lazy<Document[]> _documents = new(BuildDocuments); + private static readonly Lazy<BsqliteDocumentDto[]> _documentList = new(() => [.. _documents.Value.Select(document => document.Dto)]); + private static readonly Lazy<BsqliteDocumentSectionDto[]> _sections = new(() => [.. _documents.Value.SelectMany(document => document.Sections.Select(section => section.Dto))]); + private static readonly Lazy<FrozenDictionary<string, string>> _sourceFiles = new(BuildSourceFiles); + private static readonly Lazy<BsqliteSourceFileDto[]> _sourceFileList = new(BuildSourceFileList); + + /// <summary>The repository README, in full.</summary> + public static string Readme => GetDocument(ReadmeName) ?? string.Empty; + + /// <summary>Every document, in reading order: the README, the guides as it links them, then the packages.</summary> + public static BsqliteDocumentDto[] Documents => _documentList.Value; + + /// <summary>Every level 2, 3 and 4 heading of every document, in reading order.</summary> + public static BsqliteDocumentSectionDto[] Sections => _sections.Value; + + /// <summary>Every embedded source file, keyed by the path the tools expose.</summary> + public static BsqliteSourceFileDto[] SourceFiles => _sourceFileList.Value; + + /// <summary> + /// The documents and their sections, as the index GetBsqliteDocument answers with when it is called + /// without a name. Plain text rather than a list of objects: an index is read, not parsed, and the + /// same list as JSON costs twice the characters to say the same thing. + /// </summary> + public static string RenderDocumentIndex() + { + var builder = new StringBuilder(); + + builder.AppendLine("# Bit.Bsqlite documents").AppendLine(); + builder.AppendLine("The README, the guides under docs/ and every package's own README. Call ") + .AppendLine("`GetBsqliteDocument(name: \"...\")` for one of them whole, or add `section: \"...\"` for one of its sections; ") + .AppendLine("a section's line count is there so a heading that runs to a hundred lines is a deliberate choice rather than a surprise.") + .AppendLine(); + + foreach (var group in _documents.Value.GroupBy(document => document.Dto.Kind)) + { + builder.AppendLine($"## {KindTitle(group.Key)}").AppendLine(); + + foreach (var document in group) + { + builder.Append($"- **{document.Dto.Name}** - {document.Dto.Title} ({document.Dto.Lines} lines)"); + if (document.Dto.Summary is not null) builder.Append($" - {document.Dto.Summary}"); + builder.AppendLine(); + + foreach (var section in document.Sections) + { + var indent = new string(' ', (section.Dto.Level - 1) * 2); + var key = section.Dto.Key == section.Dto.Heading ? null : $" - key `{section.Dto.Key}`"; + + builder.AppendLine($"{indent}- {section.Dto.Heading} ({section.Dto.Lines} lines){key}"); + } + } + + builder.AppendLine(); + } + + return builder.ToString(); + } + + /// <summary> + /// The files GetBsqliteSourceFile can hand out, as the index it answers with when it is called + /// without a path. + /// </summary> + public static string RenderSourceIndex() + { + var builder = new StringBuilder(); + + builder.AppendLine("# Bit.Bsqlite source files").AppendLine(); + builder.AppendLine("Working code this server can hand out verbatim. Call `GetBsqliteSourceFile(path: \"...\")` for one.").AppendLine(); + + foreach (var group in SourceFiles.GroupBy(file => file.Kind)) + { + builder.AppendLine($"## {group.Key}").AppendLine(); + + foreach (var file in group) + { + builder.Append($"- `{file.Path}` ({file.Lines} lines)"); + if (file.Description is not null) builder.Append($" - {file.Description}"); + builder.AppendLine(); + } + + builder.AppendLine(); + } + + return builder.ToString(); + } + + /// <summary> + /// The document a caller means, or null. Case and punctuation are ignored, an extension or a docs/ + /// prefix is dropped, and a package README can be named without the package's common prefix: + /// "storage.md", "README", "Testing" and "opfs" all find their document. + /// </summary> + public static string? ResolveDocumentName(string? name) + { + if (string.IsNullOrWhiteSpace(name)) return null; + + var text = name.Trim().Replace('\\', '/'); + + if (text.EndsWith(".md", StringComparison.OrdinalIgnoreCase)) text = text[..^3]; + + var slash = text.LastIndexOf('/'); + if (slash >= 0) text = text[(slash + 1)..]; + + var wanted = Squash(text); + if (wanted.Length == 0) return null; + + var documents = _documents.Value; + + return (documents.FirstOrDefault(document => Squash(document.Dto.Name) == wanted) + ?? documents.FirstOrDefault(document => document.Dto.Kind == "Package" && Squash(document.Dto.Name) == $"bitbsqlite{wanted}") + ?? documents.FirstOrDefault(document => document.Dto.Kind == "Package" && Squash(document.Dto.Name) == $"bitbsqlitestorage{wanted}") + ?? documents.FirstOrDefault(document => Squash(document.Dto.Title) == wanted))?.Dto.Name; + } + + /// <summary>One document in full, or null when no document goes by that name.</summary> + public static string? GetDocument(string? name) => FindDocument(name)?.Text; + + /// <summary>The headings of one document, or an empty list when it does not exist.</summary> + public static BsqliteDocumentSectionDto[] GetSections(string? name) + => FindDocument(name) is { } document ? [.. document.Sections.Select(section => section.Dto)] : []; + + /// <summary> + /// The text under a heading of one document, including its sub-sections. Matching is case- and + /// punctuation-insensitive so "first party backends" finds "First-party backends"; a heading the + /// document uses more than once is told apart by its key, "Parent > Heading". + /// </summary> + public static string? GetDocumentSection(string? name, string? heading) + { + var document = FindDocument(name); + if (document is null) return null; + + var section = FindSection(document, heading); + if (section is null) return null; + + return string.Join('\n', document.Lines[section.Start..section.End]).TrimEnd(); + } + + /// <summary> + /// The text of a document above its first section - its title, its opening prose, the code it leads + /// with - or all of it when it has no sections. Null when the document does not exist. + /// </summary> + public static string? GetPreamble(string? name) + { + var document = FindDocument(name); + if (document is null) return null; + + var end = document.Sections.Length == 0 ? document.Lines.Length : document.Sections[0].Start; + + return string.Join('\n', document.Lines[..end]).Trim(); + } + + /// <summary>The content of an embedded source file, or null when the path is unknown.</summary> + public static string? GetSourceFile(string? path) + { + if (string.IsNullOrWhiteSpace(path)) return null; + + return _sourceFiles.Value.GetValueOrDefault(Normalize(path)); + } + + private static Document? FindDocument(string? name) + { + var resolved = ResolveDocumentName(name); + + return resolved is null ? null : _documents.Value.First(document => document.Dto.Name == resolved); + } + + private static Section? FindSection(Document document, string? heading) + { + if (string.IsNullOrWhiteSpace(heading)) return null; + + var sections = document.Sections; + + var separator = heading.LastIndexOf('>'); + if (separator > 0) + { + var parent = NormalizeHeading(heading[..separator]); + var child = NormalizeHeading(heading[(separator + 1)..]); + + var qualified = sections.FirstOrDefault(section => section.Dto.Parent is not null + && NormalizeHeading(section.Dto.Parent) == parent + && NormalizeHeading(section.Dto.Heading) == child); + if (qualified is not null) return qualified; + } + + var normalized = NormalizeHeading(heading); + + return sections.FirstOrDefault(section => NormalizeHeading(section.Dto.Heading) == normalized) + ?? sections.FirstOrDefault(section => NormalizeHeading(section.Dto.Key) == normalized); + } + + private static string? ReadResource(string name) + { + using var stream = _assembly.GetManifestResourceStream(name); + if (stream is null) return null; + + using var reader = new StreamReader(stream, Encoding.UTF8, detectEncodingFromByteOrderMarks: true); + + return reader.ReadToEnd(); + } + + private static Document[] BuildDocuments() + { + var documents = new List<Document>(); + + foreach (var resource in _assembly.GetManifestResourceNames()) + { + var normalized = Normalize(resource); + + if (normalized.StartsWith(DocsPrefix, StringComparison.OrdinalIgnoreCase) is false) continue; + + var text = ReadResource(resource); + if (text is null) continue; + + var relative = normalized[DocsPrefix.Length..]; + var stem = Path.GetFileNameWithoutExtension(relative); + + var (name, kind) = relative.StartsWith(GuidePrefix, StringComparison.OrdinalIgnoreCase) ? (stem, "Guide") + : relative.StartsWith(PackagePrefix, StringComparison.OrdinalIgnoreCase) ? (stem, "Package") + : (ReadmeName, "Readme"); + + documents.Add(BuildDocument(name, kind, text)); + } + + // The README's own list of guides is their reading order - getting started before storage before + // the limits - and the packages follow the order the API reference lists them in. + var readme = documents.FirstOrDefault(document => document.Dto.Kind == "Readme")?.Text ?? string.Empty; + + return [.. documents + .OrderBy(document => document.Dto.Kind switch { "Readme" => 0, "Guide" => 1, _ => 2 }) + .ThenBy(document => document.Dto.Kind switch + { + "Guide" => GuideOrder(readme, document.Dto.Name), + "Package" => PackageOrder(document.Dto.Name), + _ => 0 + }) + .ThenBy(document => document.Dto.Name, StringComparer.OrdinalIgnoreCase)]; + } + + private static int GuideOrder(string readme, string name) + { + var index = readme.IndexOf($"docs/{name}.md", StringComparison.OrdinalIgnoreCase); + + return index < 0 ? int.MaxValue : index; + } + + private static int PackageOrder(string name) + { + var index = Array.FindIndex(BsqliteApiCatalog.Assemblies, assembly => string.Equals(assembly.GetName().Name, name, StringComparison.OrdinalIgnoreCase)); + + return index < 0 ? int.MaxValue : index; + } + + private static Document BuildDocument(string name, string kind, string text) + { + text = text.TrimStart(''); + + var lines = text.Replace("\r\n", "\n", StringComparison.Ordinal).Split('\n'); + var sections = BuildSections(name, lines); + + var title = lines.Select(line => TryReadHeading(line, out var level, out var heading) && level == 1 ? heading : null) + .FirstOrDefault(heading => heading is not null) ?? name; + + return new Document(new BsqliteDocumentDto + { + Name = name, + Title = title, + Kind = kind, + Summary = Summary(lines), + Lines = CountLines(text), + Sections = [.. sections.Select(section => section.Dto.Key)] + }, text, lines, sections); + } + + /// <summary> + /// The first sentence of the document's opening paragraph - the prose under its title, skipping the + /// code, quotes and tables some documents open with. + /// </summary> + private static string? Summary(string[] lines) + { + var paragraph = new List<string>(); + var fenced = false; + var titled = false; + + foreach (var line in lines) + { + if (IsCodeFence(line)) + { + fenced = fenced is false; + if (paragraph.Count > 0) break; + continue; + } + + if (fenced) continue; + + // A guide that goes straight into its first section is summarised by that section's opening + // paragraph: what comes first is what the document is about. + if (TryReadHeading(line, out _, out _)) + { + if (paragraph.Count > 0) break; + + titled = true; + continue; + } + + var trimmed = line.Trim(); + + if (trimmed.Length == 0) + { + if (paragraph.Count > 0) break; + continue; + } + + if (titled is false || trimmed.StartsWith('>') || trimmed.StartsWith('|') || trimmed.StartsWith('-')) continue; + + paragraph.Add(trimmed); + } + + if (paragraph.Count == 0) return null; + + var text = MarkdownLinkRegex().Replace(string.Join(' ', paragraph), "${text}"); + + return Summarize(text); + } + + private static Section[] BuildSections(string document, string[] lines) + { + var headings = new List<(int Index, int Level, string Text)>(); + + var fenced = false; + for (int i = 0; i < lines.Length; i++) + { + if (IsCodeFence(lines[i])) + { + fenced = fenced is false; + continue; + } + + if (fenced) continue; + + if (TryReadHeading(lines[i], out var level, out var text) && level is >= 2 and <= 4) + { + headings.Add((i, level, text)); + } + } + + var duplicated = headings.GroupBy(heading => NormalizeHeading(heading.Text)) + .Where(group => group.Count() > 1) + .Select(group => group.Key) + .ToHashSet(StringComparer.Ordinal); + + var sections = new List<Section>(headings.Count); + var parents = new string?[5]; + + for (int i = 0; i < headings.Count; i++) + { + var (index, level, text) = headings[i]; + + parents[level] = text; + for (int deeper = level + 1; deeper < parents.Length; deeper++) parents[deeper] = null; + + string? parent = null; + for (int upper = level - 1; upper >= 2 && parent is null; upper--) parent = parents[upper]; + + // The section runs until the next heading of the same or a higher rank. + var end = lines.Length; + for (int j = i + 1; j < headings.Count; j++) + { + if (headings[j].Level > level) continue; + end = headings[j].Index; + break; + } + + sections.Add(new Section(new BsqliteDocumentSectionDto + { + Document = document, + Heading = text, + Key = parent is not null && duplicated.Contains(NormalizeHeading(text)) ? $"{parent}{KeySeparator}{text}" : text, + Level = level, + Parent = parent, + Lines = end - index + }, index, end)); + } + + return [.. sections]; + } + + private static FrozenDictionary<string, string> BuildSourceFiles() + { + var files = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase); + + foreach (var resource in _assembly.GetManifestResourceNames()) + { + var normalized = Normalize(resource); + + if (normalized.StartsWith(SourcePrefix, StringComparison.OrdinalIgnoreCase) is false) continue; + + var content = ReadResource(resource); + if (content is null) continue; + + files[normalized[SourcePrefix.Length..]] = content; + } + + return files.ToFrozenDictionary(StringComparer.OrdinalIgnoreCase); + } + + private static BsqliteSourceFileDto[] BuildSourceFileList() + { + return [.. _sourceFiles.Value + .Select(file => new BsqliteSourceFileDto + { + Path = file.Key, + Kind = KindOf(file.Key), + Description = DescribeSource(file.Key, file.Value), + Lines = CountLines(file.Value) + }) + .OrderBy(file => KindOrder(file.Kind)) + .ThenBy(file => file.Path, StringComparer.OrdinalIgnoreCase)]; + } + + /// <summary> + /// What a file is for: a demo page is a working example to copy, the hosting files are the wiring + /// every app needs once, the model is the EF Core model both sides share, and the rest is the + /// components and helpers the pages share. + /// </summary> + private static string KindOf(string path) + { + if (path.StartsWith("Demo/Client/Pages/", StringComparison.OrdinalIgnoreCase)) return "Page"; + if (path.StartsWith("Demo/Data/", StringComparison.OrdinalIgnoreCase) || + path.StartsWith("Demo/Domain/", StringComparison.OrdinalIgnoreCase)) return "Model"; + if (path.StartsWith("Demo/Server/", StringComparison.OrdinalIgnoreCase) || + path.StartsWith("Demo/Client/Extensions/", StringComparison.OrdinalIgnoreCase) || + path.StartsWith("Demo/Client/Layout/", StringComparison.OrdinalIgnoreCase) || + path.Equals("Demo/Client/Program.cs", StringComparison.OrdinalIgnoreCase) || + path.Equals("Demo/Client/Routes.razor", StringComparison.OrdinalIgnoreCase) || + path.Equals("Demo/Client/_Imports.razor", StringComparison.OrdinalIgnoreCase)) return "Hosting"; + + return "Shared"; + } + + private static int KindOrder(string kind) => kind switch + { + "Page" => 0, + "Shared" => 1, + "Model" => 2, + "Hosting" => 3, + _ => 4 + }; + + private static string KindTitle(string kind) => kind switch + { + "Readme" => "The README", + "Guide" => "Guides", + "Package" => "Package READMEs", + _ => kind + }; + + /// <summary> + /// The number of lines in a file. A trailing newline ends the last line rather than starting + /// another one, so a file that has one is not reported a line longer than it is. + /// </summary> + private static int CountLines(string content) + { + if (content.Length == 0) return 0; + + var newlines = content.Count(c => c == '\n'); + + return content[^1] == '\n' ? newlines : newlines + 1; + } + + /// <summary> + /// A one-line description of a source file, taken from whatever the file itself already says: its + /// leading razor/C# comment, its page header, the summary of the type it declares, or - for a page - + /// its <PageTitle>. + /// </summary> + private static string? DescribeSource(string path, string content) + { + // A byte order mark is not text, and it would stop every pattern anchored at the start of the file. + content = content.TrimStart(''); + + var leadingComment = LeadingRazorCommentRegex().Match(content); + if (leadingComment.Success) return Summarize(leadingComment.Groups["text"].Value); + + // A demo page opens with a PageHeader that says what it demonstrates, to the person reading it - + // which is the best description of it there is, far better than the first comment in its code. + var header = PageHeaderRegex().Match(content); + if (header.Success) + { + var intro = Summarize(XmlTagRegex().Replace(header.Groups["text"].Value, string.Empty)); + + return intro is null ? header.Groups["title"].Value : $"{header.Groups["title"].Value}: {intro}"; + } + + // A comment the file opens with describes the file; an XML summary further down describes one + // type inside it (Program.cs ends with one about the partial Program class). + var lineComment = LeadingLineCommentRegex().Match(content); + if (lineComment.Success) return Summarize(lineComment.Value.Replace("//", " ", StringComparison.Ordinal)); + + if (path.EndsWith(".razor", StringComparison.OrdinalIgnoreCase)) + { + // In a component the XML summaries document its parameters, not the component: the comment that + // follows its directives, or its title, says what it is. + var comment = RazorCommentRegex().Match(content); + if (comment.Success) return Summarize(comment.Groups["text"].Value); + + var title = PageTitleRegex().Match(content); + if (title.Success) return Summarize(title.Groups["text"].Value); + } + else + { + // The summary of the type the file declares, not the first one in it: a file that opens with an + // enum's values would otherwise be described by the first of them. + var summary = TypeSummaryRegex().Match(content); + if (summary.Success) return Summarize(Unmark(summary.Groups["text"].Value)); + } + + // Nothing said what the file is, but the first thing its code explains usually does. + var codeComment = LineCommentRegex().Match(content); + if (codeComment.Success) return Summarize(codeComment.Value.Replace("//", " ", StringComparison.Ordinal)); + + // A model file says nothing about itself, but its declaration is a summary of it all the same. + var declaration = TypeDeclarationRegex().Match(content); + + return declaration.Success ? Summarize($"Declares `{declaration.Groups["text"].Value.Trim()}`.") : null; + } + + /// <summary> + /// An XML summary as prose: a <c><see cref="..."/></c> becomes the name it refers to, and every + /// other tag goes - they are markup, and noise in a listing. + /// </summary> + private static string Unmark(string summary) + { + var text = summary.Replace("///", " ", StringComparison.Ordinal); + + text = SeeCrefRegex().Replace(text, match => match.Groups["name"].Value.Split('.')[^1]); + + return XmlTagRegex().Replace(text, string.Empty); + } + + private static string? Summarize(string text) + { + text = WhitespaceRegex().Replace(text, " ").Trim(); + if (text.Length == 0) return null; + + // The first sentence is the description; the rest is the file's own commentary. + var stop = text.IndexOf(". ", StringComparison.Ordinal); + if (stop > 0) text = text[..(stop + 1)]; + + return text.Length <= 220 ? text : $"{text[..217]}..."; + } + + /// <summary> + /// A Markdown code fence. A '#' line inside one is a shell comment or a C# preprocessor directive, + /// not a heading, and must not start or end a section. + /// </summary> + private static bool IsCodeFence(string line) + { + var text = line.TrimStart(); + + return text.StartsWith("```", StringComparison.Ordinal) || text.StartsWith("~~~", StringComparison.Ordinal); + } + + private static bool TryReadHeading(string line, out int level, out string text) + { + level = 0; + text = string.Empty; + + while (level < line.Length && line[level] == '#') level++; + + if (level is 0 or > 6 || level >= line.Length || line[level] != ' ') return false; + + text = line[(level + 1)..].Trim(); + + return text.Length > 0; + } + + /// <summary>Reduces a heading to its comparable core, so "first party backends" finds "## First-party backends".</summary> + private static string NormalizeHeading(string heading) + { + var builder = new StringBuilder(heading.Length); + + // The little words carry nothing a heading is told apart by, and they are the ones a caller most + // often drops ("moving off besql") or adds ("the live queries"). + foreach (var word in heading.Split(heading.Where(c => char.IsLetterOrDigit(c) is false).Distinct().ToArray(), StringSplitOptions.RemoveEmptyEntries)) + { + if (_fillerWords.Contains(word)) continue; + + foreach (var c in word) builder.Append(char.ToLowerInvariant(c)); + } + + return builder.ToString(); + } + + private static readonly HashSet<string> _fillerWords = new(StringComparer.OrdinalIgnoreCase) { "a", "an", "and", "the", "of" }; + + /// <summary>A name with nothing but its letters and digits, lower-cased: "Bit.Bsqlite.Testing" is "bitbsqlitetesting".</summary> + private static string Squash(string text) => new([.. text.Where(char.IsLetterOrDigit).Select(char.ToLowerInvariant)]); + + private static string Normalize(string path) => path.Replace('\\', '/').Trim('/'); + + [GeneratedRegex(@"^\s*@\*(?<text>.*?)\*@", RegexOptions.Singleline)] + private static partial Regex LeadingRazorCommentRegex(); + + [GeneratedRegex(@"@\*(?<text>.*?)\*@", RegexOptions.Singleline)] + private static partial Regex RazorCommentRegex(); + + [GeneratedRegex(@"<[^>]+>")] + private static partial Regex XmlTagRegex(); + + [GeneratedRegex(@"///\s*<summary>(?<text>(?:(?!</summary>).)*)</summary>[^\n]*\n(?:[ \t]*(?:///|\[)[^\n]*\n)*[ \t]*(?:public|internal)\b[^\n{]*\b(?:class|record|struct|enum|interface)\b", RegexOptions.Singleline)] + private static partial Regex TypeSummaryRegex(); + + [GeneratedRegex(@"<see\s+cref=""(?:[A-Z]:)?(?<name>[^""]+)""\s*/>")] + private static partial Regex SeeCrefRegex(); + + [GeneratedRegex(@"(?:^[ \t]*//(?!/)[^\n]*\n)+", RegexOptions.Multiline)] + private static partial Regex LineCommentRegex(); + + [GeneratedRegex(@"<PageHeader\b[^>]*\bTitle=""(?<title>[^""]*)""[^>]*>(?<text>.*?)</PageHeader>", RegexOptions.Singleline)] + private static partial Regex PageHeaderRegex(); + + [GeneratedRegex(@"<PageTitle>(?<text>.*?)</PageTitle>", RegexOptions.Singleline | RegexOptions.IgnoreCase)] + private static partial Regex PageTitleRegex(); + + [GeneratedRegex(@"^\s*(//[^\n]*\n)+")] + private static partial Regex LeadingLineCommentRegex(); + + [GeneratedRegex(@"^(?:public|internal)\s+(?<text>[^\n{]*\b(?:class|record|struct|enum|interface)\b[^\n{]*)", RegexOptions.Multiline)] + private static partial Regex TypeDeclarationRegex(); + + [GeneratedRegex(@"\[(?<text>[^\]]+)\]\([^)]+\)")] + private static partial Regex MarkdownLinkRegex(); + + [GeneratedRegex(@"\s+")] + private static partial Regex WhitespaceRegex(); +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteSqlCheck.cs b/src/Bsqlite/Demo/Server/Services/BsqliteSqlCheck.cs new file mode 100644 index 00000000000..b3712c3a87c --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteSqlCheck.cs @@ -0,0 +1,99 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Demo.Server.Dtos; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// Puts SQL to the library's own static rules - the feature guards that refuse what a backend cannot +/// honour, and the table scan live queries are keyed by - so "will this statement run on IndexedDB, and +/// which tables will a live query over it watch?" is answered by the code that decides it, before the +/// statement ever reaches a browser. +/// <para> +/// Nothing is executed. <see cref="BsqliteFeatureGuards"/> and <see cref="BsqliteTableNames"/> are +/// pattern matches over the text, public and static, and they are the very calls the command layer +/// makes before a statement leaves for the worker; there is no engine on this server for the SQL to +/// reach, and none is opened for it. +/// </para> +/// </summary> +public static class BsqliteSqlCheck +{ + /// <summary> + /// The longest SQL checked. A statement or a short script is what a question is about; and the + /// patterns are run over the whole text once per backend, so an unbounded input is unbounded work. + /// </summary> + public const int MaxSqlLength = 10_000; + + public static BsqliteSqlCheckDto Check(string? sql) + { + if (string.IsNullOrWhiteSpace(sql)) + { + return new BsqliteSqlCheckDto + { + Statements = [.. BsqliteStorageCatalog.GuardedStatements.Select(Report)], + Notes = + [ + "No SQL was passed, so these are the statements the guards exist for, judged on every first-party backend. " + + "Pass sql to check your own - a migration script, a PRAGMA, the text a LINQ query's ToQueryString() returns.", + .. GuardNotes + ] + }; + } + + if (sql.Length > MaxSqlLength) + { + // A partial check would read as a clean bill of health for the part that was not checked. + return new BsqliteSqlCheckDto + { + Statements = [], + Error = $"The SQL is {sql.Length} characters long; this tool checks up to {MaxSqlLength}. Check one statement or one migration at a time.", + Notes = [] + }; + } + + var report = Report(sql); + var notes = new List<string>(); + + if (report.Backends.All(verdict => verdict.Accepted)) + { + notes.Add("Every backend's guards let this through. The guards cover WAL, ATTACH and PRAGMA page_size only: whether the SQL itself is valid is the engine's to say, in the browser."); + } + else + { + notes.AddRange(GuardNotes); + } + + notes.Add(report.Tables.Length == 0 + ? "No table names were found, so a live query over this SQL would have nothing to watch - BsqliteLiveQuery treats an empty table list as \"every write\"." + : $"A live query over this SQL re-runs when {string.Join(", ", report.Tables.Select(table => $"'{table}'"))} is written - by this tab or another one. AsLiveQuery() reads the tables the same way, out of ToQueryString()."); + + if (report.LooksLikeWrite) + { + notes.Add("It writes. The worker reports what a write really changed from SQLite's update hook, triggers and cascades included, and only once the transaction commits; the table scan above is the fallback for a transport that cannot."); + } + + if (report.LooksLikeSchemaChange) + { + notes.Add("It contains DDL, which the update hook does not see: the statement text is what notifies live queries of it, and it clears the worker's prepared-statement cache."); + } + + return new BsqliteSqlCheckDto { Statements = [report], Notes = [.. notes] }; + } + + private static readonly string[] GuardNotes = + [ + "WAL is refused everywhere - no web VFS has the shared memory it needs; journaling is DELETE or TRUNCATE.", + "ATTACH is refused where the backend declares SupportsMultiDatabaseTransactions = false: open each database on its own session instead.", + "PRAGMA page_size is refused where the backend declares CanChangePageSize = false: its page size is fixed to its block size.", + "The command layer refuses before the statement leaves for the worker, and the worker refuses again, so a JavaScript caller cannot bypass the capability declaration either. OnOpen statements run outside the guards, and still cannot enable WAL.", + ]; + + private static BsqliteSqlReportDto Report(string sql) => new() + { + Sql = sql, + Backends = [.. BsqliteStorageCatalog.Providers.Select(provider => + BsqliteStorageCatalog.Judge(provider.Name, sql, BsqliteStorageCatalog.LimitsOf(BsqliteStorageCatalog.Instance(provider.Name)!.Capabilities)))], + Tables = [.. BsqliteTableNames.Extract(sql).Order(StringComparer.OrdinalIgnoreCase)], + LooksLikeWrite = BsqliteTableNames.LooksLikeWrite(sql), + LooksLikeSchemaChange = BsqliteTableNames.LooksLikeSchemaChange(sql) + }; +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteStorageCatalog.cs b/src/Bsqlite/Demo/Server/Services/BsqliteStorageCatalog.cs new file mode 100644 index 00000000000..5340bdc1ccd --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteStorageCatalog.cs @@ -0,0 +1,261 @@ +using System.Text; +using System.Reflection; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.Opfs; +using Bit.Bsqlite.Demo.Server.Dtos; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Storage.IndexedDb; +using Bit.Bsqlite.Storage.CacheStorage; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// The four first-party storage backends, read out of the library itself: every name, module URL and +/// capability comes from an instance of the provider type, every capability's meaning from the XML +/// documentation of the property that declares it, and every refused statement from running the +/// library's own guards under the limits the session factory derives from those capabilities. +/// <para> +/// Nothing here is a table someone keeps up to date. The capability rows are the properties of +/// <see cref="BsqliteStorageCapabilities"/>, so a capability added to the contract shows up the moment +/// it is declared; and a backend whose declaration changes is reported as it now is - which is the +/// point: "OPFS cannot ATTACH, IndexedDB serializes readers" is exactly the kind of fact an agent half +/// remembers from another version. +/// </para> +/// <para> +/// The providers are built without an <c>IJSRuntime</c>, which every first-party constructor +/// accepts: there is no browser here, and nothing on this server ever probes one. Name, capabilities +/// and modules are declarations, not discoveries, so they do not need one. +/// </para> +/// </summary> +public static class BsqliteStorageCatalog +{ + /// <summary> + /// The statements the guards exist for, one per rule of <see cref="BsqliteFeatureGuards"/>: the + /// product-wide WAL ban, and the two capability-gated ones. Every verdict about them is computed. + /// </summary> + public static readonly string[] GuardedStatements = + [ + "PRAGMA journal_mode = WAL", + "ATTACH DATABASE 'other.db' AS other", + "PRAGMA page_size = 8192", + ]; + + private static readonly Lazy<IBsqliteStorageProvider[]> _instances = new(() => + [ + new BsqliteOpfsStorageProvider(), + new BsqliteIndexedDbStorageProvider(), + new BsqliteCacheStorageProvider(), + new BsqliteInMemoryStorageProvider(), + ]); + + private static readonly Lazy<BsqliteStorageProviderDto[]> _providers = new(() => [.. _instances.Value.Select(Describe)]); + + /// <summary>The four backends, in the order the README recommends them: fastest first, volatile last.</summary> + public static BsqliteStorageProviderDto[] Providers => _providers.Value; + + /// <summary> + /// A first-party provider instance by its name, for the storage lab to borrow the real + /// capabilities and modules from. Null for a name no first-party backend uses. + /// </summary> + public static IBsqliteStorageProvider? Instance(string? name) + => _instances.Value.FirstOrDefault(provider => string.Equals(provider.Name, name?.Trim(), StringComparison.OrdinalIgnoreCase)); + + /// <summary>One backend by its name ("opfs"), its type name ("BsqliteOpfsStorageProvider") or its package, or null.</summary> + public static BsqliteStorageProviderDto? Find(string? key) + { + if (string.IsNullOrWhiteSpace(key)) return null; + + var text = key.Trim(); + var squashed = Squash(text); + + return Providers.FirstOrDefault(provider => string.Equals(provider.Name, text, StringComparison.OrdinalIgnoreCase)) + ?? Providers.FirstOrDefault(provider => string.Equals(provider.TypeName, text, StringComparison.OrdinalIgnoreCase)) + ?? Providers.FirstOrDefault(provider => string.Equals(provider.Package, text, StringComparison.OrdinalIgnoreCase)) + // "IndexedDB", "Cache Storage", "inmemory": the backend as a person spells it. + ?? Providers.FirstOrDefault(provider => Squash(provider.Name) == squashed) + ?? Providers.FirstOrDefault(provider => Squash(provider.TypeName) == $"bsqlite{squashed}storageprovider"); + } + + /// <summary>The backends as one Markdown table, as GetBsqliteStorageProviders answers when it is called without a name.</summary> + public static string RenderIndex() + { + var builder = new StringBuilder(); + var capabilities = CapabilityProperties(); + + builder.AppendLine("# Bit.Bsqlite storage backends").AppendLine(); + builder.AppendLine("The four first-party backends, each with the capabilities its provider type really declares - what selection, ") + .AppendLine("the SQL guards and the migrator act on. Call `GetBsqliteStorageProviders(provider: \"...\")` for one of them in full, ") + .AppendLine("with its package, its modules and what each capability means.") + .AppendLine(); + + builder.Append("| Name | Package |"); + foreach (var property in capabilities) builder.Append($" {property.Name} |"); + builder.AppendLine(); + + builder.Append("| --- | --- |"); + foreach (var _ in capabilities) builder.Append(" --- |"); + builder.AppendLine(); + + foreach (var provider in Providers) + { + builder.Append($"| `{provider.Name}` | `{provider.Package}` |"); + foreach (var capability in provider.Capabilities) builder.Append($" {capability.Value} |"); + builder.AppendLine(); + } + + builder.AppendLine() + .AppendLine("Refused by the guards (`BsqliteFeatureGuards`), per backend:") + .AppendLine(); + + foreach (var provider in Providers) + { + var refused = provider.Guarded.Where(verdict => verdict.Accepted is false).Select(verdict => $"`{verdict.Name}`").ToArray(); + + builder.AppendLine($"- `{provider.Name}`: {(refused.Length == 0 ? "nothing" : string.Join(", ", refused))}"); + } + + builder.AppendLine() + .AppendLine("Register a backend with `AddStorageProvider<T>()` and rank it with `UseStorage(s => s.Prefer(...).Fallback(...))`. ") + .AppendLine("Selection prefers that order, but a database already bound to a backend stays there; falling back to a backend ") + .AppendLine("that does not persist needs `AllowNonPersistentFallback()`. `PlanBsqliteStorageSelection` runs that selection on ") + .AppendLine("a browser you describe, and `GetBsqliteSetupGuide(scenario: \"storage\")` walks through choosing."); + + return builder.ToString(); + } + + /// <summary>The full description of one backend as Markdown, or null when no first-party backend goes by that name.</summary> + public static string? Render(string key) + { + var provider = Find(key); + if (provider is null) return null; + + var builder = new StringBuilder(); + + builder.AppendLine($"# {provider.TypeName} (`{provider.Name}`)").AppendLine(); + builder.AppendLine($"Package `{provider.Package}` - `using {provider.Namespace};`").AppendLine(); + + if (provider.Summary is not null) builder.AppendLine(provider.Summary).AppendLine(); + if (provider.Remarks is not null) builder.AppendLine(provider.Remarks).AppendLine(); + + builder.AppendLine("## Capabilities").AppendLine() + .AppendLine($"Read off `new {provider.TypeName}().Capabilities`. The core enforces every one of them generically - it never tests a provider's name.") + .AppendLine(); + + foreach (var capability in provider.Capabilities) + { + builder.Append($"- **{capability.Name}** = `{capability.Value}`"); + if (capability.Summary is not null) builder.Append($" - {capability.Summary}"); + builder.AppendLine(); + } + + builder.AppendLine().AppendLine("## What the guards refuse here").AppendLine() + .AppendLine("Each statement run through `BsqliteFeatureGuards.EnsureSupported` under the `BsqliteRuntimeLimits` a session on this backend gets:") + .AppendLine(); + + foreach (var verdict in provider.Guarded) + { + builder.AppendLine(verdict.Accepted + ? $"- `{verdict.Name}` - accepted" + : $"- `{verdict.Name}` - refused: {verdict.Error}"); + } + + builder.AppendLine().AppendLine("## Modules").AppendLine(); + + builder.AppendLine(provider.VfsModule is null + ? "- VFS: none - the storage is the engine's own memory, so there is no JavaScript to load." + : $"- VFS module: `{provider.VfsModule}`{(provider.VfsName is null ? null : $" (registers the `{provider.VfsName}` VFS)")}"); + + if (provider.AdminModule is not null) builder.AppendLine($"- Probe and admin module, imported on the main thread: `{provider.AdminModule}`"); + + builder.AppendLine() + .AppendLine("Both are static web assets of the package, named relative to the document so an app served under a sub-path still finds them; ") + .AppendLine("nothing needs adding to the host page.") + .AppendLine() + .AppendLine("## Registration").AppendLine() + .AppendLine("```csharp") + .AppendLine($"using {provider.Namespace};") + .AppendLine() + .AppendLine("builder.Services.AddBitBsqliteServices(sqlite => sqlite") + .AppendLine($" .AddStorageProvider<{provider.TypeName}>()") + .AppendLine($" .UseStorage(s => s.Prefer({provider.TypeName}.ProviderName))") + .AppendLine(" .AddDatabase(\"app.db\"));") + .AppendLine("```"); + + var readme = BsqliteSourceCatalog.ResolveDocumentName(provider.Package); + if (readme is not null) builder.AppendLine().AppendLine($"The package's own notes: `GetBsqliteDocument(name: \"{readme}\")`."); + + return builder.ToString(); + } + + /// <summary> + /// The limits a session on a backend runs under - built exactly as the session factory builds them + /// from the backend's capabilities, so the guards here judge a statement the way the connection will. + /// </summary> + public static BsqliteRuntimeLimits LimitsOf(BsqliteStorageCapabilities capabilities) => new() + { + SupportsMultiDatabaseTransactions = capabilities.SupportsMultiDatabaseTransactions, + CanChangePageSize = capabilities.CanChangePageSize, + }; + + /// <summary>What the library's guards make of one statement under one backend's limits. Nothing is executed.</summary> + public static BsqliteSqlVerdictDto Judge(string name, string sql, BsqliteRuntimeLimits limits) + { + try + { + BsqliteFeatureGuards.EnsureSupported(sql, limits); + + return new BsqliteSqlVerdictDto { Name = name, Accepted = true }; + } + catch (BsqliteException exception) + { + return new BsqliteSqlVerdictDto { Name = name, Accepted = false, Error = exception.Message }; + } + } + + /// <summary>A capability's value as C# prints it, with a [Flags] combination spelled out.</summary> + public static string FormatCapability(object? value) => value switch + { + null => "null", + bool flag => flag ? "true" : "false", + _ => value.ToString() ?? string.Empty + }; + + /// <summary>The capability properties of the contract, in declaration order - the rows of every table here.</summary> + public static PropertyInfo[] CapabilityProperties() + => [.. typeof(BsqliteStorageCapabilities).GetProperties(BindingFlags.Public | BindingFlags.Instance) + .Where(property => property.Name != "EqualityContract") + .OrderBy(property => property.MetadataToken)]; + + private static BsqliteStorageProviderDto Describe(IBsqliteStorageProvider provider) + { + var type = provider.GetType(); + var limits = LimitsOf(provider.Capabilities); + + return new BsqliteStorageProviderDto + { + Name = provider.Name, + TypeName = type.Name, + Namespace = type.Namespace ?? string.Empty, + Package = BsqliteApiCatalog.PackageOf(type), + Summary = BsqliteXmlDocs.GetSummary($"T:{type.FullName}"), + Remarks = BsqliteXmlDocs.GetRemarks($"T:{type.FullName}"), + VfsName = Constant(type, "VfsName"), + VfsModule = provider.VfsModule?.ModuleUrl, + AdminModule = Constant(type, "AdminModuleUrl"), + Capabilities = [.. CapabilityProperties().Select(property => new BsqliteCapabilityDto + { + Name = property.Name, + Value = FormatCapability(property.GetValue(provider.Capabilities)), + Summary = BsqliteXmlDocs.GetSummary($"P:{typeof(BsqliteStorageCapabilities).FullName}.{property.Name}") + })], + Guarded = [.. GuardedStatements.Select(sql => Judge(sql, sql, limits))] + }; + } + + /// <summary>A public string constant a provider type declares, such as its VFS name - read, not copied.</summary> + private static string? Constant(Type type, string name) + => type.GetField(name, BindingFlags.Public | BindingFlags.Static) is { IsLiteral: true } field ? field.GetRawConstantValue() as string : null; + + private static string Squash(string text) => new([.. text.Where(char.IsLetterOrDigit).Select(char.ToLowerInvariant)]); +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteStorageLab.cs b/src/Bsqlite/Demo/Server/Services/BsqliteStorageLab.cs new file mode 100644 index 00000000000..178f69d7157 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteStorageLab.cs @@ -0,0 +1,601 @@ +using System.Text; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Demo.Server.Dtos; +using Bit.Bsqlite.Storage.InMemory; +using Microsoft.Extensions.Logging; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// Runs Bit.Bsqlite's own storage selection on a configuration and a browser the caller describes, so +/// "which backend will this database open on, and why?" is answered by the code that decides it at +/// runtime rather than by a reading of the rules. +/// <para> +/// Nothing here re-implements a rule. The selection is built by <see cref="BsqliteStorageSelectionBuilder"/>, +/// so a configuration mistake fails with the builder's own message; the open goes through a real +/// <see cref="BsqliteSessionFactory"/> over a real <see cref="BsqliteStorageProviderResolver"/>, +/// <see cref="BsqliteStorageMigrator"/> and binding store, exactly as <c>AddBitBsqliteServices</c> +/// composes them. What stands in for the browser is the providers - each answers its probe the way the +/// caller described, keeps the capabilities the real provider type declares when it is a first-party +/// backend, and stores its databases in an in-memory admin store - and the transport, which opens +/// nothing: the library's tests drive the same composition the same way. +/// </para> +/// <para> +/// No SQL runs, here or anywhere on this server. The stand-in database a bound backend holds is a +/// SQLite header followed by zeros - enough for the migrator's header check and byte comparison to do +/// their real work, and nothing an engine could open. +/// </para> +/// </summary> +public static class BsqliteStorageLab +{ + /// <summary>The preference order a call without one plans for: the README's own, OPFS then IndexedDB.</summary> + public const string DefaultCandidates = "opfs, indexeddb"; + + public const string DefaultDatabaseName = "app.db"; + + /// <summary>The size of the stand-in database an existing binding holds, which a migration has to find room for.</summary> + public const int StandInDatabaseBytes = 64 * 1024; + + /// <summary>More names than any real configuration has: a list longer than this is a mistake, not a plan.</summary> + private const int MaxNames = 16; + + private const int MaxNameLength = 64; + + /// <summary>The migration modes, as the enum declares them - what the tool's schema offers.</summary> + public static readonly string[] MigrationModes = Enum.GetNames<BsqliteStorageMigrationMode>(); + + private static ReadOnlySpan<byte> SqliteMagic => "SQLite format 3\0"u8; + + public static async Task<BsqliteStoragePlanDto> PlanAsync( + string? candidates, + string? registered, + string? unavailable, + string? nonPersistent, + bool allowNonPersistentFallback, + string? migrationMode, + string? existingBinding, + long? freeBytes, + string? databaseName, + CancellationToken cancellationToken) + { + var candidateNames = Names(candidates ?? DefaultCandidates); + string[] registeredNames = registered is null ? [.. candidateNames.Distinct(StringComparer.OrdinalIgnoreCase)] : Names(registered); + var unavailableNames = Names(unavailable).ToHashSet(StringComparer.OrdinalIgnoreCase); + var nonPersistentNames = Names(nonPersistent).ToHashSet(StringComparer.OrdinalIgnoreCase); + var binding = string.IsNullOrWhiteSpace(existingBinding) ? null : existingBinding.Trim(); + var database = string.IsNullOrWhiteSpace(databaseName) ? DefaultDatabaseName : databaseName.Trim(); + + var notes = new List<string>(); + + var mode = BsqliteStorageMigrationMode.KeepExisting; + var configurationError = Validate(candidateNames, registeredNames, unavailableNames, nonPersistentNames, binding, database); + + // Parsed by name only: Enum.TryParse also takes "2" or "Manual, KeepExisting", neither of which is + // something WithMigrationMode could be handed in code. + if (configurationError is null && string.IsNullOrWhiteSpace(migrationMode) is false) + { + var modeName = MigrationModes.FirstOrDefault(name => string.Equals(name, migrationMode.Trim(), StringComparison.OrdinalIgnoreCase)); + + if (modeName is null) configurationError = $"'{migrationMode}' is not a migration mode. Use one of: {string.Join(", ", MigrationModes)}."; + else mode = Enum.Parse<BsqliteStorageMigrationMode>(modeName); + } + + // The builder is what UseStorage(...) runs at startup, so a preference order it refuses is refused + // with the exception the application would see, rather than with a paraphrase of it. + BsqliteStorageSelection? selection = null; + + if (configurationError is null) + { + try + { + selection = BsqliteStorageSelectionBuilder.Create(builder => + { + for (var index = 0; index < candidateNames.Length; index++) + { + if (index == 0) builder.Prefer(candidateNames[index]); + else builder.Fallback(candidateNames[index]); + } + + if (allowNonPersistentFallback) builder.AllowNonPersistentFallback(); + + builder.WithMigrationMode(mode); + }); + } + catch (Exception exception) when (exception is InvalidOperationException or ArgumentException) + { + configurationError = exception.Message; + } + } + + var providers = registeredNames.Select(name => new LabProvider( + name, + unavailableNames.Contains(name), + nonPersistentNames.Contains(name), + freeBytes)).ToArray(); + + foreach (var name in nonPersistentNames.Where(name => BsqliteStorageCatalog.Instance(name) is not null)) + { + notes.Add($"'{name}' is a first-party backend, so its persistence is what its provider type declares, not what nonPersistent says."); + } + + var configuration = Configuration(candidateNames, registeredNames, allowNonPersistentFallback, mode, database); + var log = new LabLog(); + + // Two resolvers over the same providers and bindings: one the session factory opens through, and + // one for the decision before any migration - ResolveAsync changes nothing, and keeping it apart + // keeps its log lines out of the report. Constructing them is also where two providers registered + // under one name are refused, as AddBitBsqliteServices refuses them on the first open. + BsqliteStorageProviderResolver? diagnostic = null; + BsqliteStorageProviderResolver? resolver = null; + var bindings = new BsqliteInMemoryStorageBindingStore(); + + if (configurationError is null) + { + try + { + diagnostic = new BsqliteStorageProviderResolver(providers, bindings); + resolver = new BsqliteStorageProviderResolver(providers, bindings, log.For<BsqliteStorageProviderResolver>()); + } + catch (InvalidOperationException exception) + { + configurationError = exception.Message; + } + } + + if (configurationError is not null) + { + return new BsqliteStoragePlanDto + { + Opens = false, + BindingBefore = binding, + BindingAfter = binding, + Candidates = Report(candidateNames, providers, binding, attempts: null, resolution: null, migration: null), + Error = configurationError, + Configuration = configuration, + Log = [], + Notes = [.. notes, "Nothing was planned: the configuration itself is refused - at startup, by AddBitBsqliteServices, before any database is opened."] + }; + } + + if (binding is not null) + { + await bindings.SetProviderNameAsync(database, binding, cancellationToken); + + // The data the binding points at. The in-memory backend never has any to give: a connection's + // database lives in the worker's own heap, where its admin store cannot see it - so moving away + // from it copies nothing, which is exactly what the library does with one. + var holder = providers.FirstOrDefault(provider => string.Equals(provider.Name, binding, StringComparison.OrdinalIgnoreCase)); + + if (holder is not null && string.Equals(holder.Name, BsqliteInMemoryStorageProvider.ProviderName, StringComparison.OrdinalIgnoreCase) is false) + { + await holder.Admin.ImportAsync(database, StandInDatabase(), cancellationToken); + } + } + + BsqliteStorageResolution? initial = null; + BsqliteStorageResolution? resolution = null; + IReadOnlyList<BsqliteStorageCandidateOutcome>? attempts = null; + string? error = null; + + try + { + initial = await diagnostic!.ResolveAsync(database, selection!, cancellationToken); + } + catch (BsqliteStorageUnavailableException) + { + // The open below fails the same way, and says so with the same exception. + } + + try + { + var factory = new BsqliteSessionFactory(resolver!, new LabTransportFactory(), selection!, bindings, new BsqliteStorageMigrator(), log.For<BsqliteSessionFactory>()); + + await using var session = await factory.OpenAsync(database, cancellationToken); + + resolution = session.Resolution; + attempts = resolution.Attempts; + } + catch (BsqliteStorageUnavailableException exception) + { + error = exception.Message; + attempts = exception.Attempts; + } + + var bindingAfter = await bindings.GetProviderNameAsync(database, cancellationToken); + var migration = await MigrationReportAsync(mode, initial, bindingAfter, database, log, cancellationToken); + + var plan = new BsqliteStoragePlanDto + { + Opens = resolution is not null, + Provider = resolution?.Provider.Name, + EngineBuild = resolution?.Provider.Capabilities.RequiredBuild.ToString(), + VfsModule = resolution?.Provider.VfsModule?.ModuleUrl, + IsFirstChoice = resolution?.IsFirstChoice, + WasDecidedByExistingData = resolution?.WasDecidedByExistingData, + BetterProviderAvailable = resolution?.BetterProviderAvailable?.Name, + BindingBefore = binding, + BindingAfter = bindingAfter, + Migration = migration, + Candidates = Report(candidateNames, providers, binding, attempts, resolution, migration), + Error = error, + Configuration = configuration, + Log = [.. log.Lines], + Notes = [] + }; + + return plan with { Notes = [.. notes, .. Explain(plan, selection!, providers)] }; + } + + /// <summary>The automatic migration the session factory ran, read back from what it left behind.</summary> + private static async Task<BsqliteMigrationReportDto?> MigrationReportAsync( + BsqliteStorageMigrationMode mode, + BsqliteStorageResolution? initial, + string? bindingAfter, + string database, + LabLog log, + CancellationToken cancellationToken) + { + if (initial?.BetterProviderAvailable is not { } better) return null; + + if (mode is not BsqliteStorageMigrationMode.AutomaticOnOpen) + { + return new BsqliteMigrationReportDto { From = initial.Provider.Name, To = better.Name, Outcome = "NotAttempted" }; + } + + var migrated = string.Equals(bindingAfter, better.Name, StringComparison.OrdinalIgnoreCase); + + return new BsqliteMigrationReportDto + { + From = initial.Provider.Name, + To = better.Name, + Outcome = migrated ? "Migrated" : "Failed", + // Whether the target now holds an image is the difference between a move and a rebinding: + // the migrator flips the binding without copying when the source had nothing to export. + Copied = migrated ? await better.Admin.ExistsAsync(database, cancellationToken) : null, + Error = migrated ? null : log.LastException + }; + } + + /// <summary> + /// Every provider the plan involves, with what the library decided about each: the candidates in + /// preference order, then the registered providers that are not candidates, then a bound provider that + /// is neither - each one matters to the outcome, and only the resolver's attempts say how. + /// </summary> + private static BsqliteCandidateReportDto[] Report( + string[] candidates, + LabProvider[] providers, + string? binding, + IReadOnlyList<BsqliteStorageCandidateOutcome>? attempts, + BsqliteStorageResolution? resolution, + BsqliteMigrationReportDto? migration) + { + var names = new List<string>(); + + foreach (var name in candidates.Concat(providers.Select(provider => provider.Name)).Append(binding ?? string.Empty)) + { + if (name.Length > 0 && names.Contains(name, StringComparer.OrdinalIgnoreCase) is false) names.Add(name); + } + + return [.. names.Select(name => + { + var provider = providers.FirstOrDefault(provider => string.Equals(provider.Name, name, StringComparison.OrdinalIgnoreCase)); + var attempt = attempts?.FirstOrDefault(outcome => string.Equals(outcome.ProviderName, name, StringComparison.OrdinalIgnoreCase)); + var rank = Array.FindIndex(candidates, candidate => string.Equals(candidate, name, StringComparison.OrdinalIgnoreCase)); + + return new BsqliteCandidateReportDto + { + Name = provider?.Name ?? name, + Rank = rank < 0 ? null : rank + 1, + Registered = provider is not null, + FirstParty = BsqliteStorageCatalog.Instance(name) is not null, + Persistent = provider?.Capabilities.IsPersistent, + Available = provider?.IsAvailable, + EngineBuild = provider?.Capabilities.RequiredBuild.ToString(), + Status = attempt?.Status.ToString(), + Explanation = attempt is not null + ? attempt.Explanation ?? attempt.Probe?.UnavailableReason + : NotExamined(name, provider, binding, rank, resolution, attempts, migration) + }; + })]; + } + + /// <summary>Why the resolver never looked at a provider - which is itself part of the answer.</summary> + private static string? NotExamined( + string name, + LabProvider? provider, + string? binding, + int rank, + BsqliteStorageResolution? resolution, + IReadOnlyList<BsqliteStorageCandidateOutcome>? attempts, + BsqliteMigrationReportDto? migration) + { + if (attempts is null) return null; + + if (migration is { Outcome: "Migrated" } && string.Equals(migration.From, name, StringComparison.OrdinalIgnoreCase)) + { + return $"Held the database until AutomaticOnOpen moved it to '{migration.To}'; once the copy was verified and the binding flipped, the source was deleted."; + } + + if (resolution?.BetterProviderAvailable is { } better && string.Equals(better.Name, name, StringComparison.OrdinalIgnoreCase)) + { + return "Available and ranked above the backend holding the data - reported as BetterProviderAvailable, which the migration mode decides the response to."; + } + + if (binding is not null) + { + return $"Not examined: the database already lives on '{resolution?.Provider.Name ?? binding}', and existing data outranks preference."; + } + + if (rank < 0) + { + return provider is null ? null : "Registered but not a candidate: selection never picks it for a new database, though a database bound to it still opens there."; + } + + return null; + } + + /// <summary>What the outcome means in practice, and what to change when it is not the one wanted.</summary> + private static IEnumerable<string> Explain(BsqliteStoragePlanDto plan, BsqliteStorageSelection selection, LabProvider[] providers) + { + var chosen = providers.FirstOrDefault(provider => provider.Name == plan.Provider); + + if (plan.Opens && chosen is not null) + { + yield return chosen.Capabilities.RequiredBuild is BsqliteEngineBuild.Synchronous + ? $"'{chosen.Name}' runs on the synchronous engine build - the smallest download, with no JSPI or Asyncify." + : $"'{chosen.Name}' needs the async-capable engine build: JSPI where the browser has it, Asyncify (about twice the size) otherwise."; + + if (chosen.Capabilities.IsPersistent is false) + { + yield return plan.WasDecidedByExistingData is true + ? $"'{chosen.Name}' does not survive a reload: everything written in this session is gone when the page closes." + : $"'{chosen.Name}' does not survive a reload: everything written in this session is gone when the page closes. No binding is written for it - it would protect no data - so the next load chooses again from the preference order."; + } + + if (plan.WasDecidedByExistingData is true && plan.Migration is not { Outcome: "Migrated" }) + { + yield return "The sticky binding decided: a database stays on the backend that holds it, whatever the preference order says. The binding is not rewritten."; + } + else if (plan.IsFirstChoice is false) + { + yield return "This is a fallback, so the resolver logged a warning naming every higher-ranked candidate and why it was rejected; session.Resolution.Attempts carries the same."; + } + + if (plan.WasDecidedByExistingData is not true && plan.BindingAfter is not null) + { + yield return $"The binding to '{plan.BindingAfter}' is written only after the open succeeds, so a failed open never leaves one pointing at nothing. From then on this database reopens there."; + } + } + + if (plan.BetterProviderAvailable is { } better && plan.Migration is null or { Outcome: "NotAttempted" }) + { + yield return selection.MigrationMode is BsqliteStorageMigrationMode.Manual + ? $"'{better}' is available and ranked higher, and Manual leaves it to the application: call BsqliteStorageMigrator.MigrateAsync with the database closed (BsqliteDatabase.WhileClosedAsync) when it suits - the demo's BrowserDatabase.SwitchStorageAsync does exactly that." + : $"'{better}' is available and ranked higher, but KeepExisting moves nothing. WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen) would copy the database there before opening it."; + } + + if (plan.Migration is { Outcome: "Migrated" } migrated) + { + yield return migrated.Copied is true + ? $"AutomaticOnOpen moved the database: exported from '{migrated.From}', header checked, imported into '{migrated.To}', read back and compared byte for byte, binding flipped, source deleted." + : $"AutomaticOnOpen flipped the binding to '{migrated.To}' without copying anything: '{migrated.From}' held no image its admin could export. Moving off in-memory always ends like this - its data lives in the worker's heap."; + } + + if (plan.Migration is { Outcome: "Failed" } failed) + { + yield return string.Equals(plan.Provider, failed.To, StringComparison.OrdinalIgnoreCase) + ? $"The automatic migration to '{failed.To}' failed after the copy was verified and bound - deleting '{failed.From}' failed with the source no longer whole - so the database opened on '{failed.To}', where the binding points." + : $"The automatic migration to '{failed.To}' failed and was logged; the binding still names '{failed.From}', so the database opened there with its data, and the move is offered again on the next open."; + } + + if (plan.Candidates.Any(candidate => candidate.Status == nameof(BsqliteStorageCandidateStatus.RejectedAsNonPersistent))) + { + yield return "A non-persistent backend was usable but refused as a fallback: AllowNonPersistentFallback() is the opt-in that accepts losing data on reload. A non-persistent FIRST choice needs no opt-in."; + } + + if (plan.Opens is false) + { + if (plan.BindingBefore is { } bound) + { + yield return plan.Candidates.FirstOrDefault(candidate => string.Equals(candidate.Name, bound, StringComparison.OrdinalIgnoreCase)) is { Registered: false } + ? $"The database lives on '{bound}', which is not registered. Register it again (AddStorageProvider) to reach the data - Bsqlite will not open an empty database elsewhere in its place." + : $"The database lives on '{bound}', which this browser cannot reach, and Bsqlite will not open an empty database elsewhere in its place. To start over deliberately, delete it with that provider's Admin.DeleteAsync and remove the binding (IBsqliteStorageBindingStore.RemoveAsync)."; + } + else + { + yield return "No candidate could be used, so the open throws BsqliteStorageUnavailableException; its Attempts and its message name every candidate and why it was rejected."; + } + } + + foreach (var missing in plan.Candidates.Where(candidate => candidate.Rank is not null && candidate.Registered is false)) + { + yield return missing.FirstParty + ? $"'{missing.Name}' is a candidate but not registered: add AddStorageProvider<{BsqliteStorageCatalog.Find(missing.Name)?.TypeName}>() and a reference to its package, {BsqliteStorageCatalog.Find(missing.Name)?.Package}." + : $"'{missing.Name}' is a candidate but no provider of that name is registered: register your IBsqliteStorageProvider with AddStorageProvider."; + } + + if (selection.IsStrict) + { + yield return "A single candidate is strict: it is used or the open fails - a lone candidate is never quietly substituted."; + } + } + + /// <summary>The registration this plan describes, as the C# that produces it.</summary> + private static string Configuration(string[] candidates, string[] registered, bool allowNonPersistentFallback, BsqliteStorageMigrationMode mode, string database) + { + // '\n' rather than AppendLine: the snippet is a JSON string, and it should read the same whichever + // operating system this server runs on. + var builder = new StringBuilder(); + + builder.Append("builder.Services.AddBitBsqliteServices(sqlite => sqlite\n"); + + foreach (var name in registered) + { + builder.Append(BsqliteStorageCatalog.Find(name) is { } known && BsqliteStorageCatalog.Instance(name) is not null + ? $" .AddStorageProvider<{known.TypeName}>()\n" + : $" .AddStorageProvider<YourStorageProvider>() // Name == \"{name}\"\n"); + } + + builder.Append(" .UseStorage(s => s"); + + for (var index = 0; index < candidates.Length; index++) + { + builder.Append($"\n .{(index == 0 ? "Prefer" : "Fallback")}({NameExpression(candidates[index])})"); + } + + if (allowNonPersistentFallback) builder.Append("\n .AllowNonPersistentFallback()"); + if (mode is not BsqliteStorageMigrationMode.KeepExisting) builder.Append($"\n .WithMigrationMode(BsqliteStorageMigrationMode.{mode})"); + + builder.Append($")\n .AddDatabase(\"{database}\"));"); + + return builder.ToString(); + } + + private static string NameExpression(string name) + => BsqliteStorageCatalog.Instance(name) is not null && BsqliteStorageCatalog.Find(name) is { } known + ? $"{known.TypeName}.ProviderName" + : $"\"{name}\""; + + private static string? Validate(string[] candidates, string[] registered, HashSet<string> unavailable, HashSet<string> nonPersistent, string? binding, string database) + { + foreach (var (list, label) in new[] { (candidates, "candidates"), (registered, "registered"), (unavailable.ToArray(), "unavailable"), (nonPersistent.ToArray(), "nonPersistent") }) + { + if (list.Length > MaxNames) return $"'{label}' names {list.Length} backends; this tool plans for at most {MaxNames}."; + + if (list.FirstOrDefault(name => name.Length > MaxNameLength) is { } longName) return $"'{longName[..20]}...' is longer than a provider name can usefully be ({MaxNameLength} characters)."; + } + + if (binding?.Length > MaxNameLength) return $"existingBinding is longer than a provider name can usefully be ({MaxNameLength} characters)."; + + // Both reach the library's log lines, which this tool reports back: a line break in either would forge + // lines the library never wrote. The candidate lists cannot carry one - Names splits on it. + if (binding?.Any(char.IsControl) is true) return "existingBinding contains a control character, which no provider name has."; + + if (database.Length > 256) return "databaseName is longer than 256 characters."; + + return database.Any(char.IsControl) ? "databaseName contains a control character, which no database name should." : null; + } + + /// <summary>A list of provider names as a caller writes one: "opfs, indexeddb", "opfs indexeddb" or "opfs;indexeddb".</summary> + private static string[] Names(string? text) + => string.IsNullOrWhiteSpace(text) ? [] : text.Split([',', ';', ' ', '\t', '\r', '\n'], StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries); + + /// <summary>A SQLite header followed by zeros: what the migrator checks and compares, and nothing an engine could open.</summary> + private static byte[] StandInDatabase() + { + var image = new byte[StandInDatabaseBytes]; + + SqliteMagic.CopyTo(image); + + return image; + } + + /// <summary> + /// A backend as the caller described it. A first-party name keeps the real provider's name, + /// capabilities and VFS module, so selection and the engine-build choice see exactly what they would + /// in a browser; only the probe is the caller's, since that is the browser being described. + /// </summary> + internal sealed class LabProvider : IBsqliteStorageProvider + { + private readonly BsqliteProbeResult _probe; + + public LabProvider(string name, bool unavailable, bool nonPersistent, long? freeBytes) + { + var real = BsqliteStorageCatalog.Instance(name); + + Name = real?.Name ?? name; + IsAvailable = unavailable is false; + + // A custom backend is described by the two facts that cannot be guessed safely - the same two the + // contract makes required - and the least capable answer for everything else, as the contract's + // own defaults are. + Capabilities = real?.Capabilities ?? new BsqliteStorageCapabilities + { + RequiredBuild = BsqliteEngineBuild.AsyncCapable, + IsPersistent = nonPersistent is false, + }; + + VfsModule = real?.VfsModule; + + // An admin store of the library's own: the in-memory backend's, which exports, imports and + // compares real bytes - what the migrator needs to do its real work. + Admin = new BsqliteInMemoryStorageProvider(Name).Admin; + + _probe = unavailable + ? BsqliteProbeResult.Unavailable($"'{Name}' was described as unavailable in this browser.") + : BsqliteProbeResult.Available(quotaBytes: freeBytes, usageBytes: freeBytes is null ? null : 0); + } + + public string Name { get; } + + public bool IsAvailable { get; } + + public BsqliteStorageCapabilities Capabilities { get; } + + public BsqliteJsModule? VfsModule { get; } + + public IBsqliteStorageAdmin Admin { get; } + + public ValueTask<BsqliteProbeResult> ProbeAsync(CancellationToken cancellationToken = default) => ValueTask.FromResult(_probe); + } + + /// <summary>Hands the session factory a transport that opens nothing, as the library's own session tests do.</summary> + private sealed class LabTransportFactory : IBsqliteTransportFactory + { + public IBsqliteTransport Create(IBsqliteStorageProvider provider) => new LabTransport(); + } + + /// <summary> + /// A transport with no engine behind it. Opening is all the session factory does with one, so that + /// is all it supports: a batch reaching it would mean SQL about to run, and that does not happen here. + /// </summary> + private sealed class LabTransport : IBsqliteTransport + { + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) => Task.CompletedTask; + + public Task CloseAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; + + public Task<IReadOnlyList<BsqliteCommandResult>> ExecuteAsync(IReadOnlyList<BsqliteCommandRequest> batch, CancellationToken cancellationToken = default) + => throw new InvalidOperationException("The storage lab has no engine: it plans an open, and never executes SQL."); + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; + } + + /// <summary> + /// What the resolver and the session factory log while they decide - the warning a fallback raises, + /// the information a better backend left unused is reported as, the failure of an automatic + /// migration - captured as the lines an application's log would show. + /// </summary> + private sealed class LabLog + { + private readonly List<string> _lines = []; + + public IReadOnlyList<string> Lines => _lines; + + /// <summary>The message of the last exception logged - an automatic migration's failure.</summary> + public string? LastException { get; private set; } + + public ILogger<T> For<T>() => new Sink<T>(this); + + private void Write(LogLevel level, string message, Exception? exception) + { + _lines.Add(exception is null ? $"{level}: {message}" : $"{level}: {message} ({exception.GetType().Name}: {exception.Message})"); + + if (exception is not null) LastException = exception.Message; + } + + private sealed class Sink<T>(LabLog log) : ILogger<T> + { + public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => logLevel >= LogLevel.Information; + + public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func<TState, Exception?, string> formatter) + { + if (IsEnabled(logLevel)) log.Write(logLevel, formatter(state, exception), exception); + } + } + } +} diff --git a/src/Bsqlite/Demo/Server/Services/BsqliteXmlDocs.cs b/src/Bsqlite/Demo/Server/Services/BsqliteXmlDocs.cs new file mode 100644 index 00000000000..3610b0298f6 --- /dev/null +++ b/src/Bsqlite/Demo/Server/Services/BsqliteXmlDocs.cs @@ -0,0 +1,252 @@ +using System.Text; +using System.Xml.Linq; +using System.Collections.Frozen; +using System.Text.RegularExpressions; + +namespace Bit.Bsqlite.Demo.Server.Services; + +/// <summary> +/// Reads the XML documentation the Bit.Bsqlite builds emit next to their assemblies - the core, the +/// three storage packages and the testing kit - and flattens each entry into plain text: +/// <c><see cref="T:Bit.Bsqlite.Storage.IBsqliteStorageProvider"/></c> becomes +/// <c>IBsqliteStorageProvider</c>, <c><c></c> spans become their content, and paragraphs become +/// blank lines. That text is the same prose a developer reads in IntelliSense, which makes it the most +/// accurate answer an MCP client can get about a member - no second copy to keep in sync. +/// </summary> +public static partial class BsqliteXmlDocs +{ + private const char CodeMarker = (char)1; + + /// <summary>How far an <c><inheritdoc cref="..."/></c> chain is followed before giving up on it.</summary> + private const int MaxInheritDepth = 4; + + private static readonly Lazy<FrozenDictionary<string, XElement>> _members = new(Load); + private static readonly Lazy<FrozenDictionary<string, XElement>> _overloads = new(BuildOverloads); + + /// <summary>The documentation files read, by assembly name - one per package.</summary> + public static IEnumerable<string> AssemblyNames => BsqliteApiCatalog.Assemblies.Select(assembly => assembly.GetName().Name!); + + /// <summary>How many documented members were loaded, across every file.</summary> + public static int Count => _members.Value.Count; + + /// <summary>The flattened <summary> of a documented member, or null.</summary> + public static string? GetSummary(string documentationId) => GetSection(documentationId, "summary", 0); + + /// <summary>The flattened <remarks> of a documented member, or null.</summary> + public static string? GetRemarks(string documentationId) => GetSection(documentationId, "remarks", 0); + + private static string? GetSection(string documentationId, string section, int depth) + { + // Overloads are told apart by their parameter list; when building it did not produce an exact + // hit (generics, modifiers), one overload's documentation still beats none. + if (_members.Value.TryGetValue(documentationId, out var member) is false && + _overloads.Value.TryGetValue(documentationId, out member) is false) return null; + + // A member may carry several <remarks> blocks. Every one of them is a caveat, so they are joined + // rather than all but the first being dropped. + var elements = member.Elements(section).ToArray(); + + if (elements.Length > 0) + { + var text = string.Join("\n\n", elements.Select(Flatten).Where(part => part.Length > 0)); + + return text.Length == 0 ? null : text; + } + + // The library's generic overloads say <inheritdoc cref="..."/> and nothing else: the compiler + // shows the referenced member's text for them, and so does this. An <inheritdoc/> without a + // cref is resolved by the API catalog, which knows the member's base and interfaces. + if (depth < MaxInheritDepth && member.Element("inheritdoc")?.Attribute("cref")?.Value is { Length: > 0 } cref) + { + return GetSection(cref, section, depth + 1); + } + + return null; + } + + /// <summary> + /// Every documented method by its id without the parameter list, so an inexact id still finds an + /// overload without walking the whole table. Which overload is decided by ordering the candidates + /// - a FrozenDictionary enumerates in whatever order it hashed into, so taking one off it + /// directly would answer differently per build. + /// </summary> + private static FrozenDictionary<string, XElement> BuildOverloads() + { + return _members.Value.Where(m => m.Key.Contains('(', StringComparison.Ordinal)) + .OrderBy(m => m.Key, StringComparer.Ordinal) + .GroupBy(m => m.Key[..m.Key.IndexOf('(', StringComparison.Ordinal)], StringComparer.Ordinal) + .ToFrozenDictionary(g => g.Key, g => g.First().Value, StringComparer.Ordinal); + } + + private static FrozenDictionary<string, XElement> Load() + { + var members = new Dictionary<string, XElement>(StringComparer.Ordinal); + + // One file per package. A member id is fully qualified, so the files cannot collide; the first + // one wins anyway rather than letting a duplicate throw. + foreach (var name in AssemblyNames) + { + var path = Path.Combine(AppContext.BaseDirectory, $"{name}.xml"); + + if (File.Exists(path) is false) continue; + + try + { + var document = XDocument.Load(path); + + foreach (var member in document.Descendants("member")) + { + if (member.Attribute("name")?.Value is not { } id) continue; + + members.TryAdd(id, member); + } + } + catch (Exception) + { + // Documentation is a nicety: a malformed or half-written file must not take the tools + // down, nor the other files with it. + } + } + + return members.ToFrozenDictionary(StringComparer.Ordinal); + } + + private static string Flatten(XElement element) + { + var builder = new StringBuilder(); + var codeSamples = new List<string>(); + + Write(element, builder, codeSamples); + + var text = builder.ToString().Replace("\r\n", "\n", StringComparison.Ordinal); + + // The source wraps its prose across lines, so a single newline is just a space - while the + // blank line a <para> emits is a real paragraph break and has to survive. + text = WrappedLineRegex().Replace(text, " "); + text = ParagraphBreakRegex().Replace(text, "\n\n"); + text = RepeatedSpaceRegex().Replace(text, " "); + + // A code sample only stood in as a placeholder while the prose was unwrapped: its own line + // breaks and indentation are the sample, not source wrapping, and go back in verbatim. + for (int i = 0; i < codeSamples.Count; i++) + { + text = text.Replace(CodePlaceholder(i), codeSamples[i], StringComparison.Ordinal); + } + + return text.Trim(); + } + + /// <summary> + /// Stands in for a code sample while the prose is unwrapped. U+0001 never occurs in + /// documentation text, and none of the whitespace passes above can touch it. + /// </summary> + private static string CodePlaceholder(int index) => $"{CodeMarker}{index}{CodeMarker}"; + + private static void Write(XNode node, StringBuilder builder, List<string> codeSamples) + { + switch (node) + { + case XText text: + builder.Append(text.Value); + return; + + case XElement element: + switch (element.Name.LocalName) + { + case "see" or "seealso": + builder.Append(Reference(element)); + return; + + case "paramref" or "typeparamref": + builder.Append(element.Attribute("name")?.Value); + return; + + case "para": + builder.Append('\n'); + foreach (var child in element.Nodes()) Write(child, builder, codeSamples); + builder.Append('\n'); + return; + + case "code": + // Blank lines on both sides, so the sample is a block of its own rather than + // a sentence that happens to continue in code. + builder.Append("\n\n").Append(CodePlaceholder(codeSamples.Count)).Append("\n\n"); + codeSamples.Add($"```\n{Dedent(element.Value)}\n```"); + return; + + case "param" or "typeparam": + builder.Append('\n').Append(element.Attribute("name")?.Value).Append(": "); + foreach (var child in element.Nodes()) Write(child, builder, codeSamples); + return; + + default: + foreach (var child in element.Nodes()) Write(child, builder, codeSamples); + return; + } + } + } + + /// <summary> + /// A code sample without the indentation of the comment it was written in. The compiler keeps + /// every line's leading spaces, so a sample in an indented doc comment arrives with its first line + /// flush (it follows the <code> tag) and every other line pushed right by the comment's own + /// indentation - which reads as a sample whose braces do not line up. + /// </summary> + private static string Dedent(string code) + { + var lines = code.ReplaceLineEndings("\n").Split('\n').ToList(); + + while (lines.Count > 0 && string.IsNullOrWhiteSpace(lines[0])) lines.RemoveAt(0); + while (lines.Count > 0 && string.IsNullOrWhiteSpace(lines[^1])) lines.RemoveAt(lines.Count - 1); + + if (lines.Count == 0) return string.Empty; + + // The first line is measured only when it is indented itself: one that follows <code> on the + // same line starts at column 0 whatever the rest of the sample does. + var measured = lines.Skip(char.IsWhiteSpace(lines[0], 0) ? 0 : 1).Where(line => string.IsNullOrWhiteSpace(line) is false).ToArray(); + var indent = measured.Length == 0 ? 0 : measured.Min(line => line.Length - line.TrimStart().Length); + + return string.Join('\n', lines.Select((line, index) => + index == 0 && char.IsWhiteSpace(line, 0) is false ? line : line.Length >= indent ? line[indent..].TrimEnd() : line.Trim())); + } + + /// <summary>Turns a cref such as "P:Bit.Bsqlite.BsqliteStorageSelection.Candidates" into "BsqliteStorageSelection.Candidates".</summary> + private static string Reference(XElement element) + { + var target = element.Attribute("cref")?.Value ?? element.Attribute("langword")?.Value ?? element.Value; + + if (string.IsNullOrEmpty(target)) return string.Empty; + + var kind = target.Length > 2 && target[1] == ':' ? target[0] : '\0'; + if (kind != '\0') target = target[2..]; + + // A method cref carries its parameter list ("M:Bit.Bsqlite.Interop.BsqliteDatabase.OpenAsync(System.Threading.CancellationToken)"). + // Fully qualified argument types read as noise in prose, and the '.'s inside them would + // otherwise be split on below - which would keep the arguments and drop the owning type. + var arguments = target.IndexOf('(', StringComparison.Ordinal); + if (arguments > 0) target = target[..arguments]; + + // Drop the arity markers of generic types and methods ("IBsqliteLiveQuery`1.RefreshAsync") wherever + // they are: cutting the text at the first one would drop the member a member reference is about. + target = ArityRegex().Replace(target, string.Empty); + + var parts = target.Split('.'); + + // A type reads best on its own; a member keeps the type it belongs to. + var keep = kind is 'T' or 'N' ? 1 : 2; + + return parts.Length <= keep ? target : string.Join('.', parts[^keep..]); + } + + [GeneratedRegex(@"(?<!\n)[ \t]*\n[ \t]*(?!\n)")] + private static partial Regex WrappedLineRegex(); + + [GeneratedRegex(@"[ \t]*\n[ \t\n]*")] + private static partial Regex ParagraphBreakRegex(); + + [GeneratedRegex(@"[ \t]{2,}")] + private static partial Regex RepeatedSpaceRegex(); + + [GeneratedRegex(@"`+\d+")] + private static partial Regex ArityRegex(); +} diff --git a/src/Bsqlite/Demo/Server/appsettings.json b/src/Bsqlite/Demo/Server/appsettings.json new file mode 100644 index 00000000000..d286041f99d --- /dev/null +++ b/src/Bsqlite/Demo/Server/appsettings.json @@ -0,0 +1,10 @@ +{ + "Logging": { + "LogLevel": { + "Default": "Information", + "Microsoft.AspNetCore": "Warning", + "Microsoft.EntityFrameworkCore": "Warning" + } + }, + "AllowedHosts": "*" +} diff --git a/src/Bsqlite/Demo/Server/wwwroot/css/base.css b/src/Bsqlite/Demo/Server/wwwroot/css/base.css new file mode 100644 index 00000000000..9917d59a077 --- /dev/null +++ b/src/Bsqlite/Demo/Server/wwwroot/css/base.css @@ -0,0 +1,420 @@ +/* ============================================================================= + Bsqlite - reset, typography, and the application shell + ========================================================================== */ + +*, *::before, *::after { box-sizing: border-box; } + +html { + scrollbar-gutter: stable; + scroll-behavior: smooth; + -webkit-text-size-adjust: 100%; +} + +body { + margin: 0; + min-height: 100dvh; + font-family: var(--font-sans); + font-size: var(--text-base); + line-height: var(--lh-normal); + color: var(--text); + background: var(--canvas); + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; +} + +::selection { background: var(--accent-soft); color: var(--accent-ink); } + +/* --- headings -------------------------------------------------------------- */ +h1, h2, h3, h4 { + margin: 0; + font-weight: 600; + line-height: var(--lh-tight); + letter-spacing: var(--track-tight); + color: var(--text); + text-wrap: balance; +} +h1 { font-size: var(--text-2xl); letter-spacing: var(--track-tighter); } +h2 { font-size: var(--text-lg); } +h3 { font-size: var(--text-md); } +h4 { font-size: var(--text-base); } + +p { margin: 0 0 var(--space-3); color: var(--text-muted); text-wrap: pretty; } +p:last-child { margin-bottom: 0; } + +.lead { + font-size: var(--text-md); + line-height: var(--lh-normal); + color: var(--text-muted); + max-width: var(--measure); +} + +.eyebrow { + margin: 0; + font-family: var(--font-mono); + font-size: var(--text-2xs); + font-weight: 600; + letter-spacing: var(--track-wide); + text-transform: uppercase; + color: var(--accent); +} + +a { + color: var(--accent); + text-decoration-color: color-mix(in srgb, var(--accent) 35%, transparent); + text-underline-offset: 0.18em; + transition: color var(--dur) var(--ease); +} +a:hover { color: var(--accent-hover); text-decoration-color: currentColor; } + +hr { border: 0; border-top: 1px solid var(--border); margin: var(--space-6) 0; } + +strong, b { font-weight: 600; color: var(--text); } + +/* --- focus ---------------------------------------------------------------- */ +:focus-visible { + outline: 2px solid var(--focus-ring); + outline-offset: 2px; + border-radius: var(--radius-sm); +} +:focus:not(:focus-visible) { outline: none; } + +.skip-link { + position: fixed; + top: var(--space-2); + left: var(--space-2); + z-index: 100; + padding: var(--space-2) var(--space-4); + background: var(--accent); + color: var(--text-on-accent); + border-radius: var(--radius-md); + font-weight: 600; + text-decoration: none; + transform: translateY(-200%); + transition: transform var(--dur) var(--ease); +} +.skip-link:focus-visible { transform: translateY(0); color: var(--text-on-accent); } + +/* --- inline code ----------------------------------------------------------- */ +code, pre, .mono { + /* Ligatures off: a documentation site has to show the characters you type, + not JetBrains Mono's arrow glyph for "=>". */ + font-variant-ligatures: none; + font-feature-settings: "calt" 0, "liga" 0; +} + +code { + font-family: var(--font-mono); + font-size: 0.86em; + background: var(--surface-2); + border: 1px solid var(--border-subtle); + border-radius: var(--radius-sm); + padding: 0.05em 0.35em; + color: var(--text); + white-space: nowrap; +} + +/* An identifier such as BsqliteSynchronousApiNotSupportedException is wider than a phone held upright; there it + may break anywhere rather than widen the page. */ +@media (max-width: 40rem) { + code { + white-space: normal; + overflow-wrap: anywhere; + } +} +.mono { font-family: var(--font-mono); font-size: var(--text-sm); font-variant-numeric: tabular-nums; } + +/* ============================================================================= + Shell: topbar / sidebar / content + ========================================================================== */ + +.topbar { + position: sticky; + top: 0; + z-index: 40; + height: var(--topbar-h); + display: flex; + align-items: center; + background: color-mix(in srgb, var(--canvas) 82%, transparent); + backdrop-filter: saturate(180%) blur(12px); + border-bottom: 1px solid var(--border); +} +.topbar-inner { + width: 100%; + max-width: var(--page-max); + margin: 0 auto; + padding: 0 var(--space-4); + display: flex; + align-items: center; + gap: var(--space-3); +} + +.brand { + display: inline-flex; + align-items: center; + gap: var(--space-3); + text-decoration: none; + color: var(--text); + border-radius: var(--radius-md); + padding: var(--space-1); + margin-left: calc(var(--space-1) * -1); +} +.brand:hover { color: var(--text); } +.brand-mark { + flex: none; + width: 1.75rem; + height: 1.75rem; + display: grid; + place-items: center; + border-radius: var(--radius-md); + background: var(--accent); + color: var(--text-on-accent); +} +.brand-mark svg { width: 1.05rem; height: 1.05rem; } +.brand-name { + font-weight: 600; + font-size: var(--text-md); + letter-spacing: var(--track-tight); + line-height: 1.1; +} +.brand-version { + font-family: var(--font-mono); + font-size: var(--text-2xs); + color: var(--text-muted); + border: 1px solid var(--border); + border-radius: var(--radius-full); + padding: 0.1rem 0.45rem; + line-height: 1.4; + white-space: nowrap; +} + +.topbar-links { display: flex; align-items: center; gap: var(--space-1); margin-left: var(--space-4); } +.topbar-link { + padding: var(--space-2) var(--space-3); + border-radius: var(--radius-md); + color: var(--text-muted); + text-decoration: none; + font-size: var(--text-base); + font-weight: 500; + transition: color var(--dur) var(--ease), background var(--dur) var(--ease); +} +.topbar-link:hover { color: var(--text); background: var(--surface-2); } +.topbar-link[aria-current] { color: var(--text); background: var(--surface-2); } + +.topbar-actions { margin-left: auto; display: flex; align-items: center; gap: var(--space-2); } + +/* --- icon button / theme switch ------------------------------------------- */ +.icon-btn { + display: inline-grid; + place-items: center; + width: 2.25rem; + height: 2.25rem; + padding: 0; + border: 1px solid transparent; + border-radius: var(--radius-md); + background: transparent; + color: var(--text-muted); + cursor: pointer; + transition: color var(--dur) var(--ease), background var(--dur) var(--ease), border-color var(--dur) var(--ease); +} +/* The :not(:disabled) is not decoration: components.css styles bare <button> as the primary + action, and `button:hover:not(:disabled)` out-specifies a plain `.icon-btn:hover`, which would + paint every chrome button accent-green on hover. Chrome controls match that specificity. */ +.icon-btn:hover:not(:disabled) { color: var(--text); background: var(--surface-2); border-color: var(--border); } +.icon-btn svg { width: 1.15rem; height: 1.15rem; } + +.segmented { + display: inline-flex; + gap: 2px; + padding: 2px; + background: var(--surface-2); + border: 1px solid var(--border); + border-radius: var(--radius-md); +} +.segmented button { + display: inline-grid; + place-items: center; + width: 1.85rem; + height: 1.6rem; + padding: 0; + border: 0; + border-radius: calc(var(--radius-md) - 2px); + background: transparent; + color: var(--text-faint); + cursor: pointer; + transition: color var(--dur) var(--ease), background var(--dur) var(--ease); +} +.segmented button:hover:not(:disabled) { color: var(--text); background: transparent; } +.segmented button[aria-pressed="true"], +.segmented button[aria-pressed="true"]:hover:not(:disabled) { + background: var(--surface); + color: var(--text); + box-shadow: var(--shadow-xs); +} +.segmented svg { width: 0.95rem; height: 0.95rem; } + +/* --- layout grid ----------------------------------------------------------- */ +.layout { + max-width: var(--page-max); + margin: 0 auto; + padding: 0 var(--space-4); + display: grid; + grid-template-columns: var(--sidebar-w) minmax(0, 1fr); + gap: var(--space-6); + align-items: start; +} + +.sidebar { + position: sticky; + top: var(--topbar-h); + max-height: calc(100dvh - var(--topbar-h)); + overflow-y: auto; + overscroll-behavior: contain; + min-height: calc(100dvh - var(--topbar-h)); + padding: var(--space-5) var(--space-2) var(--space-7) 0; + border-right: 1px solid var(--border-subtle); + scrollbar-width: thin; +} + +.nav-group + .nav-group { margin-top: var(--space-5); } +.nav-group-title { + margin: 0 0 var(--space-2) var(--space-3); + font-family: var(--font-mono); + font-size: var(--text-2xs); + font-weight: 600; + letter-spacing: var(--track-wide); + text-transform: uppercase; + color: var(--text-faint); +} +.nav-list { list-style: none; margin: 0; padding: 0; display: grid; gap: 1px; } + +.nav-link { + display: flex; + align-items: center; + gap: var(--space-3); + padding: var(--space-2) var(--space-3); + border-radius: var(--radius-md); + color: var(--text-muted); + text-decoration: none; + font-size: var(--text-base); + line-height: 1.3; + min-height: 2.25rem; + transition: color var(--dur) var(--ease), background var(--dur) var(--ease); +} +.nav-link:hover { color: var(--text); background: var(--surface-2); } +.nav-link svg { width: 1.05rem; height: 1.05rem; flex: none; color: var(--text-faint); transition: color var(--dur) var(--ease); } +.nav-link:hover svg { color: var(--text-muted); } +.nav-link[aria-current="page"] { + background: var(--accent-wash); + color: var(--accent-ink); + font-weight: 600; + box-shadow: inset 2px 0 0 var(--accent); +} +.nav-link[aria-current="page"] svg { color: var(--accent); } +.nav-external { margin-left: auto; opacity: 0.65; } +.nav-external svg { width: 0.85rem; height: 0.85rem; } + +.nav-toggle { display: none; } +.nav-scrim { display: none; } + +/* --- content --------------------------------------------------------------- */ +.content { min-width: 0; padding: var(--space-5) 0 var(--space-8); } +.container { width: 100%; max-width: var(--page-max); margin: 0 auto; padding: 0 var(--space-4); } + +.site-footer { + border-top: 1px solid var(--border); + background: var(--canvas-sunken); + padding: var(--space-6) 0; + margin-top: var(--space-8); +} +.footer-inner { + max-width: var(--page-max); + margin: 0 auto; + padding: 0 var(--space-4); + display: flex; + flex-wrap: wrap; + gap: var(--space-4) var(--space-6); + align-items: center; + justify-content: space-between; + font-size: var(--text-sm); + color: var(--text-muted); +} +.footer-links { display: flex; flex-wrap: wrap; gap: var(--space-4); } +.footer-links a { color: var(--text-muted); text-decoration: none; } +.footer-links a:hover { color: var(--accent); } + +/* --- responsive ------------------------------------------------------------- */ +@media (max-width: 639px) { + /* At 390px the version chip pushes the theme switch off the right edge. */ + .brand-version { display: none; } + .topbar-inner { gap: var(--space-2); padding: 0 var(--space-3); } +} + +@media (max-width: 1023px) { + .layout { grid-template-columns: minmax(0, 1fr); gap: 0; } + .topbar-links { display: none; } + + .nav-toggle { display: inline-grid; } + + .sidebar { + position: fixed; + min-height: 0; + inset: var(--topbar-h) auto 0 0; + z-index: 45; + width: min(20rem, 84vw); + max-height: none; + padding: var(--space-4) var(--space-3) var(--space-7); + background: var(--canvas); + border-right: 1px solid var(--border); + box-shadow: var(--shadow-lg); + transform: translateX(-102%); + transition: transform var(--dur-slow) var(--ease), visibility var(--dur-slow) var(--ease); + visibility: hidden; + } + .sidebar[data-open="true"] { transform: translateX(0); visibility: visible; } + + .nav-scrim { + display: block; + position: fixed; + inset: var(--topbar-h) 0 0 0; + z-index: 44; + background: var(--overlay); + border: 0; + padding: 0; + opacity: 0; + pointer-events: none; + transition: opacity var(--dur-slow) var(--ease); + } + .nav-scrim[data-open="true"] { opacity: 1; pointer-events: auto; } + .nav-scrim:hover:not(:disabled) { background: var(--overlay); } +} + +@media (prefers-reduced-motion: reduce) { + html { scroll-behavior: auto; } + *, *::before, *::after { + animation-duration: 0.01ms !important; + animation-iteration-count: 1 !important; + transition-duration: 0.01ms !important; + } +} + +/* Blazor's unhandled-error bar (Components/App.razor). Hidden until the runtime shows it - the framework only + switches its display on, so without this rule it would sit at the bottom of every page. */ +#blazor-error-ui { + position: fixed; + inset: auto 0 0 0; + z-index: 1000; + display: none; + padding: 0.7rem 1.2rem; + background: var(--danger-soft); + color: var(--danger-ink); + border-top: 1px solid var(--danger); + font-size: var(--text-sm); +} + +#blazor-error-ui .reload { margin-inline-start: 0.5rem; } + +#blazor-error-ui .dismiss { + cursor: pointer; + float: right; + font-weight: 700; +} diff --git a/src/Bsqlite/Demo/Server/wwwroot/css/components.css b/src/Bsqlite/Demo/Server/wwwroot/css/components.css new file mode 100644 index 00000000000..947ab913eaf --- /dev/null +++ b/src/Bsqlite/Demo/Server/wwwroot/css/components.css @@ -0,0 +1,554 @@ +/* ============================================================================= + Bsqlite - components + Layer 3 of the token system: every value below resolves through a semantic + token, never through a raw hex. + ========================================================================== */ + +/* --- page header ----------------------------------------------------------- */ +.page-head { + max-width: 62rem; + margin: 0 0 var(--space-6); + padding-bottom: var(--space-5); + border-bottom: 1px solid var(--border-subtle); +} +.page-head h1 { margin: var(--space-2) 0 var(--space-3); font-size: var(--text-2xl); } +.page-head .lead { margin: 0; } + +/* --- section --------------------------------------------------------------- */ +.section { margin: var(--space-8) 0 0; } +.section-head { max-width: 46rem; margin: 0 0 var(--space-5); } +.section-head h2 { + font-size: var(--text-xl); + letter-spacing: var(--track-tighter); + margin: var(--space-2) 0 var(--space-2); +} +.section-head p { margin: 0; } + +/* --- card ------------------------------------------------------------------ */ +.card { + position: relative; + min-width: 0; + background: var(--surface); + border: 1px solid var(--border); + border-radius: var(--radius-lg); + padding: var(--space-5); + box-shadow: var(--shadow-xs); +} +.card-head { + display: flex; + align-items: baseline; + flex-wrap: wrap; + gap: var(--space-2) var(--space-3); + margin-bottom: var(--space-2); +} +.card-head h2 { margin: 0; } +.card-step { + font-family: var(--font-mono); + font-size: var(--text-2xs); + font-weight: 600; + letter-spacing: var(--track-wide); + text-transform: uppercase; + color: var(--accent); + background: var(--accent-wash); + border: 1px solid var(--accent-soft); + border-radius: var(--radius-full); + padding: 0.1rem 0.5rem; + order: -1; +} +.card > p { max-width: var(--measure); } +/* Sub-headings inside a card need their own rhythm: without this a demo that lists + several sub-sections runs them together. */ +.card h3 { margin-top: var(--space-5); } +.card > h3:first-child, .card-head + h3 { margin-top: 0; } + +/* --- callouts -------------------------------------------------------------- */ +.try, .note, .warn { + display: flex; + gap: var(--space-3); + align-items: baseline; + border-radius: var(--radius-md); + padding: var(--space-3) var(--space-4); + font-size: var(--text-base); + line-height: var(--lh-snug); + margin: var(--space-4) 0; + max-width: var(--measure); +} +.try { + background: var(--accent-wash); + border: 1px solid var(--accent-soft); + border-left: 2px solid var(--accent); + color: var(--accent-ink); +} +.try b { + flex: none; + font-family: var(--font-mono); + font-size: var(--text-2xs); + letter-spacing: var(--track-wide); + text-transform: uppercase; + color: var(--accent); + padding-top: 0.1em; +} +.try a, .note a { color: currentColor; text-decoration-color: color-mix(in srgb, currentColor 40%, transparent); } +.try code, .note code, .warn code { background: transparent; border-color: currentColor; } + +.note { + background: var(--surface-2); + border: 1px solid var(--border); + color: var(--text-muted); +} +.warn { + background: var(--warn-soft); + border: 1px solid color-mix(in srgb, var(--warn) 30%, transparent); + color: var(--warn-ink); +} + +.error { + display: flex; + gap: var(--space-3); + align-items: baseline; + margin: 0 0 var(--space-4); + padding: var(--space-3) var(--space-4); + border-radius: var(--radius-md); + background: var(--danger-soft); + border: 1px solid color-mix(in srgb, var(--danger) 35%, transparent); + color: var(--danger-ink); + font-size: var(--text-base); +} + +/* --- buttons --------------------------------------------------------------- */ +button, .btn { + font: inherit; + font-weight: 500; + display: inline-flex; + align-items: center; + justify-content: center; + gap: var(--space-2); + min-height: 2.25rem; + padding: var(--space-2) var(--space-4); + border: 1px solid var(--accent); + border-radius: var(--radius-md); + background: var(--accent); + color: var(--text-on-accent); + cursor: pointer; + text-decoration: none; + white-space: nowrap; + transition: background var(--dur) var(--ease), border-color var(--dur) var(--ease), + color var(--dur) var(--ease), transform var(--dur-fast) var(--ease); +} +button:hover:not(:disabled), .btn:hover { background: var(--accent-hover); border-color: var(--accent-hover); color: var(--text-on-accent); } +button:active:not(:disabled), .btn:active { transform: translateY(1px); } +button:disabled { opacity: 0.45; cursor: not-allowed; } +button svg, .btn svg { width: 1rem; height: 1rem; flex: none; } + +button.secondary, .btn-secondary { + background: var(--surface); + color: var(--text); + border-color: var(--border-strong); +} +button.secondary:hover:not(:disabled), .btn-secondary:hover { + background: var(--surface-2); + color: var(--text); + border-color: var(--border-strong); +} + +button.danger { + background: transparent; + color: var(--danger); + border-color: color-mix(in srgb, var(--danger) 40%, transparent); +} +button.danger:hover:not(:disabled) { + background: var(--danger-soft); + color: var(--danger); + border-color: var(--danger); +} + +button.ghost { + background: transparent; + border-color: transparent; + color: var(--text-muted); +} +button.ghost:hover:not(:disabled) { background: var(--surface-2); color: var(--text); border-color: var(--border); } + +.btn-lg { min-height: 2.75rem; padding: var(--space-2) var(--space-5); font-size: var(--text-md); } + +/* --- forms ----------------------------------------------------------------- */ +form, .row { + display: flex; + gap: var(--space-2); + flex-wrap: wrap; + align-items: center; + margin: var(--space-4) 0 0; +} +form.stack-form { display: grid; gap: var(--space-3); margin: var(--space-4) 0 0; } +form.stack-form label { + display: grid; + gap: var(--space-1); + font-size: var(--text-xs); + font-weight: 600; + letter-spacing: 0.01em; + color: var(--text-muted); +} +form.stack-form label:has(> input[type="checkbox"]) { + display: flex; + align-items: center; + gap: var(--space-2); + min-height: 2.25rem; +} + +input, select, textarea { + font: inherit; + min-height: 2.25rem; + padding: var(--space-2) var(--space-3); + border: 1px solid var(--border-strong); + border-radius: var(--radius-md); + background: var(--surface); + color: var(--text); + transition: border-color var(--dur) var(--ease), box-shadow var(--dur) var(--ease); +} +input:hover, select:hover, textarea:hover { border-color: var(--text-faint); } +input:focus-visible, select:focus-visible, textarea:focus-visible { + outline: none; + border-color: var(--accent); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--accent) 22%, transparent); +} +input[type="checkbox"] { min-height: 0; width: 1rem; height: 1rem; accent-color: var(--accent); cursor: pointer; } +input[type="color"] { padding: 0.15rem; width: 3rem; cursor: pointer; } +input[type="search"] { min-width: 14rem; } +textarea { min-height: 5rem; resize: vertical; line-height: var(--lh-snug); } +select { cursor: pointer; } +::placeholder { color: var(--text-faint); } + +/* --- toolbar --------------------------------------------------------------- */ +.toolbar { + display: flex; + gap: var(--space-2); + flex-wrap: wrap; + align-items: center; + margin: var(--space-4) 0; +} +.grow { flex: 1 1 auto; } + +/* --- tables ---------------------------------------------------------------- */ +.scroll-x { + overflow-x: auto; + border: 1px solid var(--border); + border-radius: var(--radius-md); + background: var(--surface); +} +.scroll-x + .codeblock, .scroll-x + form { margin-top: var(--space-4); } + +table { width: 100%; border-collapse: collapse; font-size: var(--text-base); } +/* Inside a scroller a data table scrolls rather than wrapping every cell to four lines - which + is what a seven-column grid does in a half-width column. 38rem clears the widest grid in the + sample at desktop widths without scrolling; narrower viewports scroll the table, not the page. */ +.scroll-x > table { min-width: 38rem; } +.scroll-x td.mono, .scroll-x td:has(> .swatch) { white-space: nowrap; } +th, td { + text-align: left; + padding: var(--space-2) var(--space-3); + border-bottom: 1px solid var(--border-subtle); + vertical-align: middle; +} +thead th { + position: sticky; + top: 0; + background: var(--surface-2); + color: var(--text-muted); + font-family: var(--font-mono); + font-size: var(--text-2xs); + font-weight: 600; + letter-spacing: var(--track-wide); + text-transform: uppercase; + white-space: nowrap; + border-bottom: 1px solid var(--border); +} +tbody tr:last-child td { border-bottom: 0; } +tbody tr { transition: background var(--dur-fast) var(--ease); } +tr.clickable { cursor: pointer; } +tr.clickable:hover td { background: var(--surface-2); } +tr.selected td { background: var(--accent-wash); box-shadow: inset 2px 0 0 var(--accent); } +tr.selected:hover td { background: var(--accent-wash); } +td .actions { display: flex; gap: var(--space-1); justify-content: flex-end; } +.actions button { min-height: 1.85rem; padding: var(--space-1) var(--space-2); font-size: var(--text-xs); } +.type-table td:first-child { white-space: nowrap; } +.empty { color: var(--text-faint); font-style: italic; padding: var(--space-2) 0; } + +/* --- pills ----------------------------------------------------------------- */ +.pill { + display: inline-flex; + align-items: center; + gap: var(--space-1); + border-radius: var(--radius-full); + padding: 0.1rem 0.55rem; + background: var(--accent-soft); + color: var(--accent-ink); + font-size: var(--text-xs); + font-weight: 600; + line-height: 1.5; + white-space: nowrap; +} +.pill.warn { background: var(--warn-soft); color: var(--warn-ink); border: 0; } +.pill.danger { background: var(--danger-soft); color: var(--danger-ink); } +.pill.muted { background: var(--surface-3); color: var(--text-muted); } +.pill.ok { background: var(--ok-soft); color: var(--ok-ink); } +.pill svg { width: 0.8rem; height: 0.8rem; } + +.swatch { + display: inline-block; + width: 0.7rem; + height: 0.7rem; + border-radius: var(--radius-full); + vertical-align: -0.05rem; + margin-right: var(--space-2); + border: 1px solid var(--border-strong); +} + +/* --- code block ------------------------------------------------------------ */ +.codeblock { + border: 1px solid var(--code-border); + border-radius: var(--radius-md); + overflow: hidden; + background: var(--code-bg); + margin: var(--space-4) 0 0; +} +.codeblock + .codeblock { margin-top: var(--space-2); } +.codeblock-head { + display: flex; + align-items: center; + gap: var(--space-2); + padding: var(--space-1) var(--space-2) var(--space-1) var(--space-3); + min-height: 2.25rem; + background: var(--code-head); + border-bottom: 1px solid var(--code-border); + font-size: var(--text-xs); +} +.codeblock[data-open="false"] .codeblock-head { border-bottom: 0; } +.codeblock-lang { + flex: none; + font-family: var(--font-mono); + font-size: var(--text-2xs); + font-weight: 600; + letter-spacing: var(--track-wide); + text-transform: uppercase; + color: var(--accent); +} +.codeblock-title { + font-family: var(--font-mono); + font-size: var(--text-xs); + color: var(--text-muted); + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.codeblock-btn { + min-height: 1.85rem; + padding: var(--space-1) var(--space-2); + background: transparent; + border: 1px solid transparent; + border-radius: var(--radius-sm); + color: var(--text-muted); + font-size: var(--text-xs); + font-weight: 500; +} +.codeblock-btn:hover:not(:disabled) { + color: var(--text); + background: var(--surface); + border-color: var(--code-border); +} +.codeblock-btn.copied, .codeblock-btn.copied:hover:not(:disabled) { color: var(--ok); border-color: transparent; background: transparent; } +.codeblock-btn svg { width: 0.9rem; height: 0.9rem; } + +.codeblock pre { + margin: 0; + padding: var(--space-4); + overflow-x: auto; + font-family: var(--font-mono); + font-size: var(--text-sm); + line-height: 1.65; + color: var(--code-text); + tab-size: 4; +} +.codeblock pre code { + background: none; + border: 0; + padding: 0; + font-size: inherit; + color: inherit; + white-space: pre; +} + +.tok-com { color: var(--code-comment); font-style: italic; } +.tok-str { color: var(--code-string); } +.tok-key { color: var(--code-keyword); font-weight: 500; } +.tok-num { color: var(--code-number); } +.tok-typ { color: var(--code-type); } + +/* runtime output panes share the code surface */ +pre.output { + margin: var(--space-4) 0 0; + padding: var(--space-4); + background: var(--code-bg); + color: var(--code-text); + border: 1px solid var(--code-border); + border-radius: var(--radius-md); + overflow-x: auto; + white-space: pre-wrap; + font-family: var(--font-mono); + font-size: var(--text-sm); + line-height: 1.6; +} + +/* --- copyable command ------------------------------------------------------ */ +.command { + display: flex; + align-items: center; + gap: var(--space-3); + padding: var(--space-2) var(--space-2) var(--space-2) var(--space-4); + background: var(--code-bg); + border: 1px solid var(--code-border); + border-radius: var(--radius-md); + font-family: var(--font-mono); + font-size: var(--text-sm); + color: var(--code-text); + max-width: 34rem; +} +.command-prompt { color: var(--accent); user-select: none; flex: none; } +.command-text { overflow-x: auto; white-space: nowrap; flex: 1 1 auto; scrollbar-width: none; } +.command-text::-webkit-scrollbar { display: none; } + +/* --- lists ----------------------------------------------------------------- */ +ul.plain { margin: var(--space-3) 0 0; padding-left: var(--space-5); color: var(--text-muted); } +ul.plain li { margin: var(--space-1) 0; } + +ul.checks { list-style: none; margin: var(--space-3) 0 0; padding: 0; display: grid; gap: var(--space-2); } +ul.checks li { display: flex; gap: var(--space-3); align-items: baseline; color: var(--text-muted); } +ul.checks svg { width: 1rem; height: 1rem; flex: none; color: var(--accent); transform: translateY(0.15em); } + +/* --- generic layout helpers ------------------------------------------------- */ +.grid { display: grid; gap: var(--space-4); grid-template-columns: repeat(auto-fit, minmax(15rem, 1fr)); } +/* A seven-column CRUD grid and an editor rail do not both fit in 1000px: side by side, every + product name wrapped to five lines. The grid takes the full content width and the editor cards + sit under it, spreading out on wide screens. */ +.split { display: grid; gap: var(--space-4); grid-template-columns: minmax(0, 1fr); align-items: start; } +.split > .stack { grid-template-columns: repeat(auto-fit, minmax(20rem, 1fr)); align-items: start; } +.stack { display: grid; gap: var(--space-4); } +.stack-lg { display: grid; gap: var(--space-5); margin-top: var(--space-6); } + +@media (max-width: 700px) { + .split > .stack { grid-template-columns: minmax(0, 1fr); } +} + +/* --- stats ----------------------------------------------------------------- */ +.kpi { display: grid; gap: var(--space-1); align-content: start; } +.kpi strong { + font-size: var(--text-2xl); + font-weight: 600; + letter-spacing: var(--track-tighter); + font-variant-numeric: tabular-nums; + line-height: 1.1; +} +.kpi span { color: var(--text-muted); font-size: var(--text-sm); } + +/* --- session (storage) bar -------------------------------------------------- */ +.session-bar { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: var(--space-3) var(--space-5); + padding: var(--space-3) var(--space-4); + margin-bottom: var(--space-5); + background: var(--surface-2); + border: 1px solid var(--border); + border-radius: var(--radius-lg); +} +.session-id { display: flex; align-items: center; gap: var(--space-3); min-width: 0; } +.session-label { + font-family: var(--font-mono); + font-size: var(--text-2xs); + letter-spacing: var(--track-wide); + text-transform: uppercase; + color: var(--text-faint); + margin: 0; +} +.session-name { margin: 0; font-weight: 600; color: var(--text); line-height: 1.2; } +.session-blurb { + flex: 1 1 18rem; + min-width: 0; + margin: 0; + font-size: var(--text-xs); + color: var(--text-faint); + line-height: var(--lh-snug); +} +.session-controls { display: flex; align-items: flex-end; gap: var(--space-2); flex-wrap: wrap; } +.session-controls label { display: grid; gap: var(--space-1); font-size: var(--text-2xs); font-weight: 600; color: var(--text-muted); text-transform: uppercase; letter-spacing: var(--track-wide); } +.session-controls select { min-height: 2.25rem; } +.session-controls button { min-height: 2.25rem; } +.session-status { flex-basis: 100%; margin: 0; font-size: var(--text-sm); } +.session-status.ok { color: var(--ok); } +.session-status.failed { color: var(--danger); } + +.storage-dot { + flex: none; + width: 0.6rem; + height: 0.6rem; + border-radius: var(--radius-full); + background: var(--accent); + box-shadow: 0 0 0 4px color-mix(in srgb, var(--accent) 20%, transparent); +} +.storage-dot[data-name="indexeddb"] { background: #3b82f6; box-shadow: 0 0 0 4px color-mix(in srgb, #3b82f6 22%, transparent); } +.storage-dot[data-name="cache-storage"] { background: #d97706; box-shadow: 0 0 0 4px color-mix(in srgb, #d97706 22%, transparent); } +.storage-dot[data-name="in-memory"] { background: var(--text-faint); box-shadow: 0 0 0 4px color-mix(in srgb, var(--text-faint) 22%, transparent); } + +/* --- live indicator --------------------------------------------------------- */ +.live-dot { + display: inline-block; + width: 0.45rem; + height: 0.45rem; + border-radius: var(--radius-full); + background: var(--ok); + animation: pulse 2.4s var(--ease) infinite; +} +@keyframes pulse { + 0%, 100% { opacity: 1; box-shadow: 0 0 0 0 color-mix(in srgb, var(--ok) 45%, transparent); } + 50% { opacity: 0.75; box-shadow: 0 0 0 4px color-mix(in srgb, var(--ok) 0%, transparent); } +} + +/* --- boot splash ------------------------------------------------------------ */ +.boot { + min-height: 100dvh; + display: grid; + place-content: center; + justify-items: center; + gap: var(--space-4); + color: var(--text-muted); + font-size: var(--text-sm); +} +.boot-mark { + width: 2.5rem; + height: 2.5rem; + display: grid; + place-items: center; + border-radius: var(--radius-lg); + background: var(--accent); + color: var(--text-on-accent); +} +.boot-mark svg { width: 1.4rem; height: 1.4rem; } +.boot-bar { + width: 10rem; + height: 2px; + border-radius: var(--radius-full); + background: var(--border); + overflow: hidden; +} +.boot-bar::after { + content: ""; + display: block; + width: 40%; + height: 100%; + background: var(--accent); + animation: slide 1.4s var(--ease) infinite; +} +@keyframes slide { + 0% { transform: translateX(-100%); } + 100% { transform: translateX(350%); } +} diff --git a/src/Bsqlite/Demo/Server/wwwroot/css/landing.css b/src/Bsqlite/Demo/Server/wwwroot/css/landing.css new file mode 100644 index 00000000000..c5e4bce9cce --- /dev/null +++ b/src/Bsqlite/Demo/Server/wwwroot/css/landing.css @@ -0,0 +1,291 @@ +/* ============================================================================= + Bsqlite - landing page + Section order follows the open-source / developer-tool pattern: + hero -> install -> live proof -> capability -> quick start -> demos -> limits + ========================================================================== */ + +.landing { padding-bottom: var(--space-8); } + +/* --- hero ------------------------------------------------------------------- */ +.hero { + position: relative; + overflow: hidden; + border-bottom: 1px solid var(--border-subtle); + padding: var(--space-8) 0 var(--space-7); +} +.hero::before { + content: ""; + position: absolute; + inset: 0; + background: + radial-gradient(58rem 26rem at 12% -18%, var(--hero-glow-a) 0%, transparent 62%), + radial-gradient(48rem 24rem at 88% -12%, var(--hero-glow-b) 0%, transparent 60%); + pointer-events: none; +} +.hero::after { + content: ""; + position: absolute; + inset: 0; + background-image: + linear-gradient(to right, var(--grid-line) 1px, transparent 1px), + linear-gradient(to bottom, var(--grid-line) 1px, transparent 1px); + background-size: 3.5rem 3.5rem; + mask-image: radial-gradient(75% 70% at 28% 0%, #000 0%, transparent 72%); + pointer-events: none; +} +.hero-inner { + position: relative; + max-width: var(--page-max); + margin: 0 auto; + padding: 0 var(--space-4); + display: grid; + grid-template-columns: minmax(0, 1.05fr) minmax(0, 1fr); + gap: var(--space-7); + align-items: center; +} + +.hero-badges { display: flex; flex-wrap: wrap; gap: var(--space-2); margin-bottom: var(--space-4); } +.badge { + display: inline-flex; + align-items: center; + gap: var(--space-2); + padding: 0.15rem 0.6rem; + border: 1px solid var(--border); + border-radius: var(--radius-full); + background: var(--surface); + font-family: var(--font-mono); + font-size: var(--text-2xs); + letter-spacing: 0.03em; + color: var(--text-muted); + text-decoration: none; +} +.badge svg { width: 0.8rem; height: 0.8rem; } +a.badge:hover { color: var(--text); border-color: var(--border-strong); } + +.hero h1 { + font-size: clamp(2rem, 1.25rem + 3vw, var(--text-4xl)); + line-height: 1.05; + letter-spacing: -0.04em; + margin: 0 0 var(--space-4); + max-width: 18ch; +} +.hero h1 em { font-style: normal; color: var(--accent); } +.hero-lead { + font-size: var(--text-lg); + line-height: 1.5; + color: var(--text-muted); + max-width: 40rem; + margin: 0 0 var(--space-5); +} +.hero-actions { display: flex; flex-wrap: wrap; gap: var(--space-3); align-items: center; margin-bottom: var(--space-5); } +.hero-note { font-size: var(--text-xs); color: var(--text-faint); margin: 0; } + +/* --- hero code panel --------------------------------------------------------- */ +.hero-panel { + background: var(--code-bg); + border: 1px solid var(--code-border); + border-radius: var(--radius-xl); + box-shadow: var(--shadow-lg); + overflow: hidden; +} +.hero-panel-head { + display: flex; + align-items: center; + gap: var(--space-2); + padding: var(--space-3) var(--space-4); + background: var(--code-head); + border-bottom: 1px solid var(--code-border); +} +.hero-panel-dots { display: flex; gap: 0.3rem; } +.hero-panel-dots i { + width: 0.55rem; + height: 0.55rem; + border-radius: var(--radius-full); + background: var(--border-strong); + display: block; +} +.hero-panel-file { + font-family: var(--font-mono); + font-size: var(--text-xs); + color: var(--text-muted); + margin-left: var(--space-2); +} +.hero-panel pre { + margin: 0; + font-variant-ligatures: none; + padding: var(--space-5); + overflow-x: auto; + font-family: var(--font-mono); + font-size: var(--text-sm); + line-height: 1.7; + color: var(--code-text); + tab-size: 4; +} +.hero-panel pre code { background: none; border: 0; padding: 0; white-space: pre; } + +/* --- stat strip -------------------------------------------------------------- */ +.stat-strip { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(11rem, 1fr)); + gap: 1px; + background: var(--border); + border: 1px solid var(--border); + border-radius: var(--radius-lg); + overflow: hidden; + margin-top: var(--space-6); +} +.stat { + background: var(--surface); + padding: var(--space-4) var(--space-5); + display: grid; + gap: var(--space-1); + align-content: start; +} +.stat-value { + font-size: var(--text-2xl); + font-weight: 600; + letter-spacing: var(--track-tighter); + font-variant-numeric: tabular-nums; + line-height: 1.1; + color: var(--text); +} +.stat-value.small { font-size: var(--text-lg); } +.stat-label { font-size: var(--text-xs); color: var(--text-muted); display: flex; align-items: center; gap: var(--space-2); } + +/* --- feature grid ------------------------------------------------------------ */ +.feature-grid { display: grid; gap: var(--space-4); grid-template-columns: repeat(3, minmax(0, 1fr)); } +@media (max-width: 1100px) { .feature-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } } +@media (max-width: 700px) { .feature-grid { grid-template-columns: minmax(0, 1fr); } } +.feature { + padding: var(--space-5); + background: var(--surface); + border: 1px solid var(--border); + border-radius: var(--radius-lg); + transition: border-color var(--dur) var(--ease), box-shadow var(--dur) var(--ease); +} +.feature:hover { border-color: var(--border-strong); box-shadow: var(--shadow-sm); } +.feature-icon { + display: grid; + place-items: center; + width: 2.25rem; + height: 2.25rem; + border-radius: var(--radius-md); + background: var(--accent-wash); + color: var(--accent); + border: 1px solid var(--accent-soft); + margin-bottom: var(--space-3); +} +.feature-icon svg { width: 1.15rem; height: 1.15rem; } +.feature h3 { margin-bottom: var(--space-2); } +.feature p { font-size: var(--text-base); margin: 0; } + +/* --- steps ------------------------------------------------------------------- */ +.steps { display: grid; gap: var(--space-5); counter-reset: step; } +.step { + display: grid; + grid-template-columns: 2rem minmax(0, 1fr); + gap: var(--space-4); + align-items: start; +} +.step-num { + counter-increment: step; + display: grid; + place-items: center; + width: 2rem; + height: 2rem; + border-radius: var(--radius-full); + border: 1px solid var(--border); + background: var(--surface); + font-family: var(--font-mono); + font-size: var(--text-xs); + font-weight: 600; + color: var(--accent); +} +.step-num::before { content: counter(step); } +.step-body { min-width: 0; } +.step-body h3 { margin-bottom: var(--space-2); } +.step-body > p { max-width: var(--measure); } + +/* --- demo link cards ---------------------------------------------------------- */ +.link-grid { display: grid; gap: var(--space-3); grid-template-columns: repeat(auto-fit, minmax(19rem, 1fr)); } +.link-card { + display: grid; + grid-template-columns: 2.25rem minmax(0, 1fr) 1rem; + gap: var(--space-4); + align-items: start; + padding: var(--space-4); + background: var(--surface); + border: 1px solid var(--border); + border-radius: var(--radius-lg); + text-decoration: none; + color: var(--text); + transition: border-color var(--dur) var(--ease), background var(--dur) var(--ease), transform var(--dur) var(--ease); +} +.link-card:hover { + color: var(--text); + border-color: var(--accent); + background: var(--accent-wash); + transform: translateY(-1px); +} +.link-card-icon { + display: grid; + place-items: center; + width: 2.25rem; + height: 2.25rem; + border-radius: var(--radius-md); + background: var(--surface-2); + color: var(--text-muted); + border: 1px solid var(--border); +} +.link-card:hover .link-card-icon { color: var(--accent); border-color: var(--accent-soft); } +.link-card-icon svg { width: 1.05rem; height: 1.05rem; } +.link-card strong { display: block; font-size: var(--text-md); margin-bottom: var(--space-1); } +.link-card span { display: block; font-size: var(--text-sm); color: var(--text-muted); line-height: var(--lh-snug); } +.link-card-go { color: var(--text-faint); transition: transform var(--dur) var(--ease); align-self: center; } +.link-card:hover .link-card-go { color: var(--accent); transform: translateX(2px); } +.link-card-go svg { width: 1rem; height: 1rem; display: block; } + +/* --- comparison / matrix table ------------------------------------------------ */ +.matrix { border: 1px solid var(--border); border-radius: var(--radius-lg); overflow: hidden; background: var(--surface); } +.matrix table { font-size: var(--text-sm); } +.matrix th:first-child, .matrix td:first-child { font-weight: 600; color: var(--text); white-space: nowrap; } +.matrix td { color: var(--text-muted); } + +/* --- honest limits list --------------------------------------------------------- */ +.limits-list { display: grid; gap: var(--space-3); } +.limit-item { + display: grid; + grid-template-columns: 1.15rem minmax(0, 1fr); + gap: var(--space-3); + align-items: start; + padding: var(--space-4); + border: 1px solid var(--border); + border-radius: var(--radius-md); + background: var(--surface); +} +.limit-item svg { width: 1.05rem; height: 1.05rem; color: var(--warn); transform: translateY(0.15em); } +.limit-item strong { display: block; margin-bottom: var(--space-1); } +.limit-item p { margin: 0; font-size: var(--text-sm); } + +/* --- closing call to action ------------------------------------------------------ */ +.closing { + margin-top: var(--space-8); + padding: var(--space-7) var(--space-5); + text-align: center; + border: 1px solid var(--border); + border-radius: var(--radius-xl); + background: var(--surface-2); +} +.closing h2 { font-size: var(--text-xl); margin-bottom: var(--space-3); } +.closing p { max-width: 34rem; margin: 0 auto var(--space-5); } +.closing .hero-actions { justify-content: center; margin-bottom: 0; } + +/* --- responsive ------------------------------------------------------------------ */ +@media (max-width: 1023px) { + .hero { padding: var(--space-7) 0 var(--space-6); } + .hero-inner { grid-template-columns: minmax(0, 1fr); gap: var(--space-6); } +} +@media (max-width: 560px) { + .hero-actions .btn, .hero-actions button { width: 100%; } + .command { max-width: none; } +} diff --git a/src/Bsqlite/Demo/Server/wwwroot/css/tokens.css b/src/Bsqlite/Demo/Server/wwwroot/css/tokens.css new file mode 100644 index 00000000000..b3be5cbf225 --- /dev/null +++ b/src/Bsqlite/Demo/Server/wwwroot/css/tokens.css @@ -0,0 +1,243 @@ +/* ============================================================================= + Bsqlite - design tokens + Layer 1 primitives -> layer 2 semantics. Components never reach for a raw hex. + + Direction: Minimalism & Swiss Style, developer-tool register (see + samples/design-system/bsqlite-demo/MASTER.md). Every foreground / + background pair below is >= 4.5:1 in both themes. + ========================================================================== */ + +:root { + /* --- layer 1: primitives ------------------------------------------- */ + --ink-950: #0b1220; + --ink-900: #131c2b; + --ink-800: #1e2a3d; + --ink-700: #2c3a51; + --ink-600: #54637a; + --ink-500: #68788f; + --ink-400: #93a2b7; + --ink-300: #c3cddb; + --ink-200: #dfe5ee; + --ink-100: #eef2f7; + --ink-050: #f7f9fc; + --white: #ffffff; + + --night-950: #070a0f; + --night-900: #0a0e14; + --night-850: #0e141d; + --night-800: #111823; + --night-750: #16202d; + --night-700: #1c2836; + --night-600: #263447; + + --teal-700: #0f766e; + --teal-600: #12857c; + --teal-500: #14b8a6; + --teal-400: #2dd4bf; + --teal-300: #5eead4; + --teal-100: #dcf5f0; + --teal-050: #f0faf8; + --teal-950: #04211e; + --teal-900: #08302c; + --teal-850: #0d3f39; + + --red-700: #b42318; + --red-300: #ff8a80; + --red-100: #fdecea; + --red-900: #3a1512; + + --amber-800: #7a4a03; + --amber-400: #fbbf24; + --amber-100: #fff7e6; + --amber-900: #3a2a08; + + --green-800: #0a5c48; + --green-400: #34d399; + --green-100: #e6f6f1; + --green-900: #0a2b23; + + --violet-600: #7c3aed; + --violet-300: #c4b5fd; + --blue-700: #0369a1; + --blue-300: #7dd3fc; + --orange-700: #b45309; + + /* --- scale: type ---------------------------------------------------- */ + --text-2xs: 0.6875rem; /* 11 - eyebrows only, uppercase + tracked */ + --text-xs: 0.75rem; /* 12 */ + --text-sm: 0.8125rem; /* 13 */ + --text-base: 0.9375rem; /* 15 - UI default */ + --text-md: 1rem; /* 16 - prose */ + --text-lg: 1.125rem; /* 18 */ + --text-xl: 1.375rem; /* 22 */ + --text-2xl: 1.75rem; /* 28 */ + --text-3xl: 2.25rem; /* 36 */ + --text-4xl: 3rem; /* 48 */ + + --lh-tight: 1.15; + --lh-snug: 1.35; + --lh-normal: 1.6; + + --track-wide: 0.09em; + --track-tight: -0.02em; + --track-tighter: -0.035em; + + --font-sans: "IBM Plex Sans", "Segoe UI Variable Text", "Segoe UI", system-ui, -apple-system, sans-serif; + --font-mono: "JetBrains Mono", ui-monospace, "Cascadia Code", "SFMono-Regular", Consolas, monospace; + + /* --- scale: space (density 6/10 - standard) -------------------------- */ + --space-1: 0.25rem; + --space-2: 0.5rem; + --space-3: 0.75rem; + --space-4: 1rem; + --space-5: 1.5rem; + --space-6: 2rem; + --space-7: 3rem; + --space-8: 4rem; + --space-9: 6rem; + + /* --- scale: radius / elevation --------------------------------------- */ + --radius-sm: 0.375rem; + --radius-md: 0.5rem; + --radius-lg: 0.75rem; + --radius-xl: 1rem; + --radius-full: 999px; + + /* --- motion (dial 3/10 - subtle) ------------------------------------- */ + --ease: cubic-bezier(0.2, 0, 0.15, 1); + --dur-fast: 120ms; + --dur: 180ms; + --dur-slow: 260ms; + + /* --- layout ----------------------------------------------------------- */ + --topbar-h: 3.5rem; + --sidebar-w: 16.5rem; + --measure: 46rem; + --page-max: 84rem; +} + +/* --- layer 2: semantics - light ------------------------------------------ */ +:root { + color-scheme: light; + + --canvas: var(--white); + --canvas-sunken: var(--ink-050); + --surface: var(--white); + --surface-2: var(--ink-050); + --surface-3: var(--ink-100); + --surface-inset: var(--ink-050); + --overlay: rgba(11, 18, 32, 0.45); + + --text: var(--ink-950); + --text-muted: var(--ink-600); + --text-faint: var(--ink-500); + --text-on-accent: var(--white); + + --border: var(--ink-200); + --border-strong: var(--ink-300); + --border-subtle: var(--ink-100); + + --accent: var(--teal-700); + --accent-hover: var(--teal-600); + --accent-ink: #115e56; + --accent-soft: var(--teal-100); + --accent-wash: var(--teal-050); + + --danger: var(--red-700); + --danger-soft: var(--red-100); + --danger-ink: var(--red-700); + + --warn: var(--amber-800); + --warn-soft: var(--amber-100); + --warn-ink: var(--amber-800); + + --ok: var(--green-800); + --ok-soft: var(--green-100); + --ok-ink: var(--green-800); + + --focus-ring: var(--teal-700); + + --shadow-xs: 0 1px 2px rgba(11, 18, 32, 0.06); + --shadow-sm: 0 1px 2px rgba(11, 18, 32, 0.05), 0 1px 1px rgba(11, 18, 32, 0.04); + --shadow-md: 0 4px 12px -4px rgba(11, 18, 32, 0.12), 0 2px 4px -2px rgba(11, 18, 32, 0.06); + --shadow-lg: 0 24px 48px -24px rgba(11, 18, 32, 0.28), 0 8px 16px -12px rgba(11, 18, 32, 0.12); + + --code-bg: var(--ink-050); + --code-head: var(--ink-100); + --code-border: var(--ink-200); + --code-text: var(--ink-900); + --code-comment: var(--ink-600); + --code-keyword: var(--violet-600); + --code-string: var(--orange-700); + --code-number: var(--blue-700); + --code-type: var(--teal-700); + + --grid-line: rgba(11, 18, 32, 0.05); + --hero-glow-a: rgba(20, 184, 166, 0.16); + --hero-glow-b: rgba(3, 105, 161, 0.09); +} + +/* --- layer 2: semantics - dark ------------------------------------------- + Only one dark block exists. The boot script in index.html resolves "system" + to a concrete light/dark value and stamps data-theme before first paint, so + no prefers-color-scheme duplicate of this table is needed. + ------------------------------------------------------------------------ */ +:root[data-theme="dark"] { + color-scheme: dark; + + --canvas: var(--night-900); + --canvas-sunken: var(--night-950); + --surface: var(--night-800); + --surface-2: var(--night-750); + --surface-3: var(--night-700); + --surface-inset: var(--night-850); + --overlay: rgba(3, 5, 8, 0.66); + + --text: #e6edf6; + --text-muted: var(--ink-400); + --text-faint: #7c8ca3; + --text-on-accent: var(--teal-950); + + --border: var(--night-700); + --border-strong: var(--night-600); + --border-subtle: var(--night-750); + + --accent: var(--teal-400); + --accent-hover: var(--teal-300); + --accent-ink: var(--teal-300); + --accent-soft: var(--teal-850); + --accent-wash: var(--teal-900); + + --danger: var(--red-300); + --danger-soft: var(--red-900); + --danger-ink: var(--red-300); + + --warn: var(--amber-400); + --warn-soft: var(--amber-900); + --warn-ink: var(--amber-400); + + --ok: var(--green-400); + --ok-soft: var(--green-900); + --ok-ink: var(--green-400); + + --focus-ring: var(--teal-400); + + --shadow-xs: 0 1px 2px rgba(0, 0, 0, 0.4); + --shadow-sm: 0 1px 2px rgba(0, 0, 0, 0.45); + --shadow-md: 0 4px 14px -6px rgba(0, 0, 0, 0.6); + --shadow-lg: 0 24px 48px -24px rgba(0, 0, 0, 0.8), 0 8px 16px -12px rgba(0, 0, 0, 0.6); + + --code-bg: var(--night-850); + --code-head: var(--night-750); + --code-border: var(--night-700); + --code-text: #dbe4f0; + --code-comment: #8496ad; + --code-keyword: var(--violet-300); + --code-string: var(--amber-400); + --code-number: var(--blue-300); + --code-type: var(--teal-300); + + --grid-line: rgba(255, 255, 255, 0.04); + --hero-glow-a: rgba(45, 212, 191, 0.13); + --hero-glow-b: rgba(56, 130, 246, 0.08); +} diff --git a/src/Bsqlite/Demo/Server/wwwroot/js/demo.js b/src/Bsqlite/Demo/Server/wwwroot/js/demo.js new file mode 100644 index 00000000000..6b8697e025a --- /dev/null +++ b/src/Bsqlite/Demo/Server/wwwroot/js/demo.js @@ -0,0 +1,66 @@ +window.bsqliteDemo = { + download(filename, bytes) { + const blob = new Blob([bytes], { type: "application/vnd.sqlite3" }); + const url = URL.createObjectURL(blob); + const link = document.createElement("a"); + link.href = url; + link.download = filename; + link.click(); + URL.revokeObjectURL(url); + }, + + async copy(text) { + try { + await navigator.clipboard.writeText(text); + return true; + } catch { + // Clipboard API needs a secure context and permission; fall back to a hidden textarea. + const area = document.createElement("textarea"); + area.value = text; + area.setAttribute("readonly", ""); + area.style.position = "fixed"; + area.style.opacity = "0"; + document.body.appendChild(area); + area.select(); + const ok = document.execCommand("copy"); + document.body.removeChild(area); + return ok; + } + }, + + /** + * The boot script in index.html owns the resolution rules so that the first paint is already + * correct; this object only reads and writes the preference, then asks it to re-apply. + */ + theme: { + /** "light" | "dark" | "system" - what the reader chose, not what is painted. */ + get() { + return window.__bsqliteTheme.preference(); + }, + + /** Stores the preference and returns what is actually painted now: "light" or "dark". */ + set(mode) { + const key = window.__bsqliteTheme.key; + try { + if (mode === "light" || mode === "dark") { + localStorage.setItem(key, mode); + } else { + localStorage.removeItem(key); + } + } catch { + // Storage blocked: the choice still applies to this page, it just will not survive a reload. + } + return window.__bsqliteTheme.apply(); + }, + + /** What the page is painting right now: "light" or "dark". */ + resolved() { + return document.documentElement.getAttribute("data-theme") === "dark" ? "dark" : "light"; + }, + }, + + /** Keeps the page behind the mobile navigation drawer from scrolling with it. */ + lockScroll(locked) { + document.body.style.overflow = locked ? "hidden" : ""; + }, +}; diff --git a/src/Bsqlite/README.md b/src/Bsqlite/README.md new file mode 100644 index 00000000000..5f616ad537c --- /dev/null +++ b/src/Bsqlite/README.md @@ -0,0 +1,207 @@ +# bit Bsqlite + +SQLite in the browser for Blazor, with EF Core on top. The same entities and the same migrations run on the client as +on the server, and the database lives in the user's browser tab - in OPFS, IndexedDB, Cache Storage or memory. + +```csharp +// Program.cs +builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider<BsqliteOpfsStorageProvider>() + .AddStorageProvider<BsqliteIndexedDbStorageProvider>() + .UseStorage(s => s.Prefer("opfs").Fallback("indexeddb")) + .AddDatabase("app.db")); + +builder.Services.AddBitBsqliteDbContext<AppDbContext>(); + +// A component +@inject IBsqliteDbContextFactory<AppDbContext> Contexts + +await using var db = await Contexts.CreateDbContextAsync(); +var open = await db.Orders.Where(o => o.Status == OrderStatus.Open).ToListAsync(); +``` + +That query runs in a dedicated worker in the browser: SQLite 3.53 (via [wa-sqlite](https://github.com/rhashimoto/wa-sqlite) +1.1.2), the stock EF Core SQLite provider, and a storage backend chosen and remembered per database. + +> Targets **.NET 8, 9, and 10** with the matching EF Core · Runs on **Blazor WebAssembly**, and in a **Blazor Web App** on +> **Interactive WebAssembly** or **Interactive Auto** once the page runs on WebAssembly. Prerendered pages render on the +> server and open their database once interactive. + +Bsqlite's documentation website lives in [`Demo`](https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite/Demo/) - run the +`Demo/Server` project, or visit [bsqlite.bitplatform.dev](https://bsqlite.bitplatform.dev). + +--- + +## Table of Contents + +- [Why Bsqlite](#why-bsqlite) +- [Packages](#packages) +- [Installation](#installation) +- [Composition without dependency injection](#composition-without-dependency-injection) +- [Live queries](#live-queries) +- [Where it runs](#where-it-runs) +- [What this release does not claim](#what-this-release-does-not-claim) +- [Moving off Bit.Besql](#moving-off-bitbesql) +- [MCP server (for AI agents)](#mcp-server-for-ai-agents) +- [Tests](#tests) +- [Documentation](#documentation) +- [License](#license) + +--- + +## Why Bsqlite + +- **EF Core, unmodified.** `UseBitBsqlite` is the stock SQLite provider over a Bsqlite connection. Migrations scaffolded + against the server apply in the browser. Decimal arithmetic, `REGEXP`, and the `EF_DECIMAL` collation are provided in + the worker so query results match the server's. +- **Fast interop.** SQL batches and result sets cross the worker and Blazor boundaries as one binary buffer each, + transferred rather than copied. No JSON, no base64, 64-bit integers intact. Prepared statements are cached in the + worker, so EF's repeated statement texts are parsed once. +- **Honest storage.** Selection never substitutes a backend silently: existing data outranks preference, a + non-persistent fallback needs an opt-in, and a failed open names every candidate and why it was rejected. + Cross-backend migration verifies the copy before flipping the binding. +- **Live queries.** A write re-runs any live query that reads the table, including writes from another tab. Change + tracking comes from the engine's update hook, so triggers and cascades count too. +- **Complete ADO.NET.** Commands, batches (`DbBatch`), transactions with savepoints and rollback-on-dispose, + multi-result readers, typed exceptions by SQLite result code, and in-place export of an open database. + +## Packages + +| Package | Role | +| --- | --- | +| `Bit.Bsqlite` | `AddBitBsqliteServices`, `AddBitBsqliteDbContext`, `UseBitBsqlite(connection)`, ADO.NET and live queries, the worker transport, storage selection and migration, the storage provider contract, the in-memory backend, and the browser assets (engine, worker, host) under `_content/Bit.Bsqlite` | +| `Bit.Bsqlite.Storage.Opfs` | Persistent, synchronous engine build, several connections | +| `Bit.Bsqlite.Storage.IndexedDb` | Persistent, JSPI or Asyncify engine build, widest reach | +| `Bit.Bsqlite.Storage.CacheStorage` | Persistent, and the migration path off `Bit.Besql` | +| `Bit.Bsqlite.Testing` | An in-process transport for browser-free unit tests, plus the MSTest conformance kit every storage provider must pass | + +## Installation + +```bash +dotnet add package Bit.Bsqlite +dotnet add package Bit.Bsqlite.Storage.Opfs +dotnet add package Bit.Bsqlite.Storage.IndexedDb +``` + +Register the services in the project that runs on WebAssembly - and, in a Blazor Web App whose pages are prerendered, +in the server project too, so the prerender pass can resolve what the pages inject: + +```csharp +using Bit.Bsqlite.Storage.IndexedDb; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Storage.Opfs; + +builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider<BsqliteOpfsStorageProvider>() + .AddStorageProvider<BsqliteIndexedDbStorageProvider>() + .AddStorageProvider<BsqliteInMemoryStorageProvider>() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()) + .AddDatabase("app.db")); + +builder.Services.AddBitBsqliteDbContext<AppDbContext>(); +``` + +The engine (`.wasm`), the worker and the VFSes are static web assets under `_content/Bit.Bsqlite*/`. Nothing needs to be +added to the host page. + +## Composition without dependency injection + +`AddBitBsqliteServices` registers everything above. Without dependency injection, the same pieces compose by hand: + +```csharp +var session = await factory.OpenAsync("app.db"); // a BsqliteSessionFactory +options.UseBitBsqlite(session.Connection); +``` + +Selection is sticky: existing data outranks preference. `BsqliteStorageMigrationMode.AutomaticOnOpen` copies, checks the +SQLite header, then flips the binding. `KeepExisting` is the default. + +## Live queries + +Table-level. A committed write re-runs any live query that reads that table, including across tabs. The worker learns +what changed from SQLite's own update hook - cascades and triggers included - and reports it once the transaction +commits, so a rolled-back write is never shown and a multi-statement `SaveChangesAsync` re-runs a query once, not per +statement. + +```csharp +await using var live = ctx.Orders.Where(o => o.Open).AsLiveQuery(); +live.Changed += (_, rows) => InvokeAsync(StateHasChanged); +``` + +## Where it runs + +| Host | Support | +| --- | --- | +| Blazor WebAssembly | Everything | +| Blazor Web App, Interactive WebAssembly | Everything | +| Blazor Web App, Interactive Auto | Everything once the page runs on WebAssembly | +| Prerendering | Register the services on the server as well; they resolve there, and the database opens once the page runs in the browser (`RendererInfo.IsInteractive` on Interactive WebAssembly, `OperatingSystem.IsBrowser()` on Interactive Auto) | +| Interactive Server | Not supported: the engine runs in the user's tab, and EF Core on the server would need a native SQLite of its own | + +Every service that reaches the browser, or holds a database session, is registered as scoped: one for the whole +application on WebAssembly, and one per circuit or request on a server, so nothing is ever shared between users. + +EF Core's own feature set follows the target framework. The SQLite provider translates `SUM` over a `decimal` from EF +Core 9 on and `ORDER BY` a `decimal` from EF Core 10 on; an application on .NET 8 gets the same `NotSupportedException` +from those queries in the browser as from the stock provider on its server. + +## What this release does not claim + +- A synchronous tier (`SaveChanges()`, `ToList()`). Synchronous members throw + `BsqliteSynchronousApiNotSupportedException` naming the async overload. +- WAL journal mode. No web VFS can provide the shared memory it needs. +- Server synchronization. The demo's two sides share a model, not data. +- A synchronous worker bridge. Browsers forbid `Atomics.wait` on the main thread, a spin-wait would freeze the UI, and + multithreaded WebAssembly is not Blazor's default. +- `in-memory` has no admin view of live data. It registers no VFS, so a database opened on it lives in the engine's own + heap inside the worker; `Admin` describes only images passed to it directly. Export, list, and migration away from + this backend do not see what a connection wrote. Persistent backends do not have this gap. +- `indexeddb` serializes readers. `IDBBatchAtomicVFS` is registered with `WebLocksMixin`'s default `exclusive` lock + policy, so a read holds the same lock a write does. `SupportsConcurrentReads` reports that honestly. + +## Moving off Bit.Besql + +[Bit.Besql](https://github.com/bitfoundation/bitplatform/tree/develop/src/Besql) keeps a whole SQLite file in Cache Storage +and syncs it from Blazor's in-memory file system. `Bit.Bsqlite.Storage.CacheStorage` reads that file: a database +Bit.Besql left in its `bit-Besql` cache is imported losslessly into Bsqlite's own page layout the first time it is +opened, so an application that switches keeps its users' data. + +## MCP server (for AI agents) + +The demo website hosts a [Model Context Protocol](https://modelcontextprotocol.io) server at +`https://bsqlite.bitplatform.dev/mcp`, and it is also served from the bit platform's combined endpoint at +`https://bitplatform.dev/mcp`. Point your coding agent at either to let it search the Bsqlite API, read the guides and +the demo's source, and plan a storage setup. See the MCP server page of the demo website for the full tool list. + +## Tests + +The wa-sqlite engine and the upstream VFS files are vendored at restore, verified against the checksums in the +`*.lock.props` files, and git-ignored, so restore before building. + +```bash +cd src/Bsqlite/Tests/Bit.Bsqlite.Tests && dotnet test # ADO.NET, EF Core and storage selection, every target framework +cd src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E && dotnet test # the worker, the engine and every VFS in a real browser, plus the demo +cd src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp && dotnet test # the MCP server, hosted in-memory +cd src/Bsqlite/Tests/bit-bsqlite-js && npm ci && npm test # the JavaScript that needs no browser +``` + +See [Tests/Bit.Bsqlite.Tests.E2E/README.md](https://github.com/bitfoundation/bitplatform/blob/develop/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/README.md) +for the browser matrix, the soak suite and the benchmark page. + +## Documentation + +- [Getting started](https://github.com/bitfoundation/bitplatform/blob/develop/src/Bsqlite/docs/getting-started.md) - registration, contexts, raw ADO.NET, the shared database +- [Storage](https://github.com/bitfoundation/bitplatform/blob/develop/src/Bsqlite/docs/storage.md) - backends, selection rules, migration, admin, writing a provider +- [Live queries](https://github.com/bitfoundation/bitplatform/blob/develop/src/Bsqlite/docs/live-queries.md) - EF and ADO.NET live queries, cross-tab notifications +- [Architecture and performance](https://github.com/bitfoundation/bitplatform/blob/develop/src/Bsqlite/docs/architecture.md) - the worker, the binary wire format, the statement cache, change tracking +- [Limits](https://github.com/bitfoundation/bitplatform/blob/develop/src/Bsqlite/docs/limits.md) - what the browser cannot do and how the library tells you +- [Extending](https://github.com/bitfoundation/bitplatform/blob/develop/src/Bsqlite/docs/extending.md) - SQL functions from JavaScript, custom transports + +## License + +MIT, as is the bundled wa-sqlite, whose license ships next to the engine under `_content/Bit.Bsqlite/engine`. See +[LICENSE](https://github.com/bitfoundation/bitplatform/blob/develop/LICENSE). diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/AssemblySetup.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/AssemblySetup.cs new file mode 100644 index 00000000000..d0cfa053041 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/AssemblySetup.cs @@ -0,0 +1,32 @@ +using Bit.Bsqlite.Tests.E2E.Infrastructure; +using Microsoft.Playwright; + +// The tests run one at a time. Many of them share an origin's storage on purpose (two tabs on one OPFS +// file, a worker killed mid-transaction and reopened), every one of them boots a worker and a WebAssembly +// engine, and a second test doing the same next to it is exactly the load that turns a wait into a flake +// on a two-core runner. +[assembly: DoNotParallelize] + +namespace Bit.Bsqlite.Tests.E2E; + +[TestClass] +public static class AssemblySetup +{ + [AssemblyInitialize] + public static async Task InitializeAsync(TestContext context) + { + // Instantiating a WebAssembly engine in a fresh worker routinely takes longer than Playwright's + // 5 second default on a CI runner, without anything being wrong. + Assertions.SetDefaultExpectTimeout(20_000); + + // Started here rather than by the first test, so the boot is not charged to that test's time. Nothing + // throws: a browser or assets that are missing are reported by every test (see BsqliteTest). + await PlaywrightSession.GetAsync(); + } + + [AssemblyCleanup] + public static async Task CleanupAsync() + { + await PlaywrightSession.StopAsync(); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/BenchmarkTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/BenchmarkTests.cs new file mode 100644 index 00000000000..713ae0b9ed0 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/BenchmarkTests.cs @@ -0,0 +1,41 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// <summary> +/// Runs TestPages/benchmarks.html per backend and prints what it measured: cold open after engine init, a +/// 100-row insert batch in one round trip, 1 000 repeated parameterised inserts inside one transaction (the +/// statement cache), a 1 000-row read on the binary path and on the tagged JavaScript path, and an indexed +/// point read. +/// </summary> +/// <remarks> +/// The numbers are for a human comparing backends and encodings on one machine, not for a red build, so +/// nothing is asserted about them and the tests only run with <c>BSQLITE_E2E_BENCHMARKS=1</c>. The page also +/// works by hand: its console exposes <c>await run('opfs')</c> and the rest. +/// </remarks> +[TestClass] +public sealed class BenchmarkTests : BsqliteTest +{ + [TestMethod] + [TestCategory("Benchmark")] + [DataRow("memory")] + [DataRow("opfs")] + [DataRow("indexeddb")] + [DataRow("cache")] + public async Task Benchmark(string backend) + { + SkipWhen(E2EEnvironment.Benchmarks is false, "the benchmarks are a manual read, not a gate; set BSQLITE_E2E_BENCHMARKS=1 to run them"); + if (backend == "opfs") await SkipUnlessOpfsAsync(Page, "/benchmarks.html"); + + await Page.GotoAsync("/benchmarks.html"); + await Page.WaitForFunctionAsync("() => typeof globalThis.run === 'function'"); + + var row = await Page.EvaluateAsync<JsonElement>("backend => globalThis.run(backend)", backend); + + foreach (var measurement in row.EnumerateObject()) + { + TestContext.WriteLine($"{measurement.Name}: {measurement.Value}"); + } + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/BinaryWireTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/BinaryWireTests.cs new file mode 100644 index 00000000000..8e80ff66832 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/BinaryWireTests.cs @@ -0,0 +1,150 @@ +using System.Text.Json; +using System.Text.RegularExpressions; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// <summary> +/// The binary wire format is what the .NET transport speaks. These tests pin the layout from the JavaScript +/// side; <c>SqliteWireFormatTests</c> pins it from the .NET side with a byte-for-byte builder. +/// </summary> +[TestClass] +public sealed class BinaryWireTests : BsqliteTest +{ + [TestMethod] + public async Task A_batch_encodes_to_the_documented_header() + { + await OpenAsync(); + + var head = await Page.EvaluateAsync<int[]>( + """ + () => { + const { encodeBatchBinary } = globalThis.BitBsqlite.host; + const bytes = encodeBatchBinary([{ commandText: 'SELECT 1', resultKind: 'scalar', parameters: [] }]); + return Array.from(bytes.subarray(0, 6)); + } + """); + + // magic, version, request count (u32 LE) + CollectionAssert.AreEqual(new[] { 0xb5, 1, 1, 0, 0, 0 }, head); + } + + [TestMethod] + public async Task Every_storage_class_survives_the_round_trip() + { + await OpenAsync(); + + var row = await Page.EvaluateAsync<JsonElement>( + """ + async () => { + const { encodeBatchBinary, decodeReplyBinary, encodeValue } = globalThis.BitBsqlite.host; + const run = async batch => decodeReplyBinary(await globalThis.host.executeBinary(encodeBatchBinary(batch))); + + await run([{ commandText: 'CREATE TABLE v (i INTEGER, r REAL, t TEXT, b BLOB, n INTEGER)', parameters: [] }]); + await run([{ + commandText: 'INSERT INTO v VALUES (@i, @r, @t, @b, @n)', + parameters: [ + { name: '@i', ...encodeValue(9223372036854775807n) }, + { name: '@r', ...encodeValue(-2.5) }, + { name: '@t', ...encodeValue('héllo 🌍') }, + { name: '@b', ...encodeValue(new Uint8Array([0, 128, 255])) }, + { name: '@n', ...encodeValue(null) }, + ], + }]); + + const [result] = await run([{ commandText: 'SELECT i, r, t, b, n FROM v', resultKind: 'reader', parameters: [] }]); + return { types: result.columnTypes, t: result.rows[0].t, v: result.rows[0].v }; + } + """); + + AssertJson.AreEqual("""["INTEGER", "REAL", "TEXT", "BLOB", null]""", row.GetProperty("types")); + AssertJson.AreEqual("[1, 2, 3, 4, 5]", row.GetProperty("t")); + + var v = row.GetProperty("v"); + Assert.AreEqual("9223372036854775807", v[0].GetString()); + Assert.AreEqual(-2.5, v[1].GetDouble()); + Assert.AreEqual("héllo 🌍", v[2].GetString()); + Assert.AreEqual(Convert.ToBase64String([0, 128, 255]), v[3].GetString()); + Assert.AreEqual(JsonValueKind.Null, v[4].ValueKind); + } + + [TestMethod] + public async Task A_failure_is_a_reply_not_a_rejection() + { + await OpenAsync(); + + var outcome = await Page.EvaluateAsync<JsonElement>( + """ + async () => { + const { encodeBatchBinary, decodeReplyBinary } = globalThis.BitBsqlite.host; + const reply = await globalThis.host.executeBinary(encodeBatchBinary([ + { commandText: 'SELECT * FROM nope', resultKind: 'reader', parameters: [] }, + ])); + + try { + decodeReplyBinary(reply); + return { threw: false }; + } catch (error) { + return { threw: true, message: error.message, sqliteCode: error.sqliteCode }; + } + } + """); + + Assert.IsTrue(outcome.GetProperty("threw").GetBoolean()); + Assert.MatchesRegex(new Regex("no such table"), outcome.GetProperty("message").GetString()!); + Assert.AreEqual(1, outcome.GetProperty("sqliteCode").GetInt32()); + } + + [TestMethod] + public async Task A_corrupt_batch_is_reported_not_thrown() + { + await OpenAsync(); + + var outcome = await Page.EvaluateAsync<string?>( + """ + async () => { + const { decodeReplyBinary } = globalThis.BitBsqlite.host; + const reply = await globalThis.host.executeBinary(new Uint8Array([1, 2, 3])); + try { + decodeReplyBinary(reply); + return null; + } catch (error) { + return error.message; + } + } + """); + + Assert.IsNotNull(outcome); + Assert.MatchesRegex(new Regex("Not a Bsqlite wire message"), outcome); + } + + [TestMethod] + public async Task A_large_result_set_round_trips_intact() + { + await OpenAsync(); + + var summary = await Page.EvaluateAsync<JsonElement>( + """ + async () => { + const { encodeBatchBinary, decodeReplyBinary } = globalThis.BitBsqlite.host; + const run = async batch => decodeReplyBinary(await globalThis.host.executeBinary(encodeBatchBinary(batch))); + + await run([{ commandText: 'CREATE TABLE big (id INTEGER PRIMARY KEY, payload TEXT)', parameters: [] }]); + await run([{ + commandText: "WITH RECURSIVE n(i) AS (SELECT 1 UNION ALL SELECT i + 1 FROM n WHERE i < 5000) INSERT INTO big (payload) SELECT 'row-' || i FROM n", + parameters: [], + }]); + + const [result] = await run([{ commandText: 'SELECT id, payload FROM big ORDER BY id', resultKind: 'reader', parameters: [] }]); + return { count: result.rows.length, last: result.rows[4999].v }; + } + """); + + Assert.AreEqual(5000, summary.GetProperty("count").GetInt32()); + AssertJson.AreEqual("""[5000, "row-5000"]""", summary.GetProperty("last")); + } + + // Every test starts from a database of its own (the describe block's beforeEach). + private Task<HostVersion> OpenAsync() + => OpenHostAsync(Page, new HostOptions { DatabaseName = $"wire-{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}" }); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Bit.Bsqlite.Tests.E2E.csproj b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Bit.Bsqlite.Tests.E2E.csproj new file mode 100644 index 00000000000..813c471ba2b --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Bit.Bsqlite.Tests.E2E.csproj @@ -0,0 +1,53 @@ +<Project Sdk="Microsoft.NET.Sdk"> + + <PropertyGroup> + <TargetFramework>net10.0</TargetFramework> + <Nullable>enable</Nullable> + <ImplicitUsings>enable</ImplicitUsings> + <IsPackable>false</IsPackable> + <IsTestProject>true</IsTestProject> + <GenerateDocumentationFile>false</GenerateDocumentationFile> + <NoWarn>$(NoWarn);CS1591</NoWarn> + + <!-- The repo's global.json mandates the Microsoft.Testing.Platform runner. EnableMSTestRunner + builds that test host into the assembly itself, and it only generates the entry point for + an executable project. --> + <EnableMSTestRunner>true</EnableMSTestRunner> + <OutputType>Exe</OutputType> + </PropertyGroup> + + <ItemGroup> + <!-- Playwright itself, without a per-framework integration package: the suite manages the + browser, the contexts, the asset host and the demo host from its own infrastructure. --> + <PackageReference Include="Microsoft.Playwright" Version="1.63.0" /> + <PackageReference Include="MSTest" Version="4.5.1" /> + </ItemGroup> + + <ItemGroup> + <!-- Infrastructure/AssetHost serves the libraries' wwwroot folders with Kestrel, the way a Blazor + application maps them under _content/. --> + <FrameworkReference Include="Microsoft.AspNetCore.App" /> + </ItemGroup> + + <ItemGroup> + <!-- Built, not referenced: what the suite tests is the JavaScript the libraries compile into their + wwwroot (and the wa-sqlite engine and VFSes their restore vendors next to it), which the asset + host serves straight from those folders. Building them here keeps a test run from serving + modules a TypeScript change has already made stale. --> + <ProjectReference Include="..\..\Bit.Bsqlite\Bit.Bsqlite.csproj" ReferenceOutputAssembly="false" /> + <ProjectReference Include="..\..\Bit.Bsqlite.Storage.Opfs\Bit.Bsqlite.Storage.Opfs.csproj" ReferenceOutputAssembly="false" /> + <ProjectReference Include="..\..\Bit.Bsqlite.Storage.IndexedDb\Bit.Bsqlite.Storage.IndexedDb.csproj" ReferenceOutputAssembly="false" /> + <ProjectReference Include="..\..\Bit.Bsqlite.Storage.CacheStorage\Bit.Bsqlite.Storage.CacheStorage.csproj" ReferenceOutputAssembly="false" /> + </ItemGroup> + + <!-- Deliberately no ProjectReference to the demo. The suite boots Bit.Bsqlite.Demo.Server from its build + output as a separate process (Infrastructure/DemoHost.cs), so build the demo server first, in the + configuration the tests run in. A reference would also make this project build the WebAssembly + client a second time, into the same obj folders, on every test build. --> + + <ItemGroup> + <!-- The pages the asset host serves at its root: the tests drive the library through them. --> + <None Include="TestPages\**\*" CopyToOutputDirectory="PreserveNewest" /> + </ItemGroup> + +</Project> diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/CacheStorageTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/CacheStorageTests.cs new file mode 100644 index 00000000000..d77b5c6c1fb --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/CacheStorageTests.cs @@ -0,0 +1,187 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E; + +/// <summary>The Cache Storage provider: persistence, the admin module, and several tabs on one database.</summary> +[TestClass] +public sealed class CacheStorageTests : BsqliteTest +{ + // --------------------------------------------------------------------------------------------- + // Cache Storage persistence + // + // Playwright's WebKit runs with an ephemeral data store, and its Cache Storage keeps a cache's entries + // only while some context holds that cache open: when the worker that wrote them is the sole holder and + // is terminated, the entries are gone (the cache name survives, its contents do not). Every test here + // restarts the worker and expects to read back what it wrote, which is exactly what that store cannot + // do. The multi-tab tests below, where another tab holds the cache open the whole time, do run there. + // --------------------------------------------------------------------------------------------- + + private const string WebKitDiscardsEntries = "Playwright's WebKit discards Cache Storage entries once no context holds the cache open"; + + [TestMethod] + public async Task Persistence_survives_a_worker_restart() + { + SkipWhen(BrowserName == "webkit", WebKitDiscardsEntries); + + var databaseName = UniqueName("cache-persist"); + await OpenCacheAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Kept')"); + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + await OpenCacheAsync(Page, databaseName); + AssertJson.AreEqual("""[["Kept"]]""", (await QueryAsync(Page, "SELECT name FROM product")).Rows); + } + + [TestMethod] + public async Task Persistence_an_uncommitted_write_does_not_survive_a_killed_worker() + { + SkipWhen(BrowserName == "webkit", WebKitDiscardsEntries); + + var databaseName = UniqueName("cache-crash"); + await OpenCacheAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Committed')"); + await ExecAsync(Page, "BEGIN"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Ghost')"); + await Page.EvaluateAsync("() => globalThis.host.dispose()"); + + await OpenCacheAsync(Page, databaseName); + AssertJson.AreEqual("""[["Committed"]]""", (await QueryAsync(Page, "SELECT name FROM product ORDER BY id")).Rows); + } + + [TestMethod] + public async Task Persistence_opens_a_database_that_besql_left_in_bit_Besql() + { + SkipWhen(BrowserName == "webkit", WebKitDiscardsEntries); + + var databaseName = UniqueName("cache-besql"); + await OpenCacheAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('FromBesql')"); + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + await Page.EvaluateAsync( + """ + async name => { + const admin = await import('/_content/Bit.Bsqlite.Storage.CacheStorage/bit-bsqlite-cache-admin.js'); + const image = await admin.exportDatabase(name); + await admin.deleteDatabase(name); + const besql = await caches.open('bit-Besql'); + await besql.put(`/data/cache/${name}`, new Response(image)); + } + """, databaseName); + + await OpenCacheAsync(Page, databaseName); + AssertJson.AreEqual("""[["FromBesql"]]""", (await QueryAsync(Page, "SELECT name FROM product")).Rows); + } + + // --------------------------------------------------------------------------------------------- + // Cache Storage admin + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Admin_import_export_list_and_delete() + { + await GotoHostPageAsync(Page); + + var name = UniqueName("cache-admin"); + var image = Enumerable.Range(0, 64).Select(i => i * 7 % 251).ToArray(); + "SQLite format 3\0".Select(c => (int)c).ToArray().CopyTo(image, 0); + + var outcome = await Page.EvaluateAsync<JsonElement>( + """ + async ({ name, image }) => { + const admin = await import('/_content/Bit.Bsqlite.Storage.CacheStorage/bit-bsqlite-cache-admin.js'); + await admin.importDatabase(name, new Uint8Array(image)); + const exported = Array.from(await admin.exportDatabase(name)); + const listed = await admin.list(); + const existed = await admin.exists(name); + await admin.deleteDatabase(name); + return { + exported, + listed: listed.includes(name), + existed, + existsAfterDelete: await admin.exists(name), + }; + } + """, new { name, image }); + + AssertJson.AreEqual(JsonSerializer.Serialize(image), outcome.GetProperty("exported")); + Assert.IsTrue(outcome.GetProperty("listed").GetBoolean()); + Assert.IsTrue(outcome.GetProperty("existed").GetBoolean()); + Assert.IsFalse(outcome.GetProperty("existsAfterDelete").GetBoolean()); + } + + // --------------------------------------------------------------------------------------------- + // Cache Storage multi-tab + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Multi_tab_a_second_tab_sees_a_committed_write() + { + var databaseName = UniqueName("cache-tabs"); + var context = await NewContextAsync(); + var first = await context.NewPageAsync(); + var second = await context.NewPageAsync(); + + try + { + await OpenCacheAsync(first, databaseName); + await ExecAsync(first, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(first, "INSERT INTO product (name) VALUES ('Shared')"); + + await OpenCacheAsync(second, databaseName); + AssertJson.AreEqual("""[["Shared"]]""", (await QueryAsync(second, "SELECT name FROM product")).Rows); + } + finally + { + await context.CloseAsync(); + } + } + + /// <summary> + /// The harder order: the reader opened while the file was still empty. Its VFS cached a size of zero, and + /// every read transaction since has to notice that the other tab grew the file. + /// </summary> + [TestMethod] + public async Task Multi_tab_a_tab_that_opened_first_sees_a_write_that_grew_the_file() + { + var databaseName = UniqueName("cache-grow"); + var context = await NewContextAsync(); + var reader = await context.NewPageAsync(); + var writer = await context.NewPageAsync(); + + try + { + await OpenCacheAsync(reader, databaseName); + AssertJson.AreEqual("[]", (await QueryAsync(reader, "SELECT name FROM sqlite_master WHERE type = 'table'")).Rows); + + await OpenCacheAsync(writer, databaseName); + await ExecAsync(writer, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(writer, "INSERT INTO product (name) VALUES ('Grown')"); + + AssertJson.AreEqual("""[["Grown"]]""", (await QueryAsync(reader, "SELECT name FROM product")).Rows); + } + finally + { + await context.CloseAsync(); + } + } + + private static Task<HostVersion> OpenCacheAsync(IPage page, string databaseName) => OpenHostAsync(page, Storages.CacheStorage(databaseName)); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/DemoTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/DemoTests.cs new file mode 100644 index 00000000000..a20f4553f8a --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/DemoTests.cs @@ -0,0 +1,145 @@ +using System.Net; +using System.Text.Json; +using System.Text.RegularExpressions; +using Bit.Bsqlite.Tests.E2E.Infrastructure; +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E; + +/// <summary> +/// The demo website, end to end: prerendered on the server, hydrated on WebAssembly, and running every +/// database it opens in the browser. Where the library suites prove the modules, these prove the composition +/// an application ships - the static web assets, the registration, prerendering, and moving a database between +/// backends from the UI. +/// </summary> +[TestClass] +public sealed class DemoTests : DemoTest +{ + /// <summary> + /// Every page in the sidebar, discovered from the sidebar itself so a new page is covered without a new + /// test: the server answers it prerendered, and in the browser - reached the way a reader reaches it, through + /// the sidebar, with the runtime and the database already up - it renders against the open database without + /// anything reaching the console. + /// </summary> + [TestMethod] + public async Task Every_page_prerenders_hydrates_and_opens_its_database_without_a_console_error() + { + var errors = new List<string>(); + var page = await NewDemoPageAsync(errors); + + await GotoInteractiveAsync(page, "/catalog"); + await WaitForSessionAsync(page); + + var routes = await page.Locator("nav a.nav-link").EvaluateAllAsync<string[]>( + "links => [...new Set(links.map(a => a.getAttribute('href')).filter(h => h !== null && !/^[a-z]+:/i.test(h)))]"); + + Assert.IsGreaterThan(5, routes.Length, $"The sidebar lists {routes.Length} pages: {string.Join(", ", routes)}."); + + using var client = new HttpClient(); + foreach (var route in routes) + { + var path = "/" + route.TrimStart('/'); + using var response = await client.GetAsync(DemoUrl + path); + Assert.AreEqual(HttpStatusCode.OK, response.StatusCode, $"{path} answered {response.StatusCode}."); + Assert.MatchesRegex(new Regex("<title>[^<]*Bsqlite[^<]*"), await response.Content.ReadAsStringAsync(), $"{path} is not prerendered with a title of its own."); + } + + // The overview last: it runs full-bleed, without the sidebar or the session bar. + foreach (var route in routes.OrderBy(r => r.Trim('/').Length == 0)) + { + await page.Locator($"nav a.nav-link[href='{route}']").First.ClickAsync(); + await page.WaitForURLAsync(DemoUrl + "/" + route.TrimStart('/')); + + if (route.Trim('/').Length > 0) + { + await WaitForSessionAsync(page); + } + + await Assertions.Expect(page.Locator("main[data-interactive='true'] h1").First).ToBeVisibleAsync(); + } + + Assert.AreEqual(0, errors.Count, "Console errors:" + Environment.NewLine + string.Join(Environment.NewLine, errors)); + } + + /// + /// The prerendered HTML is the page, not a loading screen: the text is readable before WebAssembly starts, + /// which is why every page opens its database only once it is interactive. + /// + [TestMethod] + public async Task Pages_are_readable_before_WebAssembly_starts() + { + using var client = new HttpClient(); + var html = await client.GetStringAsync(DemoUrl + "/catalog"); + + StringAssert.Contains(html, "Products and categories, read and written entirely on the client"); + StringAssert.Contains(html, "data-interactive=\"false\""); + StringAssert.Contains(html, "data-session=\"closed\""); + } + + [TestMethod] + public async Task An_unknown_route_answers_404_with_the_demos_own_page() + { + var page = await NewDemoPageAsync([]); + + var response = await page.GotoAsync("/no-such-page"); + + Assert.IsNotNull(response); + Assert.AreEqual((int)HttpStatusCode.NotFound, response.Status); + await Assertions.Expect(page.Locator("h1")).ToHaveTextAsync("That page is not in this demo"); + } + + /// + /// The storage switcher's "Move data": the image is exported, imported on the target, verified, the sticky + /// binding flips, and the catalog the page reads afterwards is the one it read before. A reload then reopens + /// the database where it now lives - existing data outranks the configured preference. + /// + [TestMethod] + public async Task Moving_the_database_to_another_backend_keeps_its_data_and_the_binding_sticks() + { + var errors = new List(); + var page = await NewDemoPageAsync(errors); + + await GotoInteractiveAsync(page, "/catalog"); + var first = await WaitForSessionAsync(page); + var products = page.Locator("section.card:has(h2:text-is('Products')) tbody tr"); + await Assertions.Expect(products.First).ToBeVisibleAsync(); + var count = await products.CountAsync(); + + var target = first == "indexeddb" ? "cache-storage" : "indexeddb"; + await page.Locator("section.session-bar select").SelectOptionAsync(target); + await page.Locator("section.session-bar button", new PageLocatorOptions { HasText = "Move data" }).ClickAsync(); + + Assert.AreEqual(target, await WaitForSessionAsync(page, target)); + await Assertions.Expect(products).ToHaveCountAsync(count); + + await page.ReloadAsync(); + await page.WaitForSelectorAsync("main[data-interactive='true']"); + Assert.AreEqual(target, await WaitForSessionAsync(page, target), "The reload did not reopen the database where it was moved to."); + await Assertions.Expect(products).ToHaveCountAsync(count); + + Assert.AreEqual(0, errors.Count, "Console errors:" + Environment.NewLine + string.Join(Environment.NewLine, errors)); + } + + /// + /// The claim the demo exists to make: one AppDbContext and one migration set on both sides. The server ran + /// the migrations with the stock provider and seeded its copy; a fresh tab ran the same migrations through + /// UseBitBsqlite and seeded its own - and the two catalogs agree. + /// + [TestMethod] + public async Task The_browser_and_the_server_run_the_same_model_and_seed_the_same_catalog() + { + using var client = new HttpClient(); + var server = JsonDocument.Parse(await client.GetStringAsync(DemoUrl + "/api/products")).RootElement; + var serverSkus = server.EnumerateArray().Select(p => p.GetProperty("sku").GetString()).Order().ToArray(); + + var page = await NewDemoPageAsync([]); + await GotoInteractiveAsync(page, "/catalog"); + await WaitForSessionAsync(page); + + var products = page.Locator("section.card:has(h2:text-is('Products')) tbody tr"); + await Assertions.Expect(products).ToHaveCountAsync(serverSkus.Length); + + var browserSkus = (await products.Locator("td.mono").AllTextContentsAsync()).Select(s => s.Trim()).Order().ToArray(); + CollectionAssert.AreEqual(serverSkus, browserSkus); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineBehaviourTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineBehaviourTests.cs new file mode 100644 index 00000000000..657fc6209fd --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineBehaviourTests.cs @@ -0,0 +1,327 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E; + +/// +/// Behaviours of the worker's engine loop that the .NET layer relies on: the statement cache, the +/// engine-reported change notifications, accurate affected-row counts, open-time configuration, extension +/// modules, and in-place snapshots. +/// +[TestClass] +public sealed class EngineBehaviourTests : BsqliteTest +{ + // --------------------------------------------------------------------------------------------- + // statement cache + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Statement_cache_a_repeated_statement_runs_with_fresh_parameters_each_time() + { + await OpenForStatementCacheAsync(); + + foreach (var name in new[] { "A", "B", "C" }) + { + await ExecAsync(Page, "INSERT INTO product (name) VALUES (@name)", new WireParameter("@name", name)); + } + + var rows = (await QueryAsync(Page, "SELECT name FROM product ORDER BY id")).Rows; + AssertJson.AreEqual("""[["A"], ["B"], ["C"]]""", rows); + + // The same SELECT text, now with a parameter that changed and then a parameter that was cleared. + var one = await QueryAsync(Page, "SELECT name FROM product WHERE name = @n", new WireParameter("@n", "B")); + AssertJson.AreEqual("""[["B"]]""", one.Rows); + var none = await QueryAsync(Page, "SELECT name FROM product WHERE name = @n", new WireParameter("@n", "Z")); + AssertJson.AreEqual("[]", none.Rows); + } + + [TestMethod] + public async Task Statement_cache_a_cached_statement_survives_a_schema_change() + { + await OpenForStatementCacheAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('A')"); + CollectionAssert.AreEqual(new[] { "id", "name", "price" }, (await QueryAsync(Page, "SELECT * FROM product")).ColumnNames); + + await ExecAsync(Page, "ALTER TABLE product ADD COLUMN sku TEXT"); + + CollectionAssert.AreEqual(new[] { "id", "name", "price", "sku" }, (await QueryAsync(Page, "SELECT * FROM product")).ColumnNames); + } + + [TestMethod] + public async Task Statement_cache_a_failing_statement_does_not_poison_its_cached_handle() + { + await OpenForStatementCacheAsync(); + + await ExecAsync(Page, "CREATE UNIQUE INDEX ux ON product (name)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('dup')"); + + var error = await ExecuteExpectingFailureAsync(Page, + new WireRequest("INSERT INTO product (name) VALUES (@n)", [new WireParameter("@n", "dup")])); + Assert.IsNotNull(error); + Assert.AreEqual(19, error.SqliteCode); + + // Same text again, now with a value that works: the reset in between must have cleared the error. + var ok = await ExecAsync(Page, "INSERT INTO product (name) VALUES (@n)", new WireParameter("@n", "fresh")); + Assert.AreEqual(1, ok.RecordsAffected); + } + + [TestMethod] + public async Task Statement_cache_many_distinct_statements_do_not_exhaust_the_cache() + { + await OpenForStatementCacheAsync(); + + for (var i = 0; i < 200; i++) + { + await ExecAsync(Page, $"INSERT INTO product (name) VALUES ('n{i}') -- {i}"); + } + + AssertJson.AreEqual("[[200]]", (await QueryAsync(Page, "SELECT COUNT(*) FROM product")).Rows); + } + + // --------------------------------------------------------------------------------------------- + // engine-reported changes + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Engine_reported_changes_a_write_names_the_tables_it_changed_including_through_a_cascade() + { + await OpenForChangesAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('A')"); + await ExecAsync(Page, "INSERT INTO line (product_id) VALUES (1)"); + await NotificationsAsync(Page); + + await ExecAsync(Page, "DELETE FROM product WHERE id = 1"); + + AssertJson.AreEqual("""[["line", "product"]]""", await NotifiedAsync(Page)); + } + + [TestMethod] + public async Task Engine_reported_changes_a_read_and_a_write_that_changed_nothing_report_no_tables() + { + await OpenForChangesAsync(); + + await QueryAsync(Page, "SELECT * FROM product"); + await ExecAsync(Page, "DELETE FROM product WHERE id = 42"); + + AssertJson.AreEqual("[]", await NotifiedAsync(Page)); + } + + [TestMethod] + public async Task Engine_reported_changes_a_truncating_DELETE_which_the_update_hook_does_not_see_is_still_reported() + { + await OpenForChangesAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('A')"); + await NotificationsAsync(Page); + + await ExecAsync(Page, "DELETE FROM line"); + await ExecAsync(Page, "DELETE FROM product"); + + AssertJson.AreEqual("""[["product"]]""", await NotifiedAsync(Page)); + } + + [TestMethod] + public async Task Engine_reported_changes_DDL_names_the_table_it_touched() + { + await OpenForChangesAsync(); + + await ExecAsync(Page, "CREATE TABLE extra (id INTEGER PRIMARY KEY)"); + + AssertJson.AreEqual("""[["extra"]]""", await NotifiedAsync(Page)); + } + + [TestMethod] + public async Task Engine_reported_changes_nothing_is_reported_until_the_transaction_commits_and_nothing_at_all_on_rollback() + { + await OpenForChangesAsync(); + + await ExecAsync(Page, "BEGIN"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('A')"); + AssertJson.AreEqual("[]", await NotifiedAsync(Page)); + await ExecAsync(Page, "COMMIT"); + AssertJson.AreEqual("""[["product"]]""", await NotifiedAsync(Page)); + + await ExecAsync(Page, "BEGIN"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('B')"); + await ExecAsync(Page, "ROLLBACK"); + AssertJson.AreEqual("[]", await NotifiedAsync(Page)); + } + + [TestMethod] + public async Task Engine_reported_changes_the_cascade_also_reaches_the_notification_channel() + { + await OpenForChangesAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('A')"); + await ExecAsync(Page, "INSERT INTO line (product_id) VALUES (1)"); + + var tables = await Page.EvaluateAsync( + """ + async () => { + const seen = new Promise(resolve => { + globalThis.host.onNotify(payload => resolve(payload.tables)); + }); + await globalThis.host.execute([{ commandText: 'DELETE FROM product WHERE id = 1', resultKind: 'nonQuery' }]); + return await seen; + } + """); + + Array.Sort(tables, StringComparer.Ordinal); + CollectionAssert.AreEqual(new[] { "line", "product" }, tables); + } + + // --------------------------------------------------------------------------------------------- + // counters + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Counters_a_SELECT_after_an_INSERT_does_not_count_the_insert_twice() + { + await OpenForCountersAsync(); + + var result = await QueryAsync( + Page, + "INSERT INTO product (name) VALUES ('x'); SELECT id FROM product WHERE rowid = last_insert_rowid();"); + + Assert.AreEqual(1, result.RecordsAffected); + AssertJson.AreEqual("[[1]]", result.Rows); + } + + [TestMethod] + public async Task Counters_a_read_only_batch_reports_zero_affected_rows() + { + await OpenForCountersAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('x')"); + + Assert.AreEqual(0, (await QueryAsync(Page, "SELECT * FROM product")).RecordsAffected); + } + + [TestMethod] + public async Task Counters_last_insert_rowid_is_carried_on_the_result() + { + await OpenForCountersAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('a')"); + var second = await ExecAsync(Page, "INSERT INTO product (name) VALUES ('b')"); + + Assert.AreEqual(2L, second.LastInsertRowId); + } + + [TestMethod] + public async Task Counters_a_row_limit_stops_the_reader_early() + { + await OpenForCountersAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('a'), ('b'), ('c')"); + + var limited = (await ExecuteAsync(Page, new WireRequest("SELECT name FROM product ORDER BY id", ResultKind: "reader", MaxRows: 2)))[0]; + AssertJson.AreEqual("""[["a"], ["b"]]""", limited.Rows); + + var schemaOnly = (await ExecuteAsync(Page, new WireRequest("SELECT id, name FROM product", ResultKind: "reader", MaxRows: 0)))[0]; + CollectionAssert.AreEqual(new[] { "id", "name" }, schemaOnly.ColumnNames); + AssertJson.AreEqual("[]", schemaOnly.Rows); + } + + // --------------------------------------------------------------------------------------------- + // open-time configuration + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Open_time_configuration_foreign_keys_are_enforced_by_default() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"fk-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE parent (id INTEGER PRIMARY KEY)"); + await ExecAsync(Page, "CREATE TABLE child (id INTEGER PRIMARY KEY, parent_id INTEGER REFERENCES parent(id))"); + + var error = await ExecuteExpectingFailureAsync(Page, new WireRequest("INSERT INTO child (parent_id) VALUES (99)")); + + Assert.IsNotNull(error); + Assert.AreEqual(19, error.SqliteCode); + AssertJson.AreEqual("[[1]]", (await QueryAsync(Page, "PRAGMA foreign_keys")).Rows); + } + + [TestMethod] + public async Task Open_time_configuration_foreign_keys_stay_off_when_the_caller_opts_out() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"nofk-{Now()}", ForeignKeys = false }); + + AssertJson.AreEqual("[[0]]", (await QueryAsync(Page, "PRAGMA foreign_keys")).Rows); + } + + [TestMethod] + public async Task Open_time_configuration_open_time_statements_run_after_the_defaults() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"onopen-{Now()}", OnOpenSql = ["PRAGMA foreign_keys = OFF"] }); + + AssertJson.AreEqual("[[0]]", (await QueryAsync(Page, "PRAGMA foreign_keys")).Rows); + } + + [TestMethod] + public async Task Open_time_configuration_an_extension_module_can_add_a_function() + { + await OpenHostAsync(Page, new HostOptions + { + DatabaseName = $"ext-{Now()}", + Extensions = [new JsModule("/extension.js", "register")], + }); + + AssertJson.AreEqual("""[["HI!"]]""", (await QueryAsync(Page, "SELECT shout('hi')")).Rows); + } + + // --------------------------------------------------------------------------------------------- + // snapshots + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Snapshots_export_returns_a_SQLite_image_of_the_open_database() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"export-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('kept')"); + + var header = await Page.EvaluateAsync( + """ + async () => { + const image = await globalThis.host.export(); + return { length: image.byteLength, magic: new TextDecoder().decode(image.subarray(0, 15)) }; + } + """); + + Assert.AreEqual("SQLite format 3", header.GetProperty("magic").GetString()); + Assert.IsGreaterThanOrEqualTo(4096, header.GetProperty("length").GetInt32()); + } + + // --------------------------------------------------------------------------------------------- + // The describe blocks' beforeEach hooks + // --------------------------------------------------------------------------------------------- + + private async Task OpenForStatementCacheAsync() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"cache-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT, price REAL)"); + } + + private async Task OpenForChangesAsync() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"changes-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "CREATE TABLE line (id INTEGER PRIMARY KEY, product_id INTEGER REFERENCES product(id) ON DELETE CASCADE)"); + await RecordNotificationsAsync(Page); + } + + private async Task OpenForCountersAsync() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"counters-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + } + + /// The notifications recorded since the last call, as JSON for . + private static async Task NotifiedAsync(IPage page) => JsonSerializer.SerializeToElement(await NotificationsAsync(page)); + + /// Date.now(), which the spec put in its database names. + private static long Now() => DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineConformanceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineConformanceTests.cs new file mode 100644 index 00000000000..b47ec58490a --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineConformanceTests.cs @@ -0,0 +1,52 @@ +using System.Text.RegularExpressions; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// +/// The engine-layer rules, run against the worker's built-in memory VFS - the InMemory provider's actual +/// engine path. The C# kit runs the same rules against the in-process transport; a divergence between the two +/// is a bug in one of them. +/// +[TestClass] +public sealed class EngineConformanceTests : BsqliteTest +{ + [TestMethod] + public async Task A_rolled_back_write_is_invisible() + { + await OpenAsync(); + + await ExecAsync(Page, "BEGIN; INSERT INTO product (name) VALUES ('Ghost'); ROLLBACK;"); + + AssertJson.AreEqual("[[0]]", (await QueryAsync(Page, "SELECT COUNT(*) FROM product")).Rows); + } + + [TestMethod] + public async Task A_committed_write_is_visible() + { + await OpenAsync(); + + await ExecAsync(Page, "BEGIN; INSERT INTO product (name) VALUES ('Kept'); COMMIT;"); + + AssertJson.AreEqual("""[["Kept"]]""", (await QueryAsync(Page, "SELECT name FROM product")).Rows); + } + + [TestMethod] + public async Task Journal_mode_WAL_is_rejected_rather_than_silently_ignored() + { + await OpenAsync(); + + var error = await ExecuteExpectingFailureAsync(Page, new WireRequest("PRAGMA journal_mode=WAL", ResultKind: "nonQuery")); + + Assert.IsNotNull(error); + Assert.MatchesRegex(new Regex("WAL", RegexOptions.IgnoreCase), error.Message); + Assert.MatchesRegex(new Regex("shared-memory", RegexOptions.IgnoreCase), error.Message); + } + + // The describe block's beforeEach: the in-memory database every test here starts from. + private async Task OpenAsync() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = "conformance.db" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineTests.cs new file mode 100644 index 00000000000..56261af687d --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/EngineTests.cs @@ -0,0 +1,140 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// Engine loading and build negotiation. +[TestClass] +public sealed class EngineTests : BsqliteTest +{ + [TestMethod] + public async Task Loads_the_synchronous_build_and_reports_the_pinned_SQLite_version() + { + var (version, build) = await OpenHostAsync(Page); + + Assert.AreEqual("synchronous", build); + + // Pinned through the vendored artifacts, so a silent engine change fails here rather than showing + // up as different SQL behaviour much later. + Assert.AreEqual("3.53.0", version); + } + + /// + /// EF Core gates SQL translations on the engine's version, and it reads that version from + /// BsqlitePclProvider.sqlite3_libversion() - a stub, because WebAssembly has no e_sqlite3 to ask. A + /// stub that claims more than the vendored engine delivers is how EF ends up emitting SQL the engine cannot + /// run, at query time, in the browser only. + /// + [TestMethod] + public async Task The_engine_reports_the_version_the_dotnet_stub_claims() + { + var reported = await OpenHostAsync(Page); + + // Keep in step with BsqlitePclProvider.EngineVersion. + Assert.AreEqual("3.53.0", reported.Version); + } + + [TestMethod] + public async Task Picks_an_async_capable_build_only_when_the_VFS_needs_one() + { + await GotoHostPageAsync(Page); + + var selection = await Page.EvaluateAsync( + """ + () => { + const { selectBuild, supportsJspi } = globalThis.BitBsqlite.engine; + return { + jspi: supportsJspi(), + synchronous: selectBuild('synchronous').id, + asyncCapable: selectBuild('asyncCapable').id, + }; + } + """); + + Assert.AreEqual("synchronous", selection.GetProperty("synchronous").GetString()); + + // The point of vendoring three builds: whichever async build this browser gets, it must be the + // cheaper one when the browser can take it. + Assert.AreEqual(selection.GetProperty("jspi").GetBoolean() ? "jspi" : "asyncify", selection.GetProperty("asyncCapable").GetString()); + } + + /// + /// Loading the async build is not hypothetical work: it is what the IndexedDB and Cache Storage providers + /// will run on, and a build that cannot execute a plain query is worth discovering now rather than behind a + /// VFS. + /// + [TestMethod] + public async Task The_async_capable_build_loads_and_executes() + { + var (version, build) = await OpenHostAsync(Page, new HostOptions { DatabaseName = "async.db", RequiredBuild = "asyncCapable" }); + + CollectionAssert.Contains(new[] { "jspi", "asyncify" }, build); + Assert.AreEqual("3.53.0", version); + + AssertJson.AreEqual("[[42]]", (await QueryAsync(Page, "SELECT 6 * 7")).Rows); + } + + [TestMethod] + public async Task Rejects_an_unknown_build_requirement_instead_of_guessing() + { + await GotoHostPageAsync(Page); + + var error = await Page.EvaluateAsync( + """ + () => { + try { + globalThis.BitBsqlite.engine.selectBuild('whatever'); + return null; + } catch (e) { + return e.message; + } + } + """); + + Assert.IsNotNull(error); + Assert.Contains("whatever", error); + } + + [TestMethod] + public async Task A_worker_owns_one_database_and_says_so_when_asked_for_another() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = "first.db" }); + + var error = await Page.EvaluateAsync( + """ + async () => { + try { + await globalThis.host.send({ + kind: 'open', + databaseName: 'second.db', + requiredBuild: 'synchronous', + }); + return null; + } catch (e) { + return e.message; + } + } + """); + + Assert.IsNotNull(error); + Assert.Contains("first.db", error); + Assert.Contains("own worker", error); + } + + [TestMethod] + public async Task Reopening_the_same_database_reuses_the_session() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = "same.db" }); + + var result = await Page.EvaluateAsync( + """ + () => globalThis.host.send({ + kind: 'open', + databaseName: 'same.db', + requiredBuild: 'synchronous', + }) + """); + + Assert.IsTrue(result.GetProperty("reused").GetBoolean()); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/FunctionsTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/FunctionsTests.cs new file mode 100644 index 00000000000..d19cdb2f6f6 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/FunctionsTests.cs @@ -0,0 +1,352 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// The ef_* decimal functions, the EF_DECIMAL collation and regexp. +[TestClass] +public sealed class FunctionsTests : BsqliteTest +{ + /// + /// The S4 data set: values chosen so lexicographic TEXT order and numeric order disagree (9 vs + /// 10, 2.5 vs 100), with zero, negatives, sub-unit scales, and nulls. + /// + /// + /// Every ef_* result below is what Microsoft.Data.Sqlite returns for the same call - the oracle these + /// functions exist to imitate - which is the canonical 0.0########################### form and not a + /// plain ToString. ef_multiply('10','3') is '30.0' there, so it is '30.0' here. + /// + private static readonly (int Id, string Value, string? Optional, int Bucket, string Text)[] Rows = + [ + (Id: 1, Value: "0", Optional: null, Bucket: 0, Text: "alpha"), + (Id: 2, Value: "10", Optional: "10", Bucket: 0, Text: "Alpha"), + (Id: 3, Value: "9", Optional: null, Bucket: 0, Text: "beta-1"), + (Id: 4, Value: "-5", Optional: "-5", Bucket: 0, Text: "gamma 42"), + (Id: 5, Value: "2.5", Optional: "2.5", Bucket: 1, Text: ""), + (Id: 6, Value: "0.1", Optional: null, Bucket: 1, Text: "a.b.c"), + (Id: 7, Value: "100", Optional: "100", Bucket: 1, Text: "123"), + (Id: 8, Value: "-0.25", Optional: "-0.25", Bucket: 1, Text: "x[y]z"), + (Id: 9, Value: "1234.5678", Optional: "1234.5678", Bucket: 1, Text: " spaced "), + (Id: 10, Value: "3", Optional: "3", Bucket: 1, Text: "delta\ntail"), + ]; + + // --------------------------------------------------------------------------------------------- + // scalar decimal functions + // --------------------------------------------------------------------------------------------- + + [TestMethod] + [DataRow("ef_add(value, '1.5')", + new string?[] { "1.5", "11.5", "10.5", "-3.5", "4.0", "1.6", "101.5", "1.25", "1236.0678", "4.5" }, DisplayName = "add")] + [DataRow("ef_add(value, ef_negate('1.5'))", + new string?[] { "-1.5", "8.5", "7.5", "-6.5", "1.0", "-1.4", "98.5", "-1.75", "1233.0678", "1.5" }, DisplayName = "subtract")] + [DataRow("ef_multiply(value, '3')", + new string?[] { "0.0", "30.0", "27.0", "-15.0", "7.5", "0.3", "300.0", "-0.75", "3703.7034", "9.0" }, DisplayName = "multiply")] + [DataRow("ef_divide(value, '4')", + new string?[] { "0.0", "2.5", "2.25", "-1.25", "0.625", "0.025", "25.0", "-0.0625", "308.64195", "0.75" }, DisplayName = "divide")] + [DataRow("ef_mod(value, '3')", + new string?[] { "0.0", "1.0", "0.0", "-2.0", "2.5", "0.1", "1.0", "-0.25", "1.5678", "0.0" }, DisplayName = "modulo")] + [DataRow("ef_negate(value)", + new string?[] { "0.0", "-10.0", "-9.0", "5.0", "-2.5", "-0.1", "-100.0", "0.25", "-1234.5678", "-3.0" }, DisplayName = "negate")] + [DataRow("ef_add(optional, '1')", + new string?[] { null, "11.0", null, "-4.0", "3.5", null, "101.0", "0.75", "1235.5678", "4.0" }, DisplayName = "nullable add")] + public async Task Scalar_decimal_function(string expr, string?[] expected) + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, $"SELECT {expr} FROM rows ORDER BY id"); + AssertJson.AreEqual(JsonSerializer.Serialize(expected), Column(result)); + } + + [TestMethod] + public async Task Scalar_decimal_functions_1_over_3_is_twenty_eight_threes_not_an_IEEE_float() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, "SELECT ef_divide('1', '3')"); + AssertJson.AreEqual("""[["0.3333333333333333333333333333"]]""", result.Rows); + } + + [TestMethod] + public async Task Scalar_decimal_functions_a_zero_divisor_yields_null_rather_than_an_error() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, "SELECT ef_divide('10', '0'), ef_mod('10', '0')"); + AssertJson.AreEqual("[[null, null]]", result.Rows); + } + + // --------------------------------------------------------------------------------------------- + // aggregates + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Aggregates_sum_average_max_min() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, + "SELECT ef_sum(value), ef_avg(value), ef_max(value), ef_min(value) FROM rows"); + + AssertJson.AreEqual("""[["1353.9178", "135.39178", "1234.5678", "-5.0"]]""", result.Rows); + } + + [TestMethod] + public async Task Aggregates_nullable_aggregates_skip_nulls() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, + "SELECT ef_sum(optional), ef_avg(optional), ef_max(optional), ef_min(optional) FROM rows"); + + AssertJson.AreEqual( + """ + [[ + "1344.8178", + "192.11682857142857142857142857", + "1234.5678", + "-5.0" + ]] + """, result.Rows); + } + + [TestMethod] + public async Task Aggregates_grouped_sum_and_average() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, + "SELECT bucket, ef_sum(value), ef_avg(value) FROM rows GROUP BY bucket ORDER BY bucket"); + + AssertJson.AreEqual( + """ + [ + [0, "14.0", "3.5"], + [1, "1339.9178", "223.31963333333333333333333333"] + ] + """, result.Rows); + } + + [TestMethod] + public async Task Aggregates_an_empty_group_yields_null_not_zero() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, "SELECT ef_sum(value), ef_avg(value) FROM rows WHERE id = 0"); + AssertJson.AreEqual("[[null, null]]", result.Rows); + } + + /// + /// A scalar read abandons a grouped query after its first row, leaving the remaining groups part-way + /// through. Aggregate state is held against the address sqlite3_aggregate_context returns, which + /// SQLite frees with the statement and hands out again, so anything left behind would be read as a running + /// total by whichever aggregate lands on that address next. These pin the behaviour that matters - a later + /// aggregate starts from nothing - rather than the mechanism. + /// + [TestMethod] + public async Task Aggregates_a_grouped_query_abandoned_after_one_row_does_not_poison_the_next() + { + await OpenSeededAsync(); + + var abandoned = (await ExecuteAsync(Page, + new WireRequest("SELECT ef_sum(value) FROM rows GROUP BY bucket ORDER BY bucket", ResultKind: "scalar")))[0]; + AssertJson.AreEqual("""[["14.0"]]""", abandoned.Rows); + + // The same totals as a first run would produce. + var again = await QueryAsync(Page, + "SELECT bucket, ef_sum(value) FROM rows GROUP BY bucket ORDER BY bucket"); + AssertJson.AreEqual("""[[0, "14.0"], [1, "1339.9178"]]""", again.Rows); + + var whole = await QueryAsync(Page, "SELECT ef_sum(value) FROM rows"); + AssertJson.AreEqual("""[["1353.9178"]]""", whole.Rows); + } + + [TestMethod] + public async Task Aggregates_a_failing_aggregate_query_does_not_poison_the_next() + { + await OpenSeededAsync(); + + await ExecuteExpectingFailureAsync(Page, + new WireRequest("SELECT ef_sum(value) FROM rows WHERE no_such_column = 1")); + + var whole = await QueryAsync(Page, "SELECT ef_sum(value) FROM rows"); + AssertJson.AreEqual("""[["1353.9178"]]""", whole.Rows); + } + + // --------------------------------------------------------------------------------------------- + // ordering and comparison + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Ordering_ORDER_BY_COLLATE_EF_DECIMAL_is_numeric_not_lexicographic() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, "SELECT id FROM rows ORDER BY value COLLATE EF_DECIMAL, id"); + AssertJson.AreEqual("[4, 8, 1, 6, 5, 10, 3, 2, 7, 9]", Column(result)); + } + + [TestMethod] + public async Task Ordering_ORDER_BY_DESC_is_the_reverse_numeric_order() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, + "SELECT id FROM rows ORDER BY value COLLATE EF_DECIMAL DESC, id"); + AssertJson.AreEqual("[9, 7, 2, 3, 10, 5, 6, 1, 8, 4]", Column(result)); + } + + [TestMethod] + public async Task Ordering_nullable_ORDER_BY_puts_SQLite_nulls_first_then_numeric_order() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, + "SELECT id FROM rows ORDER BY optional COLLATE EF_DECIMAL, id"); + AssertJson.AreEqual("[1, 3, 6, 4, 8, 5, 10, 2, 7, 9]", Column(result)); + } + + [TestMethod] + public async Task Ordering_without_the_collation_9_sorts_after_100() + { + await OpenSeededAsync(); + + // The failure this prevents: a missing collation looks like a working ORDER BY until the data set + // contains values whose TEXT order and numeric order disagree. + var result = await QueryAsync(Page, "SELECT value FROM rows WHERE value IN ('9', '100') ORDER BY value"); + AssertJson.AreEqual("""["100", "9"]""", Column(result)); + } + + [TestMethod] + [DataRow("ef_compare(value, '2.5') > 0", new[] { 2, 3, 7, 9, 10 }, DisplayName = "greater than")] + [DataRow("ef_compare(value, '2.5') >= 0", new[] { 2, 3, 5, 7, 9, 10 }, DisplayName = "greater or equal")] + [DataRow("ef_compare(value, '2.5') < 0", new[] { 1, 4, 6, 8 }, DisplayName = "less than")] + [DataRow("ef_compare(value, '0') < 0", new[] { 4, 8 }, DisplayName = "less than zero")] + [DataRow("ef_compare(value, '9') = 0", new[] { 3 }, DisplayName = "equals via compare")] + [DataRow("ef_compare(value, '9') <> 0", new[] { 1, 2, 4, 5, 6, 7, 8, 9, 10 }, DisplayName = "not equals via compare")] + [DataRow("optional IS NULL", new[] { 1, 3, 6 }, DisplayName = "null check")] + [DataRow("ef_compare(ef_multiply(value, '2'), '10') > 0", new[] { 2, 3, 7, 9 }, DisplayName = "computed filter")] + public async Task Ordering_filter(string where, int[] expected) + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, $"SELECT id FROM rows WHERE {where} ORDER BY id"); + AssertJson.AreEqual(JsonSerializer.Serialize(expected), Column(result)); + } + + [TestMethod] + public async Task Ordering_TEXT_equality_is_exact_which_is_why_the_stored_form_has_to_be_canonical() + { + await OpenSeededAsync(); + + // These rows were seeded as literal TEXT, so `value` holds '9' rather than the canonical '9.0'. SQLite's + // `=` does not care that they are the same number - and EF compiles `== 9m` to exactly this comparison, + // against a literal it renders as '9.0'. That is the whole reason decimals are written through + // toSqlText: a row stored in any other form is invisible to the lookup that goes looking for it. + AssertJson.AreEqual("[3]", Column(await QueryAsync(Page, "SELECT id FROM rows WHERE value = '9'"))); + AssertJson.AreEqual("[]", Column(await QueryAsync(Page, "SELECT id FROM rows WHERE value = '9.0'"))); + AssertJson.AreEqual("[3]", Column(await QueryAsync(Page, "SELECT id FROM rows WHERE ef_compare(value, '9.0') = 0"))); + } + + [TestMethod] + public async Task Ordering_an_ef_result_is_in_the_canonical_form_EF_compares_against() + { + await OpenSeededAsync(); + + // The failure this prevents: writing `10` where Microsoft.Data.Sqlite writes `10.0`, so + // `WHERE price = '10.0'` - which is what EF generates for `== 10m` - matches nothing. + var result = await QueryAsync(Page, + "SELECT ef_multiply('5', '2') = '10.0', ef_add('9', '1') = '10.0', ef_negate('10') = '-10.0'"); + AssertJson.AreEqual("[[1, 1, 1]]", result.Rows); + } + + // --------------------------------------------------------------------------------------------- + // regexp + // --------------------------------------------------------------------------------------------- + + [TestMethod] + [DataRow(@"^a", new[] { 1, 6 }, DisplayName = @"REGEXP ^a")] + [DataRow(@"^A", new[] { 2 }, DisplayName = @"REGEXP ^A")] + [DataRow(@"a", new[] { 1, 2, 3, 4, 6, 9, 10 }, DisplayName = @"REGEXP a")] + [DataRow(@"[0-9]+", new[] { 3, 4, 7 }, DisplayName = @"REGEXP [0-9]+")] + [DataRow(@"^[0-9]+$", new[] { 7 }, DisplayName = @"REGEXP ^[0-9]+$")] + [DataRow(@"\d", new[] { 3, 4, 7 }, DisplayName = @"REGEXP \d")] + [DataRow(@"a\.b", new[] { 6 }, DisplayName = @"REGEXP a\.b")] + [DataRow(@"\[y\]", new[] { 8 }, DisplayName = @"REGEXP \[y\]")] + [DataRow(@"^$", new[] { 5 }, DisplayName = @"REGEXP ^$")] + [DataRow(@"^\s+", new[] { 9 }, DisplayName = @"REGEXP ^\s+")] + [DataRow(@"tail$", new[] { 10 }, DisplayName = @"REGEXP tail$")] + [DataRow(@"(alpha|beta)", new[] { 1, 3 }, DisplayName = @"REGEXP (alpha|beta)")] + [DataRow(@"z{1}", new[] { 8 }, DisplayName = @"REGEXP z{1}")] + [DataRow(@"^.$", new int[] { }, DisplayName = @"REGEXP ^.$")] + [DataRow(@"a(?=l)", new[] { 1 }, DisplayName = @"REGEXP a(?=l)")] + [DataRow(@"a(?!l)", new[] { 1, 2, 3, 4, 6, 9, 10 }, DisplayName = @"REGEXP a(?!l)")] + [DataRow(@"(\w)\1", new[] { 4 }, DisplayName = @"REGEXP (\w)\1")] + public async Task REGEXP(string pattern, int[] expected) + { + await OpenSeededAsync(); + + var result = await QueryAsync( + Page, + $"SELECT id FROM rows WHERE text REGEXP {SqlQuote(pattern)} ORDER BY id"); + AssertJson.AreEqual(JsonSerializer.Serialize(expected), Column(result)); + } + + [TestMethod] + public async Task Regexp_pattern_input_is_the_argument_order_SQLite_uses() + { + await OpenSeededAsync(); + + // A transposition would pass '^a' against 'alpha' if someone called IsMatch-style regexp(input, pattern) + // and the test only used the operator. + var result = await QueryAsync(Page, "SELECT regexp('^a', 'alpha'), regexp('alpha', '^a')"); + AssertJson.AreEqual("[[1, 0]]", result.Rows); + } + + [TestMethod] + public async Task Regexp_a_null_operand_yields_null() + { + await OpenSeededAsync(); + + var result = await QueryAsync(Page, "SELECT regexp(NULL, text), text REGEXP NULL FROM rows WHERE id = 1"); + AssertJson.AreEqual("[[null, null]]", result.Rows); + } + + // The file's beforeEach: every test starts from the seeded data set. + private async Task OpenSeededAsync() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = "functions.db" }); + await SeedAsync(); + } + + private async Task SeedAsync() + { + await ExecAsync(Page, + """ + CREATE TABLE rows ( + id INTEGER PRIMARY KEY, + value TEXT NOT NULL, + optional TEXT, + bucket INTEGER NOT NULL, + text TEXT NOT NULL + ) + """); + + foreach (var row in Rows) + { + await ExecAsync( + Page, + "INSERT INTO rows (id, value, optional, bucket, text) VALUES (@id, @value, @optional, @bucket, @text)", + new WireParameter("@id", row.Id), + new WireParameter("@value", row.Value), + new WireParameter("@optional", row.Optional), + new WireParameter("@bucket", row.Bucket), + new WireParameter("@text", row.Text)); + } + } + + /// The first cell of every row, as JSON for . + private static JsonElement Column(WireResult result) + => JsonSerializer.SerializeToElement(result.Rows.EnumerateArray().Select(row => row[0]).ToArray()); + + private static string SqlQuote(string value) => $"'{value.Replace("'", "''")}'"; +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/GlobalUsings.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/GlobalUsings.cs new file mode 100644 index 00000000000..974ab1211e3 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/GlobalUsings.cs @@ -0,0 +1 @@ +global using Microsoft.VisualStudio.TestTools.UnitTesting; diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/IndexedDbTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/IndexedDbTests.cs new file mode 100644 index 00000000000..b27c57e197f --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/IndexedDbTests.cs @@ -0,0 +1,343 @@ +using System.Text.Json; +using System.Text.RegularExpressions; +using Bit.Bsqlite.Tests.E2E.Infrastructure; +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E; + +/// The IndexedDB storage provider: persistence, the admin module, and several tabs on one database. +[TestClass] +public sealed class IndexedDbTests : BsqliteTest +{ + // --------------------------------------------------------------------------------------------- + // IndexedDB persistence + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Persistence_opens_on_an_async_capable_build_not_the_synchronous_one() + { + var (_, build) = await OpenIdbAsync(Page, UniqueName("idb-build")); + CollectionAssert.Contains(new[] { "jspi", "asyncify" }, build); + } + + [TestMethod] + public async Task Persistence_survives_a_worker_restart() + { + var databaseName = UniqueName("idb-persist"); + await OpenIdbAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Kept')"); + + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + await OpenIdbAsync(Page, databaseName); + AssertJson.AreEqual("""[["Kept"]]""", (await QueryAsync(Page, "SELECT name FROM product")).Rows); + } + + [TestMethod] + public async Task Persistence_an_uncommitted_write_does_not_survive_a_killed_worker() + { + var databaseName = UniqueName("idb-crash"); + await OpenIdbAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Committed')"); + await ExecAsync(Page, "BEGIN"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Ghost')"); + + await Page.EvaluateAsync("() => globalThis.host.dispose()"); + + await OpenIdbAsync(Page, databaseName); + AssertJson.AreEqual("""[["Committed"]]""", (await QueryAsync(Page, "SELECT name FROM product ORDER BY id")).Rows); + } + + [TestMethod] + public async Task Persistence_two_workers_can_open_the_same_database() + { + var databaseName = UniqueName("idb-shared"); + await OpenIdbAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('FromFirst')"); + + var second = await Page.EvaluateAsync( + """ + async options => { + const { decodeValue } = globalThis.BitBsqlite.host; + const host = await globalThis.BitBsqlite.host.open(options); + try { + const result = await host.execute([{ + commandText: 'SELECT name FROM product', + resultKind: 'reader', + }]); + return result[0].rows.map(row => + row.v.map((value, i) => decodeValue(row.t[i], value))); + } finally { + host.dispose(); + } + } + """, Storages.IndexedDb(databaseName).ToArgument()); + + AssertJson.AreEqual("""[["FromFirst"]]""", second); + } + + [TestMethod] + public async Task Persistence_PRAGMA_page_size_assignment_is_rejected() + { + await OpenIdbAsync(Page, UniqueName("idb-pagesize")); + + var error = await ExecuteExpectingFailureAsync(Page, new WireRequest("PRAGMA page_size=8192", ResultKind: "nonQuery")); + + Assert.IsNotNull(error); + Assert.MatchesRegex(new Regex("page_size", RegexOptions.IgnoreCase), error.Message); + Assert.MatchesRegex(new Regex("fixed", RegexOptions.IgnoreCase), error.Message); + } + + [TestMethod] + public async Task Persistence_the_default_page_size_is_4096() + { + await OpenIdbAsync(Page, UniqueName("idb-defaultpage")); + AssertJson.AreEqual("[[4096]]", (await QueryAsync(Page, "PRAGMA page_size")).Rows); + } + + [TestMethod] + public async Task Persistence_cache_size_is_raised_so_batch_atomic_writes_can_keep_the_journal_in_cache() + { + await OpenIdbAsync(Page, UniqueName("idb-cache")); + var cacheSize = (await QueryAsync(Page, "PRAGMA cache_size")).Rows[0][0]; + Assert.AreEqual(-8000, cacheSize.GetInt32()); + } + + [TestMethod] + public async Task Persistence_the_synchronous_build_cannot_host_this_VFS() + { + await GotoHostPageAsync(Page); + + var error = await Page.EvaluateAsync( + """ + async options => { + try { + await globalThis.BitBsqlite.host.open(options); + return null; + } catch (e) { + return e.message; + } + } + """, new HostOptions + { + DatabaseName = UniqueName("idb-sync"), + RequiredBuild = "synchronous", + Vfs = Storages.IndexedDbVfs, + }.ToArgument()); + + Assert.IsNotNull(error); + } + + // --------------------------------------------------------------------------------------------- + // IndexedDB admin + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Admin_import_export_list_and_delete() + { + await GotoHostPageAsync(Page); + + var name = UniqueName("idb-admin"); + var image = Enumerable.Range(0, 64).Select(i => i * 7 % 251).ToArray(); + + var outcome = await Page.EvaluateAsync( + """ + async ({ name, image }) => { + const admin = await import('/_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-admin.js'); + await admin.importDatabase(name, new Uint8Array(image)); + const exported = Array.from(await admin.exportDatabase(name)); + const listed = await admin.list(); + const existed = await admin.exists(name); + await admin.deleteDatabase(name); + return { + exported, + listed: listed.includes(name), + existed, + existsAfterDelete: await admin.exists(name), + }; + } + """, new { name, image }); + + AssertJson.AreEqual(JsonSerializer.Serialize(image), outcome.GetProperty("exported")); + Assert.IsTrue(outcome.GetProperty("listed").GetBoolean()); + Assert.IsTrue(outcome.GetProperty("existed").GetBoolean()); + Assert.IsFalse(outcome.GetProperty("existsAfterDelete").GetBoolean()); + } + + [TestMethod] + public async Task Admin_export_of_an_engine_created_database_is_a_SQLite_file_image() + { + var databaseName = UniqueName("idb-export"); + await OpenIdbAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Migrated')"); + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + var header = await Page.EvaluateAsync( + """ + async name => { + const admin = await import('/_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-admin.js'); + const bytes = await admin.exportDatabase(name); + return String.fromCharCode(...bytes.subarray(0, 15)); + } + """, databaseName); + + Assert.AreEqual("SQLite format 3", header); + } + + /// + /// A tab killed mid-transaction leaves its blocks behind, one version below the committed one and + /// marked by pendingVersion, until a worker next takes a shared lock. An export before + /// then - an automatic migration, say - must not read them: the image would mix in a transaction + /// that never committed and still pass for a SQLite file. + /// + [TestMethod] + public async Task Admin_export_leaves_out_the_blocks_of_a_transaction_that_never_committed() + { + var databaseName = UniqueName("idb-pending"); + await OpenIdbAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Committed')"); + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + var outcome = await Page.EvaluateAsync( + """ + async name => { + const admin = await import('/_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-admin.js'); + const committed = Array.from(await admin.exportDatabase(name)); + + // What the VFS leaves when its tab dies between writing a transaction's pages and + // committing them: a newer block for every page, and the metadata marking it pending. + const path = admin.pathOf(name); + const db = await new Promise((resolve, reject) => { + const request = indexedDB.open(admin.IDB_NAME, admin.IDB_VERSION); + request.onsuccess = () => resolve(request.result); + request.onerror = () => reject(request.error); + }); + + await new Promise((resolve, reject) => { + const tx = db.transaction(['metadata', 'blocks'], 'readwrite'); + tx.oncomplete = resolve; + tx.onerror = () => reject(tx.error); + const metadata = tx.objectStore('metadata'); + const blocks = tx.objectStore('blocks'); + metadata.get(path).onsuccess = event => { + const meta = event.target.result; + const pending = meta.version - 1; + metadata.put({ ...meta, pendingVersion: pending }); + for (let offset = 0; offset < meta.fileSize; offset += 4096) { + const data = new Uint8Array(4096).fill(0xEE); + data.set(new TextEncoder().encode('SQLite format 3\0')); + blocks.put({ path, offset: -offset, version: pending, data }); + } + }; + }); + + db.close(); + return { committed, exported: Array.from(await admin.exportDatabase(name)) }; + } + """, databaseName); + + AssertJson.AreEqual(outcome.GetProperty("committed").GetRawText(), outcome.GetProperty("exported")); + + // And the worker agrees: the VFS drops the pending blocks itself when it opens. + await OpenIdbAsync(Page, databaseName); + AssertJson.AreEqual("""[["Committed"]]""", (await QueryAsync(Page, "SELECT name FROM product")).Rows); + } + + [TestMethod] + public async Task Admin_probe_reports_available_when_indexedDB_exists() + { + await GotoHostPageAsync(Page); + + var probe = await Page.EvaluateAsync( + """ + async () => { + const admin = await import('/_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-admin.js'); + return await admin.probe(); + } + """); + + Assert.IsTrue(probe.GetProperty("available").GetBoolean()); + Assert.AreEqual("true", probe.GetProperty("environment").GetProperty("hasIndexedDB").GetString()); + } + + // --------------------------------------------------------------------------------------------- + // IndexedDB multi-tab + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Multi_tab_a_second_tab_sees_a_committed_write() + { + var databaseName = UniqueName("idb-tabs"); + var context = await NewContextAsync(); + var first = await context.NewPageAsync(); + var second = await context.NewPageAsync(); + + try + { + await OpenIdbAsync(first, databaseName); + await ExecAsync(first, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(first, "INSERT INTO product (name) VALUES ('Shared')"); + + await OpenIdbAsync(second, databaseName); + AssertJson.AreEqual("""[["Shared"]]""", (await QueryAsync(second, "SELECT name FROM product")).Rows); + } + finally + { + await context.CloseAsync(); + } + } + + /// + /// The harder order: the reader opened while the file was still empty and has to notice, at each read + /// transaction, that the other tab grew it. + /// + [TestMethod] + public async Task A_tab_that_opened_first_sees_a_write_that_grew_the_file() + { + var databaseName = UniqueName("idb-grow"); + var context = await NewContextAsync(); + var reader = await context.NewPageAsync(); + var writer = await context.NewPageAsync(); + + try + { + await OpenIdbAsync(reader, databaseName); + AssertJson.AreEqual("[]", (await QueryAsync(reader, "SELECT name FROM sqlite_master WHERE type = 'table'")).Rows); + + await OpenIdbAsync(writer, databaseName); + await ExecAsync(writer, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(writer, "INSERT INTO product (name) VALUES ('Grown')"); + + AssertJson.AreEqual("""[["Grown"]]""", (await QueryAsync(reader, "SELECT name FROM product")).Rows); + } + finally + { + await context.CloseAsync(); + } + } + + private static Task OpenIdbAsync(IPage page, string databaseName) => OpenHostAsync(page, Storages.IndexedDb(databaseName)); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/AssetHost.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/AssetHost.cs new file mode 100644 index 00000000000..a93fd73f37c --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/AssetHost.cs @@ -0,0 +1,95 @@ +using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Hosting; +using Microsoft.AspNetCore.StaticFiles; +using Microsoft.Extensions.FileProviders; +using Microsoft.Extensions.Hosting; +using Microsoft.Extensions.Logging; + +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// Serves the libraries' static assets the way a Blazor application does - each package's wwwroot under +/// _content/<package>/ - next to the test pages, so the tests exercise the same URLs and MIME +/// types production uses. Kestrel in this process, on a free loopback port, for the whole run. +/// +/// +/// +/// Deliberately sends no COOP/COEP headers: the default tier must work without cross-origin isolation, and +/// a test server that granted it would hide the day the library accidentally depended on it. +/// +/// +/// The same layout is repeated under /nested/, which is how an application published under a +/// sub-path (<base href="/nested/">) serves its assets. TestPages/nested/index.html loads the +/// library from there, so the tests can prove that no module is reached by a root-relative URL. +/// +/// +public sealed class AssetHost : IAsyncDisposable +{ + private readonly WebApplication _app; + + private AssetHost(WebApplication app, string baseUrl) + { + _app = app; + BaseUrl = baseUrl; + } + + /// The URL the assets are served from, without a trailing slash. + public string BaseUrl { get; } + + public static async Task StartAsync() + { + foreach (var package in RepoLayout.Packages) + { + var root = RepoLayout.WebRoot(package); + if (Directory.Exists(root) is false || Directory.EnumerateFiles(root, "*.js").Any() is false) + throw new DirectoryNotFoundException( + $"{root} holds no compiled modules. Build the libraries first (building this test project does), " + + "after a restore that vendored the wa-sqlite engine."); + } + + var baseUrl = $"http://127.0.0.1:{ChildProcess.FreePort()}"; + + var builder = WebApplication.CreateSlimBuilder(); + builder.WebHost.UseUrls(baseUrl); + builder.Logging.ClearProviders(); + + var app = builder.Build(); + + var contentTypes = new FileExtensionContentTypeProvider(); + contentTypes.Mappings[".mjs"] = "text/javascript"; + contentTypes.Mappings[".wasm"] = "application/wasm"; + + foreach (var basePath in new[] { "", "/nested" }) + { + foreach (var package in RepoLayout.Packages) + { + Serve(app, $"{basePath}/_content/{package}", RepoLayout.WebRoot(package), contentTypes); + } + } + + Serve(app, "", RepoLayout.TestPages(), contentTypes); + + await app.StartAsync(); + return new AssetHost(app, baseUrl); + } + + private static void Serve(WebApplication app, string requestPath, string directory, IContentTypeProvider contentTypes) + { + app.UseStaticFiles(new StaticFileOptions + { + RequestPath = requestPath, + FileProvider = new PhysicalFileProvider(directory), + ContentTypeProvider = contentTypes, + ServeUnknownFileTypes = true, + // Every test reloads modules a rebuild may just have replaced; a cached copy would pair new + // JavaScript with old. + OnPrepareResponse = context => context.Context.Response.Headers.CacheControl = "no-store", + }); + } + + public async ValueTask DisposeAsync() + { + await _app.StopAsync(); + await _app.DisposeAsync(); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/BsqliteTest.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/BsqliteTest.cs new file mode 100644 index 00000000000..50ba2b568ab --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/BsqliteTest.cs @@ -0,0 +1,250 @@ +using System.Text.Json; +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// Base of every library suite: a browser context and page of its own per test against the shared asset +/// host, and the helpers that drive the library from test code. +/// +/// +/// +/// Tests write plain values and read plain results. Batches go through the binary wire path - the one the +/// .NET transport uses - encoded and decoded in the page with the library's own codec, so every test here +/// exercises the real format rather than a parallel one. InteropTests covers the tagged JSON path +/// separately. +/// +/// +/// Values that JSON cannot carry are tagged again on the way out of the page, so that "came back as a 64-bit +/// integer" and "came back as a blob" stay observable instead of being silently coerced: a BigInt arrives as +/// { "bigint": "…" }, a blob as { "blob": [ … ] }, and a non-finite REAL - which Playwright cannot +/// hand to .NET as a at all - as { "real": "Infinity" }. +/// +/// +public abstract class BsqliteTest +{ + private readonly List _contexts = []; + private PlaywrightSession.Session? _session; + private IPage? _page; + + public TestContext TestContext { get; set; } = default!; + + /// The origin the assets and test pages are served from, without a trailing slash. + protected string BaseUrl => Session.Assets!.BaseUrl; + + protected IBrowser Browser => Session.Browser!; + + /// chromium, firefox or webkit: see . + protected static string BrowserName => E2EEnvironment.Browser; + + /// A page in a context of its own, opened for every test (Playwright Test's page fixture). + protected IPage Page => _page ?? throw new InvalidOperationException("The page is opened in TestInitialize."); + + private PlaywrightSession.Session Session => _session ?? throw new InvalidOperationException("The session is opened in TestInitialize."); + + [TestInitialize] + public async Task OpenSessionAsync() + { + _session = await PlaywrightSession.GetAsync(); + if (_session.Unavailable is { } reason) + { + // A build server must not report a suite that never ran as anything but a failure. + if (E2EEnvironment.IsContinuousIntegration) Assert.Fail("The browser tests could not run: " + reason); + Assert.Inconclusive("The browser tests could not run: " + reason); + } + + _page = await NewPageAsync(); + } + + [TestCleanup] + public async Task CloseContextsAsync() + { + foreach (var context in _contexts) + { + try { await context.CloseAsync(); } + catch (PlaywrightException) { /* the browser went away first */ } + } + _contexts.Clear(); + } + + /// + /// A fresh browser context, closed when the test ends. Relative URLs resolve against the asset host, so a + /// page navigates with GotoAsync("/index.html"). Pages of one context share their origin's storage, + /// which is what the multi-tab tests need; separate contexts do not. + /// + protected Task NewContextAsync() => NewContextAsync(new BrowserNewContextOptions { BaseURL = BaseUrl }); + + /// A fresh browser context with , closed when the test ends. + protected async Task NewContextAsync(BrowserNewContextOptions options) + { + var context = await Browser.NewContextAsync(options); + _contexts.Add(context); + return context; + } + + /// A page in a fresh context of its own. + protected async Task NewPageAsync() => await (await NewContextAsync()).NewPageAsync(); + + /// Reports the test as inconclusive - MSTest's skip - with when holds. + protected static void SkipWhen(bool condition, string reason) + { + if (condition) Assert.Inconclusive(reason); + } + + /// A database name no other test - and no earlier run on the same browser profile - has used. + protected static string UniqueName(string prefix) => $"{prefix}-{Guid.NewGuid():N}.db"; + + // --------------------------------------------------------------------------------------------- + // The test pages and the host + // --------------------------------------------------------------------------------------------- + + /// Loads a test page and waits until it has imported the library. + protected static async Task GotoHostPageAsync(IPage page, string path = "/index.html") + { + await page.GotoAsync(path); + await page.WaitForFunctionAsync("() => globalThis.BitBsqliteReady === true"); + } + + /// + /// Loads the host page and opens a database, keeping the host as globalThis.host for the helpers + /// below. Returns what the worker reports about itself. + /// + protected static async Task OpenHostAsync(IPage page, HostOptions? options = null) + { + await GotoHostPageAsync(page); + + var version = await page.EvaluateAsync( + """ + async options => { + globalThis.host = await globalThis.BitBsqlite.host.open(options); + return await globalThis.host.version(); + } + """, (options ?? new HostOptions()).ToArgument()); + + return new HostVersion(version.GetProperty("version").GetString()!, version.GetProperty("build").GetString()!); + } + + /// Runs a batch through the binary path, encoding parameters and decoding rows with the library's wire helpers. + protected static async Task ExecuteAsync(IPage page, params WireRequest[] batch) + { + var results = await page.EvaluateAsync( + """ + async batch => { + const { encodeValue, decodeValue, encodeBatchBinary, decodeReplyBinary } = globalThis.BitBsqlite.host; + + const naturalValue = p => { + if (p.bytes) return new Uint8Array(p.bytes); + if (p.big) return BigInt(p.big); + return p.value; + }; + + const wire = batch.map(request => ({ + commandText: request.commandText, + resultKind: request.resultKind ?? 'nonQuery', + maxRows: request.maxRows ?? undefined, + parameters: (request.parameters ?? []).map(p => ({ + name: p.name, + ...encodeValue(naturalValue(p)), + })), + })); + + const results = decodeReplyBinary(await globalThis.host.executeBinary(encodeBatchBinary(wire))); + + // Re-tagged for the trip to the test runner: JSON has no BigInt, no bytes and no Infinity. + const forTest = value => { + if (typeof value === 'bigint') return { bigint: String(value) }; + if (value instanceof Uint8Array) return { blob: Array.from(value) }; + if (typeof value === 'number' && !Number.isFinite(value)) return { real: String(value) }; + return value; + }; + + return results.map(result => ({ + columnNames: result.columnNames, + columnTypes: result.columnTypes, + recordsAffected: result.recordsAffected, + lastInsertRowId: Number(result.lastInsertRowId), + rows: result.rows.map(row => row.v.map((value, i) => forTest(decodeValue(row.t[i], value)))), + })); + } + """, batch.Select(r => r.ToArgument()).ToArray()); + + return [.. results.EnumerateArray().Select(WireResult.From)]; + } + + /// Convenience for the common case of one statement whose rows are wanted. + protected static async Task QueryAsync(IPage page, string commandText, params WireParameter[] parameters) + => (await ExecuteAsync(page, new WireRequest(commandText, parameters, "reader")))[0]; + + /// Convenience for one statement whose rows are not wanted. + protected static async Task ExecAsync(IPage page, string commandText, params WireParameter[] parameters) + => (await ExecuteAsync(page, new WireRequest(commandText, parameters, "nonQuery")))[0]; + + /// + /// Runs a batch expected to fail, returning the error as data - or null when it did not fail. A captured + /// rejection rather than an exception from Playwright, so the SQLite result code survives the boundary: a + /// test asserting only on the message would pass for the wrong error. + /// + protected static async Task ExecuteExpectingFailureAsync(IPage page, params WireRequest[] batch) + { + var failure = await page.EvaluateAsync( + """ + async batch => { + const { encodeValue, encodeBatchBinary, decodeReplyBinary } = globalThis.BitBsqlite.host; + + const wire = batch.map(request => ({ + commandText: request.commandText, + resultKind: request.resultKind ?? 'nonQuery', + parameters: (request.parameters ?? []).map(p => ({ name: p.name, ...encodeValue(p.value) })), + })); + + try { + decodeReplyBinary(await globalThis.host.executeBinary(encodeBatchBinary(wire))); + return null; + } catch (error) { + return { message: error.message, sqliteCode: error.sqliteCode ?? null, name: error.name }; + } + } + """, batch.Select(r => r.ToArgument()).ToArray()); + + return WireFailure.From(failure); + } + + /// + /// Starts recording the host's change notifications. Each later call to + /// returns the table lists received since the previous one, each sorted. + /// + /// + /// The worker posts a notification before the reply to the batch that caused it, and the host handles + /// messages in order, so a notification is recorded by the time that batch's promise settles. + /// + protected static Task RecordNotificationsAsync(IPage page) => page.EvaluateAsync( + """ + () => { + globalThis.notified = []; + globalThis.host.onNotify(payload => globalThis.notified.push([...payload.tables].sort())); + } + """); + + /// The notifications recorded since the last call; see . + protected static async Task NotificationsAsync(IPage page) + { + var notified = await page.EvaluateAsync("() => globalThis.notified.splice(0)"); + return [.. notified.EnumerateArray().Select(tables => tables.EnumerateArray().Select(t => t.GetString()!).ToArray())]; + } + + /// + /// Skips the current test where the browser has no OPFS. + /// + /// + /// Playwright's own WebKit builds expose no navigator.storage at all - not in the window, not in a + /// worker, secure context notwithstanding - so anything that opens an OPFS database there fails before + /// SQLite is involved. Safari proper has OPFS; the tests are skipped rather than failed so the suite still + /// reports on what that WebKit can exercise (IndexedDB, Cache Storage, the engine itself). + /// + protected static async Task SkipUnlessOpfsAsync(IPage page, string path = "/index.html") + { + await page.GotoAsync(path); + var available = await page.EvaluateAsync("() => typeof navigator.storage?.getDirectory === 'function'"); + SkipWhen(available is false, "this browser has no navigator.storage.getDirectory, so OPFS cannot be tested"); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/ChildProcess.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/ChildProcess.cs new file mode 100644 index 00000000000..79836a69c07 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/ChildProcess.cs @@ -0,0 +1,115 @@ +using System.Collections.Concurrent; +using System.Diagnostics; +using System.Net; +using System.Net.Sockets; + +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// A process the suite starts and owns: its output is drained (a full pipe would stall it) into a +/// bounded buffer that failure messages quote, and disposing kills the whole process tree. +/// +public sealed class ChildProcess : IAsyncDisposable +{ + private const int KeptOutputLines = 200; + + private readonly Process _process; + private readonly ConcurrentQueue _output = new(); + + private ChildProcess(Process process) => _process = process; + + public bool HasExited => _process.HasExited; + + /// The last lines the process wrote, for failure messages. + public string RecentOutput => string.Join(Environment.NewLine, _output); + + public static ChildProcess Start(string fileName, IEnumerable arguments, string workingDirectory, IDictionary? environment = null) + { + var startInfo = new ProcessStartInfo + { + FileName = fileName, + WorkingDirectory = workingDirectory, + UseShellExecute = false, + CreateNoWindow = true, + RedirectStandardOutput = true, + RedirectStandardError = true, + }; + foreach (var argument in arguments) startInfo.ArgumentList.Add(argument); + foreach (var (name, value) in environment ?? new Dictionary()) startInfo.Environment[name] = value; + + var child = new ChildProcess(new Process { StartInfo = startInfo }); + child._process.OutputDataReceived += (_, e) => child.Keep(e.Data); + child._process.ErrorDataReceived += (_, e) => child.Keep(e.Data); + + if (child._process.Start() is false) + throw new InvalidOperationException($"Failed to start {fileName}."); + + child._process.BeginOutputReadLine(); + child._process.BeginErrorReadLine(); + return child; + } + + /// Polls until it answers with a success status. + public async Task WaitForHttpAsync(string url, TimeSpan timeout) + { + using var client = new HttpClient { Timeout = TimeSpan.FromSeconds(5) }; + var deadline = DateTime.UtcNow + timeout; + + while (DateTime.UtcNow < deadline) + { + if (HasExited) + throw new InvalidOperationException($"The process exited before {url} answered.{Environment.NewLine}{RecentOutput}"); + + try + { + using var response = await client.GetAsync(url); + if (response.IsSuccessStatusCode) return; + } + catch (HttpRequestException) { /* not listening yet */ } + catch (TaskCanceledException) { /* not answering yet */ } + + await Task.Delay(250); + } + + throw new TimeoutException($"{url} did not answer within {timeout}.{Environment.NewLine}{RecentOutput}"); + } + + public static int FreePort() + { + // Bind to port 0 and release it: the OS hands back a free ephemeral port. + var listener = new TcpListener(IPAddress.Loopback, 0); + listener.Start(); + var port = ((IPEndPoint)listener.LocalEndpoint).Port; + listener.Stop(); + return port; + } + + public async ValueTask DisposeAsync() + { + try + { + if (_process.HasExited is false) + { + _process.Kill(entireProcessTree: true); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + await _process.WaitForExitAsync(cts.Token); + } + } + catch (Exception ex) when (ex is InvalidOperationException or OperationCanceledException or System.ComponentModel.Win32Exception) + { + // Best-effort: the process may already be gone, or refuse to die before the run ends. + } + finally + { + _process.Dispose(); + } + } + + private void Keep(string? line) + { + if (line is null) return; + + _output.Enqueue(line); + while (_output.Count > KeptOutputLines) _output.TryDequeue(out _); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/DemoHost.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/DemoHost.cs new file mode 100644 index 00000000000..83a7a7bbc43 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/DemoHost.cs @@ -0,0 +1,54 @@ +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// The demo server the demo tests drive: Bit.Bsqlite.Demo.Server started from its build output on a free +/// port, once for the whole run. Every database the demo opens lives in the browser, and every test opens +/// its own browser context, so sharing the process shares nothing but the server-side copy of the catalog. +/// +public sealed class DemoHost : IAsyncDisposable +{ + private readonly ChildProcess? _process; + + private DemoHost(string baseUrl, ChildProcess? process) + { + BaseUrl = baseUrl; + _process = process; + } + + /// The URL the demo answers at, without a trailing slash. + public string BaseUrl { get; } + + public static async Task StartAsync() + { + if (E2EEnvironment.DemoUrl is { } external) + return new DemoHost(external, process: null); + + var assembly = RepoLayout.DemoServerAssembly(E2EEnvironment.Configuration); + if (File.Exists(assembly) is false) + throw new FileNotFoundException( + $"The demo server has not been built in the {E2EEnvironment.Configuration} configuration. Build it first with " + + $"'dotnet build src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj -c {E2EEnvironment.Configuration}'", assembly); + + var baseUrl = $"http://127.0.0.1:{ChildProcess.FreePort()}"; + + // A build output only finds the static web assets of its references (the WebAssembly client, the + // worker, the engine and the VFSes among them) through the development manifest, which Development + // switches on. + var process = ChildProcess.Start("dotnet", [assembly, "--urls", baseUrl], Path.GetDirectoryName(RepoLayout.DemoServerProject())!, + new Dictionary { ["ASPNETCORE_ENVIRONMENT"] = "Development" }); + + try + { + await process.WaitForHttpAsync(baseUrl + "/", TimeSpan.FromMinutes(2)); + } + catch + { + await process.DisposeAsync(); + throw; + } + + return new DemoHost(baseUrl, process); + } + + public ValueTask DisposeAsync() => _process?.DisposeAsync() ?? ValueTask.CompletedTask; +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/DemoTest.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/DemoTest.cs new file mode 100644 index 00000000000..3fa29e7b1a7 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/DemoTest.cs @@ -0,0 +1,76 @@ +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// Base of the suites that drive the demo website rather than the library's modules: the demo server is +/// started on first use (see ), and every page is opened in a +/// context of its own whose console errors are collected. +/// +public abstract class DemoTest : BsqliteTest +{ + private PlaywrightSession.DemoSession? _demo; + + /// The origin the demo answers at, without a trailing slash. + protected string DemoUrl => (_demo?.Host ?? throw new InvalidOperationException("The demo is started in TestInitialize.")).BaseUrl; + + [TestInitialize] + public async Task StartDemoAsync() + { + _demo = await PlaywrightSession.GetDemoAsync(); + if (_demo.Unavailable is not { } reason) return; + + if (E2EEnvironment.IsContinuousIntegration) Assert.Fail("The demo tests could not run: " + reason); + Assert.Inconclusive("The demo tests could not run: " + reason); + } + + /// + /// A page of the demo in a fresh context - so a fresh origin storage, with no database and no sticky + /// binding yet - with every console error and uncaught exception recorded in . + /// + protected async Task NewDemoPageAsync(List errors) + { + var context = await NewContextAsync(new BrowserNewContextOptions { BaseURL = DemoUrl }); + + // The site loads its typefaces from Google Fonts. Answered locally, with nothing: a runner that cannot reach + // them - or reaches them slowly - says nothing about Bsqlite, and would otherwise hold every page load up + // and report a failed resource as a console error. + await context.RouteAsync(new System.Text.RegularExpressions.Regex(@"^https://fonts\.(googleapis|gstatic)\.com/"), + route => route.FulfillAsync(new RouteFulfillOptions { Status = 200, ContentType = "text/css", Body = "" })); + + var page = await context.NewPageAsync(); + + page.Console += (_, message) => + { + if (message.Type == "error") errors.Add(message.Text); + }; + page.PageError += (_, error) => errors.Add(error); + + return page; + } + + /// Navigates and waits until the page runs in the browser: every page is prerendered first. + protected static async Task GotoInteractiveAsync(IPage page, string path) + { + var response = await page.GotoAsync(path, new PageGotoOptions { Timeout = 90_000 }); + await page.WaitForSelectorAsync("main[data-interactive='true']", new PageWaitForSelectorOptions { Timeout = 90_000 }); + return response; + } + + /// + /// Waits until the session bar reports an open database and returns the backend it landed on. Only the + /// documentation pages have the bar; the overview opens its database without one. + /// + protected static async Task WaitForSessionAsync(IPage page, string? expected = null) + { + await page.WaitForFunctionAsync( + """ + expected => { + const session = document.querySelector('section.session-bar')?.dataset.session; + return expected === null ? !!session && session !== 'closed' : session === expected; + } + """, expected, new PageWaitForFunctionOptions { Timeout = 60_000 }); + + return (await page.Locator("section.session-bar").GetAttributeAsync("data-session"))!; + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/E2EEnvironment.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/E2EEnvironment.cs new file mode 100644 index 00000000000..f9185b14ba1 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/E2EEnvironment.cs @@ -0,0 +1,86 @@ +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// Everything the suite reads from the environment, in one place. Environment variables rather than +/// runsettings, which the Microsoft.Testing.Platform runner does not thread through reliably. +/// +public static class E2EEnvironment +{ + /// + /// BSQLITE_E2E_CONFIGURATION: the build configuration of the demo server the demo tests boot. + /// Defaults to the configuration this test assembly was built in, so a Release test run drives the + /// Release demo that CI builds rather than looking for a Debug build that was never produced. + /// + public static string Configuration { get; } = Read("BSQLITE_E2E_CONFIGURATION") ?? +#if DEBUG + "Debug"; +#else + "Release"; +#endif + + /// + /// BSQLITE_E2E_BROWSER: chromium (the default), firefox or webkit. The storage + /// matrix matters most on the last two - neither has JSPI, so both run the asynchronous VFSes on the + /// Asyncify build - but only Chromium is installed by CI, so they are opt-in and need + /// playwright.ps1 install firefox webkit first. + /// + public static string Browser { get; } = (Read("BSQLITE_E2E_BROWSER") ?? "chromium").ToLowerInvariant(); + + /// + /// BSQLITE_E2E_CHANNEL: e.g. chrome / msedge, to use an installed browser. Without it the suite + /// uses the Chromium Playwright installed and falls back to an installed Chrome, then Edge - which is also + /// what keeps it runnable where Playwright's own browser download is blocked. + /// + public static string? Channel { get; } = Read("BSQLITE_E2E_CHANNEL"); + + /// BSQLITE_E2E_EXECUTABLE: full path to a browser executable of 's family. + public static string? Executable { get; } = Read("BSQLITE_E2E_EXECUTABLE"); + + /// BSQLITE_E2E_HEADED=1: show the browser. + public static bool Headed { get; } = Read("BSQLITE_E2E_HEADED") == "1"; + + /// + /// BSQLITE_E2E_DEMO_URL: an already running demo (for example one started from the IDE). When set + /// the demo tests drive that instead of booting the demo server themselves. + /// + public static string? DemoUrl { get; } = Read("BSQLITE_E2E_DEMO_URL")?.TrimEnd('/'); + + /// + /// BSQLITE_SOAK=1: run the soak tests (category Soak) - a worker killed mid-transaction a + /// thousand times over per storage, and four then eight tabs committing at once. They take minutes rather + /// than seconds, so an ordinary run reports them inconclusive. + /// + public static bool Soak { get; } = Read("BSQLITE_SOAK") is "1" or "true"; + + /// BSQLITE_SOAK_TABS: tabs in the concurrent-write soak (default 8). + public static int SoakTabs { get; } = ReadSize("BSQLITE_SOAK_TABS", 8); + + /// BSQLITE_SOAK_ROWS: rows each of those tabs commits (default 50). + public static int SoakRows { get; } = ReadSize("BSQLITE_SOAK_ROWS", 50); + + /// BSQLITE_SOAK_KILLS: mid-transaction kills per storage (default 1000). + public static int SoakKills { get; } = ReadSize("BSQLITE_SOAK_KILLS", 1000); + + /// + /// BSQLITE_E2E_BENCHMARKS=1: run the benchmark page (category Benchmark) and print its numbers. + /// They are for a human comparing backends and encodings on one machine, not a gate, so an ordinary run + /// reports them inconclusive. + /// + public static bool Benchmarks { get; } = Read("BSQLITE_E2E_BENCHMARKS") is "1" or "true"; + + /// + /// True on a build server (CI=true, which GitHub Actions sets). A missing browser makes the tests + /// inconclusive on a developer machine, but on a build server that would turn "the browser tests never + /// ran" into a green build, so the tests fail there instead. + /// + public static bool IsContinuousIntegration { get; } = string.Equals(Read("CI"), "true", StringComparison.OrdinalIgnoreCase); + + private static string? Read(string name) + { + var value = Environment.GetEnvironmentVariable(name); + return string.IsNullOrWhiteSpace(value) ? null : value.Trim(); + } + + private static int ReadSize(string name, int fallback) + => int.TryParse(Read(name), out var value) && value > 0 ? value : fallback; +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/PlaywrightSession.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/PlaywrightSession.cs new file mode 100644 index 00000000000..c0eeeceba82 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/PlaywrightSession.cs @@ -0,0 +1,133 @@ +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// One Playwright driver, one launched browser and one asset host for the whole run, plus the demo server +/// the first demo test asks for. Tests isolate from each other with a browser context apiece (separate +/// storage - OPFS, IndexedDB, Cache Storage, localStorage - and separate workers), which is far cheaper than +/// a browser apiece and isolates just as well. +/// +/// +/// Whatever stops the suite from running at all - no browser, no compiled assets, a demo that does not +/// start - is captured once as an Unavailable reason, so every test reports it without waiting out +/// the same timeout again. +/// +public static class PlaywrightSession +{ + private static readonly Lazy> _session = new(StartAsync); + private static readonly Lazy> _demo = new(StartDemoAsync); + + public static Task GetAsync() => _session.Value; + + /// The demo server, started on first use: the library tests never need it. + public static Task GetDemoAsync() => _demo.Value; + + public static async Task StopAsync() + { + if (_demo.IsValueCreated && (await _demo.Value).Host is { } demo) await demo.DisposeAsync(); + + if (_session.IsValueCreated is false) return; + + var session = await _session.Value; + + if (session.Browser is not null) + { + try { await session.Browser.CloseAsync(); } + catch (PlaywrightException) { /* the browser is gone already */ } + } + + session.Playwright?.Dispose(); + + if (session.Assets is not null) await session.Assets.DisposeAsync(); + } + + private static async Task StartAsync() + { + IPlaywright playwright; + try + { + playwright = await Playwright.CreateAsync(); + } + catch (Exception ex) when (ex is PlaywrightException or FileNotFoundException or DllNotFoundException) + { + return Session.Fail("Playwright is not available: " + ex.Message); + } + + var browser = await LaunchAsync(playwright); + if (browser is null) + { + playwright.Dispose(); + return Session.Fail($"no {E2EEnvironment.Browser} browser could be launched. Run 'pwsh bin//net10.0/playwright.ps1 install {E2EEnvironment.Browser}'" + + (E2EEnvironment.Browser == "chromium" ? ", or install Chrome or Edge." : ".")); + } + + try + { + return new Session(playwright, browser, await AssetHost.StartAsync(), Unavailable: null); + } + catch (Exception ex) when (ex is DirectoryNotFoundException or FileNotFoundException or IOException) + { + await browser.CloseAsync(); + playwright.Dispose(); + return Session.Fail("the library assets could not be served: " + ex.Message); + } + } + + private static async Task StartDemoAsync() + { + try + { + return new DemoSession(await DemoHost.StartAsync(), Unavailable: null); + } + catch (Exception ex) when (ex is FileNotFoundException or InvalidOperationException or TimeoutException) + { + return new DemoSession(null, $"the demo application could not be started in the {E2EEnvironment.Configuration} configuration: {ex.Message}"); + } + } + + private static async Task LaunchAsync(IPlaywright playwright) + { + var type = E2EEnvironment.Browser switch + { + "firefox" => playwright.Firefox, + "webkit" => playwright.Webkit, + _ => playwright.Chromium, + }; + + // Preferred: the browser Playwright manages. Falling back to an installed Chrome or Edge keeps the + // suite usable where that download is blocked. A channel or an executable asked for explicitly is the + // only candidate, and only Chromium has installed channels to fall back to. + string?[] channels = E2EEnvironment.Channel is { } channel ? [channel] + : E2EEnvironment.Executable is not null || type != playwright.Chromium ? [null] + : [null, "chrome", "msedge"]; + + foreach (var candidate in channels) + { + try + { + return await type.LaunchAsync(new BrowserTypeLaunchOptions + { + Channel = candidate, + ExecutablePath = E2EEnvironment.Executable, + Headless = E2EEnvironment.Headed is false, + }); + } + catch (PlaywrightException) + { + // Try the next candidate. + } + } + + return null; + } + + /// The shared browser and asset host, or the reason the suite cannot run. + public sealed record Session(IPlaywright? Playwright, IBrowser? Browser, AssetHost? Assets, string? Unavailable) + { + public static Session Fail(string reason) => new(null, null, null, reason); + } + + /// The shared demo server, or the reason it could not start. + public sealed record DemoSession(DemoHost? Host, string? Unavailable); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/RepoLayout.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/RepoLayout.cs new file mode 100644 index 00000000000..be0bda5763e --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/RepoLayout.cs @@ -0,0 +1,44 @@ +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// +/// Finds the libraries and the demo server from the test binary's location by walking up, so a run does not +/// depend on the working directory the shell, the IDE or the test runner picked. +/// +public static class RepoLayout +{ + public const string DemoServerName = "Bit.Bsqlite.Demo.Server"; + + /// The packages whose wwwroot is served under _content/<package>/, as a Blazor app maps them. + public static readonly string[] Packages = + [ + "Bit.Bsqlite", + "Bit.Bsqlite.Storage.Opfs", + "Bit.Bsqlite.Storage.IndexedDb", + "Bit.Bsqlite.Storage.CacheStorage", + ]; + + /// The src/Bsqlite folder. + public static string BsqliteRoot() => Path.GetDirectoryName(FindUpward("Bit.Bsqlite.slnx"))!; + + /// A package's wwwroot: its compiled modules plus whatever its restore vendored there. + public static string WebRoot(string package) => Path.Combine(BsqliteRoot(), package, "wwwroot"); + + /// The pages the asset host serves at its root, copied next to the test assembly. + public static string TestPages() => Path.Combine(AppContext.BaseDirectory, "TestPages"); + + public static string DemoServerProject() => Path.Combine(BsqliteRoot(), "Demo", "Server", $"{DemoServerName}.csproj"); + + public static string DemoServerAssembly(string configuration) => + Path.Combine(Path.GetDirectoryName(DemoServerProject())!, "bin", configuration, "net10.0", $"{DemoServerName}.dll"); + + private static string FindUpward(string relativePath) + { + for (var directory = new DirectoryInfo(AppContext.BaseDirectory); directory is not null; directory = directory.Parent) + { + var candidate = Path.Combine(directory.FullName, relativePath); + if (File.Exists(candidate)) return candidate; + } + + throw new FileNotFoundException($"Could not find {relativePath} walking up from {AppContext.BaseDirectory}."); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/Wire.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/Wire.cs new file mode 100644 index 00000000000..d4d2d3769ab --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Infrastructure/Wire.cs @@ -0,0 +1,202 @@ +using System.Text.Json; +using System.Text.Json.Nodes; + +namespace Bit.Bsqlite.Tests.E2E.Infrastructure; + +/// What the worker reports about itself: the SQLite version and the engine build it loaded. +public sealed record HostVersion(string Version, string Build); + +/// +/// The options of the host module's open, as the .NET transport sends them. Unset members are left +/// out of what the page receives rather than sent as null, the same as a JavaScript caller leaving them out. +/// +public sealed record HostOptions +{ + public string DatabaseName { get; init; } = "test.db"; + + /// synchronous or asyncCapable. + public string RequiredBuild { get; init; } = "synchronous"; + + /// The storage provider's VFS module, e.g. . + public JsModule? Vfs { get; init; } + + public StorageLimits? Limits { get; init; } + + public bool? ForeignKeys { get; init; } + + public string[]? OnOpenSql { get; init; } + + public JsModule[]? Extensions { get; init; } + + internal Dictionary ToArgument() + { + var argument = new Dictionary + { + ["databaseName"] = DatabaseName, + ["requiredBuild"] = RequiredBuild, + }; + + if (Vfs is not null) argument["vfs"] = Vfs.ToArgument(); + if (Limits is not null) argument["limits"] = Limits.ToArgument(); + if (ForeignKeys is not null) argument["foreignKeys"] = ForeignKeys; + if (OnOpenSql is not null) argument["onOpenSql"] = OnOpenSql; + if (Extensions is not null) argument["extensions"] = Extensions.Select(e => e.ToArgument()).ToArray(); + return argument; + } +} + +/// A module the worker imports: a storage provider's VFS, or an extension registering functions. +public sealed record JsModule(string ModuleUrl, string RegisterExport = "register") +{ + internal Dictionary ToArgument() => new() { ["moduleUrl"] = ModuleUrl, ["registerExport"] = RegisterExport }; +} + +/// What a storage provider tells the worker it cannot do, so the worker can refuse it in SQL. +public sealed record StorageLimits(bool SupportsMultiDatabaseTransactions, bool CanChangePageSize) +{ + internal Dictionary ToArgument() => new() + { + ["supportsMultiDatabaseTransactions"] = SupportsMultiDatabaseTransactions, + ["canChangePageSize"] = CanChangePageSize, + }; +} + +/// The first-party storages, as each .NET provider describes itself to the worker. +public static class Storages +{ + public static readonly JsModule OpfsVfs = new("/_content/Bit.Bsqlite.Storage.Opfs/bit-bsqlite-opfs-vfs.js"); + public static readonly StorageLimits OpfsLimits = new(SupportsMultiDatabaseTransactions: false, CanChangePageSize: true); + + public static readonly JsModule IndexedDbVfs = new("/_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-vfs.js"); + public static readonly StorageLimits IndexedDbLimits = new(SupportsMultiDatabaseTransactions: true, CanChangePageSize: false); + + public static readonly JsModule CacheStorageVfs = new("/_content/Bit.Bsqlite.Storage.CacheStorage/bit-bsqlite-cache-register.js"); + public static readonly StorageLimits CacheStorageLimits = new(SupportsMultiDatabaseTransactions: true, CanChangePageSize: false); + + public static HostOptions Opfs(string databaseName) => new() + { + DatabaseName = databaseName, + RequiredBuild = "synchronous", + Vfs = OpfsVfs, + Limits = OpfsLimits, + }; + + public static HostOptions IndexedDb(string databaseName) => new() + { + DatabaseName = databaseName, + RequiredBuild = "asyncCapable", + Vfs = IndexedDbVfs, + Limits = IndexedDbLimits, + }; + + public static HostOptions CacheStorage(string databaseName) => new() + { + DatabaseName = databaseName, + RequiredBuild = "asyncCapable", + Vfs = CacheStorageVfs, + Limits = CacheStorageLimits, + }; +} + +/// One request of a batch. is nonQuery, scalar or reader. +public sealed record WireRequest(string CommandText, IReadOnlyList? Parameters = null, string ResultKind = "nonQuery", int? MaxRows = null) +{ + internal Dictionary ToArgument() + { + var argument = new Dictionary + { + ["commandText"] = CommandText, + ["resultKind"] = ResultKind, + ["parameters"] = (Parameters ?? []).Select(p => p.ToArgument()).ToArray(), + }; + + if (MaxRows is not null) argument["maxRows"] = MaxRows; + return argument; + } +} + +/// +/// A parameter with a natural value. Two kinds cannot cross into the page as themselves and are written as +/// for bytes and for a 64-bit integer. +/// +public sealed record WireParameter(string Name, object? Value = null) +{ + public byte[]? Bytes { get; init; } + + public string? Big { get; init; } + + public static WireParameter Blob(string name, params byte[] bytes) => new(name) { Bytes = bytes }; + + public static WireParameter BigInt(string name, long value) => new(name) { Big = value.ToString(System.Globalization.CultureInfo.InvariantCulture) }; + + public static WireParameter BigInt(string name, string value) => new(name) { Big = value }; + + internal Dictionary ToArgument() + { + var argument = new Dictionary { ["name"] = Name, ["value"] = Value }; + if (Bytes is not null) argument["bytes"] = Bytes.Select(b => (int)b).ToArray(); + if (Big is not null) argument["big"] = Big; + return argument; + } +} + +/// One result set of a reply, with the rows kept as JSON for . +public sealed record WireResult(string[] ColumnNames, string?[] ColumnTypes, int RecordsAffected, long LastInsertRowId, JsonElement Rows) +{ + internal static WireResult From(JsonElement result) => new( + [.. result.GetProperty("columnNames").EnumerateArray().Select(n => n.GetString()!)], + [.. result.GetProperty("columnTypes").EnumerateArray().Select(t => t.ValueKind == JsonValueKind.Null ? null : t.GetString())], + result.GetProperty("recordsAffected").GetInt32(), + result.GetProperty("lastInsertRowId").GetInt64(), + result.GetProperty("rows").Clone()); +} + +/// A failed batch, as the reply carried it. +public sealed record WireFailure(string Message, int? SqliteCode, string Name) +{ + internal static WireFailure? From(JsonElement failure) => failure.ValueKind is JsonValueKind.Null or JsonValueKind.Undefined + ? null + : new WireFailure( + failure.GetProperty("message").GetString()!, + failure.GetProperty("sqliteCode").ValueKind == JsonValueKind.Number ? failure.GetProperty("sqliteCode").GetInt32() : null, + failure.GetProperty("name").GetString()!); +} + +/// +/// Structural equality of a value that came out of the page and the JSON a test expects - the counterpart +/// of Playwright Test's toEqual. Numbers compare by value, so 42 and 42.0 are equal. +/// +public static class AssertJson +{ + public static void AreEqual(string expectedJson, JsonElement actual, string? message = null) + { + var expected = JsonNode.Parse(expectedJson); + var actualNode = WithoutObjectIds(JsonNode.Parse(actual.GetRawText())); + + if (JsonNode.DeepEquals(expected, actualNode) is false) + { + Assert.Fail($"{(message is null ? "" : message + " ")}Expected {expected?.ToJsonString() ?? "null"} but was {actualNode?.ToJsonString() ?? "null"}."); + } + } + + /// + /// Removes the $id member Playwright gives every object it returns as a . Its + /// protocol numbers the objects of a result so that shared references and cycles survive the trip; that + /// numbering is transport metadata rather than part of the value, and toEqual never sees it. + /// + private static JsonNode? WithoutObjectIds(JsonNode? node) + { + switch (node) + { + case JsonObject obj: + obj.Remove("$id"); + foreach (var (_, child) in obj) WithoutObjectIds(child); + break; + case JsonArray array: + foreach (var child in array) WithoutObjectIds(child); + break; + } + + return node; + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/InteropTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/InteropTests.cs new file mode 100644 index 00000000000..af6547eb6f2 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/InteropTests.cs @@ -0,0 +1,235 @@ +using System.Text.Json; +using System.Text.RegularExpressions; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// +/// The exact conversation the .NET transport has with the host: create a host, then drive every operation +/// through call so a failure is data rather than a thrown error. These tests pin the envelope shape +/// BsqliteWireFormat.DecodeCall reads; a change here without a matching change there is a broken +/// release. +/// +[TestClass] +public sealed class InteropTests : BsqliteTest +{ + [TestMethod] + public async Task CreateHost_plus_call_opens_a_database_without_throwing() + { + await GotoHostPageAsync(Page); + + var envelope = await Page.EvaluateAsync( + """ + async () => { + const host = globalThis.BitBsqlite.host.createHost(); + globalThis.host = host; + return await host.call({ + kind: 'open', + databaseName: 'interop.db', + requiredBuild: 'synchronous', + }); + } + """); + + AssertJson.AreEqual( + """ + { + "ok": true, + "result": { "build": "synchronous", "vfsName": null, "reused": false } + } + """, envelope); + } + + /// + /// wa-sqlite binds a JavaScript number outside int32 with bind_double. A value tagged as + /// an integer has to reach the engine as one, or Date.now() is stored as REAL - and in a + /// TEXT column as '1700000000000.0'. + /// + [TestMethod] + public async Task An_integer_parameter_outside_int32_is_bound_as_an_integer() + { + await GotoHostPageAsync(Page); + + var envelope = await Page.EvaluateAsync( + """ + async () => { + const { createHost, encodeValue } = globalThis.BitBsqlite.host; + const host = createHost(); + await host.call({ kind: 'open', databaseName: 'int64.db', requiredBuild: 'synchronous' }); + + await host.call({ + kind: 'execute', + batch: [{ commandText: 'CREATE TABLE t (s TEXT)', resultKind: 'nonQuery', parameters: [] }], + }); + + await host.call({ + kind: 'execute', + batch: [{ + commandText: 'INSERT INTO t (s) VALUES (@v)', + resultKind: 'nonQuery', + parameters: [{ name: '@v', ...encodeValue(1700000000000) }], + }], + }); + + const result = await host.call({ + kind: 'execute', + batch: [{ + commandText: 'SELECT typeof(@a), typeof(@b), typeof(@c), (SELECT s FROM t)', + resultKind: 'reader', + parameters: [ + { name: '@a', ...encodeValue(3000000000) }, + { name: '@b', ...encodeValue(-3000000000) }, + { name: '@c', ...encodeValue(42) }, + ], + }], + }); + + host.dispose(); + return result; + } + """); + + Assert.IsTrue(envelope.GetProperty("ok").GetBoolean(), envelope.GetRawText()); + var row = envelope.GetProperty("result")[0].GetProperty("rows")[0]; + AssertJson.AreEqual("""{ "t": [3, 3, 3, 3], "v": ["integer", "integer", "integer", "1700000000000"] }""", row); + } + + /// + /// The lock EF migrations run under: shared by every host on the origin - every tab's worker - and + /// let go of when its holder releases it, withdraws a request it gave up on, or is disposed. + /// + [TestMethod] + public async Task Hosts_share_named_locks_across_the_origin() + { + await GotoHostPageAsync(Page); + + var outcome = await Page.EvaluateAsync( + """ + async () => { + const { createHost } = globalThis.BitBsqlite.host; + const name = 'bit-bsqlite:test:' + crypto.randomUUID(); + const settle = () => new Promise(resolve => setTimeout(resolve, 100)); + const first = createHost(); + const second = createHost(); + const third = createHost(); + + await first.acquireLock(name, 1); + let secondHeld = false; + const secondWaits = second.acquireLock(name, 1).then(() => { secondHeld = true; }); + await settle(); + const heldOff = !secondHeld; + + first.releaseLock(1); + await secondWaits; + + // A request given up on while still queued is withdrawn, not granted later to nobody. + const withdrawn = first.acquireLock(name, 2).then(() => 'granted', error => error.name); + first.releaseLock(2); + + // Disposing the holder releases what it held. + second.dispose(); + await third.acquireLock(name, 1); + + third.releaseLock(1); + first.dispose(); + third.dispose(); + return { heldOff, secondHeld, withdrawn: await withdrawn }; + } + """); + + AssertJson.AreEqual("""{ "heldOff": true, "secondHeld": true, "withdrawn": "AbortError" }""", outcome); + } + + [TestMethod] + public async Task A_successful_execute_carries_tagged_rows_not_raw_JavaScript_values() + { + await GotoHostPageAsync(Page); + + var envelope = await Page.EvaluateAsync( + """ + async () => { + const { createHost, encodeValue } = globalThis.BitBsqlite.host; + const host = createHost(); + await host.call({ kind: 'open', databaseName: 'wire.db', requiredBuild: 'synchronous' }); + + await host.call({ + kind: 'execute', + batch: [{ + commandText: 'CREATE TABLE t (i INTEGER, b BLOB)', + resultKind: 'nonQuery', + parameters: [], + }], + }); + + await host.call({ + kind: 'execute', + batch: [{ + commandText: 'INSERT INTO t (i, b) VALUES (@i, @b)', + resultKind: 'nonQuery', + parameters: [ + { name: '@i', ...encodeValue(9007199254740993n) }, + { name: '@b', ...encodeValue(new Uint8Array([1, 2, 250])) }, + ], + }], + }); + + return await host.call({ + kind: 'execute', + batch: [{ commandText: 'SELECT i, b FROM t', resultKind: 'reader', parameters: [] }], + }); + } + """); + + Assert.IsTrue(envelope.GetProperty("ok").GetBoolean()); + Assert.AreEqual(1, envelope.GetProperty("result").GetArrayLength()); + + var row = envelope.GetProperty("result")[0].GetProperty("rows")[0]; + AssertJson.AreEqual("[1, 4]", row.GetProperty("t")); + // A number would have already lost the low bits; a string is the whole point of the format. + Assert.AreEqual("9007199254740993", row.GetProperty("v")[0].GetString()); + Assert.AreEqual(Convert.ToBase64String([1, 2, 250]), row.GetProperty("v")[1].GetString()); + } + + [TestMethod] + public async Task A_failing_execute_is_an_envelope_with_the_SQLite_result_code_not_a_thrown_error() + { + await GotoHostPageAsync(Page); + + var envelope = await Page.EvaluateAsync( + """ + async () => { + const host = globalThis.BitBsqlite.host.createHost(); + await host.call({ kind: 'open', databaseName: 'fail.db', requiredBuild: 'synchronous' }); + await host.call({ + kind: 'execute', + batch: [{ + commandText: 'CREATE TABLE t (id INTEGER PRIMARY KEY)', + resultKind: 'nonQuery', + parameters: [], + }], + }); + await host.call({ + kind: 'execute', + batch: [{ + commandText: 'INSERT INTO t (id) VALUES (1)', + resultKind: 'nonQuery', + parameters: [], + }], + }); + + return await host.call({ + kind: 'execute', + batch: [{ + commandText: 'INSERT INTO t (id) VALUES (1)', + resultKind: 'nonQuery', + parameters: [], + }], + }); + } + """); + + Assert.IsFalse(envelope.GetProperty("ok").GetBoolean()); + Assert.AreEqual(19, envelope.GetProperty("error").GetProperty("sqliteCode").GetInt32()); + Assert.MatchesRegex(new Regex("UNIQUE|constraint", RegexOptions.IgnoreCase), envelope.GetProperty("error").GetProperty("message").GetString()!); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/LiveQueryTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/LiveQueryTests.cs new file mode 100644 index 00000000000..72a906de635 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/LiveQueryTests.cs @@ -0,0 +1,146 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// Live queries. +[TestClass] +public sealed class LiveQueryTests : BsqliteTest +{ + [TestMethod] + public async Task A_write_notifies_the_host_with_the_table_name() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"live-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + var tables = await Page.EvaluateAsync( + """ + async () => { + const seen = new Promise(resolve => { + globalThis.host.onNotify(payload => resolve(payload.tables)); + }); + await globalThis.host.execute([{ + commandText: "INSERT INTO product (name) VALUES ('Live')", + resultKind: 'nonQuery', + }]); + return await seen; + } + """); + + CollectionAssert.Contains(tables, "product"); + } + + /// + /// Nothing goes out while a transaction is open: another tab that re-ran on the INSERT could not see the + /// row and would never hear about the commit. The COMMIT batch is what carries the name. + /// + [TestMethod] + public async Task A_transaction_is_reported_once_at_commit_with_every_table_it_wrote() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"live-tx-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "CREATE TABLE customer (id INTEGER PRIMARY KEY, name TEXT)"); + + var notifications = await Page.EvaluateAsync( + """ + async () => { + const seen = []; + globalThis.host.onNotify(payload => seen.push([...payload.tables].sort())); + + const run = sql => globalThis.host.execute([{ commandText: sql, resultKind: 'nonQuery' }]); + await run('BEGIN'); + await run("INSERT INTO product (name) VALUES ('Live')"); + await run("INSERT INTO customer (name) VALUES ('Live')"); + const beforeCommit = seen.length; + await run('COMMIT'); + return { beforeCommit, seen }; + } + """); + + Assert.AreEqual(0, notifications.GetProperty("beforeCommit").GetInt32()); + AssertJson.AreEqual("""[["customer", "product"]]""", notifications.GetProperty("seen")); + } + + [TestMethod] + public async Task A_rolled_back_transaction_is_not_reported() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"live-rollback-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + var count = await Page.EvaluateAsync( + """ + async () => { + let seen = 0; + globalThis.host.onNotify(() => seen++); + + const run = sql => globalThis.host.execute([{ commandText: sql, resultKind: 'nonQuery' }]); + await run('BEGIN'); + await run("INSERT INTO product (name) VALUES ('Ghost')"); + await run('ROLLBACK'); + return seen; + } + """); + + Assert.AreEqual(0, count); + } + + /// + /// The update hook sees what the SQL text cannot: nothing in this DELETE names the child table, yet its + /// rows are gone and a live query over it has to know. + /// + [TestMethod] + public async Task A_cascading_delete_names_the_child_table() + { + await OpenHostAsync(Page, new HostOptions { DatabaseName = $"live-cascade-{Now()}" }); + await ExecAsync(Page, "CREATE TABLE parent (id INTEGER PRIMARY KEY)"); + await ExecAsync(Page, "CREATE TABLE child (id INTEGER PRIMARY KEY, parent_id INTEGER REFERENCES parent(id) ON DELETE CASCADE)"); + await ExecAsync(Page, "INSERT INTO parent (id) VALUES (1)"); + await ExecAsync(Page, "INSERT INTO child (id, parent_id) VALUES (1, 1)"); + + var tables = await Page.EvaluateAsync( + """ + async () => { + const seen = new Promise(resolve => { + globalThis.host.onNotify(payload => resolve([...payload.tables].sort())); + }); + await globalThis.host.execute([{ commandText: 'DELETE FROM parent WHERE id = 1', resultKind: 'nonQuery' }]); + return await seen; + } + """); + + CollectionAssert.AreEqual(new[] { "child", "parent" }, tables); + } + + [TestMethod] + public async Task A_second_tab_hears_the_write_over_BroadcastChannel() + { + var databaseName = $"live-tab-{Now()}"; + var context = await NewContextAsync(); + var first = await context.NewPageAsync(); + var second = await context.NewPageAsync(); + + try + { + await OpenHostAsync(first, new HostOptions { DatabaseName = databaseName }); + await OpenHostAsync(second, new HostOptions { DatabaseName = $"{databaseName}-other" }); + await ExecAsync(first, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + var notified = second.EvaluateAsync( + """ + () => new Promise(resolve => { + globalThis.host.onNotify(payload => resolve(payload.tables)); + }) + """); + + await ExecAsync(first, "INSERT INTO product (name) VALUES ('CrossTab')"); + CollectionAssert.Contains(await notified, "product"); + } + finally + { + await context.CloseAsync(); + } + } + + /// Date.now(), which the spec put in its database names. + private static long Now() => DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/OpfsTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/OpfsTests.cs new file mode 100644 index 00000000000..dc78f310074 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/OpfsTests.cs @@ -0,0 +1,224 @@ +using System.Text.Json; +using System.Text.RegularExpressions; +using Bit.Bsqlite.Tests.E2E.Infrastructure; +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E; + +/// The OPFS storage provider: persistence, the admin module, and several tabs on one file. +/// +/// Every test here needs OPFS, including the multi-tab ones that open their own pages, so each starts with +/// . +/// +[TestClass] +public sealed class OpfsTests : BsqliteTest +{ + // --------------------------------------------------------------------------------------------- + // OPFS persistence + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Persistence_survives_a_worker_restart() + { + await SkipUnlessOpfsAsync(Page); + + var databaseName = UniqueName("opfs-persist"); + await OpenOpfsAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Kept')"); + + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + await OpenOpfsAsync(Page, databaseName); + AssertJson.AreEqual("""[["Kept"]]""", (await QueryAsync(Page, "SELECT name FROM product")).Rows); + } + + [TestMethod] + public async Task Persistence_an_uncommitted_write_does_not_survive_a_killed_worker() + { + await SkipUnlessOpfsAsync(Page); + + var databaseName = UniqueName("opfs-crash"); + await OpenOpfsAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Committed')"); + await ExecAsync(Page, "BEGIN"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('Ghost')"); + + await Page.EvaluateAsync("() => globalThis.host.dispose()"); + + await OpenOpfsAsync(Page, databaseName); + AssertJson.AreEqual("""[["Committed"]]""", (await QueryAsync(Page, "SELECT name FROM product ORDER BY id")).Rows); + } + + [TestMethod] + public async Task Persistence_two_workers_can_open_the_same_database() + { + await SkipUnlessOpfsAsync(Page); + + var databaseName = UniqueName("opfs-shared"); + await OpenOpfsAsync(Page, databaseName); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('FromFirst')"); + + var second = await Page.EvaluateAsync( + """ + async options => { + const { decodeValue } = globalThis.BitBsqlite.host; + const host = await globalThis.BitBsqlite.host.open(options); + try { + const result = await host.execute([{ + commandText: 'SELECT name FROM product', + resultKind: 'reader', + }]); + return result[0].rows.map(row => + row.v.map((value, i) => decodeValue(row.t[i], value))); + } finally { + host.dispose(); + } + } + """, Storages.Opfs(databaseName).ToArgument()); + + AssertJson.AreEqual("""[["FromFirst"]]""", second); + } + + [TestMethod] + public async Task Persistence_ATTACH_is_rejected_rather_than_corrupting_silently() + { + await SkipUnlessOpfsAsync(Page); + + await OpenOpfsAsync(Page, UniqueName("opfs-attach")); + + var error = await ExecuteExpectingFailureAsync(Page, new WireRequest("ATTACH 'other.db' AS other", ResultKind: "nonQuery")); + + Assert.IsNotNull(error); + Assert.MatchesRegex(new Regex("ATTACH", RegexOptions.IgnoreCase), error.Message); + Assert.MatchesRegex(new Regex("transaction", RegexOptions.IgnoreCase), error.Message); + } + + // --------------------------------------------------------------------------------------------- + // OPFS admin + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Admin_import_export_list_and_delete() + { + await SkipUnlessOpfsAsync(Page); + + await GotoHostPageAsync(Page); + + var name = UniqueName("opfs-admin"); + var image = Enumerable.Range(0, 64).Select(i => i * 7 % 251).ToArray(); + + var outcome = await Page.EvaluateAsync( + """ + async ({ name, image }) => { + const admin = await import('/_content/Bit.Bsqlite.Storage.Opfs/bit-bsqlite-opfs-admin.js'); + await admin.importDatabase(name, new Uint8Array(image)); + const exported = Array.from(await admin.exportDatabase(name)); + const listed = await admin.list(); + const existed = await admin.exists(name); + await admin.deleteDatabase(name); + return { + exported, + listed: listed.includes(name), + existed, + existsAfterDelete: await admin.exists(name), + }; + } + """, new { name, image }); + + AssertJson.AreEqual(JsonSerializer.Serialize(image), outcome.GetProperty("exported")); + Assert.IsTrue(outcome.GetProperty("listed").GetBoolean()); + Assert.IsTrue(outcome.GetProperty("existed").GetBoolean()); + Assert.IsFalse(outcome.GetProperty("existsAfterDelete").GetBoolean()); + } + + [TestMethod] + public async Task Admin_probe_reports_available_in_a_secure_Chrome_context() + { + await SkipUnlessOpfsAsync(Page); + + await GotoHostPageAsync(Page); + + var probe = await Page.EvaluateAsync( + """ + async () => { + const admin = await import('/_content/Bit.Bsqlite.Storage.Opfs/bit-bsqlite-opfs-admin.js'); + return await admin.probe(); + } + """); + + Assert.IsTrue(probe.GetProperty("available").GetBoolean()); + Assert.AreEqual("true", probe.GetProperty("environment").GetProperty("hasGetDirectory").GetString()); + Assert.AreEqual("true", probe.GetProperty("environment").GetProperty("secureContext").GetString()); + } + + // --------------------------------------------------------------------------------------------- + // OPFS multi-tab + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Multi_tab_a_second_tab_sees_a_committed_write() + { + await SkipUnlessOpfsAsync(Page); + + var databaseName = UniqueName("opfs-tabs"); + var context = await NewContextAsync(); + var first = await context.NewPageAsync(); + var second = await context.NewPageAsync(); + + try + { + await OpenOpfsAsync(first, databaseName); + await ExecAsync(first, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(first, "INSERT INTO product (name) VALUES ('Shared')"); + + await OpenOpfsAsync(second, databaseName); + AssertJson.AreEqual("""[["Shared"]]""", (await QueryAsync(second, "SELECT name FROM product")).Rows); + } + finally + { + await context.CloseAsync(); + } + } + + /// + /// The harder order: the reader opened while the file was still empty and has to notice, at each read + /// transaction, that the other tab grew it. + /// + [TestMethod] + public async Task A_tab_that_opened_first_sees_a_write_that_grew_the_file() + { + await SkipUnlessOpfsAsync(Page); + + var databaseName = UniqueName("opfs-grow"); + var context = await NewContextAsync(); + var reader = await context.NewPageAsync(); + var writer = await context.NewPageAsync(); + + try + { + await OpenOpfsAsync(reader, databaseName); + AssertJson.AreEqual("[]", (await QueryAsync(reader, "SELECT name FROM sqlite_master WHERE type = 'table'")).Rows); + + await OpenOpfsAsync(writer, databaseName); + await ExecAsync(writer, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(writer, "INSERT INTO product (name) VALUES ('Grown')"); + + AssertJson.AreEqual("""[["Grown"]]""", (await QueryAsync(reader, "SELECT name FROM product")).Rows); + } + finally + { + await context.CloseAsync(); + } + } + + private static Task OpenOpfsAsync(IPage page, string databaseName) => OpenHostAsync(page, Storages.Opfs(databaseName)); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/README.md b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/README.md new file mode 100644 index 00000000000..14c9456b8f0 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/README.md @@ -0,0 +1,102 @@ +# Bit.Bsqlite end-to-end tests + +The tests in `Bit.Bsqlite.Tests` run the whole .NET stack - ADO.NET, EF Core, storage selection, the wire format - on +desktop .NET, with Microsoft.Data.Sqlite standing in for the engine (`Bit.Bsqlite.Testing`'s in-process transport) and +the JavaScript side scripted. That covers the .NET half well and leaves out everything that only exists in a browser: + +- `bit-bsqlite-host.ts`, the worker, the engine loader and the wire codec never execute, so nothing proves that a batch + survives the trip into a dedicated worker and back, or that the binary format the .NET side writes is the one the + worker reads; +- none of the three wa-sqlite builds is ever instantiated, so nothing proves the synchronous build serves OPFS, that + the asynchronous VFSes get JSPI or Asyncify as the browser allows, or that the function-table patch lets `EF_DECIMAL` + register; +- OPFS, IndexedDB and Cache Storage are never touched: no persistence across a worker restart, no crash safety, no two + tabs sharing a file through Web Locks, no import of a database Bit.Besql left behind; +- the demo website - prerendered on the server, hydrated on WebAssembly - never runs. + +This suite does all of that in a real browser with Playwright. The library suites drive the library's own modules +through small test pages, the same way an application's `IJSRuntime` does; `DemoTests` boots the demo server and drives +its pages. + +## What it proves + +| Suite | What it covers | +|---|---| +| `EngineTests`, `EngineBehaviourTests`, `EngineConformanceTests` | Build negotiation and the pinned SQLite version; one database per worker; the statement cache, engine-reported changes, counters, open-time configuration, snapshots; the conformance rules on the in-memory VFS. | +| `BinaryWireTests`, `InteropTests`, `WorkerTests` | The binary batch and reply format end to end, the interop envelope that keeps SQLite result codes across the boundary, and the worker's request queue, failures, guards and lifecycle. | +| `FunctionsTests` | The `ef_*` decimal functions, aggregates, the `EF_DECIMAL` collation and `REGEXP`, against .NET's answers. | +| `LiveQueryTests` | Change notifications from the update and commit hooks, in one tab and across tabs. | +| `OpfsTests`, `IndexedDbTests`, `CacheStorageTests`, `StorageIsolationTests` | Each backend's persistence, crash safety, admin surface and multi-tab behaviour, the Bit.Besql import, and that two backends never see each other's data under the same name. | +| `SubPathTests` | Every module is reached relative to the document, so an app under `` works unchanged. | +| `DemoTests` | Every demo page is prerendered, hydrates and opens its database without a console error; moving the database between backends from the session bar keeps the data and the binding sticks; the browser and the server seed the same catalog through the same model. | +| `SoakTests` (category `Soak`) | Per storage, a worker killed mid-transaction a thousand times, and four then eight tabs committing at once, each ending on `PRAGMA integrity_check`. Opt-in: minutes, not seconds. | +| `BenchmarkTests` (category `Benchmark`) | Runs `TestPages/benchmarks.html` per backend and prints its numbers. Opt-in, and never a gate. | + +The test pages are served by an in-process Kestrel host (`Infrastructure/AssetHost.cs`) that maps each library's +`wwwroot` under `_content//` the way a Blazor app does, and the whole layout again under `/nested/`. It sends no +COOP/COEP headers on purpose: the library must work without cross-origin isolation. + +## Running locally + +From `src/Bsqlite` (inside `src`, so `src/global.json` selects the Microsoft.Testing.Platform runner): + +```bash +# Building the suite builds the libraries too: it serves their compiled modules and the engine their restore vendored. +# The demo tests run the demo server's build output, so build that first, in the configuration the tests run in. +dotnet build Demo/Server/Bit.Bsqlite.Demo.Server.csproj +dotnet build Tests/Bit.Bsqlite.Tests.E2E/Bit.Bsqlite.Tests.E2E.csproj + +# Once per machine (and after a Playwright update): the browser Playwright drives. +pwsh Tests/Bit.Bsqlite.Tests.E2E/bin/Debug/net10.0/playwright.ps1 install chromium + +cd Tests/Bit.Bsqlite.Tests.E2E && dotnet test + +# One test class, or one test +dotnet test --filter "FullyQualifiedName~.OpfsTests." +dotnet test --filter "FullyQualifiedName~Persistence_survives_a_worker_restart" +``` + +CI does the same in Release (`.github/workflows/bit.ci.Bsqlite.e2e.yml`), only when started by hand (Actions > +"bit platform CI - Bsqlite E2E" > Run workflow), and uploads the test results (TRX) as an artifact: + +```bash +dotnet build src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj -c Release +dotnet build src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/Bit.Bsqlite.Tests.E2E.csproj -c Release +pwsh src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/bin/Release/net10.0/playwright.ps1 install --with-deps chromium +cd src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E && dotnet test -c Release --no-build --report-trx --report-trx-filename bsqlite-e2e.trx +``` + +The pull-request pipeline, `bit.ci.Bsqlite.yml`, runs the build, unit, MCP server and JavaScript tests only. + +Without PowerShell 7, the browser can be installed with the Node.js runtime Playwright ships: +`bin/Debug/net10.0/.playwright/node//node bin/Debug/net10.0/.playwright/package/cli.js install chromium`. + +The tests run one at a time, each in a browser context of its own - a fresh origin storage, so no database, OPFS file +or sticky binding leaks from one test into the next. The demo server is started on the first demo test +(`dotnet Bit.Bsqlite.Demo.Server.dll --urls http://127.0.0.1:`, in the Development environment so the build +output finds its static web assets) and stopped when the run ends. The demo's Google Fonts requests are answered locally +with nothing, so a runner that cannot reach them is not slowed down or failed by it. + +When no browser can be launched, the assets have not been built or the demo does not start, every affected test +reports **Inconclusive** with the reason. On a build server (`CI=true`) the same tests **fail** instead, so a suite that +never ran cannot pass. + +### Other browsers + +Chromium is the default and the only browser CI installs. Firefox and WebKit matter most for the storage matrix - +neither has JSPI, so both run the asynchronous VFSes on the Asyncify build - and run the same suite after +`playwright.ps1 install firefox webkit`, with `BSQLITE_E2E_BROWSER=firefox` or `webkit`. Playwright's own WebKit has no +`navigator.storage`, so its OPFS tests report inconclusive, and it drops Cache Storage entries once no context holds the +cache open, so the Cache Storage persistence tests skip there too. + +### Environment variables + +| Variable | Effect | +|---|---| +| `BSQLITE_E2E_BROWSER` | `chromium` (default), `firefox` or `webkit`. | +| `BSQLITE_E2E_CHANNEL` / `BSQLITE_E2E_EXECUTABLE` | Use an installed Chrome/Edge (`chrome`, `msedge`) or a specific browser binary. Without either, the Chromium Playwright installed is used, falling back to an installed Chrome, then Edge. | +| `BSQLITE_E2E_HEADED=1` | Show the browser. | +| `BSQLITE_E2E_CONFIGURATION` | Build configuration of the demo server to boot; defaults to the test assembly's own. | +| `BSQLITE_E2E_DEMO_URL` | Drive an already running demo (for example one started from the IDE) instead of booting one. | +| `BSQLITE_SOAK=1` | Run the soak tests. `BSQLITE_SOAK_TABS` (default 8), `BSQLITE_SOAK_ROWS` (default 50) and `BSQLITE_SOAK_KILLS` (default 1000) size them; smaller runs help while investigating a failure. | +| `BSQLITE_E2E_BENCHMARKS=1` | Run the benchmark page per backend. The numbers are written to the test output (`--output detailed`, or the TRX). | diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/SoakTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/SoakTests.cs new file mode 100644 index 00000000000..bda4e03a87a --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/SoakTests.cs @@ -0,0 +1,279 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; +using Microsoft.Playwright; + +namespace Bit.Bsqlite.Tests.E2E; + +/// +/// Soak coverage: many tabs writing at once, and a worker killed mid-transaction a thousand times over. +/// +/// +/// +/// Category Soak, and reported inconclusive by an ordinary run unless BSQLITE_SOAK=1, because both +/// tests are minutes of wall clock rather than seconds. +/// +/// +/// The loops live inside page.evaluate rather than being driven a statement at a time from here: a +/// thousand round trips through the Playwright protocol would measure the protocol, and tabs told to write "at +/// the same time" from .NET would in fact take turns. +/// +/// +/// Sizes are environment-tunable so a smaller run can be used while investigating a failure (see +/// ): BSQLITE_SOAK_TABS - tabs in the concurrent-write test (default 8), +/// BSQLITE_SOAK_ROWS - rows each of those tabs commits (default 50), BSQLITE_SOAK_KILLS - +/// mid-transaction kills per storage (default 1000). The assembly runs one test at a time, which the soak tests +/// need: one already saturates a machine's storage layer, and running two at once would turn a genuine +/// contention failure into a scheduling artefact. +/// +/// +[TestClass] +public sealed class SoakTests : BsqliteTest +{ + private const string NotRequested = "the soak tests take minutes rather than seconds; set BSQLITE_SOAK=1 to run them"; + + // Killed workers are reported in batches so a long run shows progress instead of going silent. + private const int KillBatch = 25; + + private static readonly SoakStorage[] SoakStorages = + [ + new("OPFS", "opfs", Storages.Opfs), + new("IndexedDB", "idb", Storages.IndexedDb), + ]; + + // Both ends of the range in the reference targets, deduplicated in case they were tuned to meet. + private static readonly int[] TabCounts = [.. new[] { 4, E2EEnvironment.SoakTabs }.Distinct()]; + + public static IEnumerable> KillRuns => + SoakStorages.Select(storage => new TestDataRow(storage.Label) + { + DisplayName = $"@soak {storage.Label} survives {E2EEnvironment.SoakKills} kills mid-transaction", + }); + + public static IEnumerable> TabRuns => + from storage in SoakStorages + from tabs in TabCounts + select new TestDataRow<(string, int)>((storage.Label, tabs)) + { + DisplayName = $"@soak {storage.Label} loses no write across {tabs} tabs", + }; + + [TestMethod] + [TestCategory("Soak")] + [DynamicData(nameof(KillRuns))] + public async Task Survives_kills_mid_transaction(string label) + { + SkipWhen(E2EEnvironment.Soak is false, NotRequested); + + var storage = SoakStorages.Single(s => s.Label == label); + var kills = E2EEnvironment.SoakKills; + + var databaseName = UniqueName($"{storage.Prefix}-soak-kill"); + await PrepareAsync(Page, storage.Options(databaseName)); + + var committed = 0; + var openRetries = 0; + + for (var done = 0; done < kills; done += KillBatch) + { + var batch = Math.Min(KillBatch, kills - done); + var outcome = await KillBatchAsync(Page, storage.Options(databaseName), iterations: batch, startIndex: done); + + committed += outcome.GetProperty("committed").GetInt32(); + openRetries += outcome.GetProperty("openRetries").GetInt32(); + TestContext.WriteLine($"progress: {done + batch}/{kills} kills, {openRetries} open retries"); + } + + Assert.AreEqual(kills, committed); + + await OpenHostAsync(Page, storage.Options(databaseName)); + + var kept = await QueryAsync(Page, "SELECT count(*) FROM ledger WHERE label LIKE 'kept-%'"); + var ghosts = await QueryAsync(Page, "SELECT count(*) FROM ledger WHERE label LIKE 'ghost-%'"); + var distinct = await QueryAsync(Page, "SELECT count(DISTINCT label) FROM ledger"); + var integrity = await QueryAsync(Page, "PRAGMA integrity_check"); + + // Every committed row is there exactly once, no row from an unfinished transaction survived, and the + // file the kills left behind is still a valid SQLite database. + AssertJson.AreEqual($"[[{kills}]]", kept.Rows); + AssertJson.AreEqual("[[0]]", ghosts.Rows); + AssertJson.AreEqual($"[[{kills}]]", distinct.Rows); + AssertJson.AreEqual("""[["ok"]]""", integrity.Rows); + } + + [TestMethod] + [TestCategory("Soak")] + [DynamicData(nameof(TabRuns))] + public async Task Loses_no_write_across_tabs(string label, int tabs) + { + SkipWhen(E2EEnvironment.Soak is false, NotRequested); + + var storage = SoakStorages.Single(s => s.Label == label); + var rowsPerTab = E2EEnvironment.SoakRows; + + var databaseName = UniqueName($"{storage.Prefix}-soak-tabs-{tabs}"); + var context = await NewContextAsync(); + + try + { + var pages = new List(); + for (var index = 0; index < tabs; index++) + { + pages.Add(await context.NewPageAsync()); + } + + await PrepareAsync(pages[0], storage.Options(databaseName)); + + var outcomes = await Task.WhenAll(pages.Select((tab, index) => WriteBatchAsync( + tab, + storage.Options(databaseName), + tab: index, + rows: rowsPerTab))); + + CollectionAssert.AreEqual( + pages.Select(_ => rowsPerTab).ToArray(), + outcomes.Select(outcome => outcome.GetProperty("written").GetInt32()).ToArray()); + TestContext.WriteLine($"progress: {outcomes.Sum(o => o.GetProperty("retries").GetInt32())} write retries"); + + var reader = pages[0]; + await OpenHostAsync(reader, storage.Options(databaseName)); + + var total = await QueryAsync(reader, "SELECT count(*) FROM ledger"); + var perTab = await QueryAsync( + reader, + "SELECT tab, count(*) FROM ledger GROUP BY tab ORDER BY tab"); + var integrity = await QueryAsync(reader, "PRAGMA integrity_check"); + + // A lost write shows up as a short count; a write applied twice shows up as a long one. Both are + // visible per tab, so a failure names the tab that lost the race. + AssertJson.AreEqual($"[[{tabs * rowsPerTab}]]", total.Rows); + AssertJson.AreEqual( + JsonSerializer.Serialize(Enumerable.Range(0, tabs).Select(index => new[] { index, rowsPerTab })), + perTab.Rows); + AssertJson.AreEqual("""[["ok"]]""", integrity.Rows); + } + finally + { + await context.CloseAsync(); + } + } + + /// Creates the table the soak tests write to, then closes cleanly so the loops start from a good file. + private static async Task PrepareAsync(IPage page, HostOptions options) + { + await OpenHostAsync(page, options); + await ExecAsync( + page, + "CREATE TABLE ledger (id INTEGER PRIMARY KEY, tab INTEGER NOT NULL, label TEXT NOT NULL)"); + await page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + } + + /// + /// Commits one row, opens a transaction, writes a row into it, then kills the worker without committing or + /// closing - the closest a test can get to the tab being killed by the browser or the user. + /// + private static Task KillBatchAsync(IPage page, HostOptions options, int iterations, int startIndex) + { + return page.EvaluateAsync( + """ + async ({ databaseName, options, iterations, startIndex }) => { + const sleep = ms => new Promise(resolve => setTimeout(resolve, ms)); + const outcome = { committed: 0, openRetries: 0 }; + + // A killed worker releases its lock on the file asynchronously, so the next open can arrive while + // the previous one is still letting go. That is a retry, not a failure. + const openWithRetry = async () => { + let lastError; + for (let attempt = 0; attempt < 60; attempt++) { + try { + return await globalThis.BitBsqlite.host.open({ databaseName, ...options }); + } catch (error) { + lastError = error; + outcome.openRetries++; + await sleep(25 + attempt * 5); + } + } + + throw lastError; + }; + + const run = (host, commandText) => + host.execute([{ commandText, resultKind: 'nonQuery' }]); + + for (let index = 0; index < iterations; index++) { + const iteration = startIndex + index; + const host = await openWithRetry(); + + try { + await run(host, `INSERT INTO ledger (tab, label) VALUES (0, 'kept-${iteration}')`); + outcome.committed++; + + await run(host, 'BEGIN'); + await run(host, `INSERT INTO ledger (tab, label) VALUES (0, 'ghost-${iteration}')`); + } finally { + host.dispose(); + } + } + + return outcome; + } + """, new { databaseName = options.DatabaseName, options = options.ToArgument(), iterations, startIndex }); + } + + /// + /// Commits rows from one tab, retrying the statements another tab is currently + /// holding the database against. Contention is expected here; losing a row to it is not. + /// + private static async Task WriteBatchAsync(IPage page, HostOptions options, int tab, int rows) + { + await GotoHostPageAsync(page); + + return await page.EvaluateAsync( + """ + async ({ databaseName, options, tab, rows }) => { + const sleep = ms => new Promise(resolve => setTimeout(resolve, ms)); + const outcome = { written: 0, retries: 0 }; + + const withRetry = async action => { + let lastError; + for (let attempt = 0; attempt < 60; attempt++) { + try { + return await action(); + } catch (error) { + lastError = error; + outcome.retries++; + await sleep(25 + attempt * 5); + } + } + + throw lastError; + }; + + const host = await withRetry(() => globalThis.BitBsqlite.host.open({ databaseName, ...options })); + + try { + for (let row = 0; row < rows; row++) { + await withRetry(() => host.execute([{ + commandText: `INSERT INTO ledger (tab, label) VALUES (${tab}, 'tab-${tab}-row-${row}')`, + resultKind: 'nonQuery', + }])); + outcome.written++; + } + } finally { + await host.close(); + host.dispose(); + } + + return outcome; + } + """, new { databaseName = options.DatabaseName, options = options.ToArgument(), tab, rows }); + } + + /// A storage the soak runs against: its label, the prefix of its database names, and how it opens. + private sealed record SoakStorage(string Label, string Prefix, Func Options); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/StorageIsolationTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/StorageIsolationTests.cs new file mode 100644 index 00000000000..7e6c4f2002d --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/StorageIsolationTests.cs @@ -0,0 +1,69 @@ +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// +/// The reason sticky binding exists, shown at the storage layer: the same filename on a different VFS is a +/// different, empty database. Switching backends without the binding would look like success and lose the +/// rows. +/// +[TestClass] +public sealed class StorageIsolationTests : BsqliteTest +{ + [TestMethod] + public async Task An_IndexedDB_write_is_invisible_on_OPFS_under_the_same_name() + { + // Both directions involve OPFS, which Playwright's WebKit does not have. + await SkipUnlessOpfsAsync(Page); + + var databaseName = UniqueName("iso-idb"); + + await OpenHostAsync(Page, Idb(databaseName)); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('OnlyInIndexedDb')"); + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + await OpenHostAsync(Page, Opfs(databaseName)); + var tables = await QueryAsync( + Page, + "SELECT name FROM sqlite_master WHERE type='table' AND name='product'"); + AssertJson.AreEqual("[]", tables.Rows); + } + + [TestMethod] + public async Task An_OPFS_write_is_invisible_on_IndexedDB_under_the_same_name() + { + // Both directions involve OPFS, which Playwright's WebKit does not have. + await SkipUnlessOpfsAsync(Page); + + var databaseName = UniqueName("iso-opfs"); + + await OpenHostAsync(Page, Opfs(databaseName)); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('OnlyInOpfs')"); + await Page.EvaluateAsync( + """ + async () => { + await globalThis.host.close(); + globalThis.host.dispose(); + } + """); + + await OpenHostAsync(Page, Idb(databaseName)); + var tables = await QueryAsync( + Page, + "SELECT name FROM sqlite_master WHERE type='table' AND name='product'"); + AssertJson.AreEqual("[]", tables.Rows); + } + + // The two storages as the spec opened them: the VFS module and the build it needs, without limits. + private static HostOptions Opfs(string databaseName) => new() { DatabaseName = databaseName, RequiredBuild = "synchronous", Vfs = Storages.OpfsVfs }; + + private static HostOptions Idb(string databaseName) => new() { DatabaseName = databaseName, RequiredBuild = "asyncCapable", Vfs = Storages.IndexedDbVfs }; +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/SubPathTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/SubPathTests.cs new file mode 100644 index 00000000000..8e88dcecbd4 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/SubPathTests.cs @@ -0,0 +1,69 @@ +using System.Text.Json; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// +/// A Blazor application is routinely published under a sub-path - GitHub Pages, an IIS virtual directory - +/// where <base href="/app/"> puts every static asset under /app/_content/.... The .NET +/// providers therefore name their modules relative to the document, and the host resolves those names +/// against the document base before the worker imports them. This page lives under /nested/ and serves the +/// library from /nested/_content/, so a root-relative URL anywhere in the chain - the VFS module, the engine's +/// base classes the vendored VFS imports, the worker itself - fails here. +/// +[TestClass] +public sealed class SubPathTests : BsqliteTest +{ + [TestMethod] + [DataRow("opfs", "opfs-coop-sync", "synchronous", "./_content/Bit.Bsqlite.Storage.Opfs/bit-bsqlite-opfs-vfs.js", + DisplayName = "opfs opens through a document-relative module URL")] + [DataRow("indexeddb", "idb-batch-atomic", "asyncCapable", "./_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-vfs.js", + DisplayName = "indexeddb opens through a document-relative module URL")] + [DataRow("cache-storage", "cache-storage", "asyncCapable", "./_content/Bit.Bsqlite.Storage.CacheStorage/bit-bsqlite-cache-register.js", + DisplayName = "cache-storage opens through a document-relative module URL")] + public async Task Opens_through_a_document_relative_module_URL(string name, string vfsName, string requiredBuild, string moduleUrl) + { + if (name == "opfs") + { + await SkipUnlessOpfsAsync(Page, "/nested/index.html"); + } + + await GotoHostPageAsync(Page, "/nested/index.html"); + + var result = await Page.EvaluateAsync( + """ + async ({ requiredBuild, moduleUrl }) => { + const host = globalThis.BitBsqlite.host.createHost(); + try { + const opened = await host.call({ + kind: 'open', + databaseName: `sub-path-${Date.now()}.db`, + requiredBuild, + vfs: { moduleUrl, registerExport: 'register' }, + }); + + const queried = await host.call({ + kind: 'execute', + batch: [{ commandText: 'SELECT 6 * 7', resultKind: 'reader', parameters: [] }], + }); + + return { opened, queried }; + } finally { + host.dispose(); + } + } + """, new { name, vfsName, requiredBuild, moduleUrl }); + + // The build is whichever one this browser got (expect.any(String)); everything else is exact. + var opened = result.GetProperty("opened"); + var build = opened.GetProperty("result").GetProperty("build"); + Assert.AreEqual(JsonValueKind.String, build.ValueKind); + AssertJson.AreEqual( + JsonSerializer.Serialize(new { ok = true, result = new { build = build.GetString(), vfsName, reused = false } }), + opened); + + var queried = result.GetProperty("queried"); + Assert.IsTrue(queried.GetProperty("ok").GetBoolean()); + AssertJson.AreEqual("[42]", queried.GetProperty("result")[0].GetProperty("rows")[0].GetProperty("v")); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/benchmarks.html b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/benchmarks.html new file mode 100644 index 00000000000..e45ecc10305 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/benchmarks.html @@ -0,0 +1,85 @@ + + + + + Bsqlite benchmarks + + + +

Bsqlite benchmarks

+

Manual. Open the console, then:

+ 
await run('memory')
+await run('opfs')
+await run('indexeddb')
+await run('cache')
+

Each run reports, in milliseconds: cold open, a 100-row insert batch, 1 000 repeated parameterised + inserts (statement cache), a 1 000-row read on the binary path and on the tagged path, and an + indexed point read.

+ + + diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/extension.js b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/extension.js new file mode 100644 index 00000000000..3b1a9ac382e --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/extension.js @@ -0,0 +1,6 @@ +// A minimal extension module, the shape an application ships to add its own SQL functions. +export function register({ sqlite3, db }) { + sqlite3.create_function(db, 'shout', 1, 1 /* SQLITE_UTF8 */, 0, (ctx, values) => { + sqlite3.result(ctx, `${sqlite3.value_text(values[0]).toUpperCase()}!`); + }); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/index.html b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/index.html new file mode 100644 index 00000000000..9a89405eacf --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/index.html @@ -0,0 +1,21 @@ + + + + + Bsqlite browser test host + + + + + + diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/nested/index.html b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/nested/index.html new file mode 100644 index 00000000000..12e5289b827 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/TestPages/nested/index.html @@ -0,0 +1,22 @@ + + + + + + + Bsqlite browser test host (sub-path) + + + + + diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/WorkerTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/WorkerTests.cs new file mode 100644 index 00000000000..99f952cd859 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.E2E/WorkerTests.cs @@ -0,0 +1,445 @@ +using System.Text.Json; +using System.Text.RegularExpressions; +using Bit.Bsqlite.Tests.E2E.Infrastructure; + +namespace Bit.Bsqlite.Tests.E2E; + +/// The worker: SQL execution, batching, value round-tripping and request serialization. +[TestClass] +public sealed class WorkerTests : BsqliteTest +{ + // --------------------------------------------------------------------------------------------- + // foreign keys + // + // SQLite defaults foreign keys off; Microsoft.Data.Sqlite turns them on for every connection. The + // worker has to do the same, or the same model enforces its relationships on the server and silently + // does not in the browser - which is the one difference this library exists to not have. + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Foreign_keys_are_on_the_way_Microsoft_Data_Sqlite_leaves_them() + { + await OpenWithParentAndChildAsync(); + + var result = await QueryAsync(Page, "PRAGMA foreign_keys"); + AssertJson.AreEqual("[[1]]", result.Rows); + } + + [TestMethod] + public async Task Foreign_keys_reject_an_orphaned_row_rather_than_storing_it() + { + await OpenWithParentAndChildAsync(); + + var error = await ExecuteExpectingFailureAsync(Page, + new WireRequest("INSERT INTO child (id, parent_id) VALUES (1, 999)")); + + Assert.IsNotNull(error); + Assert.Contains("FOREIGN KEY", error.Message); + Assert.AreEqual(19, error.SqliteCode); + } + + [TestMethod] + public async Task Foreign_keys_cascade_a_delete_to_the_child_rows() + { + await OpenWithParentAndChildAsync(); + + await ExecAsync(Page, "INSERT INTO parent (id) VALUES (1)"); + await ExecAsync(Page, "INSERT INTO child (id, parent_id) VALUES (1, 1)"); + + await ExecAsync(Page, "DELETE FROM parent WHERE id = 1"); + + var remaining = await QueryAsync(Page, "SELECT COUNT(*) FROM child"); + AssertJson.AreEqual("[[0]]", remaining.Rows); + } + + // EF's SQLite migrations rebuild tables with the pragma toggled off and back on, so it has to stay + // something the application can change. + [TestMethod] + public async Task Foreign_keys_can_still_be_turned_off_by_the_application() + { + await OpenWithParentAndChildAsync(); + + await ExecAsync(Page, "PRAGMA foreign_keys=OFF"); + + AssertJson.AreEqual("[[0]]", (await QueryAsync(Page, "PRAGMA foreign_keys")).Rows); + await ExecAsync(Page, "INSERT INTO child (id, parent_id) VALUES (1, 999)"); + } + + // --------------------------------------------------------------------------------------------- + // executing SQL + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Executing_SQL_round_trips_a_row_through_parameters() + { + await OpenAsync(); + + var insert = await ExecAsync( + Page, + "INSERT INTO product (name, price) VALUES (@name, @price)", + new WireParameter("@name", "Widget"), new WireParameter("@price", 9.5)); + + Assert.AreEqual(1, insert.RecordsAffected); + + var selected = await QueryAsync(Page, "SELECT id, name, price FROM product"); + + CollectionAssert.AreEqual(new[] { "id", "name", "price" }, selected.ColumnNames); + AssertJson.AreEqual("""[[1, "Widget", 9.5]]""", selected.Rows); + } + + [TestMethod] + public async Task Executing_SQL_matches_parameter_names_with_or_without_their_sigil() + { + await OpenAsync(); + + // ADO.NET callers write both, and Microsoft.Data.Sqlite accepts both. + await ExecAsync(Page, "INSERT INTO product (name) VALUES (@name)", new WireParameter("name", "Bare")); + + var selected = await QueryAsync(Page, "SELECT name FROM product"); + + AssertJson.AreEqual("""[["Bare"]]""", selected.Rows); + } + + [TestMethod] + public async Task Executing_SQL_binds_positional_markers_by_position() + { + await OpenAsync(); + + await ExecAsync( + Page, + "INSERT INTO product (name, price) VALUES (?, ?)", + new WireParameter("?1", "Positional"), new WireParameter("?2", 1.25)); + + AssertJson.AreEqual("""[["Positional", 1.25]]""", (await QueryAsync(Page, "SELECT name, price FROM product")).Rows); + } + + /// + /// The failure this prevents is the quiet one: binding a name the statement never asked for would + /// otherwise leave the real parameter NULL and return wrong rows instead of an error. + /// + [TestMethod] + public async Task Executing_SQL_refuses_to_run_a_statement_with_a_parameter_left_unbound() + { + await OpenAsync(); + + var error = await ExecuteExpectingFailureAsync(Page, new WireRequest( + "INSERT INTO product (name) VALUES (@name)", + [new WireParameter("@misspelled", "Widget")], + "nonQuery")); + + Assert.IsNotNull(error); + Assert.Contains("@name", error.Message); + Assert.Contains("@misspelled", error.Message); + } + + [TestMethod] + public async Task Executing_SQL_reports_storage_classes_per_column() + { + await OpenAsync(); + + await ExecAsync( + Page, + "INSERT INTO product (name, price) VALUES ('Mixed', 3)"); + + // The alias avoids NOTHING, which SQLite reserves for `ON CONFLICT DO NOTHING`. + var selected = await QueryAsync(Page, "SELECT id, name, price, NULL AS absent FROM product"); + + // 'price' is declared REAL but was given an integer, and this reports what SQLite stored rather than + // what the column was declared as - the engine is built without declared types at all. + CollectionAssert.AreEqual(new[] { "INTEGER", "TEXT", "REAL", null }, selected.ColumnTypes); + } + + // --------------------------------------------------------------------------------------------- + // batching + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Batching_runs_a_batch_in_order_and_returns_one_result_per_request() + { + await OpenAsync(); + + var results = await ExecuteAsync(Page, + new WireRequest( + "INSERT INTO product (name) VALUES (@n)", + [new WireParameter("@n", "First")], + "nonQuery"), + new WireRequest( + "INSERT INTO product (name) VALUES (@n)", + [new WireParameter("@n", "Second")], + "nonQuery"), + new WireRequest("SELECT name FROM product ORDER BY id", ResultKind: "reader")); + + Assert.HasCount(3, results); + Assert.AreEqual(1, results[0].RecordsAffected); + Assert.AreEqual(1, results[1].RecordsAffected); + AssertJson.AreEqual("""[["First"], ["Second"]]""", results[2].Rows); + } + + /// + /// EF Core's insert pattern: the write and the identity read arrive as one command, and the reader has to + /// see the SELECT's rows while the INSERT's effect is still counted. + /// + [TestMethod] + public async Task Batching_takes_rows_from_the_first_row_producing_statement_and_still_runs_the_rest() + { + await OpenAsync(); + + var result = await QueryAsync( + Page, + """ + INSERT INTO product (name) VALUES (@n); + SELECT id FROM product WHERE rowid = last_insert_rowid(); + """, + new WireParameter("@n", "Identity")); + + CollectionAssert.AreEqual(new[] { "id" }, result.ColumnNames); + AssertJson.AreEqual("[[1]]", result.Rows); + + // Exactly one: sqlite3_changes still says 1 after the SELECT, and a count that asked it after every + // statement billed the row twice. + Assert.AreEqual(1, result.RecordsAffected); + } + + [TestMethod] + public async Task Batching_a_scalar_request_stops_after_the_first_row() + { + await OpenAsync(); + + await ExecAsync(Page, "INSERT INTO product (name) VALUES ('A'), ('B'), ('C')"); + + var result = (await ExecuteAsync(Page, new WireRequest("SELECT name FROM product ORDER BY id", ResultKind: "scalar")))[0]; + + AssertJson.AreEqual("""[["A"]]""", result.Rows); + } + + [TestMethod] + public async Task Batching_a_failing_request_does_not_leave_the_connection_unusable() + { + await OpenAsync(); + + var error = await ExecuteExpectingFailureAsync(Page, + new WireRequest("SELECT * FROM nonexistent", ResultKind: "reader")); + + Assert.IsNotNull(error); + Assert.IsNotNull(error.SqliteCode); + + // The queue must survive a rejected request; otherwise one bad query poisons the worker. + AssertJson.AreEqual("[[0]]", (await QueryAsync(Page, "SELECT COUNT(*) FROM product")).Rows); + } + + // --------------------------------------------------------------------------------------------- + // value round-tripping + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Value_round_tripping_carries_every_storage_class() + { + await OpenAsync(); + + await ExecAsync(Page, "CREATE TABLE values_ (i INTEGER, r REAL, t TEXT, b BLOB, n INTEGER)"); + await ExecAsync(Page, "INSERT INTO values_ VALUES (@i, @r, @t, @b, @n)", + new WireParameter("@i", 42), + new WireParameter("@r", 1.5), + new WireParameter("@t", "text"), + WireParameter.Blob("@b", 1, 2, 250), + new WireParameter("@n", null)); + + var result = await QueryAsync(Page, "SELECT i, r, t, b, n FROM values_"); + + var row = result.Rows[0]; + Assert.AreEqual(42, row[0].GetInt32()); + Assert.AreEqual(1.5, row[1].GetDouble()); + Assert.AreEqual("text", row[2].GetString()); + AssertJson.AreEqual("""{ "blob": [1, 2, 250] }""", row[3]); + Assert.AreEqual(JsonValueKind.Null, row[4].ValueKind); + } + + /// + /// JSON has no spelling for Infinity - JSON.stringify writes null - and the .NET transport reads the rows + /// through JSON. So it travels as a string, and NaN, which SQLite stores as NULL, comes back as one. + /// + [TestMethod] + public async Task Value_round_tripping_carries_an_infinite_REAL_and_NaN_as_the_NULL_SQLite_makes_of_it() + { + await OpenAsync(); + + await ExecAsync(Page, "CREATE TABLE reals (r REAL)"); + await ExecAsync(Page, "INSERT INTO reals VALUES (@a), (@b), (@c)", + new WireParameter("@a", double.PositiveInfinity), + new WireParameter("@b", double.NegativeInfinity), + new WireParameter("@c", double.NaN)); + + var result = await QueryAsync(Page, "SELECT r FROM reals"); + + // The decoded values are ±Infinity, which ExecuteAsync re-tags for the trip out of the page. + AssertJson.AreEqual("""[[{ "real": "Infinity" }], [{ "real": "-Infinity" }], [null]]""", result.Rows); + + // What actually crossed the wire: a string, not the null JSON would have made of it. + var raw = await Page.EvaluateAsync( + """ + async () => { + const [first] = await globalThis.host.execute([{ commandText: 'SELECT r FROM reals', resultKind: 'reader' }]); + return first.rows.map(row => row.v[0]); + } + """); + AssertJson.AreEqual("""["Infinity", "-Infinity", null]""", raw); + } + + /// + /// Integers beyond 2^53 cannot be represented exactly as a JavaScript number, and SQLite's INTEGER is + /// 64-bit. The engine hands these back as BigInt, which the .NET transport must be prepared for - JSON + /// interop cannot carry it. + /// + [TestMethod] + public async Task Value_round_tripping_returns_a_big_integer_as_a_BigInt_rather_than_losing_precision() + { + await OpenAsync(); + + await ExecAsync(Page, "CREATE TABLE big (v INTEGER)"); + await ExecAsync(Page, "INSERT INTO big (v) VALUES (9007199254740993)"); + + var result = await QueryAsync(Page, "SELECT v FROM big"); + + AssertJson.AreEqual("""[[{ "bigint": "9007199254740993" }]]""", result.Rows); + } + + /// The same boundary on the way in: a 64-bit parameter must bind as int64, not as a rounded double. + [TestMethod] + public async Task Value_round_tripping_accepts_a_big_integer_as_a_parameter_without_rounding_it() + { + await OpenAsync(); + + await ExecAsync(Page, "CREATE TABLE big (v INTEGER)"); + await ExecAsync(Page, "INSERT INTO big (v) VALUES (@v)", WireParameter.BigInt("@v", "9223372036854775807")); + + var result = await QueryAsync(Page, "SELECT v FROM big"); + + AssertJson.AreEqual("""[[{ "bigint": "9223372036854775807" }]]""", result.Rows); + } + + [TestMethod] + public async Task Value_round_tripping_round_trips_a_blob_too_large_for_a_single_fromCharCode_call() + { + await OpenAsync(); + + await ExecAsync(Page, "CREATE TABLE blobs (b BLOB)"); + + // Past the 0x8000 chunk boundary in the base64 encoder, where an unchunked implementation throws. + var bytes = Enumerable.Range(0, 70_000).Select(i => (byte)(i % 251)).ToArray(); + await ExecAsync(Page, "INSERT INTO blobs (b) VALUES (@b)", WireParameter.Blob("@b", bytes)); + + var result = await QueryAsync(Page, "SELECT b, length(b) AS size FROM blobs"); + + Assert.AreEqual(70_000, result.Rows[0][1].GetInt32()); + CollectionAssert.AreEqual(bytes, result.Rows[0][0].GetProperty("blob").EnumerateArray().Select(b => b.GetByte()).ToArray()); + } + + // --------------------------------------------------------------------------------------------- + // request serialization + // --------------------------------------------------------------------------------------------- + + /// + /// One SQLite connection cannot execute two statements at once. Requests issued without awaiting must + /// therefore queue, and the observable proof is that a counter incremented by overlapping batches lands + /// on the exact total. + /// + [TestMethod] + public async Task Request_serialization_overlapping_batches_queue_instead_of_interleaving() + { + await OpenAsync(); + + await ExecAsync(Page, "CREATE TABLE counter (n INTEGER)"); + await ExecAsync(Page, "INSERT INTO counter (n) VALUES (0)"); + + await Page.EvaluateAsync( + """ + async () => { + const bump = () => globalThis.host.execute([{ + commandText: 'UPDATE counter SET n = n + 1', + resultKind: 'nonQuery', + }]); + + await Promise.all(Array.from({ length: 25 }, bump)); + } + """); + + AssertJson.AreEqual("[[25]]", (await QueryAsync(Page, "SELECT n FROM counter")).Rows); + } + + /// + /// A worker whose script never loaded has nobody listening. The first request fails through the error + /// event; the ones after it must fail the same way rather than wait for an answer that is never coming. + /// + [TestMethod] + public async Task Request_serialization_a_worker_that_failed_to_load_rejects_every_request_not_just_the_first() + { + await OpenAsync(); + + await GotoHostPageAsync(Page); + + var outcomes = await Page.EvaluateAsync( + """ + async () => { + const host = globalThis.BitBsqlite.host.createHost('/_content/Bit.Bsqlite/no-such-worker.js'); + const attempt = async () => { + try { + await host.version(); + return 'resolved'; + } catch (error) { + return error.message; + } + }; + + const first = await attempt(); + const second = await attempt(); + host.dispose(); + return { first, second }; + } + """); + + Assert.MatchesRegex(new Regex("SQLite worker failed"), outcomes.GetProperty("first").GetString()!); + Assert.MatchesRegex(new Regex("SQLite worker failed"), outcomes.GetProperty("second").GetString()!); + } + + [TestMethod] + public async Task Request_serialization_disposing_rejects_requests_still_in_flight() + { + await OpenAsync(); + + var outcome = await Page.EvaluateAsync( + """ + async () => { + const pending = globalThis.host.execute([ + { commandText: 'SELECT 1', resultKind: 'scalar' }, + ]); + + globalThis.host.dispose(); + + try { + await pending; + return 'resolved'; + } catch (error) { + return error.message; + } + } + """); + + Assert.Contains("disposed", outcome); + } + + // The file's beforeEach: every test starts from an open database with a product table. + private async Task OpenAsync() + { + await OpenHostAsync(Page); + await ExecAsync(Page, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT, price REAL)"); + } + + // The foreign keys describe block's beforeEach, after the file's. + private async Task OpenWithParentAndChildAsync() + { + await OpenAsync(); + await ExecAsync(Page, "CREATE TABLE parent (id INTEGER PRIMARY KEY)"); + await ExecAsync(Page, + "CREATE TABLE child (id INTEGER PRIMARY KEY, parent_id INTEGER REFERENCES parent(id) ON DELETE CASCADE)"); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/Bit.Bsqlite.Tests.Mcp.csproj b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/Bit.Bsqlite.Tests.Mcp.csproj new file mode 100644 index 00000000000..b41a5b5026e --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/Bit.Bsqlite.Tests.Mcp.csproj @@ -0,0 +1,37 @@ + + + + + net10.0 + + Exe + enable + enable + false + true + $(NoWarn);CS1591 + true + false + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/BsqliteServiceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/BsqliteServiceTests.cs new file mode 100644 index 00000000000..4c6ea619920 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/BsqliteServiceTests.cs @@ -0,0 +1,206 @@ +using System.Text.RegularExpressions; +using Bit.Bsqlite.Demo.Server.Services; +using Bit.Bsqlite.Demo.Server.Controllers; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// The services the tools are built from, called directly: the rules that are awkward to reach through +/// a schema-validated protocol call - an argument a client would never send, a document longer than any +/// that exists - and the small parsing decisions every answer depends on. +/// +[TestClass] +public partial class BsqliteServiceTests +{ + public TestContext TestContext { get; set; } = null!; + + // ---- Completions ------------------------------------------------------- + + [TestMethod] + public void An_argument_with_no_set_completes_with_nothing_rather_than_failing() + { + Assert.IsEmpty(BsqliteCompletions.Complete("dataSource", "Off").Values); + Assert.IsEmpty(BsqliteCompletions.Complete(null, null).Values); + } + + [TestMethod] + public void A_typed_prefix_ranks_above_a_typed_substring() + { + var values = BsqliteCompletions.Complete("typeName", "Storage").Values; + + Assert.IsNotEmpty(values); + Assert.IsTrue(values.All(value => value.Contains("Storage", StringComparison.OrdinalIgnoreCase))); + + // "IBsqliteStorageProvider" contains the prefix, "BsqliteStorageMigrator" starts with it: the second kind first. + var prefixed = BsqliteCompletions.Complete("typeName", "BsqliteStorage").Values; + var firstSubstring = prefixed.ToList().FindIndex(value => value.StartsWith("BsqliteStorage", StringComparison.Ordinal) is false); + + Assert.IsGreaterThan(0, firstSubstring); + Assert.IsTrue(prefixed.Skip(firstSubstring).All(value => value.StartsWith("BsqliteStorage", StringComparison.Ordinal) is false)); + } + + [TestMethod] + public void A_set_larger_than_the_protocol_allows_says_there_is_more() + { + var completion = BsqliteCompletions.Complete("typeName", string.Empty); + + Assert.AreEqual(BsqliteApiCatalog.Types.Length, completion.Total); + Assert.IsLessThanOrEqualTo(100, completion.Values.Count); + Assert.AreEqual(BsqliteApiCatalog.Types.Length > 100, completion.HasMore); + } + + // ---- Names a caller writes --------------------------------------------- + + [TestMethod] + public void A_document_name_is_resolved_the_way_people_write_it() + { + Assert.AreEqual("storage", BsqliteSourceCatalog.ResolveDocumentName("Storage.md")); + Assert.AreEqual("live-queries", BsqliteSourceCatalog.ResolveDocumentName("docs\\live-queries.md")); + Assert.AreEqual("live-queries", BsqliteSourceCatalog.ResolveDocumentName("Live Queries")); + Assert.AreEqual(BsqliteSourceCatalog.ReadmeName, BsqliteSourceCatalog.ResolveDocumentName("README.md")); + Assert.AreEqual("Bit.Bsqlite.Storage.IndexedDb", BsqliteSourceCatalog.ResolveDocumentName("indexeddb")); + Assert.AreEqual("Bit.Bsqlite.Storage.CacheStorage", BsqliteSourceCatalog.ResolveDocumentName("Storage.CacheStorage")); + Assert.AreEqual("Bit.Bsqlite.Testing", BsqliteSourceCatalog.ResolveDocumentName("Testing")); + Assert.IsNull(BsqliteSourceCatalog.ResolveDocumentName("changelog")); + Assert.IsNull(BsqliteSourceCatalog.ResolveDocumentName(" ")); + } + + [TestMethod] + public void A_scenario_is_resolved_from_the_spellings_people_reach_for() + { + Assert.AreEqual("auto", BsqliteSetupGuide.Resolve("InteractiveAuto")); + Assert.AreEqual("server", BsqliteSetupGuide.Resolve("Blazor Server")); + Assert.AreEqual("wasm", BsqliteSetupGuide.Resolve("InteractiveWebAssembly")); + Assert.AreEqual("standalone-wasm", BsqliteSetupGuide.Resolve("standalone")); + Assert.AreEqual("move-from-besql", BsqliteSetupGuide.Resolve("Bit.Besql")); + Assert.AreEqual("testing", BsqliteSetupGuide.Resolve("MSTest")); + Assert.AreEqual("custom-provider", BsqliteSetupGuide.Resolve("custom_storage_provider")); + Assert.IsNull(BsqliteSetupGuide.Resolve("electron")); + + foreach (var scenario in BsqliteSetupGuide.Scenarios) Assert.AreEqual(scenario, BsqliteSetupGuide.Resolve(scenario.ToUpperInvariant())); + } + + [TestMethod] + public void A_backend_is_resolved_however_it_is_named() + { + Assert.AreEqual("cache-storage", BsqliteStorageCatalog.Find("CacheStorage")?.Name); + Assert.AreEqual("in-memory", BsqliteStorageCatalog.Find("InMemory")?.Name); + Assert.AreEqual("opfs", BsqliteStorageCatalog.Find("Bit.Bsqlite.Storage.Opfs")?.Name); + Assert.IsNull(BsqliteStorageCatalog.Find("localstorage")); + Assert.IsNull(BsqliteStorageCatalog.Instance("acme-encrypted")); + } + + // ---- Bounds ------------------------------------------------------------ + + [TestMethod] + public void A_long_answer_is_cut_and_says_so() + { + var text = new string('a', 50_000); + var cut = McpController.Truncate(text); + + Assert.StartsWith(new string('a', 40_000), cut); + Assert.Contains("[truncated", cut); + Assert.AreEqual("short", McpController.Truncate("short")); + } + + [TestMethod] + public void A_cut_never_splits_a_surrogate_pair() + { + // An emoji straddling the cut: half a pair is not text. + var text = new string('a', 39_999) + "\U0001F600" + new string('b', 100); + var cut = McpController.Truncate(text); + + Assert.IsFalse(char.IsHighSurrogate(cut[cut.IndexOf("\n\n[truncated", StringComparison.Ordinal) - 1])); + } + + [TestMethod] + public async Task The_planner_refuses_input_no_configuration_has() + { + var mode = await BsqliteStorageLab.PlanAsync(null, null, null, null, false, "Sometimes", null, null, null, TestContext.CancellationToken); + var numeric = await BsqliteStorageLab.PlanAsync(null, null, null, null, false, "2", null, null, null, TestContext.CancellationToken); + var many = await BsqliteStorageLab.PlanAsync(string.Join(",", Enumerable.Range(0, 20).Select(i => $"p{i}")), null, null, null, false, null, null, null, null, TestContext.CancellationToken); + var longName = await BsqliteStorageLab.PlanAsync(new string('x', 100), null, null, null, false, null, null, null, null, TestContext.CancellationToken); + + Assert.Contains("is not a migration mode", mode.Error!); + Assert.Contains("is not a migration mode", numeric.Error!, "A mode is named, as WithMigrationMode is written - not numbered."); + Assert.Contains("at most", many.Error!); + Assert.Contains("longer than a provider name", longName.Error!); + Assert.IsTrue(new[] { mode, numeric, many, longName }.All(plan => plan.Opens is false && plan.Log.Length == 0)); + } + + [TestMethod] + public async Task The_planner_honours_the_database_name() + { + var plan = await BsqliteStorageLab.PlanAsync("indexeddb", null, "indexeddb", null, false, null, null, null, "orders.db", TestContext.CancellationToken); + + Assert.Contains("No storage provider could open database 'orders.db'", plan.Error!); + Assert.Contains("AddDatabase(\"orders.db\")", plan.Configuration); + } + + [TestMethod] + public void Blank_sql_is_the_guard_reference() + { + Assert.HasCount(BsqliteStorageCatalog.GuardedStatements.Length, BsqliteSqlCheck.Check(" ").Statements); + } + + // ---- Documentation text ------------------------------------------------ + + [TestMethod] + public void An_inheritdoc_cref_overload_carries_the_documentation_it_points_at() + { + // The generic UseBitBsqlite overloads say and nothing else. + var extensions = BsqliteApiCatalog.GetTypeDetails("BsqliteDbContextOptionsExtensions")!; + var overloads = extensions.Members.Where(member => member.Name.StartsWith("UseBitBsqlite", StringComparison.Ordinal)).ToArray(); + + Assert.IsGreaterThan(3, overloads.Length); + foreach (var overload in overloads) Assert.IsNotNull(overload.Summary, $"{overload.Name}{overload.Signature}"); + } + + [TestMethod] + public void No_rendered_documentation_carries_a_generic_arity_marker() + { + // A cref to a member of a generic type ("IBsqliteLiveQuery`1.RefreshAsync") must read as the member, + // not be cut off at the marker. + foreach (var type in BsqliteApiCatalog.Types) + { + var details = BsqliteApiCatalog.GetTypeDetails(type.Name)!; + + foreach (var text in details.Members.SelectMany(member => new[] { member.Summary, member.Remarks }).Append(details.Summary).Append(details.Remarks)) + { + if (text is null) continue; + + Assert.IsFalse(ArityRegex().IsMatch(text), $"{type.Name}: {text}"); + } + } + } + + [GeneratedRegex(@"\w`\d")] + private static partial Regex ArityRegex(); + + [TestMethod] + public void A_document_without_sections_is_its_own_preamble() + { + Assert.AreEqual(BsqliteSourceCatalog.GetDocument("limits")!.Trim().Replace("\r\n", "\n", StringComparison.Ordinal), + BsqliteSourceCatalog.GetPreamble("limits")!.Replace("\r\n", "\n", StringComparison.Ordinal)); + + Assert.DoesNotContain("## ", BsqliteSourceCatalog.GetPreamble("storage")!); + Assert.IsNull(BsqliteSourceCatalog.GetPreamble("changelog")); + } + + [TestMethod] + public void A_source_file_is_described_by_what_it_says_about_itself() + { + string? Describe(string path) => BsqliteSourceCatalog.SourceFiles.Single(file => file.Path == path).Description; + + // The class summary, not the first enum value's above it. + Assert.StartsWith("The sample's view of the library's shared BsqliteDatabase", Describe("Demo/Client/BrowserDatabase.cs")!); + // The page header a reader sees. + Assert.StartsWith("Choosing where the file lives:", Describe("Demo/Client/Pages/Storage.razor")!); + // A component's own comment, not its parameters' documentation. + Assert.StartsWith("Shows the code that powers the demo", Describe("Demo/Client/Components/CodeBlock.razor")!); + // A file that explains itself only in its code. + Assert.StartsWith("Every page is prerendered on the server first", Describe("Demo/Client/Components/SessionLifecycle.razor")!); + // A reads as the name it refers to. + Assert.StartsWith("Named SalesOrder rather than Order", Describe("Demo/Domain/SalesOrder.cs")!); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpCall.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpCall.cs new file mode 100644 index 00000000000..ddb81ca833e --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpCall.cs @@ -0,0 +1,66 @@ +using System.Text.Json; +using ModelContextProtocol.Protocol; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// Calls a tool the way a client does and hands back what a client would actually get. +/// +/// Every call goes over the connected session rather than into the controller, so a tool that throws, +/// answers with nothing, or returns something that does not serialize fails here - with the tool's name +/// in the message - instead of being read past by the test that called it. +/// +/// +internal static class McpCall +{ + private static readonly JsonSerializerOptions _json = new(JsonSerializerDefaults.Web); + + /// The raw result, for the tests that are about the failure rather than the answer. + public static Task RawAsync(string tool, Dictionary? arguments = null) + => McpTestHost.Client.CallToolAsync(tool, arguments).AsTask(); + + /// The text of a prose tool's answer. + public static async Task TextAsync(string tool, Dictionary? arguments = null) + { + var result = await RawAsync(tool, arguments); + + AssertSucceeded(tool, result); + + var text = string.Join('\n', result.Content.OfType().Select(block => block.Text)); + + Assert.IsFalse(string.IsNullOrWhiteSpace(text), $"'{tool}' answered with no text at all."); + + return text; + } + + /// + /// A structured tool's answer, deserialized into the DTO the tool declares. This is the round trip + /// that matters for a client: the object is rebuilt out of the JSON that went over the wire, under the + /// output schema the tool published, not out of the instance the method made. + /// + public static async Task StructuredAsync(string tool, Dictionary? arguments = null) + { + var result = await RawAsync(tool, arguments); + + AssertSucceeded(tool, result); + + Assert.IsNotNull(result.StructuredContent, + $"'{tool}' returned no structured content, so a client has nothing to validate against its output schema."); + + var value = result.StructuredContent.Value.Deserialize(_json); + + Assert.IsNotNull(value, $"'{tool}' returned structured content that does not deserialize into {typeof(T).Name}."); + + return value; + } + + /// The text of a tool call that is expected to answer, rather than to fail. + private static void AssertSucceeded(string tool, CallToolResult result) + { + if (result.IsError is not true) return; + + var text = string.Join('\n', result.Content.OfType().Select(block => block.Text)); + + Assert.Fail($"'{tool}' answered with an error: {text}"); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpOracleTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpOracleTests.cs new file mode 100644 index 00000000000..44749ada004 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpOracleTests.cs @@ -0,0 +1,323 @@ +using System.Net; +using System.Reflection; +using System.Text.Json; +using System.Net.Http.Json; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Testing; +using Bit.Bsqlite.Storage.Opfs; +using System.Text.RegularExpressions; +using Bit.Bsqlite.Demo.Server.Dtos; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Storage.IndexedDb; +using Bit.Bsqlite.Demo.Server.Services; +using Bit.Bsqlite.Storage.CacheStorage; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// The server's claims put to the things they describe: the storage catalog against the provider types +/// it was read from, the storage lab against the library's own resolver, the API index against the +/// assemblies, the document index against the documents, and the HTTP mirror against the protocol it +/// mirrors. No expected answers are written down here - the library, its files and the protocol are the +/// expectations. +/// +[TestClass] +public partial class McpOracleTests +{ + private static readonly IBsqliteStorageProvider[] _firstParty = + [ + new BsqliteOpfsStorageProvider(), + new BsqliteIndexedDbStorageProvider(), + new BsqliteCacheStorageProvider(), + new BsqliteInMemoryStorageProvider(), + ]; + + [TestMethod] + public void The_storage_catalog_is_what_the_provider_types_declare() + { + CollectionAssert.AreEqual(_firstParty.Select(provider => provider.Name).ToArray(), BsqliteStorageCatalog.Providers.Select(provider => provider.Name).ToArray()); + + foreach (var provider in _firstParty) + { + var entry = BsqliteStorageCatalog.Find(provider.Name)!; + + Assert.AreEqual(provider.GetType().Name, entry.TypeName); + Assert.AreEqual(provider.GetType().Assembly.GetName().Name, entry.Package); + Assert.AreEqual(provider.VfsModule?.ModuleUrl, entry.VfsModule, provider.Name); + + foreach (var property in typeof(BsqliteStorageCapabilities).GetProperties(BindingFlags.Public | BindingFlags.Instance).Where(property => property.Name != "EqualityContract")) + { + var capability = entry.Capabilities.SingleOrDefault(capability => capability.Name == property.Name); + + Assert.IsNotNull(capability, $"{provider.Name} does not report {property.Name}."); + Assert.AreEqual(BsqliteStorageCatalog.FormatCapability(property.GetValue(provider.Capabilities)), capability.Value, $"{provider.Name}.{property.Name}"); + } + } + } + + [TestMethod] + public void Every_capability_is_documented() + { + // The capability rows explain themselves out of the contract's XML documentation; a capability + // without it is a row of values nobody can interpret. + foreach (var capability in BsqliteStorageCatalog.Providers[0].Capabilities) + { + Assert.IsFalse(string.IsNullOrWhiteSpace(capability.Summary), capability.Name); + } + } + + /// + /// The lab's verdicts against the library's resolver, run here directly over providers of the test's own: + /// the real capabilities of the first-party types and a probe that says what the configuration says. A + /// spread of configurations - preference orders, missing backends, bindings, the fallback opt-in - is + /// planned both ways, and every candidate's status has to agree. + /// + [TestMethod] + public async Task The_storage_lab_agrees_with_the_librarys_resolver() + { + var random = new Random(20261018); + string[] names = ["opfs", "indexeddb", "cache-storage", "in-memory"]; + + for (var i = 0; i < 40; i++) + { + var candidates = names.OrderBy(_ => random.Next()).Take(random.Next(1, names.Length + 1)).ToArray(); + var unavailable = names.Where(_ => random.Next(3) == 0).ToArray(); + var allow = random.Next(2) == 0; + var binding = random.Next(3) == 0 ? candidates[random.Next(candidates.Length)] : null; + var context = $"plan {i}: [{string.Join(", ", candidates)}], unavailable [{string.Join(", ", unavailable)}], allow {allow}, binding {binding ?? "none"}"; + + var plan = await McpStorageLabTests.PlanAsync(new() + { + ["candidates"] = string.Join(", ", candidates), + ["unavailable"] = string.Join(", ", unavailable), + ["allowNonPersistentFallback"] = allow, + ["existingBinding"] = binding, + }); + + var bindings = new BsqliteInMemoryStorageBindingStore(); + if (binding is not null) await bindings.SetProviderNameAsync("app.db", binding); + + var resolver = new BsqliteStorageProviderResolver( + candidates.Select(name => new DescribedProvider(_firstParty.Single(provider => provider.Name == name), unavailable.Contains(name))), + bindings); + + var selection = BsqliteStorageSelectionBuilder.Create(builder => + { + builder.Prefer(candidates[0]); + foreach (var name in candidates.Skip(1)) builder.Fallback(name); + if (allow) builder.AllowNonPersistentFallback(); + }); + + IReadOnlyList attempts; + BsqliteStorageResolution? resolution = null; + + try + { + resolution = await resolver.ResolveAsync("app.db", selection); + attempts = resolution.Attempts; + } + catch (BsqliteStorageUnavailableException exception) + { + attempts = exception.Attempts; + Assert.AreEqual(exception.Message, plan.Error, context); + } + + Assert.AreEqual(resolution is not null, plan.Opens, context); + Assert.AreEqual(resolution?.Provider.Name, plan.Provider, context); + Assert.AreEqual(resolution?.IsFirstChoice, plan.IsFirstChoice, context); + Assert.AreEqual(resolution?.WasDecidedByExistingData, plan.WasDecidedByExistingData, context); + Assert.AreEqual(resolution?.BetterProviderAvailable?.Name, plan.BetterProviderAvailable, context); + Assert.AreEqual(resolution?.Provider.Capabilities.RequiredBuild.ToString(), plan.EngineBuild, context); + + foreach (var attempt in attempts) + { + var candidate = plan.Candidates.Single(candidate => candidate.Name == attempt.ProviderName); + + Assert.AreEqual(attempt.Status.ToString(), candidate.Status, $"{context}: {attempt.ProviderName}"); + } + } + } + + [TestMethod] + public void Every_exported_type_is_in_the_api_index() + { + var exported = BsqliteApiCatalog.Assemblies + .SelectMany(assembly => assembly.GetExportedTypes()) + .Where(type => type.IsNested is false) + .Select(BsqliteApiCatalog.FriendlyName) + .ToHashSet(StringComparer.Ordinal); + + var indexed = BsqliteApiCatalog.Types.Select(type => type.Name).ToHashSet(StringComparer.Ordinal); + + CollectionAssert.AreEquivalent(exported.ToArray(), indexed.ToArray()); + } + + [TestMethod] + public void The_api_covers_every_package_and_the_entry_points() + { + CollectionAssert.AreEquivalent( + new[] { typeof(BsqliteConnection), typeof(BsqliteOpfsStorageProvider), typeof(BsqliteIndexedDbStorageProvider), typeof(BsqliteCacheStorageProvider), typeof(BsqliteInProcessTransport) } + .Select(type => type.Assembly).ToArray(), + BsqliteApiCatalog.Assemblies); + + var entryPoints = BsqliteApiCatalog.GetTypeDetails("BitBsqlite")!; + + Assert.AreEqual("Microsoft.Extensions.DependencyInjection", entryPoints.Namespace); + Assert.IsTrue(entryPoints.Members.Any(member => member.Name == "AddBitBsqliteServices")); + Assert.IsTrue(entryPoints.Members.Any(member => member.Name == "AddBitBsqliteDbContext")); + Assert.AreEqual("Microsoft.EntityFrameworkCore", BsqliteApiCatalog.GetTypeDetails("BsqliteDbContextOptionsExtensions")!.Namespace); + } + + [TestMethod] + public void Every_documented_public_type_has_its_summary_in_the_index() + { + // The XML documentation files have to reach the server's output folder for any of this to work; a + // build that stops copying them leaves every summary empty without failing anything else. + var withSummary = BsqliteApiCatalog.Types.Count(type => type.Summary is not null); + + Assert.IsGreaterThan(BsqliteApiCatalog.Types.Length * 9 / 10, withSummary, "Most public types should come with their XML summary."); + + foreach (var name in BsqliteXmlDocs.AssemblyNames) + { + Assert.IsTrue(File.Exists(Path.Combine(AppContext.BaseDirectory, $"{name}.xml")), $"{name}.xml is not next to the server."); + } + } + + [TestMethod] + public void The_sections_cover_every_heading_of_every_document() + { + Assert.IsGreaterThanOrEqualTo(12, BsqliteSourceCatalog.Documents.Length, "The README, six guides and five package READMEs are embedded."); + + foreach (var document in BsqliteSourceCatalog.Documents) + { + var text = BsqliteSourceCatalog.GetDocument(document.Name)!.Replace("\r\n", "\n", StringComparison.Ordinal); + + var fenced = false; + var headings = new List(); + + foreach (var line in text.Split('\n')) + { + if (line.TrimStart().StartsWith("```", StringComparison.Ordinal)) { fenced = fenced is false; continue; } + if (fenced is false && HeadingRegex().Match(line) is { Success: true } match) headings.Add(match.Groups["text"].Value.Trim()); + } + + var sections = BsqliteSourceCatalog.GetSections(document.Name); + + CollectionAssert.AreEqual(headings, sections.Select(section => section.Heading).ToArray(), document.Name); + Assert.HasCount(sections.Length, sections.Select(section => section.Key).Distinct(StringComparer.Ordinal), $"Every section of '{document.Name}' needs a key of its own."); + + foreach (var section in sections) + { + Assert.StartsWith($"{new string('#', section.Level)} {section.Heading}", BsqliteSourceCatalog.GetDocumentSection(document.Name, section.Key)!, $"{document.Name} > {section.Key}"); + } + } + } + + [TestMethod] + public async Task Every_tool_is_reachable_as_a_plain_get_with_the_same_answer() + { + using var http = McpTestHost.CreateHttpClient(); + + (string Tool, string Query, Dictionary? Arguments)[] calls = + [ + ("SearchBsqlite", "?query=live%20query&limit=3", new() { ["query"] = "live query", ["limit"] = 3 }), + ("GetBsqliteSetupGuide", "?scenario=storage", new() { ["scenario"] = "storage" }), + ("GetBsqliteDocument", "?name=storage§ion=Selection", new() { ["name"] = "storage", ["section"] = "Selection" }), + ("GetBsqliteApi", "?typeName=BsqliteDatabase", new() { ["typeName"] = "BsqliteDatabase" }), + ("GetBsqliteStorageProviders", "?provider=opfs", new() { ["provider"] = "opfs" }), + ("PlanBsqliteStorageSelection", "?unavailable=opfs&migrationMode=Manual", new() { ["unavailable"] = "opfs", ["migrationMode"] = "Manual" }), + ("CheckBsqliteSql", "?sql=PRAGMA%20page_size%20%3D%208192", new() { ["sql"] = "PRAGMA page_size = 8192" }), + ("GetBsqliteSourceFile", "?path=Demo/Client/Pages/Storage.razor", new() { ["path"] = "Demo/Client/Pages/Storage.razor" }), + ]; + + CollectionAssert.AreEquivalent(McpSurfaceTests.ExpectedTools, calls.Select(call => call.Tool).ToArray(), + "The HTTP mirror is being exercised for a different set of tools than the server publishes."); + + foreach (var (tool, query, arguments) in calls) + { + using var response = await http.GetAsync($"/api/mcp/{tool}{query}"); + + Assert.AreEqual(HttpStatusCode.OK, response.StatusCode, tool); + + var body = await response.Content.ReadAsStringAsync(); + + if (tool is "SearchBsqlite" or "PlanBsqliteStorageSelection" or "CheckBsqliteSql") + { + // The structured answers are compared as JSON: the same members, the same values, the nulls left + // out on both sides. + var protocol = (await McpCall.RawAsync(tool, arguments)).StructuredContent!.Value; + + Assert.IsTrue(JsonElement.DeepEquals(JsonDocument.Parse(body).RootElement, protocol), tool); + continue; + } + + Assert.AreEqual(await McpCall.TextAsync(tool, arguments), body, tool); + } + } + + [TestMethod] + public async Task The_demo_pages_catalog_lists_exactly_what_the_protocol_publishes() + { + using var http = McpTestHost.CreateHttpClient(); + + var catalog = await http.GetFromJsonAsync("/api/mcp/GetMcpCatalog"); + + Assert.IsNotNull(catalog); + CollectionAssert.AreEquivalent(McpSurfaceTests.ExpectedTools, catalog.Tools.Select(tool => tool.Name).ToArray()); + CollectionAssert.AreEquivalent((await McpTestHost.Client.ListPromptsAsync()).Select(prompt => prompt.Name).ToArray(), catalog.Prompts.Select(prompt => prompt.Name).ToArray()); + Assert.IsTrue(catalog.Tools.All(tool => tool.Title is not null && tool.Description.Length > 0)); + Assert.IsFalse(catalog.Tools.SelectMany(tool => tool.Parameters).Any(parameter => parameter.StartsWith("cancellationToken", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task The_mcp_page_and_the_existing_demo_endpoints_still_answer() + { + using var http = McpTestHost.CreateHttpClient(); + + using var page = await http.GetAsync("/mcp-server"); + using var products = await http.GetAsync("/api/products"); + + Assert.AreEqual(HttpStatusCode.OK, page.StatusCode); + Assert.Contains("MCP server", await page.Content.ReadAsStringAsync()); + Assert.AreEqual(HttpStatusCode.OK, products.StatusCode, "The server's own catalog endpoint is untouched."); + } + + /// + /// The demo is public and its server database lives in memory for the life of the process, so + /// nothing may add rows to it: anyone could grow it until the process ran out of memory. + /// + [TestMethod] + public async Task The_server_catalog_cannot_be_written_to() + { + using var http = McpTestHost.CreateHttpClient(); + + var before = JsonDocument.Parse(await http.GetStringAsync("/api/products")).RootElement.GetArrayLength(); + + using var post = await http.PostAsync( + "/api/products", + new StringContent("""{"name":"x","price":1}""", System.Text.Encoding.UTF8, "application/json")); + + Assert.IsFalse(post.IsSuccessStatusCode); + Assert.AreEqual(before, JsonDocument.Parse(await http.GetStringAsync("/api/products")).RootElement.GetArrayLength()); + } + + /// A first-party backend as the test describes it: its real capabilities, and the probe the plan says it gives. + private sealed class DescribedProvider(IBsqliteStorageProvider real, bool unavailable) : IBsqliteStorageProvider + { + public string Name => real.Name; + + public BsqliteStorageCapabilities Capabilities => real.Capabilities; + + public BsqliteJsModule? VfsModule => real.VfsModule; + + public IBsqliteStorageAdmin Admin => throw new NotSupportedException("Resolution never touches the admin surface."); + + public ValueTask ProbeAsync(CancellationToken cancellationToken = default) + => ValueTask.FromResult(unavailable ? BsqliteProbeResult.Unavailable($"'{Name}' was described as unavailable in this browser.") : BsqliteProbeResult.Available()); + } + + [GeneratedRegex(@"^#{2,4} (?.+)$")] + private static partial Regex HeadingRegex(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpResourcePromptTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpResourcePromptTests.cs new file mode 100644 index 00000000000..ada59bbb5a7 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpResourcePromptTests.cs @@ -0,0 +1,192 @@ +using Bit.Bsqlite.Demo.Server.Services; +using ModelContextProtocol; +using ModelContextProtocol.Protocol; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// The other two halves of the protocol - resources and prompts - plus the completions that let a +/// client fill in their placeholders and arguments. +/// +[TestClass] +public class McpResourcePromptTests +{ + // ---- Resources --------------------------------------------------------- + + [TestMethod] + public async Task The_fixed_resources_the_documents_and_the_setup_guides_are_listed() + { + var uris = (await McpTestHost.Client.ListResourcesAsync()).Select(resource => resource.Uri).ToArray(); + + CollectionAssert.IsSubsetOf(new[] { "bsqlite://guide", "bsqlite://api", "bsqlite://storage" }, uris); + CollectionAssert.Contains(uris, "bsqlite://docs/storage"); + CollectionAssert.Contains(uris, "bsqlite://docs/Bit.Bsqlite.Testing"); + CollectionAssert.Contains(uris, "bsqlite://setup/move-from-besql"); + + foreach (var document in BsqliteSourceCatalog.Documents) CollectionAssert.Contains(uris, $"bsqlite://docs/{Uri.EscapeDataString(document.Name)}"); + foreach (var scenario in BsqliteSetupGuide.Scenarios) CollectionAssert.Contains(uris, $"bsqlite://setup/{scenario}"); + } + + [TestMethod] + public async Task The_templates_are_published() + { + var templates = (await McpTestHost.Client.ListResourceTemplatesAsync()).Select(template => template.UriTemplate).ToArray(); + + CollectionAssert.IsSubsetOf(new[] + { + "bsqlite://docs/{name}", "bsqlite://api/{typeName}", "bsqlite://storage/{provider}", "bsqlite://setup/{scenario}", "bsqlite://source/{path}" + }, templates); + } + + [TestMethod] + public async Task Every_listed_resource_reads_back_as_text() + { + foreach (var resource in await McpTestHost.Client.ListResourcesAsync()) + { + var text = await ReadAsync(resource.Uri); + + Assert.IsFalse(string.IsNullOrWhiteSpace(text), resource.Uri); + Assert.DoesNotContain("There is no Bit.Bsqlite document", text, resource.Uri); + Assert.DoesNotContain("is not a known scenario", text, resource.Uri); + } + } + + [TestMethod] + public async Task The_templates_resolve_the_same_keys_the_tools_do() + { + Assert.AreEqual(BsqliteApiCatalog.RenderType("BsqliteDatabase"), await ReadAsync("bsqlite://api/BsqliteDatabase")); + Assert.AreEqual(BsqliteStorageCatalog.Render("opfs"), await ReadAsync("bsqlite://storage/opfs")); + Assert.AreEqual(BsqliteSetupGuide.Get("testing"), await ReadAsync("bsqlite://setup/testing")); + Assert.AreEqual(BsqliteSourceCatalog.GetDocument("live-queries"), await ReadAsync("bsqlite://docs/live-queries")); + Assert.AreEqual(BsqliteSourceCatalog.GetSourceFile("Demo/Client/Components/SessionLifecycle.razor"), + await ReadAsync("bsqlite://source/Demo%2FClient%2FComponents%2FSessionLifecycle.razor")); + Assert.AreEqual(await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "readme" }), await ReadAsync("bsqlite://guide")); + } + + [TestMethod] + public async Task An_unknown_key_reads_back_as_an_explanation() + { + Assert.Contains("no public type called 'Zebra'", await ReadAsync("bsqlite://api/Zebra")); + Assert.Contains("No first-party storage backend is called 'floppy'", await ReadAsync("bsqlite://storage/floppy")); + } + + // ---- Prompts ----------------------------------------------------------- + + private static readonly (string Name, Dictionary Arguments)[] _prompts = + [ + ("add-bsqlite-to-app", new() { ["hosting"] = "wasm" }), + ("move-from-besql", new() { ["dataSource"] = "Offline-Client.db" }), + ("debug-bsqlite-storage", new() { ["symptom"] = "the database is empty after a reload" }), + ]; + + [TestMethod] + public async Task Every_prompt_is_published() + { + var names = (await McpTestHost.Client.ListPromptsAsync()).Select(prompt => prompt.Name).ToArray(); + + CollectionAssert.AreEquivalent(_prompts.Select(prompt => prompt.Name).ToArray(), names); + } + + [TestMethod] + public async Task Every_prompt_carries_its_argument_and_names_only_real_tools() + { + foreach (var (name, arguments) in _prompts) + { + var text = await PromptTextAsync(name, arguments); + + foreach (var value in arguments.Values) Assert.Contains((string)value!, text, name); + + var mentioned = ToolNames.MentionedIn(text).ToArray(); + Assert.IsNotEmpty(mentioned, name); + + foreach (var tool in mentioned) CollectionAssert.Contains(McpSurfaceTests.ExpectedTools, tool, $"'{name}' mentions '{tool}', which is not a tool."); + } + } + + [TestMethod] + public async Task The_prompts_name_only_types_the_library_has() + { + // A prompt that tells an agent to use a type the library does not have sends it straight into the + // mistake the prompt exists to prevent. + var types = BsqliteApiCatalog.Types.Select(type => type.Name.Split('<')[0]).ToHashSet(StringComparer.Ordinal); + + foreach (var (name, arguments) in _prompts) + { + var text = await PromptTextAsync(name, arguments); + + foreach (var type in System.Text.RegularExpressions.Regex.Matches(text, @"`(I?Bsqlite[A-Z][A-Za-z]*)`").Select(match => match.Groups[1].Value)) + { + Assert.Contains(type, types, $"'{name}' names '{type}', which is not a public Bit.Bsqlite type."); + } + } + } + + [TestMethod] + public async Task The_prompts_that_take_a_default_answer_without_one() + { + var text = await PromptTextAsync("add-bsqlite-to-app", null); + + Assert.Contains("Its hosting model is: unknown.", text); + } + + [TestMethod] + public async Task An_unknown_prompt_is_an_error() + { + await Assert.ThrowsAsync(async () => await McpTestHost.Client.GetPromptAsync("write-my-app")); + } + + // ---- Completions ------------------------------------------------------- + + [TestMethod] + public async Task Resource_placeholders_complete_from_the_catalogs() + { + var types = await CompleteAsync("bsqlite://api/{typeName}", "typeName", "BsqliteData"); + var names = await CompleteAsync("bsqlite://docs/{name}", "name", "live"); + var providers = await CompleteAsync("bsqlite://storage/{provider}", "provider", "index"); + var scenarios = await CompleteAsync("bsqlite://setup/{scenario}", "scenario", "besql"); + var paths = await CompleteAsync("bsqlite://source/{path}", "path", "SessionLifecycle"); + + CollectionAssert.Contains(types, "BsqliteDatabase"); + Assert.StartsWith("BsqliteData", types[0], "A value the typed text starts should rank first."); + CollectionAssert.Contains(names, "live-queries"); + CollectionAssert.Contains(providers, "indexeddb"); + CollectionAssert.Contains(scenarios, "move-from-besql"); + CollectionAssert.Contains(paths, "Demo/Client/Components/SessionLifecycle.razor"); + } + + [TestMethod] + public async Task An_empty_prefix_completes_with_the_whole_set() + { + var scenarios = await CompleteAsync("bsqlite://setup/{scenario}", "scenario", ""); + + CollectionAssert.AreEquivalent(BsqliteSetupGuide.Scenarios, scenarios); + } + + [TestMethod] + public async Task Prompt_arguments_complete_with_the_phrasings_they_were_written_for() + { + var symptoms = (await McpTestHost.Client.CompleteAsync(new PromptReference { Name = "debug-bsqlite-storage" }, "symptom", "")).Completion.Values; + var hosting = (await McpTestHost.Client.CompleteAsync(new PromptReference { Name = "add-bsqlite-to-app" }, "hosting", "")).Completion.Values; + + Assert.IsNotEmpty(symptoms); + CollectionAssert.IsSubsetOf(new[] { "unknown", "standalone-wasm", "wasm", "auto", "server" }, hosting.ToArray()); + Assert.HasCount(hosting.Distinct().Count(), hosting, "A value declared by [AllowedValues] should not be offered twice."); + } + + private static async Task ReadAsync(string uri) + { + var result = await McpTestHost.Client.ReadResourceAsync(uri); + + return string.Join('\n', result.Contents.OfType().Select(contents => contents.Text)); + } + + private static async Task PromptTextAsync(string name, Dictionary? arguments) + { + var result = await McpTestHost.Client.GetPromptAsync(name, arguments); + + return string.Join('\n', result.Messages.Select(message => message.Content).OfType().Select(block => block.Text)); + } + + private static async Task CompleteAsync(string template, string argument, string typed) + => [.. (await McpTestHost.Client.CompleteAsync(new ResourceTemplateReference { Uri = template }, argument, typed)).Completion.Values]; +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSearchTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSearchTests.cs new file mode 100644 index 00000000000..e4544807225 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSearchTests.cs @@ -0,0 +1,213 @@ +using System.Text.RegularExpressions; +using Bit.Bsqlite.Demo.Server.Dtos; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// The search, and the paths it opens: every hit hands the caller a follow-up call, and that call has +/// to answer with the thing rather than with an apology about a key that no longer exists. +/// +[TestClass] +public partial class McpSearchTests +{ + /// Questions phrased the way a developer asks them, and a title that has to rank among the first hits. + private static readonly (string Query, string ExpectedTitle)[] _questions = + [ + ("live query", "Live queries"), + ("prerendering", "InteractiveWebAssembly with prerendering"), + ("opfs fallback indexeddb", "BsqliteIndexedDbStorageProvider"), + ("migrate to another backend", "Migration"), + ("unit test without a browser", "Unit testing with Bit.Bsqlite.Testing"), + ("attach", "Check SQL against the guards"), + ("moving off besql", "Moving off Bit.Besql"), + ("conformance kit", "The storage conformance kit"), + ("custom storage provider", "Writing a custom storage provider"), + ("export the database file", "ExportAsync"), + ("sticky binding", "UseBindingStore"), + ("interactive server", "InteractiveServer (not supported)"), + ("transactions savepoint", "BsqliteTransaction"), + ("non persistent fallback", "AllowNonPersistentFallback"), + ("hosting under a sub-path", "Hosting under a sub-path"), + ("SaveChanges synchronous", "BsqliteSynchronousApiNotSupportedException"), + ]; + + [TestMethod] + public async Task Common_questions_find_their_answer_among_the_first_hits() + { + // Every question is asked before anything is asserted, so one run reports every miss at once. + var misses = new List(); + + foreach (var (query, expected) in _questions) + { + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = query, ["limit"] = 6 }); + + if (result.Hits.Any(hit => hit.Title.Contains(expected, StringComparison.OrdinalIgnoreCase))) continue; + + misses.Add($"'{query}' did not find '{expected}' among: {string.Join(" | ", result.Hits.Select(hit => hit.Title))}"); + } + + Assert.IsEmpty(misses, string.Join(Environment.NewLine, misses)); + } + + [TestMethod] + public async Task A_word_inside_another_word_does_not_count() + { + // Every public type is named Bsqlite..., so "sqlite" counted inside a word would match all of them. + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "sqlite", ["limit"] = 50 }); + + Assert.IsNotEmpty(result.Hits); + + // A hit's snippet is cut around the occurrence that was counted, so every hit shows "sqlite" as a word of its own. + foreach (var hit in result.Hits) + { + Assert.IsTrue(WordStartRegex().IsMatch($"{hit.Title} {hit.Snippet}"), $"'{hit.Title}' was ranked for 'sqlite' without the word in it: {hit.Snippet}"); + } + } + + [GeneratedRegex("(?("SearchBsqlite", new() { ["query"] = query, ["limit"] = 50 }); + + Assert.IsFalse(result.Hits.Any(hit => hit.Kind.EndsWith(" section", StringComparison.Ordinal) && meta.Contains(hit.Title)), query); + } + } + + [TestMethod] + public async Task The_conformance_kits_rules_are_found_through_the_kit_not_one_by_one() + { + // The kit's tests are named as sentences about storage ("Export_ThrowsForADatabaseThatDoesNotExist"); hit + // one by one they would answer every question about exporting or throwing with a test. + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "export throws", ["limit"] = 50 }); + + Assert.IsFalse(result.Hits.Any(hit => hit.Context?.EndsWith("ConformanceTests", StringComparison.Ordinal) is true && hit.Kind == "API method"), + string.Join(" | ", result.Hits.Select(hit => hit.Title))); + + var kit = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "conformance", ["limit"] = 12 }); + + Assert.IsTrue(kit.Hits.Any(hit => hit.Title == "BsqliteStorageProviderConformanceTests")); + Assert.IsTrue(kit.Hits.Any(hit => hit.Title == "BsqliteStorageProviderConformanceTests.CreateProvider"), "What a test class overrides stays findable."); + } + + [TestMethod] + public async Task A_query_of_filler_words_is_told_what_to_search_for() + { + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "how do I do it" }); + + Assert.IsEmpty(result.Hits); + Assert.IsNotNull(result.Message); + Assert.Contains("Search for the thing itself", result.Message); + } + + [TestMethod] + public async Task A_query_that_matches_nothing_falls_back_to_the_indexes() + { + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "qqqq zzzz" }); + + Assert.IsEmpty(result.Hits); + Assert.IsNotNull(result.Terms); + Assert.Contains("GetBsqliteApi", result.Message!); + } + + [TestMethod] + public async Task A_near_miss_suggests_what_to_search_for_instead() + { + // A typo past the end of a title word ranks nothing - and is close enough to that word to be worth + // naming, rather than leaving the caller to conclude there is no such thing. + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "persistencex" }); + + Assert.IsEmpty(result.Hits); + Assert.IsNotNull(result.DidYouMean); + CollectionAssert.Contains(result.DidYouMean, "BsqlitePersistence"); + Assert.Contains("didYouMean", result.Message!); + } + + [TestMethod] + public async Task The_library_name_alone_still_finds_something() + { + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "bsqlite" }); + + Assert.IsNotEmpty(result.Hits); + } + + [TestMethod] + public async Task The_limit_is_honoured_and_clamped() + { + var three = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "storage", ["limit"] = 3 }); + var huge = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = "storage", ["limit"] = 5000 }); + + Assert.HasCount(3, three.Hits); + Assert.IsTrue(three.HasMore); + Assert.HasCount(50, huge.Hits); + } + + /// + /// Every follow-up call a hit suggests is made, verbatim, over the protocol. A renamed heading, a moved + /// file or a renamed tool leaves every tool working on its own and this path broken. + /// + [TestMethod] + public async Task Every_follow_up_call_a_hit_suggests_answers_with_the_thing() + { + var queries = _questions.Select(question => question.Query) + .Concat(["storage", "provider", "connection", "transaction", "live", "migration", "selection", "guard", "setup", "demo page", "readme", "binding", "limits"]); + + var calls = new HashSet(StringComparer.Ordinal); + + foreach (var query in queries) + { + var result = await McpCall.StructuredAsync("SearchBsqlite", new() { ["query"] = query, ["limit"] = 50 }); + + foreach (var hit in result.Hits) calls.Add(hit.Tool); + } + + Assert.IsGreaterThan(60, calls.Count, "The searches above should open many distinct follow-up calls."); + + foreach (var call in calls) + { + var match = FollowUpRegex().Match(call); + Assert.IsTrue(match.Success, $"'{call}' is not a call a caller can make verbatim."); + + var tool = match.Groups["tool"].Value; + CollectionAssert.Contains(McpSurfaceTests.ExpectedTools, tool, $"'{call}' names a tool that does not exist."); + + var arguments = new Dictionary(); + for (var i = 0; i < match.Groups["name"].Captures.Count; i++) + { + arguments[match.Groups["name"].Captures[i].Value] = match.Groups["value"].Captures[i].Value; + } + + if (tool is "PlanBsqliteStorageSelection") + { + var plan = await McpCall.StructuredAsync(tool, arguments); + Assert.IsTrue(plan.Opens, call); + continue; + } + + if (tool is "CheckBsqliteSql") + { + var check = await McpCall.StructuredAsync(tool, arguments); + Assert.IsNotEmpty(check.Statements, call); + continue; + } + + var text = await McpCall.TextAsync(tool, arguments.Count == 0 ? null : arguments); + + Assert.DoesNotContain("has no section called", text, call); + Assert.DoesNotContain("There is no Bit.Bsqlite document", text, call); + Assert.DoesNotContain("has no public type called", text, call); + Assert.DoesNotContain("No first-party storage backend", text, call); + Assert.DoesNotContain("is not a known scenario", text, call); + Assert.DoesNotContain("No source file at", text, call); + } + } + + [GeneratedRegex("""^(?\w+)\((?:(?\w+): "(?[^"]*)"(?:, (?\w+): "(?[^"]*)")*)?\)$""")] + private static partial Regex FollowUpRegex(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSqlCheckTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSqlCheckTests.cs new file mode 100644 index 00000000000..8f07e7b90f6 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSqlCheckTests.cs @@ -0,0 +1,166 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Demo.Server.Dtos; +using Bit.Bsqlite.Demo.Server.Services; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// CheckBsqliteSql on the statements the guards exist for, on the ones they let through, and on input +/// that is too much to check - every verdict held to what the library's own guard says for the same +/// text under the same backend's limits. +/// +[TestClass] +public class McpSqlCheckTests +{ + [TestMethod] + public async Task Without_sql_it_judges_every_guarded_statement_on_every_backend() + { + var check = await CheckAsync(null); + + CollectionAssert.AreEqual(BsqliteStorageCatalog.GuardedStatements, check.Statements.Select(statement => statement.Sql).ToArray()); + + foreach (var statement in check.Statements) + { + CollectionAssert.AreEquivalent(new[] { "opfs", "indexeddb", "cache-storage", "in-memory" }, statement.Backends.Select(verdict => verdict.Name).ToArray()); + } + + Assert.IsTrue(check.Notes.Any(note => note.Contains("Pass sql", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task Wal_is_refused_everywhere() + { + var verdicts = Verdicts(await CheckAsync("PRAGMA journal_mode=WAL")); + + Assert.IsTrue(verdicts.Values.All(verdict => verdict.Accepted is false)); + Assert.IsTrue(verdicts.Values.All(verdict => verdict.Error!.StartsWith("WAL mode is not available in the browser", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task Attach_is_refused_only_where_the_backend_declares_it_cannot_span_databases() + { + var verdicts = Verdicts(await CheckAsync("ATTACH DATABASE 'archive.db' AS archive")); + + Assert.IsFalse(verdicts["opfs"].Accepted); + Assert.Contains("ATTACH is not available on this storage backend", verdicts["opfs"].Error!); + Assert.IsTrue(verdicts["indexeddb"].Accepted); + Assert.IsTrue(verdicts["cache-storage"].Accepted); + Assert.IsTrue(verdicts["in-memory"].Accepted); + } + + [TestMethod] + public async Task Page_size_is_refused_only_where_the_backend_fixes_it() + { + var verdicts = Verdicts(await CheckAsync("PRAGMA page_size = 1024")); + + Assert.IsTrue(verdicts["opfs"].Accepted); + Assert.IsFalse(verdicts["indexeddb"].Accepted); + Assert.IsFalse(verdicts["cache-storage"].Accepted); + Assert.IsTrue(verdicts["in-memory"].Accepted); + Assert.Contains("PRAGMA page_size cannot be changed", verdicts["indexeddb"].Error!); + } + + [TestMethod] + public async Task A_query_is_accepted_and_reports_the_tables_a_live_query_over_it_watches() + { + var check = await CheckAsync("SELECT o.Number, c.DisplayName FROM \"Orders\" AS o INNER JOIN \"Customers\" AS c ON c.Id = o.CustomerId"); + var statement = check.Statements.Single(); + + Assert.IsTrue(statement.Backends.All(verdict => verdict.Accepted)); + CollectionAssert.AreEquivalent(new[] { "Customers", "Orders" }, statement.Tables); + Assert.IsFalse(statement.LooksLikeWrite); + Assert.IsFalse(statement.LooksLikeSchemaChange); + Assert.IsTrue(check.Notes.Any(note => note.Contains("re-runs when", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task A_write_and_a_schema_change_are_told_apart() + { + var write = (await CheckAsync("UPDATE Products SET Price = Price * 1.1 WHERE CategoryId = 2")).Statements.Single(); + var ddl = (await CheckAsync("CREATE INDEX IX_Products_Sku ON Products (Sku)")).Statements.Single(); + + Assert.IsTrue(write.LooksLikeWrite); + Assert.IsFalse(write.LooksLikeSchemaChange); + Assert.IsTrue(ddl.LooksLikeSchemaChange); + CollectionAssert.Contains(ddl.Tables, "Products"); + } + + [TestMethod] + public async Task Sql_longer_than_the_bound_is_not_half_checked() + { + var check = await CheckAsync(new string('x', BsqliteSqlCheck.MaxSqlLength + 1)); + + Assert.IsEmpty(check.Statements); + Assert.Contains($"{BsqliteSqlCheck.MaxSqlLength}", check.Error!); + } + + [TestMethod] + public async Task The_longest_sql_it_takes_is_checked_quickly_whatever_it_holds() + { + // The library's patterns scan from every statement boundary, and the input is a stranger's: the bound + // on its length is what keeps the worst case - a boundary every few characters, each opening a scan - + // a fraction of a second rather than a stalled request. + static string Fill(string unit) => string.Concat(Enumerable.Repeat(unit, BsqliteSqlCheck.MaxSqlLength / unit.Length + 1))[..BsqliteSqlCheck.MaxSqlLength]; + + string[] worst = [Fill("; with "), Fill(";/*"), Fill(";-- x\n"), Fill("pragma attach from ")]; + + foreach (var sql in worst) + { + var clock = System.Diagnostics.Stopwatch.StartNew(); + + var check = await CheckAsync(sql); + + Assert.IsNotEmpty(check.Statements); + Assert.IsLessThan(TimeSpan.FromSeconds(2), clock.Elapsed, $"{sql[..20]}... took {clock.Elapsed}."); + } + } + + [TestMethod] + public async Task The_verdicts_are_the_guards_own() + { + // The library's guard, run here under the limits a session on each backend gets, must say exactly + // what the tool said - message for message. + string[] statements = + [ + "PRAGMA journal_mode = 'wal'", + "attach database 'x.db' as x", + "PRAGMA page_size=65536", + "BEGIN; INSERT INTO t VALUES (1); ATTACH 'y.db' AS y; COMMIT;", + "SELECT 'attach' AS word", + ]; + + foreach (var sql in statements) + { + var verdicts = Verdicts(await CheckAsync(sql)); + + foreach (var provider in BsqliteStorageCatalog.Providers) + { + var capabilities = BsqliteStorageCatalog.Instance(provider.Name)!.Capabilities; + var limits = new BsqliteRuntimeLimits + { + SupportsMultiDatabaseTransactions = capabilities.SupportsMultiDatabaseTransactions, + CanChangePageSize = capabilities.CanChangePageSize, + }; + + string? expected = null; + try + { + BsqliteFeatureGuards.EnsureSupported(sql, limits); + } + catch (BsqliteException exception) + { + expected = exception.Message; + } + + Assert.AreEqual(expected is null, verdicts[provider.Name].Accepted, $"{sql} on {provider.Name}"); + Assert.AreEqual(expected, verdicts[provider.Name].Error, $"{sql} on {provider.Name}"); + } + } + } + + internal static Task CheckAsync(string? sql) + => McpCall.StructuredAsync("CheckBsqliteSql", sql is null ? null : new() { ["sql"] = sql }); + + private static Dictionary Verdicts(BsqliteSqlCheckDto check) + => check.Statements.Single().Backends.ToDictionary(verdict => verdict.Name, StringComparer.Ordinal); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpStorageLabTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpStorageLabTests.cs new file mode 100644 index 00000000000..8d7483f7de6 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpStorageLabTests.cs @@ -0,0 +1,263 @@ +using Bit.Bsqlite.Storage.Opfs; +using Bit.Bsqlite.Demo.Server.Dtos; +using Bit.Bsqlite.Storage.IndexedDb; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// PlanBsqliteStorageSelection on the situations an application actually meets: a first open in a +/// browser that has everything, one that lacks OPFS, a database already bound to a backend, a better +/// backend appearing under each migration mode, a migration that cannot fit, a backend that is gone, +/// and configurations the library refuses outright. Every expectation is the library's documented +/// rule; the lab is only ever the messenger. +/// +[TestClass] +public class McpStorageLabTests +{ + [TestMethod] + public async Task A_first_open_with_everything_available_lands_on_the_first_choice() + { + var plan = await PlanAsync(); + + Assert.IsTrue(plan.Opens); + Assert.AreEqual("opfs", plan.Provider); + Assert.AreEqual("Synchronous", plan.EngineBuild); + Assert.AreEqual(BsqliteOpfsStorageProvider.VfsModuleUrl, plan.VfsModule); + Assert.IsTrue(plan.IsFirstChoice); + Assert.IsFalse(plan.WasDecidedByExistingData); + Assert.IsNull(plan.BindingBefore); + Assert.AreEqual("opfs", plan.BindingAfter, "The binding is written once the open succeeds."); + Assert.AreEqual("Selected", Candidate(plan, "opfs").Status); + Assert.AreEqual("NotProbed", Candidate(plan, "indexeddb").Status); + Assert.IsEmpty(plan.Log, "A first choice is nothing to log."); + } + + [TestMethod] + public async Task Without_opfs_it_falls_back_and_says_so() + { + var plan = await PlanAsync(new() { ["unavailable"] = "opfs" }); + + Assert.IsTrue(plan.Opens); + Assert.AreEqual("indexeddb", plan.Provider); + Assert.AreEqual("AsyncCapable", plan.EngineBuild); + Assert.AreEqual(BsqliteIndexedDbStorageProvider.VfsModuleUrl, plan.VfsModule); + Assert.IsFalse(plan.IsFirstChoice); + Assert.AreEqual("Unavailable", Candidate(plan, "opfs").Status); + Assert.Contains("unavailable", Candidate(plan, "opfs").Explanation!); + Assert.IsTrue(plan.Log.Any(line => line.StartsWith("Warning:", StringComparison.Ordinal) && line.Contains("opfs", StringComparison.Ordinal)), + "A fallback is logged as a warning naming what was rejected."); + } + + [TestMethod] + public async Task A_non_persistent_fallback_needs_the_opt_in() + { + var refused = await PlanAsync(new() { ["candidates"] = "opfs, in-memory", ["unavailable"] = "opfs" }); + + Assert.IsFalse(refused.Opens); + Assert.AreEqual("RejectedAsNonPersistent", Candidate(refused, "in-memory").Status); + Assert.Contains("No storage provider could open database 'app.db'", refused.Error!); + Assert.Contains("AllowNonPersistentFallback()", Candidate(refused, "in-memory").Explanation!); + Assert.IsNull(refused.BindingAfter, "A failed open leaves no binding behind."); + + var accepted = await PlanAsync(new() { ["candidates"] = "opfs, in-memory", ["unavailable"] = "opfs", ["allowNonPersistentFallback"] = true }); + + Assert.IsTrue(accepted.Opens); + Assert.AreEqual("in-memory", accepted.Provider); + Assert.IsTrue(accepted.Notes.Any(note => note.Contains("does not survive a reload", StringComparison.Ordinal))); + Assert.Contains(".AllowNonPersistentFallback()", accepted.Configuration); + } + + [TestMethod] + public async Task A_non_persistent_first_choice_needs_no_opt_in() + { + var plan = await PlanAsync(new() { ["candidates"] = "in-memory" }); + + Assert.IsTrue(plan.Opens); + Assert.AreEqual("in-memory", plan.Provider); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("never quietly substituted", StringComparison.Ordinal)), "A lone candidate is strict."); + } + + [TestMethod] + public async Task Existing_data_outranks_preference() + { + var plan = await PlanAsync(new() { ["existingBinding"] = "indexeddb" }); + + Assert.IsTrue(plan.Opens); + Assert.AreEqual("indexeddb", plan.Provider); + Assert.IsTrue(plan.WasDecidedByExistingData); + Assert.AreEqual("opfs", plan.BetterProviderAvailable); + Assert.AreEqual("indexeddb", plan.BindingAfter); + Assert.AreEqual("NotAttempted", plan.Migration!.Outcome, "KeepExisting moves nothing."); + Assert.Contains("BetterProviderAvailable", Candidate(plan, "opfs").Explanation!); + Assert.IsTrue(plan.Log.Any(line => line.StartsWith("Information:", StringComparison.Ordinal) && line.Contains("KeepExisting", StringComparison.Ordinal))); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("AutomaticOnOpen", StringComparison.Ordinal)), "The note names the mode that would move it."); + } + + [TestMethod] + public async Task Manual_leaves_the_move_to_the_application() + { + var plan = await PlanAsync(new() { ["existingBinding"] = "indexeddb", ["migrationMode"] = "Manual" }); + + Assert.AreEqual("indexeddb", plan.Provider); + Assert.AreEqual("NotAttempted", plan.Migration!.Outcome); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("BsqliteStorageMigrator.MigrateAsync", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task AutomaticOnOpen_moves_the_database_before_opening_it() + { + var plan = await PlanAsync(new() { ["existingBinding"] = "indexeddb", ["migrationMode"] = "AutomaticOnOpen" }); + + Assert.IsTrue(plan.Opens); + Assert.AreEqual("opfs", plan.Provider); + Assert.AreEqual("indexeddb", plan.BindingBefore); + Assert.AreEqual("opfs", plan.BindingAfter); + Assert.AreEqual("Migrated", plan.Migration!.Outcome); + Assert.AreEqual("indexeddb", plan.Migration.From); + Assert.AreEqual("opfs", plan.Migration.To); + Assert.IsTrue(plan.Migration.Copied, "The stand-in database was exported, verified and imported."); + Assert.IsNull(plan.BetterProviderAvailable); + Assert.Contains("AutomaticOnOpen moved it", Candidate(plan, "indexeddb").Explanation!); + Assert.Contains(".WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen)", plan.Configuration); + } + + [TestMethod] + public async Task A_migration_without_room_fails_safely_and_opens_where_the_data_is() + { + var plan = await PlanAsync(new() { ["existingBinding"] = "indexeddb", ["migrationMode"] = "AutomaticOnOpen", ["freeBytes"] = 1000 }); + + Assert.IsTrue(plan.Opens); + Assert.AreEqual("indexeddb", plan.Provider); + Assert.AreEqual("indexeddb", plan.BindingAfter, "The migrator leaves the binding untouched."); + Assert.AreEqual("Failed", plan.Migration!.Outcome); + Assert.Contains("bytes free", plan.Migration.Error!); + Assert.IsTrue(plan.Log.Any(line => line.StartsWith("Warning:", StringComparison.Ordinal) && line.Contains("BsqliteQuotaExceededException", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task Moving_off_in_memory_copies_nothing() + { + // The in-memory backend's admin store never sees what a connection wrote, so the library's migrator + // finds nothing to export and only flips the binding - which the plan says rather than hides. + var plan = await PlanAsync(new() { ["candidates"] = "opfs, in-memory", ["existingBinding"] = "in-memory", ["migrationMode"] = "AutomaticOnOpen" }); + + Assert.AreEqual("opfs", plan.Provider); + Assert.AreEqual("Migrated", plan.Migration!.Outcome); + Assert.IsFalse(plan.Migration.Copied); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("without copying anything", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task A_database_on_a_backend_this_browser_lacks_does_not_open_elsewhere() + { + var plan = await PlanAsync(new() { ["existingBinding"] = "opfs", ["unavailable"] = "opfs" }); + + Assert.IsFalse(plan.Opens); + Assert.Contains("will not open an empty", plan.Error!); + Assert.AreEqual("Unavailable", Candidate(plan, "opfs").Status); + Assert.AreEqual("opfs", plan.BindingAfter, "The binding still points at the data."); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("Admin.DeleteAsync", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task A_database_on_a_backend_no_longer_registered_does_not_open_elsewhere() + { + var plan = await PlanAsync(new() { ["candidates"] = "opfs", ["existingBinding"] = "indexeddb" }); + + Assert.IsFalse(plan.Opens); + Assert.AreEqual("NotRegistered", Candidate(plan, "indexeddb").Status); + Assert.IsFalse(Candidate(plan, "indexeddb").Registered); + Assert.IsNull(Candidate(plan, "indexeddb").Rank, "It is the binding, not a candidate."); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("Register it again", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task A_candidate_that_is_not_registered_is_named_with_its_registration() + { + var plan = await PlanAsync(new() { ["registered"] = "indexeddb" }); + + Assert.AreEqual("indexeddb", plan.Provider); + Assert.AreEqual("NotRegistered", Candidate(plan, "opfs").Status); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("AddStorageProvider()", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task A_custom_backend_is_planned_with_the_facts_the_caller_gives() + { + var plan = await PlanAsync(new() { ["candidates"] = "acme-encrypted, indexeddb", ["nonPersistent"] = "acme-encrypted", ["unavailable"] = "acme-encrypted" }); + + var custom = Candidate(plan, "acme-encrypted"); + + Assert.IsFalse(custom.FirstParty); + Assert.IsFalse(custom.Persistent); + Assert.AreEqual("AsyncCapable", custom.EngineBuild); + Assert.AreEqual("Unavailable", custom.Status); + Assert.AreEqual("indexeddb", plan.Provider); + Assert.Contains("YourStorageProvider", plan.Configuration); + Assert.Contains(".Prefer(\"acme-encrypted\")", plan.Configuration); + Assert.Contains(".Fallback(BsqliteIndexedDbStorageProvider.ProviderName)", plan.Configuration); + } + + [TestMethod] + public async Task A_first_party_backend_keeps_its_declared_persistence() + { + var plan = await PlanAsync(new() { ["nonPersistent"] = "opfs" }); + + Assert.IsTrue(Candidate(plan, "opfs").Persistent); + Assert.IsTrue(plan.Notes.Any(note => note.Contains("first-party backend", StringComparison.Ordinal))); + } + + [TestMethod] + public async Task A_configuration_the_library_refuses_is_refused_with_its_own_message() + { + var twice = await PlanAsync(new() { ["candidates"] = "opfs, OPFS" }); + var duplicate = await PlanAsync(new() { ["registered"] = "opfs, OPFS" }); + var empty = await PlanAsync(new() { ["candidates"] = "" }); + + Assert.IsFalse(twice.Opens); + Assert.Contains("is already a candidate", twice.Error!); + Assert.Contains("Two storage providers are registered under the name", duplicate.Error!); + Assert.Contains("No storage provider was selected", empty.Error!); + Assert.IsEmpty(twice.Log); + } + + [TestMethod] + public async Task A_name_with_a_line_break_is_refused_before_it_reaches_the_log() + { + var database = await PlanAsync(new() { ["databaseName"] = "app.db\nWarning: forged" }); + var binding = await PlanAsync(new() { ["existingBinding"] = "opfs\r\nWarning: forged" }); + + Assert.IsFalse(database.Opens); + Assert.Contains("databaseName contains a control character", database.Error!); + Assert.IsEmpty(database.Log); + Assert.Contains("existingBinding contains a control character", binding.Error!); + Assert.IsEmpty(binding.Log); + } + + [TestMethod] + public async Task The_configuration_is_the_csharp_that_produces_the_plan() + { + var plan = await PlanAsync(new() { ["candidates"] = "opfs, indexeddb, in-memory", ["allowNonPersistentFallback"] = true, ["databaseName"] = "orders.db" }); + + Assert.AreEqual( + """ + builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .AddStorageProvider() + .AddStorageProvider() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()) + .AddDatabase("orders.db")); + """.ReplaceLineEndings("\n"), plan.Configuration); + } + + internal static Task PlanAsync(Dictionary? arguments = null) + => McpCall.StructuredAsync("PlanBsqliteStorageSelection", arguments); + + private static BsqliteCandidateReportDto Candidate(BsqliteStoragePlanDto plan, string name) + => plan.Candidates.SingleOrDefault(candidate => candidate.Name == name) + ?? throw new AssertFailedException($"The plan has no candidate '{name}': {string.Join(", ", plan.Candidates.Select(candidate => candidate.Name))}"); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSurfaceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSurfaceTests.cs new file mode 100644 index 00000000000..c794c4fecb7 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpSurfaceTests.cs @@ -0,0 +1,229 @@ +using System.Text.Json; +using Bit.Bsqlite.Data; +using ModelContextProtocol.Client; +using Bit.Bsqlite.Demo.Server.Services; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// The server as a client sees it before calling anything: the handshake, the published tools, the +/// annotations that decide whether a client may run a tool without asking a person, and the schemas it +/// validates arguments and answers against. None of this is visible from the C# side - an attribute +/// left off compiles perfectly and only goes missing on the wire. +/// +[TestClass] +public class McpSurfaceTests +{ + /// + /// The published set, spelled out. A tool added without being added here fails on purpose: a new tool + /// needs its own coverage, and a renamed one breaks every client that had learned the old name - + /// including bitplatform.dev/mcp, which proxies these names as they are. + /// + internal static readonly string[] ExpectedTools = + [ + "CheckBsqliteSql", + "GetBsqliteApi", + "GetBsqliteDocument", + "GetBsqliteSetupGuide", + "GetBsqliteSourceFile", + "GetBsqliteStorageProviders", + "PlanBsqliteStorageSelection", + "SearchBsqlite", + ]; + + /// The tools that answer with a DTO, and therefore have to publish an output schema. + private static readonly string[] _structuredTools = ["CheckBsqliteSql", "PlanBsqliteStorageSelection", "SearchBsqlite"]; + + /// The reference tools, and the key each one takes - optional, since leaving it out answers with the index. + private static readonly (string Tool, string Argument)[] _keyedTools = + [ + ("GetBsqliteApi", "typeName"), + ("GetBsqliteDocument", "name"), + ("GetBsqliteStorageProviders", "provider"), + ("GetBsqliteSourceFile", "path"), + ("CheckBsqliteSql", "sql"), + ]; + + private static IList _tools = []; + + [ClassInitialize] + public static async Task ListToolsAsync(TestContext context) + { + _tools = await McpTestHost.Client.ListToolsAsync(cancellationToken: context.CancellationToken); + } + + [TestMethod] + public void The_server_introduces_itself_as_the_library_at_its_own_version() + { + var info = McpTestHost.Client.ServerInfo; + + Assert.AreEqual("bit-bsqlite", info.Name); + Assert.AreEqual("Bit.Bsqlite", info.Title); + Assert.AreEqual("https://github.com/bitfoundation/bitplatform/tree/develop/src/Bsqlite", info.WebsiteUrl); + Assert.AreEqual(BsqliteServerInstructions.BsqliteVersion, info.Version); + Assert.AreNotEqual("unknown", info.Version); + Assert.DoesNotContain("+", info.Version, "The build metadata should have been stripped from the version."); + } + + [TestMethod] + public void The_version_is_the_library_package_version() + { + // The package version is stamped into the assembly by Bit.Build.props; the MCP version must be + // that, not the demo host's, since the answers are about the library. + var informational = typeof(BsqliteConnection).Assembly + .GetCustomAttributes(typeof(System.Reflection.AssemblyInformationalVersionAttribute), false) + .OfType() + .Single().InformationalVersion; + + Assert.StartsWith(BsqliteServerInstructions.BsqliteVersion, informational); + } + + [TestMethod] + public void The_handshake_advertises_tools_resources_prompts_and_completions() + { + var capabilities = McpTestHost.Client.ServerCapabilities; + + Assert.IsNotNull(capabilities.Tools); + Assert.IsNotNull(capabilities.Resources); + Assert.IsNotNull(capabilities.Prompts); + Assert.IsNotNull(capabilities.Completions); + } + + [TestMethod] + public void The_instructions_name_the_version_and_only_tools_that_exist() + { + var instructions = McpTestHost.Client.ServerInstructions; + + Assert.AreEqual(BsqliteServerInstructions.Text, instructions); + Assert.Contains(BsqliteServerInstructions.BsqliteVersion, instructions!); + + var mentioned = ToolNames.MentionedIn(instructions!).ToArray(); + + Assert.IsNotEmpty(mentioned); + foreach (var tool in mentioned) CollectionAssert.Contains(ExpectedTools, tool, $"The instructions mention '{tool}', which is not a tool."); + } + + [TestMethod] + public void The_instructions_name_only_namespaces_the_packages_declare() + { + // The instructions steer an agent's using directives; a namespace written there that no package + // declares is a compiler error handed out before the first tool call. + var namespaces = BsqliteApiCatalog.Types.Select(type => type.Namespace).ToHashSet(StringComparer.Ordinal); + + foreach (var match in System.Text.RegularExpressions.Regex.Matches(BsqliteServerInstructions.Text, @"`((?:Bit|Microsoft)\.[A-Za-z.]+?)`").Cast()) + { + var name = match.Groups[1].Value; + if (name.EndsWith(".*", StringComparison.Ordinal)) continue; + + Assert.IsTrue(namespaces.Contains(name) || namespaces.Any(ns => ns.StartsWith($"{name}.", StringComparison.Ordinal)), + $"The instructions name '{name}', which no Bit.Bsqlite package declares a public type in."); + } + } + + [TestMethod] + public void Every_expected_tool_is_published_and_nothing_else_is() + { + CollectionAssert.AreEquivalent(ExpectedTools, _tools.Select(tool => tool.Name).ToArray()); + } + + [TestMethod] + public void Every_tool_name_carries_the_library_name() + { + // bitplatform.dev/mcp merges the tools of every bit library into one list, where a name is the only + // address a client has and the first server to claim one wins. "Bsqlite" in every name is what keeps + // these from colliding with SearchBink, GetBmageDocument and the rest. + foreach (var tool in _tools) Assert.Contains("Bsqlite", tool.Name, $"'{tool.Name}' would be ambiguous next to the other libraries' tools."); + } + + [TestMethod] + public void Every_tool_carries_a_title_and_a_description_worth_reading() + { + foreach (var tool in _tools) + { + Assert.IsFalse(string.IsNullOrWhiteSpace(tool.Title), $"'{tool.Name}' has no title."); + Assert.IsGreaterThan(80, tool.Description.Length, $"'{tool.Name}' has a description too short to choose it by."); + } + } + + [TestMethod] + public void Every_tool_is_read_only_and_closed_world() + { + foreach (var tool in _tools) + { + var annotations = tool.ProtocolTool.Annotations; + + Assert.IsNotNull(annotations, $"'{tool.Name}' has no annotations."); + Assert.AreEqual(true, annotations.ReadOnlyHint, $"'{tool.Name}' is not marked read-only; a client will stop and ask before running it."); + Assert.AreEqual(false, annotations.OpenWorldHint, $"'{tool.Name}' is not marked closed-world."); + } + } + + [TestMethod] + public void Only_the_machine_actionable_tools_publish_an_output_schema() + { + foreach (var tool in _tools) + { + var structured = _structuredTools.Contains(tool.Name); + + Assert.AreEqual(structured, tool.ReturnJsonSchema is not null, + structured ? $"'{tool.Name}' answers with a DTO but publishes no output schema." : $"'{tool.Name}' answers in prose but publishes an output schema."); + } + } + + [TestMethod] + public void The_key_of_every_reference_tool_is_optional() + { + foreach (var (name, argument) in _keyedTools) + { + var tool = _tools.Single(tool => tool.Name == name); + + Assert.IsTrue(tool.JsonSchema.GetProperty("properties").TryGetProperty(argument, out _), $"'{name}' does not take '{argument}'."); + + if (tool.JsonSchema.TryGetProperty("required", out var required)) + { + Assert.IsFalse(required.EnumerateArray().Any(value => value.GetString() == argument), + $"'{name}' requires '{argument}', so it can no longer answer with the index."); + } + } + } + + [TestMethod] + public void The_scenario_is_a_closed_set_in_the_schema() + { + var tool = _tools.Single(tool => tool.Name == "GetBsqliteSetupGuide"); + + CollectionAssert.AreEquivalent(BsqliteSetupGuide.Scenarios, EnumOf(tool, "scenario")); + } + + [TestMethod] + public void The_migration_mode_is_the_libraries_own_enum_in_the_schema() + { + // [AllowedValues] takes constants, so the values are written out on the tool; this is what keeps them + // the enum's - a mode added to the library and not to the tool fails here. + var tool = _tools.Single(tool => tool.Name == "PlanBsqliteStorageSelection"); + + CollectionAssert.AreEquivalent(Enum.GetNames(), EnumOf(tool, "migrationMode")); + } + + [TestMethod] + public void The_cancellation_token_is_not_an_argument() + { + var tool = _tools.Single(tool => tool.Name == "PlanBsqliteStorageSelection"); + + Assert.IsFalse(tool.JsonSchema.GetProperty("properties").TryGetProperty("cancellationToken", out _)); + } + + [TestMethod] + public void The_tool_list_stays_cheap_to_send_with_every_request() + { + // Every tool's name, description and schemas ride along with every request an agent makes. + var size = _tools.Sum(tool => tool.Name.Length + tool.Description.Length + + JsonSerializer.Serialize(tool.JsonSchema).Length + + (tool.ReturnJsonSchema is null ? 0 : JsonSerializer.Serialize(tool.ReturnJsonSchema).Length)); + + Assert.IsLessThan(32_000, size, $"The tool list costs {size} characters per request."); + } + + private static string?[] EnumOf(McpClientTool tool, string argument) + => [.. tool.JsonSchema.GetProperty("properties").GetProperty(argument).GetProperty("enum").EnumerateArray().Select(value => value.GetString())]; +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpTestHost.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpTestHost.cs new file mode 100644 index 00000000000..5d1881dc465 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpTestHost.cs @@ -0,0 +1,61 @@ +using Microsoft.AspNetCore.Mvc.Testing; +using ModelContextProtocol.Client; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// The demo server, booted once for the whole test run, and one MCP client connected to it. +/// +/// The app is started from its own Program.cs rather than from a re-registration of the MCP +/// pieces, so what the tests exercise is the wiring that ships: the same AddMcpServer options, +/// the same MapMcp("/mcp"), the same DI container the tools resolve out of - and the same +/// in-memory server database and /api/products endpoint the demo has always had next to them. And +/// the tests talk to it through the protocol - initialize, tools/list, tools/call - instead of calling +/// the tool methods, which is the only way to catch a tool that is written correctly but never reaches +/// the wire: a missing attribute, a schema a client rejects, a return value that does not serialize. +/// +/// +[TestClass] +public static class McpTestHost +{ + private static WebApplicationFactory? _factory; + private static McpClient? _client; + + /// The running app's service provider, for the services a tool resolves per request. + public static IServiceProvider Services => (_factory ?? throw NotStarted()).Services; + + /// The connected MCP client. One session serves every test - the server is read-only. + public static McpClient Client => _client ?? throw NotStarted(); + + /// A plain HTTP client against the same app, for the /api/mcp/... endpoints that mirror the tools. + public static HttpClient CreateHttpClient() => (_factory ?? throw NotStarted()).CreateClient(); + + [AssemblyInitialize] + public static async Task InitializeAsync(TestContext context) + { + _factory = new WebApplicationFactory(); + + // The handshake happens here: a failure to connect fails the whole run loudly, which is the right + // report for "the MCP server does not come up" - far better than 200 identical per-test failures. + _client = await McpClient.CreateAsync(new HttpClientTransport(new HttpClientTransportOptions + { + Endpoint = new Uri("http://localhost/mcp"), + // Pinned rather than auto-detected: the app maps the streamable HTTP transport, and a silent + // fallback to the deprecated SSE one would hide exactly that going missing. + TransportMode = HttpTransportMode.StreamableHttp, + // Nothing here is server-initiated, and the standalone stream would otherwise hold a request + // open for the lifetime of the test run. + EnableStandaloneGetStream = false, + Name = "bit-bsqlite-tests" + }, _factory.CreateClient(), loggerFactory: null, ownsHttpClient: true), cancellationToken: context.CancellationToken); + } + + [AssemblyCleanup] + public static async Task CleanupAsync() + { + if (_client is not null) await _client.DisposeAsync(); + if (_factory is not null) await _factory.DisposeAsync(); + } + + private static InvalidOperationException NotStarted() => new("The MCP test host has not been started yet."); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpToolTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpToolTests.cs new file mode 100644 index 00000000000..3e95e55d0fc --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/McpToolTests.cs @@ -0,0 +1,434 @@ +using Bit.Bsqlite.Demo.Server.Services; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// Every reference tool's answers over the protocol: the thing a key names, the index a missing key +/// answers with, and the guidance a wrong key gets instead of an error. +/// +[TestClass] +public class McpToolTests +{ + // ---- Documents --------------------------------------------------------- + + [TestMethod] + public async Task The_document_index_lists_the_readme_the_guides_and_the_packages() + { + var index = await McpCall.TextAsync("GetBsqliteDocument"); + + foreach (var name in new[] { "readme", "getting-started", "storage", "live-queries", "architecture", "limits", "extending", "Bit.Bsqlite.Testing", "Bit.Bsqlite.Storage.Opfs" }) + { + Assert.Contains($"- **{name}** - ", index); + } + + Assert.Contains(" - Selection (", index, "A guide's sections are listed under it."); + } + + [TestMethod] + public void The_guides_are_listed_in_the_order_the_readme_links_them() + { + var guides = BsqliteSourceCatalog.Documents.Where(document => document.Kind == "Guide").Select(document => document.Name).ToArray(); + + CollectionAssert.AreEqual(new[] { "getting-started", "storage", "live-queries", "architecture", "limits", "extending" }, guides); + } + + [TestMethod] + public async Task A_whole_document_comes_back_verbatim() + { + var storage = await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "storage" }); + + Assert.AreEqual(BsqliteSourceCatalog.GetDocument("storage"), storage); + Assert.StartsWith("# Storage", storage); + } + + [TestMethod] + public async Task A_document_is_found_whatever_it_is_called() + { + foreach (var name in new[] { "STORAGE", "storage.md", "docs/storage.md" }) + { + Assert.StartsWith("# Storage", await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = name }), name); + } + + // A package README by the part of its name that says something. + Assert.StartsWith("# Bit.Bsqlite.Testing", await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "testing" })); + Assert.StartsWith("# Bit.Bsqlite.Storage.Opfs", await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "opfs" })); + Assert.StartsWith("# bit Bsqlite", await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "README" })); + } + + [TestMethod] + public async Task A_section_is_found_regardless_of_case_punctuation_and_little_words() + { + var section = await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "storage", ["section"] = "first party BACKENDS" }); + + Assert.StartsWith("## First-party backends", section); + Assert.Contains("`opfs`", section); + Assert.DoesNotContain("## Selection", section, "A section stops at the next heading of its rank."); + } + + [TestMethod] + public async Task A_section_includes_its_sub_sections() + { + var section = await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "getting-started", ["section"] = "Use EF Core" }); + + Assert.Contains("### Without the factory", section); + Assert.DoesNotContain("## Use ADO.NET", section); + } + + [TestMethod] + public async Task An_unknown_section_lists_the_close_ones_or_all_of_them() + { + var close = await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "storage", ["section"] = "backends" }); + var none = await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "storage", ["section"] = "zebra" }); + + Assert.Contains("Did you mean", close); + Assert.Contains("'First-party backends'", close); + Assert.Contains("Its sections are", none); + Assert.Contains("'Writing a provider'", none); + } + + [TestMethod] + public async Task An_unknown_document_lists_the_real_ones() + { + var answer = await McpCall.TextAsync("GetBsqliteDocument", new() { ["name"] = "changelog" }); + + Assert.Contains("There is no Bit.Bsqlite document called 'changelog'", answer); + foreach (var document in BsqliteSourceCatalog.Documents) Assert.Contains($"'{document.Name}'", answer); + } + + // ---- API --------------------------------------------------------------- + + [TestMethod] + public async Task The_api_index_groups_the_types_by_package_and_namespace() + { + var index = await McpCall.TextAsync("GetBsqliteApi"); + + foreach (var package in new[] { "Bit.Bsqlite", "Bit.Bsqlite.Storage.Opfs", "Bit.Bsqlite.Storage.IndexedDb", "Bit.Bsqlite.Storage.CacheStorage", "Bit.Bsqlite.Testing" }) + { + Assert.Contains($"## Package {package}", index); + } + + Assert.Contains("### `using Microsoft.Extensions.DependencyInjection;`", index, "The DI entry points live in Microsoft's namespace."); + + foreach (var type in new[] { "BsqliteDatabase", "BsqliteConnection", "IBsqliteStorageProvider", "BsqliteStorageSelectionBuilder", "IBsqliteDbContextFactory", "BsqliteInProcessTransport", "BitBsqlite" }) + { + Assert.Contains($"**{type}**", index); + } + + // Internal machinery stays out of the reference. + Assert.DoesNotContain("BsqlitePclProvider", index); + Assert.DoesNotContain("BsqliteSaveChangesGate", index); + Assert.DoesNotContain("BsqliteOpfsStorageAdmin", index); + } + + [TestMethod] + public async Task A_type_reference_names_its_namespace_and_package() + { + var reference = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "BsqliteOpfsStorageProvider" }); + + Assert.StartsWith("# BsqliteOpfsStorageProvider (Class)", reference); + Assert.Contains("package `Bit.Bsqlite.Storage.Opfs`", reference); + Assert.Contains("`using Bit.Bsqlite.Storage.Opfs;`", reference); + Assert.Contains("implements `IBsqliteStorageProvider`", reference); + Assert.Contains("**ProviderName** : `string` = `\"opfs\"`", reference, "A constant reads as its real value."); + Assert.Contains("## Constructor", reference); + Assert.Contains("(IJSRuntime js = null)", reference); + } + + [TestMethod] + public async Task Option_defaults_are_read_off_a_real_instance() + { + var options = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "BsqliteWorkerTransportOptions" }); + + Assert.Contains("**EnableForeignKeys** : `bool` = `true`", options); + } + + [TestMethod] + public async Task A_required_member_is_marked_and_given_no_made_up_default() + { + var reference = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "BsqliteStorageCapabilities" }); + + var build = reference.Split('\n').Single(line => line.StartsWith("- **RequiredBuild**", StringComparison.Ordinal)); + var contexts = reference.Split('\n').Single(line => line.StartsWith("- **SupportedContexts**", StringComparison.Ordinal)); + + Assert.Contains("**(required)**", build); + Assert.DoesNotContain(" = `", build, "A required member has no default: the compiler refuses to leave it out."); + Assert.Contains("= `BsqliteExecutionContexts.DedicatedWorker`", contexts); + } + + [TestMethod] + public async Task The_conformance_kit_lists_the_protected_member_a_test_class_overrides() + { + var reference = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "BsqliteStorageProviderConformanceTests" }); + + Assert.Contains("- protected **CreateProvider**() : `IBsqliteStorageProvider`", reference); + Assert.Contains("`using Bit.Bsqlite.Storage.ConformanceTests;`", reference); + } + + [TestMethod] + public async Task An_inheritdoc_member_carries_the_documentation_it_inherits() + { + var reference = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "BsqliteInMemoryStorageProvider" }); + + var nameLine = reference.Split('\n').Single(line => line.StartsWith("- **Name**", StringComparison.Ordinal)); + + Assert.Contains("Stable identifier", nameLine, "BsqliteInMemoryStorageProvider.Name says , so its summary is the interface's."); + } + + [TestMethod] + public async Task What_the_compiler_writes_for_a_record_is_left_out() + { + var reference = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "BsqliteStorageResolution" }); + + Assert.DoesNotContain("- **Equals**", reference); + Assert.DoesNotContain("- **GetHashCode**", reference); + Assert.DoesNotContain("- **ToString**", reference); + Assert.DoesNotContain("EqualityContract", reference); + Assert.DoesNotContain("", reference); + } + + [TestMethod] + public async Task A_generic_type_is_found_with_or_without_its_type_parameters() + { + var bare = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "IBsqliteDbContextFactory" }); + var closed = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "IBsqliteDbContextFactory" }); + + Assert.StartsWith("# IBsqliteDbContextFactory (Interface)", bare); + Assert.AreEqual(bare, closed); + } + + [TestMethod] + public async Task An_unprefixed_type_name_points_at_the_prefixed_one() + { + var answer = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "LiveQuery" }); + + Assert.Contains("BsqliteLiveQuery", answer); + } + + [TestMethod] + public async Task A_type_name_with_nothing_like_it_explains_the_prefix() + { + var answer = await McpCall.TextAsync("GetBsqliteApi", new() { ["typeName"] = "Zebra" }); + + Assert.Contains("prefixed 'Bsqlite'", answer); + } + + // ---- Storage backends -------------------------------------------------- + + [TestMethod] + public async Task The_backend_table_lists_every_first_party_backend() + { + var table = await McpCall.TextAsync("GetBsqliteStorageProviders"); + + foreach (var name in new[] { "opfs", "indexeddb", "cache-storage", "in-memory" }) Assert.Contains($"| `{name}` |", table); + } + + [TestMethod] + public async Task A_backend_comes_with_its_capabilities_modules_and_refused_statements() + { + var opfs = await McpCall.TextAsync("GetBsqliteStorageProviders", new() { ["provider"] = "opfs" }); + + Assert.StartsWith("# BsqliteOpfsStorageProvider (`opfs`)", opfs); + Assert.Contains("**RequiredBuild** = `Synchronous`", opfs); + Assert.Contains("**SupportsMultiDatabaseTransactions** = `false`", opfs); + Assert.Contains("`ATTACH DATABASE 'other.db' AS other` - refused: ATTACH is not available", opfs); + Assert.Contains("`PRAGMA page_size = 8192` - accepted", opfs); + Assert.Contains("bit-bsqlite-opfs-vfs.js", opfs); + Assert.Contains("`opfs-coop-sync`", opfs, "The VFS name is read off the provider's own constant."); + + var memory = await McpCall.TextAsync("GetBsqliteStorageProviders", new() { ["provider"] = "in-memory" }); + + Assert.Contains("VFS: none", memory); + Assert.Contains("**IsPersistent** = `false`", memory); + } + + [TestMethod] + public async Task A_backend_is_found_by_its_type_name_and_as_a_person_spells_it() + { + var byName = await McpCall.TextAsync("GetBsqliteStorageProviders", new() { ["provider"] = "indexeddb" }); + + Assert.AreEqual(byName, await McpCall.TextAsync("GetBsqliteStorageProviders", new() { ["provider"] = "BsqliteIndexedDbStorageProvider" })); + Assert.AreEqual(byName, await McpCall.TextAsync("GetBsqliteStorageProviders", new() { ["provider"] = "IndexedDB" })); + Assert.StartsWith("# BsqliteCacheStorageProvider", await McpCall.TextAsync("GetBsqliteStorageProviders", new() { ["provider"] = "Cache Storage" })); + } + + [TestMethod] + public async Task An_unknown_backend_lists_the_real_ones_and_the_way_to_write_one() + { + var answer = await McpCall.TextAsync("GetBsqliteStorageProviders", new() { ["provider"] = "localstorage" }); + + Assert.Contains("No first-party storage backend is called 'localstorage'", answer); + Assert.Contains("custom-provider", answer); + foreach (var provider in BsqliteStorageCatalog.Providers) Assert.Contains(provider.Name, answer); + } + + // ---- Setup guides ------------------------------------------------------ + + [TestMethod] + public async Task Every_scenario_gets_a_checklist_and_the_documentation_it_rests_on() + { + foreach (var scenario in BsqliteSetupGuide.Scenarios) + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = scenario }); + + Assert.StartsWith("# Setting Bsqlite up - ", guide, scenario); + Assert.Contains("## Checklist", guide, scenario); + Assert.Contains("## From the documentation", guide, scenario); + Assert.DoesNotContain("[truncated", guide, scenario); + + foreach (var tool in ToolNames.MentionedIn(guide)) CollectionAssert.Contains(McpSurfaceTests.ExpectedTools, tool, $"'{scenario}' mentions '{tool}', which is not a tool."); + } + } + + [TestMethod] + public async Task Every_section_and_file_a_guide_quotes_is_in_it() + { + // A guide leaves out a quote it cannot find, so a heading renamed or a file moved underneath it would + // only make the guide quietly shorter. Each one is looked for in the text a client receives. + foreach (var scenario in BsqliteSetupGuide.Scenarios) + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = scenario }); + var (quotes, files) = BsqliteSetupGuide.Sources(scenario); + + Assert.IsNotEmpty(quotes, scenario); + + foreach (var (document, section) in quotes) + { + Assert.IsNotNull(BsqliteSourceCatalog.GetDocumentSection(document, section), $"'{scenario}' quotes '{document} > {section}', which does not exist."); + Assert.Contains($"_{document}:_", guide, scenario); + } + + foreach (var file in files) + { + Assert.IsNotNull(BsqliteSourceCatalog.GetSourceFile(file), $"'{scenario}' quotes '{file}', which is not embedded."); + Assert.Contains($"### `{file}`", guide, scenario); + } + } + } + + [TestMethod] + public async Task The_hosting_guides_register_in_both_containers_and_open_only_when_interactive() + { + var wasm = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "wasm" }); + + Assert.Contains("RendererInfo.IsInteractive", wasm); + Assert.Contains("BOTH containers", wasm); + Assert.Contains("dotnet add package Bit.Bsqlite", wasm); + Assert.Contains("AddBitBsqliteServices", wasm); + Assert.Contains("BsqliteSynchronousApiNotSupportedException", wasm); + Assert.IsTrue(wasm.Split('\n').Any(line => line.TrimEnd() == "### Where it runs"), "The quoted README section should be demoted under 'From the documentation'."); + + var auto = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "auto" }); + + Assert.Contains("OperatingSystem.IsBrowser()", auto); + Assert.Contains("@rendermode InteractiveWebAssembly", auto); + } + + [TestMethod] + public async Task The_webassembly_guide_quotes_the_demo_files_verbatim() + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "wasm" }); + + Assert.Contains(BsqliteSourceCatalog.GetSourceFile("Demo/Client/Extensions/IServiceCollectionExtensions.cs")!.TrimStart('').TrimEnd(), guide); + Assert.Contains(BsqliteSourceCatalog.GetSourceFile("Demo/Client/Components/SessionLifecycle.razor")!.TrimStart('').TrimEnd(), guide); + Assert.Contains("### `Demo/Server/Program.cs`", guide); + } + + [TestMethod] + public async Task The_server_guide_says_it_is_not_supported() + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "server" }); + + Assert.Contains("not supported", guide); + Assert.Contains("InteractiveWebAssembly", guide); + } + + [TestMethod] + public async Task The_besql_guide_puts_cache_storage_first_under_besqls_own_file_name() + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "move-from-besql" }); + + Assert.Contains(".Prefer(BsqliteCacheStorageProvider.ProviderName)", guide); + Assert.Contains("AddDatabase(\"Offline-Client.db\")", guide); + Assert.Contains("/data/cache/", guide); + Assert.Contains("bit-besql.js", guide); + } + + [TestMethod] + public async Task The_storage_guide_tabulates_what_the_provider_types_declare() + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "storage" }); + + Assert.Contains("| `opfs` | `Bit.Bsqlite.Storage.Opfs` | Synchronous | true | false | refused | allowed |", guide); + Assert.Contains("| `indexeddb` | `Bit.Bsqlite.Storage.IndexedDb` | AsyncCapable | true | false | allowed | refused |", guide); + Assert.Contains("AllowNonPersistentFallback()", guide); + } + + [TestMethod] + public async Task The_provider_guide_lists_the_contract_read_off_the_interfaces() + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "custom-provider" }); + + foreach (var member in new[] { "Name", "Capabilities", "VfsModule", "Admin", "ProbeAsync", "ExistsAsync", "ListAsync", "DeleteAsync", "ExportAsync", "ImportAsync" }) + { + Assert.Contains($" {member}", guide, member); + } + + Assert.Contains("export async function register(", guide); + } + + [TestMethod] + public async Task The_testing_guide_uses_the_in_process_transport_and_the_kit() + { + var guide = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "testing" }); + + Assert.Contains("new BsqliteInProcessTransport()", guide); + Assert.Contains(": BsqliteStorageProviderConformanceTests", guide); + Assert.Contains("UseBindingStore()", guide); + } + + [TestMethod] + public async Task An_unknown_scenario_lists_the_known_ones() + { + // [AllowedValues] puts the values in the schema for a client to honour; a value that gets past it + // anyway is answered with the list rather than with an error. + var answer = await McpCall.TextAsync("GetBsqliteSetupGuide", new() { ["scenario"] = "electron" }); + + Assert.Contains("'electron' is not a known scenario", answer); + foreach (var scenario in BsqliteSetupGuide.Scenarios) Assert.Contains(scenario, answer); + } + + // ---- Source files ------------------------------------------------------ + + [TestMethod] + public async Task The_source_index_lists_the_demo_and_its_model_but_not_the_site_chrome() + { + var index = await McpCall.TextAsync("GetBsqliteSourceFile"); + + Assert.Contains("Demo/Client/Pages/Storage.razor", index); + Assert.Contains("Demo/Client/Components/SessionLifecycle.razor", index); + Assert.Contains("Demo/Data/AppDbContext.cs", index); + Assert.Contains("Demo/Server/Program.cs", index); + Assert.DoesNotContain("McpServer.razor", index); + Assert.DoesNotContain("NotFoundPage.razor", index); + Assert.DoesNotContain("ErrorPage.razor", index); + Assert.DoesNotContain("/Migrations/", index); + Assert.DoesNotContain("/obj/", index); + } + + [TestMethod] + public async Task A_source_file_comes_back_verbatim() + { + var file = await McpCall.TextAsync("GetBsqliteSourceFile", new() { ["path"] = "Demo/Client/Pages/Storage.razor" }); + + Assert.Contains("@page \"/storage\"", file); + Assert.Contains("BsqliteStorageResolution", file); + } + + [TestMethod] + public async Task A_partial_path_suggests_the_full_one() + { + var answer = await McpCall.TextAsync("GetBsqliteSourceFile", new() { ["path"] = "SessionLifecycle" }); + + Assert.Contains("Demo/Client/Components/SessionLifecycle.razor", answer); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/ToolNames.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/ToolNames.cs new file mode 100644 index 00000000000..ca79153bd01 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests.Mcp/ToolNames.cs @@ -0,0 +1,20 @@ +using System.Text.RegularExpressions; + +namespace Bit.Bsqlite.Tests.Mcp; + +/// +/// Finds the tool names inside a piece of the server's own prose - the instructions, a prompt, a +/// setup guide - so a test can hold that prose to the tools that actually exist. +/// +internal static partial class ToolNames +{ + public static IEnumerable MentionedIn(string text) + { + return ToolNameRegex().Matches(text).Select(match => match.Groups["name"].Value).Distinct(StringComparer.Ordinal); + } + + // Every tool on this server is named after what it does to Bsqlite - get, search, plan, check - which is + // what makes them findable in prose without also matching the Bsqlite... type names the same prose is full of. + [GeneratedRegex(@"\b(?(Get|Search|Plan|Check)Bsqlite[A-Za-z]*)\b")] + private static partial Regex ToolNameRegex(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/AssertSequence.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/AssertSequence.cs new file mode 100644 index 00000000000..2c77fc285e8 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/AssertSequence.cs @@ -0,0 +1,15 @@ +namespace Bit.Bsqlite; + +/// +/// Element-by-element equality of two sequences. MSTest's compares +/// a sequence as one object - two arrays with the same contents are not equal to it - so every assertion +/// about what a list holds goes through here. +/// +internal static class AssertSequence +{ + public static void Equal(IEnumerable expected, IEnumerable? actual) + { + Assert.IsNotNull(actual); + CollectionAssert.AreEqual(expected.ToArray(), actual.ToArray()); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Bit.Bsqlite.Tests.csproj b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Bit.Bsqlite.Tests.csproj new file mode 100644 index 00000000000..bb3034fa9a6 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Bit.Bsqlite.Tests.csproj @@ -0,0 +1,48 @@ + + + + net10.0;net9.0;net8.0 + enable + enable + false + true + $(NoWarn);CS1591 + true + Exe + false + + true + ..\..\..\AssemblyOriginatorKeyFile.snk + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ContextFactory.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ContextFactory.cs new file mode 100644 index 00000000000..58034c362f2 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ContextFactory.cs @@ -0,0 +1,24 @@ +using Bit.Bsqlite.Data; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Diagnostics; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +internal static class ContextFactory +{ + public static ProductContext Create(IBsqliteTransport transport, string databaseName = "ef.db") + => new(new DbContextOptionsBuilder() + .UseBitBsqlite(new BsqliteConnection(transport, databaseName)) +#if NET9_0_OR_GREATER + .ConfigureWarnings(w => w.Ignore(RelationalEventId.PendingModelChangesWarning)) +#endif + .Options); + + public static ProductContext CreateStock() + => new(new DbContextOptionsBuilder() + .UseSqlite("Data Source=:memory:") +#if NET9_0_OR_GREATER + .ConfigureWarnings(w => w.Ignore(RelationalEventId.PendingModelChangesWarning)) +#endif + .Options); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/DependencyInjectionTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/DependencyInjectionTests.cs new file mode 100644 index 00000000000..e2e970ac941 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/DependencyInjectionTests.cs @@ -0,0 +1,245 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.EntityFrameworkCore; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.DependencyInjection; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// The registration surface an application actually calls: AddBitBsqliteServices, +/// AddDatabase, and AddBitBsqliteDbContext. The worker transport is replaced by +/// the in-process one so the whole composition runs on desktop .NET. +/// +[TestClass] +public sealed class DependencyInjectionTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + private static ServiceProvider Build(Action>? configure = null) + { + var services = new ServiceCollection(); + services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .UseStorage(s => s.Prefer(BsqliteInMemoryStorageProvider.ProviderName)) + .UseBindingStore() + .AddDatabase("di.db")); + + // Desktop has no worker; the in-process engine stands in for it. + services.AddSingleton(new InProcessTransportFactory()); + + services.AddBitBsqliteDbContext(options => + { + options.Schema = BsqliteSchemaStrategy.EnsureCreated; +#if NET9_0_OR_GREATER + // EF Core 9 started refusing to migrate a model with pending changes; EF Core 8 has no such warning. + options.Configure = (_, builder) => builder.ConfigureWarnings(w => + w.Ignore(Microsoft.EntityFrameworkCore.Diagnostics.RelationalEventId.PendingModelChangesWarning)); +#endif + configure?.Invoke(options); + }); + + return services.BuildServiceProvider(); + } + + [TestMethod] + public async Task AddBitBsqliteServices_RegistersTheCompositionRoot() + { + await using var provider = Build(); + + Assert.IsNotNull(provider.GetRequiredService()); + Assert.IsNotNull(provider.GetRequiredService()); + Assert.IsNotNull(provider.GetRequiredService()); + Assert.AreEqual("di.db", provider.GetRequiredService().Name); + Assert.ContainsSingle(provider.GetServices()); + } + + [TestMethod] + public void AddBitBsqliteServices_WithoutAPreference_FailsAtRegistration() + { + var services = new ServiceCollection(); + + var error = Assert.ThrowsExactly(() => + services.AddBitBsqliteServices(sqlite => sqlite.AddStorageProvider())); + + Assert.Contains("UseStorage", error.Message); + } + + [TestMethod] + public async Task ContextFactory_OpensTheDatabase_PreparesTheSchemaOnce_AndHandsOutContexts() + { + var firstUse = 0; + await using var provider = Build(o => o.OnFirstUse = (_, _) => + { + firstUse++; + return Task.CompletedTask; + }); + + var factory = provider.GetRequiredService>(); + Assert.IsFalse(factory.Database.IsOpen); + + await using (var first = await factory.CreateDbContextAsync(Ct)) + { + first.Categories.Add(new Category { Name = "Tools" }); + await first.SaveChangesAsync(Ct); + } + + await using (var second = await factory.CreateDbContextAsync(Ct)) + { + Assert.AreEqual(1, await second.Categories.CountAsync(Ct)); + } + + Assert.IsTrue(factory.Database.IsOpen); + Assert.AreEqual(1, firstUse); + + // The generic EF factory interface resolves to the same instance. + Assert.AreSame(factory, provider.GetRequiredService>()); + } + + [TestMethod] + public async Task ContextFactory_SynchronousCreate_WorksOnceOpen_AndExplainsOtherwise() + { + await using var provider = Build(); + var factory = provider.GetRequiredService>(); + + var error = Assert.ThrowsExactly(() => factory.CreateDbContext()); + Assert.Contains("CreateDbContextAsync", error.Message); + + await using var _ = await factory.CreateDbContextAsync(Ct); + using var sync = factory.CreateDbContext(); + Assert.IsNotNull(sync); + } + + /// + /// A session opened some other way has not had its schema step run, so a synchronous context on + /// it would query tables that are not there yet - or a schema one deploy out of date. + /// + [TestMethod] + public async Task ContextFactory_SynchronousCreate_RefusesASessionItHasNotPrepared() + { + await using var provider = Build(); + var factory = provider.GetRequiredService>(); + await provider.GetRequiredService().OpenAsync(Ct); + + var error = Assert.ThrowsExactly(() => factory.CreateDbContext()); + Assert.Contains("CreateDbContextAsync", error.Message); + Assert.DoesNotContain("BsqliteDatabase.OpenAsync", error.Message); + } + + [TestMethod] + public async Task TwoDatabases_TheContextIsBoundToTheOneItNames() + { + await using var provider = BuildTwoDatabases(o => o.DatabaseName = "app.db"); + + var factory = provider.GetRequiredService>(); + + Assert.AreEqual("app.db", factory.Database.Name); + Assert.AreSame(provider.GetRequiredKeyedService("app.db"), factory.Database); + Assert.AreEqual("cache.db", provider.GetRequiredKeyedService("cache.db").Name); + } + + /// + /// The last registration would win, binding the context - and running its migrations - against + /// a database that is not its own. + /// + [TestMethod] + public async Task TwoDatabases_AContextThatNamesNeither_IsRefused() + { + await using var provider = BuildTwoDatabases(configure: null); + + var error = Assert.ThrowsExactly( + () => provider.GetRequiredService>()); + + Assert.Contains("DatabaseName", error.Message); + Assert.Contains("'cache.db'", error.Message); + } + + [TestMethod] + public void AddDatabase_TheSameNameTwice_IsRefused() + { + var services = new ServiceCollection(); + + var error = Assert.ThrowsExactly(() => services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .UseStorage(s => s.Prefer(BsqliteInMemoryStorageProvider.ProviderName)) + .AddDatabase("app.db") + .AddDatabase("app.db"))); + + Assert.Contains("already registered", error.Message); + } + + private static ServiceProvider BuildTwoDatabases(Action>? configure) + { + var services = new ServiceCollection(); + services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .UseStorage(s => s.Prefer(BsqliteInMemoryStorageProvider.ProviderName)) + .UseBindingStore() + .AddDatabase("app.db") + .AddDatabase("cache.db")); + services.AddSingleton(new InProcessTransportFactory()); + services.AddBitBsqliteDbContext(options => configure?.Invoke(options)); + return services.BuildServiceProvider(); + } + + [TestMethod] + public async Task Database_Close_RaisesClosing_AndReopenPreparesAgain() + { + var firstUse = 0; + await using var provider = Build(o => o.OnFirstUse = (_, _) => + { + firstUse++; + return Task.CompletedTask; + }); + + var database = provider.GetRequiredService(); + var factory = provider.GetRequiredService>(); + var closing = 0; + var opened = 0; + database.Closing += () => { closing++; return Task.CompletedTask; }; + database.Opened += () => { opened++; return Task.CompletedTask; }; + + await using (var _ = await factory.CreateDbContextAsync(Ct)) + { + } + + await database.CloseAsync(Ct); + Assert.IsFalse(database.IsOpen); + + await using (var _ = await factory.CreateDbContextAsync(Ct)) + { + } + + Assert.AreEqual(1, closing); + Assert.AreEqual(2, opened); + Assert.AreEqual(2, firstUse); + } + + [TestMethod] + public async Task Database_WhileClosed_RunsWorkWithTheSessionGone_ThenReopens() + { + await using var provider = Build(); + var database = provider.GetRequiredService(); + await database.OpenAsync(Ct); + + var sawClosed = false; + await database.WhileClosedAsync(_ => + { + sawClosed = !database.IsOpen; + return Task.CompletedTask; + }, Ct); + + Assert.IsTrue(sawClosed); + Assert.IsTrue(database.IsOpen); + } + + private sealed class InProcessTransportFactory : IBsqliteTransportFactory + { + public IBsqliteTransport Create(IBsqliteStorageProvider provider) => new BsqliteInProcessTransport(); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ExecutionTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ExecutionTests.cs new file mode 100644 index 00000000000..f5562ba69cf --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ExecutionTests.cs @@ -0,0 +1,190 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Testing; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// SQL generation never touches the connection. These tests drive real reads and writes through +/// UseBitBsqlite so identical SQL cannot be mistaken for a working provider. +/// +[TestClass] +public sealed class ExecutionTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task SaveChanges_And_Query_RoundTrip() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product + { + Name = "Widget", + Price = 19.99m, + CreatedUtc = new DateTime(2026, 3, 1, 12, 0, 0, DateTimeKind.Utc), + CategoryId = 1, + }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var loaded = await ctx.Products.Include(p => p.Category).SingleAsync(Ct); + + Assert.AreEqual("Widget", loaded.Name); + Assert.AreEqual(19.99m, loaded.Price); + Assert.AreEqual("Tools", loaded.Category!.Name); + Assert.AreEqual(new DateTime(2026, 3, 1, 12, 0, 0), loaded.CreatedUtc); + } + + [TestMethod] + public async Task SaveChanges_DateTime_SurvivesTheBrowserWireFormat() + { + await using var transport = new WireFormatAssertingTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product + { + Name = "Widget", + Price = 19.99m, + CreatedUtc = new DateTime(2026, 3, 1, 12, 0, 0, DateTimeKind.Utc), + CategoryId = 1, + }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var loaded = await ctx.Products.SingleAsync(Ct); + Assert.AreEqual(new DateTime(2026, 3, 1, 12, 0, 0), loaded.CreatedUtc); + } + + [TestMethod] + public async Task DecimalComparison_RunsThroughEfCompare() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.AddRange( + new Product { Name = "Cheap", Price = 5m, CategoryId = 1 }, + new Product { Name = "Pricey", Price = 500m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var query = ctx.Products.Where(p => p.Price > 10m); + + Assert.Contains("ef_compare", query.ToQueryString(), StringComparison.Ordinal); + AssertSequence.Equal(["Pricey"], await query.Select(p => p.Name).ToListAsync(Ct)); + } + +#if NET9_0_OR_GREATER + // EF Core translates SUM over a decimal from EF Core 9 on; before that the stock provider refuses it too. + [TestMethod] + public async Task DecimalSum_RunsThroughEfSum() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.AddRange( + new Product { Name = "A", Price = 1.10m, CategoryId = 1 }, + new Product { Name = "B", Price = 2.20m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var query = ctx.Products.GroupBy(p => p.CategoryId).Select(g => g.Sum(p => p.Price)); + + Assert.Contains("ef_sum", query.ToQueryString(), StringComparison.Ordinal); + Assert.AreEqual(3.30m, await query.SingleAsync(Ct)); + } +#endif + + [TestMethod] + public async Task WithoutEfFunctions_DecimalQueries_Fail() + { + await using var transport = new BsqliteInProcessTransport(registerEfFunctions: false); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "A", Price = 1.10m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var failure = await Assert.ThrowsExactlyAsync( + () => ctx.Products.Where(p => p.Price > 10m).ToListAsync(Ct)); + + Assert.Contains("no such function: ef_compare", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public async Task SynchronousDispose_DoesNotThrow_AndLeavesTransportUsable() + { + await using var transport = new BsqliteInProcessTransport(); + var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = 1m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + + ctx.Dispose(); + + await using var second = ContextFactory.Create(transport); + Assert.AreEqual(1, await second.Products.CountAsync(Ct)); + } + + [TestMethod] + public async Task Connection_SurvivesOpenCloseCycles() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + for (var i = 0; i < 3; i++) + { + await ctx.Database.OpenConnectionAsync(Ct); + await ctx.Database.CloseConnectionAsync(); + } + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = 1m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + + Assert.AreEqual(1, await ctx.Products.CountAsync(Ct)); + } + + /// + /// Desktop tests bind through Microsoft.Data.Sqlite, which converts DateTime itself. The browser + /// transport encodes with first, so this wrapper makes a leaking + /// DateTime fail here the same way it fails in WASM. + /// + private sealed class WireFormatAssertingTransport : IBsqliteTransport + { + private readonly BsqliteInProcessTransport _inner = new(); + + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + => _inner.OpenAsync(databaseName, cancellationToken); + + public Task CloseAsync(CancellationToken cancellationToken = default) + => _inner.CloseAsync(cancellationToken); + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + { + _ = BsqliteWireFormat.EncodeBatch(batch); + return _inner.ExecuteAsync(batch, cancellationToken); + } + + public ValueTask DisposeAsync() => _inner.DisposeAsync(); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/LiveQueryTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/LiveQueryTests.cs new file mode 100644 index 00000000000..d947b632f7d --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/LiveQueryTests.cs @@ -0,0 +1,176 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +[TestClass] +public sealed class LiveQueryTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task AsLiveQuery_RerunsAfterSaveChanges() + { + var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "live-ef.db"); + await connection.OpenAsync(Ct); + await using var ctx = new ProductContext(new DbContextOptionsBuilder() + .UseBitBsqlite(connection) + .Options); + + await ctx.Database.EnsureCreatedAsync(Ct); + ctx.Categories.Add(new Category { Name = "Tools" }); + await ctx.SaveChangesAsync(Ct); + + await using var live = ctx.Products.AsLiveQuery(connection); + Assert.IsEmpty(await live.RefreshAsync(Ct)); + + ctx.Products.Add(new Product { Name = "Live", Price = 1m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + + var deadline = DateTime.UtcNow.AddSeconds(2); + while ((live.Current?.Count ?? 0) == 0 && DateTime.UtcNow < deadline) + { + await Task.Delay(20, Ct); + } + + Assert.AreEqual("Live", Assert.ContainsSingle(live.Current!).Name); + } + + [TestMethod] + public async Task AsLiveQuery_RefreshWaitsForSaveChangesToFinish() + { + // The in-process transport raises TablesChanged before EF has consumed the insert result, and + // on the thread pool the refresh runs concurrently with the rest of SaveChanges. Without the + // save gate the re-read tracked the new row while EF was still assigning its key, and the + // *write* failed with "another instance with the same key value is already being tracked". + var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "live-ef-race.db"); + await connection.OpenAsync(Ct); + await using var ctx = new ProductContext(new DbContextOptionsBuilder() + .UseBitBsqlite(connection) + .Options); + + await ctx.Database.EnsureCreatedAsync(Ct); + ctx.Categories.Add(new Category { Name = "Tools" }); + await ctx.SaveChangesAsync(Ct); + + await using var live = ctx.Products.AsLiveQuery(connection); + await live.RefreshAsync(Ct); + + // Each write is followed by a wait for its refresh, so every refresh is checked - not to keep + // the two apart: the refresh runs on a context of its own. + const int writes = 50; + for (var i = 0; i < writes; i++) + { + ctx.Products.Add(new Product { Name = $"Item {i}", Price = i, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + + var deadline = DateTime.UtcNow.AddSeconds(5); + while ((live.Current?.Count ?? 0) < i + 1 && DateTime.UtcNow < deadline) + { + await Task.Delay(1, Ct); + } + + Assert.AreEqual(i + 1, live.Current!.Count); + } + } + + /// + /// A tracking query re-run on a context that already tracks its rows gets the tracked instances + /// back, old values and all. The refresh runs on a fresh context, so an update - here from raw + /// SQL, as another tab's would arrive - shows up, not just inserts and deletes. + /// + [TestMethod] + public async Task AsLiveQuery_TrackingQuery_SeesUpdatesToRowsItAlreadyLoaded() + { + var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "live-update.db"); + await connection.OpenAsync(Ct); + await using var ctx = new ProductContext(new DbContextOptionsBuilder() + .UseBitBsqlite(connection) + .Options); + + await ctx.Database.EnsureCreatedAsync(Ct); + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Before", Price = 1m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + + await using var live = ctx.Products.Include(p => p.Category).AsLiveQuery(); + Assert.AreEqual("Before", Assert.ContainsSingle(await live.RefreshAsync(Ct)).Name); + + await ctx.Database.ExecuteSqlRawAsync("UPDATE \"Products\" SET \"Name\" = 'After'", Ct); + + var deadline = DateTime.UtcNow.AddSeconds(2); + while (live.Current?.SingleOrDefault()?.Name != "After" && DateTime.UtcNow < deadline) + { + await Task.Delay(20, Ct); + } + + var row = Assert.ContainsSingle(live.Current!); + Assert.AreEqual("After", row.Name); + Assert.AreEqual("Tools", row.Category!.Name); + } + + /// + /// Refreshes arrive whenever another tab writes, so running them on the application's context + /// would collide with whatever the application starts there next. They run elsewhere, and the + /// application's context neither tracks their rows nor is kept busy by them. + /// + [TestMethod] + public async Task AsLiveQuery_RunsOutsideTheApplicationsContext() + { + var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "live-own-context.db"); + await connection.OpenAsync(Ct); + await using var ctx = new ProductContext(new DbContextOptionsBuilder() + .UseBitBsqlite(connection) + .Options); + + await ctx.Database.EnsureCreatedAsync(Ct); + ctx.Categories.Add(new Category { Name = "Tools" }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + await using var live = ctx.Categories.AsLiveQuery(); + Assert.ContainsSingle(await live.RefreshAsync(Ct)); + Assert.IsEmpty(ctx.ChangeTracker.Entries()); + + ctx.Categories.Add(new Category { Name = "Paint" }); + await ctx.SaveChangesAsync(Ct); + + var deadline = DateTime.UtcNow.AddSeconds(2); + while (live.Current?.Count != 2 && DateTime.UtcNow < deadline) + { + await Task.Delay(20, Ct); + } + + Assert.HasCount(2, live.Current!); + + // Only what the application added itself. + Assert.AreEqual("Paint", ((Category)Assert.ContainsSingle(ctx.ChangeTracker.Entries()).Entity).Name); + } + + [TestMethod] + public async Task AsLiveQuery_ResolvesConnectionFromComposedQuery() + { + // The sample calls OrderBy(...).AsLiveQuery() with no connection argument. DbSet exposes + // the context through IInfrastructure; EntityQueryable after OrderBy/Where does not. + var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "live-composed.db"); + await connection.OpenAsync(Ct); + await using var ctx = new ProductContext(new DbContextOptionsBuilder() + .UseBitBsqlite(connection) + .Options); + + await ctx.Database.EnsureCreatedAsync(Ct); + ctx.Categories.Add(new Category { Name = "Tools" }); + await ctx.SaveChangesAsync(Ct); + + await using var live = ctx.Products.OrderBy(p => p.Id).AsLiveQuery(); + Assert.IsEmpty(await live.RefreshAsync(Ct)); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/MigrationLockTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/MigrationLockTests.cs new file mode 100644 index 00000000000..357f06a4248 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/MigrationLockTests.cs @@ -0,0 +1,121 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.DependencyInjection; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// Every tab runs its own worker over the same file, so two tabs opening after a deploy race to apply +/// the same migration - and to seed the same empty database. Two in-process transports over one +/// shared-cache database stand in for the two tabs, their locks shared the way the worker's Web Locks +/// are. +/// +[TestClass] +public sealed class MigrationLockTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + private static string SharedDatabase() => $"Data Source=lock-{Guid.NewGuid():N};Mode=Memory;Cache=Shared"; + +#if NET9_0_OR_GREATER + [TestMethod] + public async Task MigrateAsync_WaitsWhileAnotherTabIsMigrating() + { + var database = SharedDatabase(); + await using var otherTab = new BsqliteInProcessTransport(database); + await otherTab.OpenAsync("app.db", Ct); + await using var thisTab = new BsqliteInProcessTransport(database); + await using var ctx = ContextFactory.Create(thisTab, "app.db"); + + var held = await otherTab.AcquireLockAsync(BsqliteHistoryRepository.MigrationLockName, Ct); + var migrating = ctx.Database.MigrateAsync(Ct); + + await Task.Delay(200, Ct); + Assert.IsFalse(migrating.IsCompleted, "The migration ran while another tab held the lock."); + + // Waiting must not hold SQLite's own write lock, or the tab it waits for could not finish. + Assert.DoesNotContain(sql => sql.StartsWith("BEGIN", StringComparison.Ordinal), thisTab.ExecutedCommands); + + await held.DisposeAsync(); + await migrating.WaitAsync(TimeSpan.FromSeconds(10), Ct); + + Assert.ContainsSingle(await ctx.Database.GetAppliedMigrationsAsync(Ct)); + } + + [TestMethod] + public async Task MigrateAsync_ReleasesTheLockForTheNextTab() + { + var database = SharedDatabase(); + await using var first = new BsqliteInProcessTransport(database); + await using var second = new BsqliteInProcessTransport(database); + + await using (var ctx = ContextFactory.Create(first, "app.db")) + { + await ctx.Database.MigrateAsync(Ct); + } + + await using (var ctx = ContextFactory.Create(second, "app.db")) + { + await ctx.Database.MigrateAsync(Ct).WaitAsync(TimeSpan.FromSeconds(10), Ct); + Assert.ContainsSingle(await ctx.Database.GetAppliedMigrationsAsync(Ct)); + } + } +#endif + + /// + /// The factory's first-use step - the schema step and the seed after it - runs under a lock of + /// its own, which is also what keeps two tabs on EF Core 8, whose migrator takes no lock, from + /// applying the same migration twice. + /// + [TestMethod] + public async Task ContextFactory_FirstUse_WaitsWhileAnotherTabPreparesTheDatabase() + { + var database = SharedDatabase(); + await using var otherTab = new BsqliteInProcessTransport(database); + await otherTab.OpenAsync("app.db", Ct); + + var seeded = 0; + var services = new ServiceCollection(); + services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .UseStorage(s => s.Prefer(BsqliteInMemoryStorageProvider.ProviderName)) + .UseBindingStore() + .AddDatabase("app.db")); + services.AddSingleton(new SharedTransportFactory(database)); + services.AddBitBsqliteDbContext(options => + { + options.Schema = BsqliteSchemaStrategy.EnsureCreated; + options.OnFirstUse = (_, _) => + { + seeded++; + return Task.CompletedTask; + }; + }); + + await using var provider = services.BuildServiceProvider(); + var factory = provider.GetRequiredService>(); + + var held = await otherTab.AcquireLockAsync(BsqliteDbContextFactory.FirstUseLockName, Ct); + var creating = factory.CreateDbContextAsync(Ct); + + await Task.Delay(200, Ct); + Assert.IsFalse(creating.IsCompleted, "The first-use step ran while another tab held the lock."); + Assert.AreEqual(0, seeded); + + await held.DisposeAsync(); + await using var ctx = await creating.WaitAsync(TimeSpan.FromSeconds(10), Ct); + + Assert.AreEqual(1, seeded); + } + + private sealed class SharedTransportFactory(string connectionString) : IBsqliteTransportFactory + { + public IBsqliteTransport Create(IBsqliteStorageProvider provider) => new BsqliteInProcessTransport(connectionString); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/Migrations/InitialCreate.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/Migrations/InitialCreate.cs new file mode 100644 index 00000000000..9bd52fa7834 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/Migrations/InitialCreate.cs @@ -0,0 +1,61 @@ +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Migrations; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests.Migrations; + +/// +/// Hand-written rather than scaffolded: dotnet ef would need a design-time factory able to +/// construct a transport. The suite only needs a migration that really applies. +/// +[DbContext(typeof(ProductContext))] +[Migration("20260101000000_InitialCreate")] +public sealed class InitialCreate : Migration +{ + protected override void Up(MigrationBuilder migrationBuilder) + { + migrationBuilder.CreateTable( + name: "Categories", + columns: table => new + { + Id = table.Column(type: "INTEGER", nullable: false) + .Annotation("Sqlite:Autoincrement", true), + Name = table.Column(type: "TEXT", maxLength: 100, nullable: false), + }, + constraints: table => table.PrimaryKey("PK_Categories", x => x.Id)); + + migrationBuilder.CreateTable( + name: "Products", + columns: table => new + { + Id = table.Column(type: "INTEGER", nullable: false) + .Annotation("Sqlite:Autoincrement", true), + Name = table.Column(type: "TEXT", maxLength: 200, nullable: false), + Price = table.Column(type: "TEXT", precision: 18, scale: 2, nullable: false), + CreatedUtc = table.Column(type: "TEXT", nullable: false), + CategoryId = table.Column(type: "INTEGER", nullable: false), + }, + constraints: table => + { + table.PrimaryKey("PK_Products", x => x.Id); + table.ForeignKey( + name: "FK_Products_Categories_CategoryId", + column: x => x.CategoryId, + principalTable: "Categories", + principalColumn: "Id", + onDelete: ReferentialAction.Cascade); + }); + + migrationBuilder.CreateIndex(name: "IX_Products_Name", table: "Products", column: "Name"); + + migrationBuilder.CreateIndex( + name: "IX_Products_CategoryId", + table: "Products", + column: "CategoryId"); + } + + protected override void Down(MigrationBuilder migrationBuilder) + { + migrationBuilder.DropTable(name: "Products"); + migrationBuilder.DropTable(name: "Categories"); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ProductContext.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ProductContext.cs new file mode 100644 index 00000000000..46e25da535e --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/ProductContext.cs @@ -0,0 +1,56 @@ +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// Exercises decimal, navigations, and indexes - the mappings that depend on the UDF host. +public sealed class ProductContext(DbContextOptions options) : DbContext(options) +{ + public DbSet Categories => Set(); + + public DbSet Products => Set(); + + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + modelBuilder.Entity(category => + { + category.HasKey(c => c.Id); + category.Property(c => c.Name).IsRequired().HasMaxLength(100); + }); + + modelBuilder.Entity(product => + { + product.HasKey(p => p.Id); + product.Property(p => p.Name).IsRequired().HasMaxLength(200); + product.Property(p => p.Price).HasPrecision(18, 2); + product.HasIndex(p => p.Name); + product.HasOne(p => p.Category) + .WithMany(c => c.Products) + .HasForeignKey(p => p.CategoryId) + .OnDelete(DeleteBehavior.Cascade); + }); + } +} + +public sealed class Category +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Products { get; } = []; +} + +public sealed class Product +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public decimal Price { get; set; } + + public DateTime CreatedUtc { get; set; } + + public int CategoryId { get; set; } + + public Category? Category { get; set; } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SchemaTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SchemaTests.cs new file mode 100644 index 00000000000..569daa6fb64 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SchemaTests.cs @@ -0,0 +1,135 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// The schema paths the S1 audit found blocked - now the exit criterion for UseBitBsqlite. +/// +[TestClass] +public sealed class SchemaTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task EnsureCreatedAsync_CreatesAUsableSchema() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + + Assert.IsTrue(await ctx.Database.EnsureCreatedAsync(Ct)); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = 9.99m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var loaded = await ctx.Products.Include(p => p.Category).SingleAsync(Ct); + Assert.AreEqual("Tools", loaded.Category!.Name); + } + + [TestMethod] + public async Task EnsureCreatedAsync_IsIdempotent() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + + Assert.IsTrue(await ctx.Database.EnsureCreatedAsync(Ct)); + Assert.IsFalse(await ctx.Database.EnsureCreatedAsync(Ct)); + } + + [TestMethod] + public async Task MigrateAsync_AppliesMigrations_AndRecordsHistory() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + + AssertSequence.Equal(["20260101000000_InitialCreate"], await ctx.Database.GetPendingMigrationsAsync(Ct)); + + await ctx.Database.MigrateAsync(Ct); + + AssertSequence.Equal(["20260101000000_InitialCreate"], await ctx.Database.GetAppliedMigrationsAsync(Ct)); + Assert.IsEmpty(await ctx.Database.GetPendingMigrationsAsync(Ct)); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = 12.34m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var loaded = await ctx.Products.SingleAsync(Ct); + Assert.AreEqual(12.34m, loaded.Price); + } + + [TestMethod] + public async Task MigrateAsync_IsIdempotent() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + + await ctx.Database.MigrateAsync(Ct); + await ctx.Database.MigrateAsync(Ct); + + Assert.IsEmpty(await ctx.Database.GetPendingMigrationsAsync(Ct)); + } + + [TestMethod] + public async Task EnsureDeletedAsync_DropsUserTables_NotSqliteBookkeeping() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + var deleted = await ctx.Database.EnsureDeletedAsync(Ct); + + Assert.IsTrue(deleted); + Assert.IsTrue(await ctx.Database.EnsureCreatedAsync(Ct)); + } + + /// + /// With foreign keys on, DROP TABLE runs an implicit DELETE that a referencing table can veto, + /// and sqlite_master lists parents first. A view left behind would break the next + /// EnsureCreated. Neither may stop the delete, and enforcement has to be back on afterwards + /// because the connection lives on. + /// + [TestMethod] + public async Task EnsureDeletedAsync_CopesWithReferencedRowsAndViews_AndLeavesForeignKeysOn() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + await ctx.Database.ExecuteSqlRawAsync("CREATE TABLE parent (id INTEGER PRIMARY KEY)", Ct); + await ctx.Database.ExecuteSqlRawAsync( + "CREATE TABLE child (id INTEGER PRIMARY KEY, parent_id INTEGER REFERENCES parent(id))", Ct); + await ctx.Database.ExecuteSqlRawAsync("INSERT INTO parent (id) VALUES (1)", Ct); + await ctx.Database.ExecuteSqlRawAsync("INSERT INTO child (id, parent_id) VALUES (1, 1)", Ct); + await ctx.Database.ExecuteSqlRawAsync("CREATE VIEW parents AS SELECT id FROM parent", Ct); + + Assert.IsTrue(await ctx.Database.EnsureDeletedAsync(Ct)); + + Assert.IsTrue(await ctx.Database.EnsureCreatedAsync(Ct)); + Assert.AreEqual(0L, await ScalarAsync(ctx, "SELECT COUNT(*) FROM sqlite_master WHERE type = 'view'")); + Assert.AreEqual(1L, await ScalarAsync(ctx, "PRAGMA foreign_keys")); + } + + private async Task ScalarAsync(DbContext ctx, string sql) + { + var connection = ctx.Database.GetDbConnection(); + await using var command = connection.CreateCommand(); + command.CommandText = sql; + return Convert.ToInt64(await command.ExecuteScalarAsync(Ct)); + } + + [TestMethod] + public async Task SynchronousSchemaApis_StillThrow() + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport); + + Assert.ThrowsExactly( + () => ctx.Database.EnsureCreated()); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SessionCompositionTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SessionCompositionTests.cs new file mode 100644 index 00000000000..dfcdc12489a --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SessionCompositionTests.cs @@ -0,0 +1,57 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// The composition UseBitBsqlite is documented as: resolve and open through +/// , then hand the session's connection to EF. +/// +[TestClass] +public sealed class SessionCompositionTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task SessionFactory_ThenUseBitBsqlite_MigratesAndQueries() + { + var provider = new BsqliteInMemoryStorageProvider(); + var store = new BsqliteInMemoryStorageBindingStore(); + var factory = new BsqliteSessionFactory( + new BsqliteStorageProviderResolver([provider], store), + new InProcessTransportFactory(), + BsqliteStorageSelectionBuilder.Create(s => s.Prefer(BsqliteInMemoryStorageProvider.ProviderName)), + store); + + await using var session = await factory.OpenAsync("app.db", Ct); + await using var ctx = new ProductContext(new DbContextOptionsBuilder() + .UseBitBsqlite(session.Connection) + .Options); + + await ctx.Database.MigrateAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = 4.50m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var loaded = await ctx.Products.Include(p => p.Category).SingleAsync(Ct); + Assert.AreEqual("Widget", loaded.Name); + Assert.AreEqual("Tools", loaded.Category!.Name); + + // In-memory holds nothing across a reload, so there is nothing for a binding to protect. + Assert.IsNull(await store.GetProviderNameAsync("app.db", Ct)); + } + + private sealed class InProcessTransportFactory : IBsqliteTransportFactory + { + public IBsqliteTransport Create(IBsqliteStorageProvider provider) + => new BsqliteInProcessTransport(); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SharedSampleModelTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SharedSampleModelTests.cs new file mode 100644 index 00000000000..18cf609c899 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SharedSampleModelTests.cs @@ -0,0 +1,51 @@ +using Bit.Bsqlite.Demo.Data; +using Bit.Bsqlite.Demo.Domain; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// The sample's load-bearing claim: one migration set applies through UseBitBsqlite. +/// +[TestClass] +public sealed class SharedSampleModelTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task SampleMigrations_ApplyAndRoundTrip() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "sample.db"); + await connection.OpenAsync(Ct); + await using var ctx = new AppDbContext(new DbContextOptionsBuilder() + .UseBitBsqlite(connection) + .Options); + + await ctx.Database.MigrateAsync(Ct); + await DemoData.SeedIfEmptyAsync(ctx, Ct); + + var product = await ctx.Products.Include(p => p.Category).SingleAsync(p => p.Sku == "TL-100", Ct); + Assert.AreEqual("Digital caliper", product.Name); + Assert.AreEqual(42.50m, product.Price); + Assert.AreEqual(TimeSpan.FromDays(2), product.LeadTime); + Assert.AreEqual("Tools", product.Category!.Name); + + var customer = await ctx.Customers.SingleAsync(c => c.Email == "stores@northwind.example", Ct); + Assert.AreEqual(new DateOnly(1978, 4, 12), customer.DateOfBirth); + Assert.IsTrue(customer.IsVip); + Assert.AreEqual(25_000m, customer.CreditLimit); + + var order = await ctx.Orders + .Include(o => o.Lines) + .ThenInclude(l => l.Product) + .SingleAsync(o => o.Number == "SO-1042", Ct); + Assert.AreEqual(OrderStatus.Shipped, order.Status); + Assert.AreEqual(2, order.Lines.Count); + Assert.Contains(l => l.Product!.Sku == "TL-100", order.Lines); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SqlEquivalenceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SqlEquivalenceTests.cs new file mode 100644 index 00000000000..74fc059abbd --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SqlEquivalenceTests.cs @@ -0,0 +1,57 @@ +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// SQL generation never inspects the connection type, so UseBitBsqlite must produce +/// SQL byte-identical to the stock provider. That is the §2 "same model both sides" guardrail. +/// +[TestClass] +public sealed class SqlEquivalenceTests +{ + public static IEnumerable<(string Name, Func> Query)> Queries => + [ + ("simple filter", ctx => ctx.Products.Where(p => p.Name == "widget")), + ("decimal comparison", ctx => ctx.Products.Where(p => p.Price > 10m)), + ("decimal arithmetic", ctx => ctx.Products.Where(p => p.Price * 2m < 100m)), +#if NET10_0_OR_GREATER + // EF Core translates ORDER BY a decimal from EF Core 10 on; before that the stock provider refuses it too. + ("order by decimal", ctx => ctx.Products.OrderBy(p => p.Price)), +#endif + ("include navigation", ctx => ctx.Products.Include(p => p.Category)), +#if NET9_0_OR_GREATER + // EF Core translates SUM over a decimal from EF Core 9 on; before that the stock provider refuses it too. + ( + "group by with aggregate", + ctx => ctx.Products + .GroupBy(p => p.CategoryId) + .Select(g => new { g.Key, Total = g.Sum(p => p.Price), Count = g.Count() }) + ), +#endif + ( + "projection with join", + ctx => ctx.Products.Select(p => new { p.Name, Category = p.Category!.Name }) + ), + ("skip take", ctx => ctx.Products.OrderBy(p => p.Id).Skip(10).Take(5)), + ("string contains", ctx => ctx.Products.Where(p => p.Name.Contains("wid"))), + ("date filter", ctx => ctx.Products.Where(p => p.CreatedUtc > new DateTime(2026, 1, 1))), + ]; + + [TestMethod] + [DynamicData(nameof(Queries))] + public async Task GeneratedSql_IsIdenticalToStockProvider( + string name, + Func> query) + { + await using var transport = new BsqliteInProcessTransport(); + using var stock = ContextFactory.CreateStock(); + using var ours = ContextFactory.Create(transport); + + var expected = query(stock).ToQueryString(); + var actual = query(ours).ToQueryString(); + + Assert.AreEqual(expected, actual); + Assert.IsFalse(string.IsNullOrWhiteSpace(actual), $"Query '{name}' produced no SQL."); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SyncSurfaceAudit.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SyncSurfaceAudit.cs new file mode 100644 index 00000000000..96660e3f876 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/SyncSurfaceAudit.cs @@ -0,0 +1,243 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// The S1 audit, now a regression suite: every entry point an application actually calls is marked +/// usable or blocked, so a new sync-only EF path shows up as a failing expectation naming the frame. +/// +[TestClass] +public sealed class SyncSurfaceAudit +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + public enum Expectation + { + Works, + SyncBound, + } + + private async Task CreateSeededAsync(BsqliteInProcessTransport transport) + { + var ctx = ContextFactory.Create(transport, "audit.db"); + await ctx.Database.EnsureCreatedAsync(Ct); + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = 9.99m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + return ctx; + } + + public static IEnumerable<(string Name, Expectation Expected, Func Operation)> DataOperations => + [ + ("ToListAsync", Expectation.Works, (ctx, ct) => ctx.Products.ToListAsync(ct)), + ("FirstOrDefaultAsync", Expectation.Works, (ctx, ct) => ctx.Products.FirstOrDefaultAsync(ct)), + ("CountAsync", Expectation.Works, (ctx, ct) => ctx.Products.CountAsync(ct)), + ("AnyAsync", Expectation.Works, (ctx, ct) => ctx.Products.AnyAsync(ct)), + ("Include + ToListAsync", Expectation.Works, (ctx, ct) => ctx.Products.Include(p => p.Category).ToListAsync(ct)), + ( + "AsAsyncEnumerable", + Expectation.Works, + async (ctx, ct) => + { + await foreach (var _ in ctx.Products.AsAsyncEnumerable().WithCancellation(ct)) + { + } + } + ), + ( + "SaveChangesAsync (insert)", + Expectation.Works, + (ctx, ct) => + { + ctx.Products.Add(new Product { Name = "New", Price = 1m, CategoryId = 1 }); + return ctx.SaveChangesAsync(ct); + } + ), + ( + "SaveChangesAsync (update)", + Expectation.Works, + async (ctx, ct) => + { + var product = await ctx.Products.FirstAsync(ct); + product.Name = "Renamed"; + await ctx.SaveChangesAsync(ct); + } + ), + ( + "SaveChangesAsync (delete)", + Expectation.Works, + async (ctx, ct) => + { + ctx.Products.Remove(await ctx.Products.FirstAsync(ct)); + await ctx.SaveChangesAsync(ct); + } + ), + ( + "ExecuteUpdateAsync", + Expectation.Works, + (ctx, ct) => ctx.Products.ExecuteUpdateAsync(s => s.SetProperty(p => p.Name, "Bulk"), ct) + ), + ("ExecuteDeleteAsync", Expectation.Works, (ctx, ct) => ctx.Products.ExecuteDeleteAsync(ct)), + ( + "FromSqlRaw + ToListAsync", + Expectation.Works, + (ctx, ct) => ctx.Products.FromSqlRaw("SELECT * FROM Products").ToListAsync(ct) + ), + ( + "ExecuteSqlRawAsync", + Expectation.Works, + (ctx, ct) => ctx.Database.ExecuteSqlRawAsync("UPDATE Products SET Name = 'raw'", ct) + ), + ("CanConnectAsync", Expectation.Works, (ctx, ct) => ctx.Database.CanConnectAsync(ct)), + ( + "OpenConnectionAsync + CloseConnectionAsync", + Expectation.Works, + async (ctx, ct) => + { + await ctx.Database.OpenConnectionAsync(ct); + await ctx.Database.CloseConnectionAsync(); + } + ), + ( + "BeginTransactionAsync + CommitAsync", + Expectation.Works, + async (ctx, ct) => + { + await using var tx = await ctx.Database.BeginTransactionAsync(ct); + ctx.Products.Add(new Product { Name = "Tx", Price = 1m, CategoryId = 1 }); + await ctx.SaveChangesAsync(ct); + await tx.CommitAsync(ct); + } + ), + ( + "BeginTransactionAsync + RollbackAsync", + Expectation.Works, + async (ctx, ct) => + { + await using var tx = await ctx.Database.BeginTransactionAsync(ct); + ctx.Products.Add(new Product { Name = "Tx", Price = 1m, CategoryId = 1 }); + await ctx.SaveChangesAsync(ct); + await tx.RollbackAsync(ct); + } + ), + ( + "Entry.ReloadAsync", + Expectation.Works, + async (ctx, ct) => + { + var product = await ctx.Products.FirstAsync(ct); + await ctx.Entry(product).ReloadAsync(ct); + } + ), + ( + "Collection.LoadAsync", + Expectation.Works, + async (ctx, ct) => + { + var category = await ctx.Categories.FirstAsync(ct); + await ctx.Entry(category).Collection(c => c.Products).LoadAsync(ct); + } + ), + ( + "GetAppliedMigrationsAsync", + Expectation.Works, + (ctx, ct) => ctx.Database.GetAppliedMigrationsAsync(ct) + ), + ( + "GetPendingMigrationsAsync", + Expectation.Works, + (ctx, ct) => ctx.Database.GetPendingMigrationsAsync(ct) + ), + ("ToList (sync)", Expectation.SyncBound, (ctx, ct) => Task.FromResult(ctx.Products.ToList())), + ("Count (sync)", Expectation.SyncBound, (ctx, ct) => Task.FromResult(ctx.Products.Count())), + ( + "SaveChanges (sync)", + Expectation.SyncBound, + (ctx, ct) => + { + ctx.Products.Add(new Product { Name = "Sync", Price = 1m, CategoryId = 1 }); + return Task.FromResult(ctx.SaveChanges()); + } + ), + ]; + + public static IEnumerable<(string Name, Expectation Expected, Func Operation)> SchemaOperations => + [ + ("EnsureCreatedAsync", Expectation.Works, (ctx, ct) => ctx.Database.EnsureCreatedAsync(ct)), + ("EnsureDeletedAsync", Expectation.Works, (ctx, ct) => ctx.Database.EnsureDeletedAsync(ct)), + ("MigrateAsync", Expectation.Works, (ctx, ct) => ctx.Database.MigrateAsync(ct)), + ("CanConnectAsync (virgin database)", Expectation.Works, (ctx, ct) => ctx.Database.CanConnectAsync(ct)), + ("EnsureCreated (sync)", Expectation.SyncBound, (ctx, ct) => Task.FromResult(ctx.Database.EnsureCreated())), + ]; + + [TestMethod] + [DynamicData(nameof(DataOperations))] + public async Task DataOperation_MatchesExpectation( + string name, + Expectation expected, + Func operation) + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = await CreateSeededAsync(transport); + await AssertExpectationAsync(name, expected, ctx, operation); + } + + [TestMethod] + [DynamicData(nameof(SchemaOperations))] + public async Task SchemaOperation_MatchesExpectation( + string name, + Expectation expected, + Func operation) + { + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = ContextFactory.Create(transport, "audit-schema.db"); + await AssertExpectationAsync(name, expected, ctx, operation); + } + + private async Task AssertExpectationAsync( + string name, + Expectation expected, + ProductContext ctx, + Func operation) + { + Exception? failure = null; + try + { + await operation(ctx, Ct); + } + catch (Exception exception) + { + failure = exception; + } + + if (expected == Expectation.SyncBound) + { + Assert.IsInstanceOfType(failure); + + TestContext.WriteLine($"{name} is sync-bound via:"); + foreach (var frame in EfFrames(failure!)) + { + TestContext.WriteLine($" {frame}"); + } + + return; + } + + if (failure is not null) + { + Assert.Fail($"'{name}' was expected to work on the async tier but threw:{Environment.NewLine}{failure}"); + } + } + + private static IEnumerable EfFrames(Exception failure) + => (failure.StackTrace ?? string.Empty) + .Split(Environment.NewLine, StringSplitOptions.RemoveEmptyEntries) + .Select(line => line.Trim()) + .Where(line => line.Contains("Microsoft.EntityFrameworkCore", StringComparison.Ordinal)) + .Take(4); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/UseBitBsqliteTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/UseBitBsqliteTests.cs new file mode 100644 index 00000000000..63377aa9f95 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/UseBitBsqliteTests.cs @@ -0,0 +1,63 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Migrations; +using Microsoft.EntityFrameworkCore.Storage; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// UseBitBsqlite is the stock SQLite provider plus two service replacements. These tests +/// pin that shape so a future change cannot quietly become a custom provider. +/// +[TestClass] +public sealed class UseBitBsqliteTests +{ + public TestContext TestContext { get; set; } = null!; + + [TestMethod] + public void ProviderName_IsStockSqlite() + { + using var stock = ContextFactory.CreateStock(); + using var ours = ContextFactory.Create(new BsqliteInProcessTransport()); + + Assert.AreEqual("Microsoft.EntityFrameworkCore.Sqlite", stock.Database.ProviderName); + Assert.AreEqual(stock.Database.ProviderName, ours.Database.ProviderName); + } + + [TestMethod] + public void ReplacesTheTwoSyncBoundServices() + { + using var ctx = ContextFactory.Create(new BsqliteInProcessTransport()); + + Assert.IsInstanceOfType(ctx.GetService()); + Assert.IsInstanceOfType(ctx.GetService()); + } + + [TestMethod] + public async Task StockUseSqlite_EnsureCreatedAsync_StillThrows() + { + // The reason the veneer exists: UseSqlite(connection) alone reaches Open() through + // RelationalDatabaseCreator.CreateAsync. If that ever becomes async in EF, this test + // fails and the replacements can be re-evaluated. + await using var transport = new BsqliteInProcessTransport(); + await using var ctx = new ProductContext(new DbContextOptionsBuilder() + .UseSqlite(new BsqliteConnection(transport, "stock.db")) + .Options); + + var failure = await Assert.ThrowsExactlyAsync( + () => ctx.Database.EnsureCreatedAsync(TestContext.CancellationToken)); + + Assert.Contains("Open", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public void StockSqliteConnection_ServerVersion_IsAtLeastJsonFunctions() + { + // EF Core 10's query translator does this before compiling any query. On WASM there is + // no e_sqlite3; UseBitBsqlite installs a PCL stub so the call does not throw. + var version = new Version(new Microsoft.Data.Sqlite.SqliteConnection().ServerVersion); + Assert.IsTrue(version >= new Version(3, 38)); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/WireFormatRoundTripTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/WireFormatRoundTripTests.cs new file mode 100644 index 00000000000..2d9f5cc3d08 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/WireFormatRoundTripTests.cs @@ -0,0 +1,165 @@ +using Microsoft.EntityFrameworkCore; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// EF over the real wire format, which is the combination the browser runs and the one the rest of +/// the desktop suite skips past. +/// +[TestClass] +public sealed class WireFormatRoundTripTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + /// + /// The bug this exists to prevent: the wire format wrote 10 where the rest of the SQLite + /// stack writes 10.0, EF compiled == 10m to WHERE "Price" = '10.0', and a + /// row the application had just saved could not be found again. Nothing threw. + /// + /// + /// The comparisons are written as literals on purpose. A captured variable is parameterised, so + /// both sides of the comparison go through the same encoder and agree even when that encoder is + /// wrong - which is exactly how this defect stayed invisible. An inlined constant is rendered by + /// EF's own type mapping, and only matches if what was stored agrees with it. + /// + [TestMethod] + public async Task ADecimalWrittenThroughTheWire_IsFoundByAConstantComparison() + { + await using var transport = new WireLoopbackTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + foreach (var price in new[] { 10m, 1.50m, 0m, 12.34m, 1234.5678m }) + { + ctx.Products.Add(new Product { Name = $"P{price}", Price = price, CategoryId = 1 }); + } + + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + Assert.AreEqual(1, await ctx.Products.CountAsync(p => p.Price == 10m, Ct)); + Assert.AreEqual(1, await ctx.Products.CountAsync(p => p.Price == 1.50m, Ct)); + Assert.AreEqual(1, await ctx.Products.CountAsync(p => p.Price == 0m, Ct)); + Assert.AreEqual(1, await ctx.Products.CountAsync(p => p.Price == 12.34m, Ct)); + Assert.AreEqual(1, await ctx.Products.CountAsync(p => p.Price == 1234.5678m, Ct)); + } + + /// + /// The same row, read back as a CLR value rather than matched in SQL. + /// + [TestMethod] + [DataRow("10")] + [DataRow("1.50")] + [DataRow("0")] + [DataRow("12.34")] + [DataRow("-3")] + [DataRow("1234.5678")] + public async Task ADecimalWrittenThroughTheWire_MaterializesBackToItself(string literal) + { + var price = decimal.Parse(literal, System.Globalization.CultureInfo.InvariantCulture); + + await using var transport = new WireLoopbackTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = price, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + Assert.AreEqual(price, (await ctx.Products.SingleAsync(Ct)).Price); + } + + /// + /// The stored text is compared against the stock provider's, because "both halves agree" is the + /// whole claim - a database written in the browser has to be one the server can read. + /// + [TestMethod] + public async Task TheStoredTextMatchesWhatTheStockProviderWrites() + { + await using var transport = new WireLoopbackTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product { Name = "Widget", Price = 10m, CategoryId = 1 }); + await ctx.SaveChangesAsync(Ct); + + var stored = await ctx.Database + .SqlQueryRaw("SELECT CAST(\"Price\" AS TEXT) AS \"Value\" FROM \"Products\"") + .SingleAsync(Ct); + + Assert.AreEqual("10.0", stored); + } + + /// + /// Ordering and comparison go through ef_compare and the EF_DECIMAL collation, so + /// they have to keep working whatever the stored text looks like. + /// +#if NET10_0_OR_GREATER + // EF Core translates ORDER BY a decimal from EF Core 10 on; before that the stock provider refuses it too. + [TestMethod] + public async Task DecimalsWrittenThroughTheWire_StillCompareNumerically() + { + await using var transport = new WireLoopbackTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + foreach (var price in new[] { 9m, 10m, 2.5m, 100m, -0.25m }) + { + ctx.Products.Add(new Product { Name = $"P{price}", Price = price, CategoryId = 1 }); + } + + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + AssertSequence.Equal( + [-0.25m, 2.5m, 9m, 10m, 100m], + await ctx.Products.OrderBy(p => p.Price).Select(p => p.Price).ToListAsync(Ct)); + + AssertSequence.Equal( + [10m, 100m], + await ctx.Products + .Where(p => p.Price > 9m) + .OrderBy(p => p.Price) + .Select(p => p.Price) + .ToListAsync(Ct)); + } +#endif + + /// + /// Every other type the binder converts, checked through the same path - a regression in any of + /// them is the same silent class of failure the decimal one was. + /// + [TestMethod] + public async Task NonDecimalValues_SurviveTheWireUnchanged() + { + var created = new DateTime(2026, 8, 17, 9, 30, 15, DateTimeKind.Utc); + + await using var transport = new WireLoopbackTransport(); + await using var ctx = ContextFactory.Create(transport); + await ctx.Database.EnsureCreatedAsync(Ct); + + ctx.Categories.Add(new Category { Name = "Tools" }); + ctx.Products.Add(new Product + { + Name = "Widget", + Price = 1m, + CreatedUtc = created, + CategoryId = 1, + }); + await ctx.SaveChangesAsync(Ct); + ctx.ChangeTracker.Clear(); + + var loaded = await ctx.Products.SingleAsync(Ct); + + Assert.AreEqual("Widget", loaded.Name); + Assert.AreEqual(created, loaded.CreatedUtc); + Assert.AreEqual(1, loaded.CategoryId); + Assert.AreEqual(1, await ctx.Products.CountAsync(p => p.CreatedUtc == created, Ct)); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/WireLoopbackTransport.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/WireLoopbackTransport.cs new file mode 100644 index 00000000000..165db129813 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/EntityFrameworkCore/WireLoopbackTransport.cs @@ -0,0 +1,71 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Testing; + +namespace Bit.Bsqlite.EntityFrameworkCore.Tests; + +/// +/// An that pushes every parameter through the real wire format +/// before executing it in-process. +/// +/// +/// +/// hands CLR values straight to Microsoft.Data.Sqlite, which +/// applies its own conversions. That is the right thing for testing SQL generation, but it means +/// the desktop suite never sees the encoding the browser actually stores - so a value the worker +/// would write differently from the server looks fine in every test and is wrong only in +/// production. +/// +/// +/// This closes that gap by doing what the worker does: encode with +/// , take the one representation the binary format carries for +/// each value, and bind that. What EF stores through this transport is byte-for-byte what it would +/// store through a worker. +/// +/// +internal sealed class WireLoopbackTransport : IBsqliteTransport +{ + private readonly BsqliteInProcessTransport _inner = new(); + + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + => _inner.OpenAsync(databaseName, cancellationToken); + + public Task CloseAsync(CancellationToken cancellationToken = default) + => _inner.CloseAsync(cancellationToken); + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + { + // The whole batch goes through the encoder, so anything it rejects is rejected here too. + _ = BsqliteWireFormat.EncodeBatch(batch); + + var rebound = new BsqliteCommandRequest[batch.Count]; + for (var i = 0; i < batch.Count; i++) + { + rebound[i] = batch[i] with + { + Parameters = [.. batch[i].Parameters.Select(p => + { + var (type, value) = BsqliteWireFormat.EncodeValue(p.Value, p.Name); + return new BsqliteParameterValue(p.Name, Decode(type, value)); + })], + }; + } + + return _inner.ExecuteAsync(rebound, cancellationToken); + } + + public ValueTask DisposeAsync() => _inner.DisposeAsync(); + + /// Mirrors decodeBatchBinary in bit-bsqlite-wire.js. + private static object? Decode(int type, object? value) => type switch + { + BsqliteWireFormat.TypeCode.Null => null, + BsqliteWireFormat.TypeCode.Integer => (long)value!, + BsqliteWireFormat.TypeCode.Real => (double)value!, + BsqliteWireFormat.TypeCode.Text => (string)value!, + BsqliteWireFormat.TypeCode.Blob => ((ReadOnlyMemory)value!).ToArray(), + _ => throw new FormatException($"Unknown wire type tag {type}."), + }; +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/GlobalUsings.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/GlobalUsings.cs new file mode 100644 index 00000000000..974ab1211e3 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/GlobalUsings.cs @@ -0,0 +1 @@ +global using Microsoft.VisualStudio.TestTools.UnitTesting; diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/AdoNetSurfaceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/AdoNetSurfaceTests.cs new file mode 100644 index 00000000000..8b77a3d8c50 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/AdoNetSurfaceTests.cs @@ -0,0 +1,452 @@ +using System.Data; +using System.Data.Common; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The ADO.NET behaviours an application relies on without thinking about them: a transaction that +/// rolls back when abandoned, savepoints, batches, multi-result readers, and parameter names with or +/// without their sigil. Run against the in-process engine, which reports changes through the same +/// update hook the worker uses. +/// +[TestClass] +public sealed class AdoNetSurfaceTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + private async Task<(BsqliteInProcessTransport Transport, BsqliteConnection Connection)> OpenAsync() + { + var transport = new BsqliteInProcessTransport(); + var connection = new BsqliteConnection(transport, "ado.db"); + await connection.OpenAsync(Ct); + await ExecAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT NOT NULL)"); + return (transport, connection); + } + + private async Task ExecAsync(BsqliteConnection connection, string sql) + { + await using var command = connection.CreateCommand(); + command.CommandText = sql; + return await command.ExecuteNonQueryAsync(Ct); + } + + private async Task CountAsync(BsqliteConnection connection) + { + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT COUNT(*) FROM product"; + return Convert.ToInt64(await command.ExecuteScalarAsync(Ct)); + } + + // --------------------------------------------------------------------------------------------- + // Transactions + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Transaction_DisposedWithoutCommit_RollsBack() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using (var tx = await connection.BeginTransactionAsync(Ct)) + { + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('Ghost')"); + Assert.AreEqual(1, await CountAsync(connection)); + } + + Assert.AreEqual(0, await CountAsync(connection)); + + // The connection is free for a new transaction, which is the observable proof of the rollback. + await using var next = await connection.BeginTransactionAsync(Ct); + await next.CommitAsync(Ct); + } + + [TestMethod] + public async Task Transaction_BeginsImmediate_ForSerializable_AndDeferredOtherwise() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using (var tx = await connection.BeginTransactionAsync(Ct)) + { + await tx.RollbackAsync(Ct); + } + + await using (var tx = await connection.BeginTransactionAsync(IsolationLevel.ReadCommitted, Ct)) + { + await tx.RollbackAsync(Ct); + } + + Assert.Contains("BEGIN IMMEDIATE", transport.ExecutedCommands); + Assert.Contains("BEGIN", transport.ExecutedCommands); + } + + [TestMethod] + public async Task Transaction_NestedBegin_IsRefusedWithAClearMessage() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using var tx = await connection.BeginTransactionAsync(Ct); + + var error = await Assert.ThrowsExactlyAsync( + async () => await connection.BeginTransactionAsync(Ct)); + + Assert.Contains("savepoint", error.Message, StringComparison.OrdinalIgnoreCase); + } + + [TestMethod] + public async Task Savepoints_RollBackPartOfATransaction() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using var tx = await connection.BeginTransactionAsync(Ct); + Assert.IsTrue(tx.SupportsSavepoints); + + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('Kept')"); + await tx.SaveAsync("before_ghost", Ct); + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('Ghost')"); + await tx.RollbackAsync("before_ghost", Ct); + await tx.ReleaseAsync("before_ghost", Ct); + await tx.CommitAsync(Ct); + + Assert.AreEqual(1, await CountAsync(connection)); + } + + [TestMethod] + public async Task Transaction_CommitTwice_Throws() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using var tx = await connection.BeginTransactionAsync(Ct); + await tx.CommitAsync(Ct); + + await Assert.ThrowsExactlyAsync(() => tx.CommitAsync(Ct)); + } + + // --------------------------------------------------------------------------------------------- + // Batches and readers + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task Batch_RunsEveryCommandInOneRoundTrip_AndReportsPerCommandCounts() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + Assert.IsTrue(connection.CanCreateBatch); + await using var batch = connection.CreateBatch(); + for (var i = 0; i < 3; i++) + { + var command = batch.CreateBatchCommand(); + command.CommandText = "INSERT INTO product (name) VALUES (@name)"; + var parameter = command.CreateParameter(); + parameter.ParameterName = "@name"; + parameter.Value = $"Item {i}"; + command.Parameters.Add(parameter); + batch.BatchCommands.Add(command); + } + + var before = transport.ExecutedCommands.Count; + Assert.AreEqual(3, await batch.ExecuteNonQueryAsync(Ct)); + foreach (BsqliteBatchCommand command in batch.BatchCommands) + { + Assert.AreEqual(1, command.RecordsAffected); + } + Assert.AreEqual(3, transport.ExecutedCommands.Count - before); + Assert.AreEqual(3, await CountAsync(connection)); + } + + [TestMethod] + public async Task Batch_Reader_WalksEveryResultSet() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('A'), ('B')"); + + await using var batch = connection.CreateBatch(); + batch.BatchCommands.Add(new BsqliteBatchCommand { CommandText = "SELECT name FROM product ORDER BY id" }); + batch.BatchCommands.Add(new BsqliteBatchCommand { CommandText = "SELECT COUNT(*) FROM product" }); + + await using var reader = await batch.ExecuteReaderAsync(Ct); + var names = new List(); + while (await reader.ReadAsync(Ct)) + { + names.Add(reader.GetString(0)); + } + + AssertSequence.Equal(["A", "B"], names); + Assert.IsTrue(await reader.NextResultAsync(Ct)); + Assert.IsTrue(await reader.ReadAsync(Ct)); + Assert.AreEqual(2L, reader.GetInt64(0)); + Assert.IsFalse(await reader.NextResultAsync(Ct)); + } + + [TestMethod] + public async Task Reader_SingleRow_LimitsTheResult() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('A'), ('B'), ('C')"); + + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT name FROM product ORDER BY id"; + await using var reader = await command.ExecuteReaderAsync(CommandBehavior.SingleRow, Ct); + + Assert.IsTrue(await reader.ReadAsync(Ct)); + Assert.IsFalse(await reader.ReadAsync(Ct)); + } + + [TestMethod] + public async Task Reader_SchemaOnly_ReturnsColumnsWithoutRows() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('A')"); + + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT id, name FROM product"; + await using var reader = await command.ExecuteReaderAsync(CommandBehavior.SchemaOnly, Ct); + + Assert.AreEqual(2, reader.FieldCount); + Assert.AreEqual("name", reader.GetName(1)); + Assert.IsFalse(reader.HasRows); + } + + [TestMethod] + public async Task Reader_GetFieldType_WorksBeforeRead_FromTheColumnType() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT id, name FROM product"; + await using var reader = await command.ExecuteReaderAsync(Ct); + + Assert.AreEqual(typeof(long), reader.GetFieldType(0)); + Assert.AreEqual(typeof(string), reader.GetFieldType(1)); + Assert.AreEqual(1, reader.GetOrdinal("NAME")); + } + + [TestMethod] + public async Task Command_LastInsertRowId_IsAvailableWithoutAQuery() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using var command = connection.CreateCommand(); + command.CommandText = "INSERT INTO product (name) VALUES ('A')"; + await command.ExecuteNonQueryAsync(Ct); + await command.ExecuteNonQueryAsync(Ct); + + Assert.AreEqual(2L, command.LastInsertRowId); + Assert.AreEqual(2L, connection.LastInsertRowId); + } + + [TestMethod] + public async Task Parameters_MatchWithOrWithoutTheirSigil() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + await using var command = connection.CreateCommand(); + command.CommandText = "INSERT INTO product (name) VALUES (@name)"; + command.Parameters.AddWithValue("@name", "Sigil"); + + Assert.IsTrue(command.Parameters.Contains("name")); + Assert.AreSame(command.Parameters["@name"], command.Parameters["name"]); + Assert.AreEqual(1, await command.ExecuteNonQueryAsync(Ct)); + } + + [TestMethod] + public async Task Command_RequiresAnOpenConnection() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "closed.db"); + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT 1"; + + await Assert.ThrowsExactlyAsync(() => command.ExecuteScalarAsync(Ct)); + } + + // --------------------------------------------------------------------------------------------- + // Change tracking and snapshots + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task TablesChanged_ComesFromTheEngine_SoCascadesAreSeen() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + await ExecAsync(connection, "PRAGMA foreign_keys = ON"); + await ExecAsync(connection, "CREATE TABLE line (id INTEGER PRIMARY KEY, product_id INTEGER REFERENCES product(id) ON DELETE CASCADE)"); + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('A')"); + await ExecAsync(connection, "INSERT INTO line (product_id) VALUES (1)"); + + var seen = new List>(); + connection.TablesChanged += (_, e) => seen.Add(e.Tables); + + await ExecAsync(connection, "DELETE FROM product WHERE id = 1"); + + var tables = Assert.ContainsSingle(seen); + Assert.Contains("product", tables); + Assert.Contains("line", tables); + } + + [TestMethod] + public async Task TablesChanged_IsNotRaised_ForAReadOrAWriteThatChangedNothing() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + var raised = 0; + connection.TablesChanged += (_, _) => raised++; + + await CountAsync(connection); + await ExecAsync(connection, "DELETE FROM product WHERE id = 999"); + + Assert.AreEqual(0, raised); + } + + [TestMethod] + public async Task RemoteChanges_ReachTheConnection_WhenTheTransportHearsThem() + { + var transport = new NotifyingTransport(); + await using var connection = new BsqliteConnection(transport, "remote.db"); + await connection.OpenAsync(Ct); + + IReadOnlySet? seen = null; + connection.TablesChanged += (_, e) => seen = e.Tables; + + transport.Raise("Orders"); + + Assert.IsNotNull(seen); + Assert.Contains("orders", seen); + } + + [TestMethod] + public async Task Export_SnapshotsTheOpenDatabase() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('Exported')"); + + var image = await connection.ExportAsync(Ct); + + Assert.IsTrue(image.Length >= 4096); + CollectionAssert.AreEqual("SQLite format 3\0"u8.ToArray(), image[..16]); + } + + // --------------------------------------------------------------------------------------------- + // Live queries + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task LiveQuery_CoalescesABurstOfWrites_IntoFewRefreshes() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + var refreshes = 0; + var gate = new SemaphoreSlim(0); + await using var live = new BsqliteLiveQuery( + connection, + async ct => + { + Interlocked.Increment(ref refreshes); + await gate.WaitAsync(ct); + return await CountAsync(connection); + }, + ["product"]); + + var first = live.RefreshAsync(Ct); + gate.Release(); + await first; + + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + await WaitUntilAsync(() => Volatile.Read(ref refreshes) == 2); + + for (var i = 1; i < 10; i++) + { + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + } + + // One refresh is in flight (blocked on the gate); the other nine writes fold into one more. + gate.Release(2); + await WaitUntilAsync(() => Volatile.Read(ref refreshes) >= 3 && live.Current == 10); + await Task.Delay(100, Ct); + + Assert.AreEqual(3, refreshes); + } + + [TestMethod] + public async Task LiveQuery_ReportsAFailedRefresh_InsteadOfSwallowingIt() + { + var (transport, connection) = await OpenAsync(); + await using var _ = transport; + await using var __ = connection; + + Exception? failure = null; + await using var live = new BsqliteLiveQuery( + connection, + _ => throw new InvalidOperationException("boom"), + ["product"]); + live.RefreshFailed += (_, ex) => failure = ex; + + await ExecAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + + await WaitUntilAsync(() => failure is not null); + Assert.AreEqual("boom", failure!.Message); + } + + private static async Task WaitUntilAsync(Func condition) + { + var deadline = DateTime.UtcNow.AddSeconds(5); + while (!condition() && DateTime.UtcNow < deadline) + { + await Task.Delay(10); + } + + Assert.IsTrue(condition()); + } + + private sealed class NotifyingTransport : IBsqliteTransport + { + public event EventHandler? TablesChanged; + + public void Raise(params string[] tables) + => TablesChanged?.Invoke(this, new BsqliteTablesChangedEventArgs(tables)); + + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) => Task.CompletedTask; + + public Task CloseAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + => Task.FromResult>([new BsqliteCommandResult()]); + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BrowserSqlitePclProviderTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BrowserSqlitePclProviderTests.cs new file mode 100644 index 00000000000..b950289204e --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BrowserSqlitePclProviderTests.cs @@ -0,0 +1,42 @@ +using Bit.Bsqlite.EntityFrameworkCore; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The version this stub reports is what EF Core gates its SQL translations on in the browser, so +/// it has to describe the engine that is actually there. The browser suite checks it against the +/// running engine; this checks the two forms of it against each other, which is the half that can +/// be got wrong on desktop. +/// +[TestClass] +public sealed class BrowserSqlitePclProviderTests +{ + [TestMethod] + public void TheReportedVersionAndVersionNumberAgree() + { + var provider = new BsqlitePclProvider(); + var version = new Version(provider.sqlite3_libversion().utf8_to_string()); + + var expected = (version.Major * 1_000_000) + + (version.Minor * 1_000) + + Math.Max(version.Build, 0); + + Assert.AreEqual(expected, provider.sqlite3_libversion_number()); + } + + /// + /// Pinned so that bumping the vendored engine without updating the stub is a failing test here + /// as well as in the browser, where the engine can actually be asked. + /// + [TestMethod] + public void TheReportedVersionIsTheVendoredEngines() + { + // Two constants on purpose: the pin is the point, so the analyzer's "always true" is the test. +#pragma warning disable MSTEST0032 + Assert.AreEqual("3.53.0", BsqlitePclProvider.EngineVersion); +#pragma warning restore MSTEST0032 + Assert.AreEqual( + BsqlitePclProvider.EngineVersion, + new BsqlitePclProvider().sqlite3_libversion().utf8_to_string()); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteDataReaderTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteDataReaderTests.cs new file mode 100644 index 00000000000..acc15a0cf83 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteDataReaderTests.cs @@ -0,0 +1,107 @@ +using System.Data.Common; +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The reader is the surface EF materializes through, so the ADO.NET contract matters here more +/// than anywhere else: schema questions must be answerable before the first row, and no answer may +/// be . +/// +[TestClass] +public sealed class BsqliteDataReaderTests +{ + [TestMethod] + public void GetFieldType_AnswersBeforeTheFirstRead() + { + using var reader = Reader( + ["id", "name"], + ["INTEGER", "TEXT"], + [[1L, "Widget"]]); + + Assert.AreEqual(typeof(long), reader.GetFieldType(0)); + Assert.AreEqual(typeof(string), reader.GetFieldType(1)); + } + + /// + /// A NULL cell has no type of its own. Answering DBNull - which is what asking the + /// current row would give - is not a CLR type any caller can materialize into. + /// + [TestMethod] + public void GetFieldType_NeverAnswersDBNull_ForAColumnThatStartsNull() + { + using var reader = Reader( + ["name"], + ["TEXT"], + [[null], ["Widget"]]); + + Assert.IsTrue(reader.Read()); + Assert.IsTrue(reader.IsDBNull(0)); + Assert.AreEqual(typeof(string), reader.GetFieldType(0)); + } + + /// + /// With no data to go on the column's reported type decides, by SQLite's affinity rules - which + /// have to work for the worker's storage-class spellings as well as declared types. + /// + [TestMethod] + [DataRow("INTEGER", typeof(long))] + [DataRow("int", typeof(long))] + [DataRow("BIGINT", typeof(long))] + [DataRow("TEXT", typeof(string))] + [DataRow("nvarchar(50)", typeof(string))] + [DataRow("CLOB", typeof(string))] + [DataRow("BLOB", typeof(byte[]))] + [DataRow("REAL", typeof(double))] + [DataRow("decimal(18,2)", typeof(double))] + [DataRow(null, typeof(object))] + public void GetFieldType_FallsBackToAffinity_ForAnEmptyResult(string? declared, Type expected) + { + using var reader = Reader(["v"], [declared], []); + + Assert.AreEqual(expected, reader.GetFieldType(0)); + } + + /// + /// Reading past the end of a blob is how a caller finds the end, so it yields nothing rather + /// than an exception out of the copy. + /// + [TestMethod] + public void GetBytes_ReturnsZero_WhenTheOffsetIsPastTheEnd() + { + using var reader = Reader(["b"], ["BLOB"], [[new byte[] { 1, 2, 3 }]]); + Assert.IsTrue(reader.Read()); + + var buffer = new byte[8]; + + Assert.AreEqual(3, reader.GetBytes(0, 0, null, 0, 0)); + Assert.AreEqual(2, reader.GetBytes(0, 1, buffer, 0, 8)); + Assert.AreEqual(0, reader.GetBytes(0, 3, buffer, 0, 8)); + } + + [TestMethod] + public void GetChars_ReturnsZero_WhenTheOffsetIsPastTheEnd() + { + using var reader = Reader(["s"], ["TEXT"], [["abc"]]); + Assert.IsTrue(reader.Read()); + + var buffer = new char[8]; + + Assert.AreEqual(3, reader.GetChars(0, 0, null, 0, 0)); + Assert.AreEqual(0, reader.GetChars(0, 3, buffer, 0, 8)); + } + + private static DbDataReader Reader( + string[] columnNames, + string?[] columnTypes, + object?[][] rows) + => new BsqliteDataReader( + [ + new BsqliteCommandResult + { + ColumnNames = columnNames, + ColumnTypes = columnTypes, + Rows = rows, + }, + ]); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteExceptionTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteExceptionTests.cs new file mode 100644 index 00000000000..f508c6c5bf7 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteExceptionTests.cs @@ -0,0 +1,48 @@ +using System.Data.Common; +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// Code shared between a server and the browser catches , as it does for +/// Microsoft.Data.Sqlite. A Bsqlite failure that slipped past those handlers would surface only in +/// the browser. +/// +[TestClass] +public sealed class BsqliteExceptionTests +{ + /// What the worker transport throws for a failure the engine reported. + [TestMethod] + public void AFailureFromTheWorker_IsCaughtAsADbException() + { + DbException? caught = null; + try + { + throw BsqliteWireFormat.CreateException("UNIQUE constraint failed: product.name", 19, "Error"); + } + catch (DbException ex) + { + caught = ex; + } + + Assert.IsInstanceOfType(caught); + } + + [TestMethod] + public void EveryBsqliteFailure_IsADbException() + { + Assert.IsInstanceOfType(new BsqliteConstraintException("UNIQUE constraint failed")); + Assert.IsInstanceOfType(new BsqliteConcurrencyException("database is locked")); + Assert.IsInstanceOfType(new BsqliteCorruptDatabaseException("file is not a database")); + Assert.IsInstanceOfType(new BsqliteQuotaExceededException("database or disk is full")); + } + + [TestMethod] + [DataRow(5, true)] + [DataRow(6, true)] + [DataRow(261, true)] // SQLITE_BUSY_RECOVERY: the extended code keeps the primary in its low byte. + [DataRow(19, false)] + [DataRow(null, false)] + public void Busy_And_Locked_AreTransient(int? code, bool transient) + => Assert.AreEqual(transient, new BsqliteException("failure", code).IsTransient); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteSessionFactoryTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteSessionFactoryTests.cs new file mode 100644 index 00000000000..9da76c066d2 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteSessionFactoryTests.cs @@ -0,0 +1,356 @@ +using System.Data; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The session factory is the only place that both chooses a backend and opens it, so the rule +/// "the binding is written only after the open succeeds" is asserted here rather than left as a +/// comment on two collaborating types. +/// +[TestClass] +public sealed class BsqliteSessionFactoryTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task Open_UsesTheResolvedProvider_AndCommitsTheBinding() + { + var opfs = Available("opfs", BsqliteEngineBuild.Synchronous); + var store = new BsqliteInMemoryStorageBindingStore(); + var transports = new RecordingTransportFactory(); + var factory = Factory(store, transports, s => s.Prefer("opfs"), opfs); + + await using var session = await factory.OpenAsync("app.db", Ct); + + Assert.AreEqual("opfs", session.Resolution.Provider.Name); + Assert.AreEqual("opfs", await store.GetProviderNameAsync("app.db", Ct)); + Assert.AreEqual("app.db", transports.Last.OpenedAs); + Assert.AreEqual(opfs, transports.Created[0].Provider); + Assert.AreEqual(ConnectionState.Open, session.Connection.State); + } + + [TestMethod] + public async Task Open_PassesTheProvidersBuildAndVfs_ThroughTheTransportFactory() + { + var indexed = Available( + "indexeddb", + BsqliteEngineBuild.AsyncCapable, + vfs: new BsqliteJsModule("./_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-vfs.js")); + + var capturing = new RecordingTransportFactory(); + var factory = Factory( + new BsqliteInMemoryStorageBindingStore(), + capturing, + s => s.Prefer("indexeddb"), + indexed); + + await using var session = await factory.OpenAsync("app.db", Ct); + + var seen = capturing.Created[0].Provider; + Assert.AreSame(indexed, seen); + Assert.AreEqual(BsqliteEngineBuild.AsyncCapable, seen.Capabilities.RequiredBuild); + Assert.AreEqual("./_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-vfs.js", seen.VfsModule!.ModuleUrl); + } + + [TestMethod] + public async Task FailedOpen_DoesNotCommitABinding_AndDisposesTheTransport() + { + var store = new BsqliteInMemoryStorageBindingStore(); + var transports = new RecordingTransportFactory( + () => new ScriptedTransport + { + OpenThrows = new BsqliteException("disk I/O error", sqliteErrorCode: 10), + }); + var factory = Factory(store, transports, s => s.Prefer("opfs"), Available("opfs")); + + var error = await Assert.ThrowsExactlyAsync( + () => factory.OpenAsync("app.db", Ct)); + + Assert.AreEqual(10, error.SqliteErrorCode); + Assert.IsNull(await store.GetProviderNameAsync("app.db", Ct)); + Assert.IsTrue(transports.Last.Disposed); + } + + [TestMethod] + public async Task ExistingData_OpensOnTheBoundBackend_NotThePreferredOne() + { + var indexed = Available("indexeddb", BsqliteEngineBuild.AsyncCapable); + var opfs = Available("opfs", BsqliteEngineBuild.Synchronous); + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", "indexeddb", Ct); + + var transports = new RecordingTransportFactory(); + var factory = Factory( + store, + transports, + s => s.Prefer("opfs").Fallback("indexeddb"), + opfs, + indexed); + + await using var session = await factory.OpenAsync("app.db", Ct); + + Assert.AreEqual("indexeddb", session.Resolution.Provider.Name); + Assert.IsTrue(session.Resolution.WasDecidedByExistingData); + Assert.AreSame(indexed, transports.Created[0].Provider); + } + + [TestMethod] + public async Task AutomaticOnOpen_WithoutABetterProvider_JustOpens() + { + var store = new BsqliteInMemoryStorageBindingStore(); + var factory = Factory( + store, + new RecordingTransportFactory(), + s => s.Prefer("opfs").WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen), + Available("opfs")); + + await using var session = await factory.OpenAsync("app.db", Ct); + + Assert.AreEqual("opfs", session.Resolution.Provider.Name); + Assert.AreEqual("opfs", await store.GetProviderNameAsync("app.db", Ct)); + } + + [TestMethod] + public async Task AutomaticOnOpen_MovesTheDatabaseToTheBetterProvider_BeforeOpening() + { + var slow = new PersistentTestProvider("slow"); + var fast = new PersistentTestProvider("fast"); + byte[] image = [.. "SQLite format 3\0"u8, 1, 2, 3]; + await slow.Admin.ImportAsync("app.db", image, Ct); + + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", "slow", Ct); + + var transports = new RecordingTransportFactory(); + var factory = Factory( + store, + transports, + s => s.Prefer("fast").Fallback("slow").WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen), + fast, + slow); + + await using var session = await factory.OpenAsync("app.db", Ct); + + Assert.AreEqual("fast", session.Resolution.Provider.Name); + Assert.AreSame(fast, transports.Created[0].Provider); + Assert.AreEqual("fast", await store.GetProviderNameAsync("app.db", Ct)); + CollectionAssert.AreEqual(image, await fast.Admin.ExportAsync("app.db", Ct)); + Assert.IsFalse(await slow.Admin.ExistsAsync("app.db", Ct)); + } + + /// + /// An opportunistic move that fails is not a reason to keep the user from data that is still + /// exactly where it was. The open proceeds on the bound backend, the binding is unchanged, and + /// the resolution still says a better backend exists. + /// + [TestMethod] + public async Task AutomaticOnOpen_WhenTheMoveFails_StillOpensWhereTheDataIs() + { + var slow = new PersistentTestProvider("slow"); + var fast = new PersistentTestProvider("fast", refusesImports: true); + await slow.Admin.ImportAsync("app.db", "SQLite format 3\0"u8.ToArray(), Ct); + + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", "slow", Ct); + + var transports = new RecordingTransportFactory(); + var factory = Factory( + store, + transports, + s => s.Prefer("fast").Fallback("slow").WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen), + fast, + slow); + + await using var session = await factory.OpenAsync("app.db", Ct); + + Assert.AreEqual("slow", session.Resolution.Provider.Name); + Assert.AreSame(fast, session.Resolution.BetterProviderAvailable); + Assert.AreEqual("slow", await store.GetProviderNameAsync("app.db", Ct)); + Assert.IsTrue(await slow.Admin.ExistsAsync("app.db", Ct)); + } + + /// + /// The copy was verified and bound, then deleting the source failed with the source still whole - + /// another tab holding it open, say. The flip is undone, so the database opens on the source and + /// the binding agrees: writes made now are where the next load reads. + /// + [TestMethod] + public async Task AutomaticOnOpen_WhenDeletingTheSourceFails_AndItSurvived_OpensOnTheSource_AndKeepsItBound() + { + var slow = new PersistentTestProvider("slow", deletes: DeleteBehavior.FailsLeavingIt); + var fast = new PersistentTestProvider("fast"); + await slow.Admin.ImportAsync("app.db", "SQLite format 3"u8.ToArray(), Ct); + + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", "slow", Ct); + + var factory = Factory( + store, + new RecordingTransportFactory(), + s => s.Prefer("fast").Fallback("slow").WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen), + fast, + slow); + + await using var session = await factory.OpenAsync("app.db", Ct); + + Assert.AreEqual("slow", session.Resolution.Provider.Name); + Assert.AreEqual("slow", await store.GetProviderNameAsync("app.db", Ct)); + Assert.IsTrue(await slow.Admin.ExistsAsync("app.db", Ct)); + Assert.IsFalse(await fast.Admin.ExistsAsync("app.db", Ct), "The unbound copy is dropped."); + } + + /// + /// Deleting the source failed part-way, so the verified copy is the only whole database left. It + /// stays bound, and the session opens on it rather than on what remains of the source. + /// + [TestMethod] + public async Task AutomaticOnOpen_WhenDeletingTheSourceFailsPartway_OpensOnTheTarget() + { + var slow = new PersistentTestProvider("slow", deletes: DeleteBehavior.FailsAfterDeleting); + var fast = new PersistentTestProvider("fast"); + byte[] image = [.. "SQLite format 3"u8, 7]; + await slow.Admin.ImportAsync("app.db", image, Ct); + + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", "slow", Ct); + + var transports = new RecordingTransportFactory(); + var factory = Factory( + store, + transports, + s => s.Prefer("fast").Fallback("slow").WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen), + fast, + slow); + + await using var session = await factory.OpenAsync("app.db", Ct); + + Assert.AreEqual("fast", session.Resolution.Provider.Name); + Assert.AreSame(fast, transports.Created[0].Provider); + Assert.AreEqual("fast", await store.GetProviderNameAsync("app.db", Ct)); + CollectionAssert.AreEqual(image, await fast.Admin.ExportAsync("app.db", Ct)); + } + + [TestMethod] + public async Task Dispose_TearsDownTheTransport() + { + var transports = new RecordingTransportFactory(); + var factory = Factory( + new BsqliteInMemoryStorageBindingStore(), + transports, + s => s.Prefer("opfs"), + Available("opfs")); + + var session = await factory.OpenAsync("app.db", Ct); + await session.DisposeAsync(); + + Assert.IsTrue(transports.Last.Disposed); + } + + private enum DeleteBehavior + { + Deletes, + + /// Throws with the database untouched - an OPFS file another tab holds open. + FailsLeavingIt, + + /// Removes the database, then throws - a related file that could not go. + FailsAfterDeleting, + } + + /// + /// A persistent, available backend with a real (in-memory) admin surface, so a migration can be + /// run against it. Optionally refuses every import, which is what an origin out of quota does, + /// and fails deletes the ways lists. + /// + private sealed class PersistentTestProvider( + string name, + bool refusesImports = false, + DeleteBehavior deletes = DeleteBehavior.Deletes) + : IBsqliteStorageProvider + { + public string Name { get; } = name; + + public BsqliteStorageCapabilities Capabilities { get; } = new() + { + RequiredBuild = BsqliteEngineBuild.Synchronous, + IsPersistent = true, + }; + + public BsqliteJsModule? VfsModule => null; + + public IBsqliteStorageAdmin Admin { get; } = new MemoryAdmin(refusesImports, deletes); + + public ValueTask ProbeAsync(CancellationToken cancellationToken = default) + => ValueTask.FromResult(BsqliteProbeResult.Available()); + + private sealed class MemoryAdmin(bool refusesImports, DeleteBehavior deletes) : IBsqliteStorageAdmin + { + private readonly Dictionary _images = new(StringComparer.Ordinal); + + public ValueTask ExistsAsync(string databaseName, CancellationToken cancellationToken = default) + => ValueTask.FromResult(_images.ContainsKey(databaseName)); + + public ValueTask> ListAsync(CancellationToken cancellationToken = default) + => ValueTask.FromResult>([.. _images.Keys]); + + public ValueTask DeleteAsync(string databaseName, CancellationToken cancellationToken = default) + { + if (deletes is DeleteBehavior.FailsLeavingIt) + { + throw new IOException("The file is in use."); + } + + _images.Remove(databaseName); + + return deletes is DeleteBehavior.FailsAfterDeleting + ? throw new IOException("The journal is in use.") + : ValueTask.CompletedTask; + } + + public ValueTask ExportAsync(string databaseName, CancellationToken cancellationToken = default) + => _images.TryGetValue(databaseName, out var image) + ? ValueTask.FromResult(image.ToArray()) + : throw new FileNotFoundException(databaseName); + + public ValueTask ImportAsync( + string databaseName, + ReadOnlyMemory contents, + CancellationToken cancellationToken = default) + { + if (refusesImports) + { + throw new BsqliteQuotaExceededException("The origin is out of storage."); + } + + _images[databaseName] = contents.ToArray(); + return ValueTask.CompletedTask; + } + } + } + + private static ConfigurableProvider Available( + string name, + BsqliteEngineBuild build = BsqliteEngineBuild.Synchronous, + BsqliteJsModule? vfs = null) + => new() + { + Name = name, + RequiredBuild = build, + Vfs = vfs, + }; + + private static BsqliteSessionFactory Factory( + BsqliteInMemoryStorageBindingStore store, + IBsqliteTransportFactory transports, + Action configure, + params IBsqliteStorageProvider[] providers) + => new( + new BsqliteStorageProviderResolver(providers, store), + transports, + BsqliteStorageSelectionBuilder.Create(configure), + store); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteTransactionTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteTransactionTests.cs new file mode 100644 index 00000000000..d2e74a0cf38 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/BsqliteTransactionTests.cs @@ -0,0 +1,180 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The transport keeps one database open for the life of a session, so an open transaction outlives +/// the scope that started it. That makes the ADO.NET rule about disposal - an uncommitted +/// transaction rolls back - the difference between a tidy unwind and a connection nothing can write +/// to again. +/// +[TestClass] +public sealed class BsqliteTransactionTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task Dispose_RollsBackATransactionNobodyCompleted() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "tx.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + await using (var transaction = await connection.BeginTransactionAsync(Ct)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('Ghost')"); + } + + Assert.AreEqual(0L, await CountAsync(connection)); + + // The connection is writable again, which it would not be if the BEGIN were still open. + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('Kept')"); + Assert.AreEqual(1L, await CountAsync(connection)); + } + + [TestMethod] + public async Task Dispose_AfterCommit_LeavesTheWriteInPlace() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "tx.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + await using (var transaction = await connection.BeginTransactionAsync(Ct)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('Kept')"); + await transaction.CommitAsync(Ct); + } + + Assert.AreEqual(1L, await CountAsync(connection)); + } + + [TestMethod] + public async Task Dispose_AfterRollback_DoesNotRollBackTwice() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "tx.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + var transaction = await connection.BeginTransactionAsync(Ct); + await transaction.RollbackAsync(Ct); + await transaction.DisposeAsync(); + + Assert.DoesNotContain( + sql => sql == "ROLLBACK", + transport.ExecutedCommands.Skip(transport.ExecutedCommands.IndexOf("ROLLBACK") + 1)); + } + + /// + /// A cancel that lands after BEGIN was sent: the worker runs it regardless, so the transaction + /// has to come back owned. Otherwise the engine sits in a transaction no object can end, holding + /// the write lock and swallowing every later write. + /// + [TestMethod] + public async Task Begin_CancelledOnceSent_StillHandsBackTheTransaction() + { + using var cancel = new CancellationTokenSource(); + await using var inner = new BsqliteInProcessTransport(); + await using var transport = new AbandoningTransport(inner, "BEGIN", cancel); + await using var connection = new BsqliteConnection(transport, "tx.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + await using (var transaction = await connection.BeginTransactionAsync(cancel.Token)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('Ghost')"); + } + + Assert.AreEqual(0L, await CountAsync(connection)); + + await using var next = await connection.BeginTransactionAsync(Ct); + await next.CommitAsync(Ct); + } + + [TestMethod] + public async Task Begin_AlreadyCancelled_SendsNothing() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "tx.db"); + await connection.OpenAsync(Ct); + + await Assert.ThrowsAsync( + async () => await connection.BeginTransactionAsync(new CancellationToken(canceled: true))); + + Assert.DoesNotContain(sql => sql.StartsWith("BEGIN", StringComparison.Ordinal), transport.ExecutedCommands); + + await using var transaction = await connection.BeginTransactionAsync(Ct); + await transaction.CommitAsync(Ct); + } + + [TestMethod] + public async Task Commit_CancelledOnceSent_StillCompletes() + { + using var cancel = new CancellationTokenSource(); + await using var inner = new BsqliteInProcessTransport(); + await using var transport = new AbandoningTransport(inner, "COMMIT", cancel); + await using var connection = new BsqliteConnection(transport, "tx.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + var transaction = await connection.BeginTransactionAsync(Ct); + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('Kept')"); + await transaction.CommitAsync(cancel.Token); + await transaction.DisposeAsync(); + + Assert.AreEqual(1L, await CountAsync(connection)); + Assert.DoesNotContain("ROLLBACK", inner.ExecutedCommands); + } + + /// + /// Runs every batch, then - for the statement it watches - cancels the caller's token and stops + /// waiting on it, which is what Blazor interop does when a token fires after the call went out. + /// + private sealed class AbandoningTransport( + BsqliteInProcessTransport inner, + string statementPrefix, + CancellationTokenSource cancel) : IBsqliteTransport + { + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + => inner.OpenAsync(databaseName, cancellationToken); + + public Task CloseAsync(CancellationToken cancellationToken = default) + => inner.CloseAsync(cancellationToken); + + public async Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + { + var results = await inner.ExecuteAsync(batch, CancellationToken.None); + + if (batch[0].CommandText.StartsWith(statementPrefix, StringComparison.Ordinal)) + { + await cancel.CancelAsync(); + cancellationToken.ThrowIfCancellationRequested(); + } + + return results; + } + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; + } + + private async Task ExecuteAsync(BsqliteConnection connection, string sql) + { + await using var command = connection.CreateCommand(); + command.CommandText = sql; + await command.ExecuteNonQueryAsync(Ct); + } + + private async Task CountAsync(BsqliteConnection connection) + { + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT COUNT(*) FROM product"; + return Convert.ToInt64(await command.ExecuteScalarAsync(Ct)); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/InProcessEngineConformanceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/InProcessEngineConformanceTests.cs new file mode 100644 index 00000000000..fe5a65a2e5d --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/InProcessEngineConformanceTests.cs @@ -0,0 +1,22 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage.ConformanceTests; +using Bit.Bsqlite.Testing; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// Runs the engine kit against the in-process transport, which is how the kit is shown to be +/// honest: a rule the reference engine cannot satisfy is a bug in the rule. Crash safety is +/// correctly skipped - there is no worker to kill and nothing persists. +/// +[TestClass] +public sealed class InProcessEngineConformanceTests : BsqliteStorageEngineConformanceTests +{ + /// + protected override async ValueTask CreateOpenTransportAsync() + { + var transport = new BsqliteInProcessTransport(); + await transport.OpenAsync("conformance.db"); + return transport; + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/LiveQueryTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/LiveQueryTests.cs new file mode 100644 index 00000000000..c94d97e3aed --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/LiveQueryTests.cs @@ -0,0 +1,248 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; + +namespace Bit.Bsqlite.Interop.Tests; + +[TestClass] +public sealed class LiveQueryTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task ReRuns_WhenAWatchedTableIsWritten() + { + var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "live.db"); + await connection.OpenAsync(Ct); + await using var create = connection.CreateCommand(); + create.CommandText = "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"; + await create.ExecuteNonQueryAsync(Ct); + + var seen = new List(); + await using var live = new BsqliteLiveQuery( + connection, + async ct => + { + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT COUNT(*) FROM product"; + return Convert.ToInt32(await command.ExecuteScalarAsync(ct)); + }, + ["product"]); + + live.Changed += (_, count) => seen.Add(count); + Assert.AreEqual(0, await live.RefreshAsync(Ct)); + + await using var insert = connection.CreateCommand(); + insert.CommandText = "INSERT INTO product (name) VALUES ('Kept')"; + await insert.ExecuteNonQueryAsync(Ct); + + await WaitUntilAsync(() => seen.Contains(1)); + Assert.AreEqual(1, live.Current); + } + + /// + /// A write from another tab reaches the application only through the transport, so the + /// connection has to relay it. Without this the cross-tab half of a live query is JavaScript + /// plumbing with nothing on the other end. + /// + [TestMethod] + public async Task ReRuns_WhenTheTransportReportsAnotherTabsWrite() + { + var transport = new RemoteWriteTransport(); + await using var connection = new BsqliteConnection(transport, "live-remote.db"); + await connection.OpenAsync(Ct); + + var runs = 0; + await using var live = new BsqliteLiveQuery( + connection, + _ => Task.FromResult(Interlocked.Increment(ref runs)), + ["product"]); + + Assert.AreEqual(1, await live.RefreshAsync(Ct)); + + transport.ReportRemoteWrite("product"); + + await WaitUntilAsync(() => live.Current == 2); + } + + [TestMethod] + public async Task Ignores_ARemoteWriteToAnUnwatchedTable() + { + var transport = new RemoteWriteTransport(); + await using var connection = new BsqliteConnection(transport, "live-remote.db"); + await connection.OpenAsync(Ct); + + await using var live = new BsqliteLiveQuery( + connection, + _ => Task.FromResult(1), + ["product"]); + + await live.RefreshAsync(Ct); + var changed = 0; + live.Changed += (_, _) => Interlocked.Increment(ref changed); + + transport.ReportRemoteWrite("customer"); + + await Task.Delay(100, Ct); + Assert.AreEqual(0, changed); + } + + /// + /// The connection unsubscribes on dispose; otherwise the transport - which outlives every + /// connection made against it - keeps every one of them alive. + /// + [TestMethod] + public async Task DisposedConnection_StopsRelayingRemoteWrites() + { + var transport = new RemoteWriteTransport(); + var connection = new BsqliteConnection(transport, "live-remote.db"); + await connection.OpenAsync(Ct); + + var seen = 0; + connection.TablesChanged += (_, _) => Interlocked.Increment(ref seen); + await connection.DisposeAsync(); + + transport.ReportRemoteWrite("product"); + + Assert.AreEqual(0, seen); + } + + /// + /// A consumer that is busy with one item while the next refresh lands must not wait for a + /// further write to see it - and when it catches up it gets the newest snapshot, not a stale + /// intermediate one. + /// + [TestMethod] + public async Task WithCancellation_CatchesUpOnRefreshesItWasNotWaitingFor() + { + var transport = new RemoteWriteTransport(); + await using var connection = new BsqliteConnection(transport, "live-enumerate.db"); + await connection.OpenAsync(Ct); + + var runs = 0; + await using var live = new BsqliteLiveQuery( + connection, + _ => Task.FromResult(Interlocked.Increment(ref runs)), + ["product"]); + + using var cts = CancellationTokenSource.CreateLinkedTokenSource(Ct); +#pragma warning disable xUnit1051 // The enumerator's own token is the subject: it is cancelled below. + await using var enumerator = live.WithCancellation(cts.Token).GetAsyncEnumerator(cts.Token); +#pragma warning restore xUnit1051 + + Assert.IsTrue(await enumerator.MoveNextAsync()); + Assert.AreEqual(1, enumerator.Current); + + // Two refreshes complete while nobody is awaiting MoveNextAsync. + await live.RefreshAsync(Ct); + await live.RefreshAsync(Ct); + + Assert.IsTrue(await enumerator.MoveNextAsync()); + Assert.AreEqual(3, enumerator.Current); + + // Nothing new now, so the next item has to wait for the next refresh. + var pending = enumerator.MoveNextAsync(); + Assert.IsFalse(pending.IsCompleted); + + await live.RefreshAsync(Ct); + Assert.IsTrue(await pending); + Assert.AreEqual(4, enumerator.Current); + + await cts.CancelAsync(); + Assert.IsFalse(await enumerator.MoveNextAsync()); + } + + /// + /// The consumer was busy with an item - rendering it, say - when the query was disposed. Coming + /// back for the next one must end the stream, not register a waiter nobody will ever complete. + /// + [TestMethod] + public async Task WithCancellation_EndsWhenDisposedWhileTheConsumerWasBusy() + { + var transport = new RemoteWriteTransport(); + await using var connection = new BsqliteConnection(transport, "live-dispose.db"); + await connection.OpenAsync(Ct); + + var live = new BsqliteLiveQuery(connection, _ => Task.FromResult(1), ["product"]); + +#pragma warning disable xUnit1051 // No token on purpose: disposal alone has to end the stream. + await using var enumerator = live.WithCancellation(CancellationToken.None).GetAsyncEnumerator(); +#pragma warning restore xUnit1051 + Assert.IsTrue(await enumerator.MoveNextAsync()); + + await live.DisposeAsync(); + + var next = enumerator.MoveNextAsync().AsTask(); + Assert.AreSame(next, await Task.WhenAny(next, Task.Delay(TimeSpan.FromSeconds(5), Ct)), "The stream hung."); + Assert.IsFalse(await next); + } + + /// + /// An explicit refresh and one a write set off must not run the query at the same time: an EF + /// query shares a context that allows one operation at a time, and two runs finishing out of + /// order would publish the older result last. + /// + [TestMethod] + public async Task Refreshes_NeverOverlap() + { + var transport = new RemoteWriteTransport(); + await using var connection = new BsqliteConnection(transport, "live-overlap.db"); + await connection.OpenAsync(Ct); + + var running = 0; + var overlapped = false; + var runs = 0; + await using var live = new BsqliteLiveQuery( + connection, + async _ => + { + overlapped |= Interlocked.Increment(ref running) > 1; + await Task.Delay(20); + Interlocked.Decrement(ref running); + return Interlocked.Increment(ref runs); + }, + ["product"]); + + var explicitRuns = Enumerable.Range(0, 5).Select(_ => live.RefreshAsync(Ct)).ToList(); + transport.ReportRemoteWrite("product"); + await Task.WhenAll(explicitRuns); + await WaitUntilAsync(() => live.Current == 6); + + Assert.IsFalse(overlapped); + } + + /// A transport that can be told to speak for another tab. + private sealed class RemoteWriteTransport : IBsqliteTransport + { + public event EventHandler? TablesChanged; + + public void ReportRemoteWrite(params string[] tables) + => TablesChanged?.Invoke(this, new BsqliteTablesChangedEventArgs(tables)); + + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + => Task.CompletedTask; + + public Task CloseAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + => Task.FromResult>( + [.. batch.Select(_ => new BsqliteCommandResult())]); + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; + } + + private static async Task WaitUntilAsync(Func condition) + { + var deadline = DateTime.UtcNow.AddSeconds(2); + while (!condition() && DateTime.UtcNow < deadline) + { + await Task.Delay(20); + } + + Assert.IsTrue(condition()); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/RecordingTransportFactory.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/RecordingTransportFactory.cs new file mode 100644 index 00000000000..2505147d55e --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/RecordingTransportFactory.cs @@ -0,0 +1,96 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// Hands out scripted transports and remembers which provider each one was created for, so a test +/// can see that the factory asked for the backend selection actually chose. +/// +internal sealed class RecordingTransportFactory : IBsqliteTransportFactory +{ + private readonly Func _create; + + public RecordingTransportFactory(Func? create = null) + => _create = create ?? (() => new ScriptedTransport()); + + public List<(IBsqliteStorageProvider Provider, ScriptedTransport Transport)> Created { get; } = []; + + public ScriptedTransport Last + => Created.Count > 0 + ? Created[^1].Transport + : throw new InvalidOperationException("No transport has been created."); + + public IBsqliteTransport Create(IBsqliteStorageProvider provider) + { + var transport = _create(); + Created.Add((provider, transport)); + return transport; + } +} + +/// An whose open can be told to fail. +internal sealed class ScriptedTransport : IBsqliteTransport +{ + public string? OpenedAs { get; private set; } + + public int OpenCount { get; private set; } + + public bool Disposed { get; private set; } + + public Exception? OpenThrows { get; init; } + + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + { + OpenCount++; + OpenedAs = databaseName; + + if (OpenThrows is not null) + { + throw OpenThrows; + } + + return Task.CompletedTask; + } + + public Task CloseAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + => Task.FromResult>([]); + + public ValueTask DisposeAsync() + { + Disposed = true; + return ValueTask.CompletedTask; + } +} + +internal sealed class ConfigurableProvider : IBsqliteStorageProvider +{ + public required string Name { get; init; } + + public required BsqliteEngineBuild RequiredBuild { get; init; } + + public bool IsPersistent { get; init; } = true; + + public bool IsAvailable { get; init; } = true; + + public BsqliteJsModule? Vfs { get; init; } + + public BsqliteStorageCapabilities Capabilities => new() + { + RequiredBuild = RequiredBuild, + IsPersistent = IsPersistent, + }; + + public BsqliteJsModule? VfsModule => Vfs; + + public IBsqliteStorageAdmin Admin => throw new NotSupportedException(); + + public ValueTask ProbeAsync(CancellationToken cancellationToken = default) + => ValueTask.FromResult(IsAvailable + ? BsqliteProbeResult.Available() + : BsqliteProbeResult.Unavailable($"'{Name}' is switched off.")); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/ScriptedJsRuntime.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/ScriptedJsRuntime.cs new file mode 100644 index 00000000000..2d756c13209 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/ScriptedJsRuntime.cs @@ -0,0 +1,115 @@ +using System.Text.Json; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// A scripted that plays back envelopes and binary replies the way the +/// real host's call and executeBinary methods would, so the transport can be tested +/// without a browser. +/// +internal sealed class ScriptedJsRuntime : IJSRuntime +{ + private readonly Queue _replies = new(); + + public List Calls { get; } = []; + + public ScriptedJsObject Module { get; } + + public ScriptedJsObject Host { get; } + + public bool HostDisposed { get; private set; } + + /// The .NET object and method the host module was asked to forward committed writes to. + public (object Target, string Method)? Subscription { get; private set; } + + public ScriptedJsRuntime() + { + Module = new ScriptedJsObject(this, "module"); + Host = new ScriptedJsObject(this, "host"); + } + + /// Queues a JSON envelope for the next call. + public void EnqueueEnvelope(string json) + { + using var document = JsonDocument.Parse(json); + _replies.Enqueue(document.RootElement.Clone()); + } + + /// Queues a binary reply for the next executeBinary. + public void EnqueueReply(byte[] reply) => _replies.Enqueue(reply); + + public ValueTask InvokeAsync(string identifier, object?[]? args) + => InvokeAsync(identifier, CancellationToken.None, args); + + public ValueTask InvokeAsync( + string identifier, + CancellationToken cancellationToken, + object?[]? args) + { + Calls.Add(new JsCall(identifier, args ?? [])); + + object result = identifier switch + { + "import" => Module, + _ => throw new InvalidOperationException($"Unexpected runtime call '{identifier}'."), + }; + + return ValueTask.FromResult((TValue)result); + } + + internal ValueTask InvokeObjectAsync( + string objectName, + string identifier, + object?[]? args) + { + Calls.Add(new JsCall($"{objectName}.{identifier}", args ?? [])); + + if (objectName == "module" && identifier == "createHost") + { + return ValueTask.FromResult((TValue)(object)Host); + } + + // Subscribes the transport to committed writes. Nothing to play back - the recorded call + // in Calls, and the target it names, are what the tests assert on. + if (objectName == "module" && identifier == "listen") + { + Subscription = (args![1]!, args.Length > 3 ? (string)args[3]! : "OnTablesChanged"); + return ValueTask.FromResult(default(TValue)!); + } + + if (objectName == "host" && identifier is "call" or "executeBinary") + { + if (_replies.Count == 0) + { + throw new InvalidOperationException("The test did not enqueue a reply for this call."); + } + + return ValueTask.FromResult((TValue)_replies.Dequeue()); + } + + if (objectName == "host" && identifier == "dispose") + { + HostDisposed = true; + return ValueTask.FromResult(default(TValue)!); + } + + throw new InvalidOperationException($"Unexpected call '{objectName}.{identifier}'."); + } +} + +internal sealed class ScriptedJsObject(ScriptedJsRuntime runtime, string name) : IJSObjectReference +{ + public ValueTask InvokeAsync(string identifier, object?[]? args) + => runtime.InvokeObjectAsync(name, identifier, args); + + public ValueTask InvokeAsync( + string identifier, + CancellationToken cancellationToken, + object?[]? args) + => runtime.InvokeObjectAsync(name, identifier, args); + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; +} + +internal sealed record JsCall(string Identifier, object?[] Args); diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteFeatureGuardsTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteFeatureGuardsTests.cs new file mode 100644 index 00000000000..6aef5bf0db7 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteFeatureGuardsTests.cs @@ -0,0 +1,65 @@ +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Interop.Tests; + +[TestClass] +public sealed class SqliteFeatureGuardsTests +{ + [TestMethod] + [DataRow("PRAGMA journal_mode=WAL")] + [DataRow("pragma journal_mode = wal")] + [DataRow("PRAGMA journal_mode='wal'")] + [DataRow("PRAGMA journal_mode=\"WAL\"")] + public void RejectsWalInAnyReasonableSpelling(string sql) + { + var error = Assert.ThrowsExactly(() => BsqliteFeatureGuards.EnsureSupported(sql)); + + Assert.Contains("WAL", error.Message); + } + + [TestMethod] + [DataRow("PRAGMA journal_mode=DELETE")] + [DataRow("PRAGMA journal_mode=TRUNCATE")] + [DataRow("SELECT journal_mode FROM pragma_journal_mode")] + [DataRow("INSERT INTO t (name) VALUES ('walrus')")] + [DataRow("ATTACH 'other.db' AS other")] + [DataRow("PRAGMA page_size=4096")] + public void DoesNotRejectUnrelatedSql(string sql) + => BsqliteFeatureGuards.EnsureSupported(sql); + + [TestMethod] + [DataRow("ATTACH 'other.db' AS other")] + [DataRow("attach database 'other.db' as other")] + public void RejectsAttach_WhenTheBackendCannotSpanDatabases(string sql) + { + var limits = BsqliteRuntimeLimits.Unrestricted with { SupportsMultiDatabaseTransactions = false }; + var error = Assert.ThrowsExactly(() => BsqliteFeatureGuards.EnsureSupported(sql, limits)); + + Assert.Contains("ATTACH", error.Message); + } + + [TestMethod] + public void RejectsPageSizeAssignment_WhenTheBackendPinsThePageSize() + { + var limits = BsqliteRuntimeLimits.Unrestricted with { CanChangePageSize = false }; + var error = Assert.ThrowsExactly( + () => BsqliteFeatureGuards.EnsureSupported("PRAGMA page_size=8192", limits)); + + Assert.Contains("page_size", error.Message); + } + + [TestMethod] + [DataRow("INSERT INTO t (name) VALUES ('attach')")] + [DataRow("SELECT page_size FROM pragma_page_size")] + [DataRow("PRAGMA page_size")] + public void DoesNotFalsePositive_OnAttachOrPageSizeWords(string sql) + { + var limits = new BsqliteRuntimeLimits + { + SupportsMultiDatabaseTransactions = false, + CanChangePageSize = false, + }; + + BsqliteFeatureGuards.EnsureSupported(sql, limits); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteParameterBindingTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteParameterBindingTests.cs new file mode 100644 index 00000000000..675d8903fac --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteParameterBindingTests.cs @@ -0,0 +1,139 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The wire format refuses dates and GUIDs. The ADO.NET binder must convert them first, using the +/// same TEXT forms Microsoft.Data.Sqlite binds, or a browser SaveChanges of a DateTime column fails. +/// +[TestClass] +public sealed class SqliteParameterBindingTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public void DateTime_MatchesMicrosoftDataSqliteText() + { + var value = BsqliteParameterBinding.ToTransportValue( + new DateTime(2026, 3, 1, 12, 0, 0, DateTimeKind.Utc)); + + Assert.AreEqual("2026-03-01 12:00:00", value); + } + + [TestMethod] + public void Guid_MatchesMicrosoftDataSqliteText() + { + var guid = Guid.Parse("d1e2f3a4-b5c6-4789-abcd-ef0123456789"); + + Assert.AreEqual("D1E2F3A4-B5C6-4789-ABCD-EF0123456789", BsqliteParameterBinding.ToTransportValue(guid)); + } + + [TestMethod] + public void ConvertedValues_AreAcceptedByTheWireFormat() + { + var date = BsqliteParameterBinding.ToTransportValue(new DateTime(2026, 1, 1)); + var guid = BsqliteParameterBinding.ToTransportValue(Guid.Empty); + + var (dateType, dateValue) = BsqliteWireFormat.EncodeValue(date, "when"); + var (guidType, guidValue) = BsqliteWireFormat.EncodeValue(guid, "id"); + + Assert.AreEqual(BsqliteWireFormat.TypeCode.Text, dateType); + Assert.AreEqual("2026-01-01 00:00:00", dateValue); + Assert.AreEqual(BsqliteWireFormat.TypeCode.Text, guidType); + Assert.AreEqual("00000000-0000-0000-0000-000000000000", guidValue); + } + + [TestMethod] + public void FromTransportValue_ParsesTheTextToTransportValueWrote() + { + var lead = TimeSpan.FromDays(2); + var born = new DateOnly(1978, 4, 12); + var ordered = new DateTimeOffset(2026, 6, 2, 10, 15, 0, TimeSpan.Zero); + + Assert.AreEqual(lead, BsqliteParameterBinding.FromTransportValue( + BsqliteParameterBinding.ToTransportValue(lead))); + Assert.AreEqual(born, BsqliteParameterBinding.FromTransportValue( + BsqliteParameterBinding.ToTransportValue(born))); + Assert.AreEqual(ordered, BsqliteParameterBinding.FromTransportValue( + BsqliteParameterBinding.ToTransportValue(ordered))); + } + + [TestMethod] + public async Task Reader_GetFieldValue_ParsesTimeSpanText() + { + var transport = new ResultTransport(new BsqliteCommandResult + { + ColumnNames = ["lead"], + ColumnTypes = ["TEXT"], + Rows = [["2.00:00:00"]], + }); + await using var connection = new BsqliteConnection(transport, "read.db"); + await connection.OpenAsync(Ct); + await using var command = connection.CreateCommand(); + command.CommandText = "SELECT lead FROM t"; + await using var reader = await command.ExecuteReaderAsync(Ct); + + Assert.IsTrue(await reader.ReadAsync(Ct)); + Assert.AreEqual(TimeSpan.FromDays(2), reader.GetFieldValue(0)); + } + + [TestMethod] + public async Task Command_ConvertsDateTimeBeforeTheTransportSeesIt() + { + var transport = new CapturingTransport(); + await using var connection = new BsqliteConnection(transport, "bind.db"); + await connection.OpenAsync(Ct); + await using var command = connection.CreateCommand(); + command.CommandText = "INSERT INTO t (d) VALUES (@p1)"; + var parameter = command.CreateParameter(); + parameter.ParameterName = "@p1"; + parameter.Value = new DateTime(2026, 3, 1, 12, 0, 0, DateTimeKind.Utc); + command.Parameters.Add(parameter); + + await command.ExecuteNonQueryAsync(Ct); + + var batch = Assert.ContainsSingle(transport.LastBatch!); + Assert.AreEqual("2026-03-01 12:00:00", Assert.ContainsSingle(batch.Parameters).Value); + _ = BsqliteWireFormat.EncodeBatch(transport.LastBatch!); + } + + private sealed class CapturingTransport : IBsqliteTransport + { + public IReadOnlyList? LastBatch { get; private set; } + + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + => Task.CompletedTask; + + public Task CloseAsync(CancellationToken cancellationToken = default) + => Task.CompletedTask; + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + { + LastBatch = batch; + return Task.FromResult>([new BsqliteCommandResult()]); + } + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; + } + + private sealed class ResultTransport(BsqliteCommandResult result) : IBsqliteTransport + { + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + => Task.CompletedTask; + + public Task CloseAsync(CancellationToken cancellationToken = default) + => Task.CompletedTask; + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + => Task.FromResult>([result]); + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteTableNamesTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteTableNamesTests.cs new file mode 100644 index 00000000000..fb5d4a97a49 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteTableNamesTests.cs @@ -0,0 +1,82 @@ +using Bit.Bsqlite.Data; + +namespace Bit.Bsqlite.Interop.Tests; + +[TestClass] +public sealed class SqliteTableNamesTests +{ + [TestMethod] + [DataRow("SELECT * FROM product", "product")] + [DataRow("INSERT INTO product (name) VALUES ('x')", "product")] + [DataRow("UPDATE product SET name = 'x'", "product")] + [DataRow("DELETE FROM product", "product")] + [DataRow("SELECT * FROM product JOIN category ON 1=1", "product", "category")] + [DataRow("SELECT * FROM \"Products\" AS p", "Products")] + [DataRow("SELECT * FROM [Products]", "Products")] + [DataRow("SELECT * FROM `Products`", "Products")] + [DataRow("INSERT INTO main.Products (Name) VALUES ('x')", "Products")] + [DataRow("INSERT INTO \"main\".\"Products\" (Name) VALUES ('x')", "Products")] + [DataRow("CREATE TABLE IF NOT EXISTS \"Orders\" (Id INTEGER)", "Orders")] + [DataRow("CREATE INDEX IX_Orders_Date ON Orders (Date)", "Orders")] + [DataRow("DROP VIEW OpenOrders", "OpenOrders")] + [DataRow("SELECT * FROM (SELECT 1)")] + // The quoting SQLite accepts, a schema prefix, and UPDATE's conflict clause: each of these hid a + // table from the pattern once, and a hidden table is a live query that never re-runs. + [DataRow("SELECT * FROM \"Products\" AS \"p\"", "Products")] + [DataRow("INSERT INTO [Products] (Name) VALUES ('x')", "Products")] + [DataRow("DELETE FROM `Products` WHERE Id = 1", "Products")] + [DataRow("SELECT * FROM main.\"Products\"", "Products")] + [DataRow("UPDATE OR REPLACE Products SET Name = 'x'", "Products")] + [DataRow("UPDATE OR IGNORE \"Products\" SET Name = 'x'", "Products")] + [DataRow("CREATE TABLE IF NOT EXISTS \"Products\" (Id INTEGER)", "Products")] + [DataRow("SELECT * FROM sqlite_master")] + public void ExtractsTables(string sql, params string[] expected) + => AssertSequence.Equal( + expected.Order(StringComparer.OrdinalIgnoreCase), + BsqliteTableNames.Extract(sql).Order(StringComparer.OrdinalIgnoreCase)); + + [TestMethod] + public void SkipsSqliteInternalTables() + => Assert.IsEmpty(BsqliteTableNames.Extract("SELECT * FROM sqlite_master")); + + [TestMethod] + [DataRow("INSERT INTO product (name) VALUES ('from')", true)] + [DataRow("SELECT * FROM product", false)] + [DataRow("BEGIN; INSERT INTO product (name) VALUES ('x'); COMMIT;", true)] + [DataRow("PRAGMA foreign_keys = ON; DELETE FROM product", true)] + [DataRow("-- comment\nUPDATE product SET name = 'x'", true)] + [DataRow("/* comment */ DELETE FROM product", true)] + [DataRow("WITH gone AS (SELECT id FROM product) DELETE FROM product WHERE id IN gone", true)] + [DataRow("SELECT 'insert into' FROM product", false)] + [DataRow("CREATE TABLE t (id INTEGER)", true)] + // A write is a write wherever it sits in the batch. Missing one of these is a live query that + // never re-runs, which is the failure this heuristic exists to prevent. + [DataRow("SELECT 1; UPDATE product SET name = 'x'", true)] + [DataRow("BEGIN; SELECT * FROM product; COMMIT;", false)] + [DataRow(" \n DELETE FROM product", true)] + [DataRow("SELECT * FROM product; SELECT * FROM category", false)] + // A common table expression in front of a write is still a write. + [DataRow("WITH ids AS (SELECT 1 AS id) INSERT INTO product (id) SELECT id FROM ids", true)] + [DataRow("WITH ids AS (SELECT 1 AS id) UPDATE product SET name = 'x' WHERE id IN (SELECT id FROM ids)", true)] + [DataRow("WITH ids AS (SELECT 1 AS id) SELECT * FROM product WHERE id IN (SELECT id FROM ids)", false)] + [DataRow("SELECT * FROM withdrawals", false)] + // Block comments whose bodies hold stars and slashes, and an unterminated one. + [DataRow("/** doc **/ DELETE FROM product", true)] + [DataRow("/* a * b / c */ /* x */ DELETE FROM product", true)] + [DataRow("/**/DELETE FROM product", true)] + [DataRow("/* unterminated DELETE FROM product", false)] + public void DistinguishesWrites(string sql, bool write) + => Assert.AreEqual(write, BsqliteTableNames.LooksLikeWrite(sql)); + + // A run of */ /* used to backtrack exponentially: at this length the old patterns ran for hours. + [TestMethod] + public void MatchesRepeatedCommentsInLinearTime() + { + var sql = "/*" + string.Concat(Enumerable.Repeat("*//*", 2_000)); + + var check = Task.Run(() => (BsqliteTableNames.LooksLikeWrite(sql), BsqliteTableNames.LooksLikeSchemaChange(sql))); + + Assert.IsTrue(check.Wait(TimeSpan.FromSeconds(5)), "The write and schema-change patterns backtracked."); + Assert.AreEqual((false, false), check.Result); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteWireFormatTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteWireFormatTests.cs new file mode 100644 index 00000000000..43d9f3d878c --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/SqliteWireFormatTests.cs @@ -0,0 +1,521 @@ +using System.Buffers.Binary; +using System.Globalization; +using System.Text; +using System.Text.Json; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The wire format is the only thing standing between a 64-bit key and a silently rounded one, so +/// every storage class and the exact byte layout are pinned here rather than left to a browser test +/// that would also be testing the engine. writes replies the way +/// bit-bsqlite-wire.js does; a layout change must be made in both places. +/// +[TestClass] +public sealed class SqliteWireFormatTests +{ + [TestMethod] + public void EncodeBatch_WritesTheLayoutTheWorkerReads() + { + var bytes = BsqliteWireFormat.EncodeBatch( + [ + new BsqliteCommandRequest + { + CommandText = "INSERT INTO t (n) VALUES (@n)", + ResultKind = BsqliteResultKind.Reader, + RowLimit = 5, + Parameters = [new BsqliteParameterValue("@n", 7)], + }, + ]); + + var reader = new SpanReader(bytes); + Assert.AreEqual(BsqliteWireFormat.Magic, reader.Byte()); + Assert.AreEqual(BsqliteWireFormat.Version, reader.Byte()); + Assert.AreEqual(1u, reader.UInt32()); + Assert.AreEqual("INSERT INTO t (n) VALUES (@n)", reader.String()); + Assert.AreEqual(2, reader.Byte()); // reader + Assert.AreEqual(5, reader.Int32()); // row limit + Assert.AreEqual(1, reader.UInt16()); // parameter count + Assert.AreEqual("@n", reader.String()); + Assert.AreEqual(BsqliteWireFormat.TypeCode.Integer, reader.Byte()); + Assert.AreEqual(7L, reader.Int64()); + Assert.AreEqual(0, reader.Remaining); + } + + [TestMethod] + public void EncodeBatch_WritesMinusOne_WhenThereIsNoRowLimit() + { + var bytes = BsqliteWireFormat.EncodeBatch([new BsqliteCommandRequest { CommandText = "SELECT 1" }]); + + var reader = new SpanReader(bytes); + reader.Skip(6); + reader.String(); + Assert.AreEqual(0, reader.Byte()); + Assert.AreEqual(-1, reader.Int32()); + } + + [TestMethod] + [DataRow(0)] + [DataRow(1)] + [DataRow(-1)] + [DataRow(9007199254740993)] + [DataRow(long.MaxValue)] + [DataRow(long.MinValue)] + public void Integers_TravelAsEightBytes_WhateverTheirMagnitude(long value) + { + var (type, encoded) = BsqliteWireFormat.EncodeValue(value, "p"); + + Assert.AreEqual(BsqliteWireFormat.TypeCode.Integer, type); + Assert.AreEqual(value, Assert.IsInstanceOfType(encoded)); + + var bytes = BsqliteWireFormat.EncodeBatch( + [new BsqliteCommandRequest { CommandText = "x", Parameters = [new BsqliteParameterValue("p", value)] }]); + Assert.AreEqual(value, BinaryPrimitives.ReadInt64LittleEndian(bytes.AsSpan(bytes.Length - 8))); + } + + [TestMethod] + public void Decode_RestoresALargeInteger_Exactly() + { + var reply = new ReplyBuilder() + .Result(["v"], [BsqliteWireFormat.TypeCode.Integer]) + .Row(long.MaxValue) + .End() + .Build(); + + var results = BsqliteWireFormat.DecodeReply(reply); + + Assert.AreEqual(long.MaxValue, Assert.ContainsSingle(Assert.ContainsSingle(Assert.ContainsSingle(results).Rows))); + } + + [TestMethod] + public void Blob_RoundTripsAsRawBytes() + { + byte[] blob = [0, 1, 250, 255]; + var encoded = BsqliteWireFormat.EncodeBatch( + [ + new BsqliteCommandRequest + { + CommandText = "INSERT INTO t (b) VALUES (@b)", + Parameters = [new BsqliteParameterValue("@b", blob)], + }, + ]); + + // The last nine bytes are the blob tag, its length, and the bytes themselves. + Assert.AreEqual(BsqliteWireFormat.TypeCode.Blob, encoded[^9]); + Assert.AreEqual(4u, BinaryPrimitives.ReadUInt32LittleEndian(encoded.AsSpan(encoded.Length - 8))); + CollectionAssert.AreEqual(blob, encoded[^4..]); + + var results = BsqliteWireFormat.DecodeReply(new ReplyBuilder() + .Result(["b"], [BsqliteWireFormat.TypeCode.Blob]) + .Row(blob) + .End() + .Build()); + + CollectionAssert.AreEqual(blob, (byte[]?)Assert.ContainsSingle(Assert.ContainsSingle(Assert.ContainsSingle(results).Rows))); + } + + [TestMethod] + public void EveryStorageClass_DecodesToItsClrType() + { + var results = BsqliteWireFormat.DecodeReply(new ReplyBuilder() + .Result( + ["i", "r", "t", "b", "n"], + [BsqliteWireFormat.TypeCode.Integer, BsqliteWireFormat.TypeCode.Real, BsqliteWireFormat.TypeCode.Text, BsqliteWireFormat.TypeCode.Blob, 0]) + .Row(42L, 1.5d, "héllo wörld", new byte[] { 9 }, null) + .End() + .Build()); + + var result = Assert.ContainsSingle(results); + var row = Assert.ContainsSingle(result.Rows); + AssertSequence.Equal(["INTEGER", "REAL", "TEXT", "BLOB", null], result.ColumnTypes); + Assert.AreEqual(42L, row[0]); + Assert.AreEqual(1.5d, row[1]); + Assert.AreEqual("héllo wörld", row[2]); + CollectionAssert.AreEqual(new byte[] { 9 }, (byte[]?)row[3]); + Assert.IsNull(row[4]); + } + + [TestMethod] + public void Decode_CarriesTheCounters() + { + var results = BsqliteWireFormat.DecodeReply(new ReplyBuilder() + .Result([], [], recordsAffected: 3, lastInsertRowId: 77) + .End() + .Build()); + + var result = Assert.ContainsSingle(results); + Assert.AreEqual(3, result.RecordsAffected); + Assert.AreEqual(77L, result.LastInsertRowId); + } + + [TestMethod] + public void Decode_ReturnsOneResultPerRequest_InOrder() + { + var results = BsqliteWireFormat.DecodeReply(new ReplyBuilder() + .Result([], [], recordsAffected: 1).End() + .Result(["n"], [BsqliteWireFormat.TypeCode.Text]).Row("First").Row("Second").End() + .Build()); + + Assert.AreEqual(2, results.Count); + Assert.AreEqual(1, results[0].RecordsAffected); + Assert.AreEqual(2, results[1].Rows.Count); + Assert.AreEqual("Second", results[1].Rows[1][0]); + } + + [TestMethod] + public void NullBoolDecimalAndText_HaveOneObviousEncodingEach() + { + Assert.AreEqual((BsqliteWireFormat.TypeCode.Null, (object?)null), BsqliteWireFormat.EncodeValue(null, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Null, (object?)null), BsqliteWireFormat.EncodeValue(DBNull.Value, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Integer, 1L), BsqliteWireFormat.EncodeValue(true, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Integer, 0L), BsqliteWireFormat.EncodeValue(false, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Text, "1.25"), BsqliteWireFormat.EncodeValue(1.250m, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Text, "hi"), BsqliteWireFormat.EncodeValue("hi", "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Real, 1.5d), BsqliteWireFormat.EncodeValue(1.5d, "p")); + } + + /// + /// The oracle is Microsoft.Data.Sqlite: whatever it would have stored for a decimal parameter is + /// what the worker has to store, or a database written in the browser stops matching one written + /// on the server - and EF's decimal equality, which is a plain TEXT comparison against a literal + /// in this same form, starts returning nothing. + /// + [TestMethod] + [DataRow("10", "10.0")] + [DataRow("1.50", "1.5")] + [DataRow("100.000", "100.0")] + [DataRow("0", "0.0")] + [DataRow("-3", "-3.0")] + [DataRow("12.34", "12.34")] + [DataRow("0.3333333333333333333333333333", "0.3333333333333333333333333333")] + public void ADecimal_TravelsInMicrosoftDataSqlitesCanonicalTextForm(string clr, string expected) + { + var value = decimal.Parse(clr, CultureInfo.InvariantCulture); + + Assert.AreEqual((BsqliteWireFormat.TypeCode.Text, expected), BsqliteWireFormat.EncodeValue(value, "p")); + Assert.AreEqual(expected, StoredByMicrosoftDataSqlite(value)); + } + + /// + /// SQLite stores NaN as NULL, so that is what it becomes on the way in; the infinities are + /// ordinary IEEE doubles and cross the binary format as eight bytes like any other REAL. + /// + [TestMethod] + public void InfinityAndNaN_SurviveTheWire() + { + Assert.AreEqual((BsqliteWireFormat.TypeCode.Real, double.PositiveInfinity), BsqliteWireFormat.EncodeValue(double.PositiveInfinity, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Real, double.NegativeInfinity), BsqliteWireFormat.EncodeValue(float.NegativeInfinity, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Null, (object?)null), BsqliteWireFormat.EncodeValue(double.NaN, "p")); + Assert.AreEqual((BsqliteWireFormat.TypeCode.Null, (object?)null), BsqliteWireFormat.EncodeValue(float.NaN, "p")); + + var results = BsqliteWireFormat.DecodeReply(new ReplyBuilder() + .Result(["v", "w"], [BsqliteWireFormat.TypeCode.Real, BsqliteWireFormat.TypeCode.Real]) + .Row(double.PositiveInfinity, double.NegativeInfinity) + .End() + .Build()); + + AssertSequence.Equal([double.PositiveInfinity, double.NegativeInfinity], Assert.ContainsSingle(Assert.ContainsSingle(results).Rows)); + } + + /// What Microsoft.Data.Sqlite actually writes for a decimal parameter. + private static string StoredByMicrosoftDataSqlite(decimal value) + { + using var connection = new Microsoft.Data.Sqlite.SqliteConnection("Data Source=:memory:"); + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = "SELECT @v"; + command.Parameters.AddWithValue("@v", value); + return (string)command.ExecuteScalar()!; + } + + [TestMethod] + public void DatesAndGuids_AreRejected_SoTheTypeMappingOwnsTheChoice() + { + var date = Assert.ThrowsExactly( + () => BsqliteWireFormat.EncodeValue(new DateTime(2026, 1, 1), "when")); + var guid = Assert.ThrowsExactly( + () => BsqliteWireFormat.EncodeValue(Guid.Empty, "id")); + + Assert.Contains("when", date.Message); + Assert.Contains("id", guid.Message); + Assert.Contains("ADO.NET binder", date.Message); + } + + [TestMethod] + public void AnUnsignedIntegerPastInt64_IsRejectedRatherThanWrapped() + { + var error = Assert.ThrowsExactly( + () => BsqliteWireFormat.EncodeValue(ulong.MaxValue, "p")); + + Assert.Contains("64-bit signed INTEGER", error.Message); + } + + [TestMethod] + public void DecodeReply_ThrowsBsqliteException_CarryingTheResultCode() + { + var error = Assert.ThrowsExactly( + () => BsqliteWireFormat.DecodeReply(ReplyBuilder.Error("UNIQUE constraint failed", 19, "SQLiteError"))); + + Assert.AreEqual("UNIQUE constraint failed", error.Message); + Assert.AreEqual(19, error.SqliteErrorCode); + } + + [TestMethod] + public void DecodeReply_ThrowsWithoutACode_WhenTheFailureIsTheTransports() + { + var error = Assert.ThrowsExactly( + () => BsqliteWireFormat.DecodeReply(ReplyBuilder.Error("No database is open in this worker.", -1, "Error"))); + + Assert.IsNull(error.SqliteErrorCode); + } + + [TestMethod] + [DataRow(5, typeof(BsqliteConcurrencyException))] + [DataRow(6, typeof(BsqliteConcurrencyException))] + [DataRow(11, typeof(BsqliteCorruptDatabaseException))] + [DataRow(13, typeof(BsqliteQuotaExceededException))] + [DataRow(19, typeof(BsqliteConstraintException))] + [DataRow(1, typeof(BsqliteException))] + public void ResultCodes_MapToTypedExceptions(int code, Type expected) + { + var error = BsqliteWireFormat.CreateException("x", code, null); + + Assert.AreEqual(expected, error.GetType()); + Assert.AreEqual(code, error.SqliteErrorCode); + } + + [TestMethod] + public void AQuotaExceededError_IsRecognisedByName_WhenTheBrowserThrewIt() + { + var error = BsqliteWireFormat.CreateException("The quota has been exceeded.", null, "QuotaExceededError"); + + Assert.IsInstanceOfType(error); + } + + [TestMethod] + public void DecodeReply_RejectsTheWrongMagic() + { + Assert.ThrowsExactly(() => BsqliteWireFormat.DecodeReply([0x00, 0x01, 0x00])); + } + + [TestMethod] + public void DecodeReply_RejectsAnotherVersion_NamingTheMismatch() + { + var error = Assert.ThrowsExactly( + () => BsqliteWireFormat.DecodeReply([BsqliteWireFormat.Magic, (byte)(BsqliteWireFormat.Version + 1), 0])); + + Assert.Contains("different release", error.Message); + } + + [TestMethod] + public void DecodeReply_RejectsATruncatedMessage() + { + var whole = new ReplyBuilder().Result(["v"], [BsqliteWireFormat.TypeCode.Text]).Row("abc").End().Build(); + + Assert.ThrowsExactly(() => BsqliteWireFormat.DecodeReply(whole.AsSpan(0, whole.Length - 2))); + } + + [TestMethod] + public void DecodeCall_ReturnsTheResult_WhenTheWorkerSucceeded() + { + var result = BsqliteWireFormat.DecodeCall(Parse("""{ "ok": true, "result": [1, 2] }""")); + + Assert.AreEqual(2, result.GetArrayLength()); + } + + [TestMethod] + public void DecodeCall_ThrowsBsqliteException_CarryingTheResultCode() + { + var error = Assert.ThrowsExactly( + () => BsqliteWireFormat.DecodeCall(Parse( + """{ "ok": false, "error": { "message": "UNIQUE constraint failed", "sqliteCode": 19 } }"""))); + + Assert.AreEqual("UNIQUE constraint failed", error.Message); + Assert.AreEqual(19, error.SqliteErrorCode); + } + + [TestMethod] + public void DecodeCall_RejectsABareArray_SoAMissingEnvelopeCannotBeMistakenForSuccess() + { + Assert.ThrowsExactly(() => BsqliteWireFormat.DecodeCall(Parse("[1]"))); + } + + /// + /// A reply that does not match the envelope is a wire problem and is reported as one, rather + /// than as whatever System.Text.Json happens to raise when a property is missing or the + /// wrong kind. + /// + [TestMethod] + [DataRow("""{ "result": 1 }""")] + [DataRow("""{ "ok": "yes", "result": 1 }""")] + [DataRow("[]")] + public void DecodeCall_ReportsAMalformedEnvelope_AsAFormatException(string json) + => Assert.ThrowsExactly(() => BsqliteWireFormat.DecodeCall(Parse(json))); + + private static JsonElement Parse(string json) + { + using var document = JsonDocument.Parse(json); + return document.RootElement.Clone(); + } + + /// Walks a batch the way the worker's `BinaryReader` does. + private ref struct SpanReader(ReadOnlySpan data) + { + private readonly ReadOnlySpan _data = data; + private int _offset = 0; + + public readonly int Remaining => _data.Length - _offset; + + public void Skip(int count) => _offset += count; + + public byte Byte() => _data[_offset++]; + + public ushort UInt16() + { + var value = BinaryPrimitives.ReadUInt16LittleEndian(_data[_offset..]); + _offset += 2; + return value; + } + + public int Int32() + { + var value = BinaryPrimitives.ReadInt32LittleEndian(_data[_offset..]); + _offset += 4; + return value; + } + + public uint UInt32() + { + var value = BinaryPrimitives.ReadUInt32LittleEndian(_data[_offset..]); + _offset += 4; + return value; + } + + public long Int64() + { + var value = BinaryPrimitives.ReadInt64LittleEndian(_data[_offset..]); + _offset += 8; + return value; + } + + public string String() + { + var length = (int)UInt32(); + var value = Encoding.UTF8.GetString(_data.Slice(_offset, length)); + _offset += length; + return value; + } + } +} + +/// Writes replies exactly as bit-bsqlite-wire.js does, so the decoder is tested against the real layout. +internal sealed class ReplyBuilder +{ + private readonly MemoryStream _stream = new(); + private readonly BinaryWriter _writer; + private int _resultCount; + private long _rowCountPosition; + private int _rowCount; + + public ReplyBuilder() + { + _writer = new BinaryWriter(_stream, Encoding.UTF8, leaveOpen: true); + _writer.Write(BsqliteWireFormat.Magic); + _writer.Write(BsqliteWireFormat.Version); + _writer.Write((byte)0); + _writer.Write(0u); // result count, patched by Build + } + + public static byte[] Error(string message, int code, string name) + { + using var stream = new MemoryStream(); + using var writer = new BinaryWriter(stream, Encoding.UTF8); + writer.Write(BsqliteWireFormat.Magic); + writer.Write(BsqliteWireFormat.Version); + writer.Write((byte)1); + WriteString(writer, message); + writer.Write(code); + WriteString(writer, name); + writer.Flush(); + return stream.ToArray(); + } + + public ReplyBuilder Result(string[] columns, int[] storageClasses, int recordsAffected = 0, long lastInsertRowId = 0) + { + _resultCount++; + _writer.Write((ushort)columns.Length); + for (var i = 0; i < columns.Length; i++) + { + WriteString(_writer, columns[i]); + _writer.Write((byte)storageClasses[i]); + } + + _writer.Write(recordsAffected); + _writer.Write(lastInsertRowId); + _rowCountPosition = _stream.Position; + _writer.Write(0u); + _rowCount = 0; + return this; + } + + public ReplyBuilder Row(params object?[] cells) + { + _rowCount++; + foreach (var cell in cells) + { + switch (cell) + { + case null: + _writer.Write((byte)BsqliteWireFormat.TypeCode.Null); + break; + case long value: + _writer.Write((byte)BsqliteWireFormat.TypeCode.Integer); + _writer.Write(value); + break; + case double value: + _writer.Write((byte)BsqliteWireFormat.TypeCode.Real); + _writer.Write(value); + break; + case string value: + _writer.Write((byte)BsqliteWireFormat.TypeCode.Text); + WriteString(_writer, value); + break; + case byte[] value: + _writer.Write((byte)BsqliteWireFormat.TypeCode.Blob); + _writer.Write((uint)value.Length); + _writer.Write(value); + break; + default: + throw new ArgumentException($"Unsupported cell {cell.GetType()}."); + } + } + + return this; + } + + public ReplyBuilder End() + { + var end = _stream.Position; + _stream.Position = _rowCountPosition; + _writer.Write((uint)_rowCount); + _stream.Position = end; + return this; + } + + public byte[] Build() + { + _writer.Flush(); + var bytes = _stream.ToArray(); + BinaryPrimitives.WriteUInt32LittleEndian(bytes.AsSpan(3), (uint)_resultCount); + return bytes; + } + + private static void WriteString(BinaryWriter writer, string text) + { + var bytes = Encoding.UTF8.GetBytes(text); + writer.Write((uint)bytes.Length); + writer.Write(bytes); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/StorageMigratorTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/StorageMigratorTests.cs new file mode 100644 index 00000000000..d465a5fe30a --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/StorageMigratorTests.cs @@ -0,0 +1,107 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage.InMemory; + +namespace Bit.Bsqlite.Interop.Tests; + +[TestClass] +public sealed class StorageMigratorTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + private static readonly byte[] Image = + [ + .. "SQLite format 3\0"u8, + .. Enumerable.Repeat((byte)7, 48), + ]; + + [TestMethod] + public async Task CopiesTheImage_FlipsTheBinding_AndDeletesTheSource() + { + var source = new BsqliteInMemoryStorageProvider("source-memory"); + var target = new BsqliteInMemoryStorageProvider("target-memory"); + var store = new BsqliteInMemoryStorageBindingStore(); + await source.Admin.ImportAsync("app.db", Image, Ct); + + await new BsqliteStorageMigrator().MigrateAsync("app.db", source, target, store, Ct); + + Assert.IsFalse(await source.Admin.ExistsAsync("app.db", Ct)); + CollectionAssert.AreEqual(Image, await target.Admin.ExportAsync("app.db", Ct)); + Assert.AreEqual(target.Name, await store.GetProviderNameAsync("app.db", Ct)); + } + + /// + /// There is nothing to copy, so the binding moves and the open continues. Exporting anyway + /// would throw, and on the AutomaticOnOpen path that turns "a better backend became + /// available" into a database the application can no longer open at all. + /// + [TestMethod] + public async Task ASourceHoldingNothing_MovesTheBindingWithoutFailing() + { + var source = new BsqliteInMemoryStorageProvider("source-memory"); + var target = new BsqliteInMemoryStorageProvider("target-memory"); + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", source.Name, Ct); + + await new BsqliteStorageMigrator().MigrateAsync("app.db", source, target, store, Ct); + + Assert.AreEqual(target.Name, await store.GetProviderNameAsync("app.db", Ct)); + Assert.IsFalse(await target.Admin.ExistsAsync("app.db", Ct)); + } + + [TestMethod] + public async Task ACorruptImage_LeavesTheSourceAndBindingUntouched() + { + var source = new BsqliteInMemoryStorageProvider("source-memory"); + var target = new BsqliteInMemoryStorageProvider("target-memory"); + var store = new BsqliteInMemoryStorageBindingStore(); + await source.Admin.ImportAsync("app.db", new byte[] { 1, 2, 3, 4 }, Ct); + + await Assert.ThrowsExactlyAsync( + () => new BsqliteStorageMigrator().MigrateAsync("app.db", source, target, store, Ct)); + + Assert.IsTrue(await source.Admin.ExistsAsync("app.db", Ct)); + Assert.IsFalse(await target.Admin.ExistsAsync("app.db", Ct)); + Assert.IsNull(await store.GetProviderNameAsync("app.db", Ct)); + } + + /// + /// The binding write landed and only its report failed - a cancel that fired once the interop + /// call was out, say. The binding names the copy now, so dropping the copy as unbound would + /// leave the binding pointing at nothing. + /// + [TestMethod] + public async Task ABindingWriteThatLandedButReportedFailure_KeepsTheCopy() + { + var source = new BsqliteInMemoryStorageProvider("source-memory"); + var target = new BsqliteInMemoryStorageProvider("target-memory"); + var store = new WritesThenThrowsBindingStore(); + await source.Admin.ImportAsync("app.db", Image, Ct); + + await Assert.ThrowsExactlyAsync( + () => new BsqliteStorageMigrator().MigrateAsync("app.db", source, target, store, Ct)); + + Assert.AreEqual(target.Name, await store.GetProviderNameAsync("app.db", Ct)); + CollectionAssert.AreEqual(Image, await target.Admin.ExportAsync("app.db", Ct)); + } + + /// A binding store whose writes land and then report a cancellation. + private sealed class WritesThenThrowsBindingStore : IBsqliteStorageBindingStore + { + private readonly BsqliteInMemoryStorageBindingStore _inner = new(); + + public ValueTask GetProviderNameAsync(string databaseName, CancellationToken cancellationToken = default) + => _inner.GetProviderNameAsync(databaseName, cancellationToken); + + public async ValueTask SetProviderNameAsync(string databaseName, string providerName, CancellationToken cancellationToken = default) + { + await _inner.SetProviderNameAsync(databaseName, providerName, cancellationToken); + throw new TaskCanceledException(); + } + + public ValueTask RemoveAsync(string databaseName, CancellationToken cancellationToken = default) + => _inner.RemoveAsync(databaseName, cancellationToken); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/WorkerSqliteTransportTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/WorkerSqliteTransportTests.cs new file mode 100644 index 00000000000..d61ba024084 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/WorkerSqliteTransportTests.cs @@ -0,0 +1,289 @@ +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage; +using Microsoft.JSInterop; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// The transport is the only .NET type that talks to the worker, so these tests pin the calls it +/// makes and the way it turns a reply into either a result or a . +/// The worker itself is covered by the browser suite; this suite does not need one. +/// +[TestClass] +public sealed class WorkerSqliteTransportTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task Open_ImportsTheHostModule_ThenCreatesAHost_ThenCallsOpen_ThenSubscribes() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous", "reused": false } }"""); + + await using var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + await transport.OpenAsync("app.db", Ct); + + AssertSequence.Equal( + ["import", "module.createHost", "module.listen", "host.call"], + js.Calls.Select(c => c.Identifier)); + // Stamped, not bare: the version in the query is what keeps a browser from answering the + // import with a module it cached under an earlier one. + Assert.AreEqual(BsqliteWorkerTransport.VersionedHostModuleUrl, js.Calls[0].Args[0]); + Assert.StartsWith( + BsqliteWorkerTransport.DefaultHostModuleUrl + "?v=", + BsqliteWorkerTransport.VersionedHostModuleUrl, + StringComparison.Ordinal); + + var request = OpenRequest(js); + Assert.AreEqual("open", Read(request, "kind")); + Assert.AreEqual("app.db", Read(request, "databaseName")); + Assert.AreEqual("synchronous", Read(request, "requiredBuild")); + Assert.IsNull(Read(request, "vfs")); + Assert.AreEqual(nameof(BsqliteWorkerTransport.OnTablesChanged), js.Subscription!.Value.Method); + } + + /// + /// The subscription is what makes a live query re-run for a committed write - this tab's or + /// another's - and it has to be in place before the open so a write during startup is not missed. + /// + [TestMethod] + public async Task Open_SubscribesToCommittedWrites_ForThisDatabaseOnly() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous", "reused": false } }"""); + + await using var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + await transport.OpenAsync("app.db", Ct); + + var listen = js.Calls.Single(c => c.Identifier == "module.listen"); + Assert.AreEqual("app.db", listen.Args[2]); + Assert.IsInstanceOfType>(listen.Args[1]); + + // The worker reports this tab's own writes too, from the engine's hooks and only once + // committed, so the command layer must not raise them again from the SQL text. + Assert.IsTrue(transport.ReportsLocalWrites); + } + + [TestMethod] + public async Task OnTablesChanged_RaisesTheTransportEvent() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous", "reused": false } }"""); + + await using var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + await transport.OpenAsync("app.db", Ct); + + BsqliteTablesChangedEventArgs? seen = null; + transport.TablesChanged += (_, e) => seen = e; + + transport.OnTablesChanged(["product", "customer"]); + + Assert.IsNotNull(seen); + Assert.Contains("product", seen.Tables); + Assert.Contains("customer", seen.Tables); + } + + /// + /// Through , not the concrete type. The interface declares + /// TablesChanged with a no-op default implementation so existing transports still + /// compile, and if the worker transport's own event failed to implement it the subscription + /// would bind to that default and cross-tab live queries would go quiet with nothing to see. + /// + [TestMethod] + public async Task ARemoteWrite_ReachesAConnectionThroughTheInterface() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous", "reused": false } }"""); + + var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + IBsqliteTransport asInterface = transport; + await asInterface.OpenAsync("app.db", Ct); + + await using var connection = new BsqliteConnection(asInterface, "app.db"); + BsqliteTablesChangedEventArgs? seen = null; + connection.TablesChanged += (_, e) => seen = e; + + transport.OnTablesChanged(["product"]); + + Assert.IsNotNull(seen); + Assert.Contains("product", seen.Tables); + } + + [TestMethod] + public async Task Open_SubscribesOnlyOnce_WhenCalledAgain() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous", "reused": false } }"""); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous", "reused": true } }"""); + + await using var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + await transport.OpenAsync("app.db", Ct); + await transport.OpenAsync("app.db", Ct); + + Assert.ContainsSingle(c => c.Identifier == "module.listen", js.Calls); + } + + [TestMethod] + public async Task Open_EnablesForeignKeys_ByDefault_AndForwardsExtraPragmas() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous" } }"""); + + var options = new BsqliteWorkerTransportOptions + { + RequiredBuild = BsqliteEngineBuild.Synchronous, + OnOpenSql = ["PRAGMA cache_size=-16000"], + }; + + await using var transport = new BsqliteWorkerTransport(js, options); + await transport.OpenAsync("app.db", Ct); + + var open = OpenRequest(js); + Assert.AreEqual(true, Read(open, "foreignKeys")); + AssertSequence.Equal(["PRAGMA cache_size=-16000"], Assert.IsInstanceOfType>(Read(open, "onOpenSql"))); + } + + [TestMethod] + public async Task Open_CanLeaveForeignKeysAlone() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous" } }"""); + + await using var transport = new BsqliteWorkerTransport(js, new BsqliteWorkerTransportOptions + { + RequiredBuild = BsqliteEngineBuild.Synchronous, + EnableForeignKeys = false, + }); + await transport.OpenAsync("app.db", Ct); + + var open = OpenRequest(js); + Assert.AreEqual(false, Read(open, "foreignKeys")); + Assert.IsEmpty(Assert.IsInstanceOfType>(Read(open, "onOpenSql"))); + } + + [TestMethod] + public async Task Open_PassesTheVfsModuleAndExtensionsTheProviderDeclared() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "jspi", "reused": false } }"""); + + var options = new BsqliteWorkerTransportOptions + { + RequiredBuild = BsqliteEngineBuild.AsyncCapable, + Vfs = new BsqliteJsModule("./_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-vfs.js"), + ExtensionModules = [new BsqliteJsModule("./functions.js", "install")], + }; + + await using var transport = new BsqliteWorkerTransport(js, options); + await transport.OpenAsync("app.db", Ct); + + var open = OpenRequest(js); + var vfs = Read(open, "vfs")!; + Assert.AreEqual("asyncCapable", Read(open, "requiredBuild")); + Assert.AreEqual("./_content/Bit.Bsqlite.Storage.IndexedDb/bit-bsqlite-idb-vfs.js", Read(vfs, "moduleUrl")); + Assert.AreEqual("register", Read(vfs, "registerExport")); + + var extension = Assert.ContainsSingle((object?[])Read(open, "extensions")!)!; + Assert.AreEqual("./functions.js", Read(extension, "moduleUrl")); + Assert.AreEqual("install", Read(extension, "registerExport")); + } + + [TestMethod] + public async Task Execute_SendsTheBinaryBatch_AndDecodesTheBinaryReply() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous" } }"""); + js.EnqueueReply(new ReplyBuilder() + .Result(["v"], [BsqliteWireFormat.TypeCode.Integer]) + .Row(9007199254740993L) + .End() + .Build()); + + await using var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + await transport.OpenAsync("app.db", Ct); + + var results = await transport.ExecuteAsync( + [ + new BsqliteCommandRequest + { + CommandText = "SELECT v FROM t", + ResultKind = BsqliteResultKind.Reader, + }, + ], Ct); + + var call = js.Calls.Last(c => c.Identifier == "host.executeBinary"); + var sent = Assert.IsInstanceOfType(call.Args[0]); + Assert.AreEqual(BsqliteWireFormat.Magic, sent[0]); + + var row = Assert.ContainsSingle(Assert.ContainsSingle(results).Rows); + Assert.AreEqual(9007199254740993L, Assert.ContainsSingle(row)); + } + + [TestMethod] + public async Task Execute_ThrowsTypedException_WhenTheReplyIsAFailure() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous" } }"""); + js.EnqueueReply(ReplyBuilder.Error("UNIQUE constraint failed: t.id", 19, "SQLiteError")); + + await using var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + await transport.OpenAsync("app.db", Ct); + + var error = await Assert.ThrowsExactlyAsync( + () => transport.ExecuteAsync( + [ + new BsqliteCommandRequest { CommandText = "INSERT INTO t (id) VALUES (1)" }, + ], Ct)); + + Assert.AreEqual(19, error.SqliteErrorCode); + Assert.Contains("UNIQUE", error.Message); + } + + [TestMethod] + public async Task Execute_BeforeOpen_Throws() + { + await using var transport = new BsqliteWorkerTransport(new ScriptedJsRuntime(), DefaultOptions()); + + await Assert.ThrowsExactlyAsync( + () => transport.ExecuteAsync([], Ct)); + } + + [TestMethod] + public async Task Close_IsANoOp_WhenNothingWasOpened() + { + var js = new ScriptedJsRuntime(); + await using var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + + await transport.CloseAsync(Ct); + + Assert.IsEmpty(js.Calls); + } + + [TestMethod] + public async Task Dispose_TerminatesTheWorker() + { + var js = new ScriptedJsRuntime(); + js.EnqueueEnvelope("""{ "ok": true, "result": { "build": "synchronous" } }"""); + + var transport = new BsqliteWorkerTransport(js, DefaultOptions()); + await transport.OpenAsync("app.db", Ct); + await transport.DisposeAsync(); + + Assert.IsTrue(js.HostDisposed); + await Assert.ThrowsExactlyAsync(() => transport.OpenAsync("app.db", Ct)); + } + + private static BsqliteWorkerTransportOptions DefaultOptions() => new() + { + RequiredBuild = BsqliteEngineBuild.Synchronous, + }; + + private static object OpenRequest(ScriptedJsRuntime js) + => js.Calls.First(c => c.Identifier == "host.call").Args[0]!; + + private static object? Read(object target, string property) + => target.GetType().GetProperty(property)?.GetValue(target); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/WriteNotificationTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/WriteNotificationTests.cs new file mode 100644 index 00000000000..5e6b84cdefa --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Interop/WriteNotificationTests.cs @@ -0,0 +1,256 @@ +using System.Data.Common; +using Bit.Bsqlite.Data; +using Bit.Bsqlite.Testing; + +namespace Bit.Bsqlite.Interop.Tests; + +/// +/// When a live query hears about a write, and what it hears. Two paths exist and both are pinned +/// here: a transport that reports its own writes from the engine's hooks (the worker, and the +/// in-process transport that mirrors it), and the command layer's fallback for a transport that +/// cannot, which derives the tables from the SQL text. +/// +[TestClass] +public sealed class WriteNotificationTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + // --------------------------------------------------------------------------------------------- + // The engine-hook path, through the in-process transport. + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task InProcessTransport_ReportsItsOwnWrites_SoTheConnectionDoesNot() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "notify.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + Assert.IsTrue(transport.ReportsLocalWrites); + + var raised = new List>(); + connection.TablesChanged += (_, e) => raised.Add(e.Tables); + + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + + // Once, not twice: the transport spoke, and the command layer stayed quiet. + var tables = Assert.ContainsSingle(raised); + Assert.Contains("product", tables); + } + + /// + /// The reason the hooks are worth having: nothing in this DELETE's text names the child table, + /// yet its rows are gone. + /// + [TestMethod] + public async Task ACascadingDelete_NamesTheChildTable() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "cascade.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "PRAGMA foreign_keys=ON"); + await ExecuteAsync(connection, "CREATE TABLE parent (id INTEGER PRIMARY KEY)"); + await ExecuteAsync( + connection, + "CREATE TABLE child (id INTEGER PRIMARY KEY, " + + "parent_id INTEGER REFERENCES parent(id) ON DELETE CASCADE)"); + await ExecuteAsync(connection, "INSERT INTO parent (id) VALUES (1)"); + await ExecuteAsync(connection, "INSERT INTO child (id, parent_id) VALUES (1, 1)"); + + var raised = new List>(); + connection.TablesChanged += (_, e) => raised.Add(e.Tables); + + await ExecuteAsync(connection, "DELETE FROM parent WHERE id = 1"); + + var tables = Assert.ContainsSingle(raised); + Assert.Contains("parent", tables); + Assert.Contains("child", tables); + } + + /// + /// SQLite's truncate optimisation bypasses the update hook for a DELETE with no WHERE, so the + /// statement text has to carry the name. + /// + [TestMethod] + public async Task ATruncatingDelete_StillNamesTheTable() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "truncate.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('x'), ('y')"); + + var raised = new List>(); + connection.TablesChanged += (_, e) => raised.Add(e.Tables); + + await ExecuteAsync(connection, "DELETE FROM product"); + + Assert.Contains("product", Assert.ContainsSingle(raised)); + } + + [TestMethod] + public async Task ATransactionsWrites_AreReportedOnceAtCommit() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "tx-notify.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + await ExecuteAsync(connection, "CREATE TABLE customer (id INTEGER PRIMARY KEY, name TEXT)"); + + var raised = new List>(); + connection.TablesChanged += (_, e) => raised.Add(e.Tables); + + await using (var transaction = await connection.BeginTransactionAsync(Ct)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + await ExecuteAsync(connection, "INSERT INTO customer (name) VALUES ('y')"); + + // Nothing yet: another reader could not see these rows, and they may still roll back. + Assert.IsEmpty(raised); + + await transaction.CommitAsync(Ct); + } + + var tables = Assert.ContainsSingle(raised); + Assert.Contains("product", tables); + Assert.Contains("customer", tables); + } + + [TestMethod] + public async Task ARolledBackTransaction_ReportsNothing() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "tx-rollback.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + var raised = 0; + connection.TablesChanged += (_, _) => raised++; + + await using (var transaction = await connection.BeginTransactionAsync(Ct)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('ghost')"); + await transaction.RollbackAsync(Ct); + } + + // Also for the implicit rollback that disposal performs. + await using (await connection.BeginTransactionAsync(Ct)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('ghost')"); + } + + Assert.AreEqual(0, raised); + } + + /// + /// A transaction driven by SQL text rather than the ADO.NET API is a transaction all the same: + /// the transport reads the engine's autocommit state, not the command that changed it. + /// + [TestMethod] + public async Task ARawSqlTransaction_IsReportedAtItsCommit() + { + await using var transport = new BsqliteInProcessTransport(); + await using var connection = new BsqliteConnection(transport, "tx-raw.db"); + await connection.OpenAsync(Ct); + await ExecuteAsync(connection, "CREATE TABLE product (id INTEGER PRIMARY KEY, name TEXT)"); + + var raised = new List>(); + connection.TablesChanged += (_, e) => raised.Add(e.Tables); + + await ExecuteAsync(connection, "BEGIN"); + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + Assert.IsEmpty(raised); + + await ExecuteAsync(connection, "COMMIT"); + Assert.Contains("product", Assert.ContainsSingle(raised)); + } + + // --------------------------------------------------------------------------------------------- + // The command-layer fallback, for a transport that cannot report its own writes. + // --------------------------------------------------------------------------------------------- + + [TestMethod] + public async Task SilentTransport_TheConnectionReportsFromTheSql() + { + var transport = new SilentTransport(); + await using var connection = new BsqliteConnection(transport, "silent.db"); + await connection.OpenAsync(Ct); + + var raised = new List>(); + connection.TablesChanged += (_, e) => raised.Add(e.Tables); + + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + + Assert.Contains("product", Assert.ContainsSingle(raised)); + } + + [TestMethod] + public async Task SilentTransport_ATransactionsWrites_WaitForCommit() + { + var transport = new SilentTransport(); + await using var connection = new BsqliteConnection(transport, "silent-tx.db"); + await connection.OpenAsync(Ct); + + var raised = new List>(); + connection.TablesChanged += (_, e) => raised.Add(e.Tables); + + await using (var transaction = await connection.BeginTransactionAsync(Ct)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('x')"); + await ExecuteAsync(connection, "UPDATE customer SET name = 'y'"); + Assert.IsEmpty(raised); + + await transaction.CommitAsync(Ct); + } + + var tables = Assert.ContainsSingle(raised); + Assert.Contains("product", tables); + Assert.Contains("customer", tables); + } + + [TestMethod] + public async Task SilentTransport_ARolledBackTransaction_ReportsNothing() + { + var transport = new SilentTransport(); + await using var connection = new BsqliteConnection(transport, "silent-rollback.db"); + await connection.OpenAsync(Ct); + + var raised = 0; + connection.TablesChanged += (_, _) => raised++; + + await using (var transaction = await connection.BeginTransactionAsync(Ct)) + { + await ExecuteAsync(connection, "INSERT INTO product (name) VALUES ('ghost')"); + await transaction.RollbackAsync(Ct); + } + + Assert.AreEqual(0, raised); + } + + private async Task ExecuteAsync(DbConnection connection, string sql) + { + await using var command = connection.CreateCommand(); + command.CommandText = sql; + await command.ExecuteNonQueryAsync(Ct); + } + + /// A transport that runs nothing and reports nothing - the default contract. + private sealed class SilentTransport : IBsqliteTransport + { + public Task OpenAsync(string databaseName, CancellationToken cancellationToken = default) + => Task.CompletedTask; + + public Task CloseAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; + + public Task> ExecuteAsync( + IReadOnlyList batch, + CancellationToken cancellationToken = default) + => Task.FromResult>( + [.. batch.Select(_ => new BsqliteCommandResult())]); + + public ValueTask DisposeAsync() => ValueTask.CompletedTask; + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/CacheStorageConformanceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/CacheStorageConformanceTests.cs new file mode 100644 index 00000000000..8f3b6bfc506 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/CacheStorageConformanceTests.cs @@ -0,0 +1,12 @@ +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.CacheStorage; +using Bit.Bsqlite.Storage.ConformanceTests; + +namespace Bit.Bsqlite.Storage.Tests; + +[TestClass] +public sealed class CacheStorageConformanceTests : BsqliteStorageProviderConformanceTests +{ + /// + protected override IBsqliteStorageProvider CreateProvider() => new BsqliteCacheStorageProvider(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/CacheStorageProviderTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/CacheStorageProviderTests.cs new file mode 100644 index 00000000000..650336017f5 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/CacheStorageProviderTests.cs @@ -0,0 +1,37 @@ +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.CacheStorage; + +namespace Bit.Bsqlite.Storage.Tests; + +[TestClass] +public sealed class CacheStorageProviderTests +{ + public TestContext TestContext { get; set; } = null!; + + [TestMethod] + public void Name_IsTheStickyBindingKey() + => Assert.AreEqual("cache-storage", new BsqliteCacheStorageProvider().Name); + + [TestMethod] + public void Capabilities_MatchJournalBasedCacheStorage() + { + var capabilities = new BsqliteCacheStorageProvider().Capabilities; + + Assert.AreEqual(BsqliteEngineBuild.AsyncCapable, capabilities.RequiredBuild); + Assert.IsTrue(capabilities.IsPersistent); + Assert.IsTrue(capabilities.SupportsMultipleConnections); + Assert.IsTrue(capabilities.SupportsConcurrentReads); + Assert.IsFalse(capabilities.SupportsRelaxedDurability); + Assert.IsTrue(capabilities.SupportsMultiDatabaseTransactions); + Assert.IsFalse(capabilities.CanChangePageSize); + } + + [TestMethod] + public async Task Probe_WithoutJavaScript_ReportsUnavailable() + { + var probe = await new BsqliteCacheStorageProvider().ProbeAsync(TestContext.CancellationToken); + + Assert.IsFalse(probe.IsAvailable); + Assert.Contains("browser", probe.UnavailableReason ?? string.Empty, StringComparison.OrdinalIgnoreCase); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/ConformanceKitTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/ConformanceKitTests.cs new file mode 100644 index 00000000000..41cdc3d805f --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/ConformanceKitTests.cs @@ -0,0 +1,107 @@ +using Bit.Bsqlite.Storage.ConformanceTests; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// Tests the conformance kit itself. A suite that only ever runs against a correct provider proves +/// nothing, so each rule is aimed at a backend that breaks exactly that rule and is expected to fail. +/// +[TestClass] +public sealed class ConformanceKitTests +{ + /// + /// Runs the kit's assertions against a chosen provider. Private, so the kit's inherited tests are + /// not collected a second time from here. + /// +#pragma warning disable MSTEST0030 // Deliberately not a [TestClass]: see above. + private sealed class Harness(IBsqliteStorageProvider provider) : BsqliteStorageProviderConformanceTests +#pragma warning restore MSTEST0030 + { + protected override IBsqliteStorageProvider CreateProvider() => provider; + } + + private static async Task RejectsAsync( + IBsqliteStorageProvider provider, + Func rule) + { + var harness = new Harness(provider); + + await using (harness) + { + await Assert.ThrowsAsync(async () => await rule(harness)); + } + } + + private static async Task RejectsAsync( + IBsqliteStorageProvider provider, + Action rule) + => await RejectsAsync(provider, harness => + { + rule(harness); + return Task.CompletedTask; + }); + + /// The correct provider is expected to pass; the rest of this class expects failures. + [TestMethod] + public async Task AcceptsAConformantBackend() + { + var harness = new Harness(new LyingStorageProvider()); + + await using (harness) + { + harness.NonPersistentBackend_DoesNotOfferRelaxedDurability(); + harness.Capabilities_SupportTheDedicatedWorker(); + await harness.Export_ReturnsACopy_NotAViewOfLiveStorage(); + await harness.Import_ReplacesAnExistingDatabaseEntirely(); + await harness.Admin_RejectsABlankDatabaseName(" "); + } + } + + [TestMethod] + public async Task RejectsDurabilityPromisedWithoutPersistence() + => await RejectsAsync( + new LyingStorageProvider { IsPersistent = false, SupportsRelaxedDurability = true }, + harness => harness.NonPersistentBackend_DoesNotOfferRelaxedDurability()); + + [TestMethod] + public async Task RejectsAnAsyncBackendWithNoVfsModule() + => await RejectsAsync( + new LyingStorageProvider { RequiredBuild = BsqliteEngineBuild.AsyncCapable }, + harness => harness.BackendNeedingTheAsyncBuild_ShipsAVfsModule()); + + [TestMethod] + public async Task RejectsABackendThatRunsNowhere() + => await RejectsAsync( + new LyingStorageProvider { SupportedContexts = BsqliteExecutionContexts.None }, + harness => harness.Capabilities_DeclareAtLeastOneExecutionContext()); + + [TestMethod] + public async Task RejectsABackendThatCannotRunInTheWorker() + => await RejectsAsync( + new LyingStorageProvider { SupportedContexts = BsqliteExecutionContexts.Window }, + harness => harness.Capabilities_SupportTheDedicatedWorker()); + + [TestMethod] + public async Task RejectsABlankName() + => await RejectsAsync( + new LyingStorageProvider { Name = " " }, + harness => harness.Name_IsNotBlank()); + + [TestMethod] + public async Task RejectsAnExportThatAliasesLiveStorage() + => await RejectsAsync( + new LyingStorageProvider { ExportsLiveArray = true }, + async harness => await harness.Export_ReturnsACopy_NotAViewOfLiveStorage()); + + [TestMethod] + public async Task RejectsAnImportThatLeavesTheOldTailBehind() + => await RejectsAsync( + new LyingStorageProvider { OverwritesInPlace = true }, + async harness => await harness.Import_ReplacesAnExistingDatabaseEntirely()); + + [TestMethod] + public async Task RejectsAnAdminThatAcceptsABlankName() + => await RejectsAsync( + new LyingStorageProvider { SkipsNameValidation = true }, + async harness => await harness.Admin_RejectsABlankDatabaseName("")); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/FakeStorageProvider.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/FakeStorageProvider.cs new file mode 100644 index 00000000000..310ab41d0d9 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/FakeStorageProvider.cs @@ -0,0 +1,72 @@ +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// A provider whose availability, persistence, and probe behaviour are dictated by the test, so that +/// selection can be driven through browser conditions that are awkward to reproduce for real. +/// +/// +/// Its admin surface answers one question - which databases it holds, from holds - which is +/// all selection asks of it, and only for a database with no binding. Anything more throws. +/// +internal sealed class FakeStorageProvider( + string name, + bool isAvailable = true, + bool isPersistent = true, + Exception? probeThrows = null, + params string[] holds) : IBsqliteStorageProvider +{ + public string Name { get; } = name; + + /// How many times the provider was probed, used to assert the resolver caches. + public int ProbeCount { get; private set; } + + public BsqliteStorageCapabilities Capabilities { get; } = new() + { + RequiredBuild = BsqliteEngineBuild.AsyncCapable, + IsPersistent = isPersistent, + }; + + public BsqliteJsModule? VfsModule => null; + + /// How many times selection asked whether a database is here. + public int ExistsCount { get; private set; } + + public IBsqliteStorageAdmin Admin => new HoldingAdmin(this, holds); + + public ValueTask ProbeAsync(CancellationToken cancellationToken = default) + { + ProbeCount++; + + if (probeThrows is not null) + { + throw probeThrows; + } + + return ValueTask.FromResult(isAvailable + ? BsqliteProbeResult.Available(quotaBytes: 1024 * 1024) + : BsqliteProbeResult.Unavailable($"'{Name}' is switched off for this test.")); + } + + private sealed class HoldingAdmin(FakeStorageProvider owner, string[] holds) : IBsqliteStorageAdmin + { + public ValueTask ExistsAsync(string databaseName, CancellationToken cancellationToken = default) + { + owner.ExistsCount++; + return ValueTask.FromResult(holds.Contains(databaseName, StringComparer.Ordinal)); + } + + public ValueTask> ListAsync(CancellationToken cancellationToken = default) + => throw new NotSupportedException("Selection only asks whether a database exists."); + + public ValueTask DeleteAsync(string databaseName, CancellationToken cancellationToken = default) + => throw new NotSupportedException("Selection only asks whether a database exists."); + + public ValueTask ExportAsync(string databaseName, CancellationToken cancellationToken = default) + => throw new NotSupportedException("Selection only asks whether a database exists."); + + public ValueTask ImportAsync(string databaseName, ReadOnlyMemory contents, CancellationToken cancellationToken = default) + => throw new NotSupportedException("Selection only asks whether a database exists."); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/FirstPartyProviderContractTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/FirstPartyProviderContractTests.cs new file mode 100644 index 00000000000..54adf6e1295 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/FirstPartyProviderContractTests.cs @@ -0,0 +1,81 @@ +using Bit.Bsqlite; +using Bit.Bsqlite.Interop; +using Bit.Bsqlite.Storage.CacheStorage; +using Bit.Bsqlite.Storage.IndexedDb; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Storage.Opfs; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// S8's freeze check: the three first-party providers sit on the same contract without the core +/// knowing their names, and sticky binding still refuses to open an empty database elsewhere. +/// +[TestClass] +public sealed class FirstPartyProviderContractTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public void FirstPartyNames_AreTheOnesSelectionAlreadyUses() + { + Assert.AreEqual("in-memory", new BsqliteInMemoryStorageProvider().Name); + Assert.AreEqual("opfs", new BsqliteOpfsStorageProvider().Name); + Assert.AreEqual("indexeddb", new BsqliteIndexedDbStorageProvider().Name); + Assert.AreEqual("cache-storage", new BsqliteCacheStorageProvider().Name); + } + + [TestMethod] + public void PersistentProviders_DoNotShareAName() + { + Assert.AreNotEqual(new BsqliteOpfsStorageProvider().Name, new BsqliteIndexedDbStorageProvider().Name); + } + + [TestMethod] + public async Task BoundToIndexedDb_DoesNotOpenOpfs_WhenIndexedDbCannotBeProbed() + { + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", "indexeddb", Ct); + + var resolver = new BsqliteStorageProviderResolver( + [new BsqliteOpfsStorageProvider(), new BsqliteIndexedDbStorageProvider()], + store); + + var failure = await Assert.ThrowsExactlyAsync( + () => resolver + .ResolveAsync( + "app.db", + BsqliteStorageSelectionBuilder.Create(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct) + .AsTask()); + + var attempt = Assert.ContainsSingle(failure.Attempts); + Assert.AreEqual("indexeddb", attempt.ProviderName); + Assert.Contains("will not open an empty database", attempt.Explanation!, StringComparison.Ordinal); + } + + [TestMethod] + public async Task BoundToOpfs_DoesNotOpenIndexedDb_WhenOpfsCannotBeProbed() + { + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync("app.db", "opfs", Ct); + + var resolver = new BsqliteStorageProviderResolver( + [new BsqliteOpfsStorageProvider(), new BsqliteIndexedDbStorageProvider()], + store); + + var failure = await Assert.ThrowsExactlyAsync( + () => resolver + .ResolveAsync( + "app.db", + BsqliteStorageSelectionBuilder.Create(s => s.Prefer("indexeddb").Fallback("opfs")), + Ct) + .AsTask()); + + var attempt = Assert.ContainsSingle(failure.Attempts); + Assert.AreEqual("opfs", attempt.ProviderName); + Assert.Contains("will not open an empty database", attempt.Explanation!, StringComparison.Ordinal); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/InMemoryStorageConformanceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/InMemoryStorageConformanceTests.cs new file mode 100644 index 00000000000..bdb4b81e387 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/InMemoryStorageConformanceTests.cs @@ -0,0 +1,15 @@ +using Bit.Bsqlite.Storage.ConformanceTests; +using Bit.Bsqlite.Storage.InMemory; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// Runs the conformance kit against the in-memory backend, which is the contract's reference +/// implementation - so a kit rule the in-memory provider cannot satisfy is a bug in the rule. +/// +[TestClass] +public sealed class InMemoryStorageConformanceTests : BsqliteStorageProviderConformanceTests +{ + /// + protected override IBsqliteStorageProvider CreateProvider() => new BsqliteInMemoryStorageProvider(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/InMemoryStorageProviderTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/InMemoryStorageProviderTests.cs new file mode 100644 index 00000000000..80f17a78e70 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/InMemoryStorageProviderTests.cs @@ -0,0 +1,86 @@ +using Bit.Bsqlite.Storage.InMemory; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// What the in-memory backend promises beyond the contract. The generic rules - round-tripping, +/// replacement, detached exports, idempotent deletes - are checked by +/// , so this file covers only the choices specific to +/// this backend. +/// +[TestClass] +public sealed class InMemoryStorageProviderTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + private static readonly byte[] Image = [0x53, 0x51, 0x4C, 0x69, 0x74, 0x65, 0x00, 0x01]; + + [TestMethod] + public void DeclaresItselfVolatileAndSynchronous() + { + var provider = new BsqliteInMemoryStorageProvider(); + + Assert.AreEqual("in-memory", provider.Name); + Assert.IsFalse(provider.Capabilities.IsPersistent); + Assert.AreEqual(BsqliteEngineBuild.Synchronous, provider.Capabilities.RequiredBuild); + Assert.IsNull(provider.VfsModule); + + // A second worker is a second heap: nothing is shared, so nothing is claimed. + Assert.IsFalse(provider.Capabilities.SupportsMultipleConnections); + Assert.IsFalse(provider.Capabilities.SupportsConcurrentReads); + } + + [TestMethod] + public async Task ProbeIsAlwaysAvailable_AndExplainsItself() + { + var probe = await new BsqliteInMemoryStorageProvider().ProbeAsync(Ct); + + Assert.IsTrue(probe.IsAvailable); + Assert.IsNull(probe.UnavailableReason); + Assert.AreEqual("false", probe.Environment["persistent"]); + } + + /// + /// Ordering is this backend's own guarantee rather than a contract requirement, and it is what + /// makes the diagnostics listing reproducible. + /// + [TestMethod] + public async Task ListReturnsEveryDatabase_InAStableOrder() + { + var admin = new BsqliteInMemoryStorageProvider().Admin; + await admin.ImportAsync("zebra.db", Image, Ct); + await admin.ImportAsync("apple.db", Image, Ct); + + AssertSequence.Equal(["apple.db", "zebra.db"], await admin.ListAsync(Ct)); + } + + /// + /// The contract only requires that exporting an absent database fails; this backend commits to the + /// exception the migration protocol distinguishes a missing source by. + /// + [TestMethod] + public async Task ExportingAnAbsentDatabase_ThrowsFileNotFound() + { + var admin = new BsqliteInMemoryStorageProvider().Admin; + + await Assert.ThrowsExactlyAsync( + () => admin.ExportAsync("never-existed.db", Ct).AsTask()); + } + + /// + /// Each provider instance owns its own databases; sharing them through static state would make + /// two independently configured stores silently interfere. + /// + [TestMethod] + public async Task ProviderInstancesDoNotShareStorage() + { + var first = new BsqliteInMemoryStorageProvider(); + var second = new BsqliteInMemoryStorageProvider(); + + await first.Admin.ImportAsync("app.db", Image, Ct); + + Assert.IsFalse(await second.Admin.ExistsAsync("app.db", Ct)); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/IndexedDbStorageConformanceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/IndexedDbStorageConformanceTests.cs new file mode 100644 index 00000000000..c7273395b30 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/IndexedDbStorageConformanceTests.cs @@ -0,0 +1,16 @@ +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.ConformanceTests; +using Bit.Bsqlite.Storage.IndexedDb; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// Runs the contract-layer kit against the IndexedDB provider. Admin tests skip on desktop because +/// there is no JavaScript runtime - the probe reports unavailable, which is the honest answer. +/// +[TestClass] +public sealed class IndexedDbStorageConformanceTests : BsqliteStorageProviderConformanceTests +{ + /// + protected override IBsqliteStorageProvider CreateProvider() => new BsqliteIndexedDbStorageProvider(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/IndexedDbStorageProviderTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/IndexedDbStorageProviderTests.cs new file mode 100644 index 00000000000..a56e95b9dc5 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/IndexedDbStorageProviderTests.cs @@ -0,0 +1,60 @@ +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.IndexedDb; + +namespace Bit.Bsqlite.Storage.Tests; + +[TestClass] +public sealed class IndexedDbStorageProviderTests +{ + public TestContext TestContext { get; set; } = null!; + + [TestMethod] + public void Name_IsTheStickyBindingKey() + { + Assert.AreEqual("indexeddb", new BsqliteIndexedDbStorageProvider().Name); + Assert.AreEqual(BsqliteIndexedDbStorageProvider.ProviderName, new BsqliteIndexedDbStorageProvider().Name); + } + + [TestMethod] + public void Capabilities_MatchIdbBatchAtomic() + { + var capabilities = new BsqliteIndexedDbStorageProvider().Capabilities; + + Assert.AreEqual(BsqliteEngineBuild.AsyncCapable, capabilities.RequiredBuild); + Assert.IsTrue(capabilities.IsPersistent); + Assert.IsTrue(capabilities.SupportsMultipleConnections); + + // bit-bsqlite-idb-vfs.js registers the VFS without a lock policy, so WebLocksMixin's 'exclusive' + // default applies and reads serialize behind each other like writes do. + Assert.IsFalse(capabilities.SupportsConcurrentReads); + + Assert.IsTrue(capabilities.SupportsRelaxedDurability); + Assert.IsTrue(capabilities.SupportsMultiDatabaseTransactions); + Assert.IsFalse(capabilities.CanChangePageSize); + Assert.IsTrue(capabilities.SupportedContexts.HasFlag(BsqliteExecutionContexts.DedicatedWorker)); + Assert.IsTrue(capabilities.SupportedContexts.HasFlag(BsqliteExecutionContexts.Window)); + } + + [TestMethod] + public void VfsModule_IsDocumentRelative_LikeEveryOtherBlazorAsset() + { + var vfs = new BsqliteIndexedDbStorageProvider().VfsModule; + + Assert.IsNotNull(vfs); + Assert.AreEqual(BsqliteIndexedDbStorageProvider.VfsModuleUrl, vfs.ModuleUrl); + // Relative to the document, not the origin root: an application hosted under a sub-path must + // still find it. The host resolves it against the document base before the worker imports it. + Assert.StartsWith("./_content/", vfs.ModuleUrl, StringComparison.Ordinal); + Assert.AreEqual("register", vfs.RegisterExport); + } + + [TestMethod] + public async Task Probe_WithoutJavaScript_ReportsUnavailable() + { + var probe = await new BsqliteIndexedDbStorageProvider().ProbeAsync(TestContext.CancellationToken); + + Assert.IsFalse(probe.IsAvailable); + Assert.Contains("browser", probe.UnavailableReason ?? string.Empty, StringComparison.OrdinalIgnoreCase); + Assert.AreEqual("false", probe.Environment["javascript"]); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/LyingStorageProvider.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/LyingStorageProvider.cs new file mode 100644 index 00000000000..2e16a8a04f7 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/LyingStorageProvider.cs @@ -0,0 +1,134 @@ +using Bit.Bsqlite.Storage; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// A backend that can be told to break any one rule the conformance kit checks, so that the kit can be +/// shown to catch each violation rather than merely to pass on a correct provider. +/// +internal sealed class LyingStorageProvider : IBsqliteStorageProvider +{ + private readonly LyingStorageAdmin _admin = new(); + + public string Name { get; init; } = "lying"; + + public bool IsPersistent { get; init; } = true; + + public bool SupportsRelaxedDurability { get; init; } + + public BsqliteEngineBuild RequiredBuild { get; init; } = BsqliteEngineBuild.Synchronous; + + public BsqliteExecutionContexts SupportedContexts { get; init; } + = BsqliteExecutionContexts.DedicatedWorker; + + /// Hands out the stored array itself, so callers can scribble on live storage. + public bool ExportsLiveArray + { + get => _admin.ExportsLiveArray; + init => _admin.ExportsLiveArray = value; + } + + /// Writes a smaller image over a larger one, leaving the old tail behind. + public bool OverwritesInPlace + { + get => _admin.OverwritesInPlace; + init => _admin.OverwritesInPlace = value; + } + + /// Accepts blank and null database names instead of rejecting them. + public bool SkipsNameValidation + { + get => _admin.SkipsNameValidation; + init => _admin.SkipsNameValidation = value; + } + + public BsqliteStorageCapabilities Capabilities => new() + { + RequiredBuild = RequiredBuild, + IsPersistent = IsPersistent, + SupportsRelaxedDurability = SupportsRelaxedDurability, + SupportedContexts = SupportedContexts, + }; + + public BsqliteJsModule? VfsModule => null; + + public IBsqliteStorageAdmin Admin => _admin; + + public ValueTask ProbeAsync( + CancellationToken cancellationToken = default) + => ValueTask.FromResult(BsqliteProbeResult.Available()); + + private sealed class LyingStorageAdmin : IBsqliteStorageAdmin + { + private readonly Dictionary _databases = new(StringComparer.Ordinal); + + internal bool ExportsLiveArray { get; set; } + + internal bool OverwritesInPlace { get; set; } + + internal bool SkipsNameValidation { get; set; } + + public ValueTask ExistsAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + Validate(databaseName); + return ValueTask.FromResult(_databases.ContainsKey(databaseName ?? string.Empty)); + } + + public ValueTask> ListAsync( + CancellationToken cancellationToken = default) + => ValueTask.FromResult>([.. _databases.Keys]); + + public ValueTask DeleteAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + Validate(databaseName); + _databases.Remove(databaseName ?? string.Empty); + return ValueTask.CompletedTask; + } + + public ValueTask ExportAsync( + string databaseName, + CancellationToken cancellationToken = default) + { + Validate(databaseName); + + if (!_databases.TryGetValue(databaseName ?? string.Empty, out var image)) + { + throw new FileNotFoundException("No such database.", databaseName); + } + + return ValueTask.FromResult(ExportsLiveArray ? image : image.ToArray()); + } + + public ValueTask ImportAsync( + string databaseName, + ReadOnlyMemory contents, + CancellationToken cancellationToken = default) + { + Validate(databaseName); + var key = databaseName ?? string.Empty; + + if (OverwritesInPlace + && _databases.TryGetValue(key, out var existing) + && existing.Length > contents.Length) + { + contents.Span.CopyTo(existing); + return ValueTask.CompletedTask; + } + + _databases[key] = contents.ToArray(); + return ValueTask.CompletedTask; + } + + private void Validate(string? databaseName) + { + if (!SkipsNameValidation) + { + ArgumentException.ThrowIfNullOrWhiteSpace(databaseName); + } + } + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/OpfsStorageConformanceTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/OpfsStorageConformanceTests.cs new file mode 100644 index 00000000000..715b6d2ebbf --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/OpfsStorageConformanceTests.cs @@ -0,0 +1,17 @@ +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.ConformanceTests; +using Bit.Bsqlite.Storage.Opfs; + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// Runs the contract-layer kit against the OPFS provider. Admin tests skip on desktop because +/// there is no JavaScript runtime - the probe reports unavailable, which is the honest answer. +/// The engine-layer kit and admin operations run in the browser suite. +/// +[TestClass] +public sealed class OpfsStorageConformanceTests : BsqliteStorageProviderConformanceTests +{ + /// + protected override IBsqliteStorageProvider CreateProvider() => new BsqliteOpfsStorageProvider(); +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/OpfsStorageProviderTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/OpfsStorageProviderTests.cs new file mode 100644 index 00000000000..c9e06af10e9 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/OpfsStorageProviderTests.cs @@ -0,0 +1,55 @@ +using Bit.Bsqlite.Storage; +using Bit.Bsqlite.Storage.Opfs; + +namespace Bit.Bsqlite.Storage.Tests; + +[TestClass] +public sealed class OpfsStorageProviderTests +{ + public TestContext TestContext { get; set; } = null!; + + [TestMethod] + public void Name_IsTheStickyBindingKey() + { + Assert.AreEqual("opfs", new BsqliteOpfsStorageProvider().Name); + Assert.AreEqual(BsqliteOpfsStorageProvider.ProviderName, new BsqliteOpfsStorageProvider().Name); + } + + [TestMethod] + public void Capabilities_MatchOpfsCoopSync() + { + var capabilities = new BsqliteOpfsStorageProvider().Capabilities; + + Assert.AreEqual(BsqliteEngineBuild.Synchronous, capabilities.RequiredBuild); + Assert.IsTrue(capabilities.IsPersistent); + Assert.IsTrue(capabilities.SupportsMultipleConnections); + Assert.IsFalse(capabilities.SupportsConcurrentReads); + Assert.IsFalse(capabilities.SupportsRelaxedDurability); + Assert.IsFalse(capabilities.SupportsMultiDatabaseTransactions); + Assert.IsTrue(capabilities.CanChangePageSize); + Assert.AreEqual(BsqliteExecutionContexts.DedicatedWorker, capabilities.SupportedContexts); + } + + [TestMethod] + public void VfsModule_IsDocumentRelative_LikeEveryOtherBlazorAsset() + { + var vfs = new BsqliteOpfsStorageProvider().VfsModule; + + Assert.IsNotNull(vfs); + Assert.AreEqual(BsqliteOpfsStorageProvider.VfsModuleUrl, vfs.ModuleUrl); + // Relative to the document, not the origin root: an application hosted under a sub-path must + // still find it. The host resolves it against the document base before the worker imports it. + Assert.StartsWith("./_content/", vfs.ModuleUrl, StringComparison.Ordinal); + Assert.AreEqual("register", vfs.RegisterExport); + } + + [TestMethod] + public async Task Probe_WithoutJavaScript_ReportsUnavailable() + { + var probe = await new BsqliteOpfsStorageProvider().ProbeAsync(TestContext.CancellationToken); + + Assert.IsFalse(probe.IsAvailable); + Assert.Contains("browser", probe.UnavailableReason ?? string.Empty, StringComparison.OrdinalIgnoreCase); + Assert.AreEqual("false", probe.Environment["javascript"]); + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/StorageProviderResolverTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/StorageProviderResolverTests.cs new file mode 100644 index 00000000000..55f76c52582 --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/StorageProviderResolverTests.cs @@ -0,0 +1,440 @@ +using Bit.Bsqlite.Interop; + +namespace Bit.Bsqlite.Storage.Tests; + +[TestClass] +public sealed class StorageProviderResolverTests +{ + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + private const string Database = "app.db"; + + private static BsqliteStorageSelection Select( + Action configure) + => BsqliteStorageSelectionBuilder.Create(configure); + + private static BsqliteStorageProviderResolver Resolver( + IBsqliteStorageBindingStore bindingStore, + params IBsqliteStorageProvider[] providers) + => new(providers, bindingStore); + + [TestMethod] + public async Task SingleAvailableProvider_IsSelected() + { + var opfs = new FakeStorageProvider("opfs"); + var resolver = Resolver(new BsqliteInMemoryStorageBindingStore(), opfs); + + var resolution = await resolver.ResolveAsync(Database, Select(s => s.Prefer("opfs")), Ct); + + Assert.AreSame(opfs, resolution.Provider); + Assert.IsTrue(resolution.IsFirstChoice); + Assert.IsFalse(resolution.WasDecidedByExistingData); + Assert.IsNull(resolution.BetterProviderAvailable); + } + + /// + /// A lone candidate is a strict instruction. Substituting something else would mean the + /// application silently got storage it never asked for. + /// + [TestMethod] + public async Task SingleUnavailableProvider_Throws_WithoutSubstituting() + { + var resolver = Resolver( + new BsqliteInMemoryStorageBindingStore(), + new FakeStorageProvider("opfs", isAvailable: false), + new FakeStorageProvider("indexeddb")); + + var failure = await Assert.ThrowsExactlyAsync( + () => resolver.ResolveAsync(Database, Select(s => s.Prefer("opfs")), Ct).AsTask()); + + Assert.AreEqual(Database, failure.DatabaseName); + var attempt = Assert.ContainsSingle(failure.Attempts); + Assert.AreEqual(BsqliteStorageCandidateStatus.Unavailable, attempt.Status); + Assert.Contains("switched off", attempt.Probe!.UnavailableReason!, StringComparison.Ordinal); + } + + [TestMethod] + public async Task FallbackIsUsed_WhenPreferredIsUnavailable() + { + var indexedDb = new FakeStorageProvider("indexeddb"); + var resolver = Resolver( + new BsqliteInMemoryStorageBindingStore(), + new FakeStorageProvider("opfs", isAvailable: false), + indexedDb); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct); + + Assert.AreSame(indexedDb, resolution.Provider); + Assert.IsFalse(resolution.IsFirstChoice); + CollectionAssert.AreEqual( + new[] { BsqliteStorageCandidateStatus.Unavailable, BsqliteStorageCandidateStatus.Selected }, + resolution.Attempts.Select(a => a.Status).ToArray()); + } + + [TestMethod] + public async Task UnregisteredCandidate_IsReported_AndSkipped() + { + var indexedDb = new FakeStorageProvider("indexeddb"); + var resolver = Resolver(new BsqliteInMemoryStorageBindingStore(), indexedDb); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct); + + Assert.AreSame(indexedDb, resolution.Provider); + Assert.AreEqual(BsqliteStorageCandidateStatus.NotRegistered, resolution.Attempts[0].Status); + } + + [TestMethod] + public async Task AllCandidatesFail_ReportsEveryReason() + { + var resolver = Resolver( + new BsqliteInMemoryStorageBindingStore(), + new FakeStorageProvider("opfs", isAvailable: false), + new FakeStorageProvider("indexeddb", isAvailable: false)); + + var failure = await Assert.ThrowsExactlyAsync( + () => resolver + .ResolveAsync(Database, Select(s => s.Prefer("opfs").Fallback("indexeddb")), Ct) + .AsTask()); + + Assert.AreEqual(2, failure.Attempts.Count); + foreach (var attempt in failure.Attempts) + { + Assert.AreEqual(BsqliteStorageCandidateStatus.Unavailable, attempt.Status); + } + Assert.Contains("opfs", failure.Message, StringComparison.Ordinal); + Assert.Contains("indexeddb", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public async Task ProviderThatThrowsWhileProbing_IsTreatedAsUnavailable() + { + var indexedDb = new FakeStorageProvider("indexeddb"); + var resolver = Resolver( + new BsqliteInMemoryStorageBindingStore(), + new FakeStorageProvider("opfs", probeThrows: new InvalidOperationException("boom")), + indexedDb); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct); + + Assert.AreSame(indexedDb, resolution.Provider); + Assert.Contains("boom", resolution.Attempts[0].Probe!.UnavailableReason!, StringComparison.Ordinal); + } + + [TestMethod] + public void DuplicateProviderNames_AreRejected() + { + var failure = Assert.ThrowsExactly( + () => Resolver( + new BsqliteInMemoryStorageBindingStore(), + new FakeStorageProvider("opfs"), + new FakeStorageProvider("OPFS"))); + + Assert.Contains("must be unique", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public async Task ProbeResultIsCached_AcrossResolves() + { + var opfs = new FakeStorageProvider("opfs"); + var resolver = Resolver(new BsqliteInMemoryStorageBindingStore(), opfs); + var selection = Select(s => s.Prefer("opfs")); + + await resolver.ResolveAsync("one.db", selection, Ct); + await resolver.ResolveAsync("two.db", selection, Ct); + + Assert.AreEqual(1, opfs.ProbeCount); + } + + [TestClass] + public sealed class NonPersistentFallback + { + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task IsRejected_WithoutOptIn() + { + var resolver = Resolver( + new BsqliteInMemoryStorageBindingStore(), + new FakeStorageProvider("opfs", isAvailable: false), + new FakeStorageProvider("in-memory", isPersistent: false)); + + var failure = await Assert.ThrowsExactlyAsync( + () => resolver + .ResolveAsync(Database, Select(s => s.Prefer("opfs").Fallback("in-memory")), Ct) + .AsTask()); + + Assert.AreEqual( + BsqliteStorageCandidateStatus.RejectedAsNonPersistent, + failure.Attempts[1].Status); + Assert.Contains( + "AllowNonPersistentFallback", + failure.Attempts[1].Explanation!, + StringComparison.Ordinal); + } + + [TestMethod] + public async Task IsAccepted_WithOptIn() + { + var inMemory = new FakeStorageProvider("in-memory", isPersistent: false); + var resolver = Resolver( + new BsqliteInMemoryStorageBindingStore(), + new FakeStorageProvider("opfs", isAvailable: false), + inMemory); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("in-memory").AllowNonPersistentFallback()), + Ct); + + Assert.AreSame(inMemory, resolution.Provider); + } + + /// + /// Choosing volatile storage outright is a decision the application already made, so it needs + /// no second opt-in - the guard exists for silent downgrades, not deliberate ones. + /// + [TestMethod] + public async Task IsAllowed_AsAnExplicitFirstChoice() + { + var inMemory = new FakeStorageProvider("in-memory", isPersistent: false); + var resolver = Resolver(new BsqliteInMemoryStorageBindingStore(), inMemory); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("in-memory")), + Ct); + + Assert.AreSame(inMemory, resolution.Provider); + } + } + + [TestClass] + public sealed class StickyBinding + { + public TestContext TestContext { get; set; } = null!; + + private CancellationToken Ct => TestContext.CancellationToken; + + [TestMethod] + public async Task ExistingData_OutranksPreference() + { + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync(Database, "indexeddb", Ct); + + var opfs = new FakeStorageProvider("opfs"); + var indexedDb = new FakeStorageProvider("indexeddb"); + var resolver = Resolver(store, opfs, indexedDb); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct); + + Assert.AreSame(indexedDb, resolution.Provider); + Assert.IsTrue(resolution.WasDecidedByExistingData); + Assert.IsFalse(resolution.IsFirstChoice); + Assert.AreSame(opfs, resolution.BetterProviderAvailable); + } + + /// + /// The failure this whole mechanism exists to prevent: OPFS becomes available, preference says + /// OPFS, and the user's IndexedDB database would be replaced by an empty one with no error. + /// + [TestMethod] + public async Task UnreachableExistingData_Throws_RatherThanOpeningAnEmptyDatabase() + { + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync(Database, "indexeddb", Ct); + + var resolver = Resolver( + store, + new FakeStorageProvider("opfs"), + new FakeStorageProvider("indexeddb", isAvailable: false)); + + var failure = await Assert.ThrowsExactlyAsync( + () => resolver + .ResolveAsync(Database, Select(s => s.Prefer("opfs").Fallback("indexeddb")), Ct) + .AsTask()); + + var attempt = Assert.ContainsSingle(failure.Attempts); + Assert.AreEqual("indexeddb", attempt.ProviderName); + Assert.Contains("will not open an empty database", attempt.Explanation!, StringComparison.Ordinal); + } + + [TestMethod] + public async Task ExistingDataOnUnregisteredProvider_ThrowsWithActionableMessage() + { + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync(Database, "cache-storage", Ct); + + var resolver = Resolver(store, new FakeStorageProvider("opfs")); + + var failure = await Assert.ThrowsExactlyAsync( + () => resolver.ResolveAsync(Database, Select(s => s.Prefer("opfs")), Ct).AsTask()); + + var attempt = Assert.ContainsSingle(failure.Attempts); + Assert.AreEqual(BsqliteStorageCandidateStatus.NotRegistered, attempt.Status); + Assert.Contains("Register it", attempt.Explanation!, StringComparison.Ordinal); + } + + [TestMethod] + public async Task NoBetterProviderIsReported_WhenBoundProviderIsTheFirstChoice() + { + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync(Database, "opfs", Ct); + + var resolver = Resolver( + store, + new FakeStorageProvider("opfs"), + new FakeStorageProvider("indexeddb")); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct); + + Assert.IsTrue(resolution.IsFirstChoice); + Assert.IsNull(resolution.BetterProviderAvailable); + } + + /// + /// Moving a database onto volatile storage is never an upgrade, however the preference list is + /// ordered. + /// + [TestMethod] + public async Task NonPersistentProviderIsNeverReportedAsBetter() + { + var store = new BsqliteInMemoryStorageBindingStore(); + await store.SetProviderNameAsync(Database, "indexeddb", Ct); + + var resolver = Resolver( + store, + new FakeStorageProvider("in-memory", isPersistent: false), + new FakeStorageProvider("indexeddb")); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("in-memory").Fallback("indexeddb")), + Ct); + + Assert.IsNull(resolution.BetterProviderAvailable); + } + + [TestMethod] + public async Task CommitBinding_MakesTheChoiceStick() + { + var store = new BsqliteInMemoryStorageBindingStore(); + var indexedDb = new FakeStorageProvider("indexeddb"); + var resolver = Resolver(store, new FakeStorageProvider("opfs", isAvailable: false), indexedDb); + var selection = Select(s => s.Prefer("opfs").Fallback("indexeddb")); + + var first = await resolver.ResolveAsync(Database, selection, Ct); + await resolver.CommitBindingAsync(first, Ct); + + Assert.AreEqual("indexeddb", await store.GetProviderNameAsync(Database, Ct)); + + var second = await resolver.ResolveAsync(Database, selection, Ct); + Assert.AreSame(indexedDb, second.Provider); + Assert.IsTrue(second.WasDecidedByExistingData); + } + + /// + /// A database on a non-persistent fallback is gone by the next load, so a binding protects + /// nothing - and would keep every later visit on in-memory even with OPFS back. + /// + [TestMethod] + public async Task CommitBinding_DoesNotBindANonPersistentBackend() + { + var store = new BsqliteInMemoryStorageBindingStore(); + var opfs = new FakeStorageProvider("opfs", isAvailable: false); + var resolver = Resolver(store, opfs, new FakeStorageProvider("in-memory", isPersistent: false)); + var selection = Select(s => s.Prefer("opfs").Fallback("in-memory").AllowNonPersistentFallback()); + + var first = await resolver.ResolveAsync(Database, selection, Ct); + Assert.AreEqual("in-memory", first.Provider.Name); + await resolver.CommitBindingAsync(first, Ct); + + Assert.IsNull(await store.GetProviderNameAsync(Database, Ct)); + + // The next visit, with OPFS working again, picks it. + var nextVisit = Resolver(store, new FakeStorageProvider("opfs"), new FakeStorageProvider("in-memory", isPersistent: false)); + Assert.AreEqual("opfs", (await nextVisit.ResolveAsync(Database, selection, Ct)).Provider.Name); + } + + /// + /// An application that clears localStorage on sign-out loses the binding but not the + /// file. Opening an empty database on the preferred backend would orphan the data without a + /// word, so the backends are asked, and the one holding it is used - and bound again. + /// + [TestMethod] + public async Task LostBinding_FindsTheExistingData_AndBindsItAgain() + { + var store = new BsqliteInMemoryStorageBindingStore(); + var opfs = new FakeStorageProvider("opfs"); + var indexedDb = new FakeStorageProvider("indexeddb", holds: [Database]); + var resolver = Resolver(store, opfs, indexedDb); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct); + + Assert.AreSame(indexedDb, resolution.Provider); + Assert.IsTrue(resolution.WasDecidedByExistingData); + Assert.IsTrue(resolution.WasFoundWithoutBinding); + Assert.AreSame(opfs, resolution.BetterProviderAvailable); + Assert.IsNull(await store.GetProviderNameAsync(Database, Ct), "Resolving records nothing."); + + await resolver.CommitBindingAsync(resolution, Ct); + Assert.AreEqual("indexeddb", await store.GetProviderNameAsync(Database, Ct)); + } + + [TestMethod] + public async Task LostBinding_WithNothingAnywhere_IsANewDatabase() + { + var store = new BsqliteInMemoryStorageBindingStore(); + var opfs = new FakeStorageProvider("opfs"); + var indexedDb = new FakeStorageProvider("indexeddb"); + var resolver = Resolver(store, opfs, indexedDb); + + var resolution = await resolver.ResolveAsync( + Database, + Select(s => s.Prefer("opfs").Fallback("indexeddb")), + Ct); + + Assert.AreSame(opfs, resolution.Provider); + Assert.IsFalse(resolution.WasDecidedByExistingData); + Assert.IsFalse(resolution.WasFoundWithoutBinding); + Assert.AreEqual(1, opfs.ExistsCount); + Assert.AreEqual(1, indexedDb.ExistsCount); + } + + /// + /// Resolving is a read-only decision; a database that never opened must not leave a binding + /// behind claiming it did. + /// + [TestMethod] + public async Task Resolve_DoesNotRecordAnything_UntilCommitted() + { + var store = new BsqliteInMemoryStorageBindingStore(); + var resolver = Resolver(store, new FakeStorageProvider("opfs")); + + await resolver.ResolveAsync(Database, Select(s => s.Prefer("opfs")), Ct); + + Assert.IsNull(await store.GetProviderNameAsync(Database, Ct)); + } + } +} diff --git a/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/StorageSelectionBuilderTests.cs b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/StorageSelectionBuilderTests.cs new file mode 100644 index 00000000000..f8d9580830d --- /dev/null +++ b/src/Bsqlite/Tests/Bit.Bsqlite.Tests/Storage/StorageSelectionBuilderTests.cs @@ -0,0 +1,83 @@ + +namespace Bit.Bsqlite.Storage.Tests; + +/// +/// Configuration mistakes here are silent in production - the wrong backend or the wrong order - so +/// the builder rejects them at the point they are written instead. +/// +[TestClass] +public sealed class StorageSelectionBuilderTests +{ + [TestMethod] + public void CandidateOrder_IsThePreferenceOrder() + { + var selection = BsqliteStorageSelectionBuilder.Create( + s => s.Prefer("opfs").Fallback("indexeddb").Fallback("cache-storage")); + + AssertSequence.Equal(["opfs", "indexeddb", "cache-storage"], selection.Candidates); + Assert.IsFalse(selection.IsStrict); + } + + [TestMethod] + public void SingleCandidate_IsStrict() + { + var selection = BsqliteStorageSelectionBuilder.Create(s => s.Prefer("opfs")); + + Assert.IsTrue(selection.IsStrict); + } + + [TestMethod] + public void Defaults_AreTheConservativeChoices() + { + var selection = BsqliteStorageSelectionBuilder.Create(s => s.Prefer("opfs")); + + Assert.IsFalse(selection.AllowNonPersistentFallback); + Assert.AreEqual(BsqliteStorageMigrationMode.KeepExisting, selection.MigrationMode); + } + + [TestMethod] + public void PreferringTwice_PointsAtFallback() + { + var failure = Assert.ThrowsExactly( + () => BsqliteStorageSelectionBuilder.Create(s => s.Prefer("opfs").Prefer("indexeddb"))); + + Assert.Contains("Fallback(\"indexeddb\")", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public void FallbackWithoutPrefer_PointsAtPrefer() + { + var failure = Assert.ThrowsExactly( + () => BsqliteStorageSelectionBuilder.Create(s => s.Fallback("opfs"))); + + Assert.Contains("Prefer(\"opfs\")", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public void RepeatingACandidate_IsRejected() + { + var failure = Assert.ThrowsExactly( + () => BsqliteStorageSelectionBuilder.Create( + s => s.Prefer("opfs").Fallback("OPFS"))); + + Assert.Contains("already a candidate", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public void SelectingNothing_IsRejected() + { + var failure = Assert.ThrowsExactly( + () => BsqliteStorageSelectionBuilder.Create(_ => { })); + + Assert.Contains("No storage provider was selected", failure.Message, StringComparison.Ordinal); + } + + [TestMethod] + public void MigrationMode_IsCarriedThrough() + { + var selection = BsqliteStorageSelectionBuilder.Create( + s => s.Prefer("opfs").WithMigrationMode(BsqliteStorageMigrationMode.AutomaticOnOpen)); + + Assert.AreEqual(BsqliteStorageMigrationMode.AutomaticOnOpen, selection.MigrationMode); + } +} diff --git a/src/Bsqlite/Tests/bit-bsqlite-js/decimal.test.js b/src/Bsqlite/Tests/bit-bsqlite-js/decimal.test.js new file mode 100644 index 00000000000..013bd38dcce --- /dev/null +++ b/src/Bsqlite/Tests/bit-bsqlite-js/decimal.test.js @@ -0,0 +1,119 @@ +import { test, expect } from 'vitest'; +import { Decimal } from '../../Bit.Bsqlite/Scripts/bit-bsqlite-decimal.ts'; + +const values = ['0', '10', '9', '-5', '2.5', '0.1', '100', '-0.25', '1234.5678', '3']; +const optional = [null, '10', null, '-5', '2.5', null, '100', '-0.25', '1234.5678', '3']; + +const parse = text => Decimal.parse(text); +const map = (xs, fn) => xs.map(x => (x === null ? null : fn(parse(x)).toString())); + +test('ToString keeps the scale, including trailing zeros', () => { + expect(parse('2.5').add(parse('1.5')).toString()).toBe('4.0'); + expect(parse('0').add(parse('1.5')).toString()).toBe('1.5'); + expect(parse('0.1').multiply(parse('10')).toString()).toBe('1.0'); + expect(parse('9.0').toString()).toBe('9.0'); + expect(parse('9').toString()).toBe('9'); + expect(parse('.5').toString()).toBe('0.5'); + expect(parse('5.').toString()).toBe('5'); + expect(parse('-0').toString()).toBe('0'); +}); + +// The oracle is Microsoft.Data.Sqlite's `0.0###########################`, which is the form EF +// renders decimal literals in and therefore the only form its `=` comparisons match. +test('toSqlText writes the canonical form the SQLite stack stores', () => { + expect(parse('10').toSqlText()).toBe('10.0'); + expect(parse('1.50').toSqlText()).toBe('1.5'); + expect(parse('100.000').toSqlText()).toBe('100.0'); + expect(parse('0').toSqlText()).toBe('0.0'); + expect(parse('-3').toSqlText()).toBe('-3.0'); + expect(parse('12.34').toSqlText()).toBe('12.34'); + expect(parse('-0.25').toSqlText()).toBe('-0.25'); + expect(parse('1').divide(parse('3')).toSqlText()).toBe('0.3333333333333333333333333333'); + expect(parse('2.5').add(parse('1.5')).toSqlText()).toBe('4.0'); +}); + +test('add matches the S4 oracle', () => { + expect(map(values, v => v.add(parse('1.5')))).toEqual(['1.5', '11.5', '10.5', '-3.5', '4.0', '1.6', '101.5', '1.25', '1236.0678', '4.5']); +}); + +test('subtract matches the S4 oracle', () => { + expect(map(values, v => v.subtract(parse('1.5')))).toEqual(['-1.5', '8.5', '7.5', '-6.5', '1.0', '-1.4', '98.5', '-1.75', '1233.0678', '1.5']); +}); + +test('multiply matches the S4 oracle', () => { + expect(map(values, v => v.multiply(parse('3')))).toEqual(['0', '30', '27', '-15', '7.5', '0.3', '300', '-0.75', '3703.7034', '9']); +}); + +// When a product has more than 28 decimals, every surplus digit is dropped at once and the whole +// remainder decides the rounding, as System.Decimal does. Rounding digit by digit gets `…51` wrong: +// 1 is below half, then 5 ties to even, and 0.51 rounds down instead of up. Oracle: .NET 10. +test('multiply drops surplus digits with one rounding, not one per digit', () => { + const tiny = parse('0.0000000000000000000000000001'); + const cases = [ + ['0.51', '0.0000000000000000000000000001'], + ['0.49', '0.0000000000000000000000000000'], + ['1.50', '0.0000000000000000000000000002'], + ['2.50', '0.0000000000000000000000000002'], + ]; + expect(cases.map(([factor]) => tiny.multiply(parse(factor)).toString())).toEqual(cases.map(([, expected]) => expected)); + + const three = parse('0.0000000000000000000000000003'); + expect(three.multiply(parse('1.51')).toString()).toBe('0.0000000000000000000000000005'); + expect(three.multiply(parse('1.50')).toString()).toBe('0.0000000000000000000000000004'); +}); + +test('divide matches the S4 oracle', () => { + expect(map(values, v => v.divide(parse('4')))).toEqual(['0', '2.5', '2.25', '-1.25', '0.625', '0.025', '25', '-0.0625', '308.64195', '0.75']); +}); + +test('remainder matches the S4 oracle, including the sign of the dividend', () => { + expect(map(values, v => v.remainder(parse('3')))).toEqual(['0', '1', '0', '-2', '2.5', '0.1', '1', '-0.25', '1.5678', '0']); +}); + +test('negate matches the S4 oracle and does not produce -0', () => { + expect(map(values, v => v.negate())).toEqual(['0', '-10', '-9', '5', '-2.5', '-0.1', '-100', '0.25', '-1234.5678', '-3']); +}); + +test('1/3 is twenty-eight threes, not an IEEE float', () => { + expect(parse('1').divide(parse('3')).toString()).toBe('0.3333333333333333333333333333'); +}); + +test('10 / 0.04 is exact', () => { + expect(parse('10').divide(parse('0.04')).toString()).toBe('250'); +}); + +test('aggregates match the S4 oracle', () => { + const sum = values.map(parse).reduce((a, b) => a.add(b)); + expect(sum.toString()).toBe('1353.9178'); + expect(sum.divide(parse('10')).toString()).toBe('135.39178'); + + const present = optional.filter(v => v !== null).map(parse); + const sumOpt = present.reduce((a, b) => a.add(b)); + expect(sumOpt.toString()).toBe('1344.8178'); + expect(sumOpt.divide(new Decimal(1, BigInt(present.length), 0)).toString()).toBe('192.11682857142857142857142857'); + + const bucket0 = ['0', '10', '9', '-5'].map(parse).reduce((a, b) => a.add(b)); + const bucket1 = ['2.5', '0.1', '100', '-0.25', '1234.5678', '3'].map(parse).reduce((a, b) => a.add(b)); + expect(bucket0.toString()).toBe('14'); + expect(bucket0.divide(parse('4')).toString()).toBe('3.5'); + expect(bucket1.toString()).toBe('1339.9178'); + expect(bucket1.divide(parse('6')).toString()).toBe('223.31963333333333333333333333'); +}); + +test('compare is numeric, so 9 is less than 10 and equal to 9.0', () => { + expect(Decimal.compare(parse('9'), parse('10'))).toBe(-1); + expect(Decimal.compare(parse('10'), parse('9'))).toBe(1); + expect(Decimal.compare(parse('9'), parse('9.0'))).toBe(0); + expect(Decimal.compare(parse('2.5'), parse('100'))).toBe(-1); + expect(Decimal.compare(parse('-5'), parse('-0.25'))).toBe(-1); + expect(Decimal.compare(parse('0'), parse('-0'))).toBe(0); +}); + +test('order of the S4 values is numeric, not lexicographic', () => { + const ids = values + .map((text, i) => ({ id: i + 1, value: parse(text) })) + .sort((a, b) => a.value.compareTo(b.value) || a.id - b.id) + .map(row => row.id); + + expect(ids).toEqual([4, 8, 1, 6, 5, 10, 3, 2, 7, 9]); +}); diff --git a/src/Bsqlite/Tests/bit-bsqlite-js/host-listen.test.js b/src/Bsqlite/Tests/bit-bsqlite-js/host-listen.test.js new file mode 100644 index 00000000000..eb2036e7842 --- /dev/null +++ b/src/Bsqlite/Tests/bit-bsqlite-js/host-listen.test.js @@ -0,0 +1,89 @@ +import { test, expect } from 'vitest'; +import { listen } from '../../Bit.Bsqlite/Scripts/bit-bsqlite-host.ts'; + +// `listen` is the seam between the host's notifications and the .NET transport. It cannot be +// exercised from the browser suite, which has no .NET side, so its filtering is pinned here. +// A host is only `onNotify` as far as this function is concerned. +function fakeHost() { + const listeners = new Set(); + return { + onNotify(listener) { + listeners.add(listener); + return () => listeners.delete(listener); + }, + emit(payload, info) { + for (const listener of listeners) { + listener(payload, info); + } + }, + }; +} + +function fakeTarget() { + const calls = []; + return { + calls, + invokeMethodAsync(method, ...args) { + calls.push({ method, args }); + return Promise.resolve(); + }, + }; +} + +const write = (databaseName, tables) => ({ kind: 'notify', databaseName, tables }); + +test('another tab\'s write reaches .NET', () => { + const host = fakeHost(); + const target = fakeTarget(); + listen(host, target, 'app.db'); + + host.emit(write('app.db', ['product']), { local: false }); + + expect(target.calls).toEqual([{ method: 'OnTablesChanged', args: [['product']] }]); +}); + +// The worker is the source of truth for this tab's own writes too: it knows the exact tables from +// SQLite's update hook and reports them only once committed. The .NET transport declares +// ReportsLocalWrites so the command layer does not raise the same write again from the SQL text. +test("this tab's own write is relayed as well", () => { + const host = fakeHost(); + const target = fakeTarget(); + listen(host, target, 'app.db'); + + host.emit(write('app.db', ['product']), { local: true }); + + expect(target.calls).toEqual([{ method: 'OnTablesChanged', args: [['product']] }]); +}); + +// The broadcast channel is origin-wide, so it carries every database's traffic. +test('a write to a different database is not relayed', () => { + const host = fakeHost(); + const target = fakeTarget(); + listen(host, target, 'app.db'); + + host.emit(write('other.db', ['product']), { local: false }); + + expect(target.calls).toEqual([]); +}); + +test('the method name is overridable and unsubscribing stops the relay', () => { + const host = fakeHost(); + const target = fakeTarget(); + const stop = listen(host, target, 'app.db', 'Custom'); + + host.emit(write('app.db', ['product']), { local: false }); + stop(); + host.emit(write('app.db', ['customer']), { local: false }); + + expect(target.calls).toEqual([{ method: 'Custom', args: [['product']] }]); +}); + +test('a notification with no tables still calls through with an empty list', () => { + const host = fakeHost(); + const target = fakeTarget(); + listen(host, target, 'app.db'); + + host.emit({ kind: 'notify', databaseName: 'app.db' }, { local: false }); + + expect(target.calls).toEqual([{ method: 'OnTablesChanged', args: [[]] }]); +}); diff --git a/src/Bsqlite/Tests/bit-bsqlite-js/package-lock.json b/src/Bsqlite/Tests/bit-bsqlite-js/package-lock.json new file mode 100644 index 00000000000..af2f3e8ebe6 --- /dev/null +++ b/src/Bsqlite/Tests/bit-bsqlite-js/package-lock.json @@ -0,0 +1,1645 @@ +{ + "name": "bit-bsqlite-js-tests", + "version": "1.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "bit-bsqlite-js-tests", + "version": "1.0.0", + "devDependencies": { + "vitest": "^3.2.4" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz", + "integrity": "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.2.tgz", + "integrity": "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.2.tgz", + "integrity": "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.2.tgz", + "integrity": "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.2.tgz", + "integrity": "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.2.tgz", + "integrity": "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.2.tgz", + "integrity": "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.2.tgz", + "integrity": "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.2.tgz", + "integrity": "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.2.tgz", + "integrity": "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.2.tgz", + "integrity": "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.2.tgz", + "integrity": "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.2.tgz", + "integrity": "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.2.tgz", + "integrity": "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.2.tgz", + "integrity": "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.2.tgz", + "integrity": "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.2.tgz", + "integrity": "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.2.tgz", + "integrity": "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.2.tgz", + "integrity": "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.2.tgz", + "integrity": "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.2.tgz", + "integrity": "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.2.tgz", + "integrity": "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.2.tgz", + "integrity": "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.2.tgz", + "integrity": "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.2.tgz", + "integrity": "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.2.tgz", + "integrity": "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@jridgewell/sourcemap-codec": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.6.0.tgz", + "integrity": "sha512-T7jf+5zgsZHwNJ4lvQ7/aezbyk0nNX+zJVWpmHA7VYsEx7a7qr5Rg5IbtJFqkgze5Y2sruq1RUY8Q837Od7iFw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@napi-rs/lzma-linux-x64-gnu": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/@napi-rs/lzma-linux-x64-gnu/-/lzma-linux-x64-gnu-1.5.1.tgz", + "integrity": "sha512-oTXEIha4SsuXdTA4Iyskj0kpdx2yVXdhd75c2v3xGrHFfVMsbhTPZU/nMPL4sWKo4pBHm3aucLaqGlF696dTyQ==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^22.20 || ^24.12 || >=25" + } + }, + "node_modules/@rollup/rollup-android-arm-eabi": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.64.9.tgz", + "integrity": "sha512-qJmhckvOleXp8R+BaXiVW/S34/3M7pvN17/QcG90A74BDaSvKv8ffY+GYG+a/xU0vEoITzdMSiM7o9HZSQWbfA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-android-arm64": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.64.9.tgz", + "integrity": "sha512-dqQgds39BNWGB2ughJ6n2LTQFXk8RAOwnXj6m387OHpygfD93QRS1ODIfpiWyAa9gsFB9l34pSqeLmA0Xk/VBw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-darwin-arm64": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.64.9.tgz", + "integrity": "sha512-scrMx4SGikOSbWknLmPD0v6OeJ1Wih3kyRLdNZnlVbBeKVWp1nRTVcpM0IxDJdc3u8UjesPHTmnX649u8cQpBw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-darwin-x64": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.64.9.tgz", + "integrity": "sha512-fkwi2OQR+QWoViRFt08Bsca6G0e9pk/4z+JgmqHq6jSRTT0LtI27KRVK8bu++6L/CBtQ3OZePbqC3bcrwGzTqQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-freebsd-arm64": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.64.9.tgz", + "integrity": "sha512-8Apr0Khn0r2MUsDtcOMbp85fDdXsX/E+iLhO17WUjG24m3O8ChArqI0isI7CAdj9Ouq177PHtVtEUpMNE1C4Gw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-freebsd-x64": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.64.9.tgz", + "integrity": "sha512-H+upy4DhBcBZlUsCAMyr8wR0Dg9kj8sGloVZdZ7va2heJM/5jnROxI47dtXseJjlD/VoQe6MrXhY3b5ikOq/Vw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-linux-arm-gnueabihf": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.64.9.tgz", + "integrity": "sha512-b50NS9TbAEK/b16vBoKrkeNwy+HvGBXmtBbzX4wcodWOh9dQZvRTN9rUyElIXa36QnwXhU9PAVVkZpqRzSi2Tg==", + "cpu": [ + "arm" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm-musleabihf": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.64.9.tgz", + "integrity": "sha512-SdqcudB/FzjLESsRMgXhV+FXGtf6usxXwlMoRW7D/XS3f9CZ4T4wycRBu2h9bVl2AOaPpunoA8ftAM/MF4altQ==", + "cpu": [ + "arm" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-gnu": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.64.9.tgz", + "integrity": "sha512-dg19FFRoovN+KZ+gMHBAq4Do8PgQmBcxpuhux57OuiR0QLavfn8uPBY8CP+UO50bO+xf0tK0GlqI/jld++5E6g==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-musl": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.64.9.tgz", + "integrity": "sha512-+jyv5ofGmpAvtrzLjSr/e4syskeikEp5L82bUzB76auwOgK1VNVwZV/F2g1Pes9bhTznbejOefAqPglHSKYmTA==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-loong64-gnu": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loong64-gnu/-/rollup-linux-loong64-gnu-4.64.9.tgz", + "integrity": "sha512-MKFXxqmkzCbtNvx/vjiJdss44eRtVabhCtDWcCFD9NHGwP+YkqnM/slnEsv/3fsoq9ukTtudraAjGhptrUPCVg==", + "cpu": [ + "loong64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-loong64-musl": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loong64-musl/-/rollup-linux-loong64-musl-4.64.9.tgz", + "integrity": "sha512-lhARcOT6n7Ni9hpQRq8LZjbuoW+G/GXaW4Jfhx2MbmjnCj9XxLMRTceJIMpiMLQVWm+qxe7oq1sYvlIGOAW34Q==", + "cpu": [ + "loong64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-ppc64-gnu": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-ppc64-gnu/-/rollup-linux-ppc64-gnu-4.64.9.tgz", + "integrity": "sha512-oORQzGxZ1oHghJwLdwc8Ni0SY9otILAeXsWdiImTF4F5HkWJUwh5s/269PF5tYILNC1z/3RwFsrgcHPguBcb0g==", + "cpu": [ + "ppc64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-ppc64-musl": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-ppc64-musl/-/rollup-linux-ppc64-musl-4.64.9.tgz", + "integrity": "sha512-J2JQHQt7oR1uutd2OSwbV2Qpb2LI5gggPolbkUwDe7vn/0dnduZ0w6EIbGnZUAw7uIKaMgfVylcH6V+Aov8kRA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-gnu": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.64.9.tgz", + "integrity": "sha512-bWaKLrfzeQ9mOzJJxXbPgw2RH6VAaTJrb/60Tqf7YSMxubG8780Hzm7Us+D6qVW8QRGJYl4VUvVXxnRB0o9zvQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-musl": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-musl/-/rollup-linux-riscv64-musl-4.64.9.tgz", + "integrity": "sha512-TFJyaCB9NsvSyvVGQgTSRD9XgxsAq99ivDDkt9KAySlrNW5Obsd1kQpvoCvBCOLq+JlJTjmtroLH0wp8M8mEsA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-s390x-gnu": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.64.9.tgz", + "integrity": "sha512-DvmOM5xz/wDnUJFVjqP32ahalxn7BDuvO25bpuFOrb50AIFCCcFPN4lWWUDD1c8Ddv9sMO9SXyqtOet8G8u9cQ==", + "cpu": [ + "s390x" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-gnu": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.64.9.tgz", + "integrity": "sha512-aFzokHK3yE00mzAvKm/PY76OsOOTjvoTGE82Vp89iBDyD2wUBlV4ZsR2tWkRECTEFk6TC4Yc3zaEWjPqq1vstQ==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-musl": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.64.9.tgz", + "integrity": "sha512-t/yMdeaaQUuNNpmT7taBmH2VvHzgT4YA/IaKVE3qVr4PYQemn/kmJkL4uG4UvTLcWXQd0MSoOa/3URYSWGmUEg==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-openbsd-x64": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-openbsd-x64/-/rollup-openbsd-x64-4.64.9.tgz", + "integrity": "sha512-4dnxKQ8GHARyjqAU87q5UWhywZTpIbR2VWhtACdC37GD1SGszmvb3TaBFVA4n+fnDHSKMut4ZJ63hVFwgnxVtg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ] + }, + "node_modules/@rollup/rollup-openharmony-arm64": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-openharmony-arm64/-/rollup-openharmony-arm64-4.64.9.tgz", + "integrity": "sha512-1qdQj6IrNoR0xFvrCQHmhBXe9HMbBGNTFi+3n4jKMt2rTx4k0aeAdraZU4WDOMfL9tAwS8/TRSuquvNuPQ+Gpw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ] + }, + "node_modules/@rollup/rollup-win32-arm64-msvc": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.64.9.tgz", + "integrity": "sha512-yc1GSEHIYhKGN5WSaJc+eMy8bZvzcvAmflEgFh8L3xcy8dYxGmfrudyBnBFde9f6JzOa1kFqocF8BHaWybjQoA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-ia32-msvc": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.64.9.tgz", + "integrity": "sha512-VzHGvosAkLIzrwqvSgJBPDmGFTF/yE8OtMT4H5z8pM1NA0NiPpvTYnUh+9xPd+d5oy/wqF317FmJ1I8Kk8pp0Q==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-gnu": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-gnu/-/rollup-win32-x64-gnu-4.64.9.tgz", + "integrity": "sha512-FB77txy1PiAcJgG72nOLA6R0S3PRZahs8Spx0z2yseDUkoMmJcrQghiebJ2JHZ74cCOVlX/MYd4dUbJtJJ8fow==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-msvc": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.64.9.tgz", + "integrity": "sha512-IoIgQLpK1DbOcQf1vX+i5oe5LXypsCVb1j/mU6BLAbOGmXH4E+j4kMfgZRwPYUohVBurNyVMgIKVM5dR6xA0kA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@types/chai": { + "version": "5.2.3", + "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz", + "integrity": "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/deep-eql": "*", + "assertion-error": "^2.0.1" + } + }, + "node_modules/@types/deep-eql": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz", + "integrity": "sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/estree": { + "version": "1.0.9", + "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz", + "integrity": "sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@vitest/expect": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-3.2.7.tgz", + "integrity": "sha512-E8eBXaKibuvH2pSZErOjdVb5vF4PbKYcrnluBTYxEk1l/VhhwZg1kZQsdtjq+CsF5CFydf2Rdkz7jDHKSisi3w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/chai": "^5.2.2", + "@vitest/spy": "3.2.7", + "@vitest/utils": "3.2.7", + "chai": "^5.2.0", + "tinyrainbow": "^2.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/mocker": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-3.2.7.tgz", + "integrity": "sha512-Trr0hYO9CM3Wj6ksWHRhK9IZpIY6wTMO5u/MqXurMxT57sWBaOPEtP3Oq60ihZuh5JsiagKfz95OcxdEP6dBrA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/spy": "3.2.7", + "estree-walker": "^3.0.3", + "magic-string": "^0.30.17" + }, + "funding": { + "url": "https://opencollective.com/vitest" + }, + "peerDependencies": { + "msw": "^2.4.9", + "vite": "^5.0.0 || ^6.0.0 || ^7.0.0-0" + }, + "peerDependenciesMeta": { + "msw": { + "optional": true + }, + "vite": { + "optional": true + } + } + }, + "node_modules/@vitest/pretty-format": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-3.2.7.tgz", + "integrity": "sha512-KUHlwqVu0sRlhCdyPdQ/wBoTfRahjUky1MubOmYw9fWfIZy1gNoHpuaaQBPAaMaVYdQYHJLurzj8ECCj5OwTqA==", + "dev": true, + "license": "MIT", + "dependencies": { + "tinyrainbow": "^2.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/runner": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-3.2.7.tgz", + "integrity": "sha512-sB9y4ovltoQP+WaUPwmSxO9WIg9Ig694Di5PalVPsYHklAdE027mehpWF2SQSVq+k6sFgaivbTjTJwZLSHbedA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/utils": "3.2.7", + "pathe": "^2.0.3", + "strip-literal": "^3.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/snapshot": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-3.2.7.tgz", + "integrity": "sha512-7C+MwShwtBSI5Buwoyg3s/iY1eHL9PKAf+O1wVh/TdnjXUtkoL/9YQtre90i4MtNXM6edP1wJ2zOBpfCyhIS7g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/pretty-format": "3.2.7", + "magic-string": "^0.30.17", + "pathe": "^2.0.3" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/spy": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-3.2.7.tgz", + "integrity": "sha512-Q2eQGI6d2L/hBtZ0qNuKcAGid68XK6cv1xsoaIma6PaJhHPoqcEJhYpXZ/5myCMqkNgtP6UKuBhbc0nHKnrkuQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "tinyspy": "^4.0.3" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/utils": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-3.2.7.tgz", + "integrity": "sha512-x6BDOd7dyo3PFLY3I9/HJ25X/6OurhGXk2/B9gOZNPF7XDVjeBK4k01lQE5uvDpbuheErh91qYuE1E2OEjK3Rw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/pretty-format": "3.2.7", + "loupe": "^3.1.4", + "tinyrainbow": "^2.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/assertion-error": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz", + "integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + } + }, + "node_modules/cac": { + "version": "6.7.14", + "resolved": "https://registry.npmjs.org/cac/-/cac-6.7.14.tgz", + "integrity": "sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=8" + } + }, + "node_modules/chai": { + "version": "5.3.3", + "resolved": "https://registry.npmjs.org/chai/-/chai-5.3.3.tgz", + "integrity": "sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==", + "dev": true, + "license": "MIT", + "dependencies": { + "assertion-error": "^2.0.1", + "check-error": "^2.1.1", + "deep-eql": "^5.0.1", + "loupe": "^3.1.0", + "pathval": "^2.0.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/check-error": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/check-error/-/check-error-2.1.3.tgz", + "integrity": "sha512-PAJdDJusoxnwm1VwW07VWwUN1sl7smmC3OKggvndJFadxxDRyFJBX/ggnu/KE4kQAB7a3Dp8f/YXC1FlUprWmA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 16" + } + }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "dev": true, + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/deep-eql": { + "version": "5.0.2", + "resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-5.0.2.tgz", + "integrity": "sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/es-module-lexer": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-1.7.0.tgz", + "integrity": "sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==", + "dev": true, + "license": "MIT" + }, + "node_modules/esbuild": { + "version": "0.28.2", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.2.tgz", + "integrity": "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.2", + "@esbuild/android-arm": "0.28.2", + "@esbuild/android-arm64": "0.28.2", + "@esbuild/android-x64": "0.28.2", + "@esbuild/darwin-arm64": "0.28.2", + "@esbuild/darwin-x64": "0.28.2", + "@esbuild/freebsd-arm64": "0.28.2", + "@esbuild/freebsd-x64": "0.28.2", + "@esbuild/linux-arm": "0.28.2", + "@esbuild/linux-arm64": "0.28.2", + "@esbuild/linux-ia32": "0.28.2", + "@esbuild/linux-loong64": "0.28.2", + "@esbuild/linux-mips64el": "0.28.2", + "@esbuild/linux-ppc64": "0.28.2", + "@esbuild/linux-riscv64": "0.28.2", + "@esbuild/linux-s390x": "0.28.2", + "@esbuild/linux-x64": "0.28.2", + "@esbuild/netbsd-arm64": "0.28.2", + "@esbuild/netbsd-x64": "0.28.2", + "@esbuild/openbsd-arm64": "0.28.2", + "@esbuild/openbsd-x64": "0.28.2", + "@esbuild/openharmony-arm64": "0.28.2", + "@esbuild/sunos-x64": "0.28.2", + "@esbuild/win32-arm64": "0.28.2", + "@esbuild/win32-ia32": "0.28.2", + "@esbuild/win32-x64": "0.28.2" + } + }, + "node_modules/estree-walker": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz", + "integrity": "sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "^1.0.0" + } + }, + "node_modules/expect-type": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.4.0.tgz", + "integrity": "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=12.0.0" + } + }, + "node_modules/fdir": { + "version": "6.5.0", + "resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz", + "integrity": "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12.0.0" + }, + "peerDependencies": { + "picomatch": "^3 || ^4" + }, + "peerDependenciesMeta": { + "picomatch": { + "optional": true + } + } + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/js-tokens": { + "version": "9.0.1", + "resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-9.0.1.tgz", + "integrity": "sha512-mxa9E9ITFOt0ban3j6L5MpjwegGz6lBQmM1IJkWeBZGcMxto50+eWdjC/52xDbS2vy0k7vIMK0Fe2wfL9OQSpQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/loupe": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/loupe/-/loupe-3.2.1.tgz", + "integrity": "sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/magic-string": { + "version": "0.30.21", + "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", + "integrity": "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.5.5" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true, + "license": "MIT" + }, + "node_modules/nanoid": { + "version": "3.3.20", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.20.tgz", + "integrity": "sha512-uKdg2G3GNCKQn9byYOpxbGqrT2fGO5KRt5J/8b3pok8rT6qxGWF6hxMyJiEYtAf+FVyYuD9hRaDqX5uPFYJ4ZQ==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "bin": { + "nanoid": "bin/nanoid.cjs" + }, + "engines": { + "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" + } + }, + "node_modules/pathe": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz", + "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==", + "dev": true, + "license": "MIT" + }, + "node_modules/pathval": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/pathval/-/pathval-2.0.1.tgz", + "integrity": "sha512-//nshmD55c46FuFw26xV/xFAaB5HF9Xdap7HJBBnrKdAd6/GxDBaNA1870O79+9ueg61cZLSVc+OaFlfmObYVQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 14.16" + } + }, + "node_modules/picocolors": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", + "dev": true, + "license": "ISC" + }, + "node_modules/picomatch": { + "version": "4.0.7", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.7.tgz", + "integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, + "node_modules/postcss": { + "version": "8.5.30", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.30.tgz", + "integrity": "sha512-zkd+e9S1AQHhmaBKye9zlFHWjOwVgqXl1q3ECYy8561K7qxR2cREf/STLskulBFd78k6AOFeA+ff1TtEzk9bAA==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/postcss/" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/postcss" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "dependencies": { + "nanoid": "^3.3.20", + "picocolors": "^1.1.1", + "source-map-js": "^1.2.2" + }, + "engines": { + "node": "^10 || ^12 || >=14" + } + }, + "node_modules/rollup": { + "version": "4.64.9", + "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.64.9.tgz", + "integrity": "sha512-mnPSQINChN7PpT+a5CI0/+87hm/+6zxSKBK1XonrzQ0zFnglaTjhkTP2Br4yNCTcLHUEvrAfZffH57jGf+8xBQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "1.0.9" + }, + "bin": { + "rollup": "dist/bin/rollup" + }, + "engines": { + "node": ">=18.0.0", + "npm": ">=8.0.0" + }, + "optionalDependencies": { + "@napi-rs/lzma-linux-x64-gnu": "1.5.1", + "@rollup/rollup-android-arm-eabi": "4.64.9", + "@rollup/rollup-android-arm64": "4.64.9", + "@rollup/rollup-darwin-arm64": "4.64.9", + "@rollup/rollup-darwin-x64": "4.64.9", + "@rollup/rollup-freebsd-arm64": "4.64.9", + "@rollup/rollup-freebsd-x64": "4.64.9", + "@rollup/rollup-linux-arm-gnueabihf": "4.64.9", + "@rollup/rollup-linux-arm-musleabihf": "4.64.9", + "@rollup/rollup-linux-arm64-gnu": "4.64.9", + "@rollup/rollup-linux-arm64-musl": "4.64.9", + "@rollup/rollup-linux-loong64-gnu": "4.64.9", + "@rollup/rollup-linux-loong64-musl": "4.64.9", + "@rollup/rollup-linux-ppc64-gnu": "4.64.9", + "@rollup/rollup-linux-ppc64-musl": "4.64.9", + "@rollup/rollup-linux-riscv64-gnu": "4.64.9", + "@rollup/rollup-linux-riscv64-musl": "4.64.9", + "@rollup/rollup-linux-s390x-gnu": "4.64.9", + "@rollup/rollup-linux-x64-gnu": "4.64.9", + "@rollup/rollup-linux-x64-musl": "4.64.9", + "@rollup/rollup-openbsd-x64": "4.64.9", + "@rollup/rollup-openharmony-arm64": "4.64.9", + "@rollup/rollup-win32-arm64-msvc": "4.64.9", + "@rollup/rollup-win32-ia32-msvc": "4.64.9", + "@rollup/rollup-win32-x64-gnu": "4.64.9", + "@rollup/rollup-win32-x64-msvc": "4.64.9", + "fsevents": "~2.3.2" + } + }, + "node_modules/siginfo": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz", + "integrity": "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==", + "dev": true, + "license": "ISC" + }, + "node_modules/source-map-js": { + "version": "1.2.2", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.2.tgz", + "integrity": "sha512-KGj/8Y43x35aZVDtt+J4mK1hoLGHULMYfSkODJNQjNDC3oW1PqPoxMwo0pLUsWM/UEGzON/NxeHywEfNXNP3Vw==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/stackback": { + "version": "0.0.2", + "resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz", + "integrity": "sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==", + "dev": true, + "license": "MIT" + }, + "node_modules/std-env": { + "version": "3.10.0", + "resolved": "https://registry.npmjs.org/std-env/-/std-env-3.10.0.tgz", + "integrity": "sha512-5GS12FdOZNliM5mAOxFRg7Ir0pWz8MdpYm6AY6VPkGpbA7ZzmbzNcBJQ0GPvvyWgcY7QAhCgf9Uy89I03faLkg==", + "dev": true, + "license": "MIT" + }, + "node_modules/strip-literal": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/strip-literal/-/strip-literal-3.1.0.tgz", + "integrity": "sha512-8r3mkIM/2+PpjHoOtiAW8Rg3jJLHaV7xPwG+YRGrv6FP0wwk/toTpATxWYOW0BKdWwl82VT2tFYi5DlROa0Mxg==", + "dev": true, + "license": "MIT", + "dependencies": { + "js-tokens": "^9.0.1" + }, + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, + "node_modules/tinybench": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz", + "integrity": "sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==", + "dev": true, + "license": "MIT" + }, + "node_modules/tinyexec": { + "version": "0.3.2", + "resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-0.3.2.tgz", + "integrity": "sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==", + "dev": true, + "license": "MIT" + }, + "node_modules/tinyglobby": { + "version": "0.2.17", + "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.17.tgz", + "integrity": "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==", + "dev": true, + "license": "MIT", + "dependencies": { + "fdir": "^6.5.0", + "picomatch": "^4.0.4" + }, + "engines": { + "node": ">=12.0.0" + }, + "funding": { + "url": "https://github.com/sponsors/SuperchupuDev" + } + }, + "node_modules/tinypool": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/tinypool/-/tinypool-1.1.1.tgz", + "integrity": "sha512-Zba82s87IFq9A9XmjiX5uZA/ARWDrB03OHlq+Vw1fSdt0I+4/Kutwy8BP4Y/y/aORMo61FQ0vIb5j44vSo5Pkg==", + "dev": true, + "license": "MIT", + "engines": { + "node": "^18.0.0 || >=20.0.0" + } + }, + "node_modules/tinyrainbow": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/tinyrainbow/-/tinyrainbow-2.0.0.tgz", + "integrity": "sha512-op4nsTR47R6p0vMUUoYl/a+ljLFVtlfaXkLQmqfLR1qHma1h/ysYk4hEXZ880bf2CYgTskvTa/e196Vd5dDQXw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/tinyspy": { + "version": "4.0.6", + "resolved": "https://registry.npmjs.org/tinyspy/-/tinyspy-4.0.6.tgz", + "integrity": "sha512-u8KszXvGfU68hVcZpRHKG28T0krMuv2G5nDhiHaMLen/gIuFEgIJhaJuO69qjnXg5paSrbPMFfx3brNuN8eVSg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/vite": { + "version": "7.3.7", + "resolved": "https://registry.npmjs.org/vite/-/vite-7.3.7.tgz", + "integrity": "sha512-AHEzTORMbi7/7aXp7w7KZ5+DCUdSeLsJz0grY8xHpc9DHdYb0vQu6DqJfm2tOw3gDExNgPgnJTXxM6syYFHaCA==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "^0.27.0 || ^0.28.0", + "fdir": "^6.5.0", + "picomatch": "^4.0.3", + "postcss": "^8.5.6", + "rollup": "^4.43.0", + "tinyglobby": "^0.2.15" + }, + "bin": { + "vite": "bin/vite.js" + }, + "engines": { + "node": "^20.19.0 || >=22.12.0" + }, + "funding": { + "url": "https://github.com/vitejs/vite?sponsor=1" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + }, + "peerDependencies": { + "@types/node": "^20.19.0 || >=22.12.0", + "jiti": ">=1.21.0", + "less": "^4.0.0", + "lightningcss": "^1.21.0", + "sass": "^1.70.0", + "sass-embedded": "^1.70.0", + "stylus": ">=0.54.8", + "sugarss": "^5.0.0", + "terser": "^5.16.0", + "tsx": "^4.8.1", + "yaml": "^2.4.2" + }, + "peerDependenciesMeta": { + "@types/node": { + "optional": true + }, + "jiti": { + "optional": true + }, + "less": { + "optional": true + }, + "lightningcss": { + "optional": true + }, + "sass": { + "optional": true + }, + "sass-embedded": { + "optional": true + }, + "stylus": { + "optional": true + }, + "sugarss": { + "optional": true + }, + "terser": { + "optional": true + }, + "tsx": { + "optional": true + }, + "yaml": { + "optional": true + } + } + }, + "node_modules/vite-node": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/vite-node/-/vite-node-3.2.4.tgz", + "integrity": "sha512-EbKSKh+bh1E1IFxeO0pg1n4dvoOTt0UDiXMd/qn++r98+jPO1xtJilvXldeuQ8giIB5IkpjCgMleHMNEsGH6pg==", + "dev": true, + "license": "MIT", + "dependencies": { + "cac": "^6.7.14", + "debug": "^4.4.1", + "es-module-lexer": "^1.7.0", + "pathe": "^2.0.3", + "vite": "^5.0.0 || ^6.0.0 || ^7.0.0-0" + }, + "bin": { + "vite-node": "vite-node.mjs" + }, + "engines": { + "node": "^18.0.0 || ^20.0.0 || >=22.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/vitest": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/vitest/-/vitest-3.2.7.tgz", + "integrity": "sha512-KrxIJ62Fd89gfysR4WotlgZABiz2dqFPgqGzX7s+CwsqLFomRH7777ZcrOD6+WVAh7khPQP41A+BKbpcJFrdEg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/chai": "^5.2.2", + "@vitest/expect": "3.2.7", + "@vitest/mocker": "3.2.7", + "@vitest/pretty-format": "^3.2.7", + "@vitest/runner": "3.2.7", + "@vitest/snapshot": "3.2.7", + "@vitest/spy": "3.2.7", + "@vitest/utils": "3.2.7", + "chai": "^5.2.0", + "debug": "^4.4.1", + "expect-type": "^1.2.1", + "magic-string": "^0.30.17", + "pathe": "^2.0.3", + "picomatch": "^4.0.2", + "std-env": "^3.9.0", + "tinybench": "^2.9.0", + "tinyexec": "^0.3.2", + "tinyglobby": "^0.2.14", + "tinypool": "^1.1.1", + "tinyrainbow": "^2.0.0", + "vite": "^5.0.0 || ^6.0.0 || ^7.0.0-0", + "vite-node": "3.2.4", + "why-is-node-running": "^2.3.0" + }, + "bin": { + "vitest": "vitest.mjs" + }, + "engines": { + "node": "^18.0.0 || ^20.0.0 || >=22.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + }, + "peerDependencies": { + "@edge-runtime/vm": "*", + "@types/debug": "^4.1.12", + "@types/node": "^18.0.0 || ^20.0.0 || >=22.0.0", + "@vitest/browser": "3.2.7", + "@vitest/ui": "3.2.7", + "happy-dom": "*", + "jsdom": "*" + }, + "peerDependenciesMeta": { + "@edge-runtime/vm": { + "optional": true + }, + "@types/debug": { + "optional": true + }, + "@types/node": { + "optional": true + }, + "@vitest/browser": { + "optional": true + }, + "@vitest/ui": { + "optional": true + }, + "happy-dom": { + "optional": true + }, + "jsdom": { + "optional": true + } + } + }, + "node_modules/why-is-node-running": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz", + "integrity": "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==", + "dev": true, + "license": "MIT", + "dependencies": { + "siginfo": "^2.0.0", + "stackback": "0.0.2" + }, + "bin": { + "why-is-node-running": "cli.js" + }, + "engines": { + "node": ">=8" + } + } + } +} diff --git a/src/Bsqlite/Tests/bit-bsqlite-js/package.json b/src/Bsqlite/Tests/bit-bsqlite-js/package.json new file mode 100644 index 00000000000..891831eadda --- /dev/null +++ b/src/Bsqlite/Tests/bit-bsqlite-js/package.json @@ -0,0 +1,14 @@ +{ + "name": "bit-bsqlite-js-tests", + "version": "1.0.0", + "private": true, + "type": "module", + "description": "Unit tests for the parts of Bit.Bsqlite/Scripts that need no browser: the .NET decimal arithmetic, the host's change-notification filter and the WebAssembly function-table patch", + "scripts": { + "test": "vitest run", + "watch": "vitest" + }, + "devDependencies": { + "vitest": "^3.2.4" + } +} diff --git a/src/Bsqlite/Tests/bit-bsqlite-js/wasm-table.test.js b/src/Bsqlite/Tests/bit-bsqlite-js/wasm-table.test.js new file mode 100644 index 00000000000..b06703f72f9 --- /dev/null +++ b/src/Bsqlite/Tests/bit-bsqlite-js/wasm-table.test.js @@ -0,0 +1,74 @@ +import { readFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { test, expect } from 'vitest'; +import { fileURLToPath } from 'node:url'; +import { raiseFunctionTableLimit } from '../../Bit.Bsqlite/Scripts/bit-bsqlite-wasm-table.ts'; + +const engineDir = join( + dirname(fileURLToPath(import.meta.url)), + '../../Bit.Bsqlite/wwwroot/engine'); + +const builds = ['wa-sqlite.wasm', 'wa-sqlite-jspi.wasm', 'wa-sqlite-async.wasm']; + +function readTableLimits(bytes) { + let offset = 8; + while (offset < bytes.length) { + const id = bytes[offset++]; + let size = 0; + let shift = 0; + while (true) { + const byte = bytes[offset++]; + size |= (byte & 0x7f) << shift; + if ((byte & 0x80) === 0) { + break; + } + + shift += 7; + } + + if (id === 4) { + const payload = bytes.subarray(offset, offset + size); + // count=1, funcref, limits=1, min, max - all LEB after the two tag bytes. + let i = 0; + const skipLeb = () => { + let n = 0; + let s = 0; + while (true) { + const b = payload[i++]; + n |= (b & 0x7f) << s; + if ((b & 0x80) === 0) { + return n >>> 0; + } + + s += 7; + } + }; + + skipLeb(); + i++; // funcref + i++; // limits flag + return { min: skipLeb(), max: skipLeb() }; + } + + offset += size; + } + + throw new Error('No table section.'); +} + +for (const name of builds) { + test(`${name} is valid after the table maximum is raised`, () => { + const original = new Uint8Array(readFileSync(join(engineDir, name))); + const before = readTableLimits(original); + expect(before.min, 'the vendored pin is a fixed-size table').toBe(before.max); + + const patched = raiseFunctionTableLimit(original, 16); + expect(patched).not.toBe(original); + expect(WebAssembly.validate(patched)).toBeTruthy(); + + const after = readTableLimits(patched); + expect(after.min).toBe(before.min); + expect(after.max).toBe(before.min + 16); + expect(readTableLimits(original)).toEqual(before); + }); +} diff --git a/src/Bsqlite/docs/architecture.md b/src/Bsqlite/docs/architecture.md new file mode 100644 index 00000000000..b52f86155c8 --- /dev/null +++ b/src/Bsqlite/docs/architecture.md @@ -0,0 +1,54 @@ +# Architecture and performance + +``` +Blazor main thread Dedicated worker +────────────────────── ──────────────────────────────── +EF Core (stock SQLite provider) + │ +BsqliteConnection / Command / Batch + │ BsqliteCommandRequest[] +BsqliteWorkerTransport + │ byte[] (binary wire format) +bit-bsqlite-host.js ── postMessage, transferred ──▶ bit-bsqlite-worker.js + ◀── postMessage, transferred ── │ statement cache, update hook + wa-sqlite (SQLite 3.53 WASM) + │ + VFS: OPFS / IndexedDB / Cache / memory +``` + +## Why a worker + +OPFS synchronous access handles - the fastest storage the web has - exist only in workers. Putting the engine in a dedicated worker also keeps SQLite's work off the UI thread. The price is that every call is asynchronous, which is why the synchronous ADO.NET surface throws rather than blocks. + +One worker owns one database. A second `BsqliteDatabase` or a second session starts a second worker - and so does every other tab, each over the same file; the VFSes coordinate through Web Locks. SQLite's locks last a transaction at most, so work that has to keep other tabs out for longer, like applying EF migrations, takes a named Web Lock through the transport (`IBsqliteLockSource`). + +## The binary wire format + +Everything the transport sends or receives is one `byte[]`: + +- A **batch** is magic, version, count, then per request the SQL, a result-kind byte, a row limit, and parameters as name, SQLite storage-class tag, and payload. +- A **reply** is magic, version, status, then either an error (message, result code, name) or one result per request: columns with their storage class, records affected, last insert rowid, and rows as tagged cells. + +Integers are eight bytes, doubles eight bytes, strings and blobs length-prefixed. Blazor marshals byte arrays across JS interop without JSON or base64, the host transfers the buffer to the worker instead of cloning it, and the worker writes rows straight from `sqlite3_column_*` into the reply buffer with no intermediate object graph. A failure is a reply with an error status, so the SQLite result code survives the boundary and becomes a typed exception. + +`BsqliteWireFormat` (C#) and `bit-bsqlite-wire.js` are the two halves; the format version is checked on every reply. The host module, the worker, and every module they import are loaded with the assembly's version in the query string, so a browser cannot pair new .NET with JavaScript it cached under an earlier release - and if something else serves a stale copy anyway, the version check says so. + +JavaScript callers get the same information in a structured-clone-friendly tagged shape (`{ t: [...], v: [...] }` rows) through `host.execute`. + +## The statement cache + +The worker keeps up to 64 prepared statements keyed by SQL text, lent out and reset after each use. EF Core sends the same text over and over with different parameters; preparing is a parse and a plan, resetting is nearly free. Only single-statement texts are cached. A DDL statement clears the cache, since column metadata is read before stepping and must reflect the new schema. + +## Change tracking + +`sqlite3_update_hook` names every table a row changed in, through triggers and cascades; the statement text covers DDL and truncating deletes, which the hook does not see. `sqlite3_commit_hook` says when those rows became visible. The worker accumulates tables until the connection is back in autocommit and then notifies - this tab's .NET transport and, over a `BroadcastChannel`, every other tab - only if a commit happened. See [Live queries](live-queries.md#how-changes-are-detected). Affected-row counts honour `sqlite3_stmt_readonly`, so an `INSERT` followed by a `SELECT` reports the insert once - the rule Microsoft.Data.Sqlite applies. + +## Open-time configuration + +Every connection runs `PRAGMA foreign_keys = ON` unless told otherwise, because the wa-sqlite build defaults to off while Microsoft.Data.Sqlite defaults to on, and a cascade rule that silently does nothing is a data bug. Providers add their own statements (IndexedDB raises `cache_size`), and applications add theirs through `OnOpen`. + +## Where the time goes + +Per round trip, roughly: Blazor interop and JSON for the envelope, the worker hop, the SQL itself, and encoding the result. The binary format removes the JSON and base64 share entirely; the statement cache removes the parse; batching amortises the hop. What remains is storage: OPFS reads are synchronous memory copies, IndexedDB and Cache Storage pay an async transaction per page group. Choose OPFS first where it is available. + +The manual benchmark page, `TestPages/benchmarks.html` in `Tests/Bit.Bsqlite.Tests.E2E`, measures cold open, a 1 000-row read on both encodings, a 100-row insert batch, and repeated parameterised statements per backend; the test project's README says how to open it. diff --git a/src/Bsqlite/docs/extending.md b/src/Bsqlite/docs/extending.md new file mode 100644 index 00000000000..334a7f81e73 --- /dev/null +++ b/src/Bsqlite/docs/extending.md @@ -0,0 +1,51 @@ +# Extending + +## SQL functions from JavaScript + +User-defined functions must live where the engine lives, which is the worker. An extension module is an ES module exporting `register`: + +```js +// wwwroot/my-functions.js +export function register({ module, sqlite3, db, databaseName }) { + sqlite3.create_function(db, 'slugify', 1, 1 /* SQLITE_UTF8 */, 0, (ctx, values) => { + const text = sqlite3.value_text(values[0]); + sqlite3.result(ctx, text.toLowerCase().replace(/[^a-z0-9]+/g, '-')); + }); +} +``` + +```csharp +builder.Services.AddBitBsqliteServices(sqlite => sqlite + // ... + .AddExtensionModule("/my-functions.js")); +``` + +```csharp +modelBuilder.HasDbFunction(() => Sql.Slugify(default!)).HasName("slugify"); +``` + +The module receives the Emscripten `module`, the wa-sqlite `sqlite3` API, the open `db` handle, and the database name - the same handles the built-in `ef_*` set uses (`bit-bsqlite-functions.js` is a worked example, including aggregates and a collation). A relative URL is resolved against the document, as every other Blazor asset is, so `./my-functions.js` and `./_content/MyPackage/functions.js` work under a sub-path too. + +Functions cannot call back into .NET: the worker has no access to the Blazor runtime, and a synchronous SQL callback could not await it if it did. + +## Open-time statements + +```csharp +.OnOpen("PRAGMA cache_size = -16000") +.OnOpen("PRAGMA temp_store = MEMORY") +``` + +These run after the provider's own statements and after `PRAGMA foreign_keys`, outside the SQL guards. They cannot enable WAL. + +## A custom transport + +`IBsqliteTransport` is the seam between the ADO.NET layer and wherever the engine runs. `Bit.Bsqlite.Testing.BsqliteInProcessTransport` runs Microsoft.Data.Sqlite in-process for desktop tests; `BsqliteWorkerTransport` drives the worker. A transport that can also: + +- report committed writes - raise `TablesChanged` once a write commits, for its own writes and other writers' alike, and return `true` from `ReportsLocalWrites`; otherwise leave both at their defaults and the connection infers tables from the SQL; +- snapshot an open database - implement `IBsqliteSnapshotSource`. + +Register it through `IBsqliteTransportFactory` to replace the worker for every session, which is how the EF test suite runs the whole composition on desktop .NET. + +## A custom storage backend + +See [Storage](storage.md#writing-a-provider). diff --git a/src/Bsqlite/docs/getting-started.md b/src/Bsqlite/docs/getting-started.md new file mode 100644 index 00000000000..e112c9c600a --- /dev/null +++ b/src/Bsqlite/docs/getting-started.md @@ -0,0 +1,125 @@ +# Getting started + +## Install + +Reference `Bit.Bsqlite` - the ADO.NET surface, the EF Core integration, the worker transport, and the in-memory backend in one package - plus the storage packages you want. Every storage package is optional; register at least one. + +``` +dotnet add package Bit.Bsqlite +dotnet add package Bit.Bsqlite.Storage.Opfs +dotnet add package Bit.Bsqlite.Storage.IndexedDb +``` + +The engine (`.wasm` and the worker) is served as static web assets under `_content/Bit.Bsqlite/`. Nothing needs to be added to the host page. + +Bit.Bsqlite targets .NET 8, 9 and 10, each with the EF Core of the same major version. + +## Register + +```csharp +using Bit.Bsqlite.Storage.IndexedDb; +using Bit.Bsqlite.Storage.InMemory; +using Bit.Bsqlite.Storage.Opfs; + +builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider() + .AddStorageProvider() + .AddStorageProvider() + .UseStorage(s => s + .Prefer(BsqliteOpfsStorageProvider.ProviderName) + .Fallback(BsqliteIndexedDbStorageProvider.ProviderName) + .Fallback(BsqliteInMemoryStorageProvider.ProviderName) + .AllowNonPersistentFallback()) + .AddDatabase("app.db")); + +builder.Services.AddBitBsqliteDbContext(options => +{ + options.Schema = BsqliteSchemaStrategy.Migrate; // the default + options.OnFirstUse = (db, ct) => DemoData.SeedIfEmptyAsync(db, ct); +}); +``` + +Register them in the project that runs on WebAssembly. In a Blazor Web App whose pages are prerendered, register them in the server project as well: the prerender pass instantiates the pages there, so what they inject has to resolve, and nothing touches the browser until a page opens the database - which it should do only once the page runs in the browser: `RendererInfo.IsInteractive` on Interactive WebAssembly, and `OperatingSystem.IsBrowser()` on Interactive Auto, where a page on the server circuit is interactive too (and on .NET 8, which has no `RendererInfo`). + +`AddBitBsqliteServices` registers, all scoped except the selection and the migrator (one scope is the whole application on WebAssembly, and a circuit or a request on a server): + +| Service | What it is | +|---|---| +| `IBsqliteSessionFactory` | Resolves a backend, starts a worker, opens the database | +| `BsqliteDatabase` | The one shared session for the database named in `AddDatabase` - also a keyed service under that name, for an application that registers several | +| `BsqliteStorageProviderResolver`, `BsqliteStorageMigrator`, `IBsqliteStorageBindingStore` | Selection, migration, and the sticky binding (in `localStorage` by default) | +| `IBsqliteTransportFactory` | The worker transport | + +Builder options: + +- `EnableForeignKeys(false)` - skip `PRAGMA foreign_keys = ON` on open. On by default, as with Microsoft.Data.Sqlite. +- `OnOpen("PRAGMA cache_size = -16000")` - statements to run after every open. +- `AddExtensionModule("/my-functions.js")` - JavaScript that registers SQL functions; see [Extending](extending.md). +- `UseBindingStore()` - replace where sticky bindings are remembered. +- `UseWorker(hostModuleUrl, workerUrl)` - host the worker from a different path. + +## Use EF Core + +```csharp +@inject IBsqliteDbContextFactory Contexts + +await using var db = await Contexts.CreateDbContextAsync(); +db.Products.Add(new Product { Name = "Widget", Price = 19.99m }); +await db.SaveChangesAsync(); +``` + +The factory opens the database on first use, runs the schema step once per session, and hands out contexts bound to the shared connection. Disposing a context does not close the database. The factory also implements `IDbContextFactory`; its synchronous `CreateDbContext()` works once an awaited `CreateDbContextAsync()` has opened the session and run the schema step on it. + +The schema step and `OnFirstUse` run under a lock shared with every other tab, and so does EF's own `MigrateAsync` on EF Core 9 and later: each tab has its own worker over the same file, and two tabs opening after a deploy would otherwise both apply the new migration - or both seed an empty database. The lock is a Web Lock, so a tab that closes mid-migration releases it. + +With more than one database registered, name the one a context belongs to: `AddBitBsqliteDbContext(o => o.DatabaseName = "cache.db")`. Leaving it out is an error then, rather than a context bound to whichever was registered last. + +Use the async EF API throughout: `ToListAsync`, `SaveChangesAsync`, `MigrateAsync`. Synchronous members throw `BsqliteSynchronousApiNotSupportedException` naming the async overload, because the engine runs in a worker the main thread cannot wait on. + +### Without the factory + +```csharp +var session = await sessionFactory.OpenAsync("app.db"); +var options = new DbContextOptionsBuilder().UseBitBsqlite(session).Options; +await using var db = new AppDbContext(options); +``` + +`UseBitBsqlite` is `UseSqlite(connection)` plus replacements for the two EF services that would otherwise open the connection synchronously. SQL generation and type mapping are the stock provider's. + +## Use ADO.NET + +```csharp +@inject BsqliteDatabase Database + +var session = await Database.OpenAsync(); +await using var command = session.Connection.CreateCommand(); +command.CommandText = "SELECT name FROM product WHERE price > @min"; +command.Parameters.AddWithValue("@min", 10m); + +await using var reader = await command.ExecuteReaderAsync(); +while (await reader.ReadAsync()) +{ + Console.WriteLine(reader.GetString(0)); +} +``` + +What the connection offers beyond the basics: + +- **Batches.** `connection.CreateBatch()` runs several commands in one worker round trip; `ExecuteReaderAsync` on a batch walks every result set with `NextResultAsync`. +- **Transactions.** `BeginTransactionAsync()` issues `BEGIN IMMEDIATE` (as Microsoft.Data.Sqlite does); any isolation level other than `Serializable` begins deferred. Disposing without committing rolls back. Savepoints: `SaveAsync`, `ReleaseAsync`, `RollbackAsync(name)`. +- **Counters.** `command.LastInsertRowId` and `connection.LastInsertRowId` after any execution. +- **Export.** `connection.ExportAsync()` returns a consistent SQLite file image of the open database. +- **Typed errors.** `BsqliteConstraintException`, `BsqliteConcurrencyException`, `BsqliteQuotaExceededException`, `BsqliteCorruptDatabaseException`, all deriving from `BsqliteException` with `SqliteErrorCode`. + +## The shared database + +`BsqliteDatabase` owns the session for the lifetime of its scope - the whole application on WebAssembly: + +```csharp +await Database.OpenAsync(); // idempotent +Database.Resolution?.Provider.Name; // "opfs", "indexeddb", ... +await Database.CloseAsync(); // raises Closing first, then terminates the worker +await Database.WhileClosedAsync(ct => provider.Admin.ImportAsync("app.db", bytes, ct)); +``` + +Subscribe to `Closing` to drop contexts and live queries before the worker goes away, and to `Opened` to reload afterwards. The demo's `SessionLifecycle` component is one way to wire that into pages. diff --git a/src/Bsqlite/docs/limits.md b/src/Bsqlite/docs/limits.md new file mode 100644 index 00000000000..533b673ecd2 --- /dev/null +++ b/src/Bsqlite/docs/limits.md @@ -0,0 +1,17 @@ +# Limits + +What the browser cannot do, and how the library tells you. + +| Limit | Behaviour | +|---|---| +| Synchronous ADO.NET and EF members (`Open`, `ExecuteReader`, `SaveChanges`, `ToList`, `EnsureCreated`) | Throw `BsqliteSynchronousApiNotSupportedException` naming the async overload. The worker cannot be waited on from the main thread. | +| `PRAGMA journal_mode = WAL` | Rejected on every backend. No web VFS provides the shared memory WAL needs. Journal mode is `DELETE` or `TRUNCATE`. | +| `ATTACH` | Rejected when the backend declares `SupportsMultiDatabaseTransactions = false` (OPFS). Open each database on its own session. | +| `PRAGMA page_size = …` | Rejected when the backend declares `CanChangePageSize = false` (IndexedDB, Cache Storage). | +| Command timeout and `Cancel()` | Accepted, not enforced. A cancellation token stops the caller waiting; the statement runs to completion in the worker. | +| Declared column types from `GetDataTypeName` | The engine is built with `SQLITE_OMIT_DECLTYPE`; the worker reports the storage class of the first row (`INTEGER`, `TEXT`, …). EF Core is unaffected. | +| `EnsureDeletedAsync` | Drops every view and table and runs `VACUUM`; it cannot unlink the file while the session holds it. Use `BsqliteDatabase.WhileClosedAsync` with `provider.Admin.DeleteAsync` to wipe storage. | +| Firefox and Safari | Supported through the Asyncify build for async backends; OPFS synchronous access handles need a recent version. The browser suite runs on Chromium by default; set `BSQLITE_E2E_BROWSER=firefox` or `webkit` to run it on either, after installing that browser for Playwright. | +| EF Core 8 and 9 | The SQLite provider translates `SUM` over a `decimal` from EF Core 9 on and `ORDER BY` a `decimal` from EF Core 10 on. Older providers throw `NotSupportedException` for those queries, in the browser exactly as on a server. | + +Guards run on both sides: the .NET command layer rejects the statement before it leaves, and the worker rejects it again so a JavaScript caller cannot bypass the capability declaration. diff --git a/src/Bsqlite/docs/live-queries.md b/src/Bsqlite/docs/live-queries.md new file mode 100644 index 00000000000..eb7b5c82668 --- /dev/null +++ b/src/Bsqlite/docs/live-queries.md @@ -0,0 +1,58 @@ +# Live queries + +A live query re-executes when any table it reads is written, by this tab or another one. + +## With EF Core + +```csharp +await using var live = db.Orders + .Include(o => o.Customer) + .Where(o => o.Status == OrderStatus.Open) + .AsLiveQuery(); + +live.Changed += (_, rows) => InvokeAsync(StateHasChanged); +live.RefreshFailed += (_, ex) => logger.LogWarning(ex, "Live query failed"); +var rows = await live.RefreshAsync(); // first run; later runs are automatic +``` + +`AsLiveQuery()` resolves the connection from the query's context and the tables from `ToQueryString()`. Pass a `BsqliteConnection` explicitly when the query's provider does not expose its context. + +Each run of the query happens in a context of its own - same type, same options - disposed once the rows are read. A refresh can arrive at any moment another tab writes, so on the application's context it would collide with whatever the application starts there next (EF allows one operation per context at a time); and a tracking query re-run on a context that already tracks its rows gets the tracked instances back with their old values, so updates would never show. The rows a live query returns are therefore current, and are not tracked by the application's context: to change one, load or attach it in the context that saves it. + +## With ADO.NET + +```csharp +await using var live = new BsqliteLiveQuery( + connection, + async ct => Convert.ToInt64(await countCommand.ExecuteScalarAsync(ct)), + tables: ["product"]); +``` + +An empty table list means every write triggers a refresh. + +## Async stream + +```csharp +await foreach (var rows in live.WithCancellation(cancellationToken)) +{ + Render(rows); +} +``` + +## How changes are detected + +The worker installs `sqlite3_update_hook` and `sqlite3_commit_hook` on every connection. The update hook names the tables whose rows were inserted, updated, or deleted - including through triggers and `ON DELETE CASCADE` - and the statement text adds the two kinds of write the hook cannot see: DDL, and a `DELETE` without `WHERE`, which SQLite runs through its truncate optimisation. A request that changed no rows and is not DDL adds nothing, so a read or a no-op write never refreshes anything. + +Nothing is reported while a transaction is open. Once the connection is back in autocommit, the accumulated tables go out if a commit happened and are dropped if it rolled back - a live query never shows data that may yet disappear, and a multi-statement `SaveChangesAsync` refreshes once rather than per statement. + +The notification goes to this tab's .NET transport, which raises `TablesChanged`, and over a `BroadcastChannel` named `bit-bsqlite-changes` to every other tab, filtered there to the same database name. + +A custom transport that cannot report its own writes leaves `ReportsLocalWrites` false, and the connection falls back to a regular-expression scan of the SQL it sends, holding the tables until an explicit `BsqliteTransaction` commits. That scan is deliberately coarse: it can produce extra refreshes, never a missed one. + +## Coalescing + +A burst of writes during a refresh produces exactly one further refresh after it, not one per write. Each `Changed` event therefore carries the latest state, and the engine is never asked to re-run a query it is about to re-run again. An explicit `RefreshAsync` waits its turn behind a refresh in flight, so two runs never overlap and never publish out of order. + +## Lifetime + +Dispose a live query when the component goes away, and before the session closes (`BsqliteDatabase.Closing` is the signal). A disposed query unsubscribes from the connection and ends its streams: a consumer waiting for the next item gets none, and one that was busy with an item comes back to the end of the stream. diff --git a/src/Bsqlite/docs/storage.md b/src/Bsqlite/docs/storage.md new file mode 100644 index 00000000000..a564b014fc5 --- /dev/null +++ b/src/Bsqlite/docs/storage.md @@ -0,0 +1,62 @@ +# Storage + +A storage backend is a `IBsqliteStorageProvider`: a .NET half that declares capabilities and answers probes, and a JavaScript VFS the worker registers with the engine. The contract is public; the core consumes nothing else. + +## First-party backends + +| Name | Package | Engine build | Persistent | Notes | +|---|---|---|---|---| +| `opfs` | `Bit.Bsqlite.Storage.Opfs` | Synchronous | Yes | `OPFSCoopSyncVFS`. Fastest. Several connections and tabs, coordinated through Web Locks. No `ATTACH`. Worker-only, no COOP/COEP needed. | +| `indexeddb` | `Bit.Bsqlite.Storage.IndexedDb` | Async-capable | Yes | `IDBBatchAtomicVFS`. Works where OPFS does not (older Safari, some embedded views). Page size fixed at 4096. `cache_size` raised to 8 MiB so batch-atomic commits skip the journal. | +| `cache-storage` | `Bit.Bsqlite.Storage.CacheStorage` | Async-capable | Yes | 4096-byte pages in the Cache API. Imports a database left by [Bit.Besql](https://github.com/bitfoundation/bitplatform/tree/develop/src/Besql) on first open. | +| `in-memory` | `Bit.Bsqlite` (namespace `Bit.Bsqlite.Storage.InMemory`) | Synchronous | No | The engine's own memory VFS. Tests, demos, and a declared last resort. | + +The engine build matters: a VFS that suspends to read storage (IndexedDB, Cache Storage) needs the JSPI build where the browser has it (Chrome 137+) and the Asyncify build otherwise. The synchronous build is the smallest and serves OPFS and in-memory. Exactly one is downloaded per session. + +## Selection + +```csharp +.UseStorage(s => s + .Prefer("opfs") + .Fallback("indexeddb") + .Fallback("in-memory") + .AllowNonPersistentFallback() + .WithMigrationMode(BsqliteStorageMigrationMode.KeepExisting)) +``` + +Two rules shape everything: + +1. **Existing data outranks preference.** A database created on IndexedDB stays on IndexedDB when OPFS later becomes available. If the backend that holds the data cannot be reached, the open fails rather than producing an empty database elsewhere. The binding is recorded in `IBsqliteStorageBindingStore` (in `localStorage` by default) only after the open succeeds. A database with no binding - after an application clears `localStorage` on sign-out, say - is looked for on every available persistent backend before it is treated as new; found, it opens there, is bound again (`Resolution.WasFoundWithoutBinding`), and a warning is logged. +2. **Nothing is substituted silently.** A fallback is logged. Falling back to a non-persistent backend needs `AllowNonPersistentFallback()`, and is never bound: the data is gone by the next load, so the binding would protect nothing, and would keep every later visit off a persistent backend that has come back. Total failure throws `BsqliteStorageUnavailableException` listing every candidate and why it was rejected. + +`session.Resolution` records the decision: the chosen provider, the probe, every attempt, whether it was the first choice, and whether a better backend is available for an existing database. + +## Migration + +`BsqliteStorageMigrationMode.AutomaticOnOpen` moves a database to a better-ranked backend before opening it. `Manual` and `KeepExisting` leave it and report through `Resolution.BetterProviderAvailable`. + +A migration is: probe the target, check quota headroom, export, verify the SQLite header, import, read the copy back and compare it byte for byte, flip the binding, delete the source. A failure before the flip leaves the source and the binding untouched. If deleting the source fails - another tab still holding an OPFS file open, say - the flip is undone when the source is still whole, and kept when it is not; the error is rethrown either way, and the binding says where the database now lives. `AutomaticOnOpen` resolves again after any failure, so the session always opens where the binding points. The demo's storage switcher drives the same `BsqliteStorageMigrator` by hand. + +## Admin + +`provider.Admin` works on databases as files, with no engine running: `ExistsAsync`, `ListAsync`, `DeleteAsync`, `ExportAsync`, `ImportAsync`. Close the session first when the database may be open: + +```csharp +await Database.WhileClosedAsync(async ct => +{ + await provider.Admin.ImportAsync("app.db", bytes, ct); + await bindings.SetProviderNameAsync("app.db", provider.Name, ct); +}); +``` + +For a backup of a database that stays open, use `connection.ExportAsync()` instead: it snapshots through `sqlite3_serialize` under the engine's own lock. + +`BsqlitePersistence.RequestAsync(js)` asks the browser to exempt the origin from storage eviction. It is origin-wide, not per backend. + +## Writing a provider + +1. Implement `IBsqliteStorageProvider`: a stable `Name`, `Capabilities` (which engine build the VFS needs, whether data persists, and the rest), `VfsModule` pointing at an ES module, `Admin`, and `ProbeAsync` that reports unavailability rather than throwing. +2. Write the VFS against the base classes served from `_content/Bit.Bsqlite/engine/` (`VFS.js`, `FacadeVFS.js`, `WebLocksMixin.js`), imported relatively - `../Bit.Bsqlite/engine/VFS.js` from your package's own `_content` folder - so the application can be served under a sub-path. Export `register({ module, sqlite3, databaseName })` returning `{ vfsName, onOpenSql? }`. +3. Inherit `BsqliteStorageProviderConformanceTests` and `BsqliteStorageEngineConformanceTests` (namespace `Bit.Bsqlite.Storage.ConformanceTests`, package `Bit.Bsqlite.Testing`) in your MSTest project, with `[TestClass]` on each derived class. They check that the declared capabilities are coherent and that the admin surface behaves the way migration assumes. + +Capability flags the core enforces for you: `SupportsMultiDatabaseTransactions` gates `ATTACH`, `CanChangePageSize` gates `PRAGMA page_size`, `IsPersistent` gates fallback. Declare the least capable answer when unsure. diff --git a/src/Templates/Boilerplate/Bit.Boilerplate/.docs/19- Project Miscellaneous Files.md b/src/Templates/Boilerplate/Bit.Boilerplate/.docs/19- Project Miscellaneous Files.md index b8b06fcb1e2..01f8feec153 100644 --- a/src/Templates/Boilerplate/Bit.Boilerplate/.docs/19- Project Miscellaneous Files.md +++ b/src/Templates/Boilerplate/Bit.Boilerplate/.docs/19- Project Miscellaneous Files.md @@ -446,7 +446,7 @@ dnx Bit.Cli --prerelease -- translate - **Model Context Protocol**: A standardized way to connect AI assistants (like GitHub Copilot) to external data sources and tools **Configured Servers**: -- **bitplatform**: The bit platform documentation server. It exposes the bit BlazorUI tools (`SearchBitBlazorUI`, `GetBitBlazorUIComponent`, `GetBitBlazorUIComponentExamples`, ...), the Butil/Bswup/Bmotion/Bink/Balette/BlazeDb/Bonboarding/Blex/Bmage/Brouter tools, the `AskGitHubRepository` tool for the third-party libraries this project builds on, the tools that read the source code of the bitfoundation/bitplatform repository itself rather than this project (`FindBitPlatformSymbols`, `GetBitPlatformSymbolSource`, ...), and the Microsoft Docs tools - all answering from the shipped assemblies, documentation and source rather than from the model's memory +- **bitplatform**: The bit platform documentation server. It exposes the bit BlazorUI tools (`SearchBitBlazorUI`, `GetBitBlazorUIComponent`, `GetBitBlazorUIComponentExamples`, ...), the Butil/Bswup/Bmotion/Bink/Balette/BlazeDb/Bonboarding/Blex/Bmage/Bsqlite/Brouter tools, the `AskGitHubRepository` tool for the third-party libraries this project builds on, the tools that read the source code of the bitfoundation/bitplatform repository itself rather than this project (`FindBitPlatformSymbols`, `GetBitPlatformSymbolSource`, ...), and the Microsoft Docs tools - all answering from the shipped assemblies, documentation and source rather than from the model's memory - **aspire**: The .NET Aspire CLI's own MCP server, for inspecting the running application's resources, logs and traces - **playwright**: Browser automation, so the agent can drive and verify the running app diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Pages/Home/Products.razor b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Pages/Home/Products.razor index 7e0a199a6cf..5f701448fa2 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Pages/Home/Products.razor +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Pages/Home/Products.razor @@ -185,6 +185,25 @@ }, new() + { + Title = "SQLite and EF Core, in the browser tab", + Subtitle = "bit Bsqlite · EF Core on SQLite for Blazor", + LinkUrl = Urls.Bsqlite, + External = true, + Body = @ + + bit Bsqlite runs SQLite and the stock EF Core provider in a worker in the user's browser, over OPFS, IndexedDB + or Cache Storage, with the same entities and migrations as the server and live queries across tabs. + +
+ builder.Services.AddBitBsqliteServices(sqlite => sqlite + .AddStorageProvider<BsqliteOpfsStorageProvider>() + .UseStorage(s => s.Prefer("opfs")) + .AddDatabase("app.db")); +
+ + }, + new() { Title = "Ink that feels like paper", Subtitle = "bit Bink · Blazor-native drawing and ink", diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Footer.razor b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Footer.razor index 53d723f908a..fcd6c2fe59e 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Footer.razor +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Footer.razor @@ -17,6 +17,7 @@ Brouter Bmotion Bmage + Bsqlite Bink Balette BlazeDb diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Header.razor.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Header.razor.cs index 849e00f303e..366922eb098 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Header.razor.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Header.razor.cs @@ -25,6 +25,7 @@ private sealed record ProductMenuItem(string Title, string Description, string U new("Brouter", "Modern declarative Blazor router", Urls.Brouter, External: true), new("Bmotion", "Blazor-native animation library", Urls.Bmotion, External: true), new("Bmage", "Blazor-native image processing", Urls.Bmage, External: true), + new("Bsqlite", "SQLite and EF Core in the browser", Urls.Bsqlite, External: true), new("Bink", "Blazor-native drawing and ink", Urls.Bink, External: true), new("Balette", "Blazor-native command palette", Urls.Balette, External: true), new("BlazeDb", "Memory-first database for Blazor", Urls.BlazeDb, External: true), diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Urls.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Urls.cs index e496701c6be..10bc869cdf3 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Urls.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Client/Shared/Urls.cs @@ -24,6 +24,7 @@ public static class Urls public const string Brouter = "https://brouter.bitplatform.dev/"; public const string Bmotion = "https://bmotion.bitplatform.dev/"; public const string Bmage = "https://bmage.bitplatform.dev/"; + public const string Bsqlite = "https://bsqlite.bitplatform.dev/"; public const string Bink = "https://bink.bitplatform.dev/"; public const string Balette = "https://balette.bitplatform.dev/"; public const string BlazeDb = "https://blazedb.bitplatform.dev/"; diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/CodebaseMemoryCli.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/CodebaseMemoryCli.cs index 2ccca128e69..2cb151c6098 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/CodebaseMemoryCli.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/CodebaseMemoryCli.cs @@ -12,7 +12,7 @@ public partial class CodebaseMemoryCli { /// /// The parts of the repository that get indexed, and so the only code the source tools reach. - /// What is left out answers for itself elsewhere on this endpoint: BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Brouter, + /// What is left out answers for itself elsewhere on this endpoint: BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Brouter, /// Butil and Bswup each have their own MCP server here, written for the library rather than grepped out of it. /// internal static readonly string[] IndexedPaths = diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpDocumentationSite.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpDocumentationSite.cs index 039d0b83bb2..dc8f22c2b9e 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpDocumentationSite.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpDocumentationSite.cs @@ -24,6 +24,7 @@ public static readonly (string UpstreamName, string ProjectPath)[] Projects = ("bitBlex", "src/Blex/Demo/Server/Bit.Blex.Demo.Server.csproj"), ("bitBmage", "src/Bmage/Bit.Bmage.Demo/Server/Bit.Bmage.Demo.Server.csproj"), ("bitBonboarding", "src/Bonboarding/Demo/Server/Bit.Bonboarding.Demo.Server.csproj"), + ("bitBsqlite", "src/Bsqlite/Demo/Server/Bit.Bsqlite.Demo.Server.csproj"), ("bitBrouter", "src/Brouter/Bit.Brouter.Demo/Server/Bit.Brouter.Demo.Server.csproj"), ("bitBswup", "src/Bswup/Bit.Bswup.Demo/Server/Bit.Bswup.Demo.Server.csproj"), ("bitButil", "src/Butil/Bit.Butil.Demo/Server/Bit.Butil.Demo.Server.csproj"), diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpFeedbackTool.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpFeedbackTool.cs index 6ad00a81226..df048d78156 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpFeedbackTool.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpFeedbackTool.cs @@ -38,7 +38,7 @@ public partial class McpFeedbackTool Title = "Send feedback to the bit platform team", Description = """ Sends a defect report, or any other feedback, about the bit platform straight to the team that maintains - it: the BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Butil, Bswup and Brouter libraries, the bit Boilerplate project template, and + it: the BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Butil, Bswup and Brouter libraries, the bit Boilerplate project template, and this endpoint itself. It reaches people in a chat rather than an issue tracker, so call it only once the user has agreed that the report should be sent - never to take a note, and never to ask whether something is a bug. Write the report so that it stands on its own without the conversation it came out of: what you diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpProxyService.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpProxyService.cs index 837cad5d9a0..e51832c7511 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpProxyService.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Services/Mcp/McpProxyService.cs @@ -43,7 +43,7 @@ Asks a question about a public GitHub repository and answers it from that reposi - ZiggyCreatures/FusionCache: hybrid caching, L2 cache backplane, distributed locking, OpenTelemetry integration and cache factory configuration. - keycloak/keycloak: Keycloak and realm configuration. - microsoft/agent-framework: agent creation, multi agent orchestration, workflows, tools and function calling, MCP, A2A communication, memory and context, and provider integrations. - Do not ask it about the bit platform libraries themselves: BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Butil, Bswup and Brouter each have + Do not ask it about the bit platform libraries themselves: BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Butil, Bswup and Brouter each have their own dedicated tools on this server, and those are written from the library rather than inferred from it. """; @@ -51,7 +51,7 @@ Asks a question about a public GitHub repository and answers it from that reposi /// Ends the description of every CodebaseMemory tool: an agent may also have a codebase-memory server of its /// own, which answers about its own project. /// - private const string bitPlatformSourceScope = "It reads the source code of the bitfoundation/bitplatform GitHub repository at the release this connection asked for (the v query of this endpoint, the newest release when it names none or one not served yet, as the server instructions say), never the caller's own workspace, not even a project created from a bit template. Of that repository it holds the bit Boilerplate project template, the bit platform websites, the build time projects and two bit libraries, Besql (src/Besql) and BlazorES2019 (src/BlazorES2019), but none of the other bit libraries: BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Brouter, Butil and Bswup each answer for themselves through their own tools on this server, so a question about a component, a theme, an animation, a drawing canvas, a command palette, a browser database, a product tour, state management, image processing, routing or the service worker goes to those rather than to a search here."; + private const string bitPlatformSourceScope = "It reads the source code of the bitfoundation/bitplatform GitHub repository at the release this connection asked for (the v query of this endpoint, the newest release when it names none or one not served yet, as the server instructions say), never the caller's own workspace, not even a project created from a bit template. Of that repository it holds the bit Boilerplate project template, the bit platform websites, the build time projects and two bit libraries, Besql (src/Besql) and BlazorES2019 (src/BlazorES2019), but none of the other bit libraries: BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Brouter, Butil and Bswup each answer for themselves through their own tools on this server, so a question about a component, a theme, an animation, a drawing canvas, a command palette, a browser database, a product tour, state management, image processing, an EF Core database in the browser, routing or the service worker goes to those rather than to a search here."; /// /// Every tool of this upstream takes a project argument naming the index to answer from, a deployment diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Server/SignalR/AppHub.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Server/SignalR/AppHub.cs index 7a1ba3a00f4..0f2ab99dd97 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Server/SignalR/AppHub.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Server/SignalR/AppHub.cs @@ -56,7 +56,7 @@ public async IAsyncEnumerable Chatbot( Name = "bitplatform", Endpoint = mcpEndpoint, TransportMode = HttpTransportMode.StreamableHttp - }), new() { }, loggerFactory, cancellationToken); // provides the per product tools (bit BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Brouter, Butil, Bswup, ...), the bitplatform source code tools and the general AskGitHubRepository tool + }), new() { }, loggerFactory, cancellationToken); // provides the per product tools (bit BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Brouter, Butil, Bswup, ...), the bitplatform source code tools and the general AskGitHubRepository tool var bitplatformMcpTools = await bitplatformMcp.ListToolsAsync(cancellationToken: cancellationToken); @@ -122,7 +122,7 @@ await scope.ServiceProvider.GetRequiredService() configuration.GetRequiredSection("AppSettings:ChatOptions").Bind(chatOptions); const string supportSystemPrompt = """ - You are the AI assistant of bitplatform (https://bitplatform.dev). You answer questions about every product the bit platform team ships - bit BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Brouter, Butil, Bswup, Besql, the bit Boilerplate project template and the rest - and you route support and sales requests to a human. + You are the AI assistant of bitplatform (https://bitplatform.dev). You answer questions about every product the bit platform team ships - bit BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Brouter, Butil, Bswup, Besql, the bit Boilerplate project template and the rest - and you route support and sales requests to a human. **RELEVANCE:** - A query is relevant only if it concerns bitplatform's products, their features, setup or usage, support topics, or product recommendations tied to bitplatform. .NET, Blazor, MAUI and ASP.NET Core topics are relevant only while they serve a bitplatform answer. diff --git a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Startup/Services.cs b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Startup/Services.cs index 4719dfbca95..40f9e182e8e 100644 --- a/src/Websites/Platform/src/Bit.Websites.Platform.Server/Startup/Services.cs +++ b/src/Websites/Platform/src/Bit.Websites.Platform.Server/Startup/Services.cs @@ -169,7 +169,7 @@ private static void AddMcpServer(IServiceCollection services) Title = "bit platform", Version = typeof(Services).Assembly.GetName().Version!.ToString() }; - options.ServerInstructions = "Provides the tools of every MCP server the bit platform team develops against, including the bit BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Brouter, Butil and Bswup documentation servers, plus a source code index of the bitfoundation/bitplatform repository."; + options.ServerInstructions = "Provides the tools of every MCP server the bit platform team develops against, including the bit BlazorUI, Bmotion, Bink, Balette, BlazeDb, Bonboarding, Blex, Bmage, Bsqlite, Brouter, Butil and Bswup documentation servers, plus a source code index of the bitfoundation/bitplatform repository."; }) // Stateless: no session state is kept between requests, so the endpoint keeps working when the // site runs behind a load balancer without session affinity. Nothing is lost by it here, since