Skip to content

Use complex-aware preconditioner in BoundaryMode solver - #921

Merged
simlapointe merged 8 commits into
mainfrom
simlapointe/waveport-precond
Sep 15, 2026
Merged

simlapointe merged 8 commits into
mainfrom
simlapointe/waveport-precond

Conversation

@simlapointe

@simlapointe simlapointe commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Improve BoundaryMode linear solver convergence when lossy boundary conditions are present by including complex terms in the real-valued preconditioner ($P = A_r + A_i$, like in the 3D case) or using an exact complex-valued preconditioner when "WavePorts.ComplexCoarseSolve" is true. Closes #920.

@hughcars hughcars left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed the block assembly has to happen before the non-port ranks return: HypreParMatrixFromBlocks allgathers on the FE-space comm. benvial's numbers on #920 are the evidence this works. Nits inline, then rebase onto main once #919 is in and merge.

Comment thread palace/models/modeeigensolver.cpp Outdated
Comment on lines +536 to +539
// the FE space communicator. Wave-port sparse-direct solvers run on a subcommunicator,
// so assemble the complex-aware preconditioner on all ranks before non-port ranks
// return. The resulting real operator is already in the form expected by the wrapped
// sparse solver, avoiding another collective assembly in MfemWrapperSolver.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// the FE space communicator. Wave-port sparse-direct solvers run on a subcommunicator,
// so assemble the complex-aware preconditioner on all ranks before non-port ranks
// return. The resulting real operator is already in the form expected by the wrapped
// sparse solver, avoiding another collective assembly in MfemWrapperSolver.
// the FE space communicator. Wave-port sparse-direct solvers run on a subcommunicator,
// so assemble the complex-aware preconditioner on all ranks before non-port ranks
// return. direct_pc_op only carries the assembled real matrix (2N x 2N in the
// doubled-real case) into MfemWrapperSolver::SetOperator; its Mult is not meaningful.

port_linear.tol = data.ksp_tol;
if (data.complex_coarse_solve)
{
port_linear.complex_coarse_solve = *data.complex_coarse_solve;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With global "PCMatReal": true, port_linear.pc_mat_real is inherited and the !linear.pc_mat_real guard in ModeEigenSolver::Solve skips this entirely, so an explicit per-port "ComplexCoarseSolve": true is silently ignored. Give the explicit override precedence:

Suggested change
port_linear.complex_coarse_solve = *data.complex_coarse_solve;
port_linear.complex_coarse_solve = *data.complex_coarse_solve;
port_linear.pc_mat_real = false;

Note this also means an explicit per-port false gives Ar + Ai rather than Ar. I think that is the right reading of a per-port choice, but if you would rather preserve PCMatReal for that case, only clear it when the override is true.

Comment thread CHANGELOG.md Outdated
1-based index of the most recently completed adaptive mesh refinement (AMR) iteration
(`1` for the initial solve, matching the `iterationXX` archive subdirectory).
[PR 902](https://github.com/awslabs/palace/pull/902)
- Improve BoundaryMode linear solver convergence when lossy boundary conditions are present

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MaxIts/KSPTol fix is a separate bug, give it its own bullet. This will also need to move to In progress when you rebase.

@simlapointe
simlapointe force-pushed the simlapointe/waveport-precond branch from f1e1a66 to b389303 Compare September 15, 2026 21:19
@simlapointe
simlapointe merged commit dff1ef9 into main Sep 15, 2026
62 of 64 checks passed
@simlapointe
simlapointe deleted the simlapointe/waveport-precond branch September 15, 2026 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

push-containers Push the containers produced by CI to ECR and S3

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BoundaryMode/WavePort solve converges slowly in cases with lossy BCs

2 participants