Let another solver own the loop

OpenPFC / Build a simulation

Let another solver own the loop

Export a borrowed host field, negotiate an interval and impose new data at the accepted endpoint.

Download runnable lesson (.zip)

Includes the model, inputs, required headers and a short run guide.

OpenPFC need not own the outermost time loop. This refactors the external coupling example into a small host orchestrator that reads a field, chooses an interval and imposes an endpoint value. It demonstrates the interface seam; the host is not an implemented finite-element solver.

Give the session a lifetime and a communicator

auto isolated = world.duplicate();
const auto domain =
    pfc::domain::create(pfc::GridSize{{8, 8, 2}}, pfc::PhysicalOrigin{{0, 0, 0}},
                        pfc::GridSpacing{{.25, .5, 1}});
const auto decomposition = pfc::decomposition::create(domain, isolated.size());
pfc::SimulationState state;
state.add_field("u", pfc::data::field_from_subdomain<double>(
                         decomposition, isolated.rank(), 0));
state.get_field<double>("u").apply([](double, double, double) { return 0.; });
pfc::Time time({0, .4, .1}, .15);
openpfc_examples::HostSourceModifier source;

Duplicating a communicator preserves the rank group while creating a separate message-matching context. The example uses that communicator for its host reduction. A coupled application must also pass the isolated communicator into each communicating OpenPFC operation it constructs; duplication alone cannot redirect an operation that still uses WORLD. The session and its fields are destroyed before MPI finalizes.

Borrow, read, impose, commit

while (!time.done()) {
  const double t0 = time.get_current();
  time.begin_attempt(std::min(.25, time.get_dt())); // Host and OpenPFC limits.
  const double dt = time.get_attempted_dt();
  const auto exported = pfc::coupling::export_host_field(state, "u");
  double local = 0, global = 0;
  for (std::size_t i = 0; i < exported.view.size(); ++i)
    local += exported.view.data()[i];
  MPI_Allreduce(&local, &global, 1, MPI_DOUBLE, MPI_SUM, isolated);
  source.apply(state, t0 + dt);
  time.commit_attempt();
  time.increment_step_success();
  output << time.get_increment() << ',' << t0 << ',' << dt << ','
         << time.get_current() << ',' << time.get_increment() << ',' << global
         << '\n';
}

The host allows 0.25 time units, while OpenPFC proposes 0.1. The clock takes the smaller limit and clips it again at save boundaries and the final time. export_host_field returns a read-only borrowed view plus geometry. It neither copies the field nor transfers ownership to the external solver. This example uses compact, unpadded storage, so its view contains only owned samples; a padded field requires more careful owned-region handling. The exported origin is the domain origin. Add owned_box.low to local indices before multiplying by spacing to recover physical coordinates.

The source adapter writes

\[u(x,y,z,t)=\sin x\cos y+0.1t. \tag{1}\]

inline double coupling_source(double x, double y, double, double t) {
  return std::sin(x) * std::cos(y) + .1 * t;
}
struct HostSourceModifier {
  std::string field_name{"u"};
  void apply(pfc::SimulationState &state, double time) const {
    state.get_field<double>(field_name).apply([time](double x, double y, double z) {
      return coupling_source(x, y, z, time);
    });
  }
};

It imposes values at the new endpoint; it is not an Euler increment or a force integrated over the interval. The initial field is zero, so the first host read is zero; subsequent reads see the last imposed field.

Start a host-owned experiment

openpfc run model.cpp input.json --launcher=none

The empty input is unused; edit the domain and time constants in C++. The sibling source.hpp is included in the CLI’s source snapshot. Two-rank execution uses --ranks=2 --launcher=mpirun. Each rank records the time negotiation and its complete final field under results/.

The checks verify the five clipped intervals, every host read against the previous accepted state, and all final samples against the imposed formula. Borrowed handles must not outlive or survive reallocation of their field. Use the modifier’s public field operations for writes so residency tracking can observe them. A rejected coupled attempt would also need to restore external solver state; this accept-only orchestrator does not hide that duty.

The complete C++ model and starting input are available with this lesson. It also uses the shared source.hpp; keep its relative location when saving the files.