Ensembles

Provides an interface for applying operators to groups of similar objects.

Extended Description

A standard model for a time-dependent system with an unknown $x^k$ at time step $k$ is

\[\begin{aligned} x^k &= f(x^{k-1}), \\ y^k &= h(x^k), \end{aligned}\]

where $f$ transitions the state to a new time and $h$ create an observation $y$.

Ensemble-based methods solve the problem of estimating the state based on observed data by simulating multiple guesses for $x$ and updating them to reflect the observed data.

For an ensemble $X = \{x_1, \ldots, x_N\}$ with $N$ members, there are three main computational steps,

\[\begin{aligned} X^k &= \{f(x^{k-1}_1), \ldots, f(x^{k-1}_N) \},\\ Y^k &= \{h(x^k_1), \ldots, h(x^k_N) \}, \\ X^k_{new} &= \text{assimilate}(X^k, Y^k, y^*_0). \end{aligned}\]

Computing $X^k$ from $X^{k-1}$ and computing $Y^k$ from $X^k$ can trivially be decomposed into $N$ independent computations, making parallelization trivial. The assimilate step uses all the ensemble members and can be one of many algorithms, such as the ensemble Kalman filter.

This package provides an interface for working with this problem.

The two features of this package are:

1. Support trivially parallelizing operators that apply to each ensemble member independently.
2. Provide a common interface for ensemble-based data assimilation algorithms.

Once the operators $f$ and $h$ have been implemented for a single ensemble member, any ensemble-based data assimilation algorithm can be applied to the problem and easily compared with another.

Represents a collection of model states.

(M::AbstractOperator)(ensemble, args...; inplace=false)

Shorter syntax for apply_operator(M, ensemble, args...) or apply_operator!(M, ensemble, args...).

Keyword argument inplace determines whether in-place version of apply_operator is called.

Unstable interface

Ensemble(members, state_keys; <keyword arguments>)
Ensemble(members; <keyword arguments>)
Ensemble(member_type, state_keys; <keyword arguments>)

AbstractEnsemble implemented with a Vector of Dicts.

Arguments

• members::Vector{Dict}: the model states.
• state_keys::Vector{Dict}: the keys used to index the model states for conversion to an array. Other keys are ignored during computations. If unspecified, the sorted keys of the first ensemble member are used.
• monolithic_storage::Bool = true: whether to store the ensemble in a single file when saved.
  • true indicates the ensemble will be saved in a single file.
  • false indicates each ensemble member will be saved in a separate file.

Examples

Specify both the members and the state keys.

julia> members = [Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]
2-element Vector{Dict{Symbol, Any}}:
 Dict(:state => [0.7], :i => 1)
 Dict(:state => [0.5], :i => 2)

julia> ensemble = Ensemble(members, [:state]);

julia> @test ensemble.members == members;

julia> @test ensemble.state_keys == [:state];

Create ensemble with keys taken from first ensemble member.

julia> ensemble = Ensemble(members);

julia> @test ensemble.members == members;

julia> @test ensemble.state_keys == [:i, :state];

Create empty ensemble with specified keys.

julia> ensemble = Ensemble(Dict{Symbol, Any}, [:state]);

julia> @test ensemble.members == [];

julia> @test ensemble.state_keys == [:state];

Ensemble(ensemble, members, state_keys)

Create a new ensemble using the given ensemble's parameters, which is just monolithic_storage for now.

Examples

julia> ensemble = Ensemble(Dict{Symbol, Any}, [:state]; monolithic_storage = false);

julia> members = [Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])];

julia> ensemble = Ensemble(ensemble, members, [:state]);

julia> @test ensemble.members == members;

julia> @test ensemble.state_keys == [:state];

julia> @test ensemble.monolithic_storage == false;

Ensemble(ensemble, members)

Create a new ensemble using the given ensemble's parameters, which is just monolithic_storage for now.

Examples

julia> ensemble = Ensemble(Dict{Symbol, Any}, [:state]; monolithic_storage = false);

julia> members = [Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])];

julia> ensemble = Ensemble(ensemble, members);

julia> @test ensemble.members == members;

julia> @test ensemble.state_keys == [:i, :state];

julia> @test ensemble.monolithic_storage == false;

Unstable interface

IndexObserver(op, i)

Create an operator that applies op and then extracts element i from each state key of op.

Examples

julia> member = Dict(:i=>1, :state=>[0.7, 0.2]);

julia> M = IndexObserver(KeyObserver([:state]), 2);

julia> obs = M(member);

julia> @test obs == Dict(:state => member[:state][2]);

KeyObserver(state_keys)

Create an operator that copies the given state keys from each ensemble member.

Examples

julia> member = Dict(:i=>1, :state=>[0.7, 0.2]);

julia> M = KeyObserver([:state]);

julia> obs = M(member);

julia> @test obs == Dict(:state => member[:state]);

NoisyObserver(op::AbstractOperator; only_noisy=nothing, params)

Create a noisy version of op that runs op and then adds noise.

If only_noisy is false, the state keys returned by op are augmented with _noisy for the noisy versions, with the original state keys representing the clean versions.

Params

params is a dictionary.

• params["observation"]["noise_scale"]: standard deviation of noise.
• params["observation"]["seed"]: seed for random number generator. Defaults to 0.
• params["observation"]["rng"]: random number generator object. Defaults to Random.MersenneTwister(seed).
• params["observation"]["only_noisy"]: if true, operator returns only the noisy data instead of both noisy and clean. Defaults to false. Overrided by keyword argument only_noisy.

NoisyObserver(state_keys; params)

Create a noisy operator that observes the given state keys.

Unstable interface

merge!(e::Ensemble, e1::Ensemble)

Overwrite the first ensemble's members with corresponding values from the second ensemble.

The two ensembles must have the same size.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]);

julia> e1 = Ensemble([Dict(:state=>[0.3], :obs=>[3.5]), Dict(:state=>[0.1], :obs=>[2.3])]);

julia> merge!(e, e1);

julia> e.members
2-element Vector{Dict{Symbol, Any}}:
 Dict(:state => [0.3], :obs => [3.5], :i => 1)
 Dict(:state => [0.1], :obs => [2.3], :i => 2)

julia> @test e.state_keys == [:i, :state];

See also merge.

Create a new ensemble using values from the first ensemble overwritten with values from the second ensemble.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]);

julia> e1 = Ensemble([Dict(:state=>[0.3], :obs=>[3.5]), Dict(:state=>[0.1], :obs=>[2.3])]);

julia> e2 = merge(e, e1);

julia> e2.members
2-element Vector{Dict{Symbol, Any}}:
 Dict(:state => [0.3], :obs => [3.5], :i => 1)
 Dict(:state => [0.1], :obs => [2.3], :i => 2)

julia> @test e2.state_keys == [:i, :obs, :state];

See also merge!.

Base.show(io::IO, e::Ensemble)

Print the ensemble information on one line.

Examples

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])], [:state]);

julia> print(e)
Ensemble(Dict{Symbol, Any}[Dict(:state => [0.7], :i => 1), Dict(:state => [0.5], :i => 2)], [:state], true)

Base.show(io::IO, ::MIME"text/plain", e::Ensemble)

Print the ensemble information in a human-readable multi-line form.

Examples

julia> Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])], [:state])
Ensemble{Symbol, Any}:

members =
 Dict(:state => [0.7], :i => 1)
 Dict(:state => [0.5], :i => 2)

state_keys = [:state]
monolithic_storage = true

julia> Ensemble(Dict{Symbol, Any}[], [:state])
Ensemble{Symbol, Any}:

members = []
state_keys = [:state]
monolithic_storage = true

Helper function for converting variety of types to a vector

Helper function for assigning a vector to a variety of types.

apply_operator!(M::AbstractOperator, ensemble, args...)

Modify the given ensemble by merging each ensemble member with the result of M applied on that member with args... passed to each invokation of M.

Note: this does not change the state keys of the ensemble.

apply_operator(M::AbstractOperator, ensemble, args...)

Return a new ensemble consisting of M applied to each member of ensemble with args... passed to each invokation of M.

Return prior_state without modification.

This does not require observations.

Example

Simple example showing that the prior is unchanged.

julia> members = [Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]
2-element Vector{Dict{Symbol, Any}}:
 Dict(:state => [0.7], :i => 1)
 Dict(:state => [0.5], :i => 2)

julia> prior = Ensemble(members, [:state]);

julia> posterior1 = assimilate_data(nothing, prior, nothing, nothing, members[1]);

julia> posterior2 = assimilate_data(nothing, prior, nothing, nothing, members[2]);

julia> @test prior.members == posterior1.members;

julia> @test prior.members == posterior2.members;

assimilate_data(filter, prior_state, [prior_obs_clean,] prior_obs, y_obs, log_data=nothing)

Return an approximation for the posterior state obtained from assimilating observation y_obs based on the given prior state and prior observation.

If log_data is a dictionary, assimilate_data may log information to it.

divvy_among_workers(N, worker_id, num_workers)

Statically partition the N jobs among num_workers workers and return the job range for this worker.

Given a number of jobs N, a number of workers num_workers, and a particular worker's idworkeridin the range 1 tonumworkers, the job's are partitioned such that every worker hasfloor(N / numworkers)jobs, exceptN % numworkers` will have one extra job.

get_ensemble_dicts(ensemble, matrix)

Convert the matrix form of an ensemble to the Dict form.

Examples

julia> data = [
           -0.9; -0.3; -0.4; -0.7 ;;
            5.9;  5.3;  5.4;  5.7 ;;
       ];

julia> members = [
           Dict(:i=>1, :state=>[[0.1, 0.3], [0.2, 0.4]]),
           Dict(:i=>2, :state=>[[-1.6, 6.4], [2.6, 5.9]]),
       ];

julia> e = Ensemble(members, [:state]);

julia> members2 = get_ensemble_dicts(e, data);

julia> @test get_member_vector(e, members2[1]) == data[:,1];

julia> @test get_member_vector(e, members2[2]) == data[:,2];

See also get_member_dict!, get_ensemble_matrix.

get_ensemble_matrix(ensemble)

Convert an ensemble with N members to a matrix with N columns.

Each ensemble member must have the same length when vectorized with get_member_vector.

Examples

Convert an ensemble to a matrix.

julia> members = [
           Dict(:i=>1, :state=>[0.1]),
           Dict(:i=>2, :state=>[2.6]),
       ];

julia> e = Ensemble(members);

julia> get_ensemble_matrix(e)
2×2 Matrix{Float64}:
 1.0  2.0
 0.1  2.6

Convert an ensemble to a matrix based on the ensemble state keys.

julia> members = [
           Dict(:i=>1, :state=>[[0.1, 0.3], [0.2, 0.4]]),
           Dict(:i=>2, :state=>[[-1.6, 6.4], [2.6, 5.9]]),
       ];

julia> e = Ensemble(members, [:state]);

julia> get_ensemble_matrix(e)
4×2 Matrix{Float64}:
 0.1  -1.6
 0.3   6.4
 0.2   2.6
 0.4   5.9

See also get_member_vector, get_ensemble_dicts.

get_ensemble_members(ensemble::Ensemble)

Get the members of an ensemble.

get_ensemble_size(ensemble::Ensemble)

Get the number of members in an ensemble.

get_member_dict!(ensemble, member::Dict, data::Vector)

Write a data Vector to an ensemble member Dict.

This is the inverse of get_member_vector.

Examples

Write data to a vector.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7])]);

julia> e.members[1];

julia> data = [5, -0.4];

julia> get_member_dict!(e, e.members[1], data)
Dict{Symbol, Any} with 2 entries:
  :state => [-0.4]
  :i     => 5.0

julia> @test get_member_vector(e, e.members[1]) == data;

Limit the vectorizing attributes by specifying them in the Ensemble definition.

julia> e = Ensemble([Dict(:i=>1, :state=>[[0.1, 0.3], [0.2, 0.4]])], [:state]);

julia> data = [-0.9, -0.3, -0.4, -0.7];

julia> get_member_dict!(e, e.members[1], data)
Dict{Symbol, Any} with 2 entries:
  :state => [[-0.9, -0.3], [-0.4, -0.7]]
  :i     => 1

julia> @test get_member_vector(e, e.members[1]) == data;

See also get_member_vector, get_ensemble_matrix.

get_member_vector(ensemble, member::Dict)

Convert an ensemble member to a Vector based on ensemble.state_keys.

Examples

Convert simple ensemble members to vectors.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7])]);

julia> get_member_vector(e, e.members[1])
2-element Vector{Float64}:
 1.0
 0.7

Limit the vectorizing attributes by specifying them in the Ensemble definition.

```jldoctest julia> member1 = Dict(:i=>1, :state=>[[0.1, 0.3], [0.2, 0.4]]) Dict{Symbol, Any} with 2 entries: :state => [[0.1, 0.3], [0.2, 0.4]] :i => 1

julia> member2 = Dict(:i=>2, :state=>[[0.2, 0.6], [0.4, 0.8]]) Dict{Symbol, Any} with 2 entries: :state => [[0.2, 0.6], [0.4, 0.8]] :i => 2

julia> e = Ensemble([member1, member2], [:state]);

julia> getmembervector(e, e.members[1]) 4-element Vector{Float64}: 0.1 0.3 0.2 0.4

julia> getmembervector(e, e.members[2]) 4-element Vector{Float64}: 0.2 0.6 0.4 0.8

get_member_vector(state_keys, member::Dict)

Convert an ensemble member to a Vector.

The Vector is generated by by one-dimensionalizing each value in the member at the keys specified by state_keys.

Examples

Convert two ensemble members to vectors.

julia> member = Dict(:i=>1, :state=>[0.7]);

julia> get_member_vector([:i, :state], member)
2-element Vector{Float64}:
 1.0
 0.7

julia> get_member_vector([:state], member)
1-element Vector{Float64}:
 0.7

Works on multi-dimensional data.

julia> member = Dict(:i=>1, :state=>[[0.1, 0.3], [0.2, 0.4]])
Dict{Symbol, Any} with 2 entries:
  :state => [[0.1, 0.3], [0.2, 0.4]]
  :i     => 1

julia> get_member_vector([:i, :state], member)
5-element Vector{Float64}:
 1.0
 0.1
 0.3
 0.2
 0.4

julia> get_member_vector([:state], member)
4-element Vector{Float64}:
 0.1
 0.3
 0.2
 0.4

See also get_member_dict!, get_ensemble_matrix.

get_state_keys(M::AbstractOperator)

Return the keys that the given operator generates when applied to an ensemble member.

Should be implemented by each AbstractOperator.

install(pkg::Symbol)

Install unregistered package that is useful with the Ensembles package.

load_ensemble(stem)

Loads the ensemble saved at the given stem.

See save_ensemble_members for examples.

See also load_ensemble_members, load_ensemble, save_ensemble, and move_ensemble.

load_ensemble_members(folder_path, N)

Load the N ensemble members from the specified folder_path; uses the pattern "folder_path/i.jld2" for the i-th ensemble member.

See save_ensemble_members for examples.

See also load_ensemble, save_ensemble, and move_ensemble.

move_ensemble(stem, stem_new)

Moves an ensembled stored at stem to stem_new.

If an ensemble is already saved at stem_new, that ensemble is moved to stem_new-1.

Examples

Create and move an ensemble.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]; monolithic_storage = false);

julia> d = mktempdir(".");

julia> stem = joinpath(d, "e");

julia> save_ensemble(e, stem)

julia> stem_new = joinpath(d, "f");

julia> move_ensemble(stem, stem_new)

julia> e2 = load_ensemble(stem_new);

julia> @test e2.members == e.members;

julia> @test e2.state_keys == e.state_keys;

julia> @test e2.monolithic_storage == e.monolithic_storage;

Move an ensemble on top of an existing one.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]; monolithic_storage = false);

julia> d = mktempdir(".");

julia> stem = joinpath(d, "e");

julia> save_ensemble(e, stem)

julia> e2 = Ensemble([Dict(:i=>3, :obs=>[55.5]), Dict(:i=>4, :obs=>[34.3])]; monolithic_storage = true);

julia> d2 = mktempdir(".");

julia> stem2 = joinpath(d2, "e");

julia> save_ensemble(e2, stem2)

julia> move_ensemble(stem, stem2)
┌ Error: ./jl_SNKG0N/e.jld2 already exists. Moving existing version to ./jl_SNKG0N/e-1
└ @ Ensembles ~/a/curr_research/JutulJUDIFilter/Ensembles.jl/src/Ensembles.jl:527

julia> e_a = load_ensemble(stem2);

julia> @test e_a.members == e.members;

julia> @test e_a.state_keys == e.state_keys;

julia> @test e_a.monolithic_storage == e.monolithic_storage;

julia> e2_a = load_ensemble("$stem2-1");

julia> @test e2_a.members == e2.members;

julia> @test e2_a.state_keys == e2.state_keys;

julia> @test e2_a.monolithic_storage == e2.monolithic_storage;

See also save_ensemble_members, load_ensemble_members, load_ensemble, and save_ensemble.

Unstable interface

Unstable interface

save_ensemble(ensemble, stem; <keyword arguments>)

Save the given ensemble at the given file stem.

For monolithic storage, the whole ensemble is saved in "stem.jld2". Otherwise, the members are saved each to their own file, with any remaining ensemble parameters saved to "stem.jld2".

Arguments

• existing_member_directory: specifies the path to an existing directory where the ensemble members are saved. This directory takes precedence over ensemble.members.
• existing_merge::Bool: if true and the existing directory is given, merge the existing directory members with ensemble.members. If false, ensemble.members are not used.
• reset_state_keys::Bool: if true and the existing directory is given and existing_merge is true, then re-initialize the ensemble's state_keys based on the merged ensemble members. If false, the state keys are unchanged.

Examples

Save an ensemble to a single file.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]; monolithic_storage = true);

julia> d = mktempdir(".");

julia> stem = joinpath(d, "e");

julia> save_ensemble(e, stem)

julia> readdir(d)
1-element Vector{String}:
 "e.jld2"

julia> e2 = load_ensemble(stem);

julia> @test e2.members == e.members;

julia> @test e2.state_keys == e.state_keys;

julia> @test e2.monolithic_storage == e.monolithic_storage;

Save an ensemble across multiple files.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]; monolithic_storage = false);

julia> d = mktempdir(".");

julia> stem = joinpath(d, "e");

julia> save_ensemble(e, stem)

julia> readdir(d)
2-element Vector{String}:
 "e.jld2"
 "e_ensemble"

julia> readdir(joinpath(d, "e_ensemble"))
2-element Vector{String}:
 "1.jld2"
 "2.jld2"

julia> e2 = load_ensemble(stem);

julia> @test e2.members == e.members;

julia> @test e2.state_keys == e.state_keys;

julia> @test e2.monolithic_storage == e.monolithic_storage;

Save but replace the ensemble members with an existing ensemble stored across files.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]; monolithic_storage = false);

julia> d = mktempdir(".");

julia> stem = joinpath(d, "e");

julia> ensemble_dir = joinpath(d, "e_ensemble");

julia> save_ensemble(e, stem)

julia> e2 = Ensemble([Dict(:i=>3, :obs=>[55.5]), Dict(:i=>4, :obs=>[34.3])]; monolithic_storage = true);

julia> d2 = mktempdir(".");

julia> stem2 = joinpath(d2, "e");

julia> save_ensemble(e2, stem2; existing_member_directory=ensemble_dir)

julia> @test isdir(ensemble_dir) == false;

julia> e3 = load_ensemble(stem2);

julia> @test e3.members == e.members;

julia> @test e3.state_keys == e2.state_keys;

julia> @test e3.monolithic_storage == e2.monolithic_storage;

Save the ensemble but merge the ensemble members with an existing ensemble stored across files.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]; monolithic_storage = false);

julia> d = mktempdir(".");

julia> stem = joinpath(d, "e");

julia> ensemble_dir = joinpath(d, "e_ensemble");

julia> save_ensemble(e, stem)

julia> e2 = Ensemble([Dict(:i=>3, :obs=>[55.5]), Dict(:i=>4, :obs=>[34.3])]; monolithic_storage = true);

julia> d2 = mktempdir(".");

julia> stem2 = joinpath(d2, "e");

julia> save_ensemble(e2, stem2; existing_member_directory=ensemble_dir, existing_merge=true)

julia> @test isdir(ensemble_dir) == false;

julia> e3 = load_ensemble(stem2);

julia> @test e3.members == merge.(e.members, e2.members);

julia> @test e3.state_keys == [:i, :obs];

julia> @test e3.monolithic_storage == e2.monolithic_storage;

Reset the keys after merging.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]; monolithic_storage = false);

julia> d = mktempdir(".");

julia> stem = joinpath(d, "e");

julia> ensemble_dir = joinpath(d, "e_ensemble");

julia> save_ensemble(e, stem)

julia> e2 = Ensemble([Dict(:i=>3, :obs=>[55.5]), Dict(:i=>4, :obs=>[34.3])]; monolithic_storage = true);

julia> d2 = mktempdir(".");

julia> stem2 = joinpath(d2, "e");

julia> save_ensemble(e2, stem2; existing_member_directory=ensemble_dir, existing_merge=true, reset_state_keys=true)

julia> @test isdir(ensemble_dir) == false;

julia> e3 = load_ensemble(stem2);

julia> @test e3.members == merge.(e.members, e2.members);

julia> @test e3.state_keys == [:i, :obs, :state];

julia> @test e3.monolithic_storage == e2.monolithic_storage;

See also save_ensemble_members, load_ensemble_members, load_ensemble, and move_ensemble.

save_ensemble_members(ensemble::Ensemble, folder_path; <keyword arguments>)

Save each member of the ensemble to separate file with path "folder_path/i.jld2" with JLD2 key "data", where i is the ensemble member's index in the ensemble.

Arguments

• existing_member_directory: specifies the path to an existing directory where the ensemble members are saved. This directory takes precedence over ensemble.members.
• existing_merge::Bool: if true and the existing directory is given, merge the existing directory members with ensemble.members. If false, ensemble.members are not used.

Examples

Save an ensemble to a directory.

julia> e = Ensemble([Dict(:i=>1, :state=>[0.7]), Dict(:i=>2, :state=>[0.5])]);

julia> d = mktempdir(".");

julia> save_ensemble_members(e, d)

julia> readdir(d)
2-element Vector{String}:
 "1.jld2"
 "2.jld2"

julia> loaded_members = load_ensemble_members(d, 2);

julia> @test e.members == loaded_members;

Use an existing ensemble directory instead of the in-memory ensemble members.

julia> e2 = Ensemble([Dict(:i=>3, :state=>[3.1]), Dict(:i=>4, :state=>[2.7])]);

julia> d2 = mktempdir("."); rm(d2);

julia> save_ensemble_members(e2, d2; existing_member_directory=d)

julia> @test isdir(d) == false;

julia> readdir(d2)
2-element Vector{String}:
 "1.jld2"
 "2.jld2"

julia> loaded_members = load_ensemble_members(d2, 2);

julia> @test e.members == loaded_members;

julia> @test e2.members != loaded_members;

Merge an existing ensemble directory over the in-memory ensemble members.

julia> e3 = Ensemble([Dict(:i=>3, :obs=>[55.5]), Dict(:i=>4, :obs=>[34.3])]);

julia> d3 = mktempdir("."); rm(d3);

julia> save_ensemble_members(e3, d3; existing_member_directory=d2, existing_merge=true)

julia> @test isdir(d2) == false;

julia> readdir(d3)
2-element Vector{String}:
 "1.jld2"
 "2.jld2"

julia> loaded_members = load_ensemble_members(d3, 2);

julia> @test merge.(e.members, e3.members) == loaded_members;

See also load_ensemble_members, load_ensemble, save_ensemble`, and move_ensemble.

split_clean_noisy(M::AbstractNoisyOperator, ensemble_obs::AbstractEnsemble)

Given an operator and the ensemble generated by that operator, return two ensembles that represent the clean operator results and the noisy operator results.

For non-noisy operators, simply return the given ensemble and nothing.

For non-noisy operators, simply return the given member and nothing.

split_clean_noisy(M::AbstractNoisyOperator, member)

Given an operator and the ensemble member generated by that operator, return two members that represent the clean operator results and the noisy operator results.

Should be implemented for each AbstractNoisyOperator.

xor_seed!(M::AbstractNoisyOperator, seed_mod::UInt)

Modify the seed for M's random number generator.

Should be implemented for each AbstractNoisyOperator.