Solver ecosystem revamp/refactor

[abelsiqueira]

Now that the NLPModels ecosystem has been refactored, it's time to think about the other half of the JSO-compliant solver.

Currently we have SolverTools.jl housing stats, logging, line search, trust region, and some other auxiliary tools for blas and bounds.

NLPModels + SolverTools define a JSO-compliant solver by the input (AbstractNLPModel) and output (GenericExecutionStats). We have following packages with JSO-compliant solvers (incomplete list):

• NLPModelsIpopt and NLPModelsKnitro: Just need stats.
• JSOSolvers: Uses everything in SolverTools.
• Percival: Uses stats, logging and bounds.
• CaNNOLeS: Use stats and logging. Could use line search if it were more general
• DCI and RipQP: Haven't looked at, but suspect only stats and logging

This suggests having stats and logging in a more basic package. Something like SolverCore or SolverBase. Then, the other tools could remain on SolverTools.

Furthermore, the tests can be factored out of JSOSolvers and generalized to other situations beyond "unconstrained" and "bound-constrained". There are 3 types of tests:

• Can the solver solve these simple problems?
• Can the solver solve this super simple problem but with type T and keep the type?
• Does the solver respect max_time and max_eval? (We don't have guidance on what should these keyword arguments be)

Opinions? This will all be breaking, so it's a good moment to add other things.

[dpo]

I think SolverCore is a good idea. I would also suggest defining an AbstractSolver struct there that other solvers will derive from. The idea is that a Solver instance should preallocate the storage necessary during a solve so that re-solves with a problem of the same size performs no allocations.

I also suggest defining structs of the form SolverParams for each solver. A call to a solver could then have the form

solver = TronSolver(model)  # only the problem dimensions are useful here
params = TronParams(...)
tron!(solver, model, params)  # stats are inside solver

[dpo]

Solver parameters that we expose and can tune.

[abelsiqueira]

I started a draft: https://github.com/JuliaSmoothOptimizers/SolverCore.jl/pull/6/files
Suggestions are welcome, specially for the parameter part. I used NamedTuple instead of a struct because they are cheaper. Although I didn't have much success defining the tuning problem.

[abelsiqueira]

It is also not clear how to deal with types.

[dpo]

Parameter types? Their domain is also important, so perhaps we need our own structure.

[abelsiqueira]

The allocated arrays types. I've updated the PR to show a possibility.

[abelsiqueira]

I've been testing the refactor/revamp, and it looks like that we can separate into a refactor and later into a revamp:
refactor

• SolverCore 0.1.0 + SolverTools 0.5.0 and update every solver that uses SolverTools to use SolverCore and maybe SolverTools.
• Separate JSOSolver tests into SolverTest 0.1.0
• Use tests from SolverTest into other solver, but nothing fancy
• Release new versions of everything

Should we try this, or try to do the revamp at the same time?

[dpo]

Refactor now, revamp later. Why do we need to split out solver tests? Aren’t they solver specific?

[abelsiqueira]

Some tests are, but some can be reused. For instance, this tests an unconstrained solver: https://github.com/JuliaSmoothOptimizers/JSOSolvers.jl/blob/master/test/solvers/unconstrained.jl

[abelsiqueira]

SolverCore 0.1.0 has been submitted and is waiting for merge in the registries. I'll submit SolverTest 0.1.0 today.

[abelsiqueira]

Current ecosystem proposal - WIP, may have erros

[frapac]

Hi all! First, I would like to thank you for this revamp. I like the new shape of SolverCore and NLPModels, which are more lightweight than before. With these changes, we are now planning to integrate the solver MadNLP more tightly into JSO's ecosystem.

Currently, MadNLP is shipped with its own interfaces to HLS and MUMPS. We are wondering whether we could reuse the interfaces from MUMPS.jl and HSL.jl. I think the only part that is missing currently is a generic abstraction for sparse linear solvers. MadNLP is currently defining its own AbstractLinearSolver structure, with the following interface:

abstract type AbstractLinearSolver end
mutable struct EmptyLinearSolver <: AbstractLinearSolver end

# dummy functions
introduce(::EmptyLinearSolver) = ""
factorize!(M::EmptyLinearSolver) = M
solve!(::EmptyLinearSolver,x) = x
is_inertia(::EmptyLinearSolver) = false
inertia(::EmptyLinearSolver) = (0,0,0)
improve!(::EmptyLinearSolver) = false
rescale!(::AbstractLinearSystemScaler) = nothing
solve_refine!(y,::AbstractIterator,x) = nothing

A first idea would be to implement something inheriting from Julia's linear algebra structures, but we won't have any proper interface to compute the inertia or to proceed to iterative refinement. That's why I like your idea of defining a new package LinearSolver to implement a generic interface for the sparse linear solvers we are using in optimization (if I read correctly the figure detailing the structure of the ecosystem).

In a nutshell, my question is: is a development of a package LinearSolver.jl in JSO still planned ? If so, I would be interested to integrate this package in MadNLP. And of course, I would be happy to bring some help in developing the package :-)

cc @sshin23

[abelsiqueira]

Hi @frapac, thanks for the interest. I am very glad to know you're planning to integrate MadNLP more closely with JSO.

We are interested in developing a more general framework for LinearSolvers. Both DCI and CaNNOLeS use HSL and LDLFactorizations for sparse symmetric systems, and that's one main motivation. We are swamped with other parts of the update, so it's great that you're willing to help in this part.

We are planning some basic requirements for solvers:

• They will inherit from (some abstract type that inherits from )^n AbstractSolver (e.g. HSLSolver <: SparseLinearSolver <: AbstractLinearSolver <: AbstractSolver)
• They will return an Output type that inherits from an AbstractOuptut (or some type... etc);
• They implement solve!(solver, problem), for some problem;
• Some constructor and parameters uniformity, yet undefined.

We decided to implement this bottom-up, to avoid breaking everything. @amontoison and @geoffroyleconte have been working on the KrylovSolvers type for a while, and I intend to do that for JSOSolvers, and later with @tmigot for DCI.

About MUMPS I don't know much, @dpo should be able to help.

[frapac]

Thanks @abelsiqueira for this detailed answer! I understand the point of implementing this bottom-up, as figuring out an interface for AbstractLinearSolver is far from being trivial. I see two potential steps to move further in integrating MadNLP more closely with JSO:

• move the HSL wrappers implemented here in HSL.jl (if you think we could benefit from integrating more solvers into HSL.jl)
• abstracting further the interface for sparse linear solver currently used in MadNLP, to give a basis for the discussion.

[abelsiqueira]

Those are great ideas. I think the PRs to HSL.jl will be a great addition. @dpo, any suggestions on how to proceed? Should we open all at once or one per solver?
I also like the idea for the abstract interface, but I'll let someone else with more experience chime in.

[dpo]

@frapac Could you provide a comparison of your interface to HSL with what’s already in HSL.jl?

I’m overdue for a complete revision of MUMPS.jl.

[sshin23]

Hey everyone! Thanks for the great ideas @frapac @abelsiqueira @dpo.

MadNLP.jl currently has its own interface to Ma27, Ma77, and Ma86 (it has own interface to Ma57 and Ma97, but HSL.jl already has interface to them). I believe we can move that to HSL.jl. But should be modified substantially before moving them to HSL.jl. E.g., they currently only work for real double precision arrays and some of the options are missing. So, I believe there are still substantial amout of work to be done.

Also we have a build script in MadNLP (here) that compiles HSL shared library from the source code. So we can also move that to HSL.jl. We may need to ask the user what type of tarball file do they have (currently MadNLP works with coinhsl-2015.06.23.tar.gz).

I can work on creating PR for HSL.jl in the next few months. I think in the PR we can include:

• interface to Ma27, Ma77, Ma86
• build script from coinhsl-2015.06.23.tar.gz

And synchronizing the abstraction for sparse solver sounds terrific. I'll check AbstractSolver and see if we can directly incorporate it into MadNLP.