Features/update existing pipes

[jnettels]

Summary

• Fix errors when using "existing" pipes for an optimization
• Update attributes of returned pipe network
• Make new simplify approach compatible (wait for Fix several regressions in 'simplify()' #166)
• Update documentation
• Update changelog
• Do some more tests in my own projects (marked as draft until then)

Problem / Motivation

dhnx has support for mixing new and existing pipes in the optimization.

In the current state, dhnx does not return existing pipe segments with the results of the optimization process. This is somewhat logical, if we ask "Dear dhnx, what new pipes do I need to invest in?"

A user can work out their own solution for combining the existing pipes from their input data with the new pipes from the dhnx results. However, to me, this feels very unintuitive. My question would be "Dear dhnx, what does the network look like that supplies heat to all the consumers?"

Furthermore, some information is lost along the way, specifically:

• What actual thermal power flow (kW) is required in each pipe segment, including the existing pipes?
• Which of the existing pipe segments are actually needed?
• What heat losses occur in the existing pipe segments?
• What is the flow direction in each existing pipe segment?

I think dhnx should provide a clean way to get this information, instead of forcing each user to understand and pick apart the oemof results. I found ways to retrieve all those information for the oemof results, but I was unsure how to store them in
network.results.optimization['components']['pipes']

My first instinct was to write a solution without breaking changes, where I only add new attributes

• "flow"
  • for new pipes, we know: "capacity" = "flow" + "losses"
  • for existing pipes, we only know: capacity >= flow + losses
• "capacity_existing"
  • This is set to status_nominal from oemof.
  • If the pipe was used, this is just the capacity the user used in the input data
  • In case an existing pipe is unused, this becomes 0.
• "direction_existing"
  • The flow direction is an optimization result

This would avoid changing the columns "capacity" and "direction" which can be interpreted as "what is the newly installed pipe capacity"
https://dhnx.readthedocs.io/en/stable/optimization_models.html#results

This prevents breaking changes, but feels less clean. What I now did instead:

Solution

• Add a new dhnx boolean setting "return_existing"
  • If False, keep columns as before
  • If True
    • "capacity" and "direction" are updated.
      • For existing pipes, "capacity" equals the input value if pipe is used, otherwise 0
    • "existing" is kept in the results
• Add the new column "flow" in any case, because for "existing pipe"-scenarios it is crucial and for new pipes it is nice to have.

This should avoid breaking changes as far as I can see.

Other noteworthy changes

• d0ffb08 Replace the deprecated nominal_value with nominal_capacity
• bbccf71 Run a fillna({"existing": 0} if that column is used in the input data. I ran into issues when I only defined existing=1 for those pipes and left the new pipes NaN.
• 2575bcb Use a more explicit condition if q["existing"] in [1]: instead of if q["existing"]: because it evaluates as True for NaN. I guess it was related to the point above.
• a4b7dfe When the setting bidirectional_pipes is False, new heatpipes are created in both directions, to allow potential flows in both directions. When trying to use existing pipes, I ran into errors like this
  ValueError: No value for uninitialized VarData object InvestmentFlowBlock.invest[infrastructure_heat_bus_forks-67,infrastructure_heat_pipe-generic_forks-67-forks-75,0]
  My network became solvable once I added the same logic for exising pipes.
• 3c0d9ee weld_segments() had to be updated, to be compatible with existing pipes.
  If there are existing pipes in the input data, we do not want to weld/merge/dissolve pipe segments with different capacities, which is what happened before. There is a new attribute retain_unique_values=['capacity'] that makes sure unique values like 'capacity' are preserved.
  • Superseded by simplify()
• d5d8dba Always derive flow direction from flow attribute. Use a consistent approach for new and existing pipes instead of using 'size' attribute, which yields inconsistencies for existing pipes and cases with 'bidirectional_pipes'=True.
• 8129f98 Add a FutureWarning for the switch from welding to simplify

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
src/dhnx
model.py
network.py
src/dhnx/gistools
connect_points.py					29
geometry_operations.py					149-153
src/dhnx/optimization
add_components.py
dhs_nodes.py					151
oemof_heatpipe.py					158-159, 247-253
optimization_models.py					524, 644
Project Total

This report was generated by python-coverage-comment-action

[jnettels]

I would delegate a new test of a complete solver workflow with cbc to a new PR. That should improve coverage massively.

[p-snft]

I have mixed feeling about this. The PR improves functionality, but even explicitly increases technical debt.

I would delegate a new test of a complete solver workflow with cbc to a new PR.

I cannot really blame you, as the architecture of dhnx isn't really modular, so (unit) testing also is an issue. At the moment, I feel like we need a big refactor anyway, but I do not see who has dedicated time for that purpose.

[jnettels]

I realized cbc was already installed in the workflows, so implementing new tests for the complete optimization process was quite simple, after all. I now included two new tests that cover several different options. Now we just crossed 70% total coverage. I feel like increasing the coverage further is beyond the scope of this PR.

[p-snft]

The test coverage does not really say anything, as we don't have unit tests here. It's only that the integration test touches 70 % of the code. (This will not help to find out where in the code stuff is broken if the test fails.)

As I think this PR is a clear improvement, I will merge. Thanks.