Restart option added in case of unexpected termination by sakim8048 · Pull Request #64 · zadorlab/pynta

sakim8048 · 2025-03-27T15:57:39Z

This PR includes two major updates. Updated restart option will restart optimization or vibration calculations if previous runs are unexpectedly terminated. Also this PR will allow users to run Pynta efficiently on ALCF Polaris machine. Details are described below:

Restart Option

Upon running restart(), it will retrieve Fireworks workflow information, including the workflow ID number, task ID number, task states, and launch directories where unexpectedly terminated calculations were running. Before rerunning Fireworks for the incomplete runs, all necessary files, such as optimization trajectory files or vib folders, will be copied and sent to the destination directory.

If task states are not completed (e.g., fizzled or lost runs), the optimization runs will restart from the last geometry of the optimization trajectory file in the previous launch directory. In the case of a vibration restart, empty vibration JSON files will be deleted from the vib folder before rerunning the vibration.

Running Pynta on ALCF Polaris

With Raymundo's efforts, this PR allows Pynta to run on ALCF Polaris with a single queue allocation. Raymundo updated the way Pynta maps tasks on each node for ALCF machines. Each task runs on a different Fireworker, and each Fireworker is associated with a node. This is available for multilauncher. The optimal approach is to set num_jobs in Pynta input script to the number of nodes.

…start job. Delete empty vibration cache file in vib folder for vibration restart job

…irectory to a destination subdirectory

add machine keyword in firework task Function copyDataAndSave() is added to copy a file from a origin subdirectory to a destination subdirectory add machine keyword to vibration keywords in setup_adsorbates

Function copyDataAndSave() is added to copy a file from a origin subdirectory to a destination subdirectory add machine keywords to optimization and vibration firework tasks add machine keyword in firework task Function copyDataAndSave() is added to copy a file from a origin subdirectory to a destination subdirectory add machine keyword to vibration keywords in setup_adsorbates add machine keywords to optimization and vibration firework tasks

…t task and delete empty json file in vib folder for vib task

mjohnson541

Thanks! I went over most of it.

mjohnson541 · 2025-04-23T17:35:33Z

+import os
+from fireworks.core.fworker import FWorker
+
+def createCommand(node, software):


I'm okay with support for NERSC and general ALCF machines, but expecting users to write a whole new file for each machine is not tenable. If this polaris specific we need to redesign. It looks more like a hack to get this to work.

We can simply establish a better protocol to pass this information. I would recommend we write commands in the format:
"mpiexec --hosts {node} ......... {binary} PREFIX...." then we can instead simply tell users in the documentation if you put {node} or {binary} in the command code Pynta will automatically insert that. Then we don't need this.

mjohnson541 · 2025-04-23T17:35:56Z

+import os
+from fireworks.core.fworker import FWorker
+
+def createCommand(node, software):


function names should be snake case (also below)

mjohnson541 · 2025-04-23T17:38:29Z

@@ -0,0 +1,297 @@
+"""


Clearly this code was taken from somewhere. How is this code licensed? Can we put it in Pynta? Where is it from and is there a reason this isn't included in fireworks?

mjohnson541 · 2025-04-23T17:41:02Z

+
+
+# TODO: why is loglvl a required parameter??? Also nlaunches and sleep_time could have a sensible default??
+def launch_multiprocess2(


This definitely needs a different name. What distinguishes this from launch_multiprocess?

mjohnson541 · 2025-04-23T23:54:19Z

 from fireworks.core.fworker import FWorker
 import fireworks.fw_config
 import logging
+#restart RHE


Remove these attribution comments. They are confusing, especially here and if anyone wants to know who edited a section of the code they can look at the blame.

mjohnson541 · 2025-04-24T00:26:38Z

    del constraint_dict["type"]
    return constructor(**constraint_dict)
+
+def copyDataAndSave(origin, destination, file):


mjohnson541 · 2025-04-24T00:28:46Z

+
+def copyDataAndSave(origin, destination, file):
+    '''
+		Function to copy a file from a origin subdirectory to a destination subdirectory.


I don't understand why this function needs to exist. What makes it unviable to just use shutil.copy?

mjohnson541 · 2025-04-24T00:29:49Z


-                vib_obj_dict = {"software": self.software, "label": adsname, "software_kwargs": software_kwargs,
-                    "machine": self.machine, "constraints": ["freeze up to "+str(self.nslab)]}
+                vib_obj_dict = {"software": self.software, "label": ad, "software_kwargs": software_kwargs,


Something is messed up with the changes in this line, I think you were doing changes in one commit and then undoing them in another.

mjohnson541 · 2025-04-24T00:30:26Z

                            print(f'No directory named "vib" found in {src}.')
                            print('No vibration calculations executed: Check optimization runs are finished and optimized geometries are collected.')              
-                            
+


Remove these whitespace changes.

mjohnson541 · 2025-04-24T00:31:20Z

                    xyz = os.path.join(self.path,"Adsorbates",adsname,str(prefix),str(prefix)+".xyz")
                    xyzs.append(xyz)
                    fwopt = optimize_firework(os.path.join(self.path,"Adsorbates",adsname,str(prefix),str(prefix)+"_init.xyz"),
-                        self.machine,self.software,"weakopt_"+str(prefix),


Most of this should have probably been squashed with the changes in main...which may help solve the issue with some of the doing-undoing changes.

It seems that #45 somehow changed the fmax used in the IRC calculation from the intended fmaxirc to fmaxopt.

…orbate_2D

The fireworks gui will not display workflows with newer versions of flask.

… within pynta

…ed with a target configuration

when we attempt to convert sites with a vdW bond only into a group RMG gives us an atom with an Xv atomtype that cannot match other sites in the isomorphism so we convert all of these into X atomtypes

the adsorbates get generated in the Adguess fireworks now

…or ts

…he TSguess

this changes the electron distribution algorithm to during iterations to always first search for atoms who only have one way of distributing bonds to achieve octet and satisfy them, only relying on the prioritization scheme when there are multiple ways to distribute electrons to achieve octet for all remaining atoms

…ate in HFSP (usually means a reactant/product is very wrong)

this should help avoid errors related to folder name length

this keeps behavior the same for pysidt v1.1.0, there's a good chance it is desirable to have weigh_node_selection_by_occurrence or at least some sort of hybrid algorithm between doing so and not, but for now we keep it consistent

…start job. Delete empty vibration cache file in vib folder for vibration restart job

…irectory to a destination subdirectory

add machine keyword in firework task Function copyDataAndSave() is added to copy a file from a origin subdirectory to a destination subdirectory add machine keyword to vibration keywords in setup_adsorbates

rayhe88 and others added 8 commits March 24, 2025 14:25

polaris mapping info added in polaris.py

d83b5f1

launch several Rockets in a parallel environment

55d8479

Get information of the workflow and restart Pynta for optimization re…

dea3cad

…start job. Delete empty vibration cache file in vib folder for vibration restart job

add machine keyword in firework task

24cfadb

Function copyDataAndSave() is added to copy a file from a origin subd…

8bac114

…irectory to a destination subdirectory

add machine keywords to optimization and vibration firework tasks

e222b67

add machine keyword in firework task Function copyDataAndSave() is added to copy a file from a origin subdirectory to a destination subdirectory add machine keyword to vibration keywords in setup_adsorbates

restart module update: copy traj file to destination directory for op…

aa61306

…t task and delete empty json file in vib folder for vib task

sakim8048 requested review from mjohnson541 and tdprice-858 and removed request for mjohnson541 March 28, 2025 00:24

mjohnson541 requested changes Apr 24, 2025

View reviewed changes

mjohnson541 added 17 commits March 25, 2026 14:20

Calculate IRC with fmaxirc not fmaxopt

8a529e4

It seems that #45 somehow changed the fmax used in the IRC calculation from the intended fmaxirc to fmaxopt.

correctly find index of the site in adslabmol to provide to add_coads…

6f8be92

…orbate_2D

fix naming issues in TSest task adaption

03e0af4

add safety ensuring Molecule updating in geometricanalysis

25de3ae

update Molecule objects after merge actions

1d6fdf5

use copies of the 2D adslabmol when looping through possible sites

f21f4d2

specify flask in environment.yml

804249e

The fireworks gui will not display workflows with newer versions of flask.

bugfixes for adsorbate guess generation, particularly for bidentates

a5248be

enable use of any ASE calculator for harmonically forced calculations…

8798433

… within pynta

take harmonically forced software inputs

1ec701c

adapt adsorbate guess generation to software flexibility

41b623c

adapt TSest to harmonically forced software flexibility

6bc5774

rename functions to remove xtb specific names

649c70a

remove unnecessary import of HarmonicallyForcedXTB

55e2c34

handle TBLite calculator import in name_to_ase_software

220ffc2

handle XTB and TBLite the same for testing purposes

1e1f58f

add test for harmonically forced optimization

43283be

mjohnson541 and others added 30 commits March 25, 2026 14:24

add analyze_covdep_sample_data for looking at covdep samples associat…

37f56c8

…ed with a target configuration

add coverage dependence postprocessing notebook

556a86d

automatically apply and enforce reaction indexing

40adfd6

postprocessing exception handling

b5d9fc7

fix bug related to Xv generation

db8339c

when we attempt to convert sites with a vdW bond only into a group RMG gives us an atom with an Xv atomtype that cannot match other sites in the isomorphism so we convert all of these into X atomtypes

get adsorbate mappings directly from their info file

bc86712

the adsorbates get generated in the Adguess fireworks now

use single bond comparisons to avoid issues with generating the map f…

fa402a1

…or ts

throw error if reactant/product is missing in TSguess

a0fd38a

properly compute num_surf_sites in TSguess

f299137

allow to keep vdW bonds if they exist in the starting adsorbate for t…

7d475db

…he TSguess

skip structures if we cannot find occupied sites matching the template

20fa801

skip configurations that cannot be converted to 2D in HFSP

0bf6edd

skip 2D configurations that cannot be properly labeled from the templ…

973c680

…ate in HFSP (usually means a reactant/product is very wrong)

handle ValueError in TS validation

8b1a3a6

add additional print statements to HFSP logging

55e2937

improve missing reactant or product logging

298bb11

fix flipped boolean in fix_bond_orders

4f07cd2

handle desorption of vdW species when adding them as a coadsorbate

543dfd5

compress the adjacency list identifier

d5c03cf

this should help avoid errors related to folder name length

prefer HFSP directions that have fewer vdW bonded species

16b1476

increment to python 3.9

6a91a78

turn off verbosity for GFN1-xTB

23f8ba3

use python 3.9 in CI

cb0f06b

tweak CI workflow

b921162

handle versioning

f818782

this keeps behavior the same for pysidt v1.1.0, there's a good chance it is desirable to have weigh_node_selection_by_occurrence or at least some sort of hybrid algorithm between doing so and not, but for now we keep it consistent

Get information of the workflow and restart Pynta for optimization re…

5d42443

…start job. Delete empty vibration cache file in vib folder for vibration restart job

add machine keyword in firework task

5428bfb

Function copyDataAndSave() is added to copy a file from a origin subd…

9206bde

…irectory to a destination subdirectory

add machine keywords to optimization and vibration firework tasks

2430336

add machine keyword in firework task Function copyDataAndSave() is added to copy a file from a origin subdirectory to a destination subdirectory add machine keyword to vibration keywords in setup_adsorbates

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Restart option added in case of unexpected termination #64

Restart option added in case of unexpected termination #64
sakim8048 wants to merge 166 commits into
masterfrom
pynta-restart-updated

sakim8048 commented Mar 27, 2025

Uh oh!

mjohnson541 left a comment

Uh oh!

mjohnson541 Apr 23, 2025

Uh oh!

mjohnson541 Apr 23, 2025

Uh oh!

mjohnson541 Apr 23, 2025

Uh oh!

mjohnson541 Apr 23, 2025

Uh oh!

mjohnson541 Apr 23, 2025

Uh oh!

mjohnson541 Apr 24, 2025

Uh oh!

mjohnson541 Apr 24, 2025

Uh oh!

mjohnson541 Apr 24, 2025

Uh oh!

mjohnson541 Apr 24, 2025

Uh oh!

mjohnson541 Apr 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants



		# TODO: why is loglvl a required parameter??? Also nlaunches and sleep_time could have a sensible default??
		def launch_multiprocess2(

		print(f'No directory named "vib" found in {src}.')
		print('No vibration calculations executed: Check optimization runs are finished and optimized geometries are collected.')

Uh oh!

Conversation

sakim8048 commented Mar 27, 2025

Uh oh!

mjohnson541 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants