Import Prior Alignment Issues

[jgermain22]

Hello!
I'm new to nextPYP so please let me know if I'm just doing something wrong. When I try to import from previous IMOD alignments Im getting a couple of errors. First, it asks for a .order file which I have found the file format for but when I make the order files in the same directory as the .xf and .tlt files, it won't link them into the project directory. If I place the order files into the directory then it will proceed without error. Is there a way to get around this?

Second, when I try to run the pre-process after adding the order files to the project folder, I get the following error

488	Tapering will be done over  16 pixels
489	
490	[DEBUG] pyp/system/local_run.py:63 | mv ts_001_driftcorr.prexg ts_001_driftcorr_first.prexg
491	[INFO] pyp/align/core.py:4954 | Error did not decrease, this will be the last iteration
492	[INFO] pyp/align/core.py:4962 | Max detected shift change at iteration 2 is 230.011 
493	[DEBUG] pyp/system/local_run.py:63 | mv ts_001_driftcorr_first.prexg ts_001_driftcorr.prexg
494	[DEBUG] pyp/system/local_run.py:63 | /opt/IMOD/bin/xfproduct ts_001_driftcorr.prexg ts_001_driftcorr_bin.xf ts_001_driftcorr.xf -scale 8,8
495	[DEBUG] pyp/system/local_run.py:68 | 
496	 FORMATTED  ro  file opened: /scratch/jgermain/7109698198339792807_1/ts_001_driftcorr/ts_001_driftcorr.prexg
497	
498	ERROR: XFPRODUCT - OPENING OR READING TRANSFORM FILE
499	
500	[INFO] pyp/utils/timer.py:75 | Tilt-series alignment took: 3.3s
501	[TRACE] pyp/system/local_run.py:63 | /opt/IMOD/bin/xf2rotmagstr ts_001_driftcorr.xf
502	[TRACE] pyp/system/local_run.py:68 | 
503	ERROR: XF2ROTMAGSTR - OPENING OR READING TRANSFORM FILE
504	
505	[INFO] pyp/utils/timer.py:75 | Total time in tomo_swarm 48.1s seconds
506	[ERROR] pyp/pyp_main.py:4932 | nextPYP (tomoswarm) failed
507	[INFO] pyp/pyp_main.py:4287 | Filesystem      Size  Used Avail Use% Mounted on
508	[INFO] pyp/pyp_main.py:4287 | /dev/nvme0n1    1.8T  824G  916G  48% /scratch
509	[INFO] pyp/pyp_main.py:7011 | Deleted temporary folder /scratch/jgermain/7109698198339792807_1
510	Traceback (most recent call last):
511	  File "/opt/pyp/bin/run/pyp", line 4927, in <module>
512	    tomo_swarm(args.path, args.file, args.debug, args.keep, args.skip)
513	    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
514	  File "/opt/pyp/src/pyp/utils/timer.py", line 97, in wrapper_timer
515	    return func(*args, **kwargs)
516	  File "/opt/pyp/bin/run/pyp", line 1997, in tomo_swarm
517	    tilt_metadata["tilt_axis_angle"] = get_tilt_axis_angle(name)
518	                                       ~~~~~~~~~~~~~~~~~~~^^^^^^
519	  File "/opt/pyp/src/pyp/inout/image/core.py", line 1486, in get_tilt_axis_angle
520	    return median(axes)
521	  File "/opt/pixi/env/envs/nextpyp/lib/python3.13/statistics.py", line 637, in median
522	    raise StatisticsError("no median for empty data")
523	statistics.StatisticsError: no median for empty data

It seems there is something wrong on the import function. Let me know if there is any way to fix this! Thank You!!

Best
Jason

[abartesaghi]

Is each of your tilt series stored as a single .mrc file? If so, you also need to provide .order and .rawtlt files so that nextPYP can determine the tilt angles and acquisition order for each tilt.

In that case, your data import directory should look something like this:

tilt_series_01.mrc
tilt_series_01.rawtlt
tilt_series_01.order
...
tilt_series_10.mrc
tilt_series_10.rawtlt
tilt_series_10.order

In most cases, the contents of the .order and .rawtlt files are identical. That means you can often create one copy of each file and make the rest symlinks.

If you are importing tilt-series alignments from IMOD, you will also need a second location containing files like this:

tilt_series_01.xf
tilt_series_01.tlt
...
tilt_series_10.xf
tilt_series_10.tlt

All of these files can be placed in the same folder if you prefer, but you still need to specify the raw data location and the alignment import location separately.

You also need to make sure that import (from path) is selected as the Alignment method in the Tilt-series alignment tab. Your logs suggest that nextPYP is trying to calculate the alignments instead of importing them.

If a run failed and not all files were linked from their original locations, for example, if you ran the import data block before adding the .order files, you may need to use Delete files and data before rerunning the block so that all files are linked properly.