registration of Immunofluorescence of continuous sections

[gxygogogo]

I want to register the Immunofluorescence of continuous sections. Can VALIS perform such alignment between consecutive slices？

[cdgatenbee]

Hi @gxygogogo,
Yes, VALIS can align consecutive slices. If the order in which the slides are aligned doesn't matter, then just follow the example here. However, if the goal is 3D reconstruction, and the order does matter, the safest strategy is to pass in a list of image file names, making sure that they are in the correct consecutive order. Once you have that, you will also need to set imgs_ordered=True when you create the Valis object. So, for example, after you have created the ordered list of file names (sorted_img_f_list below), the code would be

registrar = registration.Valis(slide_src_dir, results_dst_dir, img_list= sorted_img_f_list, imgs_ordered=True)
rigid_registrar, non_rigid_registrar, error_df = registrar.register()

Hope that helps, and if you have any more questions, please let me know.

Best,
-Chandler

[gxygogogo]

Thanks! My goal is 3D reconstruction from consecutive slices IF.

[gxygogogo]

In [2]: error_df
Out[2]:
filename from to ... rigid_time_minutes mean_non_rigid_D non_rigid_time_minutes
0 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_42.vsi 图像_HJZ_42 图像_HJZ_47 ... 4.213124 42.956213 9.188644
1 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_47.vsi 图像_HJZ_47 图像_HJZ_49 ... 4.213124 42.956213 9.188644
2 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_49.vsi 图像_HJZ_49 图像_HJZ_51 ... 4.213124 42.956213 9.188644
3 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_51.vsi 图像_HJZ_51 图像_HJZ_53 ... 4.213124 42.956213 9.188644
4 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_53.vsi 图像_HJZ_53 图像_HJZ_55 ... 4.213124 42.956213 9.188644
5 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_55.vsi 图像_HJZ_55 图像_HJZ_57 ... 4.213124 42.956213 9.188644
6 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_57.vsi 图像_HJZ_57 图像_HJZ_60 ... 4.213124 42.956213 9.188644
7 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_60.vsi 图像_HJZ_60 图像_HJZ_62 ... 4.213124 42.956213 9.188644
8 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_62.vsi 图像_HJZ_62 图像_HJZ_64 ... 4.213124 42.956213 9.188644
9 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_64.vsi 图像_HJZ_64 图像_HJZ_67 ... 4.213124 42.956213 9.188644
10 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_67.vsi 图像_HJZ_67 None ... 4.213124 42.956213 9.188644
11 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_69.vsi 图像_HJZ_69 图像_HJZ_67 ... 4.213124 42.956213 9.188644
12 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_71.vsi 图像_HJZ_71 图像_HJZ_69 ... 4.213124 42.956213 9.188644
13 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_73.vsi 图像_HJZ_73 图像_HJZ_71 ... 4.213124 42.956213 9.188644
14 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_75.vsi 图像_HJZ_75 图像_HJZ_73 ... 4.213124 42.956213 9.188644
15 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_77.vsi 图像_HJZ_77 图像_HJZ_75 ... 4.213124 42.956213 9.188644
16 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_79.vsi 图像_HJZ_79 图像_HJZ_77 ... 4.213124 42.956213 9.188644
17 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_81.vsi 图像_HJZ_81 图像_HJZ_79 ... 4.213124 42.956213 9.188644
18 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_83.vsi 图像_HJZ_83 图像_HJZ_81 ... 4.213124 42.956213 9.188644
19 /public3/Xinyu/3D_tissue/IF/data/图像_HJZ_85.vsi 图像_HJZ_85 图像_HJZ_83 ... 4.213124 42.956213 9.188644

[20 rows x 20 columns]
Why are the file names of the from column and the to column in this matrix inconsistent

[cdgatenbee]

Hi @gxygogogo,

Maybe try using valtils.sort_nicely on your image file list before passing it into the Valis object. It should ensure that the images are sorted numerically. Something like:

from valis import registration, valtils
sorted_img_f_list = os.listdir(slide_src_dir)
valtils.sort_nicely(sorted_img_f_list) # sorting is done in place

registrar = registration.Valis(slide_src_dir, results_dst_dir, img_list= sorted_img_f_list, imgs_ordered=True)
rigid_registrar, non_rigid_registrar, error_df = registrar.register()

Best,
-Chandler

[gxygogogo]

==== Non-rigid registration

Creating non-rigid mask
Preparing images for non-rigid registration: 0%| | 0/20 [00:00<?, ?image/s]Converting slide to pyvips image
Preparing images for non-rigid registration: 5%|██▊ | 1/20 [00:07<02:17, 7.24s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 10%|█████▌ | 2/20 [00:12<01:52, 6.24s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 15%|████████▍ | 3/20 [00:16<01:29, 5.24s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 20%|███████████▏ | 4/20 [00:19<01:07, 4.25s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 25%|██████████████ | 5/20 [00:22<00:54, 3.62s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 30%|████████████████▊ | 6/20 [00:25<00:50, 3.64s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 35%|███████████████████▌ | 7/20 [00:30<00:52, 4.02s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 40%|██████████████████████▍ | 8/20 [00:32<00:41, 3.50s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 45%|█████████████████████████▏ | 9/20 [00:36<00:38, 3.51s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 50%|███████████████████████████▌ | 10/20 [00:40<00:35, 3.58s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 55%|██████████████████████████████▎ | 11/20 [00:43<00:31, 3.48s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 60%|█████████████████████████████████ | 12/20 [00:47<00:28, 3.57s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 65%|███████████████████████████████████▊ | 13/20 [00:49<00:22, 3.28s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 70%|██████████████████████████████████████▌ | 14/20 [00:52<00:19, 3.20s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 75%|█████████████████████████████████████████▎ | 15/20 [00:57<00:18, 3.62s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 80%|████████████████████████████████████████████ | 16/20 [00:59<00:12, 3.23s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 85%|██████████████████████████████████████████████▊ | 17/20 [01:01<00:08, 2.86s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 90%|█████████████████████████████████████████████████▌ | 18/20 [01:03<00:05, 2.59s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 95%|████████████████████████████████████████████████████▎ | 19/20 [01:06<00:02, 2.57s/image]Converting slide to pyvips image
Preparing images for non-rigid registration: 100%|███████████████████████████████████████████████████████| 20/20 [01:09<00:00, 3.47s/image]
Finding non-rigid transforms: 100%|██████████████████████████████████████████████████████████████████████| 19/19 [03:45<00:00, 11.84s/image]

======== Non-rigid registration complete in 3.769 minutes

==== Measuring error

Measuring error: 100%|███████████████████████████████████████████████████████████████████████████████████| 20/20 [01:34<00:00, 4.71s/image]
Saving images: 0%| | 0/20 [00:00<?, ?image/s]Converting slide to pyvips image
/usr/local/src/valis/valtils.py:29: UserWarning: xml in original file is invalid or missing. Will create one
warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/usr/local/src/valis/slide_io.py", line 3398, in update_xml_for_new_img
elementTree.fromstring(current_ome_xml_str)
File "/usr/lib/python3.12/xml/etree/ElementTree.py", line 1335, in XML
parser.feed(text)
xml.etree.ElementTree.ParseError: reference to invalid character number: line 1, column 391

saving /public3/Xinyu/3D_tissue/IF/registration_20/registered_slides/图像_HJZ_42.ome.tiff (57106 x 50162 and 4 channels)

Killed-----------------------------------------------------------------------------------------------] 1.0% in 0.544 minutes

Why does the process automatically kill every time I try to save it as ome.tiff？

[gxygogogo]

...: '图像_HJZ_42',
...: '图像_HJZ_43',
...: '图像_HJZ_46',
...: '图像_HJZ_47',
...: '图像_HJZ_48',
...: '图像_HJZ_49',
...: '图像_HJZ_51',
...: '图像_HJZ_52',
...: '图像_HJZ_53',
...: '图像_HJZ_54',
...: '图像_HJZ_55',
...: '图像_HJZ_56',
...: '图像_HJZ_57',
...: '图像_HJZ_59',
...: '图像_HJZ_60',
...: '图像_HJZ_61',
...: '图像_HJZ_62',
...: '图像_HJZ_63',
...: '图像_HJZ_64',
...: '图像_HJZ_66',
...: '图像_HJZ_67',
...: '图像_HJZ_68',
...: '图像_HJZ_69',
...: '图像_HJZ_70',
...: '图像_HJZ_71',
...: '图像_HJZ_72',
...: '图像_HJZ_73',
...: '图像_HJZ_74',
...: '图像_HJZ_75',
...: '图像_HJZ_76',
...: '图像_HJZ_77',
...: '图像_HJZ_78',
...: '图像_HJZ_79',
...: '图像_HJZ_80',
...: '图像_HJZ_81',
...: '图像_HJZ_82',
...: '图像_HJZ_83',
...: '图像_HJZ_84',
...: '图像_HJZ_85',
...: '图像_HJZ_86',
...: '图像_HJZ_88 - 20x_DAPI, BNA, GNA, Cy5_01',
...: '图像_HJZ_90 - 20x_DAPI, BNA, GNA, Cy5_01',
...: '图像_HJZ_92 - 20x_DAPI, BNA, GNA, Cy5_01',
...: '图像_HJZ_94 - 20x_DAPI, BNA, GNA, Cy5_01',
...: '图像_HJZ_96 - 20x_DAPI, BNA, GNA, Cy5_01',
...: '图像_HJZ_98 - 20x_DAPI, BNA, GNA, Cy5_01',
...: '图像_HJZ_100 - 20x_DAPI, BNA, GNA, Cy5_01',
...: '图像_HJZ_102 - 20x_DAPI, BNA, GNA, Cy5_01']
...:
...:
...: # sorted_img_f_list = os.listdir(slide_src_dir)
...: # valtils.sort_nicely(sorted_img_f_list) # sorting is done in place
...:
...:
...: ## 有顺序的
...: registrar = registration.Valis(slide_src_dir, results_dst_dir, img_list= sorted_img_f_list, imgs_ordered=True)
...: rigid_registrar, non_rigid_registrar, error_df = registrar.register()
...:

==== Converting images

/usr/local/src/valis/valtils.py:29: UserWarning: Unable to find reader to open 图像_HJZ_1
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Attempting to read 图像_HJZ_1 created the following error:
'NoneType' object is not callable
warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/usr/local/src/valis/registration.py", line 2507, in create_img_reader_dict
slide_reader = slide_reader_cls(src_f=slide_f, **slide_reader_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

/usr/local/src/valis/valtils.py:29: UserWarning: cannot access local variable 'slide_reader' where it is not associated with a value
warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/usr/local/src/valis/registration.py", line 4722, in register
self.convert_imgs(series=self.series, reader_cls=reader_cls, reader_dict=reader_dict)
File "/usr/local/src/valis/registration.py", line 2535, in convert_imgs
named_reader_dict = self.create_img_reader_dict(reader_dict=reader_dict,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/src/valis/registration.py", line 2513, in create_img_reader_dict
named_reader_dict[slide_name] = slide_reader
^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'slide_reader' where it is not associated with a value

Why can't the file be read？

[gxygogogo]

In [4]: registrar = registration.Valis(slide_src_dir, results_dst_dir)
...: rigid_registrar, non_rigid_registrar, error_df = registrar.register()
...:
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_29
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_62
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_83
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_78
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_33
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_16
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_43
warnings.warn(warning_msg, warning_type)
/usr/local/src/valis/valtils.py:29: UserWarning: Can't find slide file associated with 图像_HJZ_48
If I don't specify the order of the images, it can run, but a warning will appear saying there are no associated slides. My images is a vsi format image scanned by Olympus.

[cdgatenbee]

Hi @gxygogogo,
I'm not sure what the issue is with the image not being saved. As for the latter 2, it looks like there may an issue reading your slides. I can test out valis' slide readers using some example .vsi (here), but would you be able to share 2-3 images that I could debug with, to solve all of the issues? You could share them with my email, Chandler[dot]Gatenbee[at]Moffitt[dot]org.

Best,
-Chandler

[gxygogogo]

Sure. What's your email address？I will share 5 images with you.
I can save .ome.tiff of each slide. But what puzzles me is that when I open this file with the QuPath software, it shows a pseudo-color image instead of the original color of my immunofluorescence image, and I also can't open this file with the Olympus software.

[cdgatenbee]

Sounds like there may be an issue with the metadata, so I'll also check to see what the merged ome.tiff looks like in QuPath. Anyhow, please send the .vsi to Chandler[dot]Gatenbee[at]moffitt[dot]org (trying to avoid spam, so please just replace "dot" with "." and "at" with "@").

Best,
-Chandler

[gxygogogo]

I'll send it to you via Google drive

[gxygogogo]

Hi, I have already sent to you by email.

[gxygogogo]

Hi, I have completed the registration of all the pictures. But when I focused on an obvious blood vessel, I found that these details were not aligned. What should I do？

[cdgatenbee]

Hi @gxygogogo,
Apologies for taking so long to respond, it's been a busy couple of days. Anyhow, one thing you could do is to try performing "micro-registration", which refines the alignments using higher resolution versions of the images. I need to fix the documentation on how to do this (reported in issue 226), but Github issue 210 describes the process.

Also, as you were able to perform the initial registration, does that mean you were able to resolve your initial issue related to the .vsi images? If so, would you mind sharing your solution? It could be of use to others who run into a similar issue.

Best,
-Chandler

[gxygogogo]

Hi @gxygogogo, Apologies for taking so long to respond, it's been a busy couple of days. Anyhow, one thing you could do is to try performing "micro-registration", which refines the alignments using higher resolution versions of the images. I need to fix the documentation on how to do this (reported in issue 226), but Github issue 210 describes the process.

Also, as you were able to perform the initial registration, does that mean you were able to resolve your initial issue related to the .vsi images? If so, would you mind sharing your solution? It could be of use to others who run into a similar issue.

Best, -Chandler

1.	I have completed the alignment. Densely structured tissue regions generally show good alignment performance. However, for highly deformable regions, such as the intestinal submucosa, it is difficult to achieve satisfactory alignment due to strong stretching and distortion.
2.	I referred to the micro-alignment strategy as well as the solution discussed in GitHub issue #210, and I observed a noticeable improvement in the alignment results.
3.	For very large tissue sections, my current approach is to first perform a global rigid alignment, followed by micro-alignment on selected ROIs or further local refinement following the strategy in issue #210. I am not fully sure whether this is the recommended or optimal workflow, and I would appreciate any feedback or suggestions.