Dataloader error

[deeplearningdiaries]

Hi,

I am trying to reproduce the results of your paper and I am getting an error while loading data. The error is the following:

113 ./data/rplan/45776.json
[Epoch 0/200] [Batch 111/379] [D loss: 9.999077] [G loss: 81.515854] [L1 loss: 83.728096]
Traceback (most recent call last):
File "train.py", line 114, in
for i, batch in enumerate(fp_loader):
File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 530, in next
data = self._next_data()
File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1224, in _next_data
114 ./data/rplan/16164.json
return self._process_data(data)
File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1250, in _process_data
data.reraise()
File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/_utils.py", line 457, in reraise
raise exception
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/user/houseganpp/dataset/floorplan_dataset_maps_functional_high_res.py", line 135, in getitem
graph_nodes, graph_edges, rooms_mks = self.build_graph(rms_type, fp_eds, eds_to_rms)
File "/home/user/houseganpp/dataset/floorplan_dataset_maps_functional_high_res.py", line 294, in build_graph
poly = self.make_sequence(np.array([fp_eds[l][:4] for l in eds_poly]))[0]
File "/home/user/houseganpp/dataset/floorplan_dataset_maps_functional_high_res.py", line 189, in make_sequence
v_curr = tuple(edges[0][:2])
IndexError: index 0 is out of bounds for axis 0 with size 0

I have built the database with the repository https://github.com/sepidsh/Housegan-data-reader

I attach the file with the error, but I think that the error is not in the file but in the dataloader. When I feed just this file to the model, it works.
16164.zip

Have you had this error before? Do you know how to solve it?

[sucream1004]

I had the same error. In my case, when dataloader reads a file, the length of edges was weird. I tested with 16164.json that you uploaded, it looked fine. So I don't fully understand why.

Temporarily, I resolved it simply removed the json files that has an error when it's loaded.

[sakmalh]

Let me know if you found a solution for it @sucream1004 @deeplearningdiaries

[1005wl]

我有同样的错误。就我而言，当数据加载器读取文件时，边缘的长度很奇怪。我用您上传的 16164.json 进行了测试，看起来不错。所以我不完全明白为什么。

暂时，我解决了它只是删除了加载时出错的 json 文件。

请问已经解决了吗？

[1005wl]

让我知道您是否找到了解决方案。

请问您解决这个问题了吗？

[1005wl]

Hi,

I am trying to reproduce the results of your paper and I am getting an error while loading data. The error is the following:

113 ./data/rplan/45776.json [Epoch 0/200] [Batch 111/379] [D loss: 9.999077] [G loss: 81.515854] [L1 loss: 83.728096] Traceback (most recent call last): File "train.py", line 114, in for i, batch in enumerate(fp_loader): File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 530, in next data = self._next_data() File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1224, in _next_data 114 ./data/rplan/16164.json return self._process_data(data) File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1250, in _process_data data.reraise() File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/_utils.py", line 457, in reraise raise exception IndexError: Caught IndexError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/user/miniconda3/envs/houseganpp/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/user/houseganpp/dataset/floorplan_dataset_maps_functional_high_res.py", line 135, in getitem graph_nodes, graph_edges, rooms_mks = self.build_graph(rms_type, fp_eds, eds_to_rms) File "/home/user/houseganpp/dataset/floorplan_dataset_maps_functional_high_res.py", line 294, in build_graph poly = self.make_sequence(np.array([fp_eds[l][:4] for l in eds_poly]))[0] File "/home/user/houseganpp/dataset/floorplan_dataset_maps_functional_high_res.py", line 189, in make_sequence v_curr = tuple(edges[0][:2]) IndexError: index 0 is out of bounds for axis 0 with size 0

I have built the database with the repository https://github.com/sepidsh/Housegan-data-reader

I attach the file with the error, but I think that the error is not in the file but in the dataloader. When I feed just this file to the model, it works. 16164.zip

Have you had this error before? Do you know how to solve it?

"Have you already solved this issue? Looking forward to your reply."

[MalikHaroonKhokhar]

were you able to resolve it?

[MalikHaroonKhokhar]

Ended up resolving it by skipping the rooms with abnormal edges