I don't understand your question.

[wangwei-bee]

I don't understand your question.

Originally posted by @MingjunZhong in #8 (comment)

[wangwei-bee]

#8

D:\ProgramData\Anaconda3\envs\tf1\python.exe D:/env/seq2point-nilm-master/train_main.py
2021-12-07 18:39:29.084196: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-12-07 18:39:33.692646: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-12-07 18:39:33.693571: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2021-12-07 18:39:33.734057: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:73:00.0 name: GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s
2021-12-07 18:39:33.734245: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-12-07 18:39:33.756459: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-12-07 18:39:33.756564: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-12-07 18:39:33.761675: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2021-12-07 18:39:33.763061: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2021-12-07 18:39:33.774396: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2021-12-07 18:39:33.778510: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2021-12-07 18:39:33.779184: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-12-07 18:39:33.779337: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-12-07 18:39:33.779650: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-12-07 18:39:33.786034: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:73:00.0 name: GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s
2021-12-07 18:39:33.786300: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-12-07 18:39:33.786434: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-12-07 18:39:33.786566: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-12-07 18:39:33.786698: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2021-12-07 18:39:33.786824: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2021-12-07 18:39:33.786960: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2021-12-07 18:39:33.787080: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2021-12-07 18:39:33.787196: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-12-07 18:39:33.787338: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-12-07 18:39:34.533384: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-12-07 18:39:34.533500: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2021-12-07 18:39:34.533560: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2021-12-07 18:39:34.533799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3090, pci bus id: 0000:73:00.0, compute capability: 8.6)
2021-12-07 18:39:34.535316: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
Importing training file...
Counting number of rows...
Done.
The dataset contains 300000 rows
path is : D:/env/seq2point-nilm-master/dataset_management/redd/REDD1/dishwasher_training_.csv
aggregate dishwasher
0 -0.594218 -0.699000
1 -0.594218 -0.699000
2 -0.594211 -0.699667
3 -0.594240 -0.699000
4 -0.594295 -0.699000
... ... ...
299995 -0.059237 -0.700000
299996 -0.055958 -0.700000
299997 -0.045611 -0.699500
299998 -0.052087 -0.700000
299999 -0.049189 -0.700000

[300000 rows x 2 columns]
(300000,)
2021-12-07 18:39:36.360595: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
Epoch 1/50
2021-12-07 18:39:36.836693: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-12-07 18:39:37.737411: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-12-07 18:39:37.769089: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-12-07 18:39:39.559999: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 0

2021-12-07 18:39:39.641183: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 0

2021-12-07 18:39:39.839037: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
585/585 [==============================] - ETA: 0s - loss: 0.0193 - mse: 0.0193 - msle: 5.0212e-04 - mae: 0.0561Importing training file...
Counting number of rows...
Done.
The dataset contains 33372 rows
path is : D:/env/seq2point-nilm-master/dataset_management/redd/REDD1/dishwasher_validation_.csv
aggregate dishwasher
0 -0.284920 -0.6990
1 -0.284994 -0.6990
2 -0.282996 -0.6990
3 -0.282336 -0.6990
4 -0.284320 -0.6990
... ... ...
33367 -0.619515 -0.6970
33368 -0.619501 -0.6975
33369 -0.619524 -0.6970
33370 -0.619579 -0.6970
33371 -0.619562 -0.6970

[33372 rows x 2 columns]
(33372,)
585/585 [==============================] - 48s 75ms/step - loss: 0.0193 - mse: 0.0193 - msle: 5.0214e-04 - mae: 0.0561 - val_loss: 0.0048 - val_mse: 0.0048 - val_msle: 1.2839e-05 - val_mae: 0.0179
Epoch 2/50
64/585 [==>...........................] - ETA: 36s - loss: 0.0023 - mse: 0.0023 - msle: 8.2943e-05 - mae: 0.01982021-12-07 18:40:28.745127: W tensorflow/core/framework/op_kernel.cc:1751] Invalid argument: ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected.
Traceback (most recent call last):

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\ops\script_ops.py", line 249, in call
ret = func(*args)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 620, in wrapper
return func(*args, **kwargs)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 938, in generator_py_func
(ret_array.shape, expected_shape))

ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected.

Traceback (most recent call last):
File "D:/env/seq2point-nilm-master/train_main.py", line 42, in
65/585 [==>...........................] - ETA: 36s - loss: 0.0023 - mse: 0.0023 - msle: 8.2101e-05 - mae: 0.0197 trainer.train_model()
File "D:\env\seq2point-nilm-master\seq2point_train.py", line 118, in train_model
2021-12-07 18:40:28.822680: W tensorflow/core/framework/op_kernel.cc:1751] Invalid argument: ValueError: generator yielded an element of shape (0,) where an element training_history = self.default_train(model, callbacks, steps_per_training_epoch)
of sh File "D:\env\seq2point-nilm-master\seq2point_train.py", line 162, in default_train
ape (None, None) was expected.
Traceback (most recent call last):

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\ops\script_ops.py", line 249, in call
ret = func(*args)
validation_steps=self.__validation_steps)

Fi File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1100, in fit
le "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 620, in wrapper
return func(*args, **kwargs)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 938, in generator_py_func
(ret_array.shape, expected_shape))

ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected.

tmp_logs = self.train_function(iterator)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\eager\def_function.py", line 828, in call
result = self._call(*args, **kwds)
File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\eager\def_function.py", line 855, in _call
return self._stateless_fn(*args, **kwds) # pylint: disable=not-callable
File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\eager\function.py", line 2943, in call
filtered_flat_args, captured_inputs=graph_function.captured_inputs) # pylint: disable=protected-access
File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\eager\function.py", line 1919, in _call_flat
ctx, args, cancellation_manager=cancellation_manager))
File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\eager\function.py", line 560, in call
ctx=ctx)
File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\eager\execute.py", line 60, in quick_execute
inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected.
Traceback (most recent call last):

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\ops\script_ops.py", line 249, in call
ret = func(*args)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 620, in wrapper
return func(*args, **kwargs)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 938, in generator_py_func
(ret_array.shape, expected_shape))

ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected.

 [[{{node PyFunc}}]]
 [[IteratorGetNext]]

(1) Invalid argument: ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected.
Traceback (most recent call last):

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\ops\script_ops.py", line 249, in call
ret = func(*args)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 620, in wrapper
return func(*args, **kwargs)

File "D:\ProgramData\Anaconda3\envs\tf1\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 938, in generator_py_func
(ret_array.shape, expected_shape))

ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected.

 [[{{node PyFunc}}]]
 [[IteratorGetNext]]
 [[IteratorGetNext/_10]]

0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_1084]

Function call stack:
train_function -> train_function

2021-12-07 18:40:28.958495: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]

Process finished with exit code 1

[wangwei-bee]

The above error will occur when the number of lines in the training set and verification set I input is different. Could you please answer it? thank you

[MingjunZhong]

``ValueError: generator yielded an element of shape (0,) where an element of shape (None, None) was expected'' seems that there was no data when using generator for a epoch.

[wangwei-bee]

But I don't make mistakes when I have the same number of major and minor keys in the training set and the validation set.
'MSE: ', '0.01344508957117796', ' MAE: ', '0.017234958708286285'This result is the result I tested on the REDD dataset. Is there anything wrong with it? This result is too small

[MingjunZhong]

The results look vary bad. Have you tried on kettle first? The kettle should be the one to start from, as it is easier to train

[wangwei-bee]

The error appears to be that he completed a training round on the training set and then skipped to the validation set, but the validation set does not have as much data as the training set, so the following data is empty. May I ask how you solved it? Why does this treat the validation set as the training set? thank you