fail to run the water_single job in examples

[Wanlin-Cai]

(deepks) [qwe@login01 init]$ deepks iterate machines.yaml params.yaml systems.yaml

starting step: (0,)

starting step: (0, 0)

starting step: (0, 0, 0)

Traceback (most recent call last):
File "/data/home/caiwl/anaconda3/envs/deepks/bin/deepks", line 8, in
sys.exit(main_cli())
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/main.py", line 37, in main_cli
sub_cli(args.args)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/main.py", line 259, in iter_cli
main(**argdict)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/iterate/iterate.py", line 316, in main
iterate.run()
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/workflow.py", line 49, in run
task.run(curr_tag)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/workflow.py", line 49, in run
task.run(curr_tag)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/workflow.py", line 49, in run
task.run(curr_tag)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/task.py", line 88, in run
self.execute()
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/task.py", line 219, in execute
outlog=self.outlog, errlog=self.errlog)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/dispatcher.py", line 113, in run_jobs
errlog
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/dispatcher.py", line 183, in submit_jobs
para_deg=para_deg, para_res=para_res)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/batch.py", line 143, in submit
para_deg=para_deg, para_res=para_res)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/batch.py", line 62, in do_submit
self.exec_sub_script(script_str=script_str)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/slurm.py", line 33, in exec_sub_script
stdin, stdout, stderr = self.context.block_checkcall('cd %s && %s %s' % (self.context.remote_root, 'sbatch', self.sub_script_name))
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/lazy_local_context.py", line 76, in block_checkcall
raise RuntimeError("Get error code %d in locally calling %s with job: %s " % (code, cmd, self.job_uuid))
RuntimeError: Get error code 127 in locally calling cd /data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/test_single/init/iter.init/00.scf && sbatch 8479abab-7476-48cf-ba39-3e0f688420ea.sub with job: 8479abab-7476-48cf-ba39-3e0f688420ea

[y1xiaoc]

Code 127 likely means you do not have slurm available in your cluster (check the other example where there are two run script for slurm and local). To avoid confusion, I have changed the example to run locally. You may want to pull the newest version and try again.

[Wanlin-Cai]

Thank you for you help~~~

However, it shows
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/model/train.py", line 250, in
cli()
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/main.py", line 71, in train_cli
main(**argdict)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/model/train.py", line 245, in main
train(model, g_reader, test_reader=test_reader, **train_args)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/model/train.py", line 164, in train
for batch in g_reader.sample_all_batch()])
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/model/train.py", line 164, in
for batch in g_reader.sample_all_batch()])
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/model/train.py", line 108, in call
tot_loss = tot_loss + self.e_factor * self.e_lossfn(e_pred, e_label)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/model/train.py", line 52, in loss_fn
sqdf = diff.square()
AttributeError: 'Tensor' object has no attribute 'square'

Traceback (most recent call last):
File "/data/home/caiwl/anaconda3/envs/deepks/bin/deepks", line 8, in
sys.exit(main_cli())
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/main.py", line 37, in main_cli
sub_cli(args.args)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/main.py", line 259, in iter_cli
main(**argdict)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/iterate/iterate.py", line 316, in main
iterate.run()
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/workflow.py", line 49, in run
task.run(curr_tag)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/workflow.py", line 49, in run
task.run(curr_tag)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/workflow.py", line 49, in run
task.run(curr_tag)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/task.py", line 88, in run
self.execute()
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/task.py", line 171, in execute
outlog=self.outlog, errlog=self.errlog)
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/dispatcher.py", line 115, in run_jobs
while not self.all_finished(job_handler, mark_failure) :
File "/data/home/caiwl/anaconda3/envs/deepks/lib/python3.7/site-packages/deepks/task/job/dispatcher.py", line 254, in all_finished
raise RuntimeError('Job %s failed for more than 3 times' % job_uuid)
RuntimeError: Job a29b872f-4ee9-4578-a667-5413908ec9bb failed for more than 3 times

Is this error associated with my envs cudatoolkit=9.2?

[y1xiaoc]

You are using a old pytorch version that it does not support the square function. I have update the code to use pow(2) instead. You can update deepks and try again. (it can also be solved by update pytorch)

[Wanlin-Cai]

Thanks a lot.
I've finished the test successful with pytorch1.4.