Hello, thank you for sharing your excellent work. I am trying to reproduce your results using the CHD official dataset and the preprocessing/training scripts provided in this repository, but I encountered some issues regarding the training performance and final evaluation results.
I downloaded the CHD official dataset from your provided link. Then, I converted the hexadecimal payload values into decimal integers and used the normal_16_id.csv file you provided. After that, I ran the following preprocessing scripts in the data_preprocess folder: GCL nodes normal 50_40.py, GCL nodes attack 50_40.py, GCL edges normal 50_40.py, and GCL edges attack 50_40.py. These scripts successfully generated node and edge vectors under the Show_CHD_Split/Dataset50_40 directory. I then merged the node vectors by TAG within both the train and test directories to obtain vectors0.csv, vectors1.csv, vectors2.csv, vectors3.csv, and vectors4.csv respectively.
Next, I ran the training script “graph_train_CHD-upgrade.py”. However, the training loss started to increase after the 3rd epoch, and the final model performance was poor. After the training was completed, I executed “graph_execute_CHD-upgrade.py” to load the trained model and compute the embedding vectors, which generated IVN_32_embeds_16_test0.csv and IVN_32_embeds_16_test.csv. Then, I used these embedding files for evaluation by running “example_clear_CHD_upgrade.py”, but the results were still not satisfactory.
I also tried using the .pkl files you uploaded on GitHub to compute the embeddings and rerun the evaluation, but the results did not improve either. Therefore, I would like to ask whether there are any additional preprocessing steps or specific parameter settings required before training. I am wondering if I might have missed something that affects the quality of the embeddings or the model convergence.
If possible, could you please explain how the provided .pkl files were generated or how the dataset was preprocessed for training? This information would be very helpful for reproducing your reported results.
Thank you very much for your time and for sharing your code.
Hello, thank you for sharing your excellent work. I am trying to reproduce your results using the CHD official dataset and the preprocessing/training scripts provided in this repository, but I encountered some issues regarding the training performance and final evaluation results.
I downloaded the CHD official dataset from your provided link. Then, I converted the hexadecimal payload values into decimal integers and used the normal_16_id.csv file you provided. After that, I ran the following preprocessing scripts in the data_preprocess folder: GCL nodes normal 50_40.py, GCL nodes attack 50_40.py, GCL edges normal 50_40.py, and GCL edges attack 50_40.py. These scripts successfully generated node and edge vectors under the Show_CHD_Split/Dataset50_40 directory. I then merged the node vectors by TAG within both the train and test directories to obtain vectors0.csv, vectors1.csv, vectors2.csv, vectors3.csv, and vectors4.csv respectively.
Next, I ran the training script “graph_train_CHD-upgrade.py”. However, the training loss started to increase after the 3rd epoch, and the final model performance was poor. After the training was completed, I executed “graph_execute_CHD-upgrade.py” to load the trained model and compute the embedding vectors, which generated IVN_32_embeds_16_test0.csv and IVN_32_embeds_16_test.csv. Then, I used these embedding files for evaluation by running “example_clear_CHD_upgrade.py”, but the results were still not satisfactory.
I also tried using the .pkl files you uploaded on GitHub to compute the embeddings and rerun the evaluation, but the results did not improve either. Therefore, I would like to ask whether there are any additional preprocessing steps or specific parameter settings required before training. I am wondering if I might have missed something that affects the quality of the embeddings or the model convergence.
If possible, could you please explain how the provided .pkl files were generated or how the dataset was preprocessed for training? This information would be very helpful for reproducing your reported results.
Thank you very much for your time and for sharing your code.