Failed to run lgcn, din and sasrec

[NKUShaw]

Just replace the configs as you shown in the github

[rank1]: raise RuntimeError(
[rank1]: RuntimeError: Error(s) in loading state_dict for SASRec:
[rank1]: Missing key(s) in state_dict: "item_emb.weight", "pos_emb.weight", "attention_layernorms.0.weight", "attention_layernorms.0.bias", "attention_layernorms.1.weight", "attention_layernorms.1.bias", "attention_layers.0.in_proj_weight", "attention_layers.0.in_proj_bias", "attention_layers.0.out_proj.weight", "attention_layers.0.out_proj.bias", "attention_layers.1.in_proj_weight", "attention_layers.1.in_proj_bias", "attention_layers.1.out_proj.weight", "attention_layers.1.out_proj.bias", "forward_layernorms.0.weight", "forward_layernorms.0.bias", "forward_layernorms.1.weight", "forward_layernorms.1.bias", "forward_layers.0.conv1.weight", "forward_layers.0.conv1.bias", "forward_layers.0.conv2.weight", "forward_layers.0.conv2.bias", "forward_layers.1.conv1.weight", "forward_layers.1.conv1.bias", "forward_layers.1.conv2.weight", "forward_layers.1.conv2.bias", "last_layernorm.weight", "last_layernorm.bias".
[rank1]: Unexpected key(s) in state_dict: "user_embedding.weight", "item_embedding.weight".

See my configs:

model:
arch: mini_gpt4rec_v2
model_type: pretrain_vicuna
freeze_rec: True
freeze_proj: True # stage 1: proj false, lora: false
freeze_lora: False # sateg2: proj true, lora false
max_txt_len: 1024
proj_token_num: 1
proj_mid_times: 10
proj_drop: 0
end_sym: "###"
prompt_path: "./prompts/tallrec_movie.txt"

prompt_template: '### Input: {} \n### Response:'

prompt_template: '{}'

Vicuna

llama_model: "./vicuna/weight/"
user_num: -100
item_num: -100
ans_type: 'v2'
rec_model: "sasrec" #[MF, Lightgcn, sasrec]
lora_config:
use_lora: True
r: 8
alpha: 16
target_modules: ["q_proj", "v_proj"]
dropout: 0.05
rec_config: #sasrec_config
dataset: ml1m
user_num: -100
item_num: -100
embedding_size: 64
embed_size: 64
hidden_units: 64
num_blocks: 2
num_heads: 4
dropout_rate: 0.2
l2_emb: 1e-4 # not used
maxlen: 25

pretrained_path: "./train_configs/0912_ml1m_oodv2_best_model_d256lr-0.001wd0.0001.pth" # pretrained rec model path

ckpt: /home/sist/zyang/LLM/minigpt4recLog/20230920161/checkpoint_best.pth # tune proj based on tallrec lr 1e-3

datasets:
movie_ood_sasrec:
path: "./collm-datasets/ml-1m/" #"/LLM/MiniGPT-4/dataset/ml-100k/"
# path: "/LLM/MiniGPT-4/dataset/ml-1m/"`
# path: /home/sist/zyang/LLM/datasets/ml-1m
data_type: default
build_info:
# storage: /path/to/cc_sbu_dataset/{00000..01255}.tar
storage: "./collm-datasets/ml-1m/" #~/LLM/MiniGPT-4/dataset/ml-100k/
# storage: /home/sist/zyang/LLM/datasets/ml-1m/

run:
task: rec_pretrain

optimizer

lr_sched: "linear_warmup_cosine_lr"
init_lr: 1e-3
min_lr: 8e-5
warmup_lr: 1e-5
mode: 'v2' # stage1: v1,

weight_decay: 1e-3 #0.05
max_epoch: 200
iters_per_epoch: 50 #100 #50 #200
batch_size_train: 16
batch_size_eval: 64
num_workers: 4
warmup_steps: 200 #200

seed: 42

output_dir: "output"

output_dir: ./output/minigpt4rec-log

output_dir: "home/zyang/LLM/minigpt4recLog/minigpt4rec_finetune"

amp: True
resume_ckpt_path: null

evaluate: True #False
train_splits: ["train"]
valid_splits: ["valid"]
test_splits: ["test","valid"]

device: "cuda"
world_size: 1
dist_url: "env://"
distributed: True

[zyang1580]

Sorry, I do not have the time to check the bug. You may check it first by yourself.

[NKUShaw]

Sorry, I do not have the time to check the bug. You may check it first by yourself.

never mind, I used the wrong pretrained weights. We can trained the recommended models by the baseline_xxx.py first