Zero-NatVer

https://arxiv.org/abs/2410.03341

Setup

Clone Zero-NatVer:

git clone https://github.com/marekstrong/Zero-NatVer
cd Zero-NatVer

Then, setup a new conda environment:

conda env create -f environment.yml
conda activate zeronatver

LLama3

The current Zero-NatVer codebase is based on Llama3 (meta-llama/Meta-Llama-3-8B-Instruct). To download and test the model, run the following script:

python ./llm/test_llm.py

This script runs several unit tests to test that the core functionality works and that all query types are supported.

Quick Start

You can run Zero-NatVer on a preprocessed version of SciFact using the following command:

python ./zeronatver.py \
  -j "test/data/scifact_withevidence_norm.jsonl" \
  -o "test/out.jsonl" \
  --align-constrains-type "post" \
  --claim-location "claim_preprocessed" \
  --evidence-location "evidence_preprocessed"

Citation

If you find this work useful, please cite us:

@inproceedings{strong-etal-2024-zero,
    title = "Zero-Shot Fact Verification via Natural Logic and Large Language Models",
    author = "Strong, Marek  and
      Aly, Rami  and
      Vlachos, Andreas",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.findings-emnlp.991/",
    doi = "10.18653/v1/2024.findings-emnlp.991",
    pages = "17021--17035"
}