README.md

ECon: On the Detection and Resolution of Evidence Conflicts

This is the official repo for the paper ECon: On the Detection and Resolution of Evidence Conflicts [Arxiv]. This study introduces a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios.

Answer conflicts, where two pieces of evidence are conflicting in terms of the answers they suggest.

Factoid conflicts, where some factoids in the two pieces of evidence are conflicting.

Reproduce the results

To enable prompting with LLMs, you need to implement the method get_llm_response() in utils.py first.

Conflict detection

For answer conflicts

cd src/conflict_detection/answer_conflict && python run.py

For factoid conflicts (intensity of conflict)

cd src/conflict_detection/factoid_conflict/intensity_of_conflict && python run.py

For factoid conflicts (intensity of corroboration)

cd src/conflict_detection/factoid_conflict/intensity_of_corroboration && python run.py

Conflict resolution

The code we use for generating model beliefs, predictions, etc. are at conflict_resolution/main.ipynb.

Bugs or questions?

If you have any questions related to the code or the paper, please feel free to send an email to [email protected].

Citation

@inproceedings{jiayang2024econ,
  title={ECON: On the Detection and Resolution of Evidence Conflicts},
  author={Jiayang, Cheng and Chan, Chunkit and Zhuang, Qianqian and Qiu, Lin and Zhang, Tianhang and Liu, Tengxiao and Song, Yangqiu and Zhang, Yue and Liu, Pengfei and Zhang, Zheng},
  booktitle={Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  pages={7816--7844},
  year={2024}
}