synth/games/tasks/web_of_lies

Web of Lies

This is a logical reasoning game where players need to infer whether certain people are telling the truth or lying based on a series of statements.

Game Rules

1. The game features a specified number of characters, each located at different places
2. Each character either always tells the truth or always lies
3. Through given statements, players need to reason out whether target characters are truth-tellers or liars
4. Statements include four types: character truth/lie narrator statements, character-to-character statements, irrelevant statements, and location statements

Implementation

Core Classes

• Person: Represents a character in the game, with id, name, location, and is_truth_teller attributes
• WebOfLies: Main game class, inherits from Game base class, implements generate and extract_answer methods
• WebOfLiesVerifier: Verifier class used to validate whether answers are correct

Generation Logic

1. Randomly generate a specified number of characters, each with unique names and locations
2. Determine the number of target characters to be reasoned based on difficulty
3. Randomly assign truth/lie status to some characters (directly given through truth/lie narrator statements)
4. Assign statement objects to each character, ensuring every character is involved in at least one statement
5. Find unique solutions for target characters, ensuring the game has a unique correct answer
6. Generate various types of statements, including location statements, truth/lie narrator statements, character-to-character statements, and irrelevant statements
7. Generate game prompts and answers

Verification Logic

1. Extract answers from model responses (supports Chinese, English, and multiple formats)
2. Compare extracted answers with standard answers
3. Verify whether answer content and length match

Parameter Description

• num_person: Number of characters in the game, default is 10
• difficulty: Game difficulty, range 1-5, affects reasoning difficulty and statement complexity
• num_of_data: Number of game data to generate
• max_attempts: Maximum number of attempts when generating games
• statement_type: Statement type, 0 for basic type (only includes simple statements), 1 for extended type (includes simple statements, at least one statement, and same-type statements), default is 0

Usage

Game data can be generated using the following commands:

# Using default parameters
./run.sh

# Specifying parameters
./run.sh --num_of_data 50 --num_person 15 --difficulty 4

# Using extended statement type
./run.sh --statement_type 1

Generated data will be saved in data/num_person_{num_person}/difficulty_{difficulty}/statement_type_{statement_type}/num_of_data_{num_of_data}/data.jsonl file.

Example Output

English example:

In this question, assume each person either always tells the truth or always lies. Each person corresponds to a unique location.
Clues: Emma is at the museum. The person at the museum says the person at the park lies. Olivia is at the park. The person at the cafe says the person at the museum tells the truth. The person at the park says the person at the restaurant tells the truth. The person at the restaurant lies. William is at the restaurant. The person at the park thinks their friend is lying. Mason is at the cafe.
Questions: Does the person at the park tell the truth? Does the person at the cafe tell the truth?
Think step by step, end your response in the last line with the following format: The answer is $YOUR_ANSWER. $YOUR_ANSWER should be a list of words in bold, yes or no (for example, "The answer is **yes, no, yes**."), and If you don't know, guess.

Chinese example:

在这个问题中，每个人要么总是说真话，要么总是说谎话。每个人都位于不同的地点。
线索：张伟在图书馆。在图书馆的人说在公园的人说的是假话。赵芳在公园。在超市的人说在图书馆的人说的是真话。在公园的人说在餐厅的人说的是真话。在餐厅的人说的是假话。李明在餐厅。在公园的人认为自己的朋友在撒谎。王华在超市。
问题：在公园的人说的是真话吗？在超市的人说的是真话吗？
请逐步思考，在最后一行以下列格式给出你的答案：答案是 $YOUR_ANSWER。$YOUR_ANSWER 应该是由几个加粗的词组成的列表，是或否（例如，"答案是 **是，否，是**。"），如果你不确定，请猜测。