LongRewardBench

Ability:

Reward

Path:

benchmarks/Reward/LongRewardBench

Note

← Back to Benchmarks Overview | View other Reward benchmarks in the sidebar

Overview

This benchmark evaluates LongRewardBench capabilities within the Reward dimension.

Configuration

  • Config Path: ./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml

  • Ability: Reward

  • Benchmark Name: LongRewardBench

Usage

Run with Default Settings

loom-eval.run \
    --model_path <model_name> \
    --cfg_path ./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml \
    --device 0 1 \
    --eval

Run with Custom Tag

loom-eval.run \
    --model_path Meta-Llama/Meta-Llama-3.1-8B-Instruct \
    --cfg_path ./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml \
    --device 0 1 \
    --eval \
    --save_tag my_custom_tag

Lightweight Mode

loom-eval.run \
    --model_path <model_name> \
    --cfg_path ./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml \
    --device 0 1 2 3 \
    --eval \
    --lightweight

Additional Resources