LongRewardBench
===============

:Ability: **Reward**
:Path: ``benchmarks/Reward/LongRewardBench``

.. note::
   :class: navigation
   
   :doc:`← Back to Benchmarks Overview <../index>` | 
   View other Reward benchmarks in the sidebar


Overview
--------

This benchmark evaluates LongRewardBench capabilities within the Reward dimension.

Configuration
-------------

- **Config Path**: ``./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml``
- **Ability**: Reward
- **Benchmark Name**: LongRewardBench

Usage
-----

Run with Default Settings
~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   loom-eval.run \
       --model_path <model_name> \
       --cfg_path ./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml \
       --device 0 1 \
       --eval

Run with Custom Tag
~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   loom-eval.run \
       --model_path Meta-Llama/Meta-Llama-3.1-8B-Instruct \
       --cfg_path ./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml \
       --device 0 1 \
       --eval \
       --save_tag my_custom_tag

Lightweight Mode
~~~~~~~~~~~~~~~~

.. code-block:: bash

   loom-eval.run \
       --model_path <model_name> \
       --cfg_path ./benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml \
       --device 0 1 2 3 \
       --eval \
       --lightweight

Additional Resources
--------------------

- `Benchmark Source Code <https://github.com/loomeval/loom-eval/tree/main/benchmarks/Reward/LongRewardBench>`_
- `Configuration File <https://github.com/loomeval/loom-eval/blob/main/benchmarks/Reward/LongRewardBench/configs/LongRewardBench.yaml>`_