FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024

In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models with Reinforcement Learning from Market Feedback (RLMF). The framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals.

Traditional LLMs, while competent at sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, the LLaMA-3.2-3B-Instruct model is fine-tuned using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Evaluation shows the RLMF-tuned framework outperforms baseline methods in signal consistency and achieves tighter trading outcomes.

Awarded winner of Task II at the FinRL Contest 2024. The core departure from prior LLM-for-trading approaches is the reward signal itself: rather than relying on static sentiment labels, the model is shaped directly by market feedback, so the training signal moves with the thing it is trying to predict.

Built on a relatively small 3B-parameter instruct model rather than a larger foundation model, suggesting the RLMF prompt design is doing more of the work than raw model scale. Code is released publicly on GitHub.