During our investigations, we found a significant limitation in the original prompts of LegalBench. The complex nature of these prompts, combined with the challenges encountered by open source LLMs in adhering to instructions - particularly in handling formatting - leads to a substantial drop in performance (as measured by accuracy). The generated sentences are often verbose and difficult to parse, rendering LegalBench in its current form too stringent and failing to accurately gauge improvement on the task.