Comparative Evaluation of Automated Approaches for Legal Aid Document Generation: Template-Based, Rule-Based, and LLM-Based Methods

Abstract

The accessibility of legal services remains a critical challenge in the United States, with over 80% of low-income individuals unable to obtain necessary civil legal assistance. This study presents a systematic comparative evaluation of three automated document generation approaches for legal aid applications: template-based document generation, rule-based conditional generation, and large language model (LLM)-based intelligent drafting. The study compiled and anonymized N=247 housing-related legal aid cases spanning eviction defense, security deposit claims, and repair request letters, drawn from publicly available eviction records in the Atlanta metropolitan area and supplemented by open legal datasets including Multi-LexSum and LegalBench. The evaluation framework assessed four dimensions: legal element completeness (92.3% for templates, 94.7% for rules, 96.1% for LLMs), linguistic accuracy (88.5%, 91.2%, 94.8%), jurisdictional compliance (95.1%, 93.4%, 89.7%), and practitioner usability scores (7.2/10, 8.1/10, 8.9/10). The findings reveal distinct performance trade-offs: template methods excel in standard cases with high efficiency but limited flexibility; rule-based approaches handle moderate complexity at increased maintenance costs; and LLM methods demonstrate superior adaptability in non-standard scenarios that require rigorous post-processing validation mechanisms.