Template-Type: ReDIF-Article 1.0
Author-Name: Chen, Yanhuan
Author-Name: Tang, Tianxing
Title: Evaluating Prompt Engineering Strategies for Few-Shot Cyber Threat Intelligence Entity and Relation Extraction from Multi-Source Reports
Abstract: The proliferation of multi-source cyber threat intelligence reports---spanning vulnerability databases, government advisories, vendor analyses, and open-source feeds---has outpaced the capacity of human analysts to extract structured knowledge about adversary tactics, techniques, and procedures. While large language models present a promising avenue for automating this extraction under low-resource conditions, no systematic empirical comparison of prompt engineering strategies exists for the cyber threat intelligence domain. This study evaluates six prompt engineering strategies---zero-shot, one-shot, three-shot, five-shot, retrieval-augmented five-shot, and chain-of-thought five-shot---across four publicly available cyber threat intelligence named entity recognition datasets (DNRTI, CyNER, AnnoCTR, APTNER) and one relation extraction corpus, using GPT-4, GPT-3.5-turbo, and Llama-3-70B. The retrieval-augmented five-shot strategy achieves the highest named entity recognition F1 of 0.753 on CyNER with GPT-4, narrowing the gap with the fine-tuned SecureBERT baseline to 2.8 percentage points. Chain-of-thought prompting yields the lowest expected calibration error (0.108), suggesting its value for uncertainty-aware intelligence triage. Cross-source extraction variance reaches 12.2 F1 points between the easiest and hardest corpora, underscoring the challenge of heterogeneous intelligence fusion. These findings offer actionable guidance for deploying prompt-based extraction in operational threat intelligence pipelines aligned with the NIST Cybersecurity Framework and national cyber defense priorities.
Keywords: cyber threat intelligence, named entity recognition, prompt engineering, few-shot learning
Journal: Journal of Science, Innovation & Social Impact
Pages: 153-164
Volume: 2
Issue: 2
Year: 2026
File-URL: https://pinnaclepubs.com/index.php/JSISI/article/view/734/705
File-Format: Application/pdf
Handle: RePEc:dba:jsisia:v:2:y:2026:i:2:p:153-164