Distributed Batch Processing Architecture for Cross-Platform Abuse Detection at Scale

Hongbo Wang; Kun Qian; Chunhe Ni; Jiang Wu

Authors

Hongbo Wang Computer Science, University of Southern California, Los Angeles, CA, USA Author
Kun Qian Business Intelligence, Engineering School of Information and Digital Technologies, Villejuif, France Author
Chunhe Ni Computer Science, University of Texas at Dallas, Richardson, TX, USA Author
Jiang Wu Computer Science, University of Southern California, Los Angeles, CA, USA Author

Keywords:

distributed processing, abuse detection, cross-platform analysis, deep learning

Abstract

This paper presents a distributed batch processing architecture for cross-platform abuse detection at scale, addressing the challenges of detecting coordinated malicious activities across heterogeneous online platforms. The proposed architecture integrates platform-specific preprocessing with cross-platform feature normalization through a modular design that separates data acquisition, preprocessing, distributed processing, and result aggregation. We implement a dynamic batching strategy that optimizes computational resource utilization while maintaining detection latency within acceptable bounds. The architecture employs a multi-task learning approach with specialized deep learning models for different abuse types, leveraging platform-aware adversarial encoding to learn platform-independent representations. Performance optimization techniques including adaptive content resizing and model quantization enable efficient execution across diverse hardware environments. Experimental evaluation conducted on a dataset of 3.2 million content items from five major platforms demonstrates that our approach achieves a 12.7 % improvement in cross-platform F1-score compared to platform-specific models, while providing 2.8x higher throughput than naive cross-platform approaches. The architecture's ability to identify coordinated abuse campaigns spanning multiple platforms highlights the value of integrated cross-platform analysis in detecting sophisticated abuse patterns. The implementation successfully balances detection accuracy, processing efficiency, and scalability requirements, providing an effective solution for large-scale abuse detection across diverse online environments.

References

1. M. K. Geldenhuys, B. J. Pfister, D. Scheinert, L. Thamsen, and O. Kao, "Khaos: Dynamically optimizing checkpointing for de-pendable distributed stream processing," in Proc. 17th Conf. Comput. Sci. Intell. Syst. (FedCSIS), Sept. 2022, pp. 553–561, doi: 10.15439/2022F225.

2. P. Sheth, T. Kumarage, R. Moraffah, A. Chadha, and H. Liu, "Peace: Cross-platform hate speech detection—a causality-guided framework," in Proc. Joint Eur. Conf. Mach. Learn. Knowl. Discovery Databases (ECML PKDD), Cham, Switzerland: Springer, Sept. 2023, pp. 559–575. ISBN: 9783031434129.

3. P. Yi and A. Zubiaga, "Cyberbullying detection across social media platforms via platform-aware adversarial encoding," in Proc. Int. AAAI Conf. Web Social Media, vol. 16, May 2022, pp. 1430–1434, doi: 10.1609/icwsm.v16i1.19401.

4. D. Chaves, E. Fidalgo, E. Alegre, F. Jáñez-Martino, and J. Velasco-Mata, "CPU vs GPU performance of deep learning based face detectors using resized images in forensic applications," in Proc. 9th Int. Conf. Imaging Crime Detection Prevention (ICDP), Dec. 2019, pp. 93–98, doi: 10.1049/cp.2019.1174.

5. A. A. Ojugo and R. E. Yoro, "Forging a deep learning neural network intrusion detection framework to curb the distributed denial of service attack," Int. J. Electr. Comput. Eng., vol. 11, no. 2, pp. 1498–1506, 2021, doi: 10.11591/ijece.v11i2.

6. J. Chen, L. Yan, S. Wang, and W. Zheng, "Deep reinforcement learning-based automatic test case generation for hardware verification," J. Artif. Intell. Gen. Sci. (JAIGS), vol. 6, no. 1, pp. 409–429, 2024, doi: 10.60087/jaigs.v6i1.267.

7. Y. Xu, Y. Liu, J. Wu, and X. Zhan, "Privacy by design in machine learning data collection: An experiment on enhancing user experience," Appl. Comput. Eng., vol. 97, pp. 64–68, 2024, doi: 10.54254/2755-2721/97/20241388.

8. X. Xu, Z. Xu, P. Yu, and J. Wang, "Enhancing user intent for recommendation systems via large language models," arXiv preprint arXiv:2501.10871, 2025, doi: 10.48550/arXiv.2501.10871.

9. P. Yu, Z. Xu, J. Wang, and X. Xu, "The application of large language models in recommendation systems," arXiv preprint arXiv:2501.02178, 2025, doi: 10.48550/arXiv.2501.02178.

10. D. Ma, "AI-driven optimization of intergenerational community services: An empirical analysis of elderly care communities in Los Angeles," Artif. Intell. Mach. Learn. Rev., vol. 5, no. 4, pp. 10–25, 2024, doi: 10.69987/AIMLR.2024.50402.

11. R. T. Rust, W. Rand, M. H. Huang, A. T. Stephen, G. Brooks, and T. Chabuk, "Real-time brand reputation tracking using social media," J. Marketing, vol. 85, no. 4, pp. 21-43, 2021, doi: 10.1177/0022242921995173.

12. P. Wang, M. Varvello, C. Ni, R. Yu, and A. Kuzmanovic, "Web-lego: Trading content strictness for faster webpages," in Proc. IEEE Conf. Comput. Commun. (INFOCOM), May 2021, pp. 1–10, doi: 10.1109/INFOCOM42981.2021.9488904.

13. C. Ni, C. Zhang, W. Lu, H. Wang, and J. Wu, "Enabling intelligent decision making and optimization in enterprises through data pipelines," World J. Innov. Mod. Technol., vol. 7, no. 1, 2024, doi: 10.53469/wjimt.2024.07(02).13.

14. C. Zhang, W. Lu, C. Ni, H. Wang, and J. Wu, "Enhanced user interaction in operating systems through machine learning lan-guage models," in Proc. Int. Conf. Image, Signal Process., Pattern Recognit. (ISPP), vol. 13180, Jun. 2024, pp. 1623–1630, doi: 10.1117/12.3033610.

15. H. Wang, J. Wu, C. Zhang, W. Lu, and C. Ni, "Intelligent security detection and defense in operating systems based on deep learning," Int. J. Comput. Sci. Inf. Technol., vol. 2, no. 1, pp. 359–367, 2024, doi:10.62051/ijcsit.v2n1.37.

16. W. Lu, C. Ni, H. Wang, J. Wu, and C. Zhang, "Machine learning-based automatic fault diagnosis method for operating sys-tems," World J. Innov. Mod. Technol., vol. 7, no. 1, 2024, doi: 10.53469/wjimt.2024.07(02).12.

17. C. Zhang, W. Lu, J. Wu, C. Ni, and H. Wang, "SegNet network architecture for deep learning image segmentation and its in-tegrated applications and prospects," Acad. J. Sci. Technol., vol. 9, no. 2, pp. 224–229, 2024, doi: 10.54097/rfa5x119.

18. J. Wu, H. Wang, C. Ni, C. Zhang, and W. Lu, "Data pipeline training: Integrating AutoML to optimize the data flow of machine learning models," in Proc. 7th Int. Conf. Adv. Algorithms Control Eng. (ICAACE), Mar. 2024, pp. 730–734, doi: 10.1109/ICAACE61206.2024.10549260.

19. J. Wu, H. Wang, C. Ni, C. Zhang, and W. Lu, "Case study of next-generation artificial intelligence in medical image diagnosis based on cloud computing," J. Theory Pract. Eng. Sci., vol. 4, no. 2, pp. 66–73, 2024, doi: 10.53469/jtpes.2024.04(02).10.

20. C. Ni, J. Wu, H. Wang, W. Lu, and C. Zhang, "Enhancing cloud-based large language model processing with Elasticsearch and Transformer models," in Proc. Int. Conf. Image, Signal Process., Pattern Recognit. (ISPP), vol. 13180, Jun. 2024, pp. 1648–1654, doi: 10.1117/12.3033606.

21. C. Jiang, H. Zhang, and Y. Xi, "Automated game localization quality assessment using deep learning: A case study in error pattern recognition," J. Adv. Comput. Syst., vol. 4, no. 10, pp. 25–37, 2024, doi: 10.69987/JACS.2024.41003.

22. T. Huang, Z. Xu, P. Yu, J. Yi, and X. Xu, "A hybrid transformer model for fake news detection: Leveraging Bayesian optimiza-tion and bidirectional recurrent unit," 2025, arXiv preprint arXiv:2502.09097, doi. 10.48550/arXiv.2502.09097.

23. X. Xu, P. Yu, Z. Xu, and J. Wang, "A hybrid attention framework for fake news detection with large language models," 2025, arXiv preprint arXiv:2501.11967, doi: 10.48550/arXiv.2501.11967.

24. W. Bi, T. K. Trinh, and S. Fan, "Machine learning-based pattern recognition for anti-money laundering in banking systems," J. Adv. Comput. Syst., vol. 4, no. 11, pp. 30–41, 2024, doi: 10.69987/JACS.2024.41103.

25. X. Ni, L. Yan, X. Ke, and Y. Liu, "A hierarchical Bayesian market mix model with causal inference for personalized marketing optimization," J. Artif. Intell. Gen. Sci. (JAIGS), vol. 6, no. 1, pp. 378–396, 2024, doi: 10.60087/jaigs.v6i1.261.

26. S. Wang, J. Chen, L. Yan, and Z. Shui, "Automated test case generation for chip verification using deep reinforcement learning," J. Knowl. Learn. Sci. Technol., vol. 4, no. 1, pp. 1–12, 2025, doi: 10.60087/jklst.v4.n1.001.

27. G. Rao, T. Lu, L. Yan, and Y. Liu, "A hybrid LSTM-KNN framework for detecting market microstructure anomalies: Evidence from high-frequency jump behaviors in credit default swap markets," J. Knowl. Learn. Sci. Technol., vol. 3, no. 4, pp. 361–371, 2024, doi: 10.60087/jklst.v3.n4.p361.

28. D. Ma, "Standardization of community-based elderly care service quality: A multi-dimensional assessment model in Southern California," J. Adv. Comput. Syst., vol. 4, no. 12, pp. 15–27, 2024, doi: 10.69987/JACS.2024.41202.

29. S. Srinivas, "A machine learning-based approach for predicting patient punctuality in ambulatory care centers," Int. J. Environ. Res. Public Health, vol. 17, no. 10, p. 3703, 2020, doi: 10.3390/ijerph17103703.

30. Y. Liu, F. Lei, and W. Huangcheng, "Design and implementation of cross-border e-commerce risk control system based on machine learning algorithm and computer simulation," in Proc. 2024 7th Int. Conf. Comput. Inf. Sci. Artif. Intell., 2024, pp. 179-184, doi: 10.1145/3703187.3703217.

31. S. Tolenov and B. Omarov, "Real-Time Self-Localization and Mapping for Autonomous Navigation of Mobile Robots in Un-known Environments," Int. J. Adv. Comput. Sci. Appl., vol. 15, no. 10, 2024, doi: 10.14569/ijacsa.2024.0151090.

32. M. Wei, S. Wang, Y. Pu, and J. Wu, "Multi-agent reinforcement learning for high-frequency trading strategy optimization," J. AI-Powered Med. Innov., vol. 2, no. 1, pp. 109–124, 2024, doi: 10.13140/RG.2.2.27275.09761.

33. D. Ma, M. Jin, Z. Zhou, J. Wu, and Y. Liu, "Deep learning-based ADL assessment and personalized care planning optimization in adult day health center," Appl. Comput. Eng., vol. 118, pp. 14–22, 2024, doi: 10.54254/2755-2721/2025.18474.

34. C. Jiang, G. Jia, and C. Hu, "AI-driven cultural sensitivity analysis for game localization: A case study of player feedback in East Asian markets," Artif. Intell. Mach. Learn. Rev., vol. 5, no. 4, pp. 26–40, 2024, doi: 10.69987/AIMLR.2024.50403.

35. J. Weng and X. Jiang, "Research on movement fluidity assessment for professional dancers based on artificial intelligence technology," Artif. Intell. Mach. Learn. Rev., vol. 5, no. 4, pp. 41–54, 2024, doi: 10.69987/AIMLR.2024.50404.