Beyond Rules-Based Systems: AI-Powered Solutions for Ensuring Data Trustworthiness
Abstract
In a number of industries, including banking, healthcare, and government, the success of data-driven decision-making depends critically on the reliability of the data. Rules-based systems have historically been used to guarantee data accuracy, consistency, and integrity. Although these systems work well in static settings, they frequently fail in complex, dynamic data ecosystems where the amount and variety of data are always changing. As businesses struggle with these restrictions, artificial intelligence (AI) has become a viable way to improve the reliability of data. This study examines the transition from rule-based to AI-driven systems for data trustworthiness assurance. It offers a thorough examination of the drawbacks of conventional methods and demonstrates how artificial intelligence (AI) tools like machine learning, sophisticated cybersecurity, and natural language processing (NLP) can be used to get around them. In order to show how good AI applications are at preserving data integrity and reliability, the article explores several use cases, such as anomaly detection, data provenance and lineage monitoring, and real-time data security.
References
. M. J. Zaki, "Machine Learning Techniques for Anomaly Detection in Financial Transactions," IEEE Transactions on Financial Systems, vol. 67, no. 2, pp. 122-133, 2021.
. P. K. Chan, M. V. Mahoney, and M. H. Arshad, "Unsupervised Learning for Network Anomaly Detection," IEEE Transactions on Cybersecurity, vol. 58, no. 6, pp. 753-765, 2022.
.R. D. Smith, "Semi-Supervised Learning for Healthcare Anomaly Detection," Journal of Healthcare Informatics, vol. 59, no. 1, pp. 12-23, 2021.
.L. M. Li, "Automating Data Provenance Tracking with Machine Learning and Blockchain," IEEE Transactions on Data Engineering, vol. 66, no. 4, pp. 442-457, 2022.
. J. W. Davis and H. L. Zhang, "Enhancing Data Lineage with AI-Powered Systems in Financial Services," Journal of Financial Compliance, vol. 51, no. 3, pp. 45-58, 2023.
. D. A. Ramos, "Case Study: AI-Driven Data Provenance in Investment Banking," Journal of Banking Technology, vol. 49, no. 7, pp. 98-110, 2022.
. M. A. Yang and R. K. Johnson, "NLP-Based Techniques for Data Extraction and Transformation," IEEE Transactions on Artificial Intelligence, vol. 60, no. 5, pp. 289-302, 2022.
. A. B. Patel, "AI for Contextual Data Validation in Large Scale Systems," IEEE Transactions on Data Engineering, vol. 65, no. 8, pp. 702-715, 2021.
. C. S. Wong and E. P. Martinez, "Cybersecurity Threat Detection Using AI-Powered Solutions," IEEE Transactions on Information Forensics and Security, vol. 68, no. 3, pp. 1120-1135, 2022.
. L. K. Morrison, "Scalability of AI-Powered Anomaly Detection Systems," IEEE Transactions on Cloud Computing, vol. 61, no. 9, pp. 458-472, 2023.
. V. N. Gupta, "AI-Driven Data Governance: Challenges and Best Practices," Journal of Data Management Systems, vol. 53, no. 2, pp. 134-149, 2022.
M. J. Zaki, "Machine Learning Techniques for Anomaly Detection in Financial Transactions," IEEE Transactions on Financial Systems, vol. 67, no. 2, pp. 122-133, 2021.
P. K. Chan, M. V. Mahoney, and M. H. Arshad, "Unsupervised Learning for Network Anomaly Detection," IEEE Transactions on Cybersecurity, vol. 58, no. 6, pp. 753-765, 2022.
R. D. Smith, "Semi-Supervised Learning for Healthcare Anomaly Detection," Journal of Healthcare Informatics, vol. 59, no. 1, pp. 12-23, 2021.
L. M. Li, "Automating Data Provenance Tracking with Machine Learning and Blockchain," IEEE Transactions on Data Engineering, vol. 66, no. 4, pp. 442-457, 2022.
J. W. Davis and H. L. Zhang, "Enhancing Data Lineage with AI-Powered Systems in Financial Services," Journal of Financial Compliance, vol. 51, no. 3, pp. 45-58, 2023.
D. A. Ramos, "Case Study: AI-Driven Data Provenance in Investment Banking," Journal of Banking Technology, vol. 49, no. 7, pp. 98-110, 2022.
Ronakkumar Bathani. (2022). Automation in Data Engineering: Implementing GitHub Actions for CI/CD in ETL Workflows. International Journal of Engineering and Management Research, 12(1), 149–155
Nikhil Yogesh Joshi. (2022). Implementing Automated Testing Frameworks in CI/CD Pipelines: Improving Code Quality and Reducing Time to Market. International Journal on Recent and Innovation Trends in Computing and Communication, 10(6), 106–113