MACHINE LEARNING BASED LOGANALYSIS FOR AUTOMATED ANOMALY DETECTION
DOI:
https://doi.org/10.61841/e9gbyb69Keywords:
machine learning, based log-analysis, automated, anomaly detectionAbstract
Many sensors are used in a single production process, making it difficult to pinpoint the exact source of a problem. More than one process cycle is required to make a semiconductor wafer. There are many cycles in this process, and it is difficult to spot abnormalities in time; thus, the process continues until it is complete. The cost of producing these wafers is high, and a process failure can have a significant impact on both time and money. As a result, anomaly detection in semiconductor production can benefit greatly from machine learning. A manufacturing facility may interrupt the operation and fix the problematic equipment if irregularities in the production process could be discovered or predicted sooner. As a result, semiconductor producers would see an improvement in process yield and a reduction in expenses.
Downloads
References
Astekin, M., Özcan, S., & Sözer, H. (2019, December). Incremental analysis of large-scale
system logs for anomaly detection. In 2019 IEEE International Conference on Big
Data (Big Data) (pp. 2119-2127). IEEE.
https://ieeexplore.ieee.org/abstract/document/9006593/?casa_token=hxpw6mTJT3
wAAAAA:ECu2SCHL2tEKSVVouhjmG9wvqOGc1RwecRT5iJlieiLXzN8SxEQR
IE__93o76-feOKZvbvPXwJk- [Accessed on 30-11-2021]
Bao, L., Li, Q., Lu, P., Lu, J., Ruan, T., & Zhang, K. (2018). Execution anomaly detection
in large-scale systems through console log analysis. Journal of Systems and
Software, 143, 172-186.
https://www.sciencedirect.com/science/article/pii/S0164121218301031?casa_token
=b3VD5kVjYFAAAAAA:Cua2Ru72vJh13GDLrxedhJxWEaqfxjgYKI4Q3Z3YjX
QTIhMlMcTZ5SZIw7zJvLvffChmE3WwarCt [Accessed on 30-11-2021]
Bhanage, D. A., Pawar, A. V., & Kotecha, K. (2021). IT Infrastructure Anomaly Detection
and Failure Handling: A Systematic Literature Review Focusing on Datasets, Log
Preprocessing, Machine & Deep Learning Approaches and Automated Tool. IEEE
Access. https://ieeexplore.ieee.org/abstract/document/9615039/ [Accessed on 30-11-
2021]
Cao, Q., Qiao, Y., & Lyu, Z. (2017, December). Machine learning to detect anomalies in
web log analysis. In 2017 3rd IEEE International Conference on Computer and
Communications (ICCC) (pp. 519-523). IEEE.
https://ieeexplore.ieee.org/abstract/document/8322600/?casa_token=8B_etk_kK6g
AAAAA:mqJylYHUudSJcUUqux8-
wnKTkRf_YaJl7Db4iIrJR9yXhYcUO1YXEQftBTnx5xKvpbvSYhVVKn8F
[Accessed on 30-11-2021]
Chen, Z., Liu, J., Gu, W., Su, Y., & Lyu, M. R. (2021). Experience Report: Deep Learningbased System Log Analysis for Anomaly Detection. arXiv preprint
arXiv:2107.05908. https://arxiv.org/abs/2107.05908 [Accessed on 30-11-2021]
Debnath, B., Solaimani, M., Gulzar, M. A. G., Arora, N., Lumezanu, C., Xu, J., ... & Khan,
L. (2018, July). LogLens: A real-time log analysis system. In 2018 IEEE 38th
international conference on distributed computing systems (ICDCS) (pp. 1052-
1062). IEEE.
https://ieeexplore.ieee.org/abstract/document/8416368/?casa_token=IGIQ0ao1T4c
AAAAA:bhijQZWWOino9I5C_DOI6ZU0qPejpVKxVeuqkyd_hEk6ApUZgnZX1R-Eet54dl189pdZ4zAPEl9 [Accessed on 30-11-2021]
Du, Q., Zhao, L., Xu, J., Han, Y., & Zhang, S. (2021, September). Log-Based Anomaly
Detection with Multi-Head Scaled Dot-Product Attention Mechanism.
In International Conference on Database and Expert Systems Applications (pp. 335-
347). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-030-86472-
9_31 [Accessed on 30-11-2021]
Fredriksson Franzén, M., & Tyrén, N. (2021). Anomaly detection for automated security log
analysis: Comparison of existing techniques and tools. https://www.divaportal.org/smash/record.jsf?pid=diva2:1576656 [Accessed on 30-11-2021]
Hirakawa, R., Uchida, H., Nakano, A., Tominaga, K., & Nakatoh, Y. (2021). Anomaly
detection on software log based on Temporal Memory. Computers & Electrical
Engineering, 95, 107433.
https://www.sciencedirect.com/science/article/pii/S0045790621003943?casa_token
=cfjgtm_Ev2MAAAAA:o63T4sPJTD8IhCbcLa3fQmtuMuRMAndlcJeyrENIZJ3Q
7R-eqSR-DbznKs2n-wUVq8g5aloZUdre [Accessed on 30-11-2021]
Hirakawa, R., Uchida, H., Nakano, A., Tominaga, K., & Nakatoh, Y. (2021). Large Scale
Log Anomaly Detection via Spatial Pooling1. Cognitive Robotics.
https://www.sciencedirect.com/science/article/pii/S2667241321000173 [Accessed
on 30-11-2021]
Hong, J., Park, S., Yoo, J. H., & Hong, J. W. K. (2020, November). Machine Learning based
SLA-Aware VNF Anomaly Detection for Virtual Network Management. In 2020
16th International Conference on Network and Service Management (CNSM) (pp. 1-I
7). IEEE.
https://ieeexplore.ieee.org/abstract/document/9269100/?casa_token=wytAnMa6Ips
AAAAA:iW4nmieLE8C1VaVkrZ9fom3Qr6YBgnDDrC5nhr2RGdxr2hvtEYIBGe
Y_nMCvP9uHqvgJi9P4ObhC [Accessed on 30-11-2021]
Schmidt, T., Hauer, F., & Pretschner, A. (2020, September). Automated Anomaly Detection
in CPS Log Files. In International Conference on Computer Safety, Reliability, and
Security (pp. 179-194). Springer, Cham.
https://link.springer.com/chapter/10.1007/978-3-030-54549-9_12 [Accessed on 30-
11-2021]
Shao, W., Wang, Z., Wang, X., Qiu, K., Jia, C., & Jiang, C. (2020). LSC: Online auto-update
smart contracts for fortifying blockchain-based log systems. Information
Sciences, 512, 506-517.
https://www.sciencedirect.com/science/article/pii/S0020025519309260?casa_token
=Zu98Df62yIkAAAAA:ZDpMsndd1Z8g0e3t1S0vB9ZJFYLIcwYESOq2GzjoZdV
CtF6iyH5bRhZKdgcUIGglw68ZdjG9Jj8r [Accessed on 30-11-2021]
Shin, D., Khan, Z. A., Bianculli, D., & Briand, L. (2021, October). A Theoretical Framework
for Understanding the Relationship Between Log Parsing and Anomaly Detection.
In International Conference on Runtime Verification (pp. 277-287). Springer, Cham.
https://link.springer.com/chapter/10.1007/978-3-030-88494-9_16 [Accessed on 30-
11-2021]
Skopik, F., Wurzenberger, M., & Landauer, M. (2021). Smart Log Data Analytics:
Techniques for Advanced Security Analysis. Skopik, F., Wurzenberger, M., &
Landauer, M. (2021). Smart Log Data Analytics: Techniques for Advanced Security
Analysis. [Accessed on 30-11-2021]
Tallón-Ballesteros, A. J., & Chen, C. (2020). Explainable AI: Using shapley value to explain
complex anomaly detection ML-based systems. Machine Learning and Artificial
Intelligence: Proceedings of MLIS 2020, 332, 152.
https://books.google.com/books?hl=en&lr=&id=hq4SEAAAQBAJ&oi=fnd&pg=P
A152&dq=MACHINE+LEARNING+BASED+LOGANALYSIS+FOR+AUTOMATED+ANOMALY+DETECTION&ots=2bZzHPW7
k-&sig=4IodPdO_0eiKyt_DEFC7Zm8diYM [Accessed on 30-11-2021]
Wang, Z., Tian, J., Fang, H., Chen, L., & Qin, J. (2021). LightLog: A lightweight temporal
convolutional network for log anomaly detection on the edge. Computer Networks,
108616.
https://www.sciencedirect.com/science/article/pii/S1389128621005119?casa_token
=heZpk7kfOZsAAAAA:-7ix1Wgc3Ww8JrXNhj6wGccuDeldgKXZYu8u1tH7oFfMUTjRGnRmfNtRaU-h3udOhCC5i8Zxtg3 [Accessed on 30-11-
2021]
Yadav, R. B., Kumar, P. S., & Dhavale, S. V. (2020, June). A Survey on Log Anomaly
Detection using Deep Learning. In 2020 8th International Conference on Reliability,
Infocom Technologies and Optimization (Trends and Future
Directions)(ICRITO) (pp. 1215-1220). IEEE.
https://ieeexplore.ieee.org/abstract/document/9197818/ [Accessed on 30-11-2021]
Yang, L., Chen, J., Wang, Z., Wang, W., Jiang, J., Dong, X., & Zhang, W. (2021, May).
Semi-supervised log-based anomaly detection via probabilistic label estimation.
In 2021 IEEE/ACM 43rd International Conference on Software Engineering
(ICSE) (pp. 1448-1460). IEEE.
https://ieeexplore.ieee.org/abstract/document/9401970/?casa_token=2OuRv9Pn0o
YAAAAA:cAaQy0F9zpyyakMO8EmgRE7DtuQOKkqWOrYlorc_sgMhPYBlcNapgXFCSAaaCG_vJapp4rQJCci [Accessed on 30-11-2021]
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Author
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format for any purpose, even commercially.
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
- The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notices:
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation .
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.
