Revolutionizing Log Parsing for Modern Software Systems

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Authors

  • Stefan Petrescu
View graph of relations

Details

Original languageEnglish
Title of host publicationProceedings
Subtitle of host publication2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages17-21
Number of pages5
ISBN (electronic)9798350319569
Publication statusPublished - 2023
Event34th IEEE International Symposium on Software Reliability Engineering Workshop, ISSREW 2023 - Florence, Italy
Duration: 9 Oct 202312 Oct 2023

Publication series

NameProceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023

Abstract

With the tremendous log volume generated by modern systems, automated log analysis becomes indispensable for discovering crucial insights into the behavior of running systems. The foremost step subsumed in an automated log analysis pipeline, called log parsing, significantly influences its entire performance. Despite its significance, log parsing lacks quality implementations and, in practice, suffers from fundamental limitations, thereby creating a bottleneck for discovering valuable insights at the log line level. As a consequence, in my PhD thesis, I endeavor to explore a novel paradigm, called entity parsing, which goes beyond previous work by not only addressing the current limitations of log parsing but by exploring a new avenue for log parsing that could lead to significant advances in how we perform log analysis. As preliminary results show, entity parsing, despite solving a more difficult problem than conventional log parsing, is viable and obtains a significantly better accuracy on comparable datasets. By applying a first-principles approach, entity parsing is based on three fundamental components: data, machine learning, and source code. These components, leveraged within the framework of entity parsing, process logs in a way that could significantly improve the overall dependability of systems, enhancing their availability, security, and reliability.

Keywords

    log analysis, log parsing, reliability

ASJC Scopus subject areas

Cite this

Revolutionizing Log Parsing for Modern Software Systems. / Petrescu, Stefan.
Proceedings : 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 17-21 (Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023).

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Petrescu, S 2023, Revolutionizing Log Parsing for Modern Software Systems. in Proceedings : 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023. Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023, Institute of Electrical and Electronics Engineers Inc., pp. 17-21, 34th IEEE International Symposium on Software Reliability Engineering Workshop, ISSREW 2023, Florence, Italy, 9 Oct 2023. https://doi.org/10.1109/ISSREW60843.2023.00036
Petrescu, S. (2023). Revolutionizing Log Parsing for Modern Software Systems. In Proceedings : 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023 (pp. 17-21). (Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISSREW60843.2023.00036
Petrescu S. Revolutionizing Log Parsing for Modern Software Systems. In Proceedings : 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 17-21. (Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023). doi: 10.1109/ISSREW60843.2023.00036
Petrescu, Stefan. / Revolutionizing Log Parsing for Modern Software Systems. Proceedings : 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 17-21 (Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023).
Download
@inproceedings{85a5d220bd8345338a59408f4c39a678,
title = "Revolutionizing Log Parsing for Modern Software Systems",
abstract = "With the tremendous log volume generated by modern systems, automated log analysis becomes indispensable for discovering crucial insights into the behavior of running systems. The foremost step subsumed in an automated log analysis pipeline, called log parsing, significantly influences its entire performance. Despite its significance, log parsing lacks quality implementations and, in practice, suffers from fundamental limitations, thereby creating a bottleneck for discovering valuable insights at the log line level. As a consequence, in my PhD thesis, I endeavor to explore a novel paradigm, called entity parsing, which goes beyond previous work by not only addressing the current limitations of log parsing but by exploring a new avenue for log parsing that could lead to significant advances in how we perform log analysis. As preliminary results show, entity parsing, despite solving a more difficult problem than conventional log parsing, is viable and obtains a significantly better accuracy on comparable datasets. By applying a first-principles approach, entity parsing is based on three fundamental components: data, machine learning, and source code. These components, leveraged within the framework of entity parsing, process logs in a way that could significantly improve the overall dependability of systems, enhancing their availability, security, and reliability.",
keywords = "log analysis, log parsing, reliability",
author = "Stefan Petrescu",
note = "Funding Information: This research project, initiated at TU Delft, is currently underway at Leibniz University of Hannover under the supervision of Prof. Dr. Jan Rellermeyer. Furthermore, the industry experiments were carried out with the valuable support of Floris den Hengst at the AI for Fintech Research lab, ING Netherlands. ; 34th IEEE International Symposium on Software Reliability Engineering Workshop, ISSREW 2023 ; Conference date: 09-10-2023 Through 12-10-2023",
year = "2023",
doi = "10.1109/ISSREW60843.2023.00036",
language = "English",
series = "Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "17--21",
booktitle = "Proceedings",
address = "United States",

}

Download

TY - GEN

T1 - Revolutionizing Log Parsing for Modern Software Systems

AU - Petrescu, Stefan

N1 - Funding Information: This research project, initiated at TU Delft, is currently underway at Leibniz University of Hannover under the supervision of Prof. Dr. Jan Rellermeyer. Furthermore, the industry experiments were carried out with the valuable support of Floris den Hengst at the AI for Fintech Research lab, ING Netherlands.

PY - 2023

Y1 - 2023

N2 - With the tremendous log volume generated by modern systems, automated log analysis becomes indispensable for discovering crucial insights into the behavior of running systems. The foremost step subsumed in an automated log analysis pipeline, called log parsing, significantly influences its entire performance. Despite its significance, log parsing lacks quality implementations and, in practice, suffers from fundamental limitations, thereby creating a bottleneck for discovering valuable insights at the log line level. As a consequence, in my PhD thesis, I endeavor to explore a novel paradigm, called entity parsing, which goes beyond previous work by not only addressing the current limitations of log parsing but by exploring a new avenue for log parsing that could lead to significant advances in how we perform log analysis. As preliminary results show, entity parsing, despite solving a more difficult problem than conventional log parsing, is viable and obtains a significantly better accuracy on comparable datasets. By applying a first-principles approach, entity parsing is based on three fundamental components: data, machine learning, and source code. These components, leveraged within the framework of entity parsing, process logs in a way that could significantly improve the overall dependability of systems, enhancing their availability, security, and reliability.

AB - With the tremendous log volume generated by modern systems, automated log analysis becomes indispensable for discovering crucial insights into the behavior of running systems. The foremost step subsumed in an automated log analysis pipeline, called log parsing, significantly influences its entire performance. Despite its significance, log parsing lacks quality implementations and, in practice, suffers from fundamental limitations, thereby creating a bottleneck for discovering valuable insights at the log line level. As a consequence, in my PhD thesis, I endeavor to explore a novel paradigm, called entity parsing, which goes beyond previous work by not only addressing the current limitations of log parsing but by exploring a new avenue for log parsing that could lead to significant advances in how we perform log analysis. As preliminary results show, entity parsing, despite solving a more difficult problem than conventional log parsing, is viable and obtains a significantly better accuracy on comparable datasets. By applying a first-principles approach, entity parsing is based on three fundamental components: data, machine learning, and source code. These components, leveraged within the framework of entity parsing, process logs in a way that could significantly improve the overall dependability of systems, enhancing their availability, security, and reliability.

KW - log analysis

KW - log parsing

KW - reliability

UR - http://www.scopus.com/inward/record.url?scp=85178504104&partnerID=8YFLogxK

U2 - 10.1109/ISSREW60843.2023.00036

DO - 10.1109/ISSREW60843.2023.00036

M3 - Conference contribution

AN - SCOPUS:85178504104

T3 - Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023

SP - 17

EP - 21

BT - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 34th IEEE International Symposium on Software Reliability Engineering Workshop, ISSREW 2023

Y2 - 9 October 2023 through 12 October 2023

ER -