Web portallarında kullanıcı davranışlarının yerinde tespiti ve web madenciliğinde kullanımı için yenilikçi bir yaklaşım = An innovative approach for on-premises detection of user behaviors on web portals and its use in web mining

Abstract

Web portalları ve kurumsal web uygulamalarından elde edilen etkileşim verilerinin analizi, kullanıcı davranışlarının anlaşılması açısından büyük önem taşımaktadır. Temel amacı sunucuya yapılan erişimleri kaydetmek olan sunucu günlükleri, kullanıcı etkileşimlerini açıklayabilmek için detaylı veri sağlayamazlar. Bunun yanında, uygulama günlüğü bakış açısına sahip olmayan güncel web analitik araçları, özellikle kurumsal web portallarında kullanıcıların bireysel olarak hizmetleri kullanma biçimlerini ve oturumları boyunca yaptıkları işlemleri takip etme ve anlamlandırma açısından zayıftırlar. Son yıllarda artan veri gizliliği ve egemenliği endişeleri de bulut tabanlı üçüncü parti araçların kullanımında çekincelere yol açmaktadır. Diğer taraftan, verinin değerini bilen ve ondan sonuna kadar yararlanmayı amaçlayan kuruluşlar için üzerinde web kullanım madenciliği teknikleri uygulayabilecekleri verilere sahip olmak son derece önemlidir. Bu tezde, web portalları ve büyük ölçekli kurumsal web uygulamaları için uygulama günlükleri ile web analitiğini bütüncül bir yapıda birleştiren CAWAL adında bir model ve onun pratik bir uygulaması olan yazılım çerçevesi önerilmiştir. Özellikle çok sunuculu ortamlar için tasarlanan model, alt alanlar arası oturum takibi, gerçek zamanlı analiz, yüksek doğruluğa sahip veri toplama, analitik çıkarımı ve veri depolama yetenekleriyle web analitiğinde alternatif bir çözüm sunmaktadır. Yalın mimarisine karşın zengin veri seti sunan çerçeve, gerçek bir web portalı üzerinde uygulanarak etkinliği sınanmıştır. Uygulama sonucu elde edilen bulgular çerçevenin farklı kullanıcı niteliklerini ve gezinme davranışlarını çok boyutlu ve kapsamlı biçimde analiz edebildiğini göstermiştir. Gerçekleştirilen yük testlerinde CAWAL, yaygın kullanılan açık kaynaklı web analitik araçları OWA ve Matomo'ya kıyasla çok daha düşük yanıt süreleri sunarak performansını kanıtlamıştır. Veri ambarında depolanan yapılandırılmış ilişkisel verilerin web kullanım madenciliğinde veri kaynağı olarak kullanımı ise bu alana özgün ve yenilikçi bir yaklaşım getirmiştir. Yöntem, WUM'da sunucu günlüklerinin kullanımını ve veri ön işleme aşamasını ortadan kaldırarak süreci önemli ölçüde kısaltmış; aynı zamanda zenginleştirilmiş oturum ve sayfa gösterimi verileri ile madencilik faaliyetlerinin başarımını arttırmıştır. CAWAL'ın gelişmiş veri toplama ve analiz kabiliyetleri multidisipliner araştırmaları destekleyerek gelecekteki çalışmalarda yeni ve farklı uygulama senaryolarının ortaya çıkmasına katkı sağlayacaktır.

The increasing use of web portals, corporate web applications, and online platforms necessitates new approaches to business strategies and academic research. This situation has made it imperative to analyze the application data and better understand user behavior. Web analytics and web usage mining (WUM) play an essential role in corporate decision-making processes based on the analysis of data obtained to understand user behavior and integrate this information into business strategies. Web usage mining uses server logs as a traditional data source. While these logs provide valuable data about user interactions, analyzing and processing this data often requires complex and time-consuming processes. Existing tools often provide superficial analysis, not allowing for a deep understanding of user interactions. This shortcoming has led to the need for the development of new and more effective methods in the field of web usage mining and analytics. However, today's growing concerns about data privacy and security have also led organizations to seek greater control and ownership over user data. Cloud-based web analytics tools, in particular, have limitations in terms of security and data sovereignty, while client-side user monitoring tools face challenges such as data accuracy and lack of server-side metrics. For all these reasons, there is a need for an approach to overcome the challenges faced in web analytics and usage mining from a different perspective. This study proposes an innovative model to detect user behavior on web portals and use this information in web mining processes. It also aims to test its effectiveness by developing a software framework that practically implements the model. This model, which we call CAWAL (Combined Analytics and Web Application Log), represents a holistic approach that integrates application logs and web analytics by overcoming the limitations of traditional web analytics tools. This integration aims to analyze the data obtained from web portals more effectively and to use the information obtained from these analyses in strategic decision-making processes. This innovative approach fills an essential gap in the web analytics and usage mining literature, expanding the application and research possibilities in these fields and opening new research avenues. The proposed model is expected to solve challenges that traditional methods cannot overcome, especially in complex and multi-server web environments. Another thesis objective is to apply the CAWAL model to a genuine system, evaluate its performance, and show that the data obtained can be effectively used in web mining. The development of the model will enable a more detailed analysis of user behavior for corporate web applications and portals and contribute to strategic decision-making processes. The central hypothesis of this study is that the application of the proposed model and the framework developed based on this model leads to significant improvements in the fields of web analytics and web mining and provides a more effective and comprehensive data collection and analysis process by overcoming the limitations of traditional methods used in these fields. The academic importance of the study lies in the fact that it fills crucial gaps in the fields of web analytics and web usage mining and brings an innovative approach to methodologies in these disciplines. The model's integration with application logs and its success in session identification, user identification, and data accuracy has primarily eliminated the preprocessing stage in WUM and made the analysis processes more efficient and effective. This approach offers new opportunities in user behavior analysis, enabling web portals and enterprise web applications to understand their users better and effectively integrate these insights into their business strategies. In practical terms, the CAWAL framework has provided new ways of understanding user interactions in web portal management and using this information in strategic decision-making processes. Its capacity to effectively track various user engagements, especially in multi-server environments and web portals with different (sub)domains, provides a significant advantage in enterprise-level applications. In addition, the detailed data collection and analysis capabilities offered by the model will enable organizations to be more effective in their strategic decision-making processes and help them better target their business strategies and web content using this data. The CAWAL model is designed to cover various platforms and interaction types. The model uses a server-side logging API for data collection and processing. The use of a server-side API enables both detailed analysis of user behavior and the improvement of application performance and security. The model was implemented on a university web portal, where user interactions and portal performance were analyzed extensively. These analyses included various metrics and visualization techniques to understand user behavior, preferences, and interaction patterns. Furthermore, the data collected during the implementation of the model was analyzed along various dimensions such as user types, visit durations, and preferred services. This detailed data analysis plays a critical role in understanding how users navigate the web portal, which content and services they are more interested in, and how to improve the user experience. Thus, the methodology makes valuable contributions to web analytics and web usage mining, both theoretically and practically. Integration of the CAWAL framework into an enterprise web application and performance tests have demonstrated the model's capacity to work effectively. Moreover, its superior response times and data processing capacity compared to well-known open-source tools such as Open Web Analytics (OWA) and Matomo make CAWAL stand out not only as a tool that provides accurate and detailed information but also as a high-performance and scalable solution. The analytical findings of the framework have provided in-depth insights into user behavior and web portal interactions. Analyses of user types, demographics, interaction times, and preferred content have provided valuable insights into optimizing the web portal's design and functionality according to user needs and preferences. Revealing behavioral differences between various user groups, which parts and services of the portal attract more attention, and how users use the portal provides crucial strategic information to improve the user experience of the web portal. Furthermore, CAWAL's superior performance is characterized by user tracking, session management, and data accuracy. Thanks to the model's detailed data analysis and application log integration, software bugs and performance issues were detected more effectively, which contributed to improving the overall quality of web services and user satisfaction. In the future work section of this dissertation, further research on the applicability and impact of the CAWAL model is recommended. Future work is expected to examine how the model can be applied to various user interaction scenarios in different web portals and further develop the model's data collection and analysis capabilities. Studies can be conducted on how the model can be applied in different industries and sectors, for example, e-commerce, healthcare, and public services. Such work will expand the model's potential and provide a broader range of applications in web analytics and web usage mining. In addition, detailed studies of the model's current limitations and challenges would improve its applicability and effectiveness. For example, research on the model's scalability, flexibility, and adaptability to various technological infrastructures can play an essential role in its adaptation to a wide range of applications. These studies could provide practical solutions to address technical and operational challenges. In addition, studies on the long-term effects and sustainability of the model are also critical. Further model development will enable it to rapidly adapt to technological innovations and changing user needs, making it a practical and applicable solution in web analytics and usage mining.