Günümüzde binaların enerji tüketiminde önemli bir paya sahip olan Isıtma, Soğutma ve Havalandırma (HVAC) sistemlerinin optimizasyonu, enerji verimliliği ve kullanıcı konforunu artırmak için kritik öneme sahiptir. Bu çalışmada, iç ortam sıcaklık kontrolünün optimizasyonu amacıyla makine öğrenimi tabanlı bir kontrol algoritması olan Q-learning'in performansı, geleneksel Açık-Kapalı ve PID kontrol yöntemleriyle karşılaştırmalı olarak incelenmiştir. Bu amaç doğrultusunda, dış hava sıcaklığı, güneş radyasyonu, iç ısı kazanımları ve bina yapısının termal özellikleri gibi çeşitli faktörleri dikkate alan bir simülatör yazılımı Python programlama dili kullanılarak geliştirilmiştir. Simülatörde, odanın termal dinamiği tek bölgeli bir model olarak kabul edilmiş ve 0.1 saniyelik zaman adımlarıyla 60 dakika ve 24 saatlik simülasyonlar gerçekleştirilmiştir. Farklı senaryolar altında, başlangıç oda sıcaklığı, termostat hassasiyeti, ısı kayıp oranı ve Q-learning eğitim bölümleri gibi parametreler değiştirilerek üç kontrol algoritmasının performansı değerlendirilmiştir. Performans karşılaştırmaları, enerji tüketimi (dolaylı olarak aşım ve alt geçiş süreleriyle değerlendirilmiştir), konfor seviyesi (aşım ve alt geçiş) ve iç ortam hava kalitesi gibi metrikler üzerinden gerçekleştirilmiştir. Simülasyon sonuçları, Q-learning algoritmasının, özellikle uzun süreli simülasyonlarda (24 saat) ve değişken koşullar altında (ısı kaybı oranının yüksek olduğu durumlar), geleneksel yöntemlere göre daha iyi performans gösterdiğini ortaya koymuştur. Q-learning, daha düşük aşım ve alt geçiş değerlerine sahip olarak daha iyi bir konfor seviyesi ve daha düşük enerji tüketimi sağlamıştır. Bu bulgular, Q-learning'in iç ortam sıcaklık kontrol sistemlerinin optimizasyonunda kullanılmasının, enerji verimliliğini artırma ve kullanıcı konforunu iyileştirme potansiyelini doğrulamaktadır. Tez çalışmasının özgün katkıları arasında, iç ortam sıcaklık kontrolü için Q-learning algoritmasının eğitimi ve optimizasyonu, farklı senaryolarda iç ortam sıcaklık kontrolünü simüle edebilen bir yazılımın geliştirilmesi, Q-learning, Açık-Kapalı ve PID kontrol yöntemlerinin performanslarının karşılaştırılması ve kullanılan Q-learning algoritmasının ve simülatör yazılımının açık kaynak kodlu olarak sunulması yer almaktadır. Elde edilen sonuçlar, Q-learning gibi makine öğrenimi tabanlı kontrol algoritmalarının, HVAC sistemlerinin optimizasyonunda kullanılmasının, enerji verimliliğini artırma ve kullanıcı konforunu iyileştirme potansiyelini göstermiştir. Bu çalışma, gelecekteki akıllı bina kontrol sistemlerinin geliştirilmesine ve daha sürdürülebilir bir yapı stokunun oluşturulmasına katkı sağlayabilir.
In the face of escalating global energy demands and the imperative for sustainable practices, the optimization of Heating, Ventilation, and Air Conditioning (HVAC) systems in buildings has emerged as a critical area of research. HVAC systems, while indispensable for maintaining suitable indoor environments, account for a substantial proportion of a building's energy consumption. The limitations of traditional control methods, often reliant on fixed setpoints and schedules, in adapting to dynamic environmental conditions and occupant preferences, have fueled the exploration of intelligent and adaptive control strategies. The integration of machine learning algorithms, particularly reinforcement learning, into HVAC control systems presents a promising avenue to address these challenges, paving the way for enhanced energy efficiency and personalized comfort experiences. This thesis delves into the efficacy of Q-learning, a model-free reinforcement learning algorithm, in optimizing indoor temperature control systems. The central objective is to conduct a comparative analysis of Q-learning's performance against conventional control methods, namely On-off and Proportional-Integral-Derivative (PID) control. To facilitate this investigation, a simulator software has been meticulously developed in Python. This simulator incorporates a simplified room model that accounts for various factors influencing indoor temperatures, including external weather conditions, solar radiation, internal heat gains, and heat loss through the building envelope. The simulator's capabilities extend to simulating indoor temperature control under diverse scenarios, enabling a comprehensive evaluation of the impact of different control strategies on energy consumption, comfort levels (overshoot and undershoot), and indoor air quality. The simulator leverages the Streamlit library to provide a user-friendly interface, allowing for interactive visualization and exploration of simulation results. The simulator's design emphasizes flexibility, allowing users to manipulate various parameters such as initial room temperature, thermostat setpoint, heater power, heat loss rate, simulation duration, thermostat sensitivity, Q-learning training episodes, learning rate, discount factor, exploration rate, and PID parameters. This flexibility enables a thorough investigation of the control algorithms' performance under a wide range of operating conditions and scenarios. The simulation results unequivocally demonstrate the superior performance of Q-learning, particularly in extended simulations and under dynamic conditions. Let's consider the scenario of a 1440-minute simulation with varying external temperatures and heat loss. In this scenario, Q-learning showcased its adaptive prowess by achieving a remarkably low total area of 13.41 °C-minutes, signifying minimal overshoot and undershoot, which directly translates to improved occupant comfort and reduced energy wastage. In stark contrast, the On-off control exhibited a total area of 368.00 °C-minutes, highlighting its susceptibility to temperature fluctuations and energy wastage due to its inherent binary control mechanism. The PID control, while performing better than On-off, still lagged behind Q-learning with a total area of 12.68 °C-minutes, indicating its limitations in handling complex and dynamic environments. This compelling evidence underscores Q-learning's ability to learn and adapt to the environment, leading to superior temperature regulation, enhanced occupant comfort, and reduced energy consumption. Furthermore, the Q-learning algorithm's performance was robust across different scenarios, showcasing its adaptability to varying initial room temperatures, thermostat sensitivities, and heat loss rates. The algorithm's ability to learn optimal control policies through trial and error, guided by a carefully designed reward function, enabled it to consistently outperform traditional methods. This adaptability is particularly crucial in real-world building environments where conditions are constantly changing due to factors such as occupancy, weather, and building usage patterns. The simulator software played a pivotal role in facilitating this comparative analysis, allowing for the systematic evaluation of the control algorithms under a wide range of conditions. The contributions of this thesis are multifaceted. Firstly, it offers a comprehensive implementation and optimization of the Q-learning algorithm tailored for indoor temperature control, encompassing meticulous design and tuning of critical components such as state space, action space, reward function, and hyperparameters. Secondly, it presents a user-friendly simulator software, developed in Python and leveraging the Streamlit library, capable of simulating indoor temperature control under diverse scenarios. This simulator empowers researchers and engineers to evaluate the impact of different control strategies on energy consumption, comfort levels, and indoor air quality, facilitating the development and refinement of innovative control algorithms. The interactive nature of the simulator, enabled by Streamlit, promotes accessibility and encourages exploration of the complex dynamics of indoor temperature control systems. Thirdly, the thesis provides a rigorous comparative analysis of Q-learning, On-off, and PID control methods, elucidating the strengths and weaknesses of each approach under various scenarios, such as varying initial room temperatures, thermostat sensitivities, and heat loss rates. This analysis offers valuable insights into the performance trade-offs associated with different control strategies and guides the selection of the most suitable algorithm for specific building environments and operational requirements. Finally, the open-source nature of the Q-learning algorithm and simulator software fosters reproducibility and encourages further research and development in this field, promoting collaboration and knowledge sharing within the research community. Literature Review The literature review presented in this thesis provides a comprehensive overview of the current state-of-the-art in the field of indoor temperature control, energy efficiency, and machine learning. It highlights the growing interest in leveraging machine learning techniques to optimize HVAC systems and achieve a balance between energy efficiency and occupant comfort. The review explores the potential of various machine learning models, including deep reinforcement learning and artificial neural networks, in predicting indoor temperatures and optimizing HVAC control strategies. It also underscores the importance of simulation software in evaluating the performance of these models and control algorithms under different scenarios. Additionally, the literature review emphasizes the significance of personalized thermal comfort and its potential to further enhance energy efficiency and occupant satisfaction. Findings of this Study The findings of this study illuminate the potential of machine learning-based control algorithms, notably Q-learning, in revolutionizing HVAC systems for improved energy efficiency and occupant comfort. The developed simulator, with its user-friendly interface and ability to simulate diverse scenarios, serves as a valuable tool for researchers and engineers to explore and evaluate innovative control strategies. The comparative analysis provides compelling evidence of Q-learning's superiority over traditional methods, particularly in dynamic and long-duration scenarios. Future Research Directions Future research can further explore the potential of Q-learning and other advanced machine learning models in real-world building environments, paving the way for a new generation of intelligent and energy-efficient HVAC control systems. Further exploration of personalized thermal comfort models and the integration of Q-learning with other building control systems are also promising avenues for future research. The insights gained from this thesis can contribute to the development of more sustainable and comfortable buildings, aligning with the global goals of energy conservation and environmental protection.
Except where otherwise noted, this item's license is described as http://creativecommons.org/licenses/by/4.0/
