ReeM: Hierarchical Ensemble RL for Multi-Context Building HVAC Control

Design Solution · HVAC & Energy

Design Solution · Dream about it

Hierarchical RL ensemble for adaptive HVAC control across heterogeneous buildings without model retraining.

ReeM deploys a two-tier reinforcement learning architecture—a high-level agent selects from 104 thermodynamic models (symbolic regression and multilinear regression trained across 65 rooms), while a low-level agent weights them via Temporal Convolutional Networks with cross-attention. It addresses the critical AEC deployment barrier: ML-based HVAC controllers typically require expensive per-building recalibration. ReeM aims to generalize across diverse thermal contexts by learning to choose and blend pre-trained models dynamically rather than retraining from scratch.

ReeM proposes a two-tier reinforcement learning architecture for building HVAC control: a high-level agent selects from a library of 104 pre-trained thermodynamic models (symbolic regression and multilinear regression trained across 65 rooms), while a low-level TCN agent with cross-attention weights their combination in real time — the goal being to generalise across diverse building contexts without costly per-building recalibration. The architectural idea addresses a genuine barrier to deploying ML-based HVAC optimisation at portfolio scale, where the retraining burden has been a practical obstacle. The evidence base is extremely thin: a single research building at Osaka University, four months of historical data, and a live heating trial covering two rooms over four days in February 2025 — after which the generalisation claims are made. There is no multi-building validation, no testing across cooling or shoulder seasons, no comparison against simpler baselines (standard MPC, single neural networks, straightforward transfer learning) on the same testbed to justify the architectural complexity, and no safety guarantees preventing the RL agents from optimising energy at the expense of occupant comfort or equipment protection. The computational feasibility of running two RL agents plus TCN inference plus a 104-model library at embedded controller timesteps is unquantified. This remains a research paper, not a deployable control product; the concept is worth watching in the context of portfolio-scale building energy management, but the gap between a four-day winter trial and a warranted multi-building deployment is substantial.

Strengths

• Eliminates costly per-building model recalibration by leveraging a library of pre-trained thermodynamic models and learning to select/weight them dynamically.
• Handles thermal heterogeneity (65 rooms with different exposures, mass, HVAC topology) through ensemble selection rather than one-size-fits-all tuning.
• Combines symbolic regression (interpretable) with neural weighting (adaptive), bridging explainability and learning capacity.
• Potential to scale across building portfolios without returning to domain experts for each new site.
• Temporal Convolutional Networks with cross-attention allow context-aware model selection at each control timestep.

Considerations

• Requires upfront generation and validation of a large library of thermodynamic models (104 in this case); building this library itself is costly and domain-heavy. (moderate)
• High-level agent still needs training data from target building(s); transferability claims rest on whether pre-trained models + ensemble selection reduce data requirements, not eliminate it. (moderate)
• Adds complexity in deployment and maintenance (managing 104 candidate models, two-tier RL agents, TCN inference); operational burden vs. simpler rule-based or single-model MPC. (moderate)
• Ensemble selection can mask failure modes: if all library models are systematically biased for a building type not represented in training, the ensemble will amplify that bias. (moderate)

Risks

• Evidence is extremely limited: one research building (Osaka University), 4 months historical + only 4 days live heating trial in February 2025, two rooms tested. No multi-building validation; generalization claims are not yet evidenced. (critical)
• No comparison to simpler baselines (standard MPC, single neural network, transfer learning without ensemble) on same testbed; unclear whether complexity is justified. (high)
• Temporal Convolutional Networks may struggle with irregular occupancy, setpoint changes, or equipment faults not represented in training; robustness to out-of-distribution events unknown. (high)
• No discussion of how the approach handles buildings with fundamentally different HVAC architectures (e.g., VRF vs. hydronic vs. radiant); library composition and portability assumptions unclear. (high)
• Live trial was heating-only in winter over 4 days; no evidence for cooling, shoulder seasons, or long-term stability; seasonal drift and model degradation not addressed. (high)
• Computational cost of running two RL agents + TCN inference + 104-model library at each control step not quantified; feasibility for real-time embedded HVAC controllers unknown. (moderate)
• No safety guarantees or constraints (e.g., minimum occupant comfort, equipment protection); RL agents may optimize energy at expense of thermal comfort or system wear. (moderate)

Performance

• Temperature prediction MAE improvement vs single-model baseline (15-min): 44.54%
• Temperature prediction improvement vs baseline ensemble methods: 51.65%
• On-site live trial scope: 2 rooms, 4 days (Feb 2025)
• Building dataset size: 65 rooms, 1 building, Osaka University

Reality check

Method is technically sound and real (arXiv v2 October 2025 confirms active work). Claims of 44–51% improvement are supported by controlled experiments within one institution. However: (1) all validation is in-house; (2) live trial spans only 4 days on 2 rooms; (3) no independent third-party deployment or cross-building validation; (4) paper is preprint with no confirmed peer review status; (5) scalability and generalization to other climates, building types, or HVAC configurations remain unproven; (6) no mention of failure modes, robustness to sensor drift, or edge-case performance.

#reinforcement learning #HVAC control #model predictive control #ensemble methods #multi-building generalization #thermal dynamics #adaptive control

Source