Blind vs Open fire modelling

I always wanted to start a debate on this topic and now think that a better way is using the blog.

Three years after The Dalmarnock Fire Tests, the 'a priori' vs. 'a posteriori' debate in is still not too popular in the fire modelling community. The debate seems to be mostly taking place in personal communications and during the peer reviewing of papers. Unfortunately, not much is happening publicly or at the reach of the fire community as a whole.

Figure 1: Dalmarnock Fire Test One as seen from the outside, 18.5 min into the fire. (from [ Abecassis-Empis et al, Experimental Thermal and Fluid Science 2008]).

The problem is the following (summary). When making comparisons of modelling results to experimental measurements, there are two general approaches that can be followed: a priori (aka blind) and a posteriori (aka open). In a priori simulations, the modeller knows only a description of the initial scenario. The modeller has no access to the experimental measurements of the event and thus will be providing a true forecast of the quantities of interest. In a posteriori simulations, before the simulation is run the modeller knows the initial scenario and also how the fire developed (ie via the experimental measurements). Most fire model validations in fire engineering has been conducted a posteriori.

Only comparison of a priori and a posteriori simulations of the same event allows to investigate the possible effect that maybe has been introduced by prior knowledge of how the event developed. The importance of this effect in fire safety engineering is currently an advanced research topic and under study by different research groups.

The 2006 Dalmarnock Fire Tests conducted in a high-rise building were used to look into the problem. An international study of fire modelling was conducted prior to Dalmarnock Fire Test One. The philosophy behind the tests was to provide measurements in a realistic fire scenario with very high instrumentation density (more than 450 sensors were installed in a 3.50 by 4.75 2.45 m compartment). Each of the seven participating teams independently simulated the test scenario a priori using a common detailed description. Comparison of the modelling results shows a large scatter and considerable disparity among the predictions and between predictions and experimental measurements. These results tend to shock, please and anger the audience in equal parts. See Figure 2 below.

Figure 2: Evolution of the global heat release rate within the compartment. Comparison of predictions and experimental measurements. (from [Rein et al, Fire Safety Journal 2009]).

An exception to the relative silence of the fire community are the two magazine articles of Dr Alan Beard from Heriot-Watt University. These can be accessed here:

• Reliability of computer models in fire safety design, April 2008 in the magazine Industrial Fire Journal.
• Role Models, Aug 2009 in the magazine Fire Risk Management

First note that I disagree with blanket statements like "a predicted result from a model cannot be assumed to be accurate; ie to reflect the real world". Our work also shows that fire simulations provides fire features that may be good enough to be applied towards engineering problems if a robust and conservative methodology is defined. A prerequisite for this methodology is that it can use predictions with crude levels of accuracy and that it applies appropriate safety factors. But Dr Beard has an important point in that 'real world' fire engineering applications are most frequently applied to simulate events which real behaviour had not been (and will never be) measured. These simulations are a priori simulation, not a posteriori. However, most fire model validations in fire engineering has been conducted a posteriori. I certainly agree with Dr Beard on this one; we need more a priori comparisons of fire modelling and address full model validation. What is the effect of prior knowledge of the fire development? Would the validations provide the same conclusions if conducted a priori? The problem is not unique to fire engineering and any discipline dealing with complex simulations tools should be facing this question. I do not know how other disciplines cope with it.

The differences between a priori and a posteriori modelling become patent when comparing the round-robin results with the work conducted after the Dalmarnock data was publicly disseminated. Subsequent studies (Jahn et al. 2007, Jahn et al. 2008 and Lazaro et al. 2008) show that it is possible to conduct a posteriori fire simulations that reproduce the general fire behaviour to a satisfactory level. This was achieved due to the availability experimental data of the real behaviour for reference, allowing for iterations until an adequate input file was found.

I would like to finish with the same final words I use when presenting the results in conferences and seminars. We, the authors of the Dalmarnock round-robin, are professionals of, and supporters of, fire modelling. We want fire modelling to improve and be developed further. Our daily work goes in that direction.


I am interested in hearing your comments.
.Guillermo Rein.




NOTE #1: All the relevant information, book and papers about The Dalmarnock Fire Tests are accessible in open access here.

NOTE #2: There are two points about Dalmarnock that need to be emphasised since are often misunderstood. These are:

• The aim of our a priori work was to forecast the test results as accurately as possible, and not to conduct an engineering design with adequate conservative assumptions or safety factors.
• Experimental variability was one of our greatest concerns and that is why the scenario was designed for maximum test reproducibility. The Dalmarnock Fire Test One was benchmarked against a second test to establish the potential experimental variability. Results show that the scatter of the a priori simulations is much larger than the experimental error and the experimental variability together.

NOTE#3: No matter how useful and interesting the results from blind simulations are, only three blind round-robins on fire modelling can be found in the historical records of the discipline. The other two are the 1999 CIB and the 2008 French PROMESIS project. All three round-robins overwhelming agree on the results, but the Dalmarnock one was the first to be publicly communicated and the one providing, by far, the largest instrumentation density.

NOTE #4: I initiated a related discussion on this topic in April 2008 in the FDS forum. See here.