Minutes of the 4th FROST-2014 meeting (29-31 October, Moscow)
The meeting was opened at 10:00 on 29 October 2014 at the Best Western Vega hotel, Moscow, Russia by D.Kiktev. He welcomed the participants and thanked them for their contributions during the project field phase. D.Kiktev introduced the meeting agenda that was adopted.
A.Baklanov welcomed the participants on behalf of the WMO and presented overview on the WWRP activities and projects.
D.Kiktev presented a review of the project Olympic field campaign and realization of the project goals for various project components – observations, nowcasting, deterministic forecasting, ensemble forecasting and project impacts. Further steps to follow up the project goals were also outlined.
P.Nurmi asked whether the official forecasts could be made available for potential impact assessments. D.Kiktev answered that this was possible. V.Lukyanov informed the participants that report on the meteorological support of the Olympics had been prepared in Russian and in English (available on the FROST-2014 site). A.Baklanov suggested to prepare short post-Olympic recommendations on the basis of the experience gained.
E.Astakhova presented on the FROST-2014 observation and forecast archive and discussed some data issues. Observation part of the archive includes the data of ~40 automatic meteorological stations in the Sochi region, 5 radars including 3 dual-pol Dopplers, 2 micro rain vertical pointing radars, wind profiler, 2 temperature profilers, 7 web-cameras, high-resolution upper sounding data in Sochi, special snow measurements and some extra SYNOP data that are usually not available via GTS. Forecasts of 8 deterministic models with resolutions from 7 km to 250 m, 6 ensemble prediction systems with resolutions from 11 to 2.2 km, and 6 nowcasting systems were available for the period of the Olympic and Paralympic Games 2014. It was mentioned that there were some problems with different coding of forecast data and thus it was suggested to prepare a unified archive of available forecast data following the TIGGE-LAM standards.
P.Nurmi strongly supported the idea of the unified archive and stressed that the FMI verification system needs all data to be downloaded to their computer. A.Baklanov suggested adding air quality data to the FROST-2014 archive; the suggestion will be further discussed later. M.Tsyrulnikov mentioned that spatial quality control for the region with such a complex terrain was possible only if enough observations were available. D.Kiktev explained that spatial control was performed only for observation times 0, 3, 6, 9,…21 UTC using SYNOP data for the surrounding area while temporal control and control versus forecast data were implied to more frequent observations.
A.Melnichuk presented the radar setup for the Sochi Olympics.
P.Steiner asked if the location of the Akhun radar was still considered to be the best option. V.Lukyanov explained that the radar location was optimal for forecasters because the precipitation weather systems usually came from the west. The radar provided forecasters with useful information on upstream precipitation from the sea. Initially, two radars were planned, one at the Akhun mountain and another in the mountain cluster, but installation of the radar in the vicinity of the Olympic events was not permitted. B.Bica mentioned that the position was nice from the synoptic point of view, but there were some delays in the radar data flow from Sochi to ZAMG.
A.Koldaev presented setup and results of operation of the mobile system for atmospheric vertical remote sensing during the Olympic and Paralympics Games «Sochi-2014». The mobile platform was equipped with RPG temperature profiler, Scintec-3000 wind profiler and micro rain radar MRR-2.
M.Tsyrulnikov asked about the data accuracy and vertical resolution of the temperature profiler. Depending on complexity of the profiles A.Koldaev estimated it from 0.5° (for smooth profiles) to 1.5°. He said that the comparisons with independent observations largely confirmed the characteristics given by the manufacturers. It was also mentioned that the use of this type of data requires some preliminary training and such a training was organized for the Olympic forecasters. V.Lukyanov noted that only during the Olympics the forecasters were able to use the data, while it were not completely understood during the trials. V.Lukyanov advised the Korean colleagues to be quick in introducing new observations and forecast technologies as forecasters need time to get used to new products.
M.Tsyrulnikov interested whether sensitivity of RPG micro-wave observations to rain can be used as a kind of precipitation observation. A.Koldaev answered that yes and no as there are several factors that should be taken in consideration (in particular, precipitation phase). V.Lukyanov doubted that such observations would be reliable enough, but according to his experience RPG humidity observations could serve as a kind of indicator of approaching precipitation (several hours in advance).
A.Melnichuk presented the talk by J.Reid et al on radar particle classification (PCA) results. Differences between the Environment Canada’s PCA algorithm and Vaisala’s HCA algorithm were noticeable.
M.Tsyrulnikov asked which classification worked better. In A.Melnichuk’s opinion it was PCA.
Discussion on observations and project data archive
P.Nurmi asked whether any reforecasting was expected and noted that reforecasting could introduce problems for the verification team. He stressed the need in the «final» data set of observations and forecasts. D.Kiktev answered that changes are possible for the RDP forecasts, but for the FDP data freezing of the archive is implied by FDP requirements. He also said that manual control is applied to the observations along with objective quality control. He did not expect substantial changes in the observation part of archive, but there might be some changes in the quality control flags in near future as the manual control of these data was still continued. I.-L.Frogner informed the participants that the Norwegian Meteorological Institute plans to add some new extra forecasts to the FROST-2014 archive and increase the ensemble size. A.Montani asked if there were any plans to create a gridded precipitation analysis. B.Bica commented on that INCA grid-point analysis could be used, but its quality could not be guaranteed. A.Bundel mentioned that the gridded precipitation analysis was needed for spatial verification. D.Kiktev answered that there were no plans to prepare a blended precipitation analysis and probably the radar reflectivity fields could help in spatial verifications. He noted the problem of representativeness of in situ precipitation measurements for such a complex terrain.
B.Bica mentioned that in some cases at the time of analysis both in-situ and radar data were absent or arrived late and sometimes there was a different amount of observation data at the project web-site and in INCA analysis. This effected quality of some INCA nowcasts.
P.Steiner was interested if satellite data were included to the project archive. It was answered that this information was not archived.
A.Montani advised not to include much information to the unified archive. He stressed that the unified archive should contain a limited number of parameters and it is important to prepare it quickly.
A.Baklanov said that the FROST-2014 data should serve as a testbed for future and it would be good to follow TIGGE requirements for the archive and keep it open to new model runs and evaluations. Then this would be a valuable outcome of the project.
The afternoon session was devoted to the experience of forecasting for «Sochi-2014» Olympics
E.Vasilyev presented a forecasters’s view on the forecasts/nowcasts performances during the Olympics. A string of interesting weather cases was identified for further analysis. Challenges of mountain winter weather forecasting for the Olympic region were considered.
P.Nurmi commented on the mentioned forecast challenges of precipitation timing that moments of precipitation start and end can be a subject of verification.
G.Rivin noted that COSMO forecasts with 1 km resolution were not presented in the comparisons. E.Vasilyev explained that the COSMO model with 2 km and 7 km grid spacing got more attention as the 1-km COSMO version became available only shortly before the Olympics (in January 2014).
B.Bica presented the talk “INCA and ALADIN-LAEF at the Sochi 2014 Olympics“. The technical setup of both the INCA analysis and nowcasting system and the ALADIN-LAEF ensemble forecasting system were reviewed. The timely availability of observations and radar data proved to be crucial for the quality of the generated analysis and Nowcasting fields. It was shown that the overall data availability generally improved during the preparation phase of the FROST-project, and that it was satisfactory for most parameters. However there were also some delays in the data flow of precipitation observations and radar data which – along with the naturally inhomogeneous station distribution in the selected domain – diminished the quality of the analysis.
The ALADIN LAEF ensemble members generally captured the weather phenomena well, except for some effects that are due to the complex orography in the Sochi area. Calibration of the ensemble would help to reduce bias at point forecasts or grid points, respectively.
P.Steiner presented the general overview of the COSMO consortium contribution to the FROST-2014 project that was to be specified in the subsequent COSMO presentations.
In the presentation by M.Shatunova «Severe weather forecasting for mountain region by COSMO-Ru system» (M.Shatunova, G.Rivin, I.Rozinkina, D.Blinov, Hydrometcenter of Russia) the problem of visibility forecasting was considered. Two case studies of low visibility caused by fog (February, 16-17) and heavy snowfall (February, 18) during Sochi Olympic Games were examined. Comparison of the forecasts (of humidity and precipitation rate) obtained from the COSMO-Ru models with different horizontal resolutions (7, 2.2 and 1.1 km) was presented. The problem of accurate forecasting of a weather event’s start and end time, the time of the maximum is addressed.
A.Lukyanov noted that for the purposes of fog and cloudiness forecasting, the vertical resolution of the COSMO model was quite coarse. Model grid points were located at different heights with respect to the corresponding stations. E.g., for the Biathlon Stadium this vertical difference was ~90m that is quite a lot with respect to the natural spatial variability of cloud ceiling (roughly ~30m). It was noted that some height adjustment of the model data to the station levels was needed for such a case. M.Shatunova answered that for the Biathlon Stadium there were 1 model grid point and 4 neighbourhood stations. The lowest station was used in that case.
Discussion on strong and weak points in forecasts and nowcasts in FROST-2014
The Chair of the session B.Bica initiated discussion on good and bad «surprises» in FROST-2014 forecasts/nowcasts.
D.Kiktev noted that strong and weak points of the project forecasting could be considered in multiple ways: e.g., it might be weather elements that were predicted relatively well or poorly; general experimental setup; data assimilation aspects; problems of information environment and data logistics etc. He noted that majority of the project NWP systems did not assimilate the local observations in operational mode and this might be considered as a kind of weak point. D.Kiktev also added that the project archive gives a good basis for assessment of performance and intercomparison of the state of the art forecasting systems. However, all project participants used their own global driving models while for diagnostic studies and comparison of high resolution limited area models it would be preferable to use common driving system and unified boundary and initial conditions.
A.Montani indicated that the worst predicated fields were: wind direction, wind gusts, visibility and cloud height.
For G.Rivin good surprise in the project high-resolution forecasts was their reasonable quality despite the complex terrain, lack of data and other limitations. A.Montani commented that the local radar data should be assimilated in COSMO-RU system to assess the further related potential.
In A.Montani’s view the bad surprise was the data formats issue in the project archive. P.Nurmi was keen on unification of the archive data formats. D.Kiktev mentioned that for local developers the variety of input formats was also a serious issue and they were short of time to fully unify this information collected from various providers. He asked the participants to inform him on the bugs detected in the project data archive. This information is important for the preparation of the «final» unified data set (tentatively by March 2014).
I-L.Frogner said that high efficiency of GLAMEPS forecasts calibration was a good surprise to her. Despite the relatively coarse resolution the performance of these calibrated forecasts was very competitive. She adviced this cost-efficient solution to the Korean colleagues.
For D.Kiktev performance and informativity of CARDS radar tracking nowcasting in the mountaneous area of the Olympics was also a good surprise. This information and the way of its presentation were appreciated by the Olympic forecasters.
A.Baklanov noted that further advances in such areas as transition between nowcasing and NWP or assimilation of non-synoptic observations on the basis of Sochi testbed could be potential project outcomes.
Day 2
The first part of Day 2 was devoted to the experience of forecasting for «Sochi-2014» Olympics.
I-L. Frogner and T. Nipen presented the HIRLAM contribution to the FROST-2014 project.
The HIRLAM contribution to FROST-2014 consisted of three different parts; two probabilistic systems on different scales and calibration of the coarser system. The pan-European ensemble prediction system GLAMEPS at 11 km resolution was provided as FDP. A regional ensemble prediction system HarmonEPS with 2.5 km resolution was set up and run specially for the Sochi Olympics. It provided forecasts in real time as RDP. Calibration of GLAMEPS with hourly updates was also provided in RDP mode. As a baseline to compare the systems against, the ensemble system from ECMWF (IFS ENS) was used. Both raw GLAMEPS and HarmonEPS scored perfomed well comparing to IFS ENS. HarmonEPS also had equal or larger spread than IFS ENS with its 13 members compared with 51 members from IFS ENS. Calibration greatly improved the scores for GLAMEPS. Various calibration methods were used to improve the quality of the GLAMEPS forecasts at ~40 point locations. It was found that bias-correcting each ensemble member greatly reduced the average errors of the forecasts, which were caused by the coarse grid spacing of the models (11 km). It was also found that calibrating the spread of the ensemble generally gave reliable probabilities for various thresholds. Hourly update of the temperature forecasts using the real-time observations gave further improvements. The FROST-2014 project was promoted by convincing http://yr.no management to show HIRLAM Olympic forecasts on the website.
In the further discussion the important effect of calibration was noted. It was also noted that at least for 13-14.03.2014 precipitation data from Roza-Khutor-1 station should be rechecked. M.Tsyrulnikov commented that in the calibration technique, the Gaussian model for wind speed might not be an appropriate choice. He also wondered why the time scales for adaptation of temperature is 5 days, whether that for precipitation as large as 30 days. The answer was that precipitation events can be too rare to collect sufficient statistics during short training periods.
A.Montani presented the main results obtained by the limited-area ensemble system COSMO-S14-EPS with 7km grid spacing developed to assist local forecasters in the probabilistic prediction of localised high-impact weather events. As an example of the ability of COSMO-S14-EPS, the skill of the system was shown for a case of strong wind occurred in the mountain cluster during the Olympics. For this case, the forecasts provided by COSMO-S14-EPS turned out to be very valuable. The performance of COSMO-S14-EPS was also intercompared to that of ALADIN-LAEF (the ensemble system developed and implemented at ZAMG) over the period 15 January to 15 March 2014. COSMO-S14-EPS was shown to provide slightly better scores in terms of probabilistic prediction of precipitation over the Olympic area. The super-ensemble generated by joining together COSMO-S14-EPS and ALADIN-LAEF clearly outperformed both systems, suggesting the potential of a multi-model approach to the predictability problem, especially in cases of complex orography.
For the FROST-2014 archive A.Montani suggested to use the experience obtained by previous similar exercises and to follow, whenever possible, an approach similar to that of TIGGE-LAM archive at ECMWF.
P.Nurmi commented that verification of all project EPS forecasts is needed (not only COSMO-S14-EPS and LAEF).
I.Rozinkina presented a string of technological enhancements of the COSMO-based forecasting system for the Olympic Sochi testbed: continuous nudjing-based data assimilation using the model version with 2.2 km grid spacing, orographic adjustment for COSMO-RU temperature forecasts, postprocessing for fresh snow depth prediction etc. Positive effect of the data assimilation was demonstrated first of all for the first forecast hours.
G. Isaac presented «Visibility Forecasting for Sochi from the Canadian Perspective» via Web. Environment Canada produced forecasts of visibility for FROST-2014 mountain sites using the GEM (HRDPS) 250m resolution model and the INTW Nowcast system. Forecasts of visibility reductions in fog depend on accurate predictions of relative humidity. Forecasts of visibility in snow need accurate precipitation rate predictions. In contrast with SNOW-V10 most of the visibility reductions in the mountain cluster of Sochi Olympic venues appeared to come from fog and the poor model relative humidity (RH) predictions, especially between 97% and 100%, degraded the skill of visibility forecasts. The visibility reductions in snow appeared to be forecast better than those in fog. Problems with the RH measurements were also noted. More work needs to be done to verify these conclusions. G.Isaac supposed that direct use of model predicted drop or particle size and cloud liquid or ice water content for visibility forecasting was probably the method of the future.
M.Tsyrulnikov wondered if improvement of observations and data assimilation systems could help in predicting visibility (or it is only the models who are to blame). G.Isaac thought that this could have helped, especially assimilation of surface data.
None of other Canadian colleagues was able to physically join the meeting that time, but they provided their presentations (see http://frost2014.meteoinfo.ru/presentations):
- P.Joe and J.Reid: «Preliminary Investigations into Orographic Nowcasting»;
- A.Glazer, J.Milbrandt, S.Bélair, M.Faucher, S.Leroyer: «Environment Canada’s High Resolution NWP System for FROST-2014».
This information was also introduced by M.Tsyrulnikov and met with deep interest.
The second session of the Day 2 was devoted to experience of Olympic forecasting, verification, preliminary assessments, and future plans (in particular, for the next winter Olympics in the Republic of Korea) and started with presentation by P.Nurmi, S.Näsman, and M.Brockmannon on Sochi forecast verification framework at FMI. P.Nurmi also proposed to organize a special FROST-2014 verification session at the next EMS meeting in Sofia in 2015 and provided the participants with access to the FMI web-based verification package «tuned» for the Sochi region and the FROST-2014 forecasting systems. Along with forecast verifications for deterministic and ensemble forecasts, some new impacts and end-users were identified on example of the Finnish Olympic skiing service team. The issue of freezing the project observation and forecast data for the fair «final» verification was addressed.
In the subsequent discussion E.Astakhova wondered why only HIRLAM ensemble systems were verified though forecasts from other project EPSs were available at the FROST-2014 server. She also mentioned that missed at the FROST-2014 server COSMO-Ru2-EPS and COSMO-Ru1 forecasts would be added to the project server shortly (already done). P.Nurmi explained it by the issue of different coding in the data from various EPSs.
A.Montani warned the participants on the overlap in the dates of the next COSMO General Meeting and the EMS-2015 Conference in Sofia. It can draw some potential COSMO participants away from the EMS meeting. It was also agreed that including the FROST-2014 presentations into other EMS-2015 sessions is preferable to the special FROST-2014 session.
A.Bundel presented some preliminary results of verification activities for the Sochi-2014 Olympic Games at the Hydrometcenter of Russia. Her contribution was mostly devoted to verification of deterministic forecasts. Deterministic Sochi-2014 forecast systems (models COSMO-Ru7, COSMO-Ru2, COSMO-Ru1, GEM2.5km, GEM1km, GEM250m, HARMONIE, NMMB + INCA and JOINT systems) were verified using mainly Mean Error and Mean Absolute Error for T2m, dew point temperature, RH, wind speed, wind direction, and wind gusts. Hanssen-Kuipers scores were presented for 3h precipitation accumulations. Evaluation period was 15 January-15 March 2014. It was demonstrated that all the models underestimated temperature during the evaluation period, in particular, during the daytime, except for HARMONIE up to about 1400 m. JOINT forecasts quite efficiently corrected bias. For COSMO, effect of resolution enhancement was especially positive for wind speed and gusts. For GEM model effect of resolution increase was overall positive for T2m, wind direction, and RH, especially for high mountain stations. Convection-resolving GEM-250m model added value is not evident from traditional verification (although confirmed by forecasters) - spatial methods are needed. Several cases of intense precipitation were well forecasted by most models. NMMB seemed to be the most successful system for the middle of precipitation thresholds range (>1-5 mm/3h). Analysis of previous test seasons on the basis of COSMO-RU showed that the model mean errors can change the sign from one cold season to another depending on prevailing weather regimes.
In the further discussion A.Bundel mentioned that in general the presented verification scores were in agreement with the previous results of subjective evaluation by Sochi forecasters.
A.Chaika as a representative of the local Olympic Organizing Committee during the Games presented review of the Sochi-2014 weather services and the Games operations.
The participants were highly interested in the contribution from the country of the next Olympic Games, Korea, given by J-C.Ha. He presented the KMA/NIMR Research and development plan in support of PyeongChang-2018 Winter Olympic Games.
In the subsequent discussion V.Lukyanov noted that not only the absence of snow but also heavy snow could be a problem for competitions as the new snow had to be removed manually. He mentioned that most competitions are held on artificial snow, but for its production the night temperatures should be about -3 degrees or lower.
A.Koldaev interested if any weather modification activities were planned for the PyeongChang Olympics. He noted that Roshydromet had experience in this area and ready to collaborate. It was answered that there were no plans for the weather modifications during the PyeongChang-2018 Games.
D.Kiktev noted that WWRP projects should not duplicate one another and in this respect the proposed focus of the PyeongChang project on snow microphysics and cloud analysis looked unique. In his view the project concept should be inclusive enough to attract more international participants.
Participants interested if any FDP activities were planned for the Korean project. In was answered that for the time being not as operational activities were beyond the scope of NIMR.
Day 3
L.Nikitina presented «Preliminary results of precipitation nowcasting verification for Sochi-2014 Olympic venues». At the current stage the analysis was restricted to a subset of the available FROST-2014 nowcast data including CARDS, INCA and INTW systems. Verifications were presented for the period of the winter Olympic events mostly for 1-hour precipitation totals (the available 10-minutes nowcasting data are to be verified later). Despite some misses and false alarms, CARDS was found to be the most successful system in these preliminary assessments. It was noted that in severe weather cases there is a need in 10-15 min lead time forecasts and nowcasts with high temporal resolution.
A.Muravev focused his presentation at High Impact Weather (HIW) forecast verification for the Sochi-2014 Olympic Games. Forecasting HIW was acknowledged as the main challenge for regional Ensemble Prediction Systems (EPS) in the FROST-2014 project. Verifying forecasts of rare events encounters serious difficulties in part due to degeneration of traditional measures to bound values as events become rarer. The Extremal Dependence Index (EDI) measure and its symmetric form to overcome shortcomings of common measures in HIW forecast verification were especially highlighted in the presentation. EDI was used in the presentation along with traditional ensemble forecast verification scores. Standard EPS verification scores and the EDI as functions of lead-time were applied to precipitation forecasts in the Mountain Cluster, provided by the six project ensemble systems during the Olympics. The EDI values were calculated for three «event trigger thresholds» equaling a definite ensemble event frequency: 50%, 66%, and 90%. A heuristically defined ‘EDI curve integrity’ based on ideas of continuity, smoothness, and values generally above a base rate level, was applied for an overall system evaluation. The larger the threshold at which any of the three EDI-curves did not lose its integrity over the 36-h forecast interval the higher the overall evaluated EPS performance was supposed. The EDI-integrity notion gave an opportunity to preliminary rank all the ensemble systems in their ability to forecast precipitation intensity at high thresholds. Comparisons of COSMO-RU2 vs. NMMB-1km were also presented.
Commenting on the presentation P.Nurmi mentioned the issue of translation of probabilities into actions. He considered the presented results to be relevant and interesting to the WWRP High Impact Weather project.
Discussion on verification, model intercomparison, project outcomes, further steps and anticipated joint papers
In the discussion it was noted that besides the «centralized» verification at the Hydrometcentre of Russia and «external» verification by FMI many participants calculated project forecast verifications on their own. M.Tsyrulnikov commented that this «ensemble of verifications» looked like a kind of chaos and a duplication of efforts. He said that some coordination of these activities was needed.
D.Kiktev considered that such a double check for forecast verifications looks natural and might be useful to ensure confidence in the results and their final convergence. He believed that writing of joint papers could be a right mechanism of self-organization in the further project activities. The papers were considered to be an important part of the project legacy.
A number of potential joint FROST-2014 papers was identified in the discussion. It was agreed that one paper should be prepared by D.Kiktev and all the participants for BAMS:
• «General overview of the FROST-2014 project».
A string of papers has to be prepared for a potential special issue of a relevant international peer reviewed journal. The following is the preliminary list of their titles and authors (the titles and lead authors/authors to be specified in the course of preparation of the publications):
• Remote sensing observations for meteorological support of the winter Olympic Games «Sochi-2014» – Y.&A. Melnichuks, P.Joe, A.Koldaev;
• Surface observational network of automatic meteorological stations for the Sochi Winter Olympics – N.Bocharnikov and CoAuthors;
NB: Should it be two separate observational papers or it is better to combine them into one with In-Situ + Radars, Profilers, MRRs? Something like «Observational campaign for meteorological support of the winter Olympic Games «Sochi-2014»? It seems to me (D.Kiktev) that originally it makes sense to try 2 papers and later on to see how they look separately.
• Experience of nowcasting for the XXII winter Olympic Games – B.Bica and other project nowcasting providers;
• Verification of nowcasts during the Sochi-2014 winter Olympics – L.Nikitina, S.Mostamandy? P.Nurmi?;
NB: Work on the two separate nowcasting papers above or combine them into one?
• Characteristics and predictability of orographic precipitation using Akhun radar – P.Joe and CoAuthors;
• Dual-polarization algorithm for hydrometeor particle classification – P.Joe and CoAuthors;
• The performance of INTW and ABOM during FROST 2014 - G.A. Isaac of Weather Impacts Consulting and scientists from Environment Canada;
• Visibility Forecasting for Sochi from the Canadian Perspective - G.A. Isaac of Weather Impacts Consulting and Environment Canada colleagues;
• Perfomance of the MeteoExpert system during the Sochi Olympics – T.Bazlova and CoAuthors
• Development of COSMO system for meteorological support of «Sochi-2014» Olympics (Data assimilation, resolution, postprocessing etc) – G.Rivin and CoAuthors;
• Perfomance of high-resolution deterministic numerical weather forecasts for the Sochi winter Olympics – A.Bundel ? P.Nurmi ?
• Perfomance of the COSMO system in the low visibility events during the Sochi-2014 Olympics and experiments on direct visibility forecasting – M.Shatunova and CoAuthors
• Inter-comparisons of the FROST-2014 ensemble forecasts, predictability and performance of multi-model ensemble - A.Montani, B.Bica and F.Weidle +… (basing on COSMO-S14-EPS, ALADIN LAEF… anything else?)
NB: Will this paper cover the formal FDP-comparison of EPSs? If so, then GLAMEPS and NMMB-EPS have to be added. If not then the second ensemble verification paper will be needed to address the FDP assessments of the participated EPSs. Will this paper address validation and intercomparison only for the formal 2-months FDP period or EPSs performance in the pre-Olympic period will be also considered?
• Comparison of predictability of various weather phenomena in the Olympic Sochi testbed - I.-L.Frogner and COSMO colleagues (It is planned to assess the ability of coarse-resolution EPS vs. high resolution deterministic forecasting systems);
• Combining multi-system forecasts of the FROST-2014 project for meteorological support of the Olympic Games «Sochi-2014» – D.Kiktev and CoAuthors; (It is planned to present the project multi-model blend with adjustment to the latest observations and its integration into the information system of the Sochi Olympics. Can be combined with another paper on multiple model validation and intercomparison).
• GLAMEPS and HARMON-EPS performance - I-L.Frogner, T.Nippen (including effects of calibration and role of resolution);
• Ensemble prediction systems on the basis of the COSMO model with different resolutions in the Sochi Olympic testbed – E.Astakhova, A.Montani;
• Verification of high-impact weather phenomena – A.Muravev and CoAuthors.
Participants noted that many other potential papers could be also relevant:
- Other papers on case studies;
- Other papers on individual systems;
- Impact assessment paper (together with the WWRP SERA Working Group);
- Verification of precipitation forecasts (using radars and other observations);
- Air quality paper.
International peer reviewed Journals for the potential special issue with the FROST-2014 results
Several options were considered. On behalf of Y.Wang B.Bica suggested considering the possibility of special issue in the Meteorological Zeitschrift.
A.Baklanov suggested an option of the Urban Climate, where he is a Chief Editor. He also mentioned the Boundary Layer Meteorology.
I.-L.Frogner proposed to try submitting the papers to Weather and Forecasting, Tellus or J.Appl.Meteorology as more relevant for forecasting publications.
A.Baklanov asked the participants about the effect they expected from the special issue. In high-ranked or very specialized journals a considerable part of potential papers might be filtered at the stage of reviewing. The trade-off between the number of published papers and the impact factor of the journal was discussed and opinions differed. The final choice of the journal is to be postponed till the papers are prepared.
It was agreed that the middle of 2015 should be an objective for preparation of the papers for the special issue (with submitting the papers in the second half of 2015).
Along with the journal papers a formal final project report to the WMO WWRP Science Steering Committee (SSC) is to be prepared. D.Kiktev informed the participants that it should be a comprehensive document (~100 pages or more) and the papers for the special journal issue were expected to form a basis for the WWRP report. If all the papers are prepared on time there is a chance to present the report to the SSC in 2015.
A.Baklanov commented that a list of lessons learned in FROST-2014 project should be prepared and included into the WMO report and probably in the BAMS paper. This is important for transfer of the experience gained to other projects, in particular to the PyeongChang-2018 project planned by the KMA. D.Kiktev informed the colleagues that involvement of Roshydromet in the PyeongChang-2018 WWRP project is a part of the new agreement on bilateral collaboration between the KMA and Roshydromet (signed in November 2014).
Possibilities of further post-project collaborative studies were touched in the discussion. In particular, joint work on ensemble nowcasting at ZAMG and the Hydrometcentre of Russia was mentioned by M.Tsyrulnikov.
Data archiving issues, namely the observation quality control and unification of the forecast data formats in accordance with TIGGE conventions at least for a subset of key variables (T2m, Relative Humidity, Total Precipitation, Wind Speed, Direction and Gusts) were discussed. It is planned to prepare the «final» data sets in March 2015. In the interim the participants were asked to inform the archive holders on detected issues in the observation data quality flags.
Closing the Meeting
V.Lukyanov, the Chief Meteorologist of the Sochi-2014 Olympics thanked the participants for their invaluable contribution on behalf of the Olympic forecasters team.
A.Baklanov thanked the hosts for good organization of the meeting and wished success to the team of the project participants.
D.Kiktev said that the FROST-2014 project is not finished. Completion of the field phase is just a start of another project stage – stage of analysis of the field campaign, shaping and retention of the project legacy. He expressed his deep gratitude to the participants for their efforts and input to the project.
Note: All the meeting presentations are available on the FROST-2014 web-site (http://frost2014.meteoinfo.ru/presentations).