Third meeting of the FROST-2014 project - 10-12 April 2013, Saint Petersburg
Venue: Hotel Courtyard St. Petersburg Vasilievsky
MINUTES OF THE MEETING
April 10 2013
The project coordinator Dmitry Kiktev welcomed the participants on behalf of Roshydromet.
From WMO the meeting was welcomed by Slobodan Nickovic, who presented to the participants an overview of the current WWRP activitiesrelated to the goals of the FROST-2014 project.
A meeting agenda was approved. .
D. Kiktev presented a report “FROST-2014 project: state of affairs”..
During the discussion of the report B.Bica raised the question of observation quality control, asking for quality control flags for observations on the project ftp-server.
D.Kiktev answered that only a part of quality control procedures is implemented at the moment. Several types of quality checks are planned to be implemented in the nearest future: elimination of observation data falling outside of fixed limits, spatial quality control, temporal (10-min) quality control, comparison with COSMO-RU2 model forecasts and manual quality control. Special flags will be associated with the results of these quality control procedures.
S.Belair expressed his interest to compare the snow density observations in the mountain cluster with the canadian fresh snow model results.
G.Isaac interested if blowing snow may be a problem in Krasnaya Polyana. D.Kiktev said that the valley is well fenced from the winds by surrounding mountains, and basically blowing snow is not on the list of the primary weather risks. Significant blowing snow was observed there only once during the last year.
Discussing the thresholds used for probabilistic forecasts (this issue was also risen by G.Isaac), D.Kiktev noted that in general the list of thresholds collected by Chris Doyle for the Vancouver Olympics is relevant for the «Sochi-2014». However, some corrections might be introduced by the Organizing Committee of the «Sochi-2014» Games.
E.Lim pointed out that visibility forecasts were very important for Olympics, however, KMA’s attempts of visibility forecasting were not too successful. She asked about the Sochi plans with respect to visibility prediction. D.Kiktev answered that today in Roshydromet visibility is predicted mostly subjectively, and there was a lack of appropriate automated tools. In contrast with GEM or NMMB, in COSMO model visibility is not a forecast variable. Although the Roshydromet’s experience in nowcasting and visibility prediction is very modest, it is a part of the project. It will be tried and verified. D.Kiktev hoped that the project would be an opportunity to learn in that area from other participants of FROST-2014. G.Isaac noted that visibility forecasts demonstrated some skill in SNOW-V10 project.
The experience of meteorological forecasting for test sport events in Krasnaya Polyana in winter season 2013/2013 was overviewed by E.Vasilyev.Among the forecasters needs he mentioned more detailed information about cloudiness, visibility, precipitation type, intensity and timing. The forecasts should be presented as meteograms, summary tables, and animated plots. .
E.Vasilyev noted that several sport events were put off due to low visibility. G.Isaac asked whether it was connected with snow or with clouds. E.Vasilyev answered that it was associated with low clouds that was a serious problem for forecasting in the mountain cluster.
It was mentioned in E.Vasilyev’s presentation that ARPA-SIMC ensemble forecasts arrived late, and the 00UTC EPS results were not available for preparation of morning forecasts. A.Montani explained that the mesoscale COSMO-S14-EPS ran using initial and boundary conditions from the ECMWF EPS system, so it had to wait when the ECMWF 72-hour forecast were available. That’s why it’s very difficult to deliver Italian EPS products to the forecasters earlier.
A.Montani also said that forecasters had a lot of information from different models and asked E.Vasilyev whether a multimodel ensemble forecasts would be good for forecasters. E.Vasilyev answered that multimodel forecasts would be very helpful. D.Kiktev mentioned that some work on preparing such integrated forecasts was carrying out at the Hydrometcenter of Russia.
P.Joe noticed that the height of the freezing level should be a challenge. S.Belair also pointed out the problem of cloud prediction.
J.Milbrandt was interested how the forecasters used the output of different models. The Chief Meteorologist of the Sochi Olympic Games V.Lukyanov answered that the basic model was COSMO-RU with resolutions of 7 and 2 km. The forecasters start their analysis with COSMO. In the course of the following analysis the outputs of all available models were considered. The forecasts were quite successful during this winter season though the winter was very warm. V.Lukyanov noted that COSMO-RU7 forecasts with small lead times were more realistic in cases of heavy precipitation.
S.Nickovic was interested what the reason was for the differences between COSMO-RU7 and COSMO-RU2 forecasts. It seemed strange that in some cases 7-km model performed more realistically. D.Kiktev answered that differences were due to different orography and resolved convection in the 2km model. Other physical parameterizations were the same in both COSMO-RU versions. Using two models with the same parameterizations makes it easier to investigate the effect of different resolutions.
G.Isaac interested if the forecasters faced problems with precipitation phase and freezing rain.
V.Lukyanov answered that winter 2012/2013 was unusually warm and basically forecasters did not have serious problems with precipitation phase. The freezing rain issue appeared to be not on the agenda during this winter season.
G.Isaac asked whether the models treated melting snow properly?
D.Kiktev answered that although it was not analysed, he douted that the subtle diabatic effects of metlting snow were caught well.
M.Tsyrulnikov asked about the forecasters’ experience in using ensemble products and how useful these forecasts were. V.Lukyanov answered that due to the lack of time in operational conditions sometimes the forecasters would prefer to have only one good forecast. However, in reality they deal with many different forecasts, so they work with “an ensemble” of forecasts. Integrated ensemble products are very useful as they accumulate a lot of information. V.Lukyanov expressed his concern about the training: ”Many new forecast systems are being presented and we start getting their results, but the Olympics are approaching rapidly and forecasters might be short of time to adapt to these new products”.
G.Isaac suggested to organize a support group of scientists to help the forecasters and to consult them shortly before and during the Olympics. D.Kiktev answered that the training should be organized before the Olympics. He informed the participants that international experts George Isaac and Paul Joe were invited to Sochi last autumn to help with training, and their participation in training courses was very helpful. V.Lukyanov considered the idea of the training support group attractive, but he doubted if such a consulting was possible in routine operational forecasting. D.Kiktev said that in place support group of international experts was a serious organizational issue and he hoped that the required support would be provided by the local Olympic Organizing Committee. He also mentioned the language issue.
A.Melnichuk reported about the subsystem of radar observations for «Sochi-2014».He informed the participants that the Doppler radar WRM200 and MRR2 data have been available for forecasters since December 2012. He demonstrated a first example of composite map combining Russian and Turkish radar data (although by that moment there was no regular data flow from Turkish radars). It was mentioned that precipitation data could be obtained at a height of 1 km above the sea and 2 km above the mountains.
Possibilities to expand observational data coverage using Turkish radars were discussed after the report. G.Isaac, B.Bica and P.Joe took part in this discussion.
A.Chaika was interested if any manual on radar data interpretation was prepared for forecasters. It was answered that only brief technical instructions were available.
A.Koldaev briefed the participants on “Vertical remote sounding of the atmosphere for Sochi-2014 – current state and prospects”.
E.Lim asked if there were any problems with the data transfer. A.Koldaev answered that they dealt with several telecommunication providers (Megafon, MTS, Yota), but their relative reliability and speed of data transfer have not been intercompared yet. The sounding data are transmitted via Internet to the FTP-server of RPA “Typhoon” and then to the FROST-2014 server. At the moment only temperature and humidity soundings were possible. For wind profiler permission for frequencies was expected to be got in summer 2013.
M.Tsyrulnikov asked a question about the vertical resolution of profiler data. The answer was that for the Scintec wind profiler resolution was 50 m and for the temperature-humidity profiler RPG it was 100-150 m.
G.Isaac wondered if the profiler data could distinguish water vapor from ice. A.Koldaev answered that only vertically integrated liquid water could be retrieved.
A.Koldaev also informed the participants that along with other equipment two precipitation occurrence sensor systems (POSS) should be available for Sochi region in the framework of the SPICE project.
N.Bocharnikov gave a presentation «AMS observations at the Olympic sport venues – current state»
V.Lukyanov raised a problem of erroneous snow surface temperature and height measurements at the AMSs maintained by IRAM (at four sport venues in the mountain cluster).
N.Bocharnikov answered that the problem of snow depth measurements with NovaLynx IRU9429 sensor was recently solved. Some progress was made with snow surface temperature measurements (with VAISALA DST111 sensors). Testing site was organized by IRAM near Saint Petersburg. However, further tests will be possible only after establishment of new snow cover.
V.Lukyanov asked whether Canadian colleagues had faced such a problem in Vancouver. P.Joe answered that for the «Vancouver-2010» Olympics meteorologists were not responsible for predicting and observing snow temperature.
A.Koldaev asked a question on errors in snow heights obtained by supersonic sensors. How was the problem solved?
N.Bocharnikov explained that sensors «caught» snowflakes and gave wrong results. After a series of systematically controlled experiments the problem was fixed via a choice of proper positions and heights for the sensors.
A.Chaika asked whether it was possible to calibrate the weather sensors in ATOS codes? N.Bocharnikov answered that it was possible.
A.Chaika also emphasized the importance of snow temperature and snow density data. He asked about measurements of snow wetness. N.Bocharnikov said that such observations were held, but not tested yet.
One more question was about web-cameras. N.Bocharnikov answered that 4 cameras were installed at 4 venues, but by that moment there was no agreement on the data transfer (that implied some additional financing).
B.Bica reported about the status and preliminary results of INCA and ALADIN-LAEF setup for the Sochi 2014 Olympics.
He demonstrated that ZAMG was ready to provide quite a large volume of information (both nowcasting results and ensemble forecasts) to the FROST-2014 server. Some test files were already at the project server.
The report was followed by a discussion on the radar data quality, transfer and the time when the INCA results could be at the FROST-2014 server (G.Rivin, A.Melnichuk, D.Kiktev, T.Bazlova, P.Joe).
Today ZAMG receives radar data after their processing in IRAM. IRAM transfers radar data each 10 min (if available), so ZAMG can get data in 15 min. The INCA run lasts 5 min. Some time could be saved if the radar data were obtained in agreed format directly from the radar in Sochi (A.Melnichuk).
G.Isaac made a presentation “Environment Canada Nowcasts for FROST-14 which Blend In-situ Observations and NWP Data”.
During the following discussion the participants asked about the details of the INTW method (J-B.Bremnes, D.Kiktev, E.Lim). One of the questions was if the same approach was used for precipitation intensity and other nowcasted parameters. G.Isaac answered that it was so.
INTW tests on the basis of GEM and COSMO models for the Sochi region were tried and presented. G.Isaac noted an expanded set of participating models would be of interest.
P.Joe told about the Environment Canada radar-based nowcasting activity for Sochi-2014.
A.Melnichuk asked a question why 50% of radar echo over the sea was associated with the sea clutter.
D.Kiktev was interested if the presented radar nowcasting products for Sochi gave information on the nowcasting uncertainty. P.Joe confirmed that the nowcasting product presents the most likely and upper bound assessments for precipitation totals.
M.Tsyrulnikov asked whether the radar echo was extrapolated as a solid body using the same motion vector for the whole scene. P.Joe replied that the scene was divided into 9 subscenes, so that 9 motion vectors were computed.
E.Lim gave a talk on the KMA potential contribution to FROST-2014
A wide range of potential FDP- and RDP-activities including physical downscaling from global scales down to 1 km, MOS, ensemble MOS and 2D surface wind model with resolution of 100 m was presented. The presented results showed that the increase of resolution from 1 km to 200 m did not substantially improve precipitation forecast, but it was beneficial for wind prediction.
D.Kiktev noted that KMA plans to use a 3-year training period for calibration and asked whether there were changes in the Korean forecasting system during this period. E.Lim answered that there were some, the main related to the changes in model resolution. G.Isaac mentioned that in his method he used very short training period as he wanted to avoid the heterogeneity of the training series caused by changes of forecasting systems.
S.Nickovic was interested in KMA plans to organize a WWRP FDP or RDP.
P.Nurmi reported on FMI plans and contribution to FROST-2014
P.Nurmi told about HIRLAM model and high-resoluiton (1.5 and 1 km) HARMONIE models. S.Nickovic wondered why so close resolutions were chosen for the two versions of HARMONIE model. P.Nurmi answered that it was connected with experiments with different orographical data sets.
A.Montani asked if HARMONIE model could be easily plugged into any atmospheric model of larger scale. P.Nurmi answered that it could be easily plugged at least into HIRLAM and ECMWF models.
M.Tsyrulnikov was interested in the data assimilation used for HIRLAM and HARMONIE models: the more advanced forecast model HARMONIE has 3D-Var data assimilation system, whereas the older HIRLAM uses 4D-Var. P.Nurmi said that both models would be operational, but HIRLAM would run with 4D-Var and HARMONIE with 3D-Var. E.Atlaskin confirmed that HARMONIE + 3D-Var configuration is considered not to be a problem.
The day ended by Discussion lead by G.Isaac.
The following problems were discussed:.
1) Definition of wind gusts
A.Montani said that COSMO adopted several (3-4) definitions in the model postprocessing. All methods tended to overestimate gusts over the land near the sea. The verification did not show any preference among these methods. .
G.Isaac commented that according to the WMO definition, wind gusts are maximum 2-min average wind speed in 1 h.
D.Kiktev considered this issue to be a matter of temporal upscaling or downscaling, and various assessment periods can be used depending on available observation and model data.
P.Joe, G.Isaac, D.Kiktev, S.Nickovic, S.Belair, V.Lukyanov took part in this discussion.
2) Quality and frequency of AMS observation
Time step for AMS-observations was discussed. D.Kiktev informed the participants that the frequency of AMS-observations in the Sochi region would not exceed 10 minutes. For several meteorological parametes (precipitation intensity, visibility, cloud base and wind gusts) 1-minute observations will be available at AMSs maintained by IRAM (however, this information will be transmitted once in 10-minute time slot). 10-minutes time slot between observations will be used at Roshydromet’s AMSs. All AMS observations will be available at the project ftp-server. It was added that AMS-sensors of Megafon provider were installed on the masts of mobile communication at non-standard levels (from 15 to 40 meters and higher). Although temperature, humidity, pressure and wind measurements from Megafon AMSs with 3-minutes frequency were informative, their precipitation data were unreliable.
Solid precipitation data and the issue of tipping buckets at majority of Roshydromet’s AMS were discussed. Only limited set of Roshydromet’s stations (e.g. Gornaya Karusel) were equipped by proper precipitation sensors.
It was concluded that the map with observation stations and a list of stations with their coordinates and heights must be renewed at the FROST-2014 web-site.
A.Montani proposed to save both raw and controlled data at the FROST-2014 server. D.Kiktev said that the data would not be changed after control, only flags would be added. Probably some additional flags might be added in future.
3) Work of forecasters team during the Olympics
G.Zaimskikh wondered how the work with forecasters was organized in SNOW-V10.
G.Isaac and P.Joe mentioned several points:
- Researchers’ helpdesk was organized in Vancouver to help forecasters. The helpdesk communicated to the leader of the forecast team, and he transmitted advices to the “field” forecasters. Sometimes the forecasters feedback was a reason for the near real-time model changes. Sometimes forecasters started to use new techniques. The involved researchers were on site 6 weeks prior to the event, carefully watching the performance of the techniques.
- Daily teleconferences
- Project web-site with blog was organized. Everyone could read and leave comments
- At least one representative from research team for each system was available all day long
D.Kiktev mentioned that the blog at the FROST-2014 web-site was not live enough. It was thought to be moved to password-protected area as it was subjected to spam.
4) The georgaphical points for meteograms.
It was concluded that this problem was site-specific, needed too much discussions and would be considered later in private discussions.
11 April
P.Steiner made a report “COSMO contribution to FROST2014”.
He made an overview of the CORSO priority project of the COSMO consortium, briefly summarized the activity of Russian and Italian colleagues. and paid most attention to the results on high-resolution (1 km) forecasting obtained at MeteoSWISS.
S.Belair asked a question why the probability fields from 7-km and 2-km ensembles differed so much. P.Steiner answered that most differences are due do finer orography in 2.2 km model because there were no differences in physical parameterizations (except for the fact that convection is resolved directly in the 2,2-km ensemble). .
S.Nickovic was interested in the clustering procedure used in COSMO-LEPS (and in COSMO-S14-EPS) and A.Montani provided some information on the method used. For COSMO-S14-EPS, 10 clusters are selected from ECMWF EPS forecasts, then one representative member is chosen from each cluster. The ten selected representative members are used to drive the high-resolution forecast.
M.Tsyrulnikov asked how much time was needed for the Kalman filter in MOS to adapt to changing weather. P.Steiner replied that the coefficients were recalculated daily, so the results of postprocessing would “feel’ the weather changes the same day. However, that kind of MOS was successful only in reducing biases, being not too successful in improving the RMS-scores.
A.Montani made a presentation “Ensemble forecasting for Sochi-2014 Olympics: he Sochi-ensemble system from COSMO”describing limited-area ensemble activity at ARPA –SIMC for meteorological support of the Sochi Olympics, performance of COSMO-S14 ensemble prediction system, main results and open issues. The COSMO-S14-EPS had been run twice a day in operational mode and provided ensemble forecasts to the FROST-2014 server since December 2011. A.Montani analyzed the EPS performance for several cases, demonstrating how the ensemble probabilistic information could be used by forecasters.
E.Astakhova asked a question about the multi-ensemble system that A.Montani was going to develop. A.Montani answered: ”Most of the European ensembles are calculated at ECMWF, so it’s easy to combine the results. And of course other ensemble data provided to the FROST-2014 server will be used as well”.
P.Nurmi was interested in verification tools used for ensemble forecasts, A.Montani said that a new version of COSMO verification software VERSUS with a capability of ensemble verification was recently issued.
J.-B.Bremnes presented an update on the HIRLAM contribution to FROST-2014.
HIRLAM started transferring GLAMEPS ensemble forecasts to the FROST-2014 server in September 2012 as a contribution to the FDP project component. Besides this, work on calibration of GLAMEPS ensemble forecasts (for temperature, precipitation, wind speed) and development of a configuration of high-resolution ensemble prediction system HarmonEPS for the Sochi region was in progress as a contribution of HIRLAM consortium to the RDP component of FROST-2014. J.-B.Bremnes mentioned that the information about observation stations (given in Annex1 to the FROST Concept Paper) must be updated.
Answering to P.Steiner’s question on calibration, J.-B.Bremnes said that the training period for the calibration would be about a year and no important changes were introduced to the GLAMEPS system during the last year.
J.Milbrandt presented an Update on Environment Canada's High-Resolution Deterministic Modeling System for FROST-2014.
After the report P.Steiner wondered about the differences in physical parameterizations used in the hierarchy of GEM configurations with so different resolutions.
G.Isaac mentioned that originally there was a plan to use deterministic forecasts with 1-2 km grid spacing, but now Jason start using resolution of 250 m. 7km –> 2 km –> 250 m – what would be next? J.Milbrandt replied that he did not expect much improvement to be gained from further enhancement of model resolution in the near future. He thought that finer resolution for ensembles could be the next step.
K.Park noted that for an increase in the horizontal resolution, one had to enhance the vertical resolution as well. However, in practice the latter was often not the case. S.Belair said that turbulence parameterization could require modification of resolutions to about 100 m. M.Tsyrulnikov mentioned that the high-resolution model could suffer from lack of fine-scale details in the initial data because we had no data assimilation for such a high resolution.
M.Tsyrulnikov presented the talk by Matthew Pyle “NMMB model forecast plans for Sochi 2014”..
The NMMB model was briefly introduced. Its performance in the Sochi area for the last winter was examined on several high-impact weather cases. Comparisons with observations show that precipitation forecasts were quite successful.
G.Rivin reported on the recent activities with COSMO-RU
Ability of models with different resolutions to predict precipitation was discussed afterwards. G.Isaac mentioned that more details could be predicted with higher-resolution model, but the total precipitation amount could be worse. S.Nickovic suggested that the different skills of predicting precipitation amount by 7-km and 2-km COSMO model versions could be associated with resolving convection in the higher-resolution version and probably caused by a problem with clouds simulation.
It was mentioned by Sochi forecasters that COSMO-RU7 often predicted precipitation more successfully than COSMO-RU2, and G.Rivin was asked to comment on that statement.
G.Rivin said that that statement was not correct. For some locations at meteograms one could find that COSMO-RU7 was better, but for other locations it was worse. If one examined the maps, he could see that 2-km model certainly added value. For example, very good results with COSMO-RU2 model were obtained in modeling of the Novorossiisk bora.
Basing on 5-year experience of verification of COSMO model forecasts with various resolutions over Switzerland, P.Steiner noted that COSMO2 was consistently better than COSMO7 that tended to give more precipitation (larger positive bias). However, at some locations COSMO7 meteograms could be better.
M.Tsyrulnikov presented an update on the 3D-Var data assimilation with COSMO model over the Sochi area
S.Nickovic wondered if the presented 3D-Var data assimilation technique was implemented in spectral space?
M.Tsyrulnikov answered that a mixed spectral and physical space approach was used. The 3D-Var background-error statistics was first specified in spectral space and afterwards the physical-space covariance model was produced.
S.Nickovic wondered how M.Tsyrulnikov differed between gravity wave and small scale in physical space. M.Tsyrulnikov replied that the HMC 3D-Var was based on spatial operators that were defined in physical space, but reproduced the desired spectral-space filtering properties.
K.Park asked a question on background-error co-variances.
M.Tsyrulnikov commented that currently the statistics was largely ad-hoc; in the near future ensemble based background-error covariances would be used.
P.Joe, S.Belair: Input/Comments of CAS/WWRP WGs on Nowcasting and Mesoscale Weather Forecasting Research and Discussion.
S.Belair paid attention to difficulties of the models use by the forecasters and to practical training needs. He tried to identify issues to tackle to make the models more useful (visibility, precipitation type etc.).
The list of subjects for potential project articles was proposed:
- Role of mesoscale data assimilation;
- Impact of surface orography and increased resolution;
- Model ability to predict visibility;
- Ensemble forecasting aspects;
- Nowcasting of various meteorological variables in winter mountain environment;
- Verification and intercomparison of the participating forecasting systems;
- Etc.
.
P.Joe noted the significant progress that had been achieved during the two years since the start of the project (observations, model resolution, number of participating models and nowcasting systems etc.). However, more observations were needed. E.g. more surface stations along the ridge could be important for understanding local physical processes. Extra-stations outside of Krasnaya Polyana, on the windward side of the mountains could contribute to better nowcasting.
P.Joe put forward the idea to move the remote sounding system (the vertical profilers) closer to the sea (half-way between the sea and the mountain). In that case it could be possible to see the approaching weather systems. V.Lukianov answered that there were some limitations on shifting the system, e.g. it was difficult to find a proper place for the system in the mountains. Besides that it is also important to have such a system on the site because the main problem are associated with low-level clouds that affect visibility. It was also added that a new radio sounding station was installed near the coast (4 soundings per day).
D.Kiktev noted that, unfortunately, there were no plans to increase the number of stations/observations. Thus, the main task is to improve the data quality and to make the data available to all the participants.
A string of points to work on was mentioned: observations of snow depth, solid precipitation, closer links with forecasters, better understanding of physical processes.
Discussion on societal and economical impacts of the project
D.Kiktev noted that at the moment the project was in an interesting position – a spectrum of new forecast products became available quite recently or was expected to arrive in the near future. As early as possible availability of FDP products is crucial. Forecasters have only several months before the Olympics to get acquainted and gain the necessary practical experience with the new products. So the training is a critical issue. The last chance to introduce new FDP-products will be the forecasters training in September-October 2013.
Participants agreed that it was necessary to make new forecast products available as soon as possible (even before September 2013) to give the forecasters more chances to get confidence and practical experience in their use. It’s difficult to switch from synoptic scales to the small-scale - so much details! Closer contacts between FDP-team and forecasters are needed.
S.Belair noted that preparation of joint papers would be important for the project publicity. In SNOW-V10 two BAMS articles were published before the Olympics. It was recommended to pay attention to this issue and prepare a common contribution to BAMS.
D.Kiktev informed the participants that FROST-2014 was recently presented in at the WSN12 symposium in Brazil and at the annual EGU meeting in Vienna (many thanks to G.Isaac and B.Bica). He added that two papers (general overview and the first paper on COSMO forecast verifications for Sochi) had been submitted. The first guess of a joint overview paper for BAMS could be prepared in several months.
D.Kiktev noted that assessment of the project societal and economical impacts should be first of all linked to its FDP-component. For formal assessment of the project societal impacts it is important to demonstrate the benefits and efficient use of the project FDP-products. D.Kiktev informed the colleagues about his teleconference with the World Weather Research Department (WWRD) of the WMO Secretariat which took place several months earlier. At the teleconference, during the discussion on FROST-2014, WWRD recommended to consider all the project ensemble forecasting as a part of the project RDP-component because it might be difficult to demonstrate the social/economical efficiency of the ensemble probabilistic information that is still quite new for forecasters and decision makers (Probably due to this reason the ensemble WWRP project in the Beijing-2008 Olympics was RDP). It was also recommended to clearly mark the status of all provided project products – RDP or FDP.
D.Kiktev expressed his concern that there was some lack of formally acknowledged FDP-products in the project. Originally (in the project concept paper) it was planned that in FDP mode FROST-2014 deterministic forecasts will have horizontal resolution of 2-2.5 km or finer, whereas in RDP mode 1 km or less. At the moment for many products their status (FDP or RDP) was not clear. E.g. the participants were asked:
- if 1km-version of Canadian model GEM would be FDP or RDP?
- if INTW nowcasting system would be FDP or RDP?
- if INCA nowcasts of all meteorological variables (including, say, visibility) would be FDP?
It was not simple to definitely answer these questions. G.Isaac answered that formally even in Canada INTW was not at the moment operational.
The participants discussed which products could be associated to FDP.
S.Nickovic noted that FDP meant you were ready to pass from research to operation. G.Rivin suggested that everything that came regularly in time could be considered FDP. J.Milbrandt noted that this was not only a problem oftimeliness, but also a problem of how the FDP products would be used.
However, most systems, considered in the project, were implemented and verified over the domains different from Sochi. It was asked if these systems could be considered as an FDP? S.Nickovic noted that if an operational system was successful in one region, it could be expected that it would be good enough for another domain. On the other hand some of the products might not be used by forecasters, but still considered as FDP being well tested and validated.
It was also discussed if ensemble forecasts could be considered FDP. The ensemble systems are mature enough and now are being using operationally in many countries. Forecasters are experienced in working with ensemble products. EPS can help to make a decision. So the participants concluded that 7-km and 11-km ensembles were certainly FDP. A.Montani stressed that their ensemble technology was well established, so it could be part of FDP.
S.Nickovic supported the proposal to consider participating EPSs with grid spacing of 7 km or more as a part of the project FDP-component. However, convective resolving ensembles are quite new and must be considered RDP.
Some variety was expected in the status of nowcasting products: CARDS – FDP, INTW – RDP, MeteoExpert – FDP, INCA – depending on a particular meteorological variable.
It was decided that participants would discuss at their home institutions the status of their products (RDP or FDP) and send the decision to D.Kiktev.
How to measure societal and economic impacts of the project in various socially significant areas (education, understanding, transfer of technologies, practical forecasting, support of decision making etc)? D.Kiktev noted that binary indicators (e.g. “we have not provided that product/service and we provide it now”) seemed to be easier for the progress demonstration than quantitative measures.
April 12
Yong Hee Lee reported on KMA plans for the PyeongChang-2018 Winter Olympic games..
Y.-H.Lee introduced a plan of the meteorological support of the PyeongChang-2018 Olympic Games and gave an overview of present status of the preparations. Being impressed P.Nurmi noted that it seemed that everything had been already prepared for the Games.
In the course of discussion on the possibility of a WWRP project associated with the Olympics in 2018 S.Nickovic noted that KMA should specify the focus of the project and its potential new outcomes. A possible “core” of the project could be high-resolution modeling and data assimilation.
Anatoly Muravev Forecast verification for Sochi winter sport events – preliminary results.
Preliminary results of station-based and polygonal verifications for COSMO-RU and WRF-NMMB deterministic forecasts were presented. For near-surface temperature COSMO-RU forecast biases were found to be different in sign and magnitude in winter 2011/2012 and 2012/2013. This might be linked with sensitivity of the model systematic behavior to the large-scale peculiarities of these seasons (winter 2012/2013 was very warm).
M.Tsyrulnikov wondered who was the “target audience” for the presented sophisticated verification techniques like quantile plots.
A.Muravev and P.Joe replied that this kind of diagnostics could be useful for model developers.
P. Nurmi presented “Forecast evaluation/verification visions - Comments/Input of FMI and the WWRP WG on Verification” .
Discussion:
1) “Freezing” of the participating systems
.The FDP project component implies formal evaluation of participating forecasting systems. To have a solid ground underfoot, forecasters need a practical experience with “frozen” at some stage forecasting systems. Such a “freezing” might be also needed for some postprocessing systems. Normally, e.g. COSMO model has 3-4 releases per year, and there might be changes in the driving DWD model.
S.Belair said that usually Environment Canada changes the models only once a year. He emphasized the risks of last moment changes in the forecasting systems.
The participants decided that only very moderate changes in the project FDP-systems should be allowed (e.g. fixing bugs) during the several last months before the Olympics. It was also adviced to test the changes first on the retrospective data. No changes are allowed in the last 3-4 months before the Olympics (However, it was realized that basically changes in the participating systems were beyond the external control)
2) Verification systems
.
It is important to clearly formulate the main goals of the verification before designing the verification system for the events of the Olympic Games status. The main goal of using a sophisticated ‘scientific’ verification system is to better understand a model behavior and to give recommendations for its possible upgrades. ‘Operational verification’ (conceptually simple and most likely user-oriented) is mostly designed for forecasters. Both approaches are being developed for Sochi. MET (Model Evaluation Tool / NCAR) software is currently used for the project forecasts verification. The COSMO-community verification tool VERSUS is used in parallel for COSMO-RU forecasts.
Participants agreed that that some compulsory scores should be selected, permitting other measures to be used as a complement. P.Nurmi expressed his readiness to provide a draft of such a list.
D.Kiktev noted that the formal evaluation period was needed for the participating FDP-systems. It was decided that the time slot from 15 January to 15 March would be used for the formal evaluation. Official forecasts are planned to be available for their verification along with the objective project forecasts.
D.Kiktev presented an update on the project data exchange procedures.
Quality control flags for the XML-files with observations were presented as a composition of several digits:
- low-order (rightmost) digit – means manual control;
- the 2nd digit from the right – spatial control using approach by Steinaker et al, 2000;
- the 3rd digit from the right – temporal control and control with respect to the model forecast.
Each digit can take the following values: 0 - no control (default); 1- suspicious; 5 – OK; 9 – rejected. Examples: e.g., 10, 1, 900 etc.
Unified grid is preferred for gridded forecasts providing that it is accompanied with point forecasts. If point forecasts are not provided than original model grid is suggested for the data transfer. GRIB2 is recommended for transfer of forecast fields.
The participants were asked to provide their point forecasts in the unified XML-format. The link to this XML-description is available at http://frost2014.meteoinfo.ru/doc.
Besides of digital forecast data, graphical products from the participants are welcome.
Discussion:
It was proposed by S.Nickovic and G.Rivin to always use the same fixed number of digits in the quality control flags.
D.Kiktev agreed with reservation that more than 3 digits might be needed in the future.
J.Milbrandt asked for a list of fields for the graphical input from the project participants.
D.Kiktev answered that point meteograms (not fields) were meant here.
A.Montani proposed to archive all the graphical products along with digital forecasts. It was considered to be possible.
B.Bica asked whether the forecasts were planned to be archived and whether the authors could change them. It was answered that all forecasts were being archived. For practical use and formal evaluation of FDP-products it is compulsory to have all data unchanged, but for RDP-component there is more flexibility.
M.Tsyrulnikov: Discussion on possible RDP- outcomes of the project.
M.Tsyrulnikov presented several slides where the most important tasks of the RDP part of the project were listed and asked the participants for their ideas on possible ways to move further, which themes are worth to make research on and which papers should be published.
What we expect new in utilizing observations? What we expect to obtain from deterministic forecasts? From ensemble forecasts? - A string of items was identified (P.Joe, G.Isaac, J.-B.Bremnes, S.Belair, A.Montani, S.Nickovic):
• Comparison and evaluation of observations from different sensors (in particular, measurements of snow height);
• The mutual dependence of physics and resolution, which type of cloud parameterization is preferable for small scales;
• Nowcasting and forecasing of visibility, wind gusts, precipitation occurrence and type;
• Combination of deterministic runs;
• Data assimilation (including nested data assimilation): Impact of radar data and other regional observations on forecast quality;
• Impact of land surface (snow);
• Combination of observations and short-range forecasting for nowcasting;
• 7-km ensemble FDP vs 2-km RDP;
• Forecast calibration;
• New products (e.g. polygon probabilities).
• Case studies of severe weather and poor forecast performance situations
(More details on the RDP/scientific outcomes discussion can be found in the separate presentation posted at the FROST-2014 web site).
D.Kiktev: Discussion on further steps.
D.Kiktev informed the colleagues about the current activities at the Hydrometcentre of Russia aimed at development of integrated products on the basis of forecast inputs from the FROST-2014 participants. He hoped that the integrated forecasts would help the Olympic forecasters as a kind of the first guess for preparation of official forecasts. The resulting products should be transferred to the Olympic information system (supported by IOC partner Atos Origin) and Olympic Broadcasting Services (OBS). If this plan worked it would be an integration of t
D.Kiktev asked the international colleagues if they have any objections to make their data inputs to FROST-2014 mutually available for the project needs to all its participants. There were no objections. S.Belair noted that the absent colleagues from NOAA should be asked as well.
Following the recommendations of the WWRD, D.Kiktev suggested to merge the project Working Groups 3 (IT) and 4 (Products, training, assessment and social impacts). There were no objections from the participants. As a result the current structure of the project Working Groups looks as follows:
• WG1: Observations and nowcasting (chaired by D.Kiktev and G.Isaac);
• WG2: NWP, ensembles and assimilation (chaired by M.Tsyrulnikov and A.Montani);
• WG3: IT support, products, training, assessments and social impacts (chaired by D.Kiktev).
D.Kiktev asked the participants to fill in the provided template table (available in his presentation at the project web-site) with information about their systems and products with indication of their status (RDP/FDP). The status may be different for different variables of the same system. The participants’ replies were expected in two weeks.
Time and place of the next FROST-2014 meeting were not discussed. It will be done after the budget allocation .
Closing of the Meeting .
V.Lukyanov said that he was grateful for all the interest and activities within FROST-2014 and the Sochi forecasters’ team would be very grateful for any help before and during the Olympics.
G.Zaimskikh thanked the participants for their work and inputs. She expressed her hope to see at least some of the international experts in Sochi at the training course and during the Olympics.
S.Nickovic thanked Roshydromet and the local organizing committee for good organization of the meeting. He mentioned large progress in the work and wished success to the project.
D.Kiktev on behalf of Roshydromet spoke words of gratitude to the participants for their efforts and input to the project.
The meeting was closed at 13.30.
[1] Budget for 2014 has been confirmed. The next meeting is planned for Autumn 2014.