February 13, 2020: The Smart City Institute (SCI) publishes a new report about the evaluation of Smart City projects in Wallonia.
To develop this qualitative study, 25 municipalities have answered their questions. The SCI reveals the main takeaways from these interviews.
For several years, the Smart City dynamic has been growing in Wallonia. This is illustrated by several initiatives, including the Smart Region strategy of Digital Wallonia or by the launch of the call for projects “Smart Territories” launched in 2019. The latest – encouraging – figures of the Walloon Barometer 2019 have confirmed this trend: 62.5% of Walloon municipalities are already actively involved in a Smart City approach.
While these new projects are being launched in Wallonia, a question is also emerging: How are these projects evolving? Do they really have a positive impact for the citizens? And therefore, more generally: Are there processes in place in Wallonia to evaluate these projects? Since the majority of these are initiated by the public authorities (61% – cfr Walloon Barometer 2018), our team decided to look into the following question: Do Walloon municipalities evaluate their Smart City projects?
In Wallonia, although monitoring and evaluation are generally popular, there is still no publication dealing specifically with the monitoring and the evaluation of Smart City projects. With this report, the SCI is laying the foundations for this reflection.
THE MAIN TAKEAWAYS OF THIS REPORT
Through the 25 interviews conducted, this report provides a better understanding of :
- the perception that Walloon municipalities have of the monitoring* and the evaluation of their projects (all types of projects included)
- their progress on the monitoring and the evaluating of Smart City projects
*The process of “systematic collection of data on specified indicators to provide indications of the extent of progress and achievement of targets”.
Monitoring and evaluation in general: structural constraints identified
“Although there is an interest in monitoring and evaluating projects, there is generally no culture of evaluation within Walloon municipalities. »
In fact, out of the 25 municipalities surveyed, 15 of them replied that monitoring/evaluation was important, while at the same time specifying that they did not make use of it. Why not? Several structural constraints were mentioned:
Monitoring and evaluation of Smart City projects: mixed picture
“In our Walloon municipalities, only a few mechanisms for monitoring and evaluating Smart City projects are implemented and, when it is the case, the project is rarely analysed in a systemic manner.»
Indeed, just less than half of the municipalities surveyed (12/25) indicate that they have put in place a process to monitor and/or evaluate their Smart City projects. Considering that less than half of respondents evaluate on a daily basis, a contradiction can be pointed out here. Regarding the 13 municipalities that do not evaluate their Smart City projects, they consider :
- Not being advanced enough in their projects or;
- Not having the methods and/or tools to monitor and evaluate their projects.
Nevertheless, 6 of them indicate that they would like to remedy this situation.
It is also interesting to note that 10 of the 25 municipalities surveyed said they did not have a clear idea of what Smart City represents to them.
Smart City projects: who should evaluate them?
Concerning the potential Smart City projects evaluators, municipalities are divided. Among them :
- 15 consider that monitoring and evaluation can or should be carried out by a single actor.
- Almost half of the municipalities surveyed (12/25) consider that an external entity can or should take care of this.
- 9 municipalities consider that the municipal administration should be responsible for monitoring and evaluation.
- 5 municipalities consider that this should be done by a project-specific steering committee (COMAC).
- 4 municipalities consider that the Walloon Region can or should take charge of the monitoring and evaluation of projects resulting from the “intelligent territories” call.
SOME SCI’S RECOMMENDATIONS
In order to help the municipalities in their processes, but also to clarify the situation, the SCI proposes a few recommendations at the end of its report:
#1: Switch from a logic of “control” to a logic of “results”.
#2: Clarify the purpose of the evaluation and choose its indicators carefully.
#3: Point out the reason why we evaluate
#4: Raise awareness and train the administration
#5: Partially or fully delegate evaluation
EVALUATION PROCESS: WHY?
Monitoring – or evaluation – is an important step in the development of a Smart City strategy/project, and occurs at various stages of its progress. Indeed, as indicated in our Smart City Practical Guide – Volume 1, monitoring the evolution, measuring the results and impacts of the strategy and projects implemented is crucial. This evaluation phase can not only contribute to legitimizing Smart Cities projects and policies, but also help to ensure their sustainable performance. Evaluation is therefore a real tool for public managers in their decision-making and implementation tasks.
WHO ARE THE RESPONDENTS OF THIS STUDY?
The selection of municipalities was made on the basis of the call for projects “Smart Region – Smart Territory” organised by the AdN. The 78 municipalities that submitted a project were contacted in order to participate in the study (we identified the submission of a project as a significant indicator of interest in the Smart City theme). 25 of them responded favourably to our request (detailed list available in the report).
AND TO GO FURTHER
- See our report entitled “Performance Measurement in Smart Cities“. The objective of this report is
- to introduce cities and municipalities to the concept of performance measurement in the context of Smart Cities;
- to help them to establish their own performance measurement method through an integrative method in the form of a structure/canvas, based on their own specificities and strategies.
- Coming soon – a monitoring methodology: Our team is currently working on the development of a tool to guide municipalities in their evaluation process, taking into account the contextual disparities they face. Stay tuned! To ensure that you don’t miss any of our upcoming releases: subscribe to our newsletter.