Welcome to the European Statistical System Collaborative Network (ESSnet) project and the second blog. As mentioned in the first blog, the project team is broken down into four work packages focusing on different aspects. This edition focuses on Work Package Three, exploring non-traditional sources for official statistical production via six use cases. One of these is focusing on faster economic indicators using new data sources.
In this case, faster economic indicators explore open-source traffic camera data to understand the busyness in city centres around the UK. The original project looked at street scenes and explored the numbers of different modes of transport such as pedestrians, bicycles, cars, and buses. The ONS set up the project to explore alternative data sources that might provide insights on the levels of social distancing during the Covid pandemic, and also used the data to track the upturn of the economy as lockdown conditions were relaxed.
Traffic cameras are a widely and publicly available data source in the UK, allowing transport professionals and the public to assess traffic flow in different parts of the country via the internet. The images traffic cameras produce are publicly available, low resolution, and are not designed to individually identify people or vehicles. They differ from CCTV used for public safety and law enforcement for Automatic Number Plate Recognition (ANPR) or monitoring traffic speed.
Large volumes of image and video data from CCTV present significant challenges for deploying scalable platforms for computer vision and machine learning. The UK built an end-to-end solution via the Google Cloud Platform, which offered the necessary scalability and processing capability. It enabled data collection, modelling, and experimental research outputs.
It was thought that this work could be helpful to public bodies in their ongoing response to the covid pandemic as restrictions were lifted. It could also have a broader interest to local authorities to monitor issues such as:
- Changing demand for local economies
- Shifts in modes of transport for commuting
- Environmental quality
- Leveraging existing CCTV camera investments
As previously mentioned, the system was designed to measure changes in how busy places were during the pandemic rather than monitoring traffic flow. Not all cameras were useful for measuring this, as many cameras were focused on major trunk roads or located on country roads without pavements. For this project, the camera images needed to contain certain features within the scenes they captured, such as shops, residential housing, pavements, cycle or bus lane, traffic lights and/or roundabouts. The selected features needed to be easy to identify, especially static information (like traffic lights) so that they would not be confused for a pedestrian and could be easily removed from the data. The cameras were initially manually tagged to identify those that provided the required information and were in scope for this research.
The next step was to develop a Processing pipeline to gather images and apply models to detect pedestrians and vehicle numbers. The Processing pipeline needed to gather data 24 hours a day, seven days a week and manage large quantities of data, so the solution pipeline was developed in the cloud as this had several advantages:
- Machine maintenance is provided
- Security support is provided, especially with remote working
- Large data storage is easily available
- On-demand scalable compute is available, which is useful when testing models and processing large batches of imagery.
Code in the pipeline is run in “bursts” as cloud computing is only charged when used! This delivered a potential saving in costs.
However, in practice:
- Images can be duplicated if the host does not update them in time
- Cameras could periodically be offline
- Images could also be partly or entirely distorted because of physical problems with a camera.
Therefore, data cleaning was important to improve the quality of statistical outputs. Traffic cameras need maintenance; their views may become blocked, and there may also be periods of downtime, which results in missing data. The problem of missing data is well known, and algorithms to combat it.
Traffic camera data offers excellent value to public authorities by providing real-time statistics to monitor the busyness of the local population, which can inform policy interventions, for example, those seen during the pandemic.
The traffic camera data is best suited at detecting trends in “busyness” or activity in town and city centres. It is not suited for estimating the overall amount of vehicle or pedestrian movements or for understanding the journeys made. As such, it provides insights that complement mobility and transport data where other mobility data and traffic data do not cover or are not accessible given this level of temporal granularity or immediacy.
The open codebase has been refactored to enable the Traffic Cameras project to run on a single, stand-alone laptop and Google Cloud Platform. This was developed for the ESSNet project, making the system more accessible to users who do not have the infrastructure to run it.
A mid-range laptop (without specialist GPU) can now support in the order of 50 to 100 cameras. AS the above is a summary of the project and a more detailed article about the work completed by the UK to develop this system is available. The original article was written by Ian Grimstead and published in December 2021.
The Trusted Smart Statistics – Web Intelligence Network project (number: 101035829 – 2020-SmartStat) is funded by the European Union.
Published 14 March 2022