Road and Infrastructure Condition Monitoring – Fujisawa

Fujisawa is a coastal city of roughly 440,000 people covering 69.6 km² in south-central Kanagawa, between Tokyo and Yokohama, with a dense commuter population and a heavy seasonal tourism load around Enoshima and the Shonan beaches. It hosts Keio University’s Shonan Fujisawa Campus, and the city and the university have worked under a formal cooperation agreement since 2009. Fujisawa is a UTA member city.

This pilot addresses a common problem for cities: they cannot inspect their roads often enough to know their real condition. Fujisawa manages hundreds of kilometres of road surface, markings, guardrails, and street furniture, and, like most local authorities, only finds out about problems when a resident reports them or when a manual survey happens to cover that street. UTA members Keio University and NTT East, working with Fujisawa, tried a new approach. They installed cameras on the city’s refuse collection vehicles, which already travel every street on a set schedule, and used deep neural networks to analyse the road conditions from the footage.

Cities often lack the information they need to maintain roads. Faded lane markings, cracked surfaces, damaged guardrails, and debris worsen over time, and councils usually learn about them only through resident complaints or infrequent manual surveys. Dedicated road inspection vehicles can help, but they are expensive, so most authorities use them infrequently and mainly on main roads. Residential streets, where children walk to school and clear markings matter most, often receive the least attention.

Keio University’s Shonan Fujisawa Campus and Fujisawa City had already established the platform this pilot needed. From January 2013, they had been instrumenting refuse collection vehicles as mobile environmental sensors, with the National Institute of Information and Communications Technology funding the equipment. Associate Professor Jin Nakazawa’s group chose refuse trucks for a specific reason: door-to-door collection covers every inhabited street in the city, so the fleet reaches places no fixed monitoring station, and no taxi fleet ever will. For comparison, Japan’s national Soramame-kun air monitoring network operates roughly 1,800 observation points across the entire country, and Fujisawa itself has five. By May 2016, the city had equipped all 100 of its refuse vehicles.

The road condition strand extended that platform from sensing the air to seeing the street. Each vehicle carries an omnidirectional camera mounted on the undercarriage, pointed at the road surface and its surroundings. The research question was not whether cameras could be fitted but whether a machine could reliably read condition from the resulting footage — video shot from a moving working vehicle in uncontrolled light, at whatever speed the road demanded.

Keio’s answer, published at IEEE Big Data 2017, applies an end-to-end deep neural network object detection approach that simultaneously proposes candidate regions and classifies whether the road marking within each one has deteriorated. The team trained it on drive-recorder footage from the sanitation workers themselves, annotated by hand. That detail matters: the training data came from the same class of vehicle, camera and working conditions as the deployment, rather than from a curated inspection dataset that would not survive contact with a real refuse round.

BigClouT, which stands for Big data meeting Cloud and IoT to empower citizen clout in smart cities, provided the project framework. The European Commission funded it under Horizon 2020, while Japan’s NICT supported the Japanese partners. The project ran from 1 July 2016 to 30 June 2019 in four pilot cities: Grenoble, Bristol, Fujisawa, and Tsukuba. The project describes Fujisawa’s role as combining participatory incident reporting with garbage-truck sensor-based infrastructure management, with this pilot focusing on the infrastructure side.