How smart is your city? NIST provides a framework to measure

Dive Brief:

The U.S. Department of Commerce's National Institute of Standards and Technology (NIST) recently released a measurement framework to help local leaders assess the impacts of smart city technologies.
NIST intends for its holistic key performance indicators (KPI) framework to help community leaders measure the return on investment and community impact of certain technology or projects across an entire "smart city ecosystem." The framework involves assessing data across three main levels of analysis: technologies, infrastructure services and community benefits.
The key metrics are (1) the alignment of KPIs with community priorities across districts and neighborhoods, (2) investment alignment with community priorities, (3) investment efficiency, (4) information flow density and (5) quality of infrastructure services and community benefits.

Dive Insight:

What makes a smart city "smart?" NIST defines the term as "the efficient use of digital technologies to provide prioritized services and benefits to meet community goals." But without reliable methods of assessment, city leaders are limited in their ability to understand just how "smart" a smart city plan might be, according to the agency.

The goal for developing a guideline and a framework is to provide a standard process for thinking about what gets measured and how, said NIST's David Wollman, deputy chief of smart connected systems in NIST's Communications Technology Laboratory.

Traditionally, many smart city leaders have measured the impact of a smart city project based on certain results of implementing a specific technology or service, according to NIST. For example, if a city deploys smart streetlights, local leaders might assess the effectiveness of that project by energy cost savings. NIST’s new framework calls for examining how those lights might impact the entire smart city ecosystem, which could encompass areas including community safety and climate.

Wollman said this more "holistic" approach is important because a smart city consists of interconnected infrastructures that support citizen goals and services, and better understanding those interconnections leads to progress.

To date, cost and data-sharing processes have been two major obstacles for cities surrounding these projects and making sense of their data, according to Michael Dunaway, associate director of innovation for NIST's Smart Grid and Cyber-Physical Systems Program.

Cities have improved their use of data in recent years, though. The proportion of cities that use data to monitor and analyze progress on key goals has increased from 30% to 75% since 2015, according to a 2021 report. NIST’s new framework is designed to help decision-makers use data to make better choices around challenging areas like costs, Dunaway said.

Cities should involve local residents in the evaluation of smart city projects, according to Adam Wears, a research analyst at digital technology market research firm Juniper Research. "The involvement of everyday citizens is also going to become vital as issues of data privacy, transparency and security increasingly factor into evaluations,” Wears said in an email interview. “We have to ensure that these values are recognized and taken seriously because they matter to a great number of people, and at the end of the day, building user confidence is just as vital to the future of smart cities as building efficient systems.”

An emphasis on citizen engagement in project evaluation might help address a "techlash," according to Stan Caldwell, executive director of Carnegie Mellon University’s transportation research institute, Traffic21.

The dissolution of high-profile smart city efforts like San Diego’s smart streetlights program and Sidewalk Labs' controversial Quayside project in Toronto shows residents' skepticism of certain smart city projects or technologies. “My advice for cities is that this is the time where you have to be extra cautious about being very transparent about what we're doing,” Caldwell said. Cities must engage the public, he said, which often goes counter to how smart cities originally functioned.