Over the last few months our smart cities team has had dozens of deep dive conversations with principals at various departments of transportation, innovation offices and planning agencies across the United States. While these agencies have access to more data than ever before, the cities and municipalities don’t have a true understanding how their residents move through the transportation system and whether or not the core needs of the city and/or municipality are being served by their current system.

This evolving dynamic mirrors our dynamic journey here at Arity and as we continue to dive deep into this massive topic we are learning that the potential solutions to this challenge are not easy — and its going require all of us to tackle several different problems at once, here are three that we have observed.

Square peg, round hole: Existing data is mode-centric

As we heard many times, transportation planners are drowning in data — but the data that is most readily available are often volumes and counts of traffic or ridership. However, this specific type of data does not help transportation planners who are grappling with a fundamentally different set of questions: How can transportation network companies (TNCs) complement first/last mile service for public transit? Will autonomous vehicles cause or alleviate congestion? How do we ensure equitable access to mobility services?

To understand these complex questions, mode-centric data falls short. Because there is no concept of the user or customer in these datasets, it is difficult to understand how residents link multiple modes together to reach their destination, without relying on inference.

In our line of business to "infer" something without supporting data makes it difficult, if not impossible, to forecast latent demand for new services or recalibrating the delivery of existing systems. As a result, a majority of transportation agencies are still dependent on blindly testing new services through trial and error. This is sure to increase the time and effort required to identify new, useful services that citizens want and need.

Citizens are not a package from Amazon: Logistics v. mobility

A frequent topic of discussion during our wide-ranging conversations with officials, was that transportation is “consumed” in service of attaining something else: a job, medical and social services, groceries, recreation, etc.

As a result, transportation policy and the subsequent infrastructure and services are built with consideration of personal mobility, which is so much more than transporting a product or service from A to B. The effort to attempt understand the motivation underlying a given trip is intrinsically connected to other issues of the region, whether it be the economy, housing, or social services.

Tackling these multifaceted problems will require a holistic approach, which begins with the integration and analysis of data from many different domains and agencies. In several of our meetings, we heard about challenges in centralizing data across municipalities and cities. For example, in many places, police departments collect crash data, but that information is crucial to many departments of transportation.

In other conversations, we discussed the linkages of transportation policy to foster job growth and economic development in specific regions. But it is very difficult to formulate an effective transportation programs without a foundational understanding of where a trained workforce could be found, trained and deployed.

A forest of silos: The privacy hurdles related to obtaining a user-centric understanding

Believe it or not, many of the transportation agencies and planners that we spent time with are still collecting user-centric data via travel journals and customized surveys. As we all know, these methods are very tedious and nearly impossible to properly scale. The rapid proliferation of smartphones and other connected sensors will continue to enable data collection at a much higher level. But this power comes with its own set of challenges …  specifically privacy.

For cities and municipalities, the complicated privacy question cuts two ways. First, are people would be willing to contribute granular, potentially personal, data to a public-oriented initiative? Second, and surprising to me, several agencies also voiced concerns about storing such information themselves, even if people were willing to contribute.

Many cited Freedom of Information Act (FOIA) requirements and other transparency laws that make it difficult for agencies to protect this kind of data. Of course, these questions of data collection and privacy are not unique to this area, but the inherent personal nature of transportation makes this a particularly important problem for agencies and planners to wrestle with.

Why ask why?

In the end, why is it worth trying to address this problem of user-centric design? It’s important because with the advent of the on-demand economy, as well as new technology such as autonomous vehicles, transportation planning and policy is increasingly a crucial factor in our day to day lives. This is particularly true in urban environments, where hard infrastructure is largely fixed and building more roads or rails is not an option.

To meet growing density and demand, planners are looking more and more at “throughput” as the key performance indicator (KPI). To deliver on this goal we need to develop both policy and solutions that will encourage people to ditch their personal car and opt for higher capacity modes — ride-share, transit, bikes and the electronic scooter invasion.

Without a good understanding of where and why people are moving about, it will be difficult to develop the appropriate incentives to get people where they need to go while developing an efficient system for everyone.