Why Open Data is good for everyone

The San Francisco Open Data Portal, like our municipal water system, is a public utility. While water cleans our hands and our streets, data fuels essential services, enables analysis, and increases transparency in government operations. Just as water must be collected at a source, purified, transported, stored, and delivered to residents, data must also be gathered, cleaned, structured, and made accessible for public use. And like our Hetch Hetchy water—some of the cleanest and most reliable in the nation—San Francisco’s public data is among the best (just ask our 6.3 Million API calls a month).

Yet, despite its critical role, municipal data—like water—is often taken for granted. Few think about the engineers maintaining the pipelines or the consequences of a sudden shutdown—until it happens. While San Francisco’s data remains available, a troubling shift has occurred at the federal level. Since January, datasets have disappeared from federal websites and repositories. Some have been restored, but the episode served as a wake-up call, revealing the vast and often unseen reliance on open data and the serious consequences of its removal.

The original vision for the San Francisco Open Data Portal—where the tech community would unite to build free civic applications for the public good—did not fully materialize as imagined. While tech companies do leverage open data (rideshare and mapping companies, for instance, depend on temporary street closure data to navigate city streets), they represent only a fraction of its users.

Instead, the true champions of open data have emerged from within the city and beyond—public servants, researchers, advocates, and everyday residents—who rely on it to drive meaningful change. In many cases, these groups could not do their work without it. These are the communities I see benefiting most from open data:

Businesses and industry groups regularly use open data. The Registered Businesses Dataset, for example, helps business owners verify their information and track industry trends such as new businesses opening. Contractors and architects rely on public permit data to monitor projects and understand broader industry shifts. Analysts use SFO travel data to forecast tourism trends and demand. Open data provides businesses with a free resource that increases transparency and gives them visibility into factors that impact their work.

Since 2024, we’ve recorded over 50 open data citations across major and local publications. This includes Mission Local’s award-winning election coverage, BigMoneySF, which leveraged San Francisco Ethics Commission data. The SF Chronicle Data Team has used open data to analyze city management, public health, crime, transportation, and, of course, the ever-contentious topic of public trash cans.

From NYU to UC Berkeley, UCSF to the University of Reims Champagne-Ardenne, students and postdocs have used our data for research. Every week, we field questions from infectious disease researchers, urban planners, economists, and even the occasional accounting student. The accessibility of open data allows them to conduct studies that expand the world’s knowledge—at no cost.

This group is really a hundred little groups. Whether it is someone fighting back against evictions, identifying vacant lots to build housing, or understanding the devastating impacts of COVID or traffic violence, data is necessary to tell their story. Access to open data empowers these advocates with the information needed to drive change, support their communities, and hold decision-makers accountable.

There is one user group that does not often get a call-out when it comes to open data usage, but by some measures, it’s actually the largest: city employees. The City of San Francisco was created about 150 years before modern data infrastructure became mainstream. Needless to say, most departments were not designed with modern data infrastructure in mind. In most municipal governments, there is a pressing need for a centralized source of truth when it comes to data. And this is a need that the Open Data Portal and its internal-only counterparts fill.

It’s impossible to validate an address without a list of valid addresses. Or tie key property information to a parcel without a list of active parcels. It’s impossible to see how many 311 calls happened in a certain supervisor district or neighborhood without access to the official boundaries. When data is centralized, it helps answer core questions consistently and unlocks services and analysis that could not be done without use of interdepartmental data.

Unsurprisingly, when assessing use based on API calls (an automated way to pull data from our Open Data Portal), Enterprise Addresses, Parcels – Active and Retired, 311 cases, lists of intersections, and street segments are some of our most popular datasets. All feeding data to department-built tools and processes.

Maintaining the status quo is not enough. We must also focus on expanding outreach and engaging more people in the open data conversation, ensuring that communities understand how to leverage these resources effectively. We must actively identify high-value datasets and prioritize their publication. Beyond providing raw data, we should develop interactive tools and applications that make data more accessible and actionable for the public. Additionally, expanding open data initiatives in other municipalities is critical. As a first step, we are making all of our documentation on our data inventory process, data publishing, and standards publicly available on sf.gov/data. In the near future, we hope to further increase our transparency by making our Open Data code repository public - so anyone else can use our architecture, macros and models to develop their own program.

It’s important to recognize that open data isn’t just an abstract concept—it’s a vital tool that supports businesses, journalists, researchers, advocates, and even city government itself. Whether it’s ensuring transparency, driving innovation, or making city operations more efficient, open data plays a critical role in modern society.

When data remains open and accessible, it empowers people, fuels discoveries, and strengthens democracy. Let’s commit to keeping data free, available, and impactful—because open data is good for everyone.