DataSF In Progress

Plan Progress

Our open data legislation requires 1) an inventory of City datasets and 2) publishing plans from each department. March 31, 2015 was the target date for all departments to complete the dataset inventory. We accept department submissions on a rolling monthly basis and ask for annual revisions of the inventory. Publishing plans were due contingent on inventory completion and will be updated every year.

52 Departments
Inventories Complete
Plans Complete
Read the Publishing Plans
Inventory status
Publishing plan status
Department or Division Inventory Status Publishing Plan Status

Publishing Activity

Published Datasets
Inventoried Datasets
% datasets published from inventory
Datasets published by category
% of inventoried datasets published by dataset classification

During our inventory process, we classified datasets as public, sensitive or protected. We expect departments to publish public datasets first.

% of inventoried datasets published by priority

Departments prioritize datasets for publication based on data classification and value. We expect to publish Priority 1 datasets first. Learn more about how we prioritize datasets.

Datasets published over time (toggle month and year)

*Note the publishing activity above represents when it was made available via our current open data platform provider (Socrata). 2012 is when the City officially launched the updated portal, however many of those datasets were available earlier.

Publishing Quality

Performance against targets

Every six months, departments set a target number of datasets to publish. Below is the citywide rollup of department targets. The counts and targets reflect the current planning period, January 2017 through end of December 2017.

% of datasets published against annual target
Published Datasets since Jan 2017
Targeted Datasets through Dec 2017

Timeliness of publishing

Each dataset is published with a target publishing frequency (e.g. daily). We calculate if datasets meet this target. Datasets with no update schedule are marked as unknown. Over time, all datasets will have an update target. Datasets may also be published with a time lag (e.g. a month old). Publishing lags must be justified and should not exceed a month.

% of datasets updated on time
% of datasets without a publishing lag
Median Lag in Days

Documentation and usability

Datasets are only usable if they are understood and accessible. We aim to have complete metadata including data dictionaries for every dataset on the portal. We also like data to be published "natively" meaning it is hosted in our open data platform, which provides more tools for using the data.

% of datasets with required metadata
% of datasets natively hosted