In speaking with people about open government and open data issues around New Jersey, I am often asked for my views on the state of access to information. In this post, I propose a new vision: It’s time for New Jersey to adopt a state open data law and require state & local government to proactively publish certain types of records online. This will ultimately serve to create a better experience for both public & private sector data users & reduce the burden of responding to OPRA requests for government by codifying a public policy of data sharing into state law, following other succesful open data laws.

The problem with open data in New Jersey

I have come to be of the opinion that New Jersey needs its own open data law largely due to the lack of effectiveness of the state’s existing efforts to open up government data and the shortcomings inherent in the primary tool I have used to access it, the Open Public Records Act (OPRA).

OPRA is broken

One of the best cases for creating a new paradigm for open data in the state has been the problems inherent in OPRA, New Jersey’s state freedom of information law. OPRA was already broken, and the COVID-19 pandemic saw the state legislature further gut the law by throwing the normal 7-business day deadline out at the height of the pandemic. This bill was passed hours after coming out of committee with no public input.

Municipalities show their hostility to transparency laws like OPRA by routinely railing against the volume of OPRA requests they must process, commercial requesters & other grievances they have with the law.

Furthermore, municipalities routinely pass boilerplate resolutions calling for further weakening provisions of the law, usually at the behest of advocacy organizations representing segments of government and often without any public debate or discussion. Organizations like the New Jersey League of Municipalities and the state Municipal Clerks Association have long been at odds with the public’s right of access under OPRA.

OPRA records custodians also routinely take advantage of the law’s provisions to unilaterally grant themselves extensions, sometimes to a farcical degree, as any veteran records requester could tell you. And because the law makes the possibility of records custodians facing personal sanctions for a so-called “knowing and willful” violation of OPRA exceedingly rare, wily custodians know that they can get away with pushing the envelope when it comes to OPRA responses and extensions.¹

Proactive sharing of OPRA records falls short

New Jersey needs a state open data law because municipalities cannot be trusted to reliably publish open data without a mandate from Trenton. Some public bodies have sought to voluntarily publish frequently requested records, both in an effort to accomodate public demand and to reduce their own burden in responding to OPRA requests, which can be a good thing. However, while some agencies start out with this good idea in mind, they too often fail to continue to update the data, limiting its usefulness. An example of this can be seen with the Lacey Township Police Department, who began publishing use of force reports in 2020 but mysteriously stopped updating them past the summer of 2020, leaving their web page more than a year out of date. Or, like the New Brunswick Police Department, some departments are making crime data available via a web map that prevents the download of any underlying data and has an unfriendly interface that looks like it was designed in the 1990s.

COVID-19 has been a setback for NJ open data

New Jersey state agencies have left much to be desired with their approaches to data sharing during the COVID-19 pandemic, which further shows the need for an open data law. The New Jersey Department of Health has been far less transparent than public health authorities in our neigbhoring states when it comes to making machine-readable counts of COVID-19 cases and deaths available at the municipal level.

Despite the Murphy Administration’s proclamation that “data determines dates” with respect to public health restrictions implemented during the pandemic, the New Jersey Department of Health (NJDOH) has done the bare minimum in terms of making COVID-19 data available to the public and researchers. NJDOH records custodians have been less than helpful - sometimes dragging out their response to a simple OPRA request for municipal COVID-19 data by taking over a year’s worth of extensions, with no credible justification for doing so.

This intransigence has led to others to fill-in the gaps in municipal COVID-19 data by manually collecting numbers of cases and deaths announced at press conferences, among other measures. This is far from an ideal or workable solution for the long term.

A state open data law enshrining a commitment to sharing machine-readable data would have helped to rein in this problematic behavior by NJDOH officials. Unfortunately, absent OPRA lawsuits challenging their failure to release public data, there will no incentive for state agencies to do better with sharing data.

New Jersey municipalities lack tech capacity

Making a state open data law successful cannot happen without acknowledging the technology issues faced by municipal government. One of the biggest challenges with OPRA’s implementation - and source of hostility from government toward the law - has been the lack of technology resources available to municipal officials as it relates to their burden in compiling records and fulfilling requests. Many small towns lack their own IT departments, among other challenges. This has led to frustration from civil servants and requesters alike.

Some municipalities lack basic software for processing PDF documents, so their process to redact documents in response to OPRA requests has been to print out physical paper copies of records, redact them with a black marker and then scan it back in (usually in a large, uncompressed file that is not searchable), resulting in a waste of paper and time for all concerned. Even worse, this inefficiency is often used as a basis to charge requesters a fee for accessing documents that should normally be provided free of charge via electronic methods.

The lack of technology skills in government has also had serious consequences for identity theft and personal privacy. Failure to properly redact documents by OPRA records custodians has led to breaches of private personally identifying information, such as social security numbers and drivers license numbers, as some records custodians have thought it was sufficient to redact a document by creating a black box in Microsoft Word or Excel, not realizing that the underlying text they wanted to redact was perfectly accessible.

Against this background, any state open data law needs to take into account the lack of capacity in municipalities and work to alleviate this burden. The state should not place the onus on municipal authorities for building out the infrastructure needed to support open data. Instead, existing platforms like the state open data portal, data.nj.gov could be extended to support new municipal datasets, among other options.

New Jersey’s open data initiatives are balkanized

Another factor that necessitates the adoption of a state open data law has been the lack of consistency among state agencies when it comes to their approaches to data sharing. I have long supported any effort to proactively publish data by governments, but my criticism of the state is rooted in their execution.

data.nj.gov has not lived up to its potential of being a one-stop clearinghouse for open-access administrative data, as many state agencies have set up their own data portals or data websites that either overlap with the main state data portal or have proven to be redundant, leading to a poor user experience in many cases.

Despite the existence of a single statewide data portal, other state agencies have seen fit to create their own data portals, resulting in a duplication of effort across the public sector. The Department of Community Affairs created their own DCA Data Hub, with only a subset of their data being also made available over the state data portal.

The state’s campaign finance watchdog, the Election Law Enforcement Commission (ELEC) also prefers to go their own way when it comes to their open data releases. You have to know where to look around on their website if you want to download CSV or Excel spreadsheets of campaign contribution and pay-to-play government contracting data, which can illuminate the hidden relationships between money & power that dominate New Jersey politics. And even if ELEC does post additional data about campaign finance complaints on their website, sometimes they’re inclined to just delete it with no notice. Yikes!

Other state agencies have been similarly half-hearted in their embrace of a centralized open data repository. One such example is the New Jersey Department of Transportation. A few of their datasets are updated on a yearly basis on the state data portal, but their comprehensive datasets that track all motor vehicle accidents in New Jersey can only be downloaded from an obscure page on their website and haven’t been updated since 2019. There is no reason that these data cannot be uploaded to the state’s existing data portal, but absent a consistent mandate and policy framework governing data sharing, there will be no impetus for them to do so, thus the status quo persists.

I could go on and on with more examples of this type of problematic design and half-baked data sharing schemes across the state. But I’ll stop here for now.

What might a New Jersey open data law look like?

A theoretical New Jersey open data law could be the third prong in making government more open, accessible & accountable to the public. This open data law would work in conjunction with OPRA and the Open Public Meetings Act (OPMA) to cover much of the categories of electronically stored information that should be able to be proactively published online. It would not replace either of the state’s two transparency laws, rather it would build off their prior successes and help to implement their spirit & intent through new methods of data sharing.

We don’t need to look very far to see successful open data laws in practice that New Jersey could model. One of the biggest success stories in open data has been New York City’s open data law, first adopted at the behest former mayor Michael Bloomberg in 2012 which required that the city compile and publish all of its datasets in a single open data portal by 2018. The City still must respond to requests under the New York Freedom of Information Law (FOIL), which is similar to OPRA, but many of the most commonly requested datasets can be freely and instantly downloaded through the open data portal.

Adopting an open data law on the scale of an entire state rather than in one of America’s largest cities will no doubt come with its own set of challenges & issues unique to New Jersey, but it can be done if the political will is there.

Another successful open data law can be seen at the federal level. As part of the Foundations for Evidence Based Policymaking Act, the role of the federal government’s data portal, data.gov has been explicitly codified in the law. The law provides that agencies must "..publish their information online as open data, using standardized, machine-readable data formats, with their metadata included in the Data.gov catalog."

What all of these laws have in common is that they codify a commitment to sharing open data in statute, and set specific requirements for the publication of machine-readable data and deadlines for compliance by administrative agencies. These principles have proven to be effective for setting up successful data sharing initiatives, and there is no reason why New Jersey cannot do the same & unlock the value in state data.

Critics will likely claim that such a law would be yet another so-called “unfunded mandate” from Trenton, passed down the political foodchain to overworked, understaffed & cash-strapped municipalities left to handle its implementation. To allay these concerns, such a law must carefully consider the lack of technology expertise & capacity in municipal government and help bridge the gap by providing state-funded infrastructure or encouraging academic partnerships to leverage the resources of the state’s universities, not unlike the model pursued by the New Jersey Civic Information Consortium and the partnerships between academia and new media that they have funded.

The administrative data collected by state & local government is a valuable, under-utilized public asset whose value should be made freely available to the stakeholders who funded its creation. New Jersey should follow in the footsteps of the federal government, other states & localities and adopt a state open data law.