Digital Democracy: Marrying AI with a database project makes California government meetings and activities accessible to all

Dave Lesher, senior editor and co-founder at CalMatters, consults with Ryan Sabalow, CalMatter’s Digital Democracy reporter. (Photo credit: Fred Greaves)

A California nonprofit newsroom is using AI-generated databases to examine legislative sausage-making in unprecedented ways.

CalMatters and tech-driven partners have created a centralized database that improves government transparency and the public's access to materials. Called Digital Democracy, the project launched in April using a combination of AI, publicly available databases and some real-person backups. While available to anyone, the searchable product is a news reporter's dream tool.

Thanks to funding from the Knight Foundation, Arnold Ventures and the Lodestar Foundation, the Digital Democracy project delivers:

• “Every word uttered in a public hearing or floor session” within 48 hours of the hearing
• The text, amendments, bill analysis, vote, supporters and opponents of all bills
• Campaign finance information
• District voter registration, election results and demographic data from every district

David Lesher, senior editor and co-founder at CalMatters, ushered the Digital Democracy project through to fruition. (Photo credit: Fred Greaves)

According to David Lesher, senior editor and co-founder at CalMatters, all of the information was previously available to the public, but it was not easily accessible. The information in the Digital Democracy database comes from many sources, but the public can now access all the information in one place. In some cases, the technology can cut hours of research time into minutes. It can even deliver notices whenever a lawmaker speaks on an issue.

In California, the law requires all public hearings and floor sessions to be video recorded and posted within 24 hours. CalMatters sends the videos to a transcription service that uses AI to turn those videos into searchable transcripts. It also assigns the quotes to the individuals who spoke them, using facial recognition technology, with confirmation from human editors. The technology also gathers numerical data for political context.

The project launched in April after several stops and starts. CalMatters partnered with Cal Poly San Luis Obispo, which helped build the database, and 10up, a global web development firm.

“There were two things that were converging,” said Lesher, who ushered the project along. “You have a government that has become more opaque. One of the big motivations is the policy process — how things happen and why they happen. Who these people are has become much more opaque and hard to follow. But then what came along was the opportunity with new technologies to see within the policy-making process through empirical data and recognize the relationships between legislators and interest groups … and interpret why things are happening.”

Ryan Sabalow, the Digital Democracy reporter, mines the database for interesting stories. Among the stories he’s uncovered is a tactic that California lawmakers use when they want to oppose a bill without leaving a record. Sabalow reported that Democrats almost always abstain from voting rather than voting against legislation. CBS picked up on and expanded on this story, interviewing parents about their fight in favor of a failed bill intended to address fentanyl abuse.

Other stories showed how the Democrat Party has become a lobbying powerhouse, how California environmentalists killed a plan to fast-track ‘green’ energy and what lobbying groups get their way most often in the California Legislature.

Sabalow hopes more California newspapers will use the database to generate stories about their lawmakers and topics important to their communities. Digital Democracy has created a “tip sheet” system to generate notices for those wanting specific information.

“I worked at a small newspaper in Northern California for seven years, and I didn't really care as much about what the Democrats in the state were doing, but I cared what my two individual lawmakers were doing,” Sabalow said. Digital Democracy can send automated “tip sheets” to alert them when local lawmakers speak or vote on an issue.

So far, the Digital Democracy website has seen an 80% growth from April to June with 121,000 sessions in June, Lesher said. He added that there may be applications of the technology for use in local government contexts or court systems in the future.

Bob Miller has spent more than 25 years in local newsrooms, including 12 years as an executive editor with Rust Communications. Bob also produces an independent true crime investigative podcast called The Lawless Files.