State AI Rollouts Are Outrunning Their Own Governance

By late 2024, employees in 82 percent of state CIO organizations were using generative AI tools daily — up from 53 percent a year earlier, according to NASCIO. Yet, only 24 percent of states had data governance in place for generative AI, according to a Pew survey. New York, Pennsylvania, New Jersey, Colorado, and California have all launched AI programs for state workers in the past year. Each made different procurement choices. Most haven't shown the public what governs them.

New York has taken a number of recent actions to integrate generative AI into state operations, but its rollout has exposed unresolved questions about transparency and governance. Governor Hochul announced on April 6 that AI Pro — a generative AI assistant built by the state's Office of Information Technology Services and powered by Google Gemini — would expand from a 1,200-person pilot across eight agencies to the full state workforce of more than 100,000. The state describes it as a training tool, but employees in the pilot used it to draft memos, summarize documents, and process agency work.

As is the case with any cloud-based deployment of Gemini, every prompt a government employee types passes through Google's infrastructure. The state describes AI Pro as a "secure environment" without pointing to any documentation that explains what that means. The public-facing rollout — the announcement, the ITS press release, the governor's office page — links to no terms specifying whether the deployment involves tenant isolation, data residency requirements, or limits on what Google can do with prompt content.

New York State’s Comptroller, Thomas P. DiNapoli, had already flagged the underlying concern: an April 2025 audit across four state agencies found that New York's AI policy lacked adequate guidance for agencies to meet its own expectations. DiNapoli called it "a wake-up call." Nevertheless, twelve months later, AI Pro was expanded to over 100,000 users.

New York also published Appendix C-AI, standard contract clauses that bar outside AI vendors from training on state data, require 90 days' written notice before any deployment, and grant the state the right to demand independent audits. In February 2025, ITS and Google Public Sector announced a formal two-year partnership covering data services for state employees. Whether those Appendix C-AI protections apply to the Google relationship is not addressed anywhere in the public-facing rollout.

The state also advanced broader AI oversight efforts. Nineteen days before the AI Pro announcement, Governor Hochul launched the FutureWorks Commission, charged with recommending policies to protect workers from AI-driven displacement — and then the rollout went ahead before the commission reported.

The RAISE Act, signed December 19, 2025, requires frontier model developers to submit transparency reports and safety evaluations before deployment. Hochul further proposed a new oversight body — the Office of Digital Innovation, Governance, Integrity, and Trust — to centralize AI and digital safety policy. Both are directed at private-sector AI developers. Neither touches the state's own deployments, which is precisely where the gap in AI Pro sits.

Other states appear to have approached the process differently, prioritizing governance infrastructure and labor protections before expanding AI systems statewide. In Pennsylvania, Governor Shapiro's administration sat down with a union before it handed employees the tool. In March 2025, the Office of Administration negotiated a binding side letter with SEIU Local 668 — representing nearly 10,000 Commonwealth employees — that formally established a Generative AI Labor and Management Collaboration Group and negotiated protections such as prohibiting AI from being used in disciplinary decisions.

Pennsylvania had spent two and a half years building the governance infrastructure to support that kind of agreement. Shapiro signed Executive Order 2023-19 in September 2023, partnering with Carnegie Mellon University for research support, creating a Generative AI Governing Board, and codifying ten core principles for state AI use. That infrastructure was already operational when the side letter was signed.

By the time Shapiro expanded ChatGPT Enterprise to 3,000 employees across 35 agencies in April 2026, with 6,500 more enrolled in training, the side letter and executive order structures were already operational. The state also requires employees to complete Innovate US training before they can access the tool — meaning an employee cannot log in until the training is done, not merely recommended alongside it.

New Jersey took a different approach. The NJ AI Assistant, launched in July 2024, runs on Azure OpenAI — Microsoft's commercial cloud — but uses LibreChat, an open-source interface the state controls, rather than an off-the-shelf enterprise product. When the state rebuilt the tool in March 2026, it kept that same architecture. More than 15,000 state employees have used it at a cost of roughly $1 per user per month.

In what it described as a national first, New Jersey surveyed its public-sector workforce about generative AI before finalizing its governance framework — the survey results directly informed the Task Force's recommendations to the Governor rather than arriving after the fact. The state tiers its training by job function through InnovateUS rather than requiring a single course for all employees.

Colorado, which passed one of the first comprehensive AI laws in the country, has since repealed and replaced it — moving to mandatory risk management programs with a narrower disclosure-focused framework that takes effect January 1, 2027. The state also deployed a commercial AI tool to its own workforce, rolling out Gemini Advanced across 20 agencies with 1,700 licenses, requiring mandatory training and attestation before employees could access it. Whether the state's own Gemini rollout falls under either version of the law is a question Colorado's implementation guidance has not addressed.

California similarly expanded generative AI use, while broader governance standards remain under development. The state’s “Poppy” AI assistant, powered by ChatGPT, is available to state workers alongside dedicated AI sandboxes built on cloud environments separated from production systems. Governor Newsom's Executive Order, signed March 30, 2026, sets a 120-day window — closing in late July — to develop AI vendor certification and procurement standards. Until that framework is published, the terms governing Poppy's ChatGPT integration aren't surfaced in any public-facing documentation tied to the rollout.

These contrasting approaches point toward several emerging best practices for governments deploying generative AI systems. For example, when employees handling medical records, discrimination complaints, or financial data type a prompt into a commercial AI system without training, that is a data protection problem, not a workflow question.

As a result, governance frameworks should be in place before AI tools reach the full workforce. Pennsylvania's sequence — executive order, governing board, union agreement, then statewide expansion — offers one model. New York’s Comptroller's audit suggests the risks of moving in the opposite order.

States should also publish clear public documentation governing their commercial AI deployments. Several states have developed strong contract language and vendor standards, but connecting those protections to specific rollouts in ways the public can readily find and review remains inconsistent.

Employee training should further be required before access is granted — at the individual level, not the agency level. Similarly, labor-impact studies commissioned alongside AI deployments should report before those tools reach full scale, not after.

States can set these standards for themselves. The question is whether they will.