United States Representative Zoe Lofgren (D-Calif.) speaking at a hearing of the House Committee on Science, Space, and Technology Subcommittee on Energy at the US Capitol in Washington, DC, on September 2, 2026. (Photo by Michael Brochstein/Sipa via AP Images)

Rep. Zoe Lofgren on FISA, Surveillance and the Fourth Amendment

The landmark front entrance of the Virginia State Capitol in Richmond, Virginia, the historic seat of the commonwealth government, on April 18, 2025. Shutterstock

The landmark front entrance of the Virginia State Capitol in Richmond, Virginia, the historic seat of the commonwealth government, on April 18, 2025. 

Virginia Eyes AI Chatbot Rules as Observable Risks Warrant Action

New York City Schools Chancellor Kamar Samuels and Mayor Zohran Mamdani hold a press conference at the Brooklyn STEAM Center on September 2, 2026 in New York. Photo by: NDZ/STAR MAX/IPx

America’s Two Largest School Districts Impose AI Moratoriums

The OpenAI logo is displayed on a mobile phone screen in this photo illustration, as artificial intelligence tools continue to expand across consumer and business applications in Brussels, Belgium, on February 28, 2026. (Photo by Jonathan Raa/NurPhoto via AP)

What ChatGPT’s DSA Designation Means for OpenAI and the EU

Google CEO Sundar Pichai talks before a dinner with President Donald Trump and Japan's Prime Minister Sanae Takaichi in the State Dining Room of the White House, Thursday, March 19, 2026, in Washington. (AP Photo/Julia Demaree Nikhinson)

The Google Ad Tech Remedies Matter to You, Too

The Meta logo on a smartphone against a red background.

Without a Child Rights Framework, Meta’s $17.1bn Settlement Will Only Go So Far

Demonstrators take part in a protest against the deployment of an AI-assisted police surveillance system in Berlin on July 17, 2026. Photo by Andres Trujilo/ JNA PRESS(Sipa via AP Images)

Violent Anti-Tech Extremism Has Roots in History

Illustration of a laptop displaying the OpenAI logo on its screen. A hand holds a magnifying glass over the screen with blue lightning bolts emanating from it, symbolizing scrutiny or investigation. The image uses a halftone comic style with blue and black accents.

ChatGPT, Roblox Face Tough EU Rules. Implementation Is the Hard Part

Paper collage. A photograph of the Atomium in Brussels, cut into diagonal strips and reassembled over a grid of numbers.

Seeing More — Seeing Less by Anna Riepe & FARI / 

Can the EU AI Act’s Transparency Rules Strengthen Democratic Resilience?

﻿Germany’s Federal Chancellery building on a sunny day

Germany’s Federal Chancellery building on July 19, 2025 (Photo by Michael Nguyen/NurPhoto) 

What Germany's AI Security Institute Could Learn From Other Countries

This image is a section of the Detroit Industry Murals by Diego Rivera, created between 1932 and 1933. Rivera, a renowned Mexican muralist, painted these panels inside the Detroit Institute of Arts. They are widely considered one of his greatest works and a landmark of 20th-century art.

The workers shown are laboring on automobile assembly lines, likely at the Ford Motor Company’s River Rouge Plant.

Rivera celebrated the dignity and effort of industrial labor, portraying the workers with strength and unity.

The style is Social Realism, characterized by its focus on working-class subjects and its detailed, lifelike forms.

Photograph of a detail of a 1933 Diego Rivera fresco at the Detroit Institute of Arts. Justin Hendrix/Tech Policy Press

A Labor Day Prescription for Preserving Fair Employment in the Digital Age

A large number of people take part in a gathering of boats and motorbikes in support of the people of Ceuta on 30 August 2026 in Ceuta (Spain). (Europa Press via AP)

A large number of people took part in a gathering of boats and motorbikes in support of the people of Ceuta on 30 August 2026 in Ceuta (Spain). (Europa Press via AP)

How Disinformation Fueled the Latest Ceuta Mass Crossing

Mechanical Turk is Closing. The Workers Who Built AI Are Still Here

Members of California's legal team leave court after a settlement was reached in a multi-state lawsuit accusing Meta of causing harm to children on social media outside the Ronald V. Dellums Federal Building and US Courthouse on Wednesday, Aug. 26, 2026, in Oakland, Calif. (AP Photo/Noah Berger)

August 2026 US Tech Policy Roundup

The Colorado State Capitol is framed by buildings along Sherman Street Friday, June 27, 2025, in Denver. (AP Photo/David Zalubowski)

What 12 State ‘Companion Bot’ Laws Demand of AI Providers

A tic-tac-toe board with human faces as digital blocks, symbolizing how AI works on pre-existing, biased online data for information processing and decision-making.

Amritha R Warrier & AI4Media / Better Images of AI / tic tac toe / CC-BY 4.0

Germany's Draft Law on AI in Migration Raises Rights and Bias Concerns

A collage of circuit boards interwoven with aerial photographs of water treatment facilities, agricultural lands, industrial sites and mining machinery.

Privacy by Architecture Makes AI Compliance Work

A yellow school bus is seen in Chicago, Illinois, United States, on July 28, 2026. (Photo by Marcin Golba/NurPhoto)

In the US, Caution Rules on AI Ahead of K-12 School Year

A young child holds up a smart phone in front of their face using both hands.

This Week's Meta Settlement Will Reverberate Across the World for Years

This image is a collage with a colourful Japanese vintage landscape showing a mountain, hills, flowers and other plants and a small stream. There are 3 large black data servers placed in the bottom half of the image, with a cloud of black smoke emitting from them, partly obscuring the scenery.

Deborah Lupton, "Servers in a Landscape" / 

'Technology Freedom Cooperative' To Build Infrastructure for Human Rights and Justice Groups

Jake Laperruque is a fellow at Tech Policy Press. 

Audio of this conversation is available via your favorite podcast service.

, a Democrat from California, about the the Foreign Intelligence Surveillance Act and ongoing efforts to reform it. What follows is a lightly edited transcript of the discussion.

I'm really delighted to be joined here today by Representative Zoe Lofgren. She's a longtime champion on privacy, surveillance, and a whole host of technology issues. Congresswoman Lofgren, thank you so much for joining us today.

Great. So we're going to talk today primarily about FISA, the Foreign Intelligence Surveillance Act, specifically the warrantless provision of FISA, FISA Section 702.

So for listeners who aren't too familiar with FISA in general or aren't too familiar with FISA 702, could you give them just a brief summary of what this surveillance authority is, and why Congress needs to take action on this issue?

702 of FISA allows the government to collect communications of non-Americans located overseas, and it's supposed to be for foreign intelligence purposes without an individualized warrant for each target. It can be an important national security tool, and I'm not suggesting that it be eliminated.

And the reason why you don't need a warrant is the Constitution protects everyone in the United States from unreasonable search and seizure. But if you're abroad and you're not an American, it doesn't protect you.

The problem is that you're catching Americans who might be on one end of the communication, and when they do, they become part of a database. And once that database is in place, it can be queried for information. And it's not just call data, it's the substance. It's emails, it's telephone calls, it's everything without a warrant.

And in fact, this has happened in the past where the FBI goes in and does a warrantless search of the database for Americans' communications without a warrant, and certainly without even the need for a foreign policy purpose. So it's a loophole and it needs to be closed.

United States Representative Zoe Lofgren (D-Calif.) speaking at a hearing of the House Committee on Science, Space, and Technology Subcommittee on Energy at the US Capitol in Washington, DC, on September 2, 2026. (Photo by Michael Brochstein/Sipa via AP Images)


And so just to dig in a little on that querying process where the FBI and other agencies, as you're saying, they go and they say, "Well, the American wasn't a target, but hey, pull up all the American's emails from our databases. We want to read all of Jake's emails. We want to read all of Congresswoman Lofgren's emails."

I mean, is that a rare thing or are they doing this as standard practice once they've grabbed all these emails with the warrant, well, not all your emails, but grabbed all emails that could be collected from this Foreign Surveillance Authority?

It's not rare. It's not rare. We do have some data. The data is subject to some skepticism given what the FBI and others have done, but backdoor searches have been used improperly for protestors, 19,000 campaign donors, members of Congress, journalists, government officials, a state court judge who had complained to the FBI about police misconduct. It has been abused substantially in the past.

Now, the government argues that that's no problem now. All those problems have been resolved, but I can't discuss the classified FISA court ruling. Ron Wyden and I have tried to get it released, but I will just say this: the information that is being promulgated by the government is probably not accurate.

And just to follow up on that, because I obviously can't speak to anything that classified's saying, but can describe what's been in public reporting on this.

Correct. That's all I'm able to do as well.

Yeah, which there was a FISA court opinion from I believe last March that basically said through filtering processes and tools, there's what the FISA court seems to believe are queries happening but aren't being counted as queries.

Correct, correct. That's what the public reporting has been. I want to be very cautious not to discuss the classified document.

Yeah, of course. And that's the sort of thing I think all of us working on this space or interested in this space that can't see behind that classification wall would love to have access to that opinion and know the details of what's going on, but unfortunately we haven't been able to.

Well, and in my judgment, there's no reason in the world why that opinion should not be released, but it's not been released.

So FISA, it's been on and off on Congress's agenda throughout this year. It's been a big item in the news because there's a deadline for FISA 702, and that deadline actually has passed, although current surveillance is ongoing because of basically the orders that were already in effect, they go for the duration of those orders, which goes into early next year.

Because of this deadline, it's been a big item on Congress's plate and probably will come back up again at some point in the next few months because of that looming expiration of the current surveillance.

These days it really feels like there's just so many major news events every day, things that would be a front page story for a week or the fifth-biggest story of the day when we're dealing with so many issues that aren't just important but seem really critical to our society and our democracy.

Why is FISA reform and this component of FISA that targets foreigners something that the American public should feel is a priority to talk about, and to spend our time in, and for Congress to spend its time on?

Well, for one thing, as I said initially, there is value in Section 702 when it comes to foreign intelligence. So I'm not proposing that this section be eliminated, but I am insistent that the privacy rights, constitutional rights of Americans be respected.

And there's an easy way to do that: before you query the database for an American, get a warrant. People get warrants in cities and counties and states all over the United States every day, and there's no reason why not to do it in this case.

And so as I mentioned, this is something that Congress has been voting on, but there hasn't been a vote on that concept yet this year, has there?

The idea of, "Well, maybe we extend this authority, but we say if you're doing these queries, you're looking for Americans, you have to get a warrant."

Well, they pulled the bill because I think Johnson realized that we had the votes, and so here we are in limbo.

I think the proponents of the status quo were believing that if they played chicken and said the FISA authority will expire, that the constitutional rights people would fade away. That did not happen. As you correctly point out, the programs that have been authorized continue, those don't stop, but ultimately this will come to a head.

We've had over the years successful votes in the House. I think the first one was the Tom Massie and I had an amendment more than a decade ago that passed. And I'm expecting that if we have that question put to the House that it will pass. I can't ever speak for the Senate.

So before diving in a little further on FISA, I want to step back and look at privacy and surveillance more generally because this is a topic that you've been a champion on for so long.

As a general matter, the frame we often hear from folks when discussing privacy is, "I've got nothing to hide, nothing to hide, nothing to fear." How would you respond when you hear that sort of mentality about surveillance issues and about whether we should have surveillance reform as a policy priority?

Well, there are exceptions to the warrant requirement in law. One is exigent circumstances and the other is consent.

And so if someone wants to consent to the government reading all their emails and listening to all their phone calls, fine, they're free to do that, but that should not void the Constitution when it comes to the rest of us.

So basically, this is something that you should care about as a right, even if you don't care about it for yourself, you should care about it-

Yeah. If the individuals want to give the government carte blanche to take all their private information, that's up to them. I don't plan on doing that myself.

So a flip side of that, that nothing to hide, nothing to fear mentality is you'll also hear individuals sometimes say, "Well, I think tech companies are always snooping on me. Social media is collecting my data. I have ads that are so tailored to my conversations around the dinner table that I think my phone is probably wiretapped and listening to me, and I'm always being watched and privacy's dead. There's so much surveillance. Why should I care about government surveillance?"

How do you convince people with that view of privacy that these debates are still worth having and fighting for?

Well, two things. First, people are right to be concerned about privacy. The tech world is collecting information about you, and there are bills that have been introduced.

My Online Privacy Act would give users meaningful rights over their personal information and require companies to minimize data they collect and retain and could only use it for the purposes that it was collected for. That bill has not been made into law, and hopefully it will.

But there's a difference between the government collecting information about you and Meta collecting information about you. The government can arrest you and put you in prison. Meta can't do that.

I'm curious, do you think in general that people are more apathetic about privacy now than they used to be with tech as just an….?

I have no idea what people generally think. All I can do is stand up for the Constitution.

So as far as standing up for it in terms of FISA in particular, this is an issue that FISA reform that's always gotten quite good bipartisan support.

There's been Democrats and Republicans working together on past reforms that we've won on FISA surveillance reforms. And also it's something that for the current fight that's been going on this year on that, there's also been a lot of bipartisan supporting cooperation.

So I'm kind of curious to look at that as a model when our politics are so incredibly polarized, probably the most polarized they've been in my life and many people's lives.

How does surveillance reform manage to be so bipartisan? And are there lessons that policymakers and folks across DC should learn from this issue about how to function in a more bipartisan way?

Well, I think the first effort that was made on this in the House was by myself and Thomas Massie. And I didn't know Thomas at the time.

Actually, he came up to me on the floor and he said, "Is Tesla in your district?" And I said, "No, it's just outside my district." And he said, "Well, I have a Tesla." I said, "Well, good for you." And I went back, "Who is this guy?" So I looked him up and he's got two degrees from MIT. He's an engineer. Geeky as all get out. I don't agree with Thomas on a whole bunch of things, guns and other things. But one thing about Thomas is that he is not intimidated, and when he finds that there's something that he thinks is the right thing to do, he does it.

And so he was a perfect partner to work on this, and we treated it. We never had an opportunity actually to vote on it in the House. And I had the lawyers on my staff basically brief the parliamentarian, and we spent a month submitting briefs and cases and authority so that we knew that we had the ability to actually have our amendment heard, first time it was ever heard. And the proponents of FISA, I mean, it just went nuts, but when we were able to get it on the floor, it made sense to everybody.

I think it's a coalition of civil libertarians on the Democratic side and Tea Party constitutionalists on the Republican side. And one of the lessons I've learned is that you don't have to agree with somebody on other items. Focus on the things that you agree on and work on that and disagree on the other things, but don't let that prevent you from achieving something when you agree.

And it's really interesting because I feel like a lot of the time in these discussions about surveillance and why we worry about it, both Democrats and Republicans alike will refer to things, "Oh, I'm really worried about being tracked going to an abortion clinic," or "I'm really concerned about the government watching me and logging me going to a gun show."

Things that really are those hot-button partisan issues. But I think that there always seems to be this understanding of, "Well, we care about the broader principle for surveillance," which is just very interesting.

I think the way for some people to think about it is some of my friends on the Republican right are still enamored of Donald Trump. Maybe they don't care if Donald Trump knows everything personal about them, but pretend it's a liberal Democratic president. Do they feel the same way?

The rule of law protects everyone. It shouldn't change because of which party or which person is in office.

It's one of those you can always very easily imagine the shoe being on the other foot, even if you're not concerned about the current person in office.

So kind of a wrinkle for that aspect of it, and just the bipartisan nature of this means we've always got some members of Congress who are supporting FISA reform or are considering supporting FISA reform who are in the party opposing the current president and are especially keyed in to worry about abuse.

But of course also we've got that will flip, and then suddenly the president will be someone from their own party, and a lot of members in that, when it changes in that scenario, has said, "Well, actually I'm not that worried about abuse anymore."

Or they'll say, "Look, I care about privacy. I care about these reforms, but I really trust the current president. He's the leader of my party. I trust what he says. And he says that the law is the right balance right now and I should listen to what the FBI and the NSA and the IC agencies that he's the head of are saying about what's the right balance on FISA."

State legislatures across the United States are actively developing new laws on a range of issues related to artificial intelligence. In Virginia, that process is informed by an entity established by the Virginia General Assembly called the 

Joint Commission on Technology and Science

. To learn more about the Joint Commission and some of its 

 on AI chatbots, I spoke to its chief policy analyst, 

What follows is a lightly edited transcript of the discussion.

I'm Kira Allmann. I'm the chief policy analyst for the Joint Commission on Technology and Science in the Virginia General Assembly.

Kira, I am pleased to speak to you as a native Virginian myself to learn a little bit more about what you do, but also to learn a little bit more about your recent work on 

. And I want to talk to you a little bit about the reports that you've produced recently, but in particular, I think for my listeners, might be helpful just to describe what JCOTS is, this joint commission on technology and science and what role it plays in lawmaking in Virginia.

Sure. So JCOTS is a permanent legislative agency that produces research and analysis for legislators in the state of Virginia and the Commonwealth of Virginia. We are a research arm of the General Assembly. So a lot of what we do is we look at topics in the broad categories of technology and science that legislators are interested in, and we synthesize research that's out in the public domain. We also sometimes conduct our own original research, and then we present those findings to the General Assembly, to the Commission. And we usually include some policy considerations or in some cases policy recommendations, which are meant to be actionable options that legislators could take if they wanted to translate some of the findings into legislation during session. The Commission itself is made up of 12 elected members of the General Assembly, 7 members of the House of Delegates and 5 members of the Senate.

And do you have a particular remit as a researcher there? What is your beat?

So the full-time staff to the Commission are really just two people at the moment. There are two full-time staff members, myself, I'm the Chief Policy Analyst, as I said, and Jodi Kuhn, who's the Executive Director. So we're a pretty small team. And so I am over pretty much the entire research portfolio of the Commission. So my job is to oversee the research that we conduct, all the reports that we do. I write many of them. I don't write all of them. Jodi does some and we also have summer fellows and interns that help us with some of the research over during the interim. So my job is really to do the research piece.

, I'll include in the show notes recent reports on the subject we'll talk about today, 

. It really is a kind of laundry list of some of the more important topics in tech policy. It feels like you probably have to stay up on a lot of these issues.

Yes. A big part of the job is frankly just reading the news and staying on top of what's going on in the worlds of technology and science, which are certainly really broad domains when it comes to policy. I have to give a shout-out to you, to Tech Policy Press, you're a great resource to us. Both Jodi and I read the publications that you put out and we also listen to the podcast, so it's very exciting to be here for that reason, but we do have to keep a finger on the pulse of what's going on. And then of course we could wind up studying or looking at really any topic under those umbrellas of technology and science. So yes, it's a wide range of things.

It's an interesting challenge for someone like me. I come from an academic background where typically in academia you go deep on a particular substantive topic and you become a real expert in that. But when you're working in this policy space, you often have to wear a lot of different hats at the same time or you're juggling a lot of different topics and you're synthesizing research from a wide range of different domains. So a lot of times we'll produce one report and then we deliver that to the Commission and then we move on to potentially a completely different topic. And so I'm already writing the next report, for instance. So you've got the supplementary report on chatbots, which we're going to talk about today, but I'm already working on a report on a different topic now.

Well, let's get right into it then. And you did, as I noted, produce a report on AI chatbots last year, last summer, and then have followed up with the supplementary report on AI chatbots, companionship and minors, certainly a subject that's in the news all the time. And I will say at recent national conference of state legislatures that I attended and that I know Jodi attended as well, there was a good amount of conversation about artificial intelligence, a good amount of conversation about AI with regard to child online safety, and there were various advocacy groups and others that were there working on this particular issue. Your supplementary report is longer than the original report. Maybe that just speaks to how much curiosity and how much is going on around this, but is there a particular impetus for why you came back with the supplementary report this month? Is there a legislative priority or how are you briefed to go and do these things?

That's a great question. We get sent topics to study in a variety of different ways. In this case, we were sent 2 bills that failed in the previous General Assembly session, so that's the 2026 session. There was a House bill and a Senate bill that both dealt with companionship, chatbots and minors, and they were sent to us to study because of the fact that they failed. So usually that means that the patrons, and in this case, the Commission were interested in taking a look at the topic more broadly to see what they could explore as policy options if they reintroduce this legislation in the next session.

For a bit of context, there were a number of AI-related bills in Virginia that were put forward in the previous session. Most, if not all, I think probably all were either tabled or died in the previous session because of concerns around the federal approach to AI legislation and the potential for repercussions for states that passed what were deemed onerous AI laws. That was the language used in a White House executive order that came out in December of 2025. And one of the repercussions that was suggested was the withdrawal of BEAD funding, which is broadband funding for states. And Virginia is a big recipient of BEAD funding. So this had a dramatic chilling effect on AI legislation in the Commonwealth. So a lot of bills were kind of swept up in that chilling effect last session, and these two chatbot bills were among those bills that didn't make it.

So this is HB635, SB796 for anybody that's playing at home, I guess, that wants to maybe go look these pieces of legislation up. What do the legislators ask you to figure out if they want you to look at this topic through the lens of these failed pieces of legislation and pass? What are they trying to get you to answer?

So in this case, the remit was pretty broad. It really depends case by case on the bills that we get sent, what the key questions are. But in this case, really the interest was in how to mitigate harms that were associated with chatbots. And some of these harms have surfaced quite dramatically, I think, in public consciousness. Really in the period between when we wrote the first report, which was back in 2025, you mentioned that was kind of the original report. We call it a snapshot. That kind of report just provides a bit of an informational overview on a topic that we think the General Assembly, the Commission as part of the General Assembly might be interested in. And so at that point we thought chatbots seem to be an issue with policy relevance, so let's share some information with the General Assembly.

Since then, a lot has happened with chatbots and chatbots have become more popular. More people are familiar with them, including legislators. And there have been some high profile news stories about some tragedies associated with chatbot use, people dying by suicide, delusional spirals, things like this, which has really just catapulted the issue into public awareness. And so of course, legislators keep an eye on that. And the real question here is how do we legislate to mitigate those harms? So we had these two proposed bills and that provided a bit of a framework in terms of the kinds of policy options legislators were considering in terms of how to mitigate those harms. But as a staff, we're not necessarily constrained by what was in the previous legislation. We can look at the policy issue more broadly, see what is out there in the literature, and then come back with maybe the same kinds of recommendations, maybe similar policy options, but also potentially a broader array of policy options based on what we've read on the topic.

I want to ask you about just a framing question on what is considered a chatbot, particularly around companionship, which is the subject of the supplement. I believe you state in there at some point that it's sort of not meaningful from a policy standpoint to distinguish AI companions, which may be marketed or otherwise positioned as precisely to create relationships and just generalized chatbots like ChatGPT that maybe certainly teens, others may not necessarily see such a distinction, the consumers don't regard there to be a distinction between them.

I think the language around chatbots can get a bit confusing. This was kind of a conclusion I came to in digging into this topic a little bit more deeply for the supplement. So when we talk about companion chatbots, it's very easy to think of a particular class or use case of chatbots. And so that use case might be for emotional, intimate, mental health conversations. And there are some chatbots that are specifically marketed as companion chatbots. And so I think when that label is used, companion chatbot, we can too readily kind of reach for that class of chatbots that are specifically marketed for those uses.

But in reality, what a lot of the literature shows, in fact, frankly, what a lot of the journalism around this topic shows is that companionship, it's a much broader concept and it encompasses really any kind of emotionally charged interaction a user can have with a chatbot. And the type of chatbot or the category of chatbot, how the chatbot is described, doesn't really matter as much as how the chatbot behaves in certain situations with users.

And so that's why when I was writing the report, I really moved in the direction of talking about companionship rather than this class of companion chatbots in particular, because what seems to matter in the emerging literature on these tools is that it's really about chatbot design, chatbot behavior, and how users are interacting with these tools regardless of how they're marketed, regardless of how they're classified, regardless of how they're labeled. It's those interactions and the design features that matter most, that enable those emotional connections that policymakers are most interested in. And of course, at the end of the day, as a researcher for the legislature, I'm interested in the underlying question policymakers are most concerned with. I'm interested in how to mitigate the harms that policymakers are interested in addressing.

You admit in the report, of course, that the science is not solid on any of this. This is all new phenomena. We're just getting into these things now. It will be many years, I assume, before we know the actual implications of these technologies for people and their mental health and public health more generally. But I don't know, when you step back from the science that you've reviewed as a part of this, where do you think we're at? Are we at the very beginning fumbling around in the dark stage? Are there good instincts of where things are headed on this stuff? What would you make of the general understanding of these phenomena?

That's a really good question. I think that on the one hand, we are at a very early stage when it comes to understanding human-chatbot interactions. I'm an anthropologist, so I always approach questions about technology from the perspective that outcomes we see in society are the result of complex interactions between humans and technology. So humans shape technology, technology shapes our social lives, and that is frankly really hard to study. So from a scientific perspective, it takes a while to build up a robust scientific understanding of how technology works in interaction with people, communities, and society.

And so when it comes to chatbots, chatbots are still relatively new. ChatGPT hit the mainstream in 2022, and then there's been a proliferation of other kinds of chatbots, both standalone chatbots and also chatbots embedded in other kinds of applications. And people are really just starting to interact with them. It's only been a few years, and the impacts of those interactions are kind of just coming to the surface. And so the scientific research on this is definitely at an emerging stage.

That said, there are some elements of the dynamics that we're seeing in terms of human interactions with this technology that are familiar. They're not entirely new. And so we can look to some parallels and some analogies with other technologies. And I do talk about this in the supplementary report. I have a section on some lessons that we might learn from social media, for example. Another technology that at one point was very new and was also having some observable empirical impacts in the real world at one point. However, now we're many, many years later, policies largely had stalled on that issue. And we're realizing mostly through litigation, and there is just a recent huge settlement this past week, that in fact, many of those harms could have probably been mitigated earlier on. And so there are some lessons that we can learn from how policy dealt with social media. And in the report, I talk about three lessons in particular that we can take from social media if we want to use that as an analogous situation.

One is that voluntary commitments by companies were largely ineffective, have been largely ineffective at addressing harms.

The second is that we can't actually rely on scientific evidence alone to offer those timely policy options for addressing technological harms that people are experiencing.

And the third is really crucial, especially as a researcher, the lack of public information and data has definitely delayed interventions. And this one's really crucial because your original question was, are we really at an early stage? Well, yes, we are, but there's a bit of a problem there because the lack of information is actually potentially contributing to a delay in taking policy action. And so we need to think about how do we mitigate harms, think about policy options while also building the evidence base? Because the reality is that in the policy world, we often have to do both at the same time. If we wait for the scientific evidence to be robust, to be very well-established, the policy moment to intervene can sometimes have passed. And that's a big challenge that you have to strike the balance with when you're looking at research in a policy context.

I think those points are very important. I also feel like the issue around the kind of opacity of these systems is probably worse than the early days of social media when there was a kind of a heyday of computational social science around platforms like Twitter. Even early with Facebook, tools like CrowdTangle came along, et cetera. There was a lot of great research that was possible. And of course, many of those things have now been made much more opaque following various changes that have occurred, but the AI firms are even harder to study certainly. We are not privy often to the chat logs or other dynamics that we might want to be able to look at until maybe a piece of litigation brings them out.

Yeah, I think that's absolutely right. I think that the reality is that a lot of critical information that would be really useful for researchers is held by private companies, and there isn't access for independent oversight or independent researchers to take a look at that. Certainly in the early days of social media, there was a lot more research access, and you're right, over time that wound up being more and more restricted.

Now in the world of AI, really the public sector doesn't have a lot of visibility on some crucial information, a lot of the data, a lot of the inner workings of these technologies. And also there's a huge expertise gap. A lot of experts in these areas, computer scientists working on AI, they don't go into the public sector, they go into the private sector. And so again, that leaves the public sector a little behind the curve in terms of understanding how these technologies work. A huge amount of the knowledge is locked up in private companies, and that makes it very difficult to examine what's really going on. So this is kind of establishing just the factual basis.

And then also to test different policy solutions. It's very difficult to do that without access to the technology and some of the critical data in order to do that analysis. So I often think about this in terms of the kinds of policy recommendations that we give to legislators in Virginia, and increasingly we try to look for ways that the legislature could encourage or bolster that research side of the equation. So how can legislatures help support the development of robust research? This means transparency requirements, for instance. It could mean public funding for research on these topics. It could mean thinking about recruitment into the public sector in order to get that expertise into government so that it's also on this side of the equation and not just in the private sector. I think this is a really, really critical part of doing policy on technology right now. It goes beyond just the provisions that are there to try to mitigate harms or as it's often put, put guardrails in place. This is about contributing to robust information and evidence generation.

Well, one of the things that I do like about your supplementary report, you do start to make some frameworks, some maybe taxonomies of design features, emotional relational risks, et cetera. On this podcast, we've talked a lot about these things. We've talked about various harms of chatbots and that sort of thing. When you think though about the work that you've done on this and some of these various design features and the risks associated with them, where do you feel like researchers need to focus or policymakers need to focus most?

First of all, I really want to give a shout-out to the researchers who are doing the original research on these topics. A lot of what I put in these reports is a synthesis of the work that other researchers are doing. So thank you to the researchers who are out there collecting original data and doing that analysis so that I can read about it and synthesize it and put it in these reports for legislators. So I am not an expert necessarily on this topic. I'm a policy generalist, so that's kind of my job. So I just want to make sure that the researchers get a shout-out because it's really important.

I think that based on the emerging literature around this, the emerging scholarship around this, there are a few issues that seem to be, let's say, highly correlated with risk. One of those issues is conversation prolongation or frictionless and persistent availability. This comes out repeatedly. In fact, there have been several journal articles I've seen come out even since I released this report, since I published this report that address this particular issue. So a lot of times harms, the risk of harm increase with longer conversations, with more turn-taking between users and chatbots.

And so this seems to me like an area where there's a need for more research, there's a need for greater research focus in terms of, first of all, what are the dimensions of that turn-taking that lead to a higher likelihood of risk? We're starting to see some of this in the literature, but we need more longitudinal studies, studies over longer periods of time. We need a mix of quantitative and qualitative studies to get a better picture of what is it about longer conversations that leads to these harms.

And then we also need a better understanding of what the policy options are to mitigate them. So right now in a lot of chatbot policy, including the chatbot bills that have passed in other states, one of the biggest, well, I guess there are two big mitigating approaches to dealing with conversation prolongation. One is disclosures. So having a chatbot say either in a static disclosure or in some kind of pop-up, something like this, "I am not a human. This is a machine. This is machine generated content."

The other is to have reminders of that non-human status or break reminders that come up at various intervals. A lot of bills have a three-hour interval. This seems like a reasonable suggestion in terms of trying to get users to reflect on what they're interacting with and also how long they've been on the platform.

There is some evidence that these disclosures might not be that effective, partly because a lot of users seem to be very fully aware that they're not interacting with a chatbot. And in some cases, I remember reading one research paper where users that have this pop-up appear, it can be actually very disorienting and even distressing, emotionally distressing to a user to be interrupted in this way when they're having say an emotionally charged conversation with a chatbot. And so this can actually lead to the very kinds of harms that maybe policy is trying to address.

And so this is a good example where the question that I'm interested in here is, okay, so what would work? If not disclosures, then what kinds of mitigation approaches, mechanisms would actually help deal with the harms associated with long extended turn-taking conversations with chatbots? So that's one example, but there are a few others.

I think another interesting area is anthropomorphism enhancing design. This is where chatbots do things like use personal pronouns, I, me, we express human kinds of emotions like feeling anxiety or distress. For instance, if a user is saying they want to leave a conversation saying, "I'll miss you," things like this. And these features create an effect in human users called anthropomorphism where humans interpret the machine as though it's human. They ascribe human traits and attributes to the machine. And this is a design feature that potentially could be otherwise. And so any feature like that where you kind of think, "Well, does that have to be a feature of this particular product?" Is something where I think there's opportunity to explore what the effect of that particular design feature is on users, and then also again, what the policy options are.

For me, across all of the design features that I talk about in the report, the key question that often doesn't get looked at for quite a long time when it comes to looking at human technological interactions is what are the options for mitigating the harms here? What are the possible solutions and what actually works versus what doesn't? Policy often has to take a stab in the dark. We see a harm, we suggest a possible policy solution to that harm, but sometimes it's hard to do the follow through. Does that actually result in the outcomes, the mitigation of the harm that we want to see in the world? And that's the piece where I think that there's a big gap that researchers could fill.

Well, maybe to ask about one of those places where it feels like policy is making a shot in the dark on some of these things, much of the energy around AI chatbots is directed at protecting minors or putting in restrictions on teen or child use of these things. I think maybe ... Well, you can tell me if it's a fair reading of your report to say that you're somewhat skeptical of treating minors necessarily as a special group or that age gating is necessarily a solution when it comes to these things.

I think that's a fairly accurate representation of where we landed in the report, but I'm going to give you a very researcher response, which is it is complicated. So there are totally valid arguments that children, minors are developmentally different from adults in a variety of ways that may make them more vulnerable or susceptible to the design features associated with risks when it comes to chatbots. And so that is a serious issue that deserves policy attention.

The question, of course, when it comes to offering policy options though is the how. How do we deal with that? And that's where it gets a little bit trickier for a number of reasons. When content is age gated, regardless of the kind of technology or platform, this includes chatbots, that means there needs to be some way of ascertaining the age of users. And that can be a really tricky topic because there are a number of technical means of doing this, some of which involve actually obtaining a lot more information about minors, things like facial scans or birth dates or even inferring somebody's minor status from observing and collecting data on a user's extended interactions with a platform and then making an inference based on this. This all maximizes data collection rather than minimizes it, which could present some privacy risks. And this from a policy perspective is a trade-off.

Policymakers need to weigh whether the privacy risks are worth the potential benefit of being able to monitor minors in this way online. And I am not entirely sure from a research perspective what the best approach is. I think this is an area where we need a lot more information and a lot more research on what actually works.

I think there is another dimension to this, which is that age gating inherently relies on age thresholds. Things like everyone under a certain age can access a certain set of content. Everyone over a certain age can access different content, for instance. Or let's say it's not content, design features, access certain design features above or below a certain age threshold. But age thresholds are completely imperfect. There's nothing really that says that just because somebody turns 18, for instance, or turns 16, whatever the threshold is, they're suddenly not susceptible or vulnerable to some of these design features and the risks and harms associated with technologies. So this is a very imperfect kind of way of approaching the question of mitigating harms.

Instead, the way that we went in the report is to really focus on how making platforms safer in a number of respects for all users will also make them safer for minors. So this is kind of the idea that when you design for a particularly vulnerable or marginalized community, those design changes can often actually benefit everybody. And you can see this a little bit, you can see the evidence for this a little bit, even in just the journalism around this topic. There have been some very high profile tragedies involving minors, involving teens interacting with chatbots. But frankly, there are a lot of stories about adults having negative experiences with chatbots as well.

And so designing safer spaces for everyone will just make everybody safer. This is the kind of curb cut example in design theory where if you put in a cut in the curb so that, for instance, wheelchair users can more easily cross the road, it's also great for other users of the road, like people pushing strollers. And so it's that logic applied to a platform environment where if you do things like put in guardrails for suicide prevention, dealing with emotionally charged situations, delusional spirals, frankly, if you're thinking about that for all users, you'll also be thinking about it for minors.

Well, maybe consistent with that kind of design from the margins approach, you also talk about one of the recommendations being that you should more engage affected communities. I believe you even recommend to the legislators that there should be more active outreach to youth, youth organizations, and suggest that maybe your organization could do more to help there on participatory methods. Anybody taking you up on that? Can you imagine that we'll see some good work out of Virginia soon around how young people or others are engaging with artificial intelligence?

I'm optimistic. I'm hopeful. We put the recommendation in there for the Commission to consider and the Commission hasn't voted on these recommendations, it's worth saying. So we presented this report at our last Commission meeting, which was in August, and the vote on the recommendations will take place at our next Commission meeting, which is in October. So I really can't say at this point what combination of recommendations or policy options the Commission will be most interested in taking up. But we put that recommendation in there because of course I would be really excited to undertake that kind of work for the legislature. I think that there's a huge opportunity to do more participatory work with affected communities on these issues. I have done a bit of that in my previous work before joining the Joint Commission. And I think that it's a really important way of doing policymaking is to engage the communities that are most affected by legislation directly, so most affected by policy decisions directly.

I think that, especially when it comes to minors, there is frankly too little engagement directly with children and their parents on what the issues are that they're really experiencing and all the complexities of the good and the bad of their interactions with technology. It's very easy to focus on the bad. And frankly, that's where policy operates a lot of the time because it's when harms surface in society that policymakers really are spurred to action to mitigate those harms. It makes sense. That's a common pattern.

But the reality is we actually could benefit from knowing a lot more about the full picture. And again, this is my anthropological perspective on it. And we only do that by really engaging meaningfully with minors, with children, with kids, because the reality is that their interactions with all technologies, chatbots included, are nuanced. There are good and bad things that come from these interactions, and a lot more could be done to get minors' perspectives on what would help, for instance. I'm constantly asking the question, "Okay, we want to mitigate this harm, but how? What is the evidence for which how might be the best approach?"

Well, one way of getting at that question, not the only way, you have to triangulate with scientific evidence and all sorts of other things, but one way of getting at the answer to that question is asking affected communities, asking minors what kinds of tools they want and need, what did they want to see on these platforms in order to be able to have a healthier experience, a healthier engagement? And so I think we could really do a lot more and the literature supports that. I mean, a lot of the work on ... Really good work on children and technology repeatedly argues that we don't do enough to engage children directly in the policymaking process, asking children about children's experiences and perspectives. And so I think we could do a lot more about that and I would be excited to do some research on that for the Commission.

So I have to ask, you've made a point to say that you're drawing on the work of many other researchers and trying to synthesize, et cetera, but are you aware of other entities like yours in other states working with other legislatures that you really pay close attention to whose work you're following closely as well?

Let me say, if you're out there and you sound like JCOTS, please get in touch with us. If my colleague Jodi were here, she would say that whenever she goes to conferences across the country, she's always looking for our parallel in other states. And so far we haven't found it. That doesn't mean it doesn't exist necessarily, but JCOTS is kind of a unique institution and it's a really cool and interesting one for that reason. The fact that the Virginia legislature had the foresight to think we actually need an entity that's focusing on technology and science issues and is bringing robust evidence to the table to look at these topics from a policy perspective is really exciting and cool. And I selfishly think it's great that the Commission exists for this purpose, but we haven't found a real direct parallel in other states that does a similar thing for the legislature.

So if you know of one, let us know. I will say though that there are some other resources that we turn to quite regularly on these topics. There are other entities that we go to frequently because they're doing really interesting work. I mean, I personally am always checking the NYU Tech Policy Center. I'm looking at Data & Society. I'm looking at Tech Policy Press all the time to stay up to date on these issues and to see what researchers are working on. And so those independent research entities are incredibly valuable in this policy space. So yeah, I'd be interested to know if there's anyone else doing this work elsewhere.

And what is next for you? What's the next subject you're going to have to dig into?

So the next report that I am working on, really as we speak, I will be turning to it right after this conversation, is on agile approaches to regulating emerging technologies. So last year we presented a report to the Commission on artificial intelligence in healthcare, and one of the recommendations from that report was that it would be useful for the legislature to approach these topics in a feedback loop. So not just putting provisions in place, but also thinking about how do we evaluate the impact of those provisions, then take those evaluations and feed it back into the policy process.

Well, excitingly, the Commission was interested in that particular recommendation and wanted to know more about different approaches to being more responsive and flexible and adaptive to the policy environment around emerging technology. So the report I'm working on right now is kind of exploring agile approaches to regulation from the legislative perspective, because of course we work for the legislature and what kind of tools and mechanisms might be useful for legislatures to engage with in order to respond more effectively to emerging technology. A fast-paced, highly changeable environment, not frankly, very well suited to the pretty tedious, often plotting process of establishing regulatory rules. And so some mechanisms are things like sandboxes, sunset clauses, things like that. And so we're looking at that and exploring what we could recommend Virginia do to be a bit more responsive.

Well, Kira, Dr. Allmann, I appreciate you very much joining me to tell me a little bit more about what you do there, and I hope that someone will get in touch with you and perhaps we'll see the same type of capacity grow across the states and that maybe we'll end up with ... They always say the kind of, I guess trope is that states are the laboratories in the United States. It seems like you've built a great lab there, so I appreciate you joining me.

Thank you so much for having me. It's been a pleasure.

When ChatGPT was released in the fall of 2022, the scale of the impact it could have on K-12 schools appeared obvious. Not only could the chatbot fulfill many class and homework assignments in response to simple prompts, but children and teens had the digital fluency to adopt the technology much faster than their teachers.

The New York City Department of Education was one of the first educational bodies in the United States to act on what many perceived as the threat OpenAI’s new product posed to schooling: in January 2023, it 

 to ChatGPT on district devices and networks. The nation’s largest school district made the decision shortly after the second largest, Los Angeles Unified School District, banned the chatbot in December 2022.

Other districts across the country soon followed suit, including 

The bans didn’t last though — by May of 2023, then NYC Public Schools Chancellor David Banks was ready to “

” the potential of ChatGPT, and in August of that year LAUSD Superintendent Alberto Carvalho said his system was 

Yet this week, ahead of the start of the 2026-27 school year, these two districts are once again charting out a more restrictive approach to artificial intelligence than the 

 student-facing AI in all K-8th grade classes, while allowing a limited set of approved tools to be used for high schoolers, for the upcoming school year.

This new policy is significantly tighter than the guidelines the district had proposed in March, which instituted a stoplight system for judging AI risk that put almost all student use into the “yellow light” category, kicking use decisions to individual teachers. After intense 

, the district pushed pause on that policy with Chancellor Kamar Samuels 

 the draft “missed the mark.” The district also 

 the district was placing a one-year moratorium on generative AI use on its devices for all of its estimated 378,000 students in a move that caught even school district officials by surprise.

The district had passed restrictions on screen time for students earlier in the summer, but did not previously issue specific rules on AI.

The growing coalitions of parents, teachers, and students who have been pushing for more restrictive AI policies in schools in both cities celebrated this week’s announcements.

At the Los Angeles AI committee meeting where the ban was announced, parents with the advocacy group Schools Beyond Screens had to edit their prepared speeches in real time.

“ We are asking our local institut — no, we're thanking our local institution,” said Sandra Martinez Roe, 

 to LAist. "Thank you, LAUSD for setting an example and knowing that your obligation is to do so.”

The AI Moratorium (AIM) Coalition in NYC called the partial pause a “step in the right direction.” The coalition of organizations featuring teachers, students, and parents launched last fall has been pushing for a complete moratorium on classroom AI for two years.

“We have been overall very pleased with the steps that the mayor and the chancellor have taken around AI in schools with this new policy,” Kyle DeAngelis, a 3rd grade NYC public school teacher in Harlem and member of the AIM Coalition through a rank-and-file caucus of the United Federation of Teachers, told Tech Policy Press.

The coalition still has concerns with the policy they’d like to see addressed — including the creation of AI pilots in some schools, limited restrictions on staff use of AI, and a lack of metrics for evaluating tools — and hopes that a longer term policy is adopted through a democratic process that engages parents, teachers, and students alike.

That new policy should include more restrictions for high schoolers, says Martina Meijer, a fourth grade teacher and member of the same UFT caucus as DeAngelis.

“What we know about AI chatbots, what we know about teen mental health crises, what we know about the tendency of even adults to lean on these chatbots as a form of 

 makes it really dangerous to ignore the needs of teenagers at this time,” she told Tech Policy Press. “I would just underscore that it's really important that this policy also include high school students because they are extremely vulnerable.”

The nation’s two largest public school districts issuing such restrictions — albeit temporary and, in the case of New York, partial — on AI in classrooms is especially notable given that most school districts have previously shied away from forbidding use.

The news is also indicative of bubbling grassroots opposition to AI, which has manifested itself in widespread data center backlash but is also coming up in, for example, 

Both districts now have a year to develop longer-term policies on AI in education. Rolling back restrictions like they did with the short-lived ChatGPT bans will almost certainly be met with organized opposition.

“Over the last year, one of the things that has honestly really surprised me is just how quickly this particular movement has taken off,” DeAngelis said. “It’s reflective of what parents and teachers are seeing in their schools.”

Latest

Podcast

Recent

Our content delivered to your inbox

Thank you!

Around the globe