In December 2023 I was only loosely familiar with generative artificial intelligence (AI). Like many beginners, I had dabbled with ChatGPT — using it to find recipes, write thank-you notes, or brainstorm ideas. At the time, I was four months into my first job in journalism as a digital growth strategist at Spotlight PA.
Around that time, my boss forwarded me a grant opportunity from the American Journalism Project to experiment with AI in our newsroom. We had the seed of an idea: to create an interactive, chat-based tool that could answer voters’ questions about the 2024 election in Pennsylvania, drawing on Spotlight PA’s reporting. Our goal was simple: What if we could make voting — which often feels time-consuming and confusing — easier and more accessible, providing voters with the information they need to confidently cast their ballots?
We applied, got the grant, and with little prior research but plenty of excitement, we dove into planning. Below, I’ll share what we learned, what we’d do differently if we started today, and where we hope to go next.
Play video
Video demonstrating Spotlight PA’s 2024 Election Assistant landing page interface
What we learned about AI
Spotlight PA’s 2024 Election Assistant was not an off-the-shelf chatbot. When we began brainstorming, I reached out on a whim to the team at Dewey, an AI startup known for building “personal librarian” tools for a range of clients. As a parent of two young children I was familiar with the Ask Dewey tool they built for ParentData, economist Emily Oster’s company. After several conversations, we were impressed by Dewey’s interest in and ability to set guardrails for sensitive data and their user-friendly interface that allows organizations to manage and analyze interactions easily.
Partnering with Dewey gave us the flexibility to create a more tailored tool than off-the-shelf options would allow. The collaboration let us focus on our expertise — Pennsylvania elections, reader needs, and digital growth — while bringing in external expertise in language models and product development. Over the past year we’ve gained insight from Dewey’s experience with other clients, including how to avoid common chatbot pitfalls, ensure accuracy and tone, and adapt to the constant updates in AI platforms.
Despite these advantages, ensuring the Election Assistant’s reliability and consistency was a time-consuming process. Throughout most of 2024, we rigorously tested its limits. First, we conducted internal testing among our team. Then we expanded to peers and industry groups, including the American Journalism Project’s AI cohort. Eventually we recruited Spotlight PA readers to test the tool ahead of Pennsylvania’s April primary. This testing phase was invaluable — we were pleasantly surprised by areas where live AI performed well and equally relieved that we hadn’t rushed to launch prematurely.
By summer, we shifted to serious launch planning. We partnered with the University of Pittsburgh’s Institute for Cyber Law, Policy, and Security for “red teaming,” which helped us identify vulnerabilities and standardize questions across candidates and parties to minimize bias. We also implemented a rigorous internal review process involving editors, the use of Slack alerts for high-risk answers, and a freelance fact-checker who vetted responses daily.
Ultimately, we decided against relying on live, generative AI to answer readers’ questions directly — any risk felt like too much risk given the high stakes of the election and voting information. AI was most helpful on the backend, where it identified which questions could be answered with our existing content and generated draft responses. For the user-facing tool, AI matched reader queries with pre-written, fact-checked answers and suggested follow-up questions. While this approach made the Election Assistant less dynamic than fully generative chatbots, it prioritized accuracy. Out of 3,300 answers, we had just 3 instances in which outdated information was provided.
Working with Dewey was a huge asset. Their system allowed us to track conversations, review transcripts, analyze reference materials used by the AI, and identify unanswered questions. We were able to build in tons of customized functionality from Spanish translations to alerts that notified our team of high risk responses. The only drawback working with an external vendor was that it introduced some limitations in design and user experience. More than anything, it was incredibly helpful to have a partner throughout the product development and launch that was flexible and willing to experiment alongside us.
What we learned about our newsroom
While this Election Assistant required significant effort from across our team — editors, government reporters, developers, digital growth, and team leadership — it was like a crash course in gaining familiarity with AI. It gave us a much clearer understanding of AI’s capabilities, limitations, and how it can fit into our team’s goals and processes.
Looking back, our goals were intentionally vague at the start because we weren’t sure what was achievable. However, as the project progressed, the tradeoffs became more challenging. Establishing clearer priorities for the user experience and desired outcomes earlier on would have helped us with decision making. We certainly could have figured out earlier how we wanted to engage the editorial team and fit this project into our overall editorial process.
I also think we would have benefitted from a more holistic conversation among team members about how the Election Assistant fit into our broader priorities for AI use. I think we all agreed it was a valuable learning experience, but it’s important to weigh the opportunity cost of a project like this — what other experiments or initiatives are you forgoing to pursue it?
Finally, it’s worth emphasizing how much human oversight went into the Election Assistant. Our team spent countless hours reviewing and editing answers, testing, and refining the tool. This level of involvement is why we avoided calling it a “chatbot” — it wasn’t fully automated and was heavily shaped by human input.
What we learned about our audience
One of the most valuable outcomes of this project was what we learned about the Spotlight PA audience. Between the April primary and the November general election, we answered thousands of reader questions — many of which we couldn’t have anticipated or addressed through traditional reporting alone. Even questions we couldn’t answer gave us insight into our readers’ interests. We were amazed by some of the more basic questions that came in around how to register to vote, where to vote, and what would be on the ballot. These learnings were especially important for one-time site visitors who usually leave little to no information behind. Now, we have data on their questions and the topics that resonate with them, which is already informing future reporting.
We also learned about the user experience we want to create for readers using AI tools. Setting clear expectations and providing context were essential. We published an editor’s note explaining the tool’s purpose: it wasn’t replacing our reporters’ work but rather extending its reach to more readers. We incorporated answers into the experience that emphasized our focus on accuracy over breadth of coverage and explained our focus (both for the Election Assistant and our reporting) on statewide elections.
We also learned what actually worked. While we had a dedicated landing page, most engagement came from an embed we placed within articles. Users primarily used the tool as a search function rather than for full conversations. Automated conversation starters and suggested questions drove many interactions. Moving forward, we’ll explore ways to convert these interactions into deeper engagement with articles and other content on our site.
What’s next
Was launching an AI chat tool for elections the right move? It’s hard to say. The stakes were certainly high. In many ways, I think this made our team more vigilant and determined to ensure accuracy, but it was not the easiest entry point into AI. It was a challenge to build and test in real time as the election progressed and information changed. And while we answered many questions, the Election Assistant couldn’t address the more nuanced queries posed by our most politically savvy readers.
Looking ahead, we plan to focus on areas where information doesn’t change as frequently, like election administration, voting processes, and government functions. We’re also excited to use the tool to surface new questions and content ideas, keeping readers engaged beyond their initial query.
Final thoughts
On a personal note, it’s hard to believe how different my view of AI is today in comparison to 12 months ago. The technology itself has come a long way in that time, but my exposure to what is possible and to thought leaders in the field has grown immensely. For what it’s worth, I believe our experiment was a success, and the only way we can learn new technology like AI is to try, fail, and try again.
