In many ways, newsrooms are no strangers to experimentation. They’re constantly testing their assumptions by trying different headlines, story formats and lengths, publishing times, and more. But in the hustle of daily work to put out the news, it can be difficult to design and run scientific experiments to test some of the assumptions and hunches that drive how newsrooms operate.
That’s why Joshua Darr, an associate professor at Syracuse University, created the Local News Experimental Testing Lab (Local NExT Lab), through which he worked with several newsrooms to design and carry out micro-experiments for concrete test results that could help shape their future strategy in big and small ways.
Local NExT Lab, created in partnership with the W.K. Kellogg Foundation, brings a political campaign-tested idea into local newsrooms: run small, quick experiments to see what actually works with their audiences, then adjust based on the results. These micro-experiments mirror how political campaigns constantly test messaging to see what moves voters. Darr designs each study to be statistically sound for a local or regional market so newsrooms get answers based on feedback from their own readers, in addition to results from a national pilot survey through YouGov.
The Lab just wrapped its first round of experiments with news organizations each testing a question tied to their own strategy, from story mix to social media presentation to video formats to story framing. The cohort included Mississippi Today, a statewide nonprofit newsroom; Deep South Today’s Arkansas Today, a new nonprofit outlet in the network of local newsrooms including Mississippi Today; and Gulf States Newsroom, a public media regional collaboration. (The American Journalism Project is a financial supporter of Deep South Today.)
In each of these cases, small well-designed experiments tested hypotheses, explored questions, and turned them into evidence that newsroom leaders could use to inform budgets, guide hiring decisions and rethink how they package journalism to improve impact and reach.
Lessons from the Lab
Mississippi Today
Takeaway: Investigative stories are perceived in Mississippi as more relevant, important and accurate when paired with positive community stories.
What they learned: If you put an investigative news story next to positive feature stories, readers take that investigative story more seriously than if you surround it with other hard-hitting investigations. They find it more relevant, more important and rate it higher overall. You would think stacking investigations together would make each one look more credible by association, but it’s actually the opposite. The one thing this doesn’t do is make readers think better of the news outlet as a whole: The story itself gets the boost, not the brand.
How the experiment worked: Mississippi Today had been leaning into more feel-good “community pride” stories alongside their investigative journalism, and wanted to know if that was a smart pairing. So researchers ran two national surveys (not just Mississippi readers — people across the country, shown headlines with their own state’s name swapped in) totaling 1,768 people. Each person was randomly shown a set of 5 headlines that was mostly one type: mostly community stories, mostly investigative stories or mostly legislative/political stories. It’s important to understand that everyone, no matter which set they saw, was shown the same investigative story tucked into their set. So researchers could isolate one clean question: Does changing what surrounds a story change how people judge that exact same story?
Research results: People who saw the investigative story mostly surrounded by community stories rated it 6.44 out of 10 (on a scale combining things like trustworthy, accurate, fair, relevant and important). People who saw it surrounded mostly by other investigations rated it 6.23, and surrounded by legislative stories, 6.22. That’s roughly a 0.2-point edge for the community pairing, which is statistically significant in the pooled data, and it held up even after controlling for things like readers’ politics and how attached they feel to their local community. The boost mainly showed up in these areas: Readers found the story more relevant and more important, and saw it as more accurate.
Next steps: Mississippi Today, which is already making some adjustments to newsletters to reflect the findings, plans to translate the research into practical, low-pressure action, rolling out the findings to staff and the audience team to explore story packaging and homepage changes. One idea is to pair major investigations with one lighter, community-oriented story. The findings can also help them with fundraising, particularly making the case for more dedicated community-oriented reporters.
“The research supports the idea that we want Mississippi Today to be a destination that people come to to find out what is going on in the state, not something that they only look at every three or four weeks for the big investigative pieces, but something they feel like they should check every day — or maybe two or three or four times a day,” Emily Wagster Pettus, editor-in-chief of the nonprofit outlet, told AJP.
The research supports the idea that we want Mississippi Today to be a destination that people come to to find out what is going on in the state, not something that they only look at every three or four weeks for the big investigative pieces, but something they feel like they should check every day — or maybe two or three or four times a day.
Emily Wagster Pettus, editor-in-chief of Mississippi Today
Deep South Today’s Arkansas Today
Takeaway: Social video of a story with reporters on camera makes Arkansas readers feel the issue is more important than the exact same content shown as a slideshow.
What they learned: When people watched a short social video about an AI data center story, they came away thinking the issue was more important than when they saw the exact same information laid out as a slideshow. Same facts, same wording, just a different format, and it changed how much people cared. Interestingly, it didn’t change how they rated the story itself or the news outlet, and it didn’t make them more likely to say they’d engage further. The effect was specifically about how important the issue felt.
How the experiment worked: Deep South Today is launching a new newsroom, Arkansas Today, and is trying to figure out how to invest in different kinds of content, including whether to prioritize video. So Darr designed and ran an experiment: 371 Arkansas adults, recruited through the survey firm YouGov, were randomly split into two groups. Both groups saw the identical story about AI data centers in Arkansas, but one group saw it as a short vertical video, and the other saw it as a slideshow. That’s the only thing that changed.
Research results: Readers who saw the video version rated the data center issue as meaningfully more important than those who saw the slideshow, both in terms of personal importance and importance relative to other issues facing Arkansas. The fact that the effect showed up on two separate ways of measuring “importance” strengthens confidence that this is a real pattern rather than a one-off result. The format, however, didn’t significantly change how readers evaluated the story itself or their intent to engage further.
Next steps: Deep South Today plans to turn the research into specific hiring and strategy decisions. The findings solidified their confidence in budgeting for full-time video producers across newsrooms, with active hiring already underway for a Deep South Today Studios director, a video producer and dedicated video staff for both Verité News and the upcoming Arkansas Today launch. They will still keep the cheaper carousel format as a fallback for breaking news or staffing gaps. Adam Ganucheau, Executive Editor & Chief Content Officer at Deep South Today, says they are committed to localizing on-camera talent by market so reporters reflect the communities they cover, and they are experimenting with converting video viewers into newsletter subscribers and donors.
“My hypothesis is that it’s that direct connection with the reporter or with the producer that people want,” he said. “You can see this person, they are facing you, you’re looking into their eyes. I really want to try to figure out how we can learn more about what people are thinking, because that could solve a lot of problems that have been really elusive for us journalists for a long time.”
My hypothesis is that it’s that direct connection with the reporter or with the producer that people want. You can see this person, they are facing you, you’re looking into their eyes. I really want to try to figure out how we can learn more about what people are thinking, because that could solve a lot of problems that have been really elusive for us journalists for a long time.
Adam Ganucheau, Executive Editor & Chief Content Officer at Deep South Today.
Gulf States Newsroom
Takeaway: On Instagram, the established public radio brand WWNO wins the most follows, but for getting people to engage with a post, a reporter’s face and name work best.
What they learned: When New Orleans residents were asked which Instagram profile they’d want to follow, the well-known public radio brand WWNO came out on top, with Gulf States Newsroom (GSN) doing solidly but clearly behind it. Profiles branded around an individual reporter were the least appealing choice to follow. But when people were shown a screenshot from an actual Instagram Reel, a reporter’s face paired with their name made people more likely to say they would engage with that content.
How the experiment worked: Gulf States Newsroom already puts its reporters front-and-center on Instagram — their faces, voices, bylines — while relying on public radio partner WWNO to help distribute the content. Darr ran an experiment with 391 New Orleans adults recruited through YouGov to assess following choices, and a national study with a larger sample to conduct the Reels experiment. First, in the New Orleans sample, they showed people pairs of Instagram profiles that varied by three things — who the account belonged to (GSN, WWNO or an individual reporter), how saved stories were organized (by issue, by reporter or not at all) and what kind of images were used in post previews (issue photos vs. reporter faces).
Second, in the national sample, they showed people a single Instagram Reel featuring a real GSN reporter, randomly varying whether the video showed the reporter’s face or a generic state capitol interior, and whether the on-screen label said the reporter’s name or “Gulf States Newsroom.”
Research results: In the profile-picking experiment from the New Orleans sample, WWNO was the clear favorite as a brand to follow, GSN was solidly positive but well behind WWNO, and reporter-only accounts ranked lowest of the three.
In the Reel experiment from the larger-sample national replication, however, pairing a reporter’s face with their own name drove the strongest intended engagement with that specific piece of content.
Next steps: Gulf States Newsroom plans to refine rather than overhaul their existing approach on Instagram. The finding about how reporter authenticity drives engagement more than brand affiliation confirmed what they already were seeing, so the newsroom will continue coaching reporters to be comfortable and genuine on camera. The collaborative also will expand into new content formats and partnerships, leaning into more mini-docs along with potential collaborations with content creators, while considering how to monetize the videos.
“I’m always open to trying things out,” said Ryan Vasquez, managing editor for GSN. “My goal is to make sure that we are pushing ourselves and always thinking that we have to be nimble and innovative. Otherwise, we’re going to fail. So it’s our ethos for success and viability.”
I’m always open to trying things out. My goal is to make sure that we are pushing ourselves and always thinking that we have to be nimble and innovative. Otherwise, we’re going to fail. So it’s our ethos for success and viability.
Ryan Vasquez, managing editor for GSN
What comes next
Local NExT Lab’s first cohort proved that small, rigorous experiments can give newsrooms real answers to strategic questions. AJP will be joining Syracuse University and the Kellogg Foundation and as a partner in the second cohort of the Lab, which will have eight newsrooms including four AJP portfolio organizations, with more results to share in the coming year.
Read more about all of the experiments from the first cohort, including more details on those highlighted in this post, here.
This post was authored by Mark Glaser for the American Journalism Project.