Tracking the blue wave: how our visual and data journalists helped you navigate the US midterm elections

Ahead of the Democrats’ ‘Blue-nami’, some of the team behind our epic results page explained the work that goes into preparing for every election night

On millions of screens around the world this week, the colour of the US electoral map digitally transposed. But behind the pixelated blue states and red states there were really 50 shades of purple.

Some midterm races for the Senate and House of Representatives were closer than others. Some swings didn’t represent an actual shift in party allegiance. The shades and nuances in each race were always going to be important this year, with expectations that many Republican supporters wouldn’t turn out for their party because of specific issues, such as inflation, fuel prices, the Iran war, datacentres, or Donald Trump’s toxicity.

That’s why our data and visual journalists spent months working on features that don’t just show you which side won each race, but also what issues might have swung it, and the demographic makeup of the swing voters.

I spoke to some of them ahead of the elections for a newsletter that was sent out to Guardian supporters before Tuesday’s blue wave. This article is adapted from that newsletter to give you a sense of the work and thought that went into the Guardian’s results pages, the main one of which has been viewed nearly 6 million times over a total of more than 100,000 combined hours of our readers’ time.

“It will likely become apparent quite quickly which party has won the Senate and the House,” said Ashley Kirk, the Guardian’s senior editor for visual journalism. “But then the questions will be: Why did they win, and who was significant in swinging the election a certain way?”

As well as a live results page, the team also built an interactive visual analysis feature to illustrate key swings that might be attributable to a specific issue or driven by a particular demographic group. “The only way we can tell these kinds of stories is by having the actual live voting data,” said Ashley, “and then trying to correlate the swings in small geographic areas, such as counties or districts, with demographic data or historical voting patterns for those areas – for instance, people’s educational, income or racial background. So, part of the work of planning and preparing for the midterms involves ensuring that we’ve got all of that live data feeding in.”

The Guardian had a team of 14 data and visual journalists working on our coverage of the midterm elections, alongside dozens of reporters and editors. The team included colleagues in New York and London engaged in a relay effort across different time zones as they monitored and supported our live results page, analysed the data feeds, produced and updated the interactive analysis feature, and worked on the visuals for the Guardian’s print editions.

“We always plan a full rota of editors, journalists and developers on the visuals team to monitor and support the pages because there are always surprises that mean that we have to do live fixes or change things,” said Ashley. “It’s live news – there are always events we need to respond to.”

The live feeds of granular data were essential for establishing the bigger picture. “Having all those live feeds so that we can spot correlations in the data means we can quickly get a finger on the pulse of what has been going on on the ground,” said Ashley. “And that will inform all of our coverage so that we’re not having to wait to see what others are reporting; we can shed our own light on things early on.”

As well as preparing the feeds, the team also prepared analysis for a range of different scenarios. “We have to think about every eventuality and prepare for it so that we can turn things around fast enough on the day,” explained Ashley. “There might be runoffs or unexpected changes in turnout, or challenges to certain race calls. Maybe only half the things we’ve prepared will end up getting published, but we have to do it this way because if we just wait to be reactive on election day, there just won’t be the capacity to publish everything in time.”

Like many other news outlets, the Guardian takes US election vote counts and race calls from the Associated Press (AP). The winners of Senate and House races typically get called well before every single vote has been counted, when it becomes clear that there are no paths for a trailing candidate to catch up. This means there’s always a time lag between a winner being declared and analysable voting data feeding through. Our journalists only published and updated the interactive analysis feature when they were confident there was enough data to bear out a trend, insight or interpretation.

The team had to navigate other time lags. For instance, votes in a state’s more densely-populated cities often take longer to count than votes in rural areas, which can sometimes lead to seemingly sudden surges in votes for the Democrats. However, these kinds of standard vote declaration lags have become highly politicised in today’s post-truth era of election denial. “We know that, in the past, Trump has cast doubt on elections because of a sudden blue surge in the vote count as more urban precincts in a state declare, and so Trump will question where those votes have come from,” explained Ashley. “So we have to make conscious design decisions on how our results page looks when a particular party is leading. So, for example, where Republicans are leading we’ll have a pink shade so it conveys a sense of uncertainty. In a time when people cast doubt on electoral processes, we have a responsibility to present this information in as clear and accessible a way as possible.”

Antonio Voce, an interactive journalist on the team, said the live results page for this year’s midterms had been honed by work on previous contests. “It’s been easier this year because we’ve built on extensive previous experience with live US election results coverage,” he said. “But we decided to introduce some visual changes and to focus on improving the technical infrastructure behind the results page. We also collaborated with our product and engineering colleagues to create new tools to facilitate the election coverage across the newsroom, such as graphics for key races that update live as the votes are counted across our news stories.”

The shape of the US presents particular challenges when it comes to making electoral maps work in different aspect ratios for viewing on a mobile phone as well as a desktop screen. Navigation functions have been developed to make it easier for readers to see the results in geographically smaller states. “That’s why we have things like search boxes, zoom functionality and cartograms that distort geographic shapes into something more meaningful and representative, for instance with every seat having the same area on the map. These functions allow people to surface their results as quickly and as easily as possible,” said Ashley.

“But while we have a tried and tested approach to results pages, it’s never a rinse and repeat process because every election has its own quirks,” he added. “For instance, there might be boundary changes or changes to the electoral voting system.”

Indeed, this year’s House elections had been overshadowed by a spate of politically motivated redistricting attempts. As our journalists explained in this interactive map of what to look out for ahead of the elections, nine states have redrawn congressional boundaries since Trump set off a redistricting war last year.

“As we use the AP live feed to visualise and determine seat flips and incumbents on the maps, decisions on how to treat redistricted seats for those purposes are delegated to them,” explained Antonio.

While the team had prepared for scenarios such as disputed results that have sadly become a feature of US elections, they hoped those aspects of their preparation wouldn’t be needed. “There’s a lot of work that goes in that you might not see – and we hope you don’t see, because it would mean that something bad is happening,” said Ashley. “But, again, we need to be prepared.”