Journalism and AI team up to measure missing stories

Violent organized crime is one of the biggest crises facing Mexico, and it places journalists in harm’s way. Murders are a daily occurrence in many parts of the country, and research shows that Mexico is the most deadly place in the world for reporters outside of active war zones. The natural desire to avoid becoming a target has led some journalists to choose to stay quiet to save their lives.

Something akin to a code of silence has emerged across the country. We suspected that there were entire regions where journalists were not reporting on the violence, threats, intimidation and murder that were well known to be part of daily life.

We set out to measure this silence and its impact on journalism. To do so, we partnered with the Google News Initiative to use the power of machine learning and artificial intelligence to quantify and visualize news coverage and analyze the gaps in coverage across the country.

Our first step was to establish a process to determine the absence of news. We explored articles on violence to understand how they compare to the government's official registry of homicides.

In theory, each murder that occurs ought to correspond with at least one local report about the event. If we saw a divergence, or if the government's reports were suddenly very different from local news coverage, we could deduce that journalists were being silenced.

Early on, sorting through news articles seemed impossible. We knew we needed to find a news archive with the largest number of publications in Mexico possible so we could track daily coverage across the country. Google News’ vast collection of local and national news stories across Mexico was a good fit.

The effort required us to identify the difference between the number of homicides officially recorded and the news stories of those killings on Google News. This required machine learning algorithms that were able to identify the first reported story and then pinpoint where the event took place. With that information, we were able to connect reported events by media with the government's reports on homicides across more than 2400 municipalities in Mexico.

Finally, to measure the degree of silence in each region of the country, we created a formula that allows us to see the evolution of this phenomenon over time. The resulting data shows a fascinating mix of falls or peaks in unreported deaths, which coincide with events such as the arrival of new governments or the deaths of drug dealers. Further investigation will allow us to explain these connections.

At El Universal, we’re committed to continue our search for news deserts, to enhance the vitality of journalism in Mexico and draw attention to how coverage varies according to the type of crimes committed in each region, not just homicides.

This exercise is another reminder that in Mexico, as in many other countries, we cannot take freedom of the press for granted.