Invisible Victims: Estimating Underreporting in the Armed Conflict

Figure 2. Results of testing the cluster model in the identification of duplicate records. Prepared by HRDAG.

At the end of this process it was concluded that, of the almost 13 million initial records, 8,775,884 are unique records.

The probability of a victim being registered in several databases depends, for example, on the city in which he or she resides and whether or not he or she is a public figure. Eliminating these duplicate records is essential so that the data accurately reflect the dynamics of the conflict and do not over-represent certain groups.

¹ In this context, an "oracle" is a person or tool that, like Michelle, is responsible for assigning labels of a particular category to each observation in a database.

2. Statistical imputation of missing fields

The database of unique records generated in the previous stage contains characteristics (fields) of each documented violent event. For example, an organization to which the perpetrator belonged; gender, age, and ethnicity of the victim; date and location of the event, etc... With such a large number of records, missing fields are not a problem per se. If a missing field were random, this would not have a major impact on the analysis. However, the fact that a field is missing is related to variables such as the date of the event (because the victim registration system has changed over time) or the location (because there is a disparity in the reporting capabilities of each region).

Taking these observed variables into account corrects for bias. The researchers hypothesize that the missing values of a variable follow a pattern similar to the observed values, conditional on the other observed variables. Thus, they make use of the MAR: Missing at Random, which consists of constructing chained equations with a specification conditional on the observed data, containing information on how the variables are related —for example, the gender and age of the victim— and a random component. This equation is used to impute missing data, when available, to each record. The process is repeated multiple times to estimate the parameters of interest for each data set, and finally, the estimates are combined to generate a point prediction for the missing data, applying Rubin's combination rules. This methodology was implemented using the Predictive mean matching of the R mice package. 

The different databases contain heterogeneous information, so they are divided into base variables (those that may have missing information and are relevant to the study) and support variables. The latter are constructed using additional information from the databases. For example, the murder weapon, the sidewalk or location, the victim's profession, or the account of the event. It is precisely the account of the event, after being cleaned and lemmatized, that is used for the construction of a neural network of the Long-Short Term Memory (LSTM) type that allows estimating the probability score of a supporting variable for each category of a violent event (i.e. that it was a homicide, that it was committed by guerrilla groups, etc.). 

3. Estimation of underreporting of victims

Again, underreporting (events that are not recorded in any database) does not exist by chance, but because of structural differences in the armed conflict scenario: absence of entities to record complaints, fear of reporting due to possible threats, geographical difficulties, among others. Also, there are ethnic, geographic, or ideological groups that are systematically less likely to be registered when they experience a human rights violation.

Given the above, the researchers employed Multiple Systems Estimation (MSE) to adequately address the problem of underreporting of victims. MSE is applicable when four assumptions are met:

• The estimated population is closed. That is, once a person enters a state (e.g., homicide), he or she does not leave that state. This is reasonable because of the nature of the states considered.
• The linkage of records is accurate: this is ensured by the database processing developed in step 1.
• Independence of sources, i.e., the fact that a fact is documented in one source does not affect the probability that it is also documented in other sources. Although it is difficult for this to be true, the literature has shown that the problem is reduced when three or more bases are used. The study uses more than 100. 
• Homogeneous capture probability among observation units. That is, that all events have the same probability of being documented in each database used. As discussed, this is not realistic, so a stratification approach was applied to resolve the non-compliance with this assumption, in which subsets of the data are defined in which the condition is met. For example, stratifying by year gave good results. Each stratum functions as a proxy for the homogenization of the probability mentioned above. A type of Bayesian non-parametric ESM called "Multiple Latent Class Models for Capture-Recapture" is used to implement this solution.

This approach seeks to approximate the real size of a population (number of victims of each human rights violation) based on the patterns of documentation of the events: if an event is documented multiple times by several sources² and the above assumptions are met, the size of the underreporting is not so significant. On the other hand, if the coincidences between databases are small, the underreporting is estimated to be larger.

By applying this method, a confidence interval can be estimated for the actual number of victims for each year between 1985 and 2018. Figure 3 shows the result for the case of enforced disappearance. It is concluded, among others, that the increase in this phenomenon in the early 2000s was more pronounced than previously believed and that in 2007 there was a peak in underreporting.