Gender and Racial Bias in Visual Question Answering Datasets

In this thesis, we apply the methodology suggested by Hirota, Nakashima, and Garcia (2022) to examine gender and racial biases in Visual Question Answering (VQA) datasets. Four popular VQA datasets—Visual Genome, Visual7W, VQA 2.0, and OK-VQA—are the subject of our study. Every dataset undergoes a methodical analysis aimed at identifying and measuring biases related to race and gender. To enable a focused analysis of these biases, we use a rule-based method to find samples that specifically mention gender or race. We divide questions into binary categories (men and women) for gender bias analysis based on terms that are specific to one gender over the other. Our analysis of all datasets shows that there is a notable gender representation gap, with questions about men being almost twice as common as those about women. This disparity is indicative of more general patterns found in the original COCO dataset, which provided the images for this collection. In the same way, questions mentioning race and ethnicity are identified and examined to look at instances of racial bias. Underrepresentation trends and other biases in the datasets are brought to light by the comparative analysis. Our work intends to further our knowledge of the biases present in VQA datasets and aid in the creation of AI models that are more equitable and inclusive. Our goal is to reduce negative stereotypes in AI systems and advance equity by recognizing and resolving these biases. To accomplish these objectives, this thesis emphasizes how crucial it is to critically assess AI datasets and to continuously enhance them. ⁤In this thesis, we investigate racial and gender biases in Visual Question Answering (VQA) datasets using the methods proposed by Hirota, Nakashima, and Garcia (2022). ⁤⁤We analyse four widely used VQA datasets: OK-VQA, Visual7W, VQA 2.0, and Visual Genome. ⁤⁤Each dataset is subjected to a rigorous examination with the goal of detecting and quantifying biases associated with gender and ethnicity. ⁤⁤We apply a rule-based approach to identify samples that mention gender or race specifically in order to facilitate a targeted investigation of these biases. ⁤ ⁤For gender bias analysis, we separate questions into binary groups (men and women) based on phrases that are exclusive to one gender over the other. ⁤⁤We find a significant gender representation difference in all datasets analysed, with nearly twice as many questions about males as women. ⁤⁤This discrepancy points to broader trends present in the original COCO dataset that supplied the photos for this compilation. ⁤ ⁤Similarly, inquiries pertaining to race and ethnicity are recognised and scrutinised in order to detect instances of discrimination based on race. ⁤⁤The comparative analysis reveals underrepresentation tendencies and other biases in the datasets. ⁤ In order to help make AI models more inclusive and equitable, we want to understand more about the biases present in VQA data. Our goal is to reduce negative preconceptions in AI and advance justice by identifying and addressing these prejudices. This thesis emphasises the necessity of closely examining AI data sets and continuously enhancing them in order to achieve these aims.