Black feminism and Artificial Intelligence: the possibilities and limitations of contesting discriminatory AI from a critical social theory perspective

PMCID: PMC12355979

PMID:

Abstract

Artificial Intelligence (AI) has been challenged in recent years by critical scholars inspired by American Black feminism and questions around diversity, inclusion, and justice in and through AI systems. This paper takes a closer look at the application of Black feminism as a critical social theory, which originated as a theory protesting the oppression of Black women as a group in the United States. The paper reveals the benefits and limitations of the theory in contesting AI-based sorting, classification, personalization, recommendation, and population-based predictions of different groups of technology stakeholders. Through conceptual analysis and reference to recent use cases of AI applications, the paper showcases the promise of an intersectional-type analysis and a structural perspective enabling the in-depth analysis of technology stakeholders' experiences with AI. The paper also highlights limitations of the theory in contesting AI, which inspires a discussion of constraints on critical scholarship on AI more broadly. The papers' findings and arguments are relevant to those engaging in critical conceptual, qualitative research on the design and implications of AI-based computer systems.

Full Text

Artificial Intelligence (AI) has been subject to critical evaluation in computer science (see the field of Machine Learning Fairness, Oneto and Chiappa, 2020), but also in science and technology studies (STS), philosophy, ethics, sociology, and the humanities. AI has become an integral part of everyday life covering all aspects of society at global scale: AI may include prediction and decision-making tools as well as (generative) assistant technologies in education, hiring and recruitment, criminal justice, communication, personal (intimate) relationships, medicine, health, science, climate and all areas of technology development (Hacker, 2025). Given the permeation of AI in society in combination with concerns about bias in and through AI, critical scholarship on AI has gained traction in the humanities and social sciences (Hanemaayer, 2022). Particularly powerful critiques of AI include scholarly works inspired by US-American Black feminism (Schelenz, 2022a), a theory and practice developed by Black women in the United States to theorize and protest the oppression of Black women at the intersection of race, gender, and class (Collins, 2000a). This paper discusses an American Black feminist approach to contesting AI. The approach is interesting because it adds a structural, power-centric perspective to the debate around AI ethics. While other critical theories such as Marxism, decolonial studies or queer studies similarly consider power relations in society and technology, Black feminism is interesting because it enables an intersectional-type analysis that foreground AI's impact on technology stakeholders at the intersection of race, gender, class, sexuality, ability, and more. While a Black feminist intersectional-type approach to AI is certainly not the only feminist response to AI (see the wealth of feminist approaches in Browne et al., 2023), Black feminism is distinct because it builds on the structural experiences of Black women.
Black feminisms exist in plurality and have emerged in different regions in Africa, Latin America, the Caribbean, and Europe (Emejulu and Sobande, 2019; Norwood, 2013; Rodriguez et al., 2016). This paper focuses on US-centric Black feminism because US-based Black feminist works have increasingly targeted technology design and development as a site of critical inquiry (I will come back to that in a moment). US-American Black feminism combines theory and practice of resistance to the oppression of Black women. Intersectionality is an important analytical tool developed by Black feminists to reveal intersecting experiences of marginalization (Cooper, 2016). The work that Black feminists do is grounded in a normative concern for social justice and has driven Black women's activism around social change since U.S. slavery (Guy-Sheftall, 1996).
Research inspired by Black feminism (in the following abbreviated as BF) has challenged AI-enabled technologies in recent years. This includes critiques of algorithmic bias and discrimination (Buolamwini and Gebru, 2018; Buolamwini, 2019a; Howard and Borenstein, 2018; Noble, 2018; Hampton, 2021), critiques of color-blindness in the technology industry (Daniels, 2015), accounts of racism against Black women in computer science (Thomas et al., 2018), studies of race as a system of power and its entanglement with technology (Benjamin, 2019b), frameworks for the design of technology (Costanza-Chock, 2020; Erete et al., 2023, 2018; Kumar and Karusala, 2019), and frameworks for data collection and analysis (D'Ignazio and Klein, 2020; Marshall, 2023). BF has become an important foundation of the critical study and design of society and technology, to the extent that the concepts and ideas of BF are also employed by White women and queer or transgender scholars.
Given the increased popularity of BF in the critical study of AI-enabled technology, it is important to take a closer look at the potential and limitations of the theory. This paper therefore answers the following research questions: What tools of analysis and critical thinking can the theory and practice of Black feminism offer for the study and design of AI? What are limitations of Black feminism as a critical social theory in contesting AI? The paper focuses on the contestation of AI-based technologies that specifically work with concepts of diversity in their design (e.g., in the construction of datasets or features of an application) or that consider and cater to diverse groups of technology stakeholders. The focus on diversity and AI as the overarching use case was chosen as recent years saw not only an uptake in Black feminist critique but also more awareness of the lack of diversity in design (Zou and Schiebinger, 2018; Erete et al., 2023).
Following a conceptual analysis of BF writings and their application to STS, the paper argues that US-American Black feminism as a critical social theory offers a rich foundation for contesting AI-based technologies that are harmful to, e.g., Black women. BF is particularly suited to critically evaluating diversity-related matters of AI, among other reasons because BF has a long tradition of analyzing notions of societal difference, which increasingly inform the design of automated personalization, recommendation, adaption, and population-based predictions (e.g., intersectional-type analysis, Crenshaw, 1989; Noble, 2016). Aside from arguing for the potential of contesting AI with BF, the paper also addresses its constraints. BF shows some limitations in critically studying AI: first, while BF focuses successfully on power systems, these are mostly limited to gender, race, and class. Second, while BF-inspired analyses are excellent at making visible the lived experiences of technology stakeholders (e.g., those subjected to algorithmic mediation), the way that those experiences are present may inadvertently essentialize them.
The findings of the paper inspire a discussion of constraints on the critical study and design of AI more broadly. As stated above, BF aspires to create social change because Black women's consciousness of oppression animates their call for social justice (Collins, 2000a, p. 3). Yet when transformation toward socially just AI frameworks seems unreachable, because most critique operates from within rigid power structures that make it difficult to escape established ways of thinking and doing, where does this leave critical scholarship? In the discussion, I argue in favor of a continuum of action on AI, including structural critique that aims at transforming existing AI frameworks as well as smaller, strategic steps toward betterment.
This paper contributes to the debate around contesting harmful AI applications and effects in relation to questions of diversity by discussing the potential and limitations of BF theory as a vantage point for critique. Focusing particularly on questions of diversity means that the analysis of the theory's potential and limitations considers arguments and use cases that are related to diversity. Some use cases chosen for the analysis fall into the broader category of diversity-aware or diversity-sensitive technology. To briefly define what I mean by diversity and diversity-awareness, diversity refers to a conceptual idea of difference (e.g., among users of technology) or a normative claim toward fairness, inclusion, and justice, (e.g., of different stakeholders in the digital society, Schelenz et al., 2019). Diversity-aware technology is a technology that, in its design, draws on a concept of difference and/or a normative claim related to inclusion, representation, equality, and more (Schelenz, 2023a).
The focus on diversity-related arguments and use cases was chosen because more and more computer scientists and technology designers are interested in diversity. For instance, diversity is increasingly embedded as a concept and/or a value into algorithmic models, data collection processes, and the design of user interfaces: In human-computer interaction, diversity concepts of the user are employed to make the design of a system more responsive to the diverse needs of users (Himmelsbach et al., 2019). In AI-based recommendation systems like Netflix or Amazon, different items (products or content) to be recommended to the user are perceived as diverse, plus designers diversify recommendations to achieve fairness (Sonboli et al., 2020; Ekstrand et al., 2018) and user satisfaction (McNee et al., 2006). Furthermore, diversity is considered in balanced datasets that include information about different populations to counter data and algorithmic bias (cf. Buolamwini and Gebru, 2018; Kärkkäinen and Joo, 2019). With increased interest of designers in diversity, I observed in prior work that the way that diversity is leveraged in the technology industry often constitutes oversimplified, binary, and naturalizing operationalizations of (human) difference (Schelenz, 2022b). This is where a critique inspired by Black feminism comes in, and the potential and limitations of this critique are evaluated in the paper's analysis.
The analysis is structured as follows: First, I conducted a conceptual analysis of the key features and arguments of BF as well as its potential for producing comprehensive critiques of contemporary socio-technical practices. The conceptual analysis was conducted by reviewing writings on the theory and method of BF. The literature used for this exercise is reflected in a classic canon of Black feminist writings, represented in the anthology by Guy-Sheftall (1996). I thus relied on an older, classic body of literature represented by the likes of Kimberlé Crenshaw, Patricia Hill Collins, bell hooks, and Audré Lorde. More recent post-structuralist and queer Black feminist literature including by Jennifer Nash and Jasbir Puar was added to gain a full view of the theory's strands and evolution. The literature addressing specifically technology and AI using Black feminist argumentation includes among others Benjamin (2019a,b), Noble (2018, 2016), Marshall (2023), Buolamwini (2023).
Second, BF tools and arguments were applied to different cases or examples of AI and diversity in order to identify the benefits and limitations of the theory. For instance, one use case is the study of algorithmic bias and discrimination in the field of AI ethics, which conducts research into how social groups are treated differently through algorithmic mediation. In recent years, ethicists have pointed to numerous cases of profiling, misrepresentation, erasure and disparate treatment through AI, affecting core areas of life including education, job recruitment and hiring, and criminal justice (Heesen et al., 2021; Fabris et al., 2025). I have evaluated how BF adds to AI ethics or complements arguments of algorithmic discrimination. By way of example, my analysis determined that BF primarily promotes a structural perspective on societal relations rather than the focus on individual behavior or identity, which means that AI bias can be scrutinized from a BF perspective in a way that considers the broader social fabric. BF then allows AI-based applications to be assessed as to their impact on power relations: AI may alter and transform or reinforce and cement long-held gender and racial relations as well as injustices in society.
Another use case is the design and evaluation of AI-based technology that seeks to foster the social inclusion of migrants and refugees. In this context, diversity relates to social inclusion in the sense that migrants face different challenges than the majority society, e.g., with regard to language and access to computer systems. My own previous research has looked into how an (AI-based) chatbot can support Afghan refugee women in settling in Germany (Schelenz, 2023a). In the course of this work, I tested a prototype of the chatbot with Afghan refugee women and conducted a focus group with participants (Schelenz, 2023b). This use case was chosen to evaluate how Black feminist arguments and approaches fare in a context outside the USA, because it helps make a claim about the applicability of the theory to technology-related use cases across regions. More examples for diversity-related technology development are included along the text to sharpen arguments and help the reader understand the particularities of a Black feminist critique of AI.
An important feature of BF is its structural perspective on society and technology, which is inspired by an intersectional-type analysis. Intersectionality is one of the core products of Black feminist thought and moved into mainstream feminist politics around the 2000s (Yuval-Davis, 2006, p. 194). Established as a term by Crenshaw (1989, 1991), intersectionality reveals interlocking forms of oppression when systems of power such as race and gender interact (also see Collins, 2000a, p. 227; Collins and Bilge, 2016). Power systems are conglomerates of knowledge, narratives, and symbols that materialize in the creation and operation of institutions, policies, and practices. Power systems rely on the social construction of human difference. They structure society and technology by advancing certain norms and othering those people and practices that fall outside the norm.
Power systems then become systems of oppression for those who are deemed “Other” to the norm (Collins, 2000a, p. 4). To make explicit the hidden norms produced by power systems, BF considers the social groups that are structurally advantaged and disadvantaged in society: the norm of race is White, of gender is male, of class is middle or upper class, of sexuality is heterosexual, of ability is abled-bodied. Black women as a group are structurally disadvantaged in society and technology as they are consistently “othered,” whether as people or technology users (Noble, 2018) or designers (Twine, 2018). A structural perspective thus reveals the experiences of different groups (or “structural identities”) in society.
The emphasis of a structural perspective in BF should not dismiss individual and personal influences on stakeholders' experiences with technology. However, an individual perspective often renders invisible that individuals are influenced by larger structures that shape their individual behavior (Collins, 2000a, p. 171). For example, racism is not merely an individual attitude but promoted through institutions that affect individuals (e.g., schools, administrations, etc. Collins, 2000a, p. 227; Feagin, 2006). Race as a system of power is produced and reproduced via institutions and the collective of individuals who are socialized in said institutions to internalize a series of “knowledge” (e.g., race science, Rusert, 2017; Subramaniam, 2014), narratives (of who is considered the norm, Reddy, 1998), and symbols (skin color, bodily appearance, hair, Hooks, 2015b) that degrade Black women.
A structural perspective can also help theorize diversity and difference. Black feminists have demonstrated that Black women have structurally different experiences than White women. For example, sexual violence is experienced differently by White and Black women with Black women being turned away at the women's shelter or immigrants not finding support in their native language (Crenshaw, 1995). This is because of the co-constitution of systems of power and the intersecting oppression that Black women face (for a history of Black feminist intersectional thinking, see Brah and Phoenix, 2004). Structural difference or structural diversity is then tied to diverging experiences of oppression (and privilege, Collins, 2000a, p. 25f). The idea of structural diversity as the different experiences of groups is beneficial because it helps researchers and designers understand how AI-based technology affects people differently depending on their socio-economic-political standing in society.
In application to the study of AI-based technology, BF has the capacity to shed light on the lived experiences of technology stakeholders as they are shaped by power systems (Schelenz, 2022a). BF can reveal a technology stakeholder's position at the intersection of power systems such as gender, race, and class, and how this position materializes in specific types of oppression. For instance, Black women are impacted by design systems and institutions that have been found to exhibit bias against Black women due to an overrepresentation of White male perspectives in datasets and design teams (Criado Perez, 2019; Wachter-Boettcher, 2017, p. 20). Biased and derogatory cultural representations of Black women as “Mammies,” “Jezebels,” and “Sapphires” are disseminated at scale in AI-optimized search engines and chat bots, which act as knowledge gatekeepers and cement harmful stereotypes (Salinas et al., 2024, p. 24ff; Noble, 2018, p. 98).
Black women may further experience economic injustice as AI-based job applicant rankings can disadvantage Black women because of a combination of gender and racial biases: Hannák et al. (2017) show that users who are read as Black and female are ranked lower than candidates from other social groups including White women. As a consequence, Black female applicants do not appear on the radar of employers or they are seen as less qualified. This bias originates in ratings and reviews, as written feedback contains fewer and less positive adjectives for Black women than for other groups (Hannák et al., 2017). With the help of a structural perspective, these experiences of political, cultural, and economic marginalization entangled with the use of AI can be made visible and related to larger social and historical contexts.
Apart from hiring and recruitment, another prominent topic in AI ethics that foregrounds questions of diversity is the visual representation of humanoid AI systems. Cave and Dihal (2020) have argued that imaginations of AI in visual productions are overwhelmingly racialized as White. Humanoid robots but also chatbots carry physical features that resemble a White face, and voice assistants use middle- class “White” American English instead of African-American Vernacular English. Black feminist analyses point to the not so subtle messaging behind these interpretations of AI identity. Benjamin (2019b, p. 57) discusses an advertisement from the 1960s which shows a robot as a dehumanized servant, whereas the text reads: “Slavery will be back” and “Slavery will be here to stay.” In the past leading to the present, imaginations of robots by White people often involved a desire to dehumanize and command inferior “beings” (Benjamin, 2019b, p. 56). This resembles White attitudes during slavery, when Black women were forced to work not only in the cotton fields but also as servants in the household (Hooks, 2015a, p. 24ff). Going a step further, the Whiteness of robots may also be an intentional move toward an imagined White utopia, in which Black women as domestic workers are entirely removed from the White family's home (Cave and Dihal, 2020, p. 94; Rhee, 2018, p. 94). A critical Black feminist analysis helps reveal those links and understand historically-grown dynamics of oppression which become entangled and embedded in the design of AI systems.
Since power systems follow a similar logic (of creating a norm and an “Other”), Black feminism helps to theorize social difference across geographic and cultural contexts. Black feminists have stressed that systems of power are universal to the extent that they shape the experiences of Black women and women of color in all countries and cultures. There is an important clarification to be made here. The argument is not that experiences of marginalization are the same across geographic and cultural contexts. Rather, the systems of power (race, gender, class) shaping those experiences follow a similar logic in that they establish a norm and materialize as barriers for women's self-actualization (Dhamoon, 2015, p. 26ff).
The benefits of attending to local particularities is that power relations can be considered with regard to their materialization in a specific location. For instance, racism in Europe works differently than in the USA. It is usually expressed as cultural racism (especially against Muslim immigrants, Chin, 2017) and triggered by cultural, religious or ethnic symbols such as the hijab. Despite multicultural narratives, “white cultures are still posited as superior to other cultures and are thus normative whereas non-white cultures are doomed deviant and inferior” (Salem and Thompson, 2016, p. 13). In application to diversity and AI-based technology, this means that AI design must pay attention to how power materializes at the local level.
The case of a mobile phone application for Afghan refugee women in Germany illustrates the benefits of a BF approach (Schelenz, 2023b). According to the study results, Afghan women experience insecurity in navigating their new environment in Germany, e.g., fear of getting lost and missing an appointment at the immigration office. Initially, the experience of uncertainty in a new environment seems universal, affecting well-situated travelers and refugees alike. Yet power systems (particularly gender, ethnicity, language, nation/state) produce a specific experience: due to cultural racism in Germany, Afghan women may be hesitant to approach people in the street for help in navigating the city.
An app may then be a welcome support system but requires real-time, on-the-go assistance from volunteers or peers, simple navigation, and audio communication, according to study participants (Schelenz, 2023b, p. 18). An obstacle is that technology is often offered in English and uses the Latin alphabet, both mostly foreign to Afghan refugee women (Sabie and Ahmed, 2019, p. 223). Afghan refugee women may learn German in Germany, the German government does not offer free language classes to Afghan refugees as they do to, e.g., Syrian refugees (Weibert et al., 2019, p. 4). The modalities of interaction with AI-assisted technology are thus influenced by geopolitically situated power dynamics. BF helps reveal those dynamics and contest the broader policy shift toward digital, AI-based services by the German migration management system, which are difficult to manage for many refugees.
While Black feminism is an excellent theory to make sense of the work of power, how power relations come about, and how power systems interact, there is unequal attention to different power systems. Nash (2008, p. 9) criticizes the hegemony of gender and race in Black feminism:
Collins (2015, p. 2) suggests that the following systems of power are dominating intersectional analyses: “race, class, gender, sexuality, ethnicity, nation, ability, and age.” Lorde (1980) suggests there is an overemphasis on gender and suggests to include “race, sexual preference, class, and age” in analyzing structural difference. Yuval-Davis (2006, p. 202) cites a presentation by Helma Lutz who establishes 14 systems that structure social difference: “gender; sexuality; ‘race'/skin-color; ethnicity; nation/state; class; culture; ability; age; sedentariness/origin; wealth; North–South; religion; stage of social development.” The expansive view of Lutz points to the existence of power systems that have received little attention in Black feminism and may be more visible in post-colonial and decolonial feminisms that place emphasis on women's experience in the Global South, as immigrants, and transnational women (Dhamoon, 2015, p. 27; Khan, 2018; Mirza, 2009, p. 7).
In the use case of the mobile phone app for Afghan refugee women, language and digital literacy play a significant role (Schelenz, 2023b, p. 17). The use case showcases how (socio-technical) power relations are created through “linguistic ideologies, language policies, […] and communicative practices” (Windle et al., 2020, p. 11). The dominance of the English language has shaped the design of technology from the start. Warschauer (2004, p. 203) observes that the American Standard Code for Information Interchange based on the English and Roman languages was used in the initial stages of the development of the Internet. Engineers outside of the Western context were thus unable to participate in shaping the Internet, giving American and European designers a head start before the world standard for text was switched to Unicode (Warschauer, 2004, p. 203).
On a different level, workers in call centers in the Global South that provide technology support to American customers are encouraged to fake a White English sounding name and accent to appear White to the customer (Poster, 2019, p. 152). Not only is English the dominant language in the global digital economy, but White middle-class English is preferred over African-American Vernacular English (also in voice assistants and chatbots, Cave and Dihal, 2020, p. 690; Windle et al., 2020, p. 15). BF traditionally exhibits a blind spot with regard to language as a structure shaping power relations. This becomes a challenge for contesting AI, its economic context and impact on low-literate and non-English-speaking populations. In light of the increased use of large language models, language (bias) may shape the future of digital search, information, and communication (Bender et al., 2021).
Religion is another system of power that has received less attention in Black feminism and intersectional analysis than race and gender. As Weber (2015, p. 23) observes, “discussions of intersectionality have been hesitant to engage faith and religion, other than to occasionally list religion as one in a list of relevant differences.” Yet religion as a system structures social relations and creates racialized and gendered hierarchies through “organized institutions that produce particular norms and forms of belonging” and exclusion (Weber, 2015, p. 23). The co-constitution of race and religion is traced by Robinson (2019), who draws on Black feminist writer Wynter (2015), to the construction of the racialized human subject during modern colonial expansion. The idea of White European superiority is in large parts shaped by the transformation of the “human subject” as governed by god to the “human subject” as governed by the state. In other words, secularization established a notion of the human as subject to the state but also as “homo politicus,” who governs the state. In this shift, man suddenly enacted the role of god: the European man (in particular) gains the status of the savior who is to “maintain the stability, order, and territorial expansion of the state” (Robinson, 2019, p. 259f). Furthermore, the idea of a universal monotheistic religion (as opposed to locally specific traditions, practices, beliefs) is exported with Western imperialism, and the religious norm of (White) Christianity is installed as part of colonization and nation-state formation (Robinson, 2019, p. 259f). To theorize religion as a system of power is not to say that Christianity, Judaism, or Islam are inherently oppressive. Religion, like race and gender, are macro-structures, and the way religious norms have been established, e.g., through male interpretations of religious texts, can be reframed to align with feminist notions (Turman, 2016; Kirk-Duggan, 2014; Day, 2016).
Coming back to the use case of Afghan refugee women's interaction with AI-based technology, religion plays a role. Afghan refugee women report that they have a high priority for privacy and safety when interacting with social media (Schelenz, 2023b, p. 13). This may be motivated by religious gender norms, as interpretations of Islam shape the expectation of women's and men's behavior online as much as offline. Privacy and safety of the woman (and by extension the family) are important concerns for Muslim women (Mustafa et al., 2020, 3). Afghan women are expected not to share photographs and real names (Ahmed et al., 2022) or interact with men outside the family in social media (Shahalimi, 2022, p. 113). Yet these norms are rarely taken into account when designing technology. Instead, hegemonic Christian or secular gender norms dominate technology design. A power-centric analysis can reveal this bias and shift perspective to highlight the need for women-only online spaces, which have been used by Muslim women before (Piela, 2012). Regarding religion through the lens of power can thus be an efficient element in the toolbox to critically evaluate AI systems and their impact.
Finally, wealth as a power system has gained little attention in Black feminism vis à vis gender or race. Wealth or capital is a power system that normalizes the accumulation of wealth through the extraction of human and planetary resources. According to the logic of wealth, the redistribution or minimization of wealth is bad. Kelly (2023) calls this wealth or capital bias. Kelly argues that “wealth supremacy” is entangled with White supremacy as the exploitation of Black and Brown people serves to maximize the wealth of those who own it: White people (Kelly, 2023, p. 12; 41f). However, the way that intersectional-type analysis deals with economic inequalities is through “class,” e.g., pointing to the experiences of multiple burdens of Black working class women compared to White women or Black men (Collins, 2000b). Bender et al. (2013, p. 247) argues that class fails to capture the complex macrostructures of production and domination (“gesellschaftliche Produktions- und Herrschaftsverhältnisse,” Garske, 2013, p. 248) that shape social life.
A focus on wealth or capital can account for dynamics of AI-supported data extraction that seek to maximize financial wealth, a phenomenon that Zuboff (2019) calls “surveillance capitalism.” Surveillance capitalism uses behavioral data to know, predict, and nudge the behavior of people. Instead of producing or selling services or goods (this is a side effect), surveillance capitalism seeks maximum information (Zuboff, 2019, p. 513), and thereby maximizes wealth (advertising based on behavioral data is a source of immense income for the tech industry). Although Zuboff does not relate surveillance capitalism to capitalism's co-constitution with slavery and imperialism, the practice of massive data collection has been suggested to be violent to minorities (by “including” them under the pretext of algorithmic improvement in a harmful system of data exploitation, Hoffmann, 2020) and colonizing (by extracting data from African peoples but advancing sophisticated data protection regimes in Europe, Coleman, 2019).
As we have seen in the discussion of the benefits of BF, using an intersectional-type structural analysis and perspective can reveal technology stakeholders' particular socio-technical experiences interacting with AI. There is a significant benefit of presenting experiences rather than identities in contesting AI's impact on different groups. Focusing on identities in a social critique, also known as the additive approach of intersectionality, tends to present social identities as naturalized, implying a biological or genetic constitution of the identity group in question (Yuval-Davis, 2006, p. 199). Focusing on experiences is less essentializing in a way. This said, there has been criticism in BF itself of the way that experience may inadvertently have essentializing effects as well. Nash (2008, p. 12) argues that intersectionality's focus on experiences of oppression obscures differences within the totality of Black women's experiences; some women might experience privileges:
This becomes relevant in the realm of technology design and development as some Black women may be more privileged than others, e.g., in terms of having access to a device and the Internet. This privilege of access to technology is not self-evident for resource-constrained Black people, refugees, and people of color in some areas of the Global South (Alden, 2003; Khan, 2018). At the same time, Black women in computing—despite the multi-layered discrimination they experience (Rankin et al., 2021)—can be considered privileged in terms of education and economic standing (Liang et al., 2021, p. 28). One risk of presenting experience is thus that it does injustice to a diversity of experiences within a set of experiences.
However, more importantly for contesting AI efficiently, while experiences of privilege and oppression are influenced by power systems, the context that determines experiences (i.e., the exact work of power systems/relations) is often removed from the narration of the experience, which makes the experience appear natural (Perpich, 2010, p. 14). This is problematic because experiences become perceived as facts rather than interpretations of social phenomena that point to underlying power structures, which can be changed. Mirza (2009, p. 5) writes:
“Appeals to experience risks obscuring regimes of power by naturalizing some experiences as normative, and others as not, leaving the processes that structure dominance in tact (Scott, 1992). A black and post-colonial feminist standpoint does not valorise experience as an explanation or justification in itself, but should be seen as an interpretation of the social world that needs explaining.”
For a successful critique of problematic AI application and decision-making, the risk of essentializing experience becomes quite significant. Computer scientists who are already sensitive to human-centric and critical frameworks and engage in critical analyses of AI-based technologies increasingly emphasize the importance of accounting for structural experiences of technology stakeholders, e.g., through story-telling (Ogbonnaya-Ogburu et al., 2020), autobiography (Erete et al., 2023) or ethnography (Wong-Villacres et al., 2018). Producing narratives of technology stakeholders' structural experiences is thus becoming a valued practice in contesting AI. However, stories that reflect experiences of racism through technology and in computer science may lack an immediate analysis of how such experiences came about (see for example Ogbonnaya-Ogburu et al., 2020). This is not to say that the stories do not matter or should not be presented. Rather, following Perpich (2010, p. 18), story-telling and autobiography to contest AI should be combined with the immediate historical and political contextualization of experience. Contextualizing experience conveys the idea that experiences are not inevitable but depend on how power is constructed, organized, reproduced, challenged, changed, or abandoned in and through technology and society.
Black feminist works have been criticized for overwhelmingly relying on the language and concepts (such as race and gender) that have been established in the course of modernity to subordinate Black women (Puar, 2017, 2012; Jung and Costa Vargas, 2021; McKittrick and Wynter, 2015; McKittrick, 2021). Using those concepts in contesting AI, the critique may become less effective or accidentally reinforce problematic ideas that stem from colonial times of modernity. According to Jerath (2021, p. 32), “modernity refers to social, political, and economic conditions and experiences that result from modernization and capitalism.” Modern experience is shaped by industrialization, technological advancement, enlightenment, colonialism, slavery, the emergence of modern nation states, and the institutionalization of human rights in the USA and Europe as a response to the violent effects of modernity (Jerath, 2021). BF is tied to modernity because it has emerged in response to the brutal subordination of Black women during the transatlantic slave trade and its aftermath. BF has also used the language of rights and justice to fight for the recognition of Black women as subjects to human rights. Puar (2012, p. 54) criticizes that intersectionality draws on the language and concepts of modernity because this reinforces the systems of power that enable the subordination of Black women in the first place:
Puar thus condemns that epistemologies established by Euro-American colonialists are the foundation of Black feminist argumentation against the violence enacted by settler colonialists or their descendants. Puar calls for different epistemologies and to reject subjectivity and identity. Puar proposes queer/assemblage theory as a vantage point from which identities can be theorized as flexible and temporal events that transcend boundaries (e.g., between human-animal or nature-culture, Puar, 2012, p. 58). Queer theory, as a product of post-modern and post-structuralist thought, has made important contributions to challenging established gender identities. Judith Butler has argued that feminism works against itself by remaining within the boundaries of a “woman” category which alleges female unity and naturalizes gender (Jagose, 2007, p. 83). Simply using the term “woman” can reproduce harmful gender roles associated with it. But breaking with woman as a category and building other categories walks into the trap of defining new exclusive boundaries. “Queer” resists any definition and upholds flexibility and fluidity in social relations (Jagose, 2007, p. 98).
With regard to contesting AI, queer theory is crucial to reveal the harm experienced by technology stakeholders who defy established categories. Keyes (2019) condemns the need to identify oneself as female or male in a computerized system to gain access and receive services. But even when one is out and identified as trans or queer, this information may be used for profiling (such as targeted content, advertisement, and surveillance). Keyes (2019) describes data science as a profound threat to queer people because the reduction of complex identities to data points denies queer existence.
There have been different reactions to post-modern and post-structuralist perspectives on gender, race, and Black women's subjectivity. One line of response acknowledges that race and gender are socially conceived as part of modern discourses, but abandoning those constructs in analysis would render their effects (racism, sexism) invisible. This is expressed by Hankerson et al. (2016), when they argue that racial bias in technology has real effects on users. When technology does not work for Black people, they feel excluded, which has consequences for their psychological state. The problem is that diversity is often not considered in the creation of personae or avatars in computer games and thus the White male body may be automatically centered as the primary user of a computer system (Hankerson et al., 2016, p. 478).
Another line of response argues that leveraging problematic constructs in identity politics may be the only way to alleviate the crackdown on Black women's lives. Cooper (2016, p. 395) counters Puar's argument against subject formation by saying that it neglects the struggle of Black women in the American context, where the nation state grants rights to subjects. Only those who can establish themselves as subjects will be able to access legal protection (which, arguably, is difficult enough for Black women). Indeed, identity politics has successfully drawn attention to the discriminatory effects of contemporary AI systems. The activism of computer scientists, authors, and activists Joy Buolamwini and Timnit Gebru, who explicitly promote a Black feminist perspective while also raising awareness about the experiences of trans users of technology, have shaped the regulatory efforts around facial recognition technology (Kantayya, 2020). Buolamwini has engaged the central institutions of the American state by testifying in Congress about racial and gender bias in facial recognition technology (Buolamwini, 2019b). Such an approach formulates Black women technology stakeholders as subjects of rights and accepts the American government as a necessary point of contact in recognizing those rights. While this approach alleviates the worst discrimination, underlying ways of thinking/designing that cause systemic marginalization remain unchallenged.
Black feminism is one of the most sophisticated frameworks for social justice. Working with Black feminist theory to contest AI-based algorithmic decision-making or the design of AI-based systems and their implications for different groups of technology stakeholders has enormous potential. Yet there are limitations to challenging data science or AI-based recommendation, matching, classification, evaluation and so on. This is because most of the critique is constrained by the unequal power relations that are firmly embedded in Western language and methods, including methods of programming, designing, and classifying people into subjects of algorithmic mediation (Bonilla-Silva and Zuberi, 2008). This inspires the consideration of constraints in critical research and design more broadly and how researchers and designers can deal with these constraints. In order to successfully, efficiently and lastingly contest discriminatory AI applications, I suspect that a continuum is necessary: from structural critique that aims at the transformation of AI-provoked injustice to smaller, strategic steps toward betterment. The latter may seem unsatisfying for a critical theory but being patient and carving out opportunities within constraints can gradually improve the lived reality of marginalized technology stakeholders.
How can researchers and designers contest problematic AI use considering the constraints that they face? In the process of developing a plurality of strategies toward AI justice, it is important to recognize and name the constraints that researchers and designers are facing. For instance, one constraint is the silencing and dismissal of critical scholars in mainstream discourses, which may be motivated by racism and misogyny. A strategy of working within this constraint is to wait for changed conditions in public discourse. An example of this is the 2023 rise in concern about large language models and generative AI. Leaders in the technology industry and economy have started to warn of the risk of such technology, e.g., Geoffrey Hinton and Joseph Stiglitz (Taylor and Hern, 2023; Bushwick, 2023). Black women had warned much earlier about the issue and have been discredited for naming the dangers of “stochastic parrots” (which is the title of the famous text that played a role in the firing of Black AI ethicist Timnit Gebru from Google, see Bender et al., 2021; Turner et al., 2021). However, the fact that, in 2023, AI and economic leaders publicly voiced concern can be an opportunity for scholars working with critical approaches to reiterate their arguments and move their critique of unregulated AI into the mainstream.
Another constraint is the economic context within which critical research on AI is produced, as academic and industrial spaces are shaped by neoliberal capitalism. Given this constraint, alternative business models and infrastructures for design have been proposed (cf. Scholz and Schneider, 2017). Smyth and Dimond (2014, p. 70) propose cooperatives instead of companies to make the work environment where design is facilitated anti-oppressive so that the design can be anti-oppressive as well. Smyth and Dimond (2014, p. 71) write:
Examples of existing worker cooperatives in the technology sector are Sassafras Tech Collective and Research Action Design. In the context of online platforms, Poster (2019, p. 163) suggests “platform cooperativism” as a new model for more just online environments. They propose to look to the history of African-American cooperatives for models of platform infrastructures where ownership and governance is in users' hands (e.g., inspiration can be found in W.E.B DuBois'1907 text “Economic Cooperation among Negro Americans”). While these initiatives are promising, the reality is that most research and design takes place in contexts where neoliberal capitalism dictates design decisions.
This is not only true for big technology companies that demand the alignment of their products with company interests. Funding from governments, intergovernmental organizations (such as the European Union) and foundations can similarly put constraints on researchers and designers, who may further engage in self-censorship to avoid agitating potential funders (Wolf et al., 2022, p. 444). There is also a (perceived) need to scale new AI-based products or processes as a high adoption rate is a measure for success (Wolf et al., 2022, p. 444). Additionally, there is time pressure. Research projects are usually funded for a few years (if at all), pressuring researchers to go in and out of communities with little concern for their long-term development. Time is entangled with the “publish or perish culture” in academia where a researcher is incentivized to produce numerous written outputs in a short amount of time (cf. Erete et al., 2023, p. 26f). Little or no funding is offered to community members who participate in the research or design project (Tran O'Leary et al., 2019, p. 8).

Sections

"[{\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B74\", \"B40\", \"B42\", \"B89\", \"B18\", \"B8\"], \"section\": \"1 Introduction\", \"text\": \"Artificial Intelligence (AI) has been subject to critical evaluation in computer science (see the field of Machine Learning Fairness, Oneto and Chiappa, 2020), but also in science and technology studies (STS), philosophy, ethics, sociology, and the humanities. AI has become an integral part of everyday life covering all aspects of society at global scale: AI may include prediction and decision-making tools as well as (generative) assistant technologies in education, hiring and recruitment, criminal justice, communication, personal (intimate) relationships, medicine, health, science, climate and all areas of technology development (Hacker, 2025). Given the permeation of AI in society in combination with concerns about bias in and through AI, critical scholarship on AI has gained traction in the humanities and social sciences (Hanemaayer, 2022). Particularly powerful critiques of AI include scholarly works inspired by US-American Black feminism (Schelenz, 2022a), a theory and practice developed by Black women in the United States to theorize and protest the oppression of Black women at the intersection of race, gender, and class (Collins, 2000a). This paper discusses an American Black feminist approach to contesting AI. The approach is interesting because it adds a structural, power-centric perspective to the debate around AI ethics. While other critical theories such as Marxism, decolonial studies or queer studies similarly consider power relations in society and technology, Black feminism is interesting because it enables an intersectional-type analysis that foreground AI's impact on technology stakeholders at the intersection of race, gender, class, sexuality, ability, and more. While a Black feminist intersectional-type approach to AI is certainly not the only feminist response to AI (see the wealth of feminist approaches in Browne et al., 2023), Black feminism is distinct because it builds on the structural experiences of Black women.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B33\", \"B72\", \"B84\", \"B22\", \"B39\"], \"section\": \"1 Introduction\", \"text\": \"Black feminisms exist in plurality and have emerged in different regions in Africa, Latin America, the Caribbean, and Europe (Emejulu and Sobande, 2019; Norwood, 2013; Rodriguez et al., 2016). This paper focuses on US-centric Black feminism because US-based Black feminist works have increasingly targeted technology design and development as a site of critical inquiry (I will come back to that in a moment). US-American Black feminism combines theory and practice of resistance to the oppression of Black women. Intersectionality is an important analytical tool developed by Black feminists to reveal intersecting experiences of marginalization (Cooper, 2016). The work that Black feminists do is grounded in a normative concern for social justice and has driven Black women's activism around social change since U.S. slavery (Guy-Sheftall, 1996).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B13\", \"B10\", \"B50\", \"B71\", \"B41\", \"B28\", \"B101\", \"B5\", \"B23\", \"B35\", \"B34\", \"B60\", \"B31\", \"B63\", \"fn0001\"], \"section\": \"1 Introduction\", \"text\": \"Research inspired by Black feminism (in the following abbreviated as BF) has challenged AI-enabled technologies in recent years. This includes critiques of algorithmic bias and discrimination (Buolamwini and Gebru, 2018; Buolamwini, 2019a; Howard and Borenstein, 2018; Noble, 2018; Hampton, 2021), critiques of color-blindness in the technology industry (Daniels, 2015), accounts of racism against Black women in computer science (Thomas et al., 2018), studies of race as a system of power and its entanglement with technology (Benjamin, 2019b), frameworks for the design of technology (Costanza-Chock, 2020; Erete et al., 2023, 2018; Kumar and Karusala, 2019), and frameworks for data collection and analysis (D'Ignazio and Klein, 2020; Marshall, 2023). BF has become an important foundation of the critical study and design of society and technology, to the extent that the concepts and ideas of BF are also employed by White women and queer or transgender scholars.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B115\", \"B35\"], \"section\": \"1 Introduction\", \"text\": \"Given the increased popularity of BF in the critical study of AI-enabled technology, it is important to take a closer look at the potential and limitations of the theory. This paper therefore answers the following research questions: What tools of analysis and critical thinking can the theory and practice of Black feminism offer for the study and design of AI? What are limitations of Black feminism as a critical social theory in contesting AI? The paper focuses on the contestation of AI-based technologies that specifically work with concepts of diversity in their design (e.g., in the construction of datasets or features of an application) or that consider and cater to diverse groups of technology stakeholders. The focus on diversity and AI as the overarching use case was chosen as recent years saw not only an uptake in Black feminist critique but also more awareness of the lack of diversity in design (Zou and Schiebinger, 2018; Erete et al., 2023).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B24\", \"B70\"], \"section\": \"1 Introduction\", \"text\": \"Following a conceptual analysis of BF writings and their application to STS, the paper argues that US-American Black feminism as a critical social theory offers a rich foundation for contesting AI-based technologies that are harmful to, e.g., Black women. BF is particularly suited to critically evaluating diversity-related matters of AI, among other reasons because BF has a long tradition of analyzing notions of societal difference, which increasingly inform the design of automated personalization, recommendation, adaption, and population-based predictions (e.g., intersectional-type analysis, Crenshaw, 1989; Noble, 2016). Aside from arguing for the potential of contesting AI with BF, the paper also addresses its constraints. BF shows some limitations in critically studying AI: first, while BF focuses successfully on power systems, these are mostly limited to gender, race, and class. Second, while BF-inspired analyses are excellent at making visible the lived experiences of technology stakeholders (e.g., those subjected to algorithmic mediation), the way that those experiences are present may inadvertently essentialize them.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B18\"], \"section\": \"1 Introduction\", \"text\": \"The findings of the paper inspire a discussion of constraints on the critical study and design of AI more broadly. As stated above, BF aspires to create social change because Black women's consciousness of oppression animates their call for social justice (Collins, 2000a, p. 3). Yet when transformation toward socially just AI frameworks seems unreachable, because most critique operates from within rigid power structures that make it difficult to escape established ways of thinking and doing, where does this leave critical scholarship? In the discussion, I argue in favor of a continuum of action on AI, including structural critique that aims at transforming existing AI frameworks as well as smaller, strategic steps toward betterment.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B93\", \"B91\"], \"section\": \"2 Materials and methods\", \"text\": \"This paper contributes to the debate around contesting harmful AI applications and effects in relation to questions of diversity by discussing the potential and limitations of BF theory as a vantage point for critique. Focusing particularly on questions of diversity means that the analysis of the theory's potential and limitations considers arguments and use cases that are related to diversity. Some use cases chosen for the analysis fall into the broader category of diversity-aware or diversity-sensitive technology. To briefly define what I mean by diversity and diversity-awareness, diversity refers to a conceptual idea of difference (e.g., among users of technology) or a normative claim toward fairness, inclusion, and justice, (e.g., of different stakeholders in the digital society, Schelenz et al., 2019). Diversity-aware technology is a technology that, in its design, draws on a concept of difference and/or a normative claim related to inclusion, representation, equality, and more (Schelenz, 2023a).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B46\", \"B98\", \"B32\", \"B66\", \"B13\", \"B55\", \"B90\"], \"section\": \"2 Materials and methods\", \"text\": \"The focus on diversity-related arguments and use cases was chosen because more and more computer scientists and technology designers are interested in diversity. For instance, diversity is increasingly embedded as a concept and/or a value into algorithmic models, data collection processes, and the design of user interfaces: In human-computer interaction, diversity concepts of the user are employed to make the design of a system more responsive to the diverse needs of users (Himmelsbach et al., 2019). In AI-based recommendation systems like Netflix or Amazon, different items (products or content) to be recommended to the user are perceived as diverse, plus designers diversify recommendations to achieve fairness (Sonboli et al., 2020; Ekstrand et al., 2018) and user satisfaction (McNee et al., 2006). Furthermore, diversity is considered in balanced datasets that include information about different populations to counter data and algorithmic bias (cf. Buolamwini and Gebru, 2018; K\\u00e4rkk\\u00e4inen and Joo, 2019). With increased interest of designers in diversity, I observed in prior work that the way that diversity is leveraged in the technology industry often constitutes oversimplified, binary, and naturalizing operationalizations of (human) difference (Schelenz, 2022b). This is where a critique inspired by Black feminism comes in, and the potential and limitations of this critique are evaluated in the paper's analysis.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B39\", \"B4\", \"B5\", \"B71\", \"B70\", \"B63\", \"B12\"], \"section\": \"2 Materials and methods\", \"text\": \"The analysis is structured as follows: First, I conducted a conceptual analysis of the key features and arguments of BF as well as its potential for producing comprehensive critiques of contemporary socio-technical practices. The conceptual analysis was conducted by reviewing writings on the theory and method of BF. The literature used for this exercise is reflected in a classic canon of Black feminist writings, represented in the anthology by Guy-Sheftall (1996). I thus relied on an older, classic body of literature represented by the likes of Kimberl\\u00e9 Crenshaw, Patricia Hill Collins, bell hooks, and Audr\\u00e9 Lorde. More recent post-structuralist and queer Black feminist literature including by Jennifer Nash and Jasbir Puar was added to gain a full view of the theory's strands and evolution. The literature addressing specifically technology and AI using Black feminist argumentation includes among others Benjamin (2019a,b), Noble (2018, 2016), Marshall (2023), Buolamwini (2023).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B45\", \"B36\"], \"section\": \"2 Materials and methods\", \"text\": \"Second, BF tools and arguments were applied to different cases or examples of AI and diversity in order to identify the benefits and limitations of the theory. For instance, one use case is the study of algorithmic bias and discrimination in the field of AI ethics, which conducts research into how social groups are treated differently through algorithmic mediation. In recent years, ethicists have pointed to numerous cases of profiling, misrepresentation, erasure and disparate treatment through AI, affecting core areas of life including education, job recruitment and hiring, and criminal justice (Heesen et al., 2021; Fabris et al., 2025). I have evaluated how BF adds to AI ethics or complements arguments of algorithmic discrimination. By way of example, my analysis determined that BF primarily promotes a structural perspective on societal relations rather than the focus on individual behavior or identity, which means that AI bias can be scrutinized from a BF perspective in a way that considers the broader social fabric. BF then allows AI-based applications to be assessed as to their impact on power relations: AI may alter and transform or reinforce and cement long-held gender and racial relations as well as injustices in society.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B91\", \"B92\"], \"section\": \"2 Materials and methods\", \"text\": \"Another use case is the design and evaluation of AI-based technology that seeks to foster the social inclusion of migrants and refugees. In this context, diversity relates to social inclusion in the sense that migrants face different challenges than the majority society, e.g., with regard to language and access to computer systems. My own previous research has looked into how an (AI-based) chatbot can support Afghan refugee women in settling in Germany (Schelenz, 2023a). In the course of this work, I tested a prototype of the chatbot with Afghan refugee women and conducted a focus group with participants (Schelenz, 2023b). This use case was chosen to evaluate how Black feminist arguments and approaches fare in a context outside the USA, because it helps make a claim about the applicability of the theory to technology-related use cases across regions. More examples for diversity-related technology development are included along the text to sharpen arguments and help the reader understand the particularities of a Black feminist critique of AI.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B114\", \"B24\", \"B25\", \"B18\", \"B21\"], \"section\": \"3.1.1 Benefit 1: a structural perspective offers a multi-dimensional critical analysis of AI\", \"text\": \"An important feature of BF is its structural perspective on society and technology, which is inspired by an intersectional-type analysis. Intersectionality is one of the core products of Black feminist thought and moved into mainstream feminist politics around the 2000s (Yuval-Davis, 2006, p. 194). Established as a term by Crenshaw (1989, 1991), intersectionality reveals interlocking forms of oppression when systems of power such as race and gender interact (also see Collins, 2000a, p. 227; Collins and Bilge, 2016). Power systems are conglomerates of knowledge, narratives, and symbols that materialize in the creation and operation of institutions, policies, and practices. Power systems rely on the social construction of human difference. They structure society and technology by advancing certain norms and othering those people and practices that fall outside the norm.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B18\", \"fn0002\", \"B71\", \"B105\"], \"section\": \"3.1.1 Benefit 1: a structural perspective offers a multi-dimensional critical analysis of AI\", \"text\": \"Power systems then become systems of oppression for those who are deemed \\u201cOther\\u201d to the norm (Collins, 2000a, p. 4). To make explicit the hidden norms produced by power systems, BF considers the social groups that are structurally advantaged and disadvantaged in society: the norm of race is White, of gender is male, of class is middle or upper class, of sexuality is heterosexual, of ability is abled-bodied. Black women as a group are structurally disadvantaged in society and technology as they are consistently \\u201cothered,\\u201d whether as people or technology users (Noble, 2018) or designers (Twine, 2018). A structural perspective thus reveals the experiences of different groups (or \\u201cstructural identities\\u201d) in society.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B18\", \"B18\", \"B37\", \"B85\", \"B99\", \"B81\", \"B49\"], \"section\": \"3.1.1 Benefit 1: a structural perspective offers a multi-dimensional critical analysis of AI\", \"text\": \"The emphasis of a structural perspective in BF should not dismiss individual and personal influences on stakeholders' experiences with technology. However, an individual perspective often renders invisible that individuals are influenced by larger structures that shape their individual behavior (Collins, 2000a, p. 171). For example, racism is not merely an individual attitude but promoted through institutions that affect individuals (e.g., schools, administrations, etc. Collins, 2000a, p. 227; Feagin, 2006). Race as a system of power is produced and reproduced via institutions and the collective of individuals who are socialized in said institutions to internalize a series of \\u201cknowledge\\u201d (e.g., race science, Rusert, 2017; Subramaniam, 2014), narratives (of who is considered the norm, Reddy, 1998), and symbols (skin color, bodily appearance, hair, Hooks, 2015b) that degrade Black women.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B26\", \"B7\", \"B18\"], \"section\": \"3.1.1 Benefit 1: a structural perspective offers a multi-dimensional critical analysis of AI\", \"text\": \"A structural perspective can also help theorize diversity and difference. Black feminists have demonstrated that Black women have structurally different experiences than White women. For example, sexual violence is experienced differently by White and Black women with Black women being turned away at the women's shelter or immigrants not finding support in their native language (Crenshaw, 1995). This is because of the co-constitution of systems of power and the intersecting oppression that Black women face (for a history of Black feminist intersectional thinking, see Brah and Phoenix, 2004). Structural difference or structural diversity is then tied to diverging experiences of oppression (and privilege, Collins, 2000a, p. 25f). The idea of structural diversity as the different experiences of groups is beneficial because it helps researchers and designers understand how AI-based technology affects people differently depending on their socio-economic-political standing in society.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B89\", \"B27\", \"B106\", \"B88\", \"B71\"], \"section\": \"3.1.1 Benefit 1: a structural perspective offers a multi-dimensional critical analysis of AI\", \"text\": \"In application to the study of AI-based technology, BF has the capacity to shed light on the lived experiences of technology stakeholders as they are shaped by power systems (Schelenz, 2022a). BF can reveal a technology stakeholder's position at the intersection of power systems such as gender, race, and class, and how this position materializes in specific types of oppression. For instance, Black women are impacted by design systems and institutions that have been found to exhibit bias against Black women due to an overrepresentation of White male perspectives in datasets and design teams (Criado Perez, 2019; Wachter-Boettcher, 2017, p. 20). Biased and derogatory cultural representations of Black women as \\u201cMammies,\\u201d \\u201cJezebels,\\u201d and \\u201cSapphires\\u201d are disseminated at scale in AI-optimized search engines and chat bots, which act as knowledge gatekeepers and cement harmful stereotypes (Salinas et al., 2024, p. 24ff; Noble, 2018, p. 98).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B44\", \"B44\"], \"section\": \"3.1.1 Benefit 1: a structural perspective offers a multi-dimensional critical analysis of AI\", \"text\": \"Black women may further experience economic injustice as AI-based job applicant rankings can disadvantage Black women because of a combination of gender and racial biases: Hann\\u00e1k et al. (2017) show that users who are read as Black and female are ranked lower than candidates from other social groups including White women. As a consequence, Black female applicants do not appear on the radar of employers or they are seen as less qualified. This bias originates in ratings and reviews, as written feedback contains fewer and less positive adjectives for Black women than for other groups (Hann\\u00e1k et al., 2017). With the help of a structural perspective, these experiences of political, cultural, and economic marginalization entangled with the use of AI can be made visible and related to larger social and historical contexts.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B15\", \"B5\", \"B5\", \"B48\", \"B15\", \"B82\"], \"section\": \"3.1.1 Benefit 1: a structural perspective offers a multi-dimensional critical analysis of AI\", \"text\": \"Apart from hiring and recruitment, another prominent topic in AI ethics that foregrounds questions of diversity is the visual representation of humanoid AI systems. Cave and Dihal (2020) have argued that imaginations of AI in visual productions are overwhelmingly racialized as White. Humanoid robots but also chatbots carry physical features that resemble a White face, and voice assistants use middle- class \\u201cWhite\\u201d American English instead of African-American Vernacular English. Black feminist analyses point to the not so subtle messaging behind these interpretations of AI identity. Benjamin (2019b, p. 57) discusses an advertisement from the 1960s which shows a robot as a dehumanized servant, whereas the text reads: \\u201cSlavery will be back\\u201d and \\u201cSlavery will be here to stay.\\u201d In the past leading to the present, imaginations of robots by White people often involved a desire to dehumanize and command inferior \\u201cbeings\\u201d (Benjamin, 2019b, p. 56). This resembles White attitudes during slavery, when Black women were forced to work not only in the cotton fields but also as servants in the household (Hooks, 2015a, p. 24ff). Going a step further, the Whiteness of robots may also be an intentional move toward an imagined White utopia, in which Black women as domestic workers are entirely removed from the White family's home (Cave and Dihal, 2020, p. 94; Rhee, 2018, p. 94). A critical Black feminist analysis helps reveal those links and understand historically-grown dynamics of oppression which become entangled and embedded in the design of AI systems.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B30\"], \"section\": \"3.1.2 Benefit 2: Black feminist tools of analysis can be applied across regional and cultural contexts\", \"text\": \"Since power systems follow a similar logic (of creating a norm and an \\u201cOther\\u201d), Black feminism helps to theorize social difference across geographic and cultural contexts. Black feminists have stressed that systems of power are universal to the extent that they shape the experiences of Black women and women of color in all countries and cultures. There is an important clarification to be made here. The argument is not that experiences of marginalization are the same across geographic and cultural contexts. Rather, the systems of power (race, gender, class) shaping those experiences follow a similar logic in that they establish a norm and materialize as barriers for women's self-actualization (Dhamoon, 2015, p. 26ff).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B16\", \"B87\"], \"section\": \"3.1.2 Benefit 2: Black feminist tools of analysis can be applied across regional and cultural contexts\", \"text\": \"The benefits of attending to local particularities is that power relations can be considered with regard to their materialization in a specific location. For instance, racism in Europe works differently than in the USA. It is usually expressed as cultural racism (especially against Muslim immigrants, Chin, 2017) and triggered by cultural, religious or ethnic symbols such as the hijab. Despite multicultural narratives, \\u201cwhite cultures are still posited as superior to other cultures and are thus normative whereas non-white cultures are doomed deviant and inferior\\u201d (Salem and Thompson, 2016, p. 13). In application to diversity and AI-based technology, this means that AI design must pay attention to how power materializes at the local level.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B92\"], \"section\": \"3.1.2 Benefit 2: Black feminist tools of analysis can be applied across regional and cultural contexts\", \"text\": \"The case of a mobile phone application for Afghan refugee women in Germany illustrates the benefits of a BF approach (Schelenz, 2023b). According to the study results, Afghan women experience insecurity in navigating their new environment in Germany, e.g., fear of getting lost and missing an appointment at the immigration office. Initially, the experience of uncertainty in a new environment seems universal, affecting well-situated travelers and refugees alike. Yet power systems (particularly gender, ethnicity, language, nation/state) produce a specific experience: due to cultural racism in Germany, Afghan women may be hesitant to approach people in the street for help in navigating the city.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B92\", \"B86\", \"B109\"], \"section\": \"3.1.2 Benefit 2: Black feminist tools of analysis can be applied across regional and cultural contexts\", \"text\": \"An app may then be a welcome support system but requires real-time, on-the-go assistance from volunteers or peers, simple navigation, and audio communication, according to study participants (Schelenz, 2023b, p. 18). An obstacle is that technology is often offered in English and uses the Latin alphabet, both mostly foreign to Afghan refugee women (Sabie and Ahmed, 2019, p. 223). Afghan refugee women may learn German in Germany, the German government does not offer free language classes to Afghan refugees as they do to, e.g., Syrian refugees (Weibert et al., 2019, p. 4). The modalities of interaction with AI-assisted technology are thus influenced by geopolitically situated power dynamics. BF helps reveal those dynamics and contest the broader policy shift toward digital, AI-based services by the German migration management system, which are difficult to manage for many refugees.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B69\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"While Black feminism is an excellent theory to make sense of the work of power, how power relations come about, and how power systems interact, there is unequal attention to different power systems. Nash (2008, p. 9) criticizes the hegemony of gender and race in Black feminism:\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B20\", \"B62\", \"B114\", \"B30\", \"B58\", \"B67\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"Collins (2015, p. 2) suggests that the following systems of power are dominating intersectional analyses: \\u201crace, class, gender, sexuality, ethnicity, nation, ability, and age.\\u201d Lorde (1980) suggests there is an overemphasis on gender and suggests to include \\u201crace, sexual preference, class, and age\\u201d in analyzing structural difference. Yuval-Davis (2006, p. 202) cites a presentation by Helma Lutz who establishes 14 systems that structure social difference: \\u201cgender; sexuality; \\u2018race'/skin-color; ethnicity; nation/state; class; culture; ability; age; sedentariness/origin; wealth; North\\u2013South; religion; stage of social development.\\u201d The expansive view of Lutz points to the existence of power systems that have received little attention in Black feminism and may be more visible in post-colonial and decolonial feminisms that place emphasis on women's experience in the Global South, as immigrants, and transnational women (Dhamoon, 2015, p. 27; Khan, 2018; Mirza, 2009, p. 7).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B92\", \"B110\", \"B107\", \"B107\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"In the use case of the mobile phone app for Afghan refugee women, language and digital literacy play a significant role (Schelenz, 2023b, p. 17). The use case showcases how (socio-technical) power relations are created through \\u201clinguistic ideologies, language policies, [\\u2026] and communicative practices\\u201d (Windle et al., 2020, p. 11). The dominance of the English language has shaped the design of technology from the start. Warschauer (2004, p. 203) observes that the American Standard Code for Information Interchange based on the English and Roman languages was used in the initial stages of the development of the Internet. Engineers outside of the Western context were thus unable to participate in shaping the Internet, giving American and European designers a head start before the world standard for text was switched to Unicode (Warschauer, 2004, p. 203).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B77\", \"B15\", \"B110\", \"B3\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"On a different level, workers in call centers in the Global South that provide technology support to American customers are encouraged to fake a White English sounding name and accent to appear White to the customer (Poster, 2019, p. 152). Not only is English the dominant language in the global digital economy, but White middle-class English is preferred over African-American Vernacular English (also in voice assistants and chatbots, Cave and Dihal, 2020, p. 690; Windle et al., 2020, p. 15). BF traditionally exhibits a blind spot with regard to language as a structure shaping power relations. This becomes a challenge for contesting AI, its economic context and impact on low-literate and non-English-speaking populations. In light of the increased use of large language models, language (bias) may shape the future of digital search, information, and communication (Bender et al., 2021).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B108\", \"B108\", \"B83\", \"B113\", \"B83\", \"B83\", \"B103\", \"B59\", \"B29\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"Religion is another system of power that has received less attention in Black feminism and intersectional analysis than race and gender. As Weber (2015, p. 23) observes, \\u201cdiscussions of intersectionality have been hesitant to engage faith and religion, other than to occasionally list religion as one in a list of relevant differences.\\u201d Yet religion as a system structures social relations and creates racialized and gendered hierarchies through \\u201corganized institutions that produce particular norms and forms of belonging\\u201d and exclusion (Weber, 2015, p. 23). The co-constitution of race and religion is traced by Robinson (2019), who draws on Black feminist writer Wynter (2015), to the construction of the racialized human subject during modern colonial expansion. The idea of White European superiority is in large parts shaped by the transformation of the \\u201chuman subject\\u201d as governed by god to the \\u201chuman subject\\u201d as governed by the state. In other words, secularization established a notion of the human as subject to the state but also as \\u201chomo politicus,\\u201d who governs the state. In this shift, man suddenly enacted the role of god: the European man (in particular) gains the status of the savior who is to \\u201cmaintain the stability, order, and territorial expansion of the state\\u201d (Robinson, 2019, p. 259f). Furthermore, the idea of a universal monotheistic religion (as opposed to locally specific traditions, practices, beliefs) is exported with Western imperialism, and the religious norm of (White) Christianity is installed as part of colonization and nation-state formation (Robinson, 2019, p. 259f). To theorize religion as a system of power is not to say that Christianity, Judaism, or Islam are inherently oppressive. Religion, like race and gender, are macro-structures, and the way religious norms have been established, e.g., through male interpretations of religious texts, can be reframed to align with feminist notions (Turman, 2016; Kirk-Duggan, 2014; Day, 2016).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B92\", \"B68\", \"B1\", \"B96\", \"B76\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"Coming back to the use case of Afghan refugee women's interaction with AI-based technology, religion plays a role. Afghan refugee women report that they have a high priority for privacy and safety when interacting with social media (Schelenz, 2023b, p. 13). This may be motivated by religious gender norms, as interpretations of Islam shape the expectation of women's and men's behavior online as much as offline. Privacy and safety of the woman (and by extension the family) are important concerns for Muslim women (Mustafa et al., 2020, 3). Afghan women are expected not to share photographs and real names (Ahmed et al., 2022) or interact with men outside the family in social media (Shahalimi, 2022, p. 113). Yet these norms are rarely taken into account when designing technology. Instead, hegemonic Christian or secular gender norms dominate technology design. A power-centric analysis can reveal this bias and shift perspective to highlight the need for women-only online spaces, which have been used by Muslim women before (Piela, 2012). Regarding religion through the lens of power can thus be an efficient element in the toolbox to critically evaluate AI systems and their impact.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B56\", \"B56\", \"B19\", \"B38\", \"B38\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"Finally, wealth as a power system has gained little attention in Black feminism vis \\u00e0 vis gender or race. Wealth or capital is a power system that normalizes the accumulation of wealth through the extraction of human and planetary resources. According to the logic of wealth, the redistribution or minimization of wealth is bad. Kelly (2023) calls this wealth or capital bias. Kelly argues that \\u201cwealth supremacy\\u201d is entangled with White supremacy as the exploitation of Black and Brown people serves to maximize the wealth of those who own it: White people (Kelly, 2023, p. 12; 41f). However, the way that intersectional-type analysis deals with economic inequalities is through \\u201cclass,\\u201d e.g., pointing to the experiences of multiple burdens of Black working class women compared to White women or Black men (Collins, 2000b). Bender et al. (2013, p. 247) argues that class fails to capture the complex macrostructures of production and domination (\\u201cgesellschaftliche Produktions- und Herrschaftsverh\\u00e4ltnisse,\\u201d Garske, 2013, p. 248) that shape social life.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B116\", \"B116\", \"B47\", \"B17\"], \"section\": \"3.2.1 Limitation 1: BF pays little attention to power systems like language, religion, and wealth but they are important elements of a critical analysis of AI\", \"text\": \"A focus on wealth or capital can account for dynamics of AI-supported data extraction that seek to maximize financial wealth, a phenomenon that Zuboff (2019) calls \\u201csurveillance capitalism.\\u201d Surveillance capitalism uses behavioral data to know, predict, and nudge the behavior of people. Instead of producing or selling services or goods (this is a side effect), surveillance capitalism seeks maximum information (Zuboff, 2019, p. 513), and thereby maximizes wealth (advertising based on behavioral data is a source of immense income for the tech industry). Although Zuboff does not relate surveillance capitalism to capitalism's co-constitution with slavery and imperialism, the practice of massive data collection has been suggested to be violent to minorities (by \\u201cincluding\\u201d them under the pretext of algorithmic improvement in a harmful system of data exploitation, Hoffmann, 2020) and colonizing (by extracting data from African peoples but advancing sophisticated data protection regimes in Europe, Coleman, 2019).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B114\", \"B69\"], \"section\": \"3.2.2 Limitation 2: BF-inspired analyses may inadvertently essentialize experiences of technology stakeholders, making it difficult to contest AI in a way that gets to the origins of the discontent\", \"text\": \"As we have seen in the discussion of the benefits of BF, using an intersectional-type structural analysis and perspective can reveal technology stakeholders' particular socio-technical experiences interacting with AI. There is a significant benefit of presenting experiences rather than identities in contesting AI's impact on different groups. Focusing on identities in a social critique, also known as the additive approach of intersectionality, tends to present social identities as naturalized, implying a biological or genetic constitution of the identity group in question (Yuval-Davis, 2006, p. 199). Focusing on experiences is less essentializing in a way. This said, there has been criticism in BF itself of the way that experience may inadvertently have essentializing effects as well. Nash (2008, p. 12) argues that intersectionality's focus on experiences of oppression obscures differences within the totality of Black women's experiences; some women might experience privileges:\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B2\", \"B58\", \"B80\", \"B61\"], \"section\": \"3.2.2 Limitation 2: BF-inspired analyses may inadvertently essentialize experiences of technology stakeholders, making it difficult to contest AI in a way that gets to the origins of the discontent\", \"text\": \"This becomes relevant in the realm of technology design and development as some Black women may be more privileged than others, e.g., in terms of having access to a device and the Internet. This privilege of access to technology is not self-evident for resource-constrained Black people, refugees, and people of color in some areas of the Global South (Alden, 2003; Khan, 2018). At the same time, Black women in computing\\u2014despite the multi-layered discrimination they experience (Rankin et al., 2021)\\u2014can be considered privileged in terms of education and economic standing (Liang et al., 2021, p. 28). One risk of presenting experience is thus that it does injustice to a diversity of experiences within a set of experiences.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B75\", \"B67\"], \"section\": \"3.2.2 Limitation 2: BF-inspired analyses may inadvertently essentialize experiences of technology stakeholders, making it difficult to contest AI in a way that gets to the origins of the discontent\", \"text\": \"However, more importantly for contesting AI efficiently, while experiences of privilege and oppression are influenced by power systems, the context that determines experiences (i.e., the exact work of power systems/relations) is often removed from the narration of the experience, which makes the experience appear natural (Perpich, 2010, p. 14). This is problematic because experiences become perceived as facts rather than interpretations of social phenomena that point to underlying power structures, which can be changed. Mirza (2009, p. 5) writes:\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B95\"], \"section\": \"\", \"text\": \"\\u201cAppeals to experience risks obscuring regimes of power by naturalizing some experiences as normative, and others as not, leaving the processes that structure dominance in tact (Scott, 1992). A black and post-colonial feminist standpoint does not valorise experience as an explanation or justification in itself, but should be seen as an interpretation of the social world that needs explaining.\\u201d\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B73\", \"B35\", \"B112\", \"B73\", \"B75\"], \"section\": \"3.2.2 Limitation 2: BF-inspired analyses may inadvertently essentialize experiences of technology stakeholders, making it difficult to contest AI in a way that gets to the origins of the discontent\", \"text\": \"For a successful critique of problematic AI application and decision-making, the risk of essentializing experience becomes quite significant. Computer scientists who are already sensitive to human-centric and critical frameworks and engage in critical analyses of AI-based technologies increasingly emphasize the importance of accounting for structural experiences of technology stakeholders, e.g., through story-telling (Ogbonnaya-Ogburu et al., 2020), autobiography (Erete et al., 2023) or ethnography (Wong-Villacres et al., 2018). Producing narratives of technology stakeholders' structural experiences is thus becoming a valued practice in contesting AI. However, stories that reflect experiences of racism through technology and in computer science may lack an immediate analysis of how such experiences came about (see for example Ogbonnaya-Ogburu et al., 2020). This is not to say that the stories do not matter or should not be presented. Rather, following Perpich (2010, p. 18), story-telling and autobiography to contest AI should be combined with the immediate historical and political contextualization of experience. Contextualizing experience conveys the idea that experiences are not inevitable but depend on how power is constructed, organized, reproduced, challenged, changed, or abandoned in and through technology and society.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B79\", \"B78\", \"B53\", \"B65\", \"B64\", \"B52\", \"B52\", \"B78\"], \"section\": \"3.2.3 Limitation 3: BF draws on modern Western concepts such as race and gender, thus contesting AI-based services for using related concepts appears inconsistent\", \"text\": \"Black feminist works have been criticized for overwhelmingly relying on the language and concepts (such as race and gender) that have been established in the course of modernity to subordinate Black women (Puar, 2017, 2012; Jung and Costa Vargas, 2021; McKittrick and Wynter, 2015; McKittrick, 2021). Using those concepts in contesting AI, the critique may become less effective or accidentally reinforce problematic ideas that stem from colonial times of modernity. According to Jerath (2021, p. 32), \\u201cmodernity refers to social, political, and economic conditions and experiences that result from modernization and capitalism.\\u201d Modern experience is shaped by industrialization, technological advancement, enlightenment, colonialism, slavery, the emergence of modern nation states, and the institutionalization of human rights in the USA and Europe as a response to the violent effects of modernity (Jerath, 2021). BF is tied to modernity because it has emerged in response to the brutal subordination of Black women during the transatlantic slave trade and its aftermath. BF has also used the language of rights and justice to fight for the recognition of Black women as subjects to human rights. Puar (2012, p. 54) criticizes that intersectionality draws on the language and concepts of modernity because this reinforces the systems of power that enable the subordination of Black women in the first place:\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B78\", \"B51\", \"B51\"], \"section\": \"3.2.3 Limitation 3: BF draws on modern Western concepts such as race and gender, thus contesting AI-based services for using related concepts appears inconsistent\", \"text\": \"Puar thus condemns that epistemologies established by Euro-American colonialists are the foundation of Black feminist argumentation against the violence enacted by settler colonialists or their descendants. Puar calls for different epistemologies and to reject subjectivity and identity. Puar proposes queer/assemblage theory as a vantage point from which identities can be theorized as flexible and temporal events that transcend boundaries (e.g., between human-animal or nature-culture, Puar, 2012, p. 58). Queer theory, as a product of post-modern and post-structuralist thought, has made important contributions to challenging established gender identities. Judith Butler has argued that feminism works against itself by remaining within the boundaries of a \\u201cwoman\\u201d category which alleges female unity and naturalizes gender (Jagose, 2007, p. 83). Simply using the term \\u201cwoman\\u201d can reproduce harmful gender roles associated with it. But breaking with woman as a category and building other categories walks into the trap of defining new exclusive boundaries. \\u201cQueer\\u201d resists any definition and upholds flexibility and fluidity in social relations (Jagose, 2007, p. 98).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B57\", \"B57\"], \"section\": \"3.2.3 Limitation 3: BF draws on modern Western concepts such as race and gender, thus contesting AI-based services for using related concepts appears inconsistent\", \"text\": \"With regard to contesting AI, queer theory is crucial to reveal the harm experienced by technology stakeholders who defy established categories. Keyes (2019) condemns the need to identify oneself as female or male in a computerized system to gain access and receive services. But even when one is out and identified as trans or queer, this information may be used for profiling (such as targeted content, advertisement, and surveillance). Keyes (2019) describes data science as a profound threat to queer people because the reduction of complex identities to data points denies queer existence.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B43\", \"B43\"], \"section\": \"3.2.3 Limitation 3: BF draws on modern Western concepts such as race and gender, thus contesting AI-based services for using related concepts appears inconsistent\", \"text\": \"There have been different reactions to post-modern and post-structuralist perspectives on gender, race, and Black women's subjectivity. One line of response acknowledges that race and gender are socially conceived as part of modern discourses, but abandoning those constructs in analysis would render their effects (racism, sexism) invisible. This is expressed by Hankerson et al. (2016), when they argue that racial bias in technology has real effects on users. When technology does not work for Black people, they feel excluded, which has consequences for their psychological state. The problem is that diversity is often not considered in the creation of personae or avatars in computer games and thus the White male body may be automatically centered as the primary user of a computer system (Hankerson et al., 2016, p. 478).\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B22\", \"B54\", \"B11\"], \"section\": \"3.2.3 Limitation 3: BF draws on modern Western concepts such as race and gender, thus contesting AI-based services for using related concepts appears inconsistent\", \"text\": \"Another line of response argues that leveraging problematic constructs in identity politics may be the only way to alleviate the crackdown on Black women's lives. Cooper (2016, p. 395) counters Puar's argument against subject formation by saying that it neglects the struggle of Black women in the American context, where the nation state grants rights to subjects. Only those who can establish themselves as subjects will be able to access legal protection (which, arguably, is difficult enough for Black women). Indeed, identity politics has successfully drawn attention to the discriminatory effects of contemporary AI systems. The activism of computer scientists, authors, and activists Joy Buolamwini and Timnit Gebru, who explicitly promote a Black feminist perspective while also raising awareness about the experiences of trans users of technology, have shaped the regulatory efforts around facial recognition technology (Kantayya, 2020). Buolamwini has engaged the central institutions of the American state by testifying in Congress about racial and gender bias in facial recognition technology (Buolamwini, 2019b). Such an approach formulates Black women technology stakeholders as subjects of rights and accepts the American government as a necessary point of contact in recognizing those rights. While this approach alleviates the worst discrimination, underlying ways of thinking/designing that cause systemic marginalization remain unchallenged.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B6\"], \"section\": \"4 Discussion\", \"text\": \"Black feminism is one of the most sophisticated frameworks for social justice. Working with Black feminist theory to contest AI-based algorithmic decision-making or the design of AI-based systems and their implications for different groups of technology stakeholders has enormous potential. Yet there are limitations to challenging data science or AI-based recommendation, matching, classification, evaluation and so on. This is because most of the critique is constrained by the unequal power relations that are firmly embedded in Western language and methods, including methods of programming, designing, and classifying people into subjects of algorithmic mediation (Bonilla-Silva and Zuberi, 2008). This inspires the consideration of constraints in critical research and design more broadly and how researchers and designers can deal with these constraints. In order to successfully, efficiently and lastingly contest discriminatory AI applications, I suspect that a continuum is necessary: from structural critique that aims at the transformation of AI-provoked injustice to smaller, strategic steps toward betterment. The latter may seem unsatisfying for a critical theory but being patient and carving out opportunities within constraints can gradually improve the lived reality of marginalized technology stakeholders.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B100\", \"B14\", \"B3\", \"B104\"], \"section\": \"4 Discussion\", \"text\": \"How can researchers and designers contest problematic AI use considering the constraints that they face? In the process of developing a plurality of strategies toward AI justice, it is important to recognize and name the constraints that researchers and designers are facing. For instance, one constraint is the silencing and dismissal of critical scholars in mainstream discourses, which may be motivated by racism and misogyny. A strategy of working within this constraint is to wait for changed conditions in public discourse. An example of this is the 2023 rise in concern about large language models and generative AI. Leaders in the technology industry and economy have started to warn of the risk of such technology, e.g., Geoffrey Hinton and Joseph Stiglitz (Taylor and Hern, 2023; Bushwick, 2023). Black women had warned much earlier about the issue and have been discredited for naming the dangers of \\u201cstochastic parrots\\u201d (which is the title of the famous text that played a role in the firing of Black AI ethicist Timnit Gebru from Google, see Bender et al., 2021; Turner et al., 2021). However, the fact that, in 2023, AI and economic leaders publicly voiced concern can be an opportunity for scholars working with critical approaches to reiterate their arguments and move their critique of unregulated AI into the mainstream.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B94\", \"B97\", \"B97\"], \"section\": \"4 Discussion\", \"text\": \"Another constraint is the economic context within which critical research on AI is produced, as academic and industrial spaces are shaped by neoliberal capitalism. Given this constraint, alternative business models and infrastructures for design have been proposed (cf. Scholz and Schneider, 2017). Smyth and Dimond (2014, p. 70) propose cooperatives instead of companies to make the work environment where design is facilitated anti-oppressive so that the design can be anti-oppressive as well. Smyth and Dimond (2014, p. 71) write:\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"fn0003\", \"fn0004\", \"B77\"], \"section\": \"4 Discussion\", \"text\": \"Examples of existing worker cooperatives in the technology sector are Sassafras Tech Collective and Research Action Design. In the context of online platforms, Poster (2019, p. 163) suggests \\u201cplatform cooperativism\\u201d as a new model for more just online environments. They propose to look to the history of African-American cooperatives for models of platform infrastructures where ownership and governance is in users' hands (e.g., inspiration can be found in W.E.B DuBois'1907 text \\u201cEconomic Cooperation among Negro Americans\\u201d). While these initiatives are promising, the reality is that most research and design takes place in contexts where neoliberal capitalism dictates design decisions.\"}, {\"pmc\": \"PMC12355979\", \"pmid\": \"\", \"reference_ids\": [\"B111\", \"B111\", \"B35\", \"B102\"], \"section\": \"4 Discussion\", \"text\": \"This is not only true for big technology companies that demand the alignment of their products with company interests. Funding from governments, intergovernmental organizations (such as the European Union) and foundations can similarly put constraints on researchers and designers, who may further engage in self-censorship to avoid agitating potential funders (Wolf et al., 2022, p. 444). There is also a (perceived) need to scale new AI-based products or processes as a high adoption rate is a measure for success (Wolf et al., 2022, p. 444). Additionally, there is time pressure. Research projects are usually funded for a few years (if at all), pressuring researchers to go in and out of communities with little concern for their long-term development. Time is entangled with the \\u201cpublish or perish culture\\u201d in academia where a researcher is incentivized to produce numerous written outputs in a short amount of time (cf. Erete et al., 2023, p. 26f). Little or no funding is offered to community members who participate in the research or design project (Tran O'Leary et al., 2019, p. 8).\"}]"

Metadata

"{\"section-at-acceptance\": \"Work, Employment and Organizations\"}"

PMC Articles

Black feminism and Artificial Intelligence: the possibilities and limitations of contesting discriminatory AI from a critical social theory perspective

Abstract

Full Text

Sections

Metadata