A Johns Hopkins Study Finds Language Patterns Can Change AI Workplace Writing

A study by researchers at Johns Hopkins University examined whether the language used in a prompt changes the quality and tone of workplace writing produced by AI. In tests involving emails, job applications, and resignation letters, prompts containing linguistic features more commonly associated with women elicited shorter, less formal, and less sophisticated responses across four language models.

The researchers added features such as hedging and collective references to otherwise similar prompts. Their paper, “It’s How You Ask: Gender-Associated Linguistic Bias in LLMs,” argues that the effect followed the language style more strongly than the name attached to the request. The work is scheduled for presentation at the Conference on Language Modeling.

A subtle interface problem

The finding does not establish that every woman receives worse AI assistance, or that a model can reliably identify a user’s gender. It points to a narrower concern: models may react differently to linguistic styles that are common in some groups, even when the underlying task is the same.

That matters as employees use AI to draft messages, applications, and other professional documents. If a tool systematically changes tone or complexity based on the prompt’s phrasing, the output may reinforce existing expectations about whose communication sounds authoritative.

The researchers say users should not have to erase their natural writing style to get capable assistance. The more durable fix is to evaluate models across varied language patterns and make bias testing part of product development. Organizations adopting writing assistants can also compare outputs across equivalent prompts before relying on them in hiring or performance workflows.

Sources: Johns Hopkins University summary; research paper on arXiv.