The Myth of Neutrality

The seduction of the neutral system

We like to pretend that numbers don’t discriminate.
That data is objective.
That systems merely “mirror reality”.
That design can be “neutral".

But neutrality is the most dangerous design fiction of our century ... because systems that claim no opinion often enforce the strongest ones.

“Technology is not neutral. We’re inside of what we make, and it’s inside of us. We’re living in a world of connections, and it matters which ones get made and unmade.” - Donna J. Haraway

Classification is not passive.
It is power exercised through structure.
It is design that reorganises the world in its own image.

When a system starts counting people,
it begins by deciding what counts as a person.

And we’ve been here before.

The census that redesigned a civilisation

In 1881, the British Raj launched its first empire-wide synchronous census.
This was not a simple headcount.
It was a design act.
The goal? Deciding what categories millions of people must fit into, so they can be governed and compared.

Census instructions pushed enumerators to classify Bhaarat's (India's) population into abstracted fixed buckets of race, caste, tribe, and religion.
Buckets shaped less by lived identity and more by Victorian racial theory.

Anthropologist H. H. Risley’s codified "racial types" (Indo-Aryan, Dravidian, Scythian, Aboriginal) were entered into the census tables.
Identities that had never existed in those forms.
But now suddenly printed, mapped, and measured as demographic fact.

On the ground, enumerators often confronted fluid, relational identities they couldn’t fit into the forms.

Local categories (jati, kin networks, occupational guilds, lineages, spiritual affiliations) were fluid, context-specific, and relational.

But the census demanded fixed boxes.
And so the forms won.

India's complexity was collapsed into a handful of labels, recorded in ledgers, tabulated in London, and later returned as "truth".

As historian Nicholas Dirks summarises:

“The census did not record caste. It produced it.”

This was ontological design in its rawest form.
A bureaucracy mistook its own simplifications for scientific discovery and the shape of reality ... and millions had to live inside the new shapes.
A civilisation’s identities were re-designed by administrative constraint.

The modern heir: Algorithms as the new census

Today’s classification systems operate with the same premise, but at far greater speed and scale.

A credit score sorts you into “risk.”
A hiring model sorts you into “fit”, slicing your potential.
A recommendation engine slices your identity into behavioural vectors.

Not because these systems understand you.
But because they need you to fit a category to function.
Today’s classification systems wear neutral interfaces.

This is not intelligence.
This is automated categorisation at scale.

Where the census imposed a racial and caste order, algorithms impose a behavioural one:

• Your postcode becomes your economic destiny

• Your network becomes your opportunity set

• Your scroll behaviour becomes your psychological profile

• Your pauses become vulnerability indicators

These aren’t mirrors.
They are chisels.

They carve the world into categories because systems need categories to run.
And once carved, the cuts become infrastructures.

They feel neutral because they arrive as numbers, not as ideology.
But behind the interface is the same logic:

• Reduce complexity to categories.

• Treat categories as truth.

• Enforce truth through design.

The British Empire mapped bodies.
Algorithmic systems map minds.

Both claim neutrality.
Both produce control.

Design principles of classification systems

Whether colonial census or contemporary algorithm, classification systems share a dangerous DNA:

1. Simplification as violence : Complex realities are flattened into categories the system can process, not categories that honour truth.

2. The classifier's gaze : Someone decides what counts. That someone is rarely the person being counted.

3. Feedback loops of reinforcement : Train on history, and you don't automate truth. You automate yesterday's hierarchy.

4. Invisible enforcement : The most powerful classifications don't announce themselves. They simply become "how things work."

These are not bugs.
They are features of extractive classification.

But not all classification works this way.

When classification liberates

The paradox is that
classification can be both weapon and shield.

The same act of naming that can erase you can also make you visible.
The same categories that trap can also protect.

Consider:

• Protected class status in anti-discrimination law creating a legal ground to stand on.

• Disability classifications unlocking accommodations, community, and rights.

• LGBTQA+ communities claiming identity labels; contested, and continually redefined by the communities themselves.

• Indigenous groups asserting tribal identities, sovereignty and reclaiming resources.

• Counting underrepresented groups to reveal inequality

• Tracking disparities to drive policy change

• Making invisible populations visible for resource allocation

These classifications matter because they:

• Enable solidarity : you can't organize around "people like us" without some shared language

• Make oppression visible : you can't measure a gap you refuse to name

• Unlock resources : funding, legal protection, and policy often require categorical proof

• Assert identity : sometimes claiming a label is an act of resistance

So the question isn't: Should we classify?
The question is: Who controls the classification, and for what purpose?

The line between recognition and control

Here's how to tell the difference:

Classification as control:

• Imposed from outside, not chosen

• Fixed and unchangeable

• Used to restrict, sort, or extract

• Reduces complexity for system convenience

• You cannot contest or exit

• Serves the classifier, not the classified

Classification as recognition:

• Co-created or self-determined

• Fluid and contestable

• Used to protect, connect, or empower

• Acknowledges complexity while enabling action

• You can redefine or refuse

• Serves the classified, not the system

The 1881 census? Control.
Credit scores? Control.

Accessibility standards co-designed with disabled communities? Recognition.
Pride flags and pronouns? Recognition.

The colonial census forced identities onto people.
The disability rights movement demanded recognition on their own terms.

Both involve classification.
The difference is power.

The polite violence of design methods

Even inside our own field, we mistake structure for truth.

The Double Diamond, formalised by the UK Design Council in 2005, was never a universal map of ingenuity and innovation. It was a managerial tool shaped by British institutional logic.

Yet it spread across the world as if it were neutral - the diagram of how humans everywhere think.

Design scholar Lana Petrović writes:
“Design Thinking universalizes Western epistemologies by exporting them as context-free logic.”

The Diamond does not reveal a problem.
It classifies it - through the eyes of the designer.

It decides what counts as a “real” need, what fits the frame, and what falls outside it. Like the colonial census, it makes the designer the one who names reality (even if that designer indirectly obeys other's orders) - and everyone else the named.

As Arturo Escobar in Designs for the Pluriverse (2018), reminds us:

"Design has become a way of producing worlds - but usually worlds aligned with dominant ontologies."

Where bias hides: The four fault lines

Bias embeds in the pipeline, in choices made upstream, in defaults left unquestioned.
It hides in the four fault-lines of design:

1. Proxy drift : optimizing what is measurable instead of what is meaningful.
   (e.g. Time-on-site becomes "engagement," but addiction looks identical)

2. One-world defaults : declaring a dominant group’s pattern as “universal”
   (e.g. A chatbot trained on Western psychology gaslighting collectivist users.)

3. Opaque causality : outcomes with no visible reasons.
   (e.g. Your loan denied by a score you can't contest)

4. Feedback loops : data trains decisions; decisions reshape data, and bias compounds.
   (e.g. Policing data showing more crime in over-policed areas, justifying more policing)

This is why algorithmic harm is not accidental.
It is the direct result of design decisions made upstream.

Train on history, and you don’t automate truth.
You automate yesterday’s hierarchy.

The bureaucracy of fate

Like the colonial census, algorithmic systems feel administrative, dull, inevitable. Which is why they are so dangerous.

The most consequential design work of the 21st century now happens in the boring layers:

• Data schemas

• Labeling protocols

• Proxy definitions

• Ranking functions

• Decision thresholds

These choices decide:

• Who sees opportunity.

• Who is flagged as risk.

• Who gets believed.

• Who gets erased.

This is not “just product”
This is ontological control ... deciding who someone is, and therefore what someone can become.

A colonial official once declared a village “Dravidian.”
A model now declares a teenager “high risk.”
Both times, the category becomes the reality people are forced to inhabit.

Why it matters today

We are not designing neutral systems.
We are designing power structures.

Every taxonomy is a worldview.
Every default is a decision about whose reality counts.

But classification itself is not the enemy.
Extractive classification is.

The question isn't whether our systems classify.
The questions are:

Who controls the categories?
Can people contest them?
Do they serve the system or the people inside it?

Classification at scale doesn't just sort people, it shapes them.

When you're perpetually sorted into "risk," you begin to internalise it.
When opportunities are gated by proxies for privilege, meritocracy becomes mythology.
When your identity is reduced to vectors, you start performing for the algorithm.

But when classification is co-created, contestable, and used for equity?
It can make the invisible visible.
It can protect the vulnerable.
It can enable collective action.

The spinning wheel was a tool of liberation because it let people opt out.
Sometimes classification is liberation because it lets people opt in ... on their own terms.

The difference is always the same: who holds the power to name reality.

Toward accountable classification

Classification itself isn't inherently bad and classification is unavoidable.
It's often necessary and can even be liberating. The problem is how classification is done and who controls it.
The question isn't whether our systems classify.
The question is: whose world do they classify into existence?

How do we design systems that classify with people, not to them?

1. Design for self-determination

   Let people define themselves. Build systems that honor multiplicity.
   e.g. Pronoun fields that accept free text, not dropdown menus. Gender options that include "prefer to self-describe."

2. Design for contestability

   Build systems people can interrogate, appeal, and reshape.
   e.g. Loan denials with explanations. Ranking systems with opt-outs.

3. Make the classifier visible

   Show who decided the categories, and why.
   e.g. Model cards documenting training data, known biases, intended use and limitations.

4. Hold space for the unmeasurable

   Not everything that matters can be counted. Design for what resists classification.
   e.g. Qualitative input alongside quantitative metrics. Human review for edge cases and exceptions.

5. Distribute the power to classify

   Give communities the power to contest external labels and propose their own.
   e.g. Participatory data governance. Community advisory boards for algorithmic systems.

6. Design for refusal

   The right not to be categorised is as important as the right to be seen.
   e.g. "Prefer not to say" as default, not the exception. Opt-in classification, not mandatory fields.

7. Build for fluidity, not fixedness

   People change. Contexts shift. Categories should be revisable, not permanent.
   e.g. The ability to update demographic information. Systems that don't lock identity at account creation.

The goal isn't to eliminate classification.
It's to shift power from the system to the person.

From extraction to recognition.
From control to agency.

What's your taxonomy doing?

Ask yourself, relentlessly:

• Who decided these categories and who was excluded from deciding?

• What complexity am I erasing to make this system "work"?

• If someone contests their classification, can they? Or is the system the final word?

• Am I designing a mirror or a mold?

• What are we making legible, and what are we making invisible?

🔥 Designers: you are not neutral

Your taxonomy is not neutral.
Your onboarding flow is not neutral.
Your ranking algorithm is not neutral.
Your defaults are not neutral.

Classification is power.
Every choice either repeats a hierarchy or rewrites it.

Our job is no longer to design systems people can use.
Its to design systems people can interrogate, contest, and ultimately reshape.

What systems are you building that leave space for what cannot be measured?
What systems are building that empower people to decide who they are, not who your data says they are?
What categories are you imposing, and which ones are you co-creating?

If a metric wins and a human loses = extraction.
If a human wins and the metric follows = design.

The myth of neutrality has lasted long enough.
Classification will continue.
The only question is: will it be a census
or a declaration of independence?

It's time to design with our eyes open.