Digital platforms are not simply attacked by “bots.” They are contested ecosystems in which people, scripts, account farms, compromised devices, commercial automation, and AI agents pursue conflicting goals.

The consequential question is rarely Is this a bot? It is:

What behavior threatens the ecosystem, what evidence do we have, and what intervention is justified?

Platform Integrity is a practical decision framework for answering that question. It connects security engineering, trust and safety, fraud and risk, data systems, product policy, economics, privacy, legal authority, and operational response—because production integrity failures cross all of those boundaries.

The operating model

The book treats platform defense as a governed chain of decisions:

ecosystem -> actors -> incentives -> harm -> policy -> observability
          -> inference -> confidence -> urgency -> intervention
          -> error handling -> recourse -> measurement -> adaptation

Every arrow is a possible failure point. A detector is only one evidence-producing component inside this system.

The recurring tool is the Adversarial Platform Canvas—fourteen questions that force a team to connect what it protects, who benefits, what it can know, how quickly it must act, what mistakes cost, and how the attacker will respond.

What is inside

The core decisions

By the end, a reader should be able to answer:

1. Which measurable outcome—not merely which rule violation—is harming the ecosystem?
2. Is automation the problem, or is it only the mechanism producing the behavior?
3. Which events are direct observations, and which claims are features, links, or inferences?
4. Does the evidence identify a request, session, device, account, person, or organization?
5. What evidence standard is proportionate to the severity, breadth, duration, and reversibility of the action?
6. Does the platform need a millisecond decision, a delayed campaign analysis, or both?
7. How will an innocent or compromised user understand, contest, and recover from the decision?
8. Did the intervention prevent harm, displace it, or merely teach the attacker?

What makes the approach different

Most treatments begin with classifiers, CAPTCHAs, fingerprints, or anti-cheat technology. This book begins one level earlier—with institutional authority and engineering judgment.

Its working doctrine is:

• Govern harmful behavior rather than the metaphysical category “bot.”
• Separate collection from inference, and inference from authority to act.
• Treat attribution as probabilistic, temporal, and purpose-specific.
• Use the least severe intervention that controls expected harm within the latency budget.
• Require stronger evidence for actions that are severe, broad, durable, or difficult to reverse.
• Treat privacy, accessibility, legitimate automation, and recourse as system requirements.
• Measure prevented harm and legitimate-user burden, not detector accuracy alone.
• Assume every visible defense changes attacker incentives.

Built to withstand scrutiny

This is a living research book, not a bundle of unsupported prescriptions. Claims move through a visible pipeline:

primary source -> structured research note -> concept -> synthesis -> chapter

The public research library preserves what each source says, its evidence, assumptions, weaknesses, disagreements, and implications. The living synthesis keeps an evidence ledger for the book’s major claims. Sources currently include NIST risk and identity frameworks, IETF and W3C standards, peer-reviewed abuse-market measurements, legislation and enforcement actions, platform contracts, and trust-and-safety scholarship.

That discipline is part of the product: readers can inspect not only the conclusion, but why it should be believed and where it might fail.

Who this is for

Platform Integrity is written for staff and principal engineers, engineering managers, security and fraud leaders, trust-and-safety practitioners, data scientists, investigators, product and policy leaders, and founders building systems in which abuse can distort access, money, attention, safety, or trust.

It is also a working framework for teams that need to:

• design or review an abuse-prevention architecture;
• turn an ambiguous “bot problem” into a defensible program;
• connect policy, telemetry, models, investigations, and enforcement;
• reduce false positives without surrendering to fraud;
• prepare evidence and decision systems for new forms of delegated AI automation;
• establish ownership, metrics, and review mechanisms across functions.

Read next

Begin with What Is a Bot? for the category problem, or jump to the Adversarial Platform Canvas to apply the framework to a live system.

Parts I–III are in first-draft review. Parts IV–VI are structured briefs being expanded in public. The manuscript will continue to evolve as research, cases, and practitioner review challenge its claims.

Book synthesis

This file is the book’s live argument map. Claims graduate here from reviewed research and concept notes before they become polished chapter prose.

Governing claim

Bot mitigation is a decision problem under uncertainty, not merely a detection problem.

A platform governs behavior through a chain:

ecosystem -> actors -> incentives -> harm -> policy -> observability
          -> inference -> confidence -> urgency -> intervention
          -> error handling -> recourse -> measurement -> adaptation

Each arrow is a possible failure point. A highly accurate detector can still produce harmful governance when the prohibited behavior is vague, attribution is weak, the intervention is disproportionate, or recourse is absent.

The book’s promised contribution

The book will teach an engineer, product leader, investigator, or policy partner how to decide:

1. what outcome actually threatens a platform ecosystem;
2. which automation should be permitted, limited, or prohibited;
3. what evidence the platform can responsibly collect;
4. how strongly that evidence supports identity, coordination, or harmful intent;
5. how quickly a decision is required;
6. which intervention is proportionate and reversible;
7. how to measure mistakes, support appeal, and learn from adaptation.

The Adversarial Platform Canvas

Every chapter and case study should answer the same fourteen questions:

Decision doctrine under development

These are hypotheses to research, challenge, and refine:

• Govern harmful behavior, not the metaphysical category “bot.”
• Separate evidence collection from inference and inference from enforcement authority.
• Treat attribution as probabilistic across request, session, device, account, person, and organization.
• Use the least severe intervention that controls expected harm within the latency budget.
• Require stronger evidence for interventions that are severe, broad, durable, or hard to reverse.
• Treat false positives and false negatives as asymmetric, ecosystem-specific costs.
• Design appeals, evidence packages, audit trails, and rollback into the enforcement system.
• Measure prevented harm and legitimate-user burden, not detector accuracy alone.
• Expect every observable defense to alter attacker incentives and behavior.

Part map

Part I — Define the problem before solving it

Challenge “bot” as a sufficient category, establish harm analysis, and model the economics of abuse and defense.

Part II — Decide what the platform should permit

Build an automation policy, connect it to legal and contractual authority, and quantify the asymmetric cost of mistakes.

Part III — Observe the platform

Inventory telemetry, attacker interaction surfaces, and the limits of identity and attribution.

Part IV — Detect

Compare rules, behavioral signals, statistical learning, graph methods, and client integrity as evidence-producing mechanisms.

Part V — Respond

Design a graduated intervention system, time-to-decision policy, appeals, review tooling, and bounded transparency.

Part VI — Operate an adversarial system

Choose outcome metrics, run an adaptive defense program, and establish organizational decision rights.

Part VII — Case studies

Apply one canvas to gaming, dating, commerce, and financial markets so the legitimacy of automation is seen as ecosystem-dependent.

Open synthesis questions

• Can proportionality be formalized well enough to guide system design without pretending costs are objective?
• When should an intervention target behavior, capability, account, device, network, payment instrument, or organization?
• How should a platform measure displaced harm that moves to a new account or channel?
• Which transparency improves legitimacy without creating a cheap attacker feedback oracle?
• What evidence standard is appropriate at each rung of the intervention ladder?
• How should privacy and data minimization constrain graph detection and long-lived attribution?
• When does defensive friction become a product failure or discriminatory burden?

Evidence ledger

No major claim should become chapter prose until it has at least one linked research note, known limitations, affected ecosystems, and a falsification question.

The Adversarial Platform Canvas

A reusable decision record for moving from ecosystem harm to proportionate intervention.

How to use it

Complete the canvas before selecting a detector. Cite evidence, mark assumptions, name an owner, and preserve alternatives. Revisit it after incidents, policy changes, major model changes, or attacker adaptation.

Decision record

Decision:
Ecosystem and harm:
Policy authority:
Evidence and lineage:
Inference and confidence:
Latency budget:
Chosen intervention:
Why proportionate:
Error and subgroup risks:
Recourse and rollback:
Success and guardrail metrics:
Expected adaptation:
Owner and review date:

Quality checks

• Would the action still be justified if the actor were human rather than automated?
• Does the evidence support the intervention target, or only a related request/account/device?
• Is a less invasive or more reversible action sufficient?
• Can a legitimate user understand the rule and recover from error?
• What observation would prove the intervention ineffective or harmful?

1. What Is a Bot?

Choose a governable category without confusing automation, intent, identity, and harm.

Imagine four requests arriving at a ticketing platform in the same second:

1. a customer clicks “Buy” in a browser;
2. an accessibility tool invokes the same control;
3. a shopping agent acts under the customer’s instruction and budget;
4. a reseller’s program rotates through fabricated accounts to defeat a purchase limit.

All four requests can be emitted by software. Their HTTP syntax may be indistinguishable. Yet the platform has strong reasons to treat them differently.

That is the first problem with the word bot: it appears to describe an actor, but usually describes only a mechanism. It tells us that software participated. It does not tell us who authorized the action, what rule applies, which harm may result, or what intervention is justified.

There is no useful binary

Automation is a continuum of delegated effort:

The boundary shifts with implementation. A reseller can employ people to click; a legitimate user can employ an agent to transact. “Human-like” behavior is not legitimacy, and machine-like behavior is not abuse.

Platforms themselves depend on benign automation. Search engines use crawlers to discover the web. The IETF’s Robots Exclusion Protocol standardizes how cooperative crawlers discover service-owner preferences. It calls crawlers automated clients and explicitly warns that robots.txt is not access authorization. This is an important separation: declared identity and cooperative policy are useful, but they are not security proof.

Policy definitions reveal different real concerns

There is no universal platform definition because ecosystems protect different things.

GitHub’s Acceptable Use Policies focus on excessive automated bulk activity, inauthentic interaction, rank abuse, deception, and undue server load. The modifiers do the work: excessive, inauthentic, abusive. GitHub also provides APIs and automation products because automation is part of the ecosystem’s value.

Valve takes a broader contractual approach. Steam’s Subscriber Agreement defines scripts, bots, macros, and other non-human-controlled systems as “Automation” and prohibits specified interaction with Steam services. Its examples reveal underlying concerns: fabricated statistics, rewards without genuine input, automated account creation, and scripted participation in systems that require judgment.

U.S. ticket law is narrower still. In the FTC’s first BOTS Act enforcement cases, the alleged conduct combined automated purchasing, circumvention of posted limits, fabricated accounts, IP concealment, and a resale business. The law is not a general prohibition on fast customers or purchasing software. It addresses circumvention in a defined market.

These are not inconsistent definitions waiting for a standards committee. They are different governance choices.

Classify the claim, not the creature

“Bot detected” often hides several distinct claims:

Mechanism claim:     software generated or assisted the action
Identity claim:      these requests belong to one actor or operation
Authority claim:     the actor was or was not authorized by the account holder
Behavior claim:      the action violated a rate, sequence, or interaction rule
Intent claim:        the actor sought a prohibited outcome
Harm claim:          the behavior caused or materially risked ecosystem damage

Evidence rarely supports all six at once. Regular timing may support a mechanism claim. Shared payment instruments may support an identity link. Neither alone proves harmful intent. A valid account credential proves access, not current delegated authority. A prohibited outcome can sometimes be governed without resolving whether a human or program produced it.

This yields a practical rule:

Make the narrowest claim supported by the evidence, then choose only interventions justified by that claim.

A mechanism signal might justify lower request concurrency. A strong behavior claim might justify rejecting one transaction. A permanent removal of linked accounts needs much stronger attribution and policy evidence.

A better actor model for agents

AI agents make the binary less useful, but do not make detection obsolete. The platform needs a richer authorization envelope:

principal:       who receives the benefit and bears responsibility?
delegate:        which person, program, or provider is acting?
grant:           what authority was given, by whom, and when?
action:          what operation is requested?
constraints:     budget, rate, resource, audience, and expiry
evidence:        how are identity and grant authenticated?
context:         what risk, policy, and behavioral history applies now?

An agent may have a legitimate grant and still be compromised, exceed its limit, or participate in coordinated abuse. Authorization, authentication, behavior analysis, and harm prevention are complementary.

The decision procedure

Before creating a “bot” rule, answer in order:

1. Name the protected outcome. Fair allocation, authentic judgment, system capacity, account security, or something else?
2. State the policy in observable terms. What action, scale, coordination, deception, or circumvention is prohibited?
3. Identify authorized paths. APIs, declared crawlers, accessibility tools, agents, testing, and commercial integrations.
4. Separate evidence claims. Mechanism, identity, authority, behavior, intent, and harm.
5. Choose an intervention at the supported scope. Request, capability, transaction, account, linked operation, or organization.
6. Test counterexamples. Could a legitimate user, shared network, expert player, or accessibility tool produce the same evidence?

If the team cannot complete steps one and two, it is not ready to build a classifier. It does not yet know what the classifier is for.

Research trail

• RFC 9309 note
• GitHub policy note
• Steam agreement note
• FTC BOTS Act note

Review questions

1. Which of your current “bot” signals support mechanism, identity, behavior, or harm claims?
2. What legitimate automation would your policy prohibit accidentally?
3. Could the harmful outcome be governed without determining whether the actor is human?

2. Why Does the Platform Care?

Translate “abuse” into observable ecosystem harm before choosing enforcement.

Suppose an automated buyer acquires every available unit of a scarce product. What exactly went wrong?

• The seller may have earned the intended revenue.
• The transactions may be valid and paid.
• The service may have remained available.
• No account may have been compromised.

Yet ordinary buyers may conclude the allocation was unfair, resellers may extract the scarcity premium, support volume may rise, and trust in future launches may fall. “We detected automation” is not the explanation. The platform is responding to an allocation and ecosystem-design failure.

Harm is an outcome, not a synonym for violation

Three ideas need to remain separate:

Policy violation: behavior contradicts a rule the platform adopted
Harm:             an actor or ecosystem is made materially worse off
Risk:             uncertainty about future harm, including magnitude and likelihood

A rule can be violated without measurable harm. Harm can occur before a rule exists. A platform can also adopt a rule for operational simplicity even when individual violations are harmless. Enforcement systems become brittle when these categories are treated as interchangeable.

The Stanford Trust and Safety manuscript recommends prioritizing by the severity of harm, its prevalence, and the product’s responsibility for enabling it. It also warns that user reports are incomplete and selected. The manuscript is centered on online safety, but the reasoning generalizes: count neither reports nor removed accounts as the harm itself.

Build a causal harm chain

A useful harm statement connects conduct to affected parties through a mechanism:

actor and conduct
    -> platform mechanism or market response
    -> exposure or changed allocation
    -> immediate effect
    -> downstream ecosystem effect

For scarce-inventory capture:

coordinated buyers defeat per-customer limits
    -> inventory concentrates before ordinary customers can transact
    -> access shifts toward a resale market
    -> buyers pay more or abandon the purchase
    -> perceived fairness and future participation decline

Each arrow is a research question. Perhaps concentration was caused by ordinary demand rather than circumvention. Perhaps resale improves allocation to those who value the product most while violating the seller’s fairness objective. Perhaps a queue would fail differently. The chain prevents moral vocabulary from substituting for evidence.

A platform harm register

Record harms by affected party and system property:

The last row matters. A defense is part of the ecosystem and can itself cause harm.

Measure prevalence, exposure, severity, and responsibility

Raw event count favors easy-to-count activity. Instead describe at least four dimensions:

prevalence:      how much violating or risky activity exists?
exposure:        how many affected parties encounter it, and how often?
severity:        what is the distribution of consequences, including the tail?
responsibility:  how much does this product create, amplify, monetize, or control the pathway?

Add recoverability and velocity. A fraudulent transfer that can be reversed tomorrow differs from a competitive match ruined now; an account network that grows slowly permits a different evidence strategy than real-time payment theft.

Avoid collapsing the dimensions into one impressive number too early. Ten million blocked attempts may reflect ten million independent threats, one looping script, or an instrumentation change. A decline in detected abuse may mean improvement, displacement, or detector blindness.

The distribution matters more than the average

Harm and defensive burden are rarely uniform. A global false-positive rate can conceal concentrated exclusion of users behind shared networks. Median financial loss can conceal a destructive upper tail. Report rate can underrepresent people who do not know a violation occurred or cannot navigate the reporting flow.

For every harm measure, ask:

• What is the unit: request, account, person, victim, incident, campaign, or dollar?
• Who is absent from the data?
• Is the measurement before or after current defenses?
• Can attackers manipulate the metric?
• Which population bears the worst outcome?
• What counterfactual supports the claim that the platform caused or prevented it?

Product success can create integrity failure

Ross Anderson’s economic analysis observes that security problems arise when the party able to prevent harm does not bear its cost. The same pattern appears in platforms. Growth metrics can reward engagement generated by fake accounts. A marketplace can earn fees on transactions whose externalities fall on buyers or sellers. A support organization can optimize ticket closure while leaving recurring campaigns intact.

This is not necessarily bad intent. It is organizational optimization. If integrity outcomes are absent from planning and review, teams rationally optimize what is visible and rewarded.

The decision procedure

Before funding an abuse control:

1. Name affected parties, including legitimate actors burdened by the defense.
2. Write the causal chain from conduct to harm and label uncertain links.
3. Choose outcome measures, not only detection or enforcement counts.
4. Describe the distribution by severity, population, time, and ecosystem surface.
5. Estimate platform responsibility and control. Could product design remove the pathway?
6. Define the counterfactual. What would likely happen without this intervention or with an alternative?
7. Set guardrails. Which defensive harms make the intervention unacceptable?

The output is not “bots are bad.” It is a prioritized, falsifiable claim about what the platform is protecting.

Research trail

• Stanford Trust and Safety manuscript note
• Security economics note
• FTC ticket-market case note

Review questions

1. Which integrity metric in your system is merely a detector-output count?
2. What harm does your current enforcement impose, and on whom?
3. Which link in your most important harm chain has the weakest evidence?

3. The Economics of Abuse

Model defense as a contest over expected value, capacity, and replacement cost.

An attack can be technically possible and economically irrelevant. Another can survive a 99% account-removal rate because accounts are cheap and the remaining 1% pays for the entire operation.

Platform defense therefore needs two models: how the attack works and how the operation stays viable.

Start with the production function

An abuse operation converts inputs into monetizable outcomes:

infrastructure + identities + accounts + software + human labor
    -> attempts
    -> successful actions
    -> monetizable assets or influence
    -> realized revenue

Every arrow has yield, delay, variance, and capacity. Revenue may require resale, money mules, affiliate conversion, reward withdrawal, or an audience. The apparent technical step—creating an account or defeating a challenge—may not be the bottleneck.

Research on phishing and fraudulent accounts repeatedly illustrates this. Thomas and colleagues observed an underground market selling accounts in bulk, making accounts a priced production input rather than durable identities. Other work on phishing found that converting compromised credentials into irreversible value can depend on scarce intermediaries. Defense should target the limiting production step, not automatically the most visible one.

Model expected profit, not attacker inconvenience

A starting model is:

expected profit per cycle
  = P(success) * revenue_if_success
  - account and identity cost
  - infrastructure and tooling
  - human labor
  - inventory or capital at risk
  - expected enforcement loss
  - monetization and laundering cost

Then add time and scale:

return on constrained resource
  = expected profit / scarce attacker input over time

An intervention can reduce success probability, raise marginal cost, cap throughput, increase variance, delay payout, seize accumulated value, or make targeting less efficient. These are different effects.

A challenge that costs an attacker one cent but a legitimate user thirty seconds may be economically backwards. A delayed reward can be powerful when it gives the platform time to detect a network before value exits. A ban can be negligible if replacement is instant, yet meaningful when reputation, inventory, or payout history is costly to rebuild.

Count operations, not just accounts

In the 2013 study Trafficking Fraudulent Accounts, researchers observed 27 merchants over ten months and collaborated with Twitter to identify millions of fraudulent accounts. The historical prices are not current benchmarks. The enduring lesson is structural: account creation, maturation, sale, campaign use, and replacement form a supply chain.

Account-removal metrics can reward the defender for repeatedly cutting disposable leaves while the operation persists. Better measures include:

• time and cost to replace an enforced asset;
• campaign throughput and survival;
• fraction of value removed before monetization;
• attacker labor per successful outcome;
• migration to other surfaces, accounts, or victims;
• time required to regain reputation or payout eligibility.

Economics also explains false positives

Defenses impose costs on legitimate users:

legitimate burden
  = added time + failed completion + privacy cost + exclusion
  + support effort + delayed funds + trust loss

Attackers may specialize, automate around friction, or pay low-cost labor; legitimate users face the control infrequently and cannot amortize the learning. A mechanism that “raises attacker cost” can raise defender and customer cost faster.

The relevant objective is not maximum attacker pain. It is favorable cost asymmetry:

attacker cost or harm reduction gained
---------------------------------------
legitimate-user and platform cost added

Even this ratio is incomplete when burdens fall on different populations. Accessibility, geography, device quality, language, banking access, and shared networks change who pays.

The defender has an economic system too

Ross Anderson’s economic account of security emphasizes misaligned incentives and externalities. On a platform:

• a growth team may receive credit for accounts that integrity later removes;
• a marketplace may collect fees while sellers absorb fraud;
• an anti-abuse team may optimize blocks while support absorbs appeals;
• a model vendor may sell detection volume rather than prevented harm;
• one service may reject risky traffic that migrates to another service.

Allocate costs to make the whole outcome visible. Otherwise each team can improve its dashboard while the ecosystem worsens.

Shared infrastructure creates both leverage and error

The Google study The Abuse Sharing Economy measured IP reuse across six services. Cross-service intelligence identified some abusive traffic, but direct IP blacklisting created an unacceptable false-positive volume.

This illustrates a general economic temptation: a cheap, broad control can appear attractive because it moves investigative cost onto legitimate users. Signal sharing may improve risk estimates. It does not make the signal sufficient for punishment.

Displacement is not victory

An intervention can:

1. suppress the harmful outcome;
2. substitute one attacker input for another;
3. displace activity to another surface, victim group, or platform;
4. concentrate the market among more capable attackers;
5. transform the business model.

Short evaluation windows confuse these. After a new control, observed attacks often drop immediately because attackers are testing alternatives. Measure long enough to see the new equilibrium.

The decision procedure

For one abuse operation:

1. Draw the value chain from inputs to realized value.
2. Estimate ranges, not fictional point precision, for yield, cost, capacity, and delay.
3. Find constrained inputs such as trusted accounts, payout channels, human judgment, inventory, or victim attention.
4. Map each intervention to success probability, throughput, delay, variance, seizure, or replacement cost.
5. Calculate legitimate and defender burden using the same seriousness.
6. Measure operation-level outcomes and displacement over an appropriate horizon.
7. Revisit after adaptation. The model describes a temporary equilibrium, not a permanent attacker.

The economic goal is not necessarily to make all abuse impossible. It is to prevent expected harm efficiently, make large-scale abuse unattractive or capacity-constrained, and preserve a platform worth using.

Research trail

• Security economics note
• Fraudulent-account market note
• Abuse-sharing economy note

Review questions

1. Which attacker input is actually scarce in your current abuse operation?
2. Does your principal metric count disposable accounts or durable reduction in harm?
3. Which legitimate population pays the highest cost for your cheapest defense?

4. Acceptable Automation

Design a policy spectrum that preserves beneficial automation while constraining harm.

A platform that prohibits all automation will prohibit parts of itself. Monitoring, indexing, moderation, testing, accessibility, seller operations, and customer agents all depend on software acting at machine speed.

The policy question is not whether automation exists. It is which actors may exercise which capabilities, toward which outcomes, under what constraints.

Separate capability from permission

Technical capability answers “can this client act?” Policy answers “may it act this way here?” Detection estimates what is happening. Authorization records a grant. These systems overlap, but none substitutes for the others.

capability:     client can submit 1,000 requests per minute
authorization: partner token permits 600 catalog reads per minute
policy:         reads may support price comparison but not personal-data resale
behavior:       token requests 900 records and ignores retry guidance
decision:       throttle this capability and investigate the grant

Rate alone does not decide legitimacy. An authorized integration can exceed its grant; an unidentified client can make low-rate harmful requests.

Use a policy spectrum

Classify a behavior rather than an entire tool:

The same mechanism can occupy different rows. Automated gameplay may be prohibited in ranked competition, permitted in a private sandbox, and necessary for testing. Scraping may be authorized for an index, limited for research, and prohibited for unsolicited targeting.

Write rules in layers

A workable automation policy has at least four layers:

1. Purpose: the protected ecosystem outcome—fair allocation, authentic judgment, capacity, privacy, or security.
2. Conduct: observable prohibited behavior—circumvention, fabricated engagement, exceeding grants, reward extraction, impersonation.
3. Mechanism: relevant implementations—scripts, agents, macros, modified clients—without pretending the list is exhaustive.
4. Exceptions and paths: accessibility, public-interest research, testing, APIs, partners, and user-delegated agents.

GitHub’s policy illustrates conduct-oriented modifiers such as excessive, inauthentic, and undue burden. Steam’s agreement uses a broader mechanism category and examples tied to rewards and authentic judgment. Neither is universally correct; each expresses a product choice and creates a different enforcement burden.

Authorized automation should be observable

An authorized path can improve both product value and defense:

• scoped credentials identify the principal and delegate;
• grants name permitted actions, resources, rates, and expiry;
• stable identifiers enable accountability and incident response;
• predictable quotas protect capacity;
• documentation reduces accidental violation;
• revocation targets one capability without destroying an account.

But registration is not proof of good behavior. A partner can be compromised, a customer can delegate too broadly, and a legitimate agent can violate policy. Continue to evaluate action and context.

The Robots Exclusion Protocol provides a useful negative lesson: declared crawler identity and published preferences support cooperation, but the standard explicitly says they are not authorization. Stronger actions need authenticated grants.

Accessibility is a design test

If policy equates unusual timing, non-pointer input, or assistive transformation with abuse, the policy is underspecified. Accessibility should not appear as a late exception to “real human behavior.” It should test whether the rule actually describes the protected outcome.

Ask:

• Does the rule require a particular human motor pattern, or genuine human judgment?
• Can the platform expose an accessible authorized method without weakening the invariant?
• Is a challenge completion rate measured across assistive technologies, languages, devices, and networks?
• Can a user disclose or appeal assistive automation without surrendering sensitive information unnecessarily?

AI agents require bounded delegation

For agents, policy should express who may delegate what:

principal        customer account A
delegate         shopping agent provider B
scope            search and reserve; no final purchase
budget           500 USD
inventory limit  one unit per product family
expiry           24 hours
disclosure       agent identifies provider and grant
revocation       principal or platform may revoke

The platform can then govern the action without guessing humanity. It still detects forged grants, compromised delegates, coordinated principals, and prohibited outcomes.

The decision procedure

1. Define the ecosystem property the policy protects.
2. Describe prohibited conduct in technology-resistant terms.
3. Map legitimate uses and affected populations before choosing a mechanism ban.
4. Provide authorized paths where automation creates value.
5. Scope grants by action, resource, rate, value, and time.
6. Pair every policy level with proportionate enforcement and recourse.
7. Version the policy and test it against new mechanisms and old harms.

Research trail

• Robots Exclusion Protocol note
• GitHub policy note
• Steam agreement note
• After Bots source note

Review questions

1. Which automation does your platform depend on but your public policy appears to prohibit?
2. Can one grant be revoked without disabling the principal’s entire account?
3. Which policy rule incorrectly uses “human-like” as a synonym for legitimate?

5. Policy, Contracts, and Law

Align technical enforcement with actual authority, disclosure, privacy, and user rights.

An engineer can build a system that links accounts, retains device history, automatically freezes funds, and permanently removes users. Whether the company should or may operate that system cannot be answered in code.

This chapter is not legal advice. It is a guide to recognizing when architecture contains legal and policy decisions that require qualified, jurisdiction-specific review.

Four different sources of authority

Platform action may rest on:

1. Product rules: community standards, marketplace rules, competitive formats, API conditions.
2. Contract: terms accepted by a user or partner and the remedies those terms reserve.
3. Law: obligations, prohibitions, consumer rights, privacy constraints, sector rules, and regulator authority.
4. Technical control: the practical ability to reject a request, withhold a reward, or disable an account.

Technical control is not legal authority. Contract wording is not automatically enforceable everywhere. Legal permission is not proof that an intervention is proportionate or good product policy.

Policy must precede scalable enforcement

A defensible rule should identify:

• the behavior and relevant context;
• the protected interest;
• authorized exceptions;
• possible interventions and their scope;
• evidence and review standards;
• notice, appeal, reinstatement, and asset treatment;
• effective date, version, and governing jurisdiction.

Vague rules push policy-making into models and reviewer folklore. A classifier trained on previously banned accounts may reproduce an unstated historical policy even after the written rule changes.

Data collection is an intervention

Before collecting device, network, behavioral, payment, identity, or relationship signals, document:

purpose and necessity
data subjects and affected non-users
source and expected reliability
derived inferences
access and sharing
retention and deletion
security and misuse risk
jurisdiction and disclosure
user rights and contestability

“Fraud prevention” is a goal, not an unlimited retention schedule. More data can create false linkage as well as insight. Shared IP addresses, recycled devices, family payment instruments, and compromised accounts can turn a confident graph into collateral enforcement.

Automated decisions create architectural obligations

The EU GDPR’s Article 22 addresses solely automated individual decisions that produce legal or similarly significant effects and provides exceptions and safeguards. It is not a universal prohibition on models, nor a universal explanation right. Its scope is fact-specific. Still, it forces useful engineering questions:

• Did automation recommend the action or make it?
• What information did the human reviewer receive?
• Could the reviewer actually change the result?
• Was the effect significant for this user?
• Can the person express a view and contest the decision?

Do not implement “human in the loop” as a ceremonial click that ratifies a score under impossible queue pressure.

Explanation and complaints need data models

The EU Digital Services Act requires covered services to provide reasons for specified restrictions and establishes complaint-handling duties for covered online platforms. It also makes reversal counts and other moderation information operational reporting concerns. The European Commission reported in 2026 that platforms reversed about 30% of 165 million internally appealed decisions during the first two years of DSA application. That number is not a platform-wide false-positive rate: appeal selection, service scope, decision categories, and reversal reasons matter. It is evidence that reversals occur at material scale.

Build decisions with structured provenance:

decision_id
affected object and account
policy and version
facts and evidence references
automated systems used and their role
decision-maker and review path
action, scope, duration, and effective time
reason code and user-facing explanation
appeal deadline, state, outcome, and repair actions

Without this, explanations are reconstructed from logs, appeals cannot be consistent, and policy changes cannot be audited.

Purchased assets and livelihoods change severity

An account may contain purchased licenses, virtual currency, seller funds, creative work, business reputation, or access to a professional audience. Steam and PlayStation terms show how platform enforcement can affect account access, subscriptions, devices, and virtual assets. The technical action “disable account” may therefore contain several remedies with different justification.

Prefer separable controls where feasible:

• stop one transaction;
• remove an improperly obtained reward;
• restrict a competitive mode;
• pause payout pending review;
• revoke an integration token;
• preserve access to purchased offline content;
• permanently remove an identity only under the corresponding standard.

The decision procedure

For every enforcement mechanism:

1. Identify policy, contract, and legal authority separately.
2. Choose relevant jurisdictions and versions; reject timeless generic legal claims.
3. Map collected data to necessity, retention, access, and user rights.
4. Record automation’s exact role in the decision.
5. Decompose the action by asset, capability, scope, duration, and reversibility.
6. Design reason, review, appeal, correction, and audit data before launch.
7. Obtain specialist review for material legal conclusions and high-impact workflows.

Research trail

• Digital Services Act note
• GDPR Article 22 note
• Steam contract note
• FTC BOTS Act note

Review questions

1. Which of your enforcement actions combines several remedies behind one account-status field?
2. Can you reconstruct the policy version and evidence for a six-month-old decision?
3. Does human review have authority, time, and information sufficient to be meaningful?

6. The Cost of Being Wrong

Match evidence thresholds to asymmetric error costs and intervention severity.

A detector does not decide whether its score is sufficient. A decision policy does.

The same 0.70 risk estimate might justify recording additional telemetry, reducing a request rate, delaying a payout, or doing nothing. It should rarely justify every action on the intervention ladder.

Confusion matrices hide the consequential choice

False positive and false negative are properties relative to a label and threshold. Platform decisions add an action:

expected cost(action, threshold)
  = P(false positive) * cost(false positive, action)
  + P(false negative) * cost(false negative, delay)
  + cost of friction and investigation
  + cost of operating the control

The costs depend on context. Wrongly delaying a low-value post differs from freezing a seller’s payroll. Missing one spam message differs from allowing an irreversible fraudulent withdrawal. The actors bearing each cost may also differ from the team selecting the threshold.

Base rates defeat impressive accuracy

Suppose one in a thousand transactions is abusive. A detector catches 90% of abuse and incorrectly flags 1% of legitimate transactions. Among one million transactions:

abusive:       1,000 -> 900 flagged
legitimate:  999,000 -> 9,990 flagged

precision = 900 / 10,890 ~= 8.3%

The detector can be more than 98% accurate overall while most flagged transactions are legitimate. This does not make it useless. It may be excellent for routing evidence collection and terrible for automatic permanent enforcement.

Report precision and recall at the operational base rate, calibration by population and time, and the action attached to each threshold. ROC curves alone hide the workload and harm created by false alerts in rare-event settings.

Evidence standards should rise with consequence

Evaluate an intervention along at least five axes:

As consequence rises, demand stronger and more independent evidence, better attribution, a clearer policy basis, more review, and more robust recourse.

This creates action-specific thresholds:

low confidence     -> observe, sample, or request low-cost verification
moderate confidence -> rate limit, delay value exit, narrow capability
high confidence     -> temporary restriction or reward removal
very high confidence + severe harm + review -> durable identity-wide action

These are not universal numeric bands. They illustrate that the score-to-action mapping is the core governance decision.

A useful signal can be unsafe for direct enforcement

The Abuse Sharing Economy study found that cross-service IP intelligence could identify some abusive traffic, while direct blacklisting produced untenable false positives. The lesson applies to device reputation, graph neighbors, payment reuse, and anomaly scores: predictive value is not enforcement sufficiency.

Combine evidence according to how it fails. Ten features derived from the same IP address are not ten independent observations. An account’s graph neighbors may all inherit the same mistaken entity-resolution edge.

Unknown is a valid decision state

Systems often force a binary because a database column expects allow or ban. Add explicit states:

allow
allow_with_limits
pending_more_evidence
temporary_hold
reject_action
restrict_capability
escalate_review

Uncertainty can be managed through reversible controls and time. The value of waiting is the expected error reduction from additional evidence minus harm accumulated during the delay. Chapter 16 develops this latency budget.

Appeals reveal mistakes imperfectly

Appeal overturns are important but selected. Users must notice the action, understand the route, retain access, have time and language capability, and believe review is worthwhile. Non-appeal is not proof of correctness.

Still, reversals must repair more than account state. They should update linked restrictions, funds, rewards, labels, reviewer guidance, policy interpretation, and—after careful validation—model evaluation. The DSA’s requirement for reasons and complaint handling reinforces that decision provenance is part of the system.

Evaluate the whole decision system

NIST’s AI Risk Management Framework treats trustworthiness as contextual and multidimensional. Validity, reliability, safety, transparency, privacy, explainability, and fairness involve tradeoffs; optimizing one characteristic does not make the whole system trustworthy.

For integrity systems, test:

• signal and label provenance;
• performance at actual base rates;
• calibration across time, surface, and population;
• policy-to-label consistency;
• action-specific user and ecosystem outcomes;
• reviewer workload and disagreement;
• appeal access and repair completeness;
• attacker adaptation and metric manipulation;
• shutdown, rollback, and expiry behavior.

The decision procedure

1. Define the label and time horizon behind the risk estimate.
2. Measure base rate and calibration in the actual decision population.
3. Price false positives, false negatives, delay, friction, and operation by affected party.
4. Grade interventions by scope, severity, duration, reversibility, and collateral effect.
5. Set action-specific evidence standards; do not attach one threshold directly to BAN.
6. Preserve an uncertainty state and use time or narrow controls to gather information.
7. Measure errors beyond appeals and propagate corrections through the system.

Research trail

• Abuse-sharing economy note
• NIST AI RMF note
• Digital Services Act note
• GDPR Article 22 note

Review questions

1. Which action does your model threshold trigger, and why is that consequence justified?
2. What is precision at the actual operational base rate?
3. Which users cannot or will not appear in your appeal-overturn metric?

7. What Can We Actually Know?

Build an evidence system before building a detection system.

A risk system reports that an account is using a “high-risk device.” What, precisely, does the platform know?

Perhaps the server received several requests with a cookie it had seen before. Perhaps client software submitted a hardware attribute. Perhaps a vendor combined an IP address, browser properties, and historical fraud labels into a device identifier. “High-risk device” compresses all of that lineage into three words. If an investigator, model, or policy engine treats those words as a fact, uncertainty disappears exactly where it matters most.

The first job of an integrity system is therefore epistemic: preserve the distance between what the platform observed and what it concluded.

The evidence ladder

Platform evidence usually passes through six stages:

event -> representation -> feature -> linkage -> inference -> decision

An event is something the platform directly received or produced: a request arrived, a transaction settled, or a server rejected an action. A representation is the structured record of that event. A feature is a transformation such as requests per minute, time since account creation, or the number of payment instruments shared with other accounts. A linkage asserts that records concern the same entity. An inference estimates a condition such as automation, compromise, coordination, or fraud. A decision applies policy and selects an intervention.

Each stage can introduce error. A request timestamp may be trustworthy while a client-reported interaction timestamp is fabricated. The event may be logged correctly but joined to the wrong account. A feature may use a stale window. A device link may merge a household. A calibrated risk estimate may still be insufficient for the proposed action.

This yields a practical writing and engineering rule:

State the strongest claim the evidence supports, not the most convenient label downstream systems expect.

“Five accounts used the same public IP during one hour” is an observation. “Five accounts are controlled by one person” is an attribution claim. “They form a bot farm” adds mechanism and organization. “Ban all five” is a decision. None follows automatically from the previous statement.

Build a signal register

Before adding a signal to production, document it as if it may someday appear in an appeal or incident review.

The action ceiling is especially useful. A shared IP address may justify a rate limit or a graph-expansion query. It should rarely justify permanent punishment on its own. A confirmed payment reversal may support a narrow transaction hold, but it may not establish who controlled the account.

Server facts and client claims

The server has the strongest knowledge about state it controls. It can know that it issued an item, accepted an order, matched two players, or received a particular sequence of protocol messages. Even these records can be incomplete, duplicated, delayed, or corrupted, but their provenance is under the platform’s control.

Client telemetry is different. A client can report window focus, input events, software state, device properties, or an attestation result. These reports may be valuable, but they are claims crossing a trust boundary. Legitimate clients also lose events because of crashes, old versions, privacy controls, connectivity, accessibility software, and instrumentation defects.

Consequently:

• client absence is not proof of evasion;
• client presence is not proof that the claimed event happened as represented;
• a cryptographically protected report may establish its origin and integrity without establishing that the surrounding device was benign;
• server-side invariants should protect the outcomes the platform cannot afford to trust to the client.

Chapter 14 will examine attestation. The immediate lesson is simpler: classify every field by who controls the claim.

Missingness is evidence about the pipeline

Integrity teams are tempted to interpret missing telemetry as suspicious. Sometimes it is. Attackers do suppress or distort observations. But the same symptom can be produced by deployment gaps, regional outages, consent choices, old devices, unsupported browsers, or customers with unusual access needs.

Treat missingness as its own feature with competing hypotheses:

missing = evasion | product variance | collection failure | lawful choice | unknown

A sudden change across an entire client version suggests instrumentation failure. Missingness concentrated in accounts already connected to confirmed abuse may raise risk. The raw fact is the same; its evidentiary weight depends on context.

More data can make the system worse

Collection creates capability, but also liability. Persistent identifiers and cross-context linkages can make unrelated activity correlatable. RFC 8981, which standardizes temporary IPv6 addresses, is a useful reminder that stable identifiers themselves create tracking risk. NIST’s Privacy Framework treats privacy as an organizational risk-management problem, not merely a disclosure checkbox.

For each proposed signal, ask two separate questions:

1. Can we collect it? Is it technically available and legally authorized?
2. Should we collect it? Is its marginal decision value worth its privacy, security, operational, and customer costs?

These questions should be answered for raw events, derived features, and linkages independently. A platform may need a precise event briefly to investigate fraud while retaining only an aggregate for longer-term measurement. It may need to prohibit reuse of an integrity identifier for advertising or employee analytics. Retention should follow a named decision need, not the convenience of cheap storage.

Data minimization also improves defense. Unbounded telemetry expands access paths, incident impact, deletion obligations, feature ambiguity, and the number of spurious correlations an investigator can mistake for meaning.

Evidence quality is a production concern

A model can be healthy while its evidence pipeline is broken. Integrity observability needs monitors for:

• event volume, delay, duplication, and schema changes;
• feature distributions by client version, geography, and relevant user group;
• join rates and identity-link stability;
• label age, provenance, and leakage;
• access to sensitive fields and unexpected downstream consumers;
• the rate at which reviewers find that evidence packages cannot reproduce decisions.

Preserve the version of the feature logic, policy, model, thresholds, and relevant evidence used at decision time. Recomputing a score months later with current data is not the same as explaining the original decision.

A decision procedure for new telemetry

When a team proposes a new signal:

1. Name the harmful outcome and the decision the signal will improve.
2. Write the direct observation separately from every inferred claim.
3. Identify who controls the source and how ordinary users produce false or missing values.
4. Test incremental value against existing evidence, not merely correlation with old labels.
5. Segment error analysis across relevant populations and client conditions.
6. Assign an action ceiling and require corroboration above it.
7. Minimize collection, access, reuse, and retention.
8. Instrument pipeline health and decision reproduction.
9. Define the signal’s retirement condition before it becomes permanent infrastructure.

The output is not a warehouse full of facts. It is a governed evidence supply chain.

Research trail

• NIST Privacy Framework note
• NIST AI Risk Management Framework note
• The Abuse Sharing Economy note
• Temporary IPv6 addresses note

Review questions

1. Can every enforcement feature be traced back to a direct observation and transformation version?
2. Which signal is currently being used above the action ceiling its reliability warrants?
3. What missing telemetry would reveal a collection failure rather than an attacker?
4. Which retained identifier creates more future risk than present decision value?

8. How Automation Interacts With Platforms

Map capabilities and trust boundaries without confusing mechanism with legitimacy.

The same marketplace purchase can be produced by a person clicking a button, an accessibility tool generating input, a browser test, a script calling an API, or an AI agent acting under delegated authority. At the server, these paths eventually become requests that attempt to change platform state.

The mechanism matters because it changes observability and attack cost. It does not, by itself, determine whether the action is permitted or harmful.

Start with the action, then trace the path

For every high-value action, map:

principal -> authority -> controller -> client -> protocol -> service -> state change

The principal is the account or actor represented to the platform. Authority describes what that principal may do. The controller may be a person, script, organization, or agent. The client packages actions. The protocol carries them. The service validates them and changes state.

This map prevents two common errors. First, teams sometimes put policy in a client that an adversary controls. Second, they classify an entire surface—an API, emulator, or automation framework—as abusive even when legitimate users depend on it.

Browsers and direct HTTP clients

Browser automation is ordinary infrastructure. The W3C WebDriver specification defines a remote-control interface for browsers and identifies testing, monitoring, and load testing as intended uses. A browser under automation can navigate, locate elements, execute scripts, and generate input through standardized commands.

A direct HTTP client skips the rendered interface and constructs requests itself. This can be cheaper and faster, but the server’s security obligation is unchanged: authorize each meaningful operation, validate state transitions, enforce economic and concurrency constraints, and distrust client-supplied facts it can calculate itself.

Signals that a particular automation interface is active can be useful context. They are not stable proof of abuse. Cooperative software may declare itself; adversarial software may not; legitimate tools may resemble either. The Robots Exclusion Protocol illustrates the same boundary for crawlers: declaration communicates a preference but is not access authorization.

APIs are policy surfaces

An official API makes automation legible. It can express scoped permissions, quotas, idempotency, audit identifiers, application identity, and revocation. This does more than improve developer experience: it lets a platform distinguish automation operating inside an explicit contract from clients improvising against a consumer interface.

But “used the API” still does not mean “safe.” A legitimate credential can be stolen, an authorized application can exceed user expectations, and individually valid calls can form a harmful campaign. API governance needs both authorization controls and behavior controls.

For human-directed and AI agents, define an authority envelope:

• which principal delegated authority;
• which actions and resources are in scope;
• monetary, volume, and time limits;
• whether further delegation is allowed;
• how the platform displays, revokes, and audits the authority;
• which actions require renewed confirmation.

This is stronger than trying to infer afterward whether each request “looked human.”

Persistent protocols do not create trust

WebSockets, streaming APIs, and native binary protocols keep sessions open and can support rapid sequences of actions. Persistence changes rate measurement and incident containment: a single authenticated connection may produce thousands of state transitions without repeated handshakes.

It does not make client claims more trustworthy. Authorization, quotas, sequencing rules, and server-side invariants must apply to messages and state transitions, not only to connection establishment.

Native, mobile, and game clients

Native clients can expose richer signals than browsers: build identity, local integrity state, input streams, device properties, and platform attestations. They can also be modified, instrumented, emulated, or run in unexpected environments. The defender should assume that a motivated user controls the endpoint while still recognizing that raising client-manipulation cost can be valuable.

The architectural priority is server authority over scarce or fairness-critical state. A game server should decide whether movement and resource changes obey the game’s rules. A marketplace should decide whether inventory remains available and a purchase satisfies limits. A financial service should decide whether an instruction is authorized and within risk controls.

Client integrity mechanisms can add evidence and raise cost; they should not substitute for enforceable server invariants. Chapter 14 develops this tradeoff.

Input automation, accessibility, and emulation

Macros, assistive technologies, remote desktops, emulators, and testing harnesses can produce atypical timing or interaction sequences. Their users may be disabled customers, support engineers, developers, commercial operators, or attackers.

Mechanism-only policy is sometimes justified—for example, a competitive game may prohibit all macros that alter the intended skill test. But that is a product and fairness decision, not a discovery that unusual input is inherently malicious. The platform should specify the protected outcome and provide an accessible path wherever possible.

An emulator similarly establishes an execution environment, not intent. If the environment undermines a required security control, restrict the capability that depends on that control. Avoid using the label as a proxy for guilt across unrelated actions.

Declared identity requires verification

Some beneficial automation identifies itself. A crawler may send a recognizable user-agent string; a partner integration may present an application credential. Declaration improves coordination only when the platform can verify it. Google’s crawler verification guidance, for example, tells site operators to verify source networks or DNS rather than relying only on a request’s name.

This produces three useful categories:

None of the categories proves beneficial intent over time. Verification answers “which integration?”; monitoring still asks “what is it doing?”

Review each surface systematically

For each client and protocol:

1. Identify the valuable state changes it can request.
2. Identify the principal and how authority is established, scoped, and revoked.
3. Mark every client-supplied claim the server cannot independently verify.
4. Define server-side invariants for safety, scarcity, fairness, and accounting.
5. Inventory legitimate automation, testing, accessibility, and partner use.
6. Decide which mechanism signals are corroboration and which are policy facts.
7. Add rate, economic, and campaign-level controls above individual requests.
8. Design containment that can revoke authority or capability without unnecessarily destroying identity.

The purpose of this map is defensive design. It does not require publishing bypass recipes, exploit chains, or platform-specific evasion instructions. A useful integrity book should explain where trust fails without lowering the cost of abuse.

Research trail

• W3C WebDriver note
• IETF Robots Exclusion Protocol note
• Google crawler verification note
• Acceptable automation

Review questions

1. Which platform invariant currently depends on honest client behavior?
2. Which legitimate users would be harmed by treating an automation surface as an abuse category?
3. Can an agent’s authority be understood and revoked independently of its account credential?
4. Does verifying an integration identity change its limits, or merely its label?

9. Identity and Attribution

Act on the entity the evidence identifies—not the person an identifier merely suggests.

A hundred abusive accounts share an IP address. Is that a bot farm, a university residence, a mobile carrier, a corporate network, a VPN exit, or some mixture of them?

The platform has found a relationship. It has not yet identified the actor.

Keep the entity ladder intact

Integrity systems routinely collapse several layers:

request -> session -> client instance -> device -> account -> person -> organization

A request is one attempted interaction. A session groups interactions under a platform convention. A client instance is an installation or storage context. A device is a physical or virtual execution environment. An account is a platform principal. A person may control several accounts or share one. An organization coordinates people, infrastructure, and capital.

These are relationships, not synonyms.

This table should influence enforcement scope. If the strongest claim concerns a session, revoke or challenge the session. If it concerns a transaction, hold the transaction. Expanding immediately to every account ever connected to an IP turns a weak relation into collective punishment.

Network addresses are routing evidence

IP addresses are useful. They support velocity controls, rough network reputation, incident correlation, and graph investigation. They are not durable personal identifiers.

RFC 6598 reserves shared IPv4 address space specifically to support carrier-grade network address translation. Many subscribers can therefore appear behind shared network infrastructure. In the other direction, one device may change networks frequently. RFC 8981 defines temporary IPv6 addresses whose identifiers vary over time to reduce cross-activity correlation.

The practical result is asymmetric:

• a shared address can merge unrelated actors;
• changing addresses can split one actor into many apparent identities;
• a proxy can deliberately separate the apparent network from the controller;
• a compromised host can make an innocent device part of abusive infrastructure.

IP evidence remains valuable when its claim is stated narrowly and combined with time, protocol, account, device, payment, and behavioral context.

Authentication, proofing, and authorization answer different questions

NIST’s Digital Identity Guidelines separate identity proofing, authentication, and federation into different assurance processes. In its model, a verifier confirms possession and control of authenticators bound to a subscriber account; a relying party then uses identity information and other factors for authorization decisions.

That separation matters for integrity engineering:

• Identity proofing: what evidence supports the claimed real-world identity?
• Authentication: what confidence do we have that the claimant controls the account’s authenticator now?
• Authorization: what may this authenticated principal do?
• Attribution: what evidence links the observed harmful activity to a controller or organization?

Strong authentication can reduce account takeover while leaving automated misuse by the legitimate account holder untouched. Strong proofing can raise account replacement cost while creating exclusion, privacy, and recovery burdens. Neither proves intent.

An account’s controller can also change over time through theft, sale, delegation, shared use, or recovery. Attribution edges therefore need timestamps, not just confidence scores.

Model attribution as a graph of claims

Represent each link explicitly:

(entity A) -[relationship, source, time window, confidence]-> (entity B)

Examples include “session authenticated to account,” “requests used the same payment token,” “device estimate observed both accounts,” or “reviewer confirmed coordinated operation.” Store the source and time window because relationships decay and methods change.

Confidence should reflect competing explanations, not only match strength. Two accounts sharing a rare device signal and a payment instrument in the same hour may support a stronger link than two accounts using the same IP over a year. Negative evidence matters too: simultaneous activity in distant contexts may contradict a single-device hypothesis, while a public cloud range may explain a dense network hub.

Do not convert the graph into guilt by association. High-degree infrastructure—mobile gateways, workplaces, libraries, payment processors, and hosting providers—naturally connects unrelated actors. Graph expansion should usually produce candidates for additional evidence, not automatic enforcement.

Separate coordinated control from common characteristics

Integrity teams often seek “account farms,” but several phenomena can look similar:

• one operator controls many accounts;
• many contractors follow the same playbook;
• unrelated users run the same commercial tool;
• compromised accounts receive commands from shared infrastructure;
• ordinary users converge on a popular strategy;
• a platform experiment or software update changes behavior at once.

Coordination is a claim about dependence. Similarity is only one possible sign. Stronger evidence includes shared scarce resources, synchronized action serving a common objective, money flows, repeated handoffs, and consistent infrastructure reuse—evaluated against benign base rates.

The 2013 study Trafficking Fraudulent Accounts documented specialized participants creating, selling, and using accounts. Its enduring lesson is that the account visible to a platform may be one replaceable asset in a larger production chain. The platform may need to intervene against the capability or economic network, not merely delete the presented account.

Choose the intervention scope by claim strength

When deciding whether to act on a request, session, account, device cluster, household, or organization, ask:

1. Which entity is directly implicated in the harmful event?
2. Which links are observed, which are derived, and which are inferred?
3. What benign process could produce each link?
4. How current is the relationship?
5. What additional harm will occur while evidence is gathered?
6. Can the intervention be narrowed to the capability or value transfer at risk?
7. How will a wrongly linked user separate themselves and recover?

Confidence and scope interact. An uncertain account link may justify observation. A confirmed compromised session may justify immediate revocation. Organization-wide removal demands evidence of coordinated control and policy responsibility proportionate to that breadth.

Recourse must be able to repair attribution

An appeal system that only asks “was the model score correct?” cannot fix a mistaken identity graph. Reviewers need to see which entity was acted upon, the links that expanded the action, their age and source, and plausible shared-infrastructure explanations.

Correction must propagate. If a device cluster is split, future features, watchlists, reviewer tools, and training labels should not silently preserve the old association. Otherwise an overturned decision becomes a permanent shadow record.

Attribution is probabilistic, temporal, and purpose-specific. Good integrity systems make that uncertainty operational instead of hiding it behind an identifier.

Research trail

• NIST Digital Identity Guidelines note
• Shared IPv4 address space note
• Temporary IPv6 addresses note
• Fraudulent-account markets note
• The Abuse Sharing Economy note

Review questions

1. Which production identifier is treated as a person even though it identifies something narrower?
2. Where can shared infrastructure cause enforcement to expand across innocent users?
3. Does the attribution graph preserve time, source, uncertainty, and contradictory evidence?
4. Can an appeal actually remove a bad link from every downstream system?

10. Rules and Heuristics

Use simple evidence where it is sufficient, observable, and maintainable.

Reader outcome

Design, validate, version, and retire rules without confusing explainability with correctness.

Scope

Velocity, timing constraints, repetition, navigation, session properties, impossible states, and economic activity.

Decision questions

• Is the rule a hard invariant, risk indicator, or temporary patch?
• What legitimate populations violate it?
• How quickly will it decay or invite evasion?
• Can its trigger and downstream action be audited separately?

Evidence needed

• Rule lifecycle and shadow-evaluation examples
• Base rates and conditional precision
• Operational costs of overlapping rules
• Attacker adaptation after observable thresholds

11. Behavioral Detection

Infer patterns without treating statistical human-likeness as legitimacy.

Reader outcome

Evaluate timing, entropy, sequences, trajectories, dwell time, and reaction patterns for robustness and legitimate-user variation.

Decision questions

• Which behavioral differences are causal, incidental, or population-specific?
• Can accessibility tools or expert users resemble automation?
• Can human labor or imitation make harmful behavior look human?
• How does the feature change when users learn, interfaces change, or latency shifts?

Evidence needed

• Studies with representative populations
• Temporal and cross-product validation
• Accessibility and expert-user counterexamples
• Adaptation and concept-drift measurements

12. Statistical and Machine-Learning Detection

Optimize calibrated decisions and ecosystem outcomes, not leaderboard accuracy.

Reader outcome

Choose and evaluate classification, anomaly detection, clustering, and supervised or unsupervised methods against base rates, costs, and operational constraints.

Decision questions

• Where do labels come from and which enforcement bias do they encode?
• Is probability calibrated for this population and current time?
• What precision is achievable at the real base rate?
• How will drift, feedback loops, evasion, and delayed labels be detected?

Evidence needed

• Precision-recall and calibration studies
• Offline-to-online failure cases
• Model governance and feature provenance
• Cost-sensitive and selective-classification methods

13. Graph Detection

Find coordination while resisting guilt by association.

Reader outcome

Model accounts, devices, payments, networks, transactions, interactions, and messages as a temporal graph with confidence-bearing edges.

Decision questions

• Which edges indicate coordination versus ordinary shared infrastructure?
• How do hub nodes such as VPNs, schools, cafés, and payment processors distort clusters?
• What action is justified on a node because of its neighbors?
• How are stale edges decayed, explained, and contested?

Evidence needed

• Coordinated-abuse and fraud-network research
• Graph evaluation with ground-truth limitations
• Temporal graph and entity-resolution methods
• Collateral enforcement case studies

14. Client Integrity and Attestation

Use client assurance only for claims it can support and costs the ecosystem will accept.

Reader outcome

Compare server authority, code signing, secure boot, anti-tamper, attestation, and privileged anti-cheat by guarantee, bypass cost, privacy, compatibility, and maintenance.

Decision questions

• What exact claim is attested and what remains outside the trust boundary?
• Does the mechanism increase platform attack surface or privilege risk?
• Which devices and legitimate configurations are excluded?
• Can server-authoritative design remove the need for invasive assurance?

Evidence needed

• Trusted-computing and attestation specifications
• Security reviews and bypass histories at a safe abstraction
• Compatibility and privacy impact evidence
• Gaming and high-assurance case comparisons

15. The Intervention Ladder

Turn evidence into graduated, proportionate, and reversible action—not a BAN bit.

Candidate ladder

observe -> gather evidence -> rate limit -> add friction -> challenge
        -> restrict capability -> remove rewards -> suspend
        -> manual review -> permanent removal

Reader outcome

Match action to harm, confidence, urgency, scope, reversibility, and collateral cost.

Decision questions

• Can a narrow capability restriction control harm better than identity-wide punishment?
• Which actions preserve evidence and appealability?
• When does friction burden legitimate users more than adversaries?
• What automatically expires, escalates, or rolls back?

Evidence needed

• Intervention experiments and unintended consequences
• Friction and challenge completion by user population
• Reoffense, displacement, and appeal outcomes
• Procedural safeguards for high-severity actions

16. Real-Time and After-the-Fact Enforcement

Treat time-to-decision as a harm-dependent system budget.

Reader outcome

Set a detection latency budget and divide decisions among synchronous prevention, asynchronous investigation, and retrospective remediation.

Decision questions

• How does harm accumulate while the platform waits?
• Which evidence becomes available only after more activity?
• Can a reversible hold buy time without imposing final punishment?
• How do queues, review capacity, and delayed labels affect the real budget?

Evidence needed

• Fraud, scarcity, competitive integrity, and network-abuse timelines
• Value-of-information models
• Streaming versus batch operational costs
• Consequences of premature and delayed action

17. Appeals and Human Review

Make recourse, evidence, and reviewer consistency part of the enforcement architecture.

Reader outcome

Design evidence packages, review queues, escalation paths, audit trails, explanations, and correction mechanisms.

Decision questions

• What evidence can a reviewer understand and independently verify?
• Which decisions require dual control or specialist escalation?
• How are reviewer disagreement and consistency measured?
• How do overturned decisions repair accounts, models, policies, and downstream effects?

Evidence needed

• Review-system and content-moderation operations research
• Appeal overturn and time-to-resolution data
• Human factors, fatigue, and consistency evidence
• Feedback-loop risks when appeal labels train detectors

18. The Transparency Paradox

Explain enough for legitimacy and recourse without creating a cheap evasion oracle.

Reader outcome

Separate public policy transparency, individual decision explanation, evidence disclosure, and defensive-mechanism secrecy.

Decision questions

• What must a user know to understand and contest a decision?
• Which detail would materially reduce defense cost for attackers?
• Can reasons be categorical while evidence is protected?
• How do explanations change for warnings, restrictions, and permanent removal?

Evidence needed

• Procedural-justice and explanation research
• Regulatory transparency requirements
• Security disclosure and gaming examples
• User trust and evasion outcomes

19. Metrics That Actually Matter

Measure prevented harm and legitimate-user burden across the whole decision system.

Reader outcome

Build a scorecard spanning harm, errors, friction, latency, recurrence, support, infrastructure, economics, and adaptation.

Candidate measures

Prevented harm, false-positive cost, appeal overturn, attacker replacement cost, legitimate-user friction, time to detection, time to intervention, recurrence, displacement, support burden, and defense cost.

Decision questions

• What counterfactual establishes “prevented” harm?
• Which metric becomes a target attackers or teams can game?
• How are uncertainty and delayed outcomes represented?
• Can aggregate success hide concentrated harm to a user group?

Evidence needed

• Causal evaluation and experimentation methods
• Metric gaming and feedback-loop cases
• Trust-and-safety or fraud scorecards
• Longitudinal displacement measurement

20. The Attacker Adapts

Operate defense as a repeated game in which interventions reveal information.

Core cycle

attack -> observe -> infer -> intervene -> attacker learns -> new attack

Reader outcome

Anticipate evasion, displacement, probing, poisoning, imitation, and business-model change after a defense ships.

Decision questions

• What feedback does each challenge, error, delay, or ban expose?
• Can defenses be varied without sacrificing policy consistency?
• How are novel attacks distinguished from ordinary drift?
• When should the platform tolerate low-level abuse to protect a higher-value signal?

Evidence needed

• Adversarial ML, spam, fraud, and security adaptation histories
• Controlled rollout and holdout strategies
• Defender learning-loop case studies
• Evidence on displacement rather than account-level recurrence alone

21. Running the Abuse Organization

Assign decision rights across product, engineering, data, security, legal, operations, and support.

Reader outcome

Design ownership, escalation, launch review, incident response, quality assurance, and accountability for an adversarial platform program.

Decision questions

• Who owns policy, evidence standards, models, interventions, and appeals?
• Who can authorize high-severity or bulk enforcement?
• How are product friction and prevented harm traded off?
• What independent review catches confirmation bias or metric gaming?
• How do teams investigate mistakes without exposing sensitive defenses broadly?

Evidence needed

• Trust and safety, fraud, anti-cheat, and security operating models
• Decision-right and incident-management patterns
• Quality-control and reviewer-wellbeing research
• Organizational failure case studies

Gaming Ecosystems

Competitive integrity, virtual economies, client trust, and uneven definitions of legitimate play.

Canvas focus

• Actors: players, studios, platform operators, tournament organizers, creators, account sellers, cheat and farming suppliers.
• Harms: unfair competition, ruined matches, economic extraction, account compromise, support cost, and player exit.
• Policy tension: accessibility, mods, macros, secondary markets, single-player modification, and competitive play have different stakes.
• Urgency: active-match cheating may require seconds; farming networks may support weeks of investigation.
• Interventions: server-authoritative correction, matchmaking separation, reward removal, capability restriction, suspension, and tournament review.

Research questions

• What evidence connects cheating prevalence to retention and trust?
• When are client integrity controls proportionate to the competitive stakes?
• How should linked household or café devices affect attribution?
• What differs among console, mobile, PC, and cloud gaming?

Comparative lesson sought

The same automation can be a benign mod, an accessibility aid, an efficiency macro, or competitive cheating depending on the governed mode and harm.

Dating Platforms

Spam, scams, mass interaction, scraping, synthetic personas, and safety-sensitive identity.

Canvas focus

• Actors: daters, platforms, scammers, marketers, scrapers, safety teams, payment and identity providers.
• Harms: financial fraud, harassment, privacy invasion, artificial engagement, emotional harm, and degraded matching markets.
• Policy tension: high-volume but sincere behavior, assistive writing, AI-generated profiles, travel, shared networks, and commercial solicitation.
• Observability: conversations and identity signals are sensitive; collection and review impose privacy and wellbeing costs.
• Recourse: explanations must protect reporters and detection methods while permitting meaningful contest.

Research questions

• Which behaviors predict harm rather than mere unusualness?
• How does intervention timing change victim exposure?
• What safety burden is transferred to users through challenges and reporting?
• How should synthetic media and AI assistance change disclosure policy?

E-commerce Marketplaces

Scarcity bots, seller networks, review manipulation, account takeover, and legitimate commercial automation.

Canvas focus

• Actors: buyers, sellers, marketplace, brands, logistics providers, resellers, fraud rings, and automation vendors.
• Harms: inventory capture, price distortion, counterfeit trade, review fraud, payment loss, and seller exclusion.
• Policy tension: authorized seller APIs and inventory tools coexist with prohibited scraping, purchasing, and manipulation.
• Economics: margins, account acquisition, payment instruments, fulfillment, and resale liquidity determine attack scale.
• Interventions: purchase limits, holds, queueing, reward removal, listing restrictions, payment review, and network enforcement.

Research questions

• Does friction allocate scarcity fairly or merely favor sophisticated users?
• Which graph relationships justify action against a seller network?
• How is displaced abuse measured across accounts and marketplaces?
• What obligations arise when enforcement freezes seller funds or livelihoods?

Financial Markets

An ecosystem where automation is normal and the governed categories are conduct, access, and market effect.

Canvas focus

• Actors: investors, brokers, exchanges, market makers, automated traders, surveillance teams, and regulators.
• Harms: manipulation, unfair access, operational instability, fraud, and loss of market integrity.
• Policy tension: speed and algorithmic execution are legitimate; prohibited conduct cannot be defined as automation itself.
• Observability: order lifecycle, ownership, timing, venues, and beneficial control may be distributed across institutions.
• Authority: interventions span platform controls, contractual rules, supervision, and jurisdiction-specific regulation.

Research questions

• Which lessons transfer to consumer platforms and which depend on regulated market structure?
• How are patterns separated from manipulative purpose or effect?
• What evidence standard applies to real-time controls versus formal sanction?
• How do cross-venue and linked-entity limits shape attribution?

False-Positive Cost

Decision problem

How strong must evidence be before a particular intervention is justified?

Working model

Error cost depends on more than count. Include action severity, duration, reversibility, affected assets, dependency on the platform, spillover to linked users, support burden, and trust loss.

FP cost(action, user, context, time)
!= one universal constant

Research questions

• How should platforms measure harms that affected users cannot report?
• When is appeal overturn a useful lower bound rather than the false-positive rate?
• How do error costs differ across warnings, friction, capability restrictions, and account removal?
• Which subgroups bear concentrated error or challenge burden?

Affected chapters

Chapters 6, 12, 15, 17, and 19.

Detection Latency

Decision problem

How long can the platform wait for better evidence before harm becomes unacceptable?

Working model

value of waiting = expected reduction in decision error
                 - expected harm accumulated while waiting
                 - operational and user cost of temporary controls

Separate signal availability, feature computation, model inference, decision queue, human review, action propagation, and appeal latency.

Research questions

• Which harms are step functions, compounding processes, or recoverable losses?
• When can a reversible hold purchase information safely?
• How should review backlogs change automated thresholds?
• What is the correct latency SLO when labels arrive weeks later?

Affected chapters

Chapters 6, 7, 15, 16, 17, and 19.

Appeals as System Feedback

Decision problem

How can recourse correct individual harm and improve the decision system without turning appeals into a biased label stream?

Working model

An appeal is selected by awareness, language, effort, stakes, account access, and belief that review is meaningful. Therefore overturn rate is informative but is not the full false-positive rate.

Research questions

• Which users never appeal and how can their errors be detected?
• How should reviewer evidence differ from model features?
• How are corrections propagated to linked sanctions, rewards, and training data?
• Can explanation quality improve valid appeals without giving adversaries a precise oracle?

Affected chapters

Chapters 6, 17, 18, 19, and 21.

Adversarial Adaptation

Decision problem

How should a platform evaluate a defense when attackers respond strategically rather than remaining a fixed test population?

Working model

Interventions change attacker cost, information, capacity, route choice, and target selection. Evaluation must measure ecosystem-level harm and displacement over time, not only repeat activity by sanctioned accounts.

Research questions

• Which response details create the most valuable attacker feedback?
• How can long-lived holdouts be ethical and operationally safe?
• What distinguishes evasion from benign distribution shift?
• When does raising cost select for more capable and harmful adversaries?

Affected chapters

Chapters 3, 10–16, 19, and 20.

Research Guide

Research exists to change or support a decision in the book. Do not collect links without a claim, limitation, and destination.

Workflow

1. Capture a source using templates/research-note.md.
2. Verify citation, authorship, date, source type, and stable URL.
3. Separate what the source says from our implications.
4. Record assumptions, weaknesses, disagreement, and publication-safety concerns.
5. Link affected chapters and source indexes.
6. Promote a claim to BOOK.md only after review.

Directory map

• gaming/, marketplaces/, financial-markets/, social-platforms/ — ecosystem evidence
• legal/, privacy/ — jurisdiction-specific authority and constraints
• fraud/, bot-detection/, ml/, human-verification/ — mechanism and evaluation research
• identity/ — attribution, device, account, and organization linkage

File naming

Use YYYY-author-short-title.md where a year and author exist. Use one note per source. If a source changes over time, record the version or capture date in the citation.

Evidence labels

• Primary: original paper, dataset, law, decision, specification, policy, transparency report, or first-party incident account.
• Secondary: synthesis or reporting that accurately links primary evidence.
• Commentary: expert or practitioner judgment useful for hypotheses but not sufficient alone for factual claims.

Review gate

A note is checked only when citation and material claims were verified against the original. It is synthesized only when its implications and disagreements are reflected in BOOK.md, a concept, or a chapter brief.

Source Indexes

These files are navigation indexes. Detailed summaries belong in research/ so one source has one canonical note.

• Books
• Papers and datasets
• Law, regulation, and policy
• Patents and technical disclosures
• Talks and practitioner material

Each index entry should link the canonical research note and identify the chapters it informs.

Books

Papers and Datasets

Law, Regulation, and Platform Policy

Every entry must include jurisdiction, effective/version date, and whether it is primary authority or commentary.

Patents and Technical Disclosures

Patents describe claims and possible mechanisms; they do not prove deployment, effectiveness, legality, or current product behavior.

Talks and Practitioner Material

Use practitioner material for operational hypotheses, terminology, and source discovery. Preserve conflicts of interest and seek primary corroboration for factual claims.

Research Agenda

First research milestone: Parts I and II

The first corpus should establish definitions, harm, economics, acceptable automation, authority, and error cost before detection mechanisms receive deep treatment.

Workstream A — Definitions and policy comparison

• Collect versioned automation/bot policies from contrasting platform categories.
• Identify which rules govern mechanism, behavior, outcome, scale, identity, or authorization.
• Record accessibility, testing, research, API, and commercial exceptions.

Workstream B — Harm evidence

• Find primary measurements connecting automation or coordinated abuse to user, market, infrastructure, and trust outcomes.
• Distinguish platform assertions from causal or longitudinal evidence.
• Collect examples where automated participation produces value.

Workstream C — Economics

• Research attacker unit economics, account and infrastructure markets, defender cost, displacement, and replacement time.
• Prefer ranges and sensitivity analysis to unsupported point estimates.

Workstream D — Authority and rights

• Choose initial jurisdictions explicitly.
• Map data collection, profiling, automated decision, explanation, contractual remedy, purchased goods, and appeal constraints.
• Require specialist review before the manuscript presents legal conclusions.

Workstream E — Decision cost

• Gather calibrated examples showing base-rate effects and action-dependent thresholds.
• Research appeal selection bias, concentrated error burdens, and reversible interventions.

Exit criteria

Parts I and II can enter prose drafting when each chapter has:

• at least five checked primary sources spanning technical and nontechnical evidence;
• at least one meaningful counterexample or disagreement;
• a completed canvas slice;
• a decision procedure with known residual uncertainty;
• a reviewed publication-safety note.

Fighting the Forever War: Trust and Safety Engineering

Citation

Alex Stamos and Shelby Grossman with Riana Pfefferkorn. Fighting the Forever War: Trust and Safety Engineering. Stanford Internet Observatory, living manuscript initially written 2019–2025. Open textbook. Accessed 2026-08-14.

Source type: open textbook / practitioner synthesis
Evidence quality: secondary synthesis with first-hand practitioner experience
Review status: checked

What it says

Trust and Safety is a cross-functional discipline for anticipating and responding to technology-enabled harm. Relevant functions include policy, product, operations, engineering, data science, legal, research, communications, and threat assessment. The introduction recommends prioritizing by harm, prevalence, and product responsibility rather than attention alone, and distinguishes proactive product design from reactive enforcement.

Important claims

• A technically correct use of a platform can still cause harm; this distinguishes many integrity problems from conventional control-breaking security failures.
• Measurement should cover prevalence, response efficiency, and impact, while acknowledging that user reports are selected and incomplete.
• Safety tradeoffs are unavoidable: privacy, accountability, expression, cost, speed, and accuracy cannot all be maximized independently.

Assumptions and weaknesses

The manuscript is explicitly a work in progress and says it does not fully reflect AI’s role. Its center of gravity is user-generated content and serious interpersonal/societal harm, not all transactional fraud, gaming, or automated resource abuse. Practitioner cases are valuable but do not always establish causal effects.

Implications for platform defense

Use it to map functions and tradeoffs, not as the structure of this book. Platform Integrity should extend the evidence-to-decision framework across fraud, automation, marketplaces, gaming, and agents.

Chapters this affects

Chapters 1, 2, 4–7, 15–21.

RFC 9309: Robots Exclusion Protocol

Citation

M. Koster, G. Illyes, H. Zeller, and L. Sassman. “Robots Exclusion Protocol.” IETF RFC 9309, September 2022. RFC Editor. Accessed 2026-08-14.

Source type: technical standard
Evidence quality: primary
Review status: checked

What it says

The protocol lets service owners publish rules that cooperative automated crawlers are requested to honor when accessing URIs. The standard explicitly says these rules are not access authorization and are not a substitute for security controls.

Important claims

• A crawler is an automated client, yet the standard assumes automation can be identified and governed cooperatively.
• Identity declared in a user-agent is part of policy communication, not strong attribution.
• A service can express different rules for different crawler product tokens.

Weaknesses / disagreements

Non-cooperative clients can ignore or spoof the protocol. Compliance does not establish beneficial intent, and noncompliance alone does not resolve authorization or legal questions.

Implications for platform defense

Automation can be a first-class authorized participant. Policy, identity, and enforcement cannot be collapsed into human-versus-bot classification.

Chapters this affects

Chapters 1, 4, 7–10, and 18.

GitHub Acceptable Use Policies

Citation

GitHub. “GitHub Acceptable Use Policies.” Current online policy. GitHub Docs. Accessed 2026-08-14.

Source type: platform policy
Evidence quality: primary for GitHub’s stated policy
Review status: checked

What it says

GitHub prohibits excessive automated bulk activity, coordinated inauthentic activity, fake accounts, automated inauthentic interactions, rank abuse, phishing, and automation that imposes undue server load. It separately permits some uses of public information, including open research, subject to privacy and other restrictions.

Important claims

The governed attributes are excess, burden, deception, inauthenticity, and harmful purpose—not automation alone. The same platform operates APIs and automation products while restricting particular automated outcomes.

Assumptions and weaknesses

This is a contractual policy, not evidence that enforcement is complete, consistent, or legally valid in every jurisdiction. The current page may change; citations require an access date or archived version.

Implications for platform defense

A durable policy should name harm-relevant behavior and authorization paths rather than ban a technical mechanism indiscriminately.

Chapters this affects

Chapters 1, 2, 4, 5, 10, and 15.

Steam Subscriber Agreement: Automation

Citation

Valve Corporation. “Steam Subscriber Agreement,” section 4.C–D, current online agreement. Steam. Accessed 2026-08-14.

Source type: platform contract
Evidence quality: primary for Valve’s stated terms
Review status: checked

What it says

The agreement broadly prohibits scripts, bots, macros, and other non-human-controlled systems from interacting with Steam content and services, including automated account creation, fabricated statistics, reward accumulation without genuine input, and automated participation in adjudication. It states that enforcement may combine automated detection and human review and may restrict accounts or subscriptions.

Important claims

Valve defines a mechanism category (“Automation”) but illustrates it through consequences involving authenticity, rewards, and judgment. The contract also shows that enforcement affects purchased subscriptions and account access, increasing the cost of error.

Assumptions and weaknesses

Contract wording does not demonstrate detector accuracy, actual enforcement practice, or the treatment of accessibility tools and authorized integrations. Jurisdiction and the version accepted by a user matter.

Implications for platform defense

Broad mechanism bans may be operationally convenient but need exceptions, proportional enforcement, evidence standards, and recourse appropriate to affected assets.

Chapters this affects

Chapters 1, 4–6, 14, 15, and the gaming case study.

FTC’s First BOTS Act Cases

Citation

U.S. Federal Trade Commission. “FTC Brings First-Ever Cases Under the BOTS Act,” 22 January 2021. FTC. Accessed 2026-08-14.

Source type: regulator enforcement announcement
Evidence quality: primary for allegations and proposed settlements
Review status: checked

What it says

The FTC alleged that three ticket brokers violated the Better Online Ticket Sales Act by circumventing posted purchase limits using automated purchasing software, IP concealment, and many fictitious accounts and payment cards. The agency reported purchases exceeding 150,000 tickets and millions of dollars in resale revenue.

Important claims

The prohibited conduct is not simply fast purchasing or software use. It combines circumvention of access controls and purchase rules with a specific ticket-market context. Multiple identities and payment instruments were part of one economic operation.

Assumptions and weaknesses

An enforcement announcement presents the regulator’s case and settlement, not a general causal study of ticket markets. The BOTS Act is narrow U.S. legislation and should not be generalized to all automation or jurisdictions.

Implications for platform defense

Policy, law, identity, graph relationships, and market harm can be more decisive than proving whether an individual request was automated.

Chapters this affects

Chapters 1–5, 9, 13, 15, and the marketplace case study.

Why Information Security Is Hard: An Economic Perspective

Citation

Ross Anderson. “Why Information Security is Hard—An Economic Perspective.” ACSAC, 2001. University of Cambridge PDF. Accessed 2026-08-14.

Source type: research paper
Evidence quality: primary analytical work
Review status: checked

What it says

Many security failures are better explained by incentives than by missing technical mechanisms. The paper applies externalities, asymmetric information, moral hazard, adverse selection, liability dumping, and network economics to cases where those able to protect a system do not bear failure costs.

Important claims

• Security investment follows incentives and liability allocation, not social harm automatically.
• Network effects and lock-in can reward insecure products.
• Technical correctness cannot compensate for actors being rewarded for the wrong outcome.

Weaknesses / disagreements

The examples are historically situated, and the paper is conceptual rather than an empirical model of modern platform abuse. It primarily examines defender and market incentives, not every attacker production function.

Implications for platform defense

The canvas must identify who controls a risk, who bears harm, who pays for mitigation, and whether metrics reward ecosystem integrity or only growth and throughput.

Chapters this affects

Chapters 2, 3, 6, 15, 19–21.

Trafficking Fraudulent Accounts

Citation

Kurt Thomas, Damon McCoy, Chris Grier, Alek Kolcz, and Vern Paxson. “Trafficking Fraudulent Accounts: The Role of the Underground Market in Twitter Spam and Abuse.” USENIX Security, 2013. Google Research. Accessed 2026-08-14.

Source type: measurement research
Evidence quality: primary
Review status: checked

What it says

The authors observed 27 merchants selling fraudulent Twitter accounts over ten months, measured prices and availability, and collaborated with Twitter to identify and disable millions of accounts associated with the market. The observed merchants appeared responsible for a material share of accounts later flagged for spam during active months.

Important claims

Accounts are production inputs with a market price. Registration defenses, account survival, buyer demand, and replacement cost jointly shape abuse economics. Individual accounts are less informative than the supply operation and campaign.

Assumptions and weaknesses

The market, platform, and defenses date to 2012–2013. Merchant coverage was incomplete, classification and takedown relied on platform collaboration, and current prices cannot be inferred from the study.

Implications for platform defense

Measure attacker capacity, account replacement time, campaign survival, and market substitution rather than celebrating raw account-removal counts.

Chapters this affects

Chapters 3, 6, 9, 13, 15, 19, and 20.

The Abuse Sharing Economy

Citation

Kurt Thomas et al. “The Abuse Sharing Economy: Understanding the Limits of Threat Exchanges.” RAID, 2016. Google Research. Accessed 2026-08-14.

Source type: measurement research
Evidence quality: primary
Review status: checked

What it says

Using 45 million IP addresses observed abusing six Google services during a two-week 2015 snapshot, the authors studied infrastructure reuse across spam, fake-account registration, malicious hosting, and other abuse. Cross-service intelligence could identify some abusive traffic, but outright IP blacklisting produced an untenable false-positive volume.

Important claims

• Adversaries reuse infrastructure across abuse types.
• Each service observes a biased slice of the same ecosystem.
• A predictive signal can be useful for added evidence while being unsafe as a direct ban rule.

Assumptions and weaknesses

The observation window was short, limited to Google services, and IP meaning has continued to change with mobile networks, cloud services, proxies, and shared infrastructure.

Implications for platform defense

Separate signal usefulness from enforcement sufficiency. Cross-context reputation can raise risk without establishing that a particular user or action is abusive.

Chapters this affects

Chapters 6, 7, 9, 10, 12, 13, and 15.

EU Digital Services Act: Reasons and Complaints

Citation

European Parliament and Council. Regulation (EU) 2022/2065, Digital Services Act, 19 October 2022. EUR-Lex. Accessed 2026-08-14.

Source type: legislation
Evidence quality: primary authority for covered services in the EU
Review status: checked; specialist legal review still required for manuscript conclusions

What it says

Among its platform obligations, the DSA requires qualifying services to give affected recipients reasons for certain restrictions, including account suspension or termination, and to disclose relevant use of automated means. Covered online platforms must provide internal complaint handling for specified decisions; the regulation requires qualified staff and not solely automated handling for those complaints. Transparency reporting includes complaint outcomes and reversals.

Evidence

The European Commission reports that platforms reversed roughly 30% of 165 million appealed moderation decisions in the first two years of DSA application. This is regulator-reported aggregate operational data, not a universal false-positive rate.

Assumptions and weaknesses

Scope varies by service type and size, and the DSA’s content-moderation categories do not automatically cover every fraud or anti-cheat action. Application to a concrete workflow requires current legal analysis.

Implications for platform defense

Reason codes, automated-method flags, complaint queues, human review, reversals, and audit records are system requirements in some contexts—not optional customer-support polish.

Chapters this affects

Chapters 5, 6, 15, 17–19, and 21.

GDPR Article 22: Automated Individual Decisions

Citation

European Parliament and Council. Regulation (EU) 2016/679, General Data Protection Regulation, Article 22. EUR-Lex. Accessed 2026-08-14.

Source type: legislation
Evidence quality: primary authority
Review status: checked; specialist legal review required

What it says

Article 22 addresses decisions based solely on automated processing, including profiling, that produce legal or similarly significant effects. It contains exceptions and safeguards. For specified contract- or consent-based exceptions, safeguards include the right to obtain human intervention, express a point of view, and contest the decision.

Assumptions and weaknesses

It is inaccurate to describe Article 22 as a universal ban on automation or a universal right to explanation. Whether platform enforcement is solely automated and sufficiently significant is fact- and jurisdiction-dependent, and other GDPR provisions may apply independently.

Implications for platform defense

Record whether automation recommended or made a decision, what meaningful human review occurred, and how a person can contest a consequential outcome. Avoid decorative human review that merely ratifies a score.

Chapters this affects

Chapters 5–7, 12, 15, 17, 18, and 21.

NIST AI Risk Management Framework 1.0

Citation

Elham Tabassi. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1, 26 January 2023. NIST. Accessed 2026-08-14.

Source type: voluntary government framework
Evidence quality: primary framework, not binding law
Review status: checked

What it says

The AI RMF organizes work into Govern, Map, Measure, and Manage. It treats trustworthiness as contextual and multi-dimensional: validity, reliability, safety, security, resilience, accountability, transparency, explainability, privacy, and fairness with harmful bias managed. NIST cautions that optimizing characteristics separately does not ensure a trustworthy system and that tradeoffs depend on affected actors and context.

Assumptions and weaknesses

The framework is use-case agnostic and voluntary. It does not prescribe enforcement thresholds, platform-specific harm definitions, or legal compliance. AI RMF 1.0 is under revision as of 2026.

Implications for platform defense

Evaluate the entire decision system throughout its lifecycle, not only model accuracy. Document purpose, affected populations, validity conditions, monitoring, override, and decommissioning.

Chapters this affects

Chapters 6, 7, 11, 12, 15, 17, 19, and 21.

After Bots: From Bot Detection to Agent Trust

Citation

Kevin Gosschalk. After Bots: From Bot Detection to Agent Trust. Amazon Digital Services / KDP, 2026. ISBN 9798180893611. Google Books record. Accessed 2026-08-14.

Source type: vendor-founder practitioner book
Evidence quality: commentary and operational hypotheses
Review status: captured; full text not yet reviewed

What it says

The publisher description argues that agentic traffic makes human-versus-bot detection insufficient and proposes classifying agent identity and intent, authorizing actions, and changing attacker economics.

Assumptions and weaknesses

Gosschalk is founder and CEO of Arkose Labs, a vendor selling bot and fraud defenses. The book is self-published, recent, and its broad claims—especially that economic deterrence is the only defense that scales—require independent evidence. This note is based on bibliographic and publisher material, not a completed reading.

Implications for platform defense

The delegated-authority question is promising: who or what acts, for whom, under which grant, toward what bounded action? Authorization complements rather than replaces behavioral and compromise detection.

Chapters this affects

Chapters 1, 3, 4, 8, 9, 15, and 20.

NIST Privacy Framework

Citation

National Institute of Standards and Technology. NIST Privacy Framework: A Tool for Improving Privacy Through Enterprise Risk Management, Version 1.0, 2020. Official framework.

What it says

The framework gives organizations a voluntary, risk-based structure for identifying and managing privacy risk. It organizes activities into Identify-P, Govern-P, Control-P, Communicate-P, and Protect-P and treats privacy as an organizational and system-design concern across a data-processing ecosystem.

Important claims

• Data processing can create consequences for individuals even when a system is functioning as intended.
• Privacy risk management belongs alongside enterprise risk management.
• Organizations need defined purposes, governance, data-processing inventories, controls, communication, and protection—not merely a privacy notice.

Evidence

This is normative guidance from NIST, developed through a public process. It is not an empirical evaluation of any one control.

Useful examples

The framework’s profiles can express a current state and target state. For an integrity program, a profile can expose gaps between collecting a persistent identifier and governing its purpose, access, retention, and downstream reuse.

Assumptions

Organizations can translate high-level outcomes into controls appropriate to their legal obligations, risk tolerance, and systems.

Weaknesses / disagreements

The framework is intentionally technology- and jurisdiction-neutral. It does not determine whether a particular integrity signal is lawful, proportionate, or effective, and compliance with it is not a legal safe harbor.

Implications for platform defense

Integrity telemetry should have a named purpose, accountable owner, data-flow map, access policy, retention period, and deletion path. “Useful for fraud” is too broad a purpose for unlimited cross-context linkage. Privacy risk should be assessed for derived identifiers and relationship graphs, not only raw personal data.

Chapters this affects

Chapters 5, 7, 9, 13, 17, 18, 19, and 21.

Questions raised

• How should a platform measure the privacy harm of a false attribution link?
• Which integrity uses should be technically isolated from advertising and personalization?
• When does additional collection have negative marginal decision value?

NIST SP 800-63-4 Digital Identity Guidelines

Citation

National Institute of Standards and Technology. Digital Identity Guidelines, NIST SP 800-63-4, final July 2025. Official online edition.

What it says

The suite provides risk-based requirements for identity proofing, authentication, and federation. Its digital identity model distinguishes applicants, subscribers, claimants, credential service providers, verifiers, identity providers, and relying parties. It assigns separate assurance levels to proofing, authentication, and federation and adds requirements for continuous evaluation, fraud controls, privacy, customer experience, and redress.

Important claims

• Proofing a claimed identity, authenticating control of an authenticator, and making an authorization decision are separate functions.
• Assurance should be selected according to the impacts of identity errors in the protected service.
• A verifier confirms possession and control of authenticators bound to a subscriber account; the relying party uses assertions and other context to grant access.
• Digital identity systems require privacy, usability, and redress considerations in addition to security.

Evidence

This is current US federal technical guidance produced through public drafts and extensive comments. It is authoritative for its intended government scope and useful as a conceptual model elsewhere.

Useful examples

A strongly authenticated account can still be operated abusively by its legitimate controller. Conversely, an account takeover can produce harmful behavior even though the account’s earlier proofing record was accurate. These failures require different interventions.

Assumptions

The service can identify relevant impacts and select controls appropriate to its users and transactions.

Weaknesses / disagreements

The suite explicitly does not cover every machine-to-machine or API use case. Its government-service context does not settle private-platform policy. Higher assurance can also impose cost, exclusion, collection, and recovery burdens.

Implications for platform defense

Do not label authentication, proofing, authorization, and behavioral attribution as one “identity” score. Store the claim each control supports. Select assurance and redress according to decision impact. For agents, separately represent the user or organization, the delegated principal, the authenticator, and the authority granted.

Chapters this affects

Chapters 4, 5, 7, 8, 9, 14, 15, 17, and 18.

Questions raised

• What assurance is proportionate for low-value pseudonymous participation?
• How should platforms represent delegated agent authority?
• How should identity correction propagate into integrity graphs and labels?

IETF RFC 6598: Shared IPv4 Address Space

Citation

Weil, J., et al. “IANA-Reserved IPv4 Prefix for Shared Address Space.” RFC 6598, April 2012. RFC Editor.

What it says

RFC 6598 reserves 100.64.0.0/10 for service-provider shared address space used in carrier-grade NAT deployments. The design allows providers to continue supporting IPv4 growth by placing customer networks behind shared translation infrastructure.

Important claims

• Shared address space is deliberately reusable and not globally routable.
• Carrier-grade NAT introduces an additional translation boundary between subscribers and the public internet.
• Network topology can make address-based assumptions fail.

Evidence

This is an IETF Best Current Practice defining operational address space, not a measurement of abuse detection accuracy.

Useful examples

Several subscribers can reach a platform through shared provider infrastructure. A platform that interprets a public-facing network address as a household, device, or person can therefore merge unrelated actors.

Assumptions

The platform sees the external endpoint after network translation and typically lacks the provider’s internal subscriber mapping.

Weaknesses / disagreements

The RFC alone does not quantify how many users share a particular observed address or how long assignments persist. IP reputation can still be predictive at an aggregate level.

Implications for platform defense

Use IP evidence for routing context, rate controls, correlation, and investigation while preserving its entity ambiguity. Add time, account, session, device, payment, and behavior evidence before expanding severe enforcement. Calibrate network-level controls for the legitimate population behind shared infrastructure.

Chapters this affects

Chapters 7, 9, 10, 13, 15, and 16.

Questions raised

• Which network interventions remain safe at very high sharing factors?
• Can a platform estimate sharing without increasing privacy risk?
• How quickly should negative network reputation decay?

IETF RFC 8981: IPv6 Temporary Addresses

Citation

Gont, F., et al. “Temporary Address Extensions for Stateless Address Autoconfiguration in IPv6.” RFC 8981, February 2021. RFC Editor.

What it says

The RFC standardizes temporary IPv6 addresses whose interface identifiers change over time. Its purpose is to reduce the ability to correlate a host’s activity across time and networks through a stable address-derived identifier.

Important claims

• Stable identifiers reused across contexts enable correlation.
• IPv6 addresses can expose such identifiers even when higher-layer payloads are encrypted.
• Temporary addresses narrow the window for trivial address-based correlation.

Evidence

This is an IETF standards document explaining a privacy threat and protocol mechanism. It does not evaluate platform-integrity models.

Useful examples

New temporary addresses are normally generated as older ones are deprecated. One legitimate host may therefore appear under multiple identifiers over time, while cookies or authenticated accounts preserve continuity at another layer.

Assumptions

Implementations enable and correctly operate temporary address behavior; other stable identifiers may still permit correlation.

Weaknesses / disagreements

Changing an address does not guarantee anonymity. DNS names, cookies, accounts, and other application-layer identifiers can remain stable. The RFC also does not imply that every changing address belongs to one device.

Implications for platform defense

Treat a network address as time-bounded routing evidence. Do not use address churn alone as proof of evasion. More broadly, recognize that persistent integrity identifiers impose tracking costs even when created for a defensive purpose.

Chapters this affects

Chapters 7, 9, 10, 13, and 18.

Questions raised

• Which identity links should expire when their underlying identifiers rotate?
• How should models distinguish privacy-preserving churn from adversarial churn?
• What is the minimum persistence necessary for each integrity decision?

W3C WebDriver

Citation

W3C Browser Testing and Tools Working Group. WebDriver, Working Draft, 2 July 2026. Specification.

What it says

WebDriver is a platform- and language-neutral remote-control interface for web user agents. It lets an out-of-process program navigate, inspect documents, interact with elements, generate input, execute scripts, and manage browser sessions.

Important claims

• Browser automation is a standardized capability.
• The specification’s intended users include developers and testers building automated tests, monitoring, load testing, and other tools.
• Cooperative user agents can expose that they are under WebDriver control.

Evidence

This is a current W3C Working Draft grounded in browser implementations. It defines capability, not the prevalence or abuse rate of automated browsers.

Useful examples

The same interface that can drive large-scale activity can also perform release tests, synthetic monitoring, or an accessibility-support workflow. Presence of the interface identifies a mechanism under cooperative conditions, not the governing purpose of an action.

Assumptions

Conforming implementations expose the standardized behavior. Other automation techniques need not use or accurately declare WebDriver.

Weaknesses / disagreements

The cited version is a Working Draft and may change. The standard does not prescribe platform policy or provide a dependable bot-detection guarantee.

Implications for platform defense

Do not equate browser automation with abuse. Govern authority, volume, behavior, and consequences. Treat declared automation state as one signal whose reliability depends on the client and use case.

Chapters this affects

Chapters 1, 4, 7, 8, 10, 11, and 14.

Questions raised

• When should platforms offer a supported API instead of forcing legitimate automation through a browser?
• Which accessibility and testing uses are harmed by mechanism bans?
• What evidentiary weight should cooperative declaration receive?

Google Crawler Verification Guidance

Citation

Google. “Verify Requests from Google Crawlers and Fetchers.” Google Crawling Infrastructure documentation, accessed August 2026. Official guidance.

What it says

Google documents methods for verifying whether requests that claim to come from its crawlers and fetchers actually originate from Google-controlled infrastructure. Methods include checking published IP ranges or performing forward-confirmed reverse DNS for relevant crawler categories.

Important claims

• A request’s declared crawler name is not sufficient verification.
• Mechanism identity can be backed by network or DNS evidence.
• Different crawler and fetcher categories have different verification patterns.

Evidence

This is first-party operational documentation for Google infrastructure. It demonstrates one verification design, not a universal crawler identity standard.

Useful examples

A platform may permit a verified search crawler while limiting an unverified request presenting the same user-agent string. Even verified identity does not remove the need for rate and behavior policy.

Assumptions

Operators correctly implement the verification procedure and keep dependencies current.

Weaknesses / disagreements

Network and DNS verification binds requests to infrastructure, not to beneficial intent in every transaction. It is specific to Google and can fail operationally if implemented carelessly.

Implications for platform defense

Separate declared, verified, and behaviorally trusted automation. Prefer scoped application credentials or documented verification over names in client-controlled headers. Continue monitoring verified integrations for compromised authority or policy violations.

Chapters this affects

Chapters 1, 4, 7, 8, 10, and 18.

Questions raised

• What interoperable mechanism should future agents use to present delegated identity and authority?
• How should verification failures degrade service without causing broad outages?
• Which information can a platform disclose without creating an evasion oracle?