Coterminous Impermanence · Interdisciplinary Study 004

Algorithmic Automation and the Illusion of Innocence

Coterminous Impermanence · Interdisciplinary Study 004

The Schedule and the Person

The schedule arrives before the employee has had a chance to explain what has changed at home. A tool has assigned the following week's shifts using information the institution already holds. On the screen, every required period has coverage. The employee reads the assignment and begins calculating whether they can meet it, knowing that the difficulty enters the system only if someone gives them a way to report it.

Their supervisor can see the same schedule. Whether the conversation leads to relief depends partly on what the supervisor is authorized to change. The institution may have chosen a procedure in which exceptions require another person's approval. It may have given the tool's output a status that makes deviation difficult to justify. To understand the decision, the employee has to encounter the arrangement of authority behind the interface.

The schedule appears finished. The decision it represents remains underway. Its consequences depend on whether another person's circumstances can reach those authorized to respond.

The schedule is a hypothetical illustration. It could be generated by a deterministic optimization program, a predictive model, or a system combining both. None of these architectures, by itself, determines whether the procedure treats the employee responsibly. That question depends on the purpose chosen, the information admitted, the objectives optimized, the discretion available afterward, and the institution's response to consequences it did not anticipate.

Study 002, Information Theory and the Architecture of Attention, examined how a field becomes available through selection. Here selection has become operational. A schedule carries a particular account of the institution's needs into a person's next week. The employee's request for adjustment asks whether lived experience can return to the system and change what follows.

I. Empirical Foundation

1. What automation delegates

Automation may acquire information, analyze it, recommend a decision, or implement a chosen action. Raja Parasuraman, Thomas Sheridan, and Christopher Wickens describe these as distinguishable functional stages whose levels of automation can vary.[1] A system may automatically identify a staffing shortage while leaving its remedy to a manager. Another may assign every shift and publish the result without individual review.

The distinction matters because a human is sometimes described as remaining “in the loop” when their practical role is limited to acknowledging a completed outcome. An operator who lacks the time, information, or permission to alter a result occupies a different position from one able to investigate and intervene. Meaningful supervision requires the conditions through which supervision can be exercised.

Automating a task can reduce delay and inconsistency. The appropriate comparison concerns the complete practice it replaces or changes. A careful evaluation can ask whether staffing improves, how often assignments must be corrected, whose time is consumed by appeals, and whether a documented exception receives a timely answer. A faster initial calculation may move effort into later dispute. That relocated work belongs within the cost of the system.

2. Prediction and action

A predictive model estimates an outcome under defined assumptions. Its performance can be assessed against suitable test data, although the meaning of any metric depends on how the target and data were constructed. An optimization system instead seeks assignments satisfying chosen constraints and objectives. A contemporary institutional workflow may combine forecasts, rules, and optimization before presenting a result as one coherent recommendation.

A prediction may be statistically well calibrated for its intended population while proving unsuitable for a particular decision. An optimization may meet every coded constraint while omitting a consequential requirement that was never represented. Neither achievement resolves which purposes should govern the institution.

There is an important distinction between a descriptive output and an authorized action. The transition occurs when an institution decides to act on a score, forecast, category, or assignment. Someone determines which errors are tolerable, who bears them, and what happens when the forecast conflicts with circumstances it cannot adequately represent.

3. Testing the deployment

A model tested in isolation can behave differently when inserted into a workflow. People adapt to the system, staff learn its defaults, managers respond to the incentives surrounding its use, and the affected population may change. Operational evaluation therefore needs a boundary wider than the model alone.

The National Institute of Standards and Technology's AI Risk Management Framework organizes voluntary risk-management activity under Govern, Map, Measure, and Manage. Its approach treats risk assessment as a continuing activity through the system lifecycle.[2] That framework does not confer moral legitimacy on an individual deployment. It gives organizations an account of work they can undertake and document.

Raji and colleagues propose an end-to-end internal auditing approach spanning the development and deployment lifecycle, with documentary traces connecting decisions to accountability.[3] Their proposal addresses a recurrent difficulty: when a deployed system causes harm, the organization may struggle to locate the choices that produced its behavior. Records of those choices can make later investigation possible.

An audit has limits. A clean record of decisions can show that procedures were followed, even when those procedures excluded the concern a person now raises. Effective accountability requires a mechanism through which a finding may alter practice.

II. Interpretive Boundary

4. What the evidence can establish

Empirical inquiry can investigate accuracy, error distributions, speed, resource demands, and the responses of people using a system. It can establish whether an appeal reached the relevant authority and whether an outcome changed. Such findings do not, by themselves, determine the moral worth of the affected person or the justice of the objectives being pursued.

This is the normative boundary of Study 004. Coterminous Impermanence supplies a distinct philosophical premise: where our decisions shape the conditions under which others can participate, that consequential reach carries an onus of responsible attention. The scope of the onus depends on what an agent can know, influence, authorize, and repair.

The premise is open to philosophical dispute. A person could agree with every measurement of an automated system while rejecting CI's account of stewardship. Keeping the premise explicit permits scrutiny of the argument, including questions about the limits of obligation and the distribution of responsibility.

5. The illusion of innocence

The title names a temptation within delegated decision-making. Someone commissions a system, then treats the resulting output as though responsibility has migrated into the apparatus. Another person operates it and points to the rules. A third explains that the rules faithfully implement instructions received from elsewhere.

Each statement may describe part of the process. Together they can produce an institution in which every participant can identify a boundary to their own job while the affected person finds no one who can answer for the whole encounter.

A machine can produce an output. Institutional authorization converts that output into an action with consequences. The latter step remains part of human governance, including occasions when the system is allowed to act without contemporaneous approval.

CI names the continuing consequences of participation as residue. The record of an automated decision may survive as a database entry, a changed schedule, a missed appointment, a lost wage, or a choice later made under diminished options. Some residue remains undocumented. An institution should therefore examine what happened beyond what its software recorded.

III. Case Study: The British Post Office Horizon Scandal

6. A ledger granted authority

The British Post Office Horizon scandal presents the central question of delegated institutional judgment in a setting where a calculation acquired consequences far beyond its intended accounting role. Horizon was a computerized branch accounting and transaction system developed and operated by Fujitsu on behalf of the Post Office. The system was introduced across the branch network beginning in 1999, with significant subsequent technical changes. It was not a contemporary generative AI system, and there is no need to characterize it as one. Its relevance is that conventional software can become an institution's privileged account of what occurred. [4]

Sub-postmasters operated local branches under contractual arrangements with the Post Office. Horizon calculated what cash and stock should have been present. Branch operators also counted and declared their holdings. A mismatch could appear as a shortfall for which the branch operator was expected to account. According to the Criminal Cases Review Commission, contractual practice required sub-postmasters to make up apparent differences through their own funds or by deductions from remuneration. The Horizon interface offered no option to dispute the system's figures before advancing to the following accounting period. [4]

That last design feature deserves sustained examination. The operator had to accept the figure to continue operating the branch. The required acceptance placed the operator in a procedural position that an investigator might mistake for agreement with the accounting figure. Operational necessity and evidentiary meaning had been brought perilously close. A branch operator who questioned the number remained subject to an arrangement in which proceeding required accepting it.

The software thereby helped structure an institutional encounter. A person who handled cash and stock could have direct knowledge of their actions and still lack access to the records and technical mechanisms needed to challenge the institution's account. The Post Office, and its technical provider, occupied another informational position. The difference was consequential because the organization's record was capable of initiating demands, investigations, and prosecutions.

The central failure cannot be compressed into a single defective line of code. Software errors matter. So do the choices that determine who can investigate, what evidence is treated as authoritative, whether those accused receive relevant technical material, and which officials can halt an escalating accusation. Horizon provides evidence of a wider arrangement in which those choices interacted.

7. What the courts established

In Bates and Others v Post Office Ltd (No. 6: Horizon Issues), the High Court examined the operation and reliability of Horizon. The 16 December 2019 Horizon Issues judgment addressed software bugs, errors, defects, transaction records, access to information, and remote access. The proceedings distinguished the technical system from the institutional claims made on its behalf. [5]

The judgment addresses system-level reliability and documented defects. Its findings do not attribute every discrepancy in every branch to a particular software fault. Fraser J concluded that the evidence showed a significant material risk of account inaccuracy from bugs, errors and defects, and distinguished the robustness of successive Horizon versions. [5, paras. 971–976] The judgment examined recognized technical failures and the difficulties of tracing and identifying them, including the differing access available to the organization and branch operators. A later account of Horizon should avoid treating software failure as a universal explanation for all discrepancies.

The Criminal Cases Review Commission subsequently reported significant problems with Horizon's accuracy, a material risk of apparent shortfalls being caused by bugs and defects, poor investigation into alleged losses, and the Post Office's failure to disclose the full and accurate position concerning reliability. These findings connected technical shortcomings with the fairness of the investigations and prosecutions. [4]

The distinction between a potentially erroneous record and criminal guilt is fundamental. A ledger entry may justify an inquiry. It cannot complete that inquiry. A shortfall can have more than one possible cause, including genuine loss, procedural mistakes, and faults in the accounting system. A defensible process must test alternatives before attaching blame to a person.

In April 2021, the Court of Appeal held, in cases before it where Horizon reliability was central and there was no independent evidence of actual loss, that the prosecutions had been unfair. The CCRC describes the resulting recognition of an abuse of process. The legal judgment arose within specific appeals; its importance for this study lies in the way unreliable or insufficiently examined computational evidence can become part of an unjust institutional action. [4]

The caution here cuts in several directions. The Post Office's criminal prosecutions involved legal duties that exceed the responsibilities governing an ordinary workplace schedule. The findings cannot be transplanted wholesale into every automated workflow. They provide a documented instance of what can happen when an institution grants machine-generated records authority while disabling or impoverishing the affected person's ability to interrogate them.

8. An unequal distribution of information

The Post Office and Fujitsu held different kinds of access from the sub-postmasters. Branch users could inspect the records and functions made available to them. Technical personnel had opportunities to examine underlying system behavior that ordinary users did not. The High Court found that Fujitsu could insert transactions remotely without the sub-postmaster necessarily knowing or being able to identify their remote origin in transaction data. The judgment also examined the contested statements about that capability. [5, paras. 533–545]

A disparity in access is sometimes necessary. Large systems require specialized expertise and restricted permissions. Sensitive information may need safeguards. The ethical difficulty appears when one party can make consequential accusations by relying on records that another party is structurally unable to investigate. A duty to answer a charge presupposes a meaningful opportunity to understand its basis.

Study 002 described informational environments as arrangements that shape what a participant can notice and communicate. The Horizon record shows how such an arrangement may become coercive when the institution treats its own representation as the definitive account of an encounter. Technical expertise was distributed unevenly; the consequences of accepting a disputed figure were not.

This difference should inform any institutional appeal design. Someone disputing a system's output requires more than a user manual. They may need access to audit trails, an independent examination of relevant faults, an explanation of the governing rule, and a person with authority to stop an enforcement action while evidence is reviewed.

There is also a problem of translation. The affected person may know that a figure does not match the cash handled or transactions completed. A technical specialist may speak in incident identifiers, logs, synchronization faults, and known defects. The organization must build a path through which those different forms of knowledge can be brought into a common examination.

Without such a path, a perfectly intelligible complaint can be classified as unsupported simply because it cannot satisfy an evidentiary demand that only the institution is equipped to meet.

9. The transformation of discrepancy into culpability

Consider the stages by which an apparent accounting discrepancy becomes a criminal accusation. First, a system records or calculates a difference. The institution then interprets that difference as a shortfall. A contract assigns financial responsibility. Investigators choose whether to seek alternative causes. Prosecutors decide whether the available evidence supports a charge. A court determines whether guilt has been established.

At each transition, an interpretation or exercise of authority supplements the information already held. Those choices can be obscured when the final accusation is presented as an inevitable consequence of the original number. An output cannot itself choose a prosecutorial standard, withhold a document, or decline to pursue evidence of alternative explanations.

CI's concept of onus can be applied to this chain without collapsing every participant into the same degree of responsibility. The software engineer who knows about an unresolved defect occupies one position. The investigator who receives evidence of potential error occupies another. The manager empowered to change policy has a reach different from that of a clerk required to follow it. An institution should identify how relevant information is expected to pass between those positions and what each recipient is able to do with it.

The temporal dimension is equally important. Initial trust in a system may rest on limited evidence. Reports of recurring faults change the knowledge available to its owners. Continuing to rely on a disputed system after receiving credible challenges is a decision under altered circumstances. Responsibility can intensify as the grounds for doubt become clearer and the available remedies become more apparent.

This is not a claim that all actors possessed the same information at the same moment. Historical accountability requires careful dates, records, and specific findings. It is a reason to ask whose knowledge expanded, how reports moved through the organization, and where an authority failed to act on information it had received.

10. The labor of proving what did not happen

An accusation forces work onto the accused. A sub-postmaster might need to reconstruct transactions, seek assistance, explain inconsistencies, consult advisers, and challenge the records of an organization with substantially greater resources. This work may begin while financial demands or reputational damage are already accumulating.

The burden is particularly acute when a negative proposition must be established. A person may be asked to prove that they did not take money when the alleged loss originates in a system whose complete records and workings are unavailable to them. The asymmetry is structural. A fair process must place appropriate investigative duties on the party able to access and evaluate the relevant evidence.

Study 003B explored the effort required to make a need intelligible to an institution. The Horizon scandal gives that theme another form. The person must make the system's possible error legible to those who already control the system. A complaint that could lead to repair instead becomes another test the complainant is required to pass.

An appeal mechanism can be procedurally present yet practically unreachable. The costs of using it may exceed the resources of the person whose livelihood is in jeopardy. A process may also demand repeated explanations, even when the relevant information should already be available inside the organization.

The resulting loss of agency may precede any formal ruling. Someone who cannot obtain a hearing, pause collection, or access the records needed to dispute an assertion has fewer meaningful choices about how to defend their life. Their participation is being constrained while the institution continues to describe the procedure as routine.

11. Residue beyond the accounting record

The CCRC describes the Horizon affair as the most widespread miscarriage of justice it had encountered, involving wrongful convictions for theft, fraud, and false accounting. The consequences included criminal sanctions, financial hardship, and damage to livelihoods. The statutory inquiry’s first final-report volume, published 8 July 2025, documents reported human impacts and evaluates redress. The chair expressly notes that the individual case illustrations were not intended as definitive factual findings about every disputed underlying event. [4][6]

These consequences exceed the technical system's records. A branch account may preserve a figure and a timestamp. It need not register the strain of defending against an accusation, the effect of imprisonment on family relationships, the interruption of a business, or years lived under public suspicion. The absence of those effects from a database does not diminish their reality.

CI calls attention to residue as the continuation of an action's consequences through material and social conditions. The idea gains specificity here. A disputed entry can lead to a demand for repayment. That demand can change a household's finances. An accusation can alter employment opportunities and personal relationships. Later correction may arrive after circumstances have changed irreversibly.

It is important to avoid overstating what redress can accomplish. A quashed conviction is a legally consequential repair. Compensation can help restore resources. Public acknowledgment can alter the social record. Each matters; none causes the intervening years to be lived again. Repair remains necessary precisely because consequences have already occurred.

This case also complicates the slogan that the Work remembers. Institutions sometimes preserve erroneous records and destroy or lose information needed to correct them. Collective memory can be partial, contested, or delayed. The more precise claim is that consequences persist through finite carriers, including bodies, institutions, families, documents, and changed opportunities, even when official memory fails.

12. Recognition, remedy, and institutional succession

Following the litigation, criminal reviews and legislative interventions altered the legal status of many Horizon convictions. The Post Office (Horizon System) Offences Act 2024 created a statutory route for quashing qualifying convictions. The CCRC continues to describe the distinction between those cases addressed by legislation and matters that require further review. [4]

The later statutory inquiry gathered a broader account of the system's implementation and failures. Its July 2025 report concerning human impact and redress shows that the practical work of response extends beyond establishing that an original accusation was unreliable. Redress requires institutions to identify affected people, address their losses, and establish routes through which claims can be considered fairly. [6]

These developments illustrate the reach and limitations of succession. Officials who did not make the original decisions can inherit responsibility for their consequences because they now occupy positions capable of changing them. They should not be assigned invented personal culpability for events in which they played no part. They can nonetheless bear a present duty to respond within their authority.

The lesson is demanding because an institution must sometimes acknowledge that the processes through which it pursued its purpose inflicted serious harm. Genuine correction may call for changes to evidentiary assumptions, access to records, contractual practice, software design, and the distribution of authority.

An apology has value where it recognizes the affected person's experience and the organization's role. Its significance grows when it is accompanied by practical action. A revised procedure can reduce the chance that a successor inherits the same defect. Record preservation makes independent examination possible. Accessible redress acknowledges that those harmed should not have to begin the entire struggle again.

13. What Horizon can and cannot prove for CI

Horizon demonstrates that consequential authority may attach to computational records through organizational choices. It also shows that institutional barriers to contesting those records can make errors more harmful and more enduring. Judicial findings and criminal reviews establish concrete failures in the particular historical case. [4][5]

The case does not logically prove CI's ethical premise. A person could accept the legal and technical findings without adopting its philosophy of participatory responsibility. The study's normative claim arises from a further judgment that institutions exercising power over people's conditions owe them accessible means to challenge errors and obtain repair.

Nor does Horizon prove that automation generally produces worse outcomes than human administration. Manual records are fallible, human investigators can behave unjustly, and careful automation can improve service. The important issue is the relationship among evidence, authority, error correction, and consequential reach.

The case therefore serves as an empirical constraint on the philosophy. Any CI account of automated institutions that celebrates efficiency while ignoring the people required to accept disputed results has failed to examine what its own concern with agency demands. Likewise, an account that treats every software output as inherently suspect would abandon the evidentiary discipline the scandal teaches.

The defensible position is that machine-generated information can inform institutional decisions, while the institution remains answerable for how it assesses that information, responds to doubt, and repairs consequences.

IV. Comparative Case: Dutch Childcare Benefits

14. The selection of suspicion

The Dutch childcare benefits scandal brings a different mechanism into view. Here the connection with algorithmic classification is more direct. A parliamentary investigation described a severe failure in the treatment of parents accused in relation to childcare benefits and identified violations of fundamental principles of the rule of law. Its report, Ongekend onrecht (“Unprecedented Injustice”), was presented on 17 December 2020. [7]

The parliamentary account describes an ICT-supported risk-classification model used to select applications for scrutiny. The later government letter describes its risk indicators as variably weighted and its boundaries as informed by examples of correct and incorrect applications. The Dutch Data Protection Authority found that applicants’ nationality was processed as an indicator in the risk-classification model from at least March 2016 through October 2018. It deemed this processing discriminatory and unlawful. A December 2021 government letter summarized these findings, alongside separate unlawful uses of nationality information. The government stated that this risk-classification application had ceased operation by July 2020. [7][8]

The breadth of the benefits scandal should not be attributed to one model. Parliamentary findings examined a harsh fraud-control approach, disproportionate recovery practices, administrative choices, and failures across public institutions. The risk model was one consequential component within that wider arrangement. The distinction guards against treating a complicated governmental failure as the isolated consequence of an algorithm.

The case raises a question distinct from Horizon's accounting authority. Which characteristics may legitimately determine who is classified for further scrutiny? When a model incorporates nationality, a technical feature can reproduce an ethically and legally objectionable distinction at scale. Reviewers may encounter the classification as a routine workflow without examining the premise that placed a family in its path.

15. From category to coercive consequence

A risk flag may begin as a recommendation to inspect a file. Subsequent policies determine whether a person faces demands for documents, suspension of benefits, recovery of payments, or accusations of wrongdoing. The model's contribution must be investigated alongside those decisions.

The parliamentary inquiry described a harsh fraud-enforcement regime with serious consequences for parents, and deficiencies across the institutions responsible for its operation. A minor administrative mistake could lead to a response bearing little proportion to the circumstance. [7] The harm thus depended on the coupling of selection criteria with inflexible rules and institutional assumptions about suspicion.

Study 002's concern with attention is evident: the institution's model directed scrutiny toward some applications through categories it chose to recognize. Study 003B's concern with dependence also matters. Childcare support can shape a household's ability to work, care for children, and meet ordinary obligations. The cost of responding to an accusation enters an already demanding life.

The case illustrates how the burden of correction can be displaced onto the people least able to absorb it. A parent whose income and care arrangements depend on continued benefits may have to challenge a powerful agency while complying with demands that assume substantial time, documentation, and procedural knowledge.

16. Two cases, different mechanisms

Horizon involved accounting records whose reliability and interpretation became central to demands and prosecutions. The Dutch case involved a public-benefits enforcement regime in which risk classification, discriminatory data use, and severe administrative practice contributed to wrongful treatment. They should retain their historical and technical distinctions.

Both cases expose the importance of the decision boundary. A number, flag, or system-generated entry gains force when authorized people make it a basis for action. The point of philosophical concern is the full institutional pathway, including how objections are received and what remedy is possible.

The cases also caution against defining accountability solely as explainability of a model. A perfectly transparent risk criterion can still be discriminatory. An accounting rule can be intelligible while the evidence used to apply it remains unreliable. Justice requires examination of the objective, the procedure, the affected person's standing, and the ability to change a harmful result.

The comparative case therefore complements Horizon rather than duplicating it. Together they show why the study's critique extends from conventional software to algorithmic classification and contemporary AI governance. What matters ethically is the institution's use of computation to alter the circumstances of people who must live with its decisions.

V. The Institutional Decision Boundary

A system's apparent success depends on which costs are counted. Faster assignments may free staff for other work while forcing people with unusual circumstances to pursue repeated appeals. Before deployment, an institution should identify the purpose of the system, the consequences of error, and the person authorized to examine a disputed outcome. Contracts with suppliers should preserve a practical means to investigate faults and retrieve relevant records.

The key threshold arrives when a technical output receives institutional authority. A calculated deficit becomes an accusation; a risk indicator invites enforcement; a scheduling recommendation determines the hours a person must work. The organization should record the additional judgment that authorizes each transition. That account permits later reviewers to ask whether the evidence was adequate for the consequence imposed.

The standard is demanding and finite. Decisions with substantial effects on liberty, livelihood, or essential support warrant stronger safeguards than routine clerical conveniences. Their risk calls for resources devoted to independent review, temporary relief where feasible, and correction of harms already sustained. A procedure whose costs of challenge exceed the resources of those affected may have shifted its work onto the very people it claims to serve.

VI. Responsibility After Deployment

Locating the authority to act

A procurement officer may select a supplier while an executive approves the purpose for which the system will be used. Technical specialists can correct defects within their expertise. A supervisor may receive an objection yet possess authority only to record it. These differences matter because onus follows the reach of an agent's knowledge and power to intervene. Assigning identical duties across those roles would obscure the decisions each can make.

An accountable institution names the person who can authorize immediate relief and the office able to revise the governing rule. When a complaint crosses from a supervisor to a specialist, the receiving team must know what action it is expected to consider. Otherwise, the handoff preserves a record while the person affected continues living with the original decision.

The obligation changes as information arrives. Early uncertainty may justify inquiry. Repeated warnings about the same failure make continued reliance on a procedure a decision under better knowledge. The organization needs a means to preserve reports and bring them before someone authorized to alter practice. Horizon shows the danger of a process in which technical knowledge, investigation, and enforcement fail to meet before consequences become severe.

What a person can make visible

The categories chosen for a system determine which circumstances it can receive. A scheduling tool may represent qualifications, hours, and demand while lacking a field for an urgent family obligation. The employee then has to explain the exception through a separate process. That request can require time, repeated disclosure, or documents held elsewhere. Study 002 examined how institutions organize attention; its concern becomes practical wherever a computational category controls access to relief.

Information requirements can serve legitimate purposes. Colleagues also depend on fair scheduling, and administrators may have to verify a claimed exception. The question is how much effort the procedure demands, what it must establish before acting, and whether temporary protection is possible during review. For someone managing illness, caregiving, or loss, the cost of being heard may determine whether the remedy exists in any meaningful sense.

An institution should also study exceptions as evidence about its own model of the situation. A single complaint warrants a response to the person affected. A recurring pattern may warrant a change in the available categories, the objective being optimized, or the rules that determine permissible departures. Counting appeals without examining their causes would leave the information gathered inert.

Maintenance as an allocation of care

Study 003 described maintained physical systems whose organized conditions depend upon continuing exchanges. An automated institutional procedure operates under different scientific principles. The shared concern is visibility: persistence depends on work that a functioning surface can conceal. Staffing changes, revised policies, and altered user behavior can make an initially useful system unsuitable for later conditions.

Consequential deployments require continuing review and people empowered to respond. A monitoring dashboard cannot repair a harmful assignment. Its observations must reach operators with time to investigate and managers able to revise or suspend the workflow. These capacities belong in the operating budget, alongside technical support, staff training, records preservation, and the testing of changes.

Frequent modification carries its own costs. An organization can destabilize operations by changing rules before the effects of an earlier revision are understood. Maintenance therefore requires a cadence suited to the decisions involved, together with an urgent route when severe consequences demand immediate intervention. Preserving a tool merely because it has become familiar is an institutional choice subject to the same scrutiny as adopting it.

The content of a remedy

Study 003B examined dependence and the work required to make a need legible. An appeal against an automated result has similar demands. A person needs a reachable decision-maker, an explanation of the relevant grounds, and an answer within a period in which it can still affect their circumstances. Technical information may help specialists investigate; an affected participant also needs to know which facts mattered and what can change.

A successful challenge may require a corrected record, a different assignment, restored access, financial relief, or reconsideration outside the original decision chain. The appropriate response depends on the consequences and the authority available. Some losses cannot be reversed. Where a hearing concludes after the disputed action has already imposed its cost, future safeguards and forms of repair remain relevant.

The institution should distinguish a prompt acknowledgment from an operative remedy. A complaint portal can create a ticket without altering a decision. Meaningful contestability requires a route to an authorized judgment and a means to carry that judgment into practice. Repeated complaints about the same failure may also demand action beyond an individual's case.

A review pathway put to work

Return to the employee whose schedule arrived before anyone learned about the change at home. The institution has a full week of coverage, and its software has met the objective assigned to it. The employee requests a change. To assess whether the procedure functions responsibly, an investigator would need to follow the request through its actual course.

The first question concerns intake. Can the employee reach an authorized person during the hours when a response is still useful? A service portal may accept requests at any time, yet route them only during ordinary business hours. A supervisor may read the message promptly while lacking authority to adjust the schedule. These are operational facts that shape the employee's available choices. They can be observed and evaluated without first assigning a moral label to the underlying software.

Next comes the information required. The institution may need to understand which hours are affected and what alternatives exist. It may also request extensive personal documentation whose relevance to the scheduling decision is unclear. The cost of preparing that account falls on someone already managing competing obligations. Study 002 helps identify the selective structure of the exchange. The ethical premise enters when the institution judges whose effort deserves consideration and what demands are proportionate to the decision.

The third question concerns authority. A supervisor might be able to exchange shifts between willing employees. A different official might be needed to waive an automated rule. Where every exception requires approval from an office unfamiliar with local conditions, even a justified request may remain unresolved. A defensible process documents who can authorize a temporary change, including the limits of that person's discretion and the means of obtaining higher-level review.

The response also needs a time horizon. An answer delivered after the relevant shift can establish how the institution would decide a similar request in the future, but it may be too late to preserve the employee's immediate opportunity. The appropriate deadline depends on the stakes. Routine planning can tolerate intervals that urgent care arrangements cannot. A system's review standard should account for that difference.

Suppose the request succeeds only after the employee explains the same circumstances to three people. The final schedule may appear corrected in the database while the work of securing that outcome remains invisible. This distinction matters when the organization measures effectiveness. It may count successful appeals while omitting how long they take, how many contacts they require, and whether affected people felt able to use the process.

A second employee with similar needs might never appeal. They may expect refusal or fear consequences at work. Their silence cannot safely be interpreted as satisfaction. A responsible evaluation may need confidential feedback, independent inquiry, or patterns drawn from other workplace evidence. Such methods have their own limitations and should be explained accordingly.

The purpose of tracing this encounter is to specify what the institution can learn. A review outcome should inform more than the immediate assignment when it reveals a recurring omission in the system's account of employees' circumstances. Some adjustments belong in a local supervisor's discretion. Others require changed rules, revised input fields, or a new distribution of authority. Maintaining that pathway requires time and resources that belong in the assessment of the automated arrangement.

A meaningful measure of success can therefore include the employee's access to explanation, timely reconsideration, and practical relief. These measures are distinct from forecast accuracy or shift coverage. They examine the lived conditions through which a computational result becomes an institutional demand. The system remains useful only within the purposes and safeguards the organization can continue to defend.

VII. Objections and Unresolved Questions

People also make mistakes

Human scheduling can be inconsistent, opaque, or subject to favoritism. An automated system may improve reliability and make some decisions easier to examine. That improvement matters. A philosophical criticism of delegated decisions should examine comparable alternatives rather than assume that a human-only process is inherently preferable.

The relevant question is whether the chosen arrangement better supports its legitimate purposes while protecting ways to identify and repair its harms. A system that reduces errors and allows effective correction may satisfy CI's stewardship commitment better than a less accountable manual practice.

Authority is sometimes fragmented

A supplier may withhold technical information. A local supervisor may lack override privileges. Policies may be set by a remote department. These constraints are real and can limit an individual's responsibility.

They also become facts that an institution should consider before adopting a system. If it cannot investigate consequential failures, receive effective appeals, or stop the process when necessary, it may be poorly positioned to assume the authority that deployment entails.

Contracts, procurement terms, internal responsibilities, and fallback procedures can be used to preserve avenues of correction. They remain subject to the limitations of the setting and applicable law.

Can a model itself bear onus?

This study concerns institutional responsibility and leaves broader questions of machine moral status unresolved. A model may exhibit sophisticated behavior while the question of moral standing remains contested. Assigning governance obligations to an institution does not require first settling those philosophical debates.

The practical issue is immediate. People authorize the adoption, define permitted applications, determine resource allocations, and structure remedies. These acts carry consequences regardless of disputes about the system's internal status.

What can be known in advance?

Uncertainty cannot be abolished by governance. Novel circumstances expose limits in even carefully evaluated systems. A demand for perfect prediction would make responsible action impossible.

CI instead asks for accountable participation within finite knowledge. That entails disciplined evaluation before deployment, attentive observation afterward, and willingness to correct what experience reveals. A claim of uncertainty is most useful when it identifies what should be watched and who can respond.

VIII. Further Objections: Accuracy, Cost, and the Right to Be Heard

Institutional legitimacy without technical certainty

An institution may reasonably rely on computer systems whose full operation no individual manager can inspect. Modern organizations depend upon databases, communication networks, and transactional processes assembled from many components. Requiring every operator to reproduce the engineering analysis before taking action would make ordinary administration impractical.

The appropriate duty begins with the decision being made. A system may be sufficiently reliable for producing an initial schedule while requiring additional verification before an accusation of misconduct. Reliability must therefore be evaluated in relation to the consequences that depend upon it. The same output can be useful for one purpose and inadequate for another.

Horizon makes that distinction concrete. A discrepancy can reasonably trigger reconciliation work. Treating the discrepancy as dispositive evidence of theft demands further inquiry. The difference emerges from the institution's authority to move from an accounting question to an accusation affecting a person's liberty and livelihood. Any account that locates the entire ethical problem inside the code overlooks that decision.

Who should bear the cost of uncertainty?

There is a recurring administrative incentive to place the burden of uncertainty on the person with the least authority to investigate it. A transaction appears irregular. The institution asks the affected person for proof that nothing is wrong. The person may have to acquire information controlled by the institution itself.

This pattern can appear in employment, public services, insurance, and benefits administration. Different legal standards apply across these settings. The philosophical question concerns the fair allocation of investigative effort. The organization that creates the classification and controls the underlying evidence occupies a position of distinctive responsibility for assessing its reliability.

A proportional response takes the stakes into account. A routine mismatch might warrant a request for clarification. A threatened termination, prosecution, or withdrawal of essential support can justify stronger obligations of investigation, evidence disclosure, and independent review. These safeguards may be costly. Their expense belongs in the assessment of whether the automated procedure provides a benefit that justifies its use.

Consider the familiar promise that automation reduces administrative labor. If fewer employees are needed to process ordinary cases, the organization may regard this as a successful outcome. Yet if an error forces a claimant to make several calls, complete appeals, seek professional help, and sacrifice work hours, labor has been transferred. The organizational balance sheet records one saving while the affected person carries another cost. A complete evaluation should make that distribution visible.

The danger of treating every exception as proof

A complaint is evidence of a person's experience and a reason for investigation. It does not always establish the technical cause of a disputed outcome. Some appeals arise from misunderstandings, legitimate competing constraints, or genuine disagreement about institutional priorities. Treating all complaints as evidence of a defective model would compromise the inquiry's rigor.

This observation creates a demanding standard for both sides of the process. The institution should take a challenge seriously enough to investigate it. The reviewer should retain the ability to distinguish an error from a difficult but supportable decision. The affected person deserves an explanation responsive to the actual concern. Where the decision remains unchanged, the reasons should be intelligible and available for further examination when warranted.

An appeal whose outcome is always reversal would have failed as an evaluation mechanism. An appeal whose outcome is predetermined confirmation has failed in another way. Legitimate judgment must remain possible within the established evidence and the authority granted to the reviewer.

When the affected person cannot safely object

Some procedures assume that everyone may challenge an adverse output without additional risk. In practice, a worker may fear retaliation, a benefit recipient may fear intensified scrutiny, or a contractor may fear losing the relationship through which they earn a living. The formal availability of review should therefore be examined alongside the consequences a participant anticipates from using it.

Confidential reporting, independent review, clear protection from retaliation, and transparent response periods can reduce these obstacles. The suitable arrangement depends on the institution and relevant law. The underlying ethical concern is consistent: a pathway for correction has limited value when the people most likely to need it cannot use it safely.

There is a further difficulty where people lack the time, language fluency, or cognitive resources needed to navigate the route provided. An institution can respond through assistance, accessible explanation, and alternative means of presenting evidence. These arrangements are not merely improvements to user experience. They can determine whether a person's circumstances become available to decision-makers at all.

The ethical value of withdrawal

Institutions sometimes treat continued deployment as evidence of progress. The tool has been purchased, integrated, and adopted. Substantial funds and reputation are invested in its success. Withdrawal can then be experienced as an institutional defeat, even when evidence shows that the system no longer serves its legitimate purposes.

A mature governance arrangement makes suspension and retirement normal options. A system may be retired because a replacement performs better, its assumptions no longer fit the setting, or its harms cannot be reduced at proportionate cost. The decision deserves documentation, including the treatment of people affected by earlier outcomes.

CI's conception of agency leaves room for cessation as a consequential act. Refusing further reliance on a procedure can preserve future participation when continuing it would impose foreseeable harm. The Work encompasses maintenance, revision, and responsible departure from arrangements whose purposes have ceased to justify their costs.

The point at which deference becomes a decision

A manager can reasonably begin with a presumption that an established system performs the task for which it was adopted. Routine trust makes coordinated work possible. The moment an affected person supplies a credible account that conflicts with the system's output, the manager encounters new information. What happens next depends on the permissions, procedures, and working relationships that the institution has maintained.

A supervisor may find the output persuasive because it appears precise. Numbers appear in a familiar display, with timestamps and a complete assignment. Their formal presentation supplies no independent assurance that the inputs covered the employee's circumstances. The manager needs a way to ask how the result was produced, what constraints it applied, and whether an exception belongs within their discretion. Where that examination is impossible, the institution should reconsider the authority granted to the output.

A technical team may identify a defect that affects several decisions, yet its findings may remain isolated from the people handling appeals. Those who maintain the software then know something different from those charged with enforcing its results. Information must cross that boundary through an assigned reporting path. It needs a recipient empowered to pause reliance on the affected feature, notify people whose outcomes may have changed, and establish a route for correction.

An institution could decide to use the same tool after an error, provided its continuing use is supported by evidence and proportionate safeguards. That decision deserves a recorded rationale. How frequently did the error occur? Which outcomes depended on it? Who was exposed to loss while the defect persisted? What testing supports confidence in the repair? An adequate account identifies the limits of what the organization has learned and the conditions under which it will reconsider its judgment.

These questions assign work to those capable of performing it. A branch worker cannot reasonably be expected to diagnose a proprietary accounting platform. A benefits recipient should be able to challenge a classification without reverse engineering a risk model. An employee seeking a schedule adjustment can describe the conflict in their life while a supervisor investigates the institutional rules governing relief. The distribution of technical expertise creates corresponding responsibilities for those who possess it.

CI's ethical premise remains visible at this point. The empirical record may show which errors occurred and which interventions reduced them. Responsibility enters through the judgment that an institution capable of altering people's available choices ought to make its decisions contestable and its errors repairable. The argument asks the institution to put that judgment into practice through actual authority and funded procedures.

IX. Correction and Succession

Study 001, Quantum Mechanics and the Human Condition, considered finite observation and consequential encounter while distinguishing scientific claims from their philosophical interpretation. Study 004 inherits its attention to the observer's limits. A record cannot encompass every effect of an institutional decision. Yet the incompleteness of knowledge gives reason to maintain routes through which overlooked conditions can enter later judgment.

A system may be inherited from earlier administrations, long after its original authors have left. The current operator can still encounter its consequences. Governance must therefore survive individual departures, software replacement, and organizational change.

Preserving reasons for consequential choices helps successors know what they are maintaining. Keeping an exit route available permits retirement when a tool ceases to serve its purpose. A procedure whose legitimacy depends on its never being questioned has made continued use an end in itself.

In CI, the Work remembers through material and social consequences, subject to loss, alteration, and forgetting. An accurate audit trail can support that memory. The experience of a person affected may disclose residue the trail omitted. Responsible succession permits those accounts to meet.

X. A Schedule Revised

The employee speaks to the supervisor. The first response is that the schedule has already been generated. The employee explains the problem. A useful process allows the supervisor to identify the relevant constraint, examine what can change, and reach someone with the necessary authority if the adjustment lies beyond their own.

Perhaps the schedule changes. Perhaps a competing obligation prevents the requested accommodation and a more difficult conversation follows. The institution's responsibility lies partly in the quality of that conversation and in the practical options its procedures make available.

The final schedule remains a decision made within finite resources. Its provenance matters, as does the experience of those who must live through it. The institution has the capacity to use automation to extend its reach. That extension makes the routes of answerability, reconsideration, and repair part of the Work it has chosen to undertake.

Evidence and Interpretive Limits

The Horizon findings are separated by source function. The CCRC account establishes the contractual treatment of branch discrepancies and the absence of an in-system dispute route. The 2019 Horizon Issues judgment addresses technical reliability and system access in civil litigation. The 2025 inquiry report addresses human impact and financial redress while warning that its individual illustrations do not determine every disputed fact. The Netherlands comparison relies on the 2020 parliamentary inquiry for institutional failures and the government's December 2021 letter for the specific nationality-processing periods and the data protection authority’s conclusions. The philosophical conclusions are arguments offered by CI, rather than judicial or scientific findings.

Citation Provenance and Interpretive Method

The source numbers identify the authorities used in the body. For Horizon, [4] supports the mechanics of branch reconciliation, the absence of an in-system dispute option, and the CCRC account of criminal review; [5] supports findings made in the civil Horizon Issues trial, with pinpoint paragraphs where supplied; [6] supports the Inquiry's separate account of human impact and redress. The cases reviewed by different bodies have distinct evidentiary and legal boundaries. For the Netherlands, [7] supports the parliamentary findings concerning the broader benefits scandal, while [8] supports the specific administrative processing of nationality and its dates. The remaining sources support general academic claims concerning automation design and governance. The philosophical conclusions are proposed by CI.

Sources and Scope

[1] Parasuraman, Raja, Thomas B. Sheridan, and Christopher D. Wickens. “A Model for Types and Levels of Human Interaction with Automation.” IEEE Transactions on Systems, Man, and Cybernetics, Part A 30, no. 3 (2000): 286–297. https://doi.org/10.1109/3468.844354

[2] National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1 (2023). https://www.nist.gov/itl/ai-risk-management-framework

[3] Raji, Inioluwa Deborah, et al. “Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing.” Proceedings of FAT 2020. https://doi.org/10.1145/3351095.3372873

[4] Criminal Cases Review Commission. “Post Office ‘Horizon’ Cases.” https://ccrc.gov.uk/post-office-horizon-cases/ Case chronology, branch accounting procedures, grounds of review, and appeal findings.

[5] Bates & Others v Post Office Ltd (No. 6: Horizon Issues) [2019] EWHC 3408 (QB), 16 December 2019. https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-post-office-judgment.pdf High Court findings concerning system faults, access, and reliability.

[6] Post Office Horizon IT Inquiry. Volume 1 of the Final Report: Human Impact and Redress, 8 July 2025. https://www.postofficehorizoninquiry.org.uk/reports-and-statements . Official report landing page. The Volume 1 human-impact case illustrations are presented with explicit evidentiary limits.

[7] Tweede Kamer der Staten-Generaal. Ongekend onrecht (Unprecedented Injustice), parliamentary inquiry into childcare benefits, 17 December 2020. https://www.tweedekamer.nl/kamerleden_en_commissies/commissies/pok .

[8] Netherlands Ministry of Finance, parliamentary letter on Dutch Data Protection Authority findings on nationality processing, 7 December 2021, published January 2022. https://zoek.officielebekendmakingen.nl/kst-31066-936.html .

The opening employee and institution are hypothetical. They do not represent a documented employer, vendor, or incident. The sources support the distinctions between automation functions and the need for lifecycle governance and auditability. CI's moral premises are philosophical commitments stated separately from empirical findings.