From To Do Overload to Operational Control: the technical evidence

01 / Executive summary

Executive summary

Every utility that runs Oracle Energy and Water software for billing or meter data eventually meets the same problem. The system creates a To Do whenever something goes wrong: a bill fails, a reading is missing, a validation rule fires, an integration rejects a message. The nightly run produces them faster than the staff can clear them. Old entries pile up, real problems hide among noise, and bills slip.

Artificial intelligence looks like an obvious answer, and it is part of one. But three things stand in the way. Utilities will not hand operational and customer data to a service outside their control. Software-as-a-service customers cannot reach the database at all. And "let the AI fix it" is the wrong goal, because in these systems the usual fixes (retry, suppress, close) can quietly discard data.

What we mean by "the lab". The lab is a complete Oracle Energy and Water Customer to Meter (C2M) 25.4 environment, with its application server, database and batch scheduler, loaded with Oracle's DEMO database. We use it as a safe place to develop and prove the approach: it holds real product behavior and realistic exception volumes, but no customer's data and no production risk. Every experiment ran there, and every change to it is recorded and reversible.

This paper describes a ladder of five steps, from no AI to guarded automation, with prevention running underneath all of them, and shows what we measured at each step in a working lab.

What the lab showed:

1. A large share of the backlog needs no AI. The product already has retry, auto-complete, routing, suppression and monitoring levers. In the lab, one batch run cleared 164 failed seeder records and closed their 164 To Dos with no person involved, and a single monitor algorithm closed 689 more (lab). But 1,136 of the 1,156 live To Dos (98%) sat on business objects whose states had no monitor batch assigned, so the retry that was plugged in could never run (lab).
2. The levers have side effects that must be agreed first. A product retry on usage transactions did not fix 58 of them; it discarded them. Suppressing a To Do for one message made the system discard the record that caused it, where rerouting the same message to another team kept it (lab). An age-based closer cleared 689 To Dos by date alone, which would hide real problems in production.
3. Silent configuration gaps cause both noise and blindness. An empty master configuration section sent every reading older than the current date to an error state and raised To Dos nobody owned. In the other direction, a reading roughly 9 to 15 times a meter's normal monthly use was accepted with no exception at all (lab).
4. A model grounded in numbered facts can propose useful next steps, and rules can keep it safe. (In this paper a case is our own review file for one To Do: its numbered facts, the model's proposal and the reviewer's decision. It is not a case object in the Oracle products, which means something else.) In six sample cases a general-purpose language model found inconsistencies in the evidence that a person who knew the lab had missed, and passed all of its automatic guardrails in the final version: 18 of 18 answers across three repeats. The model was never allowed to change anything.
5. The AI should go to the data, not the data to the AI. Every step runs read-only, on pseudonymized facts, with the model placed where the utility chooses and every action routed through the product's own interfaces under a narrow service account.
6. An approved proposal can reach the product safely, one narrow action at a time. In the lab a person approved a model's proposals, and a small gateway carried out three narrow actions through the product's own web services (adding a log note, forwarding a To Do to another role and forwarding it back, and completing a stale To Do), under a service user that can do only that. An independent database check confirmed every action, and a full test found gaps in what the product's own settings enforced on this path, which we closed before calling the path safe (lab).

The value for a utility is not a robot that clears queues. It is a measured path: find the noise that configuration can remove, rank what is left by what it blocks, and give the people who must decide a prepared case file instead of a blank queue.

02 / The problem: To Dos are where operations meet configuration

The problem: To Dos are where operations meet configuration

A To Do in Oracle Energy and Water is a work item addressed to a role. The framework creates one when a background process or a status change finds something a person must look at. In a typical implementation they come from four directions:

• Exception tables, such as bill and bill segment exceptions, each with a process that creates one To Do per exception.
• Lifecycle plug-ins, where a record enters an error state and an algorithm creates the To Do.
• Integration errors, such as an inbound message the system cannot map.
• Users, who create manual To Dos for follow-up.

Three things make the queue hard to keep under control. First, volume: one bad setting can generate thousands of identical entries. Second, mixed causes: a To Do about a high reading, a missing tariff map and a rejected message look alike in a queue but need different people. Third, no owner for the configuration: the team that works the queue rarely owns the setting that fills it.

In the lab, which holds Oracle's demonstration data, the picture is the familiar one (lab):

• 2,528 To Dos, of which 2,106 were open; 2,062 of the 2,115 not-yet-complete entries were more than three years old.
• Most were on the meter-data side, not billing: roughly 1,700 open entries on the two meter-data types (seeder and measurement exceptions), routed to two validation roles.
• Billing errors were few (8 bill segment errors) but pointed upstream. The messages "awaiting bill determinants from MDM" and "usage transaction in non-final status" mean the real problem is a usage transaction, which in turn depends on a validation exception on a measurement, which in turn depends on a seeder error.
• A single meter-data validation table held 32,220 exceptions, mostly closed noise: about 22,000 of them came from two configuration messages (a register multiplier and a unit of measure that the rules require but that were not set).

That chain (billing error, usage transaction, validation exception, measurement, seeder) is the key to triage: a To Do is a symptom, and the useful question is what it depends on and what else depends on it.

03 / How an Oracle Energy and Water To Do works

How an Oracle Energy and Water To Do works

The entry. A To Do has a type, a role, a status (Open, Assigned with optional In Progress or On Hold, then Complete), a priority, a message (a category and number, with parameters), a drill key that points at the source record, characteristics, and a log of everything done to it. Completed entries stay in the database for audit. Oracle: lifecycle of a To Do entry.

The levers the product offers.

Oracle describes the retry and creation pattern in Managing To Do Entries and the error-handling model in Processing Errors.

The common error pattern. Most error handling in the lab follows one shape: a record enters an error state, an enter algorithm creates the To Do, a monitor algorithm on that state retries after a delay, and an exit algorithm completes the To Do when the record leaves the state. The monitor only runs if the state has a monitor batch assigned and that batch is scheduled. That detail turned out to matter more than any other.

04 / Constraints that shape any solution

Constraints that shape any solution

Data sovereignty and privacy. Customer, account and meter data is regulated and sensitive. Most utilities will not send it to an outside service, and many are right not to.

No database access in the cloud services. Customer Cloud Service (CCS) and Customer Care and Billing Cloud Service (CCBCS) are software as a service; the customer does not get a database login. Oracle provides sanctioned paths instead:

• GoldenGate replication into a customer-owned Autonomous Transaction Processing database in the customer's own cloud tenancy, giving full access to the replicated data. It is a separately licensed, fee-based feature. Oracle: GoldenGate replication.
• File export to Object Storage, using a batch process that writes extracts to a customer bucket; Oracle's FAQ recommends replication when change capture is needed for more than about 20 tables (FAQ). Oracle: file export.
• REST inbound web services with OAuth clients, which let an external service read and act through the application's own security.
• External routing of new To Dos (F1-TDEER), an event hook that sends each new entry out as it is created.

Change control and audit. Anything that changes a record or closes a To Do must be attributable and reversible where possible, and in utilities it is often regulated.

Model risk. A model can be confidently wrong. The design has to assume it will sometimes be.

What Oracle already does. Oracle ships machine-learning anomaly scoring for residential initial-load readings on its cloud services (limited availability), which skips validation of "normal" data and creates exceptions for anomalous data. We met it in the lab as the "Waiting for ML" state. Oracle: legacy measurement processing. Oracle has also announced AI call summarization for customer service. We found nothing from Oracle aimed at To Do or billing-exception triage. Vendors and system integrators do market exception-management agents for utilities; their performance claims are not independently verified and are not relied on here.

05 / The ladder: from no AI to guarded automation

The ladder: from no AI to guarded automation

The five steps share a template: what changes, the data it needs, the guardrails, and what we measured. Each step stands alone, so a utility can stop at any rung. Prevention is not a rung; it runs through all of them, because the cheapest To Do is the one never created.

Phase 0. Measure

What. Inventory and rank. Count To Dos by type, role, status, age and message; link them to their sources; find the few messages that account for most of the volume.

Data and access. Read-only. A database account with SELECT, or a replica, or a periodic export.

Lab result. Within an hour the lab's backlog reduced to a short list of messages. The top ones were seeder "overcount" (793 entries), high consumption (374), low consumption (152), zero consumption (111), no tariff map for a template and time zone (138), low-quality data (115) (lab).

Why it matters. Every later phase depends on knowing what the noise is made of.

Phase 1. Product hygiene (no AI)

What. Use the product's own levers: monitor batches assigned and scheduled, retry and auto-complete where the end state is acceptable, rerouting to the right role, conditional suppression, priority by bill cycle.

Data and access. Configuration authority; no new data.

What we ran, and what happened (all lab):

Three lessons.

1. Monitoring is a configuration question first. The batch that runs the retry must be assigned to the state and scheduled. In the lab it had never run for most controls, and 98% of live To Dos sat on states with no batch at all.
2. Know the end state. A retry can end in DISCARD. Suppression can cause a discard. Closing by age is blunt: it cleared 689 To Dos because they were old, not because the problem was gone. Each lever needs a written end state and a business sign-off before it is switched on.
3. Source-aware beats age-based. The right monitor closes a To Do when its source record is healthy or gone, and logs why.

Phase 2. Triage

What. Group and rank. For every open To Do, follow its drill key to its source record, decide whether it is still real (live), already resolved (stale), or not linked; group by root cause; rank by what it blocks.

Data and access. Read-only extract of To Dos and their sources. No model is needed; rules suffice, with a language model optionally narrating clusters.

Lab result. Of 1,260 open To Dos, 1,156 were live and 20 stale; 84 were not linked. The live ones fell into a few classes: consumption judgement (hi/lo) 820, configuration gap 138, data quality 138, late or missing data 46. Most striking: 1,136 of the 1,156 live entries (98%) pointed at records whose states had no monitor batch, so the product's retry could not clear them. Assigning a monitor batch to those states is the largest single lever by volume that the triage found, once the end state of a retry is agreed (lab).

Limits. The impact weights are assumptions. A utility would replace them with days to bill due and dollars.

Phase 3. Copilot

What. For one To Do, assemble the evidence, ask a model for a diagnosis and next steps, check the answer against rules, and give a person an approve, edit or reject choice. The model cannot change anything.

How it works.

1. A case builder reads the system read-only and writes numbered facts about the To Do into a case, our review file for that one To Do (not a case in the Oracle products), its source record, the rules involved and how it compares with similar entries. Every statement the model makes must point back to one of these facts.
2. Long identifiers become tokens before anything leaves the machine; the token map stays local.
3. The model answers in a fixed structure: diagnosis, class, confidence, what is unknown, actions from a fixed catalogue of safe actions (for example add a log note, or escalate to a specialist), each with its end state, and a ready-to-paste log note.
4. Automatic guardrails reject answers that cite facts that do not exist, step outside the catalogue, propose changes to data, omit the risk of a discard, quote figures that are not in the evidence, or leak identifiers. A warning fires when confidence looks higher than the evidence supports.
5. A review sheet shows evidence, proposal, check results and a decision line; an audit record is written for each proposal.

What we measured (lab, six sample cases, a general-purpose language model):

Each figure is three repeat runs of six cases. (* The v2 failures are all one check that v2 was not told about; the pass-rate gap between v2 and v3 is partly built in.) The same case varied by about 0.03 in stated confidence between runs, so the differences are real, not noise. Model cost in the lab was a few cents per case.

What we learned.

• Evidence moves behaviour more than prompt wording. When the facts included a comparison with sibling entries, the model reported it every time; when they did not, never.
• Rules turn omissions into measurements. A consistent omission became countable once a simple check was added.
• The model caught what a person who knew the lab missed. In the seeder case it noticed that the To Do was dated 45 days before its source (a lab artifact of a date override) and that only half the matching To Dos sat in the routed role.
• It took the discard risk seriously. The prompt says only that a retry can end by discarding a record; the model repeatedly applied that, advising against retrying before the sender fixes its payload.
• Confidence is not accuracy. The model followed our calibration rule, but whether its stated confidence is the right number needs reviewer outcomes.
• An automatic check plus one repair round corrected a repeated over-recommendation. For a stale To Do, the model repeatedly recommended completing it while its own text still said something must be confirmed first. An automatic check now stops that; when it fires, the model is shown what failed and asked once to correct the answer, and it did so in every repeat. The check does not change the model's first answer, it catches it. A person still approves every completion.

Not tested. Accuracy against real reviewer decisions; a larger, unseen set; a repair step that was never needed in the live runs (verified only with a stub).

From proposal to action: the approval path (built and tested in the lab)

What. The missing link between a prepared case and a changed record. A named reviewer approves or rejects each proposal. For an approved one, a small gateway carries out a single narrow action through web services defined in the product's own configuration, under a dedicated service user that can do only that. Three actions are built: adding a log note, forwarding a To Do to another role (reversible), and completing a To Do (not reversible, so it needs a stated end state and written reviewer evidence). Everything else the model proposes stays a recommendation.

Rules in one line. Only an approved decision runs; the approver is a named person, never the model; only a short list of actions is allowed; the target must be open, unchanged and in a permitted role; every attempt is audited and is never repeated. The audit trail is tamper-evident (each record carries the fingerprint of the one before), and a reconciliation compares it with the database.

What we measured (lab). A full test of every capability covered log notes (including one proposed by the model), a forward and the forward back, the completion of a stale To Do with reviewer evidence, and a set of negative tests that must be refused. All but one behaved as designed first time; that one exposed a real gap, described below. For each write, a database query that did not rely on the gateway's own report showed the expected entries attributed to the service user, and a reconciliation of the audit trail against the database ended with nothing unexplained. Dry run, read-only check and live run were separate steps, each with its own recorded outcome.

What we learned.

• Least privilege shaped the design. The product's own read service needed broader execution rights than we were willing to give the service user, so we exposed a narrow read through our own web service instead. The service user keeps read and change access on To Do entries and nothing more.
• Verify with something other than the actor's own report. Our first live run reported one note as unverified. The note was fine; the gateway's check had misread an unordered list. The independent query caught it. We fixed the check and made "sent but not verified" block any repeat, so a doubt can never turn into a duplicate.
• Start with the least risky action. A log note is append-only and harmless, forwarding is reversible, and completing needs a verified end state. We built them in that order and tested each before the next.
• A service user has two halves. It needs the platform's login and the product's permissions, and both should be narrow. A copied user inherited broad access and had to be reduced before use.
• Permission settings alone were not enough to narrow a service. Our first forward interface exposed a broad product service that can also send a To Do back. A deliberate probe asked it to do that, an action the service user was never granted, and the request went through. The fix was to expose a purpose-built script that accepts only the fields the gateway needs; the same probe was then refused, and so was a stronger one that carried extra fields.
• The product's settings alone did not enforce the To Do's role on completion over the web service. Deliberate probes showed that a completion request could reach To Dos in roles the service user did not hold. The role rule had to be added where we controlled it: inside the narrow completion script, so it holds even if the gateway is bypassed, and in the gateway as a second control. An out-of-role completion is now refused and an in-role one still works. Both controls should be tested with requests that must fail, not only with requests that should succeed.
• Rules the product enforces are learned from refusals. A forward target must be a role valid for that To Do type, and the acting user must belong to the To Do's current role. Neither was obvious from the screens. Plan a test phase whose purpose is to be refused.

Not yet tested. A production system; a software-as-a-service tenant; volume; how well the automatic checks catch differently worded problems than the ones seen in the lab.

Phase 4. Guarded automation (described, not built)

What. Let the system carry out the narrowest, best-understood classes without a person: for example, completing a To Do whose source is verified healthy, or a retry on a class whose end state has been agreed.

Conditions. Shadow mode first (the system proposes, a person acts, and the two are compared); a confidence and class threshold; a defined end state per class; a kill switch; every action through the product's own web services under a dedicated service account with the minimum permissions; full audit.

Why we stop short. The experiments show how easily "fix" becomes "discard". The honest recommendation is to enter this phase only with measured reviewer agreement from Phase 3.

Prevention (through all phases)

Most of the lab's configuration findings are prevention findings: fix the cause once and stop the To Dos being created. Examples:

• A missing mapping for one template and time zone generated 138 To Dos from one setup.
• An empty section of a master configuration made every measurement older than the current date error out and raised To Dos in the default role, which nobody works.
• A validation setup with no high/low rule let a reading about 13 times normal through with no exception and no To Do. This is the quiet opposite of the noise problem, and a configuration review finds it.
• To Dos sitting in the default role, a routing gap, appeared in several different ways.

06 / Reference architecture

Reference architecture

Principle: the AI goes to the data.

Read path (by deployment):

Analysis and model placement. The case builder, the guardrails and the review service run in the utility's environment. The model runs where the utility chooses: inside its own tenancy or network, or through a managed model service under its own contract. Only pseudonymized case files are sent, and the token map never leaves the utility. A small classification task can use a small local model; diagnosis and narrative need a larger one.

Action path. Approved actions go through the product's web services under a dedicated service user with narrowly scoped permissions, never to the database directly. Three of them, adding a log note, forwarding and completing, are built and tested in the lab (see above). The product's own To Do service offered no way to add a log, forward or complete, so small custom services were added, one of them a narrow script with its own role check; it is configuration, which software-as-a-service customers can also do.

Components. Connector, case builder with a pseudonymizer, model adapter, guardrail validator, review queue, audit log, and a step-by-step view per phase. The same container image can run on a hosting platform for a demonstration with demo data, and inside the utility's environment for a pilot.

07 / Governance and risk

Governance and risk

Autonomy is a dial, not a switch: advise, then prepare, then act on narrow classes under watch. Anything that closes or changes a record should carry the evidence, the proposer, the approver and the end state.

08 / Operating model, measures and offering

Operating model, measures and offering

Measures to track from the first week: open To Dos by age; share of To Dos created per bill run; share auto-resolved by product levers; share of To Dos whose source is stale; time from open to resolved; bills blocked by To Dos; reassignment rate; share of copilot proposals approved unchanged, edited or rejected; agreement of proposals with reviewer decisions; reopened To Dos after a copilot-assisted completion.

Engagement shape. Every utility's configuration, roles and messages differ, so this is a service, not a product:

1. Assessment (Phase 0, plus configuration review). Pareto of the backlog, source linkage, root-cause classes, a list of configuration findings with expected effects.
2. Hygiene sprint (Phase 1). Assign and schedule monitor batches, agree end states, reroute orphan queues, retire stale entries with a source-aware monitor.
3. Triage and copilot pilot (Phases 2 and 3). Read-only, shadow mode, measured against reviewer decisions.
4. Optional guarded automation (Phase 4) for classes that earn it.

Each step ends with a decision to stop, continue or change course.

09 / What we did not test, and next steps

What we did not test, and next steps

Not tested: real utility data; a software-as-a-service tenant; true fresh high/low exceptions from new readings (reaching fresh high/low exceptions in the lab needed workarounds); billing exceptions in volume (the lab had 8); accuracy against reviewer decisions; executing approved actions on a production system; cost and effort at utility scale.

Next: a read-only assessment at a willing utility; the approval path tested on a non-production copy of a real system, with the client's own roles and permissions; a pilot with at least a few hundred To Dos scored against reviewer decisions; configuration proposals tested on a non-production copy of a real system.

Appendix A

Evidence index

Appendix B

Glossary

To Do: a work item addressed to a role. Case: in this paper, our own review file for one To Do (its numbered facts, the model's proposal, the checks and the reviewer's decision); not the case concept of the Oracle products. Drill key: the identifier of the source record. Monitor batch: the scheduled job that runs the retry for a state. Seeder: the entry point for incoming measurement data. IMD: initial measurement data. VEE: validation, estimation and editing of measurements. Owner flag (D1, D2, F1, C1, CM): the prefix showing which product layer owns an object; D2 has no tables of its own. Pseudonymization: replacing identifying values with tokens before data is shared. Shadow mode: the system proposes, a person acts, and the two are compared.

Appendix C

Sources

• Oracle, The Lifecycle of a To Do Entry
• Oracle, Managing To Do Entries
• Oracle, Processing Errors (CC&B 25.4)
• Oracle, GoldenGate Replication Overview
• Oracle, Creating a File Export Batch Process
• Oracle, Legacy Measurement Processing (C2M 25.4)
• Oracle, Common Frequently Asked Questions (cloud services)
• Oracle, Utilities call on Oracle AI to transform customer service (2025-05-06)
• Oracle Utilities C2M 25.4 Administrative User Guide and Server Administration Guide.